FlociClone/docs/services/kafka-connect.md

MSK Connect

Protocol: REST JSON Signing name: kafkaconnect

Floci implements the MSK Connect control plane separately from MSK. Custom plugins are ZIP archives stored in Floci S3. A connector starts a pinned stock Apache Kafka Connect worker, extracts the referenced ZIP, and mounts only that extracted plugin directory into the worker.

Supported Actions

Action	Description
CreateCustomPlugin	Register a ZIP custom plugin from an S3 bucket ARN and key
ListCustomPlugins	List custom plugins
DescribeCustomPlugin	Describe a custom plugin
DeleteCustomPlugin	Delete a custom plugin
CreateWorkerConfiguration	Create a worker properties configuration
ListWorkerConfigurations	List worker configurations
DescribeWorkerConfiguration	Describe a worker configuration
DeleteWorkerConfiguration	Delete a worker configuration
CreateConnector	Start a connector using a custom plugin
ListConnectors	List connectors
DescribeConnector	Describe a connector
UpdateConnector	Update connector configuration
DeleteConnector	Stop and delete a connector

Worker behavior

Workers use the internal Redpanda address of a Floci-managed MSK cluster. The public bootstrap address supplied to CreateConnector is resolved to that cluster but is never passed into the container. This keeps the worker on Floci's Docker network while host clients continue using the public MSK bootstrap endpoint.

No connector plugin is bundled. To run an S3 sink, upload a compatible S3 sink connector ZIP to Floci S3, create a custom plugin for it, and use that plugin in CreateConnector. The worker's container logs are streamed to CloudWatch Logs under /aws/msk-connect/{connectorName}.

Configuration

Variable	Default	Description
FLOCI_SERVICES_KAFKACONNECT_ENABLED	true	Enable or disable MSK Connect
FLOCI_SERVICES_KAFKACONNECT_MOCK	false	Persist control-plane metadata without starting workers
FLOCI_SERVICES_KAFKACONNECT_WORKER_IMAGE	apache/kafka:3.9.0	Pinned stock Kafka Connect worker image