Zakaria/FlociClone
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

bug fixes: backend services patches - Implemented and deployed systemd-backed AMIs

Browse Source
This commit is contained in:
Zakaria
2026-10-07 13:22:32 -04:00
parent 7135c81050
commit 019570d8a4
58 changed files with 1591 additions and 133 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/services/kafka-connect.md
+45
View File
@@ -0,0 +1,45 @@
# MSK Connect
**Protocol:** REST JSON
**Signing name:** `kafkaconnect`
Floci implements the MSK Connect control plane separately from MSK. Custom plugins are ZIP
archives stored in Floci S3. A connector starts a pinned stock Apache Kafka Connect worker, extracts
the referenced ZIP, and mounts only that extracted plugin directory into the worker.
## Supported Actions
| Action | Description |
|---|---|
| `CreateCustomPlugin` | Register a ZIP custom plugin from an S3 bucket ARN and key |
| `ListCustomPlugins` | List custom plugins |
| `DescribeCustomPlugin` | Describe a custom plugin |
| `DeleteCustomPlugin` | Delete a custom plugin |
| `CreateWorkerConfiguration` | Create a worker properties configuration |
| `ListWorkerConfigurations` | List worker configurations |
| `DescribeWorkerConfiguration` | Describe a worker configuration |
| `DeleteWorkerConfiguration` | Delete a worker configuration |
| `CreateConnector` | Start a connector using a custom plugin |
| `ListConnectors` | List connectors |
| `DescribeConnector` | Describe a connector |
| `UpdateConnector` | Update connector configuration |
| `DeleteConnector` | Stop and delete a connector |
## Worker behavior
Workers use the internal Redpanda address of a Floci-managed MSK cluster. The public bootstrap
address supplied to `CreateConnector` is resolved to that cluster but is never passed into the
container. This keeps the worker on Floci's Docker network while host clients continue using the
public MSK bootstrap endpoint.
No connector plugin is bundled. To run an S3 sink, upload a compatible S3 sink connector ZIP to
Floci S3, create a custom plugin for it, and use that plugin in `CreateConnector`. The worker's
container logs are streamed to CloudWatch Logs under `/aws/msk-connect/{connectorName}`.
## Configuration
| Variable | Default | Description |
|---|---|---|
| `FLOCI_SERVICES_KAFKACONNECT_ENABLED` | `true` | Enable or disable MSK Connect |
| `FLOCI_SERVICES_KAFKACONNECT_MOCK` | `false` | Persist control-plane metadata without starting workers |
| `FLOCI_SERVICES_KAFKACONNECT_WORKER_IMAGE` | `apache/kafka:3.9.0` | Pinned stock Kafka Connect worker image |