FlociClone/docs/services/kinesis.md

Kinesis

Protocol: JSON 1.1 (X-Amz-Target: Kinesis_20131202.*) Endpoint: POST http://localhost:4566/

Supported Actions

Action	Description
CreateStream	Create a stream
DeleteStream	Delete a stream
ListStreams	List all streams
DescribeStream	Get stream details and shard info
DescribeStreamSummary	Lightweight stream description
RegisterStreamConsumer	Register an enhanced fan-out consumer
DeregisterStreamConsumer	Remove a consumer
DescribeStreamConsumer	Get consumer details
ListStreamConsumers	List consumers for a stream
SubscribeToShard	Subscribe to a shard for enhanced fan-out
AddTagsToStream	Tag a stream
RemoveTagsFromStream	Remove tags
ListTagsForStream	List tags
StartStreamEncryption	Enable KMS encryption
StopStreamEncryption	Disable encryption
SplitShard	Split a shard into two
MergeShards	Merge two adjacent shards
PutRecord	Write a single record
PutRecords	Write up to 500 records
GetShardIterator	Get an iterator for reading
GetRecords	Read records from a shard
ListShards	-
IncreaseStreamRetentionPeriod	Increase retention up to 8760 hours (365 days)
DecreaseStreamRetentionPeriod	Decrease retention down to 24 hours
EnableEnhancedMonitoring	-
DisableEnhancedMonitoring	-
UpdateStreamMode	-
UpdateMaxRecordSize	-
UpdateShardCount	-

Local Inspection Endpoints

These endpoints are Floci-local read-only helpers for the UI and tests. AWS SDK traffic should continue to use the JSON 1.1 Kinesis API on /.

Endpoint	Description
GET /_aws/kinesis/streams	List streams in the resolved region with shard, mode, tag, and record counts
GET /_aws/kinesis/records?StreamName=<name>	Peek up to 100 stream records with shard attribution without consuming them
GET /_aws/kinesis/records?StreamName=<name>&Limit=<n>	Peek up to n records, capped at 1000
GET /_aws/kinesis/records?StreamName=<name>&ShardId=<id>	Peek records for one shard

Stream Addressing

Most actions accept either StreamName or StreamARN to identify a stream. When both are provided, StreamName takes precedence. CreateStream only accepts StreamName.

# By name
aws kinesis describe-stream --stream-name events --endpoint-url $AWS_ENDPOINT_URL

# By ARN
aws kinesis describe-stream --stream-arn arn:aws:kinesis:us-east-1:000000000000:stream/events --endpoint-url $AWS_ENDPOINT_URL

Shard Iterators

GetShardIterator supports all five iterator types: TRIM_HORIZON, LATEST, AT_SEQUENCE_NUMBER, AFTER_SEQUENCE_NUMBER, AT_TIMESTAMP.

A LATEST iterator is positioned at the shard tip at the moment the iterator is created, matching AWS: records written after the iterator was obtained are returned, records written before are not. This supports the standard tailing pattern: obtain a LATEST iterator, trigger the action that produces the record, then poll GetRecords following NextShardIterator.

Record Routing

PutRecord and PutRecords honor ExplicitHashKey when it is provided. The value must be a decimal integer in the Kinesis hash-key space, and records are written to the open shard whose HashKeyRange contains that value. Without ExplicitHashKey, Floci keeps using the partition key to choose a shard.

Shard Scaling (UpdateShardCount)

UpdateShardCount enforces the documented default limits: at most double the current open shard count per call, at most ten calls per rolling 24-hour period per stream, a 10000-shard ceiling, and the asymmetric rule that a stream already above 10000 shards can only move to a target strictly below it.

The minimum bound (TargetShardCount cannot go below half the current open shard count) rounds up for an odd current count, so current=5 accepts target=3 but rejects target=2. AWS's documentation says only "below half" with no stated rounding, so which direction an odd count rounds is Floci's own call rather than observed AWS behavior; this is the more conservative reading (fewer shards allowed, not more).

AWS also documents a separate 10 TPS call-rate limit on this action ("Make over 10 TPS. TPS over 10 will trigger the LimitExceededException"), independent of the ten-calls-per-24h limit above. Floci does not simulate real-time request throttling for this or any other action, so that limit is intentionally unenforced here.

Current Limitations

• ListStreams, ListShards, ListTagsForStream, and ListStreamConsumers do not yet support AWS pagination tokens. ListShards honors MaxResults, but does not return a continuation token.
• Resource policy actions are not implemented.
• KMS encryption settings are recorded in stream metadata, but record payloads are not cryptographically encrypted.

Configuration

Variable	Default	Description
FLOCI_SERVICES_KINESIS_ENABLED	true	Enable or disable the service

Enhanced Fan-Out (EFO)

SubscribeToShard uses a snapshot-and-close model: the server returns one batch of records as a binary EventStream response and closes the connection. The SDK resubscribes automatically using the ContinuationSequenceNumber from the last delivered record. All five StartingPosition types are supported: TRIM_HORIZON, LATEST, AT_SEQUENCE_NUMBER, AFTER_SEQUENCE_NUMBER, AT_TIMESTAMP.

export AWS_ENDPOINT_URL=http://localhost:4566
STREAM=my-stream

# Register a consumer
aws kinesis register-stream-consumer \
  --stream-arn $(aws kinesis describe-stream --stream-name $STREAM \
      --query StreamDescription.StreamARN --output text) \
  --consumer-name my-consumer

# Subscribe (AWS CLI streams events to stdout)
aws kinesis subscribe-to-shard \
  --consumer-arn <consumer-arn> \
  --shard-id shardId-000000000000 \
  --starting-position Type=TRIM_HORIZON

Examples

export AWS_ENDPOINT_URL=http://localhost:4566

# Create a stream
aws kinesis create-stream \
  --stream-name events \
  --shard-count 2 \
  --endpoint-url $AWS_ENDPOINT_URL

# Put a record
aws kinesis put-record \
  --stream-name events \
  --partition-key "user-123" \
  --data '{"event":"page_view","page":"/home"}' \
  --endpoint-url $AWS_ENDPOINT_URL

# Get a shard iterator
SHARD_ID=$(aws kinesis describe-stream \
  --stream-name events \
  --query 'StreamDescription.Shards[0].ShardId' --output text \
  --endpoint-url $AWS_ENDPOINT_URL)

ITERATOR=$(aws kinesis get-shard-iterator \
  --stream-name events \
  --shard-id $SHARD_ID \
  --shard-iterator-type TRIM_HORIZON \
  --query ShardIterator --output text \
  --endpoint-url $AWS_ENDPOINT_URL)

# Read records
aws kinesis get-records \
  --shard-iterator $ITERATOR \
  --endpoint-url $AWS_ENDPOINT_URL