second first commit
This commit is contained in:
@@ -0,0 +1,673 @@
|
||||
# Step Functions
|
||||
|
||||
**Protocol:** AWS JSON 1.0 (`X-Amz-Target: AWSStepFunctions.*`)
|
||||
**Endpoint:** `POST http://localhost:4566/`
|
||||
|
||||
## Supported Actions
|
||||
|
||||
<!-- floci:actions:start -->
|
||||
| Action | Description |
|
||||
| --- | --- |
|
||||
| `CreateStateMachine` | Create a state machine (Standard or Express) |
|
||||
| `UpdateStateMachine` | Update definition, role, logging, tracing, or encryption settings and optionally publish a version |
|
||||
| `DescribeStateMachine` | Get state machine definition and metadata |
|
||||
| `ListStateMachines` | List all state machines |
|
||||
| `DeleteStateMachine` | Delete a state machine |
|
||||
| `PublishStateMachineVersion` | - |
|
||||
| `ListStateMachineVersions` | - |
|
||||
| `DeleteStateMachineVersion` | - |
|
||||
| `CreateStateMachineAlias` | - |
|
||||
| `DescribeStateMachineAlias` | - |
|
||||
| `ListStateMachineAliases` | - |
|
||||
| `UpdateStateMachineAlias` | - |
|
||||
| `DeleteStateMachineAlias` | - |
|
||||
| `ValidateStateMachineDefinition` | Validate an ASL definition without creating a state machine |
|
||||
| `StartExecution` | Start a new execution |
|
||||
| `StartSyncExecution` | - |
|
||||
| `DescribeExecution` | Get execution status and output |
|
||||
| `ListExecutions` | List executions for a state machine |
|
||||
| `StopExecution` | Stop a running execution |
|
||||
| `GetExecutionHistory` | Get the full event history of an execution |
|
||||
| `DescribeMapRun` | Get the item and execution counters of a distributed Map run |
|
||||
| `SendTaskSuccess` | Report task success (for `.waitForTaskToken` tasks) |
|
||||
| `SendTaskFailure` | Report task failure |
|
||||
| `SendTaskHeartbeat` | Send a heartbeat for long-running tasks |
|
||||
| `CreateActivity` | - |
|
||||
| `DeleteActivity` | - |
|
||||
| `DescribeActivity` | - |
|
||||
| `ListActivities` | - |
|
||||
| `GetActivityTask` | - |
|
||||
| `ListTagsForResource` | - |
|
||||
| `TagResource` | - |
|
||||
| `UntagResource` | - |
|
||||
<!-- floci:actions:end -->
|
||||
|
||||
`UpdateStateMachine` returns the new revision ID and update timestamp. CloudFormation updates
|
||||
definition, role, logging, tracing, encryption, and tags without replacing the state machine;
|
||||
changes to `StateMachineName` or `StateMachineType` use replacement semantics.
|
||||
|
||||
## Execution history events
|
||||
|
||||
`GetExecutionHistory` emits an event family around a Task state's `TaskStateEntered` and
|
||||
`TaskStateExited` pair. Which family depends on the resource type.
|
||||
|
||||
- A service integration ARN (`arn:aws:states:::...`) emits `TaskScheduled`, `TaskStarted`,
|
||||
and then `TaskSucceeded` or `TaskFailed`. `taskScheduledEventDetails` carries
|
||||
`resourceType`, `resource`, `region`, and `parameters`. `parameters` is the resolved
|
||||
`Parameters` or `Arguments` payload, serialized as a JSON string. `timeoutInSeconds` and
|
||||
`heartbeatInSeconds` appear only when the state sets `TimeoutSeconds` or
|
||||
`HeartbeatSeconds` as a literal number.
|
||||
- A direct Lambda function ARN emits `LambdaFunctionScheduled`, `LambdaFunctionStarted`,
|
||||
and then `LambdaFunctionSucceeded` or `LambdaFunctionFailed`. `LambdaFunctionScheduled`
|
||||
carries `resource` and `input`. `resource` holds the full function ARN. `LambdaFunctionStarted`
|
||||
carries no details at all. This matches AWS.
|
||||
- An activity ARN emits the equivalent `Activity*` family.
|
||||
|
||||
A `Retry` re-entry emits its own Scheduled, Started, and Failed triple for each attempt. A
|
||||
mocked Task (`SFN_MOCK_CONFIG`) emits the same events as a real one, because Step Functions
|
||||
Local does the same.
|
||||
|
||||
Every event's `previousEventId` points to the id of the event right before it on the same
|
||||
chain of states. The one exception is the first state's `*StateEntered` event. Its
|
||||
`previousEventId` is `0`. That matches `ExecutionStarted`, which is always
|
||||
`id: 1, previousEventId: 0`.
|
||||
|
||||
### Parallel branches and Map iterations
|
||||
|
||||
The states inside a `Parallel` branch or an inline `Map` iteration publish their events into
|
||||
the parent execution's history, as on AWS. A `Parallel` records `ParallelStateStarted`,
|
||||
`ParallelStateSucceeded` and `ParallelStateFailed`. An inline `Map` records `MapStateStarted`,
|
||||
`MapIterationStarted`, `MapIterationSucceeded`, `MapIterationFailed`, `MapStateSucceeded` and
|
||||
`MapStateFailed`. A Distributed `Map` records `MapRunStarted`, `MapRunSucceeded` and
|
||||
`MapRunFailed` instead. Its items are child executions and publish nothing into the parent
|
||||
history. A `Task` whose failure ends its branch also records `TaskStateAborted`.
|
||||
|
||||
Branches and iterations run concurrently, so the order in which their events interleave differs
|
||||
from run to run. Each branch chains its own events through `previousEventId`, and that chain is
|
||||
the same every time.
|
||||
|
||||
### The cause of a failure
|
||||
|
||||
The `cause` of a failure the interpreter raises starts with
|
||||
`An error occurred while executing the state '<name>' (entered at the event id #<n>). `, as on
|
||||
AWS. The prefix is added once, at the innermost state. A `Fail` state's `Cause` and a cause a
|
||||
task's resource answered with pass through unchanged. A `Choice` that matches no rule and has
|
||||
no `Default`, and a payload template path that matches nothing, fail with `States.Runtime`.
|
||||
A JSONata expression that fails also records an `EvaluationFailed` event with `error`, `cause`,
|
||||
`location` and `state`, once per attempt.
|
||||
|
||||
`inputDetails` appears on `ExecutionStarted`, on `stateEnteredEventDetails`, and on
|
||||
`LambdaFunctionScheduled`/`ActivityScheduled`. `outputDetails` appears on
|
||||
`stateExitedEventDetails`, on `executionSucceededEventDetails`, and on
|
||||
`taskSucceededEventDetails`/`lambdaFunctionSucceededEventDetails`. Both are always
|
||||
`{"truncated": false}`. Floci never truncates a payload, so the value never changes.
|
||||
When the request sets `includeExecutionData` to false, the details objects stay but lose
|
||||
`input`, `inputDetails`, `output`, and `outputDetails`. Every other field stays, including
|
||||
`taskScheduledEventDetails.parameters`. This matches AWS.
|
||||
|
||||
A few gaps remain. `TaskStarted`, `LambdaFunctionStarted`, and `ActivityStarted` fire at
|
||||
scheduling time, not when a worker actually picks up the task. `TaskSubmitted`, which real
|
||||
AWS emits for `.sync` and `.waitForTaskToken` integrations, is not emitted yet. When a
|
||||
branch fails, AWS records `*StateAborted` and `MapIterationAborted` events for the states its
|
||||
sibling branches were in; Floci cancels the siblings without recording them. A Distributed
|
||||
`Map` that declares no tolerance reports a failed item's own error rather than AWS's
|
||||
`States.ExceedToleratedFailureThreshold`, and emits `MapRunFailed` with that error. A `Map` that
|
||||
declares one reports `States.ExceedToleratedFailureThreshold`, as AWS does.
|
||||
|
||||
## Map concurrency
|
||||
|
||||
Map states honor `MaxConcurrency` and, for JSONPath state machines, `MaxConcurrencyPath`.
|
||||
JSONata state machines may supply `MaxConcurrency` as an expression. A value of `0`, or an
|
||||
omitted value, uses the AWS service ceiling: 40 concurrent iterations for Inline Map states and
|
||||
10,000 for Distributed Map states. `MaxConcurrency: 1` runs iterations sequentially.
|
||||
|
||||
Results remain in input order even when iterations finish out of order. If an iteration fails,
|
||||
the Map state fails promptly, cancels its active sibling iterations, and does not start queued
|
||||
iterations.
|
||||
|
||||
## Distributed Map ItemReader
|
||||
|
||||
`ItemReader` reads a dataset from S3. The resource decides how the dataset is found, and
|
||||
`ReaderConfig.InputType` decides how it is read.
|
||||
|
||||
`arn:aws:states:::s3:getObject` reads a single object:
|
||||
|
||||
- `JSON` is either an array, or an object whose entries become `Key` and `Value` items.
|
||||
`ReaderConfig.ItemsPointer` selects a node inside it.
|
||||
- `JSONL` is one item per line. Blank lines are skipped, and `ItemsPointer` does not apply,
|
||||
matching AWS.
|
||||
- `CSV` takes its field names from the first row, or from `ReaderConfig.CSVHeaders` when
|
||||
`CSVHeaderLocation` is `GIVEN`. Every value is a string: a row shorter than the headers pads
|
||||
with empty strings, and a longer one drops the surplus. `ReaderConfig.CSVDelimiter` selects
|
||||
`COMMA`, `PIPE`, `SEMICOLON`, `SPACE` or `TAB`, and a quoted field may contain the delimiter
|
||||
or a line break without ending the record. A doubled quote stands for a single quote, in an
|
||||
unquoted field as well as a quoted one, and a backslash escapes another backslash, a quote or
|
||||
the delimiter. A backslash before anything else is dropped, as AWS documents.
|
||||
- `PARQUET` and `MANIFEST` are accepted by `CreateStateMachine` and fail the execution with
|
||||
`States.ItemReaderFailed`.
|
||||
|
||||
`arn:aws:states:::s3:listObjectsV2` reads every page under `Prefix`. Each item carries the AWS
|
||||
fields `Etag`, `Key`, `LastModified` (epoch seconds), `Size` and `StorageClass`. An empty prefix
|
||||
gives zero iterations and the Map succeeds.
|
||||
|
||||
`ReaderConfig.MaxItems` applies to every reader; `MaxItemsPath` is not supported.
|
||||
|
||||
## Distributed Map ItemBatcher
|
||||
|
||||
`ItemBatcher` hands each child execution a batch of items instead of a single item. The child input
|
||||
is `{"BatchInput": ..., "Items": [...]}`, with `BatchInput` present only when the state declares it.
|
||||
`ItemSelector` still runs per item, before the items are grouped.
|
||||
|
||||
A batch closes on `MaxItemsPerBatch`, on `MaxInputBytesPerBatch`, or on the 256 KiB child-input
|
||||
ceiling AWS applies whether or not a byte limit is declared. Either limit may be given as a
|
||||
`...Path` field, or as an expression in a JSONata state machine. With neither declared, items fill
|
||||
one batch up to that ceiling. The size measured is the serialized child payload, envelope and
|
||||
`BatchInput` included, not the items alone. An item that would exceed the ceiling on its own can
|
||||
never start a child execution, so the state fails with `States.DataLimitExceeded` rather than
|
||||
building a batch AWS would reject: reduce the item with `ItemSelector` first.
|
||||
|
||||
`MaxConcurrency` then bounds concurrent batches, and the Map result has one entry per batch rather
|
||||
than per item. `DescribeMapRun` reports items under `itemCounts` and batches under
|
||||
`executionCounts`.
|
||||
|
||||
## Tolerated failures
|
||||
|
||||
`ToleratedFailureCount` and `ToleratedFailurePercentage` let a Distributed `Map` absorb failed items
|
||||
instead of failing on the first one. Both accept a `...Path` field, or an expression in a JSONata
|
||||
state machine, and the percentage is taken over the item count. Declaring both applies the stricter
|
||||
of the two.
|
||||
|
||||
An absorbed failure contributes no result, so the `Map` output carries one entry per successful
|
||||
child execution. A `ResultWriter` still exports it: successful children go to `SUCCEEDED_0.json` and
|
||||
absorbed failures to `FAILED_0.json`, each listed under the matching key of the manifest's
|
||||
`ResultFiles`. A failed record carries `Error` and `Cause` in place of an output. Once the budget is
|
||||
spent, the state fails with `States.ExceedToleratedFailureThreshold` and the run emits
|
||||
`MapRunFailed`.
|
||||
|
||||
`DescribeMapRun` reports the declared values under `toleratedFailureCount` and
|
||||
`toleratedFailurePercentage`. A `Map` that declares neither keeps the earlier behaviour: the first
|
||||
failed item fails the state, carrying that item's own error.
|
||||
|
||||
## Retry policies
|
||||
|
||||
`Task`, `Parallel`, and `Map` states honor their `Retry` field. `ErrorEquals` matching
|
||||
follows AWS semantics, including the `States.ALL` and `States.TaskFailed` wildcards.
|
||||
`States.Runtime` is never retried. AWS defaults apply when fields are omitted
|
||||
(`MaxAttempts` 3, `IntervalSeconds` 1, `BackoffRate` 2.0), `MaxDelaySeconds` is honored,
|
||||
and each retrier keeps its own attempt counter. `Retry` is evaluated before `Catch`, and
|
||||
`$$.State.RetryCount` increments per attempt. Attempt counts, defaults, and backoff
|
||||
timing were verified against real AWS Step Functions.
|
||||
|
||||
A state with no `Retry` array runs exactly once. This is what makes CDK-generated provider-framework
|
||||
workflows converge: the `framework-isComplete-task` throws on every not-yet-complete poll and relies on
|
||||
`Retry` to poll again until the custom resource reports done — see
|
||||
[CloudFormation custom resources](cloudformation.md#custom-resources-and-the-cdk-provider-framework).
|
||||
|
||||
`JitterStrategy` supports `NONE` (the default) and `FULL`. `FULL` draws the delay
|
||||
uniformly between zero and the computed delay, as on AWS. One deviation. The delay
|
||||
between attempts is capped at 30 seconds, the same cap Floci applies to `Wait` states,
|
||||
so emulated runs stay fast.
|
||||
|
||||
## Timeouts
|
||||
|
||||
ASL carries two `TimeoutSeconds` fields and Floci enforces both, in the two terminal shapes
|
||||
AWS uses. The state machine's own field bounds every state; a `Task`'s own field only bounds
|
||||
one that waits for a task token — an activity, or a `.waitForTaskToken` integration. A Lambda
|
||||
or other SDK task that returns directly is not bound by it.
|
||||
|
||||
The state machine's own `TimeoutSeconds` is the whole execution's budget. It is checked before
|
||||
every state and inside a `Wait`, so a `Wait` longer than what is left is cut rather than slept
|
||||
out. The execution ends `TIMED_OUT` with `stopDate` set and no `error` and no `cause` at all;
|
||||
`States.Timeout` is named only in the single `ExecutionTimedOut` event, whose `previousEventId`
|
||||
is `0`. The state that was cut gets no `*StateExited` event. A `Parallel` or `Map` branch runs
|
||||
on its own thread and is not cut mid-state: the budget is enforced again as soon as the branch
|
||||
returns.
|
||||
|
||||
A `Task` that waits for a task token — an activity, or a `.waitForTaskToken` integration — has
|
||||
two independent bounds. `TimeoutSeconds` is the whole wait; `HeartbeatSeconds` is the longest gap
|
||||
allowed between two `SendTaskHeartbeat` calls, and every heartbeat pushes that gap forward, so a
|
||||
worker that reports as often as the definition asks runs until `TimeoutSeconds` runs out. Either
|
||||
clock ends the state the same way: an `ActivityTimedOut` event — `TaskTimedOut` for a
|
||||
`.waitForTaskToken` integration — carrying `States.Timeout` and no cause, then an execution that
|
||||
reads `FAILED` with that same error. A `Catch` on a heartbeat expiry matches under
|
||||
`States.HeartbeatTimeout` and under `States.Timeout` alike. Neither timeout carries a cause:
|
||||
`DescribeExecution` and the `ExecutionFailed` event both leave the key out, where every other
|
||||
failure reports one. A `Task` that declares no `TimeoutSeconds` waits 300 seconds, where AWS
|
||||
waits a year.
|
||||
|
||||
One deviation. AWS starts the `TimeoutSeconds` clock when a worker picks the task up, the instant
|
||||
it emits `ActivityStarted`. Floci emits `ActivityStarted` at schedule time, so both clocks start
|
||||
when the task is scheduled.
|
||||
|
||||
## Intrinsic arguments
|
||||
|
||||
A `$.` reference passed to a `States.*` intrinsic must find something. An argument that matches
|
||||
nothing fails the execution with `States.Runtime`, as on AWS.
|
||||
|
||||
One deviation. Indexing something that is not an array makes AWS leak its JSONPath library and
|
||||
write `Filter: [0] can only be applied to arrays. Current context is: 1`. Floci writes its own
|
||||
`The JsonPath argument for the field '$.other[0]' could not be found in the input ...` there.
|
||||
|
||||
## JSONata nulls
|
||||
|
||||
An expression that evaluates to JSON `null` produces a value, not a missing one. It keeps its key
|
||||
in `Output`, in `Assign` and in a Task's `Arguments`, at any nesting depth, and it stays in place as
|
||||
an array element:
|
||||
|
||||
```
|
||||
"Output": {"v": "{% $states.input.bar %}"} on input {"bar": null} -> {"v": null}
|
||||
```
|
||||
|
||||
The same `null` is a value inside the expression too: `$exists()` on it is true and `$type()` on it
|
||||
is `"null"`.
|
||||
|
||||
One deviation. `$count()` on a JSON null answers `0`; AWS answers `1`.
|
||||
|
||||
## An expression that returns nothing fails the state
|
||||
|
||||
An expression that returns nothing, which is what `$states.input.absent` and the functions listed
|
||||
below that evaluate to undefined do, is not a missing value the state carries on without. It fails
|
||||
the state with `States.QueryEvaluationError`, naming the field it was written in, and a `Catch` on
|
||||
that error fires:
|
||||
|
||||
```
|
||||
"Output": {"v": "{% $states.input.absent %}"}
|
||||
-> States.QueryEvaluationError
|
||||
The JSONata expression '$states.input.absent' specified for the field 'Output/v'
|
||||
returned nothing (undefined).
|
||||
```
|
||||
|
||||
The field is named relative to the state, with `/` before each object key and `[i]` for each array
|
||||
index: `Output`, `Output/a/b[0]`, `Assign/x`, `Arguments/MessageGroupId`, `Choices[1]/Condition`,
|
||||
`Seconds`, `Error`, `Cause`, `Items`, `MaxConcurrency`. A matched `Choice` rule and a matching
|
||||
`Catch` clause carry their own `Assign` and `Output`, which are named under the rule or clause:
|
||||
`Choices[1]/Output/v`, `Choices[0]/Assign/x`, `Catch[1]/Output/v`. A `Choice` stops at the first rule
|
||||
that matches, so an undefined condition in a later rule is never evaluated.
|
||||
|
||||
The cause of a real execution carries the state prefix described under
|
||||
[The cause of a failure](#the-cause-of-a-failure).
|
||||
|
||||
## JSONata functions
|
||||
|
||||
State machines with `"QueryLanguage": "JSONata"` reach the six functions Step Functions adds on
|
||||
top of the JSONata language, alongside every function JSONata itself provides.
|
||||
|
||||
| Function | Returns |
|
||||
| --- | --- |
|
||||
| `$parse(jsonString)` | the deserialized value; the replacement for `$eval`, which AWS disables and so does Floci, answering `T1006` to a call |
|
||||
| `$partition(array, chunkSize)` | `array` split into chunks of `chunkSize`, the last one holding the remainder |
|
||||
| `$range(start, end, step)` | the values from `start` to `end`, inclusive when `step` lands on `end` |
|
||||
| `$hash(str, algorithm)` | the hex digest of `str`; `algorithm` is `MD5`, `SHA-1`, `SHA-256`, `SHA-384` or `SHA-512`, case-sensitive |
|
||||
| `$random(seed)` | a number in `[0, 1)`, reproducible under the optional integer `seed` |
|
||||
| `$uuid()` | a v4 UUID |
|
||||
|
||||
Three behaviours are worth knowing before reading an unexpected result, and all three are AWS's:
|
||||
|
||||
- A non-integer argument is rounded **towards zero**, so `$range(-1.7, 2, 1)` starts at `-1` and
|
||||
`$partition(items, 2.9)` chunks by 2.
|
||||
- Several arguments evaluate to undefined rather than failing: a chunk size of zero, an empty
|
||||
array, a `$range` with no `step` or with a step whose sign disagrees with the direction, and a
|
||||
`$hash` with no algorithm. The argument itself does not fail; the field the expression was
|
||||
written in does, under `States.QueryEvaluationError`, as the section above records.
|
||||
- `$range` collapses a single-element range to the bare number, not a one-element array.
|
||||
|
||||
JSONata's own `$string` follows AWS's number notation: a whole number is written out in full below
|
||||
`1e21` and in exponent notation from there, on both signs, so `$string(1e20)` is
|
||||
`100000000000000000000` and `$string(1e21)` is `1e+21`.
|
||||
|
||||
The execution input reaching `$states.input` follows the same number model as `$parse`: an integer
|
||||
stays exact while it fits in a `long` and switches to a `double` past that boundary, matching AWS.
|
||||
|
||||
JSONata's own `$formatNumber` checks its picture string against the fourteen rules of XPath F&O
|
||||
4.7.3, as AWS does, so `$formatNumber(1, "x")` fails with `D3086` rather than answering `x1`: a
|
||||
picture with no digit in it describes no number.
|
||||
|
||||
Evaluation is bounded on three axes, as it is on AWS, and past any of them the state fails with
|
||||
`States.QueryEvaluationError`.
|
||||
|
||||
- **Depth**: one expression may nest 100 levels, which is AWS's own ceiling: AWS accepts `1+1+…+1`
|
||||
at 100 terms, 99 parentheses, 99 brackets, 99 `~>` stages, and a non-tail-recursive `$f(31)`,
|
||||
which nests `3n+5`, and refuses one level more of each. The refusal names the depth reached, as
|
||||
in `Stack overflow error: … Depth=101 max=100`.
|
||||
- **Memory**: one value an expression builds may hold 6,990,256 bytes, counting a number as eight
|
||||
bytes and a character as one. That is AWS's own bound: AWS accepts `[1..873782]` and refuses one
|
||||
element more, and it accepts a string doubled 22 times, 2^22 characters, refusing the 23rd. The
|
||||
refusal is AWS's own
|
||||
`Expression evaluation memory limit exceeded`, which is what `[1..900000] ~> $count()` and
|
||||
`$sum([1..900000])` now answer. `$range` holds a tighter bound of its own, checked before it
|
||||
allocates rather than on the array it would have built: AWS accepts `$range(1, 360145, 1)`, all
|
||||
360,145 elements, and refuses one element more with the same refusal. A lazy literal range such
|
||||
as `[1..873782]` is exempt from both bounds on AWS, and stays exempt here.
|
||||
- **Time**: five seconds, which is the library's own default and roughly fifty times the slowest
|
||||
evaluation of a payload AWS itself accepts. A recursive expression with no base case is a tail
|
||||
call, so it loops rather than nesting and only the clock ends it.
|
||||
|
||||
JSON has no literal for a non-finite number, so AWS writes each one as the string JavaScript names
|
||||
it by, wherever it lands: `1/0` is `"Infinity"`, `1e308 * 10` is `"Infinity"`, `0/0` is `"NaN"` and
|
||||
`$parseInteger("abc", "0")` is `"NaN"`, and nested, `[1/0]` is `["Infinity"]` and `{"k": 1/0}` is
|
||||
`{"k": "Infinity"}`.
|
||||
|
||||
One deviation, on the arithmetic half of that. A non-finite number a **function** answers is a value
|
||||
like any other here, so `$parseInteger("abc", "0")` is `"NaN"` on its own, inside an array and
|
||||
inside an object, as on AWS. One the **arithmetic** produces is not: the JSONata library refuses to
|
||||
carry it and raises instead, so Floci sees the value only in that refusal, after it has unwound
|
||||
whatever was being built around it. `1/0`, `-1/0` and `1e308 * 10` are answered because the
|
||||
expression's own result is what was refused; `[1/0]`, `{"k": 1/0}` and `$string(1/0)` fail the state
|
||||
with `States.QueryEvaluationError` where AWS answers a string, and `0/0` fails the field that holds
|
||||
it, because NaN is dropped as not-a-number without even a refusal to read it from. A state that
|
||||
fails is one a `Catch` fires on, which is the half of the divergence worth keeping.
|
||||
|
||||
## Execution names
|
||||
|
||||
A Standard execution name is unique per account, Region and state machine. Starting a Standard
|
||||
execution with a name and input that match one still running returns that original execution, so a
|
||||
retried call is idempotent; any other reuse of the name, whether a different input or a name whose
|
||||
execution has already closed, fails with `ExecutionAlreadyExists`.
|
||||
|
||||
One deviation. AWS frees a Standard name 90 days after the execution closes; Floci keeps it taken for
|
||||
the life of the emulator, so a closed name never becomes reusable on its own.
|
||||
|
||||
An Express execution name is not unique. Every `StartExecution` on an Express state machine is its
|
||||
own execution that runs alongside any others of the same name, and its ARN carries a per-start id
|
||||
after the name (`express:<stateMachine>:<name>:<id>`). Reusing an Express name never returns an
|
||||
earlier execution and never fails with `ExecutionAlreadyExists`.
|
||||
|
||||
One more. AWS keeps no record of an Express execution once it ends, only its logs, and does not serve
|
||||
`DescribeExecution`, `GetExecutionHistory` or `ListExecutions` for one. Floci keeps every Express
|
||||
execution, running or finished, in its execution store (and, in persistent mode, in
|
||||
`sfn-executions.json`), and nothing evicts them while the emulator runs. A workload that
|
||||
starts the same Express name in a tight loop therefore grows that store without bound.
|
||||
|
||||
## Nested workflows
|
||||
|
||||
A parent workflow calls a child workflow through one of several integrations, and they differ in
|
||||
more than syntax:
|
||||
|
||||
| Resource | Child type | A child that fails | Result |
|
||||
| --- | --- | --- | --- |
|
||||
| `arn:aws:states:::states:startExecution` | Standard | not awaited | `{executionArn, startDate}` |
|
||||
| `arn:aws:states:::states:startExecution.sync` | Standard | fails the calling task | execution envelope, `output` as a JSON string |
|
||||
| `arn:aws:states:::states:startExecution.sync:2` | Standard | fails the calling task | the child output, parsed |
|
||||
| `arn:aws:states:::aws-sdk:sfn:startExecution` | Standard | not awaited | `{ExecutionArn, StartDate}` |
|
||||
| `arn:aws:states:::aws-sdk:sfn:startSyncExecution` | Express | reported through `Status` | PascalCase envelope, `Output` as a JSON string |
|
||||
|
||||
`states:startExecution` and `aws-sdk:sfn:startExecution` are the same API through two different
|
||||
integrations, and only the casing of the result tells them apart.
|
||||
|
||||
`startSyncExecution` is the only one that does not fail the calling task when the child fails: the
|
||||
SDK call itself succeeded, so the task result carries `Status`, `Error` and `Cause` and the parent
|
||||
decides what to do next.
|
||||
|
||||
A `Name` a Standard child already used fails the calling task with the child's collision error, named
|
||||
for the integration that raised it: `StepFunctions.ExecutionAlreadyExistsException` through
|
||||
`states:startExecution` in any of its modes, and `Sfn.ExecutionAlreadyExistsException` through
|
||||
`aws-sdk:sfn:startExecution`. An Express child starts a new execution instead, so the same `Name`
|
||||
never fails it.
|
||||
|
||||
## AWS SDK task integrations
|
||||
|
||||
A resource of the form `arn:aws:states:::aws-sdk:<service>:<action>` calls the service's API and
|
||||
returns its response. Two conventions separate that result from the same API's wire response, and
|
||||
both are AWS's:
|
||||
|
||||
- **Field names are the SDK's**, so a `startDate` on the wire is a `StartDate` in the task result.
|
||||
- **A timestamp is an ISO-8601 string** such as `2026-08-28T20:34:59.712Z`, where the wire response
|
||||
carries epoch seconds.
|
||||
|
||||
A failure names the SDK exception class, which always ends in `Exception`:
|
||||
`StartExecution` answers a missing state machine with the error code `StateMachineDoesNotExist` on
|
||||
the wire and the task fails with `Sfn.StateMachineDoesNotExistException`.
|
||||
|
||||
| Resource | Result | Notable failure |
|
||||
| --- | --- | --- |
|
||||
| `arn:aws:states:::aws-sdk:sfn:startExecution` | `{ExecutionArn, StartDate}` | `Sfn.ExecutionAlreadyExistsException` when `Name` is reused on a Standard child |
|
||||
| `arn:aws:states:::aws-sdk:sfn:startSyncExecution` | execution envelope | `Sfn.StateMachineTypeNotSupportedException` for a Standard child |
|
||||
| `arn:aws:states:::aws-sdk:sfn:sendTaskSuccess` | `{}` | `Sfn.InvalidTokenException` when no task is waiting on the token |
|
||||
| `arn:aws:states:::aws-sdk:sfn:sendTaskFailure` | `{}` | `Sfn.InvalidTokenException` |
|
||||
| `arn:aws:states:::aws-sdk:scheduler:createSchedule` | `{ScheduleArn}` | `Scheduler.ConflictException` when the name is taken |
|
||||
| `arn:aws:states:::aws-sdk:scheduler:updateSchedule` | `{ScheduleArn}` | `Scheduler.ResourceNotFoundException` |
|
||||
| `arn:aws:states:::aws-sdk:scheduler:deleteSchedule` | `{}` | `Scheduler.ResourceNotFoundException` |
|
||||
| `arn:aws:states:::aws-sdk:sns:publish` | `{MessageId}` | `Sns.NotFoundException` when the topic does not exist |
|
||||
|
||||
Scheduler create and update tasks accept `StartDate` and `EndDate` as RFC 3339 strings, including
|
||||
offsets and fractional seconds. The direct Scheduler API continues to use numeric epoch seconds.
|
||||
|
||||
`sendTaskSuccess` and `sendTaskFailure` resolve a token a `.waitForTaskToken` task is parked on. A
|
||||
token nobody is waiting for fails the calling task rather than reporting a delivery that never
|
||||
happened.
|
||||
|
||||
## Publishing to SNS
|
||||
|
||||
`arn:aws:states:::sns:publish` calls the SNS Publish API with the task's parameters and returns the
|
||||
Publish response, `{MessageId}`. `TopicArn`, `TargetArn`, `PhoneNumber`, `Message`, `Subject`,
|
||||
`MessageStructure`, `MessageAttributes`, `MessageGroupId` and `MessageDeduplicationId` are the API's
|
||||
own fields, so a FIFO topic needs a `MessageGroupId` here just as it does from the SDK. A `Message`
|
||||
given as an object rather than a string is published as its JSON text, which is how a
|
||||
`.waitForTaskToken` task hands its token to the subscriber:
|
||||
|
||||
```json
|
||||
{
|
||||
"Type": "Task",
|
||||
"Resource": "arn:aws:states:::sns:publish.waitForTaskToken",
|
||||
"Parameters": {
|
||||
"TopicArn": "arn:aws:sns:us-east-1:000000000000:myTopic",
|
||||
"Message": {
|
||||
"Input.$": "$.message",
|
||||
"TaskToken.$": "$$.Task.Token"
|
||||
}
|
||||
},
|
||||
"End": true
|
||||
}
|
||||
```
|
||||
|
||||
A failure names the SDK exception class under the `SNS.` prefix, so a topic that does not exist
|
||||
fails the task with `SNS.NotFoundException` and a missing `Message` with
|
||||
`SNS.InvalidParameterException`. `arn:aws:states:::aws-sdk:sns:publish` is the same call under the
|
||||
`Sns.` prefix, as the AWS SDK integration table above shows.
|
||||
|
||||
## Publishing events
|
||||
|
||||
`arn:aws:states:::events:putEvents` returns the PutEvents response itself, `Entries` and
|
||||
`FailedEntryCount`, and one rejected entry fails the whole task with `EventBridge.FailedEntry`. The
|
||||
cause is the response serialized as a string, so a `Catch` can read which entry was rejected:
|
||||
|
||||
```json
|
||||
{"FailedEntryCount":1,"Entries":[{"EventId":"08cbdc46-…"},{"ErrorCode":"InvalidArgument","ErrorMessage":"EventBus not found: no-such-bus"}]}
|
||||
```
|
||||
|
||||
The optimized integration accepts `Detail` as a JSON object in JSONPath and JSONata workflows. It
|
||||
serializes that object once for the EventBridge request, preserving nested values and escaped text.
|
||||
The direct EventBridge API continues to accept its native string-valued `Detail` field.
|
||||
|
||||
One deviation, and it belongs to EventBridge rather than to the integration: Floci rejects an entry
|
||||
addressed to an event bus that does not exist, while AWS accepts it and returns an `EventId`.
|
||||
|
||||
## JSONata expressions are validated when the state machine is created
|
||||
|
||||
A JSONata expression runs with no context item: the execution input arrives as `$states.input`
|
||||
and a variable written by `Assign` as `$name`. A path that starts from neither reads a context
|
||||
item that does not exist, and AWS refuses the whole definition:
|
||||
|
||||
```
|
||||
An error occurred (InvalidDefinition) when calling the CreateStateMachine operation:
|
||||
Invalid State Machine Definition: 'UNSUPPORTED_JSONATA_EXPRESSION: Reference to 'phone' at the
|
||||
top level is not supported. at /States/E/Output/v'
|
||||
```
|
||||
|
||||
Floci refuses it too, from `CreateStateMachine`, `UpdateStateMachine` and
|
||||
`ValidateStateMachineDefinition`, with that message and that location. Write
|
||||
`$states.input.phone` to read the input and `$phone` to read a variable: an earlier `Assign` of
|
||||
`phone` does not put a bare `phone` in scope on AWS either.
|
||||
|
||||
Only the first step of a path is read against the top-level context, so a name in a later step,
|
||||
in a predicate, in a sort term or in an object grouping stays legal and
|
||||
`$states.input.items[value > 3]` is accepted. A lambda body keeps the context of the expression
|
||||
that defines it, so `$map($states.input.a, function($x){ b })` does name `b` at the top level.
|
||||
|
||||
A JSONata expression that fails to parse, such as `{% a[1,2) %}`, is refused the same way, with
|
||||
`INVALID_JSONATA_EXPRESSION` and the parser's own message at the field's location. `$$`, the
|
||||
reference to the top-level context, is refused under `UNSUPPORTED_JSONATA_EXPRESSION` with the
|
||||
message `Reference to '$$' is not supported.`; so is `$states.errorOutput` outside the one place it
|
||||
resolves, a catcher's own `Output` or `Assign`.
|
||||
|
||||
A definition is refused outright, with no state machine created, for a graph that never reaches a
|
||||
terminal state (`MISSING_END_STATE`), for a `StartAt`, `Next`, `Default` or `Catch[].Next` naming a
|
||||
state absent from its container or a state nothing transitions to (`MISSING_TRANSITION_TARGET`),
|
||||
and for a field the state type does not carry: `TimeoutSeconds` only on `Task`, and `Catch`/`Retry`
|
||||
only on `Task`, `Parallel` and `Map`.
|
||||
|
||||
## A state's QueryLanguage, and the fields it may carry
|
||||
|
||||
A state's query language is JSONPath when its `QueryLanguage` field is exactly the string
|
||||
`"JSONPath"`, the state machine's when the field is absent, and JSONata for any other value: the
|
||||
wrong case, an unknown string and a non-string alike. A state inside a `Map`'s `ItemProcessor` or `Iterator`, or
|
||||
inside one of a `Parallel`'s `Branches`, falls back to the **state machine's** language and not to
|
||||
the enclosing `Map`'s or `Parallel`'s, which the Amazon States Language calls independent of it. The
|
||||
enclosing state's own fields, `Items` and `MaxConcurrency` among them, do use its own language.
|
||||
|
||||
Three more refusals, each measured against `ValidateStateMachineDefinition` on real AWS:
|
||||
|
||||
- A field that belongs to the other query language, at the state's path with no field suffix:
|
||||
`The QueryLanguage is set to 'JSONPath', but field 'Output' is only supported for the 'JSONata'
|
||||
QueryLanguage`, and the mirror of it for `InputPath`, `OutputPath`, `ResultPath`,
|
||||
`ResultSelector`, `Parameters`, `Result`, `ItemsPath` and `MaxConcurrencyPath` on a JSONata
|
||||
state. `Assign` belongs to neither list: AWS accepts it on both.
|
||||
- A state declaring `"QueryLanguage": "JSONPath"` under a `JSONata` state machine:
|
||||
`'QueryLanguage' can not be 'JSONPath' if set to 'JSONata' for whole state machine`, again at the
|
||||
state's path. A JSONata machine cannot be reverted one state at a time; the upgrade in the other
|
||||
direction is allowed. That diagnostic is the whole answer for that state, so the fields its
|
||||
refused language forbids are not reported on top of it, while every other check still runs.
|
||||
- `QueryLanguage` on the `ItemProcessor`, `Iterator` or branch object itself, which is not a state:
|
||||
`Field 'QueryLanguage' is not supported` at `/States/M/ItemProcessor`.
|
||||
|
||||
A `QueryLanguage` value outside the enum is reported at the field, `/States/X/QueryLanguage` or
|
||||
`/QueryLanguage`, with `Value should be one of the following: [JSONPath, JSONata]`, and a
|
||||
non-string value with `Expected value of type [STRING]`. That diagnostic is independent of the
|
||||
resolution above, so `"jsonpath"` is a JSONata state that also carries it, and it is not a
|
||||
downgrade: only the exact string `"JSONPath"` under a JSONata machine is.
|
||||
|
||||
Locations follow AWS: the two messages above point at the state, `/States/X`, while every other
|
||||
schema error points at the offending field, `/States/X/MaxConcurrency`.
|
||||
|
||||
## Mocked service integrations
|
||||
|
||||
Floci supports the Step Functions Local mock configuration format
|
||||
(`MockConfigFile.json`). This lets a Task state return a predefined result or error
|
||||
instead of calling the integrated service. It is the standard way to unit test `Catch`
|
||||
and `Retry` branches, and it also lets you execute state machines whose integrations
|
||||
Floci does not implement yet.
|
||||
|
||||
Point `SFN_MOCK_CONFIG` at the mock configuration file and start an execution against
|
||||
`<stateMachineArn>#<testCaseName>`:
|
||||
|
||||
```bash
|
||||
# docker run -e SFN_MOCK_CONFIG=/tmp/mock.json -v ./MockConfigFile.json:/tmp/mock.json ...
|
||||
aws stepfunctions start-execution \
|
||||
--state-machine-arn "$SM_ARN#Throw422" \
|
||||
--input '{}' \
|
||||
--endpoint-url $AWS_ENDPOINT_URL
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"StateMachines": {
|
||||
"Test": { "TestCases": { "Throw422": { "Call API": "ApiFailure" } } }
|
||||
},
|
||||
"MockedResponses": {
|
||||
"ApiFailure": {
|
||||
"0": { "Throw": { "Error": "ApiGateway.422", "Cause": "Unprocessable" } }
|
||||
},
|
||||
"ApiSuccess": {
|
||||
"0-1": { "Return": { "StatusCode": 200, "ResponseBody": { "id": 1 } } }
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Each test case maps a state name to a `MockedResponses` entry. Each mocked response is
|
||||
keyed by retry attempt (`"0"`, `"1"`, or a range like `"1-2"`), so a state can fail on
|
||||
the first attempt and succeed on a retry. `Return` supplies the task result. `Throw`
|
||||
fails the task with the given `Error` and `Cause`, which flow through `Retry` and
|
||||
`Catch` unchanged. States not named in the test case run their real integration, so
|
||||
mocked and real service calls can be combined in one execution. The file is re-read
|
||||
when it changes, so it can be edited without restarting Floci.
|
||||
|
||||
Behavior was verified against Step Functions Local 2.0.0. As there, `StartSyncExecution`
|
||||
rejects a test case suffix with `UnsupportedOperation` and does not strip a bare trailing
|
||||
`#`, a bare trailing `#` on `StartExecution` runs the execution unmocked, and a retry
|
||||
attempt with no mocked entry fails the execution with `States.Runtime`. One intentional deviation: Floci reports an unknown test case and any
|
||||
invalid mock configuration (unparseable file, bad attempt key, missing `MockedResponses`
|
||||
entry, `Return` and `Throw` together, `Throw` without `Error`) as a structured 400 error
|
||||
at `StartExecution`. Step Functions Local instead returns a plain HTTP 500 for most of
|
||||
these and starts the execution only to fail it with `States.Runtime` for the last two.
|
||||
|
||||
A mocked response with no attempt entries (`{}`) is not rejected. As in Step Functions
|
||||
Local, the execution starts and fails with `States.Runtime` only if the state that names
|
||||
it is entered. This keeps a generated mock file usable when the collection it was built
|
||||
from is empty and the state is never reached.
|
||||
|
||||
## Executions abandoned by a restart
|
||||
|
||||
An execution runs in the Floci process. When Floci restarts while an execution is `RUNNING`,
|
||||
no worker survives it, so at startup every execution still stored as `RUNNING` is aborted with no
|
||||
error and no cause, the shape AWS returns for `StopExecution` called without them:
|
||||
`DescribeExecution` reports `status` `ABORTED` and a `stopDate`, and leaves the `error` and `cause`
|
||||
keys out. How many executions the sweep retired is reported once, as a `WARN` log line. Executions
|
||||
of every account are swept, each written back under its own account. Executions that already
|
||||
reached a terminal status are left untouched, and so is the status and `stopDate` of one this sweep
|
||||
aborted on an earlier boot.
|
||||
|
||||
Execution history is stored with the execution. While an execution is running, the current history
|
||||
is checkpointed every 100 events and when the execution reaches a terminal state. A graceful
|
||||
shutdown flushes the current execution state before the emulator stops. On restart, persisted
|
||||
history is retained, and a previously running execution is marked `ABORTED` with one
|
||||
`ExecutionAborted` event appended. After a further restart, the execution is already terminal, so
|
||||
no additional event is written.
|
||||
|
||||
## Configuration
|
||||
|
||||
| Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `FLOCI_SERVICES_STEPFUNCTIONS_ENABLED` | `true` | Enable or disable the service |
|
||||
| `SFN_MOCK_CONFIG` | unset | Path to a Step Functions Local compatible mock configuration file (alias: `FLOCI_SERVICES_STEPFUNCTIONS_MOCK_CONFIG_FILE`) |
|
||||
|
||||
## Examples
|
||||
|
||||
```bash
|
||||
export AWS_ENDPOINT_URL=http://localhost:4566
|
||||
|
||||
# Create a state machine
|
||||
SM_ARN=$(aws stepfunctions create-state-machine \
|
||||
--name my-workflow \
|
||||
--definition '{
|
||||
"Comment": "Simple workflow",
|
||||
"StartAt": "HelloWorld",
|
||||
"States": {
|
||||
"HelloWorld": {
|
||||
"Type": "Pass",
|
||||
"Result": {"message": "Hello, World!"},
|
||||
"End": true
|
||||
}
|
||||
}
|
||||
}' \
|
||||
--role-arn arn:aws:iam::000000000000:role/step-functions-role \
|
||||
--query stateMachineArn --output text \
|
||||
--endpoint-url $AWS_ENDPOINT_URL)
|
||||
|
||||
# Start an execution
|
||||
EXEC_ARN=$(aws stepfunctions start-execution \
|
||||
--state-machine-arn $SM_ARN \
|
||||
--input '{"key":"value"}' \
|
||||
--query executionArn --output text \
|
||||
--endpoint-url $AWS_ENDPOINT_URL)
|
||||
|
||||
# Check status
|
||||
aws stepfunctions describe-execution \
|
||||
--execution-arn $EXEC_ARN \
|
||||
--endpoint-url $AWS_ENDPOINT_URL
|
||||
|
||||
# Get event history
|
||||
aws stepfunctions get-execution-history \
|
||||
--execution-arn $EXEC_ARN \
|
||||
--endpoint-url $AWS_ENDPOINT_URL
|
||||
```
|
||||
Reference in New Issue
Block a user