# Amazon SageMaker

Floci emulates SageMaker control-plane APIs and runs real Docker containers for local training jobs and hosted endpoints. Containers use the SageMaker `/opt/ml` contract rather than mocks.

## Supported operations

| Area | Operations |
| --- | --- |
| Models | `CreateModel`, `DescribeModel`, `DeleteModel`, `ListModels` |
| Endpoint configs | `CreateEndpointConfig`, `DescribeEndpointConfig`, `DeleteEndpointConfig`, `ListEndpointConfigs` |
| Endpoints | `CreateEndpoint`, `DescribeEndpoint`, `UpdateEndpoint`, `DeleteEndpoint`, `ListEndpoints` |
| Training | `CreateTrainingJob`, `DescribeTrainingJob`, `ListTrainingJobs`, `StopTrainingJob` |
| Tags | `AddTags`, `ListTags`, `DeleteTags` |
| Runtime | `POST /endpoints/{EndpointName}/invocations` |

## Supported Actions

<!-- floci:actions:start -->
| Action | Description |
| --- | --- |
| `CreateModel` | Registers a model from `PrimaryContainer` or `Containers`; rejects an empty `Containers` array rather than dereferencing it. |
| `DescribeModel` | Returns a model by name, scoped to the calling region. |
| `DeleteModel` | Deletes a model by name, scoped to the calling region. |
| `ListModels` | Lists models for the calling region; supports `NameContains`, `MaxResults` (1-100), and `NextToken` pagination. |
| `CreateEndpointConfig` | Creates an endpoint configuration; every `ProductionVariants[].ModelName` must already exist. |
| `DescribeEndpointConfig` | Returns an endpoint configuration by name, scoped to the calling region. |
| `DeleteEndpointConfig` | Deletes an endpoint configuration by name, scoped to the calling region. |
| `ListEndpointConfigs` | Lists endpoint configurations for the calling region; supports `MaxResults` and `NextToken`. |
| `CreateEndpoint` | Starts hosting the configured model image asynchronously; the endpoint is `Creating` until the container passes its `/ping` check. |
| `DescribeEndpoint` | Returns an endpoint's status, scoped to the calling region. |
| `DeleteEndpoint` | Stops and removes the endpoint's container; a start still in flight for this endpoint discards its result instead of resurrecting the deleted record. |
| `ListEndpoints` | Lists endpoints for the calling region; supports `MaxResults` and `NextToken`. |
| `UpdateEndpoint` | Swaps an endpoint to a new endpoint configuration, restarting its container; a superseded in-flight start discards its result. |
| `CreateTrainingJob` | Runs `AlgorithmSpecification.TrainingImage` asynchronously against the `/opt/ml` contract and uploads the resulting model artifacts to S3. |
| `DescribeTrainingJob` | Returns a training job's status, scoped to the calling region. |
| `ListTrainingJobs` | Lists training jobs for the calling region; supports `NameContains`, `StatusEquals`, `MaxResults`, and `NextToken`. |
| `StopTrainingJob` | Stops a training job that is `InProgress` or `Stopping`; a no-op against a job already in a terminal state. |
| `AddTags` | Adds tags to a model, endpoint config, endpoint, or training job identified by ARN. |
| `ListTags` | Lists tags for a model, endpoint config, endpoint, or training job identified by ARN. |
| `DeleteTags` | Removes tags by key from a model, endpoint config, endpoint, or training job identified by ARN. |
<!-- floci:actions:end -->

## Training contract

`CreateTrainingJob` starts `AlgorithmSpecification.TrainingImage` with command `train` unless `ContainerEntrypoint`/`ContainerArguments` are supplied. Floci writes SageMaker config files under `/opt/ml/input/config`, downloads channel data from S3 into `/opt/ml/input/data/<channel>`, waits for container exit, and uploads `/opt/ml/model` as `model.tar.gz` under `OutputDataConfig.S3OutputPath/<TrainingJobName>/output/`.

## GPU-backed training

On AWS, `ResourceConfig.InstanceType` *is* the hardware: `ml.g5.xlarge` provisions a machine that physically has one A10G. Floci has one host with whatever cards it has, so a GPU instance type is served by substituting a local device. That substitution is off by default and the SDK request is unchanged either way.

How many accelerators each `ml.*` type carries is a property of AWS, so it ships with Floci in `sagemaker/instance-type-catalog.yaml`. Configuration covers only the local decision of which of this machine's devices may be handed out:

```yaml
floci:
  services:
    sagemaker:
      gpu:
        enabled: true
        mode: cdi            # cdi | device-ids | count
        devices:
          - nvidia.com/gpu=GPU-0e1f2a3b
```

| Setting | Meaning |
| --- | --- |
| `enabled` | Off by default. While off, every training container is CPU-only, exactly as before. |
| `mode` | How the request reaches the daemon. `cdi` names [Container Device Interface](https://github.com/cncf-tags/container-device-interface) devices, `device-ids` uses daemon device ids, and `count` asks for a number and lets the daemon choose. |
| `devices` | The devices Floci may use, for `cdi` and `device-ids`. Unset allows none, so a shared machine does not hand over every GPU by default. Ignored by `count`. |

`mode` defaults to `cdi` because daemons differ: Podman resolves only CDI and will accept a count request while attaching no device ([containers/podman#22645](https://github.com/containers/podman/issues/22645)). Choosing CDI means a misconfiguration fails the container start rather than quietly training on CPU. Use `count` or `device-ids` against Docker.

Floci does not reserve configured devices across concurrent training jobs. In `cdi` and `device-ids` modes, each job takes the first devices it needs from the allowlist, so two simultaneous one-GPU jobs can target the same local card and contend for memory.

A training job fails with an explanatory `FailureReason`, rather than running on CPU, when:

- the instance type belongs to a GPU family (`ml.g*`, `ml.p*`, `ml.inf*`, `ml.trn*`) that the catalog does not list, so its accelerator count is unknown
- fewer devices are allowed than the instance type needs
- `cdi` or `device-ids` is selected with no `devices` configured
- `InstanceCount` is above one, since Floci runs a job as a single container

Only the accelerator is substituted. An instance type also implies vCPU and memory, which Floci does not emulate for any service: EC2 instance types are metadata for `DescribeInstanceTypes` and do not size containers, and RDS records `DBInstanceClass` without acting on it. Resource limits are applied only where an API supplies an explicit number, as Lambda's `MemorySize` does.

## Endpoint hosting

`CreateEndpoint` starts the model image as a long-lived Docker container with command `serve`, port `8080`, `/ping` health checks, and `/invocations` runtime proxying. `ModelDataUrl` artifacts are downloaded from S3 and placed in `/opt/ml/model`.

Endpoint containers are CPU-only for now; the `gpu` settings above apply to training jobs.

## Examples

```python
import boto3
sm = boto3.client("sagemaker", endpoint_url="http://localhost:4566", region_name="us-east-1")
sm.create_model(ModelName="m", PrimaryContainer={"Image":"my-image"}, ExecutionRoleArn="arn:aws:iam::000000000000:role/r")
sm.create_endpoint_config(EndpointConfigName="cfg", ProductionVariants=[{"VariantName":"AllTraffic","ModelName":"m","InitialInstanceCount":1,"InstanceType":"ml.t2.medium"}])
sm.create_endpoint(EndpointName="ep", EndpointConfigName="cfg")
```

```bash
aws --endpoint-url=http://localhost:4566 sagemaker list-models
```