TheCloud

TheCloud

Any Cloud. Locally.
A self-hosted cloud console for local AWS, Azure and GCP runtimes.

License: MIT

TheCloud's Cloud Explorer showing S3 buckets and the selected bucket's details

TheCloud is an AWS-Console-style web console for locally running cloud runtimes. It is centered on a unified `Cloud Explorer` and a cloud-aware `Console Home`, and it renders only real data returned by the runtimes, with explicit placeholders for work that is not wired yet. No fake resources, no demo rows, and no mock operational data are shown in normal mode. ## Quick Start AWS-only stack: ```bash docker compose up ``` Full multi-cloud stack: ```bash docker compose --profile multicloud up ``` Open [http://localhost:4500](http://localhost:4500). ## Exposing To The Internet TheCloud requires an administrator username, an Argon2id password hash, and a session secret. Copy `.env.example` to `.env`, generate the hash with `cd packages/api && bun run scripts/hash-password.ts ''`, and generate a session secret with `openssl rand -base64 48`. Do not put plaintext passwords in `.env`. Put only TheCloud's console behind Nginx Proxy Manager, terminating TLS there and forwarding to port `4500`. Keep `FLOCI_UI_SECURE_COOKIES=true` in that deployment. Bind TheCloud's AWS runtime (`4566`) to the Docker network or localhost only: it must not be publicly reachable, because the authenticated UI proxy is the intended access path. Configure Nginx Proxy Manager to pass `X-Forwarded-Proto` and `X-Forwarded-For`, and place Cloudflare or another edge proxy in front of it for public deployments. Set `FLOCI_UI_TRUSTED_PROXIES` to the proxy's IP or CIDR (comma-separated for several hops, e.g. Cloudflare's published ranges when it sits in front of Nginx Proxy Manager); `X-Forwarded-For` is ignored from any other peer, and without it sign-in throttling treats every visitor as the proxy. Set `FLOCI_UI_AUTH_MODE=oidc` to use an OIDC provider such as Authentik. Configure `FLOCI_UI_OIDC_ISSUER`, `FLOCI_UI_OIDC_CLIENT_ID`, `FLOCI_UI_OIDC_CLIENT_SECRET`, and `FLOCI_UI_PUBLIC_BASE_URL`; the callback is `${FLOCI_UI_PUBLIC_BASE_URL}/api/auth/oidc/callback`. OIDC mode also requires an allow-list: `FLOCI_UI_OIDC_ALLOWED_GROUPS` (matched against the `groups` claim, read from UserInfo when the ID token omits it) and/or `FLOCI_UI_OIDC_ALLOWED_EMAILS` (verified emails only). Set `FLOCI_UI_OIDC_ALLOW_ALL_USERS=true` only if every user of the identity provider should have console access. ## What The UI Actually Exposes Today The sidebar and Console Home are rendered from `GET /api/clouds/:cloud/services`, so this table is derived from the service catalog and the adapter registry rather than maintained by hand. Regenerate it after any change to either: ```bash cd packages/api && bun run scripts/service-matrix.ts ``` | Group | Service | AWS | Azure | GCP | |---|---|---|---|---| | Compute | Compute | Yes (list, inspect, create, delete) | Yes (list, inspect, create, delete) | No | | Compute | EKS / AKS / GKE | Yes (list, inspect) | Yes (list, inspect) | Yes (list, create, inspect, delete) | | Compute | Serverless | Yes (list, create, update, inspect, delete) | Runtime gap | Yes (list, create, inspect, delete) | | Compute | Containers / Cloud Run | No | No | Yes (list, create, delete, inspect) | | Compute | ECS | Yes (list, create, delete, inspect) | No | No | | Compute | Fargate | Yes (list, create, delete, inspect) | No | No | | Compute | Lightsail | Yes (list, create, inspect, update, delete) | No | No | | Developer Tools | CodeBuild | Yes (list, create, delete, inspect) | No | No | | Developer Tools | CodePipeline | Yes (list, create, delete, inspect) | No | No | | Developer Tools | CodeDeploy | Yes (list, create, delete, inspect) | No | No | | Storage | Storage | Yes (list, create, delete, inspect) | Yes (list, create, delete, inspect) | Yes (list, create, delete, inspect) | | Storage | ECR / Elastic Container Registry | Yes (list, create, delete, inspect) | No | No | | Databases | Database | Yes (list, create, update, delete, inspect) | Yes (list, create, delete, inspect) | Yes (list, create, inspect, delete) | | Databases | DynamoDB / Cosmos DB NoSQL / NoSQL | Yes (list, create, delete, inspect) | Yes (list, create, delete, inspect) | No | | Databases | ElastiCache | Yes (list, create, delete, inspect) | No | No | | Networking | Networking | Yes (list) | No | No | | Networking | ELB / Load Balancing | Yes (list, create, update, delete, inspect) | No | No | | Networking | ELB Target Groups / Target Groups | Yes (list, create, delete, inspect) | No | No | | Networking | Route 53 / DNS | Yes (list, create, delete, inspect) | No | No | | Networking | CloudFront / CDN | Yes (list, create, delete, inspect) | No | No | | Integration | SQS / Messaging / Pub/Sub | Yes (list, create, inspect, delete) | Yes (list, create, delete, inspect) | Yes (list, create, inspect, delete) | | Integration | API Gateway | Yes (list, create, delete, inspect) | No | No | | Integration | SNS / Notifications | Yes (list, create, inspect, delete) | No | No | | Integration | EventBridge / Events | Yes (list, create, delete, inspect) | No | No | | Integration | SES Mailbox / Email | Yes (list, inspect) | No | No | | Integration | Cloud Scheduler | No | No | Yes (list, create, delete, inspect) | | Integration | Step Functions / Workflows | Yes (list, create, update, delete, inspect) | No | No | | Analytics | MSK / Managed Streaming for Kafka | Yes (legacy page) | No | No | | Analytics | MSK Connect / Managed Streaming for Kafka Connect | Yes (legacy page) | No | No | | Analytics | Kinesis / Kinesis Data Streams | Yes (list, create, inspect, delete) | No | No | | Provisioning | CloudFormation / Infrastructure as Code | Yes (list, create, update, delete, inspect) | No | No | | Provisioning | AppConfig / Configuration | Yes (list, create, delete, inspect) | No | No | | Security | Identity | Yes (list, create, delete, inspect) | No | No | | Security | Cognito / User Directories | Yes (list, create, delete, inspect) | No | No | | Security | Secrets Manager / Key Vault / Secret Manager | Yes (list, create, inspect, delete) | Yes (list, create, delete, inspect) | Yes (list, create, inspect, delete) | | Security | KMS / Key Management | Yes (list, create, delete, inspect) | No | No | | Security | Parameter Store | Yes (list, create, delete, inspect) | No | No | | Observability | CloudWatch / Monitoring | Yes (list, create, delete, inspect) | No | No | | Observability | CloudWatch Logs / Logs | Yes (list, create, delete, inspect) | No | No | Console Home is available for all three clouds. Runtime gaps — an adapter exists but the local runtime does not implement it: - Azure Serverless: TheCloud's Azure runtime returns 501 NotImplemented for the Azure Functions endpoint. Services marked `No` render as a disabled sidebar row whose tooltip carries the server-supplied reason. Adding one is a catalog row in `packages/api/src/cloud-spi/serviceCatalog.ts` plus an adapter — no frontend change. With Azure selected, the nav is grouped the same way, `k8s Engine` is labelled `AKS` for that provider, and every unavailable service carries a reason: Serverless reads `coming soon` because the Azure runtime answers 501 for Azure Functions, even though an adapter is registered. ## Current Capability Snapshot
Storage Cloud Explorer storage is the most complete unified category today. - AWS S3 buckets are normalized as `storage` resources with type `bucket`. - Azure Blob containers are normalized as `storage` resources with type `container`. - GCP Cloud Storage buckets are normalized as `storage` resources with type `bucket`. - Shared resource table, shared inspector, runtime status strip, and schema-driven create/delete flows. - Object/blob browser with prefix navigation. - Upload, download, delete, copy, and create-folder-prefix actions. - Azure folder markers are hidden and rendered as folders in the browser. - Size and last-modified metadata are shown when returned by the runtime. Current gaps: - No bulk multi-select actions yet. - No tag/policy/version management in the unified view. - Folder creation is prefix-based, not a real filesystem directory.
k8s Engine All three clouds, through the unified shell. - AWS EKS and Azure AKS clusters can be listed and inspected. - A selected EKS cluster lists its managed nodegroups and Fargate profiles. - Create and delete managed nodegroups, including role, subnets, instance types, and scaling configuration. - Create and delete Fargate profiles, including pod execution role, selectors, labels, and optional subnets. - These nested EKS operations use the unified Cloud Proxy, not the legacy `/api/eks/*` routes. - GCP GKE clusters can additionally be created and deleted. - Cluster metadata, node groups, and related details are surfaced when returned by the runtime. Current gaps: - EKS and AKS are read-only. On AKS this is a runtime limit rather than a choice: the shipped Azure runtime (`floci-az`) config runs AKS unmocked with no Docker socket to start k3s with, so a created cluster never leaves `provisioningState: Failed`.
Database Relational and document database workflows across providers: - AWS RDS: list, inspect, create, update, and delete DB instances (PostgreSQL, MySQL, MariaDB) with provider defaults (class `db.t3.micro`, storage 20 GB, username `root`). Updates use generic `PATCH /api/clouds/:cloud/services/:service/resources/:id` mapping to `ModifyDBInstance` for password rotation, IAM authentication, DB subnet group, VPC security groups, option group, and auto minor version upgrade. Instance class, storage, engine, and version are omitted from edit operations because TheCloud's RDS runtime does not support modifying them yet. - AWS RDS Snapshots: account-scoped Snapshots tab listing DB snapshots and supporting snapshot creation. - Azure Cosmos DB NoSQL: database, container, and document workflows. - AWS DynamoDB: table management, item browsing, and Add record. - Azure SQL and PostgreSQL Flexible Server: instance management and SQL query editor. - GCP Cloud SQL: list, inspect, create, and delete database instances. Cosmos DB includes: - List, create, and delete databases. - List, create, and delete containers. - Create, edit, and delete documents/items. - SQL query editor for documents. Choose **Explore data** beside a supported resource to open its dedicated workspace at `/cloud-explorer/:cloud/:service/:resourceId/data`. DynamoDB records, Cosmos containers and documents, and Azure SQL/PostgreSQL tables and query results use this workspace. Cosmos container and SQL database/schema/table selections remain in the URL for bookmarks and browser history. SQL credentials remain in memory and must be entered again after a reload. **Back to** returns to the service's resource-management list. Frontend regression tests use mocked `/api/*` responses and need no running emulator: `pnpm --filter @floci/frontend exec playwright install chromium`, then `pnpm --filter @floci/frontend test:e2e`. Current gaps: - AWS RDS snapshot creation: the Cloud Proxy operation is available, but TheCloud's runtime does not implement `CreateDBSnapshot` (returns a typed 501 `operation_not_implemented`). Snapshot listing returns a valid empty list. - AWS RDS instance stop/start operations are not implemented in TheCloud's runtime yet.
Compute AWS only, through the unified shell plus AWS-specific panels where the workflow is too rich for a flat generic form. - List EC2 instances and AMIs as normalized resources. - Launch instances. - Start, stop, reboot, and terminate instances. - Create AMIs. - Edit tags. - View console output. Current gaps: - No Azure VM or GCP compute adapter yet. - Compute creation still uses an AWS-specific panel because it needs dependent selectors.
Networking AWS only, through the unified shell plus an AWS-specific networking panel. - VPC list and inspect through the unified resource table. - VPC creation and delete, the VPC wizard, subnets, security groups, internet gateways, NAT gateways, route tables, and Elastic IP workflows — all in the Networking panel. Current gaps: - No Azure VNet or GCP VPC adapter yet. - Create and delete are advertised as `partial` in the unified schema and are handled by the Networking panel, because they need dependent selectors that a flat generic form cannot express. - Advanced multi-cloud networking normalization is still pending.
Identity AWS only, through the generic identity service category. - List and inspect IAM users. - Create and delete IAM users. - IAM user paths are supported during creation. Current gaps: - Roles, groups, policies, access keys, and other advanced IAM workflows are not exposed yet. - No Azure or GCP identity adapter yet.
API Gateway AWS only, through the generic apigateway service category. REST APIs (v1) and HTTP and WebSocket APIs (v2) are listed together, with a Type column. - REST APIs: create, delete and inspect; add Lambda proxy routes on a resource tree, enable CORS, test a method with TestInvokeMethod, and send a request to a deployed stage. - HTTP APIs: quick-create with a Lambda (a catch-all route on an auto-deployed `$default` stage); add routes such as `GET /notes/{id}` to a Lambda or an HTTP backend; add stages with or without automatic deployment and deploy them; add JWT authorizers, filled in from a Cognito user pool and app client or from any OpenID Connect issuer, and protect routes with them; configure CORS; send a real request to a stage through its routes and authorizers. - WebSocket APIs: create with a route selection expression and a stage; add `$connect`, `$disconnect`, `$default` and action routes to Lambda, optionally sending the function's response back; connect to a stage, send a message and read the replies. Verified against TheCloud's runtime: quick create, Lambda and HTTP routes, a Cognito JWT authorizer (401 without a token, the token's claims reach the Lambda with one), CORS preflight, a stage deployed on demand, and a WebSocket round trip. Current gaps: - WebSocket APIs: Lambda (REQUEST) authorizers, `@connections` messages pushed from a backend, and route request and response models are not exposed. - Custom domain names, API mappings, VPC links, usage plans and API keys are not exposed. - No Azure or GCP API Gateway adapter yet.
Email / SES Mailbox AWS SES email capture through the unified Cloud Explorer. - Lists emails actually captured by TheCloud's SES. - Filters by subject, sender, and recipient. - Inspects sender, recipients, timestamp, and message type. - Displays HTML in a sandboxed preview, text bodies, and captured raw MIME data. - Clears the captured inbox after an explicit confirmation. Manual verification with the AWS CLI: ```bash export AWS_ACCESS_KEY_ID=test export AWS_SECRET_ACCESS_KEY=test export AWS_DEFAULT_REGION=us-east-1 aws ses send-email \ --endpoint-url http://localhost:4566 \ --from sender@example.test \ --destination 'ToAddresses=recipient@example.test' \ --message 'Subject={Data="TheCloud SES test",Charset=utf-8},Body={Text={Data="Plain-text test email.",Charset=utf-8},Html={Data="

Hello from TheCloud

This should render in the SES preview.

",Charset=utf-8}}' ``` The email is captured by TheCloud's runtime, and is delivered externally only when an SMTP relay is configured. Open `/cloud-explorer/aws/email` and refresh the mailbox to inspect its Preview, Text, and Raw views. You can also inspect the captured messages directly with: ```bash curl http://localhost:4566/_aws/ses ``` Current gaps: - Sending a test email from the UI is not wired yet; applications continue to send through their AWS SES SDK. - SES identities, templates, bulk email, configuration sets, and suppression lists are not exposed yet. - No Azure or GCP email adapter yet.
Serverless AWS and GCP, both through the unified shell. - AWS Lambda and GCP Cloud Functions list, create, inspect, and delete. - AWS Lambda invoke is wired, including the tailed execution log and handler errors. - Lambda creation packages inline code into a real deployment archive. - The navigation entry appears for any cloud with a registered adapter. Current gaps: - Azure Functions is registered but the Azure runtime answers 501 NotImplemented, so it reports `coming_soon` with that reason rather than appearing available. - GCP Cloud Functions invoke is not wired yet; the capability is advertised as `coming_soon` instead of being silently missing. - Old AWS Lambda page is gone; all future work should stay in the unified model.
Containers GCP Cloud Run, through the unified shell. - List, inspect, deploy, and delete Cloud Run services. - Image, container port, URL, traffic split, and generation are surfaced. - Deploying really starts a container: the runtime launches the requested image. Readiness is reported honestly. A deploy settles at `PENDING` and then becomes `SUCCEEDED` or `FAILED`; the runtime's own explanation is kept in `metadata.terminalMessage`. The usual cause of `FAILED` is a container that does not listen on the port given by `$PORT` (8080 by default) — `nginx:alpine` listens on 80 and fails for exactly that reason, so the create form says so. Current gaps: - No AWS ECS or Azure Container Apps adapter yet. - No revision history, traffic splitting, or scaling controls. - Deleting a service while it is still `PENDING` can race the create; deleting a settled service is durable.
Secrets Manager This is the only dedicated AWS page still outside Cloud Explorer. - List secrets. - Inspect metadata. - Reveal current value on demand. - Create secrets. - Update values. - Delete secrets, including force delete. Current gaps: - Not migrated into the Cloud Explorer contract yet. - No Azure or GCP secret adapter yet.
## Product Direction TheCloud is a metadata-driven, cloud-aware console: one web app renders multiple local runtimes through the same shell. The guiding rules are: - The UI does not know clouds. - The proxy does not know internal implementations. - The SPI defines the contracts. - The adapters perform the translation. - The runtimes execute the real behavior. ## Architecture ![TheCloud architecture: the console calls TheCloud's API, which dispatches through one proxy and an adapter registry to AWS, Azure and GCP adapters and their local runtimes](docs/images/thecloud-architecture.svg) Short implementation notes live in [docs/implementation-notes.md](docs/implementation-notes.md). ## Project Structure ```text packages/ api/ src/ cloud-spi/ registry/ adapter-aws/ adapter-azure/ adapter-gcp/ routes/ service/ frontend/ src/ api/ components/ features/ pages/ ``` High-level runtime flow: ```text Browser -> frontend (React/Vite) -> /api/clouds/* -> Cloud Adapter Registry -> provider adapter -> local runtime ``` ## Setup ### Docker Compose Default compose stack: - `floci-ui` (the console) on `http://localhost:4500` - `floci-api` (TheCloud's API) on `http://localhost:4501` - `floci` (TheCloud's AWS runtime) on `http://localhost:4566` Start AWS-only: ```bash docker compose up ``` Start AWS + Azure + GCP: ```bash docker compose --profile multicloud up ``` Convenience targets: ```bash make up make up-multicloud make down make logs ``` ### Manual Local Development Prerequisites: - Node.js 20+ - pnpm 9+ - Bun - A running local runtime: TheCloud's AWS runtime, and optionally the Azure and GCP runtimes Install dependencies: ```bash pnpm install ``` Configure the API environment: ```bash cp .env.example packages/api/.env ``` Important: the API runs from `packages/api` and loads environment variables from `packages/api/.env`. Start TheCloud's AWS runtime from its repository: ```bash git clone https://gitea.zacksolutions.dev/Zakaria/FlociClone.git ../floci cd ../floci docker compose up -d # or ./mvnw clean quarkus:dev with JDK 25 ``` Optional local runtimes: - Azure runtime (`floci-az`) on `http://localhost:4577` - GCP runtime (`floci-gcp`) on `http://localhost:4588` Start the UI stack: ```bash pnpm dev ``` That starts: - frontend on `http://localhost:4500` - API on `http://localhost:4501` Split commands: ```bash pnpm dev:api pnpm dev:web ``` ## Environment Default API environment values: ```bash FLOCI_ENDPOINT=http://localhost:4566 FLOCI_AZURE_ENDPOINT=http://localhost:4577 FLOCI_AZURE_ACCOUNT_NAME=devstoreaccount1 FLOCI_GCP_ENDPOINT=http://localhost:4588 FLOCI_GCP_PROJECT=floci-local AWS_REGION=us-east-1 AWS_ACCESS_KEY_ID=test AWS_SECRET_ACCESS_KEY=test PORT=4501 ``` `VITE_MOCK_MODE=false` is kept in `.env.example`, but the current app is intended to run against real local runtimes. ## Verification ```bash pnpm lint pnpm type-check pnpm test pnpm build ``` ## Troubleshooting ### `http proxy error` or `ECONNREFUSED` on `/api/*` The frontend is up, but the API is not reachable on `http://localhost:4501`. Check: ```bash pnpm dev:api curl http://localhost:4501/api/clouds ``` ### `EADDRINUSE` on port `4501` Another API process is already running. Stop it first or kill the process holding port `4501`. ### Runtime shows `Not connected` or `Runtime unavailable` Check the runtime directly: ```bash curl http://localhost:4566/_floci/health curl http://localhost:4577/_floci/health curl http://localhost:4588/_floci-gcp/health curl http://localhost:4501/api/clouds/aws/status curl http://localhost:4501/api/clouds/azure/status curl http://localhost:4501/api/clouds/gcp/status ``` ### A single service shows as unavailable while the cloud is connected Cloud status reflects the runtime; each service is probed separately. Ask which service is failing and why: ```bash curl http://localhost:4501/api/clouds/azure/status?services=all curl http://localhost:4501/api/clouds/azure/services/serverless/status ``` `errorCode` distinguishes the cases: `operation_not_implemented` means the local runtime does not implement that service, `runtime_unavailable` means it cannot be reached, and `operation_not_supported` means no adapter is registered. ### Credentials or endpoint mismatch For AWS local development, keep API credentials aligned with the runtime: ```bash AWS_ACCESS_KEY_ID=test AWS_SECRET_ACCESS_KEY=test ``` ## Contributing When adding new UI surface: - Prefer the Cloud Explorer and Cloud Proxy model over new legacy pages. - Reuse the SPI contracts before creating provider-specific response shapes. - Keep placeholders explicit instead of inventing fake data. - Update this README when the visible UI surface changes. ## License [MIT](LICENSE). TheCloud builds on the open-source [Floci](https://floci.io) project.