Compare commits
1 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Hongming Wang
|
75dab2c6dc |
@@ -9,16 +9,18 @@ The `workspace/` directory is Molecule AI's unified runtime image. Every provisi
|
||||
|
||||
## Runtime Matrix In Current `main`
|
||||
|
||||
Current `main` ships six adapters:
|
||||
Current `main` ships eight adapters:
|
||||
|
||||
- `langgraph`
|
||||
- `deepagents`
|
||||
- `claude-code`
|
||||
- `langgraph`
|
||||
- `crewai`
|
||||
- `autogen`
|
||||
- `deepagents`
|
||||
- `hermes`
|
||||
- `gemini-cli`
|
||||
- `openclaw`
|
||||
|
||||
This is the merged runtime surface today. Branch-level experiments such as NemoClaw are separate and should be treated as roadmap/WIP, not merged support.
|
||||
This is the merged runtime surface today. The canonical allowlist lives in `workspace-server/internal/handlers/admin_workspace_images.go` (`AllRuntimes`). Anything not on this list — including BYO runtimes — registers via the [external workspace](../external-agents.mdx) path, not as a built-in adapter.
|
||||
|
||||
Adapter-specific behavior is documented in [Agent Runtime Adapters](./cli-runtime.md).
|
||||
|
||||
|
||||
@@ -77,8 +77,7 @@ The Platform is the central control plane responsible for:
|
||||
| `REDIS_URL` | (required) | Redis connection string |
|
||||
| `PORT` | `8080` | Server listen port |
|
||||
| `PLATFORM_URL` | `http://host.docker.internal:PORT` | URL passed to agent containers |
|
||||
| `SECRETS_ENCRYPTION_KEY` | (optional) | AES-256 key, 32 bytes — static-mode envelope encryption (dev/self-host). Set `KMS_KEY_ARN` instead for production AWS KMS envelopes. |
|
||||
| `KMS_KEY_ARN` | (optional) | AWS KMS CMK ARN — production envelope encryption with per-secret data keys |
|
||||
| `SECRETS_ENCRYPTION_KEY` | (optional) | AES-256 key, 32 bytes |
|
||||
| `CORS_ORIGINS` | `http://localhost:3000,http://localhost:3001` | Allowed CORS origins |
|
||||
| `RATE_LIMIT` | `600` | Requests per minute |
|
||||
| `MOLECULE_ENV` | (optional) | Set `production` to hide test endpoints |
|
||||
|
||||
@@ -80,7 +80,7 @@ CREATE TABLE workspace_secrets (
|
||||
);
|
||||
```
|
||||
|
||||
Stores API keys, credentials, and other secrets needed by workspace agents. Values are sealed with envelope encryption — AWS KMS (per-secret data keys via `GenerateDataKey`, AES-256-GCM payload) when `KMS_KEY_ARN` is configured (production), or static-key AES-256-GCM under `SECRETS_ENCRYPTION_KEY` (dev / self-host). The static key, when used, is read from the platform environment and is never stored in the database. Both modes coexist during a KMS cutover, distinguished by a v2 prefix byte on KMS blobs. Implementation: `workspace-server/internal/crypto/envelope.go`.
|
||||
Stores API keys, credentials, and other secrets needed by workspace agents. Values are encrypted with AES-256 at the application layer. The encryption key comes from the `SECRETS_ENCRYPTION_KEY` environment variable on the platform — never stored in the database.
|
||||
|
||||
The provisioner reads secrets from this table, decrypts them, and injects them as environment variables when spinning up workspace containers. Secrets are never included in bundles (see [Constraints — Rule 5](../development/constraints-and-rules.md)).
|
||||
|
||||
|
||||
@@ -149,7 +149,7 @@ Six runtime adapters ship production-ready on `main`: LangGraph, DeepAgents, Cla
|
||||
- Event broadcasting (Redis pub/sub → WebSocket fanout)
|
||||
- Docker provisioner with T1–T4 tier enforcement
|
||||
- Activity logging with configurable retention (default 7 days)
|
||||
- Secrets management (KMS-envelope encryption in prod; AES-256-GCM static-key mode for dev/self-host)
|
||||
- Secrets management (AES-256-GCM encryption)
|
||||
- File, terminal, bundle, template, traces APIs
|
||||
- Langfuse integration
|
||||
- Prometheus metrics endpoint
|
||||
@@ -186,7 +186,7 @@ Six runtime adapters ship production-ready on `main`: LangGraph, DeepAgents, Cla
|
||||
|-------|---------|-------------|
|
||||
| `workspaces` | Current state registry | `id`, `name`, `role`, `tier` (1-4), `status`, `parent_id`, `agent_card` (JSONB), `url`, `forwarded_to`, `last_heartbeat_at`, `last_error_rate`, `active_tasks`, `uptime_seconds`, `current_task`, `runtime` |
|
||||
| `agents` | Agent assignment history | `workspace_id`, `model`, `status`, `removed_at`, `removal_reason` |
|
||||
| `workspace_secrets` | Encrypted credentials | `workspace_id`, `key`, `encrypted_value` (BYTEA — KMS envelope in prod, AES-256-GCM static-key blob in dev/self-host) |
|
||||
| `workspace_secrets` | Encrypted credentials | `workspace_id`, `key`, `encrypted_value` (BYTEA, AES-256-GCM) |
|
||||
| `agent_memories` | HMA-scoped memory | `workspace_id`, `content`, `scope` (LOCAL/TEAM/GLOBAL) |
|
||||
| `structure_events` | **Immutable** event log (APPEND-ONLY, never UPDATE/DELETE) | `event_type`, `workspace_id`, `agent_id`, `target_id`, `payload` (JSONB) |
|
||||
| `activity_logs` | Operational activity with retention | `workspace_id`, `activity_type`, `source_id`, `target_id`, `method`, `request_body`, `response_body`, `duration_ms`, `status`, `error_detail` |
|
||||
@@ -573,14 +573,16 @@ compliance:
|
||||
|
||||
| Adapter | Core Strength | Image Tag |
|
||||
|---------|--------------|-----------|
|
||||
| **LangGraph** | Graph-based state machine, tool use, streaming | `workspace-template:langgraph` |
|
||||
| **DeepAgents** | Deep planning, multi-step task decomposition | `workspace-template:deepagents` |
|
||||
| **Claude Code** | Native coding workflows, CLI continuity, OAuth auth | `workspace-template:claude-code` |
|
||||
| **LangGraph** | Graph-based state machine, tool use, streaming | `workspace-template:langgraph` |
|
||||
| **CrewAI** | Role-based crews, structured task orchestration | `workspace-template:crewai` |
|
||||
| **AutoGen** | Multi-agent conversations, explicit strategies | `workspace-template:autogen` |
|
||||
| **DeepAgents** | Deep planning, multi-step task decomposition | `workspace-template:deepagents` |
|
||||
| **Hermes** | Multi-provider dispatch (Anthropic/Gemini native + OpenAI-compatible shim) | `workspace-template:hermes` |
|
||||
| **Gemini CLI** | Google Gemini CLI workspace | `workspace-template:gemini-cli` |
|
||||
| **OpenClaw** | CLI-native runtime, own session model | `workspace-template:openclaw` |
|
||||
|
||||
**Branch-level WIP**: NemoClaw (NVIDIA T4 + Docker socket) on `feat/nemoclaw-t4-docker`.
|
||||
The canonical allowlist lives in `workspace-server/internal/handlers/admin_workspace_images.go` (`AllRuntimes`). Anything outside this list registers via the external-workspace path.
|
||||
|
||||
Each adapter implements `setup()` + `create_executor()`. The base adapter provides shared infrastructure: system prompt assembly, skill loading, tool registration, coordinator detection, plugin injection.
|
||||
|
||||
@@ -822,7 +824,7 @@ workspace-server/
|
||||
│ ├── events/ # 3 files — event broadcasting + Postgres persistence
|
||||
│ ├── router/ # 2 files — route definitions + middleware
|
||||
│ ├── db/ # 6 files — Postgres + Redis drivers, migrations
|
||||
│ └── crypto/ # 2 files — envelope encryption (KMS or AES-256-GCM static key)
|
||||
│ └── crypto/ # 2 files — AES-256-GCM secrets encryption
|
||||
└── migrations/ # 11 SQL migration files
|
||||
```
|
||||
|
||||
@@ -905,8 +907,7 @@ Postgres + Redis + Langfuse only (for local development without containerized wo
|
||||
| `REDIS_URL` | `redis://localhost:6379` | Redis connection |
|
||||
| `PORT` | `8080` | Platform listen port |
|
||||
| `PLATFORM_URL` | `http://host.docker.internal:8080` | Injected to workspace containers |
|
||||
| `SECRETS_ENCRYPTION_KEY` | Optional | AES-256 key (32 bytes) for static-mode secret encryption — used when `KMS_KEY_ARN` is unset (dev/self-host) or to decrypt legacy blobs during a KMS cutover |
|
||||
| `KMS_KEY_ARN` | Optional | AWS KMS CMK ARN — when set, secrets use KMS envelope encryption (per-secret data keys via `GenerateDataKey`); production deployments use this path |
|
||||
| `SECRETS_ENCRYPTION_KEY` | Optional | AES-256 key (32 bytes) for secret encryption |
|
||||
| `CONFIGS_DIR` | `/configs` | Workspace config template directory |
|
||||
| `PLUGINS_DIR` | `/plugins` | Shared plugin directory |
|
||||
| `ACTIVITY_RETENTION_DAYS` | `7` | Activity log retention |
|
||||
@@ -950,7 +951,7 @@ Postgres + Redis + Langfuse only (for local development without containerized wo
|
||||
|---------|-------------|
|
||||
| **A2A streaming response** | Real-time task result delivery via SSE (`message/sendSubscribe`) |
|
||||
| **Onboarding wizard** | 4-step guided first-run experience in Canvas |
|
||||
| **Global API keys** | Platform-wide secrets with per-workspace override; KMS envelope encryption in prod (AES-256-GCM static-key mode in dev/self-host) |
|
||||
| **Global API keys** | Platform-wide secrets with per-workspace override + AES-256 encryption |
|
||||
| **Coordinator enforcement** | Team leads cannot do work, only route and aggregate |
|
||||
| **Cascade pause/resume** | Pausing a parent cascades to all children; paused children can't be individually resumed |
|
||||
| **Graceful A2A errors** | `[A2A_ERROR]` sentinel + retry with exponential backoff + fallback |
|
||||
@@ -979,7 +980,6 @@ Tools call `resp.json()` without catching JSON decode errors. Should wrap in try
|
||||
|
||||
| Branch | Feature | Status |
|
||||
|--------|---------|--------|
|
||||
| `feat/nemoclaw-t4-docker` | NemoClaw adapter (NVIDIA T4 support) | WIP |
|
||||
| Backlog | Firecracker backend (faster cold starts) | Planned |
|
||||
| Backlog | E2B backend (cloud-hosted code sandbox) | Planned |
|
||||
| Backlog | pgvector semantic memory search | Planned |
|
||||
|
||||
@@ -20,7 +20,7 @@ Canvas (Next.js :3000) ←WebSocket→ Platform (Go :8080) ←HTTP→ Postgres +
|
||||
|
||||
- **Workspace Server** (`workspace-server/`): Go/Gin control plane — workspace CRUD, registry, discovery, WebSocket hub, liveness monitoring.
|
||||
- **Canvas** (`canvas/`): Next.js 15 + React Flow (@xyflow/react v12) + Zustand + Tailwind — visual workspace graph.
|
||||
- **Workspace Runtime** (`workspace/`): Shared runtime published as [`molecule-ai-workspace-runtime`](https://pypi.org/project/molecule-ai-workspace-runtime/) on PyPI. Supports LangGraph, Claude Code, OpenClaw, DeepAgents, CrewAI, AutoGen. Each adapter lives in its own standalone template repo (e.g. `molecule-ai-workspace-template-claude-code`). See `docs/workspace-runtime-package.md` for the full picture.
|
||||
- **Workspace Runtime** (`workspace/`): Shared runtime published as [`molecule-ai-workspace-runtime`](https://pypi.org/project/molecule-ai-workspace-runtime/) on PyPI. Supports Claude Code, LangGraph, CrewAI, AutoGen, DeepAgents, Hermes, Gemini CLI, and OpenClaw. Each adapter lives in its own standalone template repo (e.g. `molecule-ai-workspace-template-claude-code`). See `docs/workspace-runtime-package.md` for the full picture.
|
||||
- **molecli** (`workspace-server/cmd/cli/`): Go TUI dashboard (Bubbletea + Lipgloss) — real-time workspace monitoring, event log, health overview, delete/filter operations.
|
||||
|
||||
## Key Architectural Patterns
|
||||
|
||||
@@ -59,7 +59,7 @@ Direct A2A calls between workspaces are unauthenticated in MVP. Access control i
|
||||
|
||||
## 11. Secrets in Postgres, Encrypted
|
||||
|
||||
Workspace secrets (API keys, credentials) are stored in Postgres under envelope encryption. Production deployments use AWS KMS (`KMS_KEY_ARN`): each secret gets a fresh data key via `GenerateDataKey`, the payload is sealed with AES-256-GCM, and the KMS-encrypted DEK is stored alongside the ciphertext — rotating the CMK is a no-op for existing blobs. Dev and self-host deployments fall back to static-key AES-256-GCM under `SECRETS_ENCRYPTION_KEY`. Secrets are never included in bundles, never logged, never exposed via API responses.
|
||||
Workspace secrets (API keys, credentials) are stored in Postgres with AES-256 encryption at the application layer. The encryption key comes from the `SECRETS_ENCRYPTION_KEY` environment variable. Secrets are never included in bundles, never logged, never exposed via API responses.
|
||||
|
||||
## 12. Last-Write-Wins for MVP
|
||||
|
||||
|
||||
@@ -26,7 +26,7 @@ lands in the watch list with a colliding term, add a row here.
|
||||
| **team** | A named cluster of workspaces under a PM (org template `expand_team`). Used for role grouping in Canvas. | **CrewAI**: a "crew" is a sequence of agents that pass a task through a declared order. Our "team" is an org-chart abstraction, not an execution order. |
|
||||
| **skill** | A directory with `SKILL.md` that an agent invokes via the `Skill` tool. Skills are documentation + optional scripts that teach an agent a recipe. | **Anthropic Skills API**: nearly identical. **CrewAI tool**: closer to our plugin's MCP tool, not our skill. |
|
||||
| **channel** | An outbound/inbound social integration (Telegram, Slack, …) per-workspace, wired in `workspace_channels`. | Slack's "channel": the container for messages. We use "channel" for the adapter + credentials, not the conversation itself. |
|
||||
| **runtime** | The execution engine image tag for a workspace: one of `langgraph`, `claude-code`, `openclaw`, `crewai`, `autogen`, `deepagents`, `hermes`. | **LangGraph runtime**: the Python process running the graph. We use "runtime" for the Docker image + adapter pairing, not the inner process. |
|
||||
| **runtime** | The execution engine image tag for a workspace: one of `claude-code`, `langgraph`, `crewai`, `autogen`, `deepagents`, `hermes`, `gemini-cli`, `openclaw`. | **LangGraph runtime**: the Python process running the graph. We use "runtime" for the Docker image + adapter pairing, not the inner process. |
|
||||
|
||||
## GitHub Awesome Copilot disambiguation
|
||||
|
||||
|
||||
@@ -59,11 +59,9 @@ documents — through tool calls, logs, or responses.
|
||||
|
||||
**Molecule AI controls:**
|
||||
|
||||
- **Encrypted secrets at rest:** Workspace secrets are sealed with envelope
|
||||
encryption before storage — AWS KMS (per-secret data keys via `GenerateDataKey`,
|
||||
AES-256-GCM payload) when `KMS_KEY_ARN` is set, or static-key AES-256-GCM
|
||||
under `SECRETS_ENCRYPTION_KEY` for dev / self-host. Plaintext never hits the
|
||||
database, and the platform refuses to start with neither configured.
|
||||
- **Encrypted secrets at rest:** Workspace secrets are encrypted with
|
||||
`SECRETS_ENCRYPTION_KEY` (AES-256) before storage. Plaintext never hits the
|
||||
database.
|
||||
- **Secrets scoped per-workspace:** A token scoped to workspace A cannot access
|
||||
workspace B's secrets.
|
||||
- **Memory access controls:** The MCP server's memory tools respect workspace
|
||||
|
||||
@@ -98,8 +98,7 @@ docker compose up
|
||||
| `PORT` | `8080` | Platform HTTP port |
|
||||
| `PLATFORM_URL` | `http://host.docker.internal:PORT` | URL passed to agent containers to reach the platform |
|
||||
| `CORS_ORIGINS` | `http://localhost:3000,http://localhost:3001` | Comma-separated allowed origins |
|
||||
| `SECRETS_ENCRYPTION_KEY` | -- | AES-256 key (32 bytes) for static-mode envelope encryption of workspace secrets (dev/self-host path) |
|
||||
| `KMS_KEY_ARN` | -- | AWS KMS CMK ARN — when set, secrets use KMS envelope encryption (per-secret data keys); production SaaS deployments use this path |
|
||||
| `SECRETS_ENCRYPTION_KEY` | -- | AES-256 key (32 bytes) for encrypting workspace secrets |
|
||||
| `WORKSPACE_DIR` | -- | Global fallback host path for `/workspace` bind-mount |
|
||||
| `MOLECULE_ENV` | -- | Set to `production` to hide E2E helper endpoints |
|
||||
| `ACTIVITY_RETENTION_DAYS` | `7` | How long activity logs are retained |
|
||||
@@ -155,9 +154,7 @@ This image serves both the API and the canvas frontend from a single container.
|
||||
|
||||
### Secrets Encryption
|
||||
|
||||
The platform supports two envelope-encryption modes for workspace secrets, picked at boot:
|
||||
|
||||
**Static mode (self-host / dev).** Set `SECRETS_ENCRYPTION_KEY` to a 32-byte AES-256 key. Each secret is sealed with AES-256-GCM under that single long-lived key. Without this variable (and without `KMS_KEY_ARN`), the platform refuses to start rather than silently storing plaintext.
|
||||
Set `SECRETS_ENCRYPTION_KEY` to a 32-byte AES-256 key to encrypt workspace secrets at rest. Without this variable, secrets are stored in plaintext.
|
||||
|
||||
```bash
|
||||
# Generate a key
|
||||
@@ -166,12 +163,6 @@ openssl rand -hex 32
|
||||
|
||||
**Warning:** `SECRETS_ENCRYPTION_KEY` cannot be rotated without a data migration. Choose carefully before deploying to production.
|
||||
|
||||
**KMS mode (production SaaS).** Set `KMS_KEY_ARN` to an AWS KMS Customer Master Key ARN. Each `Encrypt()` call asks KMS for a fresh per-secret data encryption key (`GenerateDataKey`), seals the secret payload with AES-256-GCM under that DEK, and stores the KMS-encrypted DEK alongside the ciphertext (envelope encryption). Rotating the CMK is a no-op for existing blobs — KMS tracks key versions internally.
|
||||
|
||||
The two modes coexist during cutover: a v2 prefix byte tags KMS blobs; older static-mode blobs decrypt with `SECRETS_ENCRYPTION_KEY` until they're next written. Operators migrating to KMS can leave both env vars set during the transition.
|
||||
|
||||
Implementation: `workspace-server/internal/crypto/envelope.go`.
|
||||
|
||||
### Rate Limiting
|
||||
|
||||
The `RATE_LIMIT` variable (default 600 requests/min) applies per client. Adjust based on your expected traffic.
|
||||
|
||||
Reference in New Issue
Block a user