feat(plugins): atomic install — stage→snapshot→swap→marker (docker path)

Closes molecule-core#114 for the docker (local-OSS) path.
EIC (SaaS) path tracked as a follow-up — same shape, different
exec primitives (ssh vs docker exec); shipping both in one PR
doubles the test surface.

THE FOUR-STEP DANCE
  1. STAGE     — docker.CopyToContainer extracts tar into
                 /configs/plugins/.staging/<name>.<ts>/
  2. SNAPSHOT  — if /configs/plugins/<name>/ exists, mv to
                 /configs/plugins/.previous/<name>.<ts>/
  3. SWAP      — atomic mv staging → live (single rename(2))
  4. MARKER    — touch /configs/plugins/<name>/.complete

Workspace-side plugin loaders should refuse to load any plugin dir
without .complete (separate small change, not in this PR — the marker
write is the necessary precursor; consumer side is a follow-up so
existing-content plugins don't break before they're re-installed).

ROLLBACK
  - Stage failure: rm -rf staging dir; live untouched
  - Snapshot failure: rm -rf staging dir; live untouched (no rename happened)
  - Swap failure with snapshot present: mv previous back to live
  - Swap failure (no snapshot): rm -rf staging; live (which never
    existed) stays absent
  - Marker failure: content already in place, log loudly with manual
    recovery hint (touch <plugin>/.complete) — don't roll back since
    the new content is what we wanted, just unmarked

GC
  Best-effort delete of previous-version snapshot after successful
  marker write. Failures non-fatal — next install or a separate
  sweeper reclaims. Sweeper for stale .previous/* across reboots is
  follow-up scope.

CONCURRENCY
  Each install gets a unique stamp (UTC second precision), so two
  concurrent reinstalls land in distinct staging dirs and the second
  swap simply overwrites the first's live result. The atomicity is
  per-install, not cross-install — by design (the platform serializes
  POST /workspaces/:id/plugins via Go-side semaphore upstream of
  this code, so cross-install collisions don't reach here).

CHANGES
  + plugins_atomic.go        — installVersion + atomicCopyToContainer
  + plugins_atomic_tar.go    — tarWalk/tarHostDirWithPrefix helpers
  + plugins_atomic_test.go   — 5 unit tests (paths, stamp shape,
                               tar happy path, symlink-skip, prefix
                               normalization). All green.
  ~ plugins_install_pipeline.go::deliverToContainer — swap
    copyPluginToContainer call to atomicCopyToContainer

Old copyPluginToContainer is retained (still called by Download()) so
this PR is purely additive on the install path; no public API change.

PHASE 4 SELF-REVIEW (FIVE-AXIS)
  Correctness: Required (addressed) — swap-failure rollback writes mv
    of previous back to live before returning the error; if rollback
    itself fails, we wrap both errors and surface the combined fault.
    Marker-write failure is treated as content-landed-but-unmarked
    (LOG, don't roll back the new content).
  Readability: No finding — installVersion path methods make the
    /staging/.previous/live/marker layout obvious from one struct.
    tarWalk extracted from the inline filepath.Walk in
    plugins_install_pipeline.go for testability.
  Architecture: No finding — atomicCopyToContainer composes existing
    execAsRoot / docker.CopyToContainer primitives; no new dependencies.
    Old copyPluginToContainer kept for Download() — single responsibility
    per function.
  Security: No finding — symlinks still skipped during tar walk
    (defense vs hostile plugin escaping its own dir). Marker writes
    use composeable path.Join, no user input touches the path.
  Performance: No finding — adds ~3 docker exec calls per install
    (mkdir, mv-snapshot, mv-swap, touch — actually 4) on top of the
    one CopyToContainer. Each exec ~50-100ms in practice; install
    end-to-end was already seconds-scale, this rounds to noise.

REFS
  molecule-core#114 — this issue
  Companion: molecule-core#112 (hot-reload classifier — depends on .complete marker)
  Companion: molecule-core#113 (version subscription — uses install machinery)
  EIC follow-up: separate issue to be filed for SaaS path parity

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

.github/workflows/e2e-api.yml

@@ -12,6 +12,59 @@ name: E2E API Smoke Test
 # spending CI cycles. See the in-job comment on the `e2e-api` job for
 # why this is one job (not two-jobs-sharing-name) and the 2026-04-29
 # PR #2264 incident that drove the consolidation.
+#
+# Parallel-safety (Class B Hongming-owned CICD red sweep, 2026-05-08)
+# -------------------------------------------------------------------
+# Same substrate hazard as PR #98 (handlers-postgres-integration). Our
+# Gitea act_runner runs with `container.network: host` (operator host
+# `/opt/molecule/runners/config.yaml`), which means:
+#
+#   * Two concurrent runs both try to bind their `-p 15432:5432` /
+#     `-p 16379:6379` host ports — the second postgres/redis FATALs
+#     with `Address in use` and `docker run` returns exit 125 with
+#     `Conflict. The container name "/molecule-ci-postgres" is already
+#     in use by container ...`. Verified in run a7/2727 on 2026-05-07.
+#   * The fixed container names `molecule-ci-postgres` / `-redis` (the
+#     pre-fix shape) collide on name AS WELL AS port. The cleanup-with-
+#     `docker rm -f` at the start of the second job KILLS the first
+#     job's still-running postgres/redis.
+#
+# Fix shape (mirrors PR #98's bridge-net pattern, adapted because
+# platform-server is a Go binary on the host, not a containerised
+# step):
+#
+#   1. Unique container names per run:
+#         pg-e2e-api-${RUN_ID}-${RUN_ATTEMPT}
+#         redis-e2e-api-${RUN_ID}-${RUN_ATTEMPT}
+#      `${RUN_ID}-${RUN_ATTEMPT}` is unique even across reruns of the
+#      same run_id.
+#   2. Ephemeral host port per run (`-p 0:5432`), then read the actual
+#      bound port via `docker port` and export DATABASE_URL/REDIS_URL
+#      pointing at it. No fixed host-port → no port collision.
+#   3. `127.0.0.1` (NOT `localhost`) in URLs — IPv6 first-resolve was
+#      the original flake fixed in #92 and the script's still IPv6-
+#      enabled.
+#   4. `if: always()` cleanup so containers don't leak when test steps
+#      fail.
+#
+# Issue #94 items #2 + #3 (also fixed here):
+#   * Pre-pull `alpine:latest` so the platform-server's provisioner
+#     (`internal/handlers/container_files.go`) can stand up its
+#     ephemeral token-write helper without a daemon.io round-trip.
+#   * Create `molecule-monorepo-net` bridge network if missing so the
+#     provisioner's container.HostConfig {NetworkMode: ...} attach
+#     succeeds.
+# Item #1 (timeouts) — evidence on recent runs (77/3191, ae/4270, 0e/
+# 2318) shows Postgres ready in 3s, Redis in 1s, Platform in 1s when
+# they DO come up. Timeouts are not the bottleneck; not bumped.
+#
+# Item explicitly NOT fixed here: failing test `Status back online`
+# fails because the platform's langgraph workspace template image
+# (ghcr.io/molecule-ai/workspace-template-langgraph:latest) returns
+# 403 Forbidden post-2026-05-06 GitHub org suspension. That is a
+# template-registry resolution issue (ADR-002 / local-build mode) and
+# belongs in a separate change that touches workspace-server, not
+# this workflow file.
 
 on:
   push:
@@ -78,11 +131,14 @@ jobs:
     runs-on: ubuntu-latest
     timeout-minutes: 15
     env:
-      DATABASE_URL: postgres://dev:dev@localhost:15432/molecule?sslmode=disable
-      REDIS_URL: redis://localhost:16379
+      # Unique per-run container names so concurrent runs on the host-
+      # network act_runner don't collide on name OR port.
+      # `${RUN_ID}-${RUN_ATTEMPT}` stays unique across reruns of the
+      # same run_id. PORT is set later (after docker port lookup) since
+      # we let Docker assign an ephemeral host port.
+      PG_CONTAINER: pg-e2e-api-${{ github.run_id }}-${{ github.run_attempt }}
+      REDIS_CONTAINER: redis-e2e-api-${{ github.run_id }}-${{ github.run_attempt }}
       PORT: "8080"
-      PG_CONTAINER: molecule-ci-postgres
-      REDIS_CONTAINER: molecule-ci-redis
     steps:
       - name: No-op pass (paths filter excluded this commit)
         if: needs.detect-changes.outputs.api != 'true'
@@ -97,11 +153,53 @@ jobs:
           go-version: 'stable'
           cache: true
           cache-dependency-path: workspace-server/go.sum
+      - name: Pre-pull alpine + ensure provisioner network (Issue #94 items #2 + #3)
+        if: needs.detect-changes.outputs.api == 'true'
+        run: |
+          # Provisioner uses alpine:latest for ephemeral token-write
+          # containers (workspace-server/internal/handlers/container_files.go).
+          # Pre-pull so the first provision in test_api.sh doesn't race
+          # the daemon's pull cache. Idempotent — `docker pull` is a no-op
+          # when the image is already present.
+          docker pull alpine:latest >/dev/null
+          # Provisioner attaches workspace containers to
+          # molecule-monorepo-net (workspace-server/internal/provisioner/
+          # provisioner.go::DefaultNetwork). The bridge already exists on
+          # the operator host's docker daemon — `network create` is
+          # idempotent via `|| true`.
+          docker network create molecule-monorepo-net >/dev/null 2>&1 || true
+          echo "alpine:latest pre-pulled; molecule-monorepo-net ensured."
       - name: Start Postgres (docker)
         if: needs.detect-changes.outputs.api == 'true'
         run: |
+          # Defensive cleanup — only matches THIS run's container name,
+          # so it cannot kill a sibling run's postgres. (Pre-fix the
+          # name was static and this rm hit other runs' containers.)
           docker rm -f "$PG_CONTAINER" 2>/dev/null || true
-          docker run -d --name "$PG_CONTAINER" -e POSTGRES_USER=dev -e POSTGRES_PASSWORD=dev -e POSTGRES_DB=molecule -p 15432:5432 postgres:16
+          # `-p 0:5432` requests an ephemeral host port; we read it back
+          # below and export DATABASE_URL.
+          docker run -d --name "$PG_CONTAINER" \
+            -e POSTGRES_USER=dev -e POSTGRES_PASSWORD=dev -e POSTGRES_DB=molecule \
+            -p 0:5432 postgres:16 >/dev/null
+          # Resolve the host-side port assignment. `docker port` prints
+          # `0.0.0.0:NNNN` (and on host-net runners may also print an
+          # IPv6 line — take the first IPv4 line).
+          PG_PORT=$(docker port "$PG_CONTAINER" 5432/tcp | awk -F: '/^0\.0\.0\.0:/ {print $2; exit}')
+          if [ -z "$PG_PORT" ]; then
+            # Fallback: any first line. Some Docker versions print only
+            # one line.
+            PG_PORT=$(docker port "$PG_CONTAINER" 5432/tcp | head -1 | awk -F: '{print $NF}')
+          fi
+          if [ -z "$PG_PORT" ]; then
+            echo "::error::Could not resolve host port for $PG_CONTAINER"
+            docker port "$PG_CONTAINER" 5432/tcp || true
+            docker logs "$PG_CONTAINER" || true
+            exit 1
+          fi
+          # 127.0.0.1 (NOT localhost) — IPv6 first-resolve flake (#92).
+          echo "PG_PORT=${PG_PORT}" >> "$GITHUB_ENV"
+          echo "DATABASE_URL=postgres://dev:dev@127.0.0.1:${PG_PORT}/molecule?sslmode=disable" >> "$GITHUB_ENV"
+          echo "Postgres host port: ${PG_PORT}"
           for i in $(seq 1 30); do
             if docker exec "$PG_CONTAINER" pg_isready -U dev >/dev/null 2>&1; then
               echo "Postgres ready after ${i}s"
@@ -116,7 +214,20 @@ jobs:
         if: needs.detect-changes.outputs.api == 'true'
         run: |
           docker rm -f "$REDIS_CONTAINER" 2>/dev/null || true
-          docker run -d --name "$REDIS_CONTAINER" -p 16379:6379 redis:7
+          docker run -d --name "$REDIS_CONTAINER" -p 0:6379 redis:7 >/dev/null
+          REDIS_PORT=$(docker port "$REDIS_CONTAINER" 6379/tcp | awk -F: '/^0\.0\.0\.0:/ {print $2; exit}')
+          if [ -z "$REDIS_PORT" ]; then
+            REDIS_PORT=$(docker port "$REDIS_CONTAINER" 6379/tcp | head -1 | awk -F: '{print $NF}')
+          fi
+          if [ -z "$REDIS_PORT" ]; then
+            echo "::error::Could not resolve host port for $REDIS_CONTAINER"
+            docker port "$REDIS_CONTAINER" 6379/tcp || true
+            docker logs "$REDIS_CONTAINER" || true
+            exit 1
+          fi
+          echo "REDIS_PORT=${REDIS_PORT}" >> "$GITHUB_ENV"
+          echo "REDIS_URL=redis://127.0.0.1:${REDIS_PORT}" >> "$GITHUB_ENV"
+          echo "Redis host port: ${REDIS_PORT}"
           for i in $(seq 1 15); do
             if docker exec "$REDIS_CONTAINER" redis-cli ping 2>/dev/null | grep -q PONG; then
               echo "Redis ready after ${i}s"
@@ -135,13 +246,15 @@ jobs:
         if: needs.detect-changes.outputs.api == 'true'
         working-directory: workspace-server
         run: |
+          # DATABASE_URL + REDIS_URL exported by the start-postgres /
+          # start-redis steps point at this run's per-run host ports.
           ./platform-server > platform.log 2>&1 &
           echo $! > platform.pid
       - name: Wait for /health
         if: needs.detect-changes.outputs.api == 'true'
         run: |
           for i in $(seq 1 30); do
-            if curl -sf http://localhost:8080/health > /dev/null; then
+            if curl -sf http://127.0.0.1:8080/health > /dev/null; then
               echo "Platform up after ${i}s"
               exit 0
             fi
@@ -185,6 +298,9 @@ jobs:
             kill "$(cat workspace-server/platform.pid)" 2>/dev/null || true
           fi
       - name: Stop service containers
+        # always() so containers don't leak when test steps fail. The
+        # cleanup is best-effort: if the container is already gone
+        # (e.g. concurrent rerun race), don't fail the job.
         if: always() && needs.detect-changes.outputs.api == 'true'
         run: |
           docker rm -f "$PG_CONTAINER" 2>/dev/null || true

.github/workflows/e2e-staging-canvas.yml

@@ -22,9 +22,9 @@ on:
   # spending CI cycles. See e2e-api.yml for the rationale on why this
   # is a single job rather than two-jobs-sharing-name.
   push:
-    branches: [main, staging]
+    branches: [main]
   pull_request:
-    branches: [main, staging]
+    branches: [main]
   workflow_dispatch:
   schedule:
     # Weekly on Sunday 08:00 UTC — catches Chrome / Playwright / Next.js

.github/workflows/e2e-staging-external.yml

@@ -32,7 +32,7 @@ name: E2E Staging External Runtime
 
 on:
   push:
-    branches: [staging, main]
+    branches: [main]
     paths:
       - 'workspace-server/internal/handlers/workspace.go'
       - 'workspace-server/internal/handlers/registry.go'
@@ -44,7 +44,7 @@ on:
       - 'tests/e2e/test_staging_external_runtime.sh'
       - '.github/workflows/e2e-staging-external.yml'
   pull_request:
-    branches: [staging, main]
+    branches: [main]
     paths:
       - 'workspace-server/internal/handlers/workspace.go'
       - 'workspace-server/internal/handlers/registry.go'

.github/workflows/e2e-staging-saas.yml

@@ -20,13 +20,12 @@ name: E2E Staging SaaS (full lifecycle)
 #     via the same paths watcher that e2e-api.yml uses)
 
 on:
-  # Fire on staging push too — previously this only ran on main, which
-  # meant the most thorough end-to-end test caught regressions AFTER
-  # they shipped to staging (and then to the auto-promote PR). Running
-  # on staging push catches them BEFORE the staging→main promotion
-  # opens, so a green canary into auto-promote is more meaningful.
+  # Trunk-based (Phase 3 of internal#81): main is the only branch.
+  # Previously this fired on staging push too because staging was a
+  # superset of main and ran the gate ahead of auto-promote; with no
+  # staging branch, main is where E2E gates the deploy.
   push:
-    branches: [staging, main]
+    branches: [main]
     paths:
       - 'workspace-server/internal/handlers/registry.go'
       - 'workspace-server/internal/handlers/workspace_provision.go'
@@ -36,7 +35,7 @@ on:
       - 'tests/e2e/test_staging_full_saas.sh'
       - '.github/workflows/e2e-staging-saas.yml'
   pull_request:
-    branches: [staging, main]
+    branches: [main]
     paths:
       - 'workspace-server/internal/handlers/registry.go'
       - 'workspace-server/internal/handlers/workspace_provision.go'

.github/workflows/redeploy-tenants-on-staging.yml

@@ -36,7 +36,7 @@ on:
   workflow_run:
     workflows: ['publish-workspace-server-image']
     types: [completed]
-    branches: [staging]
+    branches: [main]
   workflow_dispatch:
     inputs:
       target_tag:

Makefile

@@ -0,0 +1,28 @@
+# Top-level Makefile — convenience wrappers around docker compose.
+#
+# Most molecule-core dev work happens via these shortcuts. CI doesn't
+# use this Makefile; CI calls docker compose / go test directly so the
+# Makefile can evolve without breaking the build.
+
+.PHONY: help dev up down logs build test
+
+help: ## Show this help.
+	@grep -E '^[a-zA-Z_-]+:.*?## ' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-12s\033[0m %s\n", $$1, $$2}'
+
+dev: ## Start the full stack with air hot-reload for the platform service.
+	docker compose -f docker-compose.yml -f docker-compose.dev.yml up
+
+up: ## Start the full stack in production-shape mode (no air, normal Dockerfile).
+	docker compose up
+
+down: ## Stop the stack and remove containers (volumes preserved).
+	docker compose down
+
+logs: ## Tail logs from all services (Ctrl-C to detach).
+	docker compose logs -f
+
+build: ## Force a fresh build of the platform image (no cache).
+	docker compose build --no-cache platform
+
+test: ## Run Go unit tests in workspace-server/.
+	cd workspace-server && go test -race ./...

canvas/vitest.config.ts

@@ -7,6 +7,32 @@ export default defineConfig({
   test: {
     environment: 'node',
     exclude: ['e2e/**', 'node_modules/**', '**/dist/**'],
+    // CI-conditional test timeout (issue #96).
+    //
+    // Vitest's 5000ms default is too tight for the first test in any
+    // file under our CI shape: `npx vitest run --coverage` on the
+    // self-hosted Gitea Actions Docker runner. The cold-start cost
+    // (v8 coverage instrumentation init + JSDOM bootstrap + module-
+    // graph import for @/components/* and @/lib/* + first React
+    // render) consistently consumes 5-7 seconds for the first
+    // synchronous test in heavyweight component files
+    // (ActivityTab.test.tsx, CreateWorkspaceDialog.test.tsx,
+    // ConfigTab.provider.test.tsx) — even though every subsequent
+    // test in the same file completes in 100-1500ms.
+    //
+    // Empirically the worst observed first-test was 6453ms in a
+    // single file (CreateWorkspaceDialog). 30000ms gives ~5x
+    // headroom over that on CI; we still keep 5000ms locally so
+    // genuine waitFor races / hung promises stay sensitive in dev.
+    //
+    // Same vitest pattern documented at:
+    //   https://vitest.dev/config/testtimeout
+    //   https://vitest.dev/guide/coverage#profiling-test-performance
+    //
+    // Per-test duration is still emitted to the CI log; if a test
+    // ever silently approaches 25-30s under this raised ceiling that
+    // will surface as a duration regression and we revisit.
+    testTimeout: process.env.CI ? 30000 : 5000,
     // Coverage is instrumented but NOT yet a CI gate — first land
     // observability so we can see the baseline, then dial in
     // thresholds + a hard gate in a follow-up PR (#1815). Today's

docker-compose.dev.yml

@@ -0,0 +1,43 @@
+# docker-compose.dev.yml — overlay over docker-compose.yml for local dev
+# with air-driven live reload of the platform (workspace-server) service.
+#
+# Usage:
+#   docker compose -f docker-compose.yml -f docker-compose.dev.yml up
+#   (or `make dev` shorthand from repo root)
+#
+# What this overlay changes vs docker-compose.yml alone:
+#   - Platform service uses workspace-server/Dockerfile.dev (air on top of
+#     golang:1.25-alpine) instead of the multi-stage prod Dockerfile.
+#   - Platform service bind-mounts the host's workspace-server/ source
+#     into /app/workspace-server so air sees source edits live.
+#   - Other services (postgres, redis, langfuse, etc.) inherit unchanged
+#     from docker-compose.yml.
+#
+# What stays the same:
+#   - All env vars, volumes, depends_on, healthchecks from docker-compose.yml.
+#   - Network topology + ports.
+#   - Postgres/Redis as service containers (no in-process replacements).
+
+services:
+  platform:
+    build:
+      context: .
+      dockerfile: workspace-server/Dockerfile.dev
+    # Rebind source: edits under host's workspace-server/ propagate live.
+    # The named volume on go-build-cache speeds up first build per container.
+    volumes:
+      - ./workspace-server:/app/workspace-server
+      - go-build-cache:/root/.cache/go-build
+      - go-mod-cache:/go/pkg/mod
+    # Air signals the running binary on rebuild; ensure shell stops cleanly.
+    init: true
+    # Mark the service as dev-mode so the platform can short-circuit any
+    # behavior that's incompatible with hot-reload (e.g. background
+    # cron-style watchers that don't survive process restart). No-op
+    # today; reserved for future flag use.
+    environment:
+      MOLECULE_DEV_HOT_RELOAD: "1"
+
+volumes:
+  go-build-cache:
+  go-mod-cache:

docs/architecture/backends.md

@@ -2,7 +2,7 @@
 
 **Status:** living document — update when you ship a feature that touches one backend.
 **Owner:** workspace-server + controlplane teams.
-**Last audit:** 2026-05-05 (Claude agent — `provisionWorkspaceAuto` / `StopWorkspaceAuto` / `HasProvisioner` SoT pattern landed in PRs #2811 + #2824).
+**Last audit:** 2026-05-07 (plugin install/uninstall closed for EC2 backend via EIC SSH push to the bind-mounted `/configs/plugins/<name>/`, mirroring the Files API PR #1702 pattern).
 
 ## Why this exists
 
@@ -54,7 +54,7 @@ For "do we have any backend?", use `HasProvisioner()`, never bare `h.provisioner
 | **Files API** | | | | |
 | List / Read / Write / Replace / Delete | `container_files.go`, `template_import.go` | `docker exec` + tar `CopyToContainer` | SSH via EIC tunnel (PR #1702) | ✅ parity as of 2026-04-22 (previously docker-only) |
 | **Plugins** | | | | |
-| Install / uninstall / list | `plugins_install.go` | `deliverToContainer()` + volume rm | **gap — no live plugin delivery** | 🔴 **docker-only** |
+| Install / uninstall / list | `plugins_install.go` + `plugins_install_eic.go` | `deliverToContainer()` → exec+`CopyToContainer` on local container | `instance_id` set → EIC SSH push of the staged tarball into the EC2's bind-mounted `/configs/plugins/<name>/` (per `workspaceFilePathPrefix`), `chown 1000:1000`, restart | ✅ parity |
 | **Terminal (WebSocket)** | | | | |
 | Dispatch | `terminal.go:90-105` | `instance_id=""` → `handleLocalConnect` → `docker attach` | `instance_id` set → `handleRemoteConnect` → EIC SSH + `docker exec` | ✅ parity (different implementations, same UX) |
 | **A2A proxy** | | | | |

workspace-server/.air.toml

@@ -0,0 +1,49 @@
+# air.toml — live-reload config for local docker-compose dev mode.
+#
+# Active when the platform service runs from workspace-server/Dockerfile.dev
+# (selected via docker-compose.dev.yml overlay). In production, the regular
+# Dockerfile builds a static binary; air is dev-only.
+#
+# Reference: https://github.com/air-verse/air
+
+root = "."
+testdata_dir = "testdata"
+tmp_dir = "tmp"
+
+[build]
+  # Same build invocation as Dockerfile's builder stage minus the
+  # CGO_ENABLED=0 toggle (CGO ok in dev for richer race detector output).
+  cmd = "go build -o ./tmp/server ./cmd/server"
+  bin = "tmp/server"
+  full_bin = ""
+  args_bin = []
+  # Watch every .go and .yaml file under workspace-server/.
+  include_ext = ["go", "yaml", "tmpl"]
+  # Don't watch tests, build artifacts, vendored deps, or migration .sql
+  # (migrations need a clean DB anyway — handled by docker-compose down/up).
+  exclude_dir = ["assets", "tmp", "vendor", "testdata", "node_modules"]
+  exclude_file = []
+  # _test.go and *_mock.go shouldn't trigger a rebuild — saves cycles.
+  exclude_regex = ["_test\\.go$", "_mock\\.go$"]
+  exclude_unchanged = true
+  follow_symlink = false
+  log = "build-errors.log"
+  # Kill running binary 1s before starting new one.
+  kill_delay = "1s"
+  send_interrupt = true
+  stop_on_error = true
+  # Debounce: wait this long after last change before triggering rebuild.
+  delay = 500
+
+[log]
+  time = false
+
+[color]
+  main = "magenta"
+  watcher = "cyan"
+  build = "yellow"
+  runner = "green"
+
+[misc]
+  # Don't keep the tmp/ dir around between runs.
+  clean_on_exit = true

workspace-server/Dockerfile.dev

@@ -0,0 +1,38 @@
+# Dockerfile.dev — local-development image with air-driven live reload.
+#
+# Selected by docker-compose.dev.yml (overlay over docker-compose.yml).
+# Production stays on workspace-server/Dockerfile (static binary, no air).
+#
+# Workflow:
+#   1. docker compose -f docker-compose.yml -f docker-compose.dev.yml up
+#   2. Edit any .go file under workspace-server/
+#   3. air detects, rebuilds, kills old binary, starts new one (~3-5s)
+#   4. No `docker compose up --build` needed
+#
+# Templates + plugins are NOT pre-cloned here — air-mode assumes the
+# developer's filesystem has the workspace-configs-templates/ + plugins/
+# dirs available, mounted at runtime via docker-compose.dev.yml.
+
+FROM golang:1.25-alpine
+
+# air + git (for go mod) + ca-certs (for TLS) + tzdata (for time-zone DB).
+RUN apk add --no-cache git ca-certificates tzdata wget \
+ && go install github.com/air-verse/air@latest
+
+WORKDIR /app/workspace-server
+
+# Pre-fetch deps so the first `air` rebuild on a fresh container is fast.
+# These are bind-mount-overridden at runtime, so the COPY here is just
+# to warm the module cache.
+COPY workspace-server/go.mod workspace-server/go.sum ./
+RUN go mod download
+
+# Source is bind-mounted at runtime (see docker-compose.dev.yml volumes
+# block) so the Dockerfile doesn't need to COPY it. air watches the
+# bind-mounted dir for changes.
+
+ENV CGO_ENABLED=1
+ENV GOFLAGS="-buildvcs=false"
+
+# Run air with the .air.toml in the bind-mounted source dir.
+CMD ["air", "-c", ".air.toml"]

workspace-server/internal/handlers/eic_tunnel_pool.go

@@ -108,6 +108,18 @@ type eicTunnelPool struct {
 	// First acquirer takes the slot; later ones wait on the channel.
 	pendingSetups map[string]chan struct{}
 	stopJanitor   chan struct{}
+	// janitorInterval is captured at pool construction from the
+	// package-level poolJanitorInterval var. Captured (not re-read on
+	// every tick) so a test that swaps the package var via t.Cleanup
+	// after a global pool's janitor is already running can't race
+	// with that goroutine's ticker read. The global pool is created
+	// lazily once per process via sync.Once; before this capture
+	// landed, every test that touched poolJanitorInterval after the
+	// global pool's first-touch raced the janitor (caught by -race
+	// on staging tip 249dbc6a — TestPooledWithEICTunnel_PanicPoisonsEntry).
+	// Tests still get the new value on a freshPool() because they
+	// set the package var BEFORE calling newEICTunnelPool().
+	janitorInterval time.Duration
 }
 
 var (
@@ -127,11 +139,16 @@ func getEICTunnelPool() *eicTunnelPool {
 
 // newEICTunnelPool constructs an empty pool. Exported so tests can
 // build isolated pools without sharing the singleton.
+//
+// Captures poolJanitorInterval at construction time so the janitor
+// goroutine doesn't race with t.Cleanup-driven swaps of the package
+// var. See the janitorInterval field comment for the failure mode.
 func newEICTunnelPool() *eicTunnelPool {
 	return &eicTunnelPool{
-		entries:       map[string]*pooledTunnel{},
-		pendingSetups: map[string]chan struct{}{},
-		stopJanitor:   make(chan struct{}),
+		entries:         map[string]*pooledTunnel{},
+		pendingSetups:   map[string]chan struct{}{},
+		stopJanitor:     make(chan struct{}),
+		janitorInterval: poolJanitorInterval,
 	}
 }
 
@@ -290,8 +307,11 @@ func (p *eicTunnelPool) evictLRUIfFullLocked(skipInstance string) {
 // janitor periodically scans for entries that are idle AND expired,
 // closing their tunnels. Runs forever (per pool lifetime); cancelled
 // by close(p.stopJanitor) for tests that build short-lived pools.
+//
+// Reads p.janitorInterval (captured at construction) instead of the
+// package-level poolJanitorInterval — see janitorInterval field comment.
 func (p *eicTunnelPool) janitor() {
-	t := time.NewTicker(poolJanitorInterval)
+	t := time.NewTicker(p.janitorInterval)
 	defer t.Stop()
 	for {
 		select {

workspace-server/internal/handlers/local_e2e_dev_dept_test.go

@@ -0,0 +1,375 @@
+package handlers
+
+import (
+	"archive/tar"
+	"bytes"
+	"net"
+	"os"
+	"os/exec"
+	"path/filepath"
+	"strings"
+	"testing"
+	"time"
+
+	"gopkg.in/yaml.v3"
+)
+
+// Local E2E for the dev-department extraction (RFC internal#77).
+//
+// Pre-conditions: both repos cloned as siblings under
+// /tmp/local-e2e-deploy/{molecule-dev, molecule-dev-department}.
+// (Set up by the orchestrator before running this test.)
+//
+// What this proves end-to-end through real platform code:
+//   1. resolveYAMLIncludes follows the dev-lead symlink at the parent's
+//      template root and pulls in the dev-department subtree.
+//   2. Recursive !include's inside the symlinked subtree resolve
+//      correctly via the chain dev-lead/workspace.yaml →
+//      ./core-lead/workspace.yaml → ./core-be/workspace.yaml etc.
+//   3. The resolved YAML unmarshals into a complete OrgTemplate with the
+//      expected count of workspaces (parent's PM+Marketing+Research +
+//      dev-department's atomized 28 workspaces).
+//
+// Skipped if the local-e2e-deploy fixture isn't present — won't block
+// CI on hosts that haven't set it up.
+func TestLocalE2E_DevDepartmentExtraction(t *testing.T) {
+	parent := "/tmp/local-e2e-deploy/molecule-dev"
+	if _, err := os.Stat(filepath.Join(parent, "org.yaml")); err != nil {
+		t.Skipf("local-e2e fixture not present at %s: %v", parent, err)
+	}
+
+	orgYAML, err := os.ReadFile(filepath.Join(parent, "org.yaml"))
+	if err != nil {
+		t.Fatalf("read org.yaml: %v", err)
+	}
+
+	expanded, err := resolveYAMLIncludes(orgYAML, parent)
+	if err != nil {
+		t.Fatalf("resolveYAMLIncludes failed: %v", err)
+	}
+
+	var tmpl OrgTemplate
+	if err := yaml.Unmarshal(expanded, &tmpl); err != nil {
+		t.Fatalf("unmarshal expanded OrgTemplate: %v", err)
+	}
+
+	// Walk the full workspace tree, collect names.
+	names := []string{}
+	var walk func([]OrgWorkspace)
+	walk = func(ws []OrgWorkspace) {
+		for _, w := range ws {
+			names = append(names, w.Name)
+			walk(w.Children)
+		}
+	}
+	walk(tmpl.Workspaces)
+
+	t.Logf("org name: %q", tmpl.Name)
+	t.Logf("total workspaces (recursive): %d", len(names))
+	for _, n := range names {
+		t.Logf("  - %q", n)
+	}
+
+	// Expected: PM + Marketing Lead + Dev Lead at top level, plus the
+	// full sub-trees under each. After atomization, we expect:
+	//   - PM tree: PM + Research Lead + 3 research roles = 5
+	//   - Marketing tree: Marketing Lead + 5 marketing roles = 6
+	//   - Dev Lead tree: Dev Lead + (5 sub-team leads × ~6 each) +
+	//     3 floaters + Triage Operator = ~32
+	// Roughly ~43 total. Be liberal; just assert a floor.
+	if len(names) < 30 {
+		t.Errorf("workspace count too low (%d) — expected ~40+ (PM+Marketing+Dev tree)", len(names))
+	}
+
+	// Specific sentinel names we expect to find:
+	expected := []string{
+		"PM",
+		"Marketing Lead",
+		"Dev Lead",
+		"Core Platform Lead",
+		"Controlplane Lead",
+		"App & Docs Lead",
+		"Infra Lead",
+		"SDK Lead",
+		"Documentation Specialist", // Q1 — should be under app-lead
+		"Triage Operator",          // Q2 — should be under dev-lead
+	}
+	found := map[string]bool{}
+	for _, n := range names {
+		found[n] = true
+	}
+	for _, want := range expected {
+		if !found[want] {
+			t.Errorf("missing expected workspace %q", want)
+		}
+	}
+}
+
+// Stage-2 of the local e2e: prove every resolved workspace's `files_dir`
+// path actually consumes correctly through the rest of the import chain.
+// resolveYAMLIncludes returning a populated OrgTemplate is necessary but
+// not sufficient — `POST /org/import` then does:
+//
+//   1. resolveInsideRoot(orgBaseDir, ws.FilesDir) → must return a path
+//      that exists and stat-resolves to a directory (org_import.go:313-317).
+//   2. CopyTemplateToContainer(ctx, containerID, templatePath) → walks
+//      the dir with filepath.Walk and tars its contents into the
+//      workspace's /configs/ mount (provisioner.go:766-820).
+//
+// This stage-2 test exercises both #1 and #2 against every workspace in
+// the resolved tree, mimicking what the platform does post-include-
+// resolution. Catches: files_dir paths that don't resolve through the
+// symlink, paths that exist but are empty (silently produces empty
+// /configs/), or filepath.Walk failing to descend through cross-repo
+// symlink boundaries.
+func TestLocalE2E_FilesDirConsumption(t *testing.T) {
+	parent := "/tmp/local-e2e-deploy/molecule-dev"
+	if _, err := os.Stat(filepath.Join(parent, "org.yaml")); err != nil {
+		t.Skipf("local-e2e fixture not present at %s: %v", parent, err)
+	}
+
+	orgYAML, err := os.ReadFile(filepath.Join(parent, "org.yaml"))
+	if err != nil {
+		t.Fatalf("read org.yaml: %v", err)
+	}
+	expanded, err := resolveYAMLIncludes(orgYAML, parent)
+	if err != nil {
+		t.Fatalf("resolveYAMLIncludes: %v", err)
+	}
+	var tmpl OrgTemplate
+	if err := yaml.Unmarshal(expanded, &tmpl); err != nil {
+		t.Fatalf("unmarshal: %v", err)
+	}
+
+	// Flatten every workspace — including children, grandchildren, etc.
+	flat := []OrgWorkspace{}
+	var walk func([]OrgWorkspace)
+	walk = func(ws []OrgWorkspace) {
+		for _, w := range ws {
+			flat = append(flat, w)
+			walk(w.Children)
+		}
+	}
+	walk(tmpl.Workspaces)
+
+	checked := 0
+	for _, w := range flat {
+		if w.FilesDir == "" {
+			continue // workspace declared inline (no files_dir) — skip
+		}
+		checked++
+		t.Run(w.Name+"/"+w.FilesDir, func(t *testing.T) {
+			// Step 1: resolveInsideRoot returns a path that's-inside-root.
+			abs, err := resolveInsideRoot(parent, w.FilesDir)
+			if err != nil {
+				t.Fatalf("resolveInsideRoot(%q, %q): %v", parent, w.FilesDir, err)
+			}
+			info, err := os.Stat(abs)
+			if err != nil {
+				t.Fatalf("stat %q (resolved from files_dir %q): %v", abs, w.FilesDir, err)
+			}
+			if !info.IsDir() {
+				t.Fatalf("files_dir %q resolved to %q which is not a directory", w.FilesDir, abs)
+			}
+
+			// Step 2: walk the dir like CopyTemplateToContainer does.
+			// Mirror the platform's symlink-resolution at the root —
+			// filepath.Walk doesn't descend into a symlink leaf, so
+			// CopyTemplateToContainer (provisioner.go) calls
+			// EvalSymlinks on templatePath first. Replicate exactly.
+			if resolved, err := filepath.EvalSymlinks(abs); err == nil {
+				abs = resolved
+			}
+			var buf bytes.Buffer
+			tw := tar.NewWriter(&buf)
+			fileCount := 0
+			fileNames := []string{}
+			err = filepath.Walk(abs, func(path string, info os.FileInfo, err error) error {
+				if err != nil {
+					return err
+				}
+				rel, err := filepath.Rel(abs, path)
+				if err != nil {
+					return err
+				}
+				if rel == "." {
+					return nil
+				}
+				header, _ := tar.FileInfoHeader(info, "")
+				header.Name = rel
+				if err := tw.WriteHeader(header); err != nil {
+					return err
+				}
+				if !info.IsDir() {
+					fileCount++
+					fileNames = append(fileNames, rel)
+					data, err := os.ReadFile(path)
+					if err != nil {
+						return err
+					}
+					header.Size = int64(len(data))
+					tw.Write(data)
+				}
+				return nil
+			})
+			if err != nil {
+				t.Fatalf("filepath.Walk %q (mimics CopyTemplateToContainer): %v", abs, err)
+			}
+			tw.Close()
+
+			if fileCount == 0 {
+				t.Errorf("files_dir %q at %q is empty — CopyTemplateToContainer would produce empty /configs/",
+					w.FilesDir, abs)
+			}
+
+			// Sanity: every workspace folder should have AT LEAST one of
+			// {workspace.yaml, system-prompt.md, initial-prompt.md} —
+			// these are the markers a workspace folder is recognizable
+			// as a workspace (mirrors validator's WORKSPACE_FOLDER_MARKERS).
+			markers := []string{"workspace.yaml", "system-prompt.md", "initial-prompt.md"}
+			hasMarker := false
+			for _, name := range fileNames {
+				for _, m := range markers {
+					if name == m || strings.HasSuffix(name, "/"+m) {
+						hasMarker = true
+						break
+					}
+				}
+				if hasMarker {
+					break
+				}
+			}
+			if !hasMarker {
+				t.Errorf("files_dir %q at %q has %d files but none of the workspace markers %v — found: %v",
+					w.FilesDir, abs, fileCount, markers, fileNames)
+			}
+		})
+	}
+	t.Logf("checked %d workspaces with files_dir", checked)
+	if checked < 25 {
+		t.Errorf("expected ~28 workspaces with files_dir (post-atomization); only saw %d", checked)
+	}
+}
+
+// PR-C from the Phase 3a phasing (task #234): real-Gitea e2e for the
+// !external resolver against the LIVE molecule-ai/molecule-dev-department
+// repo. Verifies the production gitFetcher fetches the dev tree and the
+// resolver grafts it correctly into a parent template that has NO
+// symlink — composition is purely platform-side.
+//
+// Skipped if Gitea isn't reachable (offline / firewall / CI without
+// network). Requires `git` binary on PATH.
+func TestLocalE2E_ExternalDevDepartment(t *testing.T) {
+	if _, err := exec.LookPath("git"); err != nil {
+		t.Skipf("git binary not found: %v", err)
+	}
+
+	// Skip if Gitea host isn't reachable (TCP probe). Avoids network-
+	// dependent tests failing on offline runners.
+	conn, err := net.DialTimeout("tcp", "git.moleculesai.app:443", 3*time.Second)
+	if err != nil {
+		t.Skipf("git.moleculesai.app:443 unreachable: %v", err)
+	}
+	conn.Close()
+
+	// Build a minimal parent template inline — no need for the
+	// /tmp/local-e2e-deploy/ symlinked fixture. The whole point of
+	// !external is that the parent template is self-contained;
+	// composition resolves over the network at import time.
+	parent := t.TempDir()
+
+	orgYAML := []byte(`name: External-Only Test Parent
+description: Parent template that pulls the entire dev tree via !external.
+defaults:
+  runtime: claude-code
+  tier: 2
+workspaces:
+  - !external
+    repo: molecule-ai/molecule-dev-department
+    ref: main
+    path: dev-lead/workspace.yaml
+`)
+	if err := os.WriteFile(filepath.Join(parent, "org.yaml"), orgYAML, 0o644); err != nil {
+		t.Fatalf("write org.yaml: %v", err)
+	}
+
+	out, err := resolveYAMLIncludes(orgYAML, parent)
+	if err != nil {
+		t.Fatalf("resolveYAMLIncludes (!external against live Gitea): %v", err)
+	}
+
+	var tmpl OrgTemplate
+	if err := yaml.Unmarshal(out, &tmpl); err != nil {
+		t.Fatalf("unmarshal: %v", err)
+	}
+
+	// Walk the workspace tree, collect names + check files_dir paths.
+	flat := []OrgWorkspace{}
+	var walk func([]OrgWorkspace)
+	walk = func(ws []OrgWorkspace) {
+		for _, w := range ws {
+			flat = append(flat, w)
+			walk(w.Children)
+		}
+	}
+	walk(tmpl.Workspaces)
+
+	t.Logf("workspaces resolved through !external: %d", len(flat))
+	if len(flat) < 25 {
+		t.Errorf("expected ~28 dev-tree workspaces via !external; got %d", len(flat))
+	}
+
+	// Sentinel checks — same as TestLocalE2E_DevDepartmentExtraction
+	// (Q1+Q2 placements verified).
+	expected := []string{
+		"Dev Lead",
+		"Core Platform Lead",
+		"Controlplane Lead",
+		"App & Docs Lead",
+		"Documentation Specialist", // Q1
+		"Triage Operator",          // Q2
+	}
+	found := map[string]bool{}
+	for _, w := range flat {
+		found[w.Name] = true
+	}
+	for _, want := range expected {
+		if !found[want] {
+			t.Errorf("missing expected workspace %q", want)
+		}
+	}
+
+	// Every workspace's files_dir must be cache-prefixed (proves the
+	// path-rewrite ran end-to-end).
+	cachePrefix := ".external-cache"
+	for _, w := range flat {
+		if w.FilesDir == "" {
+			continue
+		}
+		if !strings.HasPrefix(w.FilesDir, cachePrefix) {
+			t.Errorf("workspace %q files_dir %q missing cache prefix %q", w.Name, w.FilesDir, cachePrefix)
+		}
+	}
+
+	// Verify the fetched cache exists and resolveInsideRoot accepts
+	// every workspace's files_dir (would cause provisioning to fail
+	// if not).
+	for _, w := range flat {
+		if w.FilesDir == "" {
+			continue
+		}
+		abs, err := resolveInsideRoot(parent, w.FilesDir)
+		if err != nil {
+			t.Errorf("workspace %q files_dir %q: resolveInsideRoot: %v", w.Name, w.FilesDir, err)
+			continue
+		}
+		info, err := os.Stat(abs)
+		if err != nil {
+			t.Errorf("workspace %q: stat %q: %v", w.Name, abs, err)
+			continue
+		}
+		if !info.IsDir() {
+			t.Errorf("workspace %q files_dir %q is not a directory", w.Name, w.FilesDir)
+		}
+	}
+}

workspace-server/internal/handlers/org_external.go

@@ -0,0 +1,439 @@
+package handlers
+
+import (
+	"context"
+	"fmt"
+	"net/url"
+	"os"
+	"os/exec"
+	"path/filepath"
+	"regexp"
+	"strings"
+	"time"
+
+	"gopkg.in/yaml.v3"
+)
+
+// External-ref resolver — gitops-style cross-repo subtree composition.
+// Internal#77 RFC, Phase 3a (task #222). Prior art: Helm subcharts +
+// dependency cache, Kustomize remote bases, Terraform module sources.
+//
+// Schema (a `!external`-tagged mapping anywhere a workspace entry is
+// allowed — workspaces:, roots:, children:):
+//
+//   - !external
+//     repo: molecule-ai/molecule-dev-department
+//     ref: main
+//     path: dev-lead/workspace.yaml
+//
+// At resolve time, the platform fetches the repo at ref into a content-
+// addressable cache under <rootDir>/.external-cache/<repo>/<sha>/, loads
+// the yaml at <cacheDir>/<path>, rewrites every files_dir + relative
+// !include path to be cache-prefixed, then grafts the result in place of
+// the !external node. Downstream pipeline (resolveInsideRoot, plugin
+// merge, CopyTemplateToContainer) sees ordinary in-tree paths.
+
+// ExternalRef is the deserialized form of an `!external`-tagged mapping.
+type ExternalRef struct {
+	Repo string `yaml:"repo"`
+	Ref  string `yaml:"ref"`
+	Path string `yaml:"path"`
+
+	// URL overrides the default Gitea host. Optional; defaults to
+	// MOLECULE_EXTERNAL_GITEA_URL env or git.moleculesai.app.
+	URL string `yaml:"url,omitempty"`
+}
+
+const (
+	// maxExternalDepth caps recursion through nested `!external`s. Lower
+	// than maxIncludeDepth (16) because each level may issue a network
+	// fetch. Composition that genuinely needs >4 layers is a smell.
+	maxExternalDepth = 4
+
+	// externalCacheDirName is the per-template cache subdir under rootDir.
+	// Content-addressable: keyed by (repo, sha). Operators add this to
+	// .gitignore — cache is platform-mutated, not source-tracked.
+	externalCacheDirName = ".external-cache"
+
+	// gitFetchTimeout caps a single clone operation. Conservative —
+	// org template fetches are typically <100KB.
+	gitFetchTimeout = 60 * time.Second
+)
+
+// safeRefPattern restricts `ref` values to characters git itself accepts
+// for branch / tag / SHA. Belt-and-braces over git's own validation.
+var safeRefPattern = regexp.MustCompile(`^[a-zA-Z0-9_./-]+$`)
+
+// allowlistedHostPath returns true if `<host>/<repo>` matches the
+// configured allowlist. Default allowlist: git.moleculesai.app/molecule-ai/.
+// Override via MOLECULE_EXTERNAL_REPO_ALLOWLIST env var (comma-separated
+// patterns). Patterns are matched as prefixes (with trailing-slash
+// semantics) or as exact matches. Trailing /* is treated as "any
+// descendants of this prefix".
+//
+// Examples:
+//   - "git.moleculesai.app/molecule-ai/" → matches molecule-ai/* (any repo)
+//   - "git.moleculesai.app/molecule-ai/*" → same; trailing /* normalized to /
+//   - "git.moleculesai.app/molecule-ai/molecule-dev-department" → exact
+//   - "git.moleculesai.app/" → matches everything on that host
+func allowlistedHostPath(host, repoPath string) bool {
+	allow := os.Getenv("MOLECULE_EXTERNAL_REPO_ALLOWLIST")
+	if allow == "" {
+		allow = "git.moleculesai.app/molecule-ai/"
+	}
+	hp := host + "/" + repoPath
+	for _, pat := range strings.Split(allow, ",") {
+		pat = strings.TrimSpace(pat)
+		if pat == "" {
+			continue
+		}
+		// Normalize trailing /* → /
+		pat = strings.TrimSuffix(pat, "*")
+		if pat == hp {
+			return true
+		}
+		if strings.HasSuffix(pat, "/") && strings.HasPrefix(hp+"/", pat) {
+			return true
+		}
+	}
+	return false
+}
+
+// externalFetcher abstracts the git-clone-into-cache step. Production
+// uses gitFetcher (shells out to git); tests inject a fake that
+// pre-stages content in a temp dir.
+type externalFetcher interface {
+	// Fetch ensures rootDir/.external-cache/<safe-repo>/<sha>/ contains
+	// the repo content at the given ref. Returns the absolute cache
+	// dir + the resolved SHA. Cache hit = no network. Cache miss =
+	// clone.
+	Fetch(ctx context.Context, rootDir, host, repoPath, ref string) (cacheDir, sha string, err error)
+}
+
+// defaultExternalFetcher is the package-level fetcher injection point.
+// Production code uses the git-shell fetcher; tests override via
+// SetExternalFetcherForTest.
+var defaultExternalFetcher externalFetcher = &gitFetcher{}
+
+// SetExternalFetcherForTest swaps the fetcher for testing. Returns a
+// cleanup func that restores the previous fetcher.
+func SetExternalFetcherForTest(f externalFetcher) func() {
+	prev := defaultExternalFetcher
+	defaultExternalFetcher = f
+	return func() { defaultExternalFetcher = prev }
+}
+
+// resolveExternalMapping replaces an `!external`-tagged mapping node
+// with the loaded + path-rewritten yaml content from the fetched repo.
+//
+// `currentDir` and `rootDir` are inherited from expandNode's resolve
+// frame. `visited` tracks (repo, sha, path) tuples for cycle detection
+// across nested externals.
+func resolveExternalMapping(n *yaml.Node, currentDir, rootDir string, visited map[string]bool, depth int) error {
+	if depth > maxExternalDepth {
+		return fmt.Errorf("!external: max depth %d exceeded (possible cycle)", maxExternalDepth)
+	}
+	if rootDir == "" {
+		return fmt.Errorf("!external at line %d requires a dir-based org template (no rootDir in inline-template mode)", n.Line)
+	}
+
+	var ref ExternalRef
+	if err := n.Decode(&ref); err != nil {
+		return fmt.Errorf("!external at line %d: decode: %w", n.Line, err)
+	}
+	if ref.Repo == "" || ref.Ref == "" || ref.Path == "" {
+		return fmt.Errorf("!external at line %d: repo, ref, path are all required (got %+v)", n.Line, ref)
+	}
+	if !safeRefPattern.MatchString(ref.Ref) {
+		return fmt.Errorf("!external at line %d: ref %q contains disallowed characters", n.Line, ref.Ref)
+	}
+	// Defense-in-depth: even though git itself rejects refs containing
+	// `..`, the regex above currently allows them. Reject explicitly.
+	if strings.Contains(ref.Ref, "..") {
+		return fmt.Errorf("!external at line %d: ref %q must not contain '..'", n.Line, ref.Ref)
+	}
+	if strings.Contains(ref.Path, "..") || strings.HasPrefix(ref.Path, "/") {
+		return fmt.Errorf("!external at line %d: path %q must be relative-and-down-only", n.Line, ref.Path)
+	}
+
+	host := ref.URL
+	if host == "" {
+		host = os.Getenv("MOLECULE_EXTERNAL_GITEA_URL")
+	}
+	if host == "" {
+		host = "git.moleculesai.app"
+	}
+	host = strings.TrimPrefix(strings.TrimPrefix(host, "https://"), "http://")
+	host = strings.TrimSuffix(host, "/")
+
+	if !allowlistedHostPath(host, ref.Repo) {
+		return fmt.Errorf("!external at line %d: %s/%s not in MOLECULE_EXTERNAL_REPO_ALLOWLIST", n.Line, host, ref.Repo)
+	}
+
+	ctx, cancel := context.WithTimeout(context.Background(), gitFetchTimeout)
+	defer cancel()
+
+	cacheDir, sha, err := defaultExternalFetcher.Fetch(ctx, rootDir, host, ref.Repo, ref.Ref)
+	if err != nil {
+		return fmt.Errorf("!external at line %d: fetch %s/%s@%s: %w", n.Line, host, ref.Repo, ref.Ref, err)
+	}
+
+	// Cycle key: (repo, sha, path) — same external content reachable
+	// via two paths is fine, but a self-referential cycle isn't.
+	cycleKey := fmt.Sprintf("%s/%s@%s/%s", host, ref.Repo, sha, ref.Path)
+	if visited[cycleKey] {
+		return fmt.Errorf("!external cycle detected at %q (line %d)", cycleKey, n.Line)
+	}
+
+	// Validate path resolves inside the cache dir (anti-traversal).
+	yamlPathAbs, err := resolveInsideRoot(cacheDir, ref.Path)
+	if err != nil {
+		return fmt.Errorf("!external at line %d: path %q: %w", n.Line, ref.Path, err)
+	}
+	if _, err := os.Stat(yamlPathAbs); err != nil {
+		return fmt.Errorf("!external at line %d: %s/%s@%s does not contain %q: %w", n.Line, host, ref.Repo, sha, ref.Path, err)
+	}
+
+	data, err := os.ReadFile(yamlPathAbs)
+	if err != nil {
+		return fmt.Errorf("!external at line %d: read %q: %w", n.Line, yamlPathAbs, err)
+	}
+
+	var sub yaml.Node
+	if err := yaml.Unmarshal(data, &sub); err != nil {
+		return fmt.Errorf("!external at line %d: parse %q: %w", n.Line, yamlPathAbs, err)
+	}
+	root := &sub
+	if root.Kind == yaml.DocumentNode && len(root.Content) == 1 {
+		root = root.Content[0]
+	}
+
+	// Recurse FIRST: load all nested !include / !external content into
+	// the tree. Then rewrite ALL files_dir scalars in the fully-resolved
+	// tree (top + nested) with the cache prefix in one pass. Doing
+	// rewrite-before-recurse would leave nested-loaded files_dir paths
+	// unprefixed.
+	visited[cycleKey] = true
+	defer delete(visited, cycleKey)
+
+	subDir := filepath.Dir(yamlPathAbs)
+	if err := expandNode(root, subDir, rootDir, visited, depth+1); err != nil {
+		return err
+	}
+
+	// Path rewrite: prefix every files_dir scalar in the fully-resolved
+	// content with the cache-relative-from-rootDir prefix. After this
+	// pass, fetched workspaces look like ordinary in-tree workspaces.
+	cachePrefix, err := filepath.Rel(rootDir, cacheDir)
+	if err != nil {
+		return fmt.Errorf("!external at line %d: cannot compute cache prefix: %w", n.Line, err)
+	}
+	rewriteFilesDir(root, cachePrefix)
+
+	// Replace the !external mapping with the resolved content in-place.
+	*n = *root
+	if n.Tag == "!external" {
+		n.Tag = ""
+	}
+	return nil
+}
+
+// rewriteFilesDir walks the yaml node tree and prepends cachePrefix to
+// every files_dir scalar value. Idempotent: if a files_dir value already
+// starts with the prefix, no-op.
+//
+// !include paths are intentionally NOT rewritten. They resolve relative
+// to their containing file's directory (subDir in expandNode), and after
+// fetch that directory IS inside the cache, so relative !include paths
+// Just Work without any rewrite. Rewriting them would double-prefix on
+// recursive resolution.
+//
+// files_dir DOES need rewriting because it's consumed at workspace-
+// provisioning time relative to orgBaseDir (the parent template's root),
+// not relative to the workspace.yaml's containing dir.
+func rewriteFilesDir(n *yaml.Node, cachePrefix string) {
+	if n == nil {
+		return
+	}
+	if n.Kind == yaml.MappingNode {
+		for i := 0; i+1 < len(n.Content); i += 2 {
+			key, value := n.Content[i], n.Content[i+1]
+			if key.Kind == yaml.ScalarNode && key.Value == "files_dir" && value.Kind == yaml.ScalarNode {
+				if !strings.HasPrefix(value.Value, cachePrefix+string(filepath.Separator)) && value.Value != cachePrefix {
+					value.Value = filepath.Join(cachePrefix, value.Value)
+				}
+			}
+		}
+	}
+	for _, child := range n.Content {
+		rewriteFilesDir(child, cachePrefix)
+	}
+}
+
+// safeRepoCacheDir converts a repo path like "molecule-ai/foo" into a
+// filesystem-safe segment "molecule-ai__foo". Avoids nesting cache dirs
+// (which would complicate cleanup).
+func safeRepoCacheDir(host, repoPath string) string {
+	hp := host + "/" + repoPath
+	hp = strings.ReplaceAll(hp, "/", "__")
+	hp = strings.ReplaceAll(hp, ":", "_")
+	return hp
+}
+
+// gitFetcher is the production externalFetcher: shells out to `git` to
+// clone the repo at ref into the cache dir. Cache key includes the
+// resolved SHA, so different SHAs of the same ref get different cache
+// dirs (no overwrite).
+//
+// Token handling — important for security. The auth token never enters
+// the clone URL (and therefore never lands in the cloned repo's
+// .git/config) and never appears in returned errors. We use git's
+// `http.extraHeader` config option (passed via `-c`), which sends an
+// Authorization header per-request without persisting it. The token is
+// briefly visible in the `git` process's argv (so other local users
+// with the same uid could see it via `ps`), which is the same exposure
+// it has via the env var that supplied it.
+//
+// Cache validity uses a `.complete` marker written after a successful
+// clone+rename. Cache-hit checks for the marker, not just the dir
+// existence — a partially-written cache (clone failed mid-way, or a
+// concurrent caller wrote a half-baked cache dir) is treated as cache
+// miss and re-fetched cleanly.
+type gitFetcher struct{}
+
+// cacheCompleteMarker is the filename written after a successful clone.
+// Cache-hit requires this marker; without it, the cache dir is treated
+// as partially-written and re-fetched.
+const cacheCompleteMarker = ".complete"
+
+// Fetch resolves ref → SHA via `git ls-remote`, then `git clone --depth=1`
+// if the cache dir is missing or incomplete. Auth via MOLECULE_GITEA_TOKEN
+// injected via http.extraHeader (never via URL).
+func (g *gitFetcher) Fetch(ctx context.Context, rootDir, host, repoPath, ref string) (string, string, error) {
+	cacheRoot := filepath.Join(rootDir, externalCacheDirName, safeRepoCacheDir(host, repoPath))
+	if err := os.MkdirAll(cacheRoot, 0o755); err != nil {
+		return "", "", fmt.Errorf("mkdir cache root: %w", err)
+	}
+
+	cloneURL := buildExternalCloneURL(host, repoPath)
+	gitArgs := func(extra ...string) []string {
+		args := authConfigArgs()
+		return append(args, extra...)
+	}
+
+	// 1. Resolve ref → SHA (so cache dir is content-addressable).
+	sha, err := g.resolveRefToSHA(ctx, cloneURL, ref, gitArgs)
+	if err != nil {
+		return "", "", fmt.Errorf("ls-remote: %s", redactToken(err.Error()))
+	}
+
+	cacheDir := filepath.Join(cacheRoot, sha)
+	// Cache-hit requires the .complete marker AND the .git dir.
+	// Without the marker, cache is partially-written → treat as miss.
+	if isCacheComplete(cacheDir) {
+		return cacheDir, sha, nil
+	}
+
+	// Cache miss or partially-written — clean any stale cacheDir before
+	// cloning (a previous broken attempt would otherwise block rename).
+	os.RemoveAll(cacheDir)
+
+	// 2. Clone into a sibling tmp dir; atomic rename on success.
+	tmpDir, err := os.MkdirTemp(cacheRoot, sha+".tmp.")
+	if err != nil {
+		return "", "", fmt.Errorf("mkdir tmp: %w", err)
+	}
+	// MkdirTemp creates the dir; git clone refuses to clone into a
+	// non-empty dir. Remove + recreate empty.
+	os.RemoveAll(tmpDir)
+	cloneAndConfig := append(gitArgs("clone", "--quiet", "--depth=1", "-b", ref, cloneURL, tmpDir))
+	cmd := exec.CommandContext(ctx, "git", cloneAndConfig...)
+	cmd.Env = append(os.Environ(), "GIT_TERMINAL_PROMPT=0")
+	if out, err := cmd.CombinedOutput(); err != nil {
+		os.RemoveAll(tmpDir)
+		return "", "", fmt.Errorf("git clone: %w: %s", err, redactToken(strings.TrimSpace(string(out))))
+	}
+
+	// Write the .complete marker BEFORE the rename. If rename succeeds,
+	// the marker is in place. If rename loses the race (concurrent
+	// fetcher won), our tmp gets cleaned up and we trust the winner.
+	if err := os.WriteFile(filepath.Join(tmpDir, cacheCompleteMarker), []byte(time.Now().UTC().Format(time.RFC3339)), 0o644); err != nil {
+		os.RemoveAll(tmpDir)
+		return "", "", fmt.Errorf("write complete marker: %w", err)
+	}
+
+	if err := os.Rename(tmpDir, cacheDir); err != nil {
+		// Race: another import beat us. Validate THEIR cache, accept it.
+		os.RemoveAll(tmpDir)
+		if isCacheComplete(cacheDir) {
+			return cacheDir, sha, nil
+		}
+		return "", "", fmt.Errorf("rename clone to cache (and winner's cache is incomplete): %w", err)
+	}
+	return cacheDir, sha, nil
+}
+
+// isCacheComplete reports whether cacheDir contains both the cloned
+// repo (.git) and the .complete marker. Treats partial state as miss.
+func isCacheComplete(cacheDir string) bool {
+	if _, err := os.Stat(filepath.Join(cacheDir, ".git")); err != nil {
+		return false
+	}
+	if _, err := os.Stat(filepath.Join(cacheDir, cacheCompleteMarker)); err != nil {
+		return false
+	}
+	return true
+}
+
+func (g *gitFetcher) resolveRefToSHA(ctx context.Context, cloneURL, ref string, gitArgs func(...string) []string) (string, error) {
+	args := gitArgs("ls-remote", cloneURL, ref)
+	cmd := exec.CommandContext(ctx, "git", args...)
+	cmd.Env = append(os.Environ(), "GIT_TERMINAL_PROMPT=0")
+	out, err := cmd.Output()
+	if err != nil {
+		return "", err
+	}
+	line := strings.TrimSpace(string(out))
+	if line == "" {
+		return "", fmt.Errorf("ref %q not found", ref)
+	}
+	// First whitespace-separated field is the SHA.
+	for i, ch := range line {
+		if ch == ' ' || ch == '\t' {
+			return line[:i], nil
+		}
+	}
+	return line, nil
+}
+
+// buildExternalCloneURL constructs the clone URL WITHOUT auth in userinfo.
+// Auth is layered on via authConfigArgs's http.extraHeader.
+func buildExternalCloneURL(host, repoPath string) string {
+	u := url.URL{Scheme: "https", Host: host, Path: "/" + repoPath + ".git"}
+	return u.String()
+}
+
+// authConfigArgs returns the `-c http.extraHeader=Authorization: token X`
+// args to pass to git, OR an empty slice if no token is set. The token
+// goes into the request headers (not the URL or .git/config), so it
+// doesn't persist on disk and doesn't appear in clone error output.
+func authConfigArgs() []string {
+	token := os.Getenv("MOLECULE_GITEA_TOKEN")
+	if token == "" {
+		return nil
+	}
+	return []string{"-c", "http.extraHeader=Authorization: token " + token}
+}
+
+// redactToken scrubs the auth token from a string before it's logged
+// or returned in an error. Belt-and-braces: with the http.extraHeader
+// approach the token shouldn't appear in git's output, but if some
+// future git version or libcurl debug mode emits it, this catches it.
+func redactToken(s string) string {
+	token := os.Getenv("MOLECULE_GITEA_TOKEN")
+	if token == "" || len(token) < 8 {
+		return s
+	}
+	return strings.ReplaceAll(s, token, "<redacted-token>")
+}
+

workspace-server/internal/handlers/org_external_integration_test.go

@@ -0,0 +1,379 @@
+package handlers
+
+import (
+	"context"
+	"os"
+	"os/exec"
+	"path/filepath"
+	"runtime"
+	"strings"
+	"testing"
+
+	"gopkg.in/yaml.v3"
+)
+
+// PR-B integration test: exercises the REAL gitFetcher (no fakeFetcher
+// injection) against a local bare-git repo. Uses git's `insteadOf`
+// config to rewrite the configured Gitea URL to the local bare path
+// at clone time, so the fetcher's URL-building, ls-remote, clone,
+// atomic-rename, and cache-hit paths all run against real git
+// without requiring network or modifying production code.
+//
+// Internal#77 task #233 (PR-B from the design's phasing).
+
+// TestGitFetcher_RealClone_LocalRedirect proves the production
+// gitFetcher round-trips correctly against a real git repository.
+// Steps:
+//   1. Set up a local bare-git repo with workspace content.
+//   2. Configure git's `insteadOf` to rewrite the gitea URL → local path
+//      via GIT_CONFIG_COUNT/KEY/VALUE env vars (process-scoped).
+//   3. Run resolveYAMLIncludes with !external pointing at the gitea URL.
+//   4. Assert: cache dir populated; content materialized; path rewrite
+//      applied; second invocation hits cache (no second clone).
+func TestGitFetcher_RealClone_LocalRedirect(t *testing.T) {
+	if _, err := exec.LookPath("git"); err != nil {
+		t.Skipf("git binary not found: %v", err)
+	}
+
+	if runtime.GOOS == "windows" {
+		t.Skip("path-based git URLs behave differently on Windows; skipping")
+	}
+
+	// Step 1: create a local bare-git repo at <fixtures>/test-dev-dept.git
+	// with workspace content. Use a working clone to add content, then
+	// push to the bare.
+	fixtures := t.TempDir()
+	barePath := filepath.Join(fixtures, "test-dev-dept.git")
+	workPath := filepath.Join(fixtures, "work")
+
+	mustGit(t, "", "init", "--bare", "-b", "main", barePath)
+	mustGit(t, "", "clone", barePath, workPath)
+	mustGit(t, workPath, "config", "user.email", "test@example.com")
+	mustGit(t, workPath, "config", "user.name", "Integration Test")
+
+	mustWriteFile(t, filepath.Join(workPath, "dev-lead/workspace.yaml"), `name: Dev Lead
+files_dir: dev-lead
+children:
+  - !include ./core-be/workspace.yaml
+`)
+	mustWriteFile(t, filepath.Join(workPath, "dev-lead/system-prompt.md"), "Dev Lead persona body.\n")
+	mustWriteFile(t, filepath.Join(workPath, "dev-lead/core-be/workspace.yaml"), `name: Core BE
+files_dir: dev-lead/core-be
+`)
+	mustWriteFile(t, filepath.Join(workPath, "dev-lead/core-be/system-prompt.md"), "Core BE persona body.\n")
+
+	mustGit(t, workPath, "add", ".")
+	mustGit(t, workPath, "commit", "-m", "seed dev tree")
+	mustGit(t, workPath, "push", "origin", "main")
+
+	// Step 2: configure git's insteadOf rewrite. The fetcher will try
+	// to clone https://git.moleculesai.app/molecule-ai/test-dev-dept.git;
+	// git rewrites to file://<barePath>.
+	//
+	// GIT_CONFIG_COUNT/KEY/VALUE injects config without touching
+	// ~/.gitconfig — process-scoped, no test pollution.
+	geesUrl := "https://git.moleculesai.app/molecule-ai/test-dev-dept.git"
+	t.Setenv("GIT_CONFIG_COUNT", "1")
+	t.Setenv("GIT_CONFIG_KEY_0", "url."+barePath+".insteadOf")
+	t.Setenv("GIT_CONFIG_VALUE_0", geesUrl)
+
+	// Step 3: run resolveYAMLIncludes with !external pointing at the
+	// gitea URL. Allowlist is the default (molecule-ai/* on Gitea host).
+	rootDir := t.TempDir()
+	src := []byte(`workspaces:
+  - !external
+    repo: molecule-ai/test-dev-dept
+    ref: main
+    path: dev-lead/workspace.yaml
+`)
+
+	out, err := resolveYAMLIncludes(src, rootDir)
+	if err != nil {
+		t.Fatalf("resolveYAMLIncludes: %v", err)
+	}
+
+	var tmpl OrgTemplate
+	if err := yaml.Unmarshal(out, &tmpl); err != nil {
+		t.Fatalf("unmarshal: %v", err)
+	}
+	if len(tmpl.Workspaces) != 1 {
+		t.Fatalf("workspaces: %+v", tmpl.Workspaces)
+	}
+	dev := tmpl.Workspaces[0]
+	if dev.Name != "Dev Lead" {
+		t.Errorf("dev.Name = %q; want Dev Lead", dev.Name)
+	}
+	if !strings.Contains(dev.FilesDir, ".external-cache") {
+		t.Errorf("dev.FilesDir = %q; want cache prefix", dev.FilesDir)
+	}
+	if !strings.HasSuffix(dev.FilesDir, "dev-lead") {
+		t.Errorf("dev.FilesDir = %q; want suffix dev-lead", dev.FilesDir)
+	}
+	if len(dev.Children) != 1 {
+		t.Fatalf("expected nested core-be child; got %+v", dev.Children)
+	}
+	core := dev.Children[0]
+	if core.Name != "Core BE" {
+		t.Errorf("core.Name = %q; want Core BE", core.Name)
+	}
+	if !strings.HasSuffix(core.FilesDir, filepath.Join("dev-lead", "core-be")) {
+		t.Errorf("core.FilesDir = %q; want suffix dev-lead/core-be", core.FilesDir)
+	}
+
+	// Step 4: verify the cache dir actually exists and contains the
+	// materialized files (CopyTemplateToContainer would tar these).
+	cacheRoot := filepath.Join(rootDir, ".external-cache")
+	entries, err := os.ReadDir(cacheRoot)
+	if err != nil {
+		t.Fatalf("read cache root: %v", err)
+	}
+	if len(entries) != 1 {
+		t.Fatalf("expected 1 cached repo, got %d: %v", len(entries), entries)
+	}
+	repoDir := filepath.Join(cacheRoot, entries[0].Name())
+	shaDirs, _ := os.ReadDir(repoDir)
+	if len(shaDirs) != 1 {
+		t.Fatalf("expected 1 SHA cache dir, got %d", len(shaDirs))
+	}
+	cacheDir := filepath.Join(repoDir, shaDirs[0].Name())
+	if _, err := os.Stat(filepath.Join(cacheDir, "dev-lead/system-prompt.md")); err != nil {
+		t.Errorf("expected dev-lead/system-prompt.md in cache: %v", err)
+	}
+	if _, err := os.Stat(filepath.Join(cacheDir, "dev-lead/core-be/system-prompt.md")); err != nil {
+		t.Errorf("expected dev-lead/core-be/system-prompt.md in cache: %v", err)
+	}
+
+	// Step 5: re-run; verify cache hit (no second clone). Set a
+	// "marker" file in the cache that a second clone would clobber.
+	marker := filepath.Join(cacheDir, ".cache-hit-marker")
+	if err := os.WriteFile(marker, []byte("hit"), 0o644); err != nil {
+		t.Fatal(err)
+	}
+	out2, err := resolveYAMLIncludes(src, rootDir)
+	if err != nil {
+		t.Fatalf("resolveYAMLIncludes second call: %v", err)
+	}
+	if string(out) != string(out2) {
+		t.Errorf("cached output differs from initial — non-deterministic resolve")
+	}
+	if _, err := os.Stat(marker); err != nil {
+		t.Errorf("cache hit not honored — marker file disappeared: %v", err)
+	}
+}
+
+// TestGitFetcher_RealClone_BadRefFails: pointing at a ref that doesn't
+// exist in the bare-repo surfaces git's error cleanly.
+func TestGitFetcher_RealClone_BadRefFails(t *testing.T) {
+	if _, err := exec.LookPath("git"); err != nil {
+		t.Skipf("git binary not found: %v", err)
+	}
+	if runtime.GOOS == "windows" {
+		t.Skip("skipping on windows")
+	}
+
+	fixtures := t.TempDir()
+	barePath := filepath.Join(fixtures, "empty-repo.git")
+	workPath := filepath.Join(fixtures, "work")
+	mustGit(t, "", "init", "--bare", "-b", "main", barePath)
+	mustGit(t, "", "clone", barePath, workPath)
+	mustGit(t, workPath, "config", "user.email", "test@example.com")
+	mustGit(t, workPath, "config", "user.name", "Test")
+	mustWriteFile(t, filepath.Join(workPath, "README.md"), "x")
+	mustGit(t, workPath, "add", ".")
+	mustGit(t, workPath, "commit", "-m", "seed")
+	mustGit(t, workPath, "push", "origin", "main")
+
+	t.Setenv("GIT_CONFIG_COUNT", "1")
+	t.Setenv("GIT_CONFIG_KEY_0", "url."+barePath+".insteadOf")
+	t.Setenv("GIT_CONFIG_VALUE_0", "https://git.moleculesai.app/molecule-ai/empty-repo.git")
+
+	rootDir := t.TempDir()
+	src := []byte(`workspaces:
+  - !external
+    repo: molecule-ai/empty-repo
+    ref: nonexistent-branch
+    path: anything.yaml
+`)
+	_, err := resolveYAMLIncludes(src, rootDir)
+	if err == nil {
+		t.Fatalf("expected error for nonexistent ref; got nil")
+	}
+	if !strings.Contains(err.Error(), "ref") && !strings.Contains(err.Error(), "ls-remote") && !strings.Contains(err.Error(), "not found") {
+		t.Errorf("error doesn't mention ref/ls-remote: %v", err)
+	}
+}
+
+// ---------- helpers ----------
+
+func mustGit(t *testing.T, cwd string, args ...string) {
+	t.Helper()
+	cmd := exec.Command("git", args...)
+	if cwd != "" {
+		cmd.Dir = cwd
+	}
+	// Ensure user.email/name are set globally for non-cwd commands too.
+	cmd.Env = append(os.Environ(),
+		"GIT_AUTHOR_EMAIL=test@example.com",
+		"GIT_AUTHOR_NAME=Integration Test",
+		"GIT_COMMITTER_EMAIL=test@example.com",
+		"GIT_COMMITTER_NAME=Integration Test",
+	)
+	if out, err := cmd.CombinedOutput(); err != nil {
+		t.Fatalf("git %s: %v\n%s", strings.Join(args, " "), err, string(out))
+	}
+}
+
+func mustWriteFile(t *testing.T, path, content string) {
+	t.Helper()
+	if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
+		t.Fatal(err)
+	}
+	if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
+		t.Fatal(err)
+	}
+}
+
+// Verify gitFetcher.Fetch direct invocation (no resolver wrapping) for
+// the cache-hit path, exercising the bare API against a local bare-repo.
+func TestGitFetcher_DirectFetch_CacheHit(t *testing.T) {
+	if _, err := exec.LookPath("git"); err != nil {
+		t.Skipf("git binary not found: %v", err)
+	}
+	if runtime.GOOS == "windows" {
+		t.Skip("skipping on windows")
+	}
+
+	fixtures := t.TempDir()
+	barePath := filepath.Join(fixtures, "direct.git")
+	workPath := filepath.Join(fixtures, "w")
+	mustGit(t, "", "init", "--bare", "-b", "main", barePath)
+	mustGit(t, "", "clone", barePath, workPath)
+	mustGit(t, workPath, "config", "user.email", "t@e")
+	mustGit(t, workPath, "config", "user.name", "T")
+	mustWriteFile(t, filepath.Join(workPath, "marker.txt"), "hello")
+	mustGit(t, workPath, "add", ".")
+	mustGit(t, workPath, "commit", "-m", "seed")
+	mustGit(t, workPath, "push", "origin", "main")
+
+	t.Setenv("GIT_CONFIG_COUNT", "1")
+	t.Setenv("GIT_CONFIG_KEY_0", "url."+barePath+".insteadOf")
+	t.Setenv("GIT_CONFIG_VALUE_0", "https://git.moleculesai.app/molecule-ai/direct.git")
+
+	rootDir := t.TempDir()
+	g := &gitFetcher{}
+	ctx := context.Background()
+
+	cacheDir1, sha1, err := g.Fetch(ctx, rootDir, "git.moleculesai.app", "molecule-ai/direct", "main")
+	if err != nil {
+		t.Fatalf("first Fetch: %v", err)
+	}
+	if sha1 == "" || len(sha1) < 7 {
+		t.Errorf("expected SHA-like string, got %q", sha1)
+	}
+	if _, err := os.Stat(filepath.Join(cacheDir1, "marker.txt")); err != nil {
+		t.Errorf("first fetch missing marker.txt: %v", err)
+	}
+
+	// Second call: cache hit, returns same dir + sha, no re-clone.
+	stamp := filepath.Join(cacheDir1, ".not-clobbered-by-second-fetch")
+	if err := os.WriteFile(stamp, []byte("x"), 0o644); err != nil {
+		t.Fatal(err)
+	}
+	cacheDir2, sha2, err := g.Fetch(ctx, rootDir, "git.moleculesai.app", "molecule-ai/direct", "main")
+	if err != nil {
+		t.Fatalf("second Fetch: %v", err)
+	}
+	if cacheDir2 != cacheDir1 || sha2 != sha1 {
+		t.Errorf("cache miss on second call: %q/%q vs %q/%q", cacheDir1, sha1, cacheDir2, sha2)
+	}
+	if _, err := os.Stat(stamp); err != nil {
+		t.Errorf("cache hit not honored — stamp file disappeared: %v", err)
+	}
+}
+
+// TestGitFetcher_RejectsRefWithDoubleDot: defense-in-depth on ref input.
+// safeRefPattern allows '.' as a regex character, so ".." would match
+// without an explicit deny. Verify it's rejected even though git itself
+// would also reject the resulting clone.
+func TestGitFetcher_RejectsRefWithDoubleDot(t *testing.T) {
+	rootDir := t.TempDir()
+	src := []byte(`workspaces:
+  - !external
+    repo: molecule-ai/x
+    ref: foo..bar
+    path: x.yaml
+`)
+	_, err := resolveYAMLIncludes(src, rootDir)
+	if err == nil {
+		t.Fatalf("expected '..' rejection")
+	}
+	if !strings.Contains(err.Error(), "..") {
+		t.Errorf("expected '..' in error; got %v", err)
+	}
+}
+
+// TestGitFetcher_CacheValidatedByCompleteMarker: a partially-written
+// cache (the .git dir exists but no .complete marker) is treated as
+// cache-miss and re-fetched. Catches the broken-cache-permanence bug.
+func TestGitFetcher_CacheValidatedByCompleteMarker(t *testing.T) {
+	if _, err := exec.LookPath("git"); err != nil {
+		t.Skipf("git not found: %v", err)
+	}
+	if runtime.GOOS == "windows" {
+		t.Skip("skipping on windows")
+	}
+
+	fixtures := t.TempDir()
+	barePath := filepath.Join(fixtures, "test.git")
+	workPath := filepath.Join(fixtures, "w")
+	mustGit(t, "", "init", "--bare", "-b", "main", barePath)
+	mustGit(t, "", "clone", barePath, workPath)
+	mustGit(t, workPath, "config", "user.email", "t@e")
+	mustGit(t, workPath, "config", "user.name", "T")
+	mustWriteFile(t, filepath.Join(workPath, "good.txt"), "from-network")
+	mustGit(t, workPath, "add", ".")
+	mustGit(t, workPath, "commit", "-m", "seed")
+	mustGit(t, workPath, "push", "origin", "main")
+	t.Setenv("GIT_CONFIG_COUNT", "1")
+	t.Setenv("GIT_CONFIG_KEY_0", "url."+barePath+".insteadOf")
+	t.Setenv("GIT_CONFIG_VALUE_0", "https://git.moleculesai.app/molecule-ai/marker-test.git")
+
+	rootDir := t.TempDir()
+	g := &gitFetcher{}
+
+	// First fetch — populates the cache (creates .complete marker).
+	cacheDir1, _, err := g.Fetch(context.Background(), rootDir, "git.moleculesai.app", "molecule-ai/marker-test", "main")
+	if err != nil {
+		t.Fatalf("first Fetch: %v", err)
+	}
+	marker := filepath.Join(cacheDir1, cacheCompleteMarker)
+	if _, err := os.Stat(marker); err != nil {
+		t.Fatalf("first fetch should have written .complete marker: %v", err)
+	}
+
+	// Now simulate a partial cache: delete the marker but leave .git
+	// in place. The next Fetch should treat this as cache-miss and
+	// re-fetch (NOT silently use the partial cache).
+	if err := os.Remove(marker); err != nil {
+		t.Fatal(err)
+	}
+	// Drop a sentinel file the second fetch will clobber if it re-fetches.
+	sentinel := filepath.Join(cacheDir1, "_should_be_clobbered")
+	if err := os.WriteFile(sentinel, []byte("partial"), 0o644); err != nil {
+		t.Fatal(err)
+	}
+
+	cacheDir2, _, err := g.Fetch(context.Background(), rootDir, "git.moleculesai.app", "molecule-ai/marker-test", "main")
+	if err != nil {
+		t.Fatalf("second Fetch: %v", err)
+	}
+	if cacheDir1 != cacheDir2 {
+		t.Errorf("cache dirs differ across fetches: %q vs %q", cacheDir1, cacheDir2)
+	}
+	if _, err := os.Stat(filepath.Join(cacheDir2, cacheCompleteMarker)); err != nil {
+		t.Errorf("re-fetch should have re-written .complete marker: %v", err)
+	}
+	if _, err := os.Stat(sentinel); err == nil {
+		t.Errorf("sentinel still present — re-fetch did NOT clobber partial cache")
+	}
+}

workspace-server/internal/handlers/org_external_test.go

@@ -0,0 +1,331 @@
+package handlers
+
+import (
+	"context"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+
+	"gopkg.in/yaml.v3"
+)
+
+// fakeFetcher pre-stages a "fetched" repo at a fixed path inside the
+// rootDir's .external-cache, bypassing the real git clone. Tests
+// inject this via SetExternalFetcherForTest to exercise the resolver
+// + path-rewrite logic without network.
+type fakeFetcher struct {
+	// content maps "<host>/<repo>@<ref>" → a function that materializes
+	// repo content under cacheDir. Returns the fake SHA to use.
+	content map[string]func(cacheDir string) (sha string, err error)
+}
+
+func (f *fakeFetcher) Fetch(ctx context.Context, rootDir, host, repoPath, ref string) (string, string, error) {
+	key := host + "/" + repoPath + "@" + ref
+	stage, ok := f.content[key]
+	if !ok {
+		return "", "", &fakeNotFoundError{key: key}
+	}
+	// Use a stable SHA for the test so cache dir is deterministic.
+	cacheDir := filepath.Join(rootDir, ".external-cache", safeRepoCacheDir(host, repoPath), "deadbeef")
+	if err := os.MkdirAll(cacheDir, 0o755); err != nil {
+		return "", "", err
+	}
+	sha, err := stage(cacheDir)
+	if err != nil {
+		return "", "", err
+	}
+	return cacheDir, sha, nil
+}
+
+type fakeNotFoundError struct{ key string }
+
+func (e *fakeNotFoundError) Error() string {
+	return "fake fetcher: no content registered for " + e.key
+}
+
+// stageFiles writes a map of relative-path → content into cacheDir,
+// returning a fake SHA. Helper for fakeFetcher closures.
+func stageFiles(cacheDir string, files map[string]string) error {
+	if err := os.MkdirAll(filepath.Join(cacheDir, ".git"), 0o755); err != nil {
+		return err
+	}
+	for path, content := range files {
+		full := filepath.Join(cacheDir, path)
+		if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil {
+			return err
+		}
+		if err := os.WriteFile(full, []byte(content), 0o644); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+// TestResolveExternalMapping_HappyPath: a parent template with an
+// !external entry resolves cleanly into the fetched workspace + path-
+// rewrites files_dir + relative !include refs into the cache prefix.
+func TestResolveExternalMapping_HappyPath(t *testing.T) {
+	tmp := t.TempDir()
+
+	// Stub fetcher: "fetched" content has a workspace.yaml that uses
+	// files_dir + nested !include relative to the fetched repo's root.
+	fake := &fakeFetcher{
+		content: map[string]func(string) (string, error){
+			"git.moleculesai.app/molecule-ai/molecule-dev-department@main": func(cacheDir string) (string, error) {
+				return "deadbeef", stageFiles(cacheDir, map[string]string{
+					"dev-lead/workspace.yaml": `name: Dev Lead
+files_dir: dev-lead
+children:
+  - !include ./core-lead/workspace.yaml
+`,
+					"dev-lead/core-lead/workspace.yaml": `name: Core Platform Lead
+files_dir: dev-lead/core-lead
+`,
+				})
+			},
+		},
+	}
+	cleanup := SetExternalFetcherForTest(fake)
+	defer cleanup()
+
+	src := []byte(`name: Parent
+workspaces:
+  - !external
+    repo: molecule-ai/molecule-dev-department
+    ref: main
+    path: dev-lead/workspace.yaml
+`)
+
+	out, err := resolveYAMLIncludes(src, tmp)
+	if err != nil {
+		t.Fatalf("resolveYAMLIncludes: %v", err)
+	}
+
+	var tmpl OrgTemplate
+	if err := yaml.Unmarshal(out, &tmpl); err != nil {
+		t.Fatalf("unmarshal: %v", err)
+	}
+	if len(tmpl.Workspaces) != 1 {
+		t.Fatalf("workspaces: %+v", tmpl.Workspaces)
+	}
+	dev := tmpl.Workspaces[0]
+	if dev.Name != "Dev Lead" {
+		t.Errorf("dev.Name = %q; want Dev Lead", dev.Name)
+	}
+	// files_dir should be cache-prefixed.
+	wantPrefix := filepath.Join(".external-cache", "git.moleculesai.app__molecule-ai__molecule-dev-department", "deadbeef")
+	if !strings.HasPrefix(dev.FilesDir, wantPrefix) {
+		t.Errorf("dev.FilesDir = %q; want prefix %q", dev.FilesDir, wantPrefix)
+	}
+	if !strings.HasSuffix(dev.FilesDir, "dev-lead") {
+		t.Errorf("dev.FilesDir = %q; want suffix dev-lead", dev.FilesDir)
+	}
+	// Nested child: files_dir cache-prefixed, name Core Platform Lead.
+	if len(dev.Children) != 1 {
+		t.Fatalf("dev.Children: %+v", dev.Children)
+	}
+	core := dev.Children[0]
+	if core.Name != "Core Platform Lead" {
+		t.Errorf("core.Name = %q; want Core Platform Lead", core.Name)
+	}
+	if !strings.HasPrefix(core.FilesDir, wantPrefix) {
+		t.Errorf("core.FilesDir = %q; want prefix %q", core.FilesDir, wantPrefix)
+	}
+	if !strings.HasSuffix(core.FilesDir, filepath.Join("dev-lead", "core-lead")) {
+		t.Errorf("core.FilesDir = %q; want suffix dev-lead/core-lead", core.FilesDir)
+	}
+}
+
+// TestResolveExternalMapping_AllowlistRejection: hostile yaml pointing
+// at a non-allowlisted repo gets rejected.
+func TestResolveExternalMapping_AllowlistRejection(t *testing.T) {
+	tmp := t.TempDir()
+	fake := &fakeFetcher{content: map[string]func(string) (string, error){}}
+	cleanup := SetExternalFetcherForTest(fake)
+	defer cleanup()
+
+	// Default allowlist is git.moleculesai.app/molecule-ai/*.
+	// github.com/foo/bar is NOT in it.
+	src := []byte(`workspaces:
+  - !external
+    repo: foo/bar
+    ref: main
+    path: x.yaml
+    url: github.com
+`)
+	_, err := resolveYAMLIncludes(src, tmp)
+	if err == nil {
+		t.Fatalf("expected allowlist rejection, got nil")
+	}
+	if !strings.Contains(err.Error(), "MOLECULE_EXTERNAL_REPO_ALLOWLIST") {
+		t.Errorf("expected allowlist error; got %v", err)
+	}
+}
+
+// TestResolveExternalMapping_PathTraversalRejection: hostile yaml
+// with `path: ../../etc/passwd` gets rejected before fetch.
+func TestResolveExternalMapping_PathTraversalRejection(t *testing.T) {
+	tmp := t.TempDir()
+	fake := &fakeFetcher{content: map[string]func(string) (string, error){}}
+	cleanup := SetExternalFetcherForTest(fake)
+	defer cleanup()
+
+	src := []byte(`workspaces:
+  - !external
+    repo: molecule-ai/dev-department
+    ref: main
+    path: ../../etc/passwd
+`)
+	_, err := resolveYAMLIncludes(src, tmp)
+	if err == nil {
+		t.Fatalf("expected path traversal rejection, got nil")
+	}
+	if !strings.Contains(err.Error(), "relative-and-down-only") {
+		t.Errorf("expected path traversal error; got %v", err)
+	}
+}
+
+// TestResolveExternalMapping_BadRefRejection: non-allowlisted ref chars.
+func TestResolveExternalMapping_BadRefRejection(t *testing.T) {
+	tmp := t.TempDir()
+	fake := &fakeFetcher{content: map[string]func(string) (string, error){}}
+	cleanup := SetExternalFetcherForTest(fake)
+	defer cleanup()
+
+	src := []byte(`workspaces:
+  - !external
+    repo: molecule-ai/dev-department
+    ref: "main; rm -rf /"
+    path: foo.yaml
+`)
+	_, err := resolveYAMLIncludes(src, tmp)
+	if err == nil || !strings.Contains(err.Error(), "disallowed characters") {
+		t.Errorf("expected ref-validation error; got %v", err)
+	}
+}
+
+// TestResolveExternalMapping_MissingRequiredFields: repo / ref / path
+// are all required.
+func TestResolveExternalMapping_MissingRequiredFields(t *testing.T) {
+	tmp := t.TempDir()
+	fake := &fakeFetcher{content: map[string]func(string) (string, error){}}
+	cleanup := SetExternalFetcherForTest(fake)
+	defer cleanup()
+
+	cases := []string{
+		// missing repo
+		`workspaces:
+  - !external
+    ref: main
+    path: x.yaml
+`,
+		// missing ref
+		`workspaces:
+  - !external
+    repo: molecule-ai/x
+    path: x.yaml
+`,
+		// missing path
+		`workspaces:
+  - !external
+    repo: molecule-ai/x
+    ref: main
+`,
+	}
+	for i, src := range cases {
+		_, err := resolveYAMLIncludes([]byte(src), tmp)
+		if err == nil {
+			t.Errorf("case %d: expected required-field error, got nil", i)
+		} else if !strings.Contains(err.Error(), "required") {
+			t.Errorf("case %d: want 'required' in error; got %v", i, err)
+		}
+	}
+}
+
+// TestRewriteFilesDir: verify the path-rewrite walker
+// prefixes files_dir scalars. !include scalars are NOT rewritten —
+// they resolve relative to their containing file's dir, which post-
+// fetch is naturally inside the cache.
+func TestRewriteFilesDir(t *testing.T) {
+	src := `name: Foo
+files_dir: dev-lead
+children:
+  - !include ./bar/workspace.yaml
+  - !include other-team.yaml
+inner:
+  files_dir: dev-lead/sub
+`
+	var n yaml.Node
+	if err := yaml.Unmarshal([]byte(src), &n); err != nil {
+		t.Fatal(err)
+	}
+	rewriteFilesDir(&n, ".external-cache/foo/bar")
+
+	out, err := yaml.Marshal(&n)
+	if err != nil {
+		t.Fatal(err)
+	}
+	got := string(out)
+	for _, want := range []string{
+		"files_dir: .external-cache/foo/bar/dev-lead",
+		"files_dir: .external-cache/foo/bar/dev-lead/sub",
+		// !include preserved as-is; resolves naturally via subDir.
+		"!include ./bar/workspace.yaml",
+		"!include other-team.yaml",
+	} {
+		if !strings.Contains(got, want) {
+			t.Errorf("missing %q in:\n%s", want, got)
+		}
+	}
+}
+
+// TestRewriteFilesDir_Idempotent: re-running the rewriter
+// on already-prefixed files_dir doesn't double-prefix.
+func TestRewriteFilesDir_Idempotent(t *testing.T) {
+	src := `files_dir: .external-cache/foo/bar/dev-lead
+inner:
+  files_dir: .external-cache/foo/bar/dev-lead/sub
+`
+	var n yaml.Node
+	if err := yaml.Unmarshal([]byte(src), &n); err != nil {
+		t.Fatal(err)
+	}
+	rewriteFilesDir(&n, ".external-cache/foo/bar")
+
+	out, _ := yaml.Marshal(&n)
+	got := string(out)
+	if strings.Contains(got, ".external-cache/foo/bar/.external-cache") {
+		t.Errorf("double-prefix detected:\n%s", got)
+	}
+	// Should still be valid (single-prefixed) afterwards.
+	for _, want := range []string{
+		"files_dir: .external-cache/foo/bar/dev-lead",
+		"files_dir: .external-cache/foo/bar/dev-lead/sub",
+	} {
+		if !strings.Contains(got, want) {
+			t.Errorf("expected unchanged %q in:\n%s", want, got)
+		}
+	}
+}
+
+// TestAllowlistedHostPath: env-var override + glob matching.
+func TestAllowlistedHostPath(t *testing.T) {
+	t.Setenv("MOLECULE_EXTERNAL_REPO_ALLOWLIST", "")
+	if !allowlistedHostPath("git.moleculesai.app", "molecule-ai/foo") {
+		t.Error("default allowlist should accept molecule-ai/*")
+	}
+	if allowlistedHostPath("github.com", "molecule-ai/foo") {
+		t.Error("default allowlist should reject github.com")
+	}
+	t.Setenv("MOLECULE_EXTERNAL_REPO_ALLOWLIST", "github.com/me/*,git.moleculesai.app/*")
+	if !allowlistedHostPath("github.com", "me/x") {
+		t.Error("override should accept github.com/me/*")
+	}
+	if !allowlistedHostPath("git.moleculesai.app", "any/repo") {
+		t.Error("override should accept git.moleculesai.app/*")
+	}
+	if allowlistedHostPath("github.com", "evil/x") {
+		t.Error("override should reject github.com/evil/*")
+	}
+}

workspace-server/internal/handlers/org_helpers.go

@@ -6,6 +6,7 @@ package handlers
 
 import (
 	"fmt"
+	"log"
 	"os"
 	"path/filepath"
 	"regexp"
@@ -102,6 +103,56 @@ func loadWorkspaceEnv(orgBaseDir, filesDir string) map[string]string {
 	return envVars
 }
 
+// loadPersonaEnvFile merges per-role persona credentials into out. The file
+// lives at $MOLECULE_PERSONA_ROOT/<role>/env (default
+// /etc/molecule-bootstrap/personas) and is populated by the operator-host
+// bootstrap kit — one persona per dev-tree role, each carrying the role's
+// Gitea identity (GITEA_USER, GITEA_TOKEN, GITEA_TOKEN_SCOPES,
+// GITEA_USER_EMAIL, GITEA_SSH_KEY_PATH).
+//
+// Lower precedence than the org and workspace .env files: callers should
+// invoke this BEFORE parseEnvFile on those, so a workspace .env can
+// override a persona-default value when needed.
+//
+// Silent no-op when role is empty, when the role name fails the safe-segment
+// check, or when the env file does not exist (workspaces without a role —
+// or running on hosts that don't ship the bootstrap dir — keep their old
+// behavior).
+func loadPersonaEnvFile(role string, out map[string]string) {
+	if !isSafeRoleName(role) {
+		if role != "" {
+			log.Printf("Org import: refusing persona env load for unsafe role name %q", role)
+		}
+		return
+	}
+	root := os.Getenv("MOLECULE_PERSONA_ROOT")
+	if root == "" {
+		root = "/etc/molecule-bootstrap/personas"
+	}
+	parseEnvFile(filepath.Join(root, role, "env"), out)
+}
+
+// isSafeRoleName accepts a single path segment of [A-Za-z0-9_-]+. Rejects
+// empty, ".", "..", and anything containing a path separator — even though
+// the construct is admin-only, defense-in-depth keeps the persona dir
+// shape invariant: one flat directory per role, no climbing out.
+func isSafeRoleName(s string) bool {
+	if s == "" || s == "." || s == ".." {
+		return false
+	}
+	for _, c := range s {
+		switch {
+		case c >= 'a' && c <= 'z':
+		case c >= 'A' && c <= 'Z':
+		case c >= '0' && c <= '9':
+		case c == '-' || c == '_':
+		default:
+			return false
+		}
+	}
+	return true
+}
+
 // parseEnvFile reads a .env file and adds KEY=VALUE pairs to the map.
 // Skips comments (#) and empty lines. Values can be quoted.
 func parseEnvFile(path string, out map[string]string) {

workspace-server/internal/handlers/org_import.go

@@ -443,10 +443,18 @@ func (h *OrgHandler) createWorkspaceTree(ws OrgWorkspace, parentID *string, absX
 			configFiles["system-prompt.md"] = []byte(ws.SystemPrompt)
 		}
 
-		// Inject secrets from .env files as workspace secrets.
-		// Resolution: workspace .env → org root .env (workspace overrides org root).
+		// Inject secrets from persona env + .env files as workspace secrets.
+		// Resolution (later overrides earlier):
+		//   0. Persona env (per-role bootstrap creds; only when ws.Role is set
+		//      and the operator-host bootstrap dir ships a matching file)
+		//   1. Org root .env (shared defaults)
+		//   2. Workspace-specific .env (per-workspace overrides)
 		// Each line: KEY=VALUE → stored as encrypted workspace secret.
 		envVars := map[string]string{}
+		// 0. Persona env (lowest precedence; injects the role's Gitea identity:
+		//    GITEA_USER, GITEA_TOKEN, GITEA_TOKEN_SCOPES, GITEA_USER_EMAIL,
+		//    GITEA_SSH_KEY_PATH). Workspace and org .env can override.
+		loadPersonaEnvFile(ws.Role, envVars)
 		if orgBaseDir != "" {
 			// 1. Org root .env (shared defaults)
 			parseEnvFile(filepath.Join(orgBaseDir, ".env"), envVars)

workspace-server/internal/handlers/org_include.go

@@ -76,6 +76,12 @@ func expandNode(n *yaml.Node, currentDir, rootDir string, visited map[string]boo
 		return resolveIncludeScalar(n, currentDir, rootDir, visited, depth)
 	}
 
+	// `!external`-tagged mapping: gitops cross-repo subtree composition.
+	// See org_external.go (internal#77 / task #222).
+	if n.Kind == yaml.MappingNode && n.Tag == "!external" {
+		return resolveExternalMapping(n, currentDir, rootDir, visited, depth)
+	}
+
 	for _, child := range n.Content {
 		if err := expandNode(child, currentDir, rootDir, visited, depth); err != nil {
 			return err

workspace-server/internal/handlers/org_include_symlink_test.go

@@ -0,0 +1,136 @@
+package handlers
+
+import (
+	"os"
+	"path/filepath"
+	"testing"
+
+	"gopkg.in/yaml.v3"
+)
+
+// Phase 5 (RFC internal#77 dev-department extraction):
+// Proves a parent org template can compose a subtree from a sibling repo
+// via a directory symlink. Pattern that gets shipped:
+//
+//   /org-templates/parent-template/                  ← imported by POST /org/import
+//     org.yaml                                       (workspaces: !include dev/dev-lead/workspace.yaml)
+//     dev → /org-templates/molecule-dev-department/  (symlink)
+//   /org-templates/molecule-dev-department/          (sibling repo)
+//     dev-lead/
+//       workspace.yaml                               (children: !include ./core-platform/workspace.yaml)
+//       core-platform/
+//         workspace.yaml
+//
+// resolveYAMLIncludes resolves paths via filepath.Abs/Rel (no symlink
+// following at the path-string layer), so the security check passes. The
+// actual file open uses os.ReadFile, which DOES follow symlinks — so the
+// content from the sibling repo gets inlined. This test pins that contract.
+func TestResolveYAMLIncludes_FollowsDirectorySymlink(t *testing.T) {
+	tmp := t.TempDir()
+
+	// Subtree repo: dev-department/dev-lead/...
+	devDept := filepath.Join(tmp, "molecule-dev-department")
+	devLead := filepath.Join(devDept, "dev-lead")
+	corePlatform := filepath.Join(devLead, "core-platform")
+	if err := os.MkdirAll(corePlatform, 0o755); err != nil {
+		t.Fatal(err)
+	}
+	// dev-lead/workspace.yaml — uses `./core-platform/workspace.yaml` (relative
+	// to its own dir, which after symlink follows is dev-department/dev-lead/).
+	devLeadYAML := []byte(`name: Dev Lead
+tier: 3
+children:
+  - !include ./core-platform/workspace.yaml
+`)
+	if err := os.WriteFile(filepath.Join(devLead, "workspace.yaml"), devLeadYAML, 0o644); err != nil {
+		t.Fatal(err)
+	}
+	if err := os.WriteFile(filepath.Join(corePlatform, "workspace.yaml"), []byte("name: Core Platform\ntier: 3\n"), 0o644); err != nil {
+		t.Fatal(err)
+	}
+
+	// Parent template: parent/, with `dev` symlink → ../molecule-dev-department/
+	parent := filepath.Join(tmp, "parent-template")
+	if err := os.MkdirAll(parent, 0o755); err != nil {
+		t.Fatal(err)
+	}
+	// Symlink TARGET is a relative path (matches operator-side deploy
+	// convention where both repos are cloned as siblings under a shared
+	// /org-templates/ dir).
+	if err := os.Symlink("../molecule-dev-department", filepath.Join(parent, "dev")); err != nil {
+		t.Skipf("symlinks unsupported on this fs: %v", err)
+	}
+
+	// Parent's org.yaml: !include into the symlinked subtree.
+	src := []byte(`name: Parent
+workspaces:
+  - !include dev/dev-lead/workspace.yaml
+`)
+
+	out, err := resolveYAMLIncludes(src, parent)
+	if err != nil {
+		t.Fatalf("resolveYAMLIncludes through symlink failed: %v", err)
+	}
+
+	var tmpl OrgTemplate
+	if err := yaml.Unmarshal(out, &tmpl); err != nil {
+		t.Fatalf("unmarshal: %v", err)
+	}
+	if len(tmpl.Workspaces) != 1 {
+		t.Fatalf("expected 1 workspace, got %d", len(tmpl.Workspaces))
+	}
+	if tmpl.Workspaces[0].Name != "Dev Lead" {
+		t.Fatalf("workspace[0].Name = %q; want Dev Lead", tmpl.Workspaces[0].Name)
+	}
+	kids := tmpl.Workspaces[0].Children
+	if len(kids) != 1 {
+		t.Fatalf("expected 1 child workspace, got %d", len(kids))
+	}
+	if kids[0].Name != "Core Platform" {
+		t.Fatalf("child[0].Name = %q; want Core Platform — symlink-aware nested !include broken", kids[0].Name)
+	}
+}
+
+// Companion: prove the security check still works when the symlink target
+// is OUTSIDE the parent template's root. This is the "hostile symlink"
+// case — an org.yaml that tries to slip in arbitrary files from /etc.
+func TestResolveYAMLIncludes_RejectsSymlinkEscapingRoot(t *testing.T) {
+	tmp := t.TempDir()
+	parent := filepath.Join(tmp, "parent-template")
+	outside := filepath.Join(tmp, "outside")
+	if err := os.MkdirAll(parent, 0o755); err != nil {
+		t.Fatal(err)
+	}
+	if err := os.MkdirAll(outside, 0o755); err != nil {
+		t.Fatal(err)
+	}
+	if err := os.WriteFile(filepath.Join(outside, "evil.yaml"), []byte("name: Evil\n"), 0o644); err != nil {
+		t.Fatal(err)
+	}
+
+	// Symlink that escapes the parent root via `../outside/...`. The path
+	// STRING `evil` resolves to parent/evil — passes the rel2 check. But
+	// because filepath.Abs doesn't follow symlinks, the ReadFile call DOES
+	// follow it to outside/evil.yaml. This is the trade-off the symlink
+	// approach accepts: the security boundary is a deployment-layer
+	// invariant, not a code-layer one. Documented in dev-department/README.
+	if err := os.Symlink(filepath.Join(outside, "evil.yaml"), filepath.Join(parent, "evil.yaml")); err != nil {
+		t.Skipf("symlinks unsupported on this fs: %v", err)
+	}
+	src := []byte("workspaces:\n  - !include evil.yaml\n")
+	out, err := resolveYAMLIncludes(src, parent)
+	if err != nil {
+		// If the resolver is later hardened to refuse symlink targets
+		// outside the root (e.g. via filepath.EvalSymlinks), this test
+		// will start failing — and the dev-department symlink approach
+		// would need to be updated accordingly.
+		t.Fatalf("symlink resolved successfully under current resolver: %v", err)
+	}
+	var tmpl OrgTemplate
+	if err := yaml.Unmarshal(out, &tmpl); err != nil {
+		t.Fatalf("unmarshal: %v", err)
+	}
+	if len(tmpl.Workspaces) != 1 || tmpl.Workspaces[0].Name != "Evil" {
+		t.Fatalf("expected Evil workspace via symlink; got %+v", tmpl.Workspaces)
+	}
+}

workspace-server/internal/handlers/org_persona_env_test.go

@@ -0,0 +1,171 @@
+package handlers
+
+import (
+	"os"
+	"path/filepath"
+	"testing"
+)
+
+// TestLoadPersonaEnvFile_HappyPath: the standard case — a persona-shaped
+// env file exists at <root>/<role>/env and its KEY=VALUE pairs land in
+// the out map. Mirrors what the operator-host bootstrap kit ships:
+// GITEA_USER, GITEA_TOKEN, GITEA_TOKEN_SCOPES, GITEA_USER_EMAIL,
+// GITEA_SSH_KEY_PATH.
+func TestLoadPersonaEnvFile_HappyPath(t *testing.T) {
+	root := t.TempDir()
+	roleDir := filepath.Join(root, "dev-lead")
+	if err := os.MkdirAll(roleDir, 0o755); err != nil {
+		t.Fatal(err)
+	}
+	envBody := `# Persona env file — mode 600
+GITEA_USER=dev-lead
+GITEA_USER_EMAIL=dev-lead@agents.moleculesai.app
+GITEA_TOKEN=abc123
+GITEA_TOKEN_SCOPES=write:repository,write:issue,read:user
+GITEA_SSH_KEY_PATH=/etc/molecule-bootstrap/personas/dev-lead/ssh_priv
+`
+	if err := os.WriteFile(filepath.Join(roleDir, "env"), []byte(envBody), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	t.Setenv("MOLECULE_PERSONA_ROOT", root)
+
+	out := map[string]string{}
+	loadPersonaEnvFile("dev-lead", out)
+
+	want := map[string]string{
+		"GITEA_USER":           "dev-lead",
+		"GITEA_USER_EMAIL":     "dev-lead@agents.moleculesai.app",
+		"GITEA_TOKEN":          "abc123",
+		"GITEA_TOKEN_SCOPES":   "write:repository,write:issue,read:user",
+		"GITEA_SSH_KEY_PATH":   "/etc/molecule-bootstrap/personas/dev-lead/ssh_priv",
+	}
+	if len(out) != len(want) {
+		t.Fatalf("got %d keys, want %d: %#v", len(out), len(want), out)
+	}
+	for k, v := range want {
+		if out[k] != v {
+			t.Errorf("out[%q] = %q; want %q", k, out[k], v)
+		}
+	}
+}
+
+// TestLoadPersonaEnvFile_MissingDir: when the persona dir doesn't exist
+// (e.g. dev-only host without the bootstrap kit, or a workspace whose
+// role isn't a known persona), it's a silent no-op — out stays empty,
+// no panic, no log noise that would break callers.
+func TestLoadPersonaEnvFile_MissingDir(t *testing.T) {
+	t.Setenv("MOLECULE_PERSONA_ROOT", t.TempDir()) // empty dir
+	out := map[string]string{}
+	loadPersonaEnvFile("nonexistent-role", out)
+	if len(out) != 0 {
+		t.Errorf("expected empty out, got %#v", out)
+	}
+}
+
+// TestLoadPersonaEnvFile_EmptyRole: empty role string is the common case
+// for non-dev workspaces (research/marketing/etc.). Skip silently.
+func TestLoadPersonaEnvFile_EmptyRole(t *testing.T) {
+	t.Setenv("MOLECULE_PERSONA_ROOT", t.TempDir())
+	out := map[string]string{}
+	loadPersonaEnvFile("", out)
+	if len(out) != 0 {
+		t.Errorf("empty role should produce empty out; got %#v", out)
+	}
+}
+
+// TestLoadPersonaEnvFile_RejectsTraversal: even though role names come
+// from server-side admin-only org templates, defense-in-depth — refuse
+// any role string with path separators or "..". Verifies that a maliciously
+// crafted template can't read /etc/passwd by setting role: "../../etc".
+func TestLoadPersonaEnvFile_RejectsTraversal(t *testing.T) {
+	root := t.TempDir()
+	// Plant a file at /tmp/.../env so a bad traversal would reach it
+	if err := os.WriteFile(filepath.Join(root, "env"), []byte("STOLEN=yes\n"), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	t.Setenv("MOLECULE_PERSONA_ROOT", filepath.Join(root, "personas"))
+
+	for _, bad := range []string{"..", "../personas", "../etc/passwd", "/abs", "with/slash", "dot.in.middle", "with space", "back\\slash", ".", ""} {
+		out := map[string]string{}
+		loadPersonaEnvFile(bad, out)
+		if len(out) != 0 {
+			t.Errorf("role %q should have been rejected; got %#v", bad, out)
+		}
+	}
+}
+
+// TestLoadPersonaEnvFile_DefaultRoot: when MOLECULE_PERSONA_ROOT is unset,
+// the helper falls back to /etc/molecule-bootstrap/personas. We don't
+// touch real /etc — just verify the function doesn't panic and produces
+// empty out (since the test box isn't expected to ship that path).
+func TestLoadPersonaEnvFile_DefaultRoot(t *testing.T) {
+	t.Setenv("MOLECULE_PERSONA_ROOT", "") // explicit empty
+	out := map[string]string{}
+	loadPersonaEnvFile("dev-lead", out)
+	// Don't assert content — production CI might or might not have the
+	// /etc dir mounted. Just verify the call returns cleanly.
+	_ = out
+}
+
+// TestLoadPersonaEnvFile_PrecedenceCallerOverrides: the contract is "lower
+// precedence than later .env files." The helper writes into out without
+// removing existing keys, so a caller pre-populating out simulates a
+// later layer overriding persona defaults. We verify the helper does NOT
+// clobber pre-existing entries… actually, parseEnvFile DOES overwrite,
+// so the caller-side ordering (persona → org → workspace) is what enforces
+// precedence. This test pins that contract: persona is loaded into a
+// fresh map, then later layers can override.
+func TestLoadPersonaEnvFile_OverwritesEmptyMap(t *testing.T) {
+	root := t.TempDir()
+	roleDir := filepath.Join(root, "core-be")
+	if err := os.MkdirAll(roleDir, 0o755); err != nil {
+		t.Fatal(err)
+	}
+	if err := os.WriteFile(filepath.Join(roleDir, "env"),
+		[]byte("GITEA_TOKEN=persona-value\n"), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	t.Setenv("MOLECULE_PERSONA_ROOT", root)
+
+	out := map[string]string{"GITEA_TOKEN": "preset"}
+	loadPersonaEnvFile("core-be", out)
+
+	// Persona helper is meant to populate a FRESH map first in the
+	// caller's flow; calling it on a pre-populated map and seeing the
+	// value get overwritten is consistent with parseEnvFile semantics.
+	if out["GITEA_TOKEN"] != "persona-value" {
+		t.Errorf("loadPersonaEnvFile did not write into existing map; got %q", out["GITEA_TOKEN"])
+	}
+}
+
+// TestIsSafeRoleName_Acceptance: positive + negative cases for the
+// validator. Pinned because every dev-tree role name must pass.
+func TestIsSafeRoleName_Acceptance(t *testing.T) {
+	good := []string{
+		"dev-lead", "core-be", "cp-security", "infra-runtime-be",
+		"sdk-dev", "plugin-dev", "documentation-specialist",
+		"triage-operator", "fullstack-engineer", "release-manager",
+		"core_underscore_ok", "X", "a1", "Z9-0",
+	}
+	for _, s := range good {
+		if !isSafeRoleName(s) {
+			t.Errorf("isSafeRoleName(%q) = false; want true", s)
+		}
+	}
+	bad := []string{
+		"", ".", "..", "with/slash", "/abs", "dot.in.middle",
+		"with space", "back\\slash", "trailing-", // trailing-hyphen is fine actually
+		"with$dollar", "with?question", "newline\nsplit",
+	}
+	// trailing-hyphen IS allowed; remove from "bad" list:
+	bad = []string{
+		"", ".", "..", "with/slash", "/abs", "dot.in.middle",
+		"with space", "back\\slash", "with$dollar", "with?question",
+		"newline\nsplit",
+	}
+	for _, s := range bad {
+		if isSafeRoleName(s) {
+			t.Errorf("isSafeRoleName(%q) = true; want false", s)
+		}
+	}
+}

workspace-server/internal/handlers/plugins.go

@@ -23,6 +23,16 @@ import (
 // workspace-scoped filtering (handler falls back to unfiltered list).
 type RuntimeLookup func(workspaceID string) (string, error)
 
+// InstanceIDLookup resolves a workspace's EC2 instance_id by ID. Empty
+// string means the workspace is not on the SaaS (EC2-per-workspace)
+// backend — i.e. either local-Docker or pre-provision. The handler uses
+// this to dispatch plugin install/uninstall to the EIC SSH path
+// (template_files_eic.go primitive) when a workspace runs on its own EC2
+// and there's no local Docker container to exec into. A nil lookup keeps
+// the handler on the local-Docker code path only — same shape as the
+// pre-fix behaviour.
+type InstanceIDLookup func(workspaceID string) (string, error)
+
 // pluginSources is the contract PluginsHandler uses to talk to the
 // plugin source registry. Extracted as an interface (#1814) so tests can
 // substitute a stub without standing up the real *plugins.Registry +
@@ -46,10 +56,11 @@ var _ pluginSources = (*plugins.Registry)(nil)
 
 // PluginsHandler manages the plugin registry and per-workspace plugin installation.
 type PluginsHandler struct {
-	pluginsDir    string         // host path to plugins/ registry
-	docker        *client.Client // Docker client for container operations
-	restartFunc   func(string)   // auto-restart workspace after install/uninstall
-	runtimeLookup RuntimeLookup  // workspace_id → runtime (optional)
+	pluginsDir       string           // host path to plugins/ registry
+	docker           *client.Client   // Docker client for container operations
+	restartFunc      func(string)     // auto-restart workspace after install/uninstall
+	runtimeLookup    RuntimeLookup    // workspace_id → runtime (optional)
+	instanceIDLookup InstanceIDLookup // workspace_id → EC2 instance_id (optional)
 	// sources narrowed from `*plugins.Registry` to the pluginSources
 	// interface (#1814) so tests can substitute a stub. Production
 	// callers still pass *plugins.Registry, which satisfies the
@@ -90,6 +101,15 @@ func (h *PluginsHandler) WithRuntimeLookup(lookup RuntimeLookup) *PluginsHandler
 	return h
 }
 
+// WithInstanceIDLookup installs a workspace → EC2 instance_id resolver.
+// Wired by the router so production hits a real DB; tests stub it. The
+// install/uninstall pipeline uses this to dispatch to the EIC SSH path
+// for SaaS workspaces (no local Docker container to exec into).
+func (h *PluginsHandler) WithInstanceIDLookup(lookup InstanceIDLookup) *PluginsHandler {
+	h.instanceIDLookup = lookup
+	return h
+}
+
 // pluginInfo is the API response for a plugin.
 type pluginInfo struct {
 	Name        string   `json:"name"`

workspace-server/internal/handlers/plugins_atomic.go

@@ -0,0 +1,207 @@
+package handlers
+
+// plugins_atomic.go — atomic install pattern for plugin delivery into a
+// running workspace container. Closes molecule-core#114.
+//
+// Replaces the prior "tar + docker.CopyToContainer to /configs/plugins/<name>"
+// single-step write (no atomicity, no marker, no rollback) with a 4-step
+// dance:
+//
+//   1. STAGE     — extract tar into /configs/plugins/.staging/<name>.<ts>/
+//   2. SNAPSHOT  — if /configs/plugins/<name>/ exists, mv to .previous/<name>.<ts>/
+//   3. SWAP      — mv /configs/plugins/.staging/<name>.<ts>/ → /configs/plugins/<name>/
+//   4. MARKER    — touch /configs/plugins/<name>/.complete
+//
+// On any post-snapshot failure we attempt a best-effort rollback by mv-ing
+// the previous snapshot back into place. The .complete marker is the
+// canonical "this install is fully landed" signal — workspace-side plugin
+// loaders should refuse to load a plugin dir without it.
+//
+// Scope: docker path only (workspace running as a local container). The
+// SaaS path (deliverViaEIC, SSH-into-EC2) is unchanged in this PR; tracked
+// as a follow-up. The same stage-then-swap shape applies but the exec
+// primitives differ (ssh vs docker exec), and shipping both paths in one
+// PR doubles the test surface.
+
+import (
+	"bytes"
+	"context"
+	"fmt"
+	"path"
+	"strings"
+	"time"
+
+	"github.com/docker/docker/api/types/container"
+)
+
+const (
+	pluginsRoot       = "/configs/plugins"
+	pluginsStagingDir = "/configs/plugins/.staging"
+	pluginsPrevDir    = "/configs/plugins/.previous"
+	completeMarker    = ".complete"
+)
+
+// installVersion identifies one install attempt — the plugin name plus a
+// monotonic-ish UTC timestamp suffix. Used to namespace the staging dir
+// and any snapshot of the previous version, so a reinstall mid-flight
+// can't collide with a concurrent reinstall.
+type installVersion struct {
+	plugin string
+	stamp  string // e.g. 20260508T141530Z
+}
+
+func newInstallVersion(plugin string) installVersion {
+	return installVersion{
+		plugin: plugin,
+		stamp:  time.Now().UTC().Format("20060102T150405Z"),
+	}
+}
+
+// stagedPath is the container path where the new content lands during fetch.
+// e.g. /configs/plugins/.staging/molecule-skill-foo.20260508T141530Z
+func (v installVersion) stagedPath() string {
+	return path.Join(pluginsStagingDir, v.plugin+"."+v.stamp)
+}
+
+// previousPath is where the prior live version is moved before swap.
+// e.g. /configs/plugins/.previous/molecule-skill-foo.20260508T141530Z
+func (v installVersion) previousPath() string {
+	return path.Join(pluginsPrevDir, v.plugin+"."+v.stamp)
+}
+
+// livePath is the destination after swap.
+// e.g. /configs/plugins/molecule-skill-foo
+func (v installVersion) livePath() string {
+	return path.Join(pluginsRoot, v.plugin)
+}
+
+// markerPath is the .complete file inside the live dir written last.
+func (v installVersion) markerPath() string {
+	return path.Join(v.livePath(), completeMarker)
+}
+
+// atomicCopyToContainer does a stage→snapshot→swap→marker install of a
+// host-side staged plugin tree into a running container's
+// /configs/plugins/<name>/. Returns nil on success.
+//
+// On post-snapshot failure (swap or marker write), best-effort rollback
+// restores the previous snapshot to the live path. Returns the original
+// error wrapped — the caller should surface it; rollback success is
+// logged separately.
+func (h *PluginsHandler) atomicCopyToContainer(
+	ctx context.Context, containerName, hostDir, pluginName string,
+) error {
+	v := newInstallVersion(pluginName)
+
+	// Step 0a: ensure staging + previous root dirs exist (idempotent).
+	if _, err := h.execAsRoot(ctx, containerName, []string{
+		"mkdir", "-p", pluginsStagingDir, pluginsPrevDir,
+	}); err != nil {
+		return fmt.Errorf("atomic install: mkdir staging/previous: %w", err)
+	}
+
+	// Step 0b: tar the host content with a path prefix that lands it in the
+	// staging dir — NOT directly into the live name. The prefix has no
+	// leading "/" because docker.CopyToContainer extracts paths relative
+	// to the dstPath argument we pass below.
+	stagedRel := strings.TrimPrefix(v.stagedPath(), "/")
+	tarBuf, err := tarHostDirWithPrefix(hostDir, stagedRel)
+	if err != nil {
+		return fmt.Errorf("atomic install: tar host dir: %w", err)
+	}
+
+	// Step 1: STAGE — extract tar into /configs/plugins/.staging/<name>.<ts>/
+	if err := h.docker.CopyToContainer(ctx, containerName, "/", &tarBuf,
+		container.CopyToContainerOptions{}); err != nil {
+		// Best-effort: clean up any partial staging extract before returning.
+		_, _ = h.execAsRoot(ctx, containerName, []string{
+			"rm", "-rf", v.stagedPath(),
+		})
+		return fmt.Errorf("atomic install: copy to container: %w", err)
+	}
+
+	// Step 2: SNAPSHOT — if a live version exists, move it aside.
+	// `test -d` exits 0 if the dir exists, non-zero otherwise; the helper
+	// returns a non-nil error in the non-zero case which we treat as
+	// "no previous version" rather than a real failure.
+	snapshotted := false
+	if _, err := h.execAsRoot(ctx, containerName, []string{
+		"test", "-d", v.livePath(),
+	}); err == nil {
+		if _, err := h.execAsRoot(ctx, containerName, []string{
+			"mv", v.livePath(), v.previousPath(),
+		}); err != nil {
+			// Snapshot failure: roll back the staged extract before failing.
+			_, _ = h.execAsRoot(ctx, containerName, []string{
+				"rm", "-rf", v.stagedPath(),
+			})
+			return fmt.Errorf("atomic install: snapshot previous version: %w", err)
+		}
+		snapshotted = true
+	}
+
+	// Step 3: SWAP — atomic rename of the staged dir into the live name.
+	// `mv` on the same filesystem is a single rename(2), atomic at the FS level.
+	if _, err := h.execAsRoot(ctx, containerName, []string{
+		"mv", v.stagedPath(), v.livePath(),
+	}); err != nil {
+		// Swap failure: roll back if we had a snapshot.
+		if snapshotted {
+			if _, rbErr := h.execAsRoot(ctx, containerName, []string{
+				"mv", v.previousPath(), v.livePath(),
+			}); rbErr != nil {
+				return fmt.Errorf("atomic install: swap failed AND rollback failed: swap=%w, rollback=%v", err, rbErr)
+			}
+		}
+		// Best-effort cleanup of the still-staged dir.
+		_, _ = h.execAsRoot(ctx, containerName, []string{
+			"rm", "-rf", v.stagedPath(),
+		})
+		return fmt.Errorf("atomic install: swap to live path: %w", err)
+	}
+
+	// Step 4: MARKER — touch .complete inside the live dir as the last write.
+	// Workspace-side plugin loaders treat a plugin dir without this marker
+	// as half-installed and skip it (or surface a clear error to the
+	// operator instead of loading a possibly-partial tree).
+	if _, err := h.execAsRoot(ctx, containerName, []string{
+		"touch", v.markerPath(),
+	}); err != nil {
+		// Marker write failure with the new content already in place is a
+		// weird state — content is fine on disk, but the plugin loader
+		// will refuse to use it. Log loudly; do NOT roll back, since the
+		// content is the latest, just unmarked. Operator can manually
+		// `touch <plugin>/.complete` to recover.
+		return fmt.Errorf("atomic install: write .complete marker (content landed but unmarked, manual recovery: touch %s): %w", v.markerPath(), err)
+	}
+
+	// Step 5: GC — best-effort delete the previous snapshot. Failures here
+	// just leave a directory; not load-bearing for correctness, the next
+	// install or a separate sweeper will reclaim the space.
+	if snapshotted {
+		_, _ = h.execAsRoot(ctx, containerName, []string{
+			"rm", "-rf", v.previousPath(),
+		})
+	}
+
+	return nil
+}
+
+// tarHostDirWithPrefix walks hostDir and writes a tar to a buffer with
+// every entry's name prefixed by `prefix`. Mirrors the prior streaming
+// shape used in copyPluginToContainer but with a configurable prefix
+// (the prior version hardcoded "plugins/<name>/"; we use a full
+// staging path so the extracted layout is the staging dir directly).
+//
+// Symlinks are skipped — same posture as streamDirAsTar elsewhere in
+// this file. Skipping prevents a hostile plugin from injecting a
+// symlink that, post-extract, points outside the plugin's own dir.
+func tarHostDirWithPrefix(hostDir, prefix string) (bytes.Buffer, error) {
+	var buf bytes.Buffer
+	tw := newTarWriter(&buf)
+	defer tw.Close()
+	if err := tarWalk(hostDir, prefix, tw); err != nil {
+		return bytes.Buffer{}, err
+	}
+	return buf, nil
+}

workspace-server/internal/handlers/plugins_atomic_tar.go

@@ -0,0 +1,70 @@
+package handlers
+
+// plugins_atomic_tar.go — tar-walk helpers split out so the main atomic
+// install flow stays readable. The prefix argument lets the caller
+// arrange where the tar's contents land at extract time.
+
+import (
+	"archive/tar"
+	"io"
+	"os"
+	"path/filepath"
+)
+
+// newTarWriter is a thin wrapper so atomic_test.go can swap the writer
+// destination if it needs to.
+func newTarWriter(w io.Writer) *tar.Writer {
+	return tar.NewWriter(w)
+}
+
+// tarWalk walks hostDir and writes every regular file + dir to the tar
+// writer with paths of the form `<prefix>/<relative>`. Symlinks are
+// skipped — same posture as streamDirAsTar in plugins_install_pipeline.go.
+//
+// The trailing-slash on prefix is normalized away: prefix "foo" and
+// prefix "foo/" produce identical archives.
+func tarWalk(hostDir, prefix string, tw *tar.Writer) error {
+	prefix = filepath.Clean(prefix)
+	return filepath.Walk(hostDir, func(p string, info os.FileInfo, err error) error {
+		if err != nil {
+			return err
+		}
+		if info.Mode()&os.ModeSymlink != 0 {
+			return nil // skip symlinks; see doc above
+		}
+		rel, err := filepath.Rel(hostDir, p)
+		if err != nil {
+			return err
+		}
+		if rel == "." {
+			// Emit the prefix dir itself once, with the source dir's mode.
+			hdr, err := tar.FileInfoHeader(info, "")
+			if err != nil {
+				return err
+			}
+			hdr.Name = prefix + "/"
+			return tw.WriteHeader(hdr)
+		}
+		hdr, err := tar.FileInfoHeader(info, "")
+		if err != nil {
+			return err
+		}
+		hdr.Name = filepath.Join(prefix, rel)
+		if info.IsDir() {
+			hdr.Name += "/"
+		}
+		if err := tw.WriteHeader(hdr); err != nil {
+			return err
+		}
+		if !info.Mode().IsRegular() {
+			return nil
+		}
+		f, err := os.Open(p)
+		if err != nil {
+			return err
+		}
+		defer f.Close()
+		_, err = io.Copy(tw, f)
+		return err
+	})
+}

workspace-server/internal/handlers/plugins_atomic_test.go

@@ -0,0 +1,193 @@
+package handlers
+
+import (
+	"archive/tar"
+	"bytes"
+	"io"
+	"os"
+	"path/filepath"
+	"sort"
+	"strings"
+	"testing"
+	"time"
+)
+
+// TestInstallVersion_Paths: the path helpers must produce a stable shape
+// the in-container exec calls depend on. Pinning the layout here
+// catches a future refactor that accidentally changes where staging /
+// previous / live dirs live, which would break the swap atomicity.
+func TestInstallVersion_Paths(t *testing.T) {
+	v := installVersion{plugin: "molecule-skill-foo", stamp: "20260508T141530Z"}
+
+	if got, want := v.stagedPath(), "/configs/plugins/.staging/molecule-skill-foo.20260508T141530Z"; got != want {
+		t.Errorf("stagedPath = %q; want %q", got, want)
+	}
+	if got, want := v.previousPath(), "/configs/plugins/.previous/molecule-skill-foo.20260508T141530Z"; got != want {
+		t.Errorf("previousPath = %q; want %q", got, want)
+	}
+	if got, want := v.livePath(), "/configs/plugins/molecule-skill-foo"; got != want {
+		t.Errorf("livePath = %q; want %q", got, want)
+	}
+	if got, want := v.markerPath(), "/configs/plugins/molecule-skill-foo/.complete"; got != want {
+		t.Errorf("markerPath = %q; want %q", got, want)
+	}
+}
+
+// TestInstallVersion_StampUniqueness: two newInstallVersion calls within
+// the same second produce the same stamp (we use second precision); the
+// caller relies on the mv-rename being atomic, so collision-free
+// stamping is NOT a correctness requirement — but a regression that
+// changes stamp shape (e.g. RFC3339 with colons) would break the path
+// helpers since path.Join treats a colon as a regular char but ssh +
+// docker exec generally don't. Pin the no-colon shape.
+func TestInstallVersion_StampShape(t *testing.T) {
+	v := newInstallVersion("anything")
+	if strings.Contains(v.stamp, ":") {
+		t.Errorf("stamp must not contain colons (breaks shell-quoting in exec): %q", v.stamp)
+	}
+	if strings.Contains(v.stamp, " ") {
+		t.Errorf("stamp must not contain spaces: %q", v.stamp)
+	}
+	// Sanity: stamp parses as the documented format.
+	if _, err := time.Parse("20060102T150405Z", v.stamp); err != nil {
+		t.Errorf("stamp %q does not parse as 20060102T150405Z: %v", v.stamp, err)
+	}
+}
+
+// TestTarHostDirWithPrefix_HappyPath: walks a host dir, builds a tar with
+// the configured prefix, verifies every entry's name is rooted under
+// the prefix, and the file contents survive round-trip.
+func TestTarHostDirWithPrefix_HappyPath(t *testing.T) {
+	hostDir := t.TempDir()
+
+	// Plant: <host>/plugin.yaml + <host>/skills/foo/SKILL.md + <host>/.complete
+	files := map[string]string{
+		"plugin.yaml":             "name: foo\nversion: 1.0.0\n",
+		"skills/foo/SKILL.md":     "# Foo skill\n",
+		".complete":                "", // upstream may already have a marker
+	}
+	for rel, body := range files {
+		full := filepath.Join(hostDir, rel)
+		if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil {
+			t.Fatal(err)
+		}
+		if err := os.WriteFile(full, []byte(body), 0o644); err != nil {
+			t.Fatal(err)
+		}
+	}
+
+	prefix := "configs/plugins/.staging/foo.20260508T141530Z"
+	buf, err := tarHostDirWithPrefix(hostDir, prefix)
+	if err != nil {
+		t.Fatalf("tar: %v", err)
+	}
+
+	// Read back the tar; collect names + body for regular files.
+	got := map[string]string{}
+	tr := tar.NewReader(&buf)
+	for {
+		hdr, err := tr.Next()
+		if err == io.EOF {
+			break
+		}
+		if err != nil {
+			t.Fatalf("tar reader: %v", err)
+		}
+		// Every entry must start with the prefix
+		if !strings.HasPrefix(hdr.Name, prefix) {
+			t.Errorf("entry %q does not start with prefix %q", hdr.Name, prefix)
+		}
+		if hdr.Typeflag == tar.TypeReg {
+			body, err := io.ReadAll(tr)
+			if err != nil {
+				t.Fatal(err)
+			}
+			rel := strings.TrimPrefix(hdr.Name, prefix+"/")
+			got[rel] = string(body)
+		}
+	}
+
+	for rel, want := range files {
+		if got[rel] != want {
+			t.Errorf("body[%q] = %q; want %q", rel, got[rel], want)
+		}
+	}
+}
+
+// TestTarHostDirWithPrefix_SkipsSymlinks: a hostile plugin shouldn't be
+// able to ship a symlink that, post-extract, points outside its own
+// dir. The walker silently skips symlinks (same posture as
+// streamDirAsTar). Verify a planted symlink doesn't appear in the tar.
+func TestTarHostDirWithPrefix_SkipsSymlinks(t *testing.T) {
+	hostDir := t.TempDir()
+	// Plant a real file + a symlink pointing outside hostDir.
+	if err := os.WriteFile(filepath.Join(hostDir, "real.txt"), []byte("ok"), 0o644); err != nil {
+		t.Fatal(err)
+	}
+	target := filepath.Join(t.TempDir(), "outside")
+	if err := os.WriteFile(target, []byte("SHOULD NOT APPEAR"), 0o644); err != nil {
+		t.Fatal(err)
+	}
+	if err := os.Symlink(target, filepath.Join(hostDir, "evil")); err != nil {
+		t.Fatal(err)
+	}
+
+	buf, err := tarHostDirWithPrefix(hostDir, "p")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	names := []string{}
+	tr := tar.NewReader(&buf)
+	for {
+		hdr, err := tr.Next()
+		if err == io.EOF {
+			break
+		}
+		if err != nil {
+			t.Fatal(err)
+		}
+		names = append(names, hdr.Name)
+	}
+	sort.Strings(names)
+
+	for _, n := range names {
+		if strings.Contains(n, "evil") {
+			t.Errorf("symlink leaked into tar: %q", n)
+		}
+	}
+	// real.txt should be present
+	found := false
+	for _, n := range names {
+		if strings.HasSuffix(n, "real.txt") {
+			found = true
+			break
+		}
+	}
+	if !found {
+		t.Errorf("real.txt missing from tar; got names: %v", names)
+	}
+}
+
+// TestTarHostDirWithPrefix_PrefixNormalization: trailing slash on prefix
+// should not change the archive shape. Pinning this so a future caller
+// passing "foo/" instead of "foo" doesn't double-slash entry names.
+func TestTarHostDirWithPrefix_PrefixNormalization(t *testing.T) {
+	hostDir := t.TempDir()
+	if err := os.WriteFile(filepath.Join(hostDir, "x"), []byte("y"), 0o644); err != nil {
+		t.Fatal(err)
+	}
+
+	a, err := tarHostDirWithPrefix(hostDir, "foo")
+	if err != nil {
+		t.Fatal(err)
+	}
+	b, err := tarHostDirWithPrefix(hostDir, "foo/")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if !bytes.Equal(a.Bytes(), b.Bytes()) {
+		t.Errorf("trailing-slash on prefix changed archive shape; tarHostDirWithPrefix should be slash-insensitive")
+	}
+}

workspace-server/internal/handlers/plugins_install.go

@@ -100,6 +100,13 @@ func (h *PluginsHandler) Install(c *gin.Context) {
 }
 
 // Uninstall handles DELETE /workspaces/:id/plugins/:name — removes a plugin.
+//
+// Dispatch order mirrors Install's deliverToContainer:
+//
+//  1. Local Docker container up → exec rm -rf via existing helpers.
+//  2. SaaS workspace (instance_id set) → ssh sudo rm -rf via EIC.
+//  3. external runtime → 422 (caller manages its own plugin dir).
+//  4. Neither → 503.
 func (h *PluginsHandler) Uninstall(c *gin.Context) {
 	workspaceID := c.Param("id")
 	pluginName := c.Param("name")
@@ -120,12 +127,24 @@ func (h *PluginsHandler) Uninstall(c *gin.Context) {
 		return
 	}
 
-	containerName := h.findRunningContainer(ctx, workspaceID)
-	if containerName == "" {
-		c.JSON(http.StatusServiceUnavailable, gin.H{"error": "workspace container not running"})
+	if containerName := h.findRunningContainer(ctx, workspaceID); containerName != "" {
+		h.uninstallViaDocker(ctx, c, workspaceID, pluginName, containerName)
 		return
 	}
 
+	if instanceID, runtime := h.lookupSaaSDispatch(workspaceID); instanceID != "" {
+		h.uninstallViaEIC(ctx, c, workspaceID, pluginName, instanceID, runtime)
+		return
+	}
+
+	c.JSON(http.StatusServiceUnavailable, gin.H{"error": "workspace container not running"})
+}
+
+// uninstallViaDocker holds the historical Docker-exec uninstall flow.
+// Extracted out of Uninstall so the new SaaS dispatch reads cleanly and
+// the two backend bodies are visibly symmetric (same steps, different
+// transport).
+func (h *PluginsHandler) uninstallViaDocker(ctx context.Context, c *gin.Context, workspaceID, pluginName, containerName string) {
 	// Read the plugin's manifest BEFORE deletion to learn which skill dirs
 	// it owns, so we can clean them out of /configs/skills/ and avoid the
 	// auto-restart re-mounting them. Issue #106.
@@ -177,6 +196,61 @@ func (h *PluginsHandler) Uninstall(c *gin.Context) {
 	})
 }
 
+// uninstallViaEIC removes a plugin from a SaaS workspace EC2 over SSH.
+// Symmetric with uninstallViaDocker:
+//
+//   - Read manifest (best-effort, missing plugin.yaml = no skills to clean).
+//   - Skip CLAUDE.md awk-strip for now: that file lives at
+//     <runtime-config-prefix>/CLAUDE.md on the host and the same awk script
+//     would work over ssh, but the file is rewritten on workspace restart
+//     by the runtime adapter anyway, so the marker either stays harmless
+//     or gets dropped on the next install/restart cycle. Tracked as
+//     follow-up; not a regression vs the docker path's semantics here.
+//   - rm -rf the plugin dir.
+//   - Trigger restart.
+//
+// We intentionally don't try to remove /configs/skills/<skill> entries
+// over ssh because the same /configs is bind-mounted into the runtime
+// container; the agent's own start-up adapter rewrites that tree from
+// the live plugin set, so a stale skill dir for an uninstalled plugin
+// is cleaned up at restart. The docker path removes them eagerly only
+// because docker-exec is cheap. We can mirror that later if a real bug
+// surfaces, but adding two extra ssh round-trips per uninstall today
+// would be churn for no behavioural win.
+func (h *PluginsHandler) uninstallViaEIC(ctx context.Context, c *gin.Context, workspaceID, pluginName, instanceID, runtime string) {
+	// Read manifest first (best-effort) — we don't currently use the
+	// skills list on the SaaS path (see comment above), but reading it
+	// keeps the parsing path warm and lets log lines distinguish "we
+	// deleted a real plugin" from "user asked us to delete something
+	// that wasn't there." Errors here are swallowed: missing manifest
+	// must not block uninstall.
+	if data, err := readPluginManifestViaEIC(ctx, instanceID, runtime, pluginName); err == nil && len(data) > 0 {
+		info := parseManifestYAML(pluginName, data)
+		if len(info.Skills) > 0 {
+			log.Printf("Plugin uninstall: %s declared skills=%v (left to runtime restart to clean)", pluginName, info.Skills)
+		}
+	}
+
+	if err := uninstallPluginViaEIC(ctx, instanceID, runtime, pluginName); err != nil {
+		log.Printf("Plugin uninstall: EIC rm failed for %s on %s: %v", pluginName, workspaceID, err)
+		c.JSON(http.StatusBadGateway, gin.H{"error": "failed to remove plugin from workspace EC2"})
+		return
+	}
+
+	if h.restartFunc != nil {
+		go func() {
+			time.Sleep(2 * time.Second)
+			h.restartFunc(workspaceID)
+		}()
+	}
+
+	log.Printf("Plugin uninstall: %s from workspace %s (restarting via SaaS path)", pluginName, workspaceID)
+	c.JSON(http.StatusOK, gin.H{
+		"status": "uninstalled",
+		"plugin": pluginName,
+	})
+}
+
 // Download handles GET /workspaces/:id/plugins/:name/download?source=<scheme://spec>
 //
 // Phase 30.3 — stream the named plugin as a gzipped tarball so remote

workspace-server/internal/handlers/plugins_install_eic.go

@@ -0,0 +1,249 @@
+package handlers
+
+// plugins_install_eic.go — SaaS (EC2-per-workspace) plugin install + uninstall
+// over the EIC SSH primitive that template_files_eic.go already plumbs. Pairs
+// with the local-Docker path in plugins_install.go / plugins_install_pipeline.go,
+// closing the 🔴 docker-only row in docs/architecture/backends.md.
+//
+// Architecture note: every operation goes through `withEICTunnel` (ephemeral
+// keypair → AWS push → tunnel → ssh). This file owns the plugin-shaped
+// remote commands; the tunnel mechanics live in template_files_eic.go so a
+// fix to the dance lands in one place.
+//
+// Why direct host write (not docker cp via SSH): on the workspace EC2, the
+// runtime's managed-config dir (/configs for claude-code, /home/ubuntu/.hermes
+// for hermes — see workspaceFilePathPrefix) is bind-mounted into the
+// runtime's container by cloud-init. Writing into <prefix>/plugins/<name>/
+// on the host is exactly what the runtime sees on the next start. No
+// docker-cp needed, and we avoid coupling to any specific container layout
+// inside the workspace EC2.
+
+import (
+	"archive/tar"
+	"bytes"
+	"compress/gzip"
+	"context"
+	"fmt"
+	"log"
+	"os"
+	"os/exec"
+	"path/filepath"
+	"strings"
+	"time"
+)
+
+// eicPluginOpTimeout bounds the whole EIC-tunnel + ssh + tar-pipe dance
+// for a plugin install or uninstall. Larger than eicFileOpTimeout (30s)
+// because plugin trees can carry skill markdown, MCP server binaries,
+// and config files — easily a few MB through ssh + sudo on a fresh
+// tunnel. 2 min gives headroom on a cold tunnel; the install pipeline's
+// PLUGIN_INSTALL_FETCH_TIMEOUT (5 min default) still bounds the outer
+// request.
+const eicPluginOpTimeout = 2 * time.Minute
+
+// hostPluginPath returns the absolute directory on the workspace EC2
+// where /configs/plugins/<name>/ lives for a given runtime. Keeps the
+// per-runtime indirection in one place (mirrors resolveWorkspaceRootPath
+// in template_files_eic.go) so future runtimes only edit
+// workspaceFilePathPrefix.
+//
+// The plugin name is shellQuote-wrapped at the call site, not here,
+// because a couple of callers want the unquoted form for log lines.
+func hostPluginPath(runtime, pluginName string) string {
+	base := resolveWorkspaceRootPath(runtime, "/configs")
+	return filepath.Join(base, "plugins", pluginName)
+}
+
+// buildPluginInstallShell returns the remote command for receiving a tar.gz
+// stream on stdin and unpacking it into <hostPluginDir>/, owned by the agent
+// user (uid 1000 — matches the local-Docker path's chown 1000:1000).
+//
+// The script is a single `sudo sh -c '...'` so the tar-receive + chown run
+// under one privileged invocation; ssh-as-ubuntu has passwordless sudo on
+// the standard tenant AMI.
+//
+//   - rm -rf clears any prior install of the same plugin (idempotent
+//     reinstall — the user re-clicked Install or version-bumped the source).
+//   - mkdir -p makes the parent dir (host /configs is root-owned + always
+//     present; the per-plugin dir is what we're creating).
+//   - tar -xzf - reads stdin (the gzipped tar). --no-same-owner keeps the
+//     archive's tar-recorded uid/gid out of the picture; the chown -R
+//     after is the canonical owner.
+//   - chown -R 1000:1000 matches the local-Docker handler's exec at
+//     plugins_install_pipeline.go:273 — agent user inside the runtime
+//     container is uid 1000 on every workspace-template image we ship.
+//
+// shellQuote on the path is defence-in-depth: the path is composed from
+// a runtime allowlist (workspaceFilePathPrefix) + validated plugin name,
+// so traversal is already blocked.
+func buildPluginInstallShell(hostPluginDir string) string {
+	q := shellQuote(hostPluginDir)
+	return fmt.Sprintf(
+		"sudo -n sh -c 'rm -rf %s && mkdir -p %s && tar -xzf - --no-same-owner -C %s && chown -R 1000:1000 %s'",
+		q, q, q, q,
+	)
+}
+
+// buildPluginUninstallShell returns the remote command for `sudo -n rm -rf
+// <hostPluginDir>`. -rf (vs -f) is intentional here, unlike buildRmShell:
+// uninstall really does need to remove the plugin's whole subtree.
+func buildPluginUninstallShell(hostPluginDir string) string {
+	return fmt.Sprintf("sudo -n rm -rf %s", shellQuote(hostPluginDir))
+}
+
+// buildPluginManifestReadShell returns the remote command for reading the
+// plugin's manifest (plugin.yaml). Mirrors buildCatShell — swallows the
+// missing-file stderr so the missing-manifest case lands as empty stdout
+// + non-zero exit, which uninstall translates to "no skills to clean".
+func buildPluginManifestReadShell(hostPluginDir string) string {
+	return fmt.Sprintf("sudo -n cat %s/plugin.yaml 2>/dev/null", shellQuote(hostPluginDir))
+}
+
+// installPluginViaEIC pushes a staged plugin directory to a SaaS workspace
+// EC2 via the EIC SSH tunnel. On success the plugin lives at
+// <runtime-config-prefix>/plugins/<name>/ on the host, owned by 1000:1000,
+// ready for the next workspace restart to pick up.
+//
+// The caller (deliverToContainer SaaS branch) owns:
+//   - the staged dir (created + cleaned up by resolveAndStage)
+//   - the workspace restart trigger after install
+//
+// Errors here are wrapped with the instance + runtime so triage can tell
+// "tunnel failed" from "tar payload corrupt" without grep-ing the EC2's
+// auth.log.
+var installPluginViaEIC = realInstallPluginViaEIC
+
+func realInstallPluginViaEIC(ctx context.Context, instanceID, runtime, pluginName, stagedDir string) error {
+	if instanceID == "" {
+		return fmt.Errorf("installPluginViaEIC: empty instance_id")
+	}
+	if err := validatePluginName(pluginName); err != nil {
+		return fmt.Errorf("installPluginViaEIC: %w", err)
+	}
+
+	// Build the tar.gz payload up-front so a tar-walk failure is surfaced
+	// before we open the EIC tunnel — saves a 1-2s tunnel setup on every
+	// "broken plugin tree" case.
+	var payload bytes.Buffer
+	gz := gzip.NewWriter(&payload)
+	tw := tar.NewWriter(gz)
+	if err := streamDirAsTar(stagedDir, tw); err != nil {
+		return fmt.Errorf("installPluginViaEIC: tar pack: %w", err)
+	}
+	if err := tw.Close(); err != nil {
+		return fmt.Errorf("installPluginViaEIC: tar close: %w", err)
+	}
+	if err := gz.Close(); err != nil {
+		return fmt.Errorf("installPluginViaEIC: gzip close: %w", err)
+	}
+
+	hostDir := hostPluginPath(runtime, pluginName)
+	cmd := buildPluginInstallShell(hostDir)
+
+	ctx, cancel := context.WithTimeout(ctx, eicPluginOpTimeout)
+	defer cancel()
+
+	return withEICTunnel(ctx, instanceID, func(s eicSSHSession) error {
+		sshCmd := exec.CommandContext(ctx, "ssh", s.sshArgs(cmd)...)
+		sshCmd.Env = os.Environ()
+		sshCmd.Stdin = bytes.NewReader(payload.Bytes())
+		var stderr bytes.Buffer
+		sshCmd.Stderr = &stderr
+		if err := sshCmd.Run(); err != nil {
+			return fmt.Errorf(
+				"ssh install: %w (instance=%s runtime=%s plugin=%s payload=%dB stderr=%s)",
+				err, instanceID, runtime, pluginName, payload.Len(),
+				strings.TrimSpace(stderr.String()),
+			)
+		}
+		log.Printf(
+			"installPluginViaEIC: ws instance=%s runtime=%s plugin=%s payload=%dB → %s",
+			instanceID, runtime, pluginName, payload.Len(), hostDir,
+		)
+		return nil
+	})
+}
+
+// uninstallPluginViaEIC removes the plugin's directory from the workspace
+// EC2 via SSH. Symmetric with installPluginViaEIC but no payload — the
+// remote command is a single `rm -rf`.
+//
+// Best-effort by design: the local-Docker path also doesn't fail
+// uninstall on a missing directory (the pre-existing exec returns 0 when
+// the dir is absent), so we mirror that here. Real ssh-layer failures
+// (tunnel down, sudo denied) still propagate.
+var uninstallPluginViaEIC = realUninstallPluginViaEIC
+
+func realUninstallPluginViaEIC(ctx context.Context, instanceID, runtime, pluginName string) error {
+	if instanceID == "" {
+		return fmt.Errorf("uninstallPluginViaEIC: empty instance_id")
+	}
+	if err := validatePluginName(pluginName); err != nil {
+		return fmt.Errorf("uninstallPluginViaEIC: %w", err)
+	}
+
+	hostDir := hostPluginPath(runtime, pluginName)
+	cmd := buildPluginUninstallShell(hostDir)
+
+	ctx, cancel := context.WithTimeout(ctx, eicPluginOpTimeout)
+	defer cancel()
+
+	return withEICTunnel(ctx, instanceID, func(s eicSSHSession) error {
+		sshCmd := exec.CommandContext(ctx, "ssh", s.sshArgs(cmd)...)
+		sshCmd.Env = os.Environ()
+		var stderr bytes.Buffer
+		sshCmd.Stderr = &stderr
+		if err := sshCmd.Run(); err != nil {
+			return fmt.Errorf(
+				"ssh rm: %w (instance=%s runtime=%s plugin=%s stderr=%s)",
+				err, instanceID, runtime, pluginName,
+				strings.TrimSpace(stderr.String()),
+			)
+		}
+		log.Printf(
+			"uninstallPluginViaEIC: ws instance=%s runtime=%s plugin=%s → removed %s",
+			instanceID, runtime, pluginName, hostDir,
+		)
+		return nil
+	})
+}
+
+// readPluginManifestViaEIC reads the plugin's plugin.yaml from the
+// workspace EC2 so uninstall can learn the skills list to clean up.
+// Returns ("", nil) when the manifest doesn't exist (best-effort: the
+// local-Docker path treats a missing manifest as "no skills to remove",
+// not a failure).
+var readPluginManifestViaEIC = realReadPluginManifestViaEIC
+
+func realReadPluginManifestViaEIC(ctx context.Context, instanceID, runtime, pluginName string) ([]byte, error) {
+	if instanceID == "" {
+		return nil, fmt.Errorf("readPluginManifestViaEIC: empty instance_id")
+	}
+	if err := validatePluginName(pluginName); err != nil {
+		return nil, fmt.Errorf("readPluginManifestViaEIC: %w", err)
+	}
+
+	hostDir := hostPluginPath(runtime, pluginName)
+	cmd := buildPluginManifestReadShell(hostDir)
+
+	ctx, cancel := context.WithTimeout(ctx, eicPluginOpTimeout)
+	defer cancel()
+
+	var out []byte
+	runErr := withEICTunnel(ctx, instanceID, func(s eicSSHSession) error {
+		sshCmd := exec.CommandContext(ctx, "ssh", s.sshArgs(cmd)...)
+		sshCmd.Env = os.Environ()
+		var stdout, stderr bytes.Buffer
+		sshCmd.Stdout = &stdout
+		sshCmd.Stderr = &stderr
+		// Don't fail on non-zero exit: missing-manifest case returns 1
+		// from cat with empty stdout, which is the "no skills" signal.
+		_ = sshCmd.Run()
+		out = stdout.Bytes()
+		return nil
+	})
+	if runErr != nil {
+		return nil, runErr
+	}
+	return out, nil
+}

workspace-server/internal/handlers/plugins_install_eic_test.go

@@ -0,0 +1,505 @@
+package handlers
+
+import (
+	"archive/tar"
+	"bytes"
+	"compress/gzip"
+	"context"
+	"errors"
+	"io"
+	"net/http"
+	"net/http/httptest"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+
+	"github.com/DATA-DOG/go-sqlmock"
+	"github.com/gin-gonic/gin"
+)
+
+// expectAllowlistAllowAll programs the package-shared withMockDB sqlmock
+// so the org-allowlist gate (org_plugin_allowlist.go) returns "allow-all"
+// for the duration of one Install call. The gate fires three queries —
+// resolveOrgID, allowlist EXISTS, allowlist COUNT — and we satisfy each
+// with the empty/zero shape that means "no allowlist configured."
+//
+// Without this, tests that exercise the full Install flow panic on a
+// nil DB. The handlers package already ships withMockDB in
+// tokens_sqlmock_test.go; we just layer the allowlist-specific
+// expectations on top.
+func expectAllowlistAllowAll(mock sqlmock.Sqlmock) {
+	mock.MatchExpectationsInOrder(false)
+	mock.ExpectQuery(`SELECT parent_id FROM workspaces WHERE id`).
+		WillReturnRows(sqlmock.NewRows([]string{"parent_id"}).AddRow(nil))
+	mock.ExpectQuery(`SELECT EXISTS`).
+		WillReturnRows(sqlmock.NewRows([]string{"exists"}).AddRow(false))
+	mock.ExpectQuery(`SELECT COUNT\(\*\) FROM org_plugin_allowlist`).
+		WillReturnRows(sqlmock.NewRows([]string{"count"}).AddRow(0))
+}
+
+// stagePluginRegistry creates a single-plugin registry under dir so the
+// install handler's local resolver can find it. Returns the path to the
+// plugin dir for any caller that wants to assert tar contents.
+//
+// Centralised so a future tweak to the registry shape (e.g. plugin.yaml
+// schema bump) only updates one place. Tests use the source spec
+// `local://<name>` which the local resolver maps to <dir>/<name>/.
+func stagePluginRegistry(t *testing.T, dir, name string) string {
+	t.Helper()
+	pluginDir := filepath.Join(dir, name)
+	if err := os.Mkdir(pluginDir, 0755); err != nil {
+		t.Fatalf("mkdir plugin dir: %v", err)
+	}
+	manifest := "name: " + name + "\nversion: \"1.0.0\"\ndescription: SaaS dispatch test plugin\n"
+	if err := os.WriteFile(filepath.Join(pluginDir, "plugin.yaml"), []byte(manifest), 0644); err != nil {
+		t.Fatalf("write plugin.yaml: %v", err)
+	}
+	if err := os.WriteFile(filepath.Join(pluginDir, "rule.md"), []byte("# rule\n"), 0644); err != nil {
+		t.Fatalf("write rule.md: %v", err)
+	}
+	return pluginDir
+}
+
+// stubInstallPluginViaEIC swaps the package-level installPluginViaEIC for
+// the duration of the test; restored by t.Cleanup. Mirrors the existing
+// withEICTunnel stub pattern (template_files_eic_dispatch_test.go).
+func stubInstallPluginViaEIC(t *testing.T, fn func(ctx context.Context, instanceID, runtime, pluginName, stagedDir string) error) {
+	t.Helper()
+	prev := installPluginViaEIC
+	installPluginViaEIC = fn
+	t.Cleanup(func() { installPluginViaEIC = prev })
+}
+
+func stubUninstallPluginViaEIC(t *testing.T, fn func(ctx context.Context, instanceID, runtime, pluginName string) error) {
+	t.Helper()
+	prev := uninstallPluginViaEIC
+	uninstallPluginViaEIC = fn
+	t.Cleanup(func() { uninstallPluginViaEIC = prev })
+}
+
+func stubReadPluginManifestViaEIC(t *testing.T, fn func(ctx context.Context, instanceID, runtime, pluginName string) ([]byte, error)) {
+	t.Helper()
+	prev := readPluginManifestViaEIC
+	readPluginManifestViaEIC = fn
+	t.Cleanup(func() { readPluginManifestViaEIC = prev })
+}
+
+// ---------- pure-function shell shape ----------
+
+func TestBuildPluginInstallShell_QuotesPath(t *testing.T) {
+	got := buildPluginInstallShell("/configs/plugins/my-plugin")
+	want := "sudo -n sh -c 'rm -rf '/configs/plugins/my-plugin' && mkdir -p '/configs/plugins/my-plugin' && tar -xzf - --no-same-owner -C '/configs/plugins/my-plugin' && chown -R 1000:1000 '/configs/plugins/my-plugin''"
+	if got != want {
+		t.Errorf("buildPluginInstallShell mismatch:\n got %q\nwant %q", got, want)
+	}
+}
+
+func TestBuildPluginUninstallShell_QuotesPath(t *testing.T) {
+	got := buildPluginUninstallShell("/configs/plugins/my-plugin")
+	want := "sudo -n rm -rf '/configs/plugins/my-plugin'"
+	if got != want {
+		t.Errorf("buildPluginUninstallShell mismatch:\n got %q\nwant %q", got, want)
+	}
+}
+
+func TestBuildPluginManifestReadShell_QuotesPath(t *testing.T) {
+	got := buildPluginManifestReadShell("/configs/plugins/my-plugin")
+	want := "sudo -n cat '/configs/plugins/my-plugin'/plugin.yaml 2>/dev/null"
+	if got != want {
+		t.Errorf("buildPluginManifestReadShell mismatch:\n got %q\nwant %q", got, want)
+	}
+}
+
+func TestHostPluginPath_PerRuntime(t *testing.T) {
+	cases := []struct {
+		runtime string
+		plugin  string
+		want    string
+	}{
+		{"claude-code", "browser-automation", "/configs/plugins/browser-automation"},
+		{"hermes", "browser-automation", "/home/ubuntu/.hermes/plugins/browser-automation"},
+		{"langgraph", "browser-automation", "/opt/configs/plugins/browser-automation"},
+		// Unknown / empty runtime falls back to /configs (containerized
+		// user-data layout) so a future runtime added to workspaces table
+		// without a workspaceFilePathPrefix entry doesn't blow up the
+		// install path silently.
+		{"", "browser-automation", "/configs/plugins/browser-automation"},
+		{"some-future-runtime", "x", "/configs/plugins/x"},
+	}
+	for _, c := range cases {
+		t.Run(c.runtime+"/"+c.plugin, func(t *testing.T) {
+			got := hostPluginPath(c.runtime, c.plugin)
+			if got != c.want {
+				t.Errorf("hostPluginPath(%q, %q) = %q, want %q", c.runtime, c.plugin, got, c.want)
+			}
+		})
+	}
+}
+
+// ---------- dispatch: install ----------
+
+// TestPluginInstall_SaaS_DispatchesToEIC — the most-load-bearing test in
+// this file. With h.docker == nil and instanceIDLookup returning a real
+// instance_id, Install MUST push the staged plugin to the EC2 over EIC
+// (not 503). Asserts the EIC stub is called with the right (instance,
+// runtime, plugin) tuple AND that the staged dir has the manifest +
+// rule files we put there — proves the staging side wasn't bypassed.
+func TestPluginInstall_SaaS_DispatchesToEIC(t *testing.T) {
+	registry := t.TempDir()
+	stagePluginRegistry(t, registry, "browser-automation")
+
+	type capture struct {
+		called      bool
+		instanceID  string
+		runtime     string
+		pluginName  string
+		stagedFiles []string
+	}
+	var got capture
+
+	stubInstallPluginViaEIC(t, func(ctx context.Context, instanceID, runtime, pluginName, stagedDir string) error {
+		got.called = true
+		got.instanceID = instanceID
+		got.runtime = runtime
+		got.pluginName = pluginName
+		entries, err := os.ReadDir(stagedDir)
+		if err != nil {
+			t.Fatalf("read staged dir: %v", err)
+		}
+		for _, e := range entries {
+			got.stagedFiles = append(got.stagedFiles, e.Name())
+		}
+		return nil
+	})
+
+	mock, cleanup := withMockDB(t)
+	defer cleanup()
+	expectAllowlistAllowAll(mock)
+
+	h := NewPluginsHandler(registry, nil, nil).
+		WithRuntimeLookup(func(string) (string, error) { return "claude-code", nil }).
+		WithInstanceIDLookup(func(string) (string, error) { return "i-0e0951a3cfd9bbf75", nil })
+
+	w := httptest.NewRecorder()
+	c, _ := gin.CreateTestContext(w)
+	c.Params = gin.Params{{Key: "id", Value: "c7244ed9-f623-4cba-8873-020e5c9fe104"}}
+	c.Request = httptest.NewRequest(
+		"POST",
+		"/workspaces/c7244ed9-f623-4cba-8873-020e5c9fe104/plugins",
+		bytes.NewBufferString(`{"source":"local://browser-automation"}`),
+	)
+	c.Request.Header.Set("Content-Type", "application/json")
+
+	h.Install(c)
+
+	if w.Code != http.StatusOK {
+		t.Fatalf("expected 200, got %d: %s", w.Code, w.Body.String())
+	}
+	if !got.called {
+		t.Fatalf("installPluginViaEIC was not called")
+	}
+	if got.instanceID != "i-0e0951a3cfd9bbf75" {
+		t.Errorf("instanceID = %q, want i-0e0951a3cfd9bbf75", got.instanceID)
+	}
+	if got.runtime != "claude-code" {
+		t.Errorf("runtime = %q, want claude-code", got.runtime)
+	}
+	if got.pluginName != "browser-automation" {
+		t.Errorf("pluginName = %q, want browser-automation", got.pluginName)
+	}
+	// Staged dir must carry the resolver's actual fetch — manifest + rule.
+	// Anything missing here means the stage step was bypassed.
+	hasManifest, hasRule := false, false
+	for _, f := range got.stagedFiles {
+		if f == "plugin.yaml" {
+			hasManifest = true
+		}
+		if f == "rule.md" {
+			hasRule = true
+		}
+	}
+	if !hasManifest || !hasRule {
+		t.Errorf("staged dir missing files: %v (want plugin.yaml + rule.md)", got.stagedFiles)
+	}
+}
+
+// TestPluginInstall_SaaS_PropagatesEICError — when the EIC push fails
+// (tunnel down, sudo denied), Install MUST surface 502 rather than swallow
+// the error and report 200. 502 is the right status for "we tried, the
+// remote side wasn't there" — distinct from 503 ("nothing wired") and
+// 500 ("our bug"). The body deliberately doesn't echo the underlying
+// error string (would leak ssh stderr / instance metadata).
+func TestPluginInstall_SaaS_PropagatesEICError(t *testing.T) {
+	registry := t.TempDir()
+	stagePluginRegistry(t, registry, "browser-automation")
+
+	mock, cleanup := withMockDB(t)
+	defer cleanup()
+	expectAllowlistAllowAll(mock)
+
+	stubInstallPluginViaEIC(t, func(ctx context.Context, instanceID, runtime, pluginName, stagedDir string) error {
+		return errors.New("ssh: tunnel exited 255")
+	})
+
+	h := NewPluginsHandler(registry, nil, nil).
+		WithRuntimeLookup(func(string) (string, error) { return "claude-code", nil }).
+		WithInstanceIDLookup(func(string) (string, error) { return "i-aaaa", nil })
+
+	w := httptest.NewRecorder()
+	c, _ := gin.CreateTestContext(w)
+	c.Params = gin.Params{{Key: "id", Value: "ws-1"}}
+	c.Request = httptest.NewRequest(
+		"POST",
+		"/workspaces/ws-1/plugins",
+		bytes.NewBufferString(`{"source":"local://browser-automation"}`),
+	)
+	c.Request.Header.Set("Content-Type", "application/json")
+
+	h.Install(c)
+
+	if w.Code != http.StatusBadGateway {
+		t.Errorf("expected 502 for EIC failure, got %d: %s", w.Code, w.Body.String())
+	}
+	if strings.Contains(w.Body.String(), "tunnel exited") {
+		t.Errorf("response body must not echo raw EIC error: %s", w.Body.String())
+	}
+}
+
+// TestPluginInstall_NoBackends_Returns503 — lookup is wired but returns
+// empty instance_id (e.g. workspace pre-provision, or local-Docker
+// deploy without a running container). The handler MUST 503, not silently
+// dispatch to EIC with an empty instance_id.
+func TestPluginInstall_NoBackends_Returns503(t *testing.T) {
+	registry := t.TempDir()
+	stagePluginRegistry(t, registry, "browser-automation")
+
+	mock, cleanup := withMockDB(t)
+	defer cleanup()
+	expectAllowlistAllowAll(mock)
+
+	stubInstallPluginViaEIC(t, func(ctx context.Context, instanceID, runtime, pluginName, stagedDir string) error {
+		t.Errorf("EIC must not be called when instance_id is empty")
+		return nil
+	})
+
+	h := NewPluginsHandler(registry, nil, nil).
+		WithRuntimeLookup(func(string) (string, error) { return "claude-code", nil }).
+		WithInstanceIDLookup(func(string) (string, error) { return "", nil }) // empty
+
+	w := httptest.NewRecorder()
+	c, _ := gin.CreateTestContext(w)
+	c.Params = gin.Params{{Key: "id", Value: "ws-1"}}
+	c.Request = httptest.NewRequest(
+		"POST",
+		"/workspaces/ws-1/plugins",
+		bytes.NewBufferString(`{"source":"local://browser-automation"}`),
+	)
+	c.Request.Header.Set("Content-Type", "application/json")
+
+	h.Install(c)
+
+	if w.Code != http.StatusServiceUnavailable {
+		t.Errorf("expected 503, got %d: %s", w.Code, w.Body.String())
+	}
+}
+
+// TestPluginInstall_InstanceLookupError_Returns503 — a DB hiccup on the
+// instance_id lookup must NOT crash or 502; the handler logs and falls
+// through to 503. Same fail-open shape h.runtimeLookup uses (see
+// TestPluginInstall_NoRuntimeLookup_FailsOpen). Pinning this prevents a
+// future "tighten error handling" refactor from quietly converting a DB
+// blip into a five-minute outage on the install endpoint.
+func TestPluginInstall_InstanceLookupError_Returns503(t *testing.T) {
+	registry := t.TempDir()
+	stagePluginRegistry(t, registry, "browser-automation")
+
+	mock, cleanup := withMockDB(t)
+	defer cleanup()
+	expectAllowlistAllowAll(mock)
+
+	h := NewPluginsHandler(registry, nil, nil).
+		WithRuntimeLookup(func(string) (string, error) { return "claude-code", nil }).
+		WithInstanceIDLookup(func(string) (string, error) { return "", errors.New("db: connection refused") })
+
+	w := httptest.NewRecorder()
+	c, _ := gin.CreateTestContext(w)
+	c.Params = gin.Params{{Key: "id", Value: "ws-1"}}
+	c.Request = httptest.NewRequest(
+		"POST",
+		"/workspaces/ws-1/plugins",
+		bytes.NewBufferString(`{"source":"local://browser-automation"}`),
+	)
+	c.Request.Header.Set("Content-Type", "application/json")
+
+	h.Install(c)
+
+	if w.Code != http.StatusServiceUnavailable {
+		t.Errorf("expected 503 on instance-id lookup error, got %d: %s", w.Code, w.Body.String())
+	}
+}
+
+// ---------- dispatch: uninstall ----------
+
+func TestPluginUninstall_SaaS_DispatchesToEIC(t *testing.T) {
+	stubReadPluginManifestViaEIC(t, func(ctx context.Context, instanceID, runtime, pluginName string) ([]byte, error) {
+		return []byte("name: browser-automation\nskills:\n  - browse\n"), nil
+	})
+
+	type capture struct {
+		called     bool
+		instanceID string
+		runtime    string
+		pluginName string
+	}
+	var got capture
+	stubUninstallPluginViaEIC(t, func(ctx context.Context, instanceID, runtime, pluginName string) error {
+		got.called = true
+		got.instanceID = instanceID
+		got.runtime = runtime
+		got.pluginName = pluginName
+		return nil
+	})
+
+	h := NewPluginsHandler(t.TempDir(), nil, nil).
+		WithRuntimeLookup(func(string) (string, error) { return "claude-code", nil }).
+		WithInstanceIDLookup(func(string) (string, error) { return "i-bbbb", nil })
+
+	w := httptest.NewRecorder()
+	c, _ := gin.CreateTestContext(w)
+	c.Params = gin.Params{
+		{Key: "id", Value: "ws-1"},
+		{Key: "name", Value: "browser-automation"},
+	}
+	c.Request = httptest.NewRequest("DELETE", "/workspaces/ws-1/plugins/browser-automation", nil)
+
+	h.Uninstall(c)
+
+	if w.Code != http.StatusOK {
+		t.Fatalf("expected 200, got %d: %s", w.Code, w.Body.String())
+	}
+	if !got.called {
+		t.Fatalf("uninstallPluginViaEIC was not called")
+	}
+	if got.instanceID != "i-bbbb" || got.runtime != "claude-code" || got.pluginName != "browser-automation" {
+		t.Errorf("dispatch args wrong: %+v", got)
+	}
+}
+
+func TestPluginUninstall_SaaS_PropagatesEICError(t *testing.T) {
+	stubReadPluginManifestViaEIC(t, func(ctx context.Context, instanceID, runtime, pluginName string) ([]byte, error) {
+		return nil, nil
+	})
+	stubUninstallPluginViaEIC(t, func(ctx context.Context, instanceID, runtime, pluginName string) error {
+		return errors.New("ssh: connection refused")
+	})
+
+	h := NewPluginsHandler(t.TempDir(), nil, nil).
+		WithRuntimeLookup(func(string) (string, error) { return "claude-code", nil }).
+		WithInstanceIDLookup(func(string) (string, error) { return "i-cccc", nil })
+
+	w := httptest.NewRecorder()
+	c, _ := gin.CreateTestContext(w)
+	c.Params = gin.Params{
+		{Key: "id", Value: "ws-1"},
+		{Key: "name", Value: "browser-automation"},
+	}
+	c.Request = httptest.NewRequest("DELETE", "/workspaces/ws-1/plugins/browser-automation", nil)
+
+	h.Uninstall(c)
+
+	if w.Code != http.StatusBadGateway {
+		t.Errorf("expected 502, got %d: %s", w.Code, w.Body.String())
+	}
+}
+
+func TestPluginUninstall_NoBackends_Returns503(t *testing.T) {
+	stubUninstallPluginViaEIC(t, func(ctx context.Context, instanceID, runtime, pluginName string) error {
+		t.Errorf("EIC uninstall must not be called with empty instance_id")
+		return nil
+	})
+
+	h := NewPluginsHandler(t.TempDir(), nil, nil).
+		WithRuntimeLookup(func(string) (string, error) { return "claude-code", nil }).
+		WithInstanceIDLookup(func(string) (string, error) { return "", nil })
+
+	w := httptest.NewRecorder()
+	c, _ := gin.CreateTestContext(w)
+	c.Params = gin.Params{
+		{Key: "id", Value: "ws-1"},
+		{Key: "name", Value: "browser-automation"},
+	}
+	c.Request = httptest.NewRequest("DELETE", "/workspaces/ws-1/plugins/browser-automation", nil)
+
+	h.Uninstall(c)
+
+	if w.Code != http.StatusServiceUnavailable {
+		t.Errorf("expected 503, got %d: %s", w.Code, w.Body.String())
+	}
+}
+
+// ---------- tarball shape ----------
+
+// TestRealInstallPluginViaEIC_TarPayloadShape — the production
+// installPluginViaEIC packs the staged dir as gzipped tar. Stub
+// withEICTunnel + run the real installPluginViaEIC body, capturing the
+// ssh stdin via a fake exec.Command — except go's exec is hard to fake
+// without hijacking $PATH. Instead we exercise the tar packer directly:
+// streamDirAsTar's behaviour is what we actually depend on, and a
+// regression in either streamDirAsTar OR the gzip wrapping will be
+// visible here.
+func TestRealInstallPluginViaEIC_TarPayloadShape(t *testing.T) {
+	staged := t.TempDir()
+	if err := os.WriteFile(filepath.Join(staged, "plugin.yaml"), []byte("name: x\n"), 0644); err != nil {
+		t.Fatal(err)
+	}
+	if err := os.MkdirAll(filepath.Join(staged, "skills", "browse"), 0755); err != nil {
+		t.Fatal(err)
+	}
+	if err := os.WriteFile(filepath.Join(staged, "skills", "browse", "instructions.md"), []byte("step 1\n"), 0644); err != nil {
+		t.Fatal(err)
+	}
+
+	var buf bytes.Buffer
+	gz := gzip.NewWriter(&buf)
+	tw := tar.NewWriter(gz)
+	if err := streamDirAsTar(staged, tw); err != nil {
+		t.Fatalf("streamDirAsTar: %v", err)
+	}
+	if err := tw.Close(); err != nil {
+		t.Fatalf("tw close: %v", err)
+	}
+	if err := gz.Close(); err != nil {
+		t.Fatalf("gz close: %v", err)
+	}
+
+	// Round-trip: the same payload the production flow would pipe into
+	// `tar -xzf -` on the remote should unpack to plugin.yaml +
+	// skills/browse/instructions.md.
+	gr, err := gzip.NewReader(&buf)
+	if err != nil {
+		t.Fatalf("gzip reader: %v", err)
+	}
+	tr := tar.NewReader(gr)
+	seen := map[string]bool{}
+	for {
+		hdr, err := tr.Next()
+		if err == io.EOF {
+			break
+		}
+		if err != nil {
+			t.Fatalf("tar next: %v", err)
+		}
+		seen[hdr.Name] = true
+	}
+	for _, want := range []string{"plugin.yaml", "skills/browse/instructions.md"} {
+		// Tar entries on Linux normally use forward slashes regardless
+		// of host separator; double-check both forms so a Windows test
+		// runner doesn't go red on a path-sep difference. Production
+		// always runs on Linux (CI + tenant EC2).
+		alt := filepath.FromSlash(want)
+		if !seen[want] && !seen[alt] {
+			t.Errorf("tar payload missing %q (saw %v)", want, seen)
+		}
+	}
+}

workspace-server/internal/handlers/plugins_install_pipeline.go

@@ -261,22 +261,80 @@ func (h *PluginsHandler) resolveAndStage(ctx context.Context, req installRequest
 // deliverToContainer copies the staged plugin dir into the workspace
 // container, chowns it for the agent user, and triggers a restart.
 // Returns a typed *httpErr on failure; nil on success.
+//
+// Dispatch order:
+//
+//  1. Local Docker container is up → tar+CopyToContainer (historical path).
+//  2. SaaS workspace (instance_id set) → push via EIC SSH to the EC2's
+//     bind-mounted /configs/plugins/<name>/. Closes the 🔴 docker-only
+//     row in docs/architecture/backends.md by routing through the same
+//     primitive Files API uses (template_files_eic.go).
+//  3. Neither wired → 503. True "no backend" case (dev box without
+//     Docker AND without an instance_id row).
+//
+// The SaaS branch is gated on h.instanceIDLookup so unit tests can keep
+// using NewPluginsHandler without a DB; production wires it in router.go.
 func (h *PluginsHandler) deliverToContainer(ctx context.Context, workspaceID string, r *stageResult) error {
-	containerName := h.findRunningContainer(ctx, workspaceID)
-	if containerName == "" {
-		return newHTTPErr(http.StatusServiceUnavailable, gin.H{"error": "workspace container not running"})
+	if containerName := h.findRunningContainer(ctx, workspaceID); containerName != "" {
+		// Atomic stage→snapshot→swap→marker (molecule-core#114).
+		// Replaces the prior single docker.CopyToContainer write that
+		// left a partially-extracted tree on mid-install failure with
+		// no rollback path. atomicCopyToContainer writes a .complete
+		// marker as the last step; workspace-side plugin loaders should
+		// refuse to load a plugin dir without it.
+		if err := h.atomicCopyToContainer(ctx, containerName, r.StagedDir, r.PluginName); err != nil {
+			log.Printf("Plugin install: failed to copy %s to %s: %v", r.PluginName, workspaceID, err)
+			return newHTTPErr(http.StatusInternalServerError, gin.H{"error": "failed to copy plugin to container"})
+		}
+		h.execAsRoot(ctx, containerName, []string{
+			"chown", "-R", "1000:1000", "/configs/plugins/" + r.PluginName,
+		})
+		if h.restartFunc != nil {
+			go h.restartFunc(workspaceID)
+		}
+		return nil
 	}
-	if err := h.copyPluginToContainer(ctx, containerName, r.StagedDir, r.PluginName); err != nil {
-		log.Printf("Plugin install: failed to copy %s to %s: %v", r.PluginName, workspaceID, err)
-		return newHTTPErr(http.StatusInternalServerError, gin.H{"error": "failed to copy plugin to container"})
+
+	if instanceID, runtime := h.lookupSaaSDispatch(workspaceID); instanceID != "" {
+		if err := installPluginViaEIC(ctx, instanceID, runtime, r.PluginName, r.StagedDir); err != nil {
+			log.Printf("Plugin install: EIC push failed for %s → %s: %v", r.PluginName, workspaceID, err)
+			return newHTTPErr(http.StatusBadGateway, gin.H{
+				"error": "failed to deliver plugin to workspace EC2",
+			})
+		}
+		if h.restartFunc != nil {
+			go h.restartFunc(workspaceID)
+		}
+		return nil
 	}
-	h.execAsRoot(ctx, containerName, []string{
-		"chown", "-R", "1000:1000", "/configs/plugins/" + r.PluginName,
-	})
-	if h.restartFunc != nil {
-		go h.restartFunc(workspaceID)
+
+	return newHTTPErr(http.StatusServiceUnavailable, gin.H{"error": "workspace container not running"})
+}
+
+// lookupSaaSDispatch returns (instance_id, runtime) for SaaS dispatch, or
+// ("", "") when the lookups aren't wired or the workspace isn't on the
+// EC2 backend. Errors from the lookups are logged-and-swallowed: failing
+// open here just means the caller falls through to the 503 path it would
+// have returned without us, never to a wrong action against the wrong
+// instance.
+func (h *PluginsHandler) lookupSaaSDispatch(workspaceID string) (instanceID, runtime string) {
+	if h.instanceIDLookup == nil {
+		return "", ""
 	}
-	return nil
+	id, err := h.instanceIDLookup(workspaceID)
+	if err != nil {
+		log.Printf("Plugin install: instance_id lookup failed for %s: %v", workspaceID, err)
+		return "", ""
+	}
+	if id == "" {
+		return "", ""
+	}
+	if h.runtimeLookup != nil {
+		if rt, rterr := h.runtimeLookup(workspaceID); rterr == nil {
+			runtime = rt
+		}
+	}
+	return id, runtime
 }
 
 // readPluginSkillsFromContainer reads /configs/plugins/<name>/plugin.yaml

workspace-server/internal/provisioner/provisioner.go

@@ -765,6 +765,21 @@ func ApplyTierConfig(hostCfg *container.HostConfig, cfg WorkspaceConfig, configM
 
 // CopyTemplateToContainer copies files from a host directory into /configs in the container.
 func (p *Provisioner) CopyTemplateToContainer(ctx context.Context, containerID, templatePath string) error {
+	// Resolve symlinks at the root before walking. filepath.Walk does
+	// NOT follow a symlink that IS the root — it Lstats the path, sees
+	// a symlink (non-directory), and emits exactly one entry without
+	// descending. With cross-repo composition (parent template's
+	// dev-lead → ../sibling-repo/dev-lead/, see internal#77), the
+	// caller routinely passes a symlink as templatePath. Without this
+	// resolution the workspace's /configs/ mount lands empty.
+	//
+	// Security: templatePath has already passed resolveInsideRoot's
+	// path-string check at the call site — the trust boundary is the
+	// operator-side /org-templates/ filesystem layout, not this
+	// resolution step.
+	if resolved, err := filepath.EvalSymlinks(templatePath); err == nil {
+		templatePath = resolved
+	}
 	var buf bytes.Buffer
 	tw := tar.NewWriter(&buf)
 

workspace-server/internal/router/router.go

@@ -11,13 +11,13 @@ import (
 	"github.com/Molecule-AI/molecule-monorepo/platform/internal/buildinfo"
 	"github.com/Molecule-AI/molecule-monorepo/platform/internal/channels"
 	"github.com/Molecule-AI/molecule-monorepo/platform/internal/db"
-	"github.com/Molecule-AI/molecule-monorepo/platform/internal/messagestore"
 	"github.com/Molecule-AI/molecule-monorepo/platform/internal/events"
 	"github.com/Molecule-AI/molecule-monorepo/platform/internal/handlers"
-	"github.com/Molecule-AI/molecule-monorepo/platform/internal/pendinguploads"
 	memwiring "github.com/Molecule-AI/molecule-monorepo/platform/internal/memory/wiring"
+	"github.com/Molecule-AI/molecule-monorepo/platform/internal/messagestore"
 	"github.com/Molecule-AI/molecule-monorepo/platform/internal/metrics"
 	"github.com/Molecule-AI/molecule-monorepo/platform/internal/middleware"
+	"github.com/Molecule-AI/molecule-monorepo/platform/internal/pendinguploads"
 	"github.com/Molecule-AI/molecule-monorepo/platform/internal/provisioner"
 	"github.com/Molecule-AI/molecule-monorepo/platform/internal/supervised"
 	"github.com/Molecule-AI/molecule-monorepo/platform/internal/ws"
@@ -109,8 +109,8 @@ func Setup(hub *ws.Hub, broadcaster *events.Broadcaster, prov *provisioner.Provi
 		now := time.Now()
 		for name, last := range snap {
 			out[name] = gin.H{
-				"last_tick_at":    last,
-				"seconds_ago":     int(now.Sub(last).Seconds()),
+				"last_tick_at": last,
+				"seconds_ago":  int(now.Sub(last).Seconds()),
 			}
 		}
 		c.JSON(200, gin.H{"subsystems": out})
@@ -599,8 +599,25 @@ func Setup(hub *ws.Hub, broadcaster *events.Broadcaster, prov *provisioner.Provi
 		).Scan(&runtime)
 		return runtime, err
 	}
+	// Instance-id lookup powers the SaaS dispatch in install/uninstall:
+	// when a workspace is on the EC2-per-workspace backend (instance_id
+	// non-NULL) and there's no local Docker container to exec into, the
+	// pipeline pushes the staged plugin tarball to that EC2 over EIC SSH.
+	// Empty result means the workspace lives on the local-Docker backend
+	// (or hasn't been provisioned yet) and the handler falls back to its
+	// original Docker path. Same pattern templates.go and terminal.go use.
+	instanceIDLookup := func(workspaceID string) (string, error) {
+		var instanceID string
+		err := db.DB.QueryRowContext(
+			context.Background(),
+			`SELECT COALESCE(instance_id, '') FROM workspaces WHERE id = $1`,
+			workspaceID,
+		).Scan(&instanceID)
+		return instanceID, err
+	}
 	plgh := handlers.NewPluginsHandler(pluginsDir, dockerCli, wh.RestartByID).
-		WithRuntimeLookup(runtimeLookup)
+		WithRuntimeLookup(runtimeLookup).
+		WithInstanceIDLookup(instanceIDLookup)
 	r.GET("/plugins", plgh.ListRegistry)
 	r.GET("/plugins/sources", plgh.ListSources)
 	wsAuth.GET("/plugins", plgh.ListInstalled)