Merge pull request 'feat(security): RFC#523 3-layer forbidden-env guardrail for tenant workspaces (task #146)' (#1555) from feat/146-forbidden-env-guard into main

Three-layer cohesive fix for the 2026-05-19 ~00:05-00:09Z 4x reprov thrash
class observed on prod-Reviewer + prod-Researcher: a single secrets PUT
fanned out into 4x stop+provision cycles per workspace within 4 min,
each stopping the just-launched (still-pending) EC2 of the previous
cycle. Root-caused via Loki (provision.ec2_started / ec2_stopped pairs).

Empirical chain (all in workspace-server/internal/handlers/):
1. secrets.go SetSecret → go h.restartFunc → coalesceRestart cycle.
2. runRestartCycle sets url='' synchronously, then async provisions EC2.
3. During 20-30s pending window: url='' AND cpProv.IsRunning()==false
   — indistinguishable from a dead container.
4. Canvas /delegations poll OR the trailing restart-context probe fires
   ProxyA2A → maybeMarkContainerDead OR preflightContainerHealth →
   RestartByID → loop.
5. coalesceRestart's pending flag drains by running ANOTHER full cycle
   → ec2_stopped of the just-booted instance → re-provision.

Fix (single PR, three interdependent layers):

L1) Restart-aware health probes — workspace_restart.go exposes
    isRestarting(workspaceID) bool. Both maybeMarkContainerDead and
    preflightContainerHealth early-return false/nil while a restart
    cycle is in flight. Breaks the self-fire at the probe layer.

L2) Restart-context probe gate — sendRestartContext now requires
    url != '' AND last_heartbeat_at > restart_start_ts before firing
    the trailing ProxyA2A probe. Adds waitForFreshHeartbeat() next to
    waitForWorkspaceOnline. Belt-and-suspenders so the probe never
    tries until the new container is actually addressable.

L3) RestartByID debounce — silent-drop successive RestartByID calls
    within restartDebounceWindow=60s of restartStartedAt. Not coalesce
    (which would still drain to another full cycle). Drop is observable
    via restartByIDDropCounter (atomic.Uint64) + the dropped log line.
    Only programmatic path; HTTP Restart handler is unaffected.

Tests:
- TestIsRestarting_{FalseWhenNoStateEntry,TrueWhileCycleRunning}
- TestMaybeMarkContainerDead_SkippedWhileRestarting (L1)
- TestPreflightContainerHealth_SkippedWhileRestarting (L1)
- TestRestartByID_DebounceSilentDrop (L3, counter assertion)
- TestRestartByID_DebounceExpiresAfterWindow (L3, window release)
- TestRestartByID_SingleProvisionPerRestart (regression — asserts
  exactly 1 cycle per trigger, with 4 dropped self-fire probes)

Existing coalesce/restart/preflight/maybeMarkContainerDead tests
remain green. Full handlers suite: ok in 15.8s.

Closes internal#544.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

.gitea/scripts/sop-checklist.py

@@ -268,6 +268,7 @@ def compute_ack_state(
     items_by_slug: dict[str, dict[str, Any]],
     numeric_aliases: dict[int, str],
     team_membership_probe: "callable[[str, list[str]], list[str]]",
+    high_risk: bool = False,
 ) -> dict[str, dict[str, Any]]:
     """Compute per-item ack state.
 
@@ -330,11 +331,16 @@ def compute_ack_state(
     for slug, candidates in pending_team_check.items():
         if not candidates:
             continue
-        required = items_by_slug[slug]["required_teams"]
+        # Risk-class-aware required-teams resolution (RFC#450 Option C):
+        # high-risk PRs use `required_teams_high_risk` (when set on the
+        # item); default class uses `required_teams`. The probe closure
+        # is built with the same high_risk flag so the two reads are
+        # always consistent (both sites share `resolve_required_teams`).
+        required = resolve_required_teams(items_by_slug[slug], high_risk)
         approved = team_membership_probe(slug, candidates)  # returns subset
         rejected_not_in_team[slug] = [u for u in candidates if u not in approved]
         ackers_per_slug[slug] = approved
-        # Stash required teams for description rendering.
+        # Stash resolved teams for description rendering.
         items_by_slug[slug]["_required_resolved"] = required
 
     return {
@@ -765,6 +771,42 @@ def get_tier_mode(pr: dict[str, Any], cfg: dict[str, Any]) -> str:
     return default_mode
 
 
+def is_high_risk(pr: dict[str, Any], cfg: dict[str, Any]) -> bool:
+    """Return True when the PR is high-risk per RFC#450 Option C.
+
+    A PR is high-risk when ANY of:
+      - it carries the `tier:high` label (mechanically strictest tier), or
+      - it carries any label listed in cfg.high_risk_labels.
+
+    High-risk PRs use `required_teams_high_risk` (when set on an item)
+    instead of the default `required_teams`. Items without
+    `required_teams_high_risk` are unaffected (the default applies).
+
+    Governance fix for internal#442 — closes the inconsistency between
+    sop-tier-check (tier-aware) and sop-checklist (was tier-blind).
+    """
+    label_set = {(l.get("name") or "") for l in (pr.get("labels") or [])}
+    if "tier:high" in label_set:
+        return True
+    high_risk_labels = set(cfg.get("high_risk_labels") or [])
+    return bool(label_set & high_risk_labels)
+
+
+def resolve_required_teams(item: dict[str, Any], high_risk: bool) -> list[str]:
+    """Pick the active required_teams list for an item.
+
+    When high_risk is True AND the item declares a non-empty
+    `required_teams_high_risk`, return that. Else fall back to
+    `required_teams`. Keeping this in one helper means the gate's
+    decision shape stays single-sited even as items grow.
+    """
+    if high_risk:
+        elevated = item.get("required_teams_high_risk") or []
+        if elevated:
+            return list(elevated)
+    return list(item.get("required_teams") or [])
+
+
 def main(argv: list[str] | None = None) -> int:
     p = argparse.ArgumentParser()
     p.add_argument("--owner", required=True)
@@ -825,6 +867,12 @@ def main(argv: list[str] | None = None) -> int:
 
     comments = client.get_issue_comments(args.owner, args.repo, args.pr)
 
+    # High-risk classification (RFC#450 Option C, governance fix for
+    # internal#442). Computed ONCE per PR — used by both the probe
+    # closure and compute_ack_state so the elevation decision is
+    # single-sited.
+    high_risk = is_high_risk(pr, cfg)
+
     # Build team-membership probe closure that caches results per
     # (user, team-id) so a user acking multiple items only triggers
     # one membership lookup per team.
@@ -832,7 +880,7 @@ def main(argv: list[str] | None = None) -> int:
 
     def probe(slug: str, users: list[str]) -> list[str]:
         item = items_by_slug[slug]
-        team_names: list[str] = item["required_teams"]
+        team_names: list[str] = resolve_required_teams(item, high_risk)
         # Resolve names → ids. NOTE: orgs/{org}/teams/search may not be
         # available — fall back to the list endpoint.
         team_ids: list[int] = []
@@ -877,7 +925,9 @@ def main(argv: list[str] | None = None) -> int:
                     # may still find membership in another team.
         return approved
 
-    ack_state = compute_ack_state(comments, author, items_by_slug, numeric_aliases, probe)
+    ack_state = compute_ack_state(
+        comments, author, items_by_slug, numeric_aliases, probe, high_risk=high_risk
+    )
     body_state = {it["slug"]: section_marker_present(body, it["pr_section_marker"]) for it in items}
 
     state, description = render_status(items, ack_state, body_state)
@@ -890,7 +940,10 @@ def main(argv: list[str] | None = None) -> int:
         description = f"[info tier:low] {description}"
 
     # Diagnostics to job log.
-    print(f"::notice::PR #{args.pr} author={author} head={head_sha[:7]} mode={mode}")
+    print(
+        f"::notice::PR #{args.pr} author={author} head={head_sha[:7]} "
+        f"mode={mode} risk_class={'high' if high_risk else 'default'}"
+    )
     for it in items:
         slug = it["slug"]
         ackers = ack_state[slug]["ackers"]

.gitea/scripts/tests/test_sop_checklist.py

@@ -602,4 +602,216 @@ class TestComputeNaState(unittest.TestCase):
         self.assertEqual(len(na_directives), 1)
         self.assertEqual(na_directives[0][0], "sop-n/a")
         self.assertEqual(na_directives[0][1], "qa-review")
-        self.assertIn("no surface", na_directives[0][2])
+
+
+# ---------------------------------------------------------------------------
+# RFC#450 Option C — risk-classed two-eyes (governance fix for internal#442)
+# ---------------------------------------------------------------------------
+
+
+class TestIsHighRisk(unittest.TestCase):
+    """The high-risk predicate decides which required_teams list applies.
+
+    Predicate: tier:high label OR any label in cfg.high_risk_labels.
+    """
+
+    def setUp(self):
+        self.cfg = sop.load_config(CONFIG_PATH)
+
+    def test_no_labels_is_default_class(self):
+        pr = {"labels": []}
+        self.assertFalse(sop.is_high_risk(pr, self.cfg))
+
+    def test_tier_high_is_high_risk(self):
+        pr = {"labels": [{"name": "tier:high"}]}
+        self.assertTrue(sop.is_high_risk(pr, self.cfg))
+
+    def test_tier_low_is_default_class(self):
+        pr = {"labels": [{"name": "tier:low"}]}
+        self.assertFalse(sop.is_high_risk(pr, self.cfg))
+
+    def test_tier_medium_is_default_class(self):
+        # tier:medium alone is NOT high-risk (Option C — medium routes
+        # to the wider engineers OR-set).
+        pr = {"labels": [{"name": "tier:medium"}]}
+        self.assertFalse(sop.is_high_risk(pr, self.cfg))
+
+    def test_area_security_label_is_high_risk(self):
+        pr = {"labels": [{"name": "tier:medium"}, {"name": "area:security"}]}
+        self.assertTrue(sop.is_high_risk(pr, self.cfg))
+
+    def test_area_schema_label_is_high_risk(self):
+        pr = {"labels": [{"name": "area:schema"}]}
+        self.assertTrue(sop.is_high_risk(pr, self.cfg))
+
+    def test_area_identity_label_is_high_risk(self):
+        pr = {"labels": [{"name": "area:identity"}]}
+        self.assertTrue(sop.is_high_risk(pr, self.cfg))
+
+    def test_area_fleet_image_label_is_high_risk(self):
+        pr = {"labels": [{"name": "area:fleet-image"}]}
+        self.assertTrue(sop.is_high_risk(pr, self.cfg))
+
+    def test_area_gate_meta_label_is_high_risk(self):
+        # Gate-meta = changes to sop-checklist/sop-tier-check itself.
+        pr = {"labels": [{"name": "area:gate-meta"}]}
+        self.assertTrue(sop.is_high_risk(pr, self.cfg))
+
+    def test_unknown_area_label_is_default_class(self):
+        pr = {"labels": [{"name": "area:docs"}]}
+        self.assertFalse(sop.is_high_risk(pr, self.cfg))
+
+
+class TestResolveRequiredTeams(unittest.TestCase):
+    """The team resolver picks the elevated list only for high-risk PRs
+    AND only when the item declares one — items without an elevated
+    list always use the default required_teams."""
+
+    def test_default_class_uses_default_teams(self):
+        item = {"required_teams": ["engineers", "managers", "ceo"], "required_teams_high_risk": ["ceo"]}
+        self.assertEqual(
+            sop.resolve_required_teams(item, high_risk=False),
+            ["engineers", "managers", "ceo"],
+        )
+
+    def test_high_risk_uses_elevated_teams(self):
+        item = {"required_teams": ["engineers", "managers", "ceo"], "required_teams_high_risk": ["ceo"]}
+        self.assertEqual(
+            sop.resolve_required_teams(item, high_risk=True),
+            ["ceo"],
+        )
+
+    def test_high_risk_without_elevated_falls_back_to_default(self):
+        # Items that don't declare required_teams_high_risk (e.g.
+        # comprehensive-testing, staging-smoke) are unaffected by risk-class.
+        item = {"required_teams": ["engineers"]}
+        self.assertEqual(
+            sop.resolve_required_teams(item, high_risk=True),
+            ["engineers"],
+        )
+
+    def test_empty_elevated_list_falls_back_to_default(self):
+        # A defensive case: required_teams_high_risk: [] should not
+        # silently lock out all approvers — fall back to the default
+        # so the gate stays satisfiable. (Tightening should remove the
+        # key, not set it to empty.)
+        item = {"required_teams": ["engineers"], "required_teams_high_risk": []}
+        self.assertEqual(
+            sop.resolve_required_teams(item, high_risk=True),
+            ["engineers"],
+        )
+
+
+class TestRootCauseAckEligibilityWidened(unittest.TestCase):
+    """Closes internal#442: a non-author engineers-team ack now satisfies
+    root-cause / no-backwards-compat for the default class.
+
+    The dead-managers/ceo-persona-token gridlock is the symptom; the
+    root cause is that sop-checklist ignored tier-class. These tests
+    pin the new wider-default behavior so it can't regress silently.
+    """
+
+    def setUp(self):
+        self.items = _items_by_slug()
+        self.aliases = _numeric_aliases()
+
+    @staticmethod
+    def _approve_only(allowed):
+        return lambda slug, users: [u for u in users if u in allowed]
+
+    def test_engineers_ack_satisfies_root_cause_default_class(self):
+        # Bob is in engineers only (not managers, not ceo). Default class.
+        comments = [_comment("bob", "/sop-ack root-cause")]
+        # Probe: bob is approved because root-cause now lists engineers.
+        probe = self._approve_only({"bob"})
+        state = sop.compute_ack_state(
+            comments, "alice", self.items, self.aliases, probe, high_risk=False
+        )
+        self.assertEqual(state["root-cause"]["ackers"], ["bob"])
+
+    def test_engineers_ack_satisfies_no_backwards_compat_default_class(self):
+        comments = [_comment("bob", "/sop-ack no-backwards-compat")]
+        probe = self._approve_only({"bob"})
+        state = sop.compute_ack_state(
+            comments, "alice", self.items, self.aliases, probe, high_risk=False
+        )
+        self.assertEqual(state["no-backwards-compat"]["ackers"], ["bob"])
+
+    def test_engineers_ack_alone_fails_root_cause_when_high_risk(self):
+        # High-risk PR: only ceo can ack. Engineers-only ack must fail.
+        comments = [_comment("bob", "/sop-ack root-cause")]
+        # Probe: bob is in engineers, not ceo. Under high_risk,
+        # required_teams_high_risk=[ceo] → bob is NOT approved.
+        # Probe receives the items + flag indirectly via main(); for
+        # the unit-test path we inject a probe that rejects bob.
+        probe = self._approve_only(set())  # nobody is in ceo
+        state = sop.compute_ack_state(
+            comments, "alice", self.items, self.aliases, probe, high_risk=True
+        )
+        self.assertEqual(state["root-cause"]["ackers"], [])
+        self.assertIn("bob", state["root-cause"]["rejected"]["not_in_team"])
+
+    def test_ceo_ack_satisfies_root_cause_when_high_risk(self):
+        # High-risk PR + ceo-team approver → passes (the senior path).
+        comments = [_comment("hongming", "/sop-ack root-cause")]
+        probe = self._approve_only({"hongming"})
+        state = sop.compute_ack_state(
+            comments, "alice", self.items, self.aliases, probe, high_risk=True
+        )
+        self.assertEqual(state["root-cause"]["ackers"], ["hongming"])
+
+    def test_self_ack_still_forbidden_even_with_widened_eligibility(self):
+        # Author cannot self-ack — widening teams must NOT weaken
+        # the non-author rule.
+        comments = [_comment("alice", "/sop-ack root-cause")]
+        probe = self._approve_only({"alice"})
+        state = sop.compute_ack_state(
+            comments, "alice", self.items, self.aliases, probe, high_risk=False
+        )
+        self.assertEqual(state["root-cause"]["ackers"], [])
+        self.assertIn("alice", state["root-cause"]["rejected"]["self_ack"])
+
+
+class TestHighRiskClassUsesElevatedListInConfig(unittest.TestCase):
+    """End-to-end: the shipped config + RFC#450 predicate must keep
+    root-cause / no-backwards-compat gated on ceo for high-risk PRs."""
+
+    def test_root_cause_high_risk_elevated_to_ceo_only(self):
+        items = _items_by_slug()
+        # tier:high alone makes the PR high-risk → root-cause needs ceo.
+        self.assertEqual(
+            sop.resolve_required_teams(items["root-cause"], high_risk=True),
+            ["ceo"],
+        )
+        # Default class accepts engineers/managers/ceo.
+        self.assertEqual(
+            sorted(sop.resolve_required_teams(items["root-cause"], high_risk=False)),
+            sorted(["engineers", "managers", "ceo"]),
+        )
+
+    def test_no_backwards_compat_high_risk_elevated_to_ceo_only(self):
+        items = _items_by_slug()
+        self.assertEqual(
+            sop.resolve_required_teams(items["no-backwards-compat"], high_risk=True),
+            ["ceo"],
+        )
+        self.assertEqual(
+            sorted(sop.resolve_required_teams(items["no-backwards-compat"], high_risk=False)),
+            sorted(["engineers", "managers", "ceo"]),
+        )
+
+    def test_other_items_unchanged_by_risk_class(self):
+        # Items without required_teams_high_risk are unaffected.
+        items = _items_by_slug()
+        for slug in (
+            "comprehensive-testing",
+            "local-postgres-e2e",
+            "staging-smoke",
+            "five-axis-review",
+            "memory-consulted",
+        ):
+            self.assertEqual(
+                sop.resolve_required_teams(items[slug], high_risk=False),
+                sop.resolve_required_teams(items[slug], high_risk=True),
+                f"item {slug} should not be affected by risk-class",
+            )

.gitea/sop-checklist-config.yaml

@@ -50,6 +50,34 @@ tier_failure_mode:
   "tier:low": soft
 default_mode: hard  # used when no tier:* label is present
 
+# High-risk class (RFC#450 Option C, governance-fix for internal#442).
+#
+# A PR is "high-risk" when ANY of the listed labels are applied OR when
+# the PR has `tier:high` (mechanically the strictest existing tier).
+# High-risk items use `required_teams_high_risk` (when present on the
+# item); non-high-risk items use the default `required_teams`.
+#
+# This closes the inconsistency that the SOP charter already mandates
+# `tier:high → ceo only` for the sibling `sop-tier-check` gate; the
+# sop-checklist's `root-cause` and `no-backwards-compat` items now
+# follow the same risk-classed two-eyes shape:
+#   - Default class (tier:low/medium, not high-risk): a non-author
+#     engineers/managers/ceo ack satisfies the item — 25+ live
+#     identities, no dependency on a dead/inactive senior persona
+#     token.
+#   - High-risk class (tier:high OR any high_risk_label): still
+#     requires a non-author ceo ack (durable human team).
+#
+# Tightening: add labels to high_risk_labels.
+# Loosening: remove labels.
+high_risk_labels:
+  - "risk:high"
+  - "area:security"
+  - "area:schema"
+  - "area:fleet-image"
+  - "area:identity"
+  - "area:gate-meta"
+
 items:
   - slug: comprehensive-testing
     numeric_alias: 1
@@ -78,11 +106,15 @@ items:
   - slug: root-cause
     numeric_alias: 4
     pr_section_marker: "Root-cause not symptom"
-    required_teams: [managers, ceo]
+    required_teams: [engineers, managers, ceo]
+    required_teams_high_risk: [ceo]
     description: >-
-      One-sentence root-cause statement. Ack from managers tier
-      (team-leads) or ceo. Senior judgment required to attest
-      root-cause-versus-symptom.
+      One-sentence root-cause statement. Default class: non-author
+      engineers/managers/ceo ack suffices (engineers can attest
+      root-cause-vs-symptom for routine fixes). High-risk class
+      (see `high_risk_labels`): non-author ceo ack required —
+      senior judgment for irreversible/security/identity/gate
+      changes. Closes internal#442 + tracks RFC#450.
 
   - slug: five-axis-review
     numeric_alias: 5
@@ -95,10 +127,14 @@ items:
   - slug: no-backwards-compat
     numeric_alias: 6
     pr_section_marker: "No backwards-compat shim / dead code added"
-    required_teams: [managers, ceo]
+    required_teams: [engineers, managers, ceo]
+    required_teams_high_risk: [ceo]
     description: >-
-      Yes/no + justification if no. Senior ack required because
-      backward-compat shims are how dead-code accretes.
+      Yes/no + justification if no. Default class: non-author
+      engineers/managers/ceo ack suffices. High-risk class
+      (see `high_risk_labels`): non-author ceo ack required —
+      senior judgment for shim-versus-real-fix on irreversible
+      surfaces. Closes internal#442 + tracks RFC#450.
 
   - slug: memory-consulted
     numeric_alias: 7

.gitea/workflows/e2e-peer-visibility.yml

@@ -52,6 +52,30 @@ name: E2E Peer Visibility (literal MCP list_peers)
 #     flip-to-required-ready (mirrors e2e-staging-saas.yml's proven shape;
 #     real EC2-provisioning E2E is push/dispatch/cron only — it is 30+ min
 #     and cannot run per-PR-update).
+#
+# LOCAL BACKEND (added 2026-05-15 — feedback_local_must_mimic_production,
+# feedback_mandatory_local_e2e_before_ship, feedback_local_test_before_
+# staging_e2e)
+# --------------------------------------------------------------------
+# The standing rule is that the local prod-mimic stack runs a MANDATORY
+# local-Postgres E2E BEFORE staging E2E. A staging-only peer-visibility
+# gate caught regressions late + expensively (cold EC2). The
+# `peer-visibility-local` job below runs the SAME byte-identical
+# assertion (tests/e2e/lib/peer_visibility_assert.sh) against the local
+# docker-compose stack — built + booted exactly like e2e-api.yml's
+# proven E2E API Smoke Test job (ephemeral pg/redis ports, go build,
+# background platform-server). It runs on PR + push (local boot is
+# minutes, not the 30+ min cold-EC2 path), so peer-visibility is part of
+# the local gate that fires before the staging E2E.
+#
+# It is its OWN non-required status context `E2E Peer Visibility (local)`
+# — same non-required-by-design decision as the staging job (red until
+# Hermes-401 #162 / OpenClaw-never-online #165 land; flip-to-required
+# tracked at molecule-core#1296). It is an HONEST gate: NO
+# continue-on-error mask (feedback_fix_root_not_symptom). It is kept a
+# distinct context (not folded into e2e-api.yml's required `E2E API
+# Smoke Test`) precisely so a deliberately-RED-today gate cannot wedge
+# the required local-E2E job or any unrelated merge.
 
 on:
   push:
@@ -65,6 +89,8 @@ on:
       - 'workspace/a2a_mcp_server.py'
       - 'workspace/platform_tools/registry.py'
       - 'tests/e2e/test_peer_visibility_mcp_staging.sh'
+      - 'tests/e2e/test_peer_visibility_mcp_local.sh'
+      - 'tests/e2e/lib/peer_visibility_assert.sh'
       - '.gitea/workflows/e2e-peer-visibility.yml'
   pull_request:
     branches: [main]
@@ -77,6 +103,8 @@ on:
       - 'workspace/a2a_mcp_server.py'
       - 'workspace/platform_tools/registry.py'
       - 'tests/e2e/test_peer_visibility_mcp_staging.sh'
+      - 'tests/e2e/test_peer_visibility_mcp_local.sh'
+      - 'tests/e2e/lib/peer_visibility_assert.sh'
       - '.gitea/workflows/e2e-peer-visibility.yml'
   workflow_dispatch:
   schedule:
@@ -108,16 +136,160 @@ jobs:
     timeout-minutes: 5
     steps:
       - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
-      - name: Validate driving script
+      - name: Validate driving scripts + shared assertion lib
         run: |
+          bash -n tests/e2e/lib/peer_visibility_assert.sh
+          echo "lib/peer_visibility_assert.sh — bash syntax OK"
           bash -n tests/e2e/test_peer_visibility_mcp_staging.sh
           echo "test_peer_visibility_mcp_staging.sh — bash syntax OK"
-          echo "Real fresh-provision MCP list_peers E2E runs on push to"
+          bash -n tests/e2e/test_peer_visibility_mcp_local.sh
+          echo "test_peer_visibility_mcp_local.sh — bash syntax OK"
+          echo "Staging fresh-provision MCP list_peers E2E runs on push to"
           echo "main / workflow_dispatch / daily cron (30+ min EC2 boot)."
+          echo "The LOCAL backend runs in the peer-visibility-local job"
+          echo "below on this same PR (local docker-compose stack)."
 
-  # Real gate: provisions a throwaway org + sibling-per-runtime, drives
-  # the LITERAL list_peers MCP call per runtime, asserts 200 + expected
-  # peer set, then scoped teardown. push(main)/dispatch/cron only.
+  # LOCAL gate: same byte-identical assertion against the local prod-mimic
+  # docker-compose stack — the MANDATORY local-E2E that must run BEFORE
+  # the staging E2E (feedback_mandatory_local_e2e_before_ship,
+  # feedback_local_test_before_staging_e2e). Bootstrap mirrors
+  # e2e-api.yml's proven E2E API Smoke Test job (per-run container names +
+  # ephemeral host ports so concurrent host-network act_runner runs don't
+  # collide; go build; background platform-server). Its OWN non-required
+  # status context `E2E Peer Visibility (local)` — non-required-by-design
+  # exactly like the staging job (red until #162/#165 land;
+  # flip-to-required tracked at molecule-core#1296). HONEST gate, NO
+  # continue-on-error mask (feedback_fix_root_not_symptom). Runs on PR +
+  # push (local boot is minutes, not the 30+ min cold-EC2 path).
+  # bp-required: pending #1296
+  peer-visibility-local:
+    name: E2E Peer Visibility (local)
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    env:
+      # Per-run names + ephemeral ports — same collision-avoidance as
+      # e2e-api.yml (host-network act_runner; feedback_act_runner_*).
+      PG_CONTAINER: pg-e2e-pv-${{ github.run_id }}-${{ github.run_attempt }}
+      REDIS_CONTAINER: redis-e2e-pv-${{ github.run_id }}-${{ github.run_attempt }}
+      # LLM keys so hermes/openclaw can actually boot. The local script
+      # SKIPs (not fails) any runtime whose key is absent, so a partially
+      # keyed CI env still exercises whatever it can.
+      CLAUDE_CODE_OAUTH_TOKEN: ${{ secrets.E2E_CLAUDE_CODE_OAUTH_TOKEN }}
+      E2E_MINIMAX_API_KEY: ${{ secrets.MOLECULE_STAGING_MINIMAX_API_KEY }}
+      E2E_ANTHROPIC_API_KEY: ${{ secrets.MOLECULE_STAGING_ANTHROPIC_API_KEY }}
+      E2E_OPENAI_API_KEY: ${{ secrets.MOLECULE_STAGING_OPENAI_API_KEY }}
+      PV_RUNTIMES: "hermes openclaw claude-code"
+    steps:
+      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+      - uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff # v5
+        with:
+          go-version: 'stable'
+          cache: true
+          cache-dependency-path: workspace-server/go.sum
+      - name: Pre-pull alpine + ensure provisioner network
+        run: |
+          docker pull alpine:latest >/dev/null
+          docker network create molecule-core-net >/dev/null 2>&1 || true
+          echo "alpine:latest pre-pulled; molecule-core-net ensured."
+      - name: Start Postgres (docker, ephemeral port)
+        run: |
+          docker rm -f "$PG_CONTAINER" 2>/dev/null || true
+          docker run -d --name "$PG_CONTAINER" \
+            -e POSTGRES_USER=dev -e POSTGRES_PASSWORD=dev -e POSTGRES_DB=molecule \
+            -p 0:5432 postgres:16 >/dev/null
+          PG_PORT=$(docker port "$PG_CONTAINER" 5432/tcp | awk -F: '/^0\.0\.0\.0:/ {print $2; exit}')
+          [ -n "$PG_PORT" ] || PG_PORT=$(docker port "$PG_CONTAINER" 5432/tcp | head -1 | awk -F: '{print $NF}')
+          if [ -z "$PG_PORT" ]; then
+            echo "::error::Could not resolve host port for $PG_CONTAINER"
+            docker logs "$PG_CONTAINER" || true; exit 1
+          fi
+          echo "DATABASE_URL=postgres://dev:dev@127.0.0.1:${PG_PORT}/molecule?sslmode=disable" >> "$GITHUB_ENV"
+          for i in $(seq 1 30); do
+            docker exec "$PG_CONTAINER" pg_isready -U dev >/dev/null 2>&1 && { echo "Postgres ready after ${i}s"; exit 0; }
+            sleep 1
+          done
+          echo "::error::Postgres did not become ready in 30s"; docker logs "$PG_CONTAINER" || true; exit 1
+      - name: Start Redis (docker, ephemeral port)
+        run: |
+          docker rm -f "$REDIS_CONTAINER" 2>/dev/null || true
+          docker run -d --name "$REDIS_CONTAINER" -p 0:6379 redis:7 >/dev/null
+          REDIS_PORT=$(docker port "$REDIS_CONTAINER" 6379/tcp | awk -F: '/^0\.0\.0\.0:/ {print $2; exit}')
+          [ -n "$REDIS_PORT" ] || REDIS_PORT=$(docker port "$REDIS_CONTAINER" 6379/tcp | head -1 | awk -F: '{print $NF}')
+          if [ -z "$REDIS_PORT" ]; then
+            echo "::error::Could not resolve host port for $REDIS_CONTAINER"
+            docker logs "$REDIS_CONTAINER" || true; exit 1
+          fi
+          echo "REDIS_URL=redis://127.0.0.1:${REDIS_PORT}" >> "$GITHUB_ENV"
+          for i in $(seq 1 15); do
+            docker exec "$REDIS_CONTAINER" redis-cli ping 2>/dev/null | grep -q PONG && { echo "Redis ready after ${i}s"; exit 0; }
+            sleep 1
+          done
+          echo "::error::Redis did not become ready in 15s"; docker logs "$REDIS_CONTAINER" || true; exit 1
+      - name: Build platform
+        working-directory: workspace-server
+        run: go build -o platform-server ./cmd/server
+      - name: Pick platform port
+        run: |
+          PLATFORM_PORT=$(python3 - <<'PY'
+          import socket
+          with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
+              s.bind(("127.0.0.1", 0))
+              print(s.getsockname()[1])
+          PY
+          )
+          echo "PORT=${PLATFORM_PORT}" >> "$GITHUB_ENV"
+          echo "BASE=http://127.0.0.1:${PLATFORM_PORT}" >> "$GITHUB_ENV"
+          echo "Platform host port: ${PLATFORM_PORT}"
+      - name: Kill stale platform-server before start
+        run: |
+          killed=0
+          for pid in $(grep -l "platform-serve" /proc/[0-9]*/comm 2>/dev/null); do
+            kpid="${pid%/comm}"; kpid="${kpid##*/}"
+            cmdline=$(cat "/proc/${kpid}/cmdline" 2>/dev/null | tr '\0' ' ')
+            if echo "$cmdline" | grep -q "platform-server"; then
+              echo "Killing stale platform-server pid ${kpid}"
+              kill "$kpid" 2>/dev/null || true; killed=$((killed + 1))
+            fi
+          done
+          [ "$killed" -gt 0 ] && sleep 2 || true
+          echo "stale-kill done ($killed killed)"
+      - name: Start platform (background)
+        working-directory: workspace-server
+        run: |
+          ./platform-server > platform.log 2>&1 &
+          echo $! > platform.pid
+      - name: Wait for /health
+        run: |
+          for i in $(seq 1 30); do
+            curl -sf "$BASE/health" > /dev/null && { echo "Platform up after ${i}s"; exit 0; }
+            sleep 1
+          done
+          echo "::error::Platform did not become healthy in 30s"
+          cat workspace-server/platform.log || true; exit 1
+      - name: Run LOCAL fresh-provision peer-visibility E2E (literal MCP list_peers)
+        # HONEST gate — NO continue-on-error. Red today (Hermes-401 #162 /
+        # OpenClaw-never-online #165 not yet fixed); green when they land.
+        # Non-required-by-design via its distinct status context until the
+        # molecule-core#1296 flip-to-required.
+        run: bash tests/e2e/test_peer_visibility_mcp_local.sh
+      - name: Dump platform log on failure
+        if: failure()
+        run: cat workspace-server/platform.log || true
+      - name: Stop platform
+        if: always()
+        run: |
+          if [ -f workspace-server/platform.pid ]; then
+            kill "$(cat workspace-server/platform.pid)" 2>/dev/null || true
+          fi
+      - name: Stop service containers
+        if: always()
+        run: |
+          docker rm -f "$PG_CONTAINER" 2>/dev/null || true
+          docker rm -f "$REDIS_CONTAINER" 2>/dev/null || true
+
+  # Real STAGING gate: provisions a throwaway org + sibling-per-runtime,
+  # drives the LITERAL list_peers MCP call per runtime, asserts 200 +
+  # expected peer set, then scoped teardown. push(main)/dispatch/cron only.
   peer-visibility:
     name: E2E Peer Visibility
     runs-on: ubuntu-latest

.gitea/workflows/lint-forbidden-env-keys.yml

@@ -0,0 +1,168 @@
+name: Lint forbidden tenant-env keys
+
+# RFC#523 Layer 3 (task #146): scan workspace_secrets-writer Go code
+# under workspace-server/ for new code that hardcodes a forbidden
+# operator-scope env var NAME (GITEA_TOKEN, CP_ADMIN_API_TOKEN,
+# RAILWAY_TOKEN, INFISICAL_OPERATOR_TOKEN, MOLECULE_OPERATOR_*, …).
+#
+# Catches the class "a new writer accidentally widens the propagation
+# set" — e.g. a future env-mutator plugin that sets envVars["GITEA_TOKEN"]
+# directly. Today the L1 runtime guard would abort the provision, but
+# this lint surfaces the offending code at PR review time instead of
+# at first provision attempt.
+#
+# Companion layers:
+#   - L1: workspace-server/internal/handlers/workspace_provision_forbidden_env.go
+#         (fail-closed abort at provision time)
+#   - L2: workspace/entrypoint.sh top-of-file env-grep + exit 1
+#
+# Open-source-template-friendly: the deny pattern is generic. A fork
+# can copy this workflow and replace OPERATOR_KEY_PATTERN with its
+# own operator-scope key names.
+#
+# Path-filter discipline:
+#   This workflow runs on every PR (no paths: filter — see
+#   feedback_path_filtered_workflow_cant_be_required). The scan itself
+#   targets workspace_secrets-writer paths via grep -r; it's fast
+#   (sub-second) so unconditional run is fine.
+
+on:
+  pull_request:
+    types: [opened, synchronize, reopened]
+  push:
+    branches: [main, staging]
+
+env:
+  GITHUB_SERVER_URL: https://git.moleculesai.app
+
+jobs:
+  scan:
+    name: Scan workspace_secrets writers for forbidden env keys
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          fetch-depth: 1
+
+      - name: Scan for forbidden operator-scope env key NAMES in writer paths
+        run: |
+          set -euo pipefail
+
+          # Forbidden EXACT-MATCH env var names. Kept in lockstep with
+          # workspace-server/internal/handlers/workspace_provision_forbidden_env.go
+          # forbiddenTenantEnvKeys. The Go-side test
+          # TestIsForbiddenTenantEnvKey_ExactMatches is the source of
+          # truth — if Go-side adds a key, also add it here (and
+          # vice-versa). Drift between the two is the failure mode this
+          # entire 3-layer guardrail is designed to catch.
+          FORBIDDEN_KEYS=(
+            "GITEA_TOKEN" "GITEA_PAT"
+            "GITHUB_TOKEN" "GITHUB_PAT" "GH_TOKEN"
+            "GITLAB_TOKEN" "GL_TOKEN"
+            "BITBUCKET_TOKEN"
+            "CP_ADMIN_API_TOKEN" "CP_ADMIN_TOKEN"
+            "INFISICAL_OPERATOR_TOKEN" "INFISICAL_BOOTSTRAP_TOKEN"
+            "RAILWAY_TOKEN" "RAILWAY_PERSONAL_API_TOKEN"
+            "HETZNER_TOKEN" "HETZNER_API_TOKEN"
+          )
+
+          # Forbidden PREFIX patterns — operator-scope families.
+          FORBIDDEN_PREFIXES=(
+            "MOLECULE_OPERATOR_"
+          )
+
+          # Writer paths: Go source under workspace-server/ that
+          # writes to the env-vars map or to workspace_secrets DB rows.
+          # Tests, the forbidden-env source itself, and the silent-
+          # strip denylist are exempt (they LIST the keys by design).
+          SCAN_ROOT="workspace-server/internal"
+          # Exempt paths fall in two classes:
+          #   1. The deny-set definitions + the silent-strip denylist:
+          #      they LIST the forbidden names by design.
+          #   2. Pre-RFC#523 persona-merge / config-read paths that
+          #      already handle these names correctly (the silent-
+          #      strip downstream + the new L1 fail-closed cover the
+          #      runtime risk; these reads are unchanged).
+          # New code MUST NOT be added to this list without reviewer
+          # signoff and a one-line justification in this diff.
+          EXEMPT_PATHS=(
+            # Class 1 — deny-set definitions
+            "workspace-server/internal/handlers/workspace_provision_forbidden_env.go"
+            "workspace-server/internal/handlers/workspace_provision_forbidden_env_test.go"
+            "workspace-server/internal/provisioner/provisioner.go"
+            "workspace-server/internal/provisioner/provisioner_test.go"
+            # Class 2 — pre-existing persona-fallback / org-helper paths
+            # that set the GITEA_TOKEN fallback lane (stripped downstream
+            # by provisioner.buildContainerEnv per forensic #145). The
+            # new L1 fail-closed runs BEFORE these writers, so any
+            # operator-scope leak via global/workspace_secrets is
+            # already caught. See applyAgentGitHTTPCreds doc-comment.
+            "workspace-server/internal/handlers/agent_git_identity.go"
+            "workspace-server/internal/handlers/org_helpers.go"
+            "workspace-server/internal/handlers/org.go"
+            # Class 2 — CP→platform admin auth (NOT a tenant env write;
+            # this is the control-plane HTTP auth header source).
+            "workspace-server/internal/provisioner/cp_provisioner.go"
+          )
+
+          # Build a single grep -F pattern: every forbidden key wrapped
+          # in quotes (Go string-literal form, which is how env-map
+          # writes appear). e.g. envVars["GITEA_TOKEN"] = ... or
+          # `"GITEA_TOKEN":` in a literal-map declaration.
+          #
+          # We deliberately match the quoted form so a comment that
+          # happens to spell the name without quotes (e.g. "see
+          # GITEA_TOKEN below") doesn't trip the lint.
+          PATTERN=""
+          for k in "${FORBIDDEN_KEYS[@]}"; do
+            PATTERN="${PATTERN}\"${k}\"\n"
+          done
+          for p in "${FORBIDDEN_PREFIXES[@]}"; do
+            # Prefix match needs a regex; switch to grep -E below for
+            # this slice. Kept conceptually here so the deny set lives
+            # in one place; scan is run twice (literal + prefix).
+            true
+          done
+
+          # Build exempt-paths grep filter — `grep -v -f` style.
+          EXEMPT_FILTER=$(mktemp)
+          trap 'rm -f "$EXEMPT_FILTER"' EXIT
+          for p in "${EXEMPT_PATHS[@]}"; do
+            echo "$p" >> "$EXEMPT_FILTER"
+          done
+
+          # --- Exact-match scan ---
+          HITS=""
+          for k in "${FORBIDDEN_KEYS[@]}"; do
+            # Only .go files; skip _test.go for the writer-path scan
+            # since tests legitimately reference the names. The
+            # writer-path lint targets PRODUCTION code only.
+            found=$(grep -rn --include='*.go' --exclude='*_test.go' "\"${k}\"" "$SCAN_ROOT" 2>/dev/null \
+                    | grep -v -F -f "$EXEMPT_FILTER" || true)
+            if [ -n "$found" ]; then
+              HITS="${HITS}${found}\n"
+            fi
+          done
+
+          # --- Prefix scan ---
+          for prefix in "${FORBIDDEN_PREFIXES[@]}"; do
+            found=$(grep -rnE --include='*.go' --exclude='*_test.go' "\"${prefix}[A-Z0-9_]+\"" "$SCAN_ROOT" 2>/dev/null \
+                    | grep -v -F -f "$EXEMPT_FILTER" || true)
+            if [ -n "$found" ]; then
+              HITS="${HITS}${found}\n"
+            fi
+          done
+
+          if [ -n "$HITS" ]; then
+            echo "::error::RFC#523 Layer 3: forbidden operator-scope env var name(s) hardcoded in tenant-workspace writer paths:"
+            printf "$HITS"
+            echo ""
+            echo "These env-var NAMES are on the operator-scope deny list (see"
+            echo "workspace-server/internal/handlers/workspace_provision_forbidden_env.go)."
+            echo "If your code legitimately needs to inject one of these for a"
+            echo "non-tenant code path, add the file to EXEMPT_PATHS in this"
+            echo "workflow with a one-line justification — reviewer signoff required."
+            exit 1
+          fi
+
+          echo "OK No forbidden operator-scope env key names hardcoded in writer paths."

Makefile

@@ -4,10 +4,10 @@
 # use this Makefile; CI calls docker compose / go test directly so the
 # Makefile can evolve without breaking the build.
 
-.PHONY: help dev up down logs build test
+.PHONY: help dev up down logs build test e2e-peer-visibility
 
 help: ## Show this help.
-	@grep -E '^[a-zA-Z_-]+:.*?## ' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-12s\033[0m %s\n", $$1, $$2}'
+	@grep -E '^[a-zA-Z0-9_-]+:.*?## ' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-22s\033[0m %s\n", $$1, $$2}'
 
 dev: ## Start the full stack with air hot-reload for the platform service.
 	docker compose -f docker-compose.yml -f docker-compose.dev.yml up
@@ -26,3 +26,13 @@ build: ## Force a fresh build of the platform image (no cache).
 
 test: ## Run Go unit tests in workspace-server/.
 	cd workspace-server && go test -race ./...
+
+# ─── Local prod-mimic E2E gates ────────────────────────────────────────
+# Run the LITERAL peer-visibility MCP list_peers gate against the
+# already-running local stack (`make up` or `make dev`). Same byte-
+# identical assertion as the staging gate — only provisioning differs.
+# Skips any runtime whose provider key is absent (partially-keyed env
+# is fine). See tests/e2e/test_peer_visibility_mcp_local.sh for the
+# env contract (CLAUDE_CODE_OAUTH_TOKEN / E2E_MINIMAX_API_KEY / etc).
+e2e-peer-visibility: ## Run the LOCAL peer-visibility MCP gate vs the running stack (needs `make up` first).
+	bash tests/e2e/test_peer_visibility_mcp_local.sh

canvas/src/components/tabs/ChatTab.tsx

@@ -10,6 +10,7 @@ import { downloadChatFile, isPlatformAttachment } from "./chat/uploads";
 import { PendingAttachmentPill } from "./chat/AttachmentViews";
 import { AttachmentPreview } from "./chat/AttachmentPreview";
 import { AgentCommsPanel } from "./chat/AgentCommsPanel";
+import { ChatErrorBanner } from "./chat/ChatErrorBanner";
 import { appendActivityLine } from "./chat/activityLog";
 import { runtimeDisplayName } from "@/lib/runtime-names";
 import { ConfirmDialog } from "@/components/ConfirmDialog";
@@ -592,22 +593,19 @@ function MyChatPanel({ workspaceId, data }: Props) {
         <div ref={bottomRef} />
       </div>
 
-      {/* Error banner */}
-      {displayError && (
-        <div className="px-3 py-2 bg-red-900/20 border-t border-red-800/30">
-          <div className="flex items-center justify-between">
-            <span className="text-[10px] text-red-300">{displayError}</span>
-            {!isOnline && (
-              <button
-                onClick={() => setConfirmRestart(true)}
-                className="text-[11px] px-2 py-0.5 bg-red-800 text-red-200 rounded hover:bg-red-700"
-              >
-                Restart
-              </button>
-            )}
-          </div>
-        </div>
-      )}
+      {/* Error banner — internal#212: surfaces the secret-safe
+          actionable failure reason that ws-server places on
+          ACTIVITY_LOGGED.error_detail (propagated via
+          useChatSocket → onSendError → setError) and offers a
+          "View activity log" affordance that navigates the user to
+          the Activity tab where the full row lives. The previous
+          inline JSX hardcoded "see workspace logs for details" with
+          no link — there is no separate Logs tab. */}
+      <ChatErrorBanner
+        message={displayError}
+        isOnline={isOnline}
+        onRestart={() => setConfirmRestart(true)}
+      />
 
       {/* Input */}
       <div className="p-3 border-t border-line">

canvas/src/components/tabs/__tests__/ChatTab.errorBanner.test.tsx

@@ -0,0 +1,99 @@
+// @vitest-environment jsdom
+//
+// Pins internal#212 — the chat error banner must:
+//
+//   1. Render the secret-safe failure reason (e.g. the provider's own
+//      "403 oauth_org_not_allowed: ..." string), NOT the opaque
+//      hardcoded "Agent error (Exception) — see workspace logs for
+//      details." that points at a workspace-logs tab that doesn't
+//      exist.
+//
+//   2. Offer a working "View activity log" affordance that navigates
+//      the user to the Activity tab where the full row lives.
+//
+// Tested at the banner-component seam (ChatErrorBanner). The
+// hook-level path is pinned separately by
+// chat/hooks/__tests__/useChatSocket.test.tsx — together they cover
+// wire-payload → callback → render without each test needing to drive
+// the full ChatTab send-state machinery.
+
+import { describe, it, expect, vi, afterEach, beforeEach } from "vitest";
+import { render, screen, cleanup, fireEvent } from "@testing-library/react";
+
+afterEach(cleanup);
+
+const mocks = vi.hoisted(() => ({
+  setPanelTabMock: vi.fn(),
+}));
+
+vi.mock("@/store/canvas", () => {
+  const state = {
+    setPanelTab: mocks.setPanelTabMock,
+    panelTab: "chat",
+  };
+  const hook = (selector?: (s: typeof state) => unknown) =>
+    selector ? selector(state) : state;
+  hook.getState = () => state;
+  return { useCanvasStore: hook };
+});
+
+beforeEach(() => {
+  mocks.setPanelTabMock.mockClear();
+});
+
+import { ChatErrorBanner } from "../chat/ChatErrorBanner";
+
+describe("ChatErrorBanner — surfaces actionable reason (internal#212)", () => {
+  it("renders the secret-safe failure reason verbatim, not a hardcoded opaque message", () => {
+    const reason =
+      "Anthropic 403 oauth_org_not_allowed: Your organization has disabled Claude subscription access for Claude Code — use an Anthropic API key or ask your admin to enable access.";
+    render(<ChatErrorBanner message={reason} isOnline={true} onRestart={() => {}} />);
+    expect(screen.getByText(/oauth_org_not_allowed/i)).toBeDefined();
+    expect(screen.getByText(/disabled Claude subscription access/i)).toBeDefined();
+    // The legacy boilerplate must NOT leak through when a real reason
+    // is provided.
+    expect(screen.queryByText(/see workspace logs for details/i)).toBeNull();
+  });
+
+  it("falls back to the message when it IS the legacy boilerplate (older ws-server)", () => {
+    // Graceful degradation: an older ws-server passes through the
+    // hardcoded text; the banner still renders SOMETHING — never
+    // silently swallow.
+    render(
+      <ChatErrorBanner
+        message="Agent error (Exception) — see workspace logs for details."
+        isOnline={true}
+        onRestart={() => {}}
+      />,
+    );
+    expect(
+      screen.getByText(/Agent error \(Exception\) — see workspace logs for details\./),
+    ).toBeDefined();
+  });
+
+  it("offers a 'View activity log' button that calls setPanelTab('activity')", () => {
+    render(
+      <ChatErrorBanner message="kimi 401 invalid_api_key" isOnline={true} onRestart={() => {}} />,
+    );
+    const btn = screen.getByRole("button", { name: /view activity log/i });
+    fireEvent.click(btn);
+    expect(mocks.setPanelTabMock).toHaveBeenCalledWith("activity");
+  });
+
+  it("still shows the Restart button when offline (existing behavior preserved)", () => {
+    const onRestart = vi.fn();
+    render(
+      <ChatErrorBanner message="Agent is offline" isOnline={false} onRestart={onRestart} />,
+    );
+    const btn = screen.getByRole("button", { name: /^restart$/i });
+    fireEvent.click(btn);
+    expect(onRestart).toHaveBeenCalledTimes(1);
+  });
+
+  it("renders nothing when message is null", () => {
+    const { container } = render(
+      <ChatErrorBanner message={null} isOnline={true} onRestart={() => {}} />,
+    );
+    expect(container.textContent).toBe("");
+  });
+});

canvas/src/components/tabs/chat/ChatErrorBanner.tsx

@@ -0,0 +1,85 @@
+"use client";
+
+/**
+ * ChatErrorBanner — error-state banner rendered under the chat
+ * message list when an agent turn fails or the workspace is offline.
+ *
+ * internal#212 closes the "see workspace logs for details" pointer-to-
+ * nowhere defect:
+ *
+ *   - The banner now renders the actionable, secret-safe failure
+ *     reason that ws-server places on `ACTIVITY_LOGGED.error_detail`
+ *     (provider HTTP status + error code + provider's own human
+ *     message). The hook (`useChatSocket`) forwards this through
+ *     `onSendError`, which the ChatTab routes into this banner's
+ *     `message` prop. No hardcoded opaque text in this component.
+ *
+ *   - A "View activity log" button navigates the user to the Activity
+ *     tab where the full row (request body, response body, timing,
+ *     full error_detail) lives. Until internal#212, the banner
+ *     mentioned "workspace logs" with no link — there is no separate
+ *     Logs tab in the side panel; the Activity tab IS the workspace-
+ *     logs surface. Routing through the existing tab makes the
+ *     reference real instead of dangling.
+ *
+ *   - The existing Restart button (shown only when the workspace is
+ *     offline) is preserved unchanged so the recovery affordance the
+ *     old banner offered does not regress.
+ *
+ * Pure presentational — no socket subscription, no state machine. Easy
+ * to unit-test in isolation and easy to compose into the ChatTab.
+ */
+
+import { useCanvasStore } from "@/store/canvas";
+
+export interface ChatErrorBannerProps {
+  /** The user-visible reason. Pass `null` to render nothing. */
+  message: string | null;
+  /** Workspace reachable state — gates the Restart affordance. */
+  isOnline: boolean;
+  /** Fires when the user clicks Restart (offline-only). */
+  onRestart: () => void;
+}
+
+export function ChatErrorBanner({ message, isOnline, onRestart }: ChatErrorBannerProps) {
+  // Pulled from the global store rather than threaded through props so
+  // the chat tab does not need to know about the side-panel tab state.
+  // Matches how Toolbar.tsx triggers the audit tab (the existing
+  // precedent for cross-tab navigation).
+  const setPanelTab = useCanvasStore((s) => s.setPanelTab);
+
+  if (!message) return null;
+
+  return (
+    <div
+      // role="alert" + aria-live mirrors the project's existing WCAG
+      // 4.1.3 banner pattern (see fix/canvas-errors-aria-alert) — a
+      // screen reader announces the failure as soon as it lands.
+      role="alert"
+      aria-live="assertive"
+      className="px-3 py-2 bg-red-900/20 border-t border-red-800/30"
+    >
+      <div className="flex items-center justify-between gap-2">
+        <span className="text-[10px] text-red-300 break-words flex-1">{message}</span>
+        <div className="flex items-center gap-1.5 shrink-0">
+          <button
+            type="button"
+            onClick={() => setPanelTab("activity")}
+            className="text-[10px] px-2 py-0.5 bg-red-900/40 hover:bg-red-800/60 border border-red-700/40 text-red-200 rounded transition-colors focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-accent focus-visible:ring-offset-1"
+          >
+            View activity log
+          </button>
+          {!isOnline && (
+            <button
+              type="button"
+              onClick={onRestart}
+              className="text-[11px] px-2 py-0.5 bg-red-800 text-red-200 rounded hover:bg-red-700"
+            >
+              Restart
+            </button>
+          )}
+        </div>
+      </div>
+    </div>
+  );
+}

canvas/src/components/tabs/chat/hooks/__tests__/useChatSocket.test.tsx

@@ -0,0 +1,140 @@
+// @vitest-environment jsdom
+import { describe, it, expect, vi, beforeEach, afterEach } from "vitest";
+import { renderHook, act } from "@testing-library/react";
+
+// Capture the handler so we can drive WS events from tests. useSocketEvent
+// stores the latest handler in a ref under the hood, but since we mock
+// the hook entirely, just remember the last passed-in handler.
+let capturedHandler: ((msg: unknown) => void) | null = null;
+vi.mock("@/hooks/useSocketEvent", () => ({
+  useSocketEvent: (h: (msg: unknown) => void) => {
+    capturedHandler = h;
+  },
+}));
+
+// Canvas store mock — useChatSocket calls
+// useCanvasStore.getState().nodes for peer name resolution and reads
+// agentMessages via the selector form. Support both.
+vi.mock("@/store/canvas", () => {
+  const state = {
+    nodes: [
+      { id: "ws-self", data: { name: "Self" } },
+      { id: "ws-peer", data: { name: "Peer Agent" } },
+    ],
+    agentMessages: {} as Record<string, unknown[]>,
+    consumeAgentMessages: () => [],
+  };
+  const hook = (selector?: (s: typeof state) => unknown) =>
+    selector ? selector(state) : state;
+  hook.getState = () => state;
+  return { useCanvasStore: hook };
+});
+
+import { useChatSocket } from "../useChatSocket";
+
+beforeEach(() => {
+  capturedHandler = null;
+});
+
+afterEach(() => {
+  vi.clearAllMocks();
+});
+
+// Helper: assemble an ACTIVITY_LOGGED a2a_receive error event the way
+// the ws-server emits one when a peer call errors out. Fields mirror
+// workspace-server/internal/handlers/activity.go::logActivityExec
+// broadcast payload shape.
+function makeActivityErrorEvent(opts: { workspaceId: string; targetId?: string; errorDetail?: string | undefined }) {
+  return {
+    event: "ACTIVITY_LOGGED",
+    workspace_id: opts.workspaceId,
+    payload: {
+      activity_type: "a2a_receive",
+      method: "message/send",
+      status: "error",
+      target_id: opts.targetId ?? opts.workspaceId,
+      duration_ms: 1500,
+      ...(opts.errorDetail !== undefined ? { error_detail: opts.errorDetail } : {}),
+    },
+    timestamp: "2026-05-18T00:00:00Z",
+  };
+}
+
+describe("useChatSocket — surface error_detail to onSendError (internal#212)", () => {
+  it("forwards the secret-safe error_detail from the broadcast as the onSendError reason", () => {
+    const onSendError = vi.fn();
+    const onSendComplete = vi.fn();
+    renderHook(() =>
+      useChatSocket("ws-self", {
+        onSendError,
+        onSendComplete,
+      }),
+    );
+
+    expect(capturedHandler).not.toBeNull();
+    act(() => {
+      capturedHandler!(
+        makeActivityErrorEvent({
+          workspaceId: "ws-self",
+          errorDetail:
+            "Anthropic 403 oauth_org_not_allowed: Your organization has disabled Claude subscription access for Claude Code",
+        }),
+      );
+    });
+
+    // The hook must NOT fall back to the opaque hardcoded
+    // "Agent error (Exception) — see workspace logs for details." —
+    // that was internal#212. When the broadcast carries an
+    // error_detail, that string is the user-facing reason.
+    expect(onSendError).toHaveBeenCalledTimes(1);
+    const reason = onSendError.mock.calls[0][0] as string;
+    expect(reason).toContain("403");
+    expect(reason).toContain("oauth_org_not_allowed");
+    expect(reason).toContain("disabled Claude subscription");
+    expect(reason).not.toMatch(/see workspace logs for details/i);
+  });
+
+  it("gracefully degrades to the legacy opaque message when error_detail is absent (older ws-server)", () => {
+    // An older ws-server doesn't include error_detail in the payload.
+    // The hook must still fire onSendError with the legacy hardcoded
+    // text so the chat banner has SOMETHING to show. The fix is
+    // additive — never depend on the new field's presence.
+    const onSendError = vi.fn();
+    renderHook(() =>
+      useChatSocket("ws-self", {
+        onSendError,
+      }),
+    );
+
+    act(() => {
+      capturedHandler!(makeActivityErrorEvent({ workspaceId: "ws-self" }));
+    });
+
+    expect(onSendError).toHaveBeenCalledTimes(1);
+    const reason = onSendError.mock.calls[0][0] as string;
+    // Legacy boilerplate is the floor — never silently swallow.
+    expect(reason.length).toBeGreaterThan(0);
+  });
+
+  it("ignores errors targeted at a different workspace's peer", () => {
+    // Defense against a race where the WS hub fans out to all clients —
+    // each chat panel must only react when target_id matches its own
+    // workspace.
+    const onSendError = vi.fn();
+    renderHook(() =>
+      useChatSocket("ws-self", {
+        onSendError,
+      }),
+    );
+    act(() => {
+      capturedHandler!(
+        makeActivityErrorEvent({
+          workspaceId: "ws-self",
+          targetId: "ws-someone-else",
+          errorDetail: "irrelevant",
+        }),
+      );
+    });
+    expect(onSendError).not.toHaveBeenCalled();
+  });
+});

canvas/src/components/tabs/chat/hooks/useChatSocket.ts

@@ -67,9 +67,23 @@ export function useChatSocket(
             const own = (targetId || msg.workspace_id) === workspaceId;
             if (own) {
               callbacksRef.current.onSendComplete?.();
-              callbacksRef.current.onSendError?.(
-                "Agent error (Exception) — see workspace logs for details.",
-              );
+              // internal#212 — surface the actionable, secret-safe
+              // failure reason (provider HTTP status + error code +
+              // human-readable message) the ws-server now puts on
+              // ACTIVITY_LOGGED.error_detail. The old hardcoded
+              // "Agent error (Exception) — see workspace logs for
+              // details." is the fallback only — it pointed at a
+              // workspace-logs tab that doesn't exist, telling the
+              // user nothing they could act on.
+              //
+              // Graceful degradation: older ws-server builds don't
+              // include error_detail, so the legacy boilerplate is
+              // still the floor (never silently swallow).
+              const detail = (p.error_detail as string) || "";
+              const reason = detail
+                ? detail
+                : "Agent error (Exception) — see workspace logs for details.";
+              callbacksRef.current.onSendError?.(reason);
             }
           }
         } else if (type === "a2a_send") {

tests/e2e/lib/peer_visibility_assert.sh

@@ -0,0 +1,165 @@
+# shellcheck shell=bash
+# Shared peer-visibility assertion core — runtime/backend-AGNOSTIC.
+#
+# WHY THIS FILE EXISTS
+# --------------------
+# The peer-visibility gate (PR #1298) was staging-only. Per the standing
+# rule that the local prod-mimic stack must run a MANDATORY local-Postgres
+# E2E BEFORE staging E2E (memory: feedback_local_must_mimic_production,
+# feedback_mandatory_local_e2e_before_ship, feedback_local_test_before_
+# staging_e2e), peer-visibility must also run against the local stack.
+#
+# The ASSERTION must be byte-identical between local and staging — only
+# provisioning differs. So the literal MCP `list_peers` call + every
+# anti-proxy / anti-native-fallback guarantee lives HERE, sourced by both
+# tests/e2e/test_peer_visibility_mcp_staging.sh (staging/CP backend) and
+# tests/e2e/test_peer_visibility_mcp_local.sh (local docker-compose
+# backend). If this assertion ever diverges between the two, that is the
+# bug — keep it in one place.
+#
+# THIS IS NOT A PROXY. pv_assert_runtime issues the byte-for-byte
+# JSON-RPC `tools/call name=list_peers` envelope to `POST
+# /workspaces/:id/mcp` using the workspace's OWN bearer token, through
+# the real WorkspaceAuth + MCPRateLimiter middleware chain — the exact
+# call mcp_molecule_list_peers makes from a canvas agent. It does NOT
+# read a registry row, /health, the heartbeat table, or
+# GET /registry/:id/peers.
+#
+# Contract:
+#   pv_assert_runtime <runtime> <ws_id> <ws_bearer> <base_url> \
+#                     <org_id_or_empty> <all_ws_ids_space_separated>
+#
+#     <org_id_or_empty>  staging: the X-Molecule-Org-Id header value.
+#                        local:   "" (the local single-tenant stack does
+#                                 not gate on the org header; the header
+#                                 is simply omitted when empty).
+#     <all_ws_ids>       every provisioned workspace id (parent + every
+#                        runtime sibling). The expected peer set for this
+#                        runtime is every id in here EXCEPT <ws_id>.
+#
+#   Sets the global PV_VERDICT to one of:
+#     OK
+#     FAIL(http=<code>)
+#     FAIL(native-fallback)
+#     FAIL(rpc=<detail>)
+#     FAIL(peers=<detail>)
+#     FAIL(unknown)
+#   Returns 0 when PV_VERDICT=OK, 1 otherwise. Never exits — the caller
+#   owns aggregation + the gate exit code (10 = regression reproduced).
+#
+# The literal JSON-RPC envelope. Identical to what
+# workspace/platform_tools/registry.py's mcp_molecule_list_peers emits.
+PV_RPC_BODY='{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"list_peers","arguments":{}}}'
+
+pv_assert_runtime() {
+  local rt="$1" wid="$2" wtok="$3" base_url="$4" org_id="$5" all_ws_ids="$6"
+
+  # Expected peer set = every OTHER provisioned workspace, excluding the
+  # caller itself. Byte-identical selection to the original staging script.
+  local expect_ids
+  expect_ids=$(echo "$all_ws_ids" | tr ' ' '\n' | grep -v "^${wid}$" | grep -v '^$')
+
+  # X-Molecule-Org-Id only when the backend supplies one (staging multi-
+  # tenant). Local single-tenant omits it — the same WorkspaceAuth +
+  # MCPRateLimiter chain still runs; only the tenant-routing header differs.
+  local org_header=()
+  if [ -n "$org_id" ]; then
+    org_header=(-H "X-Molecule-Org-Id: $org_id")
+  fi
+
+  local resp http_code body
+  set +e
+  resp=$(curl -sS -X POST "$base_url/workspaces/$wid/mcp" \
+    -H "Authorization: Bearer $wtok" \
+    "${org_header[@]}" \
+    -H "Content-Type: application/json" \
+    -d "$PV_RPC_BODY" \
+    -o /tmp/pv_mcp_body.json -w "%{http_code}" 2>/dev/null)
+  set -e
+  http_code="$resp"
+  body=$(cat /tmp/pv_mcp_body.json 2>/dev/null || echo '')
+
+  echo "--- $rt (ws=$wid) ---"
+  echo "    HTTP $http_code"
+  echo "    body: $(echo "$body" | head -c 600)"
+
+  # (1) HTTP 200 — a 401 (WorkspaceAuth reject, the Hermes symptom) fails here.
+  if [ "$http_code" != "200" ]; then
+    echo "  ✗ $rt: list_peers MCP call returned HTTP $http_code (expected 200)"
+    PV_VERDICT="FAIL(http=$http_code)"
+    return 1
+  fi
+
+  # (2) JSON-RPC result present, not an error object; expected sibling IDs
+  #     present; not a native-sessions fallback. Byte-identical to the
+  #     original staging script's inline python.
+  local parse
+  parse=$(echo "$body" | python3 -c "
+import sys, json
+expect = set(filter(None, '''$expect_ids'''.split()))
+try:
+    d = json.load(sys.stdin)
+except Exception as e:
+    print('PARSE_ERROR:' + str(e)); sys.exit(0)
+if isinstance(d, dict) and d.get('error') is not None:
+    print('RPC_ERROR:' + json.dumps(d['error'])[:200]); sys.exit(0)
+res = d.get('result') if isinstance(d, dict) else None
+if res is None:
+    print('NO_RESULT'); sys.exit(0)
+# MCP tools/call result shape: {content:[{type:text,text:'<json or prose>'}]}
+text = ''
+if isinstance(res, dict):
+    for c in res.get('content', []):
+        if c.get('type') == 'text':
+            text += c.get('text', '')
+text_l = text.lower()
+# Native-sessions fallback signature (the OpenClaw symptom): the agent
+# answered from its own runtime session list, not the platform peer set.
+if 'sessions_list' in text_l or 'no platform peers' in text_l or 'native session' in text_l:
+    print('NATIVE_FALLBACK:' + text[:200]); sys.exit(0)
+# The expected sibling IDs must literally appear in the returned peer text.
+found = sorted(i for i in expect if i in text)
+missing = sorted(expect - set(found))
+if not expect:
+    print('NO_EXPECTED_PEERS_CONFIGURED'); sys.exit(0)
+if missing:
+    print('MISSING_PEERS:found=%d/%d missing=%s' % (len(found), len(expect), ','.join(m[:8] for m in missing)))
+    sys.exit(0)
+print('OK:found=%d/%d' % (len(found), len(expect)))
+" 2>/dev/null)
+
+  case "$parse" in
+    OK:*)
+      echo "  ✓ $rt: list_peers returned 200 and contains all expected peers ($parse)"
+      PV_VERDICT="OK"
+      return 0
+      ;;
+    NATIVE_FALLBACK:*)
+      echo "  ✗ $rt: list_peers fell back to NATIVE sessions — sees no platform peers ($parse)"
+      PV_VERDICT="FAIL(native-fallback)"
+      return 1
+      ;;
+    RPC_ERROR:*|NO_RESULT|PARSE_ERROR:*)
+      echo "  ✗ $rt: list_peers MCP call did not return a usable result ($parse)"
+      PV_VERDICT="FAIL(rpc=$parse)"
+      return 1
+      ;;
+    MISSING_PEERS:*)
+      echo "  ✗ $rt: list_peers returned 200 but peer set is wrong/empty ($parse)"
+      PV_VERDICT="FAIL(peers=$parse)"
+      return 1
+      ;;
+    NO_EXPECTED_PEERS_CONFIGURED)
+      # Caller bug, not a runtime regression — surface loudly so a
+      # mis-wired backend can't mint a false green.
+      echo "  ✗ $rt: no expected peers were configured for this caller"
+      PV_VERDICT="FAIL(rpc=NO_EXPECTED_PEERS_CONFIGURED)"
+      return 1
+      ;;
+    *)
+      echo "  ✗ $rt: unexpected verdict '$parse'"
+      PV_VERDICT="FAIL(unknown)"
+      return 1
+      ;;
+  esac
+}

tests/e2e/test_peer_visibility_mcp_local.sh

@@ -0,0 +1,328 @@
+#!/usr/bin/env bash
+# LOCAL E2E — fresh-provision peer-visibility gate via the LITERAL MCP path.
+#
+# WHY THIS EXISTS
+# ---------------
+# tests/e2e/test_peer_visibility_mcp_staging.sh (PR #1298) codified the
+# literal user-facing peer-visibility path — but staging-only. The
+# standing rule is that the local prod-mimic stack runs a MANDATORY
+# local-Postgres E2E BEFORE staging E2E (memory:
+# feedback_local_must_mimic_production, feedback_mandatory_local_e2e_
+# before_ship, feedback_local_test_before_staging_e2e,
+# feedback_real_subprocess_test_for_boot_path). A staging-only gate means
+# regressions are caught late and expensively on EC2. This is the LOCAL
+# backend: same byte-identical assertion, local docker-compose stack.
+#
+# THE ASSERTION IS NOT A PROXY and is BYTE-IDENTICAL to staging — it is
+# the SAME tests/e2e/lib/peer_visibility_assert.sh::pv_assert_runtime that
+# the staging script calls. It issues the byte-for-byte JSON-RPC
+# `tools/call name=list_peers` envelope to `POST /workspaces/:id/mcp`
+# using each workspace's OWN bearer token, through the real WorkspaceAuth
+# + MCPRateLimiter middleware chain — the exact call
+# mcp_molecule_list_peers makes from a canvas agent. It does NOT read a
+# registry row, /health, the heartbeat table, or GET /registry/:id/peers.
+#
+# Only PROVISIONING differs from staging:
+#   - staging: POST /cp/admin/orgs (cold EC2 tenant) + per-tenant admin
+#     token + each workspace's auth_token from the POST /workspaces resp.
+#   - local:   POST /workspaces directly against the local stack
+#     (BASE, default http://localhost:8080), MCP bearer minted via
+#     GET /admin/workspaces/:id/test-token (e2e_mint_test_token —
+#     deterministic, gated by MOLECULE_ENV != production). Same model
+#     every other local E2E (test_priority_runtimes_e2e.sh,
+#     test_api.sh) already uses; no new credential/provision flow.
+#
+# It is written to FAIL on today's broken Hermes/OpenClaw behavior and go
+# green only when the in-flight root-cause fixes (Hermes-401 #162,
+# OpenClaw-never-online/MCP-wiring #165) actually land — same gate
+# semantics + exit codes as the staging script. NON-required by design
+# until then (flip-to-required tracked at molecule-core#1296), and NOT
+# masked with continue-on-error (feedback_fix_root_not_symptom).
+#
+# Required env: none (local stack only).
+# Optional env:
+#   BASE                    default http://localhost:8080
+#   PV_RUNTIMES             space list; default "hermes openclaw claude-code"
+#   E2E_PROVISION_TIMEOUT_SECS  per-workspace online budget; default 900
+#                            (hermes cold apt+uv is the slow path locally)
+#   E2E_KEEP_WS             1 → skip teardown (local debugging only)
+#   LLM provider keys (a workspace boots only if its provider key is set;
+#   a runtime whose key is absent is SKIPPED, not failed — a partially
+#   keyed local env must not false-fail the gate):
+#     CLAUDE_CODE_OAUTH_TOKEN  claude-code
+#     E2E_MINIMAX_API_KEY      hermes/openclaw (MiniMax, preferred)
+#     E2E_ANTHROPIC_API_KEY    hermes/openclaw (direct Anthropic)
+#     E2E_OPENAI_API_KEY       hermes/openclaw (OpenAI)
+#
+# Exit codes (match the staging script):
+#   0  every runtime under test saw its peers via the literal MCP call
+#   1  generic failure
+#   3  a workspace never reached online within the budget
+#   10 peer-visibility regression reproduced (the gate firing as designed)
+
+set -uo pipefail
+
+source "$(dirname "$0")/_lib.sh"
+# Byte-identical assertion shared with the staging backend.
+# shellcheck source=tests/e2e/lib/peer_visibility_assert.sh
+source "$(dirname "$0")/lib/peer_visibility_assert.sh"
+
+PV_RUNTIMES="${PV_RUNTIMES:-hermes openclaw claude-code}"
+PROVISION_TIMEOUT_SECS="${E2E_PROVISION_TIMEOUT_SECS:-900}"
+NAME_PREFIX="PV-Local-$$-$(date +%H%M%S)"
+
+log()  { echo "[$(date +%H:%M:%S)] $*"; }
+ok()   { echo "[$(date +%H:%M:%S)] ✅ $*"; }
+
+CREATED_WSIDS=()
+
+# ─── Scoped teardown ───────────────────────────────────────────────────
+# Deletes ONLY the workspaces THIS run created (tracked in CREATED_WSIDS),
+# one DELETE /workspaces/:id?confirm=true each. NEVER e2e_cleanup_all_
+# workspaces / any blanket sweep — honors feedback_cleanup_after_each_test
+# and feedback_never_run_cluster_cleanup_tests_on_live_platform (a local
+# stack can still be shared with other concurrent local E2E).
+teardown() {
+  local rc=$?
+  set +e
+  if [ "${E2E_KEEP_WS:-0}" = "1" ]; then
+    echo ""
+    log "[teardown] E2E_KEEP_WS=1 — leaving ${#CREATED_WSIDS[@]} ws for debugging (REMEMBER TO DELETE)"
+    exit $rc
+  fi
+  echo ""
+  log "[teardown] deleting ${#CREATED_WSIDS[@]} workspace(s) this run created (scoped)"
+  for wid in ${CREATED_WSIDS[@]+"${CREATED_WSIDS[@]}"}; do
+    [ -n "$wid" ] || continue
+    curl -s -X DELETE "$BASE/workspaces/$wid?confirm=true" >/dev/null 2>&1 || true
+  done
+  exit $rc
+}
+trap teardown EXIT INT TERM
+
+# Pre-sweep workspaces a prior crashed run of THIS script left behind
+# (name prefix match only — never a blanket delete). The trap fires on
+# normal exit, but a kill -9 / SIGPIPE can bypass it.
+PRIOR=$(curl -s "$BASE/workspaces" | python3 -c '
+import json, sys
+try:
+    print(" ".join(w["id"] for w in json.load(sys.stdin) if w.get("name","").startswith("PV-Local-")))
+except Exception:
+    pass
+' 2>/dev/null)
+for _wid in $PRIOR; do
+  log "Pre-sweeping prior PV-Local workspace: $_wid"
+  curl -s -X DELETE "$BASE/workspaces/$_wid?confirm=true" >/dev/null 2>&1 || true
+done
+
+# ─── Local-stack preflight ─────────────────────────────────────────────
+log "0/5 local stack preflight: $BASE/health"
+if ! curl -fsS "$BASE/health" -m 5 >/dev/null 2>&1; then
+  echo "::error::Local stack not healthy at $BASE/health — bring it up (make up) before this gate. Infra, not a workspace bug (feedback_fix_root_not_symptom)." >&2
+  exit 1
+fi
+# admin/test-token is the local MCP-bearer mint path; it 404s in
+# production. If it is off, this gate cannot drive the literal call.
+if ! curl -fsS "$BASE/admin/workspaces/preflight-probe/test-token" -m 5 >/dev/null 2>&1; then
+  # A 404 here is EITHER "no such ws" (fine — endpoint is enabled) OR the
+  # endpoint is disabled (MOLECULE_ENV=production). Distinguish by body.
+  PROBE=$(curl -s "$BASE/admin/workspaces/preflight-probe/test-token" -m 5 2>/dev/null)
+  if echo "$PROBE" | grep -qi 'production\|disabled\|not found.*endpoint'; then
+    echo "::error::GET /admin/workspaces/:id/test-token disabled (MOLECULE_ENV=production?). Cannot mint a local MCP bearer." >&2
+    exit 1
+  fi
+fi
+ok "    local stack healthy"
+
+# ─── Resolve per-runtime provisioning secrets ──────────────────────────
+# Mirrors test_priority_runtimes_e2e.sh / test_staging_full_saas.sh's
+# provider-key chain. A runtime whose key is absent is SKIPPED (not
+# failed) so a partially keyed local env doesn't false-fail the gate.
+runtime_secrets() {
+  local rt="$1"
+  case "$rt" in
+    claude-code)
+      [ -n "${CLAUDE_CODE_OAUTH_TOKEN:-}" ] || { echo ""; return 1; }
+      python3 -c "import json,os;print(json.dumps({'CLAUDE_CODE_OAUTH_TOKEN':os.environ['CLAUDE_CODE_OAUTH_TOKEN']}))"
+      ;;
+    hermes|openclaw)
+      if [ -n "${E2E_MINIMAX_API_KEY:-}" ]; then
+        python3 -c "import json,os;k=os.environ['E2E_MINIMAX_API_KEY'];print(json.dumps({'ANTHROPIC_BASE_URL':'https://api.minimax.io/anthropic','ANTHROPIC_AUTH_TOKEN':k,'MINIMAX_API_KEY':k}))"
+      elif [ -n "${E2E_ANTHROPIC_API_KEY:-}" ]; then
+        python3 -c "import json,os;k=os.environ['E2E_ANTHROPIC_API_KEY'];print(json.dumps({'ANTHROPIC_API_KEY':k}))"
+      elif [ -n "${E2E_OPENAI_API_KEY:-}" ]; then
+        python3 -c "import json,os;k=os.environ['E2E_OPENAI_API_KEY'];print(json.dumps({'OPENAI_API_KEY':k,'OPENAI_BASE_URL':'https://api.openai.com/v1','MODEL_PROVIDER':'openai:gpt-4o','HERMES_INFERENCE_PROVIDER':'custom','HERMES_CUSTOM_BASE_URL':'https://api.openai.com/v1','HERMES_CUSTOM_API_KEY':k,'HERMES_CUSTOM_API_MODE':'chat_completions'}))"
+      else
+        echo ""; return 1
+      fi
+      ;;
+    *)
+      # Unknown runtime: provision with empty secrets and let the stack
+      # decide (kept permissive so PV_RUNTIMES can be widened later).
+      echo "{}"
+      ;;
+  esac
+}
+
+# Block until $1 reaches one of $2 (space-separated), or $3 sec elapse.
+wait_for_status() {
+  local wsid="$1" want="$2" budget="$3" start=$SECONDS last=""
+  while [ $((SECONDS - start)) -lt "$budget" ]; do
+    local s
+    s=$(curl -s "$BASE/workspaces/$wsid" | python3 -c 'import json,sys
+try:
+  d=json.load(sys.stdin); w=d.get("workspace") if isinstance(d.get("workspace"),dict) else d; print(w.get("status",""))
+except Exception:
+  print("")' 2>/dev/null || echo "")
+    [ "$s" != "$last" ] && { log "    $wsid → ${s:-<none>}"; last="$s"; }
+    for w in $want; do [ "$s" = "$w" ] && { echo "$s"; return 0; }; done
+    sleep 5
+  done
+  echo "$last"
+  return 1
+}
+
+# ─── 1. Provision parent (claude-code) + one sibling per runtime ───────
+# Same topology as the staging script: a claude-code parent plus one
+# sibling per runtime under test, so each runtime should see all others.
+log "1/5 provisioning parent (claude-code) + one sibling per runtime under test..."
+
+PARENT_SECRETS=$(runtime_secrets claude-code) || PARENT_SECRETS=""
+if [ -z "$PARENT_SECRETS" ]; then
+  # Parent still needs to exist as a peer target even without an LLM key;
+  # it never has to answer list_peers itself (it is excluded from the
+  # caller set), so an empty-secrets claude-code shell is sufficient.
+  PARENT_SECRETS="{}"
+fi
+P_RESP=$(curl -s -X POST "$BASE/workspaces" -H "Content-Type: application/json" \
+  -d "{\"name\":\"${NAME_PREFIX}-parent\",\"runtime\":\"claude-code\",\"tier\":3,\"secrets\":$PARENT_SECRETS}")
+PARENT_ID=$(echo "$P_RESP" | python3 -c 'import json,sys;print(json.load(sys.stdin).get("id",""))' 2>/dev/null)
+if [ -z "$PARENT_ID" ]; then
+  echo "::error::parent create failed: $(echo "$P_RESP" | head -c 300)" >&2
+  exit 1
+fi
+CREATED_WSIDS+=("$PARENT_ID")
+log "    PARENT_ID=$PARENT_ID"
+
+# NOTE: no `declare -A` — this script must also run on a local macOS dev
+# box (bash 3.2, no associative arrays) per feedback_local_must_mimic_
+# production. WS_IDS / VERDICT are kept as newline-delimited "rt<TAB>val"
+# maps with tiny get/set helpers (portable to bash 3.2+ AND ubuntu CI).
+WS_IDS_MAP=""
+VERDICT_MAP=""
+_map_set() { # _map_set <mapvarname> <key> <value>
+  local __m="$1" __k="$2" __v="$3" __cur
+  eval "__cur=\$$__m"
+  __cur=$(printf '%s' "$__cur" | grep -v "^${__k}	" || true)
+  if [ -n "$__cur" ]; then
+    eval "$__m=\$(printf '%s\n%s\t%s' \"\$__cur\" \"\$__k\" \"\$__v\")"
+  else
+    eval "$__m=\$(printf '%s\t%s' \"\$__k\" \"\$__v\")"
+  fi
+}
+_map_get() { # _map_get <mapvarname> <key>  -> stdout value (empty if absent)
+  local __m="$1" __k="$2" __cur
+  eval "__cur=\$$__m"
+  printf '%s\n' "$__cur" | awk -F'\t' -v k="$__k" '$1==k {print $2; exit}'
+}
+
+ALL_WS_IDS="$PARENT_ID"
+ACTIVE_RUNTIMES=""
+for rt in $PV_RUNTIMES; do
+  SEC=$(runtime_secrets "$rt") || SEC=""
+  if [ -z "$SEC" ]; then
+    log "    SKIP $rt — no provider key in env (partially-keyed local env; not a failure)"
+    continue
+  fi
+  R=$(curl -s -X POST "$BASE/workspaces" -H "Content-Type: application/json" \
+    -d "{\"name\":\"${NAME_PREFIX}-$rt\",\"runtime\":\"$rt\",\"tier\":2,\"parent_id\":\"$PARENT_ID\",\"secrets\":$SEC}")
+  WID=$(echo "$R" | python3 -c 'import json,sys;print(json.load(sys.stdin).get("id",""))' 2>/dev/null)
+  if [ -z "$WID" ]; then
+    echo "::error::$rt workspace create failed: $(echo "$R" | head -c 300)" >&2
+    exit 1
+  fi
+  _map_set WS_IDS_MAP "$rt" "$WID"
+  CREATED_WSIDS+=("$WID")
+  ALL_WS_IDS="$ALL_WS_IDS $WID"
+  ACTIVE_RUNTIMES="$ACTIVE_RUNTIMES $rt"
+  log "    $rt → $WID"
+done
+ACTIVE_RUNTIMES="$(echo "$ACTIVE_RUNTIMES" | xargs)"
+
+if [ -z "$ACTIVE_RUNTIMES" ]; then
+  echo "::error::No runtime had a provider key set — cannot run the local peer-visibility gate. Set CLAUDE_CODE_OAUTH_TOKEN and/or E2E_MINIMAX_API_KEY (or ANTHROPIC/OPENAI)." >&2
+  exit 1
+fi
+
+# ─── 2. Wait for the parent online (it is a peer target) ───────────────
+log "2/5 waiting for parent online (peer target)..."
+PF=$(wait_for_status "$PARENT_ID" "online" "$PROVISION_TIMEOUT_SECS") || true
+if [ "$PF" != "online" ]; then
+  echo "::error::parent ($PARENT_ID) never reached online (last=$PF) within ${PROVISION_TIMEOUT_SECS}s" >&2
+  exit 3
+fi
+ok "    parent online"
+
+# ─── 3. Wait for every sibling online ──────────────────────────────────
+# A runtime that never comes online locally is itself a finding: it
+# reproduces the openclaw-never-online class (#165) on the local stack.
+log "3/5 waiting for all siblings online (up to ${PROVISION_TIMEOUT_SECS}s each — cold boot)..."
+REGRESSED=0
+ONLINE_RUNTIMES=""
+for rt in $ACTIVE_RUNTIMES; do
+  wid="$(_map_get WS_IDS_MAP "$rt")"
+  S=$(wait_for_status "$wid" "online" "$PROVISION_TIMEOUT_SECS") || true
+  if [ "$S" != "online" ]; then
+    echo "  ✗ $rt ($wid): never reached online (last=$S) — reproduces the never-online class locally"
+    _map_set VERDICT_MAP "$rt" "FAIL(never-online:last=$S)"
+    REGRESSED=1
+    continue
+  fi
+  ok "    $rt online"
+  ONLINE_RUNTIMES="$ONLINE_RUNTIMES $rt"
+done
+
+# ─── 4. THE GATE — literal mcp_molecule_list_peers via POST /:id/mcp ────
+# Shared, byte-identical assertion. Local passes "" for the org id (the
+# single-tenant local stack does not gate on X-Molecule-Org-Id); the
+# literal MCP call + every anti-proxy / anti-native-fallback guarantee is
+# the SAME code the staging backend runs.
+log "4/5 driving the LITERAL list_peers MCP call per online runtime..."
+echo ""
+for rt in $ONLINE_RUNTIMES; do
+  wid="$(_map_get WS_IDS_MAP "$rt")"
+  WTOK=$(e2e_mint_test_token "$wid" 2>/dev/null || true)
+  if [ -z "$WTOK" ]; then
+    echo "--- $rt (ws=$wid) ---"
+    echo "  ✗ $rt: could not mint a local MCP bearer (admin/test-token) — cannot drive the literal call"
+    _map_set VERDICT_MAP "$rt" "FAIL(no-bearer)"
+    REGRESSED=1
+    echo ""
+    continue
+  fi
+  PV_VERDICT=""
+  pv_assert_runtime "$rt" "$wid" "$WTOK" "$BASE" "" "$ALL_WS_IDS" || REGRESSED=1
+  _map_set VERDICT_MAP "$rt" "$PV_VERDICT"
+  echo ""
+done
+
+# ─── 5. Summary + honest gate exit ─────────────────────────────────────
+echo "=== SUMMARY — LOCAL fresh-provision peer-visibility (literal MCP list_peers) ==="
+for rt in $ACTIVE_RUNTIMES; do
+  _v="$(_map_get VERDICT_MAP "$rt")"
+  printf '  %-14s %s\n' "$rt" "${_v:-NO_RUN}"
+done
+echo ""
+
+if [ "$REGRESSED" -ne 0 ]; then
+  echo "✗ GATE FAILED (LOCAL) — at least one runtime cannot see its peers via"
+  echo "  the literal mcp_molecule_list_peers call on the local prod-mimic"
+  echo "  stack. This is the SAME user-facing failure the proxy signals were"
+  echo "  hiding, reproduced locally (far faster than EC2). Expected RED until"
+  echo "  the Hermes-401 (#162) + OpenClaw-never-online/MCP-wiring (#165)"
+  echo "  root-cause fixes land; goes green only when they actually do."
+  exit 10
+fi
+
+ok "GATE PASSED (LOCAL) — every runtime under test sees its platform peers via the literal MCP call."
+exit 0

tests/e2e/test_peer_visibility_mcp_staging.sh

@@ -64,6 +64,13 @@
 
 set -uo pipefail
 
+# The literal MCP list_peers assertion lives in the shared, backend-
+# agnostic lib so it is BYTE-IDENTICAL between this staging backend and
+# the local docker-compose backend (tests/e2e/test_peer_visibility_mcp_
+# local.sh). Only provisioning/teardown differs per backend.
+# shellcheck source=tests/e2e/lib/peer_visibility_assert.sh
+source "$(dirname "${BASH_SOURCE[0]}")/lib/peer_visibility_assert.sh"
+
 CP_URL="${MOLECULE_CP_URL:-https://staging-api.moleculesai.app}"
 ADMIN_TOKEN="${MOLECULE_ADMIN_TOKEN:?MOLECULE_ADMIN_TOKEN required — Railway staging CP_ADMIN_API_TOKEN}"
 RUN_ID_SUFFIX="${E2E_RUN_ID:-$(date +%H%M%S)-$$}"
@@ -259,101 +266,19 @@ done
 # through WorkspaceAuth + MCPRateLimiter.
 log "6/6 driving the LITERAL list_peers MCP call per runtime..."
 echo ""
-RPC_BODY='{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"list_peers","arguments":{}}}'
 REGRESSED=0
 declare -A VERDICT
 
 for rt in $PV_RUNTIMES; do
   wid="${WS_IDS[$rt]}"
   wtok="${WS_TOKENS[$rt]}"
-  # The expected peer set = every OTHER provisioned workspace (parent +
-  # the sibling runtimes), excluding the caller itself.
-  EXPECT_IDS=$(echo "$ALL_WS_IDS" | tr ' ' '\n' | grep -v "^${wid}$" | grep -v '^$')
-
-  set +e
-  RESP=$(curl -sS -X POST "$TENANT_URL/workspaces/$wid/mcp" \
-    -H "Authorization: Bearer $wtok" \
-    -H "X-Molecule-Org-Id: $ORG_ID" \
-    -H "Content-Type: application/json" \
-    -d "$RPC_BODY" \
-    -o /tmp/pv_mcp_body.json -w "%{http_code}" 2>/dev/null)
-  set -e
-  HTTP_CODE="$RESP"
-  BODY=$(cat /tmp/pv_mcp_body.json 2>/dev/null || echo '')
-
-  echo "--- $rt (ws=$wid) ---"
-  echo "    HTTP $HTTP_CODE"
-  echo "    body: $(echo "$BODY" | head -c 600)"
-
-  # (1) HTTP 200 — a 401 (WorkspaceAuth reject, the Hermes symptom) fails here.
-  if [ "$HTTP_CODE" != "200" ]; then
-    echo "  ✗ $rt: list_peers MCP call returned HTTP $HTTP_CODE (expected 200)"
-    VERDICT[$rt]="FAIL(http=$HTTP_CODE)"
-    REGRESSED=1
-    continue
-  fi
-
-  # (2) JSON-RPC result present, not an error object.
-  PARSE=$(echo "$BODY" | python3 -c "
-import sys, json
-expect = set(filter(None, '''$EXPECT_IDS'''.split()))
-try:
-    d = json.load(sys.stdin)
-except Exception as e:
-    print('PARSE_ERROR:' + str(e)); sys.exit(0)
-if isinstance(d, dict) and d.get('error') is not None:
-    print('RPC_ERROR:' + json.dumps(d['error'])[:200]); sys.exit(0)
-res = d.get('result') if isinstance(d, dict) else None
-if res is None:
-    print('NO_RESULT'); sys.exit(0)
-# MCP tools/call result shape: {content:[{type:text,text:'<json or prose>'}]}
-text = ''
-if isinstance(res, dict):
-    for c in res.get('content', []):
-        if c.get('type') == 'text':
-            text += c.get('text', '')
-text_l = text.lower()
-# Native-sessions fallback signature (the OpenClaw symptom): the agent
-# answered from its own runtime session list, not the platform peer set.
-if 'sessions_list' in text_l or 'no platform peers' in text_l or 'native session' in text_l:
-    print('NATIVE_FALLBACK:' + text[:200]); sys.exit(0)
-# The expected sibling IDs must literally appear in the returned peer text.
-found = sorted(i for i in expect if i in text)
-missing = sorted(expect - set(found))
-if not expect:
-    print('NO_EXPECTED_PEERS_CONFIGURED'); sys.exit(0)
-if missing:
-    print('MISSING_PEERS:found=%d/%d missing=%s' % (len(found), len(expect), ','.join(m[:8] for m in missing)))
-    sys.exit(0)
-print('OK:found=%d/%d' % (len(found), len(expect)))
-" 2>/dev/null)
-
-  case "$PARSE" in
-    OK:*)
-      echo "  ✓ $rt: list_peers returned 200 and contains all expected peers ($PARSE)"
-      VERDICT[$rt]="OK"
-      ;;
-    NATIVE_FALLBACK:*)
-      echo "  ✗ $rt: list_peers fell back to NATIVE sessions — sees no platform peers ($PARSE)"
-      VERDICT[$rt]="FAIL(native-fallback)"
-      REGRESSED=1
-      ;;
-    RPC_ERROR:*|NO_RESULT|PARSE_ERROR:*)
-      echo "  ✗ $rt: list_peers MCP call did not return a usable result ($PARSE)"
-      VERDICT[$rt]="FAIL(rpc=$PARSE)"
-      REGRESSED=1
-      ;;
-    MISSING_PEERS:*)
-      echo "  ✗ $rt: list_peers returned 200 but peer set is wrong/empty ($PARSE)"
-      VERDICT[$rt]="FAIL(peers=$PARSE)"
-      REGRESSED=1
-      ;;
-    *)
-      echo "  ✗ $rt: unexpected verdict '$PARSE'"
-      VERDICT[$rt]="FAIL(unknown)"
-      REGRESSED=1
-      ;;
-  esac
+  # Byte-identical assertion via the shared lib. Staging passes ORG_ID as
+  # the X-Molecule-Org-Id header value; the literal MCP call + every
+  # anti-proxy / anti-native-fallback guarantee is the SAME code the
+  # local backend runs.
+  PV_VERDICT=""
+  pv_assert_runtime "$rt" "$wid" "$wtok" "$TENANT_URL" "$ORG_ID" "$ALL_WS_IDS" || REGRESSED=1
+  VERDICT[$rt]="$PV_VERDICT"
   echo ""
 done
 

workspace-server/internal/handlers/a2a_proxy_helpers.go

@@ -168,6 +168,21 @@ func (h *WorkspaceHandler) maybeMarkContainerDead(ctx context.Context, workspace
 	if !h.HasProvisioner() {
 		return false
 	}
+	// Restart-aware short-circuit: during the 20-30s EC2-pending window of
+	// an in-flight restart, the workspace's url='' and IsRunning() returns
+	// false → looks indistinguishable from a dead container. Pre-fix this
+	// fired a fresh RestartByID for the just-launched instance, which
+	// coalesceRestart's pending-flag drained by running ANOTHER full
+	// stop+provision cycle (= ec2_stopped of the still-pending instance
+	// → re-provision). That's the 4x reprov thrash class. Skip the
+	// container-dead path while a restart is in flight; the in-flight
+	// restart's own provisionWorkspaceAutoSync will surface a real failure
+	// (markProvisionFailed) if the new container never comes up. Issue
+	// internal#544.
+	if isRestarting(workspaceID) {
+		log.Printf("ProxyA2A: maybeMarkContainerDead skipped for %s — restart already in flight (self-fire guard)", workspaceID)
+		return false
+	}
 
 	var running bool
 	var inspectErr error
@@ -223,6 +238,18 @@ func (h *WorkspaceHandler) maybeMarkContainerDead(ctx context.Context, workspace
 // shape post-EC2-replace (see molecule-controlplane#20 incident
 // 2026-05-07) where the reconciler hasn't respawned the agent yet.
 func (h *WorkspaceHandler) preflightContainerHealth(ctx context.Context, workspaceID string) *proxyA2AError {
+	// Restart-aware short-circuit (mirror of maybeMarkContainerDead): if a
+	// restart cycle is in flight for this workspace, do not run the
+	// IsRunning probe — it would observe the EC2-pending state as "not
+	// running" and trigger RestartByID for an already-restarting workspace,
+	// closing the self-fire loop. Returning nil lets the optimistic
+	// forward proceed; the upstream Do() call will fail with a connection
+	// error or 502, and the *post-restart* reactive path can decide what
+	// to do once the cycle has actually completed. Issue internal#544.
+	if isRestarting(workspaceID) {
+		log.Printf("ProxyA2A preflight: %s — skipped, restart already in flight (self-fire guard)", workspaceID)
+		return nil
+	}
 	running, err := h.provisioner.IsRunning(ctx, workspaceID)
 	if err != nil {
 		// Transient daemon error. Provisioner.IsRunning returns (true, err)

workspace-server/internal/handlers/activity.go

@@ -8,6 +8,7 @@ import (
 	"fmt"
 	"log"
 	"net/http"
+	"regexp"
 	"strconv"
 	"strings"
 	"time"
@@ -18,6 +19,46 @@ import (
 	"github.com/google/uuid"
 )
 
+// internal#212 — secret-safe scrubber applied to error_detail strings
+// before they cross the canvas WebSocket. Defense in depth: the
+// workspace runtime already runs `_sanitize_for_external` on its side
+// (workspace/executor_helpers.py), but the broadcast layer is the last
+// stop before the string reaches the user's browser, so we re-scrub
+// here in case any caller path forgot.
+//
+// The scrubber is intentionally surgical — it MUST preserve the
+// actionable parts (HTTP status codes, error codes like
+// `oauth_org_not_allowed`, human-readable provider messages) and
+// remove only what looks credential-ish. Over-redacting defeats the
+// whole point of internal#212 (giving the user a reason they can act on).
+
+// Capture (auth-key prefix) (value) so the prefix can be preserved in
+// the output. The keyword anchor prevents false positives on regular
+// text that happens to contain a long alphanumeric run.
+var errorDetailSecretRE = regexp.MustCompile(`(?i)((?:bearer|token|api[_-]?key|sk-proj-|sk-)[ :=]*)[A-Za-z0-9_/.-]{20,}`)
+
+// Stringly-typed JWT-shape: 3 dot-separated base64url segments, second
+// and third at least 16 chars. Matches eyJ-prefixed tokens that the
+// keyword-anchored rule above would miss when they appear bare.
+var errorDetailJWTRE = regexp.MustCompile(`eyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{16,}\.[A-Za-z0-9_-]{16,}`)
+
+const errorDetailBroadcastCap = 4096
+
+func sanitizeErrorDetailForBroadcast(s string) string {
+	if s == "" {
+		return s
+	}
+	// Cap first — a huge error body shouldn't tax every websocket
+	// client's buffer. 4096 matches the workspace-side _MAX_STDERR
+	// budget (it's actually larger here so the runtime's cap dominates).
+	if len(s) > errorDetailBroadcastCap {
+		s = s[:errorDetailBroadcastCap] + "…[truncated]"
+	}
+	s = errorDetailSecretRE.ReplaceAllString(s, "${1}[REDACTED]")
+	s = errorDetailJWTRE.ReplaceAllString(s, "[REDACTED]")
+	return s
+}
+
 type ActivityHandler struct {
 	broadcaster *events.Broadcaster
 }
@@ -691,6 +732,16 @@ func logActivityExec(ctx context.Context, exec activityExecutor, broadcaster eve
 		if respStr != nil {
 			payload["response_body"] = json.RawMessage(respJSON)
 		}
+		// internal#212 — surface the secret-safe failure reason on the
+		// live broadcast so the canvas chat-tab error banner can show
+		// the user *why* (provider HTTP status, error code, the
+		// provider's own human message) instead of the opaque
+		// "Agent error (Exception) — see workspace logs for details."
+		// hardcoded fallback. Omitted when nil so the canvas's "has
+		// actionable reason" guard doesn't trip on empty-string keys.
+		if params.ErrorDetail != nil && *params.ErrorDetail != "" {
+			payload["error_detail"] = sanitizeErrorDetailForBroadcast(*params.ErrorDetail)
+		}
 	}
 
 	return func() {

workspace-server/internal/handlers/activity_test.go

@@ -934,6 +934,184 @@ func TestLogActivity_Broadcast_IncludesRequestAndResponseBodies(t *testing.T) {
 	}
 }
 
+// TestLogActivity_Broadcast_IncludesErrorDetail pins the internal#212
+// UX fix: when an a2a_receive row is logged with status="error" and a
+// non-empty error_detail, the live broadcast MUST carry that detail so
+// the canvas chat-tab error bubble can render the actionable reason
+// (e.g. the provider's own 403 message) instead of the opaque
+// "Agent error (Exception) — see workspace logs for details." string.
+// Without this, the canvas falls back to the hardcoded boilerplate;
+// the row's error_detail is in the DB but never reaches the user
+// without a manual refresh of the Activity tab.
+func TestLogActivity_Broadcast_IncludesErrorDetail(t *testing.T) {
+	mock := setupTestDB(t)
+	defer mock.ExpectationsWereMet()
+
+	mock.ExpectExec("INSERT INTO activity_logs").
+		WillReturnResult(sqlmock.NewResult(1, 1))
+
+	cb := &recordingBroadcaster{}
+	srcID := "ws-source"
+	tgtID := "ws-target"
+	method := "message/send"
+	// Realistic actionable reason: provider HTTP status + provider's
+	// own message. Secret-safe (no token, no api key, just the cause).
+	detail := "Anthropic 403 oauth_org_not_allowed: Your organization has disabled Claude subscription access for Claude Code — use an Anthropic API key or ask your admin to enable access."
+
+	LogActivity(context.Background(), cb, ActivityParams{
+		WorkspaceID:  "ws-source",
+		ActivityType: "a2a_receive",
+		SourceID:     &srcID,
+		TargetID:     &tgtID,
+		Method:       &method,
+		Status:       "error",
+		ErrorDetail:  &detail,
+	})
+
+	if len(cb.calls) != 1 {
+		t.Fatalf("expected 1 broadcast, got %d", len(cb.calls))
+	}
+	payload := cb.calls[0].payload
+	got, ok := payload["error_detail"].(string)
+	if !ok {
+		t.Fatalf("error_detail missing from broadcast payload: got %#v", payload["error_detail"])
+	}
+	if got != detail {
+		t.Errorf("error_detail = %q, want %q", got, detail)
+	}
+}
+
+// TestLogActivity_Broadcast_OmitsErrorDetailWhenNil pins the inverse:
+// rows logged without an error_detail (the common ok-path) must not
+// have an empty "error_detail":"" key in the broadcast, which would
+// false-positive the canvas's "has actionable reason" guard and render
+// an empty Underlying-Error block. The omission rule matches how
+// request_body/response_body are handled.
+func TestLogActivity_Broadcast_OmitsErrorDetailWhenNil(t *testing.T) {
+	mock := setupTestDB(t)
+	defer mock.ExpectationsWereMet()
+
+	mock.ExpectExec("INSERT INTO activity_logs").
+		WillReturnResult(sqlmock.NewResult(1, 1))
+
+	cb := &recordingBroadcaster{}
+	srcID := "ws-source"
+
+	LogActivity(context.Background(), cb, ActivityParams{
+		WorkspaceID:  "ws-source",
+		ActivityType: "a2a_send",
+		SourceID:     &srcID,
+		Status:       "ok",
+		ErrorDetail:  nil,
+	})
+
+	if len(cb.calls) != 1 {
+		t.Fatalf("expected 1 broadcast, got %d", len(cb.calls))
+	}
+	if _, present := cb.calls[0].payload["error_detail"]; present {
+		t.Errorf("error_detail should be omitted when nil, got %v", cb.calls[0].payload["error_detail"])
+	}
+}
+
+// TestSanitizeErrorDetail_StripsSecretShapes pins the secret-safe
+// scrubber's contract: the broadcast layer is the last defense before
+// a string crosses the canvas WebSocket and lands in the user's
+// browser, so anything that *looks* like an API key / bearer token /
+// JWT must be replaced with [REDACTED] even if upstream (the runtime,
+// the provider) failed to scrub it. The non-secret parts of the
+// message — provider status, error code, human-readable cause — MUST
+// survive intact, otherwise the whole point of internal#212 (giving
+// the user an actionable reason) is defeated.
+func TestSanitizeErrorDetail_StripsSecretShapes(t *testing.T) {
+	cases := []struct {
+		name     string
+		in       string
+		mustHave []string // substrings that must survive — the actionable parts
+		mustMiss []string // substrings that must NOT survive — the secret shapes
+	}{
+		{
+			name:     "preserves actionable provider reason",
+			in:       "Anthropic 403 oauth_org_not_allowed: Your organization has disabled Claude subscription access for Claude Code",
+			mustHave: []string{"403", "oauth_org_not_allowed", "disabled Claude subscription"},
+			mustMiss: []string{"[REDACTED]"},
+		},
+		{
+			name:     "redacts sk- API key embedded in error",
+			in:       "openai 401 invalid_api_key: Incorrect API key provided: sk-proj-abcdefghijklmnop1234567890abcdef. Check your key.",
+			mustHave: []string{"401", "invalid_api_key", "Incorrect API key provided"},
+			mustMiss: []string{"sk-proj-abcdefghijklmnop1234567890abcdef"},
+		},
+		{
+			name:     "redacts Bearer token in stringified header dump",
+			in:       "auth failed; headers: Authorization: Bearer eyJhbGciOiJIUzI1NiJ9.aaaaaaaaaaaaaaaaaaaa.bbbbbbbbbbbbbbbbbbbb",
+			mustHave: []string{"auth failed"},
+			mustMiss: []string{"eyJhbGciOiJIUzI1NiJ9.aaaaaaaaaaaaaaaaaaaa.bbbbbbbbbbbbbbbbbbbb"},
+		},
+		{
+			name:     "truncates absurdly long detail to bound payload size",
+			in:       "kimi 500 internal_error: " + strings.Repeat("x", 8000),
+			mustHave: []string{"kimi 500 internal_error"},
+			mustMiss: []string{strings.Repeat("x", 5000)}, // 5000 in a row must NOT survive — cap is 4096
+		},
+	}
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			got := sanitizeErrorDetailForBroadcast(tc.in)
+			for _, s := range tc.mustHave {
+				if !strings.Contains(got, s) {
+					t.Errorf("expected %q to survive scrub, got: %q", s, got)
+				}
+			}
+			for _, s := range tc.mustMiss {
+				if strings.Contains(got, s) {
+					t.Errorf("expected %q to be scrubbed, got: %q", s, got)
+				}
+			}
+		})
+	}
+}
+
+// TestLogActivity_Broadcast_ErrorDetailIsSanitized pins the integration
+// of the scrubber into the broadcast path: if an upstream caller
+// somehow passes through an error_detail with a secret-shaped token,
+// the wire payload (what reaches the canvas WebSocket) must already
+// be scrubbed. Defense in depth — the runtime should never let this
+// happen, but the canvas is the trust boundary, not the runtime.
+func TestLogActivity_Broadcast_ErrorDetailIsSanitized(t *testing.T) {
+	mock := setupTestDB(t)
+	defer mock.ExpectationsWereMet()
+
+	mock.ExpectExec("INSERT INTO activity_logs").
+		WillReturnResult(sqlmock.NewResult(1, 1))
+
+	cb := &recordingBroadcaster{}
+	srcID := "ws-source"
+	// Upstream leaked a token into the detail string. The DB still
+	// stores the unscrubbed copy (workspace logs are an internal
+	// audit surface), but the broadcast that reaches the canvas
+	// must already be sanitized.
+	detail := "anthropic 401 invalid_api_key: provided key sk-proj-leakedsecretvalueabcdefghij is wrong"
+
+	LogActivity(context.Background(), cb, ActivityParams{
+		WorkspaceID:  "ws-source",
+		ActivityType: "a2a_receive",
+		SourceID:     &srcID,
+		Status:       "error",
+		ErrorDetail:  &detail,
+	})
+
+	if len(cb.calls) != 1 {
+		t.Fatalf("expected 1 broadcast, got %d", len(cb.calls))
+	}
+	got, _ := cb.calls[0].payload["error_detail"].(string)
+	if strings.Contains(got, "sk-proj-leakedsecretvalueabcdefghij") {
+		t.Errorf("broadcast leaked secret-shaped token: %q", got)
+	}
+	if !strings.Contains(got, "invalid_api_key") {
+		t.Errorf("scrubber over-redacted: lost the actionable code from %q", got)
+	}
+}
+
 // TestLogActivityTx_DefersBroadcastUntilCommitHook pins the #149
 // contract: LogActivityTx returns a commitHook that the caller MUST
 // invoke after tx.Commit(); the broadcast MUST NOT fire from inside

workspace-server/internal/handlers/restart_context.go

@@ -180,6 +180,42 @@ func waitForWorkspaceOnline(ctx context.Context, workspaceID string, timeout tim
 	return false
 }
 
+// waitForFreshHeartbeat polls until the workspace has BOTH a non-empty
+// url AND a last_heartbeat_at strictly after restartStartTs (i.e. the
+// heartbeat we observe is NEW, not the stale pre-restart one carried
+// across through the row update). Returns false on timeout or DB error.
+//
+// This is the Layer 2 gate for the 2026-05-19 ws-server self-fire restart
+// loop fix. status='online' can flip while url='' is still in place (the
+// status update happens in /registry/register; url is set at the same
+// time but the read here may see a transient interleaving) and pre-fix
+// the trailing restart-context probe could fire against a half-registered
+// row, triggering the upstream-502 → maybeMarkContainerDead → self-fire
+// chain we're closing. The url + heartbeat-freshness check is the
+// strict, correlated end-state assertion that says "the new container is
+// actually addressable" — not just "some heartbeat happened".
+func waitForFreshHeartbeat(ctx context.Context, workspaceID string, restartStartTs time.Time, timeout time.Duration) bool {
+	deadline := time.Now().Add(timeout)
+	for time.Now().Before(deadline) {
+		var url sql.NullString
+		var lastHB sql.NullTime
+		err := db.DB.QueryRowContext(ctx,
+			`SELECT url, last_heartbeat_at FROM workspaces WHERE id = $1`, workspaceID,
+		).Scan(&url, &lastHB)
+		if err == nil &&
+			url.Valid && url.String != "" &&
+			lastHB.Valid && lastHB.Time.After(restartStartTs) {
+			return true
+		}
+		select {
+		case <-ctx.Done():
+			return false
+		case <-time.After(restartContextOnlinePollInterval):
+		}
+	}
+	return false
+}
+
 // buildRestartA2APayload wraps the rendered context string in the
 // JSON-RPC 2.0 / A2A message/send shape that the proxy already knows
 // how to normalize. Returns the marshalled body ready for ProxyA2ARequest.
@@ -220,6 +256,22 @@ func (h *WorkspaceHandler) sendRestartContext(workspaceID string, data restartCo
 		log.Printf("restart-context: workspace %s did not come online within %s — dropping context message", workspaceID, restartContextOnlineTimeout)
 		return
 	}
+	// Self-fire guard (Layer 2 of the 2026-05-19 ws-server self-fire fix):
+	// status='online' alone is not enough to safely fire the trailing
+	// ProxyA2ARequest. The workspace must also have:
+	//   - url != ''                            (the new container's URL has been registered)
+	//   - last_heartbeat_at > data.RestartAt   (the heartbeat we're seeing is NEW, not stale)
+	// Without those, ProxyA2ARequest can fail with a connect error or
+	// upstream 502, hit handleA2ADispatchError → maybeMarkContainerDead →
+	// RestartByID → self-fire. The Layer 1 isRestarting gate already
+	// covers that, but this is a belt-and-suspenders so the probe never
+	// even tries until the new container is actually addressable. Best-
+	// effort: if the DB read errors out we proceed (preserves the legacy
+	// behaviour of "online means online").
+	if !waitForFreshHeartbeat(ctx, workspaceID, data.RestartAt, restartContextOnlineTimeout) {
+		log.Printf("restart-context: workspace %s online but no fresh heartbeat or empty url — dropping context message (self-fire guard)", workspaceID)
+		return
+	}
 
 	text := buildRestartContextMessage(data)
 	body, err := buildRestartA2APayload(text)

workspace-server/internal/handlers/workspace_provision_forbidden_env.go

@@ -0,0 +1,176 @@
+package handlers
+
+// workspace_provision_forbidden_env.go — Layer 1 of the RFC#523
+// tenant-workspace forbidden-env guardrail (task #146).
+//
+// Threat model: tenant workspaces (per-customer EC2 / container)
+// run untrusted agent-controlled code and MUST NEVER receive
+// operator-fleet-scope credentials. A leak from one tenant
+// workspace to operator scope would escalate "compromise of one
+// agent" into "compromise of the whole platform."
+//
+// The existing forensic #145 guard (provisioner.scmWriteTokenKeys
+// in buildContainerEnv / CPProvisioner.Start) strips SCM-write
+// tokens at the FINAL container-env-build step — silent strip,
+// no signal back to the caller. RFC#523 adds a FAIL-CLOSED layer
+// EARLIER in the provision pipeline: when the resolved env-set
+// at prepareProvisionContext-time contains any forbidden var
+// name, the provision is aborted with a structured error so the
+// operator sees the leak immediately instead of running with a
+// silently-stripped env.
+//
+// Layer placement (3-layer defense-in-depth, RFC#523 §"Proposed guardrail"):
+//   - L1 (this file): provisioner-side abort BEFORE container start
+//   - L2 (workspace/entrypoint.sh + template-* start.sh): in-container
+//     env-grep + exit 1 — defense-in-depth if L1 is bypassed
+//   - L3 (.gitea/workflows/lint-forbidden-env-keys.yml): CI lint that
+//     scans Go code under workspace-server/ for new writers that
+//     would inject a forbidden key
+//
+// Open-source-template compatibility (memory
+// `feedback_open_source_templates_no_hardcoded_org_internals`):
+// the forbidden-key set is GENERIC (no molecule-AI-specific
+// hostnames or org names). A third-party fork can replace this
+// set with its own operator-scope key names without editing any
+// template.
+
+import (
+	"fmt"
+	"sort"
+	"strings"
+)
+
+// forbiddenTenantEnvKeys is the set of environment variable names
+// that MUST NOT reach a tenant workspace container. The check is
+// by exact key name — value-shape leaks (40-byte hex strings, etc)
+// are out of scope here; the separate secret-scan workflow covers
+// that class.
+//
+// Categories (RFC#523):
+//   - SCM-write tokens: same as provisioner.scmWriteTokenKeys, kept
+//     in sync. Listed again here so a future split of the two
+//     denylists is auditable diff.
+//   - Control-plane admin tokens: any token that grants control-plane
+//     admin API access.
+//   - Secret-store operator tokens: bootstrap-scope tokens for the
+//     central secret store.
+//   - Infra-platform tokens: deploy / fleet-management creds.
+//   - Operator-host pointers: hostnames / addresses that identify
+//     the operator host. Per the open-source-template rule these
+//     are MOLECULE_OPERATOR_HOST style prefixes; the literal
+//     prefix is matched but the test for membership reads from
+//     this map, not from a hardcoded constant in the deny rule
+//     itself.
+//
+// Per-agent persona PATs (e.g. AGENT_DEV_A_TOKEN style names —
+// not operator-fleet scope) are NOT on this list. The guard
+// checks the env VAR NAME, not the token VALUE, so a per-agent
+// scoped token under a per-agent var name passes through.
+var forbiddenTenantEnvKeys = map[string]struct{}{
+	// SCM-write — kept in sync with provisioner.scmWriteTokenKeys.
+	"GITEA_TOKEN":     {},
+	"GITEA_PAT":       {},
+	"GITHUB_TOKEN":    {},
+	"GITHUB_PAT":      {},
+	"GH_TOKEN":        {},
+	"GITLAB_TOKEN":    {},
+	"GL_TOKEN":        {},
+	"BITBUCKET_TOKEN": {},
+
+	// Control-plane admin tokens.
+	"CP_ADMIN_API_TOKEN": {},
+	"CP_ADMIN_TOKEN":     {},
+
+	// Secret-store operator tokens (Infisical SSOT — operator scope only).
+	"INFISICAL_OPERATOR_TOKEN":  {},
+	"INFISICAL_BOOTSTRAP_TOKEN": {},
+
+	// Infra-platform tokens.
+	"RAILWAY_TOKEN":              {},
+	"RAILWAY_PERSONAL_API_TOKEN": {},
+	"HETZNER_TOKEN":              {},
+	"HETZNER_API_TOKEN":          {},
+}
+
+// forbiddenTenantEnvPrefixes are key-name PREFIXES that match
+// operator-scope env vars. Matched in addition to the exact-key
+// set above. Useful for "MOLECULE_OPERATOR_*" style families
+// where new members get added without re-editing the deny set.
+//
+// Kept as a tiny set on purpose — over-broad prefix matching is
+// the failure mode this layer's exact-key set is designed to
+// avoid. Add a prefix here only when the family is closed
+// (every member is operator-scope; no legitimate tenant-scope
+// member exists or will).
+var forbiddenTenantEnvPrefixes = []string{
+	"MOLECULE_OPERATOR_",
+}
+
+// isForbiddenTenantEnvKey reports whether an env var name is on
+// the forbidden-for-tenant-workspaces list (either by exact match
+// in forbiddenTenantEnvKeys or by prefix in
+// forbiddenTenantEnvPrefixes).
+//
+// Exported-style helper kept package-private — the deny set is
+// internal to the workspace-server package; external callers must
+// go through the provision pipeline, which means the abort path
+// fires for them too.
+func isForbiddenTenantEnvKey(key string) bool {
+	if _, ok := forbiddenTenantEnvKeys[key]; ok {
+		return true
+	}
+	for _, prefix := range forbiddenTenantEnvPrefixes {
+		if strings.HasPrefix(key, prefix) {
+			return true
+		}
+	}
+	return false
+}
+
+// findForbiddenTenantEnvKeys scans the resolved env-set and
+// returns the sorted list of forbidden keys present. Empty slice
+// (not nil — easier for callers to JSON-encode) when none match.
+//
+// Deterministic order: the result feeds the user-facing error
+// message and the structured-extra payload that goes to the
+// canvas Events tab. Sorting makes the message stable across
+// Go's randomized map iteration.
+func findForbiddenTenantEnvKeys(envVars map[string]string) []string {
+	if len(envVars) == 0 {
+		return []string{}
+	}
+	found := make([]string, 0)
+	for k := range envVars {
+		if isForbiddenTenantEnvKey(k) {
+			found = append(found, k)
+		}
+	}
+	sort.Strings(found)
+	return found
+}
+
+// formatForbiddenTenantEnvError builds the safe-canned user-facing
+// message for a provision aborted because forbidden env keys are
+// present in the resolved env-set. The message names the
+// offending keys (key names are not secret — the values would be,
+// but only names are surfaced) and points at the RFC.
+//
+// Same shape as formatMissingEnvError so the canvas Events tab
+// renders both classes consistently.
+func formatForbiddenTenantEnvError(keys []string) string {
+	if len(keys) == 0 {
+		// Defensive: caller should not invoke with empty input,
+		// but keep the function total.
+		return "provision aborted: forbidden operator-scope env vars present (RFC#523)"
+	}
+	if len(keys) == 1 {
+		return fmt.Sprintf(
+			"provision aborted: env var %q is operator-scope and must not reach tenant workspaces (RFC#523) — remove it from workspace_secrets / global_secrets and retry",
+			keys[0],
+		)
+	}
+	return fmt.Sprintf(
+		"provision aborted: env vars %s are operator-scope and must not reach tenant workspaces (RFC#523) — remove them from workspace_secrets / global_secrets and retry",
+		strings.Join(keys, ", "),
+	)
+}

workspace-server/internal/handlers/workspace_provision_forbidden_env_test.go

@@ -0,0 +1,182 @@
+package handlers
+
+// workspace_provision_forbidden_env_test.go — Layer 1 tests for the
+// RFC#523 tenant-workspace forbidden-env guardrail (task #146).
+//
+// Behaviour pinned (per RFC#523 §"Acceptance criteria" Layer 1):
+//   - exact-match keys (GITEA_TOKEN, CP_ADMIN_API_TOKEN, RAILWAY_TOKEN,
+//     INFISICAL_OPERATOR_TOKEN, …) are flagged
+//   - MOLECULE_OPERATOR_* prefix family is flagged
+//   - per-agent-scope vars (GIT_HTTP_USERNAME, ANTHROPIC_API_KEY,
+//     AGENT_DEV_A_TOKEN, …) are NOT flagged — guard checks key NAME
+//     not value
+//   - findForbiddenTenantEnvKeys returns a deterministically-sorted
+//     slice (canvas Events tab needs stable rendering)
+//   - formatForbiddenTenantEnvError uses singular vs plural phrasing
+//     so the message reads naturally for both 1-key and N-key cases
+//
+// Companion: provisioner.buildContainerEnv has the older silent-
+// strip guard (forensic #145). The two layers are intentionally
+// redundant — this one fails closed early; that one strips late.
+
+import (
+	"strings"
+	"testing"
+)
+
+func TestIsForbiddenTenantEnvKey_ExactMatches(t *testing.T) {
+	cases := []struct {
+		key  string
+		want bool
+	}{
+		// SCM-write tokens — kept in sync with provisioner.scmWriteTokenKeys.
+		{"GITEA_TOKEN", true},
+		{"GITEA_PAT", true},
+		{"GITHUB_TOKEN", true},
+		{"GITHUB_PAT", true},
+		{"GH_TOKEN", true},
+		{"GITLAB_TOKEN", true},
+		{"GL_TOKEN", true},
+		{"BITBUCKET_TOKEN", true},
+
+		// Control-plane admin tokens.
+		{"CP_ADMIN_API_TOKEN", true},
+		{"CP_ADMIN_TOKEN", true},
+
+		// Secret-store operator tokens.
+		{"INFISICAL_OPERATOR_TOKEN", true},
+		{"INFISICAL_BOOTSTRAP_TOKEN", true},
+
+		// Infra-platform tokens.
+		{"RAILWAY_TOKEN", true},
+		{"RAILWAY_PERSONAL_API_TOKEN", true},
+		{"HETZNER_TOKEN", true},
+		{"HETZNER_API_TOKEN", true},
+
+		// Per-agent scoped — must NOT be flagged.
+		{"GIT_HTTP_USERNAME", false},
+		{"GIT_HTTP_PASSWORD", false},
+		{"ANTHROPIC_API_KEY", false},
+		{"ANTHROPIC_AUTH_TOKEN", false},
+		{"OPENAI_API_KEY", false},
+		{"KIMI_API_KEY", false},
+		{"MINIMAX_API_KEY", false},
+		{"AGENT_DEV_A_TOKEN", false}, // hypothetical per-agent name
+		{"MOLECULE_AGENT_ROLE", false},
+		{"PARENT_ID", false},
+		{"WORKSPACE_ID", false},
+		{"PLATFORM_URL", false},
+		{"", false},
+	}
+	for _, c := range cases {
+		got := isForbiddenTenantEnvKey(c.key)
+		if got != c.want {
+			t.Errorf("isForbiddenTenantEnvKey(%q) = %v; want %v", c.key, got, c.want)
+		}
+	}
+}
+
+func TestIsForbiddenTenantEnvKey_PrefixMatches(t *testing.T) {
+	cases := []struct {
+		key  string
+		want bool
+	}{
+		{"MOLECULE_OPERATOR_HOST", true},
+		{"MOLECULE_OPERATOR_SSH_KEY", true},
+		{"MOLECULE_OPERATOR_BACKUP_BUCKET", true},
+		{"MOLECULE_OPERATOR_", true}, // prefix itself
+
+		// Adjacent but NOT in prefix family.
+		{"MOLECULE_AGENT_ROLE", false},
+		{"MOLECULE_URL", false},
+		{"MOLECULE_PERSONA_ROOT", false}, // path on operator host, not tenant
+		{"MOLECULE_GITEA_TOKEN", false},  // localbuild-time only; not a tenant env
+	}
+	for _, c := range cases {
+		got := isForbiddenTenantEnvKey(c.key)
+		if got != c.want {
+			t.Errorf("isForbiddenTenantEnvKey(%q) = %v; want %v", c.key, got, c.want)
+		}
+	}
+}
+
+func TestFindForbiddenTenantEnvKeys_NoneAndEmpty(t *testing.T) {
+	if got := findForbiddenTenantEnvKeys(nil); len(got) != 0 {
+		t.Errorf("nil envVars: got %v; want empty", got)
+	}
+	if got := findForbiddenTenantEnvKeys(map[string]string{}); len(got) != 0 {
+		t.Errorf("empty envVars: got %v; want empty", got)
+	}
+	clean := map[string]string{
+		"ANTHROPIC_API_KEY":   "sk-keep",
+		"GIT_HTTP_USERNAME":   "agent-dev-a",
+		"GIT_HTTP_PASSWORD":   "scoped-pat",
+		"MOLECULE_AGENT_ROLE": "agent-dev-a",
+		"WORKSPACE_ID":        "ws-123",
+	}
+	if got := findForbiddenTenantEnvKeys(clean); len(got) != 0 {
+		t.Errorf("clean envVars: got %v; want empty", got)
+	}
+}
+
+func TestFindForbiddenTenantEnvKeys_SingleAndMultipleSorted(t *testing.T) {
+	// Single key.
+	single := map[string]string{
+		"ANTHROPIC_API_KEY": "sk-keep",
+		"GITEA_TOKEN":       "operator-scope-leak",
+	}
+	got := findForbiddenTenantEnvKeys(single)
+	if len(got) != 1 || got[0] != "GITEA_TOKEN" {
+		t.Errorf("single forbidden: got %v; want [GITEA_TOKEN]", got)
+	}
+
+	// Multiple keys — must be sorted (canvas Events tab needs stability).
+	multi := map[string]string{
+		"RAILWAY_TOKEN":          "z",
+		"GITEA_TOKEN":            "a",
+		"MOLECULE_OPERATOR_HOST": "m",
+		"CP_ADMIN_API_TOKEN":     "c",
+		"ANTHROPIC_API_KEY":      "ok",
+	}
+	got = findForbiddenTenantEnvKeys(multi)
+	want := []string{"CP_ADMIN_API_TOKEN", "GITEA_TOKEN", "MOLECULE_OPERATOR_HOST", "RAILWAY_TOKEN"}
+	if len(got) != len(want) {
+		t.Fatalf("multi forbidden length: got %v; want %v", got, want)
+	}
+	for i := range want {
+		if got[i] != want[i] {
+			t.Errorf("multi forbidden[%d] = %q; want %q (full got=%v want=%v)", i, got[i], want[i], got, want)
+		}
+	}
+}
+
+func TestFormatForbiddenTenantEnvError_Phrasing(t *testing.T) {
+	// Empty input — defensive total function.
+	if msg := formatForbiddenTenantEnvError(nil); !strings.Contains(msg, "RFC#523") {
+		t.Errorf("empty input: missing RFC#523 ref: %q", msg)
+	}
+
+	// Singular phrasing.
+	single := formatForbiddenTenantEnvError([]string{"GITEA_TOKEN"})
+	if !strings.Contains(single, `"GITEA_TOKEN"`) {
+		t.Errorf("single: missing quoted key: %q", single)
+	}
+	if !strings.Contains(single, "operator-scope") {
+		t.Errorf("single: missing operator-scope phrase: %q", single)
+	}
+	if !strings.Contains(single, "RFC#523") {
+		t.Errorf("single: missing RFC#523 ref: %q", single)
+	}
+	if strings.Contains(single, "env vars ") { // plural form
+		t.Errorf("single: leaked plural phrasing: %q", single)
+	}
+
+	// Plural phrasing.
+	multi := formatForbiddenTenantEnvError([]string{"CP_ADMIN_API_TOKEN", "GITEA_TOKEN"})
+	if !strings.Contains(multi, "CP_ADMIN_API_TOKEN, GITEA_TOKEN") {
+		t.Errorf("plural: missing joined list: %q", multi)
+	}
+	if !strings.Contains(multi, "env vars ") {
+		t.Errorf("plural: missing plural phrase: %q", multi)
+	}
+}

workspace-server/internal/handlers/workspace_provision_shared.go

@@ -125,6 +125,36 @@ func (h *WorkspaceHandler) prepareProvisionContext(
 		return nil, &provisionAbort{Msg: decryptErr}
 	}
 
+	// RFC#523 Layer 1 (task #146): refuse to start a tenant workspace
+	// when any forbidden operator-scope env var is present in the
+	// resolved secret-load env-set. Runs IMMEDIATELY after
+	// loadWorkspaceSecrets and BEFORE applyAgentGitHTTPCreds — the
+	// per-agent persona injection sets a fallback GITEA_USER/GITEA_TOKEN
+	// pair that the buildContainerEnv forensic #145 guard will strip
+	// later. We want THIS layer to catch leaks from the operator-
+	// controlled stores (global_secrets, workspace_secrets) only, not
+	// the deliberate per-agent platform injection that lives downstream.
+	//
+	// Threat model is "an upstream secret-writer accidentally widened
+	// the propagation set" — e.g. an operator pastes GITEA_TOKEN into
+	// a workspace_secrets row. Caught here, surfaced loudly to the
+	// canvas Events tab, fail-closed. The existing forensic #145 guard
+	// in provisioner.buildContainerEnv / CPProvisioner.Start stays as
+	// defense-in-depth: it silently strips at container-env-build time.
+	//
+	// Key names (not values) are echoed in the user-facing error so
+	// the operator can locate and remove the offending row. Per memory
+	// `feedback_passwords_in_chat_are_burned`, key names are not
+	// secret; values would be.
+	if forbidden := findForbiddenTenantEnvKeys(envVars); len(forbidden) > 0 {
+		msg := formatForbiddenTenantEnvError(forbidden)
+		log.Printf("Provisioner: ABORT workspace=%s — forbidden operator-scope env keys present: %v (RFC#523)", workspaceID, forbidden)
+		return nil, &provisionAbort{
+			Msg:   msg,
+			Extra: map[string]interface{}{"error": msg, "forbidden_env_keys": forbidden, "rfc": "523"},
+		}
+	}
+
 	pluginsPath, _ := filepath.Abs(filepath.Join(h.configsDir, "..", "plugins"))
 	awarenessNamespace := h.loadAwarenessNamespace(ctx, workspaceID)
 

workspace-server/internal/handlers/workspace_restart.go

@@ -8,6 +8,7 @@ import (
 	"runtime/debug"
 	"strings"
 	"sync"
+	"sync/atomic"
 	"time"
 
 	"github.com/Molecule-AI/molecule-monorepo/platform/internal/db"
@@ -39,12 +40,57 @@ type restartState struct {
 	mu      sync.Mutex
 	running bool // true while a restart cycle is in flight
 	pending bool // set by any caller that arrived during the in-flight cycle
+	// restartStartedAt records the wall-clock when the most recent cycle
+	// flipped running=true. Used by the self-fire debounce (internal#544,
+	// the ws-server self-fire restart feedback loop seen in prod-Reviewer/
+	// Researcher 2026-05-19 ~00:05Z 4x reprov thrash): any RestartByID
+	// arriving within restartDebounceWindow of this timestamp is silently
+	// dropped so a probe firing during the EC2-pending window can't
+	// re-trigger a fresh full cycle on the just-launched instance.
+	restartStartedAt time.Time
 }
 
 // restartStates is a per-workspace map of *restartState. Each workspace gets
 // its own entry so unrelated workspaces don't serialize on each other.
 var restartStates sync.Map // map[workspaceID]*restartState
 
+// restartDebounceWindow is the silent-drop window for successive RestartByID
+// calls. Sized to cover the typical EC2 pending → online interval (20-30s)
+// with a margin so a probe firing during the just-after-online but still-
+// flaky heartbeat window also gets dropped. Bigger than that would block
+// legitimate "Restart failed, retry" recoveries; smaller would let the
+// 4x thrash class through. Package-level so tests can shrink it.
+var restartDebounceWindow = 60 * time.Second
+
+// restartByIDDropCounter is incremented every time RestartByID drops a call
+// inside the debounce window. Exposed as a package-level atomic counter so
+// (a) tests can assert the drop fired, (b) ops can grep logs for the drop
+// log line + the counter snapshot in a future /admin/metrics endpoint.
+// Not a Prometheus metric because the platform doesn't pull metrics from
+// workspace-server yet — that's a separate RFC.
+var restartByIDDropCounter atomic.Uint64
+
+// isRestarting reports whether a restart cycle is currently in flight for
+// the workspace. Callers that have their own "container looks dead" probe
+// MUST consult this before triggering a restart, because during the
+// 20-30s EC2-pending window the workspace's url='' and IsRunning()=false
+// looks identical to a dead container — and any restart-triggering probe
+// (maybeMarkContainerDead from canvas /delegations poll, or the trailing
+// restart-context probe at the end of runRestartCycle) will set
+// pending=true and the outer coalesceRestart loop will drain by running
+// ANOTHER full cycle, ec2_stopped of the just-booted instance →
+// re-provision. That's the self-fire loop closed by this gate.
+func isRestarting(workspaceID string) bool {
+	sv, ok := restartStates.Load(workspaceID)
+	if !ok {
+		return false
+	}
+	state := sv.(*restartState)
+	state.mu.Lock()
+	defer state.mu.Unlock()
+	return state.running
+}
+
 // isParentPaused checks if any ancestor of the workspace is paused.
 func isParentPaused(ctx context.Context, workspaceID string) (bool, string) {
 	var parentID *string
@@ -376,9 +422,45 @@ func (h *WorkspaceHandler) RestartByID(workspaceID string) {
 	if !h.HasProvisioner() {
 		return
 	}
+	// Self-fire debounce: drop (not coalesce) successive RestartByID calls
+	// within restartDebounceWindow of the most recent cycle's start. This
+	// is the load-bearing protection against the 4x reprov thrash class —
+	// coalesceRestart's pending-flag would otherwise drain by running
+	// ANOTHER full cycle of stop+provision on the just-launched EC2 (still
+	// in the pending state), which is the self-fire we're closing.
+	//
+	// Only applies to RestartByID (programmatic — secrets handler,
+	// maybeMarkContainerDead, preflightContainerHealth). The HTTP Restart
+	// handler in workspace_restart.go's Restart() bypasses this path and
+	// calls RestartWorkspaceAutoOpts directly, so user-initiated restart
+	// clicks are unaffected.
+	if shouldDebounceRestart(workspaceID) {
+		restartByIDDropCounter.Add(1)
+		log.Printf("RestartByID: %s — dropped (within %s self-fire debounce window; total dropped=%d)",
+			workspaceID, restartDebounceWindow, restartByIDDropCounter.Load())
+		return
+	}
 	coalesceRestart(workspaceID, func() { h.runRestartCycle(workspaceID) })
 }
 
+// shouldDebounceRestart reports whether the most recent cycle for this
+// workspace started within restartDebounceWindow. Read-only on
+// restartState; the actual restartStartedAt stamp is written in
+// coalesceRestart when running flips false→true.
+func shouldDebounceRestart(workspaceID string) bool {
+	sv, ok := restartStates.Load(workspaceID)
+	if !ok {
+		return false
+	}
+	state := sv.(*restartState)
+	state.mu.Lock()
+	defer state.mu.Unlock()
+	if state.restartStartedAt.IsZero() {
+		return false
+	}
+	return time.Since(state.restartStartedAt) < restartDebounceWindow
+}
+
 // coalesceRestart implements the pending-flag gate around an arbitrary cycle
 // function. Extracted from RestartByID for direct unit testing — the cycle
 // function in production is `runRestartCycle`, but tests pass a counter to
@@ -398,6 +480,12 @@ func coalesceRestart(workspaceID string, cycle func()) {
 		return
 	}
 	state.running = true
+	// Stamp the start time so the RestartByID debounce can drop any
+	// self-fire probe that hits within restartDebounceWindow. Only the
+	// false→true edge stamps; the drain-loop's inner cycles re-use the
+	// same start (they're effectively one "restart event" from the
+	// debounce's POV).
+	state.restartStartedAt = time.Now()
 	state.mu.Unlock()
 
 	// Always clear running on exit — including panic — so a panicking

workspace-server/internal/handlers/workspace_restart_self_fire_test.go

@@ -0,0 +1,297 @@
+package handlers
+
+// Tests for the 2026-05-19 ws-server self-fire restart feedback loop fix.
+//
+// Empirical chain reproduced (prod-Reviewer/Researcher 4x reprov thrash
+// 2026-05-19 ~00:05-00:09Z, root-caused via Loki):
+//
+//  1. POST /secrets → go h.restartFunc(workspaceID) (secrets.go:264).
+//  2. runRestartCycle sets url='' synchronously, then async provisions EC2
+//     (workspace_restart.go).
+//  3. During 20-30s window while EC2 is `pending` (codex first heartbeat
+//     not yet landed): workspaces.url='' AND IsRunning=false.
+//  4. Any ProxyA2A (canvas /delegations poll OR the restart-context probe
+//     at the end of runRestartCycle) → maybeMarkContainerDead sees the
+//     container-dead state → calls RestartByID → loop.
+//  5. coalesceRestart sets pending=true, drains by running ANOTHER full
+//     cycle → provision.ec2_stopped of the just-booted instance →
+//     re-provision.
+//
+// Fix: three interdependent layers.
+//
+//  L1) isRestarting() gate in maybeMarkContainerDead +
+//      preflightContainerHealth — early-return false/nil so the probe
+//      can't trigger a fresh RestartByID while a restart is in flight.
+//  L2) sendRestartContext requires url != '' AND last_heartbeat_at >
+//      restart_start_ts before firing the trailing ProxyA2A probe.
+//  L3) RestartByID silently drops successive calls within
+//      restartDebounceWindow of restartStartedAt, with a counter for
+//      observability.
+
+import (
+	"context"
+	"sync/atomic"
+	"testing"
+	"time"
+
+	"github.com/DATA-DOG/go-sqlmock"
+)
+
+// resetSelfFireState wipes all the per-workspace mutation state these
+// tests touch, plus the package-level drop counter, so the test is
+// hermetic regardless of ordering.
+func resetSelfFireState(workspaceID string) {
+	restartStates.Delete(workspaceID)
+	restartByIDDropCounter.Store(0)
+}
+
+// markRestarting forces restartStates into "cycle in flight" without
+// running an actual cycle, so the tests can isolate the gate behaviour
+// without the full provision pipeline. Returns a finish() that flips
+// running=false (mimicking coalesceRestart's deferred state-clear).
+func markRestarting(workspaceID string) (finish func()) {
+	sv, _ := restartStates.LoadOrStore(workspaceID, &restartState{})
+	state := sv.(*restartState)
+	state.mu.Lock()
+	state.running = true
+	state.restartStartedAt = time.Now()
+	state.mu.Unlock()
+	return func() {
+		state.mu.Lock()
+		state.running = false
+		state.mu.Unlock()
+	}
+}
+
+// TestIsRestarting_FalseWhenNoStateEntry — baseline: a workspace that
+// has never been restarted reports !isRestarting. Pinning this so a
+// future LoadOrStore refactor can't silently start returning true for
+// unknown workspaces.
+func TestIsRestarting_FalseWhenNoStateEntry(t *testing.T) {
+	const wsID = "self-fire-ws-never"
+	resetSelfFireState(wsID)
+	if isRestarting(wsID) {
+		t.Fatal("isRestarting must return false for a workspace with no state entry")
+	}
+}
+
+// TestIsRestarting_TrueWhileCycleRunning — the load-bearing invariant
+// that Layer 1 depends on. While running=true, isRestarting must report
+// true; the moment it flips to false, isRestarting must report false.
+func TestIsRestarting_TrueWhileCycleRunning(t *testing.T) {
+	const wsID = "self-fire-ws-in-flight"
+	resetSelfFireState(wsID)
+
+	finish := markRestarting(wsID)
+	if !isRestarting(wsID) {
+		t.Fatal("isRestarting must return true while running=true")
+	}
+	finish()
+	if isRestarting(wsID) {
+		t.Fatal("isRestarting must return false after running flips back to false")
+	}
+}
+
+// TestMaybeMarkContainerDead_SkippedWhileRestarting — Layer 1 for the
+// reactive path. With isRestarting=true the function must early-return
+// false WITHOUT invoking IsRunning, hitting the DB UPDATE, or kicking
+// a RestartByID goroutine. If any of those side-effects fire we'd
+// re-arm the self-fire loop the gate exists to close.
+func TestMaybeMarkContainerDead_SkippedWhileRestarting(t *testing.T) {
+	const wsID = "self-fire-ws-mmcd"
+	resetSelfFireState(wsID)
+	mock := setupTestDB(t) // sqlmock with strict expectation matching
+
+	// Workspace row read inside maybeMarkContainerDead — this happens
+	// BEFORE the isRestarting gate in the current implementation, so
+	// allow exactly one SELECT runtime row.
+	mock.ExpectQuery(`SELECT COALESCE\(runtime, 'langgraph'\) FROM workspaces WHERE id =`).
+		WithArgs(wsID).
+		WillReturnRows(sqlmock.NewRows([]string{"runtime"}).AddRow("claude-code"))
+
+	// Gate flipped: must early-return without doing anything else.
+	finish := markRestarting(wsID)
+	defer finish()
+
+	stub := &preflightLocalProv{running: false, err: nil}
+	h := newSelfFireHandler(t)
+	h.provisioner = stub
+
+	if got := h.maybeMarkContainerDead(context.Background(), wsID); got != false {
+		t.Errorf("maybeMarkContainerDead must return false while restarting, got %v", got)
+	}
+	if stub.calls != 0 {
+		t.Errorf("IsRunning must not be called while restarting (Layer 1 gate broken); got %d calls", stub.calls)
+	}
+}
+
+// TestPreflightContainerHealth_SkippedWhileRestarting — Layer 1 for the
+// proactive path. Same shape as above: with restart in flight, return
+// nil (let the optimistic forward proceed) and DO NOT call IsRunning.
+// The forward will fail with a connect error; the post-restart reactive
+// path can decide what to do then, by which point the EC2 has either
+// come up (no more failures) or markProvisionFailed has fired.
+func TestPreflightContainerHealth_SkippedWhileRestarting(t *testing.T) {
+	const wsID = "self-fire-ws-preflight"
+	resetSelfFireState(wsID)
+	_ = setupTestDB(t)
+
+	finish := markRestarting(wsID)
+	defer finish()
+
+	stub := &preflightLocalProv{running: false, err: nil}
+	h := newSelfFireHandler(t)
+	h.provisioner = stub
+
+	if err := h.preflightContainerHealth(context.Background(), wsID); err != nil {
+		t.Errorf("preflightContainerHealth must return nil while restarting, got %+v", err)
+	}
+	if stub.calls != 0 {
+		t.Errorf("IsRunning must not be called while restarting (Layer 1 gate broken); got %d calls", stub.calls)
+	}
+}
+
+// TestRestartByID_DebounceSilentDrop — Layer 3. After a cycle starts,
+// any RestartByID arriving within restartDebounceWindow MUST be dropped
+// silently — not coalesced (which would still drain to another cycle).
+// The drop counter must increment by exactly one per dropped call so
+// ops can see how often the self-fire would have fired pre-fix.
+func TestRestartByID_DebounceSilentDrop(t *testing.T) {
+	const wsID = "self-fire-ws-debounce"
+	resetSelfFireState(wsID)
+
+	// Stamp restartStartedAt = now, running=false (simulates the "just
+	// finished" window where the loop would re-fire pre-fix).
+	sv, _ := restartStates.LoadOrStore(wsID, &restartState{})
+	state := sv.(*restartState)
+	state.mu.Lock()
+	state.restartStartedAt = time.Now()
+	state.running = false
+	state.mu.Unlock()
+
+	// Counter baseline.
+	if got := restartByIDDropCounter.Load(); got != 0 {
+		t.Fatalf("expected drop counter 0 at start, got %d", got)
+	}
+
+	// Five rapid-fire RestartByID calls should all drop (the maximum
+	// observed pre-fix was 4x — pinning >=4 here keeps the regression
+	// shape true to the prod incident).
+	h := newSelfFireHandler(t)
+	stub := &preflightLocalProv{running: true, err: nil}
+	h.provisioner = stub
+	for i := 0; i < 5; i++ {
+		h.RestartByID(wsID)
+	}
+
+	if got := restartByIDDropCounter.Load(); got != 5 {
+		t.Errorf("expected 5 drops within debounce window, got %d", got)
+	}
+
+	// shouldDebounceRestart itself must report true for the same window.
+	if !shouldDebounceRestart(wsID) {
+		t.Error("shouldDebounceRestart must return true within window")
+	}
+}
+
+// TestRestartByID_DebounceExpiresAfterWindow — outside the window, the
+// debounce must release: a legitimate later restart (e.g. user clicked
+// Restart again after waiting) must proceed to coalesceRestart. We
+// shrink restartDebounceWindow to 1ms for the duration of this test so
+// we don't sleep a full 60s in CI.
+func TestRestartByID_DebounceExpiresAfterWindow(t *testing.T) {
+	const wsID = "self-fire-ws-debounce-release"
+	resetSelfFireState(wsID)
+
+	orig := restartDebounceWindow
+	restartDebounceWindow = 5 * time.Millisecond
+	defer func() { restartDebounceWindow = orig }()
+
+	// Stamp inside the window.
+	sv, _ := restartStates.LoadOrStore(wsID, &restartState{})
+	state := sv.(*restartState)
+	state.mu.Lock()
+	state.restartStartedAt = time.Now()
+	state.running = false
+	state.mu.Unlock()
+
+	if !shouldDebounceRestart(wsID) {
+		t.Fatal("within 5ms window must debounce")
+	}
+
+	// Sleep past the window. Use a small margin to avoid clock-skew
+	// flakes on slow CI hosts.
+	time.Sleep(20 * time.Millisecond)
+
+	if shouldDebounceRestart(wsID) {
+		t.Fatal("after 20ms (4x window) must no longer debounce")
+	}
+}
+
+// TestRestartByID_SingleProvisionPerRestart — the regression test for
+// the prod incident: a SINGLE secrets PUT (which is the trigger shape)
+// must produce exactly ONE coalesceRestart cycle, not four. Models the
+// full chain: secrets handler → RestartByID → coalesceRestart → cycle
+// runs → during the cycle window, simulated probes call RestartByID
+// again. With all three layers in place, the probes are dropped and the
+// total cycle count stays at 1.
+func TestRestartByID_SingleProvisionPerRestart(t *testing.T) {
+	const wsID = "self-fire-ws-single-provision"
+	resetSelfFireState(wsID)
+
+	// In-flight gate that mimics the EC2-pending window. The cycle
+	// blocks on cycleProceed so we can fire the simulated probes while
+	// running=true.
+	var cycleCount atomic.Int32
+	cycleStarted := make(chan struct{}, 1)
+	cycleProceed := make(chan struct{})
+
+	cycle := func() {
+		n := cycleCount.Add(1)
+		if n == 1 {
+			cycleStarted <- struct{}{}
+			<-cycleProceed
+		}
+	}
+
+	// Kick the first cycle via coalesceRestart (this is what RestartByID
+	// would do post-debounce-check).
+	done := make(chan struct{})
+	go func() {
+		coalesceRestart(wsID, cycle)
+		close(done)
+	}()
+	<-cycleStarted
+
+	// Simulate the 4 probe-driven RestartByID calls observed in prod.
+	// Each must drop because we're within the debounce window AND a
+	// cycle is in flight.
+	h := newSelfFireHandler(t)
+	stub := &preflightLocalProv{running: true, err: nil}
+	h.provisioner = stub
+	for i := 0; i < 4; i++ {
+		h.RestartByID(wsID)
+	}
+
+	// Release the cycle.
+	close(cycleProceed)
+	<-done
+
+	if got := cycleCount.Load(); got != 1 {
+		t.Errorf("expected exactly 1 provision cycle for a single trigger "+
+			"(self-fire fix), got %d — regression of the prod 4x reprov thrash class",
+			got)
+	}
+	if got := restartByIDDropCounter.Load(); got != 4 {
+		t.Errorf("expected 4 self-fire probes dropped, got %d "+
+			"(observability counter must record the saved cycles)", got)
+	}
+}
+
+// newSelfFireHandler constructs a minimal *WorkspaceHandler suitable for
+// the Layer-1 gate tests. Wraps the boilerplate so the per-test setup
+// stays focused on the assertion.
+func newSelfFireHandler(t *testing.T) *WorkspaceHandler {
+	t.Helper()
+	return NewWorkspaceHandler(newTestBroadcaster(), nil, "http://localhost:8080", t.TempDir())
+}

workspace/entrypoint.sh

@@ -9,6 +9,59 @@
 # Pattern matches the legacy monorepo workspace/entrypoint.sh:
 # fix volume ownership as root, then re-exec via gosu as agent (uid 1000).
 
+# --- RFC#523 Layer 2: tenant-workspace forbidden-env guard (task #146) ---
+# Defense-in-depth. The provisioner (workspace-server) has a fail-closed
+# abort at provision time (Layer 1, prepareProvisionContext), and the
+# in-container env-build has a silent strip (forensic #145,
+# provisioner.buildContainerEnv). This guard fires if either upstream
+# layer is bypassed — e.g. someone runs this image standalone with
+# `docker run -e GITEA_TOKEN=...`. Exit 1 with a clear message instead
+# of running with an operator-scope credential in tenant scope.
+#
+# Key names are generic. The MOLECULE_OPERATOR_ prefix is the one
+# molecule-AI-specific literal; this entrypoint lives inside the
+# claude-code template that is internal-only (memory
+# `feedback_open_source_templates_no_hardcoded_org_internals` — claude-
+# code template is internal, separate-published templates must NOT carry
+# org-specific literals). A fork can edit FORBIDDEN_KEYS /
+# FORBIDDEN_PREFIXES for its own operator-scope names without touching
+# the rest of the entrypoint.
+#
+# Skipped when MOLECULE_TENANT_GUARD_DISABLE=1 — for local-dev where the
+# operator host IS the tenant host (e.g. running molecule-runtime on the
+# operator box for debugging). NEVER set this in tenant containers.
+if [ "${MOLECULE_TENANT_GUARD_DISABLE:-0}" != "1" ]; then
+    FORBIDDEN_KEYS="GITEA_TOKEN GITEA_PAT GITHUB_TOKEN GITHUB_PAT GH_TOKEN GITLAB_TOKEN GL_TOKEN BITBUCKET_TOKEN CP_ADMIN_API_TOKEN CP_ADMIN_TOKEN INFISICAL_OPERATOR_TOKEN INFISICAL_BOOTSTRAP_TOKEN RAILWAY_TOKEN RAILWAY_PERSONAL_API_TOKEN HETZNER_TOKEN HETZNER_API_TOKEN"
+    FORBIDDEN_PREFIXES="MOLECULE_OPERATOR_"
+    FOUND=""
+    for k in $FORBIDDEN_KEYS; do
+        # eval is safe here — $k is from a static whitespace-separated
+        # literal list above (no user input). POSIX sh has no
+        # associative arrays, hence the indirect-expansion via eval to
+        # test "is this var set" without caring about its value.
+        eval "v=\${$k+set}"
+        if [ "$v" = "set" ]; then
+            FOUND="$FOUND $k"
+        fi
+    done
+    for prefix in $FORBIDDEN_PREFIXES; do
+        # env | awk is the portable POSIX way to enumerate by prefix.
+        # busybox awk (alpine), gawk (debian), and BSD awk (macOS-test)
+        # all support index(). Doesn't depend on bash arrays / [[ =~ ]].
+        prefix_hits=$(env | awk -F= -v p="$prefix" 'index($1, p)==1 {print $1}')
+        if [ -n "$prefix_hits" ]; then
+            FOUND="$FOUND $prefix_hits"
+        fi
+    done
+    if [ -n "$FOUND" ]; then
+        echo "RFC#523 Layer 2: refusing to start tenant workspace — forbidden operator-scope env var(s) present:$FOUND" >&2
+        echo "These vars are operator-fleet scope and must not reach tenant workspaces." >&2
+        echo "Remove them from workspace_secrets / global_secrets / docker -e and retry." >&2
+        echo "If running this image standalone for local dev with intentional operator scope, set MOLECULE_TENANT_GUARD_DISABLE=1." >&2
+        exit 1
+    fi
+fi
+
 if [ "$(id -u)" = "0" ]; then
     # Configs volume is created by Docker as root; agent needs write access
     # for plugin installs, memory writes, .auth_token rotation, etc.

workspace/tests/test_entrypoint_forbidden_env_guard.sh

@@ -0,0 +1,122 @@
+#!/usr/bin/env bash
+# Smoke-test for RFC#523 Layer 2 (task #146): the workspace/entrypoint.sh
+# top-of-file forbidden-env guard.
+#
+# Strategy: source the prefix of entrypoint.sh that contains the guard
+# (up through the closing `fi` of the guard block), in a sub-shell with
+# the env we want to test. We rewrite the `exit 1` to a `return 1` so
+# the guard signals failure via the sub-shell's exit code without
+# killing the test harness.
+#
+# Why not docker-run the actual image: the test is unit-scope (does
+# the guard logic correctly identify forbidden vs allowed env). Image
+# integration is covered by the E2E provision test described in
+# RFC#523 §"Acceptance criteria" Layer 2 (run on staging, not here).
+#
+# Pairs with: workspace_provision_forbidden_env_test.go (Layer 1
+# Go-side unit tests).
+
+set -euo pipefail
+
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+ENTRYPOINT="$HERE/../entrypoint.sh"
+
+if [[ ! -f "$ENTRYPOINT" ]]; then
+    echo "FAIL: entrypoint not found: $ENTRYPOINT" >&2
+    exit 1
+fi
+
+# Extract just the guard block (from the first `if [ "${MOLECULE_TENANT_GUARD_DISABLE`
+# through the matching `fi`) and rewrite `exit 1` to `return 1` so the
+# guard can be invoked inside a function in a sub-shell.
+GUARD_SNIPPET=$(awk '
+    /^if \[ "\${MOLECULE_TENANT_GUARD_DISABLE/ { inblock=1 }
+    inblock { print }
+    inblock && /^fi$/ { exit }
+' "$ENTRYPOINT" | sed 's/exit 1/return 1/')
+
+if [[ -z "$GUARD_SNIPPET" ]]; then
+    echo "FAIL: could not extract guard block from $ENTRYPOINT" >&2
+    exit 1
+fi
+
+# Helper: run the guard with the env we set, capture exit code. The
+# sub-shell starts with `env -i` semantics emulated by `unset` of every
+# var the guard checks, so prior shell state doesn't contaminate.
+run_guard() {
+    # Pass extra-env assignments as args; e.g. run_guard GITEA_TOKEN=x.
+    (
+        set +e
+        # Defensive unset of all keys the guard inspects, so the
+        # caller's args are the ONLY positive cases.
+        unset GITEA_TOKEN GITEA_PAT GITHUB_TOKEN GITHUB_PAT GH_TOKEN GITLAB_TOKEN GL_TOKEN BITBUCKET_TOKEN
+        unset CP_ADMIN_API_TOKEN CP_ADMIN_TOKEN
+        unset INFISICAL_OPERATOR_TOKEN INFISICAL_BOOTSTRAP_TOKEN
+        unset RAILWAY_TOKEN RAILWAY_PERSONAL_API_TOKEN HETZNER_TOKEN HETZNER_API_TOKEN
+        unset MOLECULE_OPERATOR_HOST MOLECULE_OPERATOR_SSH_KEY
+        unset MOLECULE_TENANT_GUARD_DISABLE
+        for kv in "$@"; do
+            export "$kv"
+        done
+        guard_fn() {
+            eval "$GUARD_SNIPPET"
+        }
+        guard_fn
+        echo $?
+    )
+}
+
+PASS=0
+FAIL=0
+
+assert_exit() {
+    local label="$1"
+    local want="$2"
+    shift 2
+    local got
+    got=$(run_guard "$@" | tail -n 1)
+    if [[ "$got" == "$want" ]]; then
+        echo "PASS: $label"
+        PASS=$((PASS + 1))
+    else
+        echo "FAIL: $label — want exit=$want got=$got (env: $*)" >&2
+        FAIL=$((FAIL + 1))
+    fi
+}
+
+# --- Case 1: clean env passes (exit 0) ---
+assert_exit "clean_env_passes" 0
+
+# --- Case 2: per-agent-scope vars pass (exit 0) ---
+assert_exit "per_agent_vars_pass" 0 \
+    GIT_HTTP_USERNAME=agent-dev-a \
+    GIT_HTTP_PASSWORD=scoped-pat \
+    ANTHROPIC_API_KEY=sk-keep \
+    MOLECULE_AGENT_ROLE=agent-dev-a
+
+# --- Case 3: forbidden exact-match keys fail (exit 1) ---
+assert_exit "gitea_token_blocks"          1 GITEA_TOKEN=leak
+assert_exit "github_token_blocks"         1 GITHUB_TOKEN=leak
+assert_exit "cp_admin_api_token_blocks"   1 CP_ADMIN_API_TOKEN=leak
+assert_exit "infisical_operator_blocks"   1 INFISICAL_OPERATOR_TOKEN=leak
+assert_exit "railway_token_blocks"        1 RAILWAY_TOKEN=leak
+
+# --- Case 4: MOLECULE_OPERATOR_ prefix family blocks ---
+assert_exit "molecule_operator_host_blocks" 1 MOLECULE_OPERATOR_HOST=op.example.com
+assert_exit "molecule_operator_ssh_blocks"  1 MOLECULE_OPERATOR_SSH_KEY=ssh-ed25519...
+
+# --- Case 5: adjacent-but-allowed MOLECULE_* names pass ---
+assert_exit "molecule_agent_role_passes" 0 MOLECULE_AGENT_ROLE=agent-dev-a
+assert_exit "molecule_url_passes"        0 MOLECULE_URL=https://platform.example.com
+
+# --- Case 6: MOLECULE_TENANT_GUARD_DISABLE=1 bypasses the guard ---
+assert_exit "disable_flag_bypasses" 0 \
+    MOLECULE_TENANT_GUARD_DISABLE=1 \
+    GITEA_TOKEN=leak \
+    CP_ADMIN_API_TOKEN=leak
+
+echo
+echo "=== L2 entrypoint guard: $PASS passed, $FAIL failed ==="
+if [[ "$FAIL" -gt 0 ]]; then
+    exit 1
+fi