molecule-ai/molecule-core
44
0
Fork 2
Code Issues 134 Pull Requests 156 Actions Packages Projects Releases Wiki Activity

ci(publish): retarget ship-the-fix workflows to dedicated publish lane (internal#394/#399) [BLOCKED on lane] #1315

Closed
hongming wants to merge 1 commits from feat/publish-lane-runs-on-394 into main
pull from: feat/publish-lane-runs-on-394
merge into: molecule-ai:main
Conversation 4 Commits 1 Files Changed 4
+40 -12
hongming commented 2026-05-16 07:38:58 +00:00
Owner
Copy Link
Copy Source

Summary

  • Retargets ONLY the 6 post-merge ship jobs (publish-runtime publish+cascade, publish-canvas build-and-push, publish-workspace-server build-and-push+deploy-production, redeploy-tenants redeploy) to the reserved publish lane so a merged fix ships immediately instead of queuing behind PR-CI (internal#399).
  • PR-validation jobs deliberately NOT moved (publish-runtime-autobump pr-validate stays on the general pool).

⚠ HARD MERGE BLOCK

Gitea 1.22.6: a bare runs-on: <label> with NO runner advertising it queues FOREVER (the trap that got the docker label reverted). This PR MUST NOT merge until BOTH:

  1. operator-config#63 merged + propagated, AND
  2. publish-lane-ensure.sh run under explicit Hongming GO, lane verified registered (≥2 publish runners).

Merging earlier wedges every release + production deploy.

Test plan

  • All 4 workflows valid YAML; exactly 6 ship jobs retargeted; PR-validate confirmed unchanged
  • Non-author five-axis review
  • operator-config#63 landed + lane instantiated under GO
  • After lane live: dry-run a publish (workflow_dispatch) and confirm it lands on a publish runner, not the general pool

Refs internal#394, #399, #305; pairs with operator-config#63.

## Summary - Retargets ONLY the 6 post-merge ship jobs (publish-runtime publish+cascade, publish-canvas build-and-push, publish-workspace-server build-and-push+deploy-production, redeploy-tenants redeploy) to the reserved `publish` lane so a merged fix ships immediately instead of queuing behind PR-CI (internal#399). - PR-validation jobs deliberately NOT moved (publish-runtime-autobump pr-validate stays on the general pool). ## ⚠ HARD MERGE BLOCK Gitea 1.22.6: a bare `runs-on: <label>` with NO runner advertising it queues FOREVER (the trap that got the `docker` label reverted). This PR MUST NOT merge until BOTH: 1. operator-config#63 merged + propagated, AND 2. publish-lane-ensure.sh run under explicit Hongming GO, lane verified registered (≥2 publish runners). Merging earlier wedges every release + production deploy. ## Test plan - [x] All 4 workflows valid YAML; exactly 6 ship jobs retargeted; PR-validate confirmed unchanged - [ ] Non-author five-axis review - [ ] operator-config#63 landed + lane instantiated under GO - [ ] After lane live: dry-run a publish (workflow_dispatch) and confirm it lands on a publish runner, not the general pool Refs internal#394, #399, #305; pairs with operator-config#63.
hongming added 1 commit 2026-05-16 07:39:13 +00:00
ci(publish): retarget ship-the-fix workflows to the dedicated publish lane (internal#394/#399)
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 27s
Details
cascade-list-drift-gate / check (pull_request) Successful in 28s
Details
CI / Detect changes (pull_request) Successful in 1m17s
Details
Handlers Postgres Integration / detect-changes (pull_request) Successful in 29s
Details
E2E API Smoke Test / detect-changes (pull_request) Successful in 1m59s
Details
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 1m47s
Details
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 1m59s
Details
qa-review / approved (pull_request) Failing after 43s
Details
Lint workflow YAML (Gitea-1.22.6-hostile shapes) / Lint workflow YAML for Gitea-1.22.6-hostile shapes (pull_request) Successful in 2m18s
Details
lint-continue-on-error-tracking / lint-continue-on-error-tracking (pull_request) Successful in 4m47s
Details
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 2m14s
Details
lint-required-context-exists-in-bp / lint-required-context-exists-in-bp (pull_request) Successful in 4m19s
Details
sop-checklist / all-items-acked (pull_request) Successful in 38s
Details
security-review / approved (pull_request) Failing after 40s
Details
sop-tier-check / tier-check (pull_request) Successful in 37s
Details
CI / Python Lint & Test (pull_request) Successful in 9m9s
Details
CI / Canvas (Next.js) (pull_request) Failing after 11m5s
Details
CI / Canvas Deploy Reminder (pull_request) Has been skipped
Details
CI / all-required (pull_request) Failing after 11m10s
Details
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 33s
Details
CI / Platform (Go) (pull_request) Failing after 21m28s
Details
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 26s
Details
CI / Shellcheck (E2E scripts) (pull_request) Has been cancelled
Details
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 25s
Details
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 6m29s
Details
Secret scan / Scan diff for credential-shaped strings (pull_request) Has been cancelled
Details
gate-check-v3 / gate-check (pull_request) Has been cancelled
Details
Lint pre-flip continue-on-error / Verify continue-on-error flips have run-log proof (pull_request) Has been cancelled
Details
audit-force-merge / audit (pull_request) Has been skipped
Details
813f706a71 
BLOCKED — do NOT merge until the publish lane is live (see below).

All 107 molecule-core jobs use runs-on: ubuntu-latest, so the 4
ship-the-fix workflows queue in the same FIFO as every PR-CI job.
internal#399: on a fix merge, main push image build/push/deploy waits
behind the PR backlog. This retargets ONLY the post-merge ship jobs
to the reserved `publish` lane (operator-config#63):

- publish-runtime.yml: publish + cascade
- publish-canvas-image.yml: build-and-push
- publish-workspace-server-image.yml: build-and-push + deploy-production
- redeploy-tenants-on-main.yml: redeploy

Deliberately NOT changed: publish-runtime-autobump.yml's pr-validate
job (it runs on PRs — that is PR-CI, stays on the general pool). Only
push/tag/workflow_dispatch ship jobs move.

HARD SEQUENCING PRECONDITION: Gitea 1.22.6 schedules a bare
`runs-on: <label>` job onto a runner advertising that label, and if
NONE exists the job queues INDEFINITELY (the exact trap that got the
`docker` label reverted — see the inline note retained in
publish-canvas-image.yml). Therefore this PR MUST NOT merge until
≥2 `publish`-labelled runners are registered:
  1. operator-config#63 merged + propagated, AND
  2. publish-lane-ensure.sh run under explicit Hongming GO
     (ALLOW_FLEET_MUTATION=1), lane verified registered in Gitea.
Merging earlier would wedge every release + production deploy.

Validated: all 4 workflows parse as valid YAML; exactly the 6 ship
jobs retargeted; PR-validation job confirmed unchanged.

Refs internal#394, #399, #305; pairs with operator-config#63.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
core-qa commented 2026-05-16 07:50:08 +00:00
Member
Copy Link
Copy Source

[core-qa-agent] N/A — CI workflow only

Retargets publish workflows to a dedicated lane (internal#394/#399). Pure CI infrastructure change across 4 workflow YAML files. No platform Go code, no Python workspace, no Canvas. CI gate validates.

[core-qa-agent] N/A — CI workflow only Retargets publish workflows to a dedicated lane (internal#394/#399). Pure CI infrastructure change across 4 workflow YAML files. No platform Go code, no Python workspace, no Canvas. CI gate validates.
core-devops commented 2026-05-16 07:51:06 +00:00
Member
Copy Link
Copy Source

[core-devops-agent] CI review — runs-on: publish lane changes

Reviewed all 4 workflow files in this PR:

Changes:

  • publish-canvas-image.yml, publish-runtime.yml, publish-workspace-server-image.yml, redeploy-tenants-on-main.yml — all post-merge ship jobs changed from runs-on: ubuntu-latest to runs-on: publish.

Concerns (non-blocking):

  1. Indefinite queue risk (Gitea 1.22.6): A bare runs-on: publish with no registered runners queues forever — the same trap that caused the reverted docker label (#576). The hard precondition comment in each job is correct and explicit. This PR is blocked until ≥2 publish-lane runners are registered (operator-config#63).

  2. continue-on-error: true masks lane-not-found failures: If the publish lane doesn't exist at merge time, continue-on-error: true will suppress the failure and the image won't be built/published. Consider whether this is the right Phase 3 behavior, or whether publish failures should be surfaced (fail-open vs fail-closed for production release).

  3. No change to PR-validation jobs: Confirmed that publish-runtime-autobump pr-validate and other PR-CI jobs remain on ubuntu-latest — correct per the PR description.

LGTM with the hard precondition noted. The inline comments are thorough and the rationale is clear. Merge order: (1) register ≥2 publish-lane runners, (2) verify runs-on: publish jobs run on main, (3) merge this PR.

[core-devops-agent] CI review — `runs-on: publish` lane changes Reviewed all 4 workflow files in this PR: **Changes:** - `publish-canvas-image.yml`, `publish-runtime.yml`, `publish-workspace-server-image.yml`, `redeploy-tenants-on-main.yml` — all post-merge ship jobs changed from `runs-on: ubuntu-latest` to `runs-on: publish`. **Concerns (non-blocking):** 1. **Indefinite queue risk (Gitea 1.22.6):** A bare `runs-on: publish` with no registered runners queues forever — the same trap that caused the reverted `docker` label (#576). The hard precondition comment in each job is correct and explicit. This PR is blocked until ≥2 publish-lane runners are registered (operator-config#63). 2. **`continue-on-error: true` masks lane-not-found failures:** If the publish lane doesn't exist at merge time, `continue-on-error: true` will suppress the failure and the image won't be built/published. Consider whether this is the right Phase 3 behavior, or whether publish failures should be surfaced (fail-open vs fail-closed for production release). 3. **No change to PR-validation jobs:** Confirmed that `publish-runtime-autobump pr-validate` and other PR-CI jobs remain on `ubuntu-latest` — correct per the PR description. **LGTM** with the hard precondition noted. The inline comments are thorough and the rationale is clear. Merge order: (1) register ≥2 publish-lane runners, (2) verify `runs-on: publish` jobs run on main, (3) merge this PR.
core-security commented 2026-05-16 08:10:33 +00:00
Member
Copy Link
Copy Source

[core-security-agent] N/A — CI ops. 4 workflow files retarget to dedicated 'publish' runner lane (internal#394/#399). Explicit note: 'Do NOT merge before the lane exists' (Gitea 1.22.6 bare runs-on queues indefinitely without registered runner). No production code. No security surface.

[core-security-agent] N/A — CI ops. 4 workflow files retarget to dedicated 'publish' runner lane (internal#394/#399). Explicit note: 'Do NOT merge before the lane exists' (Gitea 1.22.6 bare runs-on queues indefinitely without registered runner). No production code. No security surface.
hongming commented 2026-05-19 01:11:16 +00:00
Author
Owner
Copy Link
Copy Source

Superseded by #1376 (merged 2026-05-16T19:47:27Z, retargeted publish/deploy ship jobs to the dedicated publish lane per internal#462). Closing as duplicate; no functional gap.

Ref: https://git.moleculesai.app/molecule-ai/molecule-core/pulls/1376

Superseded by #1376 (merged 2026-05-16T19:47:27Z, retargeted publish/deploy ship jobs to the dedicated publish lane per internal#462). Closing as duplicate; no functional gap. Ref: https://git.moleculesai.app/molecule-ai/molecule-core/pulls/1376
hongming closed this pull request 2026-05-19 01:11:16 +00:00
Some required checks failed
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 27s
Details
cascade-list-drift-gate / check (pull_request) Successful in 28s
Details
CI / Detect changes (pull_request) Successful in 1m17s
Details
Handlers Postgres Integration / detect-changes (pull_request) Successful in 29s
Details
E2E API Smoke Test / detect-changes (pull_request) Successful in 1m59s
Details
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 1m47s
Details
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 1m59s
Details
qa-review / approved (pull_request) Failing after 43s
Details
Lint workflow YAML (Gitea-1.22.6-hostile shapes) / Lint workflow YAML for Gitea-1.22.6-hostile shapes (pull_request) Successful in 2m18s
Details
lint-continue-on-error-tracking / lint-continue-on-error-tracking (pull_request) Successful in 4m47s
Details
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 2m14s
Details
lint-required-context-exists-in-bp / lint-required-context-exists-in-bp (pull_request) Successful in 4m19s
Details
sop-checklist / all-items-acked (pull_request) Successful in 38s
Details
security-review / approved (pull_request) Failing after 40s
Details
sop-tier-check / tier-check (pull_request) Successful in 37s
Details
CI / Python Lint & Test (pull_request) Successful in 9m9s
Details
CI / Canvas (Next.js) (pull_request) Failing after 11m5s
Details
CI / Canvas Deploy Reminder (pull_request) Has been skipped
Details
CI / all-required (pull_request) Failing after 11m10s
Required
Details
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 33s
Details
CI / Platform (Go) (pull_request) Failing after 21m28s
Details
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 26s
Details
CI / Shellcheck (E2E scripts) (pull_request) Has been cancelled
Details
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 25s
Details
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 6m29s
Details
Secret scan / Scan diff for credential-shaped strings (pull_request) Has been cancelled
Details
gate-check-v3 / gate-check (pull_request) Has been cancelled
Details
Lint pre-flip continue-on-error / Verify continue-on-error flips have run-log proof (pull_request) Has been cancelled
Details
audit-force-merge / audit (pull_request) Has been skipped
Details

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
area/ci
CI/CD pipeline issues
 kind/infrastructure
Infrastructure-related issues
 merge-queue
Ready for serialized Gitea merge queue
 merge-queue-hold
Temporarily hold PR in merge queue
 platform/go
Go platform test issues
 release-blocker
Blocks the staging→main promotion / a release
 release-test
security
test-label-sre
tier:high
High risk per dev-sop §SOP-6 — ceo only, 24h cooldown
 tier:low
Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve
 tier:medium
Medium risk per dev-sop §SOP-6 — managers/ceo can approve
 triage-test
test
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
agent-dev-a agent-dev-b agent-pm app-fe (Molecule AI · app-fe) app-lead (Molecule AI · app-lead) app-qa (Molecule AI · app-qa) claude-ceo-assistant claude-ci-reader claude-status-reaper core-be (Molecule AI · core-be) core-devops (Molecule AI · core-devops) core-fe (Molecule AI · core-fe) core-lead (Molecule AI · core-lead) core-offsec (Molecule AI · core-offsec) core-qa (Molecule AI · core-qa) core-security (Molecule AI · core-security) core-uiux (Molecule AI · core-uiux) cp-be (Molecule AI · cp-be) cp-lead (Molecule AI · cp-lead) cp-qa (Molecule AI · cp-qa) cp-security (Molecule AI · cp-security) cui (Zhanlin Cui) dev-lead (Molecule AI · dev-lead) devops-engineer documentation-specialist (Molecule AI · documentation-specialist) fullstack-engineer (Molecule AI · fullstack-engineer) hongming hongming-codex-laptop hongming-kimi-laptop hongming-pc2 infra-lead (Molecule AI · infra-lead) infra-runtime-be (Molecule AI · infra-runtime-be) infra-sre (Molecule AI · infra-sre) integration-tester (Molecule AI · integration-tester) mc-drift-bot plugin-dev (Molecule AI · plugin-dev) pm publish-runtime-bot release-manager (Molecule AI · release-manager) sdk-dev (Molecule AI · sdk-dev) sdk-lead (Molecule AI · sdk-lead) sop-drift-bot sop-tier-bot (SOP Tier-Check Bot) technical-writer (Molecule AI · technical-writer) triage-operator (Molecule AI · triage-operator)
No Assignees
4 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#1315
Delete Branch "feat/publish-lane-runs-on-394"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?