Compare commits

..

4 Commits

Author SHA1 Message Date
core-uiux 5ff5638f53 chore: re-trigger SOP after memory-consultat typo fix
sop-checklist / all-items-acked (pull_request) [info tier:low] acked: 4/7 — missing: root-cause, no-backwards-compat, memory-consulted — body-unfilled: comprehensive-testing, local-postgr
sop-checklist / na-declarations (pull_request) N/A: (none)
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 6s
CI / Detect changes (pull_request) Successful in 4s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 11s
E2E API Smoke Test / detect-changes (pull_request) Successful in 5s
E2E Chat / detect-changes (pull_request) Successful in 4s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 4s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 3s
Harness Replays / detect-changes (pull_request) Successful in 2s
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 52s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 4s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 3s
gate-check-v3 / gate-check (pull_request) Successful in 2s
qa-review / approved (pull_request) Failing after 3s
security-review / approved (pull_request) Failing after 3s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 2s
CI / Canvas (Next.js) (pull_request) Successful in 5m33s
Harness Replays / Harness Replays (pull_request) Successful in 1s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 1s
CI / all-required (pull_request) Successful in 6m52s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
sop-tier-check / tier-check (pull_request) Successful in 3s
CI / Platform (Go) (pull_request) Successful in 4m4s
CI / Python Lint & Test (pull_request) Successful in 6m52s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 1s
E2E Chat / E2E Chat (pull_request) Failing after 4m19s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 6m3s
2026-05-17 06:54:57 +00:00
core-uiux 18b93aab66 chore: re-trigger SOP and CI
CI / Canvas Deploy Reminder (pull_request) Blocked by required conditions
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 3s
CI / Detect changes (pull_request) Successful in 4s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 9s
E2E API Smoke Test / detect-changes (pull_request) Successful in 5s
E2E Chat / detect-changes (pull_request) Successful in 6s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 7s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 5s
Harness Replays / detect-changes (pull_request) Successful in 5s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 8s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 5s
gate-check-v3 / gate-check (pull_request) Successful in 4s
qa-review / approved (pull_request) Failing after 4s
security-review / approved (pull_request) Failing after 4s
sop-checklist / na-declarations (pull_request) N/A: (none)
sop-checklist / all-items-acked (pull_request) Successful in 5s
sop-tier-check / tier-check (pull_request) Successful in 4s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 3s
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 1m5s
CI / Platform (Go) (pull_request) Successful in 5m1s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 1s
Harness Replays / Harness Replays (pull_request) Successful in 1s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 7s
CI / Canvas (Next.js) (pull_request) Successful in 6m6s
E2E Chat / E2E Chat (pull_request) Failing after 5m11s
CI / Python Lint & Test (pull_request) Successful in 7m3s
CI / all-required (pull_request) Successful in 7m9s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 7m15s
2026-05-17 06:51:40 +00:00
core-uiux 9345d3684d chore: re-trigger SOP after PR creation
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 3s
CI / Detect changes (pull_request) Successful in 4s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 14s
E2E API Smoke Test / detect-changes (pull_request) Successful in 6s
E2E Chat / detect-changes (pull_request) Successful in 7s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 6s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 8s
Harness Replays / detect-changes (pull_request) Successful in 6s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 7s
gate-check-v3 / gate-check (pull_request) Successful in 3s
qa-review / approved (pull_request) Failing after 3s
security-review / approved (pull_request) Failing after 4s
sop-tier-check / tier-check (pull_request) Successful in 3s
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 1m2s
sop-checklist / na-declarations (pull_request) N/A: (none)
sop-checklist / all-items-acked (pull_request) Successful in 3s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 8s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 3s
Harness Replays / Harness Replays (pull_request) Successful in 2s
E2E Chat / E2E Chat (pull_request) Failing after 6m4s
CI / Python Lint & Test (pull_request) Failing after 14m52s
CI / all-required (pull_request) Failing after 14m46s
Runtime PR-Built Compatibility / detect-changes (pull_request) Failing after 13m40s
CI / Platform (Go) (pull_request) Failing after 16m20s
CI / Canvas (Next.js) (pull_request) Failing after 16m18s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 7m59s
CI / Canvas Deploy Reminder (pull_request) Has been cancelled
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Has been cancelled
2026-05-17 06:12:11 +00:00
core-uiux 5f4f1b72d9 fix(canvas): add aria-live=assertive to error banners in ChatTab, ActivityTab, AuditTrailPanel
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 3s
CI / Detect changes (pull_request) Successful in 4s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 11s
E2E API Smoke Test / detect-changes (pull_request) Successful in 4s
E2E Chat / detect-changes (pull_request) Successful in 4s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 5s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 2s
Harness Replays / detect-changes (pull_request) Successful in 4s
CI / Platform (Go) (pull_request) Successful in 4m27s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 4s
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 57s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 3s
qa-review / approved (pull_request) Failing after 3s
gate-check-v3 / gate-check (pull_request) Successful in 3s
security-review / approved (pull_request) Failing after 2s
sop-checklist / all-items-acked (pull_request) Successful in 3s
sop-tier-check / tier-check (pull_request) Successful in 4s
CI / Canvas (Next.js) (pull_request) Successful in 6m11s
CI / Python Lint & Test (pull_request) Successful in 6m33s
CI / all-required (pull_request) Successful in 6m37s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 10s
Harness Replays / Harness Replays (pull_request) Successful in 2s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 9s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 9s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
E2E Chat / E2E Chat (pull_request) Failing after 6m38s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Failing after 11m33s
Add missing role="alert" and aria-live="assertive" to error state
banners in three components. WCAG 4.1.3 requires that status and error
messages be programmatically determinable via aria-live regions so
screen readers announce them immediately.

Changes:
- ChatTab.tsx: displayError banner — was missing role/aria-live
- ActivityTab.tsx: error banner — was missing role/aria-live
- AuditTrailPanel.tsx: error banner — had role="alert" but missing aria-live

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 06:11:16 +00:00
10 changed files with 38 additions and 216 deletions
+3 -86
View File
@@ -44,15 +44,9 @@ REQUIRED_CONTEXTS_RAW = _env(
"REQUIRED_CONTEXTS",
default=(
"CI / all-required (pull_request),"
"sop-checklist / all-items-acked (pull_request),"
"E2E Chat / E2E Chat (pull_request)"
"sop-checklist / all-items-acked (pull_request)"
),
)
# E2E Chat is not in branch protection's status_check_contexts, but Gitea's
# merge gate evaluates the full combined status including it. Adding it here
# prevents the queue from attempting a merge that will be 405'd by Gitea when
# E2E Chat is failing (e.g. runner-stall Quirk #9 on a flaky test).
# See: mc#420 / molecule-core runbooks/gitea-operational-quirks.md Quirk #9.
# Required contexts for push (main/staging) runs. The push CI uses the same
# aggregator names with " (push)" suffix. Checking these explicitly instead of
# the combined state avoids false-pause when non-blocking jobs (e.g. Platform
@@ -71,11 +65,6 @@ class ApiError(RuntimeError):
pass
class MergePermissionError(ApiError):
"""Merge failed with a permanent permission error (403/404/405).
The queue should skip this PR and move to the next one."""
@dataclasses.dataclass(frozen=True)
class MergeDecision:
ready: bool
@@ -325,31 +314,6 @@ def post_comment(pr_number: int, body: str, *, dry_run: bool) -> None:
api("POST", f"/repos/{OWNER}/{NAME}/issues/{pr_number}/comments", body={"body": body})
def add_hold_label(pr_number: int, *, dry_run: bool) -> None:
"""Add HOLD_LABEL to a PR if not already present."""
if not HOLD_LABEL:
return
# Check current labels first to avoid a no-op API call in dry-run.
_, current = api("GET", f"/repos/{OWNER}/{NAME}/issues/{pr_number}/labels")
current_names = {
l["name"] for l in (current if isinstance(current, list) else [])
}
if HOLD_LABEL in current_names:
print(f"::notice::PR #{pr_number} already has hold label; skipping add")
return
print(f"::notice::PR #{pr_number} adding hold label `{HOLD_LABEL}`")
if dry_run:
return
# Gitea accepts {"labels": ["label1", "label2"]} to append labels.
new_labels = list(current_names) + [HOLD_LABEL]
api(
"PATCH",
f"/repos/{OWNER}/{NAME}/issues/{pr_number}",
body={"labels": new_labels},
expect_json=False,
)
def update_pull(pr_number: int, *, dry_run: bool) -> None:
print(f"::notice::updating PR #{pr_number} with base branch via style={UPDATE_STYLE}")
if dry_run:
@@ -374,16 +338,7 @@ def merge_pull(pr_number: int, *, dry_run: bool) -> None:
print(f"::notice::merging PR #{pr_number}")
if dry_run:
return
try:
api("POST", f"/repos/{OWNER}/{NAME}/pulls/{pr_number}/merge", body=payload, expect_json=False)
except ApiError as exc:
# Re-raise permission-like errors so process_once can skip this PR.
# 403 = no push access, 404 = repo/pr not found, 405 = not allowed.
msg = str(exc)
for code in ("403", "404", "405"):
if code in msg:
raise MergePermissionError(msg) from exc
raise # re-raise other ApiErrors unchanged
api("POST", f"/repos/{OWNER}/{NAME}/pulls/{pr_number}/merge", body=payload, expect_json=False)
def process_once(*, dry_run: bool = False) -> int:
@@ -452,45 +407,7 @@ def process_once(*, dry_run: bool = False) -> int:
"deferring to next tick"
)
return 0
try:
merge_pull(pr_number, dry_run=dry_run)
except MergePermissionError as exc:
msg = str(exc)
is_status_check_failure = "not all required status checks successful" in msg
if is_status_check_failure:
# Gitea's merge gate failed due to a status check that passed our
# pre-flight but is failing at Gitea's side (e.g. runner-stall Quirk
# #9, or a context not in REQUIRED_CONTEXTS). Auto-add hold so the
# queue skips this PR and processes the next one. The hold can be
# removed once CI is green again.
add_hold_label(pr_number, dry_run=dry_run)
post_comment(
pr_number,
(
"merge-queue: merge blocked by Gitea's status-check gate "
"(E2E Chat or other non-required context failing). "
"Auto-held via `merge-queue-hold`. "
"Remove the hold label to requeue once CI is green. "
"If E2E Chat is stuck (runner stall / Quirk #9), CI will "
"self-recover after ~90 min and the hold can then be removed."
),
dry_run=dry_run,
)
return 0
else:
# Genuine permission error — token lacks Can-merge.
sys.stderr.write(f"::error::merge permission error for PR #{pr_number}: {exc}\n")
post_comment(
pr_number,
(
"merge-queue: merge failed with HTTP 405 'User not allowed to merge PR'. "
"No available token has Can-merge permission on this repo. "
"Fix: grant Can-merge to a token, or add a maintain/admin collaborator. "
"Skipping to next queued PR on next tick."
),
dry_run=dry_run,
)
return 0
merge_pull(pr_number, dry_run=dry_run)
return 0
return 0
@@ -118,13 +118,3 @@ def test_merge_decision_updates_stale_pr_before_merge():
assert decision.ready is False
assert decision.action == "update"
def test_MergePermissionError_inherits_from_ApiError():
assert issubclass(mq.MergePermissionError, mq.ApiError)
def test_MergePermissionError_message_preserved():
exc = mq.MergePermissionError("POST /merge -> HTTP 405: User not allowed")
assert "405" in str(exc)
assert "User not allowed" in str(exc)
+1 -1
View File
@@ -57,7 +57,7 @@ permissions:
# can produce duplicate comments before the title-search dedup wins.
concurrency:
group: ci-required-drift
cancel-in-progress: true
cancel-in-progress: false
jobs:
drift:
+1 -1
View File
@@ -22,7 +22,7 @@ permissions:
concurrency:
group: gitea-merge-queue-${{ github.repository }}
cancel-in-progress: true
cancel-in-progress: false
jobs:
queue:
+1 -5
View File
@@ -56,13 +56,9 @@ permissions:
# Workflow-scoped serialisation — two simultaneous runs would race on the
# `[main-red] {SHA}` open/PATCH path. Idempotent by title, but parallel
# POSTs can produce duplicates before the title search dedup wins.
# NOTE: cancel-in-progress: true is safe here — the idempotent design means
# a cancelled run produces identical output to a completed one. This also
# prevents the Gitea scheduler freeze that occurs when a cron tick fires
# while a previous run is still executing (Quirk #8).
concurrency:
group: main-red-watchdog
cancel-in-progress: true
cancel-in-progress: false
jobs:
watchdog:
@@ -172,6 +172,7 @@ export function AuditTrailPanel({ workspaceId }: Props) {
{error && (
<div
role="alert"
aria-live="assertive"
className="mx-4 mt-3 px-3 py-2 bg-red-950/30 border border-red-800/40 rounded text-xs text-bad shrink-0"
>
{error}
+5 -1
View File
@@ -189,7 +189,11 @@ export function ActivityTab({ workspaceId }: Props) {
)}
{error && (
<div className="px-3 py-1.5 bg-red-900/30 border border-red-800 rounded text-xs text-bad">
<div
role="alert"
aria-live="assertive"
className="px-3 py-1.5 bg-red-900/30 border border-red-800 rounded text-xs text-bad"
>
{error}
</div>
)}
+5 -1
View File
@@ -594,7 +594,11 @@ function MyChatPanel({ workspaceId, data }: Props) {
{/* Error banner */}
{displayError && (
<div className="px-3 py-2 bg-red-900/20 border-t border-red-800/30">
<div
role="alert"
aria-live="assertive"
className="px-3 py-2 bg-red-900/20 border-t border-red-800/30"
>
<div className="flex items-center justify-between">
<span className="text-[10px] text-red-300">{displayError}</span>
{!isOnline && (
-25
View File
@@ -77,31 +77,6 @@ does not replace the queue. The queue still performs its own current-main
check immediately before merge because branch protection alone cannot
serialize two already-green PRs.
### Correct API field names (Gitea 1.22.6)
When setting branch protection via API, use these exact field names — several
intuitively-correct names are silently ignored (see `gitea-operational-quirks.md`
Quirk #7):
```json
{
"branch_name": "main",
"enable_merge_whitelist": true,
"merge_whitelist_usernames": ["devops-engineer", "hongming", "core-devops"],
"enable_status_check": true,
"status_check_contexts": ["CI / all-required"],
"required_approvals": 1,
"block_on_rejected_reviews": true
}
```
After any `POST /branch_protections`, immediately GET and verify the values
persisted — the API returns 201 even when fields are silently dropped.
If the queue returns HTTP 405 ("User not allowed to merge"), the first
diagnostic step is `GET /branch_protections/main` and checking whether
`merge_whitelist_usernames` still contains `devops-engineer`.
## Failure Handling
If `main` is not green, the queue pauses and does not merge anything.
+21 -86
View File
@@ -196,134 +196,69 @@ primary consumer of combined status and is affected.
---
## Quirk #7 — Gitea branch protection API silently ignores some field names
## Quirk #7 — TBD
*[Placeholder — document here when a new Gitea Actions quirk is discovered.]*
### Finding
The Gitea 1.22.6 `POST /repos/{org}/{repo}/branch_protections` API accepts a
non-obvious set of field names. Several intuitively-correct names are silently
ignored — the call returns 201 but the field is dropped:
| Intended field | Correct API name | Silently ignored aliases |
|---|---|---|
| Enable merge whitelist | `enable_merge_whitelist` | `user_can_merge`, `merge_whitelist_enabled` |
| Users who can merge | `merge_whitelist_usernames` | `merge_whitelist_users`, `whitelisted_users` |
| Enable status check | `enable_status_check` | `enable_status_checks`, `require_status_checks` |
| Required status contexts | `status_check_contexts` | `required_status_checks.contexts` |
| Block on rejected reviews | `block_on_rejected_reviews` | (this one works) |
| Required approvals | `required_approvals` | `required_reviewers` |
The GET response after a POST shows the actual stored values. A naive
GET → modify → POST cycle (without using the exact GET field names) will
silently reset the merge whitelist on every call.
*[What Gitea Actions does differently from GitHub Actions.]*
### Impact
- Branch protection merge whitelist resets to empty after any API mis-invocation
- Queue AUTO_SYNC_TOKEN (`devops-engineer`) loses Can-merge permission → HTTP 405
- All queued PRs blocked until whitelist is restored
- Confirmed reset on Gitea server restart/upgrade (Gitea uses default values)
*[Which workflows or operations are affected.]*
### Workaround
1. Always GET the current protection first and use **exact** field names from the
GET response when modifying
2. After any `POST /branch_protections`, immediately GET and verify
`enable_merge_whitelist: true` and `merge_whitelist_usernames` contains
`["devops-engineer", "hongming", "core-devops"]`
3. The queue bot should verify branch protection before each merge tick
4. For queue to work: `enable_merge_whitelist: true` +
`merge_whitelist_usernames: ["devops-engineer", "hongming", "core-devops"]` +
`enable_status_check: true` + `status_check_contexts: ["CI / all-required"]`
*[How to work around this quirk.]*
### References
- SEV-1 2026-05-17: 3x branch protection resets caused 405 on all queue merges
- `feedback_gitea_branch_protection_api_field_names`
- internal#[N]: first observation
---
## Quirk #8 — Scheduled workflow with `cancel-in-progress: false` causes scheduler freeze
## Quirk #8 — TBD
*[Placeholder — document here when a new Gitea Actions quirk is discovered.]*
### Finding
When a `schedule:` workflow has `concurrency.cancel-in-progress: false`, and a
new cron tick fires while the previous run is still executing, the Gitea Actions
scheduler stops dispatching the workflow entirely. Pending entries accumulate
indefinitely — the scheduler shows the workflow as "scheduled" but never dispatches.
This is dangerous for workflows with variable execution time (e.g., workflows that
wait for downstream CI, or workflows that run on slow/degraded runners).
*[What Gitea Actions does differently from GitHub Actions.]*
### Impact
- `gitea-merge-queue.yml` with `cancel-in-progress: false` froze on 2026-05-17
starting ~16:44Z — pending runs accumulated, no new runs dispatched
- Queue appeared stalled; all 22 queued PRs blocked
- The `gitea-merge-queue` workflow itself becomes invisible to operators
*[Which workflows or operations are affected.]*
### Workaround
**Always set `cancel-in-progress: true` on `schedule:` workflows:**
```yaml
concurrency:
group: workflow-name
cancel-in-progress: true # ← always true for schedule: workflows
```
If the freeze has already occurred: the scheduler recovers automatically after the
currently-running instance completes (Gitea dispatches the next queued tick).
*[How to work around this quirk.]*
### References
- SEV-1 2026-05-17: queue frozen since 16:44Z; fixed by setting `cancel-in-progress: true`
- PR #1358: `fix(scheduled-workflows): enable cancel-in-progress` (pending merge)
- internal#[N]: first observation
---
## Quirk #9 — Gitea Actions runner accepts runs but stalls (jobs never start)
## Quirk #9 — TBD
*[Placeholder — document here when a new Gitea Actions quirk is discovered.]*
### Finding
The Gitea Actions runner on host `5.78.80.188` can enter a degraded state where:
1. It accepts new workflow runs (shows "in_progress" in the UI)
2. It never starts any jobs — pending count grows indefinitely
3. The runner shows as "online" and accepting runs
4. After ~6090 minutes, the runner self-recovers and all pending jobs start
This is distinct from a true runner crash (which would show as offline).
*[What Gitea Actions does differently from GitHub Actions.]*
### Impact
- All CI jobs for all PRs stall — no status updates posted
- Queue waits indefinitely for CI (which never posts success)
- `sop-checklist` and other workflows time out on affected PRs
- Looks like the runner is working (green in UI) but nothing executes
### How to diagnose
Add a debug step to a known-failing workflow:
```bash
# In a stalled job:
curl -s http://localhost:8088/debug/pprof/trace?seconds=5 | head
# Check runner process CPU — if near 0% while jobs are pending, runner is stalled
```
Check runner logs on the host (`/var/log/actrunner.log` or similar).
*[Which workflows or operations are affected.]*
### Workaround
No operator workaround while stalled — the runner self-recovers. Options:
1. **Wait** — runner typically recovers within 90 minutes
2. **Restart the runner service**`systemctl restart act_runner` (requires host access)
3. **Move to a second runner** — if registered, re-route dispatch
*[How to work around this quirk.]*
### References
- SEV-1 2026-05-17: runner stalled; self-recovered ~21:33Z after ~90 min
- `feedback_gitea_runner_stall_accepted_jobs_no_execution`
- internal#[N]: first observation
---