One merge, two runs, the same sha

One merge. Two runs of the same workflow. Same commit sha. Neither cancelled the other — the second one waited, then ran to completion. That is the designed behaviour of three lines at the top of .forgejo/workflows/nix-deploy.yml in the nix-config repository, and understanding it changes how you read a deploy log.

Queue, never cancel

1
2
3
concurrency:
  group: nix-deploy-$
  cancel-in-progress: false

Lines 151–153. The workflow’s own comment above them explains why this block exists at all. Forgejo’s documented default, for a workflow with no top-level concurrency, is that for on.push “whenever a new event triggers a new workflow invocation, any other invocations of the same workflow will be canceled automatically”. That default is right for a check — a superseded build is evaluating a commit nobody will merge — and wrong for a deploy. Cancelling a nixos-rebuild switch mid-activation can tear an activation half-way (some units new, some old) and skip the health gate and automatic rollback that follow it. The comment records a live instance: runs 2701 and 2712 cancelled about forty seconds apart while 2701 was mid-switch on vpn, leaving the fleet mixed.

Declaring any top-level concurrency block takes the workflow out of the automatic-cancellation regime. With a group and cancel-in-progress: false, a superseded run queues and completes instead of being killed.

The consequence for reading logs is direct: a later run in the same group waits rather than cancelling. Both runs execute, in order, and the last one to touch a host wins.

The group is keyed on the ref

group: nix-deploy-$ means a push to main and a workflow_dispatch on a PR branch are in different groups and never queue against each other. Two things follow: a dispatch and a push to the same ref share one queue, so the dispatch lines up behind the push rather than racing it; and runs on different refs do not serialise at all, even when they name the same host.

That second point is why the workflow also carries per-host job groups — deploy-mail, deploy-vpn, deploy-public, each with cancel-in-progress: false. As the file notes from Forgejo’s documentation, “Multiple jobs within a workflow are not affected by the concurrency setting”, so the job-level groups are what make two runs exclusive on the same host.

The job graph, and one deliberate always()

Three per-host deploy jobs, one real ordering edge. deploy-public waits on deploy-vpn because the client DNS tiers split their two halves across the two hosts — primary on vpn, standby on public — and the jobs would otherwise run side by side and bounce both halves in the same second. The always() is the subtle half: without it, a skipped deploy-vpn (vpn simply not in scope) would skip deploy-public too, because a skipped job skips everything that needs: it. So the public job starts even when the vpn job failed or never ran, and its pre-flight step re-checks needs.deploy-vpn.result and refuses to switch if the primary side is broken. That is a deliberate choice about publishing the edge: the fallback halves must not be left un-deployed just because the primary tier was untouched.

Identify a run by three things, not by position

With more than one run able to touch one host, “the most recent run in the UI” is not an identifier. A run is identified by workflow + commit sha + trigger event:

• The runs list items carry workflow_id, commit_sha, status and prettyref, which is enough to find the run for a specific merge per workflow — fjlog.py runs --sha <sha> --workflow nix-deploy filters on exactly those.
• The trigger event is in the branch field: prettyref on a run-list item, head_branch on a task row. PR-event runs read #<PR>; push runs read the branch name. A filter that matches only one form silently misses the other and waits out its whole deadline against a run that is already finished.
• A run’s status is not the state of the host. A push-to-main run can be cancelled after it already did its work — measured case: a cancelled nix-deploy whose target host had switched seven minutes earlier. Cancelled is not evidence a deploy did not land, and success is not evidence it did.

When two runs can touch one host, the generation on the host is evidence and the badge is not. nixos-rebuild list-generations prints newest first, so the current generation is the head of that list; the run’s own badge can only tell you what the pipeline intended.

What you can check

• The block itself: .forgejo/workflows/nix-deploy.yml, lines 151–153 — group: nix-deploy-$, cancel-in-progress: false — plus the job-level groups at deploy-mail, deploy-vpn and deploy-public.
• The gates on lines 456, 884 and 1601: the per-host contains(...) conditions, and always() && only on deploy-public.
• That a superseded run queues: dispatch the same workflow twice on one branch and read the second run — it shows waiting while the first stays running to completion.
• The host, not the run: its current generation and the timestamp on it.

For the scope job these deploys hang off — and why a green run can mean nothing was in scope at all — see the green deploy that deployed nothing.