One team's fan-out is one team's queue.
Dagweave holds runs back until there's room, and each waiting run shows its place in line.
- 0
- Workflow objects on your cluster while a run waits
Whoever submits first gets the cluster.
One team's fan-out expands into more pods than the cluster has room for, and every other team's steps sit pending behind it.
A semaphore lives in the manifest, written by whoever wrote the workflow. Nothing in there knows one team's work matters more than another's.
Held before it reaches the cluster
Each team has a limit on how many runs it can have going at once. Past that, new runs wait in Dagweave, and your cluster doesn't see them until there's room.
- The queue survives a Dagweave restart, in the same order.
- A run held for an hour goes to whichever of your clusters can take it then.
- We can lift the limit for a team whose runs all need to start straight away.
A waiting run says what is in front of it
It shows how many of your team's runs are ahead of it, and how long it's been waiting. Argo doesn't publish an order for runs waiting on a semaphore, and Dagweave never invents one.
- Waiting for room and waiting on a lock are two waits, shown in the order they happened.
- A run that has not gone out can be cancelled. Nothing on the cluster has to be stopped.
The run takes the highest choice that is open
Rank the clusters you would accept when you submit. The run goes to the highest of them that can take it when its turn comes, and to nothing you did not rank.
- Beside each cluster: how many runs there are holding a pod no node will take.
- A cluster passed over for load shows the median pod wait there, against the median where the run went.
- A run none of its ranked clusters can take waits, and says which ones and why.
What waiting cannot fix
If a cluster is offline or full, the run waits for it. If you aren't allowed to run on a cluster at all, you're told when you press Run.
- A cluster that cannot take this run stays on the list, with the reason.
The controls in the workflow still hold
Parallelism, priority, semaphores and mutexes are fields in the editor, and they end up in your YAML. The queue decides when a run is handed to Argo Workflows.
- A semaphore in one workflow knows nothing about another team's work.
- Nothing is reserved, so an unused limit costs the cluster nothing.
A team at its ceiling waits behind itself
The limit only counts a team's own runs. If one team fills up, the next team's runs don't wait behind it.
- A semaphore can't tell two teams apart. The queue can.
- Within a team, it's oldest first.
Stop one fan-out taking the cluster.
Dagweave is in early access. Leave an email and we will bring you on as we open it up.