Test, backtest, recent events, run history
Agent hint
Rules on this page that most often cause wasted work or wrong conclusions:
- Work in this order: validate, backtest, test, push.
forkline routine validatechecks the file,backtestreplays the last 7 days of stored events,testruns one event through every stage,pushsends the file. No test is required before publishing. - Test writes nothing: no task, no run record, no event, no forge write. Safe to run as often as you like.
- Backtest only covers the last 7 days of stored event summaries (at most 5,000 per owner). It does not simulate duplicates, concurrency, run limits or credits.
- Event summaries hold no bodies, comment text, command arguments or file lists. A
whencondition or a template that reads them is “not evaluated” (shown aswould_run_unverified) on a recent event. Paste the real payload to check it. - Reasons are sentences written by the server. Print them verbatim, never rebuild them from the stage or outcome.
- Read an outcome with its
stage: the stage says which rule decided. See Why didn’t my routine run?. - Run history is kept for 7 days. A run row exists for each event, routine and repository.
Four tools let you check a routine without waiting for real activity, and read what it did afterwards:
| Tool | Question | Writes anything? | Window |
|---|---|---|---|
| Recent events | What did my repositories send to Forkline? | no | last 7 days |
| Test | What would this routine do with this one event? | no | one event or payload |
| Backtest | How often would this routine have run? | no | last 7 days |
| Run history | What did this routine do, and why? | no | last 7 days |
Recent events
Recent events lists the events Forkline received from your repositories. Open it in the web app at /routines/events (the Recent events tab next to Routines), or use the CLI:
forkline routine events --repo acme/api --event pr.opened,pr.updated --since 3d --reacted none
forkline routine event 0000000e-0000-4000-8000-000000000003
Each row shows the event name, the title (the pull request or issue title, the comment command, the failed workflow names or the push summary), the number, the repository, the actor and the time. A row also shows the reactions: which routines considered the event and what each decided (up to 5 per row, newest first, with the total). “No routine reacted” means no routine considered the event, which is how you find events that nothing handles.
Opening one event (the side panel in the web app, forkline routine event <id>) shows its facts: the values you can turn into filters (event, repository, base branch, labels, author, draft, fork, bot sender, command), each with the filter it adds and whether it is checked by default, and all its reactions (up to 50).
Filters (web select, CLI flag, API query parameter):
| Filter | Web | CLI | Values |
|---|---|---|---|
| Repository | Repository select | --repo owner/name |
repository (exact full name, case-insensitive) or repository_id |
| Event | Event select | --event E[,E] |
pr.opened, pr.updated, pr.merged, pr.closed, issue.opened, comment.created, ci.failed, push |
| Routines | Routines select | --reacted any|none|some |
any (default), none (no routine considered it), some (at least one run record references it) |
| Window | --since Nd |
1 to 7 days (default 7) | |
| Source | event (forge webhook, default), schedule, webhook, manual: the last three exist only when a routine ran |
||
| Page size | Load more | --limit N |
API page size 1 to 200 (default 50). The CLI default is 50 and --limit is at most 1,000: it follows the paging cursor |
Retention. Event summaries are kept for 7 days, and each owner keeps at most 5,000: the oldest are deleted first. When that happens inside the window you are looking at, the response says older_dropped: true and the web app shows a note that older events were removed. A backtest then reports truncated: true.
Event summaries (up to 4 KiB of metadata, no bodies or comment text) are kept for 7 days, for every event of every connected repository. This happens whether or not you use routines.
A summary holds the event name, forge, repository, actor, numbers, titles, branches, labels, the comment author and command name, the CI conclusion and failed run names, and the push ref. It never holds pull request or issue bodies, comment text, command arguments, changed-file lists, CI logs or the webhook payload. See Events.
Create a routine from an event
The fastest way to a correct routine is to start from an event that should have triggered it.
In the web app
- Open Recent events, pick an event and read its facts under Use as filters.
- Check or uncheck facts. Some start checked (event, repository, base branch, and the draft, fork and bot guards when the event is not a draft, from a fork or from a bot); labels and authors start unchecked, because a label filter means “any of” and pre-checking every label makes a narrow filter that is easy to misread.
- Click Create a routine for events like this. The editor opens with a draft, and a Live backtest · last 7 days panel on the right that updates 600 ms after every change.
- Adjust, then Save draft or Publish….
Provenance tags. The editor tags the values the draft endpoint set. fromEvent marks a value set from a checked fact; suggested marks a value Forkline proposed without a fact (for example the draft, bots and self filters, and the starter prompt). A tag disappears when you change the field. Tags live only in the editor session: they are not stored in the file, and the YAML tab does not show them. Through the API the same information is provenance[], with a JSON pointer into the spec, a kind of fromEvent or suggested, the fact key and a note.
Guards that stay open. When the event is a draft, comes from a fork or came from a bot, that guard fact reads “yes” and starts unchecked. If you leave it unchecked, the draft sets the filter to include (tagged suggested, with the note Left open: this event is a draft, ... comes from a fork or ... came from a bot) instead of the default exclude, so the draft runs on the event it was made from. Check the fact to make the routine run only for such events (only).
Widen hint. When sibling repositories share a prefix, the editor offers “Widen to acme/api-*? 5 repositories, would have run 31 times in 7 days.” with Widen and Not now. Widen keeps the event’s repository and adds the pattern next to it. The repository count and the replay count cover the event’s repository plus everything the pattern matches, which is what applying the hint selects. A pattern that adds no repository beyond the event’s is not offered. The API returns at most 2 suggestions.
The event you started from is pinned first in the backtest list, marked “the event you started from”. If your edits make the routine skip it, the panel says Your routine no longer matches the event you started from: {reason}.
With the CLI
forkline routine new review-api-prs --from-event 0000000e-0000-4000-8000-000000000001 --fact event --fact repository --fact baseBranch
forkline routine new --from-event 0000000e-0000-4000-8000-000000000001 --widen --out routines/review.yaml
| Flag | Meaning |
|---|---|
[NAME] |
Routine name. Without it Forkline suggests one, for example review-acme-api-prs, unique among your routines (suffix -2, -3) |
--from-event <id> |
The recent event to start from (a UUID from forkline routine events) |
--fact KEY |
Apply this fact. Repeat it for several. Omitted means the event’s default-checked facts. Keys: event, repository, baseBranch, branch, headBranch, pr.number, label:<name>, author, draft, fork, bots, self, command, mention, ciWorkflow:<name> |
--widen |
Select the event’s repository and the first widen suggestion’s pattern |
--out PATH |
Where to write. Default .forkline/routines/<name>.yaml |
--force |
Overwrite an existing file. Without it an existing file is refused with exit code 6 |
An unknown fact key, or one that does not apply to the event, is a 422 at /facts/{i}. new writes a local file only; push sends it. The file starts with a comment header that records where each part came from:
# Created by `forkline routine new --from-event 0000000e-…-0001` on 2026-10-16.
# from event: /on/events/0, /repositories/include/0, /filters/branches/include/0
# suggested: /filters/draft, /filters/bots, /agent/prompt
# widen: acme/api-* (covers 2 repositories, would have run 7 times in 7 days): rerun with --widen
The paths are JSON pointers into the spec: the same pointers as in validation errors. The widen hints are also printed on stderr.
Test
A test runs one event, or one pasted payload, through every stage of a routine and tells you what would happen. It writes nothing: no task, no run record, no event, no forge write (a read-only forge lookup for the trigger policy is allowed). It checks the file you give it, so you can test unsaved edits.
Every filter is evaluated, not only until the first failure: the first failing stage is marked decisive, and you see the next blocker without running again. The prompt is rendered even when a filter fails, so you can read it.
Inputs
| Input | Web (test panel) | CLI | API source.type |
|---|---|---|---|
| A recent event | “Recent event” | --event <uuid> |
recent_event |
| A pasted payload | “Paste payload”: forge, event header, JSON | --event <file> --forge github|gitlab|forgejo --event-header <header> |
payload |
| A repository, for a schedule | “Repository (schedule)” | --repository owner/name |
repository |
| A webhook body, for a custom webhook | “Body (webhook)” | --repository owner/name --body <file> |
webhook_body |
- A recent event carries only the summary. Fields that are not stored (
pr.body,issue.body,comment.body,comment.command.args,push.files, CI logs) render in the prompt as‹pr.body: not stored for recent events›, and awhenthat reads them is not evaluated. Paste the payload to test it fully. - Pasted payload.
event_headeris the value of the header the forge sends. GitHub:X-GitHub-Event(pull_request,issue_comment,push). GitLab:X-Gitlab-Event(Merge Request Hook,Note Hook,Pipeline Hook,Push Hook,Issue Hook). Forgejo:X-Forgejo-Event(pull_request,issue_comment,push). The forge mapping shows which of them become which event. Find the payload in the webhook delivery log of your forge. It is used for this test only and not saved. The payload’s repository must be one of your registered repositories; an unregistered one is a422at/source/payload(“Repositoryx/yis not registered in Forkline.”), and a registered repository that the routine does not select makes therepositorystagefiltered. A payload that is not an event routines listen to is also a422at/source/payload. - A
ci.failedtest from one workflow payload lists one failure. In production, one event is created for each failed run. - Schedule and webhook routines have no event. Use
--repository(schedule) or--repositoryand--body(webhook). - Size limits. The payload or body is at most 5 MiB, otherwise
413(“Payload is 5.4 MiB; the limit is 5 MiB.”). The request is at most 6 MiB.testshares its 30 per minute with dry runs. A test refused before it is evaluated does not count toward the 30 (Webhook and Run now limits). While Forkline is busy, a test answers429Forkline is busy; retry shortlywithRetry-After: 5.
forkline routine test routines/review.yaml --event 0000000e-0000-4000-8000-000000000003
forkline routine test routines/review.yaml --event delivery.json --forge github --event-header pull_request
forkline routine test weekly-deps --repository acme/api
forkline routine test hook.yaml --repository acme/api --body body.json
forkline routine test review-prs --event 0000000e-0000-4000-8000-000000000003 --expect run --prompt-only
| Flag | Meaning |
|---|---|
<file|ref> |
An existing path is a routine file; anything else is a routine name, UUID or id prefix (tests the stored spec) |
--event <id|file> |
A UUID is a recent event; a path is a payload, and then --forge and --event-header are required (otherwise exit 2) |
--forge, --event-header <header>, --delivery-id <id> |
Describe a payload file: the forge, the event header (pull_request, Merge Request Hook) and, optionally, the delivery id |
--repository owner/name, --body FILE |
Schedule, or custom webhook |
--prompt-only |
Print only the rendered prompt |
--expect run|skip |
Exit code 10 when the result is not what you expect: run fails unless the outcome is would_run or would_run_unverified; skip fails when it is |
--json |
The full TestRoutineResponse on stdout |
An invalid routine file exits with 2 and prints the validation errors.
Stages and symbols
The result lists the stages that apply to the routine (filters you did not set are not listed), in evaluation order. Each line has a symbol:
| Symbol | Result | Meaning |
|---|---|---|
| ✓ | pass |
The stage allowed the event |
| ⊘ | filtered |
The stage blocked the event. Neutral: a filter that blocks is not an error |
| ✕ | error |
The stage could not be evaluated (a template or condition error, a failed policy check) |
| · | not_applicable or not_evaluated |
not_applicable: the event has no such thing. not_evaluated: the field is not stored in the summary |
The decisive stage (the first one that is filtered or error) is marked. Four things are never checked in a test and are listed under not_evaluated: event_key (duplicates), concurrency (active runs), run_limit and credits.
The text of a passing stage line, by stage:
| Stage | Result | Line |
|---|---|---|
event |
pass |
{event} is one of {events}. |
repository |
pass |
{repo} is selected. or {repo} is selected (matches {pattern}). |
self_guard |
pass |
exclude: {sender} is not Forkline.; include: Forkline's own activity is allowed.; only: {sender} is Forkline. |
bots |
pass |
exclude: {sender} is not a bot.; include: Bots are allowed.; only: {sender} is a bot. |
draft |
pass |
exclude: {target} is not a draft.; include: Drafts are allowed.; only: {target} is a draft. |
fork |
pass |
exclude: {target} is not from a fork.; include: Forks are allowed.; only: {target} is from a fork. |
branches |
pass |
{branchKind} {branch} is allowed. |
labels |
pass |
Labels {labels} pass the label filter. |
authors |
pass |
{authorKind} {author} is allowed. |
command |
pass |
The comment starts a line with {command}. or The comment mentions @forkline. |
paths |
pass |
{m} of {n} changed files match., or Paths not checked: the forge sent no complete file list. when the list is incomplete |
when |
pass |
The condition was true. |
trigger_policy |
pass |
{sender} is allowed by the trigger policy ({policy}). |
template |
pass |
The prompt renders ({bytes} bytes). |
| any stage | not_applicable |
Not applicable: {event} has no {sender|pull request|comment|push}. and, for events Forkline derives, Not applicable: {event} is raised by Forkline and has no sender. |
| any stage | not_evaluated |
{field} isn't stored for recent events, so {the condition|this filter} wasn't checked. |
| blocking result | filtered, error |
the matching reason from the reason table |
Predicted outcomes
| Predicted outcome | Meaning | Reason |
|---|---|---|
would_run |
Every stage passed | Would start a run for {target}. |
would_run_unverified |
Every stage that could be evaluated passed, but a when condition or filter needed a field that is not stored for recent events |
Would start a run for {target}, but {field} isn't stored for recent events, so {the condition|this filter} wasn't checked. |
filtered |
A filter blocked the event (or the event or repository does not match) | the decisive filter reason |
condition_error |
The when condition could not be evaluated |
The condition could not be evaluated: {message}. No run was started. |
template_error |
The prompt or the git ref could not be rendered | The {prompt|git ref} could not be rendered: {message}{ (line {line})}. No run was started. |
error |
Anything else, for example the trigger policy could not be checked | The run could not be started: {message}. |
The response also has the rendered_prompt (at most 64 KiB), the rendered_ref, and unverified_fields (for example ["pr.body"]).
Run now
With Run now you start a routine yourself, for real. It works for every trigger and is the counterpart of the webhook URL: connect another system with the webhook URL; start it yourself with Run now (web app, CLI or API). The same limits apply to both: concurrency, run limits, credits and the repository selection.
Only the owner can run a routine (others get 404), and only when it is published or disabled. A draft cannot be run for real, but it can be dry-run. A disabled routine runs from Run now; events, its schedule and its webhook don’t start it.
Dry run first, then Run now
In the web app, the Run drawer has two buttons. Dry run is a test of what Run now would start: it writes nothing and shows, per repository, the predicted outcome, the stages, the rendered prompt and an estimate (runs and the most credits they can cost). Run now starts the runs. The real run recomputes every check; it never trusts the dry run.
| Routine | Input | Starts |
|---|---|---|
| Webhook | A JSON payload of the same shape the webhook delivers, at most 5 MiB (default {}; the saved example is offered) |
One run per selected repository, with payload in the prompt and in when |
| Schedule | Nothing | One run per selected repository, as if the schedule fired now |
| Event | A recent event (event_id) |
A replay of that stored event, for real |
- Payload from a file. In the web app, Load from file next to the payload box reads a JSON file of up to 5 MiB. A file over 256 KiB shows as a summary with Edit as text and Clear instead of filling the box.
- Repositories. All selected repositories by default. The API and CLI can name a subset of the selection; anything else is refused with
422repository_not_selected. - Filters apply. If a
whenor other check blocks a repository, a real run is refused as a whole with422filters_not_matchedif any of its repositories is blocked, and nothing starts. With the default of all repositories, one blocked repository refuses the run. The CLI starts only the repositories a dry run says can run. - Event replay. The event must be a recent forge event (not a schedule, webhook or Run now row). Forkline keeps no bodies, comment text or file lists, so a replay is refused with
422event_fields_not_storedwhen the prompt,agent.ref,whenor apathsfilter readspr.body,issue.body,comment.body,comment.command.argsorpush.files(apathsfilter needspush.files).comment.command.argscounts only when the stored comment holds a command. - Event replay and aliases. The check follows the template however it reads the field, through an alias,
pr['body']ortojson. One case is not caught: a test on the text of a field through an alias of the whole object, such as{% set p = pr %}{% if "urgent" in p.body %}. A replay sees an empty body there, so read the field directly (pr.body). A dry run of such a replay lists the fields innot_stored, counts 0 runs, and the web app keeps Run now disabled.limits.maxRunsPerPrstill applies. The trigger policy is skipped, because you are the one asking. For a comment command the run records that. If the event’s repository is no longer registered in Forkline, the answer is422event_repository_unavailable. - A fresh run each time. A replay is not deduplicated against the original delivery. Send an
Idempotency-Keyto make a repeated request start nothing new. - Active runs. If a run for the same group is active, Run now skips it with outcome
skipped_active, even when the routine saysonConflict: replace. - Limits. 10 real runs per minute per owner, for all routines together (
429withRetry-Afterbeyond that). Dry runs count toward the 30 per minute shared with Test. A request refused before it is evaluated does not count (Webhook and Run now limits); a real run that its filters refuse (422filters_not_matchedorevent_fields_not_stored) counts. While Forkline is busy, a dry run answers429Forkline is busy; retry shortlywithRetry-After: 5, like a real run. At most 200 repositories per run, as in Repository selection (422fan_out_too_large). The payload is at most 5 MiB and the request at most 6 MiB. See Webhook and Run now limits. - Scope. An API token needs
routines:execute.
forkline routine run alert-triage --payload alert.json --dry-run
forkline routine run alert-triage --payload alert.json
forkline routine run weekly-deps --repository acme/api --yes
forkline routine run review-prs --event 0000000e-0000-4000-8000-000000000003
The CLI always does a dry run first, prints the table and the estimate, and asks before it starts anything; --yes skips the question. See CLI workflow. The API is POST /v1/routines/{id}/run.
Backtest
A backtest replays the stored event summaries of the last 1 to 7 days through the pure stages: event type, repository selection, filters, when and a template check. It tells you how often the routine would have run before you publish it. It does not simulate duplicates (event_key), concurrency, run limits or credits, and it only knows what the summaries hold.
forkline routine backtest routines/review.yaml
forkline routine backtest review-prs --since 3d --show skipped --limit 20
forkline routine backtest routines/review.yaml --fail-if-zero
| Flag | Meaning |
|---|---|
--since Nd |
Window, 1d to 7d (default 7d) |
--show would-run|skipped|all |
Which events to list (default all). The counts always cover every event |
--limit N |
At most N listed events (default 100, at most 500) |
--fail-if-zero |
Exit code 10 when nothing would run. Events: the sum of would_run and would_run_unverified is 0. Schedule: runs per week times repositories is 0. Webhook: no repository is selected. Use it in CI |
The summary line has this form:
Would run 9 (2 unverified) · skipped 31 · ≈ 9 sessions/week · up to 540 credits/week
followed by a table with the time, the event, the repository, number and title, the outcome and the reason of each listed event.
- Would run counts events that passed every stage the backtest evaluates. Skipped are events that matched the event types and repositories but failed a filter.
- Unverified (
would_run_unverified, shown as “Would run?”): the event would run, but awhencondition or filter needs a field the summary does not hold (for examplepr.body). Unverified runs count as would-run, and the line saysincl. {k} unverified. - Estimated sessions per week = (would run + would run unverified) × 7 ÷ days covered. The window can be shorter than the one you asked for when a repository was connected recently. For a schedule it is runs per week × repositories, and the response lists the next three run times; a custom webhook shows the cost per call and replays nothing.
- Estimated credits are a maximum:
ceil(sessions per week) × timeoutMinutes × size multiplier(see Run time, size and credits). - Group conflicts. The response has
group_conflicts(group,on_conflictandcount): the number of would-run events that came less thantimeoutMinutesafter an earlier would-run event of the same concurrency group. Withreplacethose would have replaced an older run, withskipthey would have been skipped. The web editor shows it as a line under the tiles. It is a rough hint: it does not simulatemaxActive, run limits or credits. - The footnote reads: “Based on stored event summaries; credits and concurrency aren’t simulated.”
In the web editor the backtest is the Live backtest · last 7 days panel: tiles for Would run, Skipped by filters and Runner sessions / week, and a list of events (All, Would run, Skipped).
Publish dialog warnings
Publish… shows the backtest, the limits and your balance, and warns when:
- the routine would have run 0 times in the last 7 days: “This routine would not have run on any event in the last 7 days. Check its filters, or publish anyway if you expect new activity.”;
- it would have run more than 50 times in a week, or the worst case is above half of your balance: “This routine would have run 214 times last week (up to 12,840 credits). Consider narrowing its repositories or filters.”;
- the balance is 0: “Your balance is empty, so runs will be refused until you add credits.”;
- Forkline’s built-in automations may run on the same events (pull request opened, CI failed, comment mention).
No test is required before publishing. The warnings are advice. From the CLI, forkline routine push --publish publishes after the files validate.
Run history
Every time an event reaches a routine that listens to it, Forkline writes a run record. The web app shows them on the routine’s Activity tab, the CLI with forkline routine runs, the API with GET /routines/{id}/runs.
forkline routine runs review-prs
forkline routine runs review-prs --outcome limited,errors --repo acme/api --pr 412 --since 3d --limit 100
forkline routine runs review-prs --outcome run_limit
| Flag | Meaning |
|---|---|
--outcome X[,Y] |
An outcome group (ran, skipped, limited, errors) or exact outcomes. An unknown value is a 400. skipped includes limited |
--repo owner/name |
Only this repository |
--pr N |
Only this pull request or issue number |
--since Nd |
1d to 7d (default 7d) |
--limit N |
Default 50, at most 1,000 (the CLI follows next_cursor) |
The output starts with the counts for the --since window (7 days by default): ran, skipped, limited, errors, for example Ran 4 · Skipped 0 · Limited 0 · Errors 0 (last 7 days). The counts are not narrowed by the other flags (--outcome, --repo, --pr). In the counts skipped does not include limited, while --outcome skipped returns both.
- One row per event, routine and repository. An event that matches 5 repositories of one routine gives 5 rows. A schedule or webhook routine that selects 6 repositories gives 6 rows for one firing; beyond 50 rows for one occurrence, the rest are grouped by outcome and stage with a count (
aggregated_count, and no repository). - Only events a routine considered appear. An event nothing listens to has no row; see it in Recent events.
- Trigger sources:
event(a forge webhook),schedule(on.schedule),webhook(POST /webhooks/custom/{id},on.webhook) andmanual(Run now). Amanualrow reads “Run now by you” in Activity: it records who started it and whether it came from the web app or an API token. A replay of an event also names the event it replays. - Retention: run records are kept for 7 days, and at most 10,000 per routine. Deleting a routine deletes its run records.
- Trace. Each row has a
trace: one entry per stage with thefieldit looked at, theobservedvalue (up to 200 characters), theruleand aresult:pass,filtered,error,not_applicableornot_evaluated.not_evaluatedappears only in tests and backtests. - A row that started a run has a
taskwith its status. If a newer run replaced it, the outcome readssupersededandtask.superseded_by_task_idnames the replacing task. - Overload rows.
dropped_overloadrows are written per routine, one every 60 seconds at most, with no event: the dropped events were never parsed, so the row has no event, repository or trigger summary.
Outcomes and stages
The outcome table
Every run record has an outcome, a stage (the rule that decided), a group (the chip in the web app and the value of --outcome) and a badge.
| Outcome | Stage | Group | Badge | task |
|---|---|---|---|---|
task_created |
submit |
ran |
Ran (green, check circle) | set |
superseded |
concurrency |
ran |
Replaced (blue, refresh) | set, with superseded_by_task_id |
filtered |
any filter stage, trigger_policy |
skipped |
Skipped (gray, minus) | null |
deduplicated |
event_key |
skipped |
Duplicate (gray, copy) | null |
skipped_active |
concurrency |
skipped |
Skipped (gray, pause) | null |
concurrency_limited |
concurrency (maxActive reached) or submit (the account’s pending-run cap) |
limited |
Limit (yellow, warning) | null |
run_limit |
run_limit |
limited |
Limit | null |
credit_exhausted |
credits |
limited |
Limit | null |
fan_out_refused |
fan_out |
limited |
Limit | null |
dropped_overload |
overload |
limited |
Limit | null |
template_error |
template |
errors |
Error (red, error) | null |
condition_error |
when |
errors |
Error | null |
error |
any | errors |
Error | null |
would_run |
submit |
(test, backtest) | Would run (green, check circle) | |
would_run_unverified |
the unverified stage | (test, backtest) | Would run? (green outline, help circle) |
supersededis derived on read: Forkline never stores it. A run stored astask_createdis returned assuperseded(stageconcurrency) once a newer run replaced its task, and itstask.superseded_by_task_idis the replacing task. The list, the counts (groupran) and the filters agree:--outcome supersededreturns the replaced runs,--outcome task_createdthe others, and--outcome ranboth. A routine’slast_run(inroutine listandview) keeps the stored outcome, so it still saystask_createdfor a replaced run.dropped_overloadis a real outcome in the list. When Forkline is overloaded it drops events before they are evaluated and writes one row per routine and 60 seconds (stageoverload, grouplimited). Filter them with--outcome dropped_overloador--outcome limited.concurrency_limitedhas two meanings, told apart by the stage. Stageconcurrency: the routine reachedmaxActive, no task exists. Stagesubmit: the account’s pending-run cap was reached, no task was created.- In the chips, Skipped includes the limit outcomes (shown with the yellow Limit badge); Ran includes replaced runs; Errors is
template_error,condition_erroranderror. filteredis also used for acomment.createdactor the trigger policy refuses (stagetrigger_policy).
The stages, in evaluation order: overload, fan_out, event, repository, self_guard, bots, draft, fork, branches, labels, authors, command, paths, when, trigger_policy, template, event_key, concurrency, run_limit, credits, submit. event and repository appear only in tests and backtests. See Filters for what each stage checks.
Rendering rules for reasons
- A reason is one sentence (two when “No run was started.” is added). It has no “Skipped:” prefix and is at most 500 characters. Only the server writes reasons. The web app and the CLI print them verbatim.
- Observed values, names, branches, labels, logins, patterns, events and repositories are in backticks and are truncated to 80 characters with
…. - A list
{list}with 1 item shows`main`; with 2 to 5 items[`main`, `release/*`]; with more than 5 the first 5 and+N more.{inList}is`main`for one item andin [`main`, `release/*`]for several. - Placeholders:
{target}:PR #412,issue #88, the push tomain(push), oracme/api(schedule, webhook, Run now).{groupTarget}:PR #412,issue #88,acme/api, orthis routine.{repo}:acme/api.{sender}:renovate[bot].{branchKind}:Base branch(pull request events),Branch(push),Head branch(ci.failed).{authorKind}:Author(pull request, issue),Comment author,Pusher.{forge}:GitHub,GitLab,Forgejo.{time}:14:03 UTCwhen on the same UTC day as the run, otherwise2026-10-15 14:03 UTC. The text is always UTC; the web app shows these times in your time zone and keeps the UTC text as a tooltip, while the CLI and the API show it as written.{n},{max},{count}: plain integers with thousands separators.
- A suffix is appended with a single space.
Reasons
Every reason template, by outcome and stage. The first rows are run records. The two rows that start with would_run are predicted outcomes (test and backtest), and the last row is a suffix.
| Outcome, stage, variant | Reason template |
|---|---|
task_created, submit |
Started a run for {target}. |
superseded, concurrency |
A newer run for {target} replaced this one at {time}. |
filtered, self_guard, self: exclude |
Caused by Forkline ({sender}); this routine skips Forkline's own activity. |
filtered, self_guard, self: only |
Not caused by Forkline; this routine only runs on Forkline's own activity. |
filtered, bots, exclude |
{sender} is a bot; bots are skipped. |
filtered, bots, only |
{sender} is not a bot; this routine only runs on bot events. |
filtered, draft, exclude |
Draft PRs are skipped. |
filtered, draft, only |
{target} is not a draft; this routine only runs on drafts. |
filtered, fork, exclude |
{target} comes from a fork; forks are skipped. |
filtered, fork, only |
{target} is not from a fork; this routine only runs on forks. |
filtered, branches, include |
{branchKind} {branch} is not {inList}. |
filtered, branches, exclude |
{branchKind} {branch} matches the exclusion {pattern}. |
filtered, labels, include |
{target} has none of the labels {list} (has: {labels}). ({labels} is a list or no labels) |
filtered, labels, exclude |
Label {label} is excluded. (the first excluded label present) |
filtered, authors, include |
{authorKind} {author} is not {inList}. |
filtered, authors, exclude |
{authorKind} {author} is excluded. |
filtered, command, names |
The comment has no {commands} command at the start of a line. ({commands} is /fix-ci, or /a or /b) |
filtered, command, mention |
The comment doesn't mention @forkline. |
filtered, command, names and mention |
The comment has no {commands} command at the start of a line and doesn't mention @forkline. |
filtered, paths, include |
None of the {n} changed files match {list}. |
filtered, paths, exclude |
All {n} changed files match the exclusions {list}. |
filtered, paths, include and exclude |
None of the {n} changed files match {include} outside the exclusions {exclude}. |
filtered, when |
The condition was false: {condition}. ({condition} is the source in backticks, at most 80 characters) |
filtered, trigger_policy |
{sender} isn't allowed to trigger routines on {repo} (trigger policy: {policy}). ({policy} is anyone, owner_only, allowlist or collaborators) |
filtered, event (test, backtest) |
This routine runs on {events}, not {event}. |
filtered, repository (test, backtest) |
{repo} is not in this routine's repositories. |
deduplicated, event_key, forge |
Already handled: {forge} delivered this event again (delivery {delivery}). ({delivery} is the id in backticks, at most 12 characters plus …) |
deduplicated, event_key, schedule |
Already handled: the run scheduled for {time} was already started. |
deduplicated, event_key, webhook |
Already handled: a call with the same idempotency key ran before. |
deduplicated, event_key, manual |
Already handled: a run with this idempotency key was already started. |
skipped_active, concurrency |
A run for {groupTarget} is still active; this routine skips new events while one runs. (for Run now: …; Run now skips while one runs.) |
concurrency_limited, concurrency |
{n} runs of this routine were already active (limit {max}). This event was not run. |
concurrency_limited, submit |
Your account already has {n} pending runs (limit {max}), so this run was not started. |
run_limit, run_limit |
PR #{number} already had {count} runs from this routine (limit {max} per pull request). (1 run when the count is 1) |
credit_exhausted, credits |
Your credit balance is empty. Add credits to resume. |
fan_out_refused, fan_out |
This {schedule|webhook} now matches {n} repositories, above the limit of 200. Narrow the repository selection. |
template_error, template |
The {prompt|git ref} could not be rendered: {message}{ (line {line})}. No run was started. |
condition_error, when |
The condition could not be evaluated: {message}. No run was started. |
dropped_overload, overload |
Forkline was overloaded and dropped {n} events for this routine between {t1} and {t2}. |
error, trigger_policy |
The trigger policy for {repo} could not be checked: {message}. No run was started. |
error, any other |
The run could not be started: {message}. |
would_run, submit |
Would start a run for {target}. |
would_run_unverified, the unverified stage |
Would start a run for {target}, but {field} isn't stored for recent events, so {the condition|this filter} wasn't checked. |
suffix on task_created and would_run when the paths stage passed on an incomplete list |
Paths not checked: the forge sent no complete file list. |
The predicted outcomes filtered, condition_error, template_error and error use the same templates as the run records of that outcome and stage.
Why didn’t my routine run?
Open the routine’s Activity tab (or forkline routine runs <routine>), find the event, and read the outcome and stage. If there is no row at all, the event never reached the routine: see the first line.
| Symptom | Outcome and stage | What to do |
|---|---|---|
| No row, and the event is not in Recent events | none | The forge did not send a webhook for it, or the webhook is not set up. Check the repository’s webhook delivery log. Actions Forkline does not map (labeling, assignment, review requests, cancelled CI) are dropped silently, see Events |
| No row, but the event is in Recent events with “no routine reacted” | none | The routine does not list the event name in on.events, does not select the repository, or is a draft or disabled. Run forkline routine test <routine> --event <id>: stage event or repository is filtered |
filtered at self_guard, bots, draft or fork |
filtered |
The default filters skip Forkline’s own activity, bots, drafts and forks. Set the filter to include or only |
filtered at branches, labels, authors, command, paths |
filtered |
The event does not match the filter: read the reason and fix the filter or the event |
filtered at when |
filtered |
The condition was false |
filtered at trigger_policy |
filtered |
The person who commented is not allowed by the repository’s trigger policy |
condition_error at when |
condition_error |
The when expression failed at run time. Test with the payload to see the message |
template_error at template |
template_error |
The prompt or agent.ref could not be rendered. Test the event and read the line number |
deduplicated at event_key |
deduplicated |
The forge delivered the same event twice, or an Idempotency-Key was reused |
skipped_active at concurrency |
skipped_active |
A run for the same group is still active and onConflict is skip. Applies to events, schedules, Run now and custom webhooks. Run now always skips, whatever onConflict says. For a Run now, wait for the active run. For events, schedules and webhooks, replace also works |
| Run shows “Replaced” | superseded at concurrency |
A newer event for the same pull request replaced it. Set onConflict: skip to keep it |
concurrency_limited at concurrency |
concurrency_limited |
The routine reached maxActive. The event is not queued. Raise maxActive or run it again later |
concurrency_limited at submit |
concurrency_limited |
Your account has too many pending runs: no task was created, and with replace the older run was not cancelled. Wait for earlier runs to start or finish |
run_limit at run_limit |
run_limit |
The pull request reached limits.maxRunsPerPr (default 3), counting replaced runs. Raise the limit or open a new pull request |
credit_exhausted at credits |
credit_exhausted |
Your balance is empty. Add credits |
fan_out_refused at fan_out |
fan_out_refused |
A schedule or webhook now matches more than 200 repositories. Narrow the selection |
dropped_overload at overload |
dropped_overload |
Forkline was overloaded. Nothing to fix in the routine |
error at any stage |
error |
See the message. |
| A schedule did not fire | row with source schedule, or none |
Check forkline routine backtest <routine> for the next three run times, and that the routine is published and enabled |
| Backtest says 0 but you expected runs | none | Window is 7 days of stored summaries. A filter that reads a field that is not stored (pr.body) shows as unverified, not as 0. Check filtered_by_stage with --json |