They share one memory, so nothing is said twice
Three layers: directive, orchestration, execution
A crawl, an issue taxonomy and a priority model, wired end to end. It produces the same artifacts every run — which is the only reason the numbers are comparable month to month.
The SOP, written down. Goals, inputs, the tools to use, the outputs and the edge cases — in plain language, versioned like code. When we learn an API limit the hard way it gets written here, so it is never learned twice.
The only probabilistic layer. It reads the directive, calls the tools in order, handles the errors and asks when something is genuinely ambiguous. It does not do the work itself.
Deterministic scripts. Same input, same output, every run. Ninety per cent accuracy per step is fifty-nine per cent over five, so anything that must be right lives down here instead.
The rendered-HTML step is the one that earns its place. Comparing raw against rendered is how you catch a page whose canonical, title or primary copy only exists after JavaScript runs — invisible to a crawler, and invisible to any audit that only looks at the finished page.
Read-only wherever read-only is enough. Nothing gets write access it doesn't need, and every write is logged.
The crawl returned more than zero URLs, and the URL count is within range of the sitemap.
The issues export is present and parsed — an empty taxonomy fails the run rather than producing a clean-looking report.
Every artifact uploaded and confirmed. A partial run is a failed run.
If a gate fails, the pipeline halts and files the error. It does not retry blindly, and it does not hand you a report built on half a run.
| Spec | Standard | Deployable |
|---|---|---|
| Where it runs | Our infrastructure | Yours |
| Who holds the API keys | Us, scoped per client | You |
| Cadence | Monthly, or weekly | Any schedule |
| Raw data retention | Rebuilt each run | Your policy |
| Artifact delivery | Your Drive and tracker | Your choice |
| Source code access | — | Full |
| Runs unattended | Yes | Yes |
| Failure alerting | Slack and email | Your channels |
Deployable hands over the software, the hosting and the keys. You run it; we stay on for support.