Determinism
DACIP's core promise is narrow and checkable: run the same analysis on the same commit twice and every artifact it writes — findings, observations, coverage, diagnostics, report, evidence, auth matrix, repro scripts — comes out byte-identical. There is no LLM anywhere in the analysis pipeline. The 22 analyzers parse source, extract facts, and compare them. No sampling, no temperature, no model version that drifts under you.
Why this matters for a CI gate
A review tool that can disagree with itself can't gate a merge. Determinism turns three chronic problems into non-problems:
| Problem | With a deterministic engine |
|---|---|
| Flaky reviews | A re-run never changes the verdict. Red means the diff, not the dice — retrying the job is never the fix. |
| Disputes | Check out the same two commits, run the same command, and you reproduce the verdict byte for byte. An argument about a finding is settled by a re-run, not by screenshots. |
| Upgrades | Every verdict change has a nameable cause: your commit or the engine version. Nightly proof runs record the engine version and commit alongside every result, so a post-upgrade verdict change is attributable, never silent noise. |
In practice the gate is dacip diff <base-ref> — exit code 1 when the diff introduces findings, 0 when clean, with --format github or --format sarif for annotations.
The nightly public proof
The claim is re-tested every night, in public, on the proof dashboard:
- Nine production-scale OSS repos (Sentry, PostHog, Zulip, Superset, NetBox, Redash, Flagsmith, Mathesar, Label Studio) are shallow-fetched at pinned commits into clean workdirs.
dacip investigate "api contract audit"runs twice per repo, each time from a freshly wiped.dacipstore.- Every artifact of each run is SHA-256 hashed — path and content, sorted — and the two hashes are compared.
- The raw output, including both hashes per repo, is published as
data.json. The job exits non-zero if any pair is not byte-identical, so CI turns red exactly when the public claim would be false.
The runner never uploads the audited source anywhere; only counts, hashes, and durations leave the machine. You can reproduce a row yourself:
git fetch --depth 1 origin <pinned-sha> && git checkout FETCH_HEAD
uvx dacip investigate "api contract audit"
# run again — same store wiped, same commit — then:
diff -r run1/.dacip/runs run2/.dacip/runs # → empty
What determinism does not claim
Honest ceilings, so you know what you're buying:
- Not completeness. Deterministic means repeatable, not all-seeing. Routes registered dynamically at runtime are visible only to an interpreter executing the code; DACIP parses, it doesn't execute. Dynamic routes and unsupported frameworks are counted and labelled in every run's diagnostics — DACIP tells you what it can't see instead of guessing.
- Not cross-machine hash equality. Byte-identity holds for repeated runs on the same checkout. Artifact bytes embed local paths — for example, a finding's repro verification command references the absolute path of its generated pytest file — so the same commit scanned on two machines produces equal verdicts but different artifact hashes.
- Not zero false positives. That's a separate, falsifiable claim backed by hand-verified, upstream-disclosed findings on the proof dashboard — dispute any asserted defect and it gets corrected in public.
See also
- Quickstart — install and run your first scan
- Proof dashboard — tonight's numbers, raw
data.json, pinned commits - FAQ — pricing, platforms, and what runs offline