DACIP DACIP

August 16, 2026 · DACIP blog

I shipped a checker that found nothing. That was the bug.

I wrote a new analyzer, tested it, and ran it against five public repositories. It reported zero problems in all five.

I nearly shipped that as a result. It was not a result. The analyzer had never run — not once, in any of them — and a checker that never runs is indistinguishable from a codebase with nothing wrong.

That is the failure mode I want to describe, because I do not think it is specific to me or to my tool. Every one of us has shipped a test that asserted nothing, a lint rule whose pattern never matched, an alert with a threshold no real incident could cross. They all pass. They all look like good news.

What I was building

DACIP checks contracts that a type checker structurally cannot see. The newest one is Prisma drift: your schema.prisma declares a model field, and your migrations are supposed to create the matching column. Prisma generates its client from the schema, so a field you forgot to migrate type-checks perfectly and then fails against the real database at runtime. Nothing in tsc can catch that, because from the compiler's point of view everything agrees.

So: parse the schema, replay prisma/migrations/*/migration.sql at the column-name level, diff the two. No database, no prisma CLI, no execution.

I built it test-first, with fixtures for the cases I could think of, and validated it on a real application I work on daily. Thirty-five models, fourteen migrations, zero drift. Correct — that app is migrated properly.

Then I pinned five public Next.js repositories to run it against something I had not written: cal.com, inbox-zero, rallly, documenso, papermark. Between them, 1,272 real migrations.

Zero findings. All five.

Zero findings is two different results

I sat with that number for a minute, and the thing that bothered me was not that it seemed too good. It was that I could not tell which of two things it meant:

  • every one of those repos is correctly migrated, or
  • my analyzer did not look.

The output was byte-for-byte identical in both cases. There is no field in the report that distinguishes them, and there was no reason for me to suspect the second — the tests passed, the reference app gave the right answer, and clean results are exactly what you expect from mature, well-maintained projects.

So I checked. analyze_prisma looked for the schema at <repo>/prisma/schema.prisma.

inbox-zero   apps/web/prisma/schema.prisma
rallly       packages/database/prisma/schema.prisma
cal.com      packages/prisma/schema.prisma
documenso    packages/prisma/schema.prisma
papermark    prisma/schema/schema.prisma

Not one of them keeps it there. They are monorepos; the schema lives under apps/ or packages/. My analyzer found no schema, returned an empty list, and the empty list rendered as a clean bill of health. Five times.

The reference app I had validated against was the outlier — the only one with a root-level prisma/ directory.

Then it ran, and it was wrong

Fixing discovery was easy. What discovery uncovered was not.

With the analyzer actually reading those 1,272 migrations, it produced six defect-grade findings on public repositories — the highest severity it can assert. Every one of them was false.

Five came from documenso, all claiming a table's columns were unmigrated. The table is created right there in the migration history. What documenso does is drop the table and recreate it in the same file:

DROP TABLE IF EXISTS "PasswordResetToken" CASCADE;

-- CreateTable
CREATE TABLE "PasswordResetToken" ( ... );

My replay applied every CREATE TABLE in a file, then every ALTER, then every DROP. Grouped by kind, not in the order the statements actually appear. So the drop ran last and erased the table the same file had just recreated. A migration is a sequence, and I had been treating it as three buckets.

The sixth was the identical bug one level down, inside a single statement:

ALTER TABLE "Payment" DROP COLUMN "processorType",
ADD COLUMN     "processorType" "ProcessorType" NOT NULL DEFAULT 'LEMON_SQUEEZY';

That is how Prisma changes a column's type. I applied all the ADDs, then all the DROPs, and deleted a column the statement had just re-added.

There was more, and none of it was exotic — it was just real SQL rather than the SQL I had imagined. Schema-qualified identifiers ("public"."Payment"). PL/pgSQL function bodies, where splitting on ; shreds a trigger into END, RETURN NEW, END IF fragments that match nothing. A Postgres exclusion constraint whose commas live inside parentheses. /* Warnings: */ blocks that Prisma emits ahead of destructive migrations.

Unparsed statements across those 1,272 migrations went from 874 to 1. The one that remains is a DO $$ block containing dynamic DDL, and it correctly suppresses the check for that repo, because DDL built at runtime is not something I can prove anything about.

Worth naming plainly: the first version of this analyzer would have been useless in production and looked flawless in every report it produced.

The check I should have had

The fix is not "test more." I had tests. They passed. They were fixtures clean enough that they never reached the code path that was broken — a two-line schema with one migration never encounters a block comment, a data backfill, or an exclusion constraint, so it can never tell you that those constructs make your analyzer give up.

What I write now, for anything permitted to stay silent, is a negative control: the same realistic input, twice.

  1. Feed it input containing every messy construct a real repository actually ships. Assert it reports nothing — and specifically assert it did not suppress itself.
  2. Inject one known defect into that same input. Assert exactly one finding, correctly attributed.

Both halves matter, and the second is the one I was missing. Silence only means something once you have watched that exact input produce a finding.

I also verified the guards by mutation, because a test that has never been seen to fail is the same trap as a checker that has never been seen to fire. I reverted each fix in turn and confirmed the new test went red, then restored it. Two of them I would otherwise have believed on faith.

What this does to "zero false positives"

I have made a zero-false-positive claim on this site, and I still stand behind it: every asserted defect has been hand-verified against source.

But a reader called Wren Calloway made a point about it on my last post that I have come around to completely. Zero false positives is half a claim. The other half is how much of your call surface the tool can actually parse — because a contract break it cannot see sails through green CI no matter how clean the findings look. Their framing was that the true-negative rate is entirely a function of coverage, and that the coverage number is doing more load-bearing work than the defect count.

I had a caveat about this. I did not have a headline about it. This episode is the strongest argument for their version that I could have produced, and I produced it by accident: coverage was not merely load-bearing, it was silently zero, and nothing in the output would have told you.

So the copy on this site changed. Every place the zero-false-positive claim appears now carries what bounds it. The proof dashboard publishes per-repo coverage alongside the findings, and the coverage docs explain what each number does and does not mean. If DACIP cannot see your frontend's HTTP client, I would rather you learn that from the report than from an outage.

Try to break it

The five repositories in this post are pinned by commit on the proof dashboard, re-scanned weekly, every scan run twice and byte-compared in public. The raw data is at /proof/data.json. If two runs ever differ, the row turns red where everyone can see it.

The more useful thing you can do is not to check my findings. It is to check my silence. Point DACIP at a repository, then look at the coverage block before the findings block:

uvx dacip scan | head -40

If symbols is zero, or calls.dynamic swamps calls.literal, or action_links is empty on an app you know is full of forms — that is the tool telling you it did not look, and I would genuinely like to hear about it.

A clean report should be something you can trust. It only earns that when the alternative explanation has been ruled out.