Social-Sourcing Into Docket: The Drip That Ships
Research output dies in folders when the pipe is a dump. The fix is a drip — claim, lookup, heartbeat, MaskGuard on the dock. Same company, two surfaces, one integrity bar.
Most research pipelines die in their folders. A two-week research sprint runs, lands in a one-shot report, then that report lives in a project folder where an operator reads it once, ships a decision, and never looks at it again. The next research sprint overwrites the previous one. The operator's records drift back to stale. The research team's value-add becomes a printout, and the lawyer working the docket carries the rough equivalent of that printout in a different tool.
social-sourcing → Docket, the way it ships today, is the version of that pipeline that doesn't end at the printout. It's a drip — same company, two surfaces, one integrity bar — and the bar is what makes the pipe.
Why folder dumps become folders
There are three failure modes that every research product hits eventually if the pipe stops at the report.
One-shot, no carryover. A report goes out. The operator reads it. Six weeks later, the underlying facts have shifted and the operator can't get the report back without running another two-week sprint. The cost of one insight becomes the cost of the next.
No operator interface. The research product interface was a memo, not a record. The operator doesn't have a place to query, refresh, or revise. The next person to do the same operator work re-runs everything.
No privacy posture. A folder dump can collect asterisks (B****) or partial values. If a downstream tool picks it up, asterisks become leak surface. The dossier is now a privacy event, regardless of how well the research itself was done.
These are the failure modes that were killing our shipments before we built the drip. They're also the failure modes a folder dump keeps killing if the pipe stops at the deliverable.
The drip as product shape
The pipe we ship is small. On the research lane we have social-sourcing pulling candidate and enrichment signals from public social sources — the same lane that's been running pilots for the past two quarters. On the Docket side we have the drip queue with the same claim→lookup→heartbeat shape as the rest of Docket's enrichment store.
The linker is social-sourcing/tools/bv/enrich_docket.py. The script walks a series of candidate records the research lane has produced. Each candidate becomes a Docket request. The request hits BeenVerified through our subscription session, gets through Cloudflare where FlareSolverr is wired in for the gate, and returns to us either with full values or with no values at all — courtesy of MaskGuard on the Docket side rejecting asterisks by hard rule. The script writes a checkpoint into jsonl so it can pick up where it left off after a session expiry or a flaky connection. There is no nightly batch. There is no folder.
That is the short version. The discipline is: every fact that lands in Docket is a candidate this script claimed, looked up, and either heartbeated back as a usable record or rejected as a no-go. There is no path through which an asterisk makes it into a fact value. There is no path through which the script ships a partial row. The pipe has the same integrity bar as the Docket wire, because the script's exit gate is MaskGuard.
What this means for the operator's day
For an operator running a docket with a list of leads, the pipe replaces the folder step that used to ship with each research sprint.
The shape of the operator's day changes like this. Instead of opening a fresh report, the operator opens Docket and the contact records are already there. They include a bv_enriched flag, a refresh timestamp, and any confirmable field that BV returned — full name, address-of-record, public-record classes that hit. They do not include asterisks. They do not include anything MaskGuard rejected. If a record came back no-go, the row is there with a bv_no_go flag and the operator understands that the upstream returned masked and we won't write a partial fact.
That's the bare-bones interface today. The operator dashboard layer is in flight. The pipe is what's shipped.
The thing that's invisible to the operator but matters to us: the link prefers Docket's API over the venerable pilot.db the old research product used for storage. The reason is that you cannot run MaskGuard at the table level of a sqlite database — you can run it at the API wire, and that's where the constraint lives. The pipe pushes everything through the wire, not the file. This is not a small difference. This is the difference between the folder dump and the drip.
How this pairs with Docket MVP and MaskGuard
This post is what happens when the social-sourcing research lane and Docket stop being two separate surfaces and become the same surface on the docket side. The integrity bar — MaskGuard hard-rejects partial values, claim/heartbeat drips, API over file — was set in the Docket MVP and is the same bar the research lane now ships into.
If you read our Docket MVP post (or didn't, and only caught this one), the through-line is the same: the wire is the constraint. The pipe inherits the constraint because the pipe ends at the wire. A folder dump cannot. A research report cannot. A weekly digest cannot.
The structural change is small enough to ship and durable enough to compound. We're not going to claim the operator workflow is finished — the audit log and the dashboard layer are pending. We are claiming the pipe lands in the same shape every time: candidate, claim, lookup, heartbeat, MaskGuard. Researcher stops watching their work die in folders. Operator gets the records. Wire keeps the records honest.
What's still honest
There are limits I won't pretend around. BV subscription session health is the system's heartbeat — if a session expires mid-run, the script picks up at the most recent checkpoint and resumes, but it doesn't retry rows the prior session had already no-go'd. We trade a few rows of completeness for a smaller blast radius on session replay. The Cloudflare gate via FlareSolverr is wired but adds a latency tax; if a session times out at the gate, the candidate stays in the queue for the next run. The operator audit log is internal today and will become a product-grade surface in a forthcoming sprint.
We're not claiming a complete research-to-product pipeline. We're claiming one pipe that ships the way it ships: candidate from the research lane, drip via enrich_docket.py, MaskGuard at the dock, never an asterisk-row, never a folder dump. The pipe is what shows up at the wire. The wire is what we shipped.
If the next sprint is the one where the research lane gets the dashboard layer that shows what's queued, what's in flight, and what's no-go — and where the operator can re-claim a no-go row through a documented operator path — that's the sprint that takes this pipe from "shipping" to "shippable." We're aiming for that sprint.