A single-tenant AI operating system for one NYC real estate broker. Built solo in roughly one week, without an engineering background.
- Built in
- ~1 week, solo
- Architecture
- Ten rules every feature obeys
- Status
- Working system, single tenant
Work
Five software systems, each built solo by first learning the domain well enough to understand the problem, then designing and shipping the system around it. Every one is documented as the problem, the operating insight, the system, the decisions behind it, the result and its limits, with the architecture open for anyone who wants to read it.
Every entry follows the same path: the problem, the operating insight, the system, the key decisions, the result, its limits, and then the technical detail behind the fold.
- 01Assemblage OS
Turn fragmented broker knowledge into an operating system that says what to do today
Case studyprivate - 02Impossible To Name
Turn a document-heavy government process into a production AI workflow
Case studyimpossibletoname.com - 03The skills library
Systematize repeatable operator work and remove it from the daily attention loop
Case studyprivate - 04The Screening Simulator
Reverse-engineer an AI decision system and stress-test applications against it
Case studyprivate - 05The Creator Deal Desk
Automate recurring founder work through a controlled multi-agent workflow
Case studyprivate
How to read the diagrams
- a component that was built
- an external system, or a person
- where a language model operates
- data flow
- control, decision
- optional, conditional
- leak point
01
Assemblage OS
Single-tenant operating system · NYC real estate
The problem
The broker's day before software.
Calls
Texts
Email
Voice notes
Listings / MLSproperty truth, separate system
Public recordsproperty truth, separate system
Board packetsproperty truth, separate system
Board approvals · school-zone datesdate-critical, tracked by hand
Client contextlives in the broker's head
The brokerone person, every channel
- Context lost between conversations.
- Follow-ups that never happen.
- No memory of what a client wants, or why.
Operating insight
The broker's product is relationship knowledge, but every tool in the category stores transactions. And real estate decisions do not happen on a schedule: they happen at life transition moments.
Explicitly not a CRM. The stated goal is that the system answers one question every day: what should I do right now, and why.
The system
Every input becomes an immutable event, events become memory, memory produces signals, signals produce a recommendation with its evidence attached, and the recommendation becomes a one-tap action. The model touches the two ends, never the middle.
Touchpointraw text or transcript
Extractionsmall model
Person recordextracted facts merge in
Pressure-signal analysispure function, no model
Person scoringpure function, no model
Task-rule tabledeclarative rules, deduplicated tasks
Today dashboardone-tap actions
Narrationsummaries, drafts, meeting prep
Outcome loggedfeedback, not discarded
Key decisions
- Decision
Scoring uses no language model
- Why
- A score that cannot be explained in one sentence violates the explainability rule, and a model's score cannot be audited after the fact.
- Result
- Intent, urgency, engagement, quality and recency each return their own score and reasons, persisted per dimension. Silence decays a lead.
- Limit
- The score only sees signals the system has already captured. A pressure no detector encodes does not exist to it.
- Decision
Ambiguous matches go to a human queue
- Why
- False positives corrupt memory permanently. Missing links are cheap to fix.
- Result
- High confidence auto-links, the middle band waits for approval, low confidence is discarded.
- Limit
- The queue moves only as fast as the one person clearing it.
- Decision
A deterministic formatter overrides the model for building facts
- Why
- Building facts are too important to let a model paraphrase.
- Result
- Facts render through tested formatting code, and the model narrates around them.
- Limit
- Every new fact type needs its own formatting code before it can appear.
Result
A working system with a specification of roughly 3,400 lines behind it, built solo in about one week. It answers one question every day: what should I do right now, and why.
Limits
What I know
- Ten rules hold across every feature, and a feature is not done until it flows from event to one-tap action.
- Scoring is deterministic and every score carries its reasons.
What I don't know
- How the detectors written ahead of their data behave on real rows, because their sources have none yet.
- Whether the confidence ladder's bands sit in the right place, because the corrections that would tune them come from a single tenant.
What I would test next
- Turn on a dormant source and watch what its detector fires.
- Compare the stored dismissals and corrections against the scores that produced them.
Architecture
The architecture contract: ten rules every feature obeys.
- 01
Event-first. Every input becomes an immutable record before anything updates.
- 02
Two memories. Structured queryable facts alongside a narrative summary per person. Raw logs are never shown without a synthesized summary.
- 03
Signals to Decisions to Actions. No recommendation may exist without the stored signal that produced it.
- 04
AI holds fixed roles and never replaces data.
- 05
Every inferred fact, task and score carries confidence, evidence, and a one-sentence explanation.
- 06
BBL is the truth for property. Not address, not BIN.
- 07
A feature is not done unless it flows Event to Memory to Signal to Recommendation to one-tap Action.
- 08
Ambiguous matches require human approval.
- 09
Push, do not pull. The system initiates.
- 10
Outcomes, dismissals and corrections are stored as feedback, not discarded.
Data boundaries
Three concentric data rings, from raw public staging inward to private user-owned records.
- 03Raw public-record stagingunowned, replaceable
- 02Shared property graphkeyed on BBL
- 01Private user-owned recordsrow-level security
Decisions
- Scoring uses no language model
- Deterministic and auditable. Intent, urgency, engagement, quality and recency each return their own score and reasons, persisted per dimension. A score that cannot be explained in one sentence violates the explainability law. Recency can go negative: silence actively decays a lead.
- Pressure stacking over single signals
- Signals bucket into financial, family, work, housing, active search, property-specific and behavioral, with weights. A transition moment fires only when two or more categories stack, because stacked pressure is qualitatively different from any single signal.
- Person ID is nullable on the event log
- An unknown inbound call still becomes a record, flagged for review, and lands in an unassigned inbox. Data is never dropped for lack of identity.
- Ambiguous matches go to a human queue
- Name similarity plus address corroboration on an explicit confidence ladder: high confidence auto-links, the middle band routes to an approval queue, low confidence is discarded. False positives corrupt memory permanently; missing links are cheap to fix.
- Task rules as data, not branching code
- An ordered table of condition, action, reason, priority, timing and channel. Adding a trigger is one entry, and every task ships with the sentence explaining why it exists.
- A deterministic formatter overrides the model
- Building facts bypass the model's prose entirely and render through tested formatting code. Those facts are too important to let a model paraphrase.
- Source priority baked into tool names
- The model sees primary source, official records and supplemental context as labels rather than as a policy it might ignore.
- Detectors written ahead of their data
- Opportunity detection probes which data sources actually have rows, then runs only those detectors. Detectors for future sources are already written and lie dormant.
Stack
InterfaceNext.js 16 · React 19 · TS strict · Tailwind v4 · shadcn/Radix
DataSupabase: Postgres, Auth, Storage
InfrastructureVercel · nightly public-records cron
ModelsClaude Sonnet 4.5 for chat, summaries and prep; a small OpenAI model for extraction; Whisper
Search, externalTavily supplemental web search
Ingestionpublic records and market data feeding the property graph
02
impossibletoname.comImpossible To Name
Consumer platform · regulated document domain
A consumer platform that prepares European visa applications end to end, priced at $399 per household. Researched, scoped, architected, built, priced and shipped by one person: six weeks from first commit to a live product in March 2026, and seven months of operation since.
- Idea to live
- 6 weeks, solo · live since March 2026
- Live coverage
- 3 countries · 9 Spanish consulates · 35+ document types
- Price
- $399 per household, one-time
- Monitoring
- 91 government sources, checked daily
- Surface
- 55 API routes · 135 task definitions
The problem
An applicant preparing a European visa. The output is a stack of documents that must be internally consistent with each other.
Government sitesseveral languages
Consulateseach one differs
Forumsoften wrong, often out of date
The applicantassembles it alone
Document stackmust be internally consistent
- Facts go stale without anyone noticing.
- The same question, answered thousands of times.
- No single source of truth.
Operating insight
The domain is unforgiving about facts. Generative models are unreliable about facts. Most products in this space either avoid the technology entirely or point it at precisely the thing it is worst at.
The system
A hard boundary between deterministic and generative. Every visa fact has one home, and values derive rather than duplicate: change the base figure and every consumer moves. Where official sources conflict, the platform uses the higher figure because it is safer for the applicant.
Knowledge hubevery fact carries a source citation, a verification date and a tier
Deterministic engines
Eligibility & routingper-country calculators
Task enginedependency DAG, urgency against move date
Document factory
Wizard stateflattened
Form fillconsulate-aware
Generatorsin parallel
Package
User journeyassessment → wizard → checklist → package
Verified-facts layercomputes this family's thresholds at request time
AI support agentsits on top of the stack, decides nothing
- Strict on visa facts
- Open on platform questions
- Conversational on general context
Retrieval runs in two passes, scoped to the user's pathway first and then unfiltered, with a tag-based pass alongside as a gap-filler, everything filtered to recently verified articles. The verified-facts block computes this specific family's thresholds at request time, including per-country dependent age rules, so the answer names the actual person and the actual number.
Key decisions
- Decision
The model narrates, never decides
- Why
- The domain is unforgiving about facts and generative models are unreliable about them.
- Result
- Eligibility, thresholds, form filling and task sequencing are deterministic. The model writes prose on top of computed facts and makes no determination.
- Limit
- Prose over computed facts can still mislead in emphasis, so generated forms are never flattened and the user's review pass stays the final checkpoint.
- Decision
Financial gates before the paywall
- Why
- The largest refund risk is paying $399 and then discovering you miss a threshold.
- Result
- Elimination happens in the free assessment, on the same threshold function and exchange-rate buffer the later stage uses. If the later stage hard-stops at a number, the first stage eliminates at that number.
- Limit
- Lower top-of-funnel conversion, accepted deliberately.
- Decision
Guidance rebuilt without the model
- Why
- The feature was AI, and a version that derived its content from the knowledge hub was better.
- Result
- Removing the model was the improvement.
- Limit
- Guidance is now exactly as current as the hub. A change the policy monitor has not caught is a change guidance does not know.
Result
Six weeks from first commit to a live product, built solo and live since late March 2026. The figures below are the state after seven months of operation and roughly 450 commits, not the six-week build.
Three pathways live end to end: Spain's digital nomad visa, Portugal's D8 and the Dutch-American Friendship Treaty. Six more priced in the free calculator but not built, with Spain's non-lucrative visa furthest along. Fifty-five API routes, 135 task definitions, 76 knowledge-base articles, 61 published guides, and a policy monitor reading 91 government sources for rule changes.
Limits
What I know
- Three pathways run end to end, from assessment through a package of filled government forms.
- The deterministic verifier catches drift in generated documents, which is what lets temperature stay high.
- Removing the model from guidance made guidance better.
- No query on a user-data table reaches the database without a user filter, because the client extension throws before it does.
What I don't know
- How the flexible-document wizard state holds up as more countries nest their data differently. The cost is paid at every read edge and grows with each country.
- Whether every low-confidence support answer is being caught. The unanswered-questions table records what the agent flagged, not what it missed.
- Whether the consulate model, built around Spain's nine offices, holds for a second country with per-office rules. Portugal's consulate data sits in the hub, but only Spain routes by jurisdiction today.
What I would test next
- Spain's non-lucrative visa, the fourth pathway, is the test of whether a pathway the platform was not designed around fits the same engines without special cases. Its hub entry is written and its wizard is scaffolded; its forms are not yet mapped.
What it looks like
The public product, rather than a diagram of it. The homepage runs a live demo of the post-paywall app on fixture data, the calculator is the free front door, and the form is one of the eight government templates the platform fills, here with the test fixture's synthetic applicant.
The three rules that follow
- One home per fact
- Every visa fact has exactly one home, wrapped with its source and its verification date. Nothing else may hardcode a threshold.
- AI narrates, never decides
- The model writes prose on top of computed facts. It makes no determination.
- Validation informs, never blocks
- After payment the user can always generate. A missing field produces a warning, not a wall.
What enforces the rules
A rule a script enforces is a different claim from a rule the author follows. Each of the three has a check behind it that fails a test, a deploy, or an edit, rather than a convention someone remembers.
- Mapping validation at build time
- A test suite asserts that every mapped field name exists in its PDF template, that every template field is mapped or on a known-exceptions list, and that a dry-run fill with synthetic data produces a valid PDF. It runs with the ordinary test command, so mapping drift fails before deployment.
- A template integrity registry
- Each of the eight government PDF templates has a SHA-256 hash, a page count and its full list of form field names on record. A pre-deploy script and an admin health route verify the files on disk against the record, so a consulate quietly replacing a form is caught before a user is handed the wrong one.
- The hub cross-check
- A verification script compares the knowledge hub against the one remaining legacy threshold file and runs self-consistency checks across Spain's consulate data. Any mismatch is a non-zero exit.
- Guard hooks on every edit
- Two hooks run on every file edit in the repo. One flags a currency amount or a processing time written into a generator or a marketing page that does not import from the hub. The other flags a hub value changed without a new verification date. Both warn rather than block, which is a weakness named below.
Tenant isolation at the query layer
The most production-grade piece of the codebase is the one a demo never shows. The database client is wrapped in an extension that inspects every query before it runs.
- No user filter, no query
- Nine user-data tables are marked tenant-isolated. Any read, update, delete or upsert on them without the user's id in its filter throws before it reaches the database. An id alone is not sufficient. Create and create-many are checked too, so a row cannot be written without an owner.
- Bypasses are explicit and few
- A pre-login assessment keyed on a visitor id, a Stripe webhook keyed on its checkout session and authenticated by signature, and admin routes behind an allowlist each carry a bypass flag, and every bypass must carry a comment justifying it.
- Soft deletes, filtered automatically
- Ten tables carry a deletion timestamp. Reads on them are filtered to live rows without the caller asking, and a delete call becomes a timestamp rather than a removal.
- One hard-delete path
- The delete-my-data flow and the auto-purge run a single raw-SQL cascade across the user's tables in foreign-key order, anonymise the model usage logs so cost accounting survives, clear sessions, and leave the audit log intact by design.
- Export returns what the user may see
- A data export returns encrypted identifiers only as their masked display copy and strips the never-stored tier entirely, rate-limited to one export a day.
Data boundaries
Three tiers of sensitivity, from ordinary personal data outward to the identifiers a passport carries and the categories the platform refuses to hold at all.
- 03Tier 1 · ordinary personal dataplaintext, tenant-isolated
- 02Tier 2 · passport and national ID numbersAES-256 at rest, a masked copy for display
- 01Tier 3 · financial and biometricnever stored
- Resolvers never return a masked string
- An encrypted value is read only through a resolver that decrypts it for the one place that needs it. The masked copy is what the interface shows, and generated supporting documents carry a placeholder rather than the real identifier.
Where the difficulty actually is
“A Spanish visa” is not one procedure. Spain is served by nine US consulates, each with its own forms, appointment method, fees, accepted payment types and submission rules, and jurisdiction follows where the applicant lives rather than where they are going. The platform detects the office and generates against it.
Free assessment5–14 adaptive questions, about five minutes, before payment
Paid wizardabout forty minutes, one session for the whole household
Jurisdiction routingthe consulate follows where the applicant lives, not where they are going
Form variantthe NIE equivalent this particular office accepts
Instruction sheetthis office's appointment method, fees, accepted payment types, submission rules
Document listwhat this office asks for that the next one does not
- Get the jurisdiction wrong and every document downstream of it is the wrong document.
Requirements that move, documents that block
The supporting documents are not a list, they are a graph. An FBI identity history summary has to exist before it can be apostilled, and the apostille has to exist before it can be sworn-translated, a chain that takes weeks and that no checklist without dependencies can warn you about in time.
FBI identity history summary
Federal apostille
Sworn translation
Packetone item on a critical path of 27 to 55 tasks, by pathway
- Requirements move on their own schedule
- Income thresholds update on a calendar (Spain's every January) and consulate procedure changes without announcement. Every fact carries a verification date, and the policy monitor watches 91 government sources for changes rather than waiting for a user to hit one.
- The checklist models dependencies, not steps
- Between 27 and 55 tasks per application depending on the pathway, each with blocked and ready states, processing times, and the critical path flagged. Knowing a document is needed is worthless without knowing it cannot be started yet.
- Employment type forks the requirements
- W-2, freelance and business owner take different Social Security paths: a W-2 employee has to request a Certificate of Coverage from the SSA, the others do not. The wizard branches rather than asking everyone for everything.
- The unit is a household, not an applicant
- Individual pre-filled forms for every person (visa applications, NIE forms, responsible declarations, parental authorizations) with thresholds adjusted for family size, all generated from one session. The price follows the unit: $399 covers the household, not each applicant in it.
- Complex cases are turned away before payment
- Prior rejections, deportation history, complex custody, a criminal record: the free assessment flags these and says so, rather than taking the money and discovering it in the wizard.
The lifecycle engine
The nine scheduled jobs in the stack are not plumbing. Four of them run a deterministic sequencing engine over every active application, deciding each day whether to say anything and, if so, which one thing.
- One email a day, highest priority wins
- Five conditions are evaluated per active track in a fixed order: everything done but nothing generated, a critical-path task overdue, 80 percent complete, a category just finished, and three days of inactivity. At most one match sends, and a 20-hour cooldown stops the daily jobs from stacking.
- Eleven transactional templates
- Documents ready, next steps, pre-departure reminders, wizard reminders, the five nudges, assessment results and a support-agent escalation, each a typed function rather than a document in a mail tool.
- Abandoned carts without a model
- The cart-abandonment email is assembled every half hour from pre-written copy blocks selected on the user's stated motivation and lifestyle. No model call, no latency, no cost, and the copy can be read in full before it is ever sent.
- The monitor on its own clock
- The policy monitor scans its 91 sources twice a day, aggregates nightly, reports on its own health weekly and on its findings monthly.
Front end and distribution
- Mobile first, because of where the traffic comes from
- Instagram is the primary traffic source, so the marketing site and the app are designed for a phone first, with a 44-pixel minimum on every touch target. The post-paywall app was redesigned in September 2026 to match the marketing site rather than the other way round.
- A free calculator as the front door
- Nine pathways across five countries checked against the year's official thresholds, adjusted for household size, with an exchange-rate buffer and a map. Six of the nine are waitlisted, and the calculator says which.
- Sixty-one published guides
- The guide library is the search surface, written from the same hub the product runs on, so a threshold that changes in the hub is a threshold that changes in the guide.
Decisions
- A claimed quote must exist in the source
- The policy monitor reads 91 government sources for rule changes. A claimed verbatim quote must be found in the source text or the alert is rejected, and a null quote is accepted as the model declining to claim one.
- Prompt-injection defense at the boundary
- User-controlled strings enter the system prompt inside structured tags, and standard escaping does not escape a closing tag, so angle brackets are escaped recursively and the output is scanned for breakout artifacts.
- Sensitive fields are tiered
- Ordinary personal data in plaintext, identifiers encrypted with a masked display copy, financial and biometric data never stored at all. Encrypted values are read only through resolvers that never return a masked string, and generated supporting documents carry a placeholder rather than the real identifier.
- Guidance is now entirely deterministic
- The feature was AI and was rebuilt to derive its content from the knowledge hub instead, because the deterministic version was better. Removing the model was the improvement.
- The eligibility invariant
- Every income-based elimination in the free assessment must use the same threshold function and exchange-rate buffer as the later evaluation. If the later stage hard-stops at a number, the first stage must eliminate at that same number. No exceptions. The anti-refund business rule expressed as a code contract.
- Date math was deleted
- Backward scheduling and computed start and end dates were removed in favor of human-readable timeline prose that never feeds a calculation, plus a red, amber, green urgency signal. The model is forbidden from emitting calendar dates. Do not manufacture precision you do not have. The same instinct governs currency conversion, buffered conservatively in both directions.
- Never flatten the generated forms
- Fields stay editable because the user's review pass is the final accuracy checkpoint. The target is high accuracy with a human check, not the illusion of perfection.
- The agent exposes its own gaps
- Confidence is detected on every turn, and anything short of full confidence writes to an unanswered-questions table that drives knowledge-base priorities.
- Financial gates before the paywall
- The largest refund risk is paying $399 and then discovering you miss a threshold, so elimination happens in the free assessment. Lower top-of-funnel conversion, deliberately, in exchange for removing the refund case.
- Two independent payment paths
- Webhooks get lost or arrive late, so a direct verification path exists alongside the webhook, both idempotent. On a metadata validation failure the system records the raw payload and the customer email for manual resolution rather than failing silently.
- Verify then retry, temperature high
- Generated documents are checked by a deterministic verifier asserting required names, phrases, placeholders and length. Failure retries once, then falls back to a template. Temperature stays high because the verifier catches drift and variety stops every applicant's letter reading identically.
- Wizard state as flexible documents
- Not typed columns. Adding a country requires zero migrations. The cost is paid at every read edge, where each country nests its data differently, and that cost was accepted knowingly.
- The support agent never dead-ends
- Automatic escalation was removed. It answers what it can, names the gap honestly, offers to flag it, and the chat stays open. Announcing that a question has been forwarded to the team is a banned phrase.
- Buffered, not streamed
- You cannot validate, strip formatting, or measure confidence on text already sent to the user. Safety over perceived latency, with an animation covering the wait.
Stack
ApplicationNext.js 16 · React 19 · TS strict · Tailwind v4 · shadcn · RHF + Zod
DataPostgres on Neon via Prisma · NextAuth sessions · Cloudflare R2
InfrastructureVercel · nine scheduled jobs · Upstash rate limiting
Payments & emailStripe · Resend · Brevo
ObservabilityPostHog · Sentry
ModelsClaude Sonnet 4.6 for prose and support; Voyage embeddings for retrieval
One application, no separate services: a deliberate solo-operator constraint. Also running: a policy-monitoring system watching 91 government sources for rule changes.
Weaknesses
- The guards warn, they do not block
- The two edit hooks print a warning and let the edit through. What actually stops a hardcoded fact is the hub cross-check and the mapping tests, and those run at test and deploy time, not at the moment of the edit.
- Isolation has a bypass flag
- The tenant guard can be switched off per query. The rule that every bypass carries a written justification is enforced by review, not by the type system, and the guard's own test script is the only thing checking that the list of legitimate bypasses has not grown.
- Nine consulates means Spain
- Jurisdiction routing exists for Spain's nine offices. Portugal's consulate data is in the hub but does not route, and the Netherlands has one national route. The consulate model was built around the hardest case and has not been proven on a second one.
- The fourth pathway is half built
- Spain's non-lucrative visa has a hub entry, a wizard scaffold and a marketing page, and no form mappings or task graph. Greece and Italy are calculator thresholds and marketing pages only; their preparation packages were produced by the skills library and nothing has been built from them.
- The build figures are seven months old
- Six weeks is the time to a live product. The 135 task definitions, 91 sources and 61 guides are the state after seven months and roughly 450 commits, and this page said so only after an audit made the conflation visible.
03
The skills library
Reusable skills and agents · removing the founder from the loop
Roughly fifteen reusable skills and agents that run the recurring work of a solo operation.
- Skills and agents
- ~15, reusable
- Functions removed
- Research · writing · spec · support · inbound
- Shared layer
- One voice and brand layer, inherited
The problem
One person running content production, domain research, specification writing, form field mapping, knowledge-base maintenance and inbound analysis by hand. Recurring work plotted against the operator's week, with the compounding items marked.
Content production
Domain research
Specification writing
Form field mapping
Knowledge-base maintenance
Inbound analysis
The operatorone person's attention
- Every one of these consumes the binding constraint.
Operating insight
A solo operation's binding constraint is not capability, it is the operator's attention. So the highest-value work is the work that removes the operator from the loop.
The system
The skills library as a map, grouped by the function each one removes, with trigger and output for each. Research passes chain into specification, specification into support coverage, and every output inherits the voice layer.
Voice & brand layerevery output inherits tone and brand rules instead of restating them
Research · removes domain research
Pathway researchcountry visa → fact sheet
Topic findercontent gaps → blog topics
Video researchtopic → research brief
Writing · removes content production
Blog generatortopic → SEO post
Content enginenews or stat → series post
Shorts creatorarticle → short-form scripts
Video producerbrief → roadmap and assets
Specification
Pathway specfact sheet → full spec
Form field mappergovernment form → mapped fields
Inbound · removes triage
Outreach analyzerbrand pitch → rate and reply
Reply drafterforum thread → reply
Support
KB writerspec → support articles
Key decisions
- Decision
Skills, not prompts
- Why
- A codified procedure runs the same way every time. A one-off prompt runs the way it was typed that day.
- Result
- About fifteen reusable skills and agents covering research, writing, specification, support and inbound.
- Limit
- The chain stops at the specification. The build itself stays manual.
- Decision
Research separated from composition
- Why
- A research pass can be verified before anything is written from it.
- Result
- Pathway research produces a fact sheet, the fact sheet a specification, and the specification requirement lists and knowledge-base articles. Each output feeds the next.
- Limit
- Adding a pathway still has manual steps. This is not end to end.
- Decision
Voice as its own layer
- Why
- Rules stated once are inherited. Rules restated per output diverge.
- Result
- Every output inherits one voice and brand layer instead of restating it.
- Limit
- One layer means one voice, so a change to it changes every output at once.
Result
The chain produced the preparation packages for the Greece and Italy retirement pathways, both priced in the platform's free calculator and not yet built, in roughly half the time the same work took by hand for the first pathway.
Limits
What I know
- The chained preparation ran in roughly half the time of the hand-run first pathway.
What I don't know
- Whether half holds on the next pathway, or whether part of it was the gain any second run has over a first.
What I would test next
- Whether the specification output is complete enough that the manual build step needs no re-research.
Decisions
- Skills, not prompts
- Codified procedures as reusable skills rather than one-off prompts. Repeatability over cleverness.
- Research separated from composition
- A research pass can be verified before anything is written from it.
- Voice as its own layer
- Voice and brand rules live in one place, so every output inherits them instead of restating them.
A chained run
Adding a visa pathway to the platform has automated steps and manual ones. This is not end to end.
The chained portion runs pathway research into a verified fact sheet, the fact sheet into a specification, and the specification into requirement lists and knowledge-base articles. Each output feeds the next.
That chain produced the preparation packages for the Greece and Italy retirement pathways, both priced in the platform's free calculator and not yet built. The chained preparation ran in roughly half the time the same work took by hand for the first pathway. The build itself stays manual, and for those two it has not started.
04
The Screening Simulator
Multi-agent system · reverse-engineered from live AI screening
A multi-agent system that measures an application before it is sent, against a reconstruction of the AI screening tier built by reverse-engineering how current AI-powered applicant tracking systems actually generate criteria and score against them. No service, no database, no runtime: markdown files and Claude Code.
- Built by
- Reverse-engineering live AI-powered ATS screening
- Reconstructions
- 3, scored independently
- Applications run
- 4, 13 to 17 artifacts each
- Agents
- 4, each with an opposed incentive
- Gates
- 2: truth, then effectiveness
- Runtime
- None. Markdown and Claude Code
The problem
The modern hiring screen. Several hundred applicants per role.
Applicants
AI screening layerreads each résumé against recruiter-written criteria
Verdict per criterion, with quoted evidence
Meets
Does not meet
Uncertain
Recruiterseconds each, in verdict order
- Rejections arrive weeks later carrying no signal: nothing is learned.
Operating insight
Candidates treat the screen as a black box, but the vendors publish how it works. Ashby documents how criteria are generated from the posting text and publishes the rules for writing them, and its own audit reports that 11.4 percent of real evaluations land on UNCERTAIN.
A failure that appears only against the unwritten gates is a channel signal, not a document signal. Pedigree and industry gates are what a warm introduction overrides and a résumé cannot.
The system
Agent topology with isolation boundaries drawn explicitly. Model tiering is part of the design: composition and research on the larger model, both gates on the small one.
Orchestratorowns the gates; writes no content, judges nothing
Researcherjob description → three criteria variants; the only agent with network access
Writercomposes only from the fact base
Fact-auditortruth gate; reads and writes nothing else
Fact baseclosed, verified facts only
Guardrailsconfirmed false, outranks all criteria
Context isolationsmall model
Evaluatorsimulates the deployed screening tier
- receives one criteria set and the blinded résumé
- cannot see the posting, fact base, guardrails, prior verdicts, reasoning, or the chat
Skiproute to a different channel
Bullet shipsrésumé + verdict sheet
Key decisions
- Decision
The evaluator must stay small
- Why
- It is simulating the deployed screening tier, not trying to be a good judge, and the vendor's credit economics imply that tier runs on a small model. A larger model resolves ambiguity in the candidate's favor and returns false passes.
- Result
- Verdicts that match the tier being simulated. Upgrading the model makes the numbers prettier and the signal worse.
- Limit
- No ground truth exists to score the reconstruction against.
- Decision
Nondeterminism is measured, not smoothed
- Why
- A pipeline reporting a clean deterministic number would be lying about the thing it simulates.
- Result
- Verdicts that move across runs on unchanged text are labeled unstable and never read as earned improvements.
- Limit
- Instability clusters in the speculative gates, which depend on the model's outside knowledge rather than the text, so those verdicts are coin flips by construction.
- Decision
Refusing to ship is a first-class output
- Why
- If an application cannot be won honestly, the system should say so and stop.
- Result
- A skip routes to a different channel instead. A skip is an instruction, not a judgment.
- Limit
- Outcome data is small, so the calibration of when to skip is provisional.
Result
Every application produces a tailored résumé, audited form answers, and a verdict sheet carrying quoted evidence per criterion across three reconstructions of the screen, plus a routing decision. Four applications run so far, 13 to 17 artifacts each.
Limits
What I know
- Criteria reconstruction follows the vendor's published rules and audit data.
- The truth gate runs before the effectiveness gate, so nothing is scored that could not ship.
- Instability clusters in the speculative gates, not the textual ones.
What I don't know
- Whether the reconstruction matches the real screen. No ground truth is available to score it against.
- Whether the calibration holds. Outcome data is small and everything about it is provisional.
What I would test next
- Revoke the evaluator's read grant so isolation is structural rather than by discipline.
- Research the Greenhouse, Workday and LinkedIn playbooks that are currently honest stubs.
- Keep accumulating outcomes and re-read the calibration against them.
The two-gate flow
The truth gate runs first; the effectiveness gate only ever sees text that has already passed it.
Writerdraft
Truth gatefact-auditor, runs first
Effectiveness gateevaluator
Ships
Reverse-engineering the screen
The screening tier is a black box to candidates, but it is not an unknowable one. The reconstruction is built from Ashby's published rules and audit data: how criteria are generated from the posting text, the rules for writing them, and an audit reporting that 11.4 percent of real evaluations land on UNCERTAIN. Ashby's credit economics, where one credit covers one candidate against up to fifty criteria, imply the screening tier runs on small, efficient models. The pipeline reconstructs that screen three ways and scores against all three, because a single reconstruction would be a guess presented as a number.
Literalreconstructs what the machine generates from the posting
Realistictrimmed to what a recruiter keeps: must-have and nice-to-have
Adversarialadds the plausible unwritten gates, each labeled speculative
The reconstruction is what makes the rest of the system possible: the evaluator is pinned to a small model precisely because that is the tier the reconstruction says is doing the screening, and the three-valued verdict exists because that is what the screen returns.
Decisions
- The evaluator must stay small
- The decision most likely to be “fixed” by someone optimizing for quality. It is not trying to be a good judge. It is simulating the deployed screening tier, and the vendor's credit economics, one credit per candidate against up to fifty criteria, imply that tier runs on a small model. A larger model resolves ambiguity in the candidate's favor, performs inference a junior screener will not, and returns false passes. Upgrading it makes the numbers prettier and the signal worse.
- Three-valued output
- Meets, does not meet, and uncertain, because uncertain is the point. For a non-standard background the dominant failure mode is not rejection, it is a screener who cannot tell. So the test for every sentence is whether a literal reader could be unsure, never whether it is impressive.
- Two gates, opposed incentives
- The auditor pressures toward provable claims, the evaluator toward stronger ones. A line ships only if it satisfies both. Truth precedes effectiveness, because scoring an unshippable draft produces pressure to keep whatever earned the score.
- A third state for untraceable claims
- Traced or deleted is the naive design, and deletion silently destroys true but unrecorded facts. The third state routes the claim to the human who can rule on it.
- Confirmed negatives outrank criteria
- The dangerous claim is not the false one, it is the adjacent-true one that is ninety percent accurate and collapses under a reference check. A plain fact base cannot express “this is almost true and the almost is fatal.” Entries are dated rulings with explicit carve-outs, not rules.
- Refusing to ship is a first-class output
- If an application cannot be won honestly, the system says so and stops, and routes to a different channel instead. A skip is an instruction, not a judgment.
- No agent without an opposed incentive
- A constraint on the architecture itself: never add an agent unless isolation buys a genuinely opposed incentive. Each agent exists because it needs to not know something, or to want something different from the writer. Anything else is an instruction inside an existing prompt.
- Nondeterminism is measured, not smoothed
- Verdicts that move across runs on unchanged text are labeled unstable and must not be read as earned improvements. Instability clusters in the speculative gates, which depend on the model's outside knowledge rather than the text, so they are coin flips by construction. A pipeline reporting a clean deterministic number would be lying about the thing it simulates.
Stack
Claude Codeexecution engine · per-agent model pinning
Markdown as stateconstitution · fact base · guardrails · playbooks · four agent definitions
Private repono hosting, no database: the filesystem as state
Weaknesses
- Isolation by discipline
- Evaluator isolation is enforced by prompt discipline rather than by the tooling. Revoking the read grant would make it structural.
- Honest stubs
- One per-ATS playbook, Ashby, is fully researched. The others (Greenhouse, Workday, LinkedIn) are honest stubs that halt rather than improvise.
- No ground truth
- Criteria reconstruction is a model of what the screen generates, grounded in the vendor's published rules, with no ground truth available to score it against.
- Small outcome data
- Outcome data is small, and everything about calibration is provisional until more accumulates.
05
The Creator Deal Desk
Single-operator back office · brand and agency inbound
A single-operator back office for one creator's brand and agency inbound: triage, research, rates, contracts, accounting, relationships. Content strategy is deliberately out of scope: this runs the business behind the feed, not the feed.
- Agents
- 8 defined, each pinned to a model. A typical inbound fires 4
- Stored
- 2 records. Every view is generated
- Escalates to Opus
- Attached contracts, novel terms, a missing comp
- Human-only
- Sending, and marking paid
The problem
Brand and agency inbound for one creator. Every email needs triage, research on the sender, a rate against the card, a contract read when one is attached, and a reply, and every list that tracks it, pipeline, obligations, comps, accounting, drifts the moment it is maintained by hand.
Operating insight
Every view is generated on request rather than maintained, so there is no stored list that can fall out of date.
Contacts are people, not deals, which is what lets the record survive an agency change: the person stays, the deal folder closes.
The system
Two things are stored: a record per deal and a record per contact. Every view (the contact list, the pipeline, obligations, comps, the accounting workbook, one list per deal type) is generated from them on request.
Inbounda brand or agency approach, pasted in
Filerfires on every paste
record.yamldeals/<slug>/, one folder per inbound
contacts/<id>.yamlpeople, not deals; survives an agency change
Filter and rendera script, run on request
views/*CONTACTS · PIPELINE · obligations · comps.csv · accounting.xlsx · one list per deal type
The operatorsends, and marks paid. Nothing else does either.
- Views are never hand-edited: anything typed there is gone on the next render.
Key decisions
- Decision
Two stored records, every view generated
- Why
- A maintained list can fall out of date. A generated one cannot.
- Result
- Contact list, pipeline, obligations, comps, the accounting workbook and one list per deal type, all produced on request and never hand-edited.
- Limit
- A view is only as correct as the two records feeding it, and those are written by an agent from a paste.
- Decision
Floors live in one place and a gate enforces them
- Why
- A check a script can run beats a judgment an agent would need taste for.
- Result
- Every draft declares its quote, the gate validates the declaration against canon floors and checks the amount appears in the body. Unreadable canon fails closed.
- Limit
- Seven inherited contradictions sit unresolved in the canon. Until they are arbitrated, rate recommendations downstream rest on a default the system picked.
- Decision
Nothing sends, nothing is marked paid
- Why
- Both are irreversible, and an irreversible action belongs to the operator.
- Result
- Eight agents file, research, price, draft and review. The human sends.
- Limit
- The operator stays in the loop for every deal that gets a reply, by design.
Result
A typical inbound fires four of the eight agents and leaves behind a filed record, research on the sender, a rate analysis, and a reply draft that has already cleared the floor gate. Cost is a design parameter: the expensive model runs only on attached contracts, novel terms, or a missing comp.
Limits
What I know
- Every quote that ships declared itself, matched the canon floors, and carried the usage-window and exclusivity asks.
- Nothing sends and nothing is marked paid without the operator.
What I don't know
- Whether a number is right for a particular deal. A gate can check that a quote is consistent, not that it is wise.
- Which side of the seven inherited contradictions the operator will take. Two of them are material.
What I would test next
- Arbitrate the two material contradictions and re-run the known-outcome fixtures against the corrected canon.
Architecture
- 00-canon/
- Rates, framework, red flags, voice, profile. The decision-time reads.
- 00-canon/CONFLICTS.md
- Inherited contradictions, left unresolved on purpose, for the operator to arbitrate.
- .claude/agents/
- Eight agent definitions, each pinned to a model.
- deals/
- One folder per inbound:
raw.md,record.yaml,research.md,analysis.md,reply-draft.md. - contacts/
- People, not deals. Survives an agency change.
- views/
- Generated:
CONTACTS,PIPELINE, obligations,comps.csv,accounting.xlsx, and one list per deal type. Never hand-edited. - fixtures/
- Known-outcome cases. Run after every canon edit.
- scripts/hooks/
- Floor gate and auto-rollup.
- log/
decisions.jsonl,learnings.md.
Model routing
Eight agents, each pinned to a model in its own frontmatter. Cost is a design parameter here, not an afterthought.
| Agent | Model | Fires |
|---|---|---|
| filer | haiku | every paste |
| researcher | sonnet | anything not ignored |
| rates | sonnet | anything getting a reply |
| drafter | sonnet | anything getting a reply |
| contract | opus | only when a document is attached |
| adversary | opus | only above a value threshold, novel terms, or a missing comp |
| accounting | haiku | scheduled |
| archivist | haiku (sonnet quarterly) | scheduled |
A typical inbound fires four agents, not eight. Each subagent costs roughly 25–35k tokens just to initialize, so fanning out on every deal would erase the savings from routing the cheap work to Haiku.
Leave CLAUDE_CODE_SUBAGENT_MODEL unset. It overrides the per-agent frontmatter and silently collapses this whole table to one model.
Scheduling
0 8 * * 1 cd /path/to/repo && claude -p "Run archivist. Report only what needs attention." >> log/weekly.txtHeadless claude -p survives a reboot and does not need the desktop app open. Each run is a full session and counts against usage limits.
Decisions
Enforcement, not judgment. Every rule below is a check a script can run; none of them require an agent to have taste.
- Every draft declares itself
- A
reply-draft.mdopens with a machine-read declaration:<!-- QUOTE: none -->or<!-- QUOTE: <rate-card-key> <amount> -->. The gate validates the declaration against canon floors, checks that the declared amount actually appears in the body, and, when quoting, requires the usage-window and exclusivity asks. Declines that mention the other side's lowball number pass cleanly. - Floors live in one place
- The frontmatter of
00-canon/rates.md, parsed at gate time. Unreadable canon fails closed for quotes. - Absence is a value
validate()flags a signed deal with nousage_window_end, because an absent end date defaults to perpetual.- Category collisions
exclusivity_check.pycatches them before a quote goes out, not after.- Nothing sends
- Nothing is marked paid by an agent either. Both irreversible actions stay with the operator.
Stack
Claude Codeeight agent definitions, each pinned to a model
YAML and markdowntwo sources of truth; every view generated from them
Python gates and hooksfloor gate · exclusivity check · fixtures
cronheadless claude -p, weekly
Weaknesses
- Canon contradicts itself
- Seven inherited contradictions sit unresolved in
CONFLICTS.md, parked there deliberately rather than papered over. Two are material: until they are arbitrated, every rate recommendation downstream of them rests on a default the system picked rather than a decision the operator made. - Gates enforce, they do not judge
- The floor gate checks that a quote declares itself, matches canon, and appears in the body. Whether the number is right for this particular deal is not a thing a gate can check.