Evidence Fields

Jev evaluates exactly the state it is given, and a field name is part of that state's meaning, not decoration. A field called website makes an implicit claim — this URL belongs to this business — and Jev cannot tell a verified fact from a labelled guess unless the field name says which one it is.

In a production lead-scoring integration, a field named website was populated with a search engine's top result whenever no real site existed: sometimes the business's own page, often an unrelated one — a national brand's homepage, a directory listing, an encyclopedia article for an ordinary word that happened to appear in the business's name. On roughly two out of every three leads without an existing site, Jev was handed an unrelated page and told it was the business's own.

The fix was not a better question. It was renaming the field to match what was actually known, with a caveat stating plainly that the result might not belong to the business at all. State engineering is field-naming discipline: every field is an implicit claim about certainty, and the claim has to match the evidence behind it.

Story tells it through a note left for a babysitter. Technical has the field names, the measured counts, and what TypeSafe's own docs say state is for.

Say you're heading out for the evening and a babysitter is coming. You leave a note on the counter. A good note says what you actually know: he ate dinner at six, homework's done, bedtime's eight-thirty — and, this part matters, anything you're not sure about, you say you're not sure about. "Might still be a little congested, keep an eye on it." That's a note the babysitter can act on.

A bad note says more than you know and dresses it as fact. "He's fine" when you haven't actually checked since noon. "No allergies" when the truth is nobody's ever tested. The babysitter can't tell your confident guess from your checked fact, because the note reads the same either way — so a decision that should have been cautious gets made like it was certain.

I built something closer to the bad note, without meaning to. I was scoring business leads with an AI model, and one of the facts I handed it was called website. For most businesses, that field held an actual website, and the model could read the page. But for a large share of them there was no website at all. What went into that field instead was whatever a search engine happened to return for the business's name: sometimes the right thing by accident, often not. A dental clinic whose name overlapped with a streaming app got handed that app's homepage. A restaurant with a common plant's name in its title got handed the plant's own encyclopedia entry. A holding company known only by a registration number got handed the city's own encyclopedia entry, as if that were its front door.

The model wasn't guessing badly. It was reading the note I left it, and the note said "this is their website" about a page that had nothing to do with them. Worse, there was a second field in the same note, one that was supposed to say whether a website existed at all, and for these same businesses it said no. The note contradicted itself in the same breath, and the model had no way to know which half to believe — from where it sat, both halves looked equally like facts I'd checked.

Fixing it wasn't about asking the question more carefully. The question was fine. What I'd handed over as a claim needed to be handed over as what it actually was: a hunch. I stopped calling that field website unless there really was one, and for everyone else I called it best search result, with a plain sentence next to it saying this is only the top hit for the business's name and might not belong to them at all. Same underlying data, same search-engine result, described plainly instead of dressed up as an address.

That's the whole lesson, and websites are almost beside the point. What matters is which sentence you'd actually be willing to sign your name to before you leave for the night — the checked fact, or the one that just sounds finished.

Jev can only judge the state it's given, and a field can misrepresent reality without a single false sentence appearing anywhere in the prompt. It only has to claim more certainty than the data underneath it actually supports.

What "website" actually held

A production lead-scoring integration builds the state sent to Jev with one function, and its first version did this:

state["website"] = row["surface"]

surface is the upstream ranking script's own field, and it holds two different things depending on the row: for a business with a real site, the site's own URL; for a business without one, whatever best_surface a search engine returned instead — a directory listing, a franchise's national homepage, an unrelated page that happened to share a word with the business name. The ranker only strips obvious social-media domains out of it. Nothing distinguishes the two cases once they're sitting in the same column.

That single line was wrong for roughly two out of every three leads without an existing site.¹ A dental clinic's "website" was a streaming app that shares its name. A restaurant's was an encyclopedia entry for a plant. A numbered holding company's was an encyclopedia entry for the city it's registered in. Meanwhile a second field in the same object, web_presence, was independently asserting that no website existed for these same businesses — two facts in one state, disagreeing with each other, with nothing telling Jev which one to trust.

The fix was a name, not a rewrite

The corrected version:

if kind == "website":
    state["website"] = row["surface"]
else:
    state["web_presence"] = "no website found; directory and search listings only"
    if row.get("surface"):
        state["best_search_result"] = row["surface"]
        state["best_search_result_caveat"] = (
            "the highest-ranking page a web search for this business name "
            "returned. It is NOT confirmed to belong to them: it may be a "
            "national brand's site, a directory listing, or an unrelated "
            "page that merely matched some of the words in the name.")

Nothing about the questions changed. The field that used to assert ownership now asserts only what was actually measured — the top hit for a text search — in words a reader would need. Dropping the field entirely, which an earlier pass tried, was the second wrong answer: surface also carries real signal. For a business whose brand and website share a name, a text search reliably surfaces the business's own site — a search finds a brand's real page precisely because the business is that brand — and removing the field outright would have thrown that signal away along with the noise.

What the state format actually asks for

The pattern generalizes past this one bug. TypeSafe's own guidance is to use a structured object "so each part of the state has a descriptive name and its relationships remain clear," reserving a bare string for state with only one piece of text in it, and to "put related information together when the decision requires comparing those parts" — state, in their framing, is "the material you would present to a panel of experts before asking them to make a judgment."² Named fields aren't a style preference. They're what makes it possible to tell two kinds of evidence apart before either one reaches the model. website and best_search_result hold the identical URL in some rows; the field name is the only thing separating a claim from a hedge.

The other real example in this codebase is homepage_text, in redesign_probe.py: when a page fetch fails, the field is set to None explicitly, never an empty string, with a sibling homepage_text_note explaining that the page couldn't be read.³ A blank page and an unreachable one are different findings, and a state packet that can't tell them apart is asking the model to guess which failure it's actually looking at.

What would falsify this

The claim is that a mislabeled field costs more than a badly worded question — not that wording never matters. It would be falsified if swapping the field name back and forth, with the same URL held constant underneath it, produced identical scores every time; that would mean Jev was reading past the label and inferring the truth from the URL alone. That specific A/B hasn't been run. What's verified is the rate — roughly two-thirds of leads without an existing site affected — and the name collisions described above. Holding the URL fixed and only swapping the field name is the test that would actually settle it, and it's the next thing to run, not a claim already proven.