Human LLM

Notes

How AI models actually work, one short note at a time.

Why a model's output is plausible rather than true, and what that means for where a human checks. Same text as John Gabrick's LinkedIn posts, dated.

A mock discharge instruction document with one line highlighted in red and flagged, illustrating AI output that reads well but contains a dangerous drug combination.

By

AI can write discharge instructions that are clean, specific, and dangerous.

This is worth understanding before you deploy AI in care.

Ask a model to draft instructions and it can produce a regimen that reads perfectly and combines two blood thinners that shouldn't go together. It isn't broken. Plausible text is all it ever produces, and plausible sometimes lands on harmful.

That's why "a clinician signs it" isn't bureaucracy. In that workflow, the signature is the safety mechanism.

Permalink
A three-stage pipeline of Index, Retrieve, and Generate, with the Index stage highlighted as the point where wrong tags cause a confident wrong answer downstream.

By

A retrieval system usually fails before anyone asks a question.

It fails at indexing.

If a document gets the wrong tags when it's stored, the retriever pulls the wrong passage, and the model writes a clean, confident answer from the wrong source. The generation step looks perfect. The mistake was upstream.

I've watched teams tune prompts for weeks when the actual problem was how their knowledge base was chunked and tagged. If retrieval matters to you, look at the index first.

Permalink
Diagram showing next-token prediction: an input phrase followed by candidate next words with probability bars, illustrating that a language model samples a plausible token rather than choosing a true one.

By

An LLM doesn't look facts up. It predicts the next token.

One word at a time, weighted by patterns it saw in training.

That single fact explains the failure everyone keeps hitting. When a model invents a drug interaction that sounds right, it isn't malfunctioning. It's doing the only thing it ever does, picking a plausible next word. Plausible and true only usually overlap.

Once you see that, "why did it make this up" stops being mysterious, and "where do we require a human to check" becomes the real design question.

Permalink