How Ask Roger works.

It will tell you when it does not know. Asked something the records do not cover, it says so instead of producing something plausible. It will not discuss compensation, references, anything confidential to an employer, or Roger’s personal life, and points you at him directly for those.

It reads more than the reviewer does. A posting submitted to the reviewer cannot reach the private half of the corpus. A question here can: it is Roger’s own record answering about Roger. Private material may inform an answer and can never be quoted at length, named, or listed as a source, which is enforced where the sources are built rather than by instructing the model. Anything he does not want speaking for him at all is excluded from the assistant outright.

It is slow, and that is real. An answer takes a few tens of seconds. The whole thing runs on one small server with no graphics card, shared with the reviewer, and the honest reason it is not faster is that it is not worth renting a bigger machine to shave off twenty seconds. Questions from the bank return instantly; a question asked while a JD review is running will be slower still, because the review has the machine.

Conversations are kept so Roger can read what was asked and grade the answers, which is how the assistant is corrected. The mechanism is the one described under “How it gets better” below, and it works the same way here: when an answer is wrong, the usual fix is that no document says the thing plainly, and the document gets written.

One model does the reading: qwen3:4b-q8_0, four billion parameters at eight-bit precision, open weights, served through Ollama on Roger’s own server with an 8,192-token context. Retrieval uses a second, smaller model, nomic-embed-text, which turns text into 768-dimension vectors stored in Postgres with pgvector. Nothing is sent to a hosted API. No posting you submit leaves that machine.

The model is not trained on Roger’s career. It has never seen it in training and nothing about it has been fine-tuned. It is a general open-weight model that is handed the relevant passages at the moment of the question and asked to judge one requirement against them. That is a deliberate choice rather than a shortcut: a model trained on a career can only be corrected by training it again, while a model that reads a corpus is corrected by fixing the corpus, and you can see which passage produced which verdict.

It is also not allowed to decide the outcome. The model returns a verdict per requirement and a sentence of reasoning. Every number, the score, the weighting, the threshold, is computed in code from those verdicts. When the model has been asked to reason about something it is reliably bad at, that judgment has been moved out of the prompt and into code: how long a span of years is, whether a named company actually appears in the evidence, and whether a résumé line is carried by the source it cites.

Four mechanisms, in the order they matter.

1. The corpus gets fixed. Most wrong verdicts are not the model reasoning badly. They are the model failing to find something because no document says it plainly. When that happens the answer is to write the missing document, not to adjust the prompt.
2. Roger grades the verdicts himself. Every verdict is logged with the exact evidence the model saw and the model and prompt version that produced it. He marks each one agree, disagree or not enough evidence to judge, in his own words. Disagreements are counted in two directions, because a reviewer that is too harsh and one that is too generous need opposite fixes.
3. A fixed set of postings guards every change. A group of real job descriptions, some Roger applied for and some drawn at random from job boards, each carrying one claim: he can do this job, or he cannot. Any change to a prompt, a model or the corpus is scored against the whole set before and after. A change that improves one posting and quietly breaks another shows up as a number, not as a feeling.
4. Rules move from prose into code. When instructing the model in words fails twice, the judgment is taken away from it and written as a rule that runs the same way every time.

A fine-tuned adapter, trained on the graded verdicts, is planned rather than built. It is listed here as a plan so that nothing above reads as a claim about something that exists.

The corpus has a public part and a private part. The private part holds client work and material under agreement, and a posting submitted through this site cannot reach it: the restriction is enforced in the database query, not by asking the model nicely. Your review is drawn from the public documents, plus two files about Roger himself that every review uses, a sheet of career facts and his master résumé.

That is a real limit and worth stating plainly: a review here sees less than Roger does when he runs the same posting for himself. It is the reason a strong result comes with a résumé and a weak one comes with Roger reading the posting personally.

Your posting is stored so the review can be reopened and so Roger can see what was asked. It is never used to train anything and never shown to anyone else.

Submit the same posting against the same corpus and you get the same verdicts and the same score. Not close to it. The same number. The table below is the check: the postings scored in the two most recent evaluations, before and after the reviewer’s prompt was rewritten, to three decimal places.

Four things make that true, and none of them is the model being reliable. The judge runs at temperature zero, so it is not sampling. Its answer is constrained to a fixed shape. That answer is one of three words, met, partial or unmet, not a number. And the score is arithmetic performed in code on those three words.

The model itself is not deterministic and this does not claim it is. Its intermediate reasoning does vary between runs on identical input. What the design does is put the decision somewhere that variation cannot reach: it would have to be large enough to move a requirement from met to partial before it could move a score at all.

Which matters for a practical reason. If you disagree with a result, it points at one requirement and the passage behind it, and you can run it again and get the same thing to argue with. And when a number does move, something really changed: the corpus, the prompt, or the model. All three are recorded with every run.

Everything below is counted from the repository at build time, not typed in. The script that does it is scripts/build_stats.py, and the build fails if the committed figures differ from a fresh count. Clone the repo and run it; you should get these numbers.