What that was measuring.

You were asked to hold a number in mind while answering questions that have an obvious wrong answer. The number was the point. The questions were the measurement.

Holding a number occupies the same limited machinery you use to check an answer that already feels right. As that load went up, the checking got worse. What the test is really looking for is whether your confidence went down with it, and for most people it does not.

Five blocks, six questions each. The number you were given at the start of each block got harder to hold as you went:

1. Three digits, given back as they were.
2. Four digits, given back as they were.
3. Four digits, with one added to every digit.
4. Four digits, with three added to every digit.
5. Three digits again, given back as they were.

Blocks three and four are the steep part, and not because four digits are harder than three. Adding to each digit moves the task from storing something to operating on it while storing it, and that second thing competes directly with the deliberate checking a trick question needs.

The fifth block is the one that makes the rest readable. It returns to the difficulty of the first, about twelve minutes later. If performance drops through the middle and then recovers at the end, the middle was the load. If it drops and stays down, some of it was simply fifteen minutes of effort, which is a different finding and worth being able to tell apart. A test that only ever got harder could not distinguish the two, and “they got tired” would be an untestable excuse for any result at all.

The questions were built around a well-documented habit: when a question is hard, people answer an easier one that came to mind instead, usually without noticing the swap. The substitute answer arrives fast, fluently and with no sense of effort, and that fluency is what gets read as confidence.

That is why being wrong here does not feel like being unsure. There is no wobble to notice. The wrong answer presents itself as the answer, and checking it requires deciding to check, which is precisely what the number in your head was consuming.

None of this is a defect peculiar to you. It is the ordinary behaviour of an efficient system that is right most of the time and is being shown the narrow class of problems built to exploit it.

Each family has a famous published counterpart, in print for decades. Those are described here rather than the test’s own items, so you can see the mechanism without learning what to answer.

Everyone takes the same thirty questions. One answer getting out spoils the measurement for everyone who comes after, and an answer key only has to escape once. So no participant is told which questions they got wrong, during the test, in the emailed results, or here. If you gave an address you get an overall score and the comparison that matters, which is your accuracy against your confidence.

That restriction ends when collection does. At that point the items and the full workings are published.

A single sitting proves nothing about you. Thirty questions is a short, noisy measurement. Have a bad block because a neighbour started drilling and the numbers move. Any individual result is as likely to reflect the fifteen minutes you happened to have as anything stable about how you think.

What it does do is demonstrate a general effect on a specific person, which is a different and more useful thing than a verdict. The effect is real and well replicated across large samples. Your run is one data point in something that only becomes a finding in aggregate, which is exactly why you were asked.

It is emphatically not an intelligence test and does not score anything that generalises. A high score means the trick questions did not catch you today.

Your answers, timings and confidence ratings are kept as a research dataset and analysed together with everyone else’s. The details you entered at the start were optional and are stored apart from the measurements, so they can be erased without destroying the data. Nothing is sold, shared or used for advertising.

The privacy page sets out what is collected, how long it is kept, and what asking for deletion does and does not remove.

This test supplements the series Better Business Decisions Require Better Information, which argues that the quality of a decision depends on the quality of the information behind it. The uncomfortable corollary is that attention is part of that information path, and attention is finite. A decision made with good data by someone with nothing left to spend on checking it is not a well-informed decision, and it will feel like one.

Thank you for the fifteen minutes. There was no way to get this measurement without asking someone for real effort, and you gave it.