More than once, on the day I thought I had every file, the one that mattered most was not among them. Not because nothing was missing, but because what was missing could not be seen. The material you have piles up in folders and gets written into lists. The material you do not have is written down nowhere.
Recently, building a system for one client, this happened to me four times. Once my reasoning was wrong, once I let something I had heard slip past, once I lumped the gaps together, and once I struck an item off the list altogether. Each looked different, but they all ended the same way: for a while, nobody knew something was missing.
The first was saying "there are none" after looking at only a few records.
Going through a handful of work records the client had sent, I concluded that a certain kind of case did not occur. It appeared nowhere in what I had received. The next day I learned those records were only part of the whole, and I deleted every conclusion that ended in "there are none."
I learned one thing properly that day. With partial material you can prove that something is there, but you cannot prove that something is not. If a case shows up even once in a few records, it definitely exists. But not showing up does not make it absent. It simply was not in the few I happened to receive.
In automation this is more dangerous still. A system built on the premise that "this case never happens" quietly gives the wrong answer on the day that case first arrives. A person hesitates in front of something new. A system does not. So a conclusion of "there are none" is worth doubting once more before you build on it.
Luckily it was before I had written the report. A little earlier I had decided not to put ratios like this out as numbers, and that decision ended up saving me. Once a number goes out, it gets quoted as fact from then on.
The second was missing where I had been told to look when the material was handed over.
I asked for any document that set out their decision criteria, and instead of a document the founder pointed me to one place to look. I saved it as a reference. Later, rereading the meeting notes, I saw that the founder had made the same point several times over while telling me this.
That point was, in effect, the core of what the system had to solve. But my draft design only handled what was written in the material I had received, and the judgement that remark pointed to had nowhere to go. I was lucky to catch it at the draft stage. Still, I had the material in hand and had let slip which part of it I was told to look at.
Looking back, it is not that I did not hear it. It was in the notes several times, so I must have heard it in the room. But I was waiting for a table of criteria, and I assumed criteria would naturally arrive as a table. Criteria handed over in words rather than in a table simply did not look like criteria.
When a client hands material over, where they tell you to look often matters more than what they give you. The file gets saved; what they said at the time does not. So now, when I receive material, I write that remark down in a line alongside the file. If I also note which file it was about, whoever opens it later gets how to read it along with it.
The third was lumping every blank under one phrase.
The rule was to leave anything unconfirmed blank rather than fill it with something plausible. I kept to that. The problem was that the screen showed every one of those blanks with the same sentence: "Not yet confirmed." A blank because we had not received the material yet, and a blank because the material was there but nobody had chosen yet, came out in the same words.
When several blanks sit in one list under the same words, people read them as a single lump — not finished yet, roughly. And buried in that lump is the fact that some of them I cannot fill at all; they can only be filled by asking the client.
Once I split them and showed "Not received yet" and "Not yet confirmed" separately, the questions for the client narrowed to exactly two. Until then they had been mixed in with the other blanks, and you could not see which ones were the client’s to answer. Leaving a blank is not enough. A blank only does its job when the reason it is blank shows up in different words.
The fourth was taking undecided items off the list.
We keep a single schedule that gathers all the work in progress. I had left out the items marked "awaiting approval," on the grounds that they were not yet work to be done. Putting something on the schedule before anyone had decided whether to do it seemed odd. The check that looks for omissions also reported "nothing missing."
When I later went through it item by item, eight were missing. The check was not broken. It counted an item as present if its number was written anywhere in the documents, and the items I had left out did have their numbers written somewhere. Deciding whether to do something is itself work to be done. "Awaiting approval" was never a reason to drop a line from the list; it was only the status of that line.
What makes this frightening is that the list looks clean. With a check result of "nothing missing" attached, no one has any reason to look again. I did not know myself until someone asked, "Why isn’t that one on the list?" The moment an undecided item disappears from the list, the person who would have decided it disappears with it.
Lay the four scenes on top of each other and the same shape appears. What exists gets counted and goes on the list; what is missing was never among the things being counted. So a result of "none" can always be read two ways. Either there really is none, or I did not look. From the outside the two look identical.
So these days, when I look at a result, I look at what was never counted as closely as at what is there. If a check comes back with zero, I first confirm what that check actually counts. A zero puts people at ease, but a zero where you do not know what was counted tells you nothing at all.
Of course you cannot find every gap. Try, and there is no end to it. But having one place where you pause before writing "there are none" down as fact catches most of them. At that pause, I check these four things.
One. Is this "none" based on everything, or on the few records you received? If only a few, write it down not as "none" but as "not seen yet."
Two. When the material was handed over, where were you told to look? Write that remark down in a line alongside the file.
Three. Is this blank because the material is missing, or because it has not been decided yet? The first has to be fetched; the second has someone else who needs to decide it.
Four. Are the items nobody has decided on yet still on the list? Deciding is work too, and if they are not on the list, no one decides them.
Even keeping all four, I cannot promise you will find every gap. This time, too, several were not found by me; I only found them after someone said that something seemed off. Lately I have started to wonder whether not letting a remark like that slip past matters more than adding questions to the list.