Analyst Field Guide · IT Help Desk & Security
Disciplined evaluation of claims, evidence, and your own reasoning — before you believe, report, or act. The capstone of the set: it's the habit underneath good troubleshooting and good root-cause analysis.
Critical thinking is the discipline of not believing things too quickly. It is the deliberate separation of what is claimed from what is shown, the active search for what would prove you wrong, and the honest statement of how sure you actually are. For an analyst it's not philosophy — it's the difference between escalating a false positive and catching a real intrusion, between a confident wrong attribution and a calibrated true one.
The failure modes critical thinking guards against are exactly the ones that bite security and support work:
The rest of this guide is a toolkit against these — structure for arguments, standards for evidence, named biases and fallacies, and structured techniques that force you to test what you'd rather assume.
Before you can evaluate a claim, decompose it. Every argument is premises plus an inference yielding a conclusion — resting on assumptions that usually go unstated.
To analyse a claim: state the conclusion in one sentence, list the premises offered for it, name the inference (how the premises are supposed to get you there), and — the part everyone skips — surface the unstated assumptions the inference depends on. A valid-looking argument fails the moment a hidden assumption is false.
Three ways to move from evidence to conclusion; each proves something different.
The RCA Field Guide diagrams these in depth and maps each to a method (5 Whys, fishbone, Pareto). Here the point is narrower: name which mode an argument is using, because it tells you what the argument can and cannot establish.
Two questions, kept separate: how reliable is the source, and how credible is this specific item? The NATO / Admiralty code grades them independently — a reliable source can still report a doubtful item, and an unreliable source can occasionally be right.
| Source reliability (A–F) | Information credibility (1–6) |
|---|---|
| A — completely reliable | 1 — confirmed by other independent sources |
| B — usually reliable | 2 — probably true (consistent, uncorroborated) |
| C — fairly reliable | 3 — possibly true |
| D — not usually reliable | 4 — doubtful |
| E — unreliable | 5 — improbable (contradicted) |
| F — reliability cannot be judged | 6 — truth cannot be judged |
Pair them as a two-character grade (e.g. B2). Then layer your own evidence tiers — confirmed / suspected / theoretical, observed-here vs. general intel — so a reader knows what's load-bearing.
Circular reporting: three sources that all trace to one origin are one source, not corroboration. Extraordinary claims: the bigger the conclusion, the stronger the evidence it needs — a single uncorroborated item shouldn't carry a high-confidence judgement.
The predictable ways the mind shortcuts. You can't delete them — you counter them with structure (next sections).
| Bias | How it shows up in analysis |
|---|---|
| Confirmation bias | Seeking and over-weighting evidence that fits your favoured theory. |
| Anchoring | The first number, label, or theory frames everything after it. |
| Availability | Recent or vivid events feel more likely than they are. |
| Base-rate neglect | Ignoring how common something actually is — the core of false-positive triage. |
| Premature attribution | Naming a threat actor or cause before the evidence is diagnostic. |
| Automation / alert bias | Treating a tool's verdict as proof rather than as one more piece of evidence. |
| Sunk cost | Sticking with a hypothesis because you've invested effort in it. |
| Hindsight | "It was obvious" — it rarely was, given the data available at the time. |
| Groupthink | Convergence without dissent; the loudest or most senior view wins. |
Flaws in the inference — the argument can be flawed even when the premises are true.
| Fallacy | The move |
|---|---|
| Ad hominem | Attacking the source instead of the claim. |
| Appeal to authority | "An expert said so" — without the supporting reasoning, or from the wrong field. |
| False dichotomy | Presenting two options when more exist. |
| Correlation ≠ causation | Two things move together, so one is assumed to cause the other. |
| Hasty generalization | A broad rule from too few cases. |
| Circular reasoning | The conclusion is smuggled into the premises. |
| Slippery slope | One step is assumed to inevitably cause an extreme chain. |
| Appeal to ignorance | "No evidence against it" treated as evidence for it. Absence of evidence isn't evidence of absence. |
| Survivorship bias | Reasoning only from the cases that made it into your data. |
The flagship structured technique (Heuer). Its one counter-intuitive move fixes most confirmation bias: you rank hypotheses by the evidence that disproves them, not the evidence that fits.
State how likely, in words tied to numbers — and keep likelihood (the estimative scale below) separate from confidence (how good your evidence is). You can be highly confident something is unlikely, or have low confidence in a "probable."
Avoid bare words like "possible" or "could" that span the whole scale. And never let a high likelihood word smuggle in confidence the evidence doesn't support — say "likely, low confidence" when that's the truth.
"Anchor on base rates" (§08) becomes concrete when you do the arithmetic, and the result surprises people. Say a detector catches 99% of true threats and false-flags only 1% of benign events — "99% accurate." If the threat is rare, 1 in 1,000: out of 100,000 events, about 100 are real (99 caught) and about 99,900 are benign (roughly 999 false-flagged). Of the ~1,098 resulting alerts, only 99 are real — about 9%. A 99%-accurate detector is right about any given alert less than one time in ten, purely because the thing it hunts is rare. That is base-rate neglect, and it is exactly the Detection Engineering guide's base-rate trap seen from the analyst's chair. The discipline: a positive indicator updates the probability (prior toward posterior), it does not confirm; independent indicators that agree update it much further, which is why convergence matters (§07); and a single hit on a rare event is a reason to investigate, not a reason to conclude.
The whole guide as a working sequence. The crux is step 3 — actively seeking what would prove you wrong.
Separate claim from evidence → grade the source → seek what would disprove you → weigh alternatives (ACH) → check your biases → state a calibrated conclusion. New evidence? Re-run it.
Conclusion · premises · inference · unstated assumptions. Attack the inference and the assumptions, not just the premises.
Grade source reliability (A–F) and item credibility (1–6) separately. Circular reporting isn't corroboration. Extraordinary claims need extraordinary evidence.
List all hypotheses first · mark evidence consistent/inconsistent across the row · the winner has the FEWEST inconsistencies, not the most fits.
Use estimative words (almost no chance → almost certain). Keep likelihood separate from confidence. Never an unqualified "secure."
Techniques and standards referenced: Analysis of Competing Hypotheses (R. J. Heuer Jr.); the NATO / Admiralty source-reliability and information-credibility code; the words of estimative probability (S. Kent, 1964) as standardized in ICD 203 (the lexicon shown reflects the ICD 203 ranges — verify the current revision before formal use). Cognitive-bias and logical-fallacy lists are standard. Worked examples are illustrative, not real incidents. Companion to the RCA Field Guide, Methods Handbook, and Troubleshooting Field Guide.