Why Does More Information Make Us Feel More Certain Without Making Us More Right?

The idea that more information should lead to better decisions feels almost self-evident. If we know more about a problem, surely we should understand it better. And in many situations, we do.

But psychologists have been studying a complication to that assumption for decades: the amount of information available to us and the accuracy of our judgment do not always improve at the same rate.

That tension feels familiar to me. I have spent a good part of my career working with and leading data and insights teams, where I have seen better information expose false assumptions, reveal hidden customer behaviour and completely change how a problem is understood. But I have also seen analyses become richer without decisions becoming much clearer: another segment added, another model, another piece of research, and the explanation becomes more detailed while the uncertainty at the heart of the decision barely moves.

That distinction has become harder for me to ignore as AI enters more of the analytical process. We can now interrogate datasets, synthesise research, identify patterns, generate explanations and even produce recommendations in a fraction of the time those activities once required. Information is becoming easier to obtain, and interpretation easier to produce.

More than sixty years ago, psychologist Stuart Oskamp conducted an experiment that captured the problem unusually clearly. Participants, including clinical psychologists, received a case study in stages and were asked to make judgments about the person it described. As they received more information, their confidence rose steadily, while their accuracy improved much less and quickly approached a ceiling.

Later research by Claire Tsai, Joshua Klayman and Reid Hastie found a similar pattern across three experiments. Additional judgment-relevant information could improve accuracy, but confidence increased more rapidly, creating a growing gap between how certain people felt and how accurate their judgments actually were.

What makes the phenomenon interesting is the gap that can open between the feeling of understanding and the quality of the judgment itself. Information may improve both, but not necessarily at the same rate.

Perhaps more information does two things at once. It can improve our understanding of the world, but it can also improve the story we are able to tell ourselves about our understanding. Those are not necessarily the same thing.

More information makes the story feel more complete

Information rarely arrives into an empty mind.

By the time we ask for another analysis, we usually have some sense of what might be happening. Sales may be slowing because the price is too high; a campaign may be underperforming because the creative is weak; a market may look attractive because demand appears to be growing.

At first, these are hypotheses. But as information accumulates, something subtle can happen: the hypothesis begins organising the evidence around itself. A customer comment, a competitor move or a particular cut of the data appears consistent with the emerging explanation, while evidence that does not fit can begin to look like an exception. Gradually, what started as one possible explanation can start to feel like the explanation.

Recent research by Peter J. Boyle and Pete Nye provides an interesting example of this in a business setting. In a managerial decision task, once one option began to lead, participants increasingly interpreted subsequent information as supporting it, and confidence rose alongside that distortion. The pattern could emerge even when people began without a strong preference.

I have seen versions of this play out in analytics work. An initial explanation begins to gain momentum and, as more analysis is added, numbers, segments and historical comparisons start to organise themselves around a coherent account of the problem. The analysis may genuinely be improving, but the emerging story is also beginning to shape how subsequent evidence is interpreted.

That is what makes coherence persuasive. A fragmented collection of observations feels uncertain, but a story that connects them feels like understanding, and the more evidence the story can absorb, the more convincing it becomes.

Yet a theory’s ability to accommodate more evidence is not automatically proof that it is becoming more accurate. Sometimes it simply means the explanation has become flexible enough to account for almost anything.

This may be one reason information can increase confidence faster than accuracy. More information can make a hypothesis easier to explain without making it more likely to be true.

AI makes information more accessible and, just as importantly, extraordinarily easy to organise into a coherent narrative. Give an AI system a dataset and a hypothesis and it can surface patterns that appear to support it. Give it contradictory evidence and it can often reconcile the pieces into a neat synthesis within seconds.

That capability is genuinely useful. But it also changes the economics of explanation. Plausible narratives that once required hours of analysis can now be generated almost instantly.

A recent preprint by Hoyoung Lee and collaborators explored one version of this problem in financial analysis. They found that AI-generated summaries of financial filings and earnings calls could remain fluent and factually plausible while omitting context that mattered to downstream investment decisions. The summary could therefore become easier to read and reason from while preserving less of the information needed to make a good judgment.

That distinction matters here. AI can reduce the complexity of evidence and make an interpretation feel more coherent, but compression can also strip away signals of uncertainty, qualification or context that should influence the decision. The result may feel like greater understanding even when some of the evidence needed for judgment has been lost.

A useful test is to ask what the additional information is actually doing. Does it discriminate between competing explanations, challenge an important assumption, or simply add support to the story we already have?

We confuse more information with more useful information

Even when additional information tests our explanation, another question remains: is it actually capable of improving the decision?

A piece of information can feel highly relevant to a decision without having much power to change it. We might want another customer survey, a more detailed forecast or one more cut of the data because each appears connected to the problem. But connection alone does not make information valuable.

Decision theory offers a useful distinction here. Information has value when learning it could lead us to make a better choice. That suggests a simple question before requesting more analysis: What would we need to learn for us to do something differently?

Psychologists Anthony Bastardi and Eldar Shafir explored a related tendency in experiments on what they called noninstrumental information: information that appears relevant to a choice but should not actually affect it. Participants were often willing to delay a decision to obtain such information, and having waited for it, tended to give it weight in the decision anyway.

There is something recognisable in that behaviour. Once we have gone looking for a piece of information, it becomes harder to treat what we find as irrelevant.

This matters in organisations because “we need more data” does not always mean the same thing. Sometimes there is a genuine uncertainty that additional evidence could resolve. At other times, it may mean we are simply not yet comfortable making the decision.

The first problem is informational. The second is psychological.

More analysis can also serve other purposes: building consensus, making a recommendation easier to defend or giving a decision greater legitimacy. Those functions are real, but they are different from reducing uncertainty. The problem begins when we mistake evidence gathered to create alignment or defensibility for evidence that materially improves the decision.

AI makes this distinction more important because the cost of producing additional analysis is falling rapidly. We can ask for another scenario, interpretation or supporting argument almost instantly. The benefit is obvious: genuinely useful evidence becomes easier to obtain. The risk is that analysis can keep expanding after the decision-relevant uncertainty has stopped shrinking, because there is very little friction telling us when another answer is no longer adding much value.

There is always another question to ask. At some point, though, strategy requires making a decision under uncertainty rather than eliminating uncertainty altogether.

That makes a different question useful, “What could we learn that would actually change what we believe or what we do?”

Information quantity is easier to see than information quality

Even when information could matter to the decision, another problem remains: it is much easier to see how much information we have than how much signal it contains.

We can see that a dashboard has thirty metrics instead of five, that a market study contains hundreds of responses, or that an analysis draws on ten different sources. What is harder to judge is whether those additional pieces of information are genuinely independent, diagnostic or relevant to the decision.

That distinction matters because more evidence is not always more signal.

Crystal Hall, Lynn Ariss and Alexander Todorov demonstrated this in a series of experiments involving predictions of basketball games. Participants were given useful statistics such as team records and halftime scores. Some were also shown the names of the teams. That additional information made knowledgeable basketball fans more confident in their predictions, but less accurate, because familiarity with particular teams drew attention away from the more predictive statistical cues.

The information was real. It was relevant to the subject. It simply was not as useful for the prediction as people believed.

I have seen a similar problem in analytics, although in a less controlled form. Different metrics, customer segments or research sources can appear to provide independent confirmation while actually reflecting much of the same underlying behaviour. We can end up counting the same signal several times because it reaches us through different measures. What looks like triangulation may simply be the same signal reflected through several mirrors.

More information gives us more ways to explain ourselves into being right

As the number of signals grows, so does the number of plausible stories we can construct about what happened.

Anyone who has worked with models knows the danger of overfitting. A model can fit the historical data extremely well and still perform poorly when it encounters something new. I wonder if our explanations can sometimes behave in a similar way.

With enough variables, customer segments, anecdotes and contextual detail, almost any business outcome can be made to look understandable after the fact. Revenue fell because pricing changed, because a competitor moved, because the customer mix shifted, because the market softened, because execution was uneven. The more information we have, the easier it becomes to build a story in which everything seems to fit.

I think of this as narrative overfitting. It is an analogy rather than an established psychological term, but the distinction it points to matters: an explanation can fit what has already happened without necessarily helping us understand what will happen next.

In her paper ‘To Explain or to Predict?’, statistician Galit Shmueli draws an important distinction between explanation and prediction. A model that explains observed data well is not necessarily the model that predicts unseen outcomes best; the two are different goals and can require different approaches.

Human judgment has its own version of this problem. Baruch Fischhoff’s classic work on hindsight showed that once people know how an event turned out, the outcome can appear more predictable than it did beforehand. Knowing what happened changes how we interpret the evidence that came before it.

That matters in strategy because explaining the past and deciding under uncertainty are not the same task. A detailed account of why a market changed may be entirely plausible, but a harder test is whether the same reasoning would have helped us anticipate the change before we knew the outcome, and whether the underlying relationships are likely to hold in the future.

AI adds another dimension to this problem by making alternative explanations extraordinarily cheap to generate. For the same outcome, we can now produce multiple plausible accounts, each supported by different patterns, assumptions or pieces of evidence. That is useful for exploration, but it also raises the bar for what should count as a convincing explanation: plausibility alone tells us less when plausible stories are abundant.

When explanations are cheap, coherence alone becomes weaker evidence of understanding.

Across these examples, the same distinction keeps resurfacing. Information can improve our model of reality, but it can also increase our confidence in the model we already have. From inside the decision, those two effects can feel remarkably similar.

Confidence and accuracy come from different places

Confidence can come from coherence, familiarity, detail, consensus or simply having spent a long time with a problem. Accuracy depends on something less comfortable: whether our understanding actually corresponds with what happens in the world.

Daniel Kahneman and Gary Klein explored this distinction in their work on expert intuition. Despite coming from different traditions, they agreed that reliable expertise is most likely to develop under two conditions: the environment contains patterns that are sufficiently regular to learn, and people have repeated opportunities to receive feedback on their judgments. Subjective confidence on its own is not a reliable indication that those conditions exist.

That distinction matters in business because not all decisions offer the same quality of feedback.

In analytics, we often work on problems where the consequences become visible relatively quickly. A campaign runs, conversion changes, churn rises or falls, or a forecast can eventually be compared with what actually happened. Over time, those feedback loops give us opportunities to discover where our judgment was wrong and adjust it.

Strategy often operates in a much weaker feedback environment. A market-entry decision may take years to play out. Competitors respond, economic conditions change and there is rarely a clean counterfactual showing what would have happened had we chosen differently. By the time the outcome is visible, it can be difficult to separate the quality of the original decision from everything that happened afterwards.

Yet we may feel just as confident in both environments.

AI adds another complication. It can make an analysis feel more comprehensive, an argument more coherent and a recommendation more polished. All of those things can increase our sense that the problem has been understood, even when the underlying environment remains difficult to predict.

The more useful question may be what has earned our confidence. Have we made similar judgments before, received meaningful feedback and worked in an environment stable enough for experience to teach us something about the future?

Confidence built under those conditions is very different from confidence that comes from a compelling explanation alone. The two may feel similar from inside the decision, even when reality treats them very differently.

So when does more information actually help?

There is an obvious danger in taking this argument too far. More information is often exactly what improves a decision. Better data can reveal a false assumption, expose a hidden pattern, establish a useful base rate or show that an apparently simple problem behaves differently across customers, markets or contexts.

Research by Shenghua Luan, Jochen Reb and Gerd Gigerenzer on managerial decision-making under uncertainty is useful here because it shows that the value of information depends on the structure of the problem. In their work on personnel selection, a relatively simple heuristic using fewer cues could outperform a more complex statistical approach under certain conditions. In other situations, another customer segment, variable or larger sample can reveal something the existing evidence missed.

AI can strengthen this side of the equation too. It can inspect volumes of information that would previously have been impractical to analyse, surface outliers, compare alternative explanations and make sophisticated analytical techniques more accessible. Used that way, additional information can genuinely challenge our current model rather than simply decorate it.

What matters is the effect the information has on the relationship between our beliefs and reality. Some evidence improves that relationship, other evidence mainly increases our confidence in the beliefs we already hold.

What changes when information becomes abundant?

Historically, obtaining and analysing information imposed real constraints on business decisions. Research took time, analysis required specialist skills, and answering another question often meant waiting days or weeks.

AI is reducing many of those constraints quickly. We can now ask for another interpretation or explanation almost immediately. The cost of producing analysis is falling, and with it the friction that once forced us to decide when we had enough.

That should create enormous value, while also shifting the bottleneck.

When obtaining and analysing information required more time and effort, those costs created natural limits on how much analysis could be done. As those limits fall, the harder problem becomes deciding which information deserves attention, which deserves belief and which should actually change the decision.

Early research on AI-assisted decision-making illustrates both sides of that shift. In two experiments comparing LLM-based search with traditional search, researchers at Microsoft found that people using LLM-based search completed tasks faster and reported a more satisfying experience. But when the system provided incorrect information, users were also more likely to rely on it, reducing decision accuracy.

That tension feels important. AI can dramatically improve our ability to find and process information without necessarily improving our ability to judge every piece of it.

Instead of asking AI only to help us complete the picture, we can also ask it to make the picture harder to believe: What evidence contradicts this explanation? What base rate are we ignoring? Which assumptions are doing most of the work? What would have to be true for the opposite conclusion to hold?

Used that way, AI can lower two costs at once: the cost of obtaining information and the cost of challenging our own interpretation of it.

As answers become easier to produce, the advantage may shift towards knowing which questions an answer must survive before it deserves our confidence.

What deserves our confidence?

I began with a simple question: why can more information make us feel more certain without making us more right?

The more I have thought about this question, the more important the quality and purpose of information seem to become. Better information can improve decisions enormously. Yet the accumulation of evidence can also make explanations more coherent, models more elaborate and confidence stronger, while understanding improves much less.

Those effects are easy to confuse because, from inside the decision, they often feel the same.

Having spent years around data and insights, I come away from this wanting analysis to work harder. A fuller picture or a more defensible recommendation has value, but the deeper test is whether the analysis improves the connection between what we believe and what is actually true.

AI makes that discipline more important by making information and explanation much easier to produce. The challenge is increasingly to decide what deserves weight, what deserves doubt and what should genuinely change our view.

So before asking whether we have enough information to make a decision, it may be worth asking a harder question: what evidence would genuinely cause us to change our mind?

If we cannot answer that, more information may make us more certain without making us more right.

• Strategy
• Decision-Making
• Data
• AI