Research shows how unreliable it is


April 22, 1991|By TOM PETERS | TOM PETERS,1991 TPG Communications

Managers get paid to make judgments. Human judgment stinks. Add those two statements up and you've got trouble.

"Judgment Under Uncertainty," edited by professors Daniel Kahneman, Paul Slovic and Amos Tversky, is a revelation. In 555 well-documented pages, they present a persuasive case for the unreliability of judgment:

* Anchoring. Take a simple case. Subjects were given a multiplication to perform: One set got 8x7x6x5x4x3x2x1; another, 1x2x3x4x5x6x7x8. The two are clearly equivalent. The catch: Subjects had five seconds to make an estimate. The average answer from the 8x7x . . . group was 2,250; the 1x2x . . . group, 512. (The correct response is 40,320.) Welcome to the world of "anchoring": Where you finish depends on where you start. When eight is on the left, where we normally begin, we lock on to it and offer an answer four times higher than those "anchored" on the left by the numeral one.

* Focus on the case, forget the context. Researchers label this the fundamental information-processing error. Subjects in one experiment were asked to read descriptions purportedly taken from a pool of 70 engineers and 30 lawyers (or 30 engineers, 70 lawyers). After reading a dossier made to sound somewhat "lawyer-like" or "engineer-like," the subjects were asked what the odds were that the description was from a lawyer or an engineer. Judgments were overwhelmed by the "story" and insensitive to the "background information" -- respondents completely ignored the statistical likelihood that weighed the odds 70-30 in one direction or the other.

* The illusion of control. "[People] behave as though chance events are subject to control," social psychologist Ellen Langer writes. In one experiment, members of adjacent offices were given an opportunity to buy tickets for a lottery. Participants in one group chose their ticket (a card emblazoned with a football player's name, team and picture) from a box. Participants in the other group were simply handed a card of the same type.

Later, participants were asked how much they'd want for selling their ticket back, prior to the drawing. People who had selected a card wanted $8.67 on average; those handed a card asked $1.96. The effect of self-selection -- in this purely chance event -- upped perceived ticket value by a factor of 4.4!

* Immutability of opinions. Confronting data that contradicts our opinion hardens rather than softens that opinion.

For example, a group supporting capital punishment reviewed "pro" and "con" studies: After perusing conclusions to the "pro" study, strength of conviction increased slightly; it remained up after examining the methodology. Upon subsequently reading the "con" study's conclusion, a minuscule drop in strength of conviction was registered. But upon scrutinizing the con study's methodology (equivalent to the pro study), the subjects' pro conviction soared to new heights.

* We ain't as complex as we think. Studies using elementary mathematical models that incorporate experts' past decisions readily predict their future decisions in complex matters. Experts get stuck in grooves, though they invariably insist they're making subtle judgments.

* Multistage events. Almost all of business life involves betting on complex, multistep events. And while research shows that we're conservative in assessing a single-step event, we are absurdly optimistic about compound doings.

Suppose a project consists of 10 key steps; further imagine that the odds of making it through any one step is 70 percent. What are the odds of surviving all 10 steps? The correct probability is .7x.7x.7x.7x.7x.7x.7x.7x.7x.7 -- or 3 percent. If success odds at any one stage are 50 percent, the overall chance of success (.5x.5 ...) plummets to one-tenth of 1 percent.

Now the research: In a typical experiment involving two to eight stages and 25 different conditions, subjects overestimated success odds by a factor of two (in a simple, two-stage state) to a factor of 900,000 (in a complex, eight-stage situation).

In short, we are rotten judges. Moreover, the professional judgment of doctors, auto mechanics and geophysicists proves only about as reliable as that of laymen.

If outrageously poor judgment is pervasive, perhaps it's one more reason for managers to empower lots of people to "get on with it." Though the judgment of those empowered is not likely to be much good either, they are closer to the action, deal with slightly less complex events and receive less distorted information.

At a minimum, such a strategy increases the number of people who try stuff. That's not a very scintillating solution, but it may be better than relying on the plans and commands of a small number of self-deluded "experts."

Baltimore Sun Articles
Please note the green-lined linked article text has been applied commercially without any involvement from our newsroom editors, reporters or any other editorial staff.