Custom Search

Friday, November 05, 2010

Top pundits: How can they consistently score higher than chance at being wrong?

They could start by striving to be the Big Expert everyone wants to quote in their paper or book for their show- and then believing their own hype.

Toronto-based Canadian journalist David Warren comments on the value (?) of punditry in predicting outcomes in the recent American election:

A piece in the Wall Street Journal over the weekend went right to the point, of pundits and their predictions. Jonah Lehrer cited a study done by a California psychologist, over a quarter century, of 284 pundits predicting political and economic trends. He did not concern himself with little things, only with big and simple ones, where the writer could be right or wrong, unambiguously -- so that results could be quantified. It came down to scoring 82,361 yes/no propositions.

As a pundit myself, let me claim I could have guessed the result. When making predictions, pundits scored significantly below random. That is to say, they would have done much better had they just flipped a coin, in each case, instead of trying to reason their way to an answer.

The good news is, anyone who took the consensus of the pundits, and assumed the exact opposite of their prediction in each and every case, would have scored significantly better than a random result.

For more, go here.

Sharon Begley for Newsweek suggests that if we can't shut pundits up, we can at least identify factors that are strongly correlated with being wrong:

At first, Tetlock's ongoing study of 82,361 predictions by 284 pundits (most but not all of them American) came up empty. He initially looked at whether accuracy was related to having a Ph.D., being an economist or political scientist rather than a blowhard journalist, having policy experience or access to classified information, or being a realist or neocon, liberal or conservative. The answers were no on all counts. The best predictor, in a backward sort of way, was fame: the more feted by the media, the worse a pundit's accuracy. And therein lay Tetlock's first clue. The media's preferred pundits are forceful, confident and decisive, not tentative and balanced.

[ ... ]

The media, of course, eat this up. Bold, decisive assertions make better sound bites; bombast, swagger and certainty make for better TV. As a result, the marketplace of ideas does not punish poor punditry. Few of us even remember who got what wrong. We are instead impressed by credentials, affiliation, fame and even looks—traits that have no bearing on a pundit's accuracy.
It's a good thing that the legacy media that inflate these balloons are on the way out.

It is all just so Darwin, isn't it? Much certainty, little evidence. Who would have guessed that predicting the future and postdicting the past could be practically indistinguishable?

Are pundits beating the odds - by scoring own goals - a legitimate argument for chance creation of information? No, because there is too little chance involved.

Warren is no fan of Darwin, and here are some of his articles on why not:

Coffee!! Oh, so now Darwinism explains why you think SHE'S beautiful? Where's my rolling pin?

Evolutionary psychology: David Warren on beautiful women
Darwinism and popular culture: Real biology vs. Darwinism

Human evolution: More on the Ida? I dunno ... files

Oppose Darwinism as scientific fraud? Well then, no birthday cake for YOU, David!

Who links to me?