Good Data and Flawed Conclusions - WSJ.com

Excerpt from an informative piece in today's WSJ by Cari Tuna

So how can the overall unemployment rate be lower today but higher among each group? The anomaly is an example of Simpson's Paradox -- a common but misleading statistical phenomenon rooted in the differing sizes of subgroups. Put simply, Simpson's Paradox reveals that aggregated data can appear to reverse important trends in the numbers being combined.

The jobless rates for each educational subgroup are higher today, but the overall rate is lower because workers are more educated. There are more college graduates, who have the lowest unemployment rate. And there are fewer high-school dropouts, who have the highest unemployment rate.

"It's the magic of weighted averages," says Princeton University economics professor Henry Farber. "We have more skilled workers than we had before, and more-skilled workers are less susceptible to unemployment." Still, he adds, compared with a similarly educated worker in 1983, "the worker today has higher unemployment at every education level."

via online.wsj.com

In planning, one of our most important functions is to interpret data dispassionately and with insight. Of course, when you make your living in the advertising business, this can sometimes be a challenge.

This is a very nice piece that looks at an important aspect of interpreting hard data, namely, why it is essential to look at weighted averages when you're looking at aggregated data. It's a reminder that what you see on the surface doesn't always tell you the truth.

There is a corollary to this problem, as well. Much of the data we get in planning is from secondary sources. We read a report here, a study there, but how often do we really know how sound the methodology was behind any of it? It's useful to keep in mind that reports like this are written by journalists with little background in understanding the fine points of methodology. They're looking for a story--a headline that telegraphs their own discovery of insight.

Planners can't fall into that trap. Insight is only insight if it's true, not because we get up on stage and claim that it is.

Comments

[ed hardy]

Yeah, I think it's hard to see how traditional trade publications can survive much longer in print