Where modern journalism failed: its lack of understanding shell games.

Watching Betsy DeVos's 60 Minutes debacle was an exercise in not getting distracted by an interviewee as you are deconstructing a seasoned interviewer, looking at the deficits others will miss because they are prime to look at an interviewee's flaws. But Lesley Stahl's own blunders in her base assumptions reminded me of why journalism horrifically failed.

clock

For a program that uses the symbol of measurement -- in this case, time -- they do not do a very good job making careful measurements themselves.

To country DeVos at one point, Stahl appeals to authority by stating that test scores in a certain state went up, meaning the DeVos' argument of school choice was flawed.

But let's take a look at what a collective of scores actually means.

Not every student got the average score. Not every student got the medium score.

So that means some students performed better than average, and others performed worse.

And that also means some schools performed better than the average, and some performed worse.

But we do not know if some schools performed worse than on previous tests, or the higher-performing ones pulled up the average, masking troubles simmering underneath.

We can look at what separates the high performing ones from the ones that did worse: did the economic fortunes in certain areas raise or lower the scores? Is there some mitigating factors we need to be aware of? An influx of immigrants? Gentrification? Redistricting? A new company hiring people -- or ones that are closing?

So even within a cluster of scores, we still have big variations.

Perhaps some bad teachers retired and a younger crop replaced them, making a difference.

Or, perhaps good teachers retired, and the younger crop -- who may be worse -- replaced them.

Or they let go teachers or hired more teachers.

Perhaps new schools opened -- or old ones closed, and they were amalgamated.

So we have so many factors we would have to consider to see why we have a variance -- as well as see if test scores are really going up -- or are things being fudged to make it appear that way.

You cannot just take test scores at face value -- but try to figure out what environmental factors can account for the changes.

And it is relevant to the DeVos interview -- if certain schools are doing worse, even when others are doing better -- we need to why it is happening -- just as we need to know why some schools are bringing up the average, so to speak.

A journalist does have a duty to address alternative theories when making a case in a report.

If someone brags that housing prices are going through the roof -- as a journalist, I cannot be one if I just blindly chirp those numbers.

I have to look at who is buying those houses.

Is it foreign investors looking to park their money?

Is it a network of con artists?

Is it people out of town?

Is it just investors?

Or is it that it is citizens in the region who are doing better financially?

You have to do your best to cross off as many alternative explanations as you can. You cannot make assumptions because behind every number, there is a world of interactions going on, making the statistic in question a mere shell.

A good journalist cracks open the shell first to make sure that those numbers add up and properly reflect reality. That is not as hard as it sounds.

But reporters don't do that. If the numbers sound good and fit that narrative, they run with it. It is not responsible journalism. It was never empirical in its approach, nor is it actually curious.

So while people were fixated one DeVos, they missed the big picture -- that despite living in decades in a place where there isn't war and all the opportunity to learn in schools and outside them -- we have adults not doing heavy thinking at all.

And we all need to overhaul our critical thinking skills to better navigate -- especially reporters.