The Case for Healthy Skepticism in K–12

Why schools must stop falling for bad ideas, and how we can identify them

Why do we shy away from skepticism in K–12?

We have a noble desire to find what works and make things work for our students under conditions of uncertainty.

In a profession fueled by hope for our students, skepticism feels too much like cynicism.

(Indeed, the most vocal skeptics on any given school faculty are often the worst cynics.)

But we don’t need to be cynical about our students or skeptical of their ability to succeed.

We just need to be skeptical about the unending slew of opportunities pushed on our profession.

Wanting to do right by our students is good. Wanting to improve is good.

But improvement doesn’t occur in a vacuum; it occurs relative to an existing baseline—a status quo.

If we want to “beat the high score” of the status quo, skepticism is our best toolkit.

Appreciating The Status Quo

The term status quo is used pejoratively in most discussions of educational improvement.

Obviously there is no reason to believe that what we’re doing now is the best that we could possibly be doing.

It would indeed be cynical to deny that improvement is possible.

But it would be equally catastrophic to throw out the status quo in favor of something—anything!—new, no matter how unproven.

The status quo is not the problem—it’s the current reality. It’s the baseline.

And if we want to improve on the status quo, new solutions must be better, not merely plausible.

As I said on Dr. Phil:

Whenever we criticize the status quo, we have to realize that the good results that we’re getting are coming from the status quo as well as the bad results. And if we’re going to shift to something completely different, we have to make sure that that alternative can deliver the good results that the status quo is currently delivering as well as address the shortcomings of it.

What standards should we have for proposed improvements on the status quo?

Proving That Something New Is Actually Better

If we want to beat the status quo, we’ll need to do so on two primary dimensions:

1. Effectiveness—the innovation must actually work better

2. Unintended consequences—the innovation must not have side effects that make it a net negative

And of course, we’ll at some point need to consider ROI—cost-effectiveness compared to the alternatives—but that can wait.

For example, let’s say we decide to give every student a laptop, citing studies like this one:

Will it work? Well, 0.18 is not a very impressive effect size—certainly not enough to justify giving out 3 million laptops.

At issue here is opportunity cost.

Everything takes time and/or costs money, and we could always do something else (including the status quo) with that time and money.

But that’s not the only question that matters—we must also concern ourselves with unintended consequences.

In his new book The Digital Delusion, neuroscientist Jared Cooney Horvath argues that ed tech is demonstrably making kids dumber:

Today, children are spending ever more hours in classrooms, yet they’re developing more slowly. The culprit lies in the meteoric rise of educational technology.

Over the past two decades, educational technology has exploded from a niche supplement into a $400 billion juggernaut woven into nearly every corner of schooling.

More than half of all students now use a computer at school for one to four hours each day, and a full quarter spend more than four hours on screens during a typical seven-hour school day.

Researchers estimate that less than half of this time is spent actually learning, with students drifting off-task up to 38 minutes of every hour when on classroom devices. Full chapter excerpt »

Both of these issues—effectiveness and unintended consequences—are widely ignored.

We’re not satisfied with the status quo, and we’re not skeptical enough about new ideas, so we race headlong into the next big thing, over and over.

Some notable examples across a wide range of issues:

• LAUSD spent $1 billion on iPads, with nothing to show for it

• Balanced Literacy crowded out proven approaches like phonics, despite no evidence of its effectiveness

• Attempts to reform school discipline have caused behavior to deteriorate rather than improve

• Attempts to reform grading have inflated grades without improving learning

• Trauma-informed approaches make students’ mental health worse, not better

• Many students now learn from literacy curriculum that features no whole books

These specific issues have nothing in common, except that we adopted them broadly without carefully studying their impact—or in some cases, despite clear red flags in the existing research base and common sense.

It doesn’t have to be this way.

Skepticism is not expensive.

It takes some time, yes, but it’s not too much to ask that a costly, disruptive, or otherwise potentially consequential change actually work.

So why isn’t skepticism the norm in K–12 education?

Structural Incentives Against Skepticism

Let’s quickly run through some of the reasons nobody is particularly incentivized to be skeptical about change in K–12 education:

• Among researchers, there are strong career incentives to publish novel results, rather than null findings or replications of existing studies. That’s why Bloom’s 2-Sigma paper is famous, but its failure to replicate is not.

• There is simply no research on thousands of issues schools must make decisions about—none at all—and even when there is research, educators often lack the time and expertise to review it before making a decision.

• Among school administrators, there are strong career incentives to implement new ideas, and these incentives operate faster than results become clear. Find a bad idea from 10 years ago, and chances are good that the responsible person has been promoted twice since then.

• Among teachers, there are strong career incentives to comply with new initiatives, and no corresponding incentives to raise concerns. Whether you’re right or wrong, you’ll likely face immediate consequences for pushing back, and if you’re ultimately right, no one will make it up to you.

• In the EdTech industry, profit is enough—a product doesn’t have to work as long as people will buy it. In fact, a company doesn’t even have to be profitable if the founders can sell their shares in time. It’s not only profitable to sell hype; in many cases, it’s the most logical plan for making the most money in the shortest time.

• Consultants (self-explanatory; pretend I wrote this in a much bigger font)

• New initiatives are immediately exploited for PR and marketing value, making everyone less likely to express skepticism once things start happening.

• Districts aren’t incentivized to own up to their bad investments; it seems crass for senior leaders to criticize their predecessors, and taxpayers are likely to ask “Then why should we give you even more money?”

• Policy analysis tends to get intertwined with politics, and tribal allegiances tend to override sober analysis.

• People tend to care much more about ideological concerns than effectiveness and unintended consequences—for example, people are likely to have very strong opinions on whether schools should address mental health in schools, without considering empirical findings.

• Old-fashioned kickbacks, corruption, district/vendor revolving doors, and perfectly legal campaign contributions are all skewed toward spending money making things happen rather than being skeptical about new ideas and maintaining the status quo.

• Hero worship—when you fly across the country to sit in a church and learn from Lucy Calkins like I did, you’re unlikely to take skepticism of her ideas seriously.

• Skepticism is not really anybody’s job—no one is paying me to write this article, and I’m likely pre-burning some bridges that might have been lucrative.

• Most people in education are pretty nice, and skepticism doesn’t seem nice.

Leave a comment if you think of other factors I haven’t touched on.

So what should we do about all of this?

How We Can Implement Healthy Skepticism

My goal with this website is to begin an ongoing and broad-reaching conversation about the proper role of skepticism in K–12.

Please leave a comment or email me at justin@principalcenter.com with your suggestions.

To start the discussion, I’ll make a few suggestions:

• Treat the status quo as the “high score” that any new idea must beat. If it can’t, it’s not worth anything more than an experiment.

• Reject the urgency of the problem as a rationale for a specific solution. Just because we need to do something doesn’t mean this is it.

• Don’t take it for granted that there is a solution at hand. Instead, describe the likely features of the kind of solution you have in mind, then consider the real possibility that it will not clear the bar—the status quo may in fact win.

• Admit that we’re constantly experimenting on kids when we try new things, and take that experimentation more seriously.

• Pre-register our hypotheses, so we can’t move the goalposts, and stop getting distracted by questions like “Do the teachers like it?” or “Did we get the grant?” or “Is our company making money?”

• Assume that everything we do will have unintended consequences, and develop a plan for detecting, measuring, and mitigating them.

• Talk to people outside of your industry, who don’t face any of the same structural incentives against skepticism.

• Talk to people in other sectors, who face different incentives and have different forms of skepticism. Policymakers and vendors and researchers and teachers and administrators should not all work in silos.

• Start with common sense, values, and aesthetics, and consider rejecting some ideas on those grounds, in the absence of overwhelming evidence, and wait for that evidence to arrive before weighing it against your priors.

On this last point, imagine that your friend tells you about an iPad app that can help your 2-year-old learn to read.

It might work. Your friend might give a glowing testimonial.

But you might say “Wait a minute. Under no circumstances will I stick my kid on an iPad.”

Your friend might reply that the results speak for themselves.

But you can remind your friend about the likelihood of unintended consequences: “It may ‘work’ but I’m also concerned about the side effects of early screen use.”

Yes, your kid might miss out on an edge of some sort. But it’s not negligent to wait for something new to clear the bar.

It’s just healthy skepticism.