Look, I get it. Really, I get it.
You saw this headline and maybe you clicked on it because you just watched a game and your voice is sore because THAT UMPIRE MISSED SO MANY CALLS and now you’re rushing to the comment section to say umpires are awful and by god when are we finally going to have an automated strike zone?
Well first off, let me assuage your concerns a little bit by telling you it sounds like an automated strike zone is on its way, at least according to Rob Manfred, so don’t worry. But I’m here to tell you that, based on the data we’ve pulled together, it doesn’t seem like an automated strike zone would make all that much of a difference, because despite what you might think, umpires are really good at calling balls and strikes, and they’ve been steadily getting better.
Just How Accurate Are Umpires?
Before we dive into the data, I want to give a shoutout to our own Andrew Kurtz who first mentioned this idea and put together this data for me. He pulled the numbers, I just wrote the pretty words (and it’s debatable just how pretty those words actually are).
Moving on—accurately calling balls and strikes at the Major League level seems like an almost Herculean task to a layman like me, especially as pitchers are throwing fastballs harder than ever, throwing breaking balls with more movement than ever, and catchers are emphasizing pitch framing more than in past years.
Yet the data we’ve pulled has found that not only are umpires really good at accurately calling balls and strikes, but they’ve been steadily getting better.
Using Statcast data going back as far as pitch tracking goes (which is 2008), here’s how umpire accuracy looks year-to-year:
In 2008, 8.1% of pitches that should have been called strikes were actually called balls (which we’re calling “missed strikes” for the context of this article), and 7.7% of pitches that should have been called balls were called strikes (which we’re calling “missed balls”).
But in every season since then, the percentage of missed strikes and missed balls has steadily dropped. Last year, umpires only missed 3.3% of strikes and 3.9% of balls. That’s roughly a 50% improvement between 2008 and 2023.
Now, if you’re like me, the very next question you’re asking is “How accurate were umpires on the balls and strikes that were hardest to call?”
We figured that out too! And we used Statcast’s Attack Zones to do it, which look like this:
Unsurprisingly, umpires virtually never miss calls in the Heart zone. That said, they’ve still improved there—missed strikes in the Heart zone dropped from 1.1% in 2008 to 0.1% in 2023. Similarly, umpires literally never (as in 0.0% of the time) miss calls in the Waste zone.
Meanwhile, in the Chase zone, umpires do miss calls sometimes, but again, they’ve gotten significantly better. The percentage of missed balls in the Chase zone dropped from 1.7% in 2008 to 0.1% in 2023.
But it’s the Shadow zone you and I are interested in. Those are the borderline calls that are arguably the most difficult ones to call correctly. Unsurprisingly, it’s also the zone umpires have the highest percentage of missed balls and strikes in, but yet again, they’ve gotten significantly better at making accurate calls even in the Shadow zone:
The percentage of missed strikes in the Shadow zone dropped from 6.9% in 2008 to 3.2% in 2023 and the percentage of missed balls dropped from 5.9% to 3.8% in the same time period. And again, umpires saw improvement every single year in missed balls and strikes in every single zone between 2008 and 2023.
Now earlier I mentioned missing the balls and strikes that were the hardest to call, and that not only applies to where the pitches are located but what kind of pitches they are. It stands to reason that a big loopy curveball or a slider with tons of break on it is probably a bit more difficult to call accurately than a pitch that comes in straight as an arrow.
However, we found the opposite to be true. Here’s a breakdown of missed strikes and missed balls by pitch type for 2023:
Surprisingly enough, it was fastballs that umpires had a harder time with than breaking balls or offspeed pitches.
Now this is all well and good, but what about missed calls in crucial situations? That’s when an umpire’s missed call can really screw things up. If they miss a call on an 0-0 count, who cares, but if they miss a strike on a 3-2 count giving a walk instead of a strikeout, that can have a significant impact on the game.
Well if you’re noticing the trend in this article, then I’m sure you can guess that umpires have gotten better year-over-year at accurately calling balls and strikes in every count. But what’s really interesting is, umpires get better as they get further into an at-bat.
Here’s the percentage of missed balls and strikes from last year by count:
The highest percentage of missed calls are in 0-0 counts, but after that, umpires really crack down, and in crucial counts like two-strike or three-ball counts, they almost never miss.
We saw a similar trend in high-pressure situations as well, including in extra innings and when there are runners in scoring position. Again, missing a call in the second inning with no runners on is one thing, but missing a call when bases are loaded in the bottom of the 10th is a whole different thing.
But again, umpires really crack down in these situations. In extra innings, we found that umpires have improved year-to-year once again in missed calls, but what’s interesting is, they weren’t missing all that many calls in extra innings as it was. In 2008, umpires missed 0.2% of balls and 0.2% of strikes, but by 2023, that number was below 0.1%.
As for situations where runners are in scoring position, here’s the year-over-year trend for missed calls:
Yet again, umpires have improved and are currently better than ever at it.
I want to take a moment to talk about why we may have this perception that umpires are really bad at their jobs or have gotten worse when the data suggests otherwise.
There’s this theory that often gets discussed in politics that goes something like this: if you talk to your average voter and ask them about the state of the country, regardless of what party is in power, you’re always going to get a good chunk of respondents who say they’re concerned about the state of the nation or disapprove of the direction it’s going in.
And when you dive more into what they disapprove of, oftentimes they’ll respond by saying that crime is bad in the United States, often they’ll even say that crime has gotten worse over the years. I know I’ve heard a million conversations where someone says “oh well nowadays you have to be more careful, there are a lot of awful people out there,” suggesting there weren’t in the past.
The thing is though, generally speaking, violent crime has been on the decline for many years, yet many people think crime has actually gotten worse. Why is that? Because unlike 30-40 years ago, right now we have 24-hour news channels and the internet to amplify every single crime that we otherwise wouldn’t have heard about in the past—we’re more aware of crime, which tricks us into thinking there’s more of it.
So why on earth did I just ramble for three paragraphs about the public’s misconceptions about crime rates in the United States? Because I think there’s a similar phenomenon going on with umpires.
Now more than ever, we’re acutely aware of whenever umpires make bad calls. Thanks to being able to watch basically every single game in Major League Baseball and how prevalent clips of games and discussions of them are on social media, we’re hyper-aware of umpires’ mistakes more than we ever have been before. I mean, there’s an entire Twitter account dedicated to developing scorecards for every umpire (they’re an awesome account and you should follow them).
All of this means that even if umpires are making fewer mistakes than they ever have before (and as we’ve established, that is the case) when umpires do make mistakes, especially egregious ones, even if they’re pretty rare in the grand scheme of things, those mistakes get significantly amplified.
So ultimately, what’s the point of all of this? I’m not asking you to get a tattoo of Ángel Hernández over your heart, nor am I secretly an umpire in a fake mustache writing umpire propaganda.
I think it’s important to remember the data when we’re having the discussion about automatic strike zones because those automatic strike zones seem to be coming whether we want them to or not, and based on the data, it doesn’t seem like they’re going to suddenly be a panacea to all of our woes.
Credit where credit is due—umpires are consistently getting better at doing the exceptionally hard job of accurately calling balls and strikes, and right now, they’re better than ever at it.
