This year, MLB is testing the Automated Ball-Strike System (ABS), (un)affectionately known as the “robo-ump,” at the AAA level. The move reflects growing trust in the technology since, much like the pitch-clock, the system has been gradually “called-up” through the minors with each passing season. Robo-umps have arrived, and their implementation in Major League Baseball seems a matter of when, not if.
Now, I’m no baseball purist, but I believe there’s been a lack of acknowledgment of the magnitude of the change that robo-umps represent. In this piece, I’ll tell you why and then pitch you an alternative: using ABS to provide umpires with real-time feedback in order to build on MLB’s recent strides in umpire accuracy.
Why not robo-umps?
Baseball fans are divided. Some shout for ABS every time a bad call goes against their team, while others lament the potential loss of the “human element”. A compromise gaining traction is an ABS-based challenge system, similar to the video challenges we’re now used to.
The challenge system seems like a win-win, but challenges can detract from the candor of the game. When calls on the field feel provisional, the excitement of the initial call dwindles and frustration mounts when challenges have run out.
In my view, bringing ABS on board isn’t just about preserving the “human element.” It’s about whether a game that’s steeped in probabilistic judgment can smoothly transition to automation. Imprecise strike-zones are often viewed as an unfortunate yet unavoidable human shortcoming. But here’s the thing – baseball’s strike-zone has inherently been probabilistic. A binary strike-zone is a mythical ideal that’s never truly existed.
Assuming ABS is consistent and precise, “computerized umpiring” might constitute the greatest re-definition of the strike-zone ever implemented, forcing an inherently probabilistic call to become a fixed entity. I would even wager that a binary strike-zone represents the greatest on-field rules change since the adoption of the foul-strike rule in 1903.
Potential Effects of ABS on Run Environment
Although the league has been testing ABS for years, the full impact ABS might have on the game remains speculative. We’ve seen measurements for pace of play and accuracy, but other forms of impact remain speculative.
This is, in part, because the run environment does not easily translate across different levels of play, but more importantly, because player development has not yet been optimized for the new game-calling system. We will only know the impact of the robo-ump several years after its MLB implementation, when teams have shifted player development to optimize performance in the new environment. Fortunately, we have some means of guessing how this development will shift.
Anecdotal accounts from minor-league players, for example, have already suggested an impact on the kinds of pitching that are effective. Some pitchers reported that success required, for example, pitching north-south rather than east-west, where pitch control and catcher framing allow pitchers to widen the zone.
If this is true, scouting, drafting, and developing pitchers will inevitably become biased toward pitchers whose arsenals and mechanics favor a north-south approach and whose stuff does not rely on expanding the zone.
This is further corroborated by an MA thesis conducted at the University of Nebraska-Lincoln in 2021, which used missed calls and RE24 to conclude that robo-umps would most negatively impact pitchers who rely on called-strikes. In a somewhat counterintuitive way, the depreciation of called strikes might mean a greater prioritization on swing-and-miss stuff and potentially counter recent MLB measures to increase balls-in-play.
As a result, the implementation of ABS will likely result in a series of tweaked strike zones, aimed at balancing the game for the new calling system. In other words, once robo-umps call the game, the zone designed for human umpires will likely need an overhaul.
And then there are the unknowns. Once the rules change, teams find ways to exploit them, and those means of exploitation aren’t always obvious.
Consider, for example, that defining the upper and lower boundaries of the strike zone is dependent upon player size and stance. The rulebook definition of the zone includes judgement-based language regarding the preparedness of the batter for the pitch: “The Strike Zone shall be determined from the batter’s stance as the batter is prepared to swing at a pitched ball.” This is something already being fine-tuned, surely, but we haven’t seen what teams will do to take advantage.
Will we see more radical mechanical changes to player stances and swing mechanics in order to narrow the zone? If the ABS system defines the zone at a precise moment of the swing, hitter will certainly try. Alternatively, if the ABS system re-defines the pitch more dynamically, how will pitchers respond to a zone that changes from pitch to pitch?
Robo-umps may be the best right path forward. It’s certainly a reasonable position and, so far, minor-league reports appear favorable. Still, I think it’s incumbent on the baseball community to reflect more deeply on a change this big. Have we sufficiently vetted the idea or are we charging headlong into a change we don’t fully understand? I’d like to think there are other options we have yet to sufficiently consider.
Allow me to propose an alternative that leverages the benefits of automation to enhance the game while retaining the probabilistic strike-zone that we all know and love (and hate). This solution, like the challenge system, is something of a middle-ground, but without the issues associated with provisional game calling.
What kind of calls do umpires miss?
To delve deeper, let’s examine the types of calls umpires typically miss. Specifically, I wanted to understand which kinds of balls are most often mistaken for strikes, since we know that umpires are generally better at calling strikes than they are at calling balls. This season’s data indicates that balls-called-strikes mostly result from an expansion of the zone:
The heat map above shows the density of balls-called-strikes by vertical and horizontal location. It’s evident that umpires mainly struggle with the east-west boundaries of the zone.
From this, we can infer a couple of things. Firstly, robo-umps will tighten the zone’s edges, which aligns with the anecdotes. Secondly, and perhaps more importantly, umpires are already pretty good at assessing the top and bottom limits of the zone based on hitters’ swing mechanics.
After chatting things over with Pitcher List’s Scott Chu, I decided to analyze the pitch movement associated with these balls-called-strikes. Looking at the pitches that were misjudged on the sides of the plate, I charted the horizontal movement profiles most commonly linked to balls-called-strikes for right-handed and left-handed pitchers.
The results are intriguing. Umpires seem to mainly struggle with arm-side movement, irrespective of the pitcher’s handedness. It’s not the back-door sliders that are racking up the most erroneous calls; it’s the pitches with arm-side run that look like they’re in the zone before veering out.
Cyber Umps and Immediate Feedback Intervention (IFI)
Over the last 15 years, umpires have made significant progress in game calling. As highlighted by Davy Andrews in FanGraphs, umpire accuracy in calling balls and strikes has climbed from 81.3% to 92.4%. This improvement is largely attributed to MLB’s umpire evaluation system, which offers feedback on the accuracy of calls.
While this kind of feedback intervention isn’t helpful in every area, there are specific contexts where the effects of immediate feedback are most beneficial: feedback provides the best results with simple task processing. The effects of feedback intervention are most optimal when feedback is clear, specific, immediate, and repeated. In a nutshell, MLB’s umpire-grading system has already proven the general benefits of feedback on umpire game-calling.
Here’s my proposition: with MLB’s Automated Ball-Strike system, why not offer umpires this feedback in real-time? Let’s call them “cyber umps”. The needed technology is already in place: MLB plans to employ an ABS relay system to allow umpires to convey automated calls on the field.
Cyber umps would involve relaying this information to the umpire immediately after they make their call, allowing them to recognize and correct any discernible trends in their missed calls. This could be done immediately after inaccurate calling or when the side is retired.
Evidence from various fields suggests that immediate feedback intervention or IFI can enhance performance in specific arenas, producing more desirable outcomes, and improved skill retention.
Much of the research on IFI comes from the realm of second language learning, where immediate feedback is compared to delayed feedback based on language acquisition. In this area, IFI consistently improves language acquisition over delayed feedback. This makes good sense. If practice makes perfect, you want to know that you are practicing the right thing. The same improvements are found when participants are given similar learning tasks using artificial grammatical rules.
Similarly, an AI-based tool providing automated feedback to teachers improved their classroom communication practices, where teachers followed a rubric for receiving, acknowledging, and engaging on student contributions. Balls and strikes, in a way, are akin to an artificial grammar or a teaching rubric: rules and frameworks by which the umpire evaluates and conveys whether a call should be made.
Umpires have already displayed the efficacy of feedback intervention on their game calling; cyber assistance would simply refine this system by bringing the feedback into the game in real time. In theory, this kind of system could allow umpires to calibrate their calls while maintaining the probabilistic boundaries of the zone that umpires are already adept at navigating. The horizontal edges of the zone could be tightened, while the vertical confines of the zone could stay tethered to their judgment-based rulebook definition.
Additionally, automated support could be a boon for baseball umpires without altering their collectively bargained compensation incentives. Since umpires are already evaluated and rewarded based on a grading system, IFI wouldn’t necessitate additional performance metrics; it would merely offer analogous feedback on performance in real time.
Calling balls and strikes is an act of instant, repeated judgment. Providing umpires with constructive feedback on the accuracy of their decision-making through a simple, wearable device could fine-tune their judgments more effectively, especially if the system conveyed detailed and exact descriptions of how their calls diverged from the ideal. The outcome would be enhanced game-calling founded on real-time feedback, as opposed to the delayed feedback currently used throughout the game.
Cyber Element of the Game
The incorporation of “cyber umps” in baseball could improve game-calling through real-time feedback, optimizing umpire performance without replacing the judgement of umpires. Drawing from the proven efficacy of immediate feedback intervention in various fields, this system would not only sharpen the accuracy of calls but also sustain the game’s tension and pace. It’s not quite a traditionalist proposal, nor is it entirely futuristic, but I think it’s worth a shot.