'You Can't Defeat the Robots!': Baseball's AI Strike Zone Is Must-Watch Television

With the bases loaded and two outs in the top of the seventh inning of Sunday’s Twins-Orioles game, Twins cleanup hitter Matt Wallner watched a knee-high 3-2 pitch sail directly over the heart of the plate for strike three. Rather than accept his fate, an emotional, frustrated Wallner tapped his helmet, signaling that he was challenging an obvious strike under Major League Baseball’s new automated ball-strike challenge system. Baseball’s new AI-powered strike zone robots confirmed the call on the field, and the Twins lost the ability to challenge for the rest of the game. This very human, very emotion-driven mistake then set up a series of events resulting in the first ever manager ejection for arguing about a robot’s decision, perhaps a glimpse at the future of baseball and, if you squint, a microcosm of various human-AI beefs in society more broadly. 

We are four days into the new baseball season, and this season’s brand new Automated Ball-Strike (ABS) system is the dominant storyline so far. Here’s how the system works, more or less: Like usual, a human umpire calls each pitch a ball or a strike. Immediately following that call, the pitcher, catcher, or batter can challenge the call by tapping on their head. The location of the pitch is then immediately shown on the stadium’s scoreboard on a graphic that includes each hitter’s strike zone; if the ball is within or clips any part of the strike zone box, it’s a strike. If not, it’s a ball. This all happens in a matter of seconds automatically on the Jumbotron and is driven by AI; its results are inarguable. There is no long human review process in a video booth in New York like there is for other umpire’s challenges. 

And yet, the ABS system feels somehow extremely human, because human beings are making the decisions on what to challenge, under what circumstances, and how to react to any given decision. ABS is also not exactly human vs robot, it is a human player’s judgment vs a human umpire’s judgment as adjudicated by an AI system, which has made it must-watch television. Anyone who has screamed “that was a strike” at their TV now gets the satisfaction of having a player’s apparently superior judgment have actual consequences in the game. And, because the home TV broadcasts have a strikezone superimposed on the proceedings, watching from home means you can, in real time, think “they should challenge that,” or “dumb challenge.” 

ABS is exposing how terrible specific umpires are at their job, in real time, in somewhat humiliating fashion. In the Reds-Red Sox game Saturday, notoriously bad umpire C.B. Bucknor made a big show of ringing up Eugenio Suarez (calling a strikeout) on two consecutive pitches that were clearly outside of the strike zone. Suarez challenged both calls and won both challenges. The crowd absolutely lost its shit at both challenges. I have heard multiple play-by-play announcers note that some of the loudest cheers of any game have been about players using the challenge system to prove the umpires wrong. In the Mariners game this weekend, Randy Arozarena was called out by the human umpire on a 3-2 pitch; Arozarena tapped his helmet and jogged to first base as though he had walked, his judgment never in doubt. ABS showed Arozarena was right. It was great theater.

“When we first talked about ABS, I said, you know what, there’s going to come a day where we have one of these challenges, and it’s going to become like cinema. It’s going to become one of the better parts of the game, talking about people getting ejected, how fun that is,” former player Trevor Plouffe said on the Baseball Today podcast Monday. “And it happened in Cincinnati, they said it was the biggest cheers of the game. Not the homers, but the overturned calls. I thought I was going to like it more, but it’s a little sad. I get sad vibes from this,” he added, referring to the humiliation of human umpires getting calls overturned. 

What the first few days of ABS are showing is that this system is somehow actually highlighting the human element of the game, and adding another layer of strategy to a game that prides itself as being the thinking person’s sport. This is because, crucially, teams can only lose two challenges, but teams have unlimited challenges as long as they get them right. Once they lose two challenges, they are not allowed to challenge any more for the rest of the game, raising all sorts of questions about which players will be good at it (well-respected veterans who have been getting borderline calls out of respect, or rookies who have a year of ABS experience from a trial run in the minors later year?), which positions should challenge (so far, catchers are good at challenging, hitters slightly less so, and pitchers are bad at challenging), and in which game circumstances challenges will be called. 

Umpires “do not like the embarrassment of it all, being up on the big board,” Baseball Today host Chris Rose responded to Plouffe. “I love it. I’m sitting here trying to think about strategy. You can tell these teams have zero strategy. Not only that, they also don’t think about it. You have teams that are leading a game in the ninth and a batter uses the last challenge at the plate, when you should be saving it for your pitcher in the bottom of the ninth. They haven’t thought about this at all.” 

This brings us back to the Orioles-Twins game, and Wallner’s horrible challenge. It was the Twins’ second failed challenge of the game. In the bottom half of the inning, Orioles shortstop Gunnar Henderson took a 3-1 pitch that was clearly a strike near the top of the zone. It was called a ball. The Twins could not challenge, and the Orioles proceeded to score three runs on the back of a series of their own successful challenges. The Twins could do nothing but sit there and suffer, and Wallner has been getting excoriated on social media for being an emotional dumbass and hurting his team. 

Then, in the top of the ninth, ABS’s first truly viral moment occurred. A 3-2 pitch from Orioles closer Ryan Helsley was called a ball. Helsley, falling off the mound, tapped his hat once, then again. ABS called the pitch a strike, which was a critical decision in a critical moment. Twins manager Derek Shelton stormed out of the dugout and argued with home plate umpire Chris Segal, eventually getting ejected from the game. “Derek Shelton’s been thrown out! He’s arguing with the robots! You can’t defeat the robots!,” Orioles announcer Kevin Brown said during the Orioles broadcast. What Shelton was actually arguing about was whether Helsley had decided to challenge quick enough, but, nevertheless, the moment has gone viral as the first-ever robot-related ejection in MLB history. Overall, there were nine challenges in the Orioles-Twins game, a new record in the very early stages of the system.

The early discourse on ABS is that it has added some excitement to the game, and has cut down on infuriating and somewhat random cases of umpires making horrendous decisions in critical situations, a problem that has plagued baseball since time immemorial but has reached crisis levels in recent years as superimposed strike zones and viral social media “umpire scorecards” highlight just how much bad umpiring has been affecting the outcome of games. 

Lots of baseball fans love the “human element” of human umpires, but the truth is that human umpires wildly vary in their ability to accurately call balls and strikes, and watching a call go against your team in a high-stakes moment is excruciating. The system that MLB has deployed feels, at the moment, like it preserves the human element of the game while adding in a new layer of strategy: Are your team’s players disciplined and unemotional enough to avoid wasting your challenges in stupid situations? Are you able to deploy them in ways that bend the game in your favor? So far it feels like this system largely strikes the right balance, and has not actually automated umpires out of a job, though it does often humiliate them in front of tens of thousands of screaming fans. In a matter of days, people have begun cheering on the trusted robots over fallible human umpires. It’s hard to say what, if anything, this means for the other ways AI and robots are being pushed into our daily lives. But in baseball, so far, the thoughtful use of robots seems to have entertainingly solved one of the game’s biggest problems. 

About the author

Jason is a cofounder of 404 Media. He was previously the editor-in-chief of Motherboard. He loves the Freedom of Information Act and surfing.

More from Jason Koebler