I love baseball. It’s partly because it’s a game with a great narrative structure. The season unfolds in acts. The energy and possibility of the spring is followed by the suspense of the early summer. In August it slows down for just long enough so you can catch your breath before the final sprint for the pennant picks up speed in September. After that, well, there’s no match for the dramatic tension of October baseball.

The other thing I love about baseball is the statistics. I don’t think there’s any other sport that’s quite as exhaustively documented as baseball. Everything is recorded, and fans aren’t surprised to hear announcers comment on facts as mundane—and apparently useless—as a pitcher’s historical performance on cloudy weekday games in July.

That number seems pretty useless. But a funny thing happens when you start naming and tracking statistics: people start caring about them. Sometimes they start caring about them a lot, even if they’re not particularly valuable.

Pitching to the numbers

Take the save stat. In 1960, sportswriter Jerome Holtzman created the stat because he felt that other pitcher stats didn’t effectively measure the contribution of relief pitchers.

He was probably right. He pitched his stat to The Sporting News, who started recording it in its baseball coverage and created an award for relief pitchers called “Fireman of the Year” that recognized relief pitchers who earned saves and wins. Major League Baseball adopted the Save as an official stat nine years later.

The adoption of the Save stat led to the creation of new, specialized role on baseball teams: the closer. Nowadays you’ll hear announcers, coaches, and managers talk about pitchers with a “closer mentality.” It’s big news when a team moves a new pitcher into the closer role, and closers can command much higher salaries than other relief pitchers who are otherwise equally effectively. This became news in the 2017 off-season when the New York Yankees argued that one of their relievers, Dellin Betances, wasn’t closer material as part of their arbitration case against him. Closers typically don’t enter the game unless there is an opportunity to earn a save, most often in the ninth inning of a close game.

Context matters

Here’s the problem. Let’s say a team’s closer is their most effective relief pitcher. Is the ninth inning in a close game the best time to deploy him? Probably not. Think about these two situations:

  1. In the seventh inning, a team is up by a run. The starting pitcher loads the bases with nobody out, with the opponent’s best hitter coming to the plate.

  2. In the ninth inning, a team is up by a run. There is nobody out and there is nobody on base.

Where does it make the most sense to put in your best pitcher? I guess the argument is that in the latter case, the pitcher’s team has a smaller opportunity to catch up if there’s a run allowed. But if the batter in the former case hits a home run, the pitcher’s team has a much bigger deficit to overcome. I’d rather have my best pitcher handle the first situation.

But in baseball that’s not the conventional wisdom. The closer’s supposed to be used in save opportunities, or so the logic goes. That’s how you end up like the Baltimore Orioles in the 2016 Wild Card game. Baltimore manager left their best pitcher—closer Zach Britton—in the bullpen and left a less effective on the mound to cough up a game-winning home run to Blue Jays slugger Edwin Encarnacion. Baltimore went home.

In contrast, Cleveland's team sent out their best reliever, Andrew Miller, in high leverage situations regardless of the inning. Miller didn't earn many saves in those playoffs, but his team came about as close as you can to winning the World Series without actually winning it.

What’s the why?

So what can this teach us about digital strategy?

In our baseball example, the save stat was created to measure pitcher performance. But over time, players and managers started to pursue the stat for its own sake. Now the statistic is over-valued. Many teams today still don't use some of their most valuable players effectively, just because somebody in 1960 made up a statistic.

The same thing can happen with your digital project.

I’ve seen projects where somebody fixates on a specific number and instructs the project team to do everything they can to improve it. Maybe they want to get more users to sign up for a newsletter. The project team probably has any number of dark patterns at their disposal that can coerce users into signing up for the newsletter. The team implements some of them, and the number increases significantly. Everybody’s happy, right?

Well, no. The person who is accountable for that number might be happy. But then they start to notice that those sign-ups aren’t actually translating into sales. The number of unsubscribes is going through the roof, too.

Meanwhile, the support team is dealing with angry people who are upset that they were forced to sign up for some newsletter they didn’t actually want. Those people are also complaining about it on social media, and the brand has lost credibility and consumer trust. Suddenly there is a host of new problems to solve.

Numbers only tell you the what. They don’t tell you the why. Key performance indicators are just that: indicators. They’re not goals; they’re diagnostic tools. If your sign-up number isn’t where you’d like it to be, that should prompt an investigation that ideally will include a close look at the user experience of your product and some research with actual users. That’s the only way you’re going to get to the crux of your problem.

Don’t let stats drive strategy. Just ask an Orioles fan.