Alone in the annals of baseball improbability is The Streak.
THE STREAK
In 1941, Joe DiMaggio recorded a record 56-game hitting streak. It is widely regarded as the one baseball record that will never be broken. The great emergent property himself, Steven Jay Gould, considered The Streak to be the greatest record in sports history[2] due to it's statistical unlikelihood.
If you care about predicting performance, then you care about the frequency of success. Baseball records rarely reflect the frequency of success. Hank Aaron hit 755 home runs and Babe Ruth hit 714. Aaron played 23 seasons as a full-time hitter (~33 HR/season, 16.4 at bats/HR) and Ruth played 22, but only 16 as a full-time hitter (~43 HR/season, 11.8 at bats/HR)[3]. Who is the better home run hitter[4]?
The Streak, however, is the record that is both an absolute number and represents the frequency of success.
How likely is it that a hitting streak of at least 56-games would occur[5]?
We can get a sense of the likelihood from an over-simplified analytical approach. Assuming that the probability of a hit in any at bat is independent of all other at bats[6], the probability of a hit in an at bat is simply the batting average (β)[7] and what we care about is the probability of getting at least one hit in a game.
The probability of not getting a hit in an at bat is (1-β). Because the individual at bat probabilities are independent, the probability of no hits in a game is:
, where α is the number of at bats in the game.
POINT TO PONDER: Intuitively, a hitter with a 0.250 batting average should get one hit per four at bats. Since the average hitter takes four at bats per game, this hitter should get one hit per game, but only on average. Actually, a 0.250 hitter will go hitless in 32% of games played.
Therefore, the probability of at least one hit in a game is:
POINT TO PONDER: The historical average batting average is 0.262, which means that the odds of an "average" pitcher on an "average" day throwing a no-hitter by chance are one in 3929 games. There are 2592 regular season games played each year. There have been 263 no-hitters since 1875 (~2.5 times as many as expected as by chance[8]).
Therefore, the probability of a hitting streak of at least n games is:
, where n is the number of games in the streak and
is the average number of at bats per game.
The probability of The Streak is dependent on batting average and the number of at bats per game[9].
Consecutive Game Hitting Streaks: A consecutive game hitting streak shall not be terminated if all the player's plate appearances (one or more) result in a base on balls, hit batsman, defensive interference or a sacrifice bunt. The streak shall terminate if the player has a sacrifice fly and no hit. The player's individual consecutive game hitting streak shall be determined by the consecutive games in which the player appears and is not determined by his club's games.
-MLB Official Rule Book, Rule 10.24
We could ask how likely a streak like The Streak is in baseball history; but that involves hard math and programming. Like an intelligent design advocate (but this is about baseball not evolution), let's ask how likely is the particular history of baseball that we observed?
How likely was Joe DiMaggio to hit in at least 56 games straight?
In one million Monte Carlo simulations[10] of Joe's career[11], he had a hit streak of at least 56 games 929 times (once every ~1100 careers).
How likely was Joe DiMaggio to hit in exactly 56 games straight?
In the million simulations above, Joe had precisely 56-game hitting streaks 185 times (once every 5405 careers).
How likely was Joe DiMaggio to hit in exactly 56 games straight in 1941[12]?
In one million simulations of 1941, Joe had precisely 56-game hitting streaks 86 times (once every ~11500 1941s).
What are the odds that Joe's streak will ever be matched?
Base on the historical, average MLB batting average (0.262)[13] and a 2000 game career the odds are one in every ~350,000 player careers (one million simulations). There have only been ~17,000 players in MLB history (~5% chance of The Streak in baseball history, very roughly).
Part of The Streak's mystique is that it has stood without serious challenge for nearly 70 years[14]. Few baseball records, especially hitting records, have that kind of longevity. But, someone has to hold the longest hitting streak record. Improbable things happen all the time, mainly because things not only happen all the time, but they have to happen. Based on higher batting averages in the early days of the MLB, it was quite likely that the longest hitting streak would be set prior to the 1950s and would continue to stand to this day.
What were the chances that Joe DiMaggio would have the longest hitting streak? The previous record hitting streak was Wee Willie Keller's 45-game streak (1896-1897).
What were Joe DiMaggio chances of breaking Keller's record?
In ten thousand simulations of Joe DiMaggio's career, he broke Keller's record 95 times (once every ~100 careers). That hardly calls for divine intervention, considering that Keller's chances of setting the original record were one in every ~30 careers. Ty Cobb, with his record 0.366 career average (one in every ~6.5 careers) and Hugh Duffy with his season record 0.440 average had the best chances of owning the hitting streak record (one in ~12 record seasons).
While The Streak is improbable, A Streak was guaranteed. See not about intelligent design or evolution at all.
NOTES
1: The overall probability of the four home run game in baseball history is ~2.4x10-6 per game.
2: American.
3: Ruth is still probably the greatest left-handed hitter in Boston Red Sox history (23-12 with a 1.75 ERA in 1916). I subtracted the 26 home runs he hit from 1914-1918 and 1935.
4: With no one on in the 9th inning and down by one, you want Ruth at the plate (11.8 AB/HR). Actually, you want Mark McGwire (10.6 AB/HR), but that is opening a whole can of worms.
5: Obviously, I'm not the first person to take a look at this problem. In fact, this is a common problem in statistics classes, but that has not stopped people with serious match chops from taking an interest in the question. The pros put the probability of a streak of at least 56-games having occurred sometime in baseball history at somewhere between 2.5 and 42%.
6: I'm aware of no data sets demonstrating otherwise, but would be
happy to be wrong on this one. This is the null assumption from random
sampling variance.
7: Batting average is not an average. It is a probability of a hit per at bat, which is philosophically determined by taking a limited number of samples from a "true" frequency distribution. The sampling variance (i.e., the accuracy of the batting average) is inversely proportional to the number of at bats observed. So, playoff batting averages theoretically should not be very reliable statistics, especially compared to season batting averages.
8: Based on 390,396 games since 1876.
9: This approach overestimates the probability of a The Streak because it assumes the mean number of at bats each game does not vary, but they do. The probability of no hits in
a game decreases exponentially with the number of at bats. One less at bat in a game has a larger
impact than one more hit a game.
10: Hacky code available upon request.
11: I used an average "Joe", based on his career statistics, adjusting the batting average by leaving out 1949 when he only played in 76 games and 1951, which was the last year of his career, because players usually have to experience being in decline before retiring, meaning that it may not be representative of their career (and wasn't). I used a 0.328 batting average and 1736 games.
12: In 1941, Joe hit 0.357 in 139 games.
13: Current average is 0.264.
14: Only one of the six hitting streaks 40 games or longer occurred after 1941 (44 games by Pete Rose in 1978).
