This article is adapted from the book, FIELDER’S CHOICE: BASEBALL’S BEST SHORTSTOPS (booklocker.com: 2003)
Anyone familiar with baseball analysis over the past 30 years is aware of the special place held by Bill James. An entertaining writer and a first-rate analyst, it is probably fair to say that he has had as great an impact on the study of baseball numbers as anyone else in that time-span. And now that he has ventured into the realm of (seemingly) evaluating defensive contributions, it is necessary to take a careful look at his work.
In chapter fifteen of this book, I refer to Bill James’s long awaited new book, The New Historical Baseball Abstract, published in October, 2001. The book is essentially an update of his earlier classic, The Historical Baseball Abstract, published in 1985. In the new book, among many other things, he ranks the top 100 players in baseball history at each position. He does this by using a system which he calls Win Shares (hereinafter referred to as WS) – which awards to individual players shares of the wins recorded by their teams. However, as I mentioned in chapter fifteen, James does not explain in the New Abstract exactly how the system works. He saves this explanation for his book, Win Shares, published in March, 2002.
I want to note at this point that I am very impressed with this latest attempt of Bill James to measure the contributions of players to their teams. It is an extraordinary achievement and, no doubt, will become one of the most important reference works for anyone (like myself) who is interested in comparing and ranking players. James has probably come closer than anyone (as of this date) to accurately measuring each player’s total contribution to his team. However, the main flaw of the book appears to be in its arbitrary allocation of credit for offensive and defensive contributions.
I believe that the point that I am raising regarding defense is an important one because, as I will show, James’s evaluation of defense in his new system represents essentially an indirect approach - almost a by-product of his main concerns. And, consequently, it is not clear that defense can be appropriately evaluated in this manner.
Bill James, together with Pete Palmer and a few other notables, are considered by many to be the founders of modern baseball analysis. And so, it is certainly reasonable to believe that many followers of analysis may now consider the WS system to be the new standard for the ranking of players. On this assumption, I feel that it is appropriate to indicate my own reservations about how he uses his new system to determine the relative defensive skills of players. I feel that this is important since I made the point earlier (in chapter fifteen) that the essential difference between my ranking of the top all-around shortstops and James’s ranking in the New Abstract is that he apparently (and arbitrarily) does not give sufficient credit to defensive skills when compared to offensive skills at this very critical defensive position. Let’s take a look at why I feel this way.
The Strange Case of Ozzie Smith
Consider the following important example. Bill James says that Ozzie Smith is the best defensive shortstop ever. No surprise there – I have established that in this book. In an essay starting on p.117 of Win Shares (entitled “Why 52?”), he explains that he has decided that 48% of the credit for a team’s wins should be given to offense while 52% should be given to pitching and defense. Unfortunately, he does not really give a satisfactory answer to his own question of Why 52%? He does say
“Why do we credit 52% of a team’s success to pitching and defense, 48% to offense? Basically for two reasons:a) I am convinced that it is as logical to do this as not to do it, and b) It causes problems if you don’t.
… Even as it is, even giving 52% of the value to pitching and defense, our values for pitchers and fielders still seem low.”
It is disappointing that one of the most important questions about the WS system (if it is going to used at all to judge defensive effectiveness) is answered so unconvincingly. What this does seem to illustrate (and James seems to admit this in the last sentence) is that while the WS system may indeed be valuable in determining the total value of a player to his team, it is very suspect when it attempts to separate the offensive and defensive components of that contribution – particularly for players who were outstanding defensive players. And Ozzie Smith is a good example of this. James states
“On defense…the Win Shares system sees Ozzie Smith as being the greatest defensive shortstop in baseball history… . But what is the relative value of Ozzie
Smith’s defense and hitting? Many of you, I suspect, would argue or assume that
70% or 80% or 90% of Ozzie’s value is in his fielding. The Win Shares system believes 57% of his value is in his offense. … But in large part, this reflects the relative scarcity of Win Shares to be credited to defensive stars.”
Yes, and it also reflects the flaw in using the WS system to compare defensive players. Bill James loves to point out the “logic” of what he does. I submit that this example points out the lack of logic in the system. Of course (as Bill says), knowledgeable fans will argue that Ozzie’s value to his team was greater in the defensive area. That is because IT WAS!!! (In chapter fifteen of this book, I show that Ozzie’s value to his team was 59% defense and 41% offense – not 70% or 80% or 90% as James suggests.)
Let us be very clear on one point. When James states that 57% of Smith’s value was in his offense, he is very clearly stating that Ozzie was more valuable to his team with his bat than with his glove. And that conclusion is ridiculous!
What James has done in his WS system is to create an arbitrary separation of offense and defense (for all players) that results in the strange (and illogical) conclusion that Ozzie Smith was more valuable to his team offensively than defensively.
We must understand that the primary focus of James’s new system is on offense (as usual in his work). His secondary focus is on pitching. Defense then gets whatever scraps are left over. This is a problem that is inherent in an indirect approach. You can only evaluate defensive skills by looking directly at what the players have done defensively – not by subtracting out what you have decided offense is worth and then assigning the rest to defense.
When the WS system tells us that a player like Willie Mays (who was a great hitter and a great fielder) contributed more to his team offensively than defensively – then we are not surprised because we would intuitively expect this result. But the fact of the matter is that Ozzie Smith was only an average offensive player at best while at the same time he was the best defensive shortstop of all time. Does anyone really believe that he had greater value to his team as an offensive player rather than as a defensive one? I think not. Can Bill James really believe that Ozzie Smith was more valuable to his team offensively than defensively? I do not think so. But his WS system produces this weird result because he has logically painted himself into a corner since his system favors offense at the expense of defense - particularly for those players who were outstanding defensive performers at crucial defensive positions.
It is important to note that I am not saying that the Win Shares book has nothing to tell us about defense. Essays like the one on p. 113 entitled “Ashburn and Hamner” offer valuable insights into discussions on defensive accomplishments. The problem arises if one advocates using the system to decide if player A was a better defensive player than player B.
“Bewilderingly Complex”
Bill James is somewhat famous (some might say infamous) for his complicated calculations and his Win Shares book is no exception. On p. 260, he acknowledges
“Still, the calculations are bewilderingly complex, and I know that most of you will think, on first reading, that there is no way that you can follow the twists and turns of the mathematical analysis.”
I have often felt that James tends at times “to hide behind” his complicated mathematical approaches. Some of his calculations seem indeed almost designed to overwhelm the average reader. After all, if most of your readers cannot understand what you have done, how can they possibly challenge your work?
Being a mathematician, I am not daunted by the complexity of the calculations. Quite the contrary, I enjoy James’s mathematical “twists and turns.” And, after studying the first 100 pages or so of his presentation in Win Shares (where most of the calculations are), my conclusion is that the author has succeeded in accomplishing the monumental task that he set for himself: He has established rather well the proportion of a team’s wins that should be awarded to each member of the team.
But, my question is, having accomplished this significant piece of research, can these results be used in a valid manner to rank players defensively?
This is the essential logical question that I am raising. And my conclusion is that there does not seem to be a logical thread between what Bill James has accomplished in his WS research and his use of this research in the ranking of players defensively.
I want to make it very clear (as I stated above) that I consider the Win Shares book to be a monumental contribution to the landscape of baseball analysis. James and his co-author have examined the careers of thousands of ball players and have made a huge contribution to the field. But as Bill James himself says in the book (p. 260), the exact extent of the contribution is yet to be determined.
“The purpose of figuring Win Shares was not to wrap up any discussion: rather, the purpose was to open up many more issues, to make additional topics and additional issues more accessible to research.”
One “additional topic” that has been raised by this work is whether this new approach is in fact a valid way to evaluate the defensive skills of players. That is, the system may be a valuable method for telling us the total contribution that a particular player made to his team’s success over a particular season – and that is a wonderful piece of research. But can this information be used to say logically, for example, that player A had a better defensive season than player B? The essential question is: Is the incredible amount of research that went into the production of Win Shares being used appropriately where defense is concerned?
It appears that Bill James is betting that his reputation in this area will itself convince most (if not all) of his many followers that he has come up with the ultimate approach to comparing players. And if you are looking for a player’s total contribution to his team – perhaps he has. But if you are not completely under the “James spell” and can still step back and examine what he has done in an objective and careful manner, then you will recognize that not everything he says is completely on the money or really logical. In using his system to judge a player’s defensive value, he has created an indirect approach to ranking players. That is, you determine how good a defensive season player A had by
1. seeing how many times his team won, then
2. seeing how much credit he should get for helping in those wins, then
3. arbitrarily saying how much should be credited to offense and how much to defense and finally,
4. concluding that his defense was worth X win shares.
This is a very convoluted journey if one is attempting to determine the quality of a player’s defense. On the surface, this approach may seem to make some sense but, in reality, as illustrated by the Ozzie Smith example above, it does not.
In trying to assess a player’s defensive contribution, it makes far more sense to take a direct approach and give credit to a player for the defensive things that we know he accomplished. This is what James has done for years on the offensive side in his runs created method and what Pete Palmer does in Total Baseball in his linear weights approach and what I do in this book in my CDT/CPT system.
Let’s summarize what we have been saying in this chapter. In his Win Shares system, Bill James may have developed what is arguably the best way to date of determining the value of a player’s total contribution to his team. And so, if we want to know if player A had a better season or a better career than player B, then a careful use of the WS system may be a good way to do this. But if the question concerns whether player A’s defensive accomplishments for a season or for his career are better than those of player B – then we are going to have to look elsewhere for our conclusions.
» Michael Hoban, Ph.D. is a professor of mathematics at Monmouth University in New Jersey (and professor emeritus of the City University of NY). He has been a fan of the game for over 50 years and he is a serious baseball researcher. He is a member of SABR (the Society for American Baseball Research) and has been published in the society’s prestigious annual, The Baseball Research Journal. He is also the author of Baseball’s Complete Players, published by McFarland in 2000.
» More submissions
Copyright © 2003 by Michael Hoban. Posted August 4, 2003.
BALLPLAYERS | TEAMS | CHRONOLOGY | TODAY | BOOKS | NEWSLETTER | ERRATA | FAQ
