I’m a little embarrassed about this. One of the reasons I started this site is because it aggravates me when people over-simplify things or fail to do proper analysis before making claims.
And I just did this myself.
What I Did Wrong
To fully understand where I went wrong, you should probably read Part 2 in my series documenting my DFS journey, “The Book and Addressing the Myths“.
I read The Book. And it’s clear. It’s right there in black and white for all to see. Don’t use BvP to make decisions.
So what’s the problem?
It might help if we first take a step back. When we’re analyzing a particular baseball statistic (pick one), we often find that they’re not predictive from one year to the next. Take batting average for instance. We know that just because a batter hit .300 last year, it’s not a safe bet to count on that repeating itself.
What do we do in this case? We disaggregate the data. We break it down into other component statistics that we can much more reliably project. Instead of looking only at batting average, we’ll start to look at strikeout rate, walk rate, batted ball profiles, historical BABIP, and more.
I Should Have Done This with Batter-Vs-Pitcher Stats
I should have thought to disagreggate things. After all, just like I get along well with certain people and others make me want to drink bleach, some hitters are going to “have an edge” on certain pitchers and they’re going to be overmatched by others.
Not all pitchers are created equal. Not all hitters are created equal.
Whether it’s a batter’s swing path, a pitcher’s arm slot, a batter’s ability to go the other way, a pitcher’s level of deception, a batter’s inability to hit the down-and-in pitch, or a pitcher’s proclivity to throw a changeup in two-strike counts, there are many variables at play in a BvP matchup.
And it is VERY likely that these variables give an edge to the pitcher or the hitter. It’s VERY possible that one of these variables could put the odds overwhelmingly in one party’s favor. It’s also VERY possible that for every factor giving an edge to the hitter that there is an equal and offsetting factor giving an advantage to the pitcher.
It’s not that BvP is useless. It’s not that it has no predictive value. Certain hitters have an edge over specific pitchers. And vice versa. Of course they do. It just makes sense.
The problem is that we don’t know how to separate out the BvP matchups that are predictive those that don’t. Due to small sample sizes, when you go to break it down, BvP matchups probably fall into these nine buckets:
|1.||Hitters with an edge over a pitcher, exceeding the expectations due to good luck|
|2.||Hitters with an edge over a pitcher, performing right at expectations due to neutral luck|
|3.||Hitters with an edge over a pitcher, underperforming the expectations due to bad luck|
|4.||Hitter with a neutral matchup against a pitcher, exceeding the expectations due to good luck|
|5.||Hitters with a neutral matchup against a pitcher, performing right at expectations due to neutral luck|
|6.||Hitters with a neutral matchup against a pitcher, underperforming the expectations due to bad luck|
|7.||Hitters overmatched by a pitcher, exceeding the expectations due to good luck|
|8.||Hitters overmatched by a pitcher, performing right at expectations due to neutral luck|
|9.||Hitters overmatched by a pitcher, underperforming the expectations due to bad luck|
And just like when you combine a bunch of bright colors of Play Doh into one and end up with brown; when you combine all of these different buckets into one, you end up thinking that batter versus pitcher matchups don’t matter.
How This Came To My Attention
It was an interview with Todd Zola, by Patrick Davitt on the Baseball HQ podcst, and this article at Fantasy Alarm.com that helped me realize the err of my ways.
Todd tells us that we should not simply be settling for the blanket answer that BvP stats are not predictive. Think about it. There’s a reason we all want to believe that BvPs matter. It’s because it makes intuitive sense. If Miguel Cabrera were to face Clayton Kershaw 1,000 times we would expect different results than if he faced John Danks.
If you read and or listen to Zola closely, his beef with the BvP argument is that many who cite The Book clearly didn’t understand what was being said. If I’m being honest, I fall into that group. I like to think I’m a smart guy. But this illustrates that you need more just intelligence and smarts to truly understand research, studies, and technical writings. They can be very easy to misinterpret, especially when they start talking in statistical terms (like predictiveness, variance, correlation, etc.).
With that in mind, Zola clarifies some of the misunderstandings I had. The study in The Book never said “all BvP data is worthless.” The study only said that for the years studied that there was no predictive value in using BvP. Here’s a quote from Zola in the interview:
If it’s non-predictive what it basically means is the results are 50-50, it’s a coin-flip… If the results are a coin flip and if BvP really really truly exists in some instances, if you were to figure out the expectations, you’ve got some that are a coin flip and some that are positive. The results should somewhat be as a group a little bit positive, but the results are showing back to being just a coin flip. Which means there have to be some people on the other side pulling that weighted average down, which means there has to be some guys that had a previous success against a pitcher that are not not having success and this “not having success” is real. It’s not a coin flip.
~ Todd Zola
If you apply this quote to my “bucket” example above, there are more buckets missing. At the very least there should be a bucket for, “Hitters who used to have an edge over a pitcher, but the pitcher has now corrected the problem or fought back in some other fashion”. And it is this bucket that is dragging down the predictiveness of BvP information.
I think this “fighting back” makes sense. Imagine this scenario… Hitter X has strong success against Pitcher Y. We’ll say 17-for-38 with 6 home runs. We can even say these 38 appearances happened in a somewhat compressed time frame. Maybe they’re in the same division so they faced each other numerous times over a four year span.
When we break Hitter X down, we could easily see that he has success against “hard stuff”. Fastballs, sliders, cutters, etc. But the soft stuff gives him problems. And maybe Pitcher Y has not always had the change up or curve to attack Hitter X’s weakness.
But we do see over time that pitchers can add new pitches. And we know they do this on purpose. They often do it to attack a weakness they have. Whether it’s to bust a platoon split against them or to add an off speed pitch to give relative strength to their hard pitches.
This is an easy explanation as to how pitchers might “battle back”.
Can I prove this is the hole in BvP? And once we plug it, we will be able to identify those matchups we can use and those we can’t? I cannot.
That’s Great. How Does This Help Me?
I get it. First, I told you BvP is useless. Now I’ve said, it might not be… but you still can’t really use it (because we don’t have a way to extract the pink Play Doh from the brown).
But a few thoughts do come to mind about how we can handle BvP going forward.
First, we could continue to ignore BvP stats. Not exactly the most proactive approach. But at the very least, ignoring them probably offers us a slight advantage over the crowd that does believe in BvP and is investing in hitters that are 6-for-10 against a pitcher (probably much too small of a sample to help any one make a judgment on a matchup).
Second, although we have no mathematical support behind doing so, we could try to identify those players that own someone. I’m not so sure this is a great approach though. Look at that table of nine buckets above. Our only method of identifying hitters that own a pitcher is to find those with extreme performances (e.g. you Paul Goldschmidts against Tim Lincecums). While they may own the interaction, the problem is that they likely have experienced extremely good luck for them to pop up on our radar. And should we really be investing in someone that has experienced extreme good luck?
Third, and perhaps most importantly, we need to continue to monitor BvP studies to see if any news comes out on how to pull that pink Play Doh from the brown. This is especially true with the advent of Statcast and some of the new metrics becoming available about batted ball data. Here’s an example of some more recent research that is attempting to cluster pitchers into similar groups in order to build larger sample sets from which to measure BvP (image below takes you to an interesting YouTube presentation by Vince Gennaro at the 2013 SABR Conference.
I still feel like I have more to learn about the strategy of the DFS game… but I’ve got the itch to start playing around with some spreadsheets.
I think that will soon come. But first, I hope to put out a survey to the SFBB readers in order to get input on what resources you use while preparing your daily lineups. Things like what is your preferred site to get Vegas run totals from, what hitter split information do you use, where do you get park factors from, what’s your favorite weather resource, and who has the most reliable and timely lineup information.
I will likely e-mail all of the SFBB Insiders and put a link to the survey on the site. It will be a great help if you can share your favorite resources. Power in numbers!
Stay tuned. Stay smart.