Should I use this projection system or that one? Why mess around with the second best system if you can easily determine the best, right?
If you search the web, you can locate previous studies that review the accuracy of baseball’s many projection models.
- Monster 2013 Projection Review, Rotovalue.com
- Evaluating 2013 Projections, Fangraphs Community, Will Larson
- Fantasy Baseball Accuracy, FantasyPros.com
- Fantasy Baseball Projections Review – 2012, Razzball.com
I Don’t Have Time To Read All That. Just Tell Me what They Say.
Understood. Here’s my summary:
- There area lot of different approaches to projecting stats (Marcel, Steamer, Zips, Oliver,PECOTA, etc.)
- Basic three year weighted average with regression to league average
- More than three year weighted averages incorporating more advanced component metrics
- Crowd sourcing
- Aging curves
- Similar player modelling
- No single projection system is consistently better than the others in all the stat categories we care about for fantasy baseball
- The most accurate projection model changes from year-to-year
- But there are some that consistently perform well
- Some systems do well in projecting offensive statistics
- Some are better at pitching
What Is Also True
A lot of research has been done on the effectiveness of combining or “aggregating” different projections or forecasts into one. This research was not done with only fantasy baseball in mind, but we can take advantage of it. Here’s one very interesting article on the topic (it’s from a website named “forecastingprinciples.com” and is a PDF of a study from the Wharton School of Business at Penn, it has to be legit, right?).
The thinking behind aggregating projections is that the wisdom of many intelligent people looking over a lot of information can lead to better results than just one isolated model for projecting future results. When you combine all of this together you’ll naturally be removing the outliers from the individual models, but hopefully you’re also improving the accuracy as a whole.
The Actual Results
It may not be appropriate to boil a 15 page research paper into a couple of sentences. But I’m going to do it anyway! Here’s what the PDF linked above concludes on the evidence on the value of combining forecasts: (more…)