Repeatability is the true holy grail of soccer statistics. Any other innovation should be considered a false prophet without it.
No matter what methodological growth happens in player evaluation, advancements will not be actionable until they provide insight into the future. For soccer clubs, this is the singular difference between useful insight and utter trivia.
The latest target of the soccer analytics community is the repeatability of goal scoring. Suspicion has arose that a player’s shot conversion rate, after controlling for the quality of the scoring opportunity, does not correlate year-on-year. This suggests that players who are exceptional goal-scorers one year are no more likely to be exceptional the following year.
If true, this shocking discovery would shake our understanding of strikers to the core. Are our heroic goal-scorers merely products of exceptional runs of luck? The implications of this research are transcending; cascading from the youngest fans to the grizzliest veterans embedded deep in club offices.
Given the pomp and circumstance in which I’ve framed this issue, it’s no surprise that I consider the research being done in this area to be the most exciting development the community has had in recent times. In this post I will summarize the crux of the arguments being made and then provide an alternative direction to previous research.
The mechanisms surrounding conversion rate and shot quality revolve around a central concept called “Expected Goals”. For those not up on the latest developments, Expected Goals is a calculation that is attached to each attempted shot and measures it’s chance of resulting in a goal. The method of calculation differs between models (and the iterations of those models), but they broadly take into account the same types of things. Such as, but not limited to, a shot’s distance from goal, what phase of play it came from, and even what body part the shot came off of. Theoretically, anything that effects a shot’s probability of resulting in a goal belongs in this model.
For further reading on ExG, check out this post by Colin Trainor. And, specifically on the topic of repeatability, this post by 11tegen11.
While Expected Goals is proving to be a better predictive indicator than constructs like TSR, not all is clear in the world of Expected Goals. The main point of contention can be illustrated in two graphs.
While there is a clear correlation between a players’ year-on-year Expected Goals rate …
… the correlation disappears when we take the obvious next step and examine the year-on-year repeatability of Goals above Expected Goals.
*2011 and 2012 BPL seasons; players of 10+ shots.
This finding puts the idea of “finishing skill” under scrutiny that was never previously imagined. Is the perceived ability of consistently converting goal scoring opportunities actually just random variation? And, by extension, are clubs chasing players for their previous runs of luck instead of actual repeatable skill?
While I do subscribe to the argument that we have historically overvalued finishing ability, this is a very hard pill to swallow given how deep implications reach into club training grounds. In my limited and anecdotal experience, a shooting drill is not an utter waste of time.
The reason our findings are suggesting the contrary is because our shooting model construct is incomplete. The reason we began valuing Expected Goals over Goals was because raw goal-scoring is heavily biased by multiple factors. It should be of no surprise that we get lost in the noise when we evaluate our Expected Goals model against the same monster we were trying avoid.
So, we need an additional step to our model that measures an opportunity “post-shot” so that we can measure the change in probability against the chance quality “pre-shot”. This exercise in model building is non-trivial for reasons I can explain through two visuals.
These are the all of the shots on goal from the 2012 BPL season where the initial goal-scoring probability was less than 5%. Red circles represent goals and grey circles represent saves.
And here are the shots on target with an initial goal-scoring probability greater than 30%:
As you would expect, goal-scoring probability increases drastically as you put the ball deeper into a corner. But, this effect diminishes (but doesn’t disappear) as the initial goal-scoring probability increases. Once we control for this interaction, and compare a player’s “Expected Goals Added” between the two phases of our model, finishing skill rears it’s head.
With a comparatively weak r-squared value of 24% we are correct to be discounting finishing skill in favor of a players’ more repeatable ability of constantly finding themselves in goal-scoring opportunities, but finishing skill does seem to persist. Over enough samples, expect that players with exceptional finishing skill to perform better than their average – but sometimes lucky – counterparts.
*Note: I expect there may be some additional gains in the repeatability of finishing skill with a few additional tweaks. In this analysis, any off-target shot is (rightfully) considered to have a expected goals value of zero. Yet, as we’ve explored via free kick efficiency in the previous OptaPro blog post, there may be some predictive value in near misses.