Assessing goalkeeping performance

The opening of this transfer window has seen an unusual amount of focus on goalkeepers. Petr Cech has swapped Chelsea for Arsenal, and now Chelsea are out for a replacement to backup Thibaut Courtois. There have been stirrings of Manchester United losing David de Gea and discussion of whether they would be better served shopping for a new number one or relying on Victor Valdes in the event de Gea does leave.

Though not as exorbitant as the 50-million-pound-plus fees mentioned in rumors involving names such as Paul Pogba and Kevin de Bruyne, the fees for goalkeepers have been part of the focus on their transfers. Some rate Petr Cech as a bargain, while others think Arsenal wildly overpaid. This is a clear illustration of how hard consensus is to come by when it comes to evaluating goalkeepers’ skill and justifying a price.

In evaluating goalkeepers, there are at least two intuitive and pretty basic areas in which we already subjectively evaluate goalkeepers. One is shot-stopping ability – how well do they prevent opponents who have goalscoring chances from converting those chances? The other area could be broadly labelled as chance prevention and includes things such as how often and how successfully they cut out crosses and through balls or claim aerial threats in the box. For now I sought to look at shot-stopping ability via the Keeping Goals Added metric.

While goals and save percentages make sense as a basic starting point for goalkeepers, they leave out a lot of information such as how likely the keeper was to stop a shot in the first place. If a keeper fails to stop a one-on-one attempt, we probably wouldn’t consider them at fault as much as with a longer shot. Conveniently, expected goals captures a lot of this information, so as we’ve done in the past, I calculated the number of goals each keeper allowed, as well as the number of goals the expected goals model predicted they would concede (based only on shots on target). I took the difference between the two to arrive at the Keeping Goals Added. In this case, I also adjusted by a constant to account for the tendency of the expected goals model to under-predict total number of goals when summed. The result is Adjusted Keeping Goals Added (aKGA) a number which indicates how many more or fewer goals a goalkeeper conceded than what they would be expected to concede based on the quality of the shots they faced. I filtered my dataset to keepers who had faced at least 50 shots on target in either year (2013/14 and 2014/15) to make the results more meaningful.

While this statistic represents a good way to assess cumulative value, it’s hard to compare across keepers who have faced a different number of shots on target. A keeper who faces more will have more chances to accumulate value in an additive statistic (this also means that having a good keeper is more valuable to teams allowing many shots on target). As in our previous analysis, I adjusted for shot volume by using aKGA per 10 shots on target faced. This gives a better representation of what we might think of as skill in shot stopping. The distribution of aKGA per 10 SoT over the last two years looks like this:

Distribution of aKGA per 10 shots on target, 2013/14 – 2014/15

Here negative numbers represent good performance. The top 15 performances across the two seasons are shown:

Top 15 performances, 2013/14 – 2014/15

Seeing David Ospina score well across two seasons with different teams is rather interesting, given that Arsenal felt the need to buy some competition (or perhaps even a replacement) for him. Vito Mannone’s high rating is echoed by his selection as Sunderland’s official player of the year for 2013-14. However, in the most recent campaign he performed worse, rating at -0.306 akGA per 10SOT over 52 shots on target, and was dropped for Costel Pantilimon, who performed at a respectable -0.504 .

A key issue not fully addressed by our past analysis of kGA is repeatability. If goalkeepers who outperform their expected goals totals are unable to sustain this performance, then the metric isn’t capturing skill so much as luck. I examined this by looking at the 69 keepers who faced at least 50 shots in each of the 2013/14 and 2014/15 seasons.

Persistence of kGA/10SOT

Here the orange line represents what would be a perfect 1:1 relationship between keeping goals added in the two years plotted. The blue line is the actual best fit, with an R-squared value of 0.183. This does not make an overwhelming case for the repeatability of akGA / 10 SOT. Though the coefficient of regression of the first year’s effect on the second year’s value is statistically significant and positive, the relatively low explanatory power seems to indicate that we haven’t quite isolated shot-stopping ability through the noise. So while we can look retroactively and assign value to what a keeper has done based on this metric, it’s hard to assess transfers when the available information on the past seems to give us little to say about the future.