Comparative Value of an NBA Roster

As promised, here is a second look at trying to put some numbers (and visuals) behind the concept of depth. The first article on the topic can be found here, and is not required reading to understand this one. Rosters are based on post-deadline information.

What is Depth?

This is a situation where what your definition of depth is will impact how you measure depth and how you report on its quality across teams. Here are a few definitions that I find more or less plausible.

1. The absolute quality of players 5-15 on the rotation
2. The absolute quality of players 3-10 on the rotation
3. The average quality of all players on the roster
4. The relative quality of starters to bench
5. etc.

Does a team of five LeBrons and ten Mike Muscala’s have more or less depth than a team of fifteen Jimmy Butlers? How does the definition of a quality bench change by the quality of the starters? The Clippers, a colloquially deep team, need their bench to score to keep pace, whereas the Warriors just need their bench to kill time until one of their Hall of Famers heats up. I don’t actually have a good definition off the top of my head that I 100% agree with — certainly not in the case of an easy single sentence definition. I’m going to try and be clear about what I’m measuring and how for each of the below sections, but your personal idea of depth might vary from mine.

Ultimately, what we care about, is if the quality (or lack thereof) of a team’s bench contributes to winning. Regardless of the metric of “quality”, we will be comparing against the winning percentage of teams in the 2019 season.

Data and Methods

There are two measures of quality that will be manipulated to achieve our goals. The first is Player Impact Plus-Minus (PIPM) and the second is Wins Added (WA). Briefly, PIPM is a single value advanced statistic that is both descriptive and predictive. WA is the conversion of PIPM into a measurement that shows how many wins over a replacement player that specific player creates. Of note, PIPM is not a cumulative number. Two players with the same PIPM with 40 games played and 82 games played will have different WA results. Therefore, moving forwards, PIPM will generally be presented as an average and WA as a total. Both were created by Jacob Goldstein, and specifics can be found at the above link. Table 1 presents the top PIPM from 2016-17 and prior to get a sense of the scale of the number.

Star Players, PIPM, and Winning

In order to get a foothold on PIPM, WA, and winning percentage, let’s look quickly at the relationship between our measurements and winning. Figure 1 is the linear relationship between the top PIPM player for each team and winning percentage, and Figure 2 is the linear relationship between the top WA player for each team and winning percentage.

Unsurprisingly, the two figures are very similar since WA is in part calculated using PIPM. The primary differences occur where the “best” player via PIPM does not play as many games as the second best player via PIPM and has a lower WA as a result. The Pelicans are an excellent example, where Anthony Davis (PIPM: 5.72) has a lower WA than Jrue Holiday (PIPM: 3.54) because Jrue has played over 500 more minutes than Anthony Davis. The big picture though - the better your best player (and the more games they play), the more games you win.

PIPM-5 & PIPM-10

The 15 man rosters and five man on the floor limits make for some easy grouping into thirds. Instead of strictly going by actual starting lineups, each player is ranked within their team by PIPM and WA, with 1st being the best PIPM or WA, and 15th as the worst. Figure 3 averages the PIPM of the top five PIPM’ers (PIPM-5) for each team and compares to winning percentage.

Again, not that shocking, but informative nonetheless, especially when we combine it with Figure 4, the same methodology but for PIPM ranks 5-10 (PIPM-10).

Well it doesn’t take a ton of effort to see that there is a similar relationship to Figure 3, except a few outlier teams. While it can be tempting to throw away data points that don’t perfectly fit the model, that’s a pretty terrible statistical practice to perform just off the cuff. Given the relatively decent R-squared values for these regressions, what I’m particularly interested in is the slope of the lines. The slope is an indication of how the y-axis (winning percentage) changes as a result of a change in the x-axis (PIPM). You might remember “rise over run” from math back in the day.

Math warning

The slope of the PIPM-5 regression line is 0.11 (Figure 3) and the slope of the PIPM-10 regression line is 0.16 (Figure 4). The interpretation of the PIPM-5 slope is “for every +1.0 increase in PIPM-5 the winning percentage additively increases by +0.11” and the reverse holds true for 1.0 decrease in PIPM-5. The interesting part here is that based on the raw numbers it appears that PIPM-10 is about 1.5x as influential on winning percentage as PIPM-5 (0.16 vs. 0.11). Should Elton go out and immediately trade Jimmy and Tobias for two really solid 8th men? No. He should not. What the above thought assumes is that a 1.0 PIPM-5 change and 1.0 PIPM-10 change is conceptually the same as well as numerically. Think of a 20mph change in speed between a runner and a car - it means the same thing based on the math, but one isn’t really sensical. Obviously, the assumption here is that players 1-5 are a different population that players 6-10, and I think that’s a generally safe thought.

The reason we know that is by comparing the slopes, +0.12 winning percentage/+1.0 PIPM-5 and +0.34 winning percentage/+1.0 PIPM-10, to the average individual PIPM values within the PIPM-5 (Mean: 1.45, SD: 1.63) and PIPM-10 (Mean: -0.53, SD: 0.673) bins. We’re going to mean standardize the values, or express them in terms of an average player within those bins. Figure 6 shows what the results of that look likes visually.

After adjustment, the slope for PIPM-5 is 0.19, or “for a 1.0 unit increase in average scaled PIPM the winning percentage additively increases by 0.19”, and for PIPM-10 (0.11) its “for a 1.0 unit increase in average scaled PIPM the winning percentage additively increases by 0.11“. What’s important to note that a 1.0 change for PIPM-5 on this scale is a 1.63 change in raw PIPM-5, whereas it is 0.673 for PIPM-10 (due to the adjustment). Taking the two models separately, PIPM-5 is roughly 50% more impactful on winning percentage than PIPM-10 based on a one unit change in their respective scaled PIPM values.

However, there is one more big step to take. Instead of dealing with PIPM-5 and PIPM-10 on their own, we will combine them into one regression in order to view their combined effect, as well as their relationship to each other. When we combine the two (and do a bit of statistical testing to see that the combined model is better than the individual ones), the slope for PIPM-5 is 0.17 and the slope for PIPM-10 is 0.06, again in terms of scaled units. The interpretation here is “for a 1.0 increase in PIPM-5 while holding PIPM-10 constant, winning percentage increases additively by 0.17” or “for a 1.0 increase in PIPM-10 while holding PIPM-5 constant, winning percentage increases additively by 0.06”.

It can be difficult to understand a multiple regression like this, so we’ll need some 3D graphics - presented below in Figure 6 as an interactive format. You can see that the pitch of the plane in the PIPM-5 direction is steeper than the PIPM-10 direction — the exact visualization of the 0.17 to 0.6 relationship.

If that doesn’t load properly for you, or you’d prefer something static, Figures 7a and 7b present the same data with two rotation angles.

So What?

With the understanding that smashing numbers together like this as a way to determine team quality has its limitations, I do believe that this concept is worth considering when evaluating roster moves for immediate impact. While you can’t fully nuke your bench to trade for increased starter quality, it likely matters less in terms of overall winning than the quality of your top five. In fact, if you use the roster data pre-deadline and post-deadline, this modeling indicates a predicted increase in win percentage of 8%, about 6.5 wins over the course of a season.

As usual, this exercise was not conducted as a determination of any absolute truth, but more of a way of thinking about absolute and relative value. I’m pretty confident that Elton et al. have a much more complex (and better) way to model swapping players in and out, as well as that whole “understands basketball” stuff. Lastly, I included the PIPM-15 as well in a a few models and such, but it ends up not being statistically significant or meaningful.