College Prospect Ratings (CPR)

By Stephen Shea, Ph.D.

 

NOTE (May 27, 2015)  CPR scores were rescaled for the 2014 Basketball Analytics Book.  The methodology and rankings largely stayed the same.  This is why recent posts on CPR refer to high scores around 10 when this post refers to high scores around 3.

I have been asked several times in the last couple years if I had an objective system to rate or evaluate the pro potential of an NCAA player.  My response was always that I had my doubts that any such system would be informative enough to have value to pro scouts.  I would look at the season average production of pro prospects and never found a strong enough correlation between the per game or per minute box score college production and later success in the NBA.

In summarizing statistics, it is common to overemphasize averages and disregard variance.  For years, I’ve complained about this phenomenon in many data supported news articles.  My major breakthrough in the construction of a college prospect rating system was to realize that I was committing the same mistake as those I was criticizing.

In recent years, full game-log data on college players has become more readily available.  Players can average 15 points in very different ways.  Specifically, college players, and even those that turn out to be great pros, are often very inconsistent.  18-19 year old freshman occasionally through up a “clunker.”  When they do so 5 times in a 35 game season, it can seriously impact the season averages and mask the player’s star potential.  Full game-log data allows us to see the variance, how great and how weak the player performed on certain occasions.   My ratings will use full game-log data so as to identify, in particular, the great performances of college prospects.

College Prospects are Inconsistent

Here, I build on and repeat some ideas from my previous blog on this topic.

Basketball Analytics presents a simple formula (on box score statistics) to track a player’s offensive efficiency (OE).  OE is positive outcomes (field goals and assists) over net possessions terminated.  Here, a possession terminated is a field goal attempt, assist or turnover.  Offensive rebounds are subtracted from the denominator (since they create another opportunity to score).  OE certainly does not chase every detail of a player’s offensive game.  For a more detailed version of offensive efficiency see OEPT, our measure of offensive efficiency with new player tracking statistics.  Instead, OE focuses on the basics of the game, and uses inputs that reflect a player’s decisions, regardless of the type of offensive player he is (perimeter vs. interior threat, for example).  Most importantly, OE can be calculated on historical data.

Chart 1 tracks Marcus Smart’s OE by game in each of the last two seasons.  To be completely unfair to Marcus, I have also included the OE of Tony Parker and Chris Paul in each of their first 34 games played this season.  Notice how in his freshman year, Smart saw is OE completely bottom out multiple times.  In his sophomore season, Smart avoided the particularly poor displays.  This is evidence that Smart has matured as a player, but also a reminder that we shouldn’t necessarily fault younger players, and freshman in particular, for inconsistent play.  The occasional deep dip in production has a significant impact on Smart’s season average production.

Chart 1: Comparing the OE by Game for Smart, Paul and Parker

chart 1

Paul was not always as efficient as he is now.  Chart 2 displays Paul’s OE by game for his freshman year at Wake Forest (2003-04), his rookie season in the NBA (2005-06) and this past season.  Paul only played 31 games his freshman year at Wake Forest.  So, we only used the first 31 games played by Paul in the subsequent seasons.  Chris Paul’s college inconsistency carried over into his rookie NBA season.  Since then, his consistency has improved dramatically.   The blue line, which marks this past season, does not have any of the significant drops in performance that the previous seasons exhibit.

Chart 2:  Chris Paul Improves Consistency

Chart 2

Chris Paul isn’t the only pro who has seen his consistency improve with experience.  In a previous blog, I demonstrated three trends in Durant’s numbers over the course of his career.  His usage has climbed, his efficiency has improved, and he has gotten more consistent.

I chose to focus on efficiency metrics for this section, but I could have easily chosen any number of other statistics.  Also, I could have chosen any number of other now NBA superstars as examples.  The moral is that consistency is not something we should expect from even the very best of prospects.  It is something that players gain with experience; it is something they can be taught.

Given that we expect inconsistency from even the best prospects, looking at a prospect’s season average production can be troublesome.  If a player experiences the drops in production we saw above with Marcus Smart or Chris Paul in college, their season average numbers will be dampened and their star potential hidden.  Instead, my prospect rating will focus on how good or poor these players were on their better days.

Weaknesses of CPR

On-the-court college statistics alone will not provide an optimal rating of a player’s pro potential.  Other components of the evaluation process are crucial.  This is especially true for 19 year olds entering the draft after just one season in college.  Before I introduce the ratings, I think it’s important to address the weaknesses of the system.

Ideally, an objective rating of pro potential will use historical data to identify the type of production that is most indicative of pro success.  Since I am interested in the pro potential of these players, I would like to witness a significant component of the prospect’s pro career before I determine the extent of his success.  This makes data on recent draft picks less valuable.  Also, for the study to have any statistical significance, it needs multiple years worth of prospects to analyze.  I have already mentioned that our ratings depend on full college game-log data.  Putting the above together means I would like to have full college game-log data going back to at least 2000 but preferably earlier.  Unfortunately, I have not located one source where this information is freely available.  (Does anyone know of such a site?)  Instead, tracking down the numbers for earlier years is a slow and tedious work-in-progress.  More information from the past could tweak the numbers presented later.

Even when I have full game-log data, the statistics reported are not a complete representation of a player’s performance.  Wouldn’t it be nice if we had 10 years of spatial tracking data on NCAA players?  So, we use what we have as proxies for true ability.  Blocks are an approximation of interior defense (and also suggest size and athleticism).  Assists suggest playmaking ability.  In general, these stats are imperfect, but a provide a reasonable approximation.

College Prospect Ratings (CPR) will not use combine measurements such as height, reach, or vertical.  CPR does not account for injury risk beyond the impact of past injuries on college performance.  Finally, CPR does not directly measure character or work ethic.  All of the above are important and should be considered in conjunction with CPR when evaluating prospects.

Methodology

As suggested above, CPR downplays inconsistency and focuses on elite potential as captured through elite college performance.

CPR uses the following inputs: projected NBA position, age, class (freshman, sophomore, etc.), 3PA, 3PM, FTA, FTM, ORB, DRB, AST, STL, BLK, and PTS.  For the performance stats (3PA, 3PM, etc.), full game-log data is used.  All of the above inputs are aggregated to produce a single raw total.  Then, the raw total for performance is adjusted according to class and age (through an exponential function).  The class is the dominating input here.  A raw total for a typical freshman that produces a rating of 1.3 would produce a .7 for a sophomore, and a .3 for a junior or senior.

2014 Draft Class Ratings

Here are the ratings for the 2014 class of college prospects.  First note that the top 5 on the board (Parker, Smart, Wiggins, Vonleh, and Embiid) could very well be the first 5 NCAA players drafted on draft night (although not in that order).  I found this surprising.  Players are drafted for what they are projected to accomplish in the pros, and performance in college is only a partial indicator of future NBA success.

Even after adjusting for year, Smart is rated higher than freshman Wiggins, Vonleh and Embiid.  It is likely that all three will be drafted ahead of Smart.  The reason is that scouts believe these players’ potential exceeds what their production suggests.  Basketball is still relatively new to Embiid.  Although his numbers do not suggest he will be an offensive force in the NBA, his rapid growth this season at Kansas suggest he has that potential.

Further down the ratings is a tale of production vs. athleticism.  Jordan Adams and Kyle Anderson rate considerably higher than their UCLA teammate Zach Lavine in terms of production.   This is true even after adjusting for year.  Adams and Anderson are sophomores, and Lavine is a freshman.  However, Adams and Anderson are knocked for their speed and athleticism, while Lavine is an athletic freak.  It is quite possible that Lavine is drafted first among the 3 UCLA products.  Similarly, Gordon is rated below where he will eventually be drafted.

2014 Draft Class CPR

RankPlayerPositionYearCPR
1Jabari Parker312.5
2Marcus Smart121.9
3Andrew Wiggins2.511.7
4Noah Vonleh4.511.5
5Joel Embiid511.4
6Tyler Ennis111.4
7Shabazz Napier141.3
8Kyle Anderson221.2
9Jordan Adams221.2
10Julius Randle411.1
11T.J. Warren321.1
12Gary Harris220.9
13K.J. McDaniels330.9
14James Young210.9
15Aaron Gordon410.8
16Doug McDermott3.540.7
17Isaiah Austin520.7
18Elfrid Payton130.6
19Jahii Carson120.6
20Nik Stauskas220.5
21Rodney Hood320.5
22Kendall Williams140.5
23Alec Brown4.540.4
24Shane Whittington540.4
25Alex Kirk530.4
26Markel Brown240.4
27Russ Smith140.4
28Jordan Bachynski540.4
29Deonte Burton140.4
30Semaj Christon120.3
31Zach Lavine210.3
32Deandre Daniels330.3
33C.J. Wilcox240.3
34Devyn Marble240.3
35Nick Johnson230.3
36Richard Solomon540.3
37Glenn Robinson III320.3
38Adreian Payne440.3
39Jarnell Stokes430.2
40LaQuinton Ross330.2
41Johnny O'Bryant430.2
42Dwight Powell440.2
43Cleanthony Early340.2
44Jabari Brown230.2
45Cameron Bairstow440.2
46Cory Jefferson440.2
47Jerami Grant320.1
48Spencer Dinwiddie230.1
49James McAdoo430.1
50Mitch McGary520.1
51Jordan Clarkson130.1
52Patric Young440.1

Looking back at 2012

In the next few sections, I will reveal ratings for players drafted in past years.  These results will demonstrate that CPR has been a good predictor of pro success, often better than pick position (and thus, certain NBA teams’ rankings).

Here are the ratings for the top 10 drafted in 2012.  All 10 were NCAA players, and after 2 years, we are developing a sense of how good these players might be.

CPR for the 2012 Top 10 Picks

2012 Table

First notice that the best player in this group, Anthony Davis, rates approximately the same as Jabari Parker.  After Davis though, there is a much steeper decline in ratings than we see in the 2014 class.  This suggests that the 2014 class has more depth of talent in the top 10 than the 2012 class.  If the 2012 draft followed the CPR ratings, the order would have been Davis, Beal, Lillard, Kidd-Gilchrist, Drummond, Ross, Robinson, Barnes, Rivers, Waiters.  If teams were able to redraft the 2012 top 10 today, I suspect the picks would more closely resemble the CPR order than the original draft order.  Had Cleveland believed in CPR, it would have been hard for them to draft Waiters over Lillard or Drummond.  In retrospect, it appears CPR could have helped the Cavs.

Other Notables

Let’s now try to provide some context for the ratings of recent CPR classes.  The highest rating among the 2012 and 2014 classes was 2.5.  As we will soon see, the highest from 2013 was 1.7.  At least according to CPR, the last three years have not held a no-doubt superstar such as Carmelo Anthony, Kevin Durant or even Kevin Love.  To put things in perspective, here are some other ratings from previous seasons.

Notable CPRs

notable table

Kevin Durant is completely off the charts.  Love, Carmelo, Cousins and Oden all rated 3.3 or higher, considerably above any prospects of the last 3 classes.  Paul George rates well enough to have been 2nd in CPR in 2012 and 2014 and first in 2013.  George also rated considerably higher than Ekpe Udoh, who Golden State drafted 5 spots ahead of George in 2010 (oops).  CPR accurately predicted a bust in Adam Morrison, and suggested Korver should have been drafted higher than 51st in 2003.  Duncan’s CPR of 2.8 is better than any prospect from the last 3 classes, but still an underestimate of his ability.  In the recent past, no player close to Duncan’s caliber stayed in college for 4 years.  So, the system breaks a bit when trying to assess his performance.  It’s hard to fit Duncan to a statistical norm when he is such an anomaly.

2013 Class

It is still far too soon to judge the pro success of the 2013 class.  However, we can look back and compare the CPR ratings to this year’s class.  Here are all NCAA prospects drafted in the 2013 first round.  The highest rating in the 2013 class was Bennett at 1.7.  Only 7 players had a CPR of at least 1.0.  The draft lacked high-end talent and depth.  I believe raw totals (before the year and age adjustment) can be good indicators of immediate impact.  From 2013, the highest raw totals were Caldwell-Pope, Carter-Williams, Porter Jr., Dieng, Burke, Bullock and Oladipo.  Steven Adams was last in that category.

CPR for the 2013 First Round Picks

2013 table

Conclusions

Oden is the only player I have found that rated above 2.0 and has not had an extremely successful career.  This bodes well for Jabari Parker who rated 2.5.  Of the group that has rated between 1.0 and 2.0, it has been a bit of a mixed bag.  Players like Wade, George, and Harden blossomed.  Others, such as Thabeet (CPR=1.7) did not.  Of those that rate below 1.0, it has been hard to find individuals that have become great NBA players.  Parsons (CPR=0.2) is maybe one of the greater anomalies.  Of course, the value he has provided out of the 2nd round is also a great anomaly.  Someone like Oladipo could eventually join the list of low CPR ratings that turned out to be great pros.

Overall, CPR provides a good baseline approximation of pro potential.  It is best used as one piece of the scouting puzzle.  Assessments of health, athleticism, work ethic, and character constitute most of the rest.

Previous Post
Leave a comment

8 Comments

  1. Mark

     /  June 14, 2014

    How do you weed out the inconsistent performances? Do you only use games where they have an OE above a certain number?

    Reply
    • Thanks for the question. I’m not ready to release all of the details, but I can say that each input/stat is considered independently of the others. So, what might be a low game in assists, could be a high game in points.

      Reply
  2. This is great thanks. I had two questions…

    1) can you expand a bit on the importance of class vs. age? This is actually a question I have been interested in for awhile but never looked at. Did you find that class explained most of the variance accounted for by age?

    2) Are the retrodictions posted here “out of sample”?

    I hope you are able to expand on your methods more at some point, because I am really interested in your game-by-game approach. Do you have anything you are willing to reveal that demonstrates the utility of that over season averages?

    Reply
    • Layne, I don’t have any answers on the predictive power of class vs. age. I simply meant that this model predominantly uses class. I only carry the information on age at the margins (under-agers such as Gordon or over-agers such as Bachynski). However, I too am very interested in the question and hope to test the importance of each once I fill in more of my database on past college performance.

      I would label all numbers at this point as estimates. The ratings on earlier draft prospects, such as Duncan, are out of sample in the sense that I don’t have complete data on their draft classes. So, Duncan is largely being rated based on the more recent history of college to pro transitions.

      Reply
  3. Hi Steve,

    Is there any concern about sample size problems in just looking at the best performances, given that you only have 35-40 observations of these players to start with?

    Reply
    • Kevin, thanks for the question. Sample size can be a problem, but I’m not sure it’s more of a problem with this methodology than using season averages. It depends on whether the specific outcomes of players’ particularly poor performances help predict pro success. Early indications are that they don’t, but I need more years of college game log data to say anything with confidence.

      To be clear, I am using more than just the 1 best performance on the season. Also, since the vague description in the post is somewhat misleading, I can reveal some more detail on the methods. I do use full season averages on FT% and 3P% (to maximize sample size). I use the details of full game logs on the counting stats. Thanks again.

      Reply
  1. Comparing Objective Draft Models | Basketball Analytics
  2. College Prospect Rating (CPR) Version 3 | Basketball Analytics

Leave a Reply

Your email address will not be published.