Reference Points and Prospect Evaluation—It’s All Relative
Updated: Jul 5, 2020
It’s officially #draftSZN. Time to start grinding that tape! But objectively assessing talent may be trickier for more than just not being able to find the All-22 film. This article examines how evaluations—including scouting—often contain hidden reference points that can skew our judgements. It may feel like we’re conducting impartial evaluations of talent and value, but in reality it’s not so simple...
What’s a Reference Point?
Classic economic theory holds that the way a person feels about their money is directly dependent on how much they have. That is, if Timmy’s total assets sum up to $10,000 and Tammy’s total assets also sum up to $10,000, then they will feel the same way about their current wealth. In other words, $10K is $10K, end of story.
In contrast, behavioral economics posits that a person’s attitude towards their money does not just depend on how much they currently have. It also depends on a reference point. To see why let’s revisit Timmy and Tammy, who both have $10K. But—and here’s where reference points are important—yesterday Timmy had $5K and Tammy had $15K. In this scenario it’s fairly obvious that they would not feel the same way about their current wealth. Timmy would feel great because he has doubled his assets, and Tammy would be miserable because she has lost one third of hers.
Critically, because their unique reference points give them different feelings about their wealth, Timmy and Tammy will also act differently with regards to how they approach financial. This is largely because Timmy views his wealth as a gain, while Tammy views hers as a loss. We will explore the implications of this for prospect evaluation below. But for now let’s focus on a key distinction that will help illuminate the confusion that reference points can cause: absolute value and reference-dependent value.
Absolute value refers to something’s current worth, irrespective of a reference point. In Timmy and Tammy’s case the absolute value of their assets was identical—$10K. Reference-dependent value, on the other hand, produces a positive or negative valence. In Timmy and Tammy’s case, the absolute value of their current worth was the same—$10K—but the reference dependent value was quite different—positive $5K for Timmy and negative $5K for Tammy.
Absolute worth is what we’re typically after when performing evaluations. But people’s assessments—and their resulting decisions—tend to stem from reference-dependent values.
Reference Points in Scouting
For Timmy and Tammy, the reference point was their previous state of wealth. So what is the reference point when scouting?
Whether you’re a professional scout or an amateur hobbyist that enjoys grinding film, it’s nearly impossible to engage in a scouting process without picking up on how the broader football community views a prospect. This background knowledge shapes your expectations, and in so doing, forms a reference point.
If you sense that you are higher on a prospect than the broader scouting community, you will form a positive view of that player. And conversely, if you are lower on a prospect than the broader scouting community, you will form a negative view of that player.
This feels obvious, but that's precisely the point. It's easy to lose sight of the fact that our feelings are tied to reference-dependent—not absolute—value.
Think back to the 2018 draft and how you felt about Josh Allen as a prospect. You likely felt strongly—people tended to either love him or hate him. Allen’s proponents saw him as a mobile gunslinger with elite tools, including a cannon arm, great size that made him tough to bring down, and a propensity for making off-script plays. His detractors saw him as an erratic decision-maker that couldn’t be counted on to hit the broad side of a barn with a pass, let alone an open receiver.
Really, both sides were right about what they saw. Allen possessed all those traits, and the combination of major tools and major flaws probably put his absolute value in the early round two range. Speaking as an Allen detractor myself, it was easy to lose sight of this fact because I felt so negatively about him.
Those negative feelings, of course, were reference dependent, set against what I viewed as a massive amount of overpraise. But it’s not hard to imagine a world where I felt positively about Allen as a prospect. If a large portion of the scouting community hadn’t become hugely vocal Allen stans (which is easily could have happened considering he was a moderately successful QB from the Mountain West Conference that threw for 1,812 yards in the year he declared for the draft), he would have been a late-round prospect and I probably would have loved him.
This simply illustrates that the way we feel about a prospect is more closely tied to the prospect’s reference-dependent value than their absolute value. But ultimately, it's absolute value that should drive our decisions. So it’s important to not overreact to your feelings—remember, they're reference-dependent! Instead, focus on not losing sight of the absolute value.
Finding the Absolute Value
Once you are aware of how reference points work, you can use them to check your intuitions and hone in on your true beliefs about a player's worth. To do this, ask yourself how much a reference point—in this case, the general consensus—would have to change in order to flip your opinion of a player. This is essentially what we did with Josh Allen in the section above, but let's revisit the process.
Say you are evaluating a wide receiver, and you find him to be skilled in some areas but generally unexciting as a prospect. If the general consensus is that he’s a mid-second round pick, ask yourself how far that would have to drop before you do start to get excited. What if the general consensus was closer to a mid-third. Would you be excited then? What about an early fourth? Figuring out the point at which your feelings flip is a good way to identify absolute value, and avoid making decisions that are skewed by reference points you weren’t paying attention to.