Ratings as Incentives

Let’s assume, just for the moment & until we replace it, that your company has a performance review process. Periodically, maybe once or twice a year, every employee get some sort of rating—1-5, doesn’t meet/meets/exceeds expectations, something like that. (I’ll write in the future about the perverse incentives created by such a process, but let’s assume the process as a given.) What is an effective way to regard these ratings?

Faux Objectivity

I talked with one of original designers of Facebook’s performance review process. They noted that at the time different people doing the same job were getting wildly different compensation. They wanted fairness above all.

This resulted in ever-more-precise rubrics describing the expectations of each level on the engineering ladder (I assume this applied to other parts of Facebook but I didn’t see them). The individual contributor would go through the list and check off the boxes, then the manager would assess for themselves how the report stacked up. These ratings were then “calibrated” with similar engineers to make sure someone didn’t get “exceeds” when they only deserved “meets”.

I call this “faux” objectivity because the rubrics themselves are written by and applied by biased humans. Those being rated are incentivized to twist their behavior to fit the rubric. In practice, your rating in this system is not a rating of your work performance, it is a rating of your manager’s storytelling in these calibration meetings.

If these supposedly-fair ratings are actually as biased as any other evaluation process, should we give up on them entirely? Long-term yes, but short-term whaddayagonnado?

Incentives

Since you’re here reading Geek Incentives you won’t be surprised that my answer is to treat ratings as incentives. Ratings can be used to send signals to the person being rated. Great programmer but toxic human? A high rating tells them that being toxic is acceptable behavior. A low rating sends the opposite message. “But look at everything I accomplished.” “Doesn’t matter because you sucked the life out of your teammates.”

It’s tempting to think of ratings as private, between the manager and the report. Ratings (and the incentives they create) leak. Other managers see the ratings being given and the reasons for them. Individuals talk. Individual incentives are an illusion.

As a manager, ratings offer energy for a difficult conversation. “We’ve talked about your effect on people. It’s not getting better. Your rating is just ‘meets’ as a result. I’m available when you’re ready to work on the issue.”

Red Team

Here’s an exercise: take a recent set of ratings. For each imagine the least-aligned, most-malign action an employee could take to improve that rating. How would you feel if that’s how they acted?

I’m not saying people will maximize self-interest. You are, however, pushing them in that direction. Is that a direction you want to be pushing the organization? Oh, you didn’t intend to push in that direction? That’s part of the responsibility of leadership, becoming aware of when you are playing push-me-pull-you.