Reliable, accurate, and bias-free measures of employees’ job performance are notoriously elusive. And while companies are awash with data about their employees, their ability to translate them into trustworthy markers of performance is at best a work in progress. Research shows that self-ratings and supervisory ratings of job performance overlap by merely 4%. While true meritocracy in organizations is impossible and performance won’t ever be entirely measurable, organizations can become more merit-based, fairer, and overall more successful if they identify KPIs about performance within their organizations and use the right data to measure it.
Meritocracy matters. A company that rewards talent, effort, and achievement can be expected to outperform those where nepotism, systemic biases, toxic politics, and sheer incompetence are prevalent; it’s just a matter of time. The rise of people analytics, innovations in the field of HR technologies, as well as the inevitable integration of AI and machine learning algorithms into talent management practices, are all signals of the same underlying phenomenon: a relentless quest for a more rational, fair, and evidence-based approach to managing workers, and unlocking human potential at work.
Yet reliable, accurate, and bias-free measures of employees’ job performancekey to managing performance more fairly, remain notoriously elusive. Indeed, despite ubiquitous tech tools, not to mention avalanches of fancy data, visualizations, and dashboards, the reliable quantification of workers’ value creation remains as distant from real-world management realities today as it was 40 years ago.
To be sure, large companies are awash with data from internal communication systems, project management software, survey platforms, and even sensors, but their ability to translate them into trustworthy markers of human performance is at best a work in progress. Despite clear advancements on the tech sideand some undeniable achievements in data science, there is still a large gap between workers’ output, performance, or value generation and their career success, status, or seniority. Ask any organization to identify their top employees, including managers and leaders, and to prove their selection with hard evidence or data, and they will look at you perplexed. In most organizations, success is more likely a reflection of winning a popularity contestor harnessing a strong reputation, than actually contributing to the organization’s success.
To overcome this problem, one first needs to identify what constitutes real, meaningful performance, tying to the high-level goals of the organization. This means creating a continuum of performance metrics from individual employees to teams, divisions, and the entire company.
Quantifying the Theoretical
A common reason for this organizational deficit is that job performance, defined as an employee’s contribution to organizational effectiveness (including not just their ability to effectively carry out valuable tasks, but also good citizenship and the absence of detrimental or counterproductive work behaviors), cannot be directly measured, especially by relying on spontaneous or intuitive observation. Job performance is a theoretical construct, just like happiness, integrity, or narcissism. We can at best observe its indicators or manifestations, but to observe these accurately requires the right incentives, a model, expertise, effort, and reliable evaluation tools that need to be refined and internally validated. That is: not our instincts.
As decades of academic research in organizational psychology indicate, the most common approachto quantifying someone’s job performance is to rely on subjective ratings, whether by the employee (self-rating of performance) or their manager (supervisory ratings). The typical correlation between self-ratings and supervisory ratings of job performance is merely 0.22which translates to a trivial 4% overlap between the two. In other words, 96% of the variability in employees’ self-rated job performance is unrelated to how their managers’ view their performance. Unsurprisingly, employees are often surprised when they discover what their bosses think of their performance, a feeling that managers reciprocate when they discover what employees think of their own performance.
Much of this tension stems from organizations not putting in the effort to define quantitative measures of performance across different roles and levels of the organization. This will only ever capture some of what could be considered the true, holistic performance of the group or individual. Without agreed upon, quantitative KPIs, performance evaluation becomes even more political, emotional, and prone to biases.
While employees are generally too generousin their self-evaluations of performance, there is not much evidence for the superior accuracy of supervisory ratings in measuring workers’ true contributed value or output, though aggregating ratings of different managers or sources, including peers, will significantly boost reliability. Needless to say, it is not just possible, but also desirable, to improve how others see us through factors unrelated to our actual job performance. For example, if you are a boss, being friendly with your employees, giving them freedom and flexibility, and ensuring that they have a good time at work, may all translate into positive 360-degree feedback ratings of your reputation, without actually boosting your team’s performance. Same goes for employee engagement, which only correlates with team performance at 0.3 (that’s merely a 9% overlap).
Creating Your Organization’s Hierarchy of Needs
After individual performance metrics are identified, the next challenging problem is building the connective KPI tissue to top level business such as financial KPIs, revenues, profits, turnover, growth, and innovation.
This requires organizations to define a hierarchy of quantitative and qualitative KPIs. The KPI definition process needs to include nearly everyone in the organization, from the CEO to frontline employees, identifying what metrics matter for their role/team/division and how that relates to outcomes defined as important in related groups. These definitions then need to be validated by an analytics team (at least for the quantitative KPIs). Tooling can then be put in place to continuously measure these metrics and provide feedback across the organization. As the predictive power of these KPIs inevitably changes with business conditions, the definition process needs to be repeated.
It is also clear that people don’t enjoy having their workplace data microanalyzed, even if they live in a world in which their data has been extensively traded and commoditized by apps, phones, and wearables. Some surveillance algorithms are in fact Orwellian. Not getting any feedback about what data is being scraped and analyzed undermines trust. And getting feedback – for instance, an automated email that generalizes “you’re in too many meetings” — may trivialize the tech being used, or confirm suspicions that the tech is intrusive. As a result, employees will likely regard the underlying analytics as either creepy or crappy. This is why the KPI definition process has to be transparent and inclusive. Executives shouldn’t think that they can design metrics in a vacuum, or that just because a metric is predictive that it’s ethical to use. Creating ethics committees with independent members, engaging in regular discussions with stakeholders across the organization, and admitting when certain metrics are a best guess rather than the absolute “right” metric are essential for building a fairer, faster, and more data-driven organization.
The ever-increasing complexity of work has also cemented the importance of effective, large-scale collaboration for organizational success. The success or failure of large projects is less about the performance of a single individual and more about how people work together. Organizational success is fundamentally not an individual phenomenon. So assessing and rewarding performance at the individual level, while necessary because individuals are ultimately the ones who get paid and promoted, is scientifically at odds with what is actually important in modern organizations: working with others, cooperating effectively, and making others better.
