Why External 360-Degree Feedback Benchmarks Should Be Avoided

External or cross-company performance benchmarks are a popular request on many 360-degree feedback and performance review RFPs these days. We're often asked about the feature from prospective clients both large and small. Dig a little deeper, though, and the appropriateness and usefulness of these nebulous metrics becomes questionable.

As a policy, we do not provide cross-company benchmarks. This is because, in short, cross-company benchmarks are unsound from a research and performance measurement perspective and provide little actionable information. Creating them would also require that we intermingle client data, and that runs counter to our confidentiality-focused belief that one customer's data should never be used to serve another's account.

It's understandable to want to know how a particular employee's performance or behaviors compare to his or her peers outside of the organization. This is a particularly attractive proposition when internal benchmarks are not available. However, external benchmarks are generally nothing more than gimmicks when it comes to employee performance management systems, and worse, they could land you in hot water legally if used to render decisions of employment. Further, internal benchmarks are easy to create with the right tools.

Below are a few more detailed reasons as to why we think external benchmarks aren't worthwhile, along with tips on how to create benchmarks that do provide valuable comparative measures.

Too Many Variables

Not all 360 reviews or performance review processes are alike, even when automated with the same tool. In fact, across companies, they are usually very different. From a research perspective, comparisons of datasets are most reliable when variance is minimal. This is rarely the case when dealing with HR surveys and performance reviews across companies.

• Differences in content. Competencies and Items, though worded similarly might have totally different purposes from one company to the next. Comparing "communication" as a competency between two companies is difficult because the behaviors evaluated might be completely different.
   
• Rating scales vary. Companies on the EchoSpan system can employ custom rating scales. Comparing results on unlike scales is very difficult, even if scales have the same number of values if they are measuring different things.
   
• The role, tenure and skill level of participants vary. Though two of our client companies might have the same exact review content (unlikely), they could be using that content to survey two or more different levels of employees. Comparing a line manager's "Team Building" scores against those of Senior Vice Presidents isn't fair. Even if results are segmented by title group, the problem isn't solved as a Senior Vice President at a bank has a very different skill set than a Senior Vice President at an aircraft manufacturing company.
   
• Environmental factors influence review content and results. Companies facing difficult economic times may use our performance evaluation tools to decide the best approach for downsizing. During times of economic ease, they may be used for decisions of promotion. Lumping datasets together for the purposes of aggregate benchmarks often ignores environmental factors that impact review results.
   
• Benchmarks get stale quickly. Companies that offer external benchmarks as a feature of their performance management tools often tout that their benchmarks span years or even decades. Benchmarking data gathered in the 1990s have little bearing on the performance of the modern worker. Think how much the importance of an item pertaining to "Uses and understands communications technology" will have changed between then and now.
   
• The purpose of reviews vary. Some companies use 360-degree feedback reviews as developmental aids. Others use them as replacements or complements to performance reviews. Scores are difficult to aggregate reliably when the purpose of the review isn't taken into account.

Population Not Representative

Another complicating factor in relying on external benchmarks provided by a performance management tool is the size and relevance of their population. EchoSpan is a larger provider of review tools spanning six continents and more than 3,000 client companies. We're regularly running 3,000 to 5,000 reviews per day. Still, this is a tiny population size for extrapolating overall employee performance statistics and trends. What's more, our client base can be skewed to particular industries or employee levels, making comparisons to other types of organizations hard.

And Then, There's Confidentiality

Creating benchmarks between clients means intermingling data. Even if done in the aggregate and only with opted-in companies, this introduces data security risks. It's our policy that all client information should always be separated and never mixed. With EchoSpan, you can rest assured that any data entered into the system is protected and only used for the purposes of serving your company.

Use Internal Benchmarks Instead

With all of this in mind, does it really matter what other companies' employees are scoring in competencies with the same names as yours?

All of this is not to say that benchmarks themselves aren't valuable. They are. Internal benchmarking of peer employees is a better way to go and can be incredibly useful in determining an employee's relative standing. By their nature, internal benchmarks do away with all of the problems listed above and are also easier to create.

With the EchoSpan system, internal benchmarks are generated and presented automatically as part of every performance review or 360-degree report. They are even automatically isolated to an employee's peer group, so you are assured that comparisons between similar individuals are being made. These benchmarks also adapt themselves over time so that you're always updating your comparative data points. As your database of reviews grows, so does your benchmarking database.

Another alternative is to create benchmarks by surveying management as to expected employee proficiency levels for reviewed behaviors. This process can be conducted using an online tool like EchoSpan, and then superimposed on employee feedback reports.

Questions or comments about this post? Contact us at contact@echospan.com.