Skill ratings drop 22% once the leaderboard shows a rank

The number arrived in a dashboard I didn't expect it in: a product team at a payments company had added a public rank to a merchant-facing performance score, and within a quarter the average quality of submissions from mid-tier merchants fell by roughly a fifth. Nobody changed the underlying scoring formula. Nobody changed the incentives. All that changed was that people could see where they stood relative to everyone else. The question worth sitting with is why visibility of rank so reliably degrades the thing being ranked — and what that means for the leaderboards quietly multiplying across banking, credit, and the card networks.

Rank changes the task from doing to positioning

When a score is private, it functions as feedback. When it's public and ordered, it functions as a verdict. Those are different psychological objects, and people respond to them differently.

The mechanism is well documented. In achievement-goal research, the distinction between mastery goals (get better at the thing) and performance-avoidance goals (don't look bad relative to others) predicts very different behaviour under pressure. A visible rank is a standing invitation to switch from the first to the second. Once you're ranked 4,112th out of 9,000, the productive question — "what would make my next submission better?" — gets crowded out by a less productive one: "how do I move up without doing the harder work?"

That substitution is where the 22% goes. The effort doesn't vanish; it gets redirected toward the metric's surface rather than its substance. People optimise the number that's visible and let the thing the number was supposed to represent drift.

The middle of the board is where the damage happens

Here's the detail that matters most, and it's the one most leaderboard designs get wrong. The drop wasn't uniform. Top performers held steady or improved slightly — they had something to defend. Bottom performers were already disengaged. The collapse concentrated in the middle, among people who could see a climb was theoretically possible but not close.

Kahneman and Tversky's work on reference points explains a lot of this. People don't evaluate their standing in absolute terms; they evaluate it against a salient anchor. A rank makes the anchor brutally concrete. Being 400 places from the top decile feels like a gap you can close. Being 3,000 places away feels like a statement about who you are. The same objective performance produces completely different motivation depending on which comparison the interface puts in front of you.

There's a second layer. Loss aversion means a visible rank is something you can lose, not just gain. Once you have a position, defending it becomes emotionally cheaper than improving it. That's fine for the top of the board. For everyone else, it produces cautious, conservative behaviour — the opposite of the exploratory risk-taking that actually generates improvement.

Variable rewards are a design choice, not a law of nature

The standard defence of leaderboards is that they motivate through competition, and competition is good. But the motivational engine most consumer leaderboards actually run on is closer to variable-ratio reinforcement — the pattern where a reward arrives unpredictably, which is the most persistent schedule of reinforcement known. It works. It also produces behaviour aimed at the next reward rather than at the underlying skill.

The distinction matters for financial products because the stakes are higher than a merchant score. Consider the gamified savings and credit-building apps that have spread across emerging markets and, increasingly, mature ones. Streaks, badges, tiered status, public rankings of savers. Some of these genuinely help people build habits. But when the reward is tied to a visible rank rather than to the balance or the repayment record, you get the same substitution: users who are excellent at the game and no better at the financial behaviour it was meant to encourage.

A concrete reference point: research on workplace ranking and feedback consistently finds that ranking information improves performance for those near the top and degrades it for the majority, while also reducing intrinsic motivation for the task itself over time. The effect is not subtle, and it replicates across cultures more than most people expect — though the size of the effect varies with how much the surrounding culture emphasises social comparison.

What this looks like when money is the score

Payment networks and banks have a particular exposure here that consumer apps don't. Their leaderboards tend to be about trustworthiness: credit scores, fraud-risk tiers, merchant quality ratings, chargeback ratios. Ranking these publicly, or even semi-publicly within a cohort, changes how participants behave in ways that can be actively harmful.

A merchant who can see they're ranked poorly on dispute rate has an obvious move: suppress disputes rather than fix the underlying cause. Make refunds frictionless enough that customers don't escalate. That improves the rank and degrades the actual relationship. A payments provider that ranks its small-business customers by transaction volume creates a visible incentive to inflate volume with low-margin or circular activity. None of this is fraud. It's just people responding rationally to the thing you made visible.

The behavioural economics literature calls this Campbell's law, or Goodhart's law in its bluntest form: when a measure becomes a target, it ceases to be a good measure. Leaderboards are target-ification machines. They take a diagnostic and turn it into a competition, and competitions select for competitors, not for the underlying competence.

Designing for the thing you actually want

None of this argues for abandoning feedback. It argues for being precise about what you make visible and to whom.

A few directions that hold up better than public rank:

Private, comparative-to-self feedback. Showing a merchant or a saver how they've moved relative to their own history preserves the motivational benefit of progress without the corrosive effect of social comparison. The reference point is their last self, not the person above them.

Cohort bands instead of ordinal ranks. "You're in the top third of similar-sized merchants" carries most of the information a rank does, with a fraction of the positioning behaviour. It tells people where they stand without handing them a ladder to game.

Reward the behaviour, not the standing. If the goal is better repayment, reward consistent repayment directly. If it's better submission quality, reward the improvements in quality. Tying rewards to rank means you're paying for rank, and you'll get rank.

Make the path legible. The most damaging feature of a distant rank is that it makes improvement feel abstract. Showing the next concrete step — the specific behaviour that would move the score — converts a verdict back into feedback.

The forward-looking question for anyone building in payments and banking isn't whether to measure. It's whether the measurement you display is doing the work you think it's doing. The 22% didn't come from people getting worse at the task. It came from people getting better at the board. If your product has a leaderboard, it's worth checking which of those two things your users are optimising — and whether the difference shows up in the metric you actually care about, or only in the one you put on the screen.