QuickFlipGame now has a working global leaderboard for all three of its games. Looking at it from the outside, you'd think the top name is the best player. But after watching the numbers, I no longer think that. Here's why.
The Three Games Score Three Different Things
Each of our games tracks a single number, but the numbers mean wildly different things.
| Game | What's Tracked | What the Number Tells You |
|---|---|---|
| 2048 | Cumulative tile-merge score | How long you can keep a board alive + how aggressive your merges are |
| Rock Paper Scissors | Win rate (% of wins over non-draws) | How well you exploit the computer's (or Markov model's) patterns |
| Guess the Number | Guesses used (lower is better) | How well you binary-search the answer space |
A 100,000-point 2048 score and a 100% RPS win rate don't even live on the same axis. Comparing them is like comparing a marathon time to a chess Elo — you can rank each within its own game, but not across games.
The Sample Size Problem
The more pernicious issue: sample size. We rate-limit to 5 submissions per IP per game per day. That means a top-of-the-leaderboard player might be there because they played once and got lucky.
Concrete example (RPS)
Player A submits once with a 100% win rate (they won their first game and the AI never caught up). Player B submits 5 times averaging 65%. Player A ranks higher. Player B is genuinely the better player — they've demonstrated skill across 50+ rounds; Player A has demonstrated it across 4.
There's no way to fix this from the leaderboard surface alone. The number next to a name says "best result", not "best player".
What I'd Track Instead
If I were rebuilding the leaderboard, I'd track three numbers per player per game:
- Total games played — the sample size. Disqualifies lucky single-round runs.
- Best score / win rate — the existing headline metric, kept for compatibility.
- Consistency score — a moving average that rewards players who perform well across many sessions, not just one peak.
For RPS in particular, the existing formula (win rate) is especially vulnerable to sample-size effects. A better default would be "win rate across at least 50 games, with low variance". That single constraint would push most casual players off the top of the board within a week.
Why I Haven't Fixed This Yet
Two practical reasons. First, privacy: the leaderboard is anonymous (no accounts, no email, no cookie). Sample-size weighting requires comparing across submissions, which we can do, but adding "games played" to the public view makes it slightly easier to de-anonymize a player who plays a lot from the same IP. Our current 5-per-day cap is a privacy commitment, not just a rate limit.
Second, the codebase is a static site with a single leaderboard page that already does its job. Adding "games played" and a consistency score is a small change to the data model but a bigger change to the way people understand the rankings. I'd rather do it once, after we have at least one month of real submissions to learn from, instead of designing in the dark.
The Honest Takeaway
A leaderboard is a narrative as much as a ranking. "Who is the best?" is a different question from "Who has played the most?" — and the data we collect answers the second, not the first. Until we add the missing dimensions, take the top of any casual-game leaderboard with the same skepticism you'd take a single game score from a stranger.
If you want a real read on player skill in 2048, look at the reverse math article — the proof shows that the difference between a 1024 and a 2048 average is almost entirely about whether you commit to the corner strategy, not about "talent".