How ranking works

Seeded starting ratings, the default 1000-point baseline, and blind local votes: a practical guide to the Elo-style leaderboard.

Two anonymous model implementations connected by a review, vote, and reveal process

A leaderboard compresses starting assumptions and recorded choices into a small table. That is useful only if readers understand what each represents. HDL Arena starts from explicit reference ratings where available, then adjusts them using account votes on AI-generated RTL. Ratings do not measure functional correctness, physical area, or timing, and are not calculated from AI review scores. Local preference evidence and external starting assumptions must remain distinct.

The comparison is blind before the vote

A new battle uses two distinct AI models and one shared language choice: Verilog, SystemVerilog, or VHDL. Both models receive the same hardware request. Implementations appear under neutral labels, and their display positions are shuffled independently of completion speed. You review the code and choose one implementation, a tie, or neither. Generation and AI review must complete before voting is enabled.

Model identities, AI review text and scores, and compilation/synthesis reports are revealed after the vote. This matters because a famous model name, a high AI score, or a lower generic cell count could otherwise bias the comparison. The code and specification are the basis of the blind choice. The revealed information adds context afterward rather than deciding the result for you.

Accounts generate and vote on their own completed battles. Ordinary battle pages are creator-only; the leaderboard does not publish the underlying prompts or RTL. Each account can vote once on its own completed battle. This is aggregate feedback from the people using the arena, not a randomized survey of every hardware engineer or a curated suite of verified design tasks.

Starting ratings: external references and the 1000 baseline

Models explicitly mapped to the frozen Design Arena Overall Agentic reference start at their imported reference rating. Unmatched models start at the default baseline of 1000. Those external coding/design ratings are not HDL Arena votes or proof of RTL ability. The leaderboard identifies the source and starting rating. A category with zero local votes can still show an ordering, but that ordering is the prior alone, not local hardware evidence.

The server replays recorded local votes in chronological order, breaking equal timestamps by vote ID. Before processing a vote, it snapshots participating models’ ratings so both sides use the same pre-vote values. Ratings are updated through the history and rounded for display. The frozen starting references are not automatically refreshed during replay; changing them would change the result even without a new local vote.

expected_A = 1 / (1 + 10 ** ((rating_B - rating_A) / 400))
new_rating_A = rating_A + 24 * (actual_A - expected_A)

The 400-point divisor controls how rating differences become expected scores. The update factor K is 24. If A and B are both rated 1000, each has an expected score of 0.5. A win gives A an actual score of 1 and B an actual score of 0: A gains 12 points and B loses 12. Their new ratings are 1012 and 988. This example describes one vote, not a claim about any current model’s ranking.

If A is rated 1200 and B 1000, A’s expected score is about 0.760. An A win adds about 5.77 points to A, while an upset loss subtracts about 18.23 points. The rule rewards a surprising outcome more strongly than an expected one. Because the application replays sequential updates, vote order can affect the final rating; this is not a batch-fitted strength model or a statistical confidence interval.

Tie and neither are different

A tie assigns an actual score of 0.5 to both participants. Equally rated models do not move after a tie. Unequally rated models move toward one another because the stronger model did worse than its expectation and the weaker one did better. Choose tie when the implementations are comparably preferable, not simply because both source files happen to compile.

Neither assigns an actual score of zero to both participants. Starting at 1000 each, both lose 12 points and finish at 988. This special rule is deliberately not zero-sum: it records that neither implementation was acceptable rather than treating two unacceptable answers as a shared half-win. Do not assume that the sum of all ratings must stay fixed when neither votes occur.

Historical battles can contain more than two participating models. The implementation averages each participant’s pairwise rating changes across its opponents in that vote. For a selected winner, comparisons between unselected participants receive a half score; tie and neither keep their respective half and zero rules. New battles use two models, so the ordinary head-to-head formula is the clearest way to understand current usage.

Win rate is a separate count

Wins count local ballots explicitly selecting that model. Win rate is wins divided by local voted participations, multiplied by 100 and rounded to one decimal place by the server. Ties and neither votes increase the denominator without adding wins. A model with no local votes has no observed win rate. The server separately counts distinct voted battles. Some battle counts in the current interface are display-scaled by 1000; that multiplier does not create additional ballots or statistical evidence. Use the actual local vote count when judging sample size.

The optional HDL-language and RTL-category filters replay only matching local votes, starting from the same model-specific priors. They do not just hide rows from the overall table. The category views are Overall, DSP RTL, Generic RTL, Memory RTL, and Complex RTL. Legacy battles without a category contribute to Overall only; their categories are not guessed. A filtered rating can differ substantially from the overall rating, especially with few votes. Model search is a display filter and does not rerun the rating calculation.

Read the table with its limits

  • AI review scores, compilation statuses, and generic cell counts never drive local rating updates.
  • Separate seeded starting assumptions from actual local votes; neither display-scaled battle counts nor an external reference establish an RTL sample size.
  • Prompt difficulty, voter preferences, and model pairing affect the observed comparisons.
  • A preferred implementation still needs independent functional verification and target-specific implementation checks before use.