How we measure — and when this number should not be trusted

No search tools are attached when we ask. A model can only answer from what it retained during training, so both lists measure how much room a brand occupies in that model's head at the moment it was trained — not what the model could look up today.

That is a different metric from what AI can find on the web right now. The latter shifts daily with published content and can genuinely be moved by PR and distribution budgets. Ours barely moves between model releases.

This choice has a direct consequence: our scores cannot be changed by publishing a burst of articles this week. That is the technical basis for refusing paid ranking changes — not merely a promise.

It cuts the other way too. Our data inherently lags reality: a brand founded last month has no place in any model's training data, which is a limit of the metric, not a verdict on the brand.

One question goes out in Chinese, then the same question goes out in English. The same models answer both, and both runs happen on the same day. The six cards re-measured weekly on the home page are three questions in two languages.

The two lists are not the same, and you can check that yourself: in the 2026-09-02 run of Most Popular AI Glasses Brands, the Chinese list placed XREAL ahead of Rokid and the English list — same day, same models — reversed them; Oakley, Amazon and Google show up in the English top ten and are absent from the Chinese one.

For a team selling out of China, the pair reads as: how a model remembers your brand in your own language, and how it remembers you in your buyer's. For an English-speaking reader it is simply the same question in two languages, same panel, same day.

All calls go through OpenRouter. Within one evaluation the 12 models are queried in parallel and cannot see one another's answers.

Models that support structured output are constrained by a JSON schema; the rest return text that we then parse. The verification page labels which path each response took, so you can tell them apart.

Each model is asked once per evaluation. This is the single most important collection limit we have, and the data-grade section below addresses its consequences head-on.

Every raw response is stored in full, append-only, never rewritten. When a certificate is issued we additionally freeze a scoring snapshot, so even if we later recompute history, the numbers on an issued certificate do not move.

Cross-model brand-name normalisation (merging "特斯拉", "Tesla" and "Tesla Inc." into one entity) is done by a single model. When normalisation fails we degrade to not merging — the same brand then splits into several entries and its score is diluted. That is a known degradation, and it is logged.

Each model's ranking is converted to a position score: rank 1 scores 10, rank 2 scores 9, down to rank 10 scoring 1; not mentioned scores 0. The position score is multiplied by that model's weight to give its contribution for that brand.

Contributions from every model that mentioned the brand are summed, divided by the theoretical maximum — every model ranking it first — and multiplied by 100 to give the 0–100 index.

So the index reads literally as: what percentage of a unanimous first place did this brand achieve.

If one model repeats the same brand within its own list, it counts once, at its highest position — this stops a single malformed response from inflating a score through repetition.

One brand, three spellings: model A writes 特斯拉, model B writes Tesla, model C writes Tesla Inc. Left unmerged, it becomes three brands each with a sliver of the score, and none of them ranks.

Method: all raw brand names returned by every model in one evaluation are de-duplicated and handed, in batches of at most 30, to a normalising model (currently deepseek-v4-flash), which merges them into canonical names. So that a brand stays one line across periods, we also pass in every canonical name used in past evaluations, with the instruction "if this new name refers to a brand already in the list, reuse that name".

We deliberately do not take the shortcut of "seen it before, copy last time's answer". We tried; a single miss became permanent. We would rather ask every time.

The batching is not a stylistic choice: when everything was submitted at once, the evaluations with the most names failed to merge at all — 4 of 25 evaluations merged not a single pair, scores were split across duplicates, and nothing warned us. Batches now fail independently, every call's raw response and token usage is kept, and each evaluation records its normalisation state as ok, degraded or failed.

Known limitation: if a batch fails, the fallback still maps each raw name to itself — those names go unmerged that period and the score is diluted. This is a logged degradation, flagged on the verify page. A change of normalising model changes brand groupings; see the Panel changes section.

When normalisation misses, one company occupies two or three rows on the same ranking and its score is split across them. A September 2026 review of 25 evaluations found 53 name groups in this state, and 7 evaluations where two rows for the same company were actually printed on the public ranking. Merging happens in three stages, each more cautious than the last.

Stage one, spelling only: two spellings that are identical after Unicode folding and after removing case, spaces and hyphens — Browser Use and Browser-use, for instance. There is no judgement in this stage. The rule is a few dozen lines of pure function (src/ask10ais/brand_key.py) with test cases, so anyone can replay it, which is why it is allowed to apply on its own.

Stage two, candidates only: identical after stripping a company suffix (Monex and Monex Securities), or the same raw name filed under different canonical names in different periods (マネックス証券 was filed as Monex in one evaluation and stood alone in another). These clues come from our own history and can be checked against the evaluation numbers, but they only enter a review queue; they never move a score by themselves.

Stage three, a person decides: candidates are confirmed one by one, written into a CSV and only then loaded. Rejections are kept on file too. Whether a cross-language name or a sub-brand is the same company is a business judgement, not a data one — no script can settle whether Xiaomi and Mijia, or Binance and Binance Japan, are one brand. The judgement can be wrong; when it is, it goes in the corrections log.

Merging happens at the display layer only. The merged rows are still in the database exactly as they were, their canonical names unchanged to the byte, so certificate numbers and any certificate or badge link already issued keep resolving to the same data. Each merged row lists its members underneath precisely so the pre-merge picture stays visible.

Panel completeness = the summed weight of models that returned a usable answer this run ÷ the summed weight of the whole panel. Two thresholds, set from the distribution of 25 real evaluations, not guessed:

Every certificate carries a number. Enter it on the verification page and you can read the exact question asked, the rank each model on the panel gave, each model's weight and contribution, and the full raw response from each model — no login, no payment.

The scoring breakdown comes from the snapshot frozen at issuance; the raw responses are read from the append-only response log. They are two records of the same call, and the page labels which is which.

If you find anything that does not reconcile, write to us. Leaving an error on this page hurts us far more than reporting it does.