The Moto Académie meta-score
Every motorcycle spec sheet carries a score out of 100: the Moto Académie meta-score. It sums up, in a single number, what the specialist press, in France and abroad, thought of the bike after riding it.
What is the meta-score?
A meta-score is a score that sums up other scores. The principle is the same as on the websites that aggregate film reviews: instead of relying on a single review, you can see at a glance what the press thinks of a bike on average.
Each Moto Académie spec sheet brings together reviews published by motorcycle media in several countries. When these outlets give a score, we convert it to a score out of 100. When they don’t, we derive one from their verdict, using a method we developed ourselves. The meta-score is the weighted average of all of this.
The meta-score covers one generation of the bike (same engine, same frame), not a specific model year. It comes with a confidence level, because an average of two scores is not worth the same as an average of eight scores that point the same way: the larger the sample, the higher the confidence.
Where do the scores come from?
The scores we analyse come from reviews published by specialist media. For each bike, we look for reviews published, for example, by Motorcycle News, Bennetts BikeSocial and Visordown in the United Kingdom, Cycle World, Motorcycle.com and RevZilla in the United States, Motorrad and 1000PS in Germany and Austria, Moto.it and Motociclismo in Italy and Spain and, of course, Moto Magazine, Moto Journal, Le Repaire des Motards, Moto-Net and Caradisiac in France.
Every review we use is dated and linked to the exact version and model year that was tested. The full list is on the spec sheet, with a link to each article, so readers can read the full text on the original website.
Scores are converted to a 100-point scale so they can be compared: 4 stars out of 5 becomes 80, 17.4 out of 20 becomes 87, 8.5 out of 10 becomes 85, and so on. Of course, each outlet only counts once per generation: if a publication has tested the same bike three times, only the most recent review is used.
Forums and comments on videos and social media are not included in the meta-score. That’s not because riders’ opinions don’t matter, but because we cannot check who is speaking, what their level of experience is or how many kilometres they have covered, and we cannot even tell whether they have actually ridden the bike. That is why the opinion of real motorcycle journalists is so valuable: a motorcycle journalist is someone who has ridden dozens and dozens of bikes, and has therefore built up a frame of reference that nothing can replace.
When a review gives no score
This happens a lot. Many motorcycle outlets publish a full review with a fairly clear verdict, but never put a number on it. That’s a shame, because leaving these reviews out would mean calculating a meta-score from just one or two scores, which would be meaningless.
So we designed a system that gives these reviews an estimated score, based on the following grid:
- the reviewer’s verdict: they recommend the bike, recommend it with reservations, or advise against it;
- the number and severity of criticisms: for example, saying that a seat is a little too firm does not carry the same weight as saying that a gearbox is temperamental;
- the bike’s ranking in group tests, where there are any;
- the conclusion.
Each criterion has a fixed value, and the sum of these values gives a score out of 100. The points identified are stored with the score, so we can always trace why a review got 78 rather than 85.
This reading is done by a text-analysis tool, then checked against reviews that do have a score. The approach is inspired by AI systems that improve step by step: we hide the number, have it estimated, and compare. As long as the average gap stays small, the method is considered reliable. Even so, an estimated score counts for less than a real score in the calculation, with a weighting of 0.6 instead of 1.
On the spec sheet, the two are kept apart: we show how many scores are published scores and how many are estimated.
How is the score calculated?
The meta-score is also a weighted average: each review is given a weight, and the average of the scores is calculated taking these weights into account.
Two main factors change the weight:
- the type of score: 1 for a published score, 0.6 for an estimated score, as explained above;
- relevance: 1 if the review covers exactly the version and model year of the spec sheet, 0.8 if it covers another version or model year of the same generation. Reviews of a previous generation do not count because it is no longer the same bike, and it would be very tricky to let a past score affect a current one, especially if the manufacturer has fixed a common criticism.
Here is an example with three fictitious reviews:
| Review | Score | Out of 100 | Type | Relevance | Weight |
|---|---|---|---|---|---|
| Outlet A | 4/5 | 80 | published (1) | exact version (1) | 1 |
| Outlet B | 17.4/20 | 87 | published (1) | other version (0.8) | 0.8 |
| Outlet C | none | 82 estimated | estimated (0.6) | exact version (1) | 0.6 |
Meta-score = (80 × 1 + 87 × 0.8 + 82 × 0.6) ÷ (1 + 0.8 + 0.6) = 198.8 ÷ 2.4 ≈ 82.8, i.e. 83.
Without the third review, the score would also have been 83, but calculated from two scores instead of three: the same number, but with lower confidence. That is why the number alone isn’t quite enough.
Why a confidence level?
As we have seen, a meta-score of 84 based on two scores and a meta-score of 84 based on eight scores are not the same thing. In the first case, the score can swing by ten points with just one more review, whereas in the second case a new review will only move the meta-score slightly.
So a confidence level is needed. It takes two things into account: first, the total weight of the reviews used, and second, how widely their scores are spread. Does the press agree, or do the scores range from 55 to 95? If so, the spread points to a problem.
| Confidence | When | What we show |
|---|---|---|
| Insufficient | total weight below 2 (for example, a single published score and one estimated review) | no number; only “press opinion: favourable”, “mixed” or “unfavourable” |
| Low | total weight from 2 to under 4, or at least 4 with widely spread scores (standard deviation above 8 points) | the number, labelled “low confidence” |
| High | total weight of at least 4 and closely grouped scores (standard deviation of 8 points or less) | the number, labelled “high confidence” |
In every case, the spec sheet shows the number of published scores and the number of estimated scores. We would rather show no score than one that looks precise but isn’t. That said, when it comes to buying a motorcycle, the best way to decide is to ride it. The moto-academie.fr scores are only there to help you narrow down the models you might want to test-ride…
How to read a meta-score
The motorcycle press tends to score generously. A truly disappointing bike rarely drops below 60, and most recent bikes sit between 75 and 90. The scale is therefore compressed towards the top. Perhaps that also means there is no such thing as a truly disappointing bike any more? We’ll let you be the judge. Either way, the meta-score needs to be read with this in mind.
| Meta-score | What it means in practice |
|---|---|
| 85 and above | Very favourable consensus: reviewers recommend it almost without reservation, often as the benchmark in its category |
| 75 to 84 | A good bike, with identified flaws shared by several reviews: read the cons carefully |
| 65 to 74 | Mixed opinion: some reviewers like the bike and others don’t, or it is considered outdated |
| Below 65 | Serious reservations in most reviews |
Between two bikes scoring 83 and 86, the gap is not enough to say that one is better than the other: three points is no more than the difference between two reviewers riding the same bike. You can see this clearly in some YouTube reviews featuring three journalists: each one has a different opinion depending on their personal preferences, their riding style, and so on. That is the limit of personal opinion.
So the meta-score helps you spot the bikes the press likes and those it doesn’t, but it is not meant to separate two closely matched bikes. For that, there is the comparison tool, reading the reviews themselves and, above all, getting on the bike and riding it.
What the meta-score is not
The meta-score is a general figure that doesn’t tell you whether the bike is right for you. An adventure bike scoring 88 is still an adventure bike: tall, wide, with a seat height of around 850 mm. A roadster scoring 79 may well be a much better choice for a 1.70 m rider who uses the bike to commute. The meta-score sums up the quality perceived by professional reviewers, not how well a bike suits a particular rider.
It mostly reflects launch reviews. Most reviews are carried out over a few days, often on roads chosen by the manufacturer in a pleasant setting (southern Spain, Morocco…), on a new and carefully prepared bike. Long-term reviews are rarer, but they carry the same weight in the meta-score, because we simply cannot create a rule for every possible case. Reliability at 30,000 km, servicing costs and resale value are not included in the meta-score.
Scales vary from one outlet to another. A 4/5 from a reviewer who almost never goes below 4 is an average score; the same 4/5 from a publication that is usually harsh is a good score. We convert to a 100-point scale without correcting for these quirks: doing so reliably would take a large number of reviews from each outlet.
Some of the scores are estimated. Even with a fixed grid and checks against reviews that do have a score, an estimated score is still our reading of a text, sometimes of a text translated into French. That is why it carries less weight and is flagged.
It does not reflect owners’ opinions. Feedback from riders who use the bike every day is rare. When we can find it, it is handled separately, on the spec sheet, but never mixed with the press score, because we don’t know who the rider leaving the review is.
The meta-score can change. One more review can shift the score, especially when confidence is low. The date of the last update is shown on the spec sheet.
Spotted an error?
Each score is linked to the review it comes from. If you spot a mistake, write to us using the contact form, stating the spec sheet and the link to the review. Corrections will be made as quickly as possible.
Frequently asked questions
Is the meta-score Moto Académie’s opinion? No. Our opinion is in the “Our review” section of each spec sheet, written out in full. The meta-score only sums up what the specialist press has written.
Why do some spec sheets have no meta-score? Because there are not enough reviews for a number to be reliable. The spec sheet then only states whether the press opinion is favourable, mixed or unfavourable.
What is an estimated score? It is a score given to a review based on its verdict, its criticisms and its conclusion, using the same grid for every review. It counts 0.6 times as much as a published score and is flagged as such.
Why doesn’t a review of the previous generation count? A new engine, a fixed flaw or a new frame changes the bike. Reviews of the previous generation can still be read on that generation’s spec sheet, where one exists.
Does the meta-score take price into account? Only indirectly: a reviewer who finds the bike too expensive for what it offers usually says so, and their score reflects it. But the meta-score is not a value-for-money rating.
Do riders’ comments on forums count? No. We cannot check who is speaking or whether they have actually ridden the bike. Owner feedback, when we find it, is handled separately and never mixed with the press score.