LegitShow is the trusted source on every newly launched software product: what it does, who it’s for, how it actually holds up, and whether the AI engines are already reading it. Built to be what AI cites.

Web apps, SaaS, AI tools, MCP servers and developer tools. How we measure →


Legit.Show benchmarks every launched service it lists — measured deterministically from the public surface. See the methodology →

Cross-links · Directory · Reports · Insights · What AI reads · Methodology · About

Privacy · Terms · @Legit_Show on X · GitHub · operated by Madeflo Inc., a Delaware corporation. Benchmark engine powered by commit.show.

AI & Agents

Umwelt

Last updated 2026-11-04 · benchmark measured 2026-11-20 — deterministic & reproducible

First Silicon: a benchmark testing whether AI models can find real hardware bugs.

Umwelt at a glance

Umwelt's First Silicon is a benchmark dataset of 86 real hardware bug fixes from open-source chip designs, used to test how well AI language models can detect genuine defects without raising false alarms.

Good for

  • AI researchers and model developers wanting to measure how well a language model finds real hardware bugs
  • Chip design teams or formal verification specialists checking whether an AI model can be trusted to review RTL code
  • Teams wanting an independent, blind evaluation of their own AI model or verification tool against real bug fixes

Questions people ask

What does the First Silicon benchmark actually test?
It gives an AI model one version (buggy or fixed) of a real hardware module and asks it to write SystemVerilog assertions describing the intended behavior; a formal checker (Yosys and SymbiYosys) then checks those assertions against both the buggy and fixed versions to see if the model truly caught the bug without raising a false alarm.

How well do current AI models perform on this benchmark?
The best-performing configuration, Claude Opus 5.5 at maximum effort, caught only 16 of 86 cases cleanly, and most bugs escaped detection by every model tested.

Where do the bugs in the dataset come from?
The 86 cases are real bug fixes taken from open-source chip projects: 70 from OpenTitan (Google's open-source security chip) and 16 from 10 other open-source projects.

Can I get my own AI model or verification tool tested on First Silicon?
Yes, Umwelt offers to run your model or verification tool on First Silicon and share the results only with you, or to check your own RTL design for bugs, which you can arrange by contacting them.

Is the benchmark data publicly available?
Umwelt publishes the outcome for every bug along with five bugs in full detail and a salted SHA-256 commitment for each bug and prompt, but keeps the other 81 bugs private so the benchmark stays usable for evaluating future models.

Written by Legit.Show from umwelt.vision on 2026-11-09. It does not change the score.

74/100
7/7 frames
Top 72% of 19,654 measured · #37 of 69 in ai model benchmark leaderboard
Legit Benchmark — the simple average of 7 measured frames. Frames we could not measure are left out of the average, never counted as zero. Every frame is shown below with its evidence.
Checked 2026-11-20 · scores move as sites change

Are you the maker of Umwelt? Claim this listing. It is free, takes a meta tag, a DNS record or GitHub admin rights, and never changes the score.

Launched something? Add your product, free.

To cite this score: legit.show/s/umwelt-vision/2026-11-20. That address never changes; this page moves with every re-measure.

Legit.Show scored Umwelt 74/100 on 2026-11-20, measured across all 7 frames from its public surface. legit.show/s/umwelt-vision/2026-11-20

Is Umwelt production-ready?

Legit.Show scores Umwelt 74 out of 100 — the simple average of its 7 measured frames. Legit.Show ran its deterministic 7-Frame production-readiness benchmark on Umwelt (public-surface assessment), measured from the public surface with no LLM in the scoring path. Its strongest frame is Discoverability; its weakest is Privacy. Every frame it averages is published with its evidence on the Legit.Show listing.

The 7 Frames

What we measured

Who it's for

AI researchers · Hardware engineers · Chip design teams · AI model developers · Formal verification specialists

Sources and updates

Description
Taken from umwelt.vision's own website on 2026-11-04.
Benchmark
Measured by Legit.Show from the live site, the way any visitor sees it — with no access to its code or accounts. Same method for every product, and no AI decides the score. Last checked 2026-11-20.

Put together from public information, without Umwelt's involvement. If anything here is wrong, tell us and a person will check it.

Visit Umwelt → · Alternatives to Umwelt → · How this was measured →

See how Umwelt ranks among tested ai model benchmark leaderboard products →

Other tested ai model benchmark leaderboard products