LegitShow is the trusted source on every newly launched software product: what it does, who it’s for, how it actually holds up, and whether the AI engines are already reading it. Built to be what AI cites.

Web apps, SaaS, AI tools, MCP servers and developer tools. How we measure →


Legit.Show benchmarks every launched service it lists — measured deterministically from the public surface. See the methodology →

Cross-links · Directory · Reports · Insights · What AI reads · Methodology · About

Privacy · Terms · @Legit_Show on X · GitHub · operated by Madeflo Inc., a Delaware corporation. Benchmark engine powered by commit.show.

AI & Agents

SWE-sweep

Last updated 2026-10-01 · benchmark measured 2026-11-10 — deterministic & reproducible

A benchmark measuring how many bugs AI agents can find and fix in real codebases.

SWE-sweep at a glance

SWE-sweep is a benchmark that tests how well AI coding agents can find and fix multiple real bugs in a codebase without being told what or where the bugs are, for researchers and teams evaluating language model agents.

Good for

  • Evaluating an LLM agent's ability to discover bugs on its own, with no hints about type or location
  • Comparing different coding agents and models on resolved bug rate, cost, turns, and tokens used
  • Testing whether an agent's fixes introduce regressions or break existing tests

Questions people ask

What does SWE-sweep actually measure?
It measures how many bugs a coding agent can find and fix on its own in a real repository that contains many bugs at once, with no information given about what the bugs are or where they are located.

How are the test repositories and bugs chosen?
Tasks are built from real issue-pull request pairs in open-source repositories, picking a single commit where the maximum number of bugs exist at the same time, and filtering to keep only bugs that can be discovered just by reading the repository.

How is an agent's fix judged as successful?
A bug counts as resolved only if hidden tests tied to that bug all pass after the fix and the repository's original test suite still passes, which also checks for regressions.

Does the agent get any hints about the bugs?
No, the agent only receives the repository at a fixed starting commit and an instruction to find and fix as many bugs as possible, with no issue descriptions, filenames, or line ranges.

Is there a way to compare different AI models and agents on this benchmark?
Yes, SWE-sweep includes a public leaderboard that compares models and agents by percentage of bugs resolved, total and per-repository cost, turns, and tokens used.

Written by Legit.Show from swesweep.com on 2026-10-26. It does not change the score.

70/100
7/7 frames
Top 80% of 19,545 measured · #45 of 68 in ai model benchmark leaderboard
Legit Benchmark — the simple average of 7 measured frames. Frames we could not measure are left out of the average, never counted as zero. Every frame is shown below with its evidence.
Checked 2026-11-10 · scores move as sites change

Are you the maker of SWE-sweep? Claim this listing. It is free, takes a meta tag, a DNS record or GitHub admin rights, and never changes the score.

Launched something? Add your product, free.

To cite this score: legit.show/s/swesweep-com/2026-11-10. That address never changes; this page moves with every re-measure.

Legit.Show scored SWE-sweep 70/100 on 2026-11-10, measured across all 7 frames from its public surface. legit.show/s/swesweep-com/2026-11-10

Is SWE-sweep production-ready?

Legit.Show scores SWE-sweep 70 out of 100 — the simple average of its 7 measured frames. Legit.Show ran its deterministic 7-Frame production-readiness benchmark on SWE-sweep (public-surface assessment), measured from the public surface with no LLM in the scoring path. Its strongest frame is Reliability; its weakest is Privacy. Every frame it averages is published with its evidence on the Legit.Show listing.

The 7 Frames

What we measured

Who it's for

AI researchers · LLM developers · Software engineering teams · Benchmark creators

Sources and updates

Description
Taken from swesweep.com's own website on 2026-10-01.
Operator
Meta Platforms, Inc, as named in swesweep.com's terms, privacy page or footer.
Benchmark
Measured by Legit.Show from the live site, the way any visitor sees it — with no access to its code or accounts. Same method for every product, and no AI decides the score. Last checked 2026-11-10.

Put together from public information, without SWE-sweep's involvement. If anything here is wrong, tell us and a person will check it.

Visit SWE-sweep → · Alternatives to SWE-sweep → · How this was measured →

See how SWE-sweep ranks among tested ai model benchmark leaderboard products →

Other tested ai model benchmark leaderboard products