Changelog
What's new in Reviewsaur, newest first. No version numbers — just dates and what changed.
October 2026 — The walkthrough is now a short review guide
The walkthrough used to retell the whole diff, which got long on big pull requests. Now it tells a reviewer where to spend their time.
- A short summary — one to three sentences on what the pull request does.
- Look closely — the few places worth reading slowly. Each one says why it matters and asks one Check: question to answer from the code. Each place links to the exact line in the pull request on GitHub.
- Safe to skim — the files with routine changes.
- Points and asks, never judges — a tip can ask a pointed question, but it never says the code is right or wrong.
- Shown open — the guide is open in the PR comment and in the quiz sidebar, not hidden behind a click. The complexity verdict works as before.
Walkthroughs made before this change are no longer shown. Their verdict still counts, and the next regeneration makes a new guide.
October 2026 — See the code behind each question
- A code column on the quiz — next to each question you now see the part of the diff it is about, with syntax colors. Switch between Diff (added and removed lines) and Code (the code as it is after the change).
- Make room when you need it — the code column and the PR info column can be folded into a thin strip at the side. The quiz page remembers how you left them.
- A question that is not about one place in the code shows no code column.
- The code is deleted when the quiz is done — the excerpts are stored only while someone can still take the quiz. They are deleted from your quiz as soon as you finish it, and from the whole quiz when the pull request is closed or merged, or a newer quiz replaces it. None are kept longer than 30 days. If you reopen a pull request, Reviewsaur makes a new quiz, so the code is back. See How do I delete my data?.
October 2026 — One quiz per pull request, a new one when you ask
Before, every push made a new quiz and a new walkthrough, even when nobody looked at them. Now Reviewsaur makes them once and lets your team decide when a new one is worth it.
- Made once, when the pull request is opened — a new push no longer replaces the quiz. The current quiz keeps working, and anyone who already passed still counts.
- The comment says when it is out of date — after a push, the PR comment shows how many new commits came since the quiz, with a link to regenerate for the latest commit. Every comment also shows when the quiz and walkthrough were made, and from which commit (times in UTC).
- Regenerate in one click — any team member can open the link from the PR comment, or click Regenerate quiz on the quiz page. Reviewsaur makes a new quiz and walkthrough for the latest commit and takes you to the new quiz. If you are halfway through the current quiz, it asks first.
- The check follows your pushes — the Reviewsaur Quiz check moves to the newest commit by itself, so a required check never gets stuck on "Expected — waiting for status".
- A failed attempt on old code gets fresh everything — if you fail a quiz that is from an older commit, your next quiz and the walkthrough are made for the latest commit.
- Bot pull requests are skipped — pull requests from dependabot, renovate and other bots get a green check and no quiz or comment.
- Quiet PR comments — the pass comment no longer shows a score or attempt number, and a failed attempt posts nothing. A retry just posts the new quiz.
See When a new quiz is generated.
October 2026 — Clearer quiz questions
- Plain English — questions read like a teammate asking in chat: short, direct, one situation per question, no formal words.
- Only what the diff shows — questions are built from the title, description and diff only, never from guesses about code outside the change. No more questions about small facts like an exact status code or a message text, unless that value is the point of the pull request.
- 4 questions per quiz — down from 5. A trivial change that still gets a quiz gets 2 instead of 3.
- Difficulty is per repository only — the per-developer difficulty setting is gone. The per-developer "I don't know" setting stays.
September 2026 — Reviewsaur now decides when a review is needed
- A complexity verdict on every pull request — the walkthrough now rates each change trivial, normal or complex, judged from the diff alone, and shows the reason on the pull request and next to the quiz.
- Skip the quiz for trivial changes — turn it on in quiz settings and a trivial pull request gets a green check with no quiz, plus one comment giving the reason. Pro, on by default.
- Shorter quizzes for trivial changes — when skipping is off, a trivial pull request gets 3 questions instead of 5. Every plan.
- Never trivial paths — glob patterns for files that always deserve a full quiz, whatever the verdict. Every plan.
September 2026 — A straight path to your next attempt
- Your next attempt finds you — fail a quiz and the result page now waits for the new one and links straight to it, instead of leaving you to hunt for a new comment on the pull request. It usually takes under a minute, and you can read your per-question breakdown while you wait. The new quiz is still posted on the PR as well.
September 2026 — Question categories, focus on what matters, per-developer settings
Quizzes now spend their questions where the change matters, and admins can tune the quiz per person.
- Focus on what matters — generation uses the PR walkthrough as an importance ranking. Cosmetic changes (styling, copy, renames, formatting) get no questions unless the PR is only cosmetic.
- Question categories — every question has a lens: Understanding, Risk, Architecture, Trade-off or Verification, shown as a badge on the quiz and the results. Pick which ones a repository uses in quiz settings. Architecture and Risk, and choosing the mix yourself, are Pro.
- Per-developer settings — give one person a different "I don't know" setting from the Members page (docs). The link on the PR stays the same for everyone. Pro.
September 2026 — Difficulty levels and an honest "I don't know"
Two settings that change how hard a quiz is, and what a reviewer can do when they are stuck.
- Difficulty — set a repository to Easy, Medium or Hard in quiz settings. It changes how deep the questions dig, not how many there are: Easy questions can be answered from a single hunk of the diff, Hard ones need several parts of the change put together. Hard also grades open-ended answers more strictly, rejecting vague ones. Medium is the default and matches how quizzes worked before.
- The level is fixed when the quiz is generated, so changing the setting never changes how a quiz already in progress is graded.
- "I don't know" — turn it on in quiz settings and every question gets an I don't know button. It counts as wrong, but the explanation appears right away and the result page marks it apart from a wrong guess.
Both are available on every plan.
August 2026 — Insights
A new Insights page for team admins, at /t/<team>/insights. It answers one
question: how well does your team understand the code it approves?
- Four headline numbers — quizzes completed, average score, first-attempt pass rate, and how many merged pull requests were never covered by a passing quiz. Each one shows the change against the previous period of the same length.
- Pick a time range — last 30 days, 90 days, or 12 months. The 30-day view is bucketed by day, the longer ones by week.
- Pass rate and volume over time — a trend chart. Periods with no completed quizzes break the line instead of drawing a straight guess across the gap.
- Attempts to pass — of the reviewers who eventually passed, how many did it in one try, two tries, or three or more.
- By repository — completed quizzes, pass rate, and average score per repo. A repo where everyone scores low points at unclear code, not at people.
- Merge tracking — Reviewsaur now records when a pull request is merged or closed, and latches whether it had reached its approval threshold at that moment. The threshold is frozen at merge time, so changing required approvals later never rewrites past results.
- Honest about small samples — with only a handful of quizzes in a range, the page says so, because a single result can swing the rates a lot.
July 2026 — New ways to sign in
Authentication is now powered by Stytch:
- More sign-in options — alongside GitHub, you can now sign in with Google, a one-time code sent to your email, or an email and password (with self-serve password reset).
- One account, any method — sign-in methods sharing the same email address resolve to the same account, so you can switch between them freely.
- GitHub linking for quizzes — taking quizzes still requires a linked GitHub account, so every result stays attributed to a person. If you signed in another way, you'll be prompted to link GitHub before your first quiz.
July 2026 — More quiz settings on the Free plan
- All question formats for everyone — Mixed and Open-ended question formats are no longer Pro-only. Every plan can now choose any question format.
- Custom pass thresholds for everyone — the minimum score to pass is now editable on the Free plan too, instead of being fixed at the default.
July 2026 — Faster, more reliable status checks
- Instant status check — the “Reviewsaur Quiz” check now appears on your PR within seconds of opening it (as “Generating quiz…”), instead of only after the quiz is ready.
- Failures are visible — if quiz generation fails, the check is marked as failed with the reason, instead of silently never appearing. Hitting your plan's quiz limit completes the check as neutral, so a billing limit never blocks a merge.
- Correct check on new pushes — pushing new commits to a PR now creates a fresh check on the latest commit; the old one is closed as superseded.
June 2026 — Initial release
The first release of Reviewsaur:
- Pull request quizzes generated automatically from each PR diff.
- GitHub status check (“Reviewsaur Quiz”) you can mark as required to gate merging on a passing quiz.
- Fresh quiz on failure — fail and Reviewsaur generates a new set of questions so you can try again.
- Per-repo settings: question format, pass threshold, required approvals, and excluded paths.
- Team dashboard with active repos, quizzes this month, and pass rate.
- Members & roles and billing (Free / Pro / Enterprise). Reviewers sign in with GitHub to take quizzes, so every result is attributed to a person.
- Documentation: this docs section (getting started, how it works, settings, the GitHub status-check walkthrough, plans & limits, and an FAQ), plus inline “Learn more” links from repository settings into the docs.