reviewsaur

How it works

Reviewsaur's job is to make sure that when someone approves a PR, they actually understood the change. It quizzes every pull request automatically; whether that quiz also blocks the merge is up to you (see Why a status check).

The flow

  1. A PR is opened. GitHub notifies Reviewsaur via webhook.
  2. Reviewsaur reads the diff. Files matching your excluded paths are skipped.
  3. A quiz is generated. Reviewsaur produces a handful of questions about the change — what it does, why, and where the risks are — in the format you chose (multiple-choice, short-answer, or mixed).
  4. A link is posted on the PR, and the Reviewsaur Quiz check is set to pending.
  5. A reviewer takes the quiz. They see the questions alongside a summary of the changed files for context.
  6. It's graded instantly. Pass, and the check turns green. Fail, and they can retry.
  7. The check goes green once enough reviewers have passed (see required approvals). If you've made it a required check, that's also when the PR becomes mergeable.

Why a status check

The quiz only blocks a merge if you've added Reviewsaur Quiz as a required status check in branch protection. That's intentional: Reviewsaur can't change your repo's protection rules for you, so you stay in control. Set it up once per repo — see GitHub status check.

What it's solving

  • Rubber-stamp reviews. A large share of reviews are approved in under five minutes. A quiz makes “LGTM” mean something.
  • AI-generated code. When a diff was written by a model, the human approving it should still understand it. The quiz checks that.
  • Knowledge silos. Quizzes spread understanding of a change across the team, not just its author.

Who sees what

  • Reviewers take quizzes and see their score and per-question feedback.
  • Admins can view all responses, reset a quiz, or manually mark it passed/failed.
  • Authors see the check status on their PR.

For the full set of knobs, head to Quiz settings.