SnagFrogThe short answer: tag every error and bug report with the version it came from, then compare the new version's rates per day with the version before it, over the same days since release. If one rate roughly doubles, roll back or fix it now. SnagFrog's release health does the comparison for 14 days after every release.
What should you compare after a release?
- Errors and crashes per day, from what the app captures on its own.
- Crashes per day on their own, since one crash costs more than ten warnings.
- Bug reports per day, because some regressions never throw an error.
- Errors first seen in this version, which is where a regression usually lives.
Step 1: Send the version with everything
On the web, add data-version to the widget's script tag along with data-errors="auto". From any other app, send version to the errors API. The Swift package and the Godot addon send it for you. See how to track JavaScript errors for the setup.
Step 2: Publish a release with the same version
When you ship, publish the release in SnagFrog with the same version number. That is the moment the comparison starts.
Step 3: Read the health card
Each recent release on the Releases tab gets a card with a verdict:
| Verdict | What it means |
|---|---|
| Too early to tell | A few hours old, and nothing has gone wrong yet. |
| Healthy | No rise in errors, crashes or bug reports, and nothing new crashes. |
| Worth a look | Something rose, but not enough to call it a regression. |
| Looks like a regression | At least one rate is about twice the previous version's, often enough that it is not one unlucky user. |
Below the verdict are the reasons, a table of each rate against the previous version, bars for each day since release, the errors first seen in this version and its top crashes. One click opens the reports and errors from that version.
Step 4: Act on a regression
A release that looks like a regression shows a banner on the dashboard's Home and turns the app's Stability quadrant to Act now. Ask Claude to look at the errors first seen in that version, or roll back. After launch week, the Release Day page keeps a week-one report card that the next release is compared against.
Questions
How do I know if a new version of my app has a regression?
Compare rates, not totals: errors, crashes and bug reports per day on the new version against the version before it. A rate that roughly doubles, often enough that it is not one unlucky user, is a regression. SnagFrog does this for 14 days after every release.
Why compare per day instead of total errors?
A new version starts with few users and grows. Totals make an old version look worse just because it has been out longer. Per-day rates over the same days since release are a fair comparison.
What do the verdicts mean?
Too early to tell: only a few hours old and nothing has gone wrong yet. Healthy: no rise and nothing new crashing. Worth a look: something rose, but not enough to call it a regression. Looks like a regression: at least one rate is about twice the previous version's.
What if my errors have no version?
They are put on whichever version was the latest that day, and the health card says so. Send the version with every error for an exact count.
Related guides
- How to publish a changelog for your app
- How to add a public roadmap with feature voting
- How to collect testimonials from your users
- How to launch a new version of your app: a launch day checklist
All 21 guides, from an idea to a launched app.
Contact us · More guides · Docs · Compare with other tools · Changelog