Back

How Meticulous works

How Meticulous works

The bottleneck has shifted from writing the code to reviewing and safely shipping it. If you can see the full set of impacts a change would have, before shipping it, then you can knock out that bottleneck -- often with quite dramatic results.

"Once we started using Meticulous, we couldn't imagine working without it." - Tony Xu, Dropbox

However, this only works if developers can trust that the view is showing every impact of the change. A partial view is not worth much, while a view you can trust to be complete changes the way you work & ship.

Exhaustiveness is therefore the key property. But also an extremely hard one to achieve. Our customers, such as Lovable, Grafana, Dropbox, ElevenLabs, Replit, and Notion, often have frontend codebases spanning millions of lines of code. A single change can have impacts across countless configurations, data edge cases, screens, and feature flag combinations. Every percentage point closer to 100% exhaustiveness gets exponentially harder to achieve.

But we've found that every inch closer to 100% creates step-changes in the speed our customers can ship and iterate at:

  • The review bottleneck starts to fall away
  • Agents start to generate far better changes in the first place, by being able to iterate against a full & detailed view of the impacts of the changes they are making
  • Product quality improves, as developers see the impact of their change across varied, representative cases, making it easier to apply the right product judgment

And so we've been relentlessly focused over the years on building a system that achieves true exhaustiveness. We’re not there yet, but we’re remarkably close — already at a level far beyond any other approach, and with dramatic results on how our customers work. Ship a major React upgrade to a multi-billion dollar project without blinking an eye. This post details the inner workings that make that possible.

An ever-evolving asset

In Meticulous your agents and engineers are presented with a view onto the full impact a change would have if it were shipped:

  • The impacts on existing workflows, split out by edge case
  • The behaviour of new features, split out by edge case
  • Performance and accessibility impacts

The tricky part is ensuring this view is exhaustive. Meticulous achieves this by evolving a repository of 'sessions', each containing the workflow and data needed to access a particular edge case. While a typical enterprise test suite may have 10,000 tests, Meticulous often stores millions of sessions to allow it to cover every sub-case across the application surface.

These are populated by a combination of three primary sources:

  1. Via agents: agents iterate against code coverage to pre-populate the initial repository, and, for each subsequent PR, an agent swarm is spun up to cover any gaps or new code. These sessions feed back into the repository.
  2. [Optional] Via passive recording: add a script tag to record on internal staging stacks and developer and cloud agent local development environments. This captures the data and workflows engineers and agents are accessing across the company. Often this data is directly or indirectly shaped by engineers, and provides data examples that are maximally representative of production data (crucial when applying judgements to changes in UX).
  3. Via multiplication: the system learns the many axes of variation that are unique to your domain (from permission levels to variants on the data itself) and the transformations that multiply existing sessions to cover the many variants.

The data is indexed to allow pulling all possible relevant sessions for a given code change, and to pull the subset within these that test the particular code change as efficiently as possible per unit of compute while still being fully exhaustive. The primary index maps each session to the lines of JS and CSS code it executes. Meticulous automatically maintains and updates these sessions and their indexes as your application’s code, network APIs and frontend surface evolve.

These indexes are used internally by Meticulous, but can also be leveraged by your own agents locally to access cases they would not be able to otherwise, and discover the full set of relevant cases -- they can pull all the relevant needle-in-a-haystack data and workflows which allow them to target a particular JS, CSS or assets change, and discover its many edge cases.

* If configuring Meticulous for full-stack testing then we also encourage adding a dependency on Meticulous to your backend services.

Testing multi-million line codebases, without blind spots

When Meticulous is triggered on a given code change it analyzes the change statically to trace the full impact through the codebase, and from this finds every edge case that could be impacted within the blast radius of the change. It:

  1. Pulls existing sessions from the session repository
  2. Generates new sessions on the fly by spinning up browser-use agents as needed to cover any gaps or QA new features. These agents leverage the data already in the session repository to discover the relevant edge cases and representative test data examples.

The data Meticulous builds catalogues every edge case across the codebase along with the data and instructions needed to access them. A large engineering organisation may ship 100,000 pull requests a year. By building this catalogue over time, rather than testing from scratch on each pull request, Meticulous can effectively leverage 3–5 orders of magnitude more tokens and compute. This is what enables Meticulous's agents to test PRs with such speed, cost efficiency, and exhaustiveness.

Once the set of sessions that exhaustively tests the change has been selected, Meticulous spins up browsers in within our cluster and replays those sessions against both the version of the application before your code change (where relevant) and the version of the application after your code change.

As it runs these flows it produces:

  • A sequence of videos of how your application behaves before the code change (where relevant)
  • A sequence of videos of how your application behaves after the code change

It then diffs each corresponding key frame across each pair of videos, detecting every pixel difference, small or large.

Every flow is run in Meticulous’s deterministic browser. This allows Meticulous to hit the extreme scale that’s required to provide exhaustive guarantees while ensuring near-zero noise, allowing agents to trust the differences returned. Changes with a large potential blast radius often require every key frame in thousands of pairs of videos to be compared for a single change. Many regressions, product issues or edge cases will present via a diff of a handful of pixels in a single frame of 10,000. Eliminating noise is therefore essential to detect these with reliability.

This whole process normally takes 2–4 minutes, even for code changes that require simulating 1000s of flows to test exhaustively.

Once ready, the diffs are grouped and deduplicated for fast review, by human or agent. If the Meticulous test run was triggered from a pull request we'll post a comment, summarising the results and linking into Meticulous to review the diffs visually, along with videos of any new features, split out by edge case -- with bugs and quality issues flagged. If it was triggered by an agent then the agent is provided with tools to understand the set of differences and results.

Meticulous can also check performance and accessibility. Since the flows are executed in a deterministic browser Meticulous is able to detect even subtle performance regressions introduced by a pull request with high signal and low noise.

Ready to try?

We’d love to chat! Book a demo.

Meticulous is used by every engineering team that touches the frontend or full-stack at companies like Lovable, Dropbox, Notion, ElevenLabs, Wiz and many more of the world’s leading engineering organizations. Join them to enable a verification loop without gaps, and ship code at the speed AI can write it.