Visual Regression Testing in a React App | Step-by-Step Tutorial
A guide on how to do visual regression testing in a React app - a step-by-step tutorial. This includes an introduction to visual regression testing, spinning up an example app and a step-by-step guide on creating some visual regression tests.

- What is Visual Regression Testing
- Why do Visual Regression Testing
- Prerequisites
- Example App
- Visual Regression Testing with Meticulous
- Register on Meticulous
- Install Meticulous and Bootstrap meticulous.json File
- Add your First Meticulous Test
- Fixing a Broken Visual Regression Test
- Conclusion
- Meticulous
Visual regression testing in simple words is a mechanism to make sure that the changes made to a system, primarily code, do not adversely affect the visible appearance of the application’s user interface. In this post, you will learn how to do visual regression testing for a React.js application, let’s get going!
What is Visual Regression Testing
Regression means a return to a former or less developed state. Regression testing means to ensure that system changes in one part do not unintentionally break something that was working in another part of the system.
Along the same lines, visual regression testing means to verify that changes do not negatively impact the visual appearance of the existing user interface of the application. For example, an engineer changed a button used for the Login form but as the button was used for “Saving” things as well unintentionally broke the saving part.
Why do Visual Regression Testing
Similar to any other form of testing like unit testing or smoke testing, setting up visual regression testing helps you to catch any visual discrepancies much earlier in the development phase. Visual regression testing is usually done by comparing two screenshots and figuring out which elements look different than the previous screenshot and if it is intended.
A simple example can be, a profile picture that is expected to be 100x100 px on the desktop web browser, but due to an unintended change in CSS now the profile picture is 150x150 px. These kinds of visual issues can be caught and solved before it reaches the end users which is the main reason to use visual regression testing.
Prerequisites
In the next section, you will start dabbling with some code. So to do that, some of the prerequisites are as follows:
-
Some experience with Node.js, React.js, and NPM CLI is expected.
-
You should have a working knowledge of Git and GitHub including cloning a repository, creating and switching branches in Git, and a general understanding of pull requests.
-
A basic experience with React.js is expected to understand the React.js code snippet.
Given you are aware of the skills needed to proceed, next the React application used to demonstrate visual regression testing in a React.js app will be discussed.
Example App
To show how to do visual regression testing on a React.js app, you will use the Name to Nationality guessing application. You can know more about how it is built in the Jest SpyOn post. It is a simple app where given the first name it tries to guess the Nationality percent of that name using the Nationalize.io API. You can view a quick working example for the name chris
below:
It is also hosted on Netlify if you want to play around with the app and the code is also available on GitHub. With that context, next, you will learn how to add visual regression tests on a React.js app with Meticulous.
Visual Regression Testing with Meticulous
Before you proceed further, you should clone the Name Nationality application from GitHub and make sure you are on the apply-meticulous
branch. Please also make sure that you have run npm install
to install the packages. You can also execute npm start
and check the application is running locally on port 3000.
Register on Meticulous
As the next step, register on Meticulous, use your Gmail, GitHub or Email to do so on the screen below:
That should be easy to do, then you will be presented with the onboarding. Select an organization name:
In the above picture, it is Geshan-Meticulous
but you can choose it to be what you like as long as it is available. The consequent step in onboarding is to choose a Project name:
The name for the project is chosen as Name-Nationality
as the example React app does that mapping with percent of nationality guessed for the name. After that, you will see the project page similar to the one below:
This is where the fun starts, in the next section you will install meticulous and create your first meticulous test. Get those fingers ready to copy paste and type some commands/code.
Install Meticulous and Bootstrap meticulous.json
File
To install Meticulous on the Name-Nationality
project you have cloned run the “Install Meticulous” command visible on the Project page, which will look like the below:
npm install --save-dev @alwaysmeticulous/cli @alwaysmeticulous/replayer @alwaysmeticulous/record
This example is of the npm
version as the Name nationality app uses NPM, not yarn. The above command will install 3 Meticulous packages the CLI, replayer, and record as dev dependencies. These will be used later in this guide.
Next, you will create a meticulous.json file. This file will list all your test cases with their title, sessionId, and baseRepayId. You can create an empty meticulous.js
file with the following command:
npx meticulous bootstrap
It will create an empty meticulous.json file and also show an output like the below:
Setting up meticulous.json...
Setting up test:meticulous script...
The bootstrap command also adds a command to the package.json file which looks like this:
"test:meticulous": "meticulous run-all-tests --headless --parallelize --deflake"
It can run with npm run test:meticulous
later to run all the Meticulous tests in headless mode.
Add your First Meticulous Test
To add your first meticulous test first make sure the Name Nationality app is running locally by executing npm start
and making sure you see the form with the name field at http://localhost:3000
on your browser of choice.
After you are confirmed that the Name Nationality app is working on your local, create a happy path test by running:
npx meticulous create-test --apiToken=”<get-your-token-from-meticulous-project-page>”
You will find your API token on the meticulous project page, search for “Record a” on that page if you don’t find it. Copy that and replace it in the above command. The above command will open up an instrumented browser (Chromium) controlled by Meticulous. Next, type in https://localhost:3000
and in the name put in chris then
click ‘Get Nationalities”, once you see the flags, close the browser.
Consequently, you will be asked 3 questions, answer them as follows:
? Does the end state screenshot match your expectation? Yes
? Would you like to save this as a test to meticulous.json? Yes
? Test name: Flags show up correctly
The whole output of this command with the browser opening and closing will look like the below:
Creating a new Meticulous test
Step 1: record a new test
Preparing recording...
Opening browser...
Browser ready
Recording session: https://app.meticulous.ai/projects/Geshan-Meticulous/Name-Nationality/sessions/2022-09-03T12:12:39.095Z_vuQfgdDIxnWihFOLHQpg9
Step 2: validating the new test...
Starting simulation...
Replay done!
Simulation time: 8.508 seconds
Sending simulation results to Meticulous
Simulation artifacts successfully sent to Meticulous
=======
View simulation at: https://app.meticulous.ai/projects/Geshan-Meticulous/Name-Nationality/simulations/iiqHJzO-S1XFg2frx40Cu
=======
? Does the end state screenshot match your expectation? Yes
? Would you like to save this as a test to meticulous.json? Yes
? Test name: Flags show up correctly
Test saved to meticulous.json.
After this, if you open up the meticulous.json
file you will see that the test has been successfully saved with the needed values. If you also visit the “Simulations” and “Session” links on the Meticulous’ project page you will see that these have been sent to Meticulous too as seen below:
At this point, your first meticulous test is saved to the meticulous.json
file and also sent to the Meticulous servers as seen on the project’s page. The meticulous.json
file looks like the following with the first test added:
{
"testCases": [
{
"title": "Flags show up correctly",
"sessionId": "2022-09-03T12:12:39.095Z_vuQfgdDIxnWihFOLHQpg9",
"baseReplayId": "iiqHJzO-S1XFg2frx40Cu"
}
]
}
Now to run the test again, without creating a new now, you can execute:
npx meticulous run-all-tests --apiToken="<get-your-token-from-meticulous-project-page>" --headless
The above command can be found in the “Commands” tab on the project’s page by searching for “Run all tests”. Please make sure the API token is correct. It will yield output like:
Test run URL: https://app.meticulous.ai/projects/Geshan-Meticulous/Name-Nationality/test-runs/Ex6wxaJiv0z3T1cDpfINC
Starting simulation...
Replay done!
Simulation time: 9.173 seconds
Sending simulation results to Meticulous
Simulation artifacts successfully sent to Meticulous
=======
View simulation at: https://app.meticulous.ai/projects/Geshan-Meticulous/Name-Nationality/simulations/ptvPReRd1Cq2Yq5lnPWz0
=======
Diffing final state screenshot against replay iiqHJzO-S1XFg2frx40Cu
0% pixel mismatch (threshold is 1%) => PASS
View screenshot diff at https://app.meticulous.ai/projects/Geshan-Meticulous/Name-Nationality/simulations/ptvPReRd1Cq2Yq5lnPWz0/diff/iiqHJzO-S1XFg2frx40Cu
Results
=======
URL: https://app.meticulous.ai/projects/Geshan-Meticulous/Name-Nationality/test-runs/Ex6wxaJiv0z3T1cDpfINC
=======
Flags show up correctly => pass
Hurray! Your first Meticulous test has passed and the good thing is till this point you have not written a single line of code. As seen above the was a 0% pixel mismatch
between the saved screenshot of the simulation and your local code. As mentioned by Alister Scott, “Never Trust an Automated Test You Haven’t Seen Fail”. Next, you will make the test fail by making a change. The change is to make the flags smaller. You will make the flags 50px wide.
To do this, you will add width: "50px"
on line 58 of src/App.js
file which it will look as follows:
<div className="nationalities">
{Array.isArray(nationalities) && nationalities.map(
nationality => {
const flagUrl = `https://flagcdn.com/w160/${nationality.country_id.toLowerCase()}.jpg`;
const altText = `${nationality.country_id} flag`;
return <div key={nationality.country_id}><h3>{nationality.country_id} - {(nationality.probability * 100).toFixed(2)}%</h3> <img src={flagUrl} alt={altText} style={{
border: "1px solid black",
width: "50px"
}} /></div>
}
)}
</div>
After you save the file when you rerun the test with the npx meticulous run-all-tests
command in headless mode providing the API key like:
npx meticulous run-all-tests --apiToken="<get-your-token-from-meticulous-project-page>" --headless --appUrl="http://localhost:3000/"
It will fail with the following output:
Test run URL: https://app.meticulous.ai/projects/Geshan-Meticulous/Name-Nationality/test-runs/oROsG3mdGSyuS69hWYt8D
Starting simulation...
Replay done!
Simulation time: 10.085 seconds
Sending simulation results to Meticulous
Simulation artifacts successfully sent to Meticulous
=======
View simulation at: https://app.meticulous.ai/projects/Geshan-Meticulous/Name-Nationality/simulations/Izp3KA5jCPUD8gb05gflV
=======
Diffing final state screenshot against replay iiqHJzO-S1XFg2frx40Cu
4% pixel mismatch (threshold is 1%) => FAIL!
View screenshot diff at https://app.meticulous.ai/projects/Geshan-Meticulous/Name-Nationality/simulations/Izp3KA5jCPUD8gb05gflV/diff/iiqHJzO-S1XFg2frx40Cu
Screenshots do not match!
Results
=======
URL: https://app.meticulous.ai/projects/Geshan-Meticulous/Name-Nationality/test-runs/oROsG3mdGSyuS69hWYt8D
=======
Flags show up correctly => fail
Here the main message is 4% pixel mismatch (threshold is 1%) => FAIL!
here by the test has failed. If you check the app locally on port 300 after the 50 px
style change it will look as follows:
Now it has smaller flags than earlier. So how can this be fixed?
Fixing a Broken Visual Regression Test
To fix the tests, you will not need to write any more code. The test can be fixed easily by running the following command:
npx meticulous update-tests --apiToken="<get-your-token-from-meticulous-project-page>" --testRunId="<test-run-id-from-previous-failing-run>" --acceptAll
The command updates the tests and accepts the new replay result as passed. The testRunId
is jZK8Fl_yWjZ-G5WxOJWvU
in my case which was taken from the above shown previous run.
It will also replace the “baseReplayId” in the meticulous.json
file with the correct simulation id from the new test run. In my case, after running the update-tests
with --acceptAll
, it looked like the below:
{
"testCases": [
{
"title": "Flags show up correctly",
"sessionId": "2022-09-03T12:12:39.095Z_vuQfgdDIxnWihFOLHQpg9",
"baseReplayId": "LYYwD2jJILt-GwnSy98vi"
}
]
}
At this juncture, you can run the tests again to see if it will pass now. To run the tests again, you can execute:
npx meticulous run-all-tests --apiToken="<get-your-token-from-meticulous-project-page>" --headless --appUrl="http://localhost:3000/"
Because the test has been updated, it should pass showing an output like the below:
Test run URL: https://app.meticulous.ai/projects/Geshan-Meticulous/Name-Nationality/test-runs/KiKq-ZDtEoCTjRE5shDOj
Starting simulation...
Replay done!
Simulation time: 9.579 seconds
Sending simulation results to Meticulous
Simulation artifacts successfully sent to Meticulous
=======
View simulation at: https://app.meticulous.ai/projects/Geshan-Meticulous/Name-Nationality/simulations/e7aC3WC2DICnORgFX3U4N
=======
Diffing final state screenshot against replay LYYwD2jJILt-GwnSy98vi
0% pixel mismatch (threshold is 1%) => PASS
View screenshot diff at https://app.meticulous.ai/projects/Geshan-Meticulous/Name-Nationality/simulations/e7aC3WC2DICnORgFX3U4N/diff/LYYwD2jJILt-GwnSy98vi
Results
=======
URL: https://app.meticulous.ai/projects/Geshan-Meticulous/Name-Nationality/test-runs/KiKq-ZDtEoCTjRE5shDOj
=======
Flags show up correctly => pass
Great! Your failing test is passing and now it can be pushed to your favorite source control management software like GitHub.
Conclusion
In this step-by-step guide, you first learn about what is visual regression testing. After that, you looked at adding a meticulous test to an existing React.js application. Then, you changed the code which failed the added test.
Consequently, you updated the tests to match the code changes that passed the first test. You can look at the steps followed to do it all in this branch comparison on GitHub.
The main takeaway from this post is, that it is a great idea to add visual regression testing on any app React.js included. With Meticulous, changing the test when the code is updated is so easy with the provided commands in the easy-to-use Meticulous CLI.
Meticulous
Meticulous is a tool for software engineers to catch visual regressions in web applications without writing or maintaining UI tests.
Inject the Meticulous snippet onto production or staging and dev environments. This snippet records user sessions by collecting clickstream and network data. When you post a pull request, Meticulous selects a subset of recorded sessions which are relevant and simulates these against the frontend of your application. Meticulous takes screenshots at key points and detects any visual differences. It posts those diffs in a comment for you to inspect in a few seconds. Meticulous automatically updates the baseline images after you merge your PR. This eliminates the setup and maintenance burden of UI testing.
Meticulous isolates the frontend code by mocking out all network calls, using the previously recorded network responses. This means Meticulous never causes side effects and you don’t need a staging environment.
Learn more here.
Authored by Geshan Manandhar