Shopify A/B testing is the cleanest way to stop arguing about design opinions and start counting money. Instead of shipping a change because someone “feels” it will convert better, you show two versions to real traffic, measure the difference, and keep the winner.
Here is a quick illustration with example numbers: a store doing 50,000 sessions a month at 1.8% conversion and $85 AOV. A relative lift of 8% in conversion rate would add roughly $6,120 in monthly revenue if it holds. Whether a store can actually prove a lift like that depends on its traffic, which we cover below.
An A/B test (also called a split test) shows version A (the control) to one group of visitors and version B (the variation) to another. You measure the same primary metric on both groups under the same conditions and decide which version produces more revenue. You will also see it called Shopify AB testing or Shopify split testing. The mechanics are the same: the store divides visitors at random and the data picks the winner.
On Shopify this can mean:
The question to answer is which version makes more money per visitor.
Monthly revenue lift ≈ Sessions × Current conversion rate × Relative lift × AOV
Here is an illustration for three common store sizes (illustrative numbers; your results will differ):
| Monthly sessions | Current CR | AOV | +5% relative lift | +10% relative lift |
| 20,000 | 1.5% | $70 | +$1,050 | +$2,100 |
| 50,000 | 1.8% | $85 | +$3,825 | +$7,650 |
| 150,000 | 2.0% | $95 | +$14,250 | +$28,500 |
The columns show what a lift is worth if it is real. Confirming it is a separate question: at these volumes, only the 10% lift for the 150,000-session store can be proven in about a month, as the next section shows.
Traffic decides what you can prove. A rough guide: a test needs about 16 × p × (1 − p) ÷ d² visitors per variant, where p is your conversion rate and d is the absolute lift you want to detect (80% power, 95% confidence). At a 2% conversion rate, confirming a 20% relative lift takes about 19,600 visitors per variant, roughly 39,000 in total. A 10% lift takes four times that, around 157,000.
So if the page you want to test gets fewer than roughly 40,000 sessions a month, only big changes can be confirmed within a month, and small ones may never reach a clear answer. In that case, testing is the wrong tool for now.
What to do instead:
Shopify Rollouts (introduced with the Winter ’26 Edition) is the native way to run experiments. You find it under Markets > Rollouts in your Shopify admin, and Shopify’s Rollouts documentation covers the details. A rollout comes in three types: a Launch merges changes into your store permanently, an Event publishes them for a set period and then rolls them back, and an Experiment compares a treatment against a control. A Shopify Rollouts A/B test is the Experiment type.
Key capabilities:
Rollouts are available on the Basic plan or higher. Experiments need the Grow plan or higher.
Rollouts covers many storefront and checkout-configuration tests, but it has limits that matter for a testing program:
Third-party testing apps fill many of these gaps. Depending on the tool, they add revenue and RPV reporting, device and segment targeting, element-level tests and price tests.
When the change lives in theme code or needs more control than Rollouts offers, we build custom test variants in your theme code or use a lightweight testing layer.
If you are looking for A/B test ideas for Shopify, start with these four. They cover common friction points: the product page, the delivery promise, the offer and the cart.
Mobile often makes up most of a store’s traffic, so check your own analytics first. Test a tighter above-the-fold experience: primary image, clear title and price, prominent Add to Cart, and minimal competing elements. This layout change fits a Rollouts Experiment when you make it in the theme editor, but results can’t be filtered by device, so judging the mobile effect takes a testing tool or custom tracking. Our guide to UX fixes worth testing has more ideas for this page.
Free shipping thresholds and estimated delivery dates can affect trust and AOV. Test different threshold amounts, the wording of the progress bar, and whether the messaging appears in the cart drawer, on the product page, or both.
Offer a simple “Buy 2, save 10%” or a product + complementary item bundle and measure both conversion rate and revenue per visitor. Odyssey, a functional energy drink brand, saw subscription conversion grow 41% and average order value 18% after we built its Shopify store with subscriptions and bundles. That result came from a store build without an A/B test, so the same ideas are good candidates for a formal test on your store.
One Rollouts caveat applies here. If an Experiment changes prices, discounts or catalogs, the change shows only in your online store, while ads, listings and product feeds keep the old details. Google and Meta can flag or delist offers that don’t match, so pause ads for the affected products during the test.
Some stores convert better with a full cart page; others prefer the speed of a drawer. Test both experiences with the same upsells and shipping messaging to see which path produces higher revenue per visitor.
Good hypothesis format:
“If we [change], then [metric] will improve because [reason]. We will know this is true if [success criterion].”
Example:
“If we move the Add to Cart button higher and reduce above-the-fold elements on mobile product pages, then add-to-cart rate and revenue per visitor will increase because fewer shoppers bounce before seeing the primary CTA. Success = statistically significant lift in RPV with no drop in average order value.”
A hypothesis you are willing to lose forces clearer thinking and cleaner test design. A backlog from a conversion audit gives you hypotheses grounded in your own data.
How long should an A/B test run? Long enough to reach the sample size you calculated before launch, and never less than one full week, so weekday and weekend shoppers are both in the data. Most tests take 1–3 weeks on a busy page and longer on a quiet one.
Conversion rate alone can mislead. A variation that converts more people at a lower average order value can lose money. Always look at revenue per visitor (RPV) or revenue per session. A Rollouts Experiment doesn’t report revenue or AOV, so RPV needs a testing app or custom tracking alongside it.
KXKS, a sports and travel bags brand, saw revenue per visitor grow 27% after we built custom product logic into its Shopify store. That result also came without a test, which is why a change like this is worth testing before you rely on it.
Speed and SEO. Rollouts assign each visitor to a group before the page loads, so there is no flicker. For organic traffic, Shopify’s guidance is to keep product URLs stable, avoid temporary page errors on indexed products, and use a rollout that reaches all visitors for important availability changes. Browser-based scripts from third-party tools can add weight, so check Core Web Vitals before and after.
A practical ongoing program looks like this, and our Growth package follows the same steps:
The report is the piece teams often skip. Owners need a clear answer: “This change is projected to add $X per month. Here is what we recommend testing next.”
A useful business report for each test includes:
The first 30 days can look like this. In week one we confirm baselines for conversion rate, AOV and RPV by device and do the sample-size math. Then we build and QA the variants, launching the product page and shipping tests first, with bundles and the cart after. Tests on different parts of the funnel can run side by side, and Rollouts flags conflicts between overlapping rollouts before you save.
If you do not yet have a prioritized list of hypotheses, begin with a CRO audit. The audit produces the first set of high-confidence test ideas. From there the monthly Growth package turns those ideas into a steady stream of measured experiments (up to 4 tests a month, each with a business report).
Yes. Shopify Rollouts, found under Markets > Rollouts in your admin, includes an Experiment type that compares a treatment against a control. Experiments require the Grow plan or higher, while launches and scheduled events are available from Basic.
About 40,000 sessions a month on the page you are testing lets you confirm a large change, around a 20% relative lift, in roughly a month at a 2% conversion rate. Smaller lifts need far more traffic: a 10% lift takes about four times as many visitors. Lower-traffic stores should focus on qualitative research and high-confidence fixes first.
Until it reaches the sample size you calculated before launch, and for at least one full week so weekday and weekend behavior are both covered. A common range is 1–3 weeks on a busy page, and quieter pages take longer. Do not stop early because one version looks ahead.
Not by itself. Rollouts assign each visitor to a group before the page loads, so visitors don’t see a flicker between versions. Shopify also advises keeping product URLs stable and avoiding temporary page errors to protect indexed pages. Third-party scripts that run in the browser can add weight, so measure Core Web Vitals before and after any test.
With native Shopify Rollouts you can test theme changes, a replacement checkout and accounts configuration, discounts, and catalogs. Common starting points are the mobile product page layout, shipping threshold messaging, bundles or quantity breaks, and the cart drawer versus a cart page.
Yes, based on Shopify’s rollout documentation: Experiments require the Grow plan or higher, and Shopify Plus is not listed as a requirement. A checkout test replaces your checkout and accounts configuration for part of your visitors and reports checkout conversion rate. Checkouts on headless and custom storefronts can’t be tested with rollouts.
Black Friday 2026 falls on November 27, and most Shopify stores stop changing their theme about a week before it. That leaves a short window for conversion fixes, and the stores that use it well tend to fix the same few pages: the homepage, collection pages, the product page and the cart. The stakes are […]
A BigCommerce to Shopify migration isn’t a data export and import. It touches your product catalog, your custom templates, your app stack, and every search ranking you’ve built up over the years. Brands that treat it as a simple platform swap usually rebuild half the store anyway, just after launch instead of before it. This […]
If you have priced a Magento to Shopify Plus migration and the quotes barely resembled each other, nobody is necessarily overcharging you. “Migration” covers two different jobs under one name. A 300-SKU catalog on a stock checkout is one project. A 12,000-SKU B2B catalog with three custom extensions and contract pricing built into checkout is […]
