How to Choose a CRO Agency: Signs of a Good One

How to spot a good conversion optimization agency, when hiring one pays off and what to ask on the first call. With red flags and a sample calculation.

Jason Spizert

Jason Spizert

Updated on October 21, 2026

Published 6 min read

The short answer

You can spot a good conversion optimization agency by the order in which it works: it wants to see your data first, builds a priority list from it, designs and implements the changes itself and measures properly whether they work, ideally by revenue per visitor. Anyone who promises you fixed percentages before the analysis or starts with button colors is guessing. An agency generally makes sense from about €150,000 in revenue per month; below that, targeted changes usually bring more than a testing program.

Why so many people have had bad experiences with CRO agencies

“When it comes to agencies, we're a burned market.”
Founder, intro call 2026 (translated from German)

I hear that sentence often, and it's fair. Many stores have already put money into conversion optimization and afterwards didn't know what it brought.

“At [our store] you ran various ‘visual’ tests. But that doesn't help us right now.”
Store owner, 2026 (translated from German)

That's the typical pattern: what gets tested is what's easy to test, not what costs the most revenue. A test without reasoning isn't a test; it's a coin toss with statistics.

What's easy to test is rarely what costs revenue
easy · low revenueOften tested firstButton color, homepage hero image, button copy
easy · high revenueWhere a good agency startsShow shipping costs earlier, reviews next to the buy box, size guidance
complex · low revenueRarely worth itElaborate rebuilds in spots with little revenue
complex · high revenueBig, but plannableOffer, product page, positioning: needs data and time
Examples, not measurements. Where an idea lands in the grid is decided by your store's data.

Then there's cold outreach: almost every store owner gets messages saying their store has “room for improvement”.

“But I'm skeptical about trusting them with my money. Has anyone here had experience with agencies that actually deliver what they promise?”
Store owner, Shopify Community (translated from German)

Remember

Bad CRO agencies test what's easy. Good ones test what costs revenue.

Do you even need an agency?

As a rule, an agency pays off from about €150,000 in revenue per month. From there, even a small increase in revenue per visitor brings noticeably more money, and there are enough orders to measure changes.

How fast a test delivers depends on three things
01Visitors per monthAt a 1% conversion rate, detecting a 20% difference takes about 79,000 visits. At 5,000 visits per month, that's almost 16 months. The calculation is in A/B testing in your online store.
02Traffic sourceIf you work with direct-response ads, you test offer, landing page and message together with the ads, for example with Meta Ads. If traffic is organic, the lever is more on product pages and trust.
03Breadth of the catalogIf revenue depends on a few bestsellers, you can test faster than with a broad catalog.
How long an A/B test takes to reach a result
5,000 visits per month
16 months
20,000 visits per month
4 months
80,000 visits per month
1 month
Example: 1% conversion rate, 20% difference between variants, about 79,000 visits to a reliable result, values rounded. With little traffic, a single test takes more than a year.

Below that, you usually don't need a testing program, but someone who finds the three biggest problem areas and changes them in a targeted way. And if there's no demand for the product, no optimization helps; then the offer and positioning come first. How to tell is covered in Shopify traffic but no sales.

An agency pays off when your traffic delivers enough orders to measure changes.

Three types of providers, and who they fit

Pure CRO agencyClassic Shopify agencyBoth in one place
StrengthTesting programs, statistics, analysisDesign, development, store buildsAnalysis, design, implementation, measurement
Typical clientStores with eight- to nine-figure annual revenue and their own design and development teamStores before launch or relaunchGrowing stores without a large team of their own
GapOften doesn't implement itself, doesn't design to fit the brandOften doesn't measure after building whether more is soldHas to genuinely do both, not just claim it

Pure CRO agencies are strong when you already have people in-house who implement the results. Without them, you end up with a report full of recommendations. Many also just keep testing while fundamental problems stay untouched: a design that doesn't fit the brand, or a product page that leaves the most important question open.

For most stores, the third column fits, if the agency can genuinely do both. The best agency is the one whose gap you can fill yourself. Or one that has none.

How to spot a good agency

Six traits, one order
1Data
2Break down the numbers
3Priority list
4Design and build
5Measure revenue
6Document
Each step builds on the one before. If you start at step 4, you design by gut feeling.

1. It starts with data, not ideas

Everyone has ideas for a store, a hundred of them in an hour. The work is finding the three that bring the most. For that, an agency needs three sources:

Three sources before the first idea
MarketWhat are competitors doing with ads, offers and pages?
StoreAnalytics, heatmaps and session recordings: where do visitors drop off, on which device?
CustomersSurveys, reviews, messages to customer service: why do they buy, and what holds them back?
AI image: a glowing heatmap, a line chart and a group of speech bubbles whose lines converge in a bright point
Heatmap, numbers and customer feedback converge. Only from that does a decision emerge. (Illustration, AI-generated)

2. It breaks down the numbers before it changes the store

When the conversion rate drops, it isn't always the store. Often a new ad channel is added whose visitors buy less often and pull the average down. A good agency therefore splits by channel and device first.

The store hasn't changed, the average has
Existing channel10,000 visits × 2%= 200 purchases
New ad channel5,000 visits × 0.8%= 40 purchases
Whole store240 purchases ÷ 15,000 visits1.6%instead of 2% conversion rate
Example numbers. Neither channel got worse; the new one just buys less often. That's why you split by channel and device first, then work on the store.

Why every dashboard shows a different number in the process is covered in Four dashboards, four different revenue numbers.

3. It builds a priority list

Every idea is scored beforehand: supporting data, revenue affected, reasoning, effort, measurability. At the top is what affects a lot of revenue and can be done with reasonable effort. Work starts at the top.

From an idea list to a priority listIdeas, unsorted
·Change the button color2/10
·New hero image on the homepage4/10
·Show shipping costs earlier9/10
·Size guidance on the product page7/10
·Reviews next to the buy box8/10
Example. Every idea is scored beforehand: is there supporting data, how much revenue runs through that spot, how big is the effort, can the result be measured?

4. It designs and implements itself, on brand

An insight is only worth something once it's live in the store, and on brand rather than a foreign object from a template kit. Ask who does the design at the agency and who develops in the Shopify theme.

5. It measures properly, and it measures revenue

A test runs until the result holds, not until the number looks good. Otherwise false winners get implemented that actually cost revenue. What gets measured is revenue per visitor (RPU), not just the conversion rate: more orders with smaller carts can end up as less revenue.

Same conversion rate, different revenue per visitor
Variant A2% conversion rate · cart €60.00
€1.20
Variant B2% conversion rate · cart €72.00
€1.44
Example. Revenue per visitor = conversion rate × average cart value. Both variants sell equally often; B brings 20% more revenue per visit. If you only measure the conversion rate, you see no difference.

6. It documents what it learns

Every test tells you something about your customers, even one that loses. A good agency collects this in one place you can access and feeds it back into ads, copy and email marketing.

You don't recognize a good agency by its ideas, but by the order in which it works.

What to ask on the first call

QuestionA good answer sounds like this
Where do you start?With access to your data and an analysis, not with a list of ideas.
How do you decide what to test first?With a scored list: supporting data, revenue affected, effort, measurability.
Do tests even pay off at my traffic?An honest calculation, even if the answer is “not yet”.
Who designs and who implements?Specific people on their own team, not “a partner”.
How do you measure success?By revenue per visitor, not just the conversion rate.
How long does a test run?Until the result is statistically sound. The duration is estimated beforehand.
What do I get to see?Running tests, results and insights in one place, at any time.
Who is my point of contact?One dedicated person who knows your store.
Jason Spizert in conversation at a table, pointing at a tablet
With us, Jason runs the first call himself.

The last question sounds trivial. It isn't.

“Then you have a weekly call where some employee who just joined […] wants to tell you how you should do it.”
Founder, intro call 2026 (translated from German)

Red flags

Five red flags in a proposal
01Fixed percentages before the analysis“We'll increase your conversion by 30%.” How would they know?
02The first test is a colorCosmetic tests without reasoning are the sign of a program without a plan.
03No statement on test durationIf they don't say how long a test runs at your traffic, they haven't calculated it.
04Reports with conversion rate onlyWithout revenue per visitor, you don't know whether more money is coming in.
05No clear answer about the first weeksAsk what the start looks like and how you'll both recognize whether the collaboration works.
“I just don't want to sign a long-term contract up front. My experience with the last German agency is still too fresh for that.”
Founder, 2026 (translated from German)

Anyone who promises you a number before the analysis is selling you hope.

What has to pay off

Work backwards before you compare proposals. The calculation shows what an agency can cost, and why conversion optimization often doesn't pay off yet for small stores.

Working backwards: 5% more revenue per visitor at the same traffic
Store with €150,000 per month€150,000 × 5%= €7,500 per month× 12 months= €90,000 per year+€7,500more revenue per month
Store with €20,000 per month€20,000 × 5%= €1,000 per month× 12 months= €12,000 per year+€1,000more revenue per month
Sample calculation, not a promised result. How much of it stays as profit depends on your margin. The calculation shows what an agency can be allowed to cost.
AI image: two identical streams of glowing dots, ending in a higher and a lower bar
The same number of visitors, the same number of purchases. Still, one track brings in more revenue because the carts are bigger. (Illustration, AI-generated)

How we work

We're a Shopify agency focused on buying psychology. Analysis, design, implementation in the theme and measurement come from one place, and we test with our own A/B testing tool. In the test cockpit, you can see at any time what's running, what won and what didn't.

NextPercent test cockpit (German interface) with additional revenue, visitors, conversion rate, running tests and next steps
Our test cockpit, shown here with sample data: running tests, winners and insights in one place.

If tests don't pay off at your traffic yet, we tell you and make targeted changes with a before-and-after comparison. And if you have a large in-house team and are only looking for a pure testing program, a specialized CRO agency may be a better fit.

How we work as a conversion optimization agency is on our service page. Where your store stands today is put in context in Your conversion rate is 1%. Is that bad?, and which spots are costing you buyers is what we look at together on the free call below.

Frequently asked questions

One that understands your store and your market before it tests: it starts with data, builds a prioritized list, designs and implements changes itself and measures by revenue per visitor whether they work. A pure CRO agency fits stores with a large in-house team; for most growing stores, a provider that handles analysis and implementation in one place fits better.

As a rule, from about €150,000 in revenue per month. What matters is whether there are enough visits and orders to measure changes. Below that, targeted changes with a before-and-after comparison usually bring more than a testing program.

The conversion rate shows how many visits end in an order. Revenue per visitor (RPU) shows how much revenue each visit brings on average, so conversion rate times average cart value. It's the better success metric because it also captures changes in cart value.

By fixed percentage promises before the analysis, cosmetic tests without reasoning, no statement on test duration and reports that only show the conversion rate.

A tool only measures what you test. It doesn't decide which change has the biggest lever, how it's designed and built, or when a result holds. If your team can do that, you only need the tool.

The analysis delivers findings in the first few weeks. Whether a change works shows after a few weeks to months, depending on traffic. A good agency estimates the test duration for your store beforehand.

See all insights

  • Reply within 24 hours
  • No obligation, no sales pitch

Request an intro call

How can we help?*
What budget do you have in mind?*

By submitting, you agree to the processing of your data as described in our privacy policy.

Rather pick a time directly?

Book a call directly
Patricia from the NextPercent team working on a laptop in an armchair

Jennifer Rosbach

Beauty brand (nagelati.de)

“As a young brand, it mattered to us to work with a partner who doesn't just execute, but really understands how ecommerce brands grow. That's exactly what we found. Clear communication, honest assessments and a good feel for conversion and brand presence.”