How to Choose a CRO Agency: Signs of a Good One
How to spot a good conversion optimization agency, when hiring one pays off and what to ask on the first call. With red flags and a sample calculation.

Jason Spizert
Updated on October 21, 2026
Published 6 min read
The short answer
You can spot a good conversion optimization agency by the order in which it works: it wants to see your data first, builds a priority list from it, designs and implements the changes itself and measures properly whether they work, ideally by revenue per visitor. Anyone who promises you fixed percentages before the analysis or starts with button colors is guessing. An agency generally makes sense from about €150,000 in revenue per month; below that, targeted changes usually bring more than a testing program.
Why so many people have had bad experiences with CRO agencies
“When it comes to agencies, we're a burned market.”Founder, intro call 2026 (translated from German)
I hear that sentence often, and it's fair. Many stores have already put money into conversion optimization and afterwards didn't know what it brought.
“At [our store] you ran various ‘visual’ tests. But that doesn't help us right now.”Store owner, 2026 (translated from German)
That's the typical pattern: what gets tested is what's easy to test, not what costs the most revenue. A test without reasoning isn't a test; it's a coin toss with statistics.
Then there's cold outreach: almost every store owner gets messages saying their store has “room for improvement”.
“But I'm skeptical about trusting them with my money. Has anyone here had experience with agencies that actually deliver what they promise?”Store owner, Shopify Community (translated from German)
Remember
Bad CRO agencies test what's easy. Good ones test what costs revenue.
Do you even need an agency?
As a rule, an agency pays off from about €150,000 in revenue per month. From there, even a small increase in revenue per visitor brings noticeably more money, and there are enough orders to measure changes.
Below that, you usually don't need a testing program, but someone who finds the three biggest problem areas and changes them in a targeted way. And if there's no demand for the product, no optimization helps; then the offer and positioning come first. How to tell is covered in Shopify traffic but no sales.
An agency pays off when your traffic delivers enough orders to measure changes.
Three types of providers, and who they fit
| Pure CRO agency | Classic Shopify agency | Both in one place | |
|---|---|---|---|
| Strength | Testing programs, statistics, analysis | Design, development, store builds | Analysis, design, implementation, measurement |
| Typical client | Stores with eight- to nine-figure annual revenue and their own design and development team | Stores before launch or relaunch | Growing stores without a large team of their own |
| Gap | Often doesn't implement itself, doesn't design to fit the brand | Often doesn't measure after building whether more is sold | Has to genuinely do both, not just claim it |
Pure CRO agencies are strong when you already have people in-house who implement the results. Without them, you end up with a report full of recommendations. Many also just keep testing while fundamental problems stay untouched: a design that doesn't fit the brand, or a product page that leaves the most important question open.
For most stores, the third column fits, if the agency can genuinely do both. The best agency is the one whose gap you can fill yourself. Or one that has none.
How to spot a good agency
1. It starts with data, not ideas
Everyone has ideas for a store, a hundred of them in an hour. The work is finding the three that bring the most. For that, an agency needs three sources:

2. It breaks down the numbers before it changes the store
When the conversion rate drops, it isn't always the store. Often a new ad channel is added whose visitors buy less often and pull the average down. A good agency therefore splits by channel and device first.
Why every dashboard shows a different number in the process is covered in Four dashboards, four different revenue numbers.
3. It builds a priority list
Every idea is scored beforehand: supporting data, revenue affected, reasoning, effort, measurability. At the top is what affects a lot of revenue and can be done with reasonable effort. Work starts at the top.
4. It designs and implements itself, on brand
An insight is only worth something once it's live in the store, and on brand rather than a foreign object from a template kit. Ask who does the design at the agency and who develops in the Shopify theme.
5. It measures properly, and it measures revenue
A test runs until the result holds, not until the number looks good. Otherwise false winners get implemented that actually cost revenue. What gets measured is revenue per visitor (RPU), not just the conversion rate: more orders with smaller carts can end up as less revenue.
6. It documents what it learns
Every test tells you something about your customers, even one that loses. A good agency collects this in one place you can access and feeds it back into ads, copy and email marketing.
You don't recognize a good agency by its ideas, but by the order in which it works.
What to ask on the first call
| Question | A good answer sounds like this |
|---|---|
| Where do you start? | With access to your data and an analysis, not with a list of ideas. |
| How do you decide what to test first? | With a scored list: supporting data, revenue affected, effort, measurability. |
| Do tests even pay off at my traffic? | An honest calculation, even if the answer is “not yet”. |
| Who designs and who implements? | Specific people on their own team, not “a partner”. |
| How do you measure success? | By revenue per visitor, not just the conversion rate. |
| How long does a test run? | Until the result is statistically sound. The duration is estimated beforehand. |
| What do I get to see? | Running tests, results and insights in one place, at any time. |
| Who is my point of contact? | One dedicated person who knows your store. |

The last question sounds trivial. It isn't.
“Then you have a weekly call where some employee who just joined […] wants to tell you how you should do it.”Founder, intro call 2026 (translated from German)
Red flags
“I just don't want to sign a long-term contract up front. My experience with the last German agency is still too fresh for that.”Founder, 2026 (translated from German)
Anyone who promises you a number before the analysis is selling you hope.
What has to pay off
Work backwards before you compare proposals. The calculation shows what an agency can cost, and why conversion optimization often doesn't pay off yet for small stores.

How we work
We're a Shopify agency focused on buying psychology. Analysis, design, implementation in the theme and measurement come from one place, and we test with our own A/B testing tool. In the test cockpit, you can see at any time what's running, what won and what didn't.

If tests don't pay off at your traffic yet, we tell you and make targeted changes with a before-and-after comparison. And if you have a large in-house team and are only looking for a pure testing program, a specialized CRO agency may be a better fit.
How we work as a conversion optimization agency is on our service page. Where your store stands today is put in context in Your conversion rate is 1%. Is that bad?, and which spots are costing you buyers is what we look at together on the free call below.
Frequently asked questions
One that understands your store and your market before it tests: it starts with data, builds a prioritized list, designs and implements changes itself and measures by revenue per visitor whether they work. A pure CRO agency fits stores with a large in-house team; for most growing stores, a provider that handles analysis and implementation in one place fits better.
As a rule, from about €150,000 in revenue per month. What matters is whether there are enough visits and orders to measure changes. Below that, targeted changes with a before-and-after comparison usually bring more than a testing program.
The conversion rate shows how many visits end in an order. Revenue per visitor (RPU) shows how much revenue each visit brings on average, so conversion rate times average cart value. It's the better success metric because it also captures changes in cart value.
By fixed percentage promises before the analysis, cosmetic tests without reasoning, no statement on test duration and reports that only show the conversion rate.
A tool only measures what you test. It doesn't decide which change has the biggest lever, how it's designed and built, or when a result holds. If your team can do that, you only need the tool.
The analysis delivers findings in the first few weeks. Whether a change works shows after a few weeks to months, depending on traffic. A good agency estimates the test duration for your store beforehand.










