A/B testing your landing page: how to know which design sells more

You built a landing page for your product, the ad is running and orders are coming in. But one question keeps nagging you: if you changed the design, would orders go up? Maybe a big headline over the photo sells more, maybe the order form at the top of the page works better, maybe calm colors reassure customers more than loud ones. The only way to really know is an A/B test.

In this article we explain what A/B testing a landing page means, what exactly to test, how many visitors you need, and how to read the result without fooling yourself. At the end we show you how to run it on HalaOrders in minutes.

What is an A/B test?#

An A/B test means showing more than one version of the same page to different visitors at the same time, and comparing which version brought more orders relative to its number of visitors.

• Version A: your current page.
• Version B (and C, D...): the same product, the same price and the same offers, but a different design.
• Each visitor sees only one version, and if they come back to the page they see the same version again.
• In the end you compare the conversion rate of each version: orders ÷ visitors.

Visitors are split randomly, so any large difference in the result is most likely caused by the design itself, not by the type of visitors, the day or the ad.

This is what we call a dynamic landing page: instead of a static page with one design, the link in your ad shows several designs, learns from real orders which design sells best, and then settles on it by itself.

Why not just compare one week with the next?#

The easiest thing is to run one design for a week and another the following week and compare. The problem is that lots of things change between the two weeks: the ad budget, the audience Facebook reaches, the start and end of the month, seasons and competing offers. So you might think the new design is better when in fact it was the week that was better.

In an A/B test all versions run at the same time, on the same ad and the same audience, so the comparison is fair.

Why A/B testing raises sales and improves your ad results#

In ads you pay for the visit, not for the order. So every increase in conversion rate means more orders from the same money, and a lower cost per order. The math is simple:

Cost per order = cost per visit ÷ conversion rate

Take an example: a budget of 10,000 EGP, with each visit costing 5 EGP, which means 2,000 visitors to your page.

So if the better design raises conversion from 2% to 3%, you get 50% more orders and the cost per order drops by a third, without adding a pound to your ads. That matters in more than one place:

• More profit from the same product: the ad cost per order drops, so your margin grows. Calculate it for your product in the profit calculator.
• You can scale your ads with confidence: when the cost per order drops, a budget that was losing money can start making money, so you can increase it confidently.
• The ad itself performs better: if you have the pixel installed and your ad optimizes for purchases, every extra order gives Facebook and TikTok more data to learn who buys, so they reach more people like them.
• Decisions based on numbers, not gut feeling: instead of changing the design every now and then and waiting, the test tells you exactly which design is better and by how much.

Important to know: not every test produces a big winner. Sometimes the designs sell about the same, and that in itself is useful information: the design isn't the problem, and you would do better working on the offer, the photos or the audience.

What should you test on a landing page?#

Start with the things that affect the customer's decision most:

• Page order: the order form at the top of the page, or after the benefits and photos?
• The first screen: a large photo with the headline over it, or a photo next to the headline and price?
• Colors and overall feel: a calm, premium design, or a promotional design with bold colors?
• Headline and message: "The best price in Egypt" or "Quality that lasts for years"?
• Sections: a product video, customer reviews, a comparison table, FAQs.

Important: the price and offers must stay the same in every version. If you change the price and the design together, you won't know which one made the difference.

How many visitors do you need?#

This is the most important point and the one people get wrong most often. Take an example:

• Version A: 100 visitors and 3 orders (3%).
• Version B: 100 visitors and 2 orders (2%).

It looks like A is 50% better! But in reality that difference is just one order, and it could flip in the next hour. Statistically, the probability that A really is better than B is only about 67%, close to a coin toss.

If the same rates held up to 500 visitors per version (15 orders against 10), confidence reaches about 84%. And at 1,500 visitors per version (45 orders against 30), confidence passes 95%, the level at which we say the difference is real.

In short:

1. Don't judge early. The first 50 or 100 visitors give you a feeling, not a result.
2. Orders matter more than visitors. A test with 1,000 visitors and two orders tells you nothing.
3. The smaller the difference between designs, the more visitors you need to confirm it.
4. Don't test 5 designs on a very small budget. Visitors are split across the versions, so each one gets a small share. If you have few visitors, test only two or three designs.

How do you read the result correctly?#

• Compare conversion rates, not order counts. A version that got more visitors will naturally have more orders.
• Cancelled orders don't count. A design that brings lots of orders that get cancelled is not a successful design.
• Look at the confidence level. If the leader is ahead with less than 95% confidence, it is better to keep the test running a little longer.
• Tie the result to money: a one-point difference in conversion rate may look small, but on a monthly ad budget it means dozens of orders.

Take this example of a test with 4 designs (made-up numbers for illustration):

Version B brought 48 orders from 1,500 visitors against 33 orders for the original design from the same number. That means 45% more orders from the same ad, a cost per order about 31% lower, and 95.5% confidence that it really is better, so the system can declare it the winner and show it to every visitor.

• If there is no clear difference after a long time, the designs sell about the same. Stay with the original and try a bigger change next (a different offer, new photos, a headline with a different angle).

A/B testing landing pages on HalaOrders#

In HalaOrders landing pages this test runs by itself, with no complicated setup:

1. From the dashboard open Landing pages, and from the page's menu choose A/B test.
2. Press Add versions and choose from the ready templates (up to 4 versions). Each version is created with the same product, prices and offers, and you can edit its design and content in the visual editor.
3. Press Start test and set the number of visitors per version in each round (100 is a good start), the number of rounds, and how many versions qualify from each round.
4. Use the same page link in your ad. Visitors are split randomly across the designs, and each visitor keeps seeing the same design.
5. After each round, the best converting designs qualify for the next one. If orders are still few, the round extends by itself instead of judging on too little data.
6. In the final round, the winner is decided once we are 95% sure it beats the runner-up, and from then on every visitor sees the winning design automatically.

Throughout the test you can follow each design's visitors, orders, conversion rate and probability of beating the original page, plus the result of each round. If you prefer to decide yourself, you can pick the winner at any time or restart the test.

Bots and link-preview visits (like the one Facebook makes when you put the link in an ad) don't count as visitors, and each order is recorded against the design the customer ordered from, so you can see it in your orders like any normal order.

Tips before you start#

• Start with designs that are really different. Changing only a button color rarely makes a difference you can measure on a small budget. Templates that differ in layout and feel give a clearer difference.
• Leave the ad as it is for the whole test, so any change in the result comes from the page.
• Install the pixel to tie the results to your ads. The steps are in the guide to installing the Meta Pixel and TikTok Pixel.
• Calculate profit, not just orders, with the online store profit calculator.
• If you don't have a landing page yet, start with the guide on how to build a landing page that sells from Facebook and TikTok ads.

The bottom line#

A/B testing turns the question "which design is better?" from opinion and gut feeling into numbers. Test designs that are really different with the same price and offers, wait until you have enough orders, and decide by conversion rate and confidence. On HalaOrders the whole test runs by itself, from splitting visitors until the winner is shown to all your customers.

You can follow every new feature on the What's new in HalaOrders page.