Research Note #18
How To Test Changes Properly
Changing five things at once then trying to work out what worked is one of the easiest mistakes in conversion optimisation. The more variables you introduce, the harder it becomes to learn anything useful. Good tests are boring, focused and controlled.
The Problem With Random Changes
When you change lots of things at the same time, you create lots of possible explanations. Maybe the new headline worked. Maybe the testimonials worked. Maybe the traffic changed. Maybe nothing actually changed at all.
The more variables you introduce, the harder it becomes to learn anything useful.
The Three Rules Of Better Testing
1. Change one thing at a time. If you change the headline, the testimonials and the pricing all at once, you'll never know which one mattered.
2. Run tests long enough. A few days isn't enough. Give your test enough time to reach statistical significance.
3. Measure what matters. Not clicks. Not traffic. Measure the outcome you actually want — enquiries, sales, bookings.
Practical Takeaway
Pick one thing to test. One variable. Run it for at least two weeks. Measure the result. Then move to the next one.