Which banner is better, which ad headline is better, which LP is better... In order to make such decisions, "AB testing" is often performed.
However, if AB testing is not done correctly, it will not produce valuable results.
This time, we will introduce tips for conducting AB testing correctly using "Fisher's Three Principles."
>>Download "10 checkpoints for improving your ad operations that you can judge even if you have no experience in ad operations!"
Table of Contents [ Hide ]
What are "Fisher's Three Laws"?
Randomization
Repetition
Localized management (blocking)
Example: Check the accuracy of AB testing
Answers and Solutions
summary
What are "Fisher's Three Laws"?
"Fisher's three principles" are the principles phone number database philippines proposed by statistician R. A. Fisher to reduce bias and error in data in experimental planning. Specifically, they refer to the following three points:
Let's take a closer look.
Randomization
For example, randomization means "in an experiment in which subjects compare the taste of several different sweets, the order in which they eat the sweets is randomly changed for each subject."
In this experiment, we want to compare the sweets, so we want to make other factors as uniform as possible. However, we cannot make the order in which the sweets are eaten completely uniform (if all the sweets could be eaten at the same time, it would be uniform, but that would not be a comparison).
In such cases, bias can be prevented by at least randomizing the order in which each subject eats the food. This is called randomization .
Repetition
Repetition means something like "in a sweet tasting experiment, having a certain number of people try the different sweets ." If there is only one subject in a taste-comparison experiment, the results will reflect that person's preferences.
To avoid bias, it is necessary to conduct experiments with a relatively large number of subjects.
Localized management (blocking)
When dividing subjects into groups, it is desirable to keep variability small except for the parts being compared. If subjects are divided into two groups, both groups should have similar age and gender composition so that there are no differences.
For this reason, the experiment may be divided into several blocks where the experimental conditions are as uniform as possible. This is called local control (blocking) .
Example: Check the accuracy of AB testing
Taking Fisher’s Three Laws into consideration, let’s consider whether AB testing can be carried out accurately for the following ad deliveries.
Example:
To conduct an AB test of the "headline" of an advertisement, Ad A and Ad B were created with the same landing page and ad copy, but with different "headlines." These were placed in the same ad group and delivered for two weeks. After two weeks, the performance of each advertisement was checked, and the one with the higher click-through rate was determined to be the "headline" with the best results.
What do you think?
Answers and Solutions
In fact, this "AB testing" may not be accurate in the following ways:
Reason for inaccuracy 1: Poor local control
In the case of distribution media such as Google Ads, appropriate advertisements are delivered taking into account the user information held by the media.
That is, if Ad A is found to be effective with men, it may be preferentially delivered to men.
In such cases, the attributes of users exposed to Ad A and Ad B may be different.
