2. An A/B test is a randomised experiment widely used in marketing and business
ID: 3361879 • Letter: 2
Question
2. An A/B test is a randomised experiment widely used in marketing and business analytics. An advertiser creates two versions of a video advertisement. To find out which version is more appealing, this advertiser conducts an A/B test. These two versions were randomly displayed on Youtube. During A/B testing, these video advertisements were presented to a total of 1050 Youtube users. 500 users (denoted Group A) watched Version 1 and 550 users (denoted Group B) watched Version 2. On average, users in Group A skipped the advert after an average of 6.7 seconds with standard error of 3.5 seconds. For Group B, users skipped the advert after an average of 7.4 seconds with standard error of 5.2 seconds. Which version of the advert is more appealing? Please use a hypothesis test to support your answer.
Explanation / Answer
Null hypothesis H0: The average watch time of Version 1 and Version 2 advert are equal. That is the mean difference in watch times of version 1 and version 2 is 0.
Alternative hypothesis Ha: The average watch time of Version 2 advert is greater than the watch time of Version 1 advert. That is the mean difference in watch times of version 2 and version 1 is greater than 0.
Difference in average watch time = 7.4 - 6.7 = 0.7 sec.
We will be using the two-sample z-test to determine whether the difference between means found in the sample is significantly different from the hypothesized difference between means.
The standard error (SE) of the sampling distribution.
SE = sqrt[ (s12/n1) + (s22/n2) ] where s1 and s2 are the standard errors of version 1 and version 2 advert and n1 and n2 are the sample size of two groups
SE = sqrt[ (3.52/500) + (5.22/550) ] = 0.2714
Z = (Observed difference - Hypothesized difference) / SE
= (0.7 - 0) / 0.2714 = 2.58
P-value for Z = 2.58 is given as,
P(Z > 2.58) = 0.005
As, p-value is less than the assumed significance level of 0.05, we reject the null hypothesis and conclude that the average watch time of Version 2 advert is greater than the watch time of Version 1 advert. So, we can conclude that at 0.05 significance level, version 2 of the advert is significantly more appealing than the version 1.
Related Questions
Navigate
Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.