A/B Test: Shortened form copy

Booking.com

Background

During my time at Booking.com, I worked for several years on the reviews team and I was the designer in charge of the reviews collection flow. After a user completed a trip, we asked them for reviews of hotels, cities, and points of interest they visited. This information was then used to inform other travelers and help them to have the best possible trip.

This experiment focused on the form we used to collect reviews of cities after a user’s trip.

Hypothesis

Our hypothesis here was simple: shorter copy on the city review form would allow users to read the questions faster and be more likely to complete their review of the city. This is especially important for mobile devices because screen space is more limited.

Results

We ran the experiment for 1 week, with a total of 292k visitors in both variants. Our primary goal destination review submitted increased 11.57% (± 0.63%) in the “B” version of the test.

Learnings

One of the things I learned over and over again while working at Booking.com was that small changes can yield big results and that even something as small as a copy change can produce dramatic results. This is clearly one of those cases. The tricky part is identifying which small ideas have big potential and that comes with time, practice, and a high volume of experiments. I personally created hundreds of experiments during my four years at Booking.com and participated in thousands more beyond that. This experience helped to hone my eye for low-effort, high-impact experiments.