UX case study · A/B testing

Adding pay later to a hotel booking flow

Findhotel already sold rooms that a traveller could pay for at the hotel instead of at checkout. Nobody could choose that while booking. I designed the first version of that choice, and a five-week test measured what it was worth.

5wks
The test ran
800
Bookers a day, split evenly
+4.2%
Conversion, after cancellations
Client
Role UX Designer, sole designer on the feature
Team 1 UX designer, 1 product owner, 1 data scientist, 1 copywriter, a scrum team
Timeline October to December 2020, results in January 2021
Platform The booking funnel, on desktop, tablet and mobile web
Tools Figma, Zeplin, FullStory
The room selection page on desktop, with the pay now and pay later options presented as a choice

The choice ended up on the room selection page, before checkout. I wanted it in the checkout itself, and further down is why it could not go there.

Problem

Why this project?

Findhotel is a smaller player in hotel booking. It works the local markets that Booking, Expedia and TripAdvisor tend to leave alone. Letting travellers pay at the hotel is one way a platform like that can stand out.

Pay later could go a thousand directions. Everyone had a view on how it should look and what belonged in a first version. Several scrum teams were involved, and the feature was meant to move conversion on the company's main product, so everyone was right to have a view. Something had to settle it.

Goals

Desired outcome

We knew travellers would welcome the feature. Nobody knew by how much, so the first version was designed to find out.

The smallest version that is still true
Bring it live on the least effort that still gives a traveller a real choice.
Borrow what is already proven
Take an approach the rest of the industry has tested at a scale we could not reach.
Built to be measured
A design that could go into an A/B test with the fewest possible changes to the flow around it.

All three are one goal: get a number. A small first version was not a way to do less work. It was the only way to measure the feature, and only a measurement would win a second round. Everything else I wanted to design had to wait for it.

Scope and constraints

Limitations

Three constraints shaped this more than any design decision I made.

The first was the checkout page. A technical limit meant we could barely change it, so the switch in checkout I wanted was off the table, along with several other designs. That left one place for the choice: the room selection page. Asking someone to pick when to pay before they reach checkout is odd. But it could ship, and shipping something measurable was the point.

The second was the search page, where a traveller decides which hotel to open. Another team owned it, and it could not change in time for the test. It showed only the top three providers for each hotel, and Findhotel's own pay-later offers were not always among them. So a traveller could see pay-later offers from Booking, Agoda and Expedia, and none from Findhotel, even where Findhotel had them.

The third was language. The first weeks of the test ran in English only, because the translation system was too slow for copy that would change several times.

Team

My part, and the team around it

I was the only designer on the feature. I studied the competition, ran the ideation and scoping sessions, prototyped and tested the concept, drew the final screens, tested the build by hand and watched real sessions once it was live. A product owner, a data scientist and a scrum team of developers built it with me.

Deep dive

The team, and who I worked with

Team map

Team's setup

Our product owner brought the business case and the priorities. Our data scientist brought the numbers the decisions were made against, and designed the A/B test itself, which matters more than it sounds: a test designed by the people who want a particular answer is not a test.

A dedicated copywriter wrote the words. An external agency translated them. I was the only designer on the feature, and the developers were a scrum team inside an organisation that had several.

My responsibilities

Benchmarking the competition, running the team's ideation and scoping sessions, wireframing, building a prototype low-fidelity enough to argue with, guerrilla testing it, turning the result into a development plan, and drawing the high-fidelity screens against the design system.

After that: keeping the copy pointed at what the test was measuring, testing the build by hand against every scenario we had agreed on, watching real sessions once it was live, and writing the whole thing up for colleagues who were never in the room.

The team Search listing team Other teams PD PO DES Dev PO DES Dev PO DS DES Dev Dev FE The team Search team Other teams PD PO DES Dev PO DES Dev PO DS DES Dev Dev FE

Inside the box, the product owner, the data scientist and the developers worked alongside me, one of the developers mainly on the front end. There was no scrum master. The product owner and I answered to the product director. Around the box are the other teams, built the same way: a designer, a product owner and developers. I worked mostly with their designers, and with the designer of the Search listing team on keeping the customer journey aligned. Its product owner and I communicated indirectly.

Process

How it was made

This was the first version of pay later at Findhotel, and it had to earn the rest.

01

What the industry already did

I took apart every major booking platform, and several smaller ones, screen by screen. I wanted findings the team could act on: what the industry had settled on, where the gaps were, and which of our arguments bigger players had already answered.

Pay later, platform by platform

Search
Room selection
Checkout
Confirmation page
Confirmation email
Booking management

Free cancellation

Reserve now, pay later

On the result card
Payment Type

Free cancellation

Reserve now, pay later

In the filters

Free cancellation

Before Tue, Nov 24

Reserve now, pay later

On the room offer

Your payment options

Free cancellation before Tue, Nov 24You can change or cancel this stay if plans change. Because flexibility matters.
Pay now
  • More ways to pay: use Debit/Credit card or PayPal
  • You can use a valid Expedia coupon
MX$88per night, $222 for 2 nights, includes taxes & feesPay now
Pay when you stay
  • You will not be charged until your stay
  • Pay the property directly in their local currency
MX$88per night, $222 for 2 nights, includes taxes & feesPay at property
In the pay-later options
Total
$221.94
Pay now
$0.00
Pay at property
$221.94
In the price summary

Team note Try to find a non-refundable pay later deal.

MX$0.00 due now. Payment information is only needed to hold your reservation.

Above the card form
Nothing here about where the guest will pay
On the confirmation page

Collected at the property Unless specified otherwise, rates are quoted in Euros. Rate quotes in EUR are based on current exchange rates, which may vary at time of travel. Final payment will be settled in local currency directly with the hotel.

Under the price total
On the booking

Collected by the hotel Rate quotes in EUR are based on current exchange rates, which may vary at time of travel. Final payment will be settled in local currency directly with the hotel. Unless specified otherwise, rates are quoted in USD.

On the pricing page, under the total

NO PREPAYMENT NEEDED

pay at the property

On the result card
Reservation policy

Free cancellation546

Book without credit card4

No prepayment194

In the filters
Set your filters

FREE Cancellation, NO PREPAYMENT NEEDED29

Breakfast included24

In the filters above the rooms

FREE cancellation before 20 November 2020

NO PREPAYMENT NEEDED, pay at the property

Between the room options

Book now, PAY AT THE PROPERTYWith free cancellation on most options
On the room offer

Now you'll pay€ 0

No payment needed now. You'll pay at the property.

At the top of the price summary, on both steps
Risk free: You can cancel before 20 November 2020, so lock in this great price today.

NO PREPAYMENT NEEDED, pay at the property

Under the price summary
Your credit card will not be charged. It's only needed to guarantee your booking. You'll pay at the property.
Above the card form

Team note We don't know per hotel, so we will need to mention something vague.

Only needed to guarantee your booking
On the card number field
The property handles all payments. See payment details below.

Further down the confirmation page

You have guaranteed your booking by credit card

On the confirmation page
Prepayment

This property doesn't require prepayment. However, the property may temporarily hold an amount on your card, to test your card and guarantee your booking. This test payment will be returned to you.

About pre-authorisation
  1. What is a pre-authorisation?
  2. What's the difference between a pre-authorisation and an actual charge to my credit card?
  3. How will I know if my card has been pre-authorised?
  4. How long will the pre-authorisation hold affect my available balance?
  5. Will the hold always equal the exact amount of my reservation?
  6. Will this happen with all bookings made through Booking.com?
Behind Payment info

Your payment will be handled by the property. The 'Payment' section below has more details

You'll pay when you stay at Travelodge by Wyndham Las Vegas Airport Near The Strip
At the top of the email
Prepayment
You don't need to prepay. However, the property may take a deposit from the card you booked with.
Cancellation cost
Until 14 February 2021 23:59 (PST): US$0. From 15 February 2021 00:00 (PST): US$63.99
In the email, under reservation details
Cancel bookingIt costs US$0 to cancel this booking until 23:59 (Las Vegas (Nevada) time), 14 February 2021
On the booking

Book now, pay later

On the result card
Payment options

Free cancellation526

Pay at the hotel365

Book now, pay later97

In the filters

Pay later, through Agoda

BOOK NOWPAY LATERFREE Cancellation
On the booking button

Pay nothing until 23 November 2020

In the room summary
Choose your payment option
Pay later

RISK FREE! Pay nothing until 23 November 2020

Free cancellation before 23:59 hrs. on 24 November 2020 (property local time)

Depending on your payment currency, Agoda's currency exchange rate may be applied as per Agoda's terms of use

On the payment step
BOOK NOW & PAY LATER!
On the last button
Not followed past checkout

Reserve now, pay at stay

On the result card
Deals

Free cancellation

Reserve now, pay at stay

Properties with special offers

In the filters

Free cancellation until 11/25/20

Reserve now, pay at stay

On the provider offers

Team note On the Expedia and Hotels.com offers, not on Agoda's.

Free Cancellation until Nov 25, 2020

Book now, pay at stay!

On the room offer, desktop
1Your Room Selection

Book now, pay at stay!

In the room summary

Book now. Pay later when you stay.

In the price summary
3Enter Your Reservation Guarantee

€0 due now. Payment information is only needed to hold your reservation.

Above the card form
Book now, pay when you stay

Your reservation is with Hotels.com, who will handle your customer service needs.There are no surprises and no hidden fees. This is your total price.You can cancel for free before Nov 25, 2020.

Under the billing address
Not followed past checkout
ZenHotels
Not shown on mobile

On the result card, desktop

Pay on the spot

On the mobile results

Team note ZenHotels seems to have forgotten to add pay later to its mobile results.

Payment and booking

No card required for booking

Free cancellation available

Pay now

Pay on the spot

In the filters

Free cancellation before November 20*

Pay on the spot, Card is required for booking

On the room offer

Pay on the spot

Credit card required

In the room summary
To be paid at the hotel
$ 145.44≈ € 122
Service fee and vat included.
To be paid upon arrival
Environmental fee
MXN 40
In the price summary
Reservation Guarantee

Your card details will be transferred to the hotel for your booking guarantee. According to the booking conditions, the hotel can withdraw the amount from your card before checking-in.

In the reservation guarantee
Not followed past checkout
Not shown on mobile

On the result card, desktop

Pay now or at property

On the mobile results

Team note Strangely enough, Hotels.com does not display pay at property on mobile, only on tablet or desktop.

Pay now or at hotel

In the window that asks when to pay

Choose when you'd like to pay

Pay in full online
  • We'll process your payment in your local currency
  • Cancel for free right up until the cancellation window of the property
  • You can use a valid Hotels.com coupon
  • Hotels.com® Rewards
  • Collect
  • Redeem
€538for 1 room for 2 nightsPay in full online
Reserve with deposit
  • Pay the property in their currency
  • Cancel for free right up until the cancellation window of the property
  • Hotels.com® Rewards
  • Collect
  • Redeem
€539for 1 room for 2 nightsReserve with deposit
On the room offer, desktop
Total price
approx. 539,22€
You will pay in the property's currency
USD 639.50
In the price summary
Payment schedule
Deposit
First night plus tax (after booking)
In the payment schedule

Team note Nothing in the payment tells you how much the so called “Deposit” will be.

This price may increase if you book later.
In the payment schedule
Not followed past checkout
Did not seem to support it yet
  • Airbnb
  • Trip.com
  • Lol.travel
  • Skyscanner

Six platforms let a guest book now and pay at the property, and no two of them call it the same thing.

What the team liked most was a plain statement of what happens next: nothing is due now, and the card is only there to hold the booking. The two platforms that were followed past checkout kept saying who collects the payment, in the email and again under booking management.

02

Choosing one way, and cutting it down

I ran the ideation sessions, and the benchmark made them work. A room of people with strong opinions is hard to lead. A room that has just seen what five competitors shipped is easier: the material did the arguing.

Before the sessions, I wrote down why a traveller would want to pay later, from my own research into payment motivations.

User motivations

  • I want to reserve something without tying up my money

  • I prefer to pay in cash

  • It is low risk if I need to cancel, and it feels more secure

  • I do not have the money at the moment

  • I want to lock in the price I see

Four ways to offer pay later

Duplicate mixed offers

Booking.com's approach

Now you'll pay€ 0

No payment needed now. You'll pay at the property.

Your credit card will not be charged. It's only needed to guarantee your booking. You'll pay at the property.

Votes V.

Selected approach

Pop-up before checkout

Expedia's approach

Pay now
  • More ways to pay: use Debit/Credit card or PayPal
  • You can use a valid Expedia coupon
MX$88per night, $222 for 2 nights, includes taxes & feesPay now
Pay when you stay
  • You will not be charged until your stay
  • Pay the property directly in their local currency
MX$88per night, $222 for 2 nights, includes taxes & feesPay at property

Votes A., M.

Switch in checkout

Agoda's approach

Choose your payment option
Pay later

RISK FREE! Pay nothing until 23 November 2020

Free cancellation before 23:59 hrs. on 24 November 2020 (property local time)

Depending on your payment currency, Agoda's currency exchange rate may be applied as per Agoda's terms of use

Pay now

You can also choose to pay now with a credit card or other payment methods below.

Votes Gil

Only list pay later

Where a property offers both, list only pay later

Anything that comes from Back-office FindHotel and has both the Pay now and Pay later option should be Pay later.

No votes

Deep dive

What the team wrote under each

Pros and cons from each approach
Duplicate mixed offers
Pros
  • It defaults to the easy option, avoiding friction in checkout
  • Side by side comparison of pay later and pay now in performance
Cons
  • The user can only switch by going back to the room selection page and choosing the right room in a table of deals
  • There is no way to see how it would be otherwise
  • Forces duplicate listings on the room selection page because of a deal feature
  • The number of offers per room would need limiting, or the list gets too long and restricts the user's options
  • Too many options to select
Pop-up before checkout
Pros
  • The user can see both scenarios neatly laid out
  • The user can switch from one scenario to the other
  • Prevents duplicate listings on the room selection page
  • Better explanations of the value proposition for users
  • Side by side comparison of pay later and pay now in performance
  • Most options offered to the user
  • Option offered in a timely manner
Cons
  • It forces the user to make a decision in order to proceed, which creates friction in checkout
  • For one night it makes very little difference, yet it forces the user to make a decision (decision paralysis)
  • The user must go back to switch scenario
  • One more step in the funnel, one more opportunity for a dropoff
  • More building involved to validate?
  • We would need to map the pay now and pay later duplicates together, to choose which two offers to show in the pop-up
Notes left on the screen
  • Gil We do not have to copy the solution.
  • I. Take into consideration not asking the user twice.
  • I. Try it on the room selection page? A link under the pay later button.
Switch in checkout
Pros
  • Defaults automatically to one option yet lets the user change, avoiding additional friction
  • The user can see both scenarios neatly laid out
  • The user can switch to either scenario without having to go back
  • Prevents duplicate listings on the room selection page
  • Most options offered to the user
  • Option offered in a timely manner
  • The payment method shown is contextually accurate
Cons
  • Yet another field to add to our already long checkout form
  • The payment breakdown would have to change when the user toggles, which could be confusing
  • A user who selected something by default in the past discovers extra information on the payment page
  • A user who could be interested in pay now might think it is not available
  • Changes on the fly to the payment page depending on the selection, so room for errors
  • The most UI changes needed
Only list pay later
Pros
  • The leanest option for A/B testing: it gives us the cleanest data
  • The fewest UI changes needed
Cons
  • The user does not have the option to pay now at all
  • We would never find out how much it cannibalises pay now
  • A certain share of users may dislike pay later, taking it as a sign of untrustworthy hotels
  • A user who could be interested in pay now might think it is not available

I wanted the choice in the checkout itself, the way Agoda does it. The checkout could barely change, so we took Expedia's approach instead: a choice made before checkout. Expedia had the most mature pay-later system in the benchmark, which made it the one to learn from.

Scoping ran alongside, with the product owner and the data scientist. We kept two questions apart: what is the ideal version, and what is the smallest version worth testing? Keeping both meant every cut was written down, ready for a second round.

The complete plan, and the first version

  • 14In the first version
  • 11Cut, kept on file
  1. Meta
    • Cut, kept on file: Bid for pay later only properties, if they are good and mapped
    • Cut, kept on file: Pay later only properties in the EPS master inventory, validated with Content. How many match between EPS and Findhotel in the new master inventory file?
  2. Search
    • In the first version: Offers carry the pay later tag
    • Cut, kept on file: Pay later offers sorted higher
    • Cut, kept on file: Filtering for pay later
  3. Room selection
    • In the first version: Properties carry the pay later tag
    • In the first version: Pay later details added to the deal details
    • Cut, kept on file: Filtering for pay later
    • Cut, kept on file: Pay later sorted to the top by default, as the primary offer
    • Cut, kept on file: A switch between pay now and pay later (a fork)
  4. Checkout
    • In the first version: Pay later in the price breakdown
    • In the first version: Currency conversion for local currencies
    • In the first version: UX explaining why we ask for credit card details
    • In the first version: Terms and conditions of pay later deals (no show and so on)
    • In the first version: A UX change on the payment page to make clear that the amount will not be charged, or that a check amount might be
    • Cut, kept on file: Deposits: we are not going to do it (5% of Expedia's pay later bookings)
    • Cut, kept on file: Reinforcing "No payment needed" and the like in the UX
    • Cut, kept on file: Booking error message: should we change it?
    • Cut, kept on file: Pay at property information changed to include pay later
  5. Confirmation
    • In the first version: Pay later in the price breakdown
    • In the first version: Currency conversion for local currencies
  6. Booking management
    • In the first version: Pay later in the price breakdown
    • In the first version: Currency conversion for local currencies
    • In the first version: Price breakdown moved to the B side, if the B side wins
    • In the first version: Pay at property tooltip hidden for pay later
03

The screen

Before anything else, I mapped what a traveller could do and what would happen when they did. Not just the happy path: every outcome, including the ones nobody wants.

Deep dive

The scenario map: every path a traveller could take

Interactive map

The development plan is only as good as this map. Mapping the scenarios first turns edge cases into design decisions instead of bugs found in the last sprint. It does not catch everything, and this one did not.

01Setting up

  • User inputs the first details to begin a search

02Search

  • User finds a list of properties
  • Hotel supports pay later
      • Pay later isn't shown as a feature of that hotel
      • Pay later is shown as a feature of that hotel
  • User opens the filters
  • Is there at least one hotel that supports pay later?
      • Pay later is disabled, with an explanation of why it can't be enabled
      • Pay later is enabled
      • User selects pay later
      • System filters for hotels that have at least one deal that supports pay later
      • Number of possible deals with that combination is shown to the user as a preview
      • Filter is enabled, along with an option for the user to disable it
  • User selects a listing with at least one pay later deal

03Room selection

  • User finds a list of deals in Promobox
  • Deal in Promobox supports pay later
      • Pay later isn't shown in that deal as a feature
      • Pay later is shown in that deal as a feature
  • User selects deal details
  • Deal supports pay later
      • Pay later isn't shown in that deal as a feature
      • Pay later is shown in that deal as a feature
  • User finds a list of deals
  • Room deal supports pay later
      • Pay later isn't shown in that deal as a feature
      • Pay later is shown in that deal as a feature
  • User opens the filters
  • Is there at least one deal that supports pay later?
      • Pay later is disabled, with an explanation of why it can't be enabled
      • Pay later is enabled
      • User selects pay later
      • System sorts the pay later deals first, with clear labels
      • Number of available deals is shown to the user
      • Filter is enabled, along with an option for the user to disable it
  • User selects a deal that supports pay later

04Checkout

  • User must select whether to pay now or later
  • User selects pay later
      • User is sent to a normal checkout
      • User is allowed to change to pay later if they want
      • User is sent to a pay later checkout
      • User is still asked for the same details, including a credit card for verification
      • The room price per night becomes part of what is paid at the property
      • A message explains why the credit card details are needed for the booking
      • User is made aware that the property could charge a pre-authorisation or a nominal amount ahead of the stay
      • User is made aware of other significant conditions, like no-show
      • User is allowed to change to pay now if they want
  • User checks out

05Confirmation

  • User sees a success message
  • Did the user select pay later?
      • Success as usual
      • User is given the conditions of pay later along with the success message
      • No-show fee is displayed
      • Late cancellation is displayed
      • Option to cancel is displayed
      • What type of booking policy did the user book?
          • Payment later must happen on the specified date
          • If it is cancelled, a partial refund should be issued only after the credit card has been charged
          • User has to pay nothing
  • User is given access to the itinerary page

06Booking management

  • User gets an email about the booking
  • Did the user select pay later?
      • Same email as usual
      • Same email as usual, except that everything is paid at the property
  • User opens the itinerary page
  • Did the user select pay later?
      • Nothing changes
      • All charges are paid at the property
      • No-show fee is displayed
      • Late cancellation is displayed
      • Option to cancel is displayed
      • What type of booking policy did the user book?
          • Payment later must happen on the specified date
          • If it is cancelled, a partial refund should be issued only after the credit card has been charged
          • User has to pay nothing

07Cancellation

  • User opens the cancellation dialog
  • Did the user select pay later?
      • Cancellation conditions are shown as they would be
      • What type of booking policy did the user book?
          • User is notified that a charge will happen nevertheless
          • User is notified that a partial charge will happen nevertheless
          • User is notified that no charges will be made
      • Cancellation policy is shown

Every interaction, and what follows from it. The edge cases are the point of this drawing, not the middle.

The proposal selector for a booking of one night and two rooms, offering three proposals: Number of nights & rooms, Stay total and Laissez faire, and a button to see the case for one night and two rooms

Proposal selector

The wireframe became a clickable prototype. It opens on a choice of three proposals for showing the price, and each one carries on through search, room selection, the choice between paying now and paying at the property, and the payment page. Feedback on something people have clicked through is worth more than feedback on a diagram, and it comes sooner.

The Figma prototype, wired the way it was built, for a booking of one night and two rooms. The payment choice is the screen the whole feature turns on, where paying now and paying at the property become two different things.

Then I drew the final screens with Findhotel's design system, so the developers built a layout instead of guessing one. Spacing, colours and icons all came from the system, and the key desktop and mobile screens went to the developers in Zeplin.

From the final screens I drew the development plan: the happy path, the main edge cases and the screen each one lands on. The product owner and the developers wrote their tickets from it.

Deep dive

The development plan, and Expedia's rules

Two diagrams

Two questions sort a booking into one of three versions, and a third question, about currency, adds one message along the way.

The development plan

Is it a pay later deal?

Does the room offer both pay later and pay now?

Pay now dealPaid online, as before
Pay later onlyThe room is paid at the property
Pay later or pay nowThe traveller chooses
Search
Room selection
Checkout
Confirmation
Booking management
Cancellation
SearchAs today

What gets builtDon't show the Pay later label.

Movenpick Hotel Amsterdam City Centre

★★★★8.8Superb

€181 €166
Free cancellation before Tue, Aug 27
SearchChanged

What gets builtShow a Pay later label on the deal, the same in the results list and on the hotel page.

Movenpick Hotel Amsterdam City Centre

★★★★8.8Superb

€181 €166
Pay later
Free cancellation before Tue, Aug 27
Room selectionAs today

What gets builtDon't show the Pay later label.

Save 10%€181 €166

Standard double or Twin room

Breakfast included

Free cancellation before Tue, Aug 27
Deal details Reserve ›
Room selectionChanged

What gets builtReserve goes straight to the payment page. Back from there returns to the room page.

Save 10%€181 €166

Standard double or Twin room

Breakfast included

Reserve now, pay later

Free cancellation before Tue, Aug 27
Deal details Reserve ›

Go to the next screen (payment page)

Room selectionChanged

What gets builtReserve opens Select payment preference, so the traveller picks pay later or pay now.

Save 10%€181 €166

Standard double or Twin room

Breakfast included

Reserve now, pay later

Free cancellation before Tue, Aug 27
Deal details Reserve ›
Select payment preference

Pay at property

  • Pay when you stay
  • You will pay in the property’s local currency
€181 €166Pay later

Pay now online

  • Pay online with your credit or debit card
€181 €166Pay now
CheckoutTooltip

What gets builtHide the pay later messages and the label. Display the tooltip that explains why part of the price is paid at the property.

Price details

Total€316.00
Pay now
€306.00
Pay at property
€10.00
Pay at property amount will be collected by the property in the local currency.
CheckoutChanged

What gets builtShow Pay now as zero and the whole amount under Pay at property, with the label and the card notice.

Price details

Total€316.00
Pay nowNo payment needed now, you’ll pay at the property
€0
Pay at property Pay later
€316.00

Your credit card information is needed to guarantee your booking. You’ll pay at the property.

ConfirmationTooltip

What gets builtDisplay the tooltip that explains why part of the price is paid at the property.

Payment

Total€316.00
Already paid
€306.00
Pay at property
€10.00
Pay at property amount will be collected by the property in the local currency.
ConfirmationChanged

What gets builtShow what is already paid and what is due at the property, on the page and in the email.

Payment

Total€316.00
Already paid
€0.00
Pay at property Pay later
€316.00
Booking managementAs today

What gets builtNothing to add.

No changes

Booking managementChanged

What gets builtShow the same amounts above the price details.

Already paid
€0.00
Pay at property
€316.00
CancellationAs today

What gets builtA cancelled booking looks as it does today.

No changes

Is there a cross-currency conversion?

Asked at checkout, confirmation and booking management

Yes

Rates quoted in EUR are based on current exchange rates, which may vary at the time of payment. The property will charge you the full amount of €316.00 in USD.

No

Don't display the message.

Beyond the screens

Two pages of words had to change as well: the Terms page, to describe nominal fees in legal language, and the help page that answers “Do I need a deposit when making a reservation?”. Both followed Expedia's example. Expedia's rules, below

It also settled something design could not decide alone. This version ran on Expedia's data, so it had to follow Expedia's rules for handling a booking after checkout. I read those rules and redrew them as a decision flow anyone could follow. The developers found it as useful as everyone else: it showed which features we inherited, and which we chose not to build yet.

Expedia's rules

Card check
Payment
After checkout

The traveller checks out

Was the transaction pre-authorised?

The booking does not succeed

The traveller sees a message about the unsuccessful booking, and goes back to checkout

Is it a pay later booking (property collect)?

The traveller is charged right away (the pay now path)

Is it a property collect with a deposit?

The traveller is charged the deposit

The traveller is told the schedule of payment and how much will be charged

Is there a nominal amount to be charged?

The property holds the funds on the card to verify it, without taking the money out

A nominal amount is charged to the traveller's card

The traveller is told a nominal amount will be charged, and how much

At the property, the nominal amount is deducted from the total payment, and the traveller is told

Was the booking consumed?

Was the cancellation done on time?

The cancellation policy is taken into account

The traveller is recommended to contact the property directly, to ensure refunds

A penalty is applied to the traveller, who is told

Was the booking partly refundable?

The traveller is charged or refunded, with the necessary deductions

The traveller is given the contact details of the property

The cancellation policy is shown to the traveller

Was the booking non-refundable?

The cancellation policy is shown to the traveller

The booking was free cancellation

The cancellation policy is shown to the traveller

The traveller is recommended to contact the property directly if they see a charge on their card

The traveller is given the contact details of the property

Does the traveller deserve a refund?

The traveller gets a refund email

The booking is cancelled

The traveller gets a cancellation email

Did the traveller show up on the first night?

The cancellation policy is taken into account

Does the traveller deserve a refund?

The traveller gets a refund email

The traveller is charged the deducted amount

The traveller is charged

04

What we tested

Side A was the booking flow as it was. Side B added the choice, and side B is what shipped. I drew the two sides next to each other, so a colleague who missed every meeting could see in seconds what had changed.

Side Athe control

Side Bthe change

Between brackets: what pay later added or changed

Room selection

Does the deal support pay later?

Side A
Doesn't support itUnchanged

Palms Place

★★★★8.4Very good 10,553 reviews

4381 West Flamingo Road, West of the Las Vegas Strip, Las Vegas, Nevada, United States

You found the best deal
$145

Studio Suite, One King Bed, No B…

Free cancellation before Dec 08, 2020
Deal details Reserve ›

Compared with 1 other site

TTripStudio SuiteNon-refundable rate$147

Nightly price including taxes and fees

Side B
Supports itChanged

Palms Place

★★★★8.4Very good 10,553 reviews

4381 West Flamingo Road, West of the Las Vegas Strip, Las Vegas, Nevada, United States

You found the best deal
$145

Studio Suite, One King Bed, No B…

Pay later
Free cancellation before Dec 08, 2020
Deal details Reserve ›

Compared with 1 other site

TTripStudio SuiteNon-refundable rate$147

Nightly price including taxes and fees

Side B
Select payment preferenceNew
Select payment preference
Free cancellation before Dec 08, 2020

Pay at property

  • Pay when you stay
  • You will pay in the property’s local currency
$145Pay later

Pay now online

  • Pay online with your credit or debit card
$145Pay now
Checkout

Which way did the guest pay?

Side A
Pays nowUnchanged

Palms Place

★★★★8.4Very good 10,553 reviews

4381 West Flamingo Road, West of the Las Vegas Strip, Las Vegas, Nevada, United States

Check-inDec 9, 202016:00Check-outJan 6, 202111:00
1Room2Guests28Nights

Studio Suite, One King Bed, No Balcony

Free cancellation before Dec 08, 2020

Price details

28 nights x 1 room$88.49 per night
$2,477.75
Taxes and fees $11.84 per night
$331.47
Resort fee $44.22 per night
$1,238.16
Total$4,047.38
Pay now
$2,809.22
Pay at property
$1,238.16

Pay at property amount will be collected by the property in the local currency.

Confirm Payment
Side B
Pays laterChanged

Palms Place

★★★★8.4Very good 10,553 reviews

4381 West Flamingo Road, West of the Las Vegas Strip, Las Vegas, Nevada, United States

Check-inDec 9, 202016:00Check-outJan 6, 202111:00
1Room2Guests28Nights

Studio Suite, One King Bed, No Balcony

Pay later
Free cancellation before Dec 08, 2020

Price details

28 nights x 1 room$88.49 per night
$2,477.75
Taxes and fees $11.84 per night
$331.53
Resort fee $44.22 per night
$1,238.16
Total$4,047.44
Pay nowNo payment needed now, you’ll pay at the property
$0.00
Pay at property
$4,047.44

Pay at property amount will be collected by the property in the local currency.

Your credit card information is needed to guarantee your booking. You’ll pay at the property.

Payment

We accept the following payment methods

VISA●●AMEXD

Card number *

Only needed to guarantee your booking

Expiration MM/YY *

CVV code *

First name of card holder *

Last name of card holder *

We use secure transmission

We protect your personal information

Important information

By clicking “Confirm Payment”, you agree you have read and accepted our Terms & Conditions, Privacy Policy

Confirm Payment
Confirmation
Side A
Confirmation pageUnchanged

Did you find a good deal?

Let other travelers know on Trustpilot

★★★★★

Congratulations!

Your booking has been confirmed

Booking ID:

653164925677003

We’ve sent a confirmation email to gilberto@findhotel.net. This might take a few minutes.

Manage booking
Palms Place4381 West Flamingo Road, West of the Las Vegas Strip, Las Vegas, Nevada, United StatesView on map
Check-inDec 9, 202016:00Check-outJan 6, 202111:00
Studio Suite, One King Bed, No Balcony28 Nights, 2 Guests

Price details

Total$4,047.38
Already paid
$2,809.22
Pay at property
$1,238.16

Pay at property amount will be collected by the property in the local currency.

Side A
Confirmation emailUnchanged
Gilberto Sepulveda Bradford <gilberto@findhotel.net>

FindHotel Confirmed Booking at Palms Place

FindHotel customer service <support@mail.findhotel.net>
To: gilberto@findhotel.net

Thank you for booking with us, Test!

Palms Place has confirmed your stay 🎉

Booking ID653164925677003
Manage bookingBook again

Palms Place4381 West Flamingo RoadContact details+17029327777

Booking overview

StatusConfirmed

Reserved for Test Test last name
Studio Suite, One King Bed, No Balcony
28 nights, 2 adults 0 children

Check-in
Dec 9, 2020 from 16:00

Check-out
Jan 6, 2021 from 11:00

Price breakdown

1 room, 28 nightsUSD 2477.75

Taxes and feesUSD 331.47

Payment

Already paidUSD 2809.22

Due at propertyUSD 1238.16

Total price (with taxes & fees)USD 4047.38

Check-in instructions

Extra-person charges may apply and vary depending on property policy. Government-issued photo identification and a credit card may be required at check-in for incidental charges. Special requests are subject to availability upon check-in and may incur additional charges; special requests cannot be guaranteed. The name on the credit card used at check-in to pay for incidentals must be the primary name on the guestroom reservation.

Side B
Confirmation pageChanged

Did you find a good deal?

Let other travelers know on Trustpilot

★★★★★

Congratulations!

Your booking has been confirmed

Booking ID:

544952678778708

We’ve sent a confirmation email to gilberto@findhotel.net. This might take a few minutes.

Manage booking
Palms Place4381 West Flamingo Road, West of the Las Vegas Strip, Las Vegas, Nevada, United StatesView on map
Check-inDec 9, 202016:00Check-outJan 6, 202111:00
Studio Suite, One King Bed, No Balcony28 Nights, 2 Guests

Price details

Total$4,047.44
Already paid
$0.00
Pay at property
$4,047.44

Pay at property amount will be collected by the property in the local currency.

Cancellation Policy

Free cancellation before Dec 08, 2020Dates and times in the policy refer to the property’s local time.

Free cancellation until 08 December 2020 12:00 AM (GMT -08:00). Cancellations made after 08 December 2020 12:00 AM (GMT -08:00) are subject to a 1 night penalty plus taxes and fees. If you fail to check-in for this reservation, or if you cancel or change this reservation after check-in, you may incur penalty charges at the discretion of the property of up to 100% of the booking value.

Questions?

Contact us+31 20 200 8439

Email uscustomercare@ian.com

Side B
Confirmation emailChanged
Gilberto Sepulveda Bradford <gilberto@findhotel.net>

FindHotel Confirmed Booking at Palms Place

FindHotel customer service <support@mail.findhotel.net>
To: gilberto@findhotel.net

Thank you for booking with us, Test!

Palms Place has confirmed your stay 🎉

Booking ID544952678778708
Manage bookingBook again

Palms Place4381 West Flamingo RoadContact details+17029327777

Booking overview

StatusConfirmed

Reserved for Test Test last name
Studio Suite, One King Bed, No Balcony
28 nights, 2 adults 0 children

Check-in
Dec 9, 2020 from 16:00

Check-out
Jan 6, 2021 from 11:00

Price breakdown

1 room, 28 nightsUSD 2477.75

Taxes and feesUSD 331.53

Payment

Already paidUSD 0.00

Due at propertyUSD 4047.44

Total price (with taxes & fees)USD 4047.44

Check-in instructions

Extra-person charges may apply and vary depending on property policy. Government-issued photo identification and a credit card may be required at check-in for incidental charges. Special requests are subject to availability upon check-in and may incur additional charges; special requests cannot be guaranteed. The name on the credit card used at check-in to pay for incidentals must be the primary name on the guestroom reservation.

Before release, I checked the build by hand. Once it was live, I watched real travellers use it.

Deep dive

Eleven things we checked by hand

Device, browser, language, tax display and more

Once the developers had something unreleased, I worked through that sprint's scenarios by hand. This is where the scenario map and the scope diagrams paid off: without them, “did we test everything?” is a question for memory.

Alongside the scenarios, the team checked the build against each of these, wherever they applied. It is a long list for a small feature. That is what adding a payment choice to an existing booking funnel takes.

  • Device mobile, tablet, desktop
  • Browser Chrome, Safari, Firefox, Edge
  • Stay length a single night against several
  • Room configuration one room against several
  • Signed in against signed out
  • Private deals present, absent, and mixed
  • Availability available, sold out, price mismatch
  • Tax display excluded, as in the US, against included, as in the Netherlands
  • Payment breakdown prices carrying pay-at-property taxes and fees against pay-now-only prices
  • Currency converted from the provider against unconverted
  • Language English against German, which is longer and breaks layouts first
Deep dive

Five real sessions, recorded

Five videos

Once it was live, the question changed from whether it works to what people actually do with it. I used FullStory, which records real sessions and filters them by behaviour or by country. Every problem, bug or idea I found, I wrote down. Bugs slip through, and interfaces do strange things in some browsers. This is how you find out which.

Five sessions of travellers checking out with pay later, on different devices. Two are on side A of the test and three on side B.

Taiwan, desktop, A side

India, desktop, B side

Ohio, phablet, B side

Qatar, mobile, A side

Spain, tablet, B side

Conclusion

Results

+4.2%

Conversion on Expedia offers, the only offers in the test. It already counts the extra cancellations below, so it is what the feature was worth after them.

+3.3%

More cancellations on pay-later bookings than on pay-now ones, as a relative increase. We expected it: an unpaid booking is easier to walk away from. It is the cost of the gain above.

The search page problem from the start also shaped the result. It often hid Findhotel's own pay-later offers, so an unknown share of the travellers in this test never found the thing being tested.

So 4.2% is a floor, not the full measure. The feature was probably worth more, and we do not know how much more. The search page was fixed later, but the test never ran again. What the test gave us was a direction, and that was enough to justify the work that followed.

Deep dive

How the test was set up

Five weeks, 800 bookers a day, Expedia only

Our data scientist designed the test. Development ran from October to December 2020, the test went live in December, and the results came in January 2021.

Duration
Five weeks from December 2020, which put the test across the winter holidays in Europe and the Americas.
Audience
Four hundred bookers a day on each side, eight hundred a day in total, split evenly.
Room offers
Expedia inventory only. It was the one provider integrated at the time, and it bounds every number below.

Lessons

The checkout got harder, not easier
Checkout errors ran slightly higher for pay later than for pay now. Nobody measured by how much. The likely cause is cheap to fix: the button still said Confirm Payment when no payment was taken. The last words on the screen have to match the choice made two screens earlier.
A bug upstream can hide a win
The search page sat outside our team and outside the test, and it still shaped the result. If people cannot find the offer, no work on the offer converts them. When a test depends on another team's page, that page is part of the test design.
A first version buys the argument
Almost nothing I wanted to design made this release. It gave us a number instead, and the number made a second round possible. When several teams disagree about a feature, shipping something measurable beats shipping something good, because only the measurable one lets you go back.