Discover UI Design Decisions Of Leading Companies,
From Leaked Screenshots & A/B Tests.

Zalando (Germany) has been experimenting with at least two interesting cascade variations on their product page. Both of these variations seem to have been rejected which is consistent with other evidence in favor of larger buttons.  View Leak

Netflix ran an experiment with two variations of appended secondary links underneath the main "Try It Now" button. One variation had a link urging users to sign-in, while another urged to resume signing-up. Both ideas didn't make it and have been rejected. Here are my thoughts and comparisons on this ... View Leak

What's more important first - search results or recommended alternative questions based on what other users ask? It's now clear that Google ran such an experiment on their search results with a clear decision in favor of the former (showing real search results first). View Leak

If you might recall, some of the recent Netflix homepage experiments weren't so positive as seen here and here. Today however we finally detected what might be considered a successful roll out decision of one variation tested over a month ago. Gladly Netflix doesn't give up. Here are the 3 core changes which we think might have influenced a positive outcome. View Leak

Netflix has been offering a 30 day free trial for Canadians. Sometime in May however we spotted something way more interesting - they began a/b testing different trial durations (30, 14 and 7 days), as well as an immediate payment-first approach (without a trial). View Leak

It seems like Booking just completed running an experiment that is very similar to a recent one from Google. The experiment evaluated icons with labels in the top navigation against a control version with labels only.  View Leak

Perhaps it's no surprise that when people wish to book a getaway, expressing a location is of higher importance than anything else. Interestingly, this has been confirmed in a recent Booking experiment which tested the position of various elements on their homepage. The A/B test, along with its implementation decision, revealed that shifting the location tiles higher up seems to have performed better. View Leak

Take a look at these two screenshots, both taken on May 15, 2019. With some cookie clearing magic in between it's pretty clear that Google ran an icon experiment on their search results page. The control version had labels only in the top nav, whereas the variant contained icons in addition to the labels. The variation was implemented a month later in June. View Leak

Here is a perfectly simple optimization of a "Get started" button on Airbnb's host signup landing page. Airbnb ran an experiment of a smaller vs larger button size. I know because I managed to capture two diverse screenshots with the same date stamp. :) More so, a few months later Airbnb rolled out the later button to 100% of their traffic - hinting at a successful experiment outcome. View Leak

Videos are usually considered a popular growth tactic that people believe will improve conversion or signup rates. Hence I find it really interesting that Airbnb A/B tested a series of videos on their host signup landing page and then rejected them altogether. So why where these video removed? View Leak

I've been watching this Booking experiment closely ever since sharing a very similar concept some months ago. Their homepage was openly challenged with the UI hypothesis of exposing a "room quantity" field right in the search bar (instead of hiding it in a pulldown menu). And their team took the initiative to run a test. Based on the observed outcome and roll out decision it turns out that the UI concept was better than their control. View Leak

Based on two screenshots from 2018 and 2019, we noticed at least three interesting ways in how the property screen of Airbnb continues to evolve. Here are the changes we found: View Leak

Netflix has just finished testing at least four interesting button labels on their homepage. Since one of them looks like it has been implemented, we just might be able to make a few subtle comparisons which I'll attempt to capture in this leak. Of course, given the fact that we're just scratching the surface of this experiment, please take this analysis as highly hypothetical (ripe for further experimentation). View Leak

This leaked experiment captures a very simple and isolated change on Booking's apartment landing page. The tested change was the automatic exposure of the calendar menu instead of keeping it collapsed (requiring an extra click to open the date picker). A few weeks later, the experiment was observed to have been completed with the exposed calendar version (B) as implemented. View Leak

Netflix has been yet again noticed experimenting with a range of variations on their homepage - all of which yet again seem to have been rejected as before. Below is the leaked experiment with its key changes, the decision, as well as our own followup experiment recommendation (if we had the privilege of doing so).  View Leak

It's now clear that Booking ran somewhat of a larger leap experiment of two very diverse search layouts on their Apartments landing page. I'm glad we discovered this one as we now know for sure their awesome team doesn't only run microscopic single change experiments. View Leak