Reverse Engineering Facebook.com's Homepage Design Optimizations

Facebook is known to run countless online experiments - most of which are kept secret. Because of their high traffic we can also assume that they have made numerous design decisions on tiny UI changes with immense statistical power (testing sensitivity). In other words, Facebook is a perfect source of patterns to be considered for imitation and all we need is to look at how their screens evolved over time. Luckily for us we have the Internet Archive that allows us to pry into these possible design optimizations. Here is my attempt at reverse engineering Facebook.com's homepage changes (signed out state) by comparing screenshots from 2012, 2015, and 2018:

The Changes

1. Headline Was Changed To Start With An Empowering Verb (See Similar Tests)
   The word "Facebook" has been moved from the beginning of the sentence towards the end. This places the reader (and what the reader can accomplish) at the center. Starting with an empowering verb is somewhat similar to our existing Empowering Headline pattern (based on 4 similar test results).

2. Abstract Image Was Replaced With Clearer Bullet Benefits (See Similar Tests)
   The global map of connected people icons might have been a little ambiguous. Eventually it was replaced with 3 bullets about the benefits of signing up to Facebook - possibly clearer. This particular change is also very similar to our Bulleted Reassurances pattern (also with matching a/b tests).

3. Flexible Phone or Email Signup Was Introduced
   The Facebook site evolved to allow signing up with either an email address or a phone number - clever. (We don't have a pattern for this yet, but this seems like a great candidate for testing).

4. Size Of Secondary Call To Action Headline Was Increased
   The right column's "Sign Up" headline gained an increased font size. Typically I always considered the standard page headline to deserve the largest font size - and it's interesting that this was broken here due to the headline being the call to action section.

5. Form Labels Were Placed Inside The Form Fields (See Similar Tests)
   Facebook shifted their form labels from left-alignment towards inline labels (inside the form fields). Although I suspect that left-aligned labels are not optimal, I'm surprised by the inline alignment. This is because we have a pattern for Top Aligned Labels (with 1 test in favor of top alignment).

6. Birthday Fields Were Shifted Higher
   The full set of birthday fields were shifted above the gender field.

7. Gender Options Were Exposed From A Pulldown Menu (See Similar Tests)
   The "male" and "female" options were shown more visibly by being exposed from within a pulldown menu (and we have quite a few tests on this change already).

8. Primary Button Size Increased
   The primary "Sign Up" button almost doubled in size for more visibility.

9. Additional Footer Links Were Added (See Similar Tests)
   The footer grew in size from additional links to other landing pages.

10. Keep Me Logged In Option Was Removed
    The checkbox option to remember a logged in person has been removed from the header. Perhaps lowering the barrier to personalized content for return visitors (by remembering everyone by default) may have increased overall engagement.

11. Duplicate Confirmation Fields Were Removed (See Similar Tests)
    The duplicate confirmation field asking to re-enter a phone or email, has been removed. I'm not surprised given the high reproducibility of this popular pattern across numerous tests.

12. Password Field Label Was Partially Unmasked
    In previous years, the password field (containing the label) appeared to be masked at all times. The newer version of the password field however has an unmasked blank state that allows to read the form label in the beginning. Only when users enter a desired password, does the masking activate. I;m somewhat surprised that they haven't implemented any additional usability suggestions such as the ones from Nielsen Norman.

13. Primary Button Shifted Higher (See Similar Tests)
    As a result of one fewer form field, the primary button gained the benefit of being positioned slightly higher up on the page.

How Confident Can We Be In These Observed Changes?

When extracting patterns from web sites in this way, without seeing actual a/b tests I would only attribute a neutral or balanced probability (maybe or maybe not having any impact). It's important to remind ourselves that these patterns may or may not have actually been tested at all. We are just guessing that they might have implemented as a result of experimentation with some good reasons to believe so. Additionally, if they were tested, some of these changes may even have been grouped together into a larger test with multiple changes. If that was the case then some of the changes might have had no or a negative impact with an overshadowing positive impact from the remaining changes. As you can see there is a lot of guess work here. Overall, I still do believe that such patterns are perfect candidates worthwhile of imitation in your own experiments.

How Confident Can We Be In Patterns That Are Backed By A/B Tests?

Interestingly, some of the patterns above actually overlap with our own patterns backed by real experiments. For these tested patterns containing direct evidence, our confidence actually shifts higher or lower depending on how many positive or negative experiments we have for a given pattern. Having access to A/B tests we can no longer maintain a neutral confidence (unless we turn ignorant of the past results starring at us).

Does It Make Sense To Imitate Others?

It's also interesting to see that many of the patterns that we do have positive data on, have also been implemented by Facebook. Seeing these implemented patterns that worked elsewhere, subtly validates this approach in imitation. When doing such reverse engineering however, the key element to respect is to make sure that a company has a mature experimentation culture in place - the more experiments they run, the higher the likelihood that a change was a result of rational experimentation.

Share Your Observations

Are there any other interesting changes between these screens that you think might have been by design (or more importantly by experimentation)? Please share your observations.