A Short Anecdote About WWII Bombers

Maybe you’ve heard this story cited before:  During World War II, researcher Abraham Wald was tasked with determining how to maximize the number of airplanes that were able to return safely home despite taking enemy fire. Common thinking was to add additional armor to the areas where returning planes had taken the most damage. Wald, however, recognized that if those planes came back, it meant that they were able to sustain that damage and still fly home. He was concerned with the planes that didn’t come back — and so he proposed reinforcing areas where the returning aircraft were not damaged. This change led to a massive uptick in the number of planes returning safely. Per Wikipedia, “His work is considered seminal in the then-nascent discipline of operational research.”

I can’t help but think of that story any time we’re watching user sessions as part of UX research or digital marketing strategy. There is so much that the analytics don’t pick up, and it could lead to the wrong conclusion or at least the wrong prioritization of UX and content optimization tasks. At this point, we recommend session recording (using tools like FullStory or Microsoft Clarity) as a part of any UX design or website optimization engagement.

Insights From Session Recordings That Aren’t Visible in Analytics

Here are a few things you can see from session recordings that aren’t immediately apparent from analytics:

Dead Clicks

Dead clicks happen when a user thinks something is a link, but it’s not. They try to click, maybe even multiple times, but nothing happens. The user may become frustrated and may or may not continue to search for alternative ways to reach their destination. But if not, your bounce rate just went up and you may have lost a conversion.

Frustrating UI Interactions

Analytics will tell you if a user reached a given page or performed a particular action, but it might not tell you that they had to struggle to use your megamenu, or that the link they tried to click “jumped” away from them because an image loaded late (leading them to have to re-find it on the page). Core Web Vitals can give some insight here, but session recordings will allow you to almost immediately identify friction in your UI.

Missed Calls to Action

We used session recording to analyze a web portal application used by 30,000 visitors per month to pay their automotive leases. The goal was to decrease customer service calls from folks who were having difficulty using the portal.

Just one example from our research: Even though the “Pay Now” button was clearly at the top of the dashboard, users would immediately scroll* down to the bottom of the page, then slowly back up. They ultimately found their way, but clearly had been scanning in search of the “Pay now” button. Ultimately, we added it to both the top and bottom of the screen.

Bots, Bots, and More Bots

Looking at website analytics, bot visits can be masked as being like any other visit. Unless they’re filtered out, they’ll count toward your page views, bounce rate, and can even impact your conversion goals.

Bots are getting trickier and tricker, and they’re using “real” browser sessions to mimic mouse clicks and work around other prevention measures.

We were recently optimizing a PPC campaign that was supposed to be targeting the U.S. only. We saw that a significant portion of the PPC-driven sessions were not only coming from outside the U.S., but were incredibly short visits (less than 5 seconds) with no interactions. After reviewing the sessions, we immediately updated the PPC sessions to explicitly exclude non-U.S. countries.

Conclusion

Session recordings provide valuable insight that can help inform your UX research and UX design strategy. They add context to the analytics data and can help you make far better decisions about how to optimize your site.