UX research and redesign of Travel Fairy, an Artificial-Intelligence powered travel application that helps users create personalized and complete trips in minutes.
Download Travel Fairy from the app store HERE.
Travel Fairy launched in late 2019 on the app store with a big marketing push. The app performed poorly, with very low retention metrics. Travel Fairy had no customer feedback platform, and insights available from behavioral data were not being used in product development. We supported Travel Fairy in three phases:
Phase 1: Design and conduct user experience research to help inform and drive product development. The goals of the research were:
- Evaluate the comprehension and valuation of the product by users
- Evaluate usability and create recommendations for improvements in user experience
Phase 2: Fix the usability issues found. Working collaboratively with Rove Media and Full Cycle Product Development, we redesigned the UX, with iterative usability testing at each stage.
Phase 3: Develop and test a new content strategy for Travel Fairy.
Mike Tokar (Product Owner, Unikoom)
Bryan Rill (UX Research & Design, Content Writer)
Eric Weiss (Product Strategist)
Matt Eeles and Zak Karst (UX/ UI Designers)
To meet Phase 1 goals, we developed a hybrid research design to provide actionable insights, creating a deep understanding of product users and delivering usability insights that helped drive product development. The hybrid methodology included generative methods to uncover motivations and find broad insights about users, and usability testing to evaluate the current product. Specific tools included:
- Amplitude for behavioral analytics captured in UX Dashboards
- SurveyMonkey for surveys and unmoderated usability tests
- Lookback.io for moderated usability tests
- Userlytics for Phase 2 feature testing (unmoderated usability tests)
- DScout for a diary study evaluating the product in context of use
For data analysis, we used Miro to collate insights from moderated and unmoderated tests. Using the newly released table feature, we developed a custom board that presented insights as cards within each stage of the user journey. The cards were linked directly to source data and to related observations, making it easy for project stakeholders to connect the dots and dive deeper into what users had to say.
We complimented the qualitative data with data visualizations from rated scales that measured task ease, confidence, satisfaction, and other salient metrics. The mix of qualitative and quantitative data enabled the team to easily see and prioritize usability issues in a facilitated insights workshop. From this, we developed a second Miro board that translated findings into prioritized tasks, which the team then used to roadmap UX improvements.
In Phase 2 we worked together in Miro and Adobe XD to define and design new UX flows and UI elements. To guide the process, we synthesized insights into a user journey/ problem statements matrix that gave us specific UX targets.
In a series of moderated usability tests in Lookback, we tested initial solution ideas at mid-fidelity to inform design decisions. The fidelity was then raised for final designs, which we tested in Userlytics- a platform that enabled us to rapidly test design iterations and collect robust data to drive development decisions. The slider below shows our process for one feature, from a wireframe in Miro that started the dialogue, to its development in Adobe Xd and the prototype screens of one flow.
A wireframe designed to start a conversation about UX
New UX ideas wireframed in Adobe XD
Itinerary for the day
In phase 3, we worked on developing and testing a new content strategy for Travel Fairy. Working closely with the team, we developed and tested a series of icons and narratives for the app store and website. We used PickFu to collect and analyze data.
Phase I research revealed a host of usability issues. We knew we had to fix the UX of certain features, but more importantly, we discovered something about users that would impact everything else we worked on. Users wanted a personalized interaction with the AI…they wanted Lucy (our avatar) to get to know them well enough to trust her recommendations and manage their trips. While her presence was strong in the onboarding stage of the user journey, she had little role thereafter. We had to fix that, developing customized, authentic feeling interactions that would build and maintain trust throughout the user experience. If we could create that bond, then Travel Fairy would be a truly unique assistant.
The desire for more interaction enabled a new perspective on the help feature, which user testing revealed to be a central need. While the AI recommendation engine is robust, users did not understand the logic of what they were being presented with. The lack of an intuitive interface when editing a trip was causing critical errors, which helped explain the low retention we were seeing in the analytics dashboard. While we did redesign the UI to be more intuitive, we used the desire for interaction with Lucy as an excuse for her to pop up in a chatbot and explain the interface to first time users. Once they got past that hurdle, critical errors sharply dropped. At the same time, we created a new bonding experience with Lucy shepherding the user through the interface. Win win.
Developing a new UX to resolve the edit feature led to our greatest problem, an “existential crisis” centered on choice. We found ourselves faced with the question, “What is the right balance between freedom and constraint?” The app was initially designed to limit choice, giving users a small set of options to choose from when replacing an event. But users overwhelmingly asked for more control. “How much? And how do we deliver it?” became central design questions. The hypotheses to test were:
- Users want a large degree of control over their itinerary, and prefer to choose from a list of options
- Users don’t want to work and would prefer to choose from recommendations.
To solve this crisis, we developed a UX prototype test design specifically to address this question, with two different flows that represent each user orientation.
The results, overwhelmingly, came back as “both and.” Users felt that, if the AI engine was good enough, they would first like to see its recommendations to make their choice as simple as possible. Yet they rejected the idea of being forced to choose from only those options. Users wanted control over choice, and the ability to see beyond the AI engine’s recommendations. These findings contradicted the original design logic of the app, which assumed that people were lazy and did not want to spent the time making their own decisions. With travel, it seems, most people are willing and want to spend the extra effort to carefully pick among alternatives, but that lazy side of us (do it for me) is the starting point of decision making.
The end result was an experience designed to give choice, allowing the user to decide between Lucy’s recommendations and a list of nearby options they can choose from. Problem solved.
Phase 1 results were (1) a user-centered perspective on how well Travel Fairy delivers on its promise, and 2) a UX dashboard with metrics and benchmarks to help further development.
Specific deliverables included:
- Virtual assumptions workshop to help define the research questions and hypothesis by doing a gap analysis of knowns, unknowns, and assumptions.
- Personas and User Journey Maps
- Iterative Usability Testing Protocols
- UX Research Findings Report
- Insights Summary Reports
- UX Dashboard, using the HEART framework developed by Google to connect metrics to specific business objectives. Success, in business terms, was defined in terms of:
- Number of Active users
- Planning Efficiency
- Planning Ease/ Satisfaction
- Trip Satisfaction Rating (Happiness)
We co-designed metrics for each, then build the dashboard in Amplitude.
Phase 2 results were UX improvements, iterative testing reports, and synapses.
Phase 3 results were a content playbook, new copy for the app store, and icon testing reports.