AR Mobile APP
A better way to find your own lipstick
My Roles and Responsibilities
UX Designer / UX Researcher
Collaborated with teammates to go through all parts from research, ideation to design and iteration
Conducted user research and usability testing process
Produced all design outputs including site maps, user flows, style guide and hi-fi prototypes
Elena Gonzales Melinger, Yu-Hsuan Liu, Ai Sano, Pei-Syuan Jhang
February 2022 - June 2022
Covid-19 has supercharged all things virtual, propelling industries like retail well into the future. According to IBM’s 2020 U.S. Retail Index report, the pandemic has accelerated the shift to digital shopping by roughly five years.
(Eliana Rodgers/For The Washington Post)
Virtual try-on technology has been a game-changer for fashion and beauty brands during the pandemic, allowing users to preview products before purchase. However, some users still encounter issues when searching for color-related products. This highlights the need for ongoing improvement and development of virtual try-on tools.
#About the product
New lipstick engine
The app provides three innovative and effective ways for people to search a lipstick that meets their expectations.
Search by color palette/ color family
Pick a color from any image
Try-on the lipstick with AR
Our survey aims to understand consumer behavior when it comes to buying makeup online. We sought to identify the most commonly purchased products online and the pain points consumers face when selecting products that suit them. In addition, the survey served as a recruitment tool for the next phase of our early research, which involved user sessions specifically with online lipstick shoppers.
We had 85 respondents to our survey. After screening out those who didn’t buy makeup, we had 74 potential users responding to our survey.
Can we build a lipstick searching engine that improves the experience we collected from the survey?
We then asked ourselves "what sort of behaviors do our target users engage in with these existing products to meet their needs while shopping for lipstick?". To answer the question, we decided to conduct a user session. In the session, we asked the participants to perform tasks on searching engines and makeup try-on APPs. Our goal was to collect user feedback for our first design.
No. of users interviewed:
10 qualified participants from the survey.
Searching for lipsticks - Mac & Sephora.
Virtual makeup try-on - YouCam Makeup & Makeup Plus APP.
The interview took place in person or over call.
Each interview lasted for 45 mins- 1 hour.
The audio was recorded and notes were taken.
Searching for lipsticks
Users place high importance on product photos when using a makeup or lipstick search engine. In addition, users identified several other factors that are crucial in their decision-making process, including color, texture, brand, ratings, finish, and formulation. By considering these aspects, makeup and lipstick search engines can better meet the needs and preferences of the users.
Users were most focused on lip swatches and arm swatches - photos that integrate the lip color with a skin tone to contextualize it. Users expressed disappointment if their skin tone was not represented in the product photos.
Users had a slight preference for Sephora’s approach over Mac’s approach. Sephora caters to these needs by designing a search results page with many filters, and Mac capitalizes on only selling their own products and shows pictures of the lipstick applied.
The number of photos available played a role - Participants mentioned that too few photos available would affect her decision to buy a product.
Virtual makeup try-on
Users rated the try-on tool highly (4.45 out of 5) for its realistic AR filter effects, which help them see how lipstick colors look on their skin tone. This addresses a longstanding problem for lipstick wearers who struggle to find the right shade online.
Users preferred the selfie try-on path to the photo-uploading path because the latter was more like a photo editing app than a try-on app. Also, the looks generated from the photo-uploading were not as natural and real as when using the selfie path.
Of all the comments and suggestions, the most mentioned are “the icons are hard to recognize, ” “don't like it uses a default skin-beautifying filter when trying on lipstick,” “inconsistency on features between selfie and upload,” and “does not well-represent the finish of lipsticks.”
When users tried on lipsticks with the AR makeup app, many of them tended to try on every color. The reasons why they tended to try on more colors with the app are, first, because they felt it was easy and fun; second, it can help them explore more different colors without applying and removing makeup physically.
The user flow in our app maps out the key steps that users take to achieve their desired task. Through a carefully planned sequence of actions, users are guided towards a successful outcome, which could be anything from visiting the brand's official website to completing a purchase. By analyzing the flow, we identify areas for improvement and create a better user experience, enhancing engagement.
Sketches & Wireframes
We quickly drew out sketches of these functionalities on MIRO to make sure that everyone's on the same page.
We then created the following wireframes and the blueprint to our potential users for feedback.
All users liked the customized lipsticks recommendations and the function to directly look up product information. This was reassuring for us, since the customization for users is our primary focus, and what distinguishes us from existing apps.
Make the design of “selfie” trying on and ”photo-uploading” trying on page consistent, avoiding users learning new things and features again.
Icon or UI design for a feature may be unrecognizable to users, we can consider providing the first-time-use tutorial page to teach users what it is and how it works.
Consider users' environments
Be mindful of the context people are in when shopping for makeup on their phone - often in bad light, no makeup, or pajamas.
It's better to add a blurring/softening filter to the default AR try-on feature to be more flattering, just like Chanel's site.
Providing more product details and after-use information can help support the effect of AR try-on feature, making users feel more confident in buying.
Make important things first and clear
Lipstick wearers care about the brand of lipstick, colors, finishes, style of the product, and feel on the lips. Be sure to provide a brand search path for them, in addition to ability to search by color (main functionality).
Mood board & Style tile
We chose beige as our brand color because it creates a luxurious, elegant, and neutral mood that complements our lip product images and color swatches. This non-intrusive background helps our visuals stand out and create a cohesive, sophisticated brand identity for our target audience.
High-Fidelity Design - Iteration 1
For the first iteration, our primary goal is to validate whether the design helps users be more confident in selecting lipsticks and feel comfortable using AR functions.
SELECT COLOR FAMILY
SEARCH BY PICKING A COLOR
WRITE A REVIEW
EVALUATION & ITERATIONS
After our initial round of testing on our wireframe for baseline feedback on our core functions and pain points. For this phase, we did 2 rounds of usability testing on the same hi-fi prototype - one round was moderated and the other was unmoderated. The testings aim to (1) validate our prototype, (2) gather unbiased user opinions and (3) get the insights that help create a better overall user experience.
Moderated Usability Testing
User Testing Process
No. of users interviewed : 10
Here's a brief overview of the testing process:
Introduce the project goal, and put users at ease.
Describe the scenario, set up the context, and let users use the app while thinking aloud.
Ask follow up questions and gather feedback.
Let users fill out a questionnaire. Thank them for participation.
Try on (home page)
Add a lipstick to favorites
Try on (product page)
Write a review
Use the color picker
Unmoderated Usability Testing
This unmoderated test took place at the same time and on the same prototype as the moderated usability testing. For this test design, we were interested in first impressions, navigability, and feedback on our core features from users, so the questions we asked were slightly different from the moderated testing.
No. of users : 6
Search by color
Try on from selfie
Try on from the image
UserTesting.com allowed us to gain some quantitative insight on how people interacted with our app.
To benchmark this iteration of our app and gauge user impressions, we collected Net Promoter Scores. During our first hi-fi test, we had six unmoderated users, resulting in a Net Promoter Score of 50%. Of the six users, three were promoters, and three were neutrals. While a larger sample size would be more valuable, this feedback still provided helpful insights as we continue to improve the app.
Two of our participants had notably longer test times than others. Upon reviewing the footage, we discovered that the extended test time for P6 was due to technical issues and long load times on their computer. Conversely, P5 spent an extended amount of time on the app due to thoroughly enjoying the features and playing around with them. Despite having the longest task time, P5's feedback remained predominantly positive throughout the session.
User Feedback & Design Iterations
Simplified unnecessary steps
Since the following flows of clicking two buttons are the same, we combined the flows together.
Users expected to see the camera immediately after clicking, and not the color family selection. By going straight to camera, we can get rid of an extra click. Users could still navigate to their preferred color family through the color family menu on the camera page.
Increased the visibility of the photo
Added "Automatically zoom in according to the lip position". Some people didn’t notice when the lip color was changing because the photo was zoomed out too much.
Reassessed the "Go back" user flow
Allows another color selection instead of jumping to the homepage.
Finding lipsticks by
selecting a color in an image
AR Lipstick try on filter
Finding lipsticks by
picking a color family
Challenge for camera interface design
There are unique challenges in camera design that pushed me to think more about how to display the prototype. For instance, a camera interface requires thorough testing and could be rather hard to communicate without a proper visualization.
Challenge for AR application design
An immersive design’s UI is tricky since we don’t want to take users out of an experience. So, how do we design an interface that maintains immersion is crucial. During the whole process, we always asked ourselves" How do you make something like augmented reality feel real all the way through?" These are questions without easy answers, but they’re questions that we must ask ourselves.
My first unmoderated testing experience
For this project, I had my first unmoderated testing experience. I found that unmoderated studies do not include any direct interaction between the researcher and the study participants, which is both their biggest benefit and their greatest drawback. First, it can always be the hi-hi prototyping testing since early-prototype testing is difficult without a moderator to explain. Second, the understandable design is very important or participants tend to do tasks that depend on imagination.
See my other projects