AR Mobile APP


A better way to find your own lipstick


MacBook Pro 14_ - 2.png

My Roles and Responsibilities

Team Members

  • UX Research

  • User Flows

  • Ideation, Brainstorming and Sketching

  • Wireframing and Prototyping

  • User Testing

Elena Gonzales Melinger, Yu-Hsuan Liu, Ai Sano, Pei-Syuan Jhang


February 2022 - June 2022

Problem Statement

Covid-19 has supercharged all things virtual, propelling industries like retail well into the future. According to IBM’s 2020 U.S. Retail Index report, the pandemic has accelerated the shift to digital shopping by roughly five years.


(Eliana Rodgers/For The Washington Post)


Throughout the pandemic, virtual try-on has offered a sweet sense of relief to fashion and beauty brands unable to demo their products offline. The “try-on” technology allows users to see how products look on them before purchase. Nonetheless, searching for some color related products are still always a problem for users

#About the product

New lipstick engine

The app provides three innovative and effective ways for people to search a lipstick that meets their expectations.


Search by color palette/ color family


Pick a color from any image


Try-on the lipstick with AR

Project Journey




The aim of the survey is to dig into people’s habits when it comes to buying makeup online. We were interested in which products people were most likely to purchase online, and what pain points they could identify in choosing a product that suited them online.


This survey was also used as a recruiting tool to identify candidates who shopped for lipstick online for the next phase of our early research: the user session.

We had 85 respondents to our survey. After screening out those who didn’t buy makeup, we had 74 potential users responding to our survey. 

Can we build a lipstick searching engine that improves the experience we collected from the survey?
Key insight 

User Session

We then asked ourselves "what sort of behaviors do our target users engage in with these existing products to meet their needs while shopping for lipstick?". To answer the question, we decided to conduct a user session. In the session, we asked the participants to perform tasks on searching engines and makeup try-on APPs. Our goal was to collect user feedback for our first design.

Research Methodology ​

No. of users interviewed:

  • 10 qualified participants from the survey​​.

Task: ​

  • Searching for lipsticks - Mac & Sephora.

  • Virtual makeup try-on -  YouCam Makeup & Makeup Plus APP.


  • The interview took place in person or over call. 

  • Each interview lasted for 45 mins- 1 hour. 

  • The audio was recorded and notes were taken.

Key insight 

Searching for lipsticks

For users, photos for a makeup/lipstick searching engine is vitally important. Besides, the users mentioned the following aspects of the product as important: color, texture, brand, ratings, finish, and formulation. 


Users were most focused on lip swatches and arm swatches - photos that integrate the lip color with a skin tone to contextualize it. Users expressed disappointment if their skin tone was not represented in the product photos. 


Users had a slight preference for Sephora’s approach over Mac’s approach. Sephora caters to these needs by designing a search results page with many filters, and Mac capitalizes on only selling their own products and shows pictures of the lipstick applied.


The number of photos available played a role - Participants mentioned that too few photos available would affect her decision to buy a product. 

Virtual makeup try-on

The average rating of willingness to use the try-on tool before buying was 4.45 of 5. Results showed that users would like to use this mainly because the AR filter effects look great and real on their faces, making them can see how the lipstick colors look on their skin color. For most lipstick wearers, finding the exact same skin tone as them to refer to when shopping for lipsticks online is not easy, and this tool helps them address this long-standing problem by the AR filter. 

Users preferred the selfie try-on path to the photo-uploading path because the latter was more like a photo editing app than a try-on app. Also, the looks generated from the photo-uploading were not as natural and real as when using the selfie path.


Of all the comments and suggestions, the most mentioned are “the icons are hard to recognize, ” “don't like it uses a default skin-beautifying filter when trying on lipstick,” “inconsistency on features between selfie and upload,” and “does not well-represent the finish of lipsticks.”


When users tried on lipsticks with the AR makeup app, many of them tended to try on every color. The reasons why they tended to try on more colors with the app are, first, because they felt it was easy and fun; second, it can help them explore more different colors without applying and removing makeup physically.

User Flow

Our user flow describes the steps taken by a typical user on the app to complete a task. As the user enters, they progress through a set of steps towards a successful outcome and final action, such as going to the brand official website.



Sketches & Wireframes

We quickly drew out sketches of these functionalities on MIRO to make sure that everyone's on the same page.


We then created the following wireframes and the blueprint to our potential users for feedback.


User Feedback

All users liked the customized lipsticks recommendations and the function to directly look up product information. This was reassuring for us, since the customization for users is our primary focus, and what distinguishes us from existing apps.

Minimize learning
  • Make the design of “selfie” trying on and ”photo-uploading” trying on page consistent, avoiding users learning new things and features again.

  • Icon or UI design for a feature may be unrecognizable to users, we can consider providing the first-time-use tutorial page to teach users what it is and how it works.

Consider users' environments
  • Be mindful of the context people are in when shopping for makeup on their phone - often in bad light, no makeup, or pajamas.

  • It's better to add a blurring/softening filter to the default AR try-on feature to be more flattering, just like Chanel's site.

  • Providing more product details and after-use information can help support the effect of AR try-on feature, making users feel more confident in buying.

Make important things first and clear
  • ​​Lipstick wearers care about the brand of lipstick, colors, finishes, style of the product, and feel on the lips. Be sure to provide a brand search path for them, in addition to ability to search by color (main functionality).

Mood board & Style tile

Our decision to employ beige as our brand color was mainly due to that the color can evoke a mood and aesthetics that is luxury and elegant, yet simple and neutral. We believe that the color’s unimposing nature will work well for our product where its majority of space will be dedicated to images such as lip product photos and color swatches, which can be quite colorful.

Mood board.png
Style tile.png

High-Fidelity Design - Iteration 1

For the first iteration, our primary goal is to validate whether the design helps users be more confident on selecting lipsticks and feel comfortable using AR functions.

On boarding.png
Try on.png
Color Picker.png


After our initial round of testing on our wireframe for baseline feedback on our core functions and pain points. For this phase, we did 2 rounds of usability testing on the same hi-fi prototype - one round was moderated and the other was unmoderated. The testings aim to (1) validate our prototype, (2) gather unbiased user opinions and (3) get the insights that help create a better overall user experience.

Moderated Usability Testing

User Testing Process

No. of users interviewed : 10

Here's a brief overview of the testing process:

  • Introduce the project goal, and put users at ease.

  • Describe the scenario, set up the context, and let users use the app while thinking aloud.

  • Ask follow up questions and gather feedback.

  • Let users fill out a questionnaire. Thank them for participation.


Account Creation


Try on (home page)


Add a lipstick to favorites


Try on (product page)


Write a review


Use the color picker

Unmoderated Usability Testing 

This unmoderated test took place at the same time and on the same prototype as the moderated usability testing. For this test design, we were interested in first impressions, navigability, and feedback on our core features from users, so the questions we asked were slightly different from the moderated testing.

No. of users : 6


First Impression


Search by color


Try on from selfie


Try on from the image

Screen Shot 2022-05-03 at 16.21.31.png
Screen Shot 2022-05-03 at 16.22.33.png
Quantitative Result allowed us to gain some quantitative insight on how people interacted with our app. 


We collected Net Promoter Scores as a way of benchmarking this iteration of our app with user’s impressions. For our first hi-fi, we tested with 6 users in an unmoderated test. We had a Net Promoter Score of 50%, with 3 promoters and 3 neutrals. Net Promoter Score is more useful with a larger sample size, but it was still good feedback as we continue to improve.


2 of our participants had significantly longer test times than others. However, when reviewing the footage, the length of P6’s session was due to technical issues and load time on her computer, and the length of P5’s session was because she was enjoying the app and playing around very thoroughly on all the features. P5, with the longest task time, was mostly positive throughout her session.

User Feedback & Design Iterations
Simplified unnecessary steps

Since the following flows of clicking two buttons are the same, we combined the flows together.


Users expected to see the camera immediately after clicking, and not the color family selection. By going straight to camera, we can get rid of an extra click. Users could still navigate to their preferred color family through the color family menu on the camera page.

Increased the visibility of the photo

Added "Automatically zoom in according to the lip position". Some people didn’t notice when the lip color was changing because the photo was zoomed out too much.

Reassessed the "Go back" user flow

Allows another color selection instead of jumping to the homepage.

Latest Prototype

Finding lipsticks by
selecting a color in an image
Finding lipsticks by
picking a color family
AR Lipstick try on filter
My Favorites

My Learning

Challenge for camera interface design

There are unique challenges in camera design that pushed me to think more about how to display the prototype. For instance, a camera interface requires thorough testing and could be rather hard to communicate without a proper visualization.

Challenge for AR application design


An immersive design’s UI is tricky since we don’t want to take users out of an experience. So, how do we design an interface that maintains immersion is crucial. During the whole process, we always asked ourselves" How do you make something like augmented reality feel real all the way through?" These are questions without easy answers, but they’re questions that we must ask ourselves. 


My first unmoderated testing experience


For this project, I had my first unmoderated testing experience. I found that unmoderated studies do not include any direct interaction between the researcher and the study participants, which is both their biggest benefit and their greatest drawback. First, it can always be the hi-hi prototyping testing since early-prototype testing is difficult without a moderator to explain. Second, the understandable design is very important or participants tend to do tasks that depend on imagination.

See my other projects