top of page

AR Mobile APP


A better way to find your own lipstick


MacBook Pro 14_ - 2.png

My Roles and Responsibilities

Team Members


UX Designer / UX Researcher 

  • Collaborated with teammates to go through all parts from research, ideation to design and iteration

  • Conducted user research and usability testing process

  • Produced all design outputs including site maps, user flows, style guide and hi-fi prototypes 

Elena Gonzales Melinger, Yu-Hsuan Liu, Ai Sano, Pei-Syuan Jhang


February 2022 - June 2022

Problem Statement

Covid-19 has supercharged all things virtual, propelling industries like retail well into the future. According to IBM’s 2020 U.S. Retail Index report, the pandemic has accelerated the shift to digital shopping by roughly five years.


(Eliana Rodgers/For The Washington Post)


Virtual try-on technology has been a game-changer for fashion and beauty brands during the pandemic, allowing users to preview products before purchase. However, some users still encounter issues when searching for color-related products. This highlights the need for ongoing improvement and development of virtual try-on tools.

#About the product

New lipstick engine

The app provides three innovative and effective ways for people to search a lipstick that meets their expectations.


Search by color palette/ color family


Pick a color from any image


Try-on the lipstick with AR

Project Journey




Our survey aims to understand consumer behavior when it comes to buying makeup online. We sought to identify the most commonly purchased products online and the pain points consumers face when selecting products that suit them. In addition, the survey served as a recruitment tool for the next phase of our early research, which involved user sessions specifically with online lipstick shoppers.

We had 85 respondents to our survey. After screening out those who didn’t buy makeup, we had 74 potential users responding to our survey. 

Can we build a lipstick searching engine that improves the experience we collected from the survey?
Key insight 

User Session

We then asked ourselves "what sort of behaviors do our target users engage in with these existing products to meet their needs while shopping for lipstick?". To answer the question, we decided to conduct a user session. In the session, we asked the participants to perform tasks on searching engines and makeup try-on APPs. Our goal was to collect user feedback for our first design.

Research Methodology ​

No. of users interviewed:

  • 10 qualified participants from the survey​​.

Task: ​

  • Searching for lipsticks - Mac & Sephora.

  • Virtual makeup try-on -  YouCam Makeup & Makeup Plus APP.


  • The interview took place in person or over call. 

  • Each interview lasted for 45 mins- 1 hour. 

  • The audio was recorded and notes were taken.

Key insight 
Searching for lipsticks

Users place high importance on product photos when using a makeup or lipstick search engine. In addition, users identified several other factors that are crucial in their decision-making process, including color, texture, brand, ratings, finish, and formulation. By considering these aspects, makeup and lipstick search engines can better meet the needs and preferences of the users.


Users were most focused on lip swatches and arm swatches - photos that integrate the lip color with a skin tone to contextualize it. Users expressed disappointment if their skin tone was not represented in the product photos. 


Users had a slight preference for Sephora’s approach over Mac’s approach. Sephora caters to these needs by designing a search results page with many filters, and Mac capitalizes on only selling their own products and shows pictures of the lipstick applied.


The number of photos available played a role - Participants mentioned that too few photos available would affect her decision to buy a product. 

Virtual makeup try-on

Users rated the try-on tool highly (4.45 out of 5) for its realistic AR filter effects, which help them see how lipstick colors look on their skin tone. This addresses a longstanding problem for lipstick wearers who struggle to find the right shade online.

Users preferred the selfie try-on path to the photo-uploading path because the latter was more like a photo editing app than a try-on app. Also, the looks generated from the photo-uploading were not as natural and real as when using the selfie path.


Of all the comments and suggestions, the most mentioned are “the icons are hard to recognize, ” “don't like it uses a default skin-beautifying filter when trying on lipstick,” “inconsistency on features between selfie and upload,” and “does not well-represent the finish of lipsticks.”


When users tried on lipsticks with the AR makeup app, many of them tended to try on every color. The reasons why they tended to try on more colors with the app are, first, because they felt it was easy and fun; second, it can help them explore more different colors without applying and removing makeup physically.

User Flow

The user flow in our app maps out the key steps that users take to achieve their desired task. Through a carefully planned sequence of actions, users are guided towards a successful outcome, which could be anything from visiting the brand's official website to completing a purchase. By analyzing the flow, we identify areas for improvement and create a better user experience, enhancing engagement.



Sketches & Wireframes

We quickly drew out sketches of these functionalities on MIRO to make sure that everyone's on the same page.


We then created the following wireframes and the blueprint to our potential users for feedback.


User Feedback

All users liked the customized lipsticks recommendations and the function to directly look up product information. This was reassuring for us, since the customization for users is our primary focus, and what distinguishes us from existing apps.

Minimize learning
  • Make the design of “selfie” trying on and ”photo-uploading” trying on page consistent, avoiding users learning new things and features again.

  • Icon or UI design for a feature may be unrecognizable to users, we can consider providing the first-time-use tutorial page to teach users what it is and how it works.

Consider users' environments
  • Be mindful of the context people are in when shopping for makeup on their phone - often in bad light, no makeup, or pajamas.

  • It's better to add a blurring/softening filter to the default AR try-on feature to be more flattering, just like Chanel's site.

  • Providing more product details and after-use information can help support the effect of AR try-on feature, making users feel more confident in buying.

Make important things first and clear
  • ​​Lipstick wearers care about the brand of lipstick, colors, finishes, style of the product, and feel on the lips. Be sure to provide a brand search path for them, in addition to ability to search by color (main functionality).

Mood board & Style tile

We chose beige as our brand color because it creates a luxurious, elegant, and neutral mood that complements our lip product images and color swatches. This non-intrusive background helps our visuals stand out and create a cohesive, sophisticated brand identity for our target audience.

Mood board.png
Style tile.png

High-Fidelity Design - Iteration 1

For the first iteration, our primary goal is to validate whether the design helps users be more confident in selecting lipsticks and feel comfortable using AR functions.

On boarding.png
Try on.png
Color Picker.png


After our initial round of testing on our wireframe for baseline feedback on our core functions and pain points. For this phase, we did 2 rounds of usability testing on the same hi-fi prototype - one round was moderated and the other was unmoderated. The testings aim to (1) validate our prototype, (2) gather unbiased user opinions and (3) get the insights that help create a better overall user experience.

Moderated Usability Testing

User Testing Process

No. of users interviewed : 10

Here's a brief overview of the testing process:

  • Introduce the project goal, and put users at ease.

  • Describe the scenario, set up the context, and let users use the app while thinking aloud.

  • Ask follow up questions and gather feedback.

  • Let users fill out a questionnaire. Thank them for participation.


Account Creation


Try on (home page)


Add a lipstick to favorites


Try on (product page)


Write a review


Use the color picker

Unmoderated Usability Testing 

This unmoderated test took place at the same time and on the same prototype as the moderated usability testing. For this test design, we were interested in first impressions, navigability, and feedback on our core features from users, so the questions we asked were slightly different from the moderated testing.

No. of users : 6


First Impression


Search by color


Try on from selfie


Try on from the image

Screen Shot 2022-05-03 at 16.21.31.png
Screen Shot 2022-05-03 at 16.22.33.png
Quantitative Result allowed us to gain some quantitative insight on how people interacted with our app. 


To benchmark this iteration of our app and gauge user impressions, we collected Net Promoter Scores. During our first hi-fi test, we had six unmoderated users, resulting in a Net Promoter Score of 50%. Of the six users, three were promoters, and three were neutrals. While a larger sample size would be more valuable, this feedback still provided helpful insights as we continue to improve the app.


Two of our participants had notably longer test times than others. Upon reviewing the footage, we discovered that the extended test time for P6 was due to technical issues and long load times on their computer. Conversely, P5 spent an extended amount of time on the app due to thoroughly enjoying the features and playing around with them. Despite having the longest task time, P5's feedback remained predominantly positive throughout the session.

User Feedback & Design Iterations
Simplified unnecessary steps

Since the following flows of clicking two buttons are the same, we combined the flows together.


Users expected to see the camera immediately after clicking, and not the color family selection. By going straight to camera, we can get rid of an extra click. Users could still navigate to their preferred color family through the color family menu on the camera page.

Increased the visibility of the photo

Added "Automatically zoom in according to the lip position". Some people didn’t notice when the lip color was changing because the photo was zoomed out too much.

Reassessed the "Go back" user flow

Allows another color selection instead of jumping to the homepage.

Latest Prototype

Finding lipsticks by
selecting a color in an image
AR Lipstick try on filter
Finding lipsticks by
picking a color family
My Favorites

My Learning

Challenge for camera interface design

There are unique challenges in camera design that pushed me to think more about how to display the prototype. For instance, a camera interface requires thorough testing and could be rather hard to communicate without a proper visualization.

Challenge for AR application design


An immersive design’s UI is tricky since we don’t want to take users out of an experience. So, how do we design an interface that maintains immersion is crucial. During the whole process, we always asked ourselves" How do you make something like augmented reality feel real all the way through?" These are questions without easy answers, but they’re questions that we must ask ourselves. 


My first unmoderated testing experience


For this project, I had my first unmoderated testing experience. I found that unmoderated studies do not include any direct interaction between the researcher and the study participants, which is both their biggest benefit and their greatest drawback. First, it can always be the hi-hi prototyping testing since early-prototype testing is difficult without a moderator to explain. Second, the understandable design is very important or participants tend to do tasks that depend on imagination.

See my other projects

bottom of page