We first conducted a user study to capture all the nuances of the buying process<\/strong>, not only the mere test of makeup but also the unconscious psychological aspects that are activated during the experience in the store. For decades the users have been used to a certain shopping experience and changing it might have dangerous repercussions for the business.<\/p>\n <\/p>\n To conduct this analysis, we have adopted the typical tools of Design Thinking, condensed into five days of intense work.<\/p>\n It\u2019s enlightening the \u00a0definition of Design Thinking <\/strong>shared by IDEO<\/a>: \u201cDesign thinking is a human-centred approach to innovation that draws from the designer\u2019s toolkit to integrate the needs of people, the possibilities of technology, and the requirements for business success.\u201d<\/em> \u2014TIM BROWN, EXECUTIVE CHAIR OF IDEO<\/em><\/p>\n <\/p>\n Design Thinking<\/strong> helped us to clarify and set solid principles from which to build functional solutions.<\/strong><\/p>\n There are 5 steps, <\/strong>which are the classical Design Thinking moments:<\/p>\n Translating this flow of activities, here the tasks that the design team has worked on:<\/p>\n \u00a0<\/strong><\/p>\n In the first step, we have started with the users\u2019 research, doing some phone interview on a sample of 15 people of different ages. The goal was to collect information on their approach to buying make-up and their experience and open mind approach towards a Virtual Try-On app.<\/p>\n Then the results have been organised in specific files summing up each person, useful to define the users\u2019 flow based on their real needs.<\/p>\n Specifically, it\u2019s interesting understanding that from the user research we can group the interviewers in two types of users:<\/p>\n <\/p>\n In this step is important defining the users\u2019 flow<\/strong> in order to provide a flawless and effective experience for the Minimum viable product.<\/p>\n In particular, we have used flowmapp<\/a> to create and define the user flow. From the research phase and from the tech team we agreed to use a lipstick from a specific beauty brand.<\/p>\n There already are some Virtual Try-On apps in the market, with their weaknesses and strengths. The world of make-up is full of colours and light effects, so our colour palette had to use light and neutral ones<\/strong> to not get in contrast with the lipstick colours. We have then chosen a Font (TT Norms) with a feeling simple and geometric, giving good readability. In this way, we have been able to conduct a \u201cvisual brainstorming\u201d which has helped us to explore all the possible options and ideas.<\/p>\n With all those elements we\u2019ve passed to the prototype. Using Sketch and InVision we have developed the mock-ups, <\/strong>the graphic representation of our user interface, focusing on the lips. For every corner of the face, we have designed some icons which they initially serve to recognise the face and they will help the user for the rest of the experience, allowing the choice of the different type of lipsticks, colours, mate trying out.<\/p>\n The final step has been to validate the design with a selected amount of people. The goal of the User Testing is to define the details and understand if there\u2019s something not very clear from a user perspective to iterate the design and fix the problem before the final development of the app.<\/strong><\/p>\n Right from the start of the sprint, the main goal was clear to everyone: to create realistic graphics in a virtual makeup app.<\/strong><\/p>\n In augmented reality applications, realism is fundamental<\/strong>, especially if the user has to interact with elements of AR. In addition to realism, we decided to focus on a secondary objective: looking for alternatives to using ARKit<\/strong> (SDK to develop Apple’s AR).<\/p>\n It is therefore important to study the current market and analyze different solutions to develop virtual makeup.<\/p>\n For this project, the development team split up into two teams: one part working with Unity and ARKit,<\/strong> developing an application for iPad Pro that provides, in addition to good tracking and graphical realism, also high quality UX and UI. The second part of the team was in charge of studying alternative solutions, in this case it was decided they would develop the virtual makeup with Facebook’s SparkAR<\/strong>.<\/p>\n <\/p>\n ARKit allows us to trace the face of a user and recreate a 3D model of the face in real-time, so we isolate the area of the lips. Using Unity you can apply a 3D model to a texture with transparency, which makes the model “invisible” where we want it to be. This is the result:<\/p>\n We made the face invisible, showing only the user’s lips. The next step was to graphically simulate the lipstick, to do so we used PBR (Physically Based Rendering) materials. These materials allow us to make a surface more or less shiny and reflective and allows us to simulate wrinkles and irregularities. This last point is very important to simulate the movement of the lips. Starting from the previous image, we were able to generate a very good Normal Map.<\/p>\n Thanks to Light Estimation, we recreated a realistic illumination<\/strong> on the virtual surfaces based on the real light perceived by the camera of the device.<\/p>\n The post-processing continued with targeted interventions:<\/p>\n Spark AR studio is the framework to create AR filters for Facebook and Instagram<\/strong>. The creation system is limited compared to editors like Unity and provides a basic set of features to work with.<\/p>\n Interestingly, face tracking is very advanced<\/strong>; this works by simply analyzing the phone’s camera feed.<\/p>\n The interactions in Spark AR studio are created by means of graphic language via a node.<\/p>\n The end result is a mono-brand filter that you can use directly on your phone, within the Facebook and Instagram app. Users can access the content simply by scanning a QR code or following a URL.<\/p>\n The user can choose the color of the lipstick by selecting one of the colors on the right side of the screen. With Spark AR there is an added possibility of taking snapshots or recording videos that are saved directly on your device. From the point of view of distribution, this choice gives the opportunity of having a very fast system for updates and use.<\/p>\n <\/p>\n As for ARKit,<\/em> during the development of the app we discovered that the tracking is limited. <\/strong>There are situations and cases where lip tracking is not accurate. This is based on the lighting in the room, the shape of the person’s face, whether the person has a beard and other factors. However, we can assume that Apple will improve the quality of ARKit in the future, with much more accurate tracking.<\/p>\n The limits of SparkAR mainly concern the User Interface<\/strong>: the system is very limited and hardly supports the various phone resolutions. For this reason, we decided to modify the interface with a simpler and functional version for dynamic adaptation to the resolution of the various device screens.<\/p>\n The results obtained were more than satisfactory for both ARKit and SparkAR. We were able to implement the design team’s vision, resulting in a simple and easy to use app that delivers high quality AR content.<\/p>\n A tool of this kind allows all the players in the world of cosmetic to let their customers try their products, without the need to apply them directly on the face, with the consequent decrease in the costs<\/strong> of samples and various equipment and an increase in in-store hygiene<\/strong>. The last aspect is anything but secondary nowadays.<\/p>\n New technologies are our superpower and they help companies in the transition toward the future. Other successful cases are the projects made for Safilo<\/a>, Universit\u00e0 di Padova<\/a> and Tecnolaser<\/a>.<\/p>\n Are you interested in these projects\u2019 insights? We share them with the subscribers to our Exploratorium, find out more about this experimental lab clicking here \ud83d\udc49 https:\/\/exploratorium.uqido.com\/<\/a><\/strong><\/span><\/p>\n Exploratorium is Uqido\u2019s laboratory where every week we project, develop and launch on the market a new digital product.<\/p>\n![\"iPad](\"http:\/\/staging.uqido.com\/wp-content\/uploads\/2020\/06\/img-try-on.png\")
<\/span><\/h3>\nPlanning and Design Thinking<\/strong><\/h3>\n
![\"design](\"http:\/\/staging.uqido.com\/wp-content\/uploads\/2020\/06\/2-design-thinking.png\")
<\/p>\n\n
![\"post-it](\"http:\/\/staging.uqido.com\/wp-content\/uploads\/2020\/06\/4I3B1488-scaled.jpg\")
<\/span><\/p>\n\n
User Research<\/h4>\n
\n
User flow\u00a0<\/span><\/h4>\n
Moodboard\u00a0<\/span><\/h4>\n
Mock-up and prototype<\/h4>\n
User Testing<\/span><\/h4>\n
![\"mockup](\"http:\/\/staging.uqido.com\/wp-content\/uploads\/2020\/06\/8-mockup-scaled.png\")
<\/span><\/h3>\nAugmented Reality, Virtual Try On, Face Recognition: how much technology!<\/strong><\/h3>\n
ARKit<\/span><\/h4>\n
\n
<\/h4>\n
SparkAR<\/span><\/h4>\n
The limits of the two approaches<\/h4>\n
![\"Virtual](\"http:\/\/staging.uqido.com\/wp-content\/uploads\/2020\/06\/https___bucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com_public_images_4067a9c1-2689-4b86-b3a6-70a48e7def4a_600x800.gif\")
<\/span><\/h3>\nWhat will the future hold for us?<\/strong><\/h3>\n
![\"Future](\"http:\/\/staging.uqido.com\/wp-content\/uploads\/2020\/06\/code-projected-over-woman-3861969-scaled.jpg\")
<\/span><\/p>\nWHAT\u2019S EXPLORATORIUM<\/h3>\n