Augmented Reality

Leveraging From Artificial Intelligence (AI) In Advancing The Augmented Reality (AR) Grocery Shopping Experience

Abstract
Globally, people are extremely eager to find modern day technologies that could save them time and money while maintaining healthy life choices, especially when performing essential activities such as in-store grocery shopping. This paper presents a system that integrates Artificial Intelligence (AI) methods with Augmented Reality (AR) techniques to enhance the grocery shopping experience through the use of smart glasses. Our proposed framework deploys a Convolutional Neural Network (CNN) object detection model that allows for item identification. By simultaneously retrieving data from a large nutrition database, personal medical reports, and other grocery store related datasets, our intelligent system is able to provide user-centric nutrition facts, health and wellness tips, and unhealthy selection warnings that are augmented on a real time broadcasting of the smart glasses. Our state-of-the-art framework (CoShopper) demonstrates high accuracy in detecting grocery items, improves product selection, increases cost efficiency, and reduces the time spent in the process.
About the Author
Yasmeen Alhamdan

Yasmeen graduated from Ted Rogers School of Management with a bachelor’s degree in Marketing Management. During her last semester as an undergraduate student, Yasmeen had the opportunity to develop a strategic marketing plan for the International Exchange Department at IESEG International School of Management in Paris, France. She believes that applying graphic design for commercial purposes is one of the fastest ways to capture consumer’s attention in an artistic manner. Her passion for graphic design motivated her to complete the Graphic Design for Print and Web program, with honour, at Humber School of Media Studies and Information Technology. She worked as a freelance graphic designer creating innovative digital and print assets.

About the Project

Project Summary

This research paper describes the architecture of CoShopper, a prototype of a system that employs AR as an intelligent guide that transforms the traditional grocery shopping into a personalized and rich user experience. CoShopper system adds a real-time digital layer to the traditional grocery store environment through the use of smart glasses. The main intention of our proposed system is to analyze the health status of the user then provide assistance in making healthy food choices to the fullest extent possible directly at the grocery store. Besides, CoShopper framework improves cost efficiency by featuring current discount deals offered by each individual grocery store. 

The Architecture of CoShopper Framework

Key Components

Our system involves several components that are collaboratively functioning to provide user-centric assistance while at the grocery store. These components are, first and foremost, the health organization that will be feeding our databases with up-to-date medical information including medical reports and lab results specific to every user of our system. Accessing such information allows CoShopper to become aware of the amount of each nutritional element that the user needs to maintain a healthy diet. Our intelligent system will also consider all the diagnosed food allergies data, and use them to warn the user from purchasing certain products. In other words, when facing products at the grocery store, our state-of-the-art system has the ability to directly distinguish between healthy and unhealthy products based on the user’s own health records. This feature is achieved through the integration between the user’s medical database and Nutritionix.com, which is the second component of CoShopper and the largest verified database of nutrition information in the world.  Saving money is another feature that we considered in the development process. The usefulness of our system extends to display weekly-deals applied on certain items in real-time while at the grocery store. The autonomous flow of such data is derived from the connection between our system and each individual store’s up-to-date discount deals.

‍Augmented Reality (AR) Grocery Shopping - CoShopper Framework

‍Augmented Reality Grocery Shopping - Artifacts From The Future


MY NEONORMAL STORY
''
Certain factors such as the weather, the political atmosphere, and even a pandemic can influence the supermarkets and grocery stores industry. The unforeseen crisis of COVID-19 pandemic has brought the online grocery shopping market to the forefront as one of the most growing fields in the past months. The number of people staying home has exponentially increased and therefore spiking numbers of online purchases for essential groceries was shown. However, that transition is preventing customers from enjoying the advantages of the in-store grocery shopping experience. This raises new horizons of research that are primarily focused on allowing individuals to take advantage of our CoShopper framework while the new guidelines of COVID-19 are remain intact for the protection of the employees and the customers. During the current health crisis, giant grocery stores are searching for innovative ways to improve store operations, increase cost efficiency, and ensure health and safety for the shoppers and essential workers. According to experts, robots and AI is the ultimate solution that will help grocers stay competitive during the crisis. Given the direction that many grocery stores are heading towards, we believe that CoShopper is able perform its full potential for the users while they are safe at their own place. With a combination of our framework installed on smart glasses, computer screen, and in-store robots controlled by the user via a web service, we provide a rich and personalized grocery shopping experience.