Creating an image-based food tracking tool for people living with MS

dr-vivienne-guan

Vivienne Guan

University of Wollongong

| Better treatments | Social And Applied Research | Incubator | 2022 | Investigator Led Research |
SUPPORT PROJECTS WITH THIS RESEARCH FOCUS

Summary

Goal-setting and appropriate feedback related to dietary intake play critical roles in a person living with MS complying with dietary advice. The tracking of a person’s intake at eating occasions is important when formulating tailored dietary advice. However, due to the impact of symptoms of MS such as fatigue, pain and cognitive issues, the required effort for accurately tracking dietary intake can be particularly challenging for people living with MS. Condition-specific disabilities, such as reduced fine motor skills or blurred vision also need to be considered when using tools for tracking the dietary intake of people living with MS. 

This project builds on previous work exploring MS-specific needs for dietary advice and machine learning. It specifically focused on combining various new models of machine learning with professional judgement (supported by over 17 years of dietetic practice within the research team) to develop a state-of-the-art automated food recognition system integrated into an app. The app will allow users living with MS to easily track their food consumption by simply taking a picture of their food. This relatively effortless tool can enhance the user’s self-efficacy (the belief in one's ability to successfully accomplish tasks and achieve goals) and self-esteem, reduce stress, improve coping skills and improve general quality of life, with lower costs and greater availability for people living with MS. 

Project Outcomes

In this project, Dr Vivienne Guan and her team have developed a tool to identify recipes from food images.  

Matching images to recipes is challenging yet useful because recipes are detailed and varied, and food looks different in each photo. Traditional methods often miss important details because they rely heavily on human input. To mitigate this, Dr Guan and her team created a new approach called Mask-Augmentation-Based Local Matching. This method breaks down images and text into smaller parts and matches them more precisely, using hidden parts of images to improve learning and avoid mistakes. 

The app prototype was evaluated by people living with MS and Accredited Practising Dietitians. The evaluation provided information on how they used the app, which features they found useful and the challenges they encountered, as well as how to improve this image-based tool to track dietary intakes of people living with MS.  

The acceptability of an AI-powered tool for users living with MS was found to largely depend on its interface. People living with MS have suggested that the benefits of using AI in nutrition apps may outweigh any potential harms, particularly in improving accessibility and inclusiveness. Dr Guan and her team have published several scientific articles on this research and is in the process of publishing another. 

Publications

  • Guan V, Zhou C, Wan H, Zhou R, Zhang D, Zhang S, Yang W, Voutharoja BP, Wang L, Win KT, Wang P. A novel mobile app for personalized dietary advice leveraging persuasive technology, computer vision and cloud computing: development and usability study. JMIR Formative Research. 2023; doi:10.2196/46839
  • Voutharoja BP, Wang P, Wang L, Guan V. MALM: Mask Augmentation based Local Matching for Food- Recipe Retrieval. arXiv preprint arXiv:230511327. 2023.

Updated 31 March 2024 

Updated: 14 February, 2022

Stages of the research process

Fundamental laboratory
Research

Laboratory research that investigates scientific theories behind the possible causes, disease progression, ways to diagnose and better treat MS.

Lab to clinic timeline: 10+ years
Translational
Research

Research that builds on fundamental scientific research to develop new therapies, medical procedures or diagnostics and advances it closer to the clinic.

Lab to clinic timeline: 5+ years
Clinical Studies
and Clinical Trials

Clinical research is the culmination of fundamental and translational research turning those research discoveries into treatments and interventions for people with MS.

Lab to clinic timeline: 1-5 years

Investigator

Grant Awarded

  • Incubator Grant

Total Funding

  • $24,790

Duration

  • 1.5 years

Read More
Professor Kilpatrick and Professor KalincikDr Yvonne Learmonth

Newsletter subscription

  • Enter your details

Creating an image-based food tracking tool for people living with MS