Food Image Data Collection App

Design an app to allow users’ to correct machine learning predictions identifying food items in an image. The user will be shown an image with individual food items labeled and marked with a bounding box.  User are tasked confirm or correct the guess. This requires a user interface which permits the user to draw and label rectangular bounding boxes. 

Objectives

  1. Design a cross-platform mobile app (in Flutter) that retrieves an image from a cloud database, displays the image to a user, and records the user response. 
  2. Maintain cloud-based databases of images to annotate and of user's annotations previously collected.
  3. Validate product with a small and informal field study. 

Motivations

We are currently engaged in research to develop computer vision models to automatically identify which foods are present in an image. We work towards identifying the quantity of each food items, so that we have (1) determine the number of calories consumed and (2) identify the amount of edible food wasted after a meal. These machine learning models are currently in ongoing development. We need assistance from a human-in-the-loop to further improve our models. We seek a mobile app to enable us to crowd-source this task. 

Qualifications

Minimum Qualifications:
  • CS 340 (database)

Preferred Qualifications:
  • CS 492 (mobile) or CS 493 (cloud)
  • experience with machine learning or computer vision


Details

Project Partner:

Patrick Donnelly

NDA/IPA:

No Agreement Required

Number Groups:

1

Project Status:

Accepting Applicants

Website:
https://soundbendor.org/
Card Image Capstone