Lidar to 3D Sound Application for the Seeing-impaired

This app harnesses the point-cloud data from the front-facing camera Lidar system, and turns the data into a form that can be used by the seeing impaired to navigate their surroundings. Some trial and error will be necessary to access the Lidar data in near real-time, and convert it into a 3D sound “map” that is useful. The final app ideally would be deployed to a smartphone to test in the field, but simulations are also acceptable.

IMPORTANT: This project does not have a formal industry or faculty mentor and is student team managed and directed.

Objectives

>>>>> You will need a recent Mac with Apple Silicon chip and recent iOS device with Lidar capability in order to build and test. <<<<<

Goals:

  • A native application (iOS or Android)
  • Access near-realtime point cloud data from the Lidar system
  • Point-cloud depth info converted into 3D sound (sound that can be localized by the listener)
  • Deployed to a smartphone device OR run on the emulator in a simulation

Stretch Goals:

  • Performance is good enough to enable actual use by a human (in a safe area!!)
  • GPS is somehow blended in with the 3D soundscape to allow walking navigation

Possibly Helpful Links:  

IMPORTANT: This project does not have a formal industry or faculty mentor and is student team managed and directed. If you choose this project for a one-term Capstone such as CS 467, make sure to manage the deliverables for that time frame. Alternatively, if you choose this project for a 3-term capstone (such as CS 461, 462, and 463) you will be expected to make the project more robust (and to determine how best to do this). The Instructor and TAs are available to answer any questions you may have about the project or provide resources, etc.

Motivations

This aims to be an open-source, affordable, and easy to use solution for the seeing-impaired. With nothing more than a chest-mounted smartphone and earbuds, a person could navigate their surroundings.

Qualifications

Minimum Qualifications:
None Listed

Preferred Qualifications:
None Listed


Details

Project Partner:

William Pfeil

NDA/IPA:

No Agreement Required

Number Groups:

1

Project Status:

Accepting Applicants

Keywords:
GamingAndroidiOSlidar
Card Image Capstone