Drone-based Advanced Situational Awareness for Airborne Firefighting

Oftentimes in wildfire situations, the environment is too dangerous to bring pilots into them to seek out hotspots. Extreme conditions often restrict the view of airborne firefighting efforts which forces them to fly through extreme conditions to drop flame-retardant payloads. This project seeks to provide advanced situational awareness to aerial firefighters through the use of a remotely controlled drone that will geolocate hotspots within the fire and provide their location to a pilot via an augmented reality heads up display. The project is composed of three main subsystems: the drone, the cloud, and the AR Headset. The drone is a DJI Matrice with an attached Nvidia Xavier and a Boson FLIR camera that performs georeferencing and sends hotspot temperature and location data over to the cloud. The cloud packages the data into a human readable model that can be viewed via a webpage. Finally, the AR headset is a Unity app that is implemented on a Hololens2. The app pulls data from a user specified session and displays it in AR to the user, mapping the data to the physical world. At the end of our project timeline, we had successfully implemented georeferencing from thermal imagery, as well as between the cloud server and the Hololens2/Unity app. While we were able to upload and download to/from the cloud model, there were some issues with connecting between the drone and the cloud for live updates. The Unity app could handle updates at a rate of 1Hz and was able to successfully display changes in the user specified cloud model.

1 Lifts 

Artifacts

Name Description
Executive Summary Provides an overview of the entire project and the reasoning behind its creation   Download
Project Closeout Closeout document   Download
Project Summary Summary video for the project including system overview, limitations, and project successes   Link