M.S. AAI Capstone Chronicles 2024

2

Introduction

The main challenge facing unmanned aerial vehicles (UAVs) is maintaining a safe distance from

obstacles, a task known as sense and avoid (SAA). Despite careful route planning and generally sparse

airspace, autonomous drones may still unexpectedly encounter airborne or static obstacles in their path.

The autonomous SAA system for UAVs is responsible for handling situational awareness, decision

making, and control of the aircraft to execute evasive maneuvers effectively. To address this issue

various sensing options are available on UAVs, such as radar, LIDAR, passive electro-optical sensors, and

passive acoustic sensors. However, using visual cameras for the SAA task is appealing due to their

lightweight and cost-effectiveness (Amazon Prime Air, 2021).

For this analysis, a SAA solution is developed by applying computer vision techniques to

monocular video and images produced by a singular vision camera. The designed model is intended for

use on a UAV, to assess data from the cameras in real-time. For static objects, the task is confined to the

detection and location of the object. For airborne objects, trajectory planning is necessary to determine

if rout e modifications need to be made. This task can be completed through the analysis of the object’s

motion over time by detecting and tracking the object across successive video frames.

The data used to develop the necessary model is generated by two aircraft equipped with high

resolution cameras and made available by Amazon Web Services (AWS) (Amazon, 2021). The dataset

consists of 4,943 flight sequences of around 120 seconds each and over 5.9 million images. About half of

the images contain airborne objects and are labeled accordingly with an average of 1.3 labels per image.

The remaining images contain no airborne objects and have no corresponding label.

The goal of this analysis is to develop a model that can detect airborne objects within a distance

that allows for corrective maneuvering if necessary and track airborne objects to allow for future motion

prediction.

116