An Image-based Obstacle Detection Approach for Drones Using Deep Learning

Abstract

Drones have been used in multiple sectors, particularly in the civilian and military sectors, including agriculture,healthcare, security, and surveillance. This expansion has raised additional concerns about the structural safety of drones dueto their prohibitive cost. As a result, developing onboard obstacle detection systems for drones has become an importantresearch area. Most existing systems face significant challenges in detecting and classifying obstacles, particularly indynamic environments which include various objects such as birds, trees, power pylons, and even other drones. Overcomingthese challenges requires developing robust and effective models that are trained in a variety of multi-obstacle imageswith diverse backgrounds. This research proposes an obstacle detection approach for drones based on deep learning. Theproposed approach was applied in complex, low-visibility scenarios using the You Only Look Once (YOLOv11s) modeland ByteTrack tracking algorithm. In addition, we propose a newly curated multi-source dataset of images containingmultiple objects namely drones, birds, trees, and power pylons. The images were collected from different environmentsto ensure diversity. Using Roboflow and other datasets, the dataset was systematically processed, optimized, and developed.YOLOv11s achieved an overall precision of 94.2%, recall of 89.6%, mAP@0.5 of 93.7%, and mAP@0.5:0.95 of 73.0% onthe evaluation set. In contrast to prior research where only one or two classes have been considered, this paper considers theperformance of a unified framework for four classes of obstacles that are relevant for drones in complicated visual scenarios.Hence, the results must be regarded as demonstrating superior performance in the presented multi-class scenario and not ingeneral terms over all past approaches. In addition, the YOLOv11s detector has a measured throughput rate of 65.7 framesper second at an input resolution of 640 x 640 pixels, which corresponds to an average processing time of approximately15.22 milliseconds per image.