Discover Magazine: 'Predator' Vision Drones get AI to Spot Poachers

The thermal infrared camera imagery taken by a drone operated by the Air Shepherd conservation group during a field demonstration

The thermal infrared camera imagery taken by a drone operated by the Air Shepherd conservation group during a field demonstration. Credit: Air Shepherd Poachers illegally hunting elephants and rhinoceroses under the supposed cover of darkness may soon find themselves being tracked by "Predator" vision drones armed with artificial intelligence. The new AI system that enables surveillance drones to automatically detect both humans and animals could help conservation experts and rangers protect endangered wildlife starting in 2018. A wildlife conservation group called Air Shepherd has already tested the AI system in a field demonstration and hopes to eventually expand such operations to various national parks in Africa. Like the alien hunters from the "Predator" science fiction films, Air Shepherd's drones use thermal infrared vision to detect the heat coming from warm objects such as human or animal bodies. The AI system developed by researchers from Carnegie Mellon, the University of Southern California and Microsoft gives such drones the added capability to identify potential poachers and alert park rangers or other law enforcement agents. "Our main goal was to assist Air Shepherd in unmanned autonomous vehicle (UAV) operations and reduce the human effort needed to monitor the UAV cameras at night," says Fei Fang, an assistant professor at the Institute for Software Research at Carnegie Mellon University in Pittsburgh. "In the future, we also want to automatically adjust the flight route of the UAV to automatically track the poachers."

When Poachers Become the Hunted

Most poachers try to hunt wildlife by night in order to avoid the daytime crowds of tourists and ranger patrols, Fang explains. Conservation groups such as Air Shepherd have already begun trying to pierce the cloak of darkness by sending drones to provide overhead surveillance that can aid ranger patrols at night. But operating the drones and monitoring the drone camera footage remains a time-intensive task for humans without the help of AI tools. https://www.youtube.com/watch?v=Ny0XGUPpawg&feature=youtu.be Fang and her colleagues trained a neural network—the basic architecture for the popular AI technique known as deep learning—to recognize the patterns in the thermal camera imagery associated with humans or animals. The SPOT (Systematic POacher deTector) system can run on common laptop GPU hardware. That means the SPOT system merely requires a wireless link between the laptop running the program and the flying drone in order to analyze the live video feed coming from the drone. This marks an early stepping stone toward the goals of a larger AI-enabled project called PAWS (Protection Assistant for Wildlife Security). That effort aims to eventually enable the drones to automatically detect suspicious humans below and then adjust their flight paths to track those humans. Eventually, Fang wants to deploy deep learning algorithms that can even help rangers predict the possible hotspots for poaching activities and automatically suggest the best patrol routes for intercepting such illegal hunting activities.

How to Train Your AI

Deep learning algorithms have already become common tools used by Silicon Valley tech giants such as Google and Facebook in services that require computers to visually identify human faces or various objects in various images. But Fang and her colleagues still faced certain challenges in training deep learning algorithms to identify possible humans in the thermal infrared infrared imagery provided by Air Shepherd's drones. (A research paper detailing the findings was posted online in November and was recently accepted for presentation at the AAAI 2018 conference.)

Air Shepherd drone

Credit: Air Shepherd One challenge came from the fact that the thermal infrared videos provide less information than the color images from RGB cameras. Another challenge comes from the generally lower resolution images available from the thermal infrared cameras mounted on the Air Shepherd drones. A third problem comes from the fact that much of the video footage taken of humans or animals is from far away and high above, which means that the deep learning algorithms often have to identify each possible warm body based on just a few pixels. Furthermore, most deep learning algorithms designed for detecting certain objects in videos are based on simpler scenarios that assume a fixed camera view. By comparison, the SPOT system's AI has to identify and track moving objects while the camera's point of view is also moving because of the drone's own flight movements. Fang and her team also had to balance how much computation the SPOT system performs on the laptop versus centralized cloud servers operated by companies such as Microsoft. More powerful laptop hardware can handle more of the computation without wirelessly offloading as much of the work to the cloud servers. In any case, researchers still had to figure out how to efficiently transmit images and video footage from the drones to the laptop and cloud severs based on the available wireless links.

Security for More than Wildlife

For now, the automated surveillance provided by the AI-boosted drones could greatly expand the area of protection for endangered wildlife by spotting and stopping more poachers. In addition, the SPOT system could also potentially protect rangers by preventing them from running into larger groups of heavily-armed poachers. Future use of the SPOT system in the field will provide more useful data that could help train SPOT's neural network to become even better at identifying humans and animals. Fang has a special passion for wildlife conservation that has extended beyond the cooperation with Air Shepherd to attending World Wildlife Fund conferences and personally accompanying rangers on patrol in China. But she acknowledged that such AI-powered drones could also provide automated surveillance in many cases beyond just protecting endangered species. "This whole framework will be useful for security surveillance at nighttime in an industrial park or any areas that need any security surveillance," Fang says.

Read Discover Magazine article