Drones have changed how movies, television shows, commercials, and even the news are filmed. When a director wants to provide a fully immersive experience, they need to be able to film from all angles, including aerial shots. The traditional way to capture these aerial shots is with a camera crew in a helicopter. But as any director will tell you, when using a helicopter for filming, you have one shot to get it right. Renting a helicopter for aerial filming and photography can cost thousands of dollars an hour. And while Hollywood productions tend to have large budgets, it is still important to save money when possible.

Drones are inexpensive and can be deployed within minutes, whenever a director needs an aerial shot. The cameras on a drone have become so sophisticated that they can capture images of the highest quality. But, as Carnegie Mellon School of Computer Science Ph.D. graduate Rogerio Bonatti points out, controlling drones still requires a certain level of skill. For Rogerio’s Ph.D. thesis, titled Active Vision: Autonomous Aerial Cinematography with Learned Artistic Decision-Making, he researched how drones could be programmed to not only fly autonomously but learn how to film specific moods for cinematography.

In the paper’s abstract Rogerio states, “We develop a system for active vision: in other words, one that not only passively processes the incoming sensor feed, but on the contrary, actively reasons about the cinematographic quality of viewpoints and safely generates sequences of shots.” To enable a drone to autonomously film a rich and interesting shot takes 4 key steps, according to Rogerio. The first step was to teach the drone’s brain about trajectory in line with cinematic guidelines. The drone needs to know what type of shot to capture at a given location and how to incorporate safety parameters to avoid collisions. The next step was to test how the drone can fly in simulations and real world scenarios while keeping the subject in frame safely.

This is not an entirely new concept. Skydio was the first drone company to offer a track and follow drone. This type of drone is programmed to follow a subject and has become popular for filming in some extreme sports. But for Rogerio’s third step, he wanted to take the track and follow concept to a new level by teaching the drone to do more, what he calls the intangible art of cinematography. To do this, Rogerio collected hundreds of drone videos of varying styles. He then showed the videos to thousands of volunteers and asked them to describe how the videos made them feel. Were their reactions calming or exciting, uplifting or nerve-wracking?

The final step in Rogerio’s research was to upload the data to an AI program for the drone and design a complete program that would allow multiple drones to film in tandem. As Rogerio explains, realistic cinematography has to be captured and edited together from multiple angles. With Rogerio’s AI platform, a seamless aerial film using multiple drones can autonomously be dynamic, creative, and evoke the emotions desired by the director. “Aerial cinematography is revolutionizing industries that require live and dynamic camera viewpoints such as entertainment, sports, and security,” Rogerio says. “Fundamentally, it is a tool with immense potential to improve human creativity, expressiveness, and sharing of experiences.”

Previous Drone News: