Drones make excellent remote sensing systems, perfect for deployment into disaster areas for emergency rescue operations. Drones can move quickly over large areas or maneuver through buildings, distinguishing debris from survivors by using different types of cameras and sensors.  Even though the data provided by a drone is invaluable, management and operations of the drones can sometimes be difficult to operate, especially when they get beyond line-of-sight.

In Austria, researchers at the Graz University of Technology, are changing the way humans and drones interact.  They want to use augmented reality (AR) to make drones less complicated and easier for unskilled operators to use and control them.  The researcher will use a mixed reality display by Microsoft called HoloLens. With this technology, drones will have a kind of X-ray vision that allows operators to see through walls.

The system will combine live videos with a 3D views of the environment.  It will use image based rendering from the drone’s video stream to deliver real time video with live updates.  By combining both viewpoints (live video and 3D renderings), the user’s experience is sort of like super human vision.

An operator wearing the HoloLens is able to make walls transparent through the drone’s viewpoint.  HoloLens uses augmented reality to project images into the operator’s field of view. It follows the user’s head motion in the direction it is pointing, and it can overlap a video feed through a wall so it looks like the wall is transparent.  Depending on the type of camera being used, the view can be limited to just your line of sight, so if you move your head in one direction (or move it around), the drone will reposition the lenses giving you the exact view that you want.

There is also a control interface that allows operators to move the drone’s camera around by using body gestures.  The drone’s location is usually visible through the HoloLens and all a user has to do is to reach out and move their hands around allowing them to interact with the video.  We can expect more emergency response teams start to adopt this new drone technology to use in their daily rescue missions to help save lives.