Teleoperated rescue robot

Summary

I collaborated with the DCXR Lab of Dr. Craig Yu at George Mason University, as well as the SubT Lab of Dr. Nick Wang at NYCU in Hsinchu, Taiwan, to build a robot for use in search-and-rescue missions.  The robot is piloted remotely, and is designed to facilitate spatial interpretation of sounds, to assist its pilot in locating calls for help and other critical sounds with high accuracy.

The pilot controls the robot via a VR setup, for easy delivery of spatial audio (which is captured by an array of 3 microphones), and to allow for intuitive viewing of the robot's surroundings (which are captured by a 360 degree array of cameras).

I made contributions to the project's audio pipeline, which combines audio from the microphones with proper panning and transmits it to the pilot.  I also contributed to a training simulation intended to familiarize pilots with operation of the robot (refer to this page for more information).  Finally, I helped build and test the system for remote control of the robot.  I am an author on an upcoming paper regarding the robot.

Skills gained

By co-designing the sound system, I gained new skills in spatial audio capture and transmission.

By helping build and test the remote control system, I learned how to build websocket connections that communicate with Unity Engine.