In a recent client work, we had the wonderful opportunity to provide hardware prototyping development services, combining wearable sensors and drones to support a Swiss innovation project. As we enter the Fourth Industrial Revolution, our lives are being exposed to more and more computing devices and robotic systems. However, the question of how we can interact naturally with robotic systems is still evolving. We were tasked with developing the hardware and software to enable Natural User Interface (NUI) control of a robotic drone. This exciting project gave us the opportunity to integrate NUI with a Parrot drone in order to control a variety of robotic devices based on the Parrot SDK.
At the heart of the drone control system was a wearable sensor based on a novel conductive composite developed at Empa in the Laboratory for High Performance Ceramics. The sensor fiber was integrated into a watchband design for integration with a smart watch platform as part of an innovation project with STBL Medical Research AG, currently being marketed under (www.carunda24.com). The sensor combines nano-textured carbon particles with a thermoplastic elastomer, resulting in a highly elastic piezoresistive fiber sensor. It has been investigated for various applications, including vital sign measurement (Piezoresistive Soft Condensed Matter Sensor for Body-Mounted Vital Function Applications).
Motion Capture Approach
Human Activity Recognition (HAR) is focused on extracting human activity from wearable sensors. This most often includes activity recognition based on Inertial Measurement Unit (IMU) sensors including accelerometers and gyroscopes. However, these sensors are not able to accurately capture the finger and hand motions such as opening and closing of the fist. The smart strap with integrated sensor allows the opening and closing motion of the hand to be easily determined by monitoring the surface strain of the skin in the region of the tendons. A gesture recognition approach was developed that would extract gestures from the smart strap sensor data and then map those gestures to drone flight commands.
Hardware / Software Development