Robots Gain Touch and Proximity Awareness Through Vision-Enabled Arms

The ProTac system uses in-arm cameras for both proximity sensing (left) and tactile sensing (lower right)
Van Anh Ho from JAIST

For robots to interact safely with humans, they must recognize when a person is nearby and detect when physical contact occurs. A newly developed system enables them to handle both tasks by placing cameras inside their arms.

Introducing the ProTac System

Prof. Van Anh Ho, Dr. Quan Khanh Luu, and their team at the Japan Advanced Institute of Science and Technology (JAIST) are advancing an approach called ProTac (short for Proximity and Tactile sensing).They integrated the technology into a cylindrical robotic arm segment known as the ProTac link, but they could also adapt it to other robotic components.

This arm section is hollow and wrapped in a soft, transparent polymer skin. On top of it lies a flexible layer of polymer-dispersed liquid crystal (PDLC). By applying voltage, the PDLC can rapidly shift between transparent and opaque states. Beneath it sits a series of marker dots that remain opaque at all times.

Embedded Camera System

Two stereo cameras, placed at either end of the arm, capture visual data along its interior.

A diagram of the ProTac link
Van Anh Ho from JAIST

When the PDLC is transparent, the vision system allows the cameras to look outward and identify objects in the environment.If the robot detects a person approaching, it either pauses its movement to avoid harm or adjusts its path to prevent contact.

Sensing Touch Through Opacity

When the PDLC turns opaque, the cameras only see the dots against a dark background. A touch on the flexible surface alters the spacing of those dots at the point of pressure, enabling the system to identify where the contact occurred and estimate its force.

Additionally, ProTac can operate in a “flicker mode,” where the PDLC switches rapidly between clear and opaque. This lets the system alternate between monitoring approaching objects and detecting touch, effectively combining both sensing abilities at once.

ProTac could also be used by robots that are designed to lift objects, letting them know how much pressure they’re applying
Van Anh Ho from JAIST

“ProTac can enhance robotic dexterity in environments where both safety and gentle handling are crucial,” explains Dr. Luu.

The team’s findings were recently detailed in the journal IEEE Transactions on Robotics.

T-RO: Vision-based Proximity and Tactile Sensing for Robot Arms: Design, Perception, and Control

Read the original article on: New Atlas

Read more: Robotics Combined with Virtual Reality Allows Seamless, Natural Engagement