Wearable SuperLimbs:

Design, Communication, and Control

An IROS 2020 Workshop

Thursday, October 29, 2020 – Full Day

Supernumerary Robotic Limbs (or SuperLimbs for short) are wearable robots that assist and augment the human physical abilities as well as compensate for lost abilities with extra arms and legs attached to the human. SuperLimbs differ from other types of wearable robotics, such as powered exoskeletons and prosthetics. Exoskeletons are attached along the human limbs to augment the joint strength. Therefore, they neither change the number of limbs nor change the anatomical structure of the limbs. Prosthetic devices, too, do not add any new limb, but replace the lost limb by a mechanical limb and compensate for the lost function. SuperLimbs are supernumerary; they increase the number of limbs to create new functionality and strength.

This difference has created a new frontier of robotics research. How can a human communicate with and control extra limbs together with his/her own natural limbs? How can a human exploit the extra limbs to perform otherwise difficult and/or laborious tasks? Does the human brain support extra limbs attached to the body? SuperLimbs have attracted researchers from diverse disciplinary fields, ranging from brain and cognitive science, neuroscience, and human factors to robot design, human-computer interface, rehabilitation and biomechanics. Recently novel technologies, including soft robotics, lightweight structures using additive manufacturing, and flexible wearable sensors, have been applied to SuperLimbs, opening up a new direction of robotics. Numerous groups worldwide are now developing various types of wearable SuperLimbs for a variety of applications. These  range from workers assistance in aerospace, construction, and infrastructure industries to rehabilitation, elderly support, entertainment, and housekeeping assistance. At this IROS workshop we will display and demonstrate the state-of-the-art of SuperLimbs, and exchange ideas among researchers and practitioners from diverse disciplinary fields.

The Workshop will start with a keynote talk by the organizer, introducing the key design concept, new functionalities, and potential applications and impact of SuperLimbs. Two talks will focus on communications with SuperLimbs from the human-computer interface perspective. Two talks will be on soft robotics and new actuator applications to SuperLimbs. Two talks will be on biomechanics of human-SuperLimb systems, and their rehabilitation applications. Two talks will be on a broad spectrum of industrial applications. SuperLimbs are presenting a rich variety of challenging issues in both science and engineering, and are opening up new opportunities and alternative solutions to critical areas of robotics applications. We will discuss these challenges and opportunities further in a panel discussion and general discussion, both of which include industrial participants and practitioners.

 

We plan to display and demonstrate real SuperLimbs at the workshop in addition to oral presentations by leading researchers of SuperLimbs. Furthermore, we plan to have a panel discussion by inviting scientists, researchers, practitioners, and entrepreneurs from diverse fields as panelists. Participants will have plenty of opportunities to interact with each other.Harry Asada

Harry Asada, Massachusetts Institute of Technology

Wenlong Zhang, Arizona State University

Masahiko Inami, University of Tokyo

Topics of interest

• Design concepts of SuperLimbs / Ergonomics, wearability, machine design
• Soft robotics applied to SuperLimbs / Soft structure, soft actuators, robot design
• Human-SuperLimbs communication / Gesture communication, wearable sensors
• Natural, implicit communication and control / Haptics, neuromotor control synergy,   biomechanics
• Cognitive aspects of SRL / Cognitive load and brain support
• Human-SuperLimbs gait synchronization and coordination / Dynamic control, stability analysis
•  Industrial applications of SuperLimbs / Aerospace and automobile manufacturing, construction industries, infrastructure field maintenance
• Biomedical and home applications of SuperLimbs / Rehabilitation, elderly care, entertainment

 

Invited Speakers

• H. Harry Asada, MIT, Confirmed
• Wenlong Zhang, Arizona State University, Confirmed
• Masahiko Inami, University of Tokyo, Confirmed
• Daniel Gonzalez, West Point, Confirmed
• Alexandre Girard, University of Sherbrooke, Confirmed
• Domenico Prattichizzo, University of Siena, Confirmed

Live demonstrations of Wearable SuperLimbs developed by the speaker’s groups will be given. Presentors will wear SuperLimbs and interactively discuss their technologies, challenges, and applications with the workshop participants. Two demo-sessions, one in the morning and the other in the afternoon, are scheduled during coffee break time.

Invited Talks

H. Harry Asada,

MIT

“SuperLimbs Research at MIT: New Design Concepts, Haptic Feedback, Bimanual Synergy, Dual-Motor Actuation, and Applications to Eldercare and Hemiplegic Patient Support”

Abstract

Three research thrusts on SuperLimbs will be presented in this talk. First, new design concepts based on high torque, high speed dual-motors and human-robot hybrid actuation will be discussed. Second, fundamental issues on communication and control will be addressed, focusing on haptic feedback and bimanual synergy. Third, new applications of SuperLimbs to eldercare and support for hemiplegic patients will be described.

Actuation: Dual-Motor Actuation and Human-Robot Hybrid Actuation

Developing lightweight, compact, and energy-efficient actuators is a key challenge in designing wearable robots, especially for building SuperLimbs. Here, we introduce two approaches to overcome the difficulty. One is to use the Dual-Motor architecture, where two electric motors having diverse gear ratios are integrated into one actuator so that the actuator system can work efficiently at a broad speed range. In SuperLimb applications this allows us to bear a heavy load for a long time as well as to move SuperLimbs quickly. There are two methods for connecting two motors: one is differential connection using differential gearing, and the other is direct connection where two motors are directly coupled with a fixed gear ratio. In this talk we will present the latter, the direct connection type, and apply it to an eldercare robot, where SuperLimbs have to lift a human body for assisting sitting-to-standing transition and move the legs quickly for stabilizing the gait and preventing the elderly person from falling. The second method utilizes human power to activate a few Degrees of Freedom (DOF) manually. The human-powered DOF are combined with active DOF of the SuperLimb to span all the DOF necessary for performing a task. Two distinct features can be obtained with this approach. One is to reduce the number of actuators and, thereby, reduces the weight and energy consumption. Second, the human-powered DOF are controlled directly by the human and, thereby, communication of the human intention can be eliminated or reduced. This hybrid human-robot design concept is applied to the SuperLimb for assisting a hemiplegic patient.

Communication and Control: Inherent Haptic Feedback and Bimanual Synergy

SuperLimbs are worn by a human. In consequence, force and moment acting on the SuperLimbs are transmitted to the human body. The human can detect and sense the contact of the SuperLimbs with the surrounding environment, forming an inherent haptic feedback loop. This is an important feature of SuperLimbs, differentiating them from those standard robots sitting on a floor. We argue whether this haptic feedback assists the wearer in monitoring and controlling the SuperLimbs. We hypothesize that humans can control a contact force acting at the tip of a SuperLimb based on the inherent haptic feedback, and that humans can monitor autonomous operations of SuperLimbs through haptic stimuli transmitted to the body. We have conducted human subject experiments to test these hypotheses. Interestingly, we have found that humans can regulate the force acting at the SuperLimb end-effector while busily performing another task with vision. Also, we have found that humans can detect physical interactions between SuperLimbs and the environment through haptic stimuli with no visual observation. Although the human is simultaneously performing a different task using vision, the inherent haptic stimuli transmitted to the human allow the human to monitor the interactions with the environment without visual observation. These findings give us valuable insights into the human-SuperLimbs communication. Inherent haptic feedback due to the wearing of SuperLimbs provides the wearer with a unique communication channel that supplements visual observation and that is independent of visual tasks. In many applications, human arms perform a main task while SuperLimbs play a supportive role. Although the visual cognitive channel is tied up with the main task, the human can still monitor the SuperLimbs through the inherent haptic channel.

We have obtained another interesting result inspired by biomechanics and neuromotor control literature. When performing a task with both hands, we coordinate the two hands in certain manners, which have been characterized with a Kinematic Chain Model (Y. Guiard, 1987) and Bimanual Synergy in the neuromotor control literature. We have applied those theories to a SuperLimb system for assisting a hemiplegic patient in Activities of Daily Living (ADL), such as eating. The patient wears a SuperLimb to replace an affected arm that cannot be moved. The patient, however, can move the unaffected arm and other unaffected parts of the body. The SuperLimb is partially powered by the human body movements, while the remaining axes of movements are generated by the robot, i.e. Human-Robot Hybrid Actuation. Coordinated control of the SuperLimb and the unaffected arm can be achieved based on the neuromotor control theories. First, Bimanual Synergy allows us to predict the desired robot motion in coordination with the unaffected body motion. The central nervous system does not control individual muscles independently, but controls a group of muscles in a coordinated manner (Synergy). Principal Component Analysis (PCA) of human movement data reveals that multiple muscle movements are described as a combination of significant  modes with large eigenvalues. Using Principal Component Regressor (PCR) we can control the actuators of the SuperLimb in concert with the unaffected arm and other unaffected parts of the body.

Applications: Eldercare and Hemiplegic Patient Support

We have recently developed two unique SuperLimbs for applications of social importance. One is for eldercare, and the other for assisting hemiplegic patients. Eldercare SuperLimbs are wearable robots that support the body of elderly people with two SuperLimbs. The system integrates multiple functions into one unified system. It can assist an elderly in walking as well as in transitioning between sitting and standing. It can also prevent elderly people from falling, which is one of the most dangerous accidents leading to wheelchair-bound, fatal injuries. To catch a fall, the SuperLimbs must move their joints rapidly. To lift an elderly person in transitioning from sitting to standing, the SuperLimb actuators must generate large torques. To meet these conflicting requirements, we developed Dual-Motor Actuation technology for the eldercare SuperLimbs.

The other newly developed SuperLimb is an extra arm for assisting hemiplegic patients. We focused on feeding assistance, where a hemiplegic patient wishes to use both knife and folk in the same manner as healthy people. This is a complex bimanual task needing highly coordinated motion control between two hands. We achieved this with one unaffected human hand and one SuperLimb. We adopted the human-robot hybrid actuation technology for two objectives. One is to reduce weight and energy consumption, and the other is to control the SuperLimb more directly. Two DOF of the SuperLimb are activated with the patient’s own body movements. We used an unaffected leg for activating the two DOF of the SuperLimb by transmitting the leg movements through cables in sheaths. These two DOF are directly controlled by the human. Another two DOF are powered with electric motors, and are controlled by using PCR based on the Bimanual Synergy theory. Desired actuator movements are predicted in relation to the two other DOF powered by the human leg and the measurement of the unaffected arm motion. Data were taken from a healthy person demonstrating how knife and folk movements are coordinated.

From the data, three statistical properties were analyzed. One is to quantify the direction of motion in the 6-dimensional space where each hand moves least significantly. These are the directions where a SuperLimb does not have to move, when the hand is replaced by the robot hand. The two least significant directions were eliminated from further analysis. The remaining 4 DOF were then divided into the two DOF powered by human movements and the two activated with electric motors. The latter are also the directions of motion that are to be estimated using PCR. We picked the two principal components that can be best predicted using PCR. The third statistical analysis, which is an ongoing research effort, is to determine which of the two hands takes the lead and which one follows. According to the Kinematic Chain Model, this distinction is critically important for coordinating the two hands. We hope to report this part of result at the conference and this Workshop.

Wenlong Zhang

Arizona State University

“Wearable Soft SuperLimbs for Task Assistance: Design, Modeling, and Control“

Abstract

Soft robotics has been extensively explored to enable safe interactions with the users and environments, by utilizing materials that are inherently compliant, low-cost, and lightweight. Wearable soft robots can assist the users in different ways, either as exosuits to provide joint support or as supernumerary limbs to augment the users. In this talk, we will present several designs of soft robotic supernumerary arms (we then called Soft Poly Limbs, SPLs), inspired by elephant trunks. The SPLs are enabled by fluidic-driven soft actuators made of elastomeric materials and multi-layered fabrics. The soft actuators are mechanically programmed to achieve bending, elongation, twisting, and a combination of these motion patterns upon inflation. We will introduce the actuator design, fabrication, characterization, computational modeling, and system integration of the SPLs. We will also discuss our work on the extended rigid-arm dynamic models, system identification, and learning controller design for individual soft actuators and the entire SPL to generate versatile motion patterns, and precisely manipulate objects using end effectors and its soft body. Our experimental results highlight the SPLs’ abilities to safely interact with the user while demonstrating promising performance in assisting with a wide variety of tasks in the workplace and daily life. This talk will be concluded by proposing some new opportunities and challenges for building synergies between the human users and SuperLimbs outside the lab environment.

Masahiko Inami

University of Tokyo

To be announced

Daniel Gonzalez

West Point

“Safety Critical Design, Dynamics, and Control of Human-Wearable Extra Robotic Legs”

Abstract

Supernumerary Robotic Limbs (SuperLimbs) can augment emergency response workers, who we entrust to work quickly and skillfully in extreme conditions, by allowing them to carry more equipment and carry out their jobs with a lower risk of injury. With the promise of useful, powerful, and helpful SRLs comes the need for each aspect of the robotic system to be designed with operator safety in mind. Unlike most industrial robots, which can be physically separated from people in engineered environments, SRLs are connected directly to the operator’s body. Thus, operator safety must be carefully considered in every aspect of the design of an SuperLimb. This talk will focus on the Extra Robotic Legs (XRL) system, a bipedal “backpack with legs” that bears its own weight, walks in sync with the operator like a quadruped, and allows them to carry heavy 50lb payloads while taking strenuous postures with ease. Because the XRL is in constant physical contact with the human operator, its electromechanical design, passive locomotion dynamics, and stance control were designed for safety. A near-direct drive transmission allows for proprioceptive motion of the joints, which can apply forces on the user and exhibit compliance when the user applies forces back, despite the heavy payload. To be able to reduce the gearing as much as possible while bearing such large loads, the motion strategy during the worst-case loading configuration was designed to exploit a closed structural loop and redundant actuation. These physical traits allow for a Failsafe Hybrid Open-Loop/Closed-Loop Control Architecture that can apply a constant and easily backdrivable force along some Cartesian axes while strictly controlling stance balance along others. This architecture is also robust to potential communication failures from the central computer because each joint can act independently to control the overall robot balance. To facilitate walking with the XRL, a spring-dashpot coupler between the human and robot can be designed that will passively synchronize the gait cycle of the human-robot robot system to one matching that of many quadrupedal animals. These naturally stable limit cycle dynamics can then be supplemented with feedback control to increase the rate and envelope of synchronization and ensure human operator safety and comfort while using the XRL.

Alexandre Girard

University of Sherbrooke

“SuperLimbs : actuation challenges”

Abstract

Wearable robotic systems open a wide range of new opportunities, but they also face important technological challenges regarding both the hardware and the software. Wearable systems must be very lightweight to avoid burdening the user, actuators must be fast enough to coordinate with the natural motions of a human, the robot software must be able to understand the user intent, etc. This talk will focus on actuations challenges and present research efforts investigating novel actuations schemes. First, we will discuss the impacts of the type of transducer (electric motor, pneumatic cylinder, etc.), the type of transmission (highly geared, direct-drive, etc.) and the actuator location (collocated on the joint, non-collocated with a cable transmission, etc.) on the performance of supernumerary robotics limbs. Second, opportunities of supernumerary robotics limbs leveraging “fast” actuators for many applications will be detailed. Finally, experimental results highlighting the advantages of a powerful and compliant actuation system with prototypes using hydro-static transmission will be presented. A 3 DoF supernumerary arm strong enough to lift power-tools yet compliant enough to interact smoothly with its user, a 2 DoF supernumerary leg fast enough to push on the ground twice during a gait cycle and more prototypes and experiments will be presented.

Domenico Prattichizzo

“Supernumerary Limbs and Wearable Sensorimotor Interfaces”

University of Siena and Istituto Italiano di Tecnologia

Abstract

Supernumerary limbs and wearable sensorimotor interfaces are emerging research trends that will enable novel forms of communication and cooperation between humans and robots. It is at the cross boundary of these two research areas where we can find extraordinary elements of innovation in human-robot interaction and cooperation.

In this talk I will present our recent research on supernumerary limbs and in particular on the robotic sixth finger, designed for rehabilitating the function of the paretic human hand. The robotic extra finger, i.e. the supernumerary limb, and the paretic hand act like the two parts of a gripper working together to stabilize the grasp of objects and let the subject, with upper limb impairments, to use both hands in bimanual tasks.

The wearable robotic extra finger works together with a wearable haptic device playing the role of a sensorimotor interface augmenting the level of embodiment of the device. This is a clear case of synergistic use of wearable haptics and wearable robotics to support people with impairments in everyday life that opens new research perspectives in robotics.