Research

Shared Autonomous Mobile Humanoid Robots for Remote Nursing and Living Assistance

  • [2015-2016] In response to the outbreak of highly infectious diseases, such as Ebola (2015) and Zika (2016), a Tele-Robotic Intelligent Nursing Assistant (TRINA) was developed at Duke University to enable healthcare workers to perform routine patient care tasks without direct contact, and to handle contaminated materials and protective gear. (see demo)
  • [2017] At WPI, we continue to improve the intelligence of robot autonomy and human-robot interface. Our work enables healthcare workers are novice to robot teleoperation to efficiently, intuitively and effortlessly control the robot to perform nursing assistance tasks that involve dexterous manipulation, navigation and active perception.
  • [2022] Recently, we integrated a new mobile humanoid robot platform. Check the GO-FOR Nursing Assistance Robot.

[2022] Check Our New Nursing Assistance Robot GO-FORGopher-Interface

[2016] TRINA – Tele-Robotic Intelligent Nursing Assistance

TRINA hardware notesTRINA console notes

TRINA_Evaluation

Selected Publications

 


Shared Autonomy for Assisting Remote Robotic Manipulation

EMG-AnalysisHow to Reduce the Physical Workload in Teleoperation Via Motion Mapping?

Tele-nursing robots provide a safe approach for patient-caring in quarantine areas. For effective nurse-robot collaboration, ergonomic teleoperation and intuitive interfaces with low physical and cognitive workload must be developed. We propose a framework to evaluate the control interfaces to iteratively develop an intuitive, efficient, and ergonomic teleoperation interface. The framework is a hierarchical procedure that incorporates general to specific assessment and its role in design evolution. We first present pre-defined objective and subjective metrics used to evaluate three representative contemporary teleoperation interfaces. The results indicate that teleoperation via human motion mapping outperforms the gamepad and stylus interfaces. The trade-off with using motion mapping as a teleoperation interface is the non-trivial physical fatigue. To understand the impact of heavy physical demand during motion mapping teleoperation, we propose an objective assessment of physical workload in teleoperation using electromyography (EMG). We find that physical fatigue happens in the actions that involve precise manipulation and steady posture maintenance. We further implemented teleoperation assistance in the form of shared autonomy to eliminate the fatigue-causing component in robot teleoperation via motion mapping. The experimental results show that the autonomous feature effectively reduces the physical effort while improving the efficiency and accuracy of the teleoperation interface.

Selected Publications

Haptic+VisualCuesPerception and Action Assistance for Freeform Tele-manipulation Control

Teleoperation enables controlling complex robot systems remotely, providing the ability to impart human expertise from a distance. However, these interfaces can be complicated to use as it is difficult to contextualize information about robot motion in the workspace from the limited camera feedback of the workspace. Thus, it is required to study the best manner in which assistance can be provided to the operator that reduces interface complexity and effort required for teleoperation. Some techniques that provide assistance to the operator while freeform teleoperating include: 1) perception augmentation, like augmented reality visual cues and additional camera angles, increasing the information available to the operator; 2) action augmentation, like assistive autonomy and control augmentation, optimized to reduce the effort required by the operator while teleoperating. In this paper we investigate:1) Which aspects of dexterous tele-manipulation require assistance 2) The impact of perception and action augmentation in improving teleoperation performance; 3) What factors impact the usage of assistance and how to tailor these interfaces based on the operators’ needs and characteristics. The findings from this user study and resulting post-study surveys will help us optimize a design for teleoperation assistance that is more task and user specific. (see demo)

Selected publications

  • Tsung-Chi Lin, Achyuthan Unni Krishnan and Zhi Li, “Perception and Action Augmentation for Teleoperation Assistance in Freeform Tele-manipulation”, Submitted to ACM Transactions on Human-Robot Interaction, 2022/07.
  • Tsung-Chi Lin, Achyuthan Unni Krishnan and Zhi Li. “Impact of Unreliable Assistive Autonomy on Shared and Supervisory Control for Remote Robot Manipulation.”, Submitted to 2023 International Conference on Robotics and Automation (ICRA), 2023.
  • Achyuthan Unni Krishnan, Tsung-Chi Lin, and Zhi Li. “Design Interface Mapping for Efficient Free-form Tele-manipulation.” In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2022). 2022. PDF
  • Lin, Tsung-Chi, Achyuthan Unni Krishnan, and Zhi Li. “Comparison of Haptic and Augmented Reality Visual Cues for Assisting Tele-manipulation.” In 2022 International Conference on Robotics and Automation (ICRA), pp. 9309-9316. IEEE, 2022. PDF [Demo]

camera_objectives

Perception and Action Assistance for Supervisory Control of Robotic Manipulation

Supervisory control of a humanoid robot in a manipulation task requires coordination of remote perception with robot action, which becomes more demanding with multiple moving cameras available for task supervision. We explore the use of autonomous camera control and selection to reduce operator workload and improve task performance in a supervisory control task. We design a novel approach to autonomous camera selection and control. Our user study shows that autonomous camera control does improve task performance and operator experience, but autonomous camera selection requires further investigation to benefit the operator’s confidence and maintain trust in the robot autonomy. (see demo)

robot-picking-up-cup

We also developed a graphical user interface for high-level robot control. The framework of the interface enables teleoperators to control a robot at the action level, and incorporates a simple but effective design that enables teleoperators to recover from task failure in a number of ways. Our user study with N = 25 participants show that high-level interface able to handle the most frequent errors is resilient to the effects of unreliable robot autonomy. Although the total task completion time increased as the robot autonomy becomes unreliable, the users’ perception of workload and task performance are not affected.

Selected Publication

 


Eye-handPerception-Action Coordination When Using Active Telepresence Cameras

Teleoperation enables complex robot platforms to perform tasks beyond the scope of the current state-of-the-art robot autonomy by imparting human intelligence and critical thinking to these operations. For seamless control of robot platforms, it is essential to facilitate optimal situational awareness of the workspace for the operator through active telepresence cameras. However, the control of these active telepresence cameras adds an additional degree of complexity to the task of teleoperation. In this paper we present our results from the user study that investigates: 1) how the teleoperator learns or adapts to performing the tasks via active cameras modeled after camera placements on the TRINA humanoid robot; 2) the perception-action coupling operators implement to control active telepresence cameras, and 3) the camera preferences for performing the tasks. These findings from the human motion analysis and post-study survey will help us determine desired design features for robot teleoperation interfaces and assistive autonomy. (see demo)

Selected Pulication

  • Tsung-Chi Lin, Achyuthan Unni Krishnan and Zhi Li, “Perception-Motion Coupling in Active Telepresence: Human Behavior and Teleoperation Interface Design”, Accepted by ACM Transactions on Human-Robot Interaction, Accepted on Nov 2022. PDF
  • Tsung-Chi Lin, Achyuthan Unni Krishnan and Zhi Li, “How People Use Active Telepresence Cameras in Tele-manipulation”, IEEE International Conference on Robotics and Automation (ICRA), 2021. PDF [Demo]
  • Alexandra Valiton and Zhi Li, “Perception-Action Coupling in Usage of Telepresence Cameras“,  in 2020 IEEE International Conference on Robotics and Automation (ICRA),  pp. 3846-3852. IEEE, 2020,  Finalist of Best Paper for Human-Robot Interaction PDF [Demo]

 


ROS-Unity Simulation: Nursing Assistance Robot in Hospital

Gopher_in_Unity hospital

We developed an open source ROS-Unity simulation for the mobile humanoid nursing robots working on various mobile manipulation nursing assistance task in a hospital environment. See our Github repository and demo video of our recent user study using this simulation.