Supervisory control interfaces for nursing robot team (2020)

Effective supervision and teleoperation of nursing robots working together with humans in a hospital setting is important so that patients can receive proper care. Studies have been conducted on interfaces that supervise and offer teleoperation of swarms of robots for use in the military and in other commercial fields. These studies offer valuable insight and guidelines on how to design a similar interface for nursing robots. In this work, we present a method for teleoperation of a multi-robot system using two interfaces. This project hopes to improve upon current teleoperation of multi-robot systems (MRS) by having two different managers control different areas of the workflow. One manager is known as the supervisor and the other manager is known as the operator. The supervisor is responsible for task allocation to the robots. However, robot autonomy is not often capable of performing reliably in uncertain environments. In these cases, the operator can take control of the stalled robot, correct the performance issue, and return the robot to the supervisor where the robot can continue its autonomous operation.

Demo

Supervisory Control Interface for Multi-Robot Fetch-and-Delivery

 Computational Modeling of Human-vehicle Interaction (2017-2019)

This project developed computational model to simulate and investigate the physical human-robot interaction (pHRI) when human driver maneuver a vehicle with and without driving assistance. The computational model integrated several contemporary technologies, including: 1) an OpenSim biomechanical human model, 2) the dynamic and geometric models of  a vehicle (Toyota), 3) a motion planner that learns the driving motion primitives (e.g., operating the steering wheel, pressing gas/brake pedals) from the human demonstrations, and 4) the controllers to ensure the stability of human motion control and physical human-robot interaction. The pHRI framework was also used to how humans response to the sudden force change (of haptic driving assistance).

Selected Publications

1. Hideyuki Kimpara, Kenechukwu Mbanisi; Zhi Li, Karen Troy, Danil Prokhorov, Michael Gennert, “Force anticipation and its potential implications on feedforward and feedback human motor control“, in Human Factors: The Journal of the Human Factors and Ergonomics Society, 2020. PDF
2. Hideyuki Kimpara, Kenechukwu Mbanisi, Jie Fu, Zhi Li, Danil Prokhorov, and Michael Gennert, “Human Model-based Active Driving System in Vehicular Dynamic Simulation“, IEEE Transactions on Intelligent Transportation Systems 21, no. 5 (2019): 1903-1914. PDF
3. Kenechukwu C. Mbanisi, Hideyuki Kimpara, Tess Meier, Michael Gennert and Zhi Li “Learning Coordinated Vehicle Maneuver Motion Primitives from Human Demonstration”, In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 6560-6565. PDF

Estimate Object Transfer Point for Fluent Human-Robot Handover (2018-2019)

We proposed an efficient method for predicting the Object Transfer Point (OTP), which synthesizes (1) an offline OTP calculated based on human preferences observed in a human-robot motion study with (2) a dynamic OTP predicted based on the observed human motion. Our proposed integration significantly improved the OTP prediction accuracy and the fluency of human-robot handover.

Selected Publilcations

1. Heramb Nemlekar, Dharini Dutia, Zhi Li, “Object Transfer Point Estimation for Fluent Human-Robot Handovers”, in 2019 International Conference on Robotics and Automation (ICRA), pp. 2627-2633. IEEE, 2019. PDF

Upper Limb Exoskeleton for Stroke Rehabilitation (2010-2014)

This project studied the human arm motions in point-to-point reaching, and reaching-to-grasp tasks, discovered new models to predict the human arm postures, and developed novel motion planners  enable upper limb exoskeleton for stroke rehabilitation to provide appropriate motion augmentation and correction when the patients practice freeform manipulation tasks.

Selected publications

1. Zhi Li, Dejan Milutinovic and Jacob Rosen, “Spatial Map of Synthesized Criteria for the Redundancy Resolution of Human Arm Movements”. IEEE Transactions on Neural Systems & Rehabilitation Engineering, 23(6), pp. 1020 – 1030, Nov. 2015. PDF
2. Zhi Li, Hyunchul Kim, Dejan Milutinovic and Jacob Rosen, “Synthesizing Redundancy Resolution Criteria of the Human Arm Posture in Reaching Movements”, In Redundancy in Robot Manipulators and Multi-robot systems (pp. 201-240). Springer Berlin Heidelberg, 2013. PDF

Multi-arm Surgical Robot System (2009-2010)

This research effort aims to optimize the mechanical design, configuration and motion coordination of the Raven IV surgical robot. For best manipulation dexterity in the common workspace of the four surgical robot arms, we proposed a performance index that considers link rigidity, operation area, and manipulability for all possible robot arm configurations. We optimized the link lengths, robot base orientation and port spacing.

Selected publications

1. Zhi Li, Daniel Glozman, Dejan Milutinovic, Jacob Rosen, “Maximizing Dexterous Workspace and Optimal Port Placement of a Multi-Arm Surgical Robot”, 2011 IEEE International Conference on Robotics and Automation (ICRA 2011), Shanghai, China, 2011. PDF

Multi-user Cooperative Manipulation via Networked Haptics (2007-2009)

Networked haptics enables cooperative manipulation of shared virtual objects and direction interactions with multiple remote users. Control architectures are necessary for synchronizing the distributed copies of the share virtual objects, as well as maintaining the contact stability of multi-agent interaction. To this end, we propose to use remote dynamic proxies with wave-based controllers to represent users in remote virtual environment. We found the remote dynamic proxies can render smooth motion and force feedback in the presence of long and time-varying time delay, and provide better user state synchronization.

Selected publications

1. Zhi Li and Daniela Constantinescu, “Networked Haptic Cooperation Using Remote Dynamic Proxies”. The Second International Conferences on Advances in Computer-Human Interactions (ACHI 2009), Cancun, Mexico, February 1-6 2009. PDF
2. Zhi Li and Daniela Constantinescu, “Remote Dynamic Proxies for Wave-Based Peer-to-Peer Haptic Interaction”. The Third Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, Salt Lake City, USA, March 18-20 2009. PDF
3. Zhi Li and Daniela Constantinescu, “Comparison of Power- and Wave-based Control of Remote Dynamic Proxies for Networked Haptic Cooperation”. The 2009 IEEE International Conference on Mechatronics and Automation (ICMA 2009), Jilin, China August 9-12, 2009. PDF