Improving Teleoperation Interfaces

Students:
Brian DeJong


In collaboration with:
Argonne National Lab 		Professors:
Ed Colgate
Michael Peshkin

Funded by:
Dept. Of Energy

Teleoperation Overview

Teleoperation is the controlling of a robot or system over a distance. The more specific type of teleoperation that this research focuses on is the controlling of a robot that is out of sight from the user. Common uses for teleoperation are controlling robots:
in space from earth
in a nuclear reactor
to diffuse a bomb
for minimally-invasive surgery.

Since the user cannot see the robot directly, he or she must rely on feedback from the robot's worksite, presented to the user via the interface. The most common form of feedback is live video from video cameras. Other types include haptic (touch, such as vibration of the manipulandum), auditory (human ear range), temperature, contact sensors, and even sonar images. Unfortunately, many difficulties arise from this restricted feedback; improving the interface so as to make teleoperation control easier for the user and to improve task performance is an important area of research.

Sample Teleoperation Setup

Northwestern's Test Bed

A teleoperation test bed has been established at Northwestern for implementing and studying improvement techniques for teleoperation interfaces. The test bed includes a 6-DOF PUMA arm-robot, a Windows PC, a in-house Visual C++ program (called "TIFT"), a 6-DOF Spaceball input device, several video cameras, and two moveable monitors.

The Northwestern Teleoperation Test Bed

The test bed also has the capability of augmented reality (AR), or the overlaying and incorporating of computer graphics onto the live video.

Sample of the AR Capability

Eliminating Mental Transformations: Inside/Outside the Box

Suppose a teleoperaion interface consists of the manipuladum and two monitors with video images of the robot (on the left):

Here, the control coordinate frame does not align with both image frames. For example, pushing right on the manipulandum moves the robot right in one view but away / into-the-screen in the other. The user must continually mentally switch between control and image frames to perform a task. This mental transformation gets guickly tiring, especially with several views and/or tasks that require using multiple views simultaneously (such as motion along a line not orthogonal to the camera views). When the user switches attention from one view to another, he/she has to recall or relearn the new tranformation.

There is a very simple solution, though. But rotating (and rearranging, if necessary) the views either away from the user (top right; called "Outside the Box") or around the user (bottom right; called "Inside the Box"), the frames match up. Now pushing right on the manipulandum moves the robot to the right in both views. One can imagine having several views, all at odd angles, arranged such that the frames all align with the control frame.

To evaluate the significance of this technique, an experiment testing teleoperation performance for three monitor/image setups, including Inside the Box was run. The experiment’s results show that Inside the Box improves task time and is considered by users to be intuitive and easy. Example AVI clips of the experiment can be downloaded from here:
overview clip, monitors clip, and pipes clip.

Papers:

DeJong, B.P., Faulring, E.L., Colgate, J.E., Peshkin, M.A., Kang, H., Park, Y.S., Ewing, T.F. "Lessons learned from a novel teleoperation testbed." Industrial Robot, 33(3): 187-193, 2006. pdf

An academic approach:
DeJong, B.P., Colgate, J.E., Peshkin, M.A. "Improving Teleoperation: Reducing Mental Rotations and Translations," Proc. of IEEE International Conference on Robotics and Automation, New Orleans, April 2004. pdf

An industrial approach:
DeJong, B.P., Colgate, J.E., Peshkin, M.A. "Improving Teleoperation: Reducing Mental Rotations and Translations," American Nuclear Society 10th International Conference on Robotics and Remote Systems for Hazardous Environments, Gainesville, Florida, March 2004. pdf

DeJong, B.P. "Improving Teleoperation Interfaces: Creation of a Testbed and Thinking Inside the Box," Masters Thesis, Northwestern University, May 2003. pdf
home | about us | faculty | members | research
Last updated BPD 11/13/06.