The economics of robot manufacturing is driving us toward situations in which a single human operator will be expected to split attention across multiple semi-autonomous vehicles, and remotely intercede if necessary. We present an analysis of such situations, with the goal of creating decision aids. Toward this end, the concept of special regions is introduced. In one set of situations special regions designate areas that are dangerous, and require teleoperation. We show how to move through single route and multi-route situations, and prove the later problem NP-Complete. In another set of situations, special regions can be used to represent areas outside direct radio contact. We present a way to minimize communication distance and plan for interventions. We relate our findings to concepts of neglect time, interaction time, and fan-out. We discuss a measure of effective fan-out for transportation tasks, and present simulation results. The work has potential impact to those engaged in emergency response and search and rescue.

The goal of this study is to determine the effect of `maintain position' and `relax' tasks on the dynamic of the foot while manipulating a gas pedal. The foot is viewed as a mass-spring-damper system, of which the visco-elasticity can be altered by reflexive feedback and muscle (co-)contraction. The dynamic properties of the foot are described by the mechanical admittance, which determines the foot position as a dynamic function of an external force perturbation. It is hypothesized that humans will change their admittance based on task perception. Experiments were done to estimate the endpoint foot admittance with Frequency Response Functions (FWs). An experimental setup that can simulate a gas pedal with different static and dynamic properties was used to apply continuous force perturbations to the foot. Eight subjects were instructed to either minimize the resulting pedal deviations (`maintain position 3 or do nothing and just rest their foot on the pedal (`relax'). The force perturbation, pedal position, and reaction force were measured, and transformed tot the frequency domain to estimate the closed-loop admittance. All subjects showed a considerable difference between the two tasks, confirming the hypothesis that drivers change the dynamics of their foot to best accomplish a perceived task.

The goal of this study is to determine the effect of amplitude and frequency of force sinusoids on force perception of the foot, in order to design an effective haptic feedback system for gas pedals. Eight subjects were asked to push a gas pedal to a constant workpoint position against a background force of 25 N Force perception was determined for three frequencies and three types of footwear by requiring subjects to respond with `yes' or `no' after each force sinusoid. Psychometric functions were calculated from the data, relating the ratio of yes answers (averaged over all subjects) to the amplitude of the force sinusoid. Although large standard deviations were found for low ratios, a statistically signifcant Just Noticeable Difference (JND) could be determined for the upper boundary of perception. Increasing the frequency of the stimulus decreased the JND. Footwear was shown to have a substantial impact on the JND at all frequencies, the largest effect occurring at the lowest frequency.

Instrument controls in motor vehicles have haptic properties (force and compliance) designed to enhance the ease of use. In conventional control knobs, these properties are obtained mechanically via springs, detents, hard stops and the like. The work reported here concerns the use of a single force-feedback knob to emulate the feel of various conventional control knobs, thus retaining their desirable haptic properties while allowing multiple functions to be controlled through one device. Compared to existing haptic knobs, this one is novel in its use of a brake to provide high torque capability in a small volume.

The design of interfaces for automotive information systems is a critical task. In fact, such design must take into account that user is busy in the primary driving task, and any visual distraction determined by telematics systems can cause serious safety problems. To limit such distraction and enhance safety, in this paper we propose a novel multimodal user interface. The key element of the proposal is a new interaction device, named Handy, conceived to exploit the drivers tactile channel to minimize the workload of visual channel. Moreover Handy is suitably integrated with the graphical user interface, which is characterized by a reduced number of choices for each state and has been designed in agreement with the self-revealing approach.

Rapid and accurate visuomotor coordination requires tight spatial and temporal sensorimotor synchronization. The introduction of a sensorimotor or intersensory misalignment (either spatial or temporal) impairs performance on most tasks. For more than a century, it has been known that a few minutes of exposure to a spatial misalignment can induce a recalibration of sensorimotor spatial relationships, a phenomenon that may be referred to as spatial visuomotor adaptation. Here, we use a high-fidelity driving simulator to demonstrate that the sensorimotor system can adapt to temporal misalignments on very complex tasks, a phenomenon that we refer to as temporal visuomotor adaptation. We demonstrate that adapting on a single street produces an adaptive state that generalizes to other streets. This shows that temporal visuomotor adaptation is not specific to a single visuomotor transformation, but generalizes across a class of transformations. Temporal visuomotor adaptation is strikingly parallel to spatial visuomotor adaptation, and has strong implications for the understanding of visuomotor coordination and intersensory integration.

Three on-road studies were conducted to determine how headway maintenance and collision warning displays influence driver behavior. Visual perspective, visual perspective with a pointer, visual perspective combined with an auditory warning, discrete visual warning, and discrete auditory warning were assessed during both coupled headway and deceleration events. Results indicate that when drivers are provided with salient visual information regarding safe headways, they utilize the information and increase their headway when appropriate. Auditory warnings were less effective than visual warnings for increasing headways but may be helpful for improving reaction time during events that require deceleration. Drivers were somewhat insensitive to false alarm rates, at least during short-term use. Finally, and most important, driver headway maintenace increased by as much as 0.5 s when the appropriate visual display was used. however, a study to investigate the long-term effects of such displays on behavior is strongly recommended prior to mass marketing of headway maintenance/collision warning devices.

Sensory overloaded environments present an opportunity for innovative design in the area of Human-Machine Interaction. In this paper we study the usefulness of a tactile display in the automobile environment. Our approach uses a simple pneumatic pump to produce pulsations of varying frequencies on the driver's hands through a car steering wheel fitted with inflatable pads. The goal of the project is to evaluate the effectiveness of such a system in alerting the driver of a possible problem, when it is used to augment the visual display presently used in automobiles. A steering wheel that provides haptic feedback using pneumatic pockets was developed to test our hypothesis. The steering wheel can pulsate at different frequencies. The system was tested in a simple multitasking paradigm on several subjects and their reaction times to different stimuli were measured and analyzed. For these experiments, we found that using a tactile feedback device lowers reaction time significantly and that modulating frequency of vibration provides extra information that can reduce the time necessary to identify a problem.

The bulk of current haptics human-factors research focuses on mapping basic human perceptual limits. However, many realistic applications demand a better understanding of how to construct more life-like but often less controllable experiment scenarios. In this paper, we study this problem in the context of advanced automobile interfaces. We employ a throttle pedal with programmable force feedback to indicate potentially undesirable situations in the external environment and to gently but steadily guide the driver away from them. We have found evidence that within this scenario, errors in such a warning signal can have a negative effect on the behavior of the driver within the conditions studied. These experiments required a complex protocol and necessarily permitted a variety of participant tactics. Post-experiment analysis revealed that very subtle variations in participant instruction produced large differences in tactics and consequent experiment outcome.

During manual steering of a vehicle, the information to the driver is restricted by his perception and recognition. The driver may lose such information under severe weather conditions. Look-ahead information has been shown to have great importance for vehicle steering problems. A supplementary visual display has been developed to provide both the fiture road information and the future position of the vehicle based on current vehicle state and steering input for safer driving. A state observer is proposed to estimate the current vehicle state. Both the current steering input and the estimated vehicle state are utilized to predict the future position of the vehicle at a prescribed look-ahead distance. Integration of the vehicle trajectory into future is avoided by an approximation method, which can be applied in real-time implementation.

In this paper, a paradigm for shared control is described in which a machine's manual control interface is motorized to allow a human and an automatic controller to simultaneously exert control. The manual interface becomes a haptic display, relaying information to the human about the intentions of the automatic controller while retaining its role as a manual control interface. The human may express his control intentions in a way that either overrides the automation or conforms to it. The automatic controller, by design, aims to create images in the mind of the human of fixtures in the shared workspace that can be incorporated into efficient task completion strategies. The fixtures are animated under the guidance of an algorithm designed to automate part of the human/machine task. Results are presented from 2 experiments in which 11 subjects completed a path following task using a motorized steering wheel on a fixed-base driving simulator. These results indicate that the haptic assist through the steering wheel improves lane keeping by at least 30% reduces visual demand by 29% (p<0.0001) and improves reaction time by 18 ms (p=0.0009).

We report two experiments designed to investigate the potential use of vibrotactile warning signals to present spatial information to car drivers. Participants performed an attention-demanding rapid serial visual presentation (RSVP) monitoring task. Meanwhile, whenever they felt a vibrotactile stimulus presented on either their front or back, they had to check the front and the rearview mirror for the rapid approach of a car, and brake or accelerate accordingly. We investigated whether speeded responses to potential emergency driving situations could be facilitated by the presentation of spatially-predictive (80% valid; Experiment 1) or spatially-nonpredictive (50% valid; Experiment 2) vibrotactile cues. Participants responded significantly more rapidly following both spatially-predictive and spatially-nonpredictive vibrotactile cues from the same rather than the opposite direction as the critical driving events. These results highlight the potential utility of vibrotactile warning signals in automobile interface design for directing a drivers visual attention to time-critical events or information.

The potential use of non-visual warning signals to present spatial information to car drivers has been successfully demonstrated in several recent studies (Ho & Spence, submitted, in preparation; Ho, Tan, & Spence, submitted). Among the three types of spatial warning signals investigated (namely auditory, visual, and vibrotactile), spatial vibrotactile cues were found to be particularly effective in directing a driver's visual spatial attention to potentially dangerous events on the road. We conducted the present study in order to examine the factors governing the relative effectiveness of auditory, visual, and vibrotactile warning signals. The speeded discrimination of warning signals presented in the various different modalities was investigated in order to explore whether the differences found in our previous research were a result of the relative speed with which people can detect warning signals presented in a given modality, or whether they were attributable to differences in the efficacy with which people can relate the warning signal to the subsequent visually-specified target driving events.

This article reports the development of a multidimensional measure of subjective fatigue states, and its associations with personality in an experimentally-controlled context. In a study of simulated driving, 256 subjects completed a new 24-item fatigue scale as well as other subjective state measures, before and after performing a fatiguing drive. An item factor analysis identified four correlated dimensions: visual fatigue, muscular fatigue, boredom and malaise. The scales were sensitive to increased fatigue following the fatiguing drive, and showed a high degree of internal consistency. The fatigue scales correlated substantially with general state measures, such as mood and motivation. A factor analysis of fatigue and other state scales identified second-order factors of task disengagement (including boredom), physical fatigue (including the other three fatigue scales), and a distress factor. The fatigue scale was also correlated with the EPQ-R and with a measure of traits related specifically to driving, the Driving Behaviour Inventory (DBI), which includes a Fatigue Proneness scale. Bivariate and multivariate analyses showed that Fatigue Proneness was the strongest single predictor of task-induced fatigue symptoms, as predicted from an interactionist analysis of relationships between traits and states. However, the relationship between traits and states associated with fatigue was complex, and other EPQ-R and DBI traits, including neuroticism, were independently associated with fatigue.

The research reported in this article considers the effects of automating driver's control tasks. Driving can be broken down into 3 general subtasks: navigation, control, and hazard avoidance. Control can be further subdivided into lateral control (positioninlane)and longitudinal control (speed and leading headway). Lateral control can be automated by an active steering (AS) system, and longitudinal control can be automated by an adaptive cruise control (ACC) system. Previous research has used driving simulators to consider the effects of driver workload and the ability to reclaim control with these systems. There are, however, some questions about the validity of driving simulators, and this research sought to validate a driving simulator. This was achieved by comparing responses on a secondary task and driving style questionnaire in both a road car and a driving simulator. When validity was established, a comparison of 4 levels of automation was undertaken: manual, ACC, AS, and ACC plus AS. The results showed no reduction in workload associated with ACC over manual driving, but reduction in workload associated with AS and further reduction in workload associated with ACC plus AS. Despite these reductions in workload, there were no adverse affects on normal driving performance. However, the presence of vehicle automation seemed to make drivers less likely to reclaim control in an emergency-braking scenario.

When humans interface with machines, the control interface is usually passive and its response contains little information pertinent to the state of the environment. Usually, information flows through the interface from human to machine but not so often in the reverse direction. This work proposes a control architecture in which bi-directional information transfer occurs across the control interface, allowing the human to use the interface to simultaneously exert control and extract information. In this alternative control architecture, which we call shared control, the human utilizes the haptic sensory modality to share control of the machine interface with an automatic controller. We present a fixed-base driving simulator experiment in which subjects take advantage of a haptic steering wheel, which aids them in a path following task. Results indicate that the haptic steering wheel allows a significant reduction in visual demand while improving path following performance.

Although it is not always possible to derive from statstics the causes of an accident, data clearly show that they are mostly due to a lack of driver's perception, with high social costs for society (direct and not). Therefore, a strong integration among all the actors involved (that is the drivers, the vehicle or technical system in general and the environment) is really necessary. From first years of 80's, there has been a shift in sysfem concept design, moving from a technological approach towards a human centred design approach. Under this point of view, if is necessary to take into account on one side the user's needs in the design process of a certain system, and on the other side also the ``machine needs''. Thus, this paper aims at presenting such types of needs as well as the mutual interaction between machine and users and both with the surrounding environment.

This paper describes the potential of using vibro-tactile displays for automobile drivers. Technological developments in the field of driver support systems and tactile displays, combined with the ever increasing need to enlarge the capacity of the driver's information channel, form the reason to review the possibilities of in-car tactile displays and to identify some promising applications. In the second part of the paper, we describe a feasibility study in which we tested an in-car tactile display in a driving simulator. The results show that the tactile navigation display resulted in better performance compared to a visual display, and that it reduces the driver's workload. This study gives a first indication that employing the tactile modality may be a major step to accomplish safety improvements.

A comparison of walking against vertical (gradient) and horizontal (trailing weight) forces was made during steady-rate exercise at 0,250, 500, and 750 kg.m/min with speeds of 3.0, 4.5, and 6.0 km/h. In all cases exponential relationships between energy expenditure (calculated from the steady-rate respiration) and increasing work rate and speed were observed which indicated that muscular efficiency during walking is inversely related to speed and work rate. ``Work'' (level, unloaded walking as the baseline correction), ``delta'' (measured work rate as the baseline correction), and ``instantaneous'' (derived from the equation describing the caloric cost of work) efficiencies were computed. All definitions yielded decreasing efficiencies with increasing work rates. At work rates above 250 kg' m/min the curves describing the relationship between energy expenditure and work rate were parallel for vertical and horizontal forces, indicating equivalent efficiencies in this range. Only the delta and instantaneous definitions accurately described these relationships for vertical and horizontal work. Determinations of combined work loads (gradient plus trailing weight) were made and the energy costs of both types of work found to be additive.

The use of self-selected intensities of exercise may increase adherence to exercise programs by allowing the participant more freedom to choose activities that are enjoyable. However, self-selected intensities may vary across exercise modes, and participants may not choose an intensity that is adequate to produce health benefits. The purpose of this study was to determine influence of exercise mode on self-selected exercise intensities. Eighteen subjects (12 men and 6 women) between the ages of 18 and 25 participated in this study. Preferred intensity tests were performed for 3 modes of exercise (treadmill, cycle ergometer, and stairstepper). VO2 values were obtained continuously and 1-minute averages were recorded at minutes 5, 10, 15, and 20 for each submaximal test. Comparisons were made using a repeated-measures analysis of variance (mode, time, mode time). Scheffe's F test was used to test simple effects. There was a significant increase in the relative VO2 for all 3 modes of exercise across the 20-minute trials (p 0.0001). Relative VO2 (%VO2peak) increased from 52.20 to 64.71% for cycle exercise, 43.27 to 63.25% for the treadmill, and 47.16 to 61.17% for stairstepping. The average relative VO2 for the cycle ergometer (60.40 +/- 15.55%) was significantly higher (p 0.02) than both the treadmill (53.65 +/- 18.36%) and stairstepper (54.66 +/- 11.98%). Relative heart rate (HR) (%HRR) for the stairstepper (80.21+/- 9.67%) and the cycle ergometer (80.03 +/- 10.59%) were significantly higher than the treadmill (74.77 +/- 13.13%) (P 0.0003). There were no significant differences in rating of perceived exertion (RPE) among the 3 modes of exercise. Similar RPE values were reported for the stairstepper (12.79 +/- 2.97), cycle ergometer (12.57 +/- 2.90), and the treadmill (12.50 +/- 2.87). The results indicate that subjects allowed to choose exercise intensity by self-selection chose work rates that were within the moderate range of American College of Sports Medicine guidelines of 50--85% VO2max for treadmill, cycle ergometer, and stairstepping exercise.

The primary purpose of this study was to determine if the aerobic demand for production of specified power outputs is altered by distribution of work between the arms and legs compared with when all the work is performed by the legs. Because of the important exercise training implications, a secondary purpose of this study was to determine if the exercising muscle mass affects the cardiorespiratory demands at specified rating of perceived exertion (RPE) levels and blood lactate concentrations. Nine healthy adults completed leg cycling and combined arm and leg exercise on an Airdyne using a discontinuous protocol. Repeated measures ANOVA revealed that oxygen uptake for the combined arm and leg exercise averaged 0.04 l - min)1 greater (p < 0.05) than for leg cycling at the same external power outputs. However, RPE levels at specified power outputs were lower (p < 0.05) with combined arm and leg exercise than leg cycling. At specified RPE levels and blood lactate concentrations, oxygen uptake and heart rate values were higher (p < 0.05) for combined arm and leg exercise than leg cycling. From these findings we conclude that: (1) the addition of arm exercise to leg cycling results in a reduction in RPE, but a minimal increase in oxygen consumption to perform a given power output, and (2) if training intensity is established by RPE or blood lactate concentration, use of a muscle mass larger than that used in leg cycling should allow a greater cardiorespiratory training effect.

The power output of an athlete decreases rapidly with time. The author describes ditferent ways of measuring power output, including a unique method, developed at Loughborough University for measuring the power developed by a sprint runner.

Purpose: Based on the resistance-rpm relationship for cycling, which is not unlike the force-velocity relationship of muscle, it is hypothesized that the cadence which requires the minimal muscle activation will be progressively higher as power output increases. Methods: To test this hypothesis, subjects were instrumented with surface electrodes placed over seven muscles that were considered to be important during cycling. Measurements were made while subjects cycled at 100, 200, 300, and 400 W at each cadence: 50, 60, 80, 100, and 120 rpm. These power outputs represented effort which was up to 32% of peak power output for these subjects. Results: When all seven muscles were averaged together, there was a proportional increase in EMG amplitude each cadence as power increased. A second-order polynomial equation fit the EMG:cadence results very well (r2 0.87-- 0.996) for each power output. Optimal cadence (cadence with lowest amplitude of EMG for a given power output) increased with increases in power output: 57 3.1, 70 3.7, 86 7.6, and 99 4.0 rpm for 100, 200, 300, and 400 W, respectively. Conclusion: The results confirm that the level of muscle activation varies with cadence at a given power output. The minimum EMG amplitude occurs at a progressively higher cadence as power output increases. These results have implications for the sense of effort and preferential use of higher cadences as power output is increased.

A cycle ergometer was modified to measure power (P) with resistance provided solely by the moment of inertia (I) of the flywheel. P was calculated as the product of I, angular velocity ([omega]), and angular acceleration([alpha]). Flywheel [omega] and [alpha] were determined by means of an optical sensor and a micro-controller based computer interface which measured time(+/-1 microsecond) and allowed P to be calculated instantaneously(PI) every 3[degrees] of pedal crank rotation or averaged over one complete revolution of the pedal cranks (PREV). Values for maximum P were identified from each bout (PI max and PREV max). Mechanical calibration of torque via a resistive strap proved this method to be both valid and accurate. Thirteen active male subjects performed four bouts of maximal acceleration lasting approximately 3-4 s with 2 min resting recovery. The mean coefficient of variation for PREV max was 3.3 +/- 0.6% and the intraclass correlation was 0.99. PREV max averaged 1317+/- 66 W at 122 +/- 2 rpm, and PI max averaged 2137 +/- 101 W at 131 +/- 2 rpm. PREV max and PI max were highly correlated (r = 0.86 and r = 0.80 respectively, P < 0.002) with estimated lean thigh volume. Therefore, the inertial-load method provides a valid and reliable determination of cycling power in one short exercise bout.

A novel apparatus, composed by a controllable treadmill, a computer, and an ultrasonic range finder, is here proposed to help investigation of many aspects of spontaneous locomotion. The acceleration or deceleration of the subject, detected by the sensor and processed by the computer, is used to accelerate or decelerate the treadmill in real time. The system has been used to assess, in eight subjects, the self-selected speed of walking and running, the maximum ``reasonable'' speed of walking, and the minimum reasonable speed of running at different gradients (from level up to 25%). This evidenced the speed range at which humans neither walk nor run, from 7.2 0.6 to 8.4 1.1 km/h for level locomotion, slightly narrowing at steeper slopes. These data confirm previous results, obtained indirectly from stride frequency recordings. The self-selected speed of walking decreases with increasing gradient (from 5.0 0.8 km/h at 0% to 3.0 0.9 km/h at 25%) and seems to be 30% higher than the speed that minimizes the metabolic energy cost of walking, obtained from the literature, at all the investigated gradients. The advantages, limitations, and potential applications of the newly proposed methodology in physiology, biomechanics, and pathology of locomotion are discussed in this paper.

Robotic devices hold much promise for use as rehabilitation aids but their success depends on identifying effective strategies for controlling human--robot interaction forces. We developed a robotic device to test a novel method of controlling interaction forces with the intent of improving force symmetry in the limbs. Users perform lower limb extensions against a computer-controlled resistive load. The control software increases resistance above baseline in proportion to lower limb force asymmetry (balance between left and right limb forces). As a preliminary trial to test the device and controller, we conducted two experiments on neurologically intact subjects. In experiment 1, one group of subjects received symmetry-based resistance while performing lower limb extensions (n = 10). A control group performed the same movements with constant resistance (n = 10). The symmetry-based resistance group improved lower limb symmetry during training (ANOVA, p<0.05), whereas the control subjects did not. In experiment 2, subjects (n = 10) successfully used symmetry-based resistance to alter their lower limb force production towards a target asymmetry (ANOVA, p<0.05). These studies suggest that symmetry-based resistance may hold rehabilitation benefits after orthopedic or neurological injury. Specifically, performing strength training therapy with this controller may allow hemiparetic individuals to focus better on increasing strength and neuromuscular recruitment in their paretic limb while experiencing symmetric limb forces.

Aspects of anaerobic and aerobic energy conversion are investigated using a mathematical model of running in conjunction with world-record statistics. Analysis of the data shows that over distances from 1500 to 10,000 m the anaerobic energy utilised is constant and independent of running distance. This result is consistent with the view that the full potential of the anaerobic capacity is available for conversion during extended periods of running; the opinions of Gollnick and Hermansen (1973) and Peronnet and Thibault (1989) that the anaerobic energy contribution declines with race duration are not corroborated. The analysis supports the finding of Peronnet and Thibault (1989) that, for running times below about 1"420 s, the maximum sustainable aerobic power is constant, and that for larger 1 it then declines progressively. The present analysis shows it falls by some 4.5% over 10,000 m, 1+1600 s, indicating that in establishing current world records at 5000 and 10,000 m athletes did not rely solely on glycogen as the source of aerobic metabolism; limited use was made of free fatty acids. For elite male runners, the anaerobic capacity and maximal aerobic power are evaluated as 1570 J/ kg and 27.1 W/ kg, respectively.

The IM4Sports music system helps exercisers select music that suits their training programs, reflects and guides sport performance, and collects data for adapting training programs and music selections.

A multitude of indoor exercise machines are promoted for improving aerobic fitness and for controlling body weight. Previous investigations have performed physiological comparisons of some of these exercise machines. However, a comprehensive comparison of the aerobic and energy demands relative to level of perceived exertion has not been performed. Such a comparison is important because the intensity of exercise is often determined by the perceived effort. The primary purpose of this study was to compare 6 commonly used, popular indoor aerobic exercise machines to determine which would elicit the greatest rate of energy expenditure at specified levels of perceived exertion.

Ever wondered if your glutes, lats and pecs are getting the workout they deserve? When your exercise equipment is retrofitted with the Mytrak Health System, you'll know for sure. Developed for the health club market by InCorp Ventures Inc. (mytrakhealth.com) of Mississauga, ON, the system adds intelligence and automation to exercise equipment.

This paper describes the simulation of stairs on a treadmill style locomotion interface using torso force feedback. The active mechanical tether of the Sarcos Treadport locomotion interface applies a specialized force profile to simulate the forces of stair walking. The biomechanics of subjects walking on real stairs versus walking under the specialized force prolile were compared. It was found that the tether force was able to adjust the subject's motion from standard slope walking towards that of stairs.

The control of a one degree of freedom exercise machine is considered. The control objective consists in making the human user exercise in a manner that maximizes his consumption of power. The optimality condition is determined by the muscle mechanics which is assumed to satisfy a force-position-velocity relationship. In general, the parameters of this relationship are unknown and vary with the configuration of the exercise machine. As a consequence, the control scheme must simultaneously i) identify the user's strength characteristic, ii) optimize the controller, and iii) stabilize the system to the estimated optimal state. In this paper we present control systems in the form of a nonlinear dynamic or static dampers that make the controlled system interact passively with the user. Adaptive and self-optimizing control strategies are discussed, which achieve the control objectives described above. Results of a clinical study are presented which corroborate many of the assumptions used in this paper and verify the efficacy of the proposed control schemes.

This article discusses the dynamics and design of multiple-degree-of-freedom robotic systems built as general purpose exercise machines for the human arm. These machines may be programmed to give the human arm the sensation of forces associated with various arbitrary maneuvers. As examples, these machines can give the human the sensation that he/she is maneuvering a mass, or pushing onto a spring or a damper. In general, the machines may be programmed for any trajectory-dependent force. To illustrate and verify the analysis of these machines, a two-degree-of-freedom electrically-powered exercise machine was designed and built at the Motion Control Laboratory of the University of California-Berkeley.

In a self-optimizing control problem, it is desired that the plans with a priori unknown parameters perform a task that optimizes a performance index. If the optimal task can be specified explicitly, an adaptive controller can often be designed to enable that task to be performed. However in many cases, the optimal task cannot be explicitly specified becausee it may depend on the unknown plan's parameters, and must also be determined explicitly on-line. In the context of intelligent controllers for exercise machines, the resistive/assistive force on the machine is manipulated to cause the user of the machine to maximize his/her mechanical power output while exercising. The optimal manner in which the user exercises is represented by a velocity field which is a functio of the individual's unknown biomechanic characteristics. The proposed self-optimizing control approach combines i) a continuous state adaptive controller which enables an arbitrary explicitly specified task to be performed; and ii) a finite state excitation supervisor which switches the desired task between a training task and the estimated optimal task based on current system parameter estimates. Depending on the switching scheme chosen, it is shown that the user will asymptotically either execute the true optimal exercise with probability one or operate close to it. Experimental results of the implementation verifies the efficacy of the design.

In order to achieve the general exercise objectives of increasing i) strength, ii) stamina, and iii) cardiovascular workout, a proper exercise regime has to achieve certain tradeoffs specific to the individual's muscle mechanics and the fatigue states. An adaptive approach is proposed to control a 1 degree of freedom (DOF) exercise machine in which the exercise regime is modified according to the estimates of the force-velocity relationship (Hill's curve) of the muscle. An optimal velocity profile is determined from the Hill's curve via a class of power related criteria. The adaptive controller simultaneously identifies the Hill's curve, and causes the user to operate at the estimated optimal velocity profile.The controller always interacts with the user passively. It is believed that the adaptive controller is able to modify the exercise regime accordingly an fatigue acts in. Stability and convergence results have been rigorously proven and in here some preliminary experimental results are presented.

The control of a one degree of freedom exercise machine is considered. The control objective is to cause the human user to exercise at a rate which optimizes a prescribed weigthed power criterion. The optimality condition is determined by the muscle mechanics which is assumed to satisfy a force-velocity relationship. In general, the parameters of this relationship are unknown and vary with the configuration of the exercise machine. As a consequence, the control scheme must simultaneously i) identify the parameters, ii) optimize the controller, and iii) stabilize the system to the estimated optimal states. In this paper we derive a controller which is in the form of a nonlinear dynamic damper and makes the controlled system interact passively with the user. Assuming that the human's force-velocity muscular biomechanics relationship is known, this controller allows the user to excercise in an optimal manner.

This is the first part of a two-part paper on the design of intelligent controllers for a class of exercise machines. The control objective is to cause the user to exercise in a manner which optimizes a criterion related to the user's mechanical power. The optimal exercise strategy is determined by a biomechanical behavior of the individual user, which is assumed to satisfy an affine force--velocity relationship dependent on the body geometric configuration. Consequently, the control scheme must simultaneously: 1) identify the user's biomechanical behavior; 2) optimize the controller; and 3) stabilize the system to the estimated optimal states. Moreover, to ensure that the exercise machine is safe to operate, the control system guarantees that the interaction between the exercise machine and the user is passive. In this first part of the paper, we formulate the control problem and propose a controller structure which satisfies the safety requirement and is capable of causing the user to execute an arbitrary exercise strategy if the user's biomechanical behavior is known. The controller is of the form of a dynamic damper and can be implemented using only passive mechanical components. Part II of this paper is concerned with the self-optimization problem, in which both the determination of the optimal exercise strategy and the execution of that strategy, when the user's biomechanical behavior is unknown, must be considered.

This is the second part of a two-part paper on the design of an intelligent controller for a class of exercise machines. The control objective is to cause the user to exercise in a manner that optimizes a criterion related to the user's mechanical power. The optimal exercise strategy is determined by an a priori unknown biomechanical behavior, called the Hill surface, of the individual user. Consequently, the control scheme must simultaneously: 1) identify the user's biomechanical behavior; 2) optimize the controller; and 3) stabilize the system to the estimated optimal states. In Part I of this paper, a dynamic damping controller was proposed which satisfies the safety requirement and is capable of causing the user to execute an arbitrary exercise strategy if the user's biomechanical behavior is known. In this second part of the paper, we address the self-optimization problem in which both the determination and the eventual execution of the optimal exercise strategy are accomplished, when the user's biomechanical behavior is unknown. This is achieved by a combination of an adaptive controller and a reference generator. The latter switches the desired exercise strategy between a training strategy and the estimated optimal strategy. Depending on the switching scheme chosen, it is shown that, asymptotically, the user will either execute the optimal exercise with probability one or operate close to it. Experimental results of the overall system verify the efficacy of the design.

Purpose: Based on the resistance-rpm relationship for cycling, which is not unlike the force-velocity relationship of muscle, it is hypothesized that the cadence which requires the minimal muscle activation will be progressively higher as power output increases. Methods: To test this hypothesis, subjects were instrumented with surface electrodes placed over seven muscles that were considered to be important during cycling. Measurements were made while subjects cycled at 100, 200, 300, and 400 W at each cadence: 50, 60, 80, 100, and 120 rpm. These power outputs represented effort which was up to 32% of peak power output for these subjects. Results: When all seven muscles were averaged together, there was a proportional increase in EMG amplitude each cadence as power increased. A second-order polynomial equation fit the EMG:cadence results very well (r2 0.87-- 0.996) for each power output. Optimal cadence (cadence with lowest amplitude of EMG for a given power output) increased with increases in power output: 57 3.1, 70 3.7, 86 7.6, and 99 4.0 rpm for 100, 200, 300, and 400 W, respectively. Conclusion: The results confirm that the level of muscle activation varies with cadence at a given power output. The minimum EMG amplitude occurs at a progressively higher cadence as power output increases. These results have implications for the sense of effort and preferential use of higher cadences as power output is increased.

A novel apparatus, composed by a controllable treadmill, a computer, and an ultrasonic range finder, is here proposed to help investigation of many aspects of spontaneous locomotion. The acceleration or deceleration of the subject, detected by the sensor and processed by the computer, is used to accelerate or decelerate the treadmill in real time. The system has been used to assess, in eight subjects, the self-selected speed of walking and running, the maximum ``reasonable'' speed of walking, and the minimum reasonable speed of running at different gradients (from level up to 25%). This evidenced the speed range at which humans neither walk nor run, from 7.2 0.6 to 8.4 1.1 km/h for level locomotion, slightly narrowing at steeper slopes. These data confirm previous results, obtained indirectly from stride frequency recordings. The self-selected speed of walking decreases with increasing gradient (from 5.0 0.8 km/h at 0% to 3.0 0.9 km/h at 25%) and seems to be 30% higher than the speed that minimizes the metabolic energy cost of walking, obtained from the literature, at all the investigated gradients. The advantages, limitations, and potential applications of the newly proposed methodology in physiology, biomechanics, and pathology of locomotion are discussed in this paper.

This paper discusses the design of a compatible controller for the Expert-Based Variable Resistance/Assistance (EVRA) exercise machine that removes the shortcomings found in the currently available constant-resistance and other variable resistance exercise machines. A mathematical model of the EVRA prototype is used to simulate its dynamic behavior using the Matlab/Simulink software package. The feed-back controller generates control signals to engage an electric motor to provide either assistance/resistance as per demand. The proposed controller is able to detect significant changes in the kinematic and neurophysiological movement profiles, compare this data with an existing database and then provide the appropriate level of mechanical assistance to the moving limb to maintain a coordinated movement profile. A comparative study on the various types of controllers (such as PI and Neuro-controllers) is also presented.

In this dissertation, the design of resistance controllers for exercise machines is considered. Whereas most exercise machines perform a preprogrammed routine, the controllers designed in this dissertation are sensitive to the performance of the user by incorporating feedback on the state of the user. Since the performance of different users can vary greatly, the controllers are designed to be adaptive to the different strength levels and behaviors of a subject. Two examples of incorporating feedback from the user into the controller of an exercise machine are demonstrated in this dissertation. The first machine optimizes the power produced by the user during a workout. Two examples of this type of exercise machine are considered. One is an arm cranking exercise that uses a variable reluctance direct drive motor to provide the resistance for the exercise. The second machine is a recumbant bicycle that usees an eddy current brake to provide resistance. The kinematics and dynamics of the exercise motions for these machines are similar. Using a combination of modeling and experimentation, the inertia, Coriolis, gravity, and friction of the exercise motions are identified and validated. In these control systems, the user's strength is parameterized based on a model of their force output. The model accounts for the position, velocity, and time dependence of the user's force. The velocity dependence of force is assumed to satisfy a linear relationship that decreases with increasing velocity. The strength is then used to define an optimal velocity profile that maximizes the power generated by the user. A velocity controller is designed for these machines based on damping resistance that can track within error bounds the optimal velocity profile, or any other desired velocity profile, such as the common isokinetic exercise. The controller identifies the parameters of the strength model and uses the model to determine the proper amount of feedforward damping. Feedback of the tracking error is used to adjust the impedance of the controller, so that tracking of the desired velocity profile is improved. Using Lyapunov methods, bounded tracking error and passivity of the controller with respect to the user's force ar proven for the controllers described above. Clinical studies are performed toverify the utility of the strength model, the tracking ability of the controller, and optimality of the maximum power regime relative to benchmark isokinetic exercises. The results of the clinical studies indicate the optimal power regime is short erm optimal and long term optimal, despite the larger fatigue rate associated with the high velocity optimal exercise. The force-velocity relationship for the subjects was found to be linear and the power-velocity relationship was found to be parabolic, as the strength model used would predict. The position dependence of the force for both the arm and recumbent bicycle machines was found to agree with the kinematics of the motion. The force-velocity characteristics for each subject was found to be highly variable, validating the need for an adaptive scheme, such as the one proposed in this dissertation, to implement the optimal power workout. The second type of machine designed in this dissertation is a stair stepper with a controller designed to regulate the step rate. This machine essentially performs the dual task of the first machine, by removing the variable behavior of the user from the controlled system, instead of reacting to it. The resistance mechanism in this machine is similar to the eddy current brake used in the recumbant bicycle, except that a permanent magnet is used, and the resistance is adjusted by positioning the magnets with a servo mechanism. Experiments are performed to identify both the servo dynamics and nonlienar step rate dynamics. An adaptive feedforward step rate regulator is designed for this machine that rejects changes in gait and in the effective weight applied to the step pedals, to maintain a constant desired step rate. These type of disturbances are referred to as ``psychological disturbances'' because they are dependent on decisions that the user makes. The multivariable circle criterion is used to prove the local asymptotic stability of the controlled system. Experiments are completed that demonstrate the regulation of the step rate both under nominal circumstances and when disturbances are acting. Considerations for extending the approach used to control the step rate to control of the heart rate are briefly discussed. Since any complete controller design for an exercise machine must incoporate a synthesis of the control with the resistance mechanism, mechatronic issues relating to the calibration and design of the resistance mechanisms for all three exercise machines considered in this dissertation is a recurring subtheme.

In this paper adaptive control of a stair stepper exercise machine is considered. The dynamics of the functional components of the exercise machine are modeled and experimentally identifed. Based on this identified system, a regulator is designed to control the step rate both under nomimal conditions and when disturbances are acting. The disturbances acting on the system result from the unpredictable behavior of the user. Using the multivariable circle criterion, the closed loop error dynamics of the system are proven to be locally asymptotically stable. Experimental results confirm the analysis and demonstrate the nominal and disturbance rejection properties of the controller.

Robotic devices hold much promise for use as rehabilitation aids but their success depends on identifying effective strategies for controlling human--robot interaction forces. We developed a robotic device to test a novel method of controlling interaction forces with the intent of improving force symmetry in the limbs. Users perform lower limb extensions against a computer-controlled resistive load. The control software increases resistance above baseline in proportion to lower limb force asymmetry (balance between left and right limb forces). As a preliminary trial to test the device and controller, we conducted two experiments on neurologically intact subjects. In experiment 1, one group of subjects received symmetry-based resistance while performing lower limb extensions (n = 10). A control group performed the same movements with constant resistance (n = 10). The symmetry-based resistance group improved lower limb symmetry during training (ANOVA, p<0.05), whereas the control subjects did not. In experiment 2, subjects (n = 10) successfully used symmetry-based resistance to alter their lower limb force production towards a target asymmetry (ANOVA, p<0.05). These studies suggest that symmetry-based resistance may hold rehabilitation benefits after orthopedic or neurological injury. Specifically, performing strength training therapy with this controller may allow hemiparetic individuals to focus better on increasing strength and neuromuscular recruitment in their paretic limb while experiencing symmetric limb forces.

A multitude of indoor exercise machines are promoted for improving aerobic fitness and for controlling body weight. Previous investigations have performed physiological comparisons of some of these exercise machines. However, a comprehensive comparison of the aerobic and energy demands relative to level of perceived exertion has not been performed. Such a comparison is important because the intensity of exercise is often determined by the perceived effort. The primary purpose of this study was to compare 6 commonly used, popular indoor aerobic exercise machines to determine which would elicit the greatest rate of energy expenditure at specified levels of perceived exertion.

Experimental results are obtained for a single degree of freedom prototype next generation exercise machine that aims to maximize the user's power output and ensure passivity with the user. In an effort to optimize the user's power expenditure, the desired velocity trajectory is developed that seeks the unknown user-dependent optimal velocity setpoint. A numerical extremum-seeking algorithm is utilized to seek the optimal velocity setpoint while ensuring the trajectory is sufficiently differentiable. To track the reference trajectory and to ensure passivity, a nonlinear controller is utilized.

The goal of this study is to determine the effect of `maintain position' and `relax' tasks on the dynamic of the foot while manipulating a gas pedal. The foot is viewed as a mass-spring-damper system, of which the visco-elasticity can be altered by reflexive feedback and muscle (co-)contraction. The dynamic properties of the foot are described by the mechanical admittance, which determines the foot position as a dynamic function of an external force perturbation. It is hypothesized that humans will change their admittance based on task perception. Experiments were done to estimate the endpoint foot admittance with Frequency Response Functions (FWs). An experimental setup that can simulate a gas pedal with different static and dynamic properties was used to apply continuous force perturbations to the foot. Eight subjects were instructed to either minimize the resulting pedal deviations (`maintain position 3 or do nothing and just rest their foot on the pedal (`relax'). The force perturbation, pedal position, and reaction force were measured, and transformed tot the frequency domain to estimate the closed-loop admittance. All subjects showed a considerable difference between the two tasks, confirming the hypothesis that drivers change the dynamics of their foot to best accomplish a perceived task.

The goal of this study is to determine the effect of amplitude and frequency of force sinusoids on force perception of the foot, in order to design an effective haptic feedback system for gas pedals. Eight subjects were asked to push a gas pedal to a constant workpoint position against a background force of 25 N Force perception was determined for three frequencies and three types of footwear by requiring subjects to respond with `yes' or `no' after each force sinusoid. Psychometric functions were calculated from the data, relating the ratio of yes answers (averaged over all subjects) to the amplitude of the force sinusoid. Although large standard deviations were found for low ratios, a statistically signifcant Just Noticeable Difference (JND) could be determined for the upper boundary of perception. Increasing the frequency of the stimulus decreased the JND. Footwear was shown to have a substantial impact on the JND at all frequencies, the largest effect occurring at the lowest frequency.

This paper presents a set of interaction techniques for hands-free multi-scale navigation through virtual environments. We believe that hands-free navigation, unlike the majority of navigation techniques based on hand motions, has the greatest potential for maximizing the interactivity of virtual environments since navigation modes are of loaded from modal hand gestures to more direct motions of the feet and torso. Not only are the users' hands freed to perform tasks such as modeling, notetaking and object manipulation, but we also believe that foot and torso movements may inherently be more natural for some navigation tasks. The particular interactions that we developed include a leaning technique for moving small and medium distances, a foot-gesture controlled Step WIM that acts as a floor map for moving larger distances, and a viewing technique that enables a user to view a full 360 degrees in only a three-walled semi-immersive environment by subtly amplifying the mapping between their torso rotation and the virtual world. We formatively designed and evaluated our techniques in existing projects related to archaeological reconstructions, free-form modeling, and interior design. In each case, our informal observations have indicated that motions such as walking and leaning are both appropriate for navigation and are effective in cognitively simplifying complex virtual environment interactions since functionality is more evenly distributed across the body.

The development of a device that enables haptic foot interaction for communication over a network is presented. Considering the physical properties of feet we demonstrate that feet are suited for personal, concealed communication over a computer network. First experiments to investigate both the usability and fun of using foot interaction indicate promising results and concrete opportunities for further development.

A new method for researching haptic interaction styles is presented, based on a layered interaction model and a classification of existing devices. The method is illustrated by designing a new foot interaction device. The aim of which is to enhance non-verbal communication over a computer network. A layered protocols interaction model allows to consider all aspects of the haptic communication process: the intention to perform an action, limitations of the human body, and specifications of the communication device and the network. We demonstrate how this model can be used to derive design-guidelines by analyzing and classifying existing communication devices. By designing and evaluating a foot interaction device, we not only demonstrate that feet are suited for personal, concealed communication over a network, but also show the added value of the design-guidelines. Results of user tests provide clues for designing stimuli for foot interaction and indicate applications of foot communication devices.

Unique from other senses, the human haptic system supports two-way communications between humans and interactive systems, enabling bidirectional interaction between users and their surroundings. More specifically, haptic interaction offers an independent sensory channel that the brain can process to further enhance a user's experience in a multimodal environment. Such interaction can speed reaction time and reduce hand--eye coordination errors for computer-related tasks. Many interactive systems use visual and, to a lesser extent, audio cues to present users with information about their surroundings and interactions with objects. Haptic feedback can enhance interactive systems' realism through more natural interaction with objects and the environment. Rupert,1for example, used tactile actuators to provide cues for resolving spatial disorientation in aviation environments when visual cues are absent or misleading. Tan, Lim, and Traylor2designed a haptic car navigation guidance system by leveraging sensory saltation,a spatiotemporal illusion of movement across a person's back. Determining how to best design haptic interfaces is essential for further advancement of such novel haptic devices. An examination of haptic sensory systems and associated cognitive and motor processes should help direct the design of haptic interaction devices. This article surveys the haptics literature and identifies conditions under which haptic interaction displays can enhance human perception and performance. Sidebars present the guiding principles and issues associated with haptic interaction devices as well as haptics design guidelines for multimodal interactive systems.

Haptic feedback has been shown to improve user performance in Graphical User Interface (GUI) targeting tasks in a number of studies. These studies have typically focused on interactions with individual targets, and it is unclear whether the performance increases reported will generalise to the more realistic situation where multiple targets are presented simultaneously. This paper addresses this issue in two ways. Firstly two empirical studies dealing with groups of haptically augmented widgets are presented. These reveal that haptic augmentations of complex widgets can reduce performance, although carefully designed feedback can result in performance improvements. The results of these studies are then used in conjunction with the previous literature to generate general design guidelines for the creation of haptic widgets.

A new method for researching haptic interaction styles is presented, based on a layered interaction model and a classification of existing devices. The method is illustrated by designing a new foot interaction device. The aim of which is to enhance non-verbal communication over a computer network. A layered protocols interaction model allows to consider all aspects of the haptic communication process: the intention to perform an action, limitations of the human body, and specifications of the communication device and the network. We demonstrate how this model can be used to derive design-guidelines by analyzing and classifying existing communication devices. By designing and evaluating a foot interaction device, we not only demonstrate that feet are suited for personal, concealed communication over a network, but also show the added value of the design-guidelines. Results of user tests provide clues for designing stimuli for foot interaction and indicate applications of foot communication devices.

Vibro-tactile displays convey messages by presenting vibration to the user's skin. In recent years, the interest in and application of vibro-tactile displays is growing. Vibratory displays are introduced in mobile devices, desktop applications and even in aircraft [1]. Despite the growing interest, guidelines on the design of vibro-tactile displays are still lacking. Existing guidelines are mainly concerned with passive displays, such as Braille labels on controls, nibs on keyboards and notches on smart cards [2, 3, 4]. In this paper we focus on active displays, either consisting of a single vibrating element (used in for example mobile phones and computer mice) or numerous elements (used in for example active Braille displays and body suits [5, 6]). This paper discusses a first set of guidelines, dealing with the basic vibro-tactile parameters. The set is mainly derived from neurophysiological and psychophysical data. The guidelines indicate the relevant parameters as well as possible pitfalls. As such they can serve as a point of departure for interface designers. Important expansions of the set can come from user evaluation studies and examples of best practices.

In this article we examine earcons, which are audio messages used in the user-computer interface to provide information and feedback to the user about computer entities. (Earcons include messages and functions, as well as states and labels.) We identify some design principles that are common to both visual symbols and auditory messages, and discuss the use of representational and abstract icons and earcons. We give some examples of audio patterns that may be used to design modules for earcons, which then may be assembled into larger groupings called families. The modules are single pitches or rhythmicized sequences of pitches called motives. The families are constructed about related motives that serve to identify a family of related messages. Issues concerned with learning and remembering earcons are discussed.

One important issue for proactive computing is how users control and interact with the systems they will carry and have access to when they are out in the field. One solution is to use multimodal interaction (interaction using different combinations of sensory modalities) to allow people to interact in a range of different ways. This paper discusses gestural interaction as an alternative for input. This is advantageous as it does not require users to look at a display. For output non-speech audio and tactile displays are presented as alternatives to visual displays. The advantages with these types of displays are that they can be unobtrusive and do not require a user's visual attention. The combination of these underutilised senses has much potential to create effective interfaces for proactive systems.

There is a lack of guidelines for designers to use when creating sounds for their interfaces. This paper proposes a set of general guidelines for the creation of earcons based upon six experiments we have performed. Using them a designer will be able to create effective earcons.

This paper describes a novel form of display using tactile output. Tactons, or tactile icons, are structured tactile messages that can be used to communicate message to users non-visually. A range of different parameters can be used to construct Tactons, e.g.: frequency, amplitude, waveform and duration of a tactile pulse, plus body location. Tactons have the potential to improve interaction in a range of different areas, particularly where the visual display is overloaded, limited in size or not available, such as interfaces for blind people or on mobile and wearable devices.

Tactile displays are now becoming available in a form that can be easily used in a user interface. This paper describes a new form of tactile output. Tactons, or tactile icons, are structured, abstract messages that can be used to communicate messages non-visually. A range of different parameters can be used for Tacton construction including: frequency, amplitude and duration of a tactile pulse, plus other parameters such as rhythm and location. Tactons have the potential to improve interaction in a range of different areas, particularly where the visual display is overloaded, limited in size or not available, such as interfaces for blind people or in mobile and wearable devices. This paper describes Tactons, the parameters used to construct them and some possible ways to design them. Examples of where Tactons might prove useful in user interfaces are given.

This paper presents an initial investigation into the use of Tactons, or tactile icons, to present progress information in desktop human-computer interfaces. Progress bars are very common in a wide range of interfaces but have problems. For example, they must compete for screen space and visual attention with other visual tasks such as document editing or web browsing. To address these problems we created a tactile progress indicator, encoding progress information into a series of vibrotactile cues. An experiment comparing the tactile progress indicator to a standard visual one showed a significant improvement in performance and an overall preference for the tactile display. These results suggest that a tactile display is a good way to present such information and this has many potential applications from computer desktops to mobile telephones.

This paper reports two experiments relating to the design of Tactons (or tactile icons). The first experiment investigated perception of vibro-tactile ``roughness'' (created using amplitude modulated sinusoids), and the results indicated that roughness could be used as a parameter for constructing Tactons. The second experiment is the first full evaluation of Tactons, and uses three values of roughness identified in the first experiment, along with three rhythms to create a set of Tactons. The results of this experiment showed that Tactons could be a successful means of communicating information in user interfaces, with an overall recognition rate of 71%, and recognition rates of 93% for rhythm and 80% for roughness.

While the sense of touch is capable of processing complex stimuli, the vibration feedback used in mobile phones is generally very simple. Using more complex vibrotactile messages would enable the communication of more information through phone alerts, however it has been suggested that phone vibration motors are not capable of presenting complex messages. This paper reports a study investigating the use of Tactons (tactile icons), presented using a standard mobile phone vibration motor, to represent mobile phone alerts. The recognition rate of 72% achieved for Tactons encoding two pieces of information is comparable to results achieved in a previous experiment with a high specification transducer, indicating that it is possible to communicate multi-dimensional information in mobile phone alerts. These results will help designers to understand the possibilities offered by standard phone vibration motors for communicating complex information.

Tactile icons (Tactons) are structured vibrotactile messages which can be used for non visual information display. Information is encoded in Tactons by manipulating vibrotactile parameters. This research investigates the possibilities of applying musical techniques to tactile icon design in order to define such parameters. Tactile versions of musical dynamics were created by manipulating the amplitude of vibrations to create increasing, decreasing, and level stimuli and an experiment was carried out to test perception of these stimuli. Identification rates of 92%-100% indicate that these tactile dynamics can be identified and distinguished from each other, and that tactile dynamics could be used in Tacton design.

In this paper, we survey guidance on tactile and haptic interactions provided by various researchers who were not in attendance at GOTHI-05. Its main purpose is to identify potential guidelines that might be incorporated into an international standard on tactile and haptic interaction. This survey also identified a number of controversial areas that will need to be dealt with in developing such a standard. Results are presented in a manner consistent with a companion paper "Initiating Guidance on Tactile and Haptic Interactions", by Fourney and Carter [8].

This work addresses the use of vibrotactile haptic feedback to transmit background information with variable intrusiveness, when recipients are engrossed in a primary visual and/or auditory task. We describe two studies designed to (a) perceptually optimize a set of vibrotactile "icons" and (b) evaluate users' ability to identify them in the presence of varying degrees of workload. Seven icons learned in approximately 3 minutes were each typically identified within 2.5 s and at 95% accuracy in the absence of workload. An extended version of this paper can be found as a technical report at http://www.cs.ubc.ca/labs/spin/.

A new breed of mobile phones has been designed to enable concurrent vibration and audio stimulation, or audio-haptics. This paper aims to share techniques for creating and optimizing audio-haptic effects to enhance the user interface. The authors present audio manipulation techniques specific to the multifunction transducer (MFT) technology. In particular two techniques, the Haptic Inheritance and Synthesis and Matching methods are discussed. These two methods of haptic media generation allow simple creation of vibration content, and also allow for compatibility with non-haptic mobile devices. The authors present preliminary results of an evaluation of 42 participants comparing audio-based haptic user interface (UI) feedback with audio-only feedback. The results show that users were receptive to audio-haptic UI feedback. The results also suggest that audio-haptics seems to enhance the perception of audio quality.

The design of interfaces for automotive information systems is a critical task. In fact, such design must take into account that user is busy in the primary driving task, and any visual distraction determined by telematics systems can cause serious safety problems. To limit such distraction and enhance safety, in this paper we propose a novel multimodal user interface. The key element of the proposal is a new interaction device, named Handy, conceived to exploit the drivers tactile channel to minimize the workload of visual channel. Moreover Handy is suitably integrated with the graphical user interface, which is characterized by a reduced number of choices for each state and has been designed in agreement with the self-revealing approach.

The goal for this research is to study a new class of force feedback applications based on abstract messages we call haptic icons. With the introduction of active haptic displays, a single knob or joystick can be used to control several different, sometimes non-related, functions. The functions associated with these multi-function handles can no longer be identified from one another by position, shape or texture differences. Haptic icons are brief programmed forces applied to a user through a haptic interface conveying an object's or event's state, function or content in a manner similar to visual or auditory icons. This thesis begins with a presentation of several tools that were developed to aid this research. It then describes a series of psychophysical tests designed to obtain the basic perceptual limits for our haptic interface. Knowing these perceptual limits is a prerequisite for proper haptic icon design. We analyzed a set of synthetically constructed haptic icons using Multidimensional Scaling, in order to discover the underlying perceptual processes in identifying different haptic stimuli. Results show that a set of icons constructed by varying the frequency, magnitude and shape of 2-sec, time-invariant waveforms map to two perceptual axes, which differ depending on the signals' frequency range, and suggest that expressive capability is maximized in one frequency subspace. I finish by proposing future work to be done on this area.

We define haptic icons, or ``hapticons'', as brief programmed forces applied to a user through a haptic interface, with the role of communicating a simple idea in manner similar to visual or auditory icons. In this paper we present the design and implementation of an innovative software tool and graphical interface for the creation and editing of hapticons. The tool's features include various methods for creating new icons including direct recording of manual trajectories and creation from a choice of basis waveforms; novel direct-manipulation icon editing mechanisms, integrated playback and convenient storage of icons to file. We discuss some ways in which the tool has aided our research in the area of haptic iconography and present an innovative approach for generating and rendering simple textures on a low degree of freedom haptic device using what we call terrain display.

A haptic phoneme represents the smallest unit of a constructed haptic signal to which a meaning can be assigned. These haptic phonemes can be combined serially or in parallel to form haptic words, or haptic icons, which can hold more elaborate meanings for their users. Here, we use phonemes which consist of brief (<2 seconds) haptic stimuli composed of a simple waveform at a constant frequency and amplitude. Building on previous results showing that a set of 12 such haptic stimuli can be perceptually distinguished, here we test learnability and recall of associations for arbitrarily chosen stimulus-meaning pairs. We found that users could consistently recall an arbitrary association between a haptic stimulus and its assigned arbitrary meaning in a 9-phoneme set, during a 45 minute test period following a reinforced learning stage.

The Guidelines On Tactile and Haptic Interactions Conference (GOTHI-05) is the result of the realization of the need for the International Organization for Standardization (ISO) to standardize guidance on tactile/haptic interactions. This paper reviews existing international standards on tactile/haptic interactions and suggests ways to construct a relevant ISO standard. It proposes potential dimensions and boundaries for a future standard and provides a preliminary collection of draft tactile/haptic interactions guidelines based on available guidance.

Haptic interfaces have the potential to enhance communication and interaction via the computer---enabling a fective expressive interpersonal communication and enriching interaction by haptic feedback. Still, what exactly their potential is and how we can design in order to fulfil it remains topic of contemporary debate. My contribution to this debate shall be to place some of the current developments into a philosophical and cultural context and to introduce social science based methodologies, which will help broaden the discussion and scope of input. Through semiotic analysis, we can predict `meaning making' in haptic communication that goes beyond linguistic description. Examples of Haptic interfaces shall be positioned as case studies in this typology. Also, I describe the Haptic Box and PinKom as my way of investigating a semiotic system of touch. In conclusion, this paper hopes to inform and catalyse the development of a haptic design palette.

In this paper, we review research and applications in the area of mediated or remote social touch. Whereas current communication media rely predominately on vision and hearing, mediated social touch allows people to touch each other over a distance by means of haptic feedback technology. Overall, the reviewed applications have interesting potential, such as the communication of simple ideas (e.g., through Hapticons), establishing a feeling of connectedness between distant lovers, or the recovery from stress. However, the beneficial effects of mediated social touch are usually only assumed and have not yet been submitted to empirical scrutiny. Based on social psychological literature on touch, communication, and the effects of media, we assess the current research and design efforts and propose future directions for the field of mediated social touch.

We report two experiments designed to investigate the potential use of vibrotactile warning signals to present spatial information to car drivers. Participants performed an attention-demanding rapid serial visual presentation (RSVP) monitoring task. Meanwhile, whenever they felt a vibrotactile stimulus presented on either their front or back, they had to check the front and the rearview mirror for the rapid approach of a car, and brake or accelerate accordingly. We investigated whether speeded responses to potential emergency driving situations could be facilitated by the presentation of spatially-predictive (80% valid; Experiment 1) or spatially-nonpredictive (50% valid; Experiment 2) vibrotactile cues. Participants responded significantly more rapidly following both spatially-predictive and spatially-nonpredictive vibrotactile cues from the same rather than the opposite direction as the critical driving events. These results highlight the potential utility of vibrotactile warning signals in automobile interface design for directing a drivers visual attention to time-critical events or information.

This paper describes a novel form of display using crossmodal output. A crossmodal icon is an abstract icon that can be instantiated in one of two equivalent forms (auditory or tactile). These can be used in interfaces as a means of non-visual output. This paper discusses how crossmodal icons can be constructed and the potential benefits they bring to mobile human computer interfaces.

This paper presents a set of interaction techniques for hands-free multi-scale navigation through virtual environments. We believe that hands-free navigation, unlike the majority of navigation techniques based on hand motions, has the greatest potential for maximizing the interactivity of virtual environments since navigation modes are of loaded from modal hand gestures to more direct motions of the feet and torso. Not only are the users' hands freed to perform tasks such as modeling, notetaking and object manipulation, but we also believe that foot and torso movements may inherently be more natural for some navigation tasks. The particular interactions that we developed include a leaning technique for moving small and medium distances, a foot-gesture controlled Step WIM that acts as a floor map for moving larger distances, and a viewing technique that enables a user to view a full 360 degrees in only a three-walled semi-immersive environment by subtly amplifying the mapping between their torso rotation and the virtual world. We formatively designed and evaluated our techniques in existing projects related to archaeological reconstructions, free-form modeling, and interior design. In each case, our informal observations have indicated that motions such as walking and leaning are both appropriate for navigation and are effective in cognitively simplifying complex virtual environment interactions since functionality is more evenly distributed across the body.

Handheld devices are enhancing many aspects of our lives. As increasingly complex devices appear with decreasing form factors, haptics may become an essential tool for interacting with them. In this regime of operation, issues of power, weight and volume are of significant importance The haptic -holds of the index finger for active exploration of a two dimensional virtual environment for two icon alignments and two finger motions were measured. Using all possible combinations of two finger motions, Aexiodentensioo and finger abdnctinnladduction, and two icon alignments, vertical and horizontal, were measured separately. Haptic thresholds ranged fmm 15 to 24 millflewtons. Thmbolds were fleeted by finger motion, hut not by icon alignment.

Mobile interaction can potentially be enhanced with well-designed haptic control and display. However, advances have been limited by a vicious cycle whereby inadequate haptic technology obstructs inception of vitalizing applications. We present the first stages of a systematic design effort to break that cycle, beginning with specific usage scenarios and a new handheld display platform based on lateral skin stretch. Results of a perceptual device characterization inform mappings between device capabilities and specific roles in mobile interaction, and the next step of hardware re-engineering.

The most common applications for haptic feedback employ ``direct rendering'' techniques wherein a user touches a virtual model, usually displayed graphically as well. We propose a new class of applications based on abstract messages, ranging from ``haptic icons'' -- brief signals conveying an object's or event's state, function or content -- to an expressive haptic language for interpersonal communication. Building this language requires us to understand how synthetic haptic signals are perceived, and what they can mean to us. Experiments presented here address the perception question by using Multidimensional Scaling (MDS) to extract perceptual axes for complex haptic icons: once this space is mapped, icons can be designed to maximize both differentiability and individual salience. Results show that a set of icons constructed by varying the frequency, magnitude and shape of 2-sec, time-invariant wave shapes map to two perceptual axes, which differ depending on the signals' frequency range; and suggest that expressive capability is maximized in one frequency subspace.

The bulk of applications for haptic feedback employ direct rendering approaches wherein a user touches a virtual model of some ``real'' thing, often displayed graphically as well. We propose a new class of applications based on abstract messages, ranging from ``haptic icons'' -- brief signals conveying an object's or event's state, function or content -- to an expressive haptic language for interpersonal communication. Building this language requires us to understand how synthetic haptic signals are perceived, and what they can mean to us. Experiments presented here address the perception question by using an efficient version of Multidimensional Scaling (MDS) to extract perceptual axes for complex haptic icons: once this space is mapped, icons can be designed to maximize both differentiability and individual salience. Results show that a set of icons constructed by varying the frequency, magnitude and shape of 2-sec, time-invariant wave shapes map to two perceptual axes, which differ depending on the signals' frequency range; and suggest that expressive capability is maximized in one frequency subspace.

Haptic feedback has been shown to improve user performance in Graphical User Interface (GUI) targeting tasks in a number of studies. These studies have typically focused on interactions with individual targets, and it is unclear whether the performance increases reported will generalise to the more realistic situation where multiple targets are presented simultaneously. This paper addresses this issue in two ways. Firstly two empirical studies dealing with groups of haptically augmented widgets are presented. These reveal that haptic augmentations of complex widgets can reduce performance, although carefully designed feedback can result in performance improvements. The results of these studies are then used in conjunction with the previous literature to generate general design guidelines for the creation of haptic widgets.

The design of usable haptic icons (brief informational signals delivered through the sense of touch) requires a tool for measuring perceptual distances between icons that will be used together as a set. Our experiences with one potentially powerful approach, Multidimensional Scaling (MDS) analysis of perceptual data acquired using an efficient cluster sorting technique, raised questions relating to the methodology for data collection. In this paper, we review key issues relating to perceptual data collection method, describe an example data set and present its initial MDS analysis, and then examine the impact of collection method on MDS outcome through a secondary analysis of the data and the inherent structure of the algorithm components. Our analysis suggests that an understanding of these issues is important for the method's effective use, but has not exposed any major flaws with the process.

Haptic devices can be used to visualize information. As well as representing tangible surfaces and forces to enhance virtual training simulators for instance, haptic devices have been used to realize tactile versions of diagrams and visualizations (such as line graphs and bar charts). Such depictions enable blind or partial sighted users to perceive and understand information. However, there are multiple challenges when presenting information tactically: (1) it is difficult to understand a summary of the information, and (2) it is challenging to represent multivariate information through these haptic representations. In this paper we present how hlyphs (haptic versions of the graphical glyph) can be created, describe design guidelines, and detail how they can be used to represent both summaries and multivariate information.

Instant Messaging (IM) is a popular chatting platform on the internet and increasingly permeates teenage life. Even intimate and emotional content is discussed. As touch is a powerful signal for emotional content, haptic signals, and especially hapticons could contribute to overcome the inevitable loss of subtle non-verbal communication cues. This paper introduces hapticons (small force or vibration patterns that can be used to communicate emotions and feelings) to enrich instant messaging. The open source Hapticon Design Tool and the Haptic Instant Messenger that are introduced, provide a framework to combine communication of textual messages with haptic effects. Also the design process and evaluation methods for meaningful hapticon design are discussed.

Instant Messaging (IM) is a popular chatting platform on the internet and increasingly permeates teenage life. Even intimate and emotional content is discussed. As touch is a powerful signal for emotional content, haptic signals, and especially hapticons can contribute to overcome the inevitable loss of subtle non-verbal communication cues. Audio-visual extensions of IM to share emotions, in particular emoticons, have been received enthusiastically by IM users. This indicates a realistic user-need for hapticons in IM. The Haptic Instant Messaging (HIM) framework introduced in this paper combines communication of textual messages with haptic effects and hapticons. The application is build as an open framework and supports small chatting communities to explore the design and use of hapticons and haptic IO devices. Researchers can use the HIM framework to monitor the use of haptics in communication and how haptics contribute to the fun and meaning of instant messaging.

The development of a device that enables haptic foot interaction for communication over a network is presented. Considering the physical properties of feet we demonstrate that feet are suited for personal, concealed communication over a computer network. First experiments to investigate both the usability and fun of using foot interaction indicate promising results and concrete opportunities for further development.

A new method for researching haptic interaction styles is presented, based on a layered interaction model and a classification of existing devices. The method is illustrated by designing a new foot interaction device. The aim of which is to enhance non-verbal communication over a computer network. A layered protocols interaction model allows to consider all aspects of the haptic communication process: the intention to perform an action, limitations of the human body, and specifications of the communication device and the network. We demonstrate how this model can be used to derive design-guidelines by analyzing and classifying existing communication devices. By designing and evaluating a foot interaction device, we not only demonstrate that feet are suited for personal, concealed communication over a network, but also show the added value of the design-guidelines. Results of user tests provide clues for designing stimuli for foot interaction and indicate applications of foot communication devices.

Haptic representations need to support human goals, capabilities, and desires. Thus a challenge is to create usable and intuitive haptic representations of intangible real-world ideas. To support this challenge of designing better haptic interfaces, we need better abstract representations of haptic signals, and of the space the signals represent. These representations will (a) help designers explore the design space in a structured way; (b) facilitate communication among haptic practitioners -- improving design practices more broadly; and, (c) potentially enlarge the design space, because exploration of alternatives can reveal gaps and areas for improvement. In this paper, we introduce a possible design space with the goal of raising questions about how to represent relationships between social & technical information and physical haptic signals.

Visual information overload is a threat to the interpretation of displays presenting large data sets or complex application environments. To combat this problem, researchers have begun to explore how haptic feedback can be used as another means for information transmission. In this paper, we show that people can perceive and accurately process haptically rendered ordinal data while under cognitive workload. We evaluate three haptic models for rendering ordinal data with participants who were performing a taxing visual tracking task. The evaluation demonstrates that information rendered by these models is perceptually available even when users are visually busy. This preliminary research has promising implications for haptic augmentation of visual displays for information visualization.

This paper describes the potential of using vibro-tactile displays for automobile drivers. Technological developments in the field of driver support systems and tactile displays, combined with the ever increasing need to enlarge the capacity of the driver's information channel, form the reason to review the possibilities of in-car tactile displays and to identify some promising applications. In the second part of the paper, we describe a feasibility study in which we tested an in-car tactile display in a driving simulator. The results show that the tactile navigation display resulted in better performance compared to a visual display, and that it reduces the driver's workload. This study gives a first indication that employing the tactile modality may be a major step to accomplish safety improvements.

Vibro-tactile displays convey messages by presenting vibration to the user's skin. In recent years, the interest in and application of vibro-tactile displays is growing. Vibratory displays are introduced in mobile devices, desktop applications and even in aircraft [1]. Despite the growing interest, guidelines on the design of vibro-tactile displays are still lacking. Existing guidelines are mainly concerned with passive displays, such as Braille labels on controls, nibs on keyboards and notches on smart cards [2, 3, 4]. In this paper we focus on active displays, either consisting of a single vibrating element (used in for example mobile phones and computer mice) or numerous elements (used in for example active Braille displays and body suits [5, 6]). This paper discusses a first set of guidelines, dealing with the basic vibro-tactile parameters. The set is mainly derived from neurophysiological and psychophysical data. The guidelines indicate the relevant parameters as well as possible pitfalls. As such they can serve as a point of departure for interface designers. Important expansions of the set can come from user evaluation studies and examples of best practices.

Multimodal interfaces offer great potential to humanize interactions with computers by employing a multitude of perceptual channels. This paper reports on a novel multimodal interface using auditory, haptic, and visual feedback in a direct manipulation task to establish new recommendations for multimodal feedback, in particular uni-, bi-, and trimodal feedback. A close examination of combinations of uni-, bi-, and trimodal feedback is necessary to determine which enhances performance without increasing workload. Thirty-two participants were asked to complete a task consisting of a series of 'drag-and-drops' while the type of feedback was manipulated. Each participant was exposed to three unimodal feedback conditions, three bimodal feedback conditions and one trimodal feedback condition that used auditory, visual, and haptic feedback alone, and in combination. Performance under the different conditions was assessed with measures of trial completion time, target highlight time, and a self-reported workload assessment captured by the NASA Task Load Index (NASA-TLX). The findings suggest that certain types of bimodal feedback can enhance performance while lowering self-perceived mental demand.

The goal of this study is to determine the effect of `maintain position' and `relax' tasks on the dynamic of the foot while manipulating a gas pedal. The foot is viewed as a mass-spring-damper system, of which the visco-elasticity can be altered by reflexive feedback and muscle (co-)contraction. The dynamic properties of the foot are described by the mechanical admittance, which determines the foot position as a dynamic function of an external force perturbation. It is hypothesized that humans will change their admittance based on task perception. Experiments were done to estimate the endpoint foot admittance with Frequency Response Functions (FWs). An experimental setup that can simulate a gas pedal with different static and dynamic properties was used to apply continuous force perturbations to the foot. Eight subjects were instructed to either minimize the resulting pedal deviations (`maintain position 3 or do nothing and just rest their foot on the pedal (`relax'). The force perturbation, pedal position, and reaction force were measured, and transformed tot the frequency domain to estimate the closed-loop admittance. All subjects showed a considerable difference between the two tasks, confirming the hypothesis that drivers change the dynamics of their foot to best accomplish a perceived task.

The goal of this study is to determine the effect of amplitude and frequency of force sinusoids on force perception of the foot, in order to design an effective haptic feedback system for gas pedals. Eight subjects were asked to push a gas pedal to a constant workpoint position against a background force of 25 N Force perception was determined for three frequencies and three types of footwear by requiring subjects to respond with `yes' or `no' after each force sinusoid. Psychometric functions were calculated from the data, relating the ratio of yes answers (averaged over all subjects) to the amplitude of the force sinusoid. Although large standard deviations were found for low ratios, a statistically signifcant Just Noticeable Difference (JND) could be determined for the upper boundary of perception. Increasing the frequency of the stimulus decreased the JND. Footwear was shown to have a substantial impact on the JND at all frequencies, the largest effect occurring at the lowest frequency.

This paper describes preliminary work in the use of a virtual environment to derive just noticeable differences (JNDs) for force. Specifically, we look for thresholds of force sensitivity so that we may ultimately construct therapeutic force feedback distortions that stay below these thresholds. Initially, we have concentrated on JNDs as they are applied to the index finger; preliminary data in healthy individuals shows an average JND of approximately 10%. More significantly, the data indicate that visual feedback distortions in a virtual environment can be created to encourage increased force productions by up to 10%, and that this can be done without a patient's awareness.

Instrument controls in motor vehicles have haptic properties (force and compliance) designed to enhance the ease of use. In conventional control knobs, these properties are obtained mechanically via springs, detents, hard stops and the like. The work reported here concerns the use of a single force-feedback knob to emulate the feel of various conventional control knobs, thus retaining their desirable haptic properties while allowing multiple functions to be controlled through one device. Compared to existing haptic knobs, this one is novel in its use of a brake to provide high torque capability in a small volume.

Investigating virtual environments has become an increasingly interesting research topic for engineers, computer and cognitive scientists, and psychologists. Although there have been several recent studies focused on the development of multimodal virtual environments (VEs) to study human-machine interactions, less attention has been paid to human-human and human-machine interactions in shared virtual environments (SVEs), and to our knowledge, no attention paid at all to what extent the addition of haptic communication between people would contribute to the shared experience. We have developed a multimodal shared virtual environment and performed a set of experiments with human subjects to study the role of haptic feedback in collaborative tasks and whether haptic communication through force feedback can facilitate a sense of being and collaborating with a remote partner. The study concerns a scenario where two participants at remote sites must co-operate to perform a joint task in a SVE. The goals of the study are (1) to assess the impact of force feedback on task performance, (2) to better understand the role of haptic communication in human-human interactions, (3) to study the impact of touch on the subjective sense of collaborating with a human as reported by the participants based on what they could see and feel, and (4) to investigate if gender, personality, or emotional experiences of users can affect haptic communication in SVEs. The outcomes of this research can have a powerful impact on the development of next generation human-computer interfaces and network protocols that integrate touch and force feedback technology into the Internet, development of protocols and techniques for collaborative teleoperation such as hazardous material removal, space station repair, and remote surgery, and enhancement of virtual environments for performing collaborative tasks in shared virtual worlds on a daily basis such as co-operative teaching, training, planning and design, cybergames, and social gatherings. Our results suggest that haptic feedback significantly improves the task performance and contributes to the feeling of `sense of togetherness' in SVEs. In addition, the results show that the experience of visual feedback only at first, and then subsequently visual plus haptic feedback elicits a better performance than presentation of visual plus haptic feedback first followed by visual feedback only.

In this paper, we introduce a new approach for applying haptic feedback technology to interpersonal communication. We present the design of our prototype inTouch system which provides a physical link between users separated by distance.

Current Systems for real-time distributed CSCW are largely rooted in traditional GUI-based groupware and voice/video coferencing methodologies. In these approaches, interactions are limited to visual and auditory media and shared environments are confined to the digital world. This paper presents a new approach to enhance remote collaboration and communication, based on the idea of Tangible interaces, which places a greater emphasis on touch and physicality. The approach is grounded in a concept called Synchronized Distributed Physical Objects, which employs telemanipulation technology to create the illusion that distant users are interacting with shared physical objects. We describe two applications of this approach: PSyBench, a physical shared workspace, and inTouch, a device for haptic interpersonal communication.

This study investigates the effect of force-feedback in computer-mediated communication. Participants completed a screen-based maze task with an alleged remote participant in a 2 (task characteristics: cooperative vs. competitive) by 2 (modality: haptic/force-feedback vs. visual) balanced, between-participants experiment. There were a number of cross-over interactions. In the competitive task, participants felt more powerful and more positively overall when interacting through force-feedback than when interacting visually. They also liked the other participant more and trusted them more. The opposite results were obtained for the cooperating participants. Implications for including force-feedback in computer-mediated communication are outlined.

One important issue for proactive computing is how users control and interact with the systems they will carry and have access to when they are out in the field. One solution is to use multimodal interaction (interaction using different combinations of sensory modalities) to allow people to interact in a range of different ways. This paper discusses gestural interaction as an alternative for input. This is advantageous as it does not require users to look at a display. For output non-speech audio and tactile displays are presented as alternatives to visual displays. The advantages with these types of displays are that they can be unobtrusive and do not require a user's visual attention. The combination of these underutilised senses has much potential to create effective interfaces for proactive systems.

This paper reports two experiments relating to the design of Tactons (or tactile icons). The first experiment investigated perception of vibro-tactile ``roughness'' (created using amplitude modulated sinusoids), and the results indicated that roughness could be used as a parameter for constructing Tactons. The second experiment is the first full evaluation of Tactons, and uses three values of roughness identified in the first experiment, along with three rhythms to create a set of Tactons. The results of this experiment showed that Tactons could be a successful means of communicating information in user interfaces, with an overall recognition rate of 71%, and recognition rates of 93% for rhythm and 80% for roughness.

In this paper, we survey guidance on tactile and haptic interactions provided by various researchers who were not in attendance at GOTHI-05. Its main purpose is to identify potential guidelines that might be incorporated into an international standard on tactile and haptic interaction. This survey also identified a number of controversial areas that will need to be dealt with in developing such a standard. Results are presented in a manner consistent with a companion paper "Initiating Guidance on Tactile and Haptic Interactions", by Fourney and Carter [8].

We describe the design of ComTouch, a device that augments remote voice communication with touch, by converting hand pressure into vibrational intensity between users in real-time. The goal of this work is to enrich interpersonal communication by complementing voice with a tactile channel. We present preliminary user studies performed on 24 people to observe possible uses of the tactile channel when used in conjunction with audio. By recording and examining both audio and tactile data, we found strong relationships between the two communication channels. Our studies show that users developed an encoding system similar to that of Morse code, as well as three original uses: emphasis, mimicry, and turn-taking. We demonstrate the potential of the tactile channel to enhance the existing voice communication channel.

This paper presents an overview of the tactile properties based on tactile languages, such as Braille and fingerspelling. The unique spatial and temporal properties of touch through use of exploratory procedures highlight the amount information available through touch. The authors make recommendations for haptic visualization to actively engage exploratory procedures and exploit context from other modalities. The authors present a mobile device augmented with tactile UI feedback. General observations based on public awareness to the haptic phenomenon are described.

The design of interfaces for automotive information systems is a critical task. In fact, such design must take into account that user is busy in the primary driving task, and any visual distraction determined by telematics systems can cause serious safety problems. To limit such distraction and enhance safety, in this paper we propose a novel multimodal user interface. The key element of the proposal is a new interaction device, named Handy, conceived to exploit the drivers tactile channel to minimize the workload of visual channel. Moreover Handy is suitably integrated with the graphical user interface, which is characterized by a reduced number of choices for each state and has been designed in agreement with the self-revealing approach.

This research investigates the ability of subjects to detect small haptic effects and the associated gains in task performance with various configurations of haptic stimuli. Variations in force amplitude, shape, and pulse duration used to create the effects are studied. An adaptive-threshold method is used to obtain subjects' detection thresholds for actively explored haptic icons ranging in size from 3 to 5 mm. Detection thresholds are compared for smooth versus rough actively-explored icons. Subjects' detection thresholds for ``staticicons'' (force pulses of 50 to 150 ms duration) are also measured. Results indicate that rough (sawtooth) hapticicons are more easily detected by a human subject than smooth (sinusoidal) icons of the same size. Transient vibrotactile cues may contribute to these observations. Mean subject performance, as measured by Fitts' information-processing rate and by clicks per minute, is shown to improve with the amplitude of haptic stimulus.

The bulk of current haptics human-factors research focuses on mapping basic human perceptual limits. However, many realistic applications demand a better understanding of how to construct more life-like but often less controllable experiment scenarios. In this paper, we study this problem in the context of advanced automobile interfaces. We employ a throttle pedal with programmable force feedback to indicate potentially undesirable situations in the external environment and to gently but steadily guide the driver away from them. We have found evidence that within this scenario, errors in such a warning signal can have a negative effect on the behavior of the driver within the conditions studied. These experiments required a complex protocol and necessarily permitted a variety of participant tactics. Post-experiment analysis revealed that very subtle variations in participant instruction produced large differences in tactics and consequent experiment outcome.

A haptic phoneme represents the smallest unit of a constructed haptic signal to which a meaning can be assigned. These haptic phonemes can be combined serially or in parallel to form haptic words, or haptic icons, which can hold more elaborate meanings for their users. Here, we use phonemes which consist of brief (<2 seconds) haptic stimuli composed of a simple waveform at a constant frequency and amplitude. Building on previous results showing that a set of 12 such haptic stimuli can be perceptually distinguished, here we test learnability and recall of associations for arbitrarily chosen stimulus-meaning pairs. We found that users could consistently recall an arbitrary association between a haptic stimulus and its assigned arbitrary meaning in a 9-phoneme set, during a 45 minute test period following a reinforced learning stage.

Intelligent systems are increasingly able to offer real-time information relevant to a user's manual control of an interactive system; however, effective presentation of this information creates many challenges. We consider how to use force feedback to convey information to a user about dynamic system control space constraints that have been computed by an intelligent system. Effective display of control constraints will require careful consideration of the usability of the forces, in addition to good technical design, to assure user acceptance of the feedback. Possible dynamic systems that can benefit from this kind of interaction feedback are tasks such as driving and the control of physically-based animations. In this thesis, we studied the haptic display of control constraints in a simple driving simulation. We developed a `look-ahead' guidance method to display usable haptic guidance suggestions to a driver based upon the predicted location of the vehicle relative to the road, and implemented this using a custom vehicle simulator based on Reynolds's Open-Steer framework. The performance and usability of our Look-Ahead Guidance method are compared to a baseline of No-Guidance, and to Potential Field Guidance, the current state-of-the-art haptic path guidance method. Our experimental results show that Look-Ahead Guidance was more usable and showed performance benefits in our task compared to both No-Guidance and to Potential Field Guidance. We identified several factors that we suspect affect the usability of haptic path guidance and suggest future work based on these observations.

Intelligent systems are increasingly able to offer real-time information relevant to a user's manual control of an interactive system, such as dynamic system control space constraints for animation control and driving. However, it is difficult to present this information in a usable manner and other approaches which have employed haptic cues for manual control in ``slow'' systems often lead to instabilities in highly dynamic tasks. We present a predictive haptic guidance method based on a look-ahead algorithm, along with a user evaluation which compares it with other approaches (no guidance and a standard potential-field method) in a 1-DoF steered path-following scenario. Look-ahead guidance outperformed the other methods in both quantitative performance and subjective preference across a range of path complexity and visibility and a force analysis demonstrated that it applied smaller and fewer forces to users. These results (which appear to derive from the predictive guidance's supporting users in taking earlier and more subtle corrective action) suggest the potential of predictive methods in aiding manual control of dynamic interactive tasks where intelligent support is available.

The Guidelines On Tactile and Haptic Interactions Conference (GOTHI-05) is the result of the realization of the need for the International Organization for Standardization (ISO) to standardize guidance on tactile/haptic interactions. This paper reviews existing international standards on tactile/haptic interactions and suggests ways to construct a relevant ISO standard. It proposes potential dimensions and boundaries for a future standard and provides a preliminary collection of draft tactile/haptic interactions guidelines based on available guidance.

In this paper I will describe the term Tactile Semiotics -- the meanings we ascribe to different types of haptic sensations. When designing devices for haptic communication or haptic input devices for HCI, it makes sense firstly to look to psychophysics as an informant. However, to gain insight into tactility as both experience and meaning, it may also be beneficial to look at the cultural and social context of touch. Also, I will explain how it is possible to explore this field with the help of low-tech prototypes and methodologies more common in art and design.

Haptic interfaces have the potential to enhance communication and interaction via the computer---enabling a fective expressive interpersonal communication and enriching interaction by haptic feedback. Still, what exactly their potential is and how we can design in order to fulfil it remains topic of contemporary debate. My contribution to this debate shall be to place some of the current developments into a philosophical and cultural context and to introduce social science based methodologies, which will help broaden the discussion and scope of input. Through semiotic analysis, we can predict `meaning making' in haptic communication that goes beyond linguistic description. Examples of Haptic interfaces shall be positioned as case studies in this typology. Also, I describe the Haptic Box and PinKom as my way of investigating a semiotic system of touch. In conclusion, this paper hopes to inform and catalyse the development of a haptic design palette.

In this paper, we review research and applications in the area of mediated or remote social touch. Whereas current communication media rely predominately on vision and hearing, mediated social touch allows people to touch each other over a distance by means of haptic feedback technology. Overall, the reviewed applications have interesting potential, such as the communication of simple ideas (e.g., through Hapticons), establishing a feeling of connectedness between distant lovers, or the recovery from stress. However, the beneficial effects of mediated social touch are usually only assumed and have not yet been submitted to empirical scrutiny. Based on social psychological literature on touch, communication, and the effects of media, we assess the current research and design efforts and propose future directions for the field of mediated social touch.

The word ``haptics'' is now well accepted. Hundreds of papers are published each year on the topic of haptic devices and interfaces. Haptics, as a technological niche, has become rich with opportunities and challenges. The field borrows from, and lends to, many sub jects in science and technology. Among these, two are particularly relevant: ``mechatronics'' on one hand, and ``robot-human interaction'' on the other. Haptic devices belong to the family of mechatronic devices because their fundamental function is to take advantage of mechanical signals to provide for communication between people and machines. It follows that haptic devices must include include transducers to convert mechanical signals to electrical signals and vice-versa used in conjunction with one or several computational or data processing systems. These transducers appeal to a variety of technologies: electromechanical devices, optoelectronics, fluids, smart materials, exploiting the possibilities that exist to build highly integrated and cost effective devices. The popularization of haptics as an area of investigation is due to the work of such pioneers as Brooks and Iwata [1,6].

We report two experiments designed to investigate the potential use of vibrotactile warning signals to present spatial information to car drivers. Participants performed an attention-demanding rapid serial visual presentation (RSVP) monitoring task. Meanwhile, whenever they felt a vibrotactile stimulus presented on either their front or back, they had to check the front and the rearview mirror for the rapid approach of a car, and brake or accelerate accordingly. We investigated whether speeded responses to potential emergency driving situations could be facilitated by the presentation of spatially-predictive (80% valid; Experiment 1) or spatially-nonpredictive (50% valid; Experiment 2) vibrotactile cues. Participants responded significantly more rapidly following both spatially-predictive and spatially-nonpredictive vibrotactile cues from the same rather than the opposite direction as the critical driving events. These results highlight the potential utility of vibrotactile warning signals in automobile interface design for directing a drivers visual attention to time-critical events or information.

The potential use of non-visual warning signals to present spatial information to car drivers has been successfully demonstrated in several recent studies (Ho & Spence, submitted, in preparation; Ho, Tan, & Spence, submitted). Among the three types of spatial warning signals investigated (namely auditory, visual, and vibrotactile), spatial vibrotactile cues were found to be particularly effective in directing a driver's visual spatial attention to potentially dangerous events on the road. We conducted the present study in order to examine the factors governing the relative effectiveness of auditory, visual, and vibrotactile warning signals. The speeded discrimination of warning signals presented in the various different modalities was investigated in order to explore whether the differences found in our previous research were a result of the relative speed with which people can detect warning signals presented in a given modality, or whether they were attributable to differences in the efficacy with which people can relate the warning signal to the subsequent visually-specified target driving events.

This paper describes a novel form of display using crossmodal output. A crossmodal icon is an abstract icon that can be instantiated in one of two equivalent forms (auditory or tactile). These can be used in interfaces as a means of non-visual output. This paper discusses how crossmodal icons can be constructed and the potential benefits they bring to mobile human computer interfaces.

The term kinesthesia refers to the perception of limb movement and position, and is often broadly defined to include the perception of force as well. These sensory perceptions originate primarily from the activity of mechanoreceptors in muscles, which provides the central nervous system with information about the static length of muscles, the rate at which muscle length changes, and the forces muscles generate. From these signals comes our awareness of where our limbs are in space, when our limbs have moved, and the mechanical properties of objects (e.g. weight, compliance) with which they interact. Sensory information about changes in limb position and movement also arises from other sources, namely receptors in the skin and joints. These inputs appear to be particularly important for kinesthesia in the hand, as both joint (Clark et al., 1989; Ferrell et al., 1987) and (or) cutaneous anesthesia (Clark et al., 1986) impairs the ability to detect finger movements and perceive finger positions. For more proximal joints, such as the knee, joint and (or) skin anesthesia does not have a significant influence on the perception of limb position (Clark et al., 1979). It appears that for the hand, cutaneous receptors provide an important facilitatory input to the central nervous system that is used to interpret position and movement signals arising from other sources. Cutaneous receptors in the hairy skin on the dorsum of the hand are capable, however, of encoding joint movement very precisely via their responses to stretch of the skin overlying the active joint (Collins & Prochazka, 1996; Edin, 1992). The importance of cutaneous sensory feedback to the perception of finger movements and positions is not surprising in view of the high innervation density of cutaneous mechanoreceptors in the hand, and its specialization for tactile exploration and manipulation. This feedback may also be more important for kinesthesia in the hand than for other parts of the body because of the complex anatomical arrangement of muscles, with most muscles acting over several finger joints, which would result in a considerable ambiguity of muscle spindle receptor discharges. In addition to these peripherally originating signals, there is evidence that central (cortical) feedback pathways provide information that is used to decode muscle afferent signals and in the perception of force.

This paper presents a set of interaction techniques for hands-free multi-scale navigation through virtual environments. We believe that hands-free navigation, unlike the majority of navigation techniques based on hand motions, has the greatest potential for maximizing the interactivity of virtual environments since navigation modes are of loaded from modal hand gestures to more direct motions of the feet and torso. Not only are the users' hands freed to perform tasks such as modeling, notetaking and object manipulation, but we also believe that foot and torso movements may inherently be more natural for some navigation tasks. The particular interactions that we developed include a leaning technique for moving small and medium distances, a foot-gesture controlled Step WIM that acts as a floor map for moving larger distances, and a viewing technique that enables a user to view a full 360 degrees in only a three-walled semi-immersive environment by subtly amplifying the mapping between their torso rotation and the virtual world. We formatively designed and evaluated our techniques in existing projects related to archaeological reconstructions, free-form modeling, and interior design. In each case, our informal observations have indicated that motions such as walking and leaning are both appropriate for navigation and are effective in cognitively simplifying complex virtual environment interactions since functionality is more evenly distributed across the body.

How the relative order in which 4 property classes of haptically perceived surfaces becomes available for processing after initial contact was studied. The classes included material, abrupt-surface discontinuity, relative orientation, and continuous 3-D surface contour properties. Relative accessibility was evaluated by using the slopes of haptic search functions obtained with a modified version ofA. Treisman's (A. Treisman & S. Gormican, 1988) visual pop-out paradigm; the Y0 intercepts were used to confirm and fine-tune order of accessibility. Target and distractors differed markedly in terms of their value on a single dimension. The results of 15 experiments show that coarse intensive discriminations are haptically processed early on. In marked contrast, most spatially encoded dimensions become accessible relatively later, sometimes considerably so.

Attention and motivation are two factors, which are important for motor relearning following stroke. In this paper we introduce and present the novel concepts for the use of haptics technology to deliver therapy to patients with arm impairments following stroke. The results of the ongoing initial pilot studies have shown the feasibility of the system presented here. The new approach will potentially improve the patient's attention and motivation and hence enhance therapy effectiveness.

Mobile interaction can potentially be enhanced with well-designed haptic control and display. However, advances have been limited by a vicious cycle whereby inadequate haptic technology obstructs inception of vitalizing applications. We present the first stages of a systematic design effort to break that cycle, beginning with specific usage scenarios and a new handheld display platform based on lateral skin stretch. Results of a perceptual device characterization inform mappings between device capabilities and specific roles in mobile interaction, and the next step of hardware re-engineering.

The bulk of applications for haptic feedback employ direct rendering approaches wherein a user touches a virtual model of some ``real'' thing, often displayed graphically as well. We propose a new class of applications based on abstract messages, ranging from ``haptic icons'' -- brief signals conveying an object's or event's state, function or content -- to an expressive haptic language for interpersonal communication. Building this language requires us to understand how synthetic haptic signals are perceived, and what they can mean to us. Experiments presented here address the perception question by using an efficient version of Multidimensional Scaling (MDS) to extract perceptual axes for complex haptic icons: once this space is mapped, icons can be designed to maximize both differentiability and individual salience. Results show that a set of icons constructed by varying the frequency, magnitude and shape of 2-sec, time-invariant wave shapes map to two perceptual axes, which differ depending on the signals' frequency range; and suggest that expressive capability is maximized in one frequency subspace.

This paper describes preliminary work in a project to add force feedback to user interface elements of the X Window System in an attempt to add true ``feel'' to the window system's ``look and feel''. Additions include adding ridges around icons and menu items to aid interaction, alignment guides for moving windows, and other enhancements to window manipulation. The motivation for this system is the observation that people naturally have many skills for and intuitions about a very rich environment of interaction forces in the non-computer world; however, these skills are largely unused in computer applications. We expect that haptic modifications to conventional graphical user interfaces, such as those we present, can lead to gains in performance, intuition, learnability, and enjoyment of the interface. This paper describes details of the implementation of the haptic window system elements, in addition to higher-level haptic design principles and informal observations of users of the system.

A set of experiments was conducted to investigate the relationship of environment damping to performance of a Fitts one-shot tapping task, and especially the relationship of environment damping to the haptic perception of target position. In the experiments, subjects were asked to locate a narrow target region that had a different level of viscous damping than the background regions. The task was performed using a one degree-of-freedom manipulandum. Movement time to target was measured as a function of the damping in the target and background regions. Different visual feedback conditions were also tested. The most striking result of the experiments was that when the targets were not visible to subjects, performance was very closely correlated with the absolute magnitude of the difference in target and environment damping (i.e. target damping minus the background damping). Performance did not vary with the percentage difference between target damping and background damping, nor with the sign of the difference between target and background damping, nor with the level of background damping. When target positions were visible to subjects, performance depended very weakly on the environment damping.

This paper is about using the sense of touch, the haptic system, as part of our everyday interface with computationally created worlds. In Part I, we discuss a particular system, called Sandpaper, designed for experimenting with feeling texture. In part II, we discuss how control analysis helps us understand the behavior of the types of hardware and software we use to implement force display.

Users of collaborative systems are typically restricted to communication through voice and video links. Users find this difficult -- it does not encompass the richness of communication they are accustomed to in the real world. Attempting to address this problem we describe the implementation of a novel mechanism for haptic communication based around interactions between users' cursors. An initial, and mainly observational, evaluation is described, along with some promising results. We show improvements in subjective experience and suggest several, more formal, avenues for future research.

Communication between users in shared editors takes place in a deprived environment -- distributed users find it difficult to communicate. While many solutions to the problems this causes have been suggested this paper presents a novel one. It describes one possible use of haptics as a channel for communication between users. User's telepointers are considered as haptic avatars and interactions such as haptically pushing and pulling each other are afforded. The use of homing forces to locate other users is also discussed, as is a proximity sensation based on viscosity. Evaluation of this system is currently underway.

Machine-mediated training of dynamic task completion is typically implemented with passive intervention via virtual fixtures or active assist by means of record and replay strategies. During interaction with a real dynamic system however, the user relies on both visual and haptic feedback real-time in order to elicit desired motions. This work investigates human performance in a Fitts' type targeting task with an underactuated dynamic system. Performance, in terms of number of hits and between-target tap times, is measured while various passive and active control modes are displayed concurrently with the haptic feedback from the simulated system's own dynamic behavior. It is hypothesized that passive and active assist modes that are implemented during manipulation of simulated underactuated systems could be beneficial in rehabilitation applications. Results indicate that human performance can be improved significantly with the passive and active assist modes.

This paper presents a shared-control interaction paradigm for haptic interface systems, with experimental data from two user studies. Shared control, evolved from its initial telerobotics applications, is adapted as a form of haptic assistance in that the haptic device contributes to execution of a dynamic manual target-hitting task via force commands from an automatic controller. Compared to haptic virtual environments, which merely display the physics of the virtual system, or to passive methods of haptic assistance for performance enhancement based on virtual fixtures, the shared-control approach offers a method for actively demonstrating desired motions during virtual environment interactions. The paper presents a thorough review of the literature related to haptic assistance. In addition, two experiments were conducted to independently verify the efficacy of the shared-control approach for performance enhancement and improved training effectiveness of the task. In the first experiment, shared control is found to be as effective as virtual fixtures for performance enhancement, with both methods resulting in significantly better performance in terms of time between target hits for the manual target-hitting task than sessions where subjects feel only the forces arising from the mass-spring-damper system dynamics. Since shared control is more general than virtual fixtures, this approach may be extremely beneficial for performance enhancement in virtual environments. In terms of training enhancement, shared control and virtual fixtures were no better than practice in an unassisted mode. For manual control tasks, such as the one described in this paper, shared control is beneficial for performance enhancement, but may not be viable for enhancing training effectiveness.

In these experiments, two plates were grasped between the thumb and forefinger and squeezed together along a linear track. An electromechanical system presented a constant resistance force during the squeeze up to a predetermined location on the track, whereupon the force effectively went to infinity (simulating a wall) or to zero (simulating a cliff). The task of the subject was to discriminate between two alternative levels of the constant resistance force (a reference level and a reference-plus-increment level). Results of these experiments indicate a just noticeable difference of roughly 7% of the reference force using a one-interval paradigm with trial-by-trial feedback over the ranges 2.5 less than or equal to F0 less than or equal to 10.0 newtons, 5 less than or equal to D less than or equal to 30 mm, 45 less than or equal to S less than or equal to 125 mm, and 25 less than or equal to V less than or equal to 160 mm/sec, where F0 is the reference force, D is the distance squeezed, S is the initial fingerspan, and V is the mean velocity of the squeeze. These results, based on tests with 5 subjects, are consistent with a wide range of previous results, some of which are associated with other body surfaces and muscle systems and many of which were obtained with different psychophysical methods.

Human communication and interaction comprise a wide range of verbal and nonverbal cues. Further adoption of novel telecommunication methods such as e-mail, chat, instant messaging (IM), mobile phone SMS text messaging, and videoconferencing; have augmented our mediated interaction abilities. However, a significant (and important) amount of human expression and interaction information is never captured, transmitted, or expressed with current computer mediated communication (CMC) tools. We also lack ambient methods of maintaining contact when not co-located with family and friends. Communal Interfaces is a new research effort aimed at the study of nonverbal human cues: their intent, motion, meaning, subtleties, and importance in communication. In this paper we address issues involved in the design, construction, and evaluation of Connexus, one such communal interface.

In physical and occupational therapy two people interact through force and motion. Other common examples of this interaction include lifting and moving a bulky object, teaching manual skills, dancing, and handing off a baton or a drinking glass. These tasks involve kinesthetic interaction, a communication channel distinct from spoken language and gestures. Understanding kinesthetic interaction should be important in designing robots to assist with physical and occupational therapy. In this paper we describe our experiments on kinesthetic interaction between two people cooperating on a 1 degree of freedom task. We characterize the interaction forces between the two people, dividing them into a productive ``net force'' and an orthogonal ``difference force.'' Our results suggest three effects (1) an emergent specialization of the two participants into different roles, (2) an oscillation of forces at about 8 Hz, and (3) a steady force in opposition to one another that could be analogous to co-contraction in an individual.

Haptic interaction between people and machines might benefit from an understanding of haptic communication between one person and another. We recently reported results showing that two people performing a physically shared dyadic task can outperform either person alone, even when the perception of each participant is that the other is a hindrance. Evidently a dyad quickly negotiates a more efficient motion strategy than is available to individuals. This negotiation must take place through a haptic channel of communication, and it is apparently at a level below the awareness of the participants. Here we report results on the motion strategy that emerged. By recording forces and motions we show that the dyads ``specialized'' temporally such that one member took on early parts of the motion and the other late parts. Tests in which one participant's contribution was surreptitiously replaced by a motor did not elicit a similar cooperative response from the remaining human participant, showing that the language of haptic communication between people must be rather subtle.

Machines that interact with humans might benefit from an understanding of the subtlety of human-human interaction. We recently reported results that two people working cooperatively will temporally specialize such that one member will command the early parts of motion and the other the late parts. In our current study, we replaced one of the humans with a robot designed to simulate this specialization. We expected the remaining human to respond to the robot the same way as another human, but they did not. Even though the subjects believed that they were interacting with a person, the subjects did not work with the robot in a similar fashion; their force profile was remarkably similar to how they worked alone. The subjects consciously believed they were working with a person, but their force profile showed that they were not. Understanding how two people physically work together is the beginning of understanding how a robot can work fluidly with a robot in a human like way.

Haptic devices can be used to visualize information. As well as representing tangible surfaces and forces to enhance virtual training simulators for instance, haptic devices have been used to realize tactile versions of diagrams and visualizations (such as line graphs and bar charts). Such depictions enable blind or partial sighted users to perceive and understand information. However, there are multiple challenges when presenting information tactically: (1) it is difficult to understand a summary of the information, and (2) it is challenging to represent multivariate information through these haptic representations. In this paper we present how hlyphs (haptic versions of the graphical glyph) can be created, describe design guidelines, and detail how they can be used to represent both summaries and multivariate information.

The fundamental purpose of a telepresence system is to extend an operator`s sensory-motor facilities and problem solving abilities to a remote environment. Telepresence is achieved by projecting the operator's manipulatory dexterity to a remote site while reflecting sensory feedback from that site so realistically that the operator actually feels present in the remote location. In order to enhance operator performance and understanding within remote environments, most research and development of telepresence systems has been directed towards improving the fidelity of the link between operator and environment. Although higher fidelity interfaces are important to the advancement of telepresence system, the work described in this paper actually looks at the benificial effects of corrupting the link between operator and remote environment by introducing abstract perceptual information into the interface called virtual fixtures.

Instant Messaging (IM) is a popular chatting platform on the internet and increasingly permeates teenage life. Even intimate and emotional content is discussed. As touch is a powerful signal for emotional content, haptic signals, and especially hapticons can contribute to overcome the inevitable loss of subtle non-verbal communication cues. Audio-visual extensions of IM to share emotions, in particular emoticons, have been received enthusiastically by IM users. This indicates a realistic user-need for hapticons in IM. The Haptic Instant Messaging (HIM) framework introduced in this paper combines communication of textual messages with haptic effects and hapticons. The application is build as an open framework and supports small chatting communities to explore the design and use of hapticons and haptic IO devices. Researchers can use the HIM framework to monitor the use of haptics in communication and how haptics contribute to the fun and meaning of instant messaging.

The development of a device that enables haptic foot interaction for communication over a network is presented. Considering the physical properties of feet we demonstrate that feet are suited for personal, concealed communication over a computer network. First experiments to investigate both the usability and fun of using foot interaction indicate promising results and concrete opportunities for further development.

A new method for researching haptic interaction styles is presented, based on a layered interaction model and a classification of existing devices. The method is illustrated by designing a new foot interaction device. The aim of which is to enhance non-verbal communication over a computer network. A layered protocols interaction model allows to consider all aspects of the haptic communication process: the intention to perform an action, limitations of the human body, and specifications of the communication device and the network. We demonstrate how this model can be used to derive design-guidelines by analyzing and classifying existing communication devices. By designing and evaluating a foot interaction device, we not only demonstrate that feet are suited for personal, concealed communication over a network, but also show the added value of the design-guidelines. Results of user tests provide clues for designing stimuli for foot interaction and indicate applications of foot communication devices.

When humans interface with machines, the control interface is usually passive and its response contains little information pertinent to the state of the environment. Usually, information flows through the interface from human to machine but not so often in the reverse direction. This work proposes a control architecture in which bi-directional information transfer occurs across the control interface, allowing the human to use the interface to simultaneously exert control and extract information. In this alternative control architecture, which we call shared control, the human utilizes the haptic sensory modality to share control of the machine interface with an automatic controller. We present a fixed-base driving simulator experiment in which subjects take advantage of a haptic steering wheel, which aids them in a path following task. Results indicate that the haptic steering wheel allows a significant reduction in visual demand while improving path following performance.

Haptic representations need to support human goals, capabilities, and desires. Thus a challenge is to create usable and intuitive haptic representations of intangible real-world ideas. To support this challenge of designing better haptic interfaces, we need better abstract representations of haptic signals, and of the space the signals represent. These representations will (a) help designers explore the design space in a structured way; (b) facilitate communication among haptic practitioners -- improving design practices more broadly; and, (c) potentially enlarge the design space, because exploration of alternatives can reveal gaps and areas for improvement. In this paper, we introduce a possible design space with the goal of raising questions about how to represent relationships between social & technical information and physical haptic signals.

This paper discusses the design criteria imposed by the capabilities of the human user on the design of force reflecting controllers for hands and arms. A framework of issues regarding human capabilities is presented that maps directly to mechanical design requirements. The state of knowledge for each capability is briefly summarized along with presentation of new experimental measurements. Finally, the implications of the human factors data to haptic interface design are discussed.

Visual information overload is a threat to the interpretation of displays presenting large data sets or complex application environments. To combat this problem, researchers have begun to explore how haptic feedback can be used as another means for information transmission. In this paper, we show that people can perceive and accurately process haptically rendered ordinal data while under cognitive workload. We evaluate three haptic models for rendering ordinal data with participants who were performing a taxing visual tracking task. The evaluation demonstrates that information rendered by these models is perceptually available even when users are visually busy. This preliminary research has promising implications for haptic augmentation of visual displays for information visualization.

This paper describes the potential of using vibro-tactile displays for automobile drivers. Technological developments in the field of driver support systems and tactile displays, combined with the ever increasing need to enlarge the capacity of the driver's information channel, form the reason to review the possibilities of in-car tactile displays and to identify some promising applications. In the second part of the paper, we describe a feasibility study in which we tested an in-car tactile display in a driving simulator. The results show that the tactile navigation display resulted in better performance compared to a visual display, and that it reduces the driver's workload. This study gives a first indication that employing the tactile modality may be a major step to accomplish safety improvements.

Vibro-tactile displays convey messages by presenting vibration to the user's skin. In recent years, the interest in and application of vibro-tactile displays is growing. Vibratory displays are introduced in mobile devices, desktop applications and even in aircraft [1]. Despite the growing interest, guidelines on the design of vibro-tactile displays are still lacking. Existing guidelines are mainly concerned with passive displays, such as Braille labels on controls, nibs on keyboards and notches on smart cards [2, 3, 4]. In this paper we focus on active displays, either consisting of a single vibrating element (used in for example mobile phones and computer mice) or numerous elements (used in for example active Braille displays and body suits [5, 6]). This paper discusses a first set of guidelines, dealing with the basic vibro-tactile parameters. The set is mainly derived from neurophysiological and psychophysical data. The guidelines indicate the relevant parameters as well as possible pitfalls. As such they can serve as a point of departure for interface designers. Important expansions of the set can come from user evaluation studies and examples of best practices.

Tactile torso displays convey information by presenting localised vibrations to the torso. Since these stimuli are directly mapped to body coordinates, tactile displays are able to present 3D spatial information in an intuitive way. This paper (1) provides a short description of the tactile torso display that we developed, and (2) gives a brief overview of a simulator study on the usefulness of tactile torso displays in maintaining a stable hover with a helicopter. The results prove the potential of intuitive tactile torso displays in reducing drift during hover, and -- more generally -- prove that tactile displays can be applied in fast man-in-the-loop tasks. The demonstration during Eurohaptics 2003 will consist of our tactile torso display connected to a helicopter flight simulator.

Multimodal interfaces offer great potential to humanize interactions with computers by employing a multitude of perceptual channels. This paper reports on a novel multimodal interface using auditory, haptic, and visual feedback in a direct manipulation task to establish new recommendations for multimodal feedback, in particular uni-, bi-, and trimodal feedback. A close examination of combinations of uni-, bi-, and trimodal feedback is necessary to determine which enhances performance without increasing workload. Thirty-two participants were asked to complete a task consisting of a series of 'drag-and-drops' while the type of feedback was manipulated. Each participant was exposed to three unimodal feedback conditions, three bimodal feedback conditions and one trimodal feedback condition that used auditory, visual, and haptic feedback alone, and in combination. Performance under the different conditions was assessed with measures of trial completion time, target highlight time, and a self-reported workload assessment captured by the NASA Task Load Index (NASA-TLX). The findings suggest that certain types of bimodal feedback can enhance performance while lowering self-perceived mental demand.

This paper seeks to establish the quantitative effects of providing force feedback on user performance in human computer interaction. A reciprocal tapping test is employed in conjunction with Fitts' law in order to establish a measure of human performance in a simple target selection task. The test was performed using a PHANToM haptic interface, under conditions with and without the provision of force feedback. It was found that providing force feedback significantly improved subjects' movement times, but had no effect on the rate of information processing (IP) as defined by Fitts' law. However, it was shown that for conditions of ballistic movement (corresponding to a low task difficulty (ID)), there was a highly significant improvement in IP for the condition with force feedback, but no improvement when force feedback was not employed. This was deemed due to the fact that for the non-haptic condition no force cues were available, therefore the user had to rely on visual cues, hence, ballistic movement was not possible.

Investigating virtual environments has become an increasingly interesting research topic for engineers, computer and cognitive scientists, and psychologists. Although there have been several recent studies focused on the development of multimodal virtual environments (VEs) to study human-machine interactions, less attention has been paid to human-human and human-machine interactions in shared virtual environments (SVEs), and to our knowledge, no attention paid at all to what extent the addition of haptic communication between people would contribute to the shared experience. We have developed a multimodal shared virtual environment and performed a set of experiments with human subjects to study the role of haptic feedback in collaborative tasks and whether haptic communication through force feedback can facilitate a sense of being and collaborating with a remote partner. The study concerns a scenario where two participants at remote sites must co-operate to perform a joint task in a SVE. The goals of the study are (1) to assess the impact of force feedback on task performance, (2) to better understand the role of haptic communication in human-human interactions, (3) to study the impact of touch on the subjective sense of collaborating with a human as reported by the participants based on what they could see and feel, and (4) to investigate if gender, personality, or emotional experiences of users can affect haptic communication in SVEs. The outcomes of this research can have a powerful impact on the development of next generation human-computer interfaces and network protocols that integrate touch and force feedback technology into the Internet, development of protocols and techniques for collaborative teleoperation such as hazardous material removal, space station repair, and remote surgery, and enhancement of virtual environments for performing collaborative tasks in shared virtual worlds on a daily basis such as co-operative teaching, training, planning and design, cybergames, and social gatherings. Our results suggest that haptic feedback significantly improves the task performance and contributes to the feeling of `sense of togetherness' in SVEs. In addition, the results show that the experience of visual feedback only at first, and then subsequently visual plus haptic feedback elicits a better performance than presentation of visual plus haptic feedback first followed by visual feedback only.

In this paper, we introduce a new approach for applying haptic feedback technology to interpersonal communication. We present the design of our prototype inTouch system which provides a physical link between users separated by distance.

Current Systems for real-time distributed CSCW are largely rooted in traditional GUI-based groupware and voice/video coferencing methodologies. In these approaches, interactions are limited to visual and auditory media and shared environments are confined to the digital world. This paper presents a new approach to enhance remote collaboration and communication, based on the idea of Tangible interaces, which places a greater emphasis on touch and physicality. The approach is grounded in a concept called Synchronized Distributed Physical Objects, which employs telemanipulation technology to create the illusion that distant users are interacting with shared physical objects. We describe two applications of this approach: PSyBench, a physical shared workspace, and inTouch, a device for haptic interpersonal communication.

This study investigates the effect of force-feedback in computer-mediated communication. Participants completed a screen-based maze task with an alleged remote participant in a 2 (task characteristics: cooperative vs. competitive) by 2 (modality: haptic/force-feedback vs. visual) balanced, between-participants experiment. There were a number of cross-over interactions. In the competitive task, participants felt more powerful and more positively overall when interacting through force-feedback than when interacting visually. They also liked the other participant more and trusted them more. The opposite results were obtained for the cooperating participants. Implications for including force-feedback in computer-mediated communication are outlined.

One important issue for proactive computing is how users control and interact with the systems they will carry and have access to when they are out in the field. One solution is to use multimodal interaction (interaction using different combinations of sensory modalities) to allow people to interact in a range of different ways. This paper discusses gestural interaction as an alternative for input. This is advantageous as it does not require users to look at a display. For output non-speech audio and tactile displays are presented as alternatives to visual displays. The advantages with these types of displays are that they can be unobtrusive and do not require a user's visual attention. The combination of these underutilised senses has much potential to create effective interfaces for proactive systems.

This paper describes a novel form of display using tactile output. Tactons, or tactile icons, are structured tactile messages that can be used to communicate message to users non-visually. A range of different parameters can be used to construct Tactons, e.g.: frequency, amplitude, waveform and duration of a tactile pulse, plus body location. Tactons have the potential to improve interaction in a range of different areas, particularly where the visual display is overloaded, limited in size or not available, such as interfaces for blind people or on mobile and wearable devices.

Tactile displays are now becoming available in a form that can be easily used in a user interface. This paper describes a new form of tactile output. Tactons, or tactile icons, are structured, abstract messages that can be used to communicate messages non-visually. A range of different parameters can be used for Tacton construction including: frequency, amplitude and duration of a tactile pulse, plus other parameters such as rhythm and location. Tactons have the potential to improve interaction in a range of different areas, particularly where the visual display is overloaded, limited in size or not available, such as interfaces for blind people or in mobile and wearable devices. This paper describes Tactons, the parameters used to construct them and some possible ways to design them. Examples of where Tactons might prove useful in user interfaces are given.

This paper presents an initial investigation into the use of Tactons, or tactile icons, to present progress information in desktop human-computer interfaces. Progress bars are very common in a wide range of interfaces but have problems. For example, they must compete for screen space and visual attention with other visual tasks such as document editing or web browsing. To address these problems we created a tactile progress indicator, encoding progress information into a series of vibrotactile cues. An experiment comparing the tactile progress indicator to a standard visual one showed a significant improvement in performance and an overall preference for the tactile display. These results suggest that a tactile display is a good way to present such information and this has many potential applications from computer desktops to mobile telephones.

This paper describes preliminary work in a project to add force feedback to user interface elements of the X Window System in an attempt to add true ``feel'' to the window system's ``look and feel''. Additions include adding ridges around icons and menu items to aid interaction, alignment guides for moving windows, and other enhancements to window manipulation. The motivation for this system is the observation that people naturally have many skills for and intuitions about a very rich environment of interaction forces in the non-computer world; however, these skills are largely unused in computer applications. We expect that haptic modifications to conventional graphical user interfaces, such as those we present, can lead to gains in performance, intuition, learnability, and enjoyment of the interface. This paper describes details of the implementation of the haptic window system elements, in addition to higher-level haptic design principles and informal observations of users of the system.

Users of collaborative systems are typically restricted to communication through voice and video links. Users find this difficult -- it does not encompass the richness of communication they are accustomed to in the real world. Attempting to address this problem we describe the implementation of a novel mechanism for haptic communication based around interactions between users' cursors. An initial, and mainly observational, evaluation is described, along with some promising results. We show improvements in subjective experience and suggest several, more formal, avenues for future research.

Direct manipulation has been lauded as a good form of interface design, and some interfaces that have this property have been well received by users. In this article we seek a cognitive account of both the advantages and disadvantages of direct manipulation interfaces. We identify two underlying phenomena that give rise to the feeling of directness. One deals with the information processing distance between the user's intentions and the facilities provided by the machine. Reduction of this distance makes the interface feel direct by reducing the effort required of the user to accomplish goals. The second phenomenon concerns the relation between the input and output vocabularies of the interface language. In particular, direct manipulation requires that the system provide representations of objects that behave as if they are the objects themselves. This provides the feeling of directness of manipulation.

Modes of human-computer interaction in the control of dynamic systems are discussed, the problem of allocating tasks between human and computer considered. Models of human performance in a variety of tasks associated with the control of dynamic systems are reviewed. These models are evaluated in the context of a design example involving human-computer interaction in aircraft operations. Other examples include power plants, chemical plants, and ships.

Instant Messaging (IM) is a popular chatting platform on the internet and increasingly permeates teenage life. Even intimate and emotional content is discussed. As touch is a powerful signal for emotional content, haptic signals, and especially hapticons can contribute to overcome the inevitable loss of subtle non-verbal communication cues. Audio-visual extensions of IM to share emotions, in particular emoticons, have been received enthusiastically by IM users. This indicates a realistic user-need for hapticons in IM. The Haptic Instant Messaging (HIM) framework introduced in this paper combines communication of textual messages with haptic effects and hapticons. The application is build as an open framework and supports small chatting communities to explore the design and use of hapticons and haptic IO devices. Researchers can use the HIM framework to monitor the use of haptics in communication and how haptics contribute to the fun and meaning of instant messaging.

Visual information overload is a threat to the interpretation of displays presenting large data sets or complex application environments. To combat this problem, researchers have begun to explore how haptic feedback can be used as another means for information transmission. In this paper, we show that people can perceive and accurately process haptically rendered ordinal data while under cognitive workload. We evaluate three haptic models for rendering ordinal data with participants who were performing a taxing visual tracking task. The evaluation demonstrates that information rendered by these models is perceptually available even when users are visually busy. This preliminary research has promising implications for haptic augmentation of visual displays for information visualization.

Vibro-tactile displays convey messages by presenting vibration to the user's skin. In recent years, the interest in and application of vibro-tactile displays is growing. Vibratory displays are introduced in mobile devices, desktop applications and even in aircraft [1]. Despite the growing interest, guidelines on the design of vibro-tactile displays are still lacking. Existing guidelines are mainly concerned with passive displays, such as Braille labels on controls, nibs on keyboards and notches on smart cards [2, 3, 4]. In this paper we focus on active displays, either consisting of a single vibrating element (used in for example mobile phones and computer mice) or numerous elements (used in for example active Braille displays and body suits [5, 6]). This paper discusses a first set of guidelines, dealing with the basic vibro-tactile parameters. The set is mainly derived from neurophysiological and psychophysical data. The guidelines indicate the relevant parameters as well as possible pitfalls. As such they can serve as a point of departure for interface designers. Important expansions of the set can come from user evaluation studies and examples of best practices.

Much previous research has established that perceived ease of use is an important factor influencing user acceptance and usage behavior of information technologies. However, very little research has been conducted to understand how that perception forms and changes over time. The current work presents and tests an anchoring and adjustment-based theoretical model of the determinants of system-specific perceived ease of use. The model proposes control (internal and external---conceptualized as computer self-efficacy and facilitating conditions, respectively), intrinsic motivation (conceptualized as computer playfulness), and emotion (conceptualized as computer anxiety) as anchors that determine early perceptions about the ease of use of a new system. With increasing experience, it is expected that system-specific perceived ease of use, while still anchored to the general beliefs regarding computers and computer use, will adjust to reflect objective usability, perceptions of external control specific to the new system environment, and system-specific perceived enjoyment. The proposed model was tested in three different organizations among 246 employees using three measurements taken over a three-month period. The proposed model was strongly supported at all points of measurement, and explained up to 60% of the variance in system-specific perceived ease of use, which is twice as much as our current understanding. Important theoretical and practical implications of these findings are discussed.

This paper seeks to establish the quantitative effects of providing force feedback on user performance in human computer interaction. A reciprocal tapping test is employed in conjunction with Fitts' law in order to establish a measure of human performance in a simple target selection task. The test was performed using a PHANToM haptic interface, under conditions with and without the provision of force feedback. It was found that providing force feedback significantly improved subjects' movement times, but had no effect on the rate of information processing (IP) as defined by Fitts' law. However, it was shown that for conditions of ballistic movement (corresponding to a low task difficulty (ID)), there was a highly significant improvement in IP for the condition with force feedback, but no improvement when force feedback was not employed. This was deemed due to the fact that for the non-haptic condition no force cues were available, therefore the user had to rely on visual cues, hence, ballistic movement was not possible.

Three on-road studies were conducted to determine how headway maintenance and collision warning displays influence driver behavior. Visual perspective, visual perspective with a pointer, visual perspective combined with an auditory warning, discrete visual warning, and discrete auditory warning were assessed during both coupled headway and deceleration events. Results indicate that when drivers are provided with salient visual information regarding safe headways, they utilize the information and increase their headway when appropriate. Auditory warnings were less effective than visual warnings for increasing headways but may be helpful for improving reaction time during events that require deceleration. Drivers were somewhat insensitive to false alarm rates, at least during short-term use. Finally, and most important, driver headway maintenace increased by as much as 0.5 s when the appropriate visual display was used. however, a study to investigate the long-term effects of such displays on behavior is strongly recommended prior to mass marketing of headway maintenance/collision warning devices.

This paper discusses the design criteria imposed by the capabilities of the human user on the design of force reflecting controllers for hands and arms. A framework of issues regarding human capabilities is presented that maps directly to mechanical design requirements. The state of knowledge for each capability is briefly summarized along with presentation of new experimental measurements. Finally, the implications of the human factors data to haptic interface design are discussed.

The mobile service robots will share their workspaces e.g. offices, hospitals, or households with humans. Thus a direct contact between man and machine is inevitably. Robots equipped with appropriate sensors can sense the touch. In this paper we present how an untrained user can intuitively interact with the new DLR light-weight robot just by touching the arm. The robot with 7 dof's will react by an evasiveness motion of the touched links while remaining the orientation of the TCP. This feature can also be used for programming the robot. Programming by ``touch'' is very intuitive as you take the robot at hand and demonstrat the movements.

In this paper an new intelligent robot control scheme is presented which enables a cooperative work of humans and robots through direct contact interaction in a partially known environment. The control of the degrees of freedom of the manipulator is shared through a hybrid controller between the operator and the intelligent control system. The allocation of the control among human and robot occurs dynamically according to the state of the task execution and the capabilities of the human and the robot. Experiments were conducted to prove the effectiveness of our concept.

A human's ability to perform physical tasks is limited not by intelligence, but by physical strength. If. in an appropriate environment, a machine's mechanical power is closely integrated with a human arm's mechanical power under the control of the human intellect, the resulting system will be superior to a loosely integrated combination of a human and a fully automated robot. Therefore, a fundamental solution to the problem of ``extending'' human mechanical power must be developed. The work presented defines ``extenders'' as a class of robot manipulators worn by humans to increase human mechanical strength while the wearer's intellect remains the central control system for manipulating the extender. The human, in physical contact with the extender, exchanges power and information signals with the extender. This analysis focuses on the dynamics and control of human-robot interaction in the sense of the transfer of power and information signals. General models for the human, the extender, and the interaction between the human and the extender are developed. The stability of the system of human, extender, and the object being manipulated is analyzed and the conditions for stable maneuvers are derived. An expression for the extender performance is defined to quantify the force augmentation. The trade-off between stability and performance is described. The theoretical predictions are verified experimentally.

The economics of robot manufacturing is driving us toward situations in which a single human operator will be expected to split attention across multiple semi-autonomous vehicles, and remotely intercede if necessary. We present an analysis of such situations, with the goal of creating decision aids. Toward this end, the concept of special regions is introduced. In one set of situations special regions designate areas that are dangerous, and require teleoperation. We show how to move through single route and multi-route situations, and prove the later problem NP-Complete. In another set of situations, special regions can be used to represent areas outside direct radio contact. We present a way to minimize communication distance and plan for interventions. We relate our findings to concepts of neglect time, interaction time, and fan-out. We discuss a measure of effective fan-out for transportation tasks, and present simulation results. The work has potential impact to those engaged in emergency response and search and rescue.

The fundamental purpose of a telepresence system is to extend an operator`s sensory-motor facilities and problem solving abilities to a remote environment. Telepresence is achieved by projecting the operator's manipulatory dexterity to a remote site while reflecting sensory feedback from that site so realistically that the operator actually feels present in the remote location. In order to enhance operator performance and understanding within remote environments, most research and development of telepresence systems has been directed towards improving the fidelity of the link between operator and environment. Although higher fidelity interfaces are important to the advancement of telepresence system, the work described in this paper actually looks at the benificial effects of corrupting the link between operator and remote environment by introducing abstract perceptual information into the interface called virtual fixtures.

Kinematic and dynamic performance of robotic manipulators is poor when they are near singularities and workspace edges. This problem becomes more complex in teleoperated systems where there are two robots whose kinematics can be different. In these cases, neither should approach singularities or workspace boundaries. This work proposes a means of preventing the robots approaching places of risk. The algorithm computes a force proportional to the distance to singularity and/or workspace boundary. This force stops the robot moving towards that direction. It should be stressed that this strategy works in a very intuitive way in bilateral controlled systems, since the operator feels as if there are virtual springs which serve to prevent him entering forbidden regions. Finally, the algorithm has been tested successfully in a telerobotic system that consists of a Stewart platform and a 6DOF open-chain manipulator.

Strong requirements of flexibility and reactivity in human-robot-co-operation lead to disappearing distinctions between planning, reactive planning and reactive control. This mainly happens because intuitive control of a robot is not possible without considering the actual task, which already touches an aspect of task planning. To solve this we decided to create a whole set of control mechanisms instead of only three or four basic control modes as usual. By doing this we try to regain a strict distinction between reactive planning which selects control modes and delivers corresponding parameters and actual control. In this paper we present the desgined control modes necessary for human-robot co-operation.

Human robot co-operation strongly requires flexibility and reactivity of a robotic system. This leads to disappearing distinctions between planning, reactive planning and reactive control, as intuitive control of a robot is not possible without considering the actual task, which already touches an aspect of task planning. We have created a set of different control mechanisms instead of three to four standard control modes in order to regain a strict distinction between reactive planning, which selects control modes and delivers corresponding parameters, and actual control. In this paper we present the designed control modes necessary for human robot co-operation, their usage by distributed planning modules and the switching between different control modes.

A human's ability to perform physical tasks is limited not by intelligence, but by physical strength. If. in an appropriate environment, a machine's mechanical power is closely integrated with a human arm's mechanical power under the control of the human intellect, the resulting system will be superior to a loosely integrated combination of a human and a fully automated robot. Therefore, a fundamental solution to the problem of ``extending'' human mechanical power must be developed. The work presented defines ``extenders'' as a class of robot manipulators worn by humans to increase human mechanical strength while the wearer's intellect remains the central control system for manipulating the extender. The human, in physical contact with the extender, exchanges power and information signals with the extender. This analysis focuses on the dynamics and control of human-robot interaction in the sense of the transfer of power and information signals. General models for the human, the extender, and the interaction between the human and the extender are developed. The stability of the system of human, extender, and the object being manipulated is analyzed and the conditions for stable maneuvers are derived. An expression for the extender performance is defined to quantify the force augmentation. The trade-off between stability and performance is described. The theoretical predictions are verified experimentally.

Modes of human-computer interaction in the control of dynamic systems are discussed, the problem of allocating tasks between human and computer considered. Models of human performance in a variety of tasks associated with the control of dynamic systems are reviewed. These models are evaluated in the context of a design example involving human-computer interaction in aircraft operations. Other examples include power plants, chemical plants, and ships.

This paper describes preliminary work in the use of a virtual environment to derive just noticeable differences (JNDs) for force. Specifically, we look for thresholds of force sensitivity so that we may ultimately construct therapeutic force feedback distortions that stay below these thresholds. Initially, we have concentrated on JNDs as they are applied to the index finger; preliminary data in healthy individuals shows an average JND of approximately 10%. More significantly, the data indicate that visual feedback distortions in a virtual environment can be created to encourage increased force productions by up to 10%, and that this can be done without a patient's awareness.

A set of experiments was conducted to investigate the relationship of environment damping to performance of a Fitts one-shot tapping task, and especially the relationship of environment damping to the haptic perception of target position. In the experiments, subjects were asked to locate a narrow target region that had a different level of viscous damping than the background regions. The task was performed using a one degree-of-freedom manipulandum. Movement time to target was measured as a function of the damping in the target and background regions. Different visual feedback conditions were also tested. The most striking result of the experiments was that when the targets were not visible to subjects, performance was very closely correlated with the absolute magnitude of the difference in target and environment damping (i.e. target damping minus the background damping). Performance did not vary with the percentage difference between target damping and background damping, nor with the sign of the difference between target and background damping, nor with the level of background damping. When target positions were visible to subjects, performance depended very weakly on the environment damping.

Intelligent systems are increasingly able to offer real-time information relevant to a user's manual control of an interactive system; however, effective presentation of this information creates many challenges. We consider how to use force feedback to convey information to a user about dynamic system control space constraints that have been computed by an intelligent system. Effective display of control constraints will require careful consideration of the usability of the forces, in addition to good technical design, to assure user acceptance of the feedback. Possible dynamic systems that can benefit from this kind of interaction feedback are tasks such as driving and the control of physically-based animations. In this thesis, we studied the haptic display of control constraints in a simple driving simulation. We developed a `look-ahead' guidance method to display usable haptic guidance suggestions to a driver based upon the predicted location of the vehicle relative to the road, and implemented this using a custom vehicle simulator based on Reynolds's Open-Steer framework. The performance and usability of our Look-Ahead Guidance method are compared to a baseline of No-Guidance, and to Potential Field Guidance, the current state-of-the-art haptic path guidance method. Our experimental results show that Look-Ahead Guidance was more usable and showed performance benefits in our task compared to both No-Guidance and to Potential Field Guidance. We identified several factors that we suspect affect the usability of haptic path guidance and suggest future work based on these observations.

Intelligent systems are increasingly able to offer real-time information relevant to a user's manual control of an interactive system, such as dynamic system control space constraints for animation control and driving. However, it is difficult to present this information in a usable manner and other approaches which have employed haptic cues for manual control in ``slow'' systems often lead to instabilities in highly dynamic tasks. We present a predictive haptic guidance method based on a look-ahead algorithm, along with a user evaluation which compares it with other approaches (no guidance and a standard potential-field method) in a 1-DoF steered path-following scenario. Look-ahead guidance outperformed the other methods in both quantitative performance and subjective preference across a range of path complexity and visibility and a force analysis demonstrated that it applied smaller and fewer forces to users. These results (which appear to derive from the predictive guidance's supporting users in taking earlier and more subtle corrective action) suggest the potential of predictive methods in aiding manual control of dynamic interactive tasks where intelligent support is available.

From the Preface: "Technological and biological control processes both are expressions of the life process. Man's effort to understand control processes is, however, a later, higher developement, in which the life process turns inward and examines an aspect of itself. More accurately, living individuals attempt to examine how they and other men are able to exert control over their environments. This boo, then, is the result of one such examination by an individual working in the technology of control. "I have endeavored to develop and to apply a theory of the human control process in both manual and automatic control systems. THis theory holds that control activities involving mechanisms always originate in the concious processes of men. To understand these control activities, therefore, one must begin with man and consider how man exercises control when there is no mechanism to aid him... "The roots of this theory of control lie not in the behavioristic psychology of today but in the 'hormic" psychology of William McDougall... McDougall proposed a theory of human action based on the concept that living things organize their behavior around goals. The movements and actions of livign organisms cannot be understood, he felt, except in terms of purpose, of goals, and of the striving toward them."

Machine-mediated training of dynamic task completion is typically implemented with passive intervention via virtual fixtures or active assist by means of record and replay strategies. During interaction with a real dynamic system however, the user relies on both visual and haptic feedback real-time in order to elicit desired motions. This work investigates human performance in a Fitts' type targeting task with an underactuated dynamic system. Performance, in terms of number of hits and between-target tap times, is measured while various passive and active control modes are displayed concurrently with the haptic feedback from the simulated system's own dynamic behavior. It is hypothesized that passive and active assist modes that are implemented during manipulation of simulated underactuated systems could be beneficial in rehabilitation applications. Results indicate that human performance can be improved significantly with the passive and active assist modes.

This paper presents a shared-control interaction paradigm for haptic interface systems, with experimental data from two user studies. Shared control, evolved from its initial telerobotics applications, is adapted as a form of haptic assistance in that the haptic device contributes to execution of a dynamic manual target-hitting task via force commands from an automatic controller. Compared to haptic virtual environments, which merely display the physics of the virtual system, or to passive methods of haptic assistance for performance enhancement based on virtual fixtures, the shared-control approach offers a method for actively demonstrating desired motions during virtual environment interactions. The paper presents a thorough review of the literature related to haptic assistance. In addition, two experiments were conducted to independently verify the efficacy of the shared-control approach for performance enhancement and improved training effectiveness of the task. In the first experiment, shared control is found to be as effective as virtual fixtures for performance enhancement, with both methods resulting in significantly better performance in terms of time between target hits for the manual target-hitting task than sessions where subjects feel only the forces arising from the mass-spring-damper system dynamics. Since shared control is more general than virtual fixtures, this approach may be extremely beneficial for performance enhancement in virtual environments. In terms of training enhancement, shared control and virtual fixtures were no better than practice in an unassisted mode. For manual control tasks, such as the one described in this paper, shared control is beneficial for performance enhancement, but may not be viable for enhancing training effectiveness.

Haptic interaction between people and machines might benefit from an understanding of haptic communication between one person and another. We recently reported results showing that two people performing a physically shared dyadic task can outperform either person alone, even when the perception of each participant is that the other is a hindrance. Evidently a dyad quickly negotiates a more efficient motion strategy than is available to individuals. This negotiation must take place through a haptic channel of communication, and it is apparently at a level below the awareness of the participants. Here we report results on the motion strategy that emerged. By recording forces and motions we show that the dyads ``specialized'' temporally such that one member took on early parts of the motion and the other late parts. Tests in which one participant's contribution was surreptitiously replaced by a motor did not elicit a similar cooperative response from the remaining human participant, showing that the language of haptic communication between people must be rather subtle.

Investigating virtual environments has become an increasingly interesting research topic for engineers, computer and cognitive scientists, and psychologists. Although there have been several recent studies focused on the development of multimodal virtual environments (VEs) to study human-machine interactions, less attention has been paid to human-human and human-machine interactions in shared virtual environments (SVEs), and to our knowledge, no attention paid at all to what extent the addition of haptic communication between people would contribute to the shared experience. We have developed a multimodal shared virtual environment and performed a set of experiments with human subjects to study the role of haptic feedback in collaborative tasks and whether haptic communication through force feedback can facilitate a sense of being and collaborating with a remote partner. The study concerns a scenario where two participants at remote sites must co-operate to perform a joint task in a SVE. The goals of the study are (1) to assess the impact of force feedback on task performance, (2) to better understand the role of haptic communication in human-human interactions, (3) to study the impact of touch on the subjective sense of collaborating with a human as reported by the participants based on what they could see and feel, and (4) to investigate if gender, personality, or emotional experiences of users can affect haptic communication in SVEs. The outcomes of this research can have a powerful impact on the development of next generation human-computer interfaces and network protocols that integrate touch and force feedback technology into the Internet, development of protocols and techniques for collaborative teleoperation such as hazardous material removal, space station repair, and remote surgery, and enhancement of virtual environments for performing collaborative tasks in shared virtual worlds on a daily basis such as co-operative teaching, training, planning and design, cybergames, and social gatherings. Our results suggest that haptic feedback significantly improves the task performance and contributes to the feeling of `sense of togetherness' in SVEs. In addition, the results show that the experience of visual feedback only at first, and then subsequently visual plus haptic feedback elicits a better performance than presentation of visual plus haptic feedback first followed by visual feedback only.

In this article we examine earcons, which are audio messages used in the user-computer interface to provide information and feedback to the user about computer entities. (Earcons include messages and functions, as well as states and labels.) We identify some design principles that are common to both visual symbols and auditory messages, and discuss the use of representational and abstract icons and earcons. We give some examples of audio patterns that may be used to design modules for earcons, which then may be assembled into larger groupings called families. The modules are single pitches or rhythmicized sequences of pitches called motives. The families are constructed about related motives that serve to identify a family of related messages. Issues concerned with learning and remembering earcons are discussed.

This study investigates the effect of force-feedback in computer-mediated communication. Participants completed a screen-based maze task with an alleged remote participant in a 2 (task characteristics: cooperative vs. competitive) by 2 (modality: haptic/force-feedback vs. visual) balanced, between-participants experiment. There were a number of cross-over interactions. In the competitive task, participants felt more powerful and more positively overall when interacting through force-feedback than when interacting visually. They also liked the other participant more and trusted them more. The opposite results were obtained for the cooperating participants. Implications for including force-feedback in computer-mediated communication are outlined.

One important issue for proactive computing is how users control and interact with the systems they will carry and have access to when they are out in the field. One solution is to use multimodal interaction (interaction using different combinations of sensory modalities) to allow people to interact in a range of different ways. This paper discusses gestural interaction as an alternative for input. This is advantageous as it does not require users to look at a display. For output non-speech audio and tactile displays are presented as alternatives to visual displays. The advantages with these types of displays are that they can be unobtrusive and do not require a user's visual attention. The combination of these underutilised senses has much potential to create effective interfaces for proactive systems.

Tactile icons (Tactons) are structured vibrotactile messages which can be used for non visual information display. Information is encoded in Tactons by manipulating vibrotactile parameters. This research investigates the possibilities of applying musical techniques to tactile icon design in order to define such parameters. Tactile versions of musical dynamics were created by manipulating the amplitude of vibrations to create increasing, decreasing, and level stimuli and an experiment was carried out to test perception of these stimuli. Identification rates of 92%-100% indicate that these tactile dynamics can be identified and distinguished from each other, and that tactile dynamics could be used in Tacton design.

The design of interfaces for automotive information systems is a critical task. In fact, such design must take into account that user is busy in the primary driving task, and any visual distraction determined by telematics systems can cause serious safety problems. To limit such distraction and enhance safety, in this paper we propose a novel multimodal user interface. The key element of the proposal is a new interaction device, named Handy, conceived to exploit the drivers tactile channel to minimize the workload of visual channel. Moreover Handy is suitably integrated with the graphical user interface, which is characterized by a reduced number of choices for each state and has been designed in agreement with the self-revealing approach.

Three on-road studies were conducted to determine how headway maintenance and collision warning displays influence driver behavior. Visual perspective, visual perspective with a pointer, visual perspective combined with an auditory warning, discrete visual warning, and discrete auditory warning were assessed during both coupled headway and deceleration events. Results indicate that when drivers are provided with salient visual information regarding safe headways, they utilize the information and increase their headway when appropriate. Auditory warnings were less effective than visual warnings for increasing headways but may be helpful for improving reaction time during events that require deceleration. Drivers were somewhat insensitive to false alarm rates, at least during short-term use. Finally, and most important, driver headway maintenace increased by as much as 0.5 s when the appropriate visual display was used. however, a study to investigate the long-term effects of such displays on behavior is strongly recommended prior to mass marketing of headway maintenance/collision warning devices.

Unique from other senses, the human haptic system supports two-way communications between humans and interactive systems, enabling bidirectional interaction between users and their surroundings. More specifically, haptic interaction offers an independent sensory channel that the brain can process to further enhance a user's experience in a multimodal environment. Such interaction can speed reaction time and reduce hand--eye coordination errors for computer-related tasks. Many interactive systems use visual and, to a lesser extent, audio cues to present users with information about their surroundings and interactions with objects. Haptic feedback can enhance interactive systems' realism through more natural interaction with objects and the environment. Rupert,1for example, used tactile actuators to provide cues for resolving spatial disorientation in aviation environments when visual cues are absent or misleading. Tan, Lim, and Traylor2designed a haptic car navigation guidance system by leveraging sensory saltation,a spatiotemporal illusion of movement across a person's back. Determining how to best design haptic interfaces is essential for further advancement of such novel haptic devices. An examination of haptic sensory systems and associated cognitive and motor processes should help direct the design of haptic interaction devices. This article surveys the haptics literature and identifies conditions under which haptic interaction displays can enhance human perception and performance. Sidebars present the guiding principles and issues associated with haptic interaction devices as well as haptics design guidelines for multimodal interactive systems.

We report two experiments designed to investigate the potential use of vibrotactile warning signals to present spatial information to car drivers. Participants performed an attention-demanding rapid serial visual presentation (RSVP) monitoring task. Meanwhile, whenever they felt a vibrotactile stimulus presented on either their front or back, they had to check the front and the rearview mirror for the rapid approach of a car, and brake or accelerate accordingly. We investigated whether speeded responses to potential emergency driving situations could be facilitated by the presentation of spatially-predictive (80% valid; Experiment 1) or spatially-nonpredictive (50% valid; Experiment 2) vibrotactile cues. Participants responded significantly more rapidly following both spatially-predictive and spatially-nonpredictive vibrotactile cues from the same rather than the opposite direction as the critical driving events. These results highlight the potential utility of vibrotactile warning signals in automobile interface design for directing a drivers visual attention to time-critical events or information.

The potential use of non-visual warning signals to present spatial information to car drivers has been successfully demonstrated in several recent studies (Ho & Spence, submitted, in preparation; Ho, Tan, & Spence, submitted). Among the three types of spatial warning signals investigated (namely auditory, visual, and vibrotactile), spatial vibrotactile cues were found to be particularly effective in directing a driver's visual spatial attention to potentially dangerous events on the road. We conducted the present study in order to examine the factors governing the relative effectiveness of auditory, visual, and vibrotactile warning signals. The speeded discrimination of warning signals presented in the various different modalities was investigated in order to explore whether the differences found in our previous research were a result of the relative speed with which people can detect warning signals presented in a given modality, or whether they were attributable to differences in the efficacy with which people can relate the warning signal to the subsequent visually-specified target driving events.

This paper describes a novel form of display using crossmodal output. A crossmodal icon is an abstract icon that can be instantiated in one of two equivalent forms (auditory or tactile). These can be used in interfaces as a means of non-visual output. This paper discusses how crossmodal icons can be constructed and the potential benefits they bring to mobile human computer interfaces.

As robots enter the human environment, there are increasing need for novice users to be able to program robots with ease. A successful robot programming system should be intuitive, interactive, and intention mare. Inhritiveness refers to the use of infuifive user interfaces such as speech and hand gestures. Interactivify refers to the system's abilig to let the user interact preemptively with the robot to toke its confrol at any given time. Intention awareness refers to the system's abilify to recognize andadapt to user intent. This paper focuses on the infention awareness problem for interactive multi-modal robot programming system. In our framework, user intent takes on the form of a robot program, which in our context is a sequential set of common& with parameters. To solve the intention recognition and adaptation problem, the system converts robot program into a set of Markov chains. The system con then deduce the most likely program the user intend to execute based on a given observation sequence. It then odapfs this program based on additional interaction. The system is implemented on a mobile vacuum cleaning robot with a user who is wearing sensor gloves, inductive position sensors, and a microphone.

Research into the visualization of abstract data has resulted in a number of domain specific solutions as well as some more generic techniques for visualising information. Similarly, the field of sonification has explored the display of information to the human auditory sense. As haptic displays such as force-feedback devices become more readily available, the sense of touch is also being used to help understand information. While applications that use multisensory information displays are becoming more common, frameworks to assist in design of these displays need to be developed. This paper extends a previously proposed structure of the visual design space to include hearing and touch and hence define a multi-sensory design space. It then correlates this space with another classification of the design space based on metaphors. Metaphors are often used as a starting point in designing information displays. Metaphors allow the user to take advantage of existing cognitive models as well as ecologically-developed perceptual skills. Metaphors provide another useful structuring of this multi- sensory design space. Throughout the paper all discussions are illustrated using the UML modeling notation. UML is a standard notation frequently used to document the design of software systems.

The ongoing evolution of modern technology from reactive tool to powerful and independent agent has created problems that are related to breakdowns in human-automation coordination. These breakdowns can be explained, in part, by the fact that machines can initiate actions on their own and without explicit operator consent but do not possess the communication skills that are required to know when and how to share information with operators concerning their intentions, actions, and limitations. One problem in particular is the extensive and increasing use of automation feedback that requires focal visual attention, especially in the context of unexpected changes and events. One possible solution to the problem is suggested by multiple-resource theory: the distribution of tasks and information across various sensory channels. In this manuscript, we will provide an overview of our recent series of simulator studies on the effectiveness of multimodal feedback for supporting early stages of attention management. Also, some of the many remaining challenges associated with supporting more complex coordinative functions in human-machine teams will be discussed.

This experiment investigates the interaction of different sensory cues in the control of propulsive forces in human gait which in turn allow the body's forward progression to be regulated. The aim of this work was to determine how optic flow and leg-somatosensory feedback interact in this control. We therefore determined whether the responses to sinusoidal perturbations of optic flow were accentuated when leg-somatosensory feedback was modified by varying the support resistance. Subjects walked on a treadmill which was driven by their own locomotor activity (1) with a sinusoidal variation of optic flow velocity, (2) with a sinusoidal variation of support resistance which modified leg-somatosensory information and (3) with both visual and leg-somatosensory modification at different frequencies. The response of the subject was measured as changes in speed and propulsive power. The response to sinusoidal perturbations of optic flow was found to be increased and time delayed when visual perturbations are coupled with support perturbations in comparison with the response observed with visual perturbations only. This result shows the influence of leg-somatosensory feedback on the weighting of optic flow. Inversely, it was also found that the motor response to support perturbation was different when the flow was congruent (i.e., corresponding to the subject's virtual speed) and when it was not. This latter result shows the influence of optic flow on the weighting of leg-somatosensory feedback. The interaction between optic flow and leg-somatosensory feedback argues in favor of a multimodal sensory control of propulsive forces. This multimodal sensory control would be based on all the sensory feedback and all their mutual sensorial interaction. Therefore, the modification of one sensory input modifies not only this input but also the integration of the other inputs.

Multimodal interfaces offer great potential to humanize interactions with computers by employing a multitude of perceptual channels. This paper reports on a novel multimodal interface using auditory, haptic, and visual feedback in a direct manipulation task to establish new recommendations for multimodal feedback, in particular uni-, bi-, and trimodal feedback. A close examination of combinations of uni-, bi-, and trimodal feedback is necessary to determine which enhances performance without increasing workload. Thirty-two participants were asked to complete a task consisting of a series of 'drag-and-drops' while the type of feedback was manipulated. Each participant was exposed to three unimodal feedback conditions, three bimodal feedback conditions and one trimodal feedback condition that used auditory, visual, and haptic feedback alone, and in combination. Performance under the different conditions was assessed with measures of trial completion time, target highlight time, and a self-reported workload assessment captured by the NASA Task Load Index (NASA-TLX). The findings suggest that certain types of bimodal feedback can enhance performance while lowering self-perceived mental demand.

Tactile icons (Tactons) are structured vibrotactile messages which can be used for non visual information display. Information is encoded in Tactons by manipulating vibrotactile parameters. This research investigates the possibilities of applying musical techniques to tactile icon design in order to define such parameters. Tactile versions of musical dynamics were created by manipulating the amplitude of vibrations to create increasing, decreasing, and level stimuli and an experiment was carried out to test perception of these stimuli. Identification rates of 92%-100% indicate that these tactile dynamics can be identified and distinguished from each other, and that tactile dynamics could be used in Tacton design.

The music therapist must have an understanding of the various disease and neurological processes to adequately assess functional ability and plan appropriate use of music for the course of treatment. We have memories for not only the particulars of a song, such as the melody or lyrics, but also the rich associations that keep the melodies alive for us throughout our life. Memories are not actually lost with dementia or with other brain injuries; rather, the ability to retrieve and gain access to these is damaged. As we possess memories for factual information, we also possess memories or `motor templates' for physical movements. Through music therapy, we have the ability to help patients reintegrate the sense of movement that they have lost. Despite the increased acceptance and understanding of the therapeutic benefits of music therapy in work with persons with neurologic impairments and challenges, more research needs to done to demonstrate it's efficacy in application with a wide range of diagnosis.

The IM4Sports music system helps exercisers select music that suits their training programs, reflects and guides sport performance, and collects data for adapting training programs and music selections.

This paper discusses a formal, mathematically-based, approach to the analysis of operator interaction with machines, in general, and with complex and automated control systems, in particular. It addresses the problem of correctness of displays by asking whether the display provides the necessary information about the machine to enable the operator to perform a specified task successfully and unambiguously. A formal methodology for verification of interface correctness is outlined. Additionally, a formal procedure for display synthesis, whose objective is to provide a succinct and correct interface for the specified task, is briefly discussed. Special attention is placed on the analysis of pilots' interaction with automated flight control systems onboard a modern commercial aircraft.

This paper proposes a complementary approach to Rasmussen's taxonomy of the human skill-, rule-, and knowledge-based performance models by combining the ecological concept of affordances with the neural concepts of human emotion and intuition. The classical cognitive engineering framework is extended through the neuro-ecological approach including personal human attributes important in exercising control over the work environment. The proposed affordance-, emotion-, and intuition-based models correspond to the three types of human performance, namely: learning, adaptive and tuning control, respectively. The new framework is not a predictive model of the operator behavior, but rather it describes the processes of neuro-ecological control of the human environment.

Unique from other senses, the human haptic system supports two-way communications between humans and interactive systems, enabling bidirectional interaction between users and their surroundings. More specifically, haptic interaction offers an independent sensory channel that the brain can process to further enhance a user's experience in a multimodal environment. Such interaction can speed reaction time and reduce hand--eye coordination errors for computer-related tasks. Many interactive systems use visual and, to a lesser extent, audio cues to present users with information about their surroundings and interactions with objects. Haptic feedback can enhance interactive systems' realism through more natural interaction with objects and the environment. Rupert,1for example, used tactile actuators to provide cues for resolving spatial disorientation in aviation environments when visual cues are absent or misleading. Tan, Lim, and Traylor2designed a haptic car navigation guidance system by leveraging sensory saltation,a spatiotemporal illusion of movement across a person's back. Determining how to best design haptic interfaces is essential for further advancement of such novel haptic devices. An examination of haptic sensory systems and associated cognitive and motor processes should help direct the design of haptic interaction devices. This article surveys the haptics literature and identifies conditions under which haptic interaction displays can enhance human perception and performance. Sidebars present the guiding principles and issues associated with haptic interaction devices as well as haptics design guidelines for multimodal interactive systems.

Ecological task analysis (ETA) is proposed as a technique for enhancing the acquisition of skilled interaction through the design of perceptually augmented information displays. Motivated by Brunswik and Gibson approaches, ETA rests on the primary assumption that skilled interaction will be most readily supported by a display on which perceptual information is available to fully specify the environmental constraints upon productive behavior. Demands for learning and resource-intensive cognitive activity are assumed to result when such information is not perceptually available. ETA can thus be used to identify perceptually impoverished conditions in order to indicate how displays should be augmented with perceptual information to enhance skill acquisition. The results of an experiment using a laboratory videogame, StarCruiser, which compared human performance using a perceptually augmented display with a baseline interface, are reported. Results indicated that the display based on the ETA framework significantly enhanced the acquisition of skill

As automation increasingly takes its place in industry, especially high-risk industry, it is often blamed for causing harm and increasing the chance of human error when failures occur. I propose that the problem is not the presence of automation, but rather its inappropriate design. The problem is that the operations under normal operating conditions are performed appropriately, but there is inadequate feed back and interaction with the humans who must control the overall conduct of the task. When the situations exceed the capabilities of the automatic equipment, then the inadequate feedback leads to difficulties for the human controllers. The problem, I suggest, is that the automation is at an intermediate level of intelligence, powerful enough to take over control that used to be done by people, but not powerful enough to handle all abnormalities. Moreover, its level of intelligence is insufficient to provide the continual, appropriate feedback that occurs naturally among human operators. This is the source of the current difficulties. To solve this problem, the automation should either be made less intelligent or more so, but the current level is quite inappropriate. The overall message is that it is possible to reduce error through appropriate design considerations. Appropriate design should assume the existence of error, it should continually provide feedback, it should continually interact with operators in an effective manner, and it should allow for the worst of situations. What is needed is a soft, compliant technology, not a rigid, formal one.

In these experiments, two plates were grasped between the thumb and forefinger and squeezed together along a linear track. An electromechanical system presented a constant resistance force during the squeeze up to a predetermined location on the track, whereupon the force effectively went to infinity (simulating a wall) or to zero (simulating a cliff). The task of the subject was to discriminate between two alternative levels of the constant resistance force (a reference level and a reference-plus-increment level). Results of these experiments indicate a just noticeable difference of roughly 7% of the reference force using a one-interval paradigm with trial-by-trial feedback over the ranges 2.5 less than or equal to F0 less than or equal to 10.0 newtons, 5 less than or equal to D less than or equal to 30 mm, 45 less than or equal to S less than or equal to 125 mm, and 25 less than or equal to V less than or equal to 160 mm/sec, where F0 is the reference force, D is the distance squeezed, S is the initial fingerspan, and V is the mean velocity of the squeeze. These results, based on tests with 5 subjects, are consistent with a wide range of previous results, some of which are associated with other body surfaces and muscle systems and many of which were obtained with different psychophysical methods.

The ongoing evolution of modern technology from reactive tool to powerful and independent agent has created problems that are related to breakdowns in human-automation coordination. These breakdowns can be explained, in part, by the fact that machines can initiate actions on their own and without explicit operator consent but do not possess the communication skills that are required to know when and how to share information with operators concerning their intentions, actions, and limitations. One problem in particular is the extensive and increasing use of automation feedback that requires focal visual attention, especially in the context of unexpected changes and events. One possible solution to the problem is suggested by multiple-resource theory: the distribution of tasks and information across various sensory channels. In this manuscript, we will provide an overview of our recent series of simulator studies on the effectiveness of multimodal feedback for supporting early stages of attention management. Also, some of the many remaining challenges associated with supporting more complex coordinative functions in human-machine teams will be discussed.

I examine the central theoretical construct of ecological psychology, the concept of an affordance. In the first part of the paper, I illustrate the role affordances play in Gibson's theory of perception. In the second part, I argue that affordances are to be understood as dispositional properties, and explain what I take to be their characteristic background circumstances, triggering circumstances and manifestations. The main purpose of my analysis is to give affordances a theoretical identity enriched by Gibson's visionary insight, but independent of the most controversial claims of the Gibsonian movement.

This paper is both a reply to Fodor and Pt_lyshyn (198 1) and a systematic explication of one of Gibson's (1979) basic claims, namely, that there are ecological laws relating organisms tc the a.ffordances of the environment. Gibson's theory of affordances holds great promise for psychology for a number of reasons: it provides a framework for the precise formulation and testing of hypotheses about behavior and perception (e.g. E. J. Gibson, in press; Johnston and Turvey, 1980; Lee, 19110; Shaw and Bransford, 1977); it suggests a way to integrate theOphenomenological and mechanistic aspects of psychology without succumbing to either one-sided point of view (Reed, 1980; Runeson, 1977;Shaw et al., in press;Tua-:rey and Shaw, 1979); and it promises to put psychology back on the track of seeking lawful relations-as Gibson (1967, p. 122) once said, in science: ``You either find causal relations or you do not'``.

Actions must be controlled prospectively. This requires the the behavioral possibilities of surface layouts and events to be perceived. In this article, the ontological basis for an understanding of prospective control in realis terms is outlined. The foundational idea is that of affordance and the promoted ontology is materialist and dynamicist. It is argued that research in the ecological approach to prospective control is ultimately the search for objective laws. Because lawfulness is equated with real possibility, this amounts to the study of the affordance (the real possibilities) underlying prospective control and the circumstances that actualize them. The ontological assumptions and hypotheses bearing on this latter proposal are articulated. It is suggested that critical evaluation of the identified ontological themes may benefit the experimental and theoretical study of perception in the service of activity.

Two experiments pursued previous studies (P. Viviani & P. Mounoud, 1990; P. Viviani & N. Stucchi, 1989) on motor-perceptual interactions. The right arm of blindfolded participants was moved passively along elliptic trajectories, Kinematics was either coherent or at variance with the relation (two-thirds power law) observed in active movements. In Experiment 1 participants compared the horizontal and vertical extent of the ellipses. Kinematics affected aspect ratio discrimination: The direction along which the movement decelerated was subjectively stretched. In Experiment 2 participants used the left arm to reproduce in real time the movement of the right ann. The trajectories of the left arm presented a stretch similar to the perceptual illusion demonstrated in Experiment 1. Between-arm asynchrony suggests that the motor control system cannot use kinesthetic information that is at variance with the flow of reafferences normally associated with voluntary movements. It is argued that these interactions occur at the level of a central amodal representation of the stimuli.

This paper describes preliminary work in the use of a virtual environment to derive just noticeable differences (JNDs) for force. Specifically, we look for thresholds of force sensitivity so that we may ultimately construct therapeutic force feedback distortions that stay below these thresholds. Initially, we have concentrated on JNDs as they are applied to the index finger; preliminary data in healthy individuals shows an average JND of approximately 10%. More significantly, the data indicate that visual feedback distortions in a virtual environment can be created to encourage increased force productions by up to 10%, and that this can be done without a patient's awareness.

Attention and motivation are two factors, which are important for motor relearning following stroke. In this paper we introduce and present the novel concepts for the use of haptics technology to deliver therapy to patients with arm impairments following stroke. The results of the ongoing initial pilot studies have shown the feasibility of the system presented here. The new approach will potentially improve the patient's attention and motivation and hence enhance therapy effectiveness.

This paper discusses the design of a compatible controller for the Expert-Based Variable Resistance/Assistance (EVRA) exercise machine that removes the shortcomings found in the currently available constant-resistance and other variable resistance exercise machines. A mathematical model of the EVRA prototype is used to simulate its dynamic behavior using the Matlab/Simulink software package. The feed-back controller generates control signals to engage an electric motor to provide either assistance/resistance as per demand. The proposed controller is able to detect significant changes in the kinematic and neurophysiological movement profiles, compare this data with an existing database and then provide the appropriate level of mechanical assistance to the moving limb to maintain a coordinated movement profile. A comparative study on the various types of controllers (such as PI and Neuro-controllers) is also presented.

Machine-mediated training of dynamic task completion is typically implemented with passive intervention via virtual fixtures or active assist by means of record and replay strategies. During interaction with a real dynamic system however, the user relies on both visual and haptic feedback real-time in order to elicit desired motions. This work investigates human performance in a Fitts' type targeting task with an underactuated dynamic system. Performance, in terms of number of hits and between-target tap times, is measured while various passive and active control modes are displayed concurrently with the haptic feedback from the simulated system's own dynamic behavior. It is hypothesized that passive and active assist modes that are implemented during manipulation of simulated underactuated systems could be beneficial in rehabilitation applications. Results indicate that human performance can be improved significantly with the passive and active assist modes.

Successful motor rehabilitation of stroke, TBI and SCI subjects requires an intensive and task-specific therapy approach. Budget constrains limit a hand-to-hand therapy approach, so that intelligent machines may offer a solution to further promote motor recovery and to better understand motor control. This new field of automated or robot-assisted motor rehabilitation has emerged since the nineties. The article presents clinically viable devices for the upper and lower extremity rehabilitation, which have been developed by our group. The Bi-Manu-Track, enables the bilateral practice of a forearm and wrist movement, and is currently tested. For gait rehabilitation after stroke, the electromechanical gait trainer GT I has proven effective as compared to treadmill training with Body weight support (BWS). The latest development is the HapticWalker, a robotic walking simulator for gait training, which allows the training of arbitrary daily life foot trajectories. The foot trajectories can be individu- ally adjusted to each patient, in addition the machine will offer a sophisticated patient-machine-interaction. A prototype of the machine has been designed and built succesfully and is being evaluated at present. Technical possibilities are one aspect, but multi-centre trials and the consideration of unsubstantiated fears among therapists of being replaced by the machines will decide on the successful implementation of this most promising field to the benefit of patients.

Robotic devices hold much promise for use as rehabilitation aids but their success depends on identifying effective strategies for controlling human--robot interaction forces. We developed a robotic device to test a novel method of controlling interaction forces with the intent of improving force symmetry in the limbs. Users perform lower limb extensions against a computer-controlled resistive load. The control software increases resistance above baseline in proportion to lower limb force asymmetry (balance between left and right limb forces). As a preliminary trial to test the device and controller, we conducted two experiments on neurologically intact subjects. In experiment 1, one group of subjects received symmetry-based resistance while performing lower limb extensions (n = 10). A control group performed the same movements with constant resistance (n = 10). The symmetry-based resistance group improved lower limb symmetry during training (ANOVA, p<0.05), whereas the control subjects did not. In experiment 2, subjects (n = 10) successfully used symmetry-based resistance to alter their lower limb force production towards a target asymmetry (ANOVA, p<0.05). These studies suggest that symmetry-based resistance may hold rehabilitation benefits after orthopedic or neurological injury. Specifically, performing strength training therapy with this controller may allow hemiparetic individuals to focus better on increasing strength and neuromuscular recruitment in their paretic limb while experiencing symmetric limb forces.

Experimental and control groups of 10 hemiparetic stroke patients each underwent a 6 week, twice daily gait training program. The control group participated in a conventional physical therapy gait program. The experimental group trained in the same basic program with the addition of rhythmic auditory stimulation (RAS). Patients entered the study as soon as they could complete 5 strides with hand-held assistance. The training program had to be completed within 3 months of the patients' stroke. In the experimental group RAS was used as a timekeeper to synchronize step patterns and gradually entrain higher stride frequencies. Study groups were equated by gender, lesion site, and age. Motor function was assessed at pretest using Barthel, Fugl-Meyer, and Berg Scales. Walking patterns were assessed during pre- and post-test without RAS present. Pre- vs post-test measures revealed a statistically significant (P<0.05) increase in velocity (164% vs 107%), stride length (88% vs 34%), and reduction in EMG amplitude variability of the gastrocnemius muscle (69% vs 33%) for the RAS-training group compared to the control group. The difference in stride symmetry improvement (32% in the RAS-group vs 16% in the control group) was statistically not significant. The data offer evidence that RAS is an efficient tool to enhance efforts in gait rehabilitation with acute stroke patients.

The music therapist must have an understanding of the various disease and neurological processes to adequately assess functional ability and plan appropriate use of music for the course of treatment. We have memories for not only the particulars of a song, such as the melody or lyrics, but also the rich associations that keep the melodies alive for us throughout our life. Memories are not actually lost with dementia or with other brain injuries; rather, the ability to retrieve and gain access to these is damaged. As we possess memories for factual information, we also possess memories or `motor templates' for physical movements. Through music therapy, we have the ability to help patients reintegrate the sense of movement that they have lost. Despite the increased acceptance and understanding of the therapeutic benefits of music therapy in work with persons with neurologic impairments and challenges, more research needs to done to demonstrate it's efficacy in application with a wide range of diagnosis.

The potential for augmented reality (AR) technologies to impact work habits and collaborative work is perhaps most striking for individuals with sensory or perceptual impairments. Commercial display and sensing technologies, in combination with on-board computation capabilities (either in the form of specialized hardware or general-purpose wearable computers), are introducing a new generation of adaptive aids. Spectacles and traditional hearing aids are being replaced by customized and context-sensitive conditional display systems. These technologies will enable much broader access, both to day-to-day interaction and to our increasingly information-based workspace. Two specific examples illustrate many of the technological and human factors challenges presented in the development of these new AR sensory aids. The lessons and technological advances gained from these efforts may have much broader implications for the design and implementation of generic AR systems.

Complex bodily rhythms are ubiquitous in living organisms. These rhythms arise from stochastic, nonlinear biological mechanisms interacting with a fluctuating environment. Disease often leads to alterations from normal to pathological rhythm. Fundamental questions concerning the dynamics of these rhythmic processes abound. For example, what is the origin of physiological rhythms? How do the rhythms interact with each other and the external environment? Can we decode the fluctuations in physiological rhythms to better diagnose human disease? And can we develop better methods to control pathological rhythms? Mathematical and physical techniques combined with physiological and medical studies are addressing these questions and are transforming our understanding of the rhythms of life.

Low latency processing is usually a goal in real-time audio applications; however, it is not clear how little latency is to be considered low enough. We discuss currently available experimental data on human perception and argue that somewhat high latencies (around 20--30ms) are probably perfectly acceptable for typical musical applications. We also argue that it should be possible to accept various levels of latency on a system if we can be aware of the effects of this latency on the users of the system; therefore, we still need more experimental data on latency perception to be able to better assess the effects of latency on musical applications. Such an experiment is suggested.

Main aim of these studies is to figure out basic central mechanisms in the timing of simple motor actions by magnetoencephalography (MEG) which permits noninvasive recordings of cortical activity with high temporal and spatial resolution. Our basic paradigm is a sensorimotor synchronization task which, on the one hand, is quite wellknown in behavioral parameters whereas, on the other hand, almost nothing is known about underlying central processes, in particular, which sensory or motor events are synchronized by subjects.

A coordinated rhythmic movement pattern is a dynamical activity involving many hidden layers of rhythmic subtasks. To investigate this dynamical substructure, spectroscopic concepts and methods were applied to an interlimb rhythmic movement task requiring 1:1 frequency locking of two hand-held pendulums in 180" phase relation. The pendulums could be of identical or very different dimensions, thereby providing different values of the ratio fl of uncoupled frequencies. Analyses focused on the power spectrum of continuous relative phase as a function of variation in ft. Predictions were derived from the theories of mode locking and fractal time. Experimental results were in agreement with theoretical expectations and were discussed in terms of the possible recruiting of rhythmic subtasks in the assembling of inteflimb absolute coordination, the interdependence of these subtasks, and the general dynamical principles that relate coordinative processes occurring at different length and time scales.

Experimental and control groups of 10 hemiparetic stroke patients each underwent a 6 week, twice daily gait training program. The control group participated in a conventional physical therapy gait program. The experimental group trained in the same basic program with the addition of rhythmic auditory stimulation (RAS). Patients entered the study as soon as they could complete 5 strides with hand-held assistance. The training program had to be completed within 3 months of the patients' stroke. In the experimental group RAS was used as a timekeeper to synchronize step patterns and gradually entrain higher stride frequencies. Study groups were equated by gender, lesion site, and age. Motor function was assessed at pretest using Barthel, Fugl-Meyer, and Berg Scales. Walking patterns were assessed during pre- and post-test without RAS present. Pre- vs post-test measures revealed a statistically significant (P<0.05) increase in velocity (164% vs 107%), stride length (88% vs 34%), and reduction in EMG amplitude variability of the gastrocnemius muscle (69% vs 33%) for the RAS-training group compared to the control group. The difference in stride symmetry improvement (32% in the RAS-group vs 16% in the control group) was statistically not significant. The data offer evidence that RAS is an efficient tool to enhance efforts in gait rehabilitation with acute stroke patients.

The music therapist must have an understanding of the various disease and neurological processes to adequately assess functional ability and plan appropriate use of music for the course of treatment. We have memories for not only the particulars of a song, such as the melody or lyrics, but also the rich associations that keep the melodies alive for us throughout our life. Memories are not actually lost with dementia or with other brain injuries; rather, the ability to retrieve and gain access to these is damaged. As we possess memories for factual information, we also possess memories or `motor templates' for physical movements. Through music therapy, we have the ability to help patients reintegrate the sense of movement that they have lost. Despite the increased acceptance and understanding of the therapeutic benefits of music therapy in work with persons with neurologic impairments and challenges, more research needs to done to demonstrate it's efficacy in application with a wide range of diagnosis.

Investigating virtual environments has become an increasingly interesting research topic for engineers, computer and cognitive scientists, and psychologists. Although there have been several recent studies focused on the development of multimodal virtual environments (VEs) to study human-machine interactions, less attention has been paid to human-human and human-machine interactions in shared virtual environments (SVEs), and to our knowledge, no attention paid at all to what extent the addition of haptic communication between people would contribute to the shared experience. We have developed a multimodal shared virtual environment and performed a set of experiments with human subjects to study the role of haptic feedback in collaborative tasks and whether haptic communication through force feedback can facilitate a sense of being and collaborating with a remote partner. The study concerns a scenario where two participants at remote sites must co-operate to perform a joint task in a SVE. The goals of the study are (1) to assess the impact of force feedback on task performance, (2) to better understand the role of haptic communication in human-human interactions, (3) to study the impact of touch on the subjective sense of collaborating with a human as reported by the participants based on what they could see and feel, and (4) to investigate if gender, personality, or emotional experiences of users can affect haptic communication in SVEs. The outcomes of this research can have a powerful impact on the development of next generation human-computer interfaces and network protocols that integrate touch and force feedback technology into the Internet, development of protocols and techniques for collaborative teleoperation such as hazardous material removal, space station repair, and remote surgery, and enhancement of virtual environments for performing collaborative tasks in shared virtual worlds on a daily basis such as co-operative teaching, training, planning and design, cybergames, and social gatherings. Our results suggest that haptic feedback significantly improves the task performance and contributes to the feeling of `sense of togetherness' in SVEs. In addition, the results show that the experience of visual feedback only at first, and then subsequently visual plus haptic feedback elicits a better performance than presentation of visual plus haptic feedback first followed by visual feedback only.

Current Systems for real-time distributed CSCW are largely rooted in traditional GUI-based groupware and voice/video coferencing methodologies. In these approaches, interactions are limited to visual and auditory media and shared environments are confined to the digital world. This paper presents a new approach to enhance remote collaboration and communication, based on the idea of Tangible interaces, which places a greater emphasis on touch and physicality. The approach is grounded in a concept called Synchronized Distributed Physical Objects, which employs telemanipulation technology to create the illusion that distant users are interacting with shared physical objects. We describe two applications of this approach: PSyBench, a physical shared workspace, and inTouch, a device for haptic interpersonal communication.

To see two expert partners, one leading and the other following, swing dance together is to watch a remarkable two-agent communication and control system in action. Even blindfolded, the follower can decode the leader's moves from haptic cues. The leader composes the dance from the vocabulary of known moves so as to complement the music he is dancing to. Systematically addressing questions about partner dance communication is of scientific interest and could improve human-robotic interaction, and imitating the leader's choreographic skill is an engineering problem with applications beyond the dance domain. Swing dance choreography is a finite state machine, with moves that transition between a small number of poses. Two automated choreographers are presented. One uses an optimization and randomization scheme to compose dances by a sequence of shortest path problems, with edge lengths measuring the dissimilarity of dance moves to each bar of music. The other solves a two-player zero-sum game between the choreographer and a judge. Choosing moves at random from among moves that are good enough is rational under the game model. Further, experiments presenting conflicting musical environments to two partners demonstrate that although musical expression clearly guides the leader's choice of moves, the follower need not hear the same music to properly decode the leader's signals. Dancers embody gentle interaction, in which each participant extends the capabilities of the other, and their cooperation is facilitated by a shared understanding of the motions to be performed. To demonstrate that followers use their understanding of the move vocabulary to interact better with their leaders, an experiment paired a haptic robot leader with human followers in a haptically cued dance to a swing music soundtrack. The subjects' performance differed significantly between instances when the subjects could determine which move was being led and instances when the subjects could not determine what the next move would be. Also, two-person teams that cooperated haptically to perform cyclical aiming tasks showed improvements in the Fitts' law or Schmidt's law speed-accuracy tradeoff consistent with a novel endpoint compromise hypothesis about haptic collaboration.

In this paper, a paradigm for shared control is described in which a machine's manual control interface is motorized to allow a human and an automatic controller to simultaneously exert control. The manual interface becomes a haptic display, relaying information to the human about the intentions of the automatic controller while retaining its role as a manual control interface. The human may express his control intentions in a way that either overrides the automation or conforms to it. The automatic controller, by design, aims to create images in the mind of the human of fixtures in the shared workspace that can be incorporated into efficient task completion strategies. The fixtures are animated under the guidance of an algorithm designed to automate part of the human/machine task. Results are presented from 2 experiments in which 11 subjects completed a path following task using a motorized steering wheel on a fixed-base driving simulator. These results indicate that the haptic assist through the steering wheel improves lane keeping by at least 30% reduces visual demand by 29% (p<0.0001) and improves reaction time by 18 ms (p=0.0009).

In this paper an new intelligent robot control scheme is presented which enables a cooperative work of humans and robots through direct contact interaction in a partially known environment. The control of the degrees of freedom of the manipulator is shared through a hybrid controller between the operator and the intelligent control system. The allocation of the control among human and robot occurs dynamically according to the state of the task execution and the capabilities of the human and the robot. Experiments were conducted to prove the effectiveness of our concept.

Users of collaborative systems are typically restricted to communication through voice and video links. Users find this difficult -- it does not encompass the richness of communication they are accustomed to in the real world. Attempting to address this problem we describe the implementation of a novel mechanism for haptic communication based around interactions between users' cursors. An initial, and mainly observational, evaluation is described, along with some promising results. We show improvements in subjective experience and suggest several, more formal, avenues for future research.

Communication between users in shared editors takes place in a deprived environment -- distributed users find it difficult to communicate. While many solutions to the problems this causes have been suggested this paper presents a novel one. It describes one possible use of haptics as a channel for communication between users. User's telepointers are considered as haptic avatars and interactions such as haptically pushing and pulling each other are afforded. The use of homing forces to locate other users is also discussed, as is a proximity sensation based on viscosity. Evaluation of this system is currently underway.

Machine-mediated training of dynamic task completion is typically implemented with passive intervention via virtual fixtures or active assist by means of record and replay strategies. During interaction with a real dynamic system however, the user relies on both visual and haptic feedback real-time in order to elicit desired motions. This work investigates human performance in a Fitts' type targeting task with an underactuated dynamic system. Performance, in terms of number of hits and between-target tap times, is measured while various passive and active control modes are displayed concurrently with the haptic feedback from the simulated system's own dynamic behavior. It is hypothesized that passive and active assist modes that are implemented during manipulation of simulated underactuated systems could be beneficial in rehabilitation applications. Results indicate that human performance can be improved significantly with the passive and active assist modes.

This paper presents a shared-control interaction paradigm for haptic interface systems, with experimental data from two user studies. Shared control, evolved from its initial telerobotics applications, is adapted as a form of haptic assistance in that the haptic device contributes to execution of a dynamic manual target-hitting task via force commands from an automatic controller. Compared to haptic virtual environments, which merely display the physics of the virtual system, or to passive methods of haptic assistance for performance enhancement based on virtual fixtures, the shared-control approach offers a method for actively demonstrating desired motions during virtual environment interactions. The paper presents a thorough review of the literature related to haptic assistance. In addition, two experiments were conducted to independently verify the efficacy of the shared-control approach for performance enhancement and improved training effectiveness of the task. In the first experiment, shared control is found to be as effective as virtual fixtures for performance enhancement, with both methods resulting in significantly better performance in terms of time between target hits for the manual target-hitting task than sessions where subjects feel only the forces arising from the mass-spring-damper system dynamics. Since shared control is more general than virtual fixtures, this approach may be extremely beneficial for performance enhancement in virtual environments. In terms of training enhancement, shared control and virtual fixtures were no better than practice in an unassisted mode. For manual control tasks, such as the one described in this paper, shared control is beneficial for performance enhancement, but may not be viable for enhancing training effectiveness.

Haptic interaction between people and machines might benefit from an understanding of haptic communication between one person and another. We recently reported results showing that two people performing a physically shared dyadic task can outperform either person alone, even when the perception of each participant is that the other is a hindrance. Evidently a dyad quickly negotiates a more efficient motion strategy than is available to individuals. This negotiation must take place through a haptic channel of communication, and it is apparently at a level below the awareness of the participants. Here we report results on the motion strategy that emerged. By recording forces and motions we show that the dyads ``specialized'' temporally such that one member took on early parts of the motion and the other late parts. Tests in which one participant's contribution was surreptitiously replaced by a motor did not elicit a similar cooperative response from the remaining human participant, showing that the language of haptic communication between people must be rather subtle.

Machines that interact with humans might benefit from an understanding of the subtlety of human-human interaction. We recently reported results that two people working cooperatively will temporally specialize such that one member will command the early parts of motion and the other the late parts. In our current study, we replaced one of the humans with a robot designed to simulate this specialization. We expected the remaining human to respond to the robot the same way as another human, but they did not. Even though the subjects believed that they were interacting with a person, the subjects did not work with the robot in a similar fashion; their force profile was remarkably similar to how they worked alone. The subjects consciously believed they were working with a person, but their force profile showed that they were not. Understanding how two people physically work together is the beginning of understanding how a robot can work fluidly with a robot in a human like way.

The ongoing evolution of modern technology from reactive tool to powerful and independent agent has created problems that are related to breakdowns in human-automation coordination. These breakdowns can be explained, in part, by the fact that machines can initiate actions on their own and without explicit operator consent but do not possess the communication skills that are required to know when and how to share information with operators concerning their intentions, actions, and limitations. One problem in particular is the extensive and increasing use of automation feedback that requires focal visual attention, especially in the context of unexpected changes and events. One possible solution to the problem is suggested by multiple-resource theory: the distribution of tasks and information across various sensory channels. In this manuscript, we will provide an overview of our recent series of simulator studies on the effectiveness of multimodal feedback for supporting early stages of attention management. Also, some of the many remaining challenges associated with supporting more complex coordinative functions in human-machine teams will be discussed.

When humans interface with machines, the control interface is usually passive and its response contains little information pertinent to the state of the environment. Usually, information flows through the interface from human to machine but not so often in the reverse direction. This work proposes a control architecture in which bi-directional information transfer occurs across the control interface, allowing the human to use the interface to simultaneously exert control and extract information. In this alternative control architecture, which we call shared control, the human utilizes the haptic sensory modality to share control of the machine interface with an automatic controller. We present a fixed-base driving simulator experiment in which subjects take advantage of a haptic steering wheel, which aids them in a path following task. Results indicate that the haptic steering wheel allows a significant reduction in visual demand while improving path following performance.

Haptic interfaces have the potential to enhance communication and interaction via the computer---enabling a fective expressive interpersonal communication and enriching interaction by haptic feedback. Still, what exactly their potential is and how we can design in order to fulfil it remains topic of contemporary debate. My contribution to this debate shall be to place some of the current developments into a philosophical and cultural context and to introduce social science based methodologies, which will help broaden the discussion and scope of input. Through semiotic analysis, we can predict `meaning making' in haptic communication that goes beyond linguistic description. Examples of Haptic interfaces shall be positioned as case studies in this typology. Also, I describe the Haptic Box and PinKom as my way of investigating a semiotic system of touch. In conclusion, this paper hopes to inform and catalyse the development of a haptic design palette.

Human communication and interaction comprise a wide range of verbal and nonverbal cues. Further adoption of novel telecommunication methods such as e-mail, chat, instant messaging (IM), mobile phone SMS text messaging, and videoconferencing; have augmented our mediated interaction abilities. However, a significant (and important) amount of human expression and interaction information is never captured, transmitted, or expressed with current computer mediated communication (CMC) tools. We also lack ambient methods of maintaining contact when not co-located with family and friends. Communal Interfaces is a new research effort aimed at the study of nonverbal human cues: their intent, motion, meaning, subtleties, and importance in communication. In this paper we address issues involved in the design, construction, and evaluation of Connexus, one such communal interface.

Instant Messaging (IM) is a popular chatting platform on the internet and increasingly permeates teenage life. Even intimate and emotional content is discussed. As touch is a powerful signal for emotional content, haptic signals, and especially hapticons can contribute to overcome the inevitable loss of subtle non-verbal communication cues. Audio-visual extensions of IM to share emotions, in particular emoticons, have been received enthusiastically by IM users. This indicates a realistic user-need for hapticons in IM. The Haptic Instant Messaging (HIM) framework introduced in this paper combines communication of textual messages with haptic effects and hapticons. The application is build as an open framework and supports small chatting communities to explore the design and use of hapticons and haptic IO devices. Researchers can use the HIM framework to monitor the use of haptics in communication and how haptics contribute to the fun and meaning of instant messaging.

This paper reports two experiments relating to the design of Tactons (or tactile icons). The first experiment investigated perception of vibro-tactile ``roughness'' (created using amplitude modulated sinusoids), and the results indicated that roughness could be used as a parameter for constructing Tactons. The second experiment is the first full evaluation of Tactons, and uses three values of roughness identified in the first experiment, along with three rhythms to create a set of Tactons. The results of this experiment showed that Tactons could be a successful means of communicating information in user interfaces, with an overall recognition rate of 71%, and recognition rates of 93% for rhythm and 80% for roughness.

While the sense of touch is capable of processing complex stimuli, the vibration feedback used in mobile phones is generally very simple. Using more complex vibrotactile messages would enable the communication of more information through phone alerts, however it has been suggested that phone vibration motors are not capable of presenting complex messages. This paper reports a study investigating the use of Tactons (tactile icons), presented using a standard mobile phone vibration motor, to represent mobile phone alerts. The recognition rate of 72% achieved for Tactons encoding two pieces of information is comparable to results achieved in a previous experiment with a high specification transducer, indicating that it is possible to communicate multi-dimensional information in mobile phone alerts. These results will help designers to understand the possibilities offered by standard phone vibration motors for communicating complex information.

We describe the design of ComTouch, a device that augments remote voice communication with touch, by converting hand pressure into vibrational intensity between users in real-time. The goal of this work is to enrich interpersonal communication by complementing voice with a tactile channel. We present preliminary user studies performed on 24 people to observe possible uses of the tactile channel when used in conjunction with audio. By recording and examining both audio and tactile data, we found strong relationships between the two communication channels. Our studies show that users developed an encoding system similar to that of Morse code, as well as three original uses: emphasis, mimicry, and turn-taking. We demonstrate the potential of the tactile channel to enhance the existing voice communication channel.

A new breed of mobile phones has been designed to enable concurrent vibration and audio stimulation, or audio-haptics. This paper aims to share techniques for creating and optimizing audio-haptic effects to enhance the user interface. The authors present audio manipulation techniques specific to the multifunction transducer (MFT) technology. In particular two techniques, the Haptic Inheritance and Synthesis and Matching methods are discussed. These two methods of haptic media generation allow simple creation of vibration content, and also allow for compatibility with non-haptic mobile devices. The authors present preliminary results of an evaluation of 42 participants comparing audio-based haptic user interface (UI) feedback with audio-only feedback. The results show that users were receptive to audio-haptic UI feedback. The results also suggest that audio-haptics seems to enhance the perception of audio quality.

This paper presents an overview of the tactile properties based on tactile languages, such as Braille and fingerspelling. The unique spatial and temporal properties of touch through use of exploratory procedures highlight the amount information available through touch. The authors make recommendations for haptic visualization to actively engage exploratory procedures and exploit context from other modalities. The authors present a mobile device augmented with tactile UI feedback. General observations based on public awareness to the haptic phenomenon are described.

As autonomous machines become more pervasive, situations will arise when human decision-makers will receive advice from both machines and other humans. When these instructions conflict, a new social situation is defined for which we have little precedent. The authors propose a model for investigating these situations. The model synthesizes research from several different fields, including machine autonomy, affect, initial trust, individual differences, and training. The model is explained, and a set of propositions is described. The model is used to analyze the case of an air collision in which machines and humans provided conflicting advice. The model is also applied to situations in which unmanned aerial vehicles and piloted aircraft seek to avoid collisions with each other. Ways of testing the model through human subject experiments are discussed.

Much previous research has established that perceived ease of use is an important factor influencing user acceptance and usage behavior of information technologies. However, very little research has been conducted to understand how that perception forms and changes over time. The current work presents and tests an anchoring and adjustment-based theoretical model of the determinants of system-specific perceived ease of use. The model proposes control (internal and external---conceptualized as computer self-efficacy and facilitating conditions, respectively), intrinsic motivation (conceptualized as computer playfulness), and emotion (conceptualized as computer anxiety) as anchors that determine early perceptions about the ease of use of a new system. With increasing experience, it is expected that system-specific perceived ease of use, while still anchored to the general beliefs regarding computers and computer use, will adjust to reflect objective usability, perceptions of external control specific to the new system environment, and system-specific perceived enjoyment. The proposed model was tested in three different organizations among 246 employees using three measurements taken over a three-month period. The proposed model was strongly supported at all points of measurement, and explained up to 60% of the variance in system-specific perceived ease of use, which is twice as much as our current understanding. Important theoretical and practical implications of these findings are discussed.

Investigating virtual environments has become an increasingly interesting research topic for engineers, computer and cognitive scientists, and psychologists. Although there have been several recent studies focused on the development of multimodal virtual environments (VEs) to study human-machine interactions, less attention has been paid to human-human and human-machine interactions in shared virtual environments (SVEs), and to our knowledge, no attention paid at all to what extent the addition of haptic communication between people would contribute to the shared experience. We have developed a multimodal shared virtual environment and performed a set of experiments with human subjects to study the role of haptic feedback in collaborative tasks and whether haptic communication through force feedback can facilitate a sense of being and collaborating with a remote partner. The study concerns a scenario where two participants at remote sites must co-operate to perform a joint task in a SVE. The goals of the study are (1) to assess the impact of force feedback on task performance, (2) to better understand the role of haptic communication in human-human interactions, (3) to study the impact of touch on the subjective sense of collaborating with a human as reported by the participants based on what they could see and feel, and (4) to investigate if gender, personality, or emotional experiences of users can affect haptic communication in SVEs. The outcomes of this research can have a powerful impact on the development of next generation human-computer interfaces and network protocols that integrate touch and force feedback technology into the Internet, development of protocols and techniques for collaborative teleoperation such as hazardous material removal, space station repair, and remote surgery, and enhancement of virtual environments for performing collaborative tasks in shared virtual worlds on a daily basis such as co-operative teaching, training, planning and design, cybergames, and social gatherings. Our results suggest that haptic feedback significantly improves the task performance and contributes to the feeling of `sense of togetherness' in SVEs. In addition, the results show that the experience of visual feedback only at first, and then subsequently visual plus haptic feedback elicits a better performance than presentation of visual plus haptic feedback first followed by visual feedback only.

The potential for augmented reality (AR) technologies to impact work habits and collaborative work is perhaps most striking for individuals with sensory or perceptual impairments. Commercial display and sensing technologies, in combination with on-board computation capabilities (either in the form of specialized hardware or general-purpose wearable computers), are introducing a new generation of adaptive aids. Spectacles and traditional hearing aids are being replaced by customized and context-sensitive conditional display systems. These technologies will enable much broader access, both to day-to-day interaction and to our increasingly information-based workspace. Two specific examples illustrate many of the technological and human factors challenges presented in the development of these new AR sensory aids. The lessons and technological advances gained from these efforts may have much broader implications for the design and implementation of generic AR systems.

This paper proposes a new locomotion haptic interface, the Virtual Walking Machine (VWM) that can simulate human walking more naturally. The VWM allows walking in a virtual environment with uneven terrain, which can improve immersiveness significantly. The proposed system is composed mainly of: (i) a fifteen degree-of-freedom hybrid parallel mechanism that allows relative rotation between toe and heel, omni-directional rotation, and natural walking distance, (ii) Pulse Width Modulation (PWM) driven pneumatic actuators for vertical motion and servo-motors for planar motion, (iii) sensors that allow motion detection. Design requirements on natural walking are analyzed and used in conjunction with kinematic analyses to optimally design the proposed VWM.

This work addresses the use of vibrotactile haptic feedback to transmit background information with variable intrusiveness, when recipients are engrossed in a primary visual and/or auditory task. We describe two studies designed to (a) perceptually optimize a set of vibrotactile "icons" and (b) evaluate users' ability to identify them in the presence of varying degrees of workload. Seven icons learned in approximately 3 minutes were each typically identified within 2.5 s and at 95% accuracy in the absence of workload. An extended version of this paper can be found as a technical report at http://www.cs.ubc.ca/labs/spin/.

Visual information overload is a threat to the interpretation of displays presenting large data sets or complex application environments. To combat this problem, researchers have begun to explore how haptic feedback can be used as another means for information transmission. In this paper, we show that people can perceive and accurately process haptically rendered ordinal data while under cognitive workload. We evaluate three haptic models for rendering ordinal data with participants who were performing a taxing visual tracking task. The evaluation demonstrates that information rendered by these models is perceptually available even when users are visually busy. This preliminary research has promising implications for haptic augmentation of visual displays for information visualization.

Multimodal interfaces offer great potential to humanize interactions with computers by employing a multitude of perceptual channels. This paper reports on a novel multimodal interface using auditory, haptic, and visual feedback in a direct manipulation task to establish new recommendations for multimodal feedback, in particular uni-, bi-, and trimodal feedback. A close examination of combinations of uni-, bi-, and trimodal feedback is necessary to determine which enhances performance without increasing workload. Thirty-two participants were asked to complete a task consisting of a series of 'drag-and-drops' while the type of feedback was manipulated. Each participant was exposed to three unimodal feedback conditions, three bimodal feedback conditions and one trimodal feedback condition that used auditory, visual, and haptic feedback alone, and in combination. Performance under the different conditions was assessed with measures of trial completion time, target highlight time, and a self-reported workload assessment captured by the NASA Task Load Index (NASA-TLX). The findings suggest that certain types of bimodal feedback can enhance performance while lowering self-perceived mental demand.