Force and Tactile Feedback in a Virtual Environment

Permission is hereby granted for unlimited use and distribution of these images of CGSD's TOPIT system.

Please click on the images for larger views.


A system has been constructed to provide force and tactile feedback to a user of a virtual cockpit. The user wears a head mounted display that presents stereo imagery of a cockpit interior, including the instrument panel, as well as the out- the-window scenery. A representation of the user's hand is also rendered in the scene. The user may actuate a variety of controls on the instrument panel, and can accurately feel the forces and surface textures of the controls. The objective is to provide a simulator that can be reconfigured entirely in software to represent different cockpits.

The feel of the instrument panel controls is provided by a servomechanism device that places actual physical controls in their correct positions, orientations, and configurations. A tracker and data glove continually provide the position of the user's hand and fingers to a computer. The computer extrapolates the user's hand position as the user reaches for a control. Using the extrapolated data, the computer commands the servomechanism system to place the correct type of control in the correct position to be actuated. The servo system has a "touchpanel" that contains examples of a dozen or so different types of controls, such as toggle switches, knobs, and push buttons, that are used re peatedly to represent any number of instrument panel controls. Because the visual representation of controls is accurate and because it is difficult to recognize an object by touch alone, a relatively small number of examples of controls suffices for many instruments. The touchpanel portion of the system may be interchanged for further flexibility . The touchpanel is driven by servo mechanism that place switches and rotary controls in their correct orientations to correspond to the particular control on the virtual panel, with programmable détentes and stops provided for rotary switches.

The system is called "Touched Objects Positioned In Time," (TOPIT tm). A press release provides a summary of the system and announces the granting of a U.S. patent on the system.

A key aspect of the system is building a servo system that moves fast enough to have the control in place before the user's hand reaches it. The design objectives in this regard were achieved. The switch panel moves at 100 inches per second with 4 g's of acceleration along with a positioning accuracy of about 0.01".

Another key aspect is achieving precise low-latency tracking of both the user's head and the user's hand. The tracking must be accomplished in the presence of the moving metal elements and the electric motors of the servo system; a hybrid magnetic/inertial tracker was developed to meet the requirements. The tracker uses a Kalman filter to combi ne accelerometer, angular rate sensor, and magnetic tracker data. To minimize magnetic interference, the moving mechanism is built of non-magnetic stainless steel, and the main servo motors are kept distant with the use of cable drive mechanisms. Despite all of the efforts made, the magnetic tracking never achieved the accuracy desired. We requir e about 0.1 inch accuracy, but the magnetic distortions could not be calibrated reliably to better than about 0.25 inch. Consequently, we now recommend use of a more expensive optical tracker with the system.

The system uses three computers: an SGI Onyx/RealityEngine2 that does the imagery, a Pentium-based PC that does the tracking, and a VME-based servo control system.

Software was custom written for the device. The software for running the image generator is written using Silicon Graphics' Performer as a starting point. Software in a Personal Computer runs under a real time operating system called QNX. The tracking, switch selection logic, and high level control logic is performed in a PC to lower costs and to simplify interfacing with other image generators. The databases for the cockpit and environment were developed using MultiGen. Custom servo control software resides in the dedicated controllers in the VME system.

The system provides high fidelity in reproducing force and tactile feedback for instrument and control panel environments. Unlike systems carried on the user's hand, the positioned objects provide external forces and torques. A limitation of the approach, however, is that it only lends itself to a limited class of problems, such as control pane ls, in which there are relatively few types of touched objects and the objects are small enough to be moved rapidly into position to be touched.

The prototype system is complete and a demo videotape is available. Those mature enough to view the pieces of an unfinished system may wish to see our progress photos of system when it was under construction. Remember, we warned you that this is not for the squeamish.

This project was sponsored in part by the U.S. Army Simulation, Training, and Instrumentation Command (STRICOM). The development is a Phase II Small Business Independent Research project.

The system is expected to be useful for the development of instrument panels for aircraft and for automobiles, for industrial control panel design, and for aircraft and other types of simulators. We are quoting a selling price of about US$ 300,000 for the TOPIT mechanism, servo controller, tracking computer, precision optical tracker, and instr umented glove. An image generator and head mounted display are available for approximately US $50K, depending upon image generator choice and display resolution. The active area of the TOPIT device can be varied in follow-on units without too much difficulty. Exact prices of all items depend upon details of the configuration.

Questions? E-mail Roy Latham,

Copyright CGSD Corp. 1999fmb