Step 5 Recognize Which Sensory Modes Are Important and Should Can Be Simulated

Surgeons rely on a variety of sensory feedback to execute complex hand-eye motor tasks and make important decisions. It is therefore important to determine which sensory feedback modalities are important to relay in a simulated environment. The types of feedback will vary form procedure to procedure.

In endoscopic procedures, the simulation of how objects appear and interact visually is the most dominate.

TABLE 3 ■ Levels of Performance

Novice

Competent

Proficient

Expert

Master

Source: From Ref. 27.

Another important factor to consider when addressing sensory feedback is the concept of clue conflict. Motion sickness can result when the body tries to interpret conflicting clues as they are received over multiple senses.

The sounds of the operating room can provide very important cues that the surgeon needs to respond to, such as the sounds of electrocautery, verbal cues from the patient or other operating room personnel, and the pulse oximeter.

In fact, in robotic laparoscopic applications, the only feedback to the surgeon is visual. It can be debated whether or not the addition of haptic feedback (force feedback and tactile feedback) would add a significant benefit. In contrast, the inclusion of hap-tic feedback is very relevant when simulating open procedures. If such feedback cannot be adequately represented, then a simulator may prove ineffective.

Glossary of sensory feedback in surgery simulation

Presence: The sensation of actually being "there."

■ Immersion: The state where the individual is absolutely focused on the task at hand.

■ Navigation: The ability to move through the environment.

■ Interaction: The ability to move and feel/touch objects in the environment.

The terms "presence" and "immersion" are often mistakenly interchanged, but they do have distinct meanings. For example, a child playing a video game may be immersed in the game and not realize what is going on around them. However, the child probably does not believe to be actually in the video game world. If the video game established a sense of presence, then the children would feel that they themselves are jumping over mushrooms and eating coins. As demonstrated, any video game (simulator) can create immersion, but as far as surgical simulation is concerned, a simulator must represent the multimodal senses of visual, tactile, auditory, and olfactory realism to enable the feeling of presence (28).

With new-generation tracking devices, navigation through virtual environments is possible as the trainee physically performs the hand/eye motor tasks used in real surgical procedures. However, in many cases, common tracking electronics may not fit within the construct of a variety of surgical instrument or there accuracy might be effected my metal, electric, and magnetic fields. This can impose limitations on a number of procedures that can be completely simulated to endoscopic procedures, laparo-scopic procedures, procedures using Seldinger technique, and simple open procedures such as intubation, incisions, and simple suturing.

Another important factor to consider when addressing sensory feedback is the concept of clue conflict. Motion sickness can result when the body tries to interpret conflicting clues as they are received over multiple senses.

For example, when visual cues of physical orientation do not match vestibular feedback, an individual is likely to become sick. In simulation, this can occur when the frame refresh rate is too slow. The trainee can also become disoriented when visual cues do not match physical motion, such as when a trainee attempts to move an object in one plane but sees the object moving in a different plane. When one is first learning endoscopy, this may occur during actual procedures due to the fulcrum effect (Fig. 1), but with repeated patterned sensory stimuli (training), these senses adapt and this conflict resolves.

Auditory realism is important in surgical virtual reality applications.

The sounds of the operating room can provide very important cues that the surgeon needs to respond to, such as the sounds of electrocautery, verbal cues from the patient or other operating room personnel, and the pulse oximeter.

FIGURE 1 ■ Laparoscopic simulators.

FIGURE 1 ■ Laparoscopic simulators.

While such sounds are usually quite simple to represent in a virtual environment, they can add a significant degree of realism to a simulator. One concept, or technique, which allows for enhanced auditory realism, is known as convolving. Convolving involves filters and intertwines signals and renders them in three-dimensional space. These are used in virtual reality applications to recreate sounds that give directional cues. Such directional feedback can also prove quite valuable in simulation.

Haptics is the study of the psychomotor experience of tactile proprioception and force feedback.

Force feedback is the counterbalanced sense of a magnitude of force perceived when a force is applied to an object and is directional in nature. This needs to be discriminated from tactile feedback, which is nondirectional sensation. Some examples of tactile feedback are vibration, texture, and temperature. Such information is transmitted to the user through an "actuator" via mechanical and/or electric means.

Relatively speaking, simulation of haptics is very computationally expensive. When considering integrating haptics into a simulator, one must pay special attention to the magnitude and number of forces that must be applied. Basic physical principals dictate that all forces on an object must sum to zero or the object will be set in motion. Therefore, if a force feedback device is resisting the push from a trainee, the force feedback device must also push on a grounded object (such as the floor or a secure desk) in the exactly opposite direction so that all force vectors sum to zero. Otherwise, the force feedback device might fly right out of the simulator. Representing one point of force feedback is feasible, such as pushing/pulling on one point of a surgical instrument. A single robotic arm might be connected between the floor and the instrument to effectively render one vector of force by controlling motor torques in the joints of the robotic arm. However, force feedback becomes much more complicated as more points of interaction are added. Imagine what would be necessary to accurately render the appropriate sense of touch over the entire hand as might be required for some open surgical procedure. This would necessitate several different vectors of force applied over a number of contact points on the hand. It simply is not practical to attach a robotic arm to each of these points of force feedback.

Haptic fidelity refers to the rate at which haptic signals are updated. Different haptic phenomena require different degrees of fidelity to convince the trainee that a sensation is realistic. In general, directional force feedback requires less fidelity than tactile force feedback. The deformability of objects within the environment is also a factor to consider. The more deformable an object, the less fidelity is required. Accurate simulation of collision with rigid objects requires higher fidelity, as the degrees of change in force can be dramatic for small displacement. In other words, small, quick changes in motion can generate large forces, which are difficult to keep up with. While it may be possible to simulate contact of soft deformable objects at only 300 updates per second, rigid object contact may require well over 1000 updates for accurate feedback. In comparison, human vision only requires about 30 updates in order for animation to be perceived as smooth.

Another factor to consider is the complexity of the object surface being manipulated. If the instrument collides with more than one structure in a short amount of time, sampling update rate again needs to compensate.

Fidelity also depends upon the speed of movement of an object through the environment. Obviously, a fast moving object is more likely to collide with more surfaces and require higher fidelity than a slow moving object.

Force feedback can occur over multiple degrees of freedom. There may be directional forces along the x, y, and z coordinate planes. There may be torques resulting from rotations. Grip may need to be accommodated. The more the degrees of force feedback, the more difficult it is to accommodate a complex task in simulation.

Many applications do not require force feedback such as those designed to just assess cognitive decision-making skills. Others rely on physical models to provide force feed back such as the PercMentor (Simbionix). The UW transurethral resection of the prostate simulator utilizes a tension-based system,a while many of the laparoscopic force feedback device uses a linkage-based device such as the Laparoscopic Surgical Workstation by Immersion.

Open procedures require high-fidelity force feedback and tactile feedback. Unlike laparoscopic procedures, open surgery tools do not work through a fulcrum that diminishes the forces as well as tactile feedback.

Haptics is the study of the psychomotor experience of tactile proprioception and force feedback.

Haptic fidelity refers to the rate at which haptic signals are updated.

Fidelity also depends upon the speed of movement of an object through the environment.

Open procedures require high-fidelity force feedback and tactile feedback. Unlike laparoscopic procedures, open surgery tools do not work through a fulcrum that diminishes the forces as well as tactile feedback.

aMantis, Mimic Technologies, Seattle, WA.

The use of sense of smell is probably underappreciated during surgical procedures.

Augmented reality is the concept of superimposing transparent (not opaque) virtual images in real time over the real world as a frame of reference. Such a concept has already been demonstrated in an application that highlights the location of nerves during robotic radical prostatectomy.

Latency is a measure of the time between when a person moves a tracking device and when the computer displays the result of that tracking on the video screen. As latency exceeds 1/10th of a second, most people will become distracted by the disconnection between their actual movement and what they see moving graphically.

The use of sense of smell is probably underappreciated during surgical procedures.

Digiscents, Inc. is a company that has manufactured the ability to reproduce thousands of every day scents with a small cartridge that contains 128 primary odors. The combinations of these primary odors claim to effectively simulate virtually every smell. Some potential applications could include the smell associated with bowel injury and excessive cautery.

It is not always possible to represent all sources of sensory feedback in a virtual environment. In some cases, a possible alternative is cross-sensory substitution. In this case, one sensory stimulus is substituted for another. For example, speech might be converted into a visual display of text that can be read. It is important to be careful when employing cross-sensory substitution because the more you divert from standard sensory feedback, the greater the potential that simulation training will not carry over to the operating room.

In many applications, it is advantageous to incorporate additional sensory feedback that may not be found in the real procedure. Extra visual, audio, or haptics cues may help a beginner to initially learn a particularly difficult task. As the trainee becomes more comfortable with the task, this extra feedback is removed and the trainee is forced to rely on feedback that is actually found in the operating room. Sometime in the future, it may even be possible to add additional feedback to real procedures.

Augmented reality is the concept of superimposing transparent (not opaque) virtual images in real time over the real world as a frame of reference. Such a concept has already been demonstrated in an application that highlights the location of nerves during robotic radical prostatectomy.

0 0

Post a comment