Haptic Feedback Interface 
Motivation: To design, develop and evaluate an ergonomic haptic feedback interface which will serve both as a master to control a robot arm (and the modular laparoscopic tool) as well as provide force feedback to the user during manipulation tasks.
Design of the 7-DOF haptic feedback device:
 |
 |
| Prototype of the 7-DOF haptic device with various coordinate frames |
| Joint | Range of motion |
|---|---|
| User Interface | |
| Yaw joint | -40° to 35° |
| Prismatic joint | -76.2mm to 76.2mm |
| Pitch joint | 0° to 40° |
| Roll Joint | -75° to 35° |
| Spatial Force Feedback Mechanism | |
| Prismatic joint 1 (X axis) | 0mm to 223mm |
| Prismatic joint 2 (Y axis) | 0mm to 223mm |
| Prismatic joint 3 (Z axis) | 0mm to 183mm |
The device consists of a closed kinematic chain with two halves; a user interface and a spatial force feedback mechanism. The user interface consists of an arm rest with four degrees-of-freedom position feedback (roll, pitch, yaw, and linear motion of the arm rest) and a grasping/dissecting mechanism at the end of the arm rest . Therefore, a user could insert their hand and forearm into the user interface and use the haptic device as a master device for controlling a slave robot, while receiving force feedback. The joints of the user interface proceed from the base → yaw joint → prismatic joint → pitch joint → roll joint → end effector joint (universal joint). All of these joints, except for the prismatic joint, are equipped with encoders for tracking the position of the user interface (see Table for joint limits). The grasping/dissecting mechanism contains two thimbles for the user’s fingers (thumb and index finger, for example) that are coupled to a DC motor with an encoder. This allows the user to fully control a grasping mechanism, such as a laparoscopic tool at the end of a robotic surgical system, and also receive force feedback as measured by sensors in the laparoscopic tool. The spatial force feedback mechanism consists of a three degree-of-freedom positioning stage that attaches to the user interface at the grasping mechanism through the use of a universal joint. This mechanism was designed to provide force feedback in three directions through orthogonally-mounted linear actuators. Therefore, this force feedback mechanism can relay manipulation forces, such as the pulling or pushing of an object (e.g. soft tissue in MIS), to the user in addition to the gripping forces felt through the grasping mechanism. This mechanism was designed to apply all forces to the user at the grasping mechanism rather than through the joints of the arm rest. This enhances the transparency of the haptic device by providing feedback to the user, which is more analogous to conventional open surgery where the surgeon primarily receives feedback at the point of contact with the soft tissue and/or organs.
In the design of the spatial force feedback mechanism, we used a cable-driven transmission powered by DC motors to actuate the prismatic joints. Each of the motors uses a 6.35 mm diameter grooved pulley on its shaft with one (X and Z axes) or two (Y axis) idle pulleys at the limits of travel. We selected brushed DC motors with encoders (RE40 manufactured by Maxon Motors) for each axis. The DC motor is capable of providing up to 181 mNm of continuous torque, which equates to approximately 56N of force. However, frictional losses reduce this number to approximately 40N (as measured experimentally). The grasping/parting force feedback mechanism also uses a cable-driven transmission powered by a DC motor; however, the transmission involves two stages (see Figure below). This two stage transmission allows the placement of the motor close to the pitch axis of the user interface to reduce the moment on that axis. The DC motor has a 6.35mm diameter pulley mounted to its shaft with a steel cable that transmits the force to an intermediate pulley that is 19mm in diameter. Connected to this pulley is a 6.35mm diameter pulley that transmits the motion further to a 19mm diameter pulley, to which the thimbles are attached. This transmission represents an increase in torque of 9:1 from the motor pulley to the thimble pulley. The brushed DC motor with encoder (RE36 manufactured by Maxon Motors) is capable of producing up to 88.8mNm of continuous torque. This equates to approximately 12.5N of force at the tip of the thimbles as desired from research in the literature. Additionally, a redesign of the cable transmission from the DC motor to the thimble was recently completed to increase the theoretical force feedback in the thimbles from 12.5N to 22.7N; however, the friction in the mechanism will slightly reduce this value.
 |
| Two-stage transmission system for force-feedback during grasping tasks (using thimbles). |
Design of the Modular laparoscopic tool with tri-directional force feedback capability: The design consists of two main components; namely, the actuation mechanism and modular tool (see Figure below). The actuation mechanism contains a DC motor with gearbox and encoder that powers a linear positioning assembly. This mechanism also includes a quick-connect mechanism used to attach/detach the modular tool. The modular tool consists of a hollow tool shaft with enclosed push rod for actuation of the jaw assembly via a linkage assembly similar to those incorporated in conventional laparoscopic tools. The modular tool also utilizes a flex shaft section along the tool shaft with strain gages mounted close to the jaw assembly to measure the manipulation forces in surgery. A resistive sensor is located in one of the jaws of the tool for measurement of the forces normal to the jaw surface during grasping and palpation tasks. The prototype specifications are listed in the Table below.
| Mechanism and Motor Specifications | |||
|---|---|---|---|
| Weight | 0.98 kg | Max. Backlash at Tool Tip | 0.076 deg. |
| Length | 530 mm | Resolution at Tool Tip | 0.001 deg. |
| Max. Width | 61 mm | Max. Force at Tool Tip | 375 N (theoretical) |
| Max. Height | 68.6 mm | Max. Angular Travel of Jaws | 30 deg. (each) |
| Jaw with Sensors Specifications | |||
| Width | 5.60 mm | Height | 4.83 mm |
| Length | 31.75 mm | Gripping Surface | 1.24 sq. cm |
| Resistive Force Sensor | |||
| Range | 0-29N | ||
| Resolution | <0.10 N (as measured) | ||
| Strain Gages | |||
| Range | 100 N (theoretical) | ||
| Resolution | <0.10 N (as measured) | ||
 |
| Various components of the modular laparoscopic grasper developed with tri-directional force measurement capability. The figures show quick connect capability of the modular tool as well as the sensor locations for force measurement. |
 |
 |
| Knot tying with and without force feedback through the laparoscopic tool. With force feedback the forces were 2.31N (0.36N standard deviation) and 3.83N without force feedback (1.03N standard deviation). | |
Application notes:
1. Teleoperation platform for application in robot-assisted minimally invasive surgery:
 |
| Experimental set-up for teleoperation using the 7-DOF haptic feedback device. |
The purpose of this platform was to evaluate the addition of force feedback to a robotic surgical system. The teleoperation platform consisted of using the 7 DOF haptic device to control the Mitsubishi PA-10 robot arm and automated laparoscopic grasper attached to the robot arm’s end-effector. The user (surgeon) is able to manipulate the laparoscopic grasper through six degrees-of-freedom, as well as, control the opening/closing of the jaws of the grasper through actuation of the thimbles on the haptic device.
 |
| Teleoperation platform controller diagram. |
System Architecture and Components: The teleoperation platform which we have developed has the capability to measure tool-tissue interaction forces and reflect these forces to the surgeon. We use the haptic device as a master controller of the Mitsubishi PA-10 robot arm with attached laparoscopic grasper. This system is controlled by a computer using the QNX Real-time Operating System. Communication from the control software to the robot arm is achieved through the ARCNET motion control card to the servo driver and then the robot arm. ARCNET is a token passing LAN protocol which was developed by Datapoint Corporation. Communication from the control software to the haptic device and laparoscopic grasper is achieved through two Sensoray Model 626 data acquisition cards.
The hierarchy of steps followed in the teleoperation setup are: 1) the system obtains the positions of the joints on the haptic device from the Sensoray cards, 2) the inverse kinematic solution is calculated to determine the new position of the joints of the robot arm, 3) the control software obtains the position and velocity of the laparoscopic grasper jaws and calculates the desired position of the jaws based on the corresponding grasping joint of the haptic device using a PD control law, 4) the measurements from the force sensors on the laparoscopic grasper are obtained, filtered, and converted to a force through an offline calibration, 5) the friction torques of the four active joints of the haptic device are obtained using a “look-up” table containing friction values determined from previous experiments, and 6) the motor torques for the Mitsubishi robot arm, the haptic device, and the laparoscopic grasper are sent via the aforementioned communication methods. This control loop currently operates at 425Hz, although, improvements to increase the speed of the system are being investigated.
The two Sensoray Model 626 data acquisition cards were used to interface both the laparoscopic grasper and haptic device with the computer. Each Sensoray Model 626 card include six 24-bit counters for encoders/timers, sixteen 16-bit differential A/D inputs operating at 15 kHz with either ±5V or ±10V input range, four 14-bit D/A outputs operating at 20 kHz with ±10V output range, and 48 digital I/O channels. Overall, the haptic device contains seven encoders and four DC motors and the laparoscopic grasper contains an additional encoder and DC motor. Therefore, two Sensoray Model 626 cards were required to provide the necessary eight encoder counters and five D/A channels. Also, five A/D channels were used to obtain the force measurements recorded by the sensors on the laparoscopic grasper.
2. Simulating needle puncture through tissue on the haptic feedback interface: We used a 7DOF haptic feedback device to replay the forces encountered during experimental measurements of needle insertion and withdrawal tasks. The purpose of the simulation is to demonstrate an integration of this work with a suitable haptic feedback device for training surgeons to perform needle insertion tasks. Since the force and position information is recorded from actual needle insertion experiments, replaying this data on the haptic device helps to assess the transparency of the device (after modeling the friction in the transmission) and the realism of the needle insertion task. The figure below shows the experimental force profile and the modeled force profile for a typical needle insertion event. The force profile shows two puncture events which occur on the surface of the tissue. A puncture event comprises of initial deformation (leading to a rise in the force) followed by puncture (sudden drop in the force). The two puncture events are due to the fact that the needle tip and cannula do not have a seamless boundary. In our preliminary work, the haptic device is capable of replaying the forces for both needle insertion and withdrawal.
 |
 |
| Needle insertion simulation on the 7-DOF haptic feedback interface. | |
Relevant archival publications:
- Gregory Tholey and Jaydev P. Desai, “ A Compact and Modular Laparoscopic Grasper with Tri-directional Force Measurement Capability , Currently under review in Transactions of the ASME-Journal of Medical Devices, 2007.
- James T. Hing, Ari D. Brooks, and Jaydev P. Desai, “A Biplanar Fluoroscopic Approach for the Measurement, Modeling, and Simulation of Needle and Soft tissue Interaction”, Medical Image Analysis, pp. 62-78, Volume 11, Issue 1, February 2007.
- Gregory Tholey and Jaydev P. Desai, “A General Purpose 7 DOF Haptic Device: Applications towards Robot-Assisted Surgery”. Accepted for publication in IEEE/ASME Transactions on Mechatronics, 2006.
- Gregory Tholey, Jaydev P. Desai, and Andres E. Castellanos, “ Force Feedback plays a significant role in Minimally Invasive Surgery – Results and Analysis”, Annals of Surgery, 241(1): 102-109, 2005.
- Tie Hu, Gregory Tholey, Jaydev P. Desai, and Andres E. Castellanos, “Evaluation of a Laparoscopic Grasper with Force Feedback”, Surgical Endoscopy, 18(5): 863-867, 2004.
- Gregory Tholey, Anand Pillarisetti, and Jaydev P. Desai, “On-Site Three Dimensional Force Sensing Capability in a Laparoscopic Grasper”, Industrial Robot, 31(6): 509-518, 2004.
For further information, please contact:
Prof. Jaydev P. Desai
Director, RAMS Laboratory
Department of Mechanical Engineering
Room 0160, Building 088
Glenn L. Martin Hall
University of Maryland
College Park, MD, 20742
Email: jaydev (at) umd.edu
Phone: 301-405-4427
Fax: 301-314-9477