Augmented-reality glasses to help surgeons remove tumours

Scientists have developed augmented reality (AR) glasses that superimpose virtual images on the body and can assist surgeons in removing tumours.

Malignant tumours often form metastases that spread to other parts of the body via the lymphatic system. High surgical skills are required to identify the precise location of the affected lymph nodes, enabling them to be completely removed.

The 3D-ARILE system developed by Fraunhofer Institute for Computer Graphics Research IGD in Germany superimposes a virtual image of the exact position of the lymph nodes using data glasses.

Malignant melanoma, or black skin cancer, is the diagnosis that most people fear. Cancer cells can, for example, be transported by the lymph fluid to the lymph nodes.

This leads to the growth of secondary tumours, or metastases. The first lymph nodes to be affected are referred to as sentinel lymph nodes.

Despite advances in medical science, it is still difficult for doctors to determine the precise anatomical location of sentinel lymph nodes during surgery and to check that the affected lymph nodes have been completely removed.

3D-ARILE provides doctors with a navigation aid for lymph node removal. The novel augmented reality system, based on the use of data glasses, helps surgeons locate lymph nodes with the aid of virtual markers.

What makes these AR glasses special is that they work in combination with powerful medical navigation software, a stereoscopic, near infrared (NIR) camera system, and indocyanine green (ICG) fluorescent dye.

"To make the affected lymph nodes visible, the fluorescent dye is injected into the patient in the direct vicinity of the primary tumour," said Stefan Wesarg, research scientist at Fraunhofer Institute.

"It then spreads along the lymphatic pathways and collects in the sentinel lymph nodes," said Wesarg.

The dye fluoresces when exposed to infrared light, in this case generated by infrared LEDs. NIR cameras capture the fluorescence and produce a 3D reconstruction of the affected lymph node.

This virtual image showing their exact position is superimposed in real time and can be viewed by the surgeon through the data glasses.

"In our case, the malignant tissue is marked in green. In this way, the surgeon can verify that every last trace has been removed," said Wesarg.