XR in healthcare

The modern world is full of high technologies affecting all spheres of human life: from school to complex robotic surgeries. We take our high-tech world for granted, never thinking about how many advancements appeared last century. And most of all, it concerns the sphere of health and medicine. People died from regular measles or infections that got into the body during surgeries just a hundred years ago. Today we can use artificial intelligence or XR medicine to solve complex medical issues.

In this article, we will talk about augmented reality and telesurgery as part of XR medicine.

But let’s go back to the beginning and consider how these innovations are related and what opportunities they open up for the world community.

The history of medicine dates back thousands of years, but only in the late 19th – early 20th centuries were several vital discoveries (that brought modern science to a new level). For a clear picture, let’s look at the most important recent achievements.

Let’s start with one of the most critical factors. Thanks to modern advances, human average life expectancy has increased from 50 years to 75 compared to the beginning of the 20th century. Today, it is 83 years.

 

The first important reason was a significant decrease in total and child mortality, especially mortality from infectious diseases at the beginning of the century. And the following milestones are the basis for that:

– the introduction of general anaesthesia and vaccination, the discovery of blood groups (early 20th century);

– the invention of the electrocardiograph (1900);

– the discovery of penicillin in the late 1920s and the first clinical trials of insulin injections (1922).

In the first half of the century, the most crucial achievement in surgery in the first plastic surgery in England. Doctors helped the soldiers who suffered in the battles.

All this made it possible to defeat mortality from many previously incurable diseases and thereby increase life expectancy.

Other important milestones of the second half of the 20th century, which became the start for the medical practice of a higher level and such modern areas as genetic engineering, plastic surgery, and innovative medicine:

– 1953 – the discovery of DNA structures and stem cells. The beginning of gene therapy, metabolomics, metagenomics.

– 1972 – the invention and widespread of CT scanners, followed by the invention of MRI and other imaging methods. Without them, we could not have reached a high level of advanced technologies today.

However, the most critical advances in medicine over the past century are related more to the development of surgery. They show how much medicine has changed and how much its evolution has changed. What was utterly impossible several decades ago became an everyday reality.

1967 – the first heart transplantation;

1978 – the IVF method is invented. Louise Brown was born as the first test-tube baby;

1988 – the invention of laparoscopy. For the first time, that type of surgery was conducted in France in 1985. However, the general introduction to mass use took place in the United States in 1988.

After, an incredible leap in the development of world medicine takes place in terms of speed. Modern surgery (including modern technologies) started to develop rapidly. So, the era of minimally invasive surgery was born. Surgeons began to use alternative approaches to treat the internal organs. In addition to laparoscopic surgery, video endoscopic intervention has also appeared.

Methods for identifying tumour stages, modern diagnostic techniques, and performing regenerative operations have fundamentally changed current approaches to surgery. At the same time, traumatology, prosthetics, and transplantation reached a new level, but they were still ineffective conservative treatment.

Today is a new milestone in the history of medicine, both practical and scientific. Medicine is developing at a tremendous speed, without requiring numerous human victims. Today, “experimental” patients are replaced by simulation mannequins, hologram patients, or humanoid robots that can talk and simulate real human feelings (Dentaroid or Simroid, Nissin).

To get an appointment with a doctor, you no longer need to stand in long lines at clinics and hospitals. There is no need to wait for weeks or even months for recording to the doctors. For this, we have remote consultations – telemedicine. You just need to press a button on the computer, turn on the camera – and the doctor is already “at your home” 24/7. Not, of course, but virtually.

Moreover, we don’t even need to spend hours looking for the best specialist. Artificial intelligence performs that for us. It selects a suitable doctor by using a specially designed algorithm.

The 21^st century today is the era of Augmented Reality or XR medicine.

The term telemedicine was presented to publicity several years ago, and we all know what it is. But the XR medicine term, which includes virtual, augmented, and augmented reality, is a concept that is still unfamiliar to many people.

Let’s try to figure out what augmented reality is and how it relates to XR medicine. So, we will start with the concept of virtual reality as the first component of XR medicine.

Virtual reality (VR) is the immersion of a user in a wholly digital world. In medicine, VR is used for training, for example, surgery. The student practices surgical manipulations on a virtual patient plunging into the world of virtual reality.  This way, skills are trained with minimal risk for an actual patient (OssoVR, Oculus, Fundamental VR).

Augmented Reality (AR) is the imposition of digital information on the real world, which means the camera becomes the “eyes”. It recognizes objects in the real world and transfers them to the virtual environment. Thus, one layer of reality is positioned on another, creating the augmented reality. In medicine, the technology is used in teaching (allowing you to view organs or bones in volume) and experimental surgery in planning and conducting surgery and postoperative control (Augmedics, Imperial College London, Medivis).

Mixed Reality (MR) is a fusion of the natural and digital world (VR/AR). Information about patients (CT, MRI, ultrasound, X-ray images) is converted into a hologram using software and the work of a specialist in 3D models, after which the hologram is transferred to glasses and placed in the real world. Thus, it is the combination of holograms and the real world. For example, a full-size hologram of an object is superimposed on a patient in the right place. The doctor on an actual patient sees the damage to the organ or bone “from the inside”, but without surgical intervention. So, it is possible to simulate surgeries in teaching medical students and planning and conducting surgeries for postoperative control (Apoqlar, Dicom director). It can also be used for telemedicine.

Extended Reality (XR) combines all types of reality (VR/AR/MR). It is a new type of augmented reality medical technology that appeared about 2.5 years ago. The real world is expanded with holograms that look three-dimensional and are perceived through special glasses – such as Microsoft’s HoloLens, Magic Leap, and others. The user interacts with holograms that can be placed anywhere: in the air, on a patient, or training mannequin. At the same time, the user can perform various manipulations with the hologram: enlarge, rotate, cut, or work with only a part of it. XR technologies can be used in all areas of medicine, from training and diagnostics to conducting complex surgical operations involving the world’s famous surgeons. In real-time, the consultant surgeon using special glasses can suggest or correct the actions of the colleagues regardless of distance (XR Doctor).

Computer vision is used to recognize and match real-world objects and holograms through augmented reality glasses. In healthcare and medical education, it is used in preoperative planning to draw various lines superimposed on holograms for drawing lines of osteotomies, etc.

Haptics creates a sense of touch. It is used for objects in the virtual world and augmented reality. Haptic technology allows you to feel “touch” the hologram according to its programmed properties: its thickness, curvature, density, bone porosity based on the CT scan of which the hologram was made. When simulating surgical manipulations using instruments, you can practice tactile skills—for example, muscle memory through the formation of pressing force, etc. Special haptic sensors and gloves can act as haptic devices.

Biometrics in medicine is used to analyze a person’s physical and behavioural reactions to actions performed based on a set of indicators such as heart rate, etc. When simulating a surgery using holograms and haptic technologies, it is possible to assess the physiological state during the reproduction of surgical procedures. This approach will mimic the copy of planned actions during the surgery, emergencies and check the reaction and degree of readiness of the student, resident, and medical practitioner.

AI and neural networks are one of the main trends in the healthcare world. It is used in medicine in various ways. In this case, the input neurons receive data, process them on the inner layer of the neural network, and the results are obtained at the output. For example, a patient makes a request “high fever”, “chills”, “sore throat”, and the neural network analyzes millions of medical records of other people. Based on their diagnoses, it gives a suitable disease for the person who made the request. To date, many medical technologies have been developed based on the use of neural networks, which are used in clinics around the world.

Examples of high-tech technologies in medicine

XR technology opens new possibilities for diagnostics, surgery, dentistry, patient care, and more. So all medical data can become available in the form of three-dimensional holograms to doctors, hospital staff, and even patients. We can visualize internal organs, bones, teeth, medical instruments in the form of three-dimensional holograms and use them to practice surgical and dental procedures, diagnose and study pathologies, and even help with actual surgery.

Holograms in Surgery

Doctors began to use virtual reality in medicine about ten years ago. But augmented reality is being introduced into various medical approaches right now. Augmented reality technology allows the doctor to look inside “through the patient’s body”, converting the patient’s data into a hologram organ with the correct position on the patient’s body. Drawing and measuring tools for working with holograms using Haptic technology and unique algorithms let you know, touch, and feel the shape and thickness of an organ. The 3D printing approach allows holograms to be transferred into a real surgical phantom.

Telesurgery: the future is here

XR medicine raises telemedicine to a new level and makes telesurgery possible. A doctor in another hospital, city or even country can appear in the operating room and even jointly conduct the surgery using augmented reality glasses. On the contrary, another option is also possible: holograms of damaged organs or bones are projected in a room with another doctor. So, someone can consult him using the glasses too.

What are the main advantages of XR medicine?

Visualizing medical images, disease progression, and medical data is a significant advantage that provides the highest accuracy, faster diagnosis, more accurate localization, and improved patient care.

The most crucial advantage of XR medicine is conducting complex surgeries remotely, using several doctors. And also, create a virtual workspace in the operating room or classroom, even if you are thousands of kilometres away from each other.

XR Doctor – a doctor of the new generation

So, XR medicine introduces a new type of doctor into our environment. XR Doctor is the new generation doctor obtaining the highest medical level standards and working with XR equipment.

The future is already here. We develop it here and today. We have learned to communicate contactless in the virtual space. With the help of computer vision, we can diagnose diseases and prevent their further development. At thousands of kilometres, we can study as if we were in the same classroom. We can conduct surgeries simultaneously while being on different continents and in different time zones.

We have achieved a lot, but a lot is still ahead.