The field of thoracic surgery is increasingly supported by virtual, augmented, and mixed reality technologies that enhance both clinical workflows and surgical education. Spatial imaging offers significant visual advancements, altering the way clinicians interact with medical data. The resulting enhanced anatomical understanding supports informed decision-making, strengthens surgical preparation, and expands opportunities for high-fidelity training.

Immersive digital twins

Zalepugas D, Buermann J, Senkel S, Schmidt N, Ziegler AM, Kurz R, Schmidt J, Feodorovici P, Arensmeyer J (2025) The potential of Real-Time volume-rendered 3D Imaging in immersive virtual reality (VR) for surgical planning in infants with congenital thoracic malformation (CTM). Computational and Structural Biotechnology Journal.

Congenital thoracic malformations are a group of rare, structural deformities of the lungs and airways that often require early surgical intervention to prevent the development of respiratory complications. To determine optimal surgical strategies, infants’ CT scans were first assessed using conventional 2D imaging, followed by review of the same anatomy converted into interactive 3D models in Medicalholodeck. The VR visualization improved overall diagnostic accuracy, particularly for surgeons with limited prior experience, potentially leading to changes in the planned surgical approach.

(A) Actual surgical approach. The incision line and the patient’s posture were decided on the basis of the surgeon’s approach angle, as observed with virtual reality. (B) Discrepancy between proximal diameter and distal diameter.

VR technology was also applied in the management of a patient with an expanding thoracic aortic aneurysm in the distal arch. The 3D reconstruction of the patient’s CTA data enabled precise evaluation of the aneurysm’s relationship to surrounding structures, including areas of severe adhesion. This detailed visualization supported the development of a carefully tailored operative plan. The aneurysm was successfully replaced with a prosthetic graft, and the patient experienced an uncomplicated postoperative recovery.

Holographic surgical planning

Medicalholodeck integrates mixed and augmented reality capabilities, allowing surgeons to project interactive 3D models as holographic overlays onto patients. This approach was applied in three thoracic oncology cases involving tumors of the chest wall and ribs. Real-time holograms aligned with the patients’ anatomy enabled the surgical team to adjust tissue renderings and assess spatial relationships between tumors and adjacent structures. This highlights mixed reality’s potential to improve anatomical orientation and support more precise preoperative planning in thoracic surgery.

AI Pulmonary Map

Applied key points using bifurcation landmarks on the preoperative model (A) and intraoperative image (B).

The thoracic surgery field benefits from AI-segmentation – the process which detects and separates anatomical parts in much shorter time compared to manual tracing methods. When combined with augmented reality, these segmented 3D models give surgeons the ability to visualize both exposed and concealed anatomy directly within the operative view. In a thoracoscopic segmentectomy, this integration enabled surgeons to superimpose AI-generated pulmonary models onto the live thoracoscopic feed, revealing the location of veins, bronchi, and tumor tissue even when hidden by deflated lung segments. This enhanced anatomical orientation supported safer dissection and more precise execution.

Case-based teaching

A patient-centered approach is fundamental in surgical practice, as every case presents distinct anatomical and clinical challenges. For this reason, the case-based learning technique has been prominent in medical education. Virtual reality technology builds on this method by offering immersive learning environments.

Presentation of a serial rib fracture with fragment dislocation (left) and consecutive haemato-pneumothorax (right).

Graduate medical students participated in a VR curriculum that allowed them to explore 3D patient data across thoracic surgical traumatology, thoracic infectiology, and thoracic surgical oncology. Engaging with the models in Medicalholodeck strengthened their understanding of complex spatial relationships and deepened their insight into relevant surgical pathologies.

Spatial imaging is advancing thoracic surgery by enabling detailed digital twin reconstructions that strengthen diagnostic assessment and preoperative planning. Building on this, AR and MR introduce holographic overlays, improving spatial orientation at the bedside and in the operating room. AI-driven segmentation further enhances these platforms by rapidly generating precise anatomical models that clarify critical structures. Together, these technologies are shaping a more accurate, informed, and efficient approach to thoracic surgical care.

How to get started

Medicalholodeck integrates with secure hospital systems, providing PACS access, HIPAA-compliant data handling, and full patient data security. It runs on stereoscopic 3D displays, VR headsets, mobile devices, and standard Windows systems, enabling flexible use in hospitals, classrooms, and training centers.

Specialized features for surgical planning are exclusive to Medical Imaging XR PRO. Currently, Medicalholodeck is available only for educational use. The platform is undergoing FDA and CE certification, and we expect Medical Imaging XR PRO to be available soon in the U.S. and EU markets.

For updates, regulatory news, availability, or questions contact info@medicalholodeck.com.