Advanced Brain Tumor Segmentation : A Deep Learning-Based Approach for Augmented Reality Visualization and Interaction in Medical Imaging

Abstract

Brain cancer is a critical and life-threatening condition that requires precise diagnosis and treatment planning. Accurate identification and localization of brain tumors are essential for effective intervention. This project explores the integration of artificial intelligence (AI) for brain MRI tumor segmentation with augmented reality (AR) to provide enhanced 3D visualization and interaction. The primary goal is to develop a robust system that processes MRI images, generates accurate 3D segmentation masks using AI, and offers an interactive, semi-immersive AR experience for medical professionals. To achieve this, we trained two state-of-the-art deep learning networks: a 3D U-Net with residual blocks and a 3D Attention U-Net. These models were trained on the BraTS2021 dataset. Our rigorous training and validation process resulted in Dice scores of 0.887 on the validation set and 0.891 on the test set. The AR component of our system enhances the utility of these AI-generated segmentation masks by allowing users to visualize and interact with the 3D brain and tumor models in a semi-immersive environment. This functionality enables neurologists and surgeons to better understand tumor morphology and localization, thereby improving surgical planning and intervention strategies.