2D/3D Image Fusion for Image Guided Intervention

The main goal of the project is to propose a complete methodological approach for 2D/3D image fusion, based on 2D/3D registration. We assume two types of 2D data, fluoroscopy and USG data. The USG data represents a more challenging scenario due to their nature. As 3D data, the CT scans are to be used. Based on this we focus mainly on the hip anatomical area and thoracic area.

The scientific objectives of the project:

1. Create visualization tools for aligning and registering 2D and 3D images, which is necessary for visual validation during development. We will utilize existing tools to adapt them to our needs and implement features for manual adjustments and initial alignment checks.
2. Development of Digitally Reconstructed Radiographs - The goal is to produce DRRs that closely mimic real X-ray images, ensuring they are optimally aligned for registration with 2D imaging modalities like USG and Fluoroscopy.
3. Intensity-based 2D/3D registration - we aim to align 2D medical images, such as USG or Fluoroscopy, with 3D CT scans using an innovative intensity loss function coupled with heuristic algorithms.
4. Segmentation mask prediction and landmarks detection - we aim to develop robust systems for segmentation and landmark detection, which are critical components for feature-based registration in the project.
5. Feature-based 2D/3D registration - the goal is to create a robust registration framework that utilizes anatomical features extracted from both 2D and 3D images. These features serve as reference points for aligning the 2D and 3D datasets. The challenge lies in accurately matching these features between the two types of images, which often differ significantly in terms of perspective, scale, and detail.
6. Improving computational efficiency - this involves the optimization of the registration process for speed and accuracy. This involves fine-tuning the feature extraction and matching algorithms to improve performance, especially considering the computational constraints and the need for near realtime processing in clinical settings.