Artificial Intelligence, Radiomics, and Deep Learning in Neuro-Oncology

Norbert Galldiks, Gelareh Zadeh and Philipp Lohmann

Besides the histomolecular evaluation of tissue samples obtained from resection or biopsy, neuroimaging forms the basis for the diagnosis of brain cancer. Contrast-enhanced MRI is the method of choice for brain tumor diagnostics, treatment planning, and follow-up. Currently, advanced MRI techniques as well as amino acid PET are increasingly applied, generating a large variety of imaging parameters for brain tumor diagnostics. This is also driven by the increasing availability of hybrid PET/CT and PET/MRI scanners. Evaluation of the complex, multiparametric imaging data can be achieved by methods from the emerging field of artificial intelligence, potentially supporting physicians in clinical routine. For example, time-consuming steps such as manual detection and segmentation of lesions can be performed fully automatically. Since computer-aided image analysis is independent of the experience level of the evaluating physician, the results are more standardized and improve the inter-institutional comparability.

Moreover, artificial intelligence offers the potential to extract yet undiscovered features from routinely acquired neuroimages. These features quantify image characteristics that cannot be captured visually. Especially in combination with clinical parameters or molecular markers, mathematical models can be developed for an improved assessment of prognosis and treatment response, or the non-invasive prediction of molecular markers. The development of these mathematical models based on quantitative features computed from medical images is called radiomics.1–4

The present special issue “Artificial Intelligence, Radiomics, and Deep Learning in Neuro-Oncology” provides a comprehensive overview for the readers of Neuro-Oncology Advances on various methods and several clinical applications in this rapidly evolving field.

In the first article of this special issue, Beig and colleagues provided a concise introduction about feature-based radiomics in Neuro-Oncology including a profound discussion on implications and challenges.5 Feature-based radiomics uses a set of mathematically predefined features for model generation related to the clinical question, e.g., the identification of a subgroup of brain tumor patients with favorable outcome. Another important clinical application of feature-based radiomics is “Radiogenomics.” It allows statistical correlations of radiomic features with genetic aberrations obtained from mutational analyses or next-generation sequencing data and has the potential to provide “virtual biopsy” maps.

Predominantly MRI studies have been used to demonstrate the potential of radiomics in patients with brain tumors. Nevertheless, the growing acceptance and application of PET imaging for brain cancer diagnostics has triggered further efforts to investigate multimodal radiomics including MRI and PET. Recent developments in terms of feature-based PET/MRI radiomics are summarized by Lohmann et al.6 Several studies demonstrated that the combination of amino acid PET and MRI radiomics encodes more diagnostic information than either modality alone, resulting in increased diagnostic performance of combined PET/MRI radiomics models. For example, combined PET/MRI radiomics allow the non-invasive prediction of molecular markers such as the O6-methylguanine-DNA-methyltransferase promoter methylation status or the alpha thalassemia/mental retardation syndrome X-linked (ATRX) genotype.

This review of the current status of feature-based PET/MRI radiomics is supplemented by a study of Rathore and colleagues. Using multicenter neuroimaging data, the authors successfully identified biomarkers based on MRI radiomics for an isocitrate dehydrogenase mutation or the epidermal growth factor receptor variant III in glioma patients. Although established and newer tumor tissue biomarkers are usually molecularly characterized, this study underlines the potential of radiomics for the identification of not-yet-defined diagnostic biomarkers from imaging data alone.

Deep learning-based radiomics offers another innovative approach for model generation. Wiestler and Menze provided a comprehensive introduction to deep learning for medical image analysis.8 In contrast to feature-based radiomics, deep learning-based radiomics uses mainly artificial neural networks that generate and extract high-dimensional features that are not predefined. Hence, the workflow of deep learning-based radiomics is fundamentally different from feature-based radiomics. Besides other clinical applications, the authors highlighted particularly the great clinical value of deep learning for the automatic lesion detection and segmentation of brain tumors and discussed challenges and limitations.

Bangalore Yogananda and colleagues demonstrated the successful implementation of deep learning for the prediction of molecular markers.9 Here, a deep learning network was developed for a simultaneous tumor segmentation and classification of the 1p/19q co-deletion status in glioma patients.

Despite promising results derived from radiomics and deep learning in patients with brain tumors, most of the developed radiomics models are not yet implemented in clinical routine. One important reason is currently the premature level of standardization of methodology. This hampers the reproducibility of results and prevents the application of developed radiomics models to different institutions and clinical settings. In the last article of this special issue, Li and Huang reviewed the ongoing efforts and discussed challenges of standardization of imaging and artificial intelligence methods in neuro-oncology.10

Artificial intelligence, radiomics, and deep learning offer great opportunities towards an improved diagnosis of brain cancer, especially in conjunction with clinical parameters. In light of the technical developments in neuroimaging, the importance of these innovative techniques in the field of neuro-oncology will increase in the future.

References

1. Lambin P, Rios-Velazquez E, Leijenaar R, et al.  Radiomics: extracting more information from medical images using advanced feature analysisEur J cancer. 2012;48(4):441–446. [PMC free article] [PubMed[]
2. Lambin P, Leijenaar RTH, Deist TM, et al.  Radiomics: the bridge between medical imaging and personalized medicineNat rev clin oncol. 2017;14(12):749–762. [PubMed[]
3. Gillies RJ, Kinahan PE, Hricak H. Radiomics: images are more than pictures, they are dataRadiology. 2016;278(2):563–577. [PMC free article] [PubMed[]
4. Aerts HJ, Velazquez ER, Leijenaar RT, et al.  Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approachNat commun. 2014;5:4006. [PMC free article] [PubMed[]
5. Beig N, Bera K, Tiwari P. Introduction to radiomics and radiogenomics in neuro-oncology: implications and challengesNeuro Oncol Adv. 2020;2(S4): iv3–iv14. []
6. Lohmann P, Meißner AK, Kocher M, et al.  Feature-based PET/MRI radiomics in patients with brain tumorsNeuro Oncol Adv. 2020;2(S4): iv15–iv21. []
7. Rathore S, Mohan S, Bakas S, et al.  Multi-institutional non-invasive in vivo characterization of IDH, 1p/19q, and EGFRvIII in glioma using Neuro-Cancer Imaging Phenomics Toolkit (neuro-CaPTk)Neuro Oncol Adv. 2020;2(S4): iv22–iv34. []
8. Wiestler B, Menze B. Deep learning for medical image analysis: A brief introductionNeuro Oncol Adv. 2020;2(S4):iv35–iv41. []
9. Bangalore Yogananda CG, Shah BR, Yu FF, et al.  A novel fully automated MRI-based deep learning method for classification of 1p/19q co-deletion status in brain gliomasNeuro Oncol Adv. 2020;2(S4):iv42–iv48. [PMC free article] [PubMed[]
10. Li XT, Huang RY. Standardization of imaging methods for machine learning in neuro-oncologyNeuro Oncol Adv. 2020;2(S4):iv49–iv55. []

Artificial Intelligence, Radiomics, and Deep Learning in Neuro-Oncology

Norbert Galldiks, Gelareh Zadeh and Philipp Lohmann

Besides the histomolecular evaluation of tissue samples obtained from resection or biopsy, neuroimaging forms the basis for the diagnosis of brain cancer. Contrast-enhanced MRI is the method of choice for brain tumor diagnostics, treatment planning, and follow-up. Currently, advanced MRI techniques as well as amino acid PET are increasingly applied, generating a large variety of imaging parameters for brain tumor diagnostics. This is also driven by the increasing availability of hybrid PET/CT and PET/MRI scanners. Evaluation of the complex, multiparametric imaging data can be achieved by methods from the emerging field of artificial intelligence, potentially supporting physicians in clinical routine. For example, time-consuming steps such as manual detection and segmentation of lesions can be performed fully automatically. Since computer-aided image analysis is independent of the experience level of the evaluating physician, the results are more standardized and improve the inter-institutional comparability.

Moreover, artificial intelligence offers the potential to extract yet undiscovered features from routinely acquired neuroimages. These features quantify image characteristics that cannot be captured visually. Especially in combination with clinical parameters or molecular markers, mathematical models can be developed for an improved assessment of prognosis and treatment response, or the non-invasive prediction of molecular markers. The development of these mathematical models based on quantitative features computed from medical images is called radiomics.1–4

The present special issue “Artificial Intelligence, Radiomics, and Deep Learning in Neuro-Oncology” provides a comprehensive overview for the readers of Neuro-Oncology Advances on various methods and several clinical applications in this rapidly evolving field.

In the first article of this special issue, Beig and colleagues provided a concise introduction about feature-based radiomics in Neuro-Oncology including a profound discussion on implications and challenges.5 Feature-based radiomics uses a set of mathematically predefined features for model generation related to the clinical question, e.g., the identification of a subgroup of brain tumor patients with favorable outcome. Another important clinical application of feature-based radiomics is “Radiogenomics.” It allows statistical correlations of radiomic features with genetic aberrations obtained from mutational analyses or next-generation sequencing data and has the potential to provide “virtual biopsy” maps.

Predominantly MRI studies have been used to demonstrate the potential of radiomics in patients with brain tumors. Nevertheless, the growing acceptance and application of PET imaging for brain cancer diagnostics has triggered further efforts to investigate multimodal radiomics including MRI and PET. Recent developments in terms of feature-based PET/MRI radiomics are summarized by Lohmann et al.6 Several studies demonstrated that the combination of amino acid PET and MRI radiomics encodes more diagnostic information than either modality alone, resulting in increased diagnostic performance of combined PET/MRI radiomics models. For example, combined PET/MRI radiomics allow the non-invasive prediction of molecular markers such as the O6-methylguanine-DNA-methyltransferase promoter methylation status or the alpha thalassemia/mental retardation syndrome X-linked (ATRX) genotype.

This review of the current status of feature-based PET/MRI radiomics is supplemented by a study of Rathore and colleagues. Using multicenter neuroimaging data, the authors successfully identified biomarkers based on MRI radiomics for an isocitrate dehydrogenase mutation or the epidermal growth factor receptor variant III in glioma patients. Although established and newer tumor tissue biomarkers are usually molecularly characterized, this study underlines the potential of radiomics for the identification of not-yet-defined diagnostic biomarkers from imaging data alone.

Deep learning-based radiomics offers another innovative approach for model generation. Wiestler and Menze provided a comprehensive introduction to deep learning for medical image analysis.8 In contrast to feature-based radiomics, deep learning-based radiomics uses mainly artificial neural networks that generate and extract high-dimensional features that are not predefined. Hence, the workflow of deep learning-based radiomics is fundamentally different from feature-based radiomics. Besides other clinical applications, the authors highlighted particularly the great clinical value of deep learning for the automatic lesion detection and segmentation of brain tumors and discussed challenges and limitations.

Bangalore Yogananda and colleagues demonstrated the successful implementation of deep learning for the prediction of molecular markers.9 Here, a deep learning network was developed for a simultaneous tumor segmentation and classification of the 1p/19q co-deletion status in glioma patients.

Despite promising results derived from radiomics and deep learning in patients with brain tumors, most of the developed radiomics models are not yet implemented in clinical routine. One important reason is currently the premature level of standardization of methodology. This hampers the reproducibility of results and prevents the application of developed radiomics models to different institutions and clinical settings. In the last article of this special issue, Li and Huang reviewed the ongoing efforts and discussed challenges of standardization of imaging and artificial intelligence methods in neuro-oncology.10

Artificial intelligence, radiomics, and deep learning offer great opportunities towards an improved diagnosis of brain cancer, especially in conjunction with clinical parameters. In light of the technical developments in neuroimaging, the importance of these innovative techniques in the field of neuro-oncology will increase in the future.

References

1. Lambin P, Rios-Velazquez E, Leijenaar R, et al.  Radiomics: extracting more information from medical images using advanced feature analysisEur J cancer. 2012;48(4):441–446. [PMC free article] [PubMed[]
2. Lambin P, Leijenaar RTH, Deist TM, et al.  Radiomics: the bridge between medical imaging and personalized medicineNat rev clin oncol. 2017;14(12):749–762. [PubMed[]
3. Gillies RJ, Kinahan PE, Hricak H. Radiomics: images are more than pictures, they are dataRadiology. 2016;278(2):563–577. [PMC free article] [PubMed[]
4. Aerts HJ, Velazquez ER, Leijenaar RT, et al.  Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approachNat commun. 2014;5:4006. [PMC free article] [PubMed[]
5. Beig N, Bera K, Tiwari P. Introduction to radiomics and radiogenomics in neuro-oncology: implications and challengesNeuro Oncol Adv. 2020;2(S4): iv3–iv14. []
6. Lohmann P, Meißner AK, Kocher M, et al.  Feature-based PET/MRI radiomics in patients with brain tumorsNeuro Oncol Adv. 2020;2(S4): iv15–iv21. []
7. Rathore S, Mohan S, Bakas S, et al.  Multi-institutional non-invasive in vivo characterization of IDH, 1p/19q, and EGFRvIII in glioma using Neuro-Cancer Imaging Phenomics Toolkit (neuro-CaPTk)Neuro Oncol Adv. 2020;2(S4): iv22–iv34. []
8. Wiestler B, Menze B. Deep learning for medical image analysis: A brief introductionNeuro Oncol Adv. 2020;2(S4):iv35–iv41. []
9. Bangalore Yogananda CG, Shah BR, Yu FF, et al.  A novel fully automated MRI-based deep learning method for classification of 1p/19q co-deletion status in brain gliomasNeuro Oncol Adv. 2020;2(S4):iv42–iv48. [PMC free article] [PubMed[]
10. Li XT, Huang RY. Standardization of imaging methods for machine learning in neuro-oncologyNeuro Oncol Adv. 2020;2(S4):iv49–iv55. []

Online Therapy Session Request

Book your first 30min session for free

Answer these questions about yourself to get started.

This will help us match you with the right counselor