Stroke-p2pHD: Cross-modality Generation Model of Cerebral Infarction from CT to DWI images_Journal of Biomedical Engineering

Authors：

WANG Qing ¹ , ZHAO Xinyao ² , LIU Xinyue ³ , ZOU Zhimeng ³ , NAN Haiwang ¹ ,  ZHENG Qiang ¹

1. School of Computer and Control Engineering Yantai University, Yantai, Shandong 264005, P. R. China;
2. Yantaishan Hospital, Yantai, Shandong 264001, P. R. China;
3. Muping District Hospital of Traditional Chinese Medicine, Yantai, Shandong 264100, P. R. China;

Corresponding author：

ZHENG Qiang, Email: zhengqiang@ytu.edu.cn

Keywords：

Deep learning; Generative adversarial networks; Computed tomography (CT); Diffusion weighted imaging (DWI); Image synthesis

DOI：

10.7507/1001-5515.202407017

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Among numerous medical imaging modalities, diffusion weighted imaging (DWI) is extremely sensitive to acute ischemic stroke lesions, especially small infarcts. However, magnetic resonance imaging is time-consuming and expensive, and it is also prone to interference from metal implants. Therefore, the aim of this study is to design a medical image synthesis method based on generative adversarial network, Stroke-p2pHD, for synthesizing DWI images from computed tomography (CT). Stroke-p2pHD consisted of a generator that effectively fused local image features and global context information (Global_to_Local) and a multi-scale discriminator (M²Dis). Specifically, in the Global_to_Local generator, a fully convolutional Transformer (FCT) and a local attention module (LAM) were integrated to achieve the synthesis of detailed information such as textures and lesions in DWI images. In the M²Dis discriminator, a multi-scale convolutional network was adopted to perform the discrimination function of the input images. Meanwhile, an optimization balance with the Global_to_Local generator was ensured and the consistency of features in each layer of the M²Dis discriminator was constrained. In this study, the public Acute Ischemic Stroke Dataset (AISD) and the acute cerebral infarction dataset from Yantai Yantaishan Hospital were used to verify the performance of the Stroke-p2pHD model in synthesizing DWI based on CT. Compared with other methods, the Stroke-p2pHD model showed excellent quantitative results (mean-square error = 0.008, peak signal-to-noise ratio = 23.766, structural similarity = 0.743). At the same time, relevant experimental analyses such as computational efficiency verify that the Stroke-p2pHD model has great potential for clinical applications.

1.	中华医学会神经病学分会, 中华医学会神经病学分会脑血管病学组. 中国急性缺血性脑卒中诊治指南2018. 中华神经科杂志, 2018, 51(9): 666-682.
2.	Nagaraja N. Diffusion weighted imaging in acute ischemic stroke: A review of its interpretation pitfalls and advanced diffusion imaging application. J Neurol Sci, 2021, 425: 117435.
3.	Dill T. Contraindications to magnetic resonance imaging. Heart, 2008, 94(7): 943-948.
4.	Oliveri S, Pricolo P, Pizzoli S, et al. Investigating cancer patient acceptance of Whole Body MRI. Clin Imaging, 2018, 52: 246-251.
5.	靳志嘉, 李彦, 陆勇, 等. 医用磁共振成像设备临床需求调研及发展趋势分析. 磁共振成像, 2019, 10(2): 96-100.
6.	Li W, Li Y, Qin W, et al. Magnetic resonance image (MRI) synthesis from brain computed tomography (CT) images based on deep learning methods for magnetic resonance (MR)-guided radiotherapy. Quant Imag Med Surg, 2020, 10(6): 1223.
7.	Hu N, Zhang T, Wu Y, et al. Detecting brain lesions in suspected acute ischemic stroke with CT-based synthetic MRI using generative adversarial networks. Ann Transl Med, 2022, 10(2): 35.
8.	Jiang J, Hu Y-C, Tyagi N, et al. Tumor-aware, adversarial domain adaptation from CT to MRI for lung cancer segmentation// Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference. Granada: MICCAI, 2018: 777-785.
9.	Jin C-B, Kim H, Liu M, et al. Deep CT to MR synthesis using paired and unpaired data. Sensors, 2019, 19(10): 2361.
10.	Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv, 2020: 2010.11929.
11.	Tragakis A, Kaul C, Murray-Smith R, et al. The fully convolutional transformer for medical image segmentation// IEEE Workshop on Applications of Computer Vision (WACV). Waikoloa: WACV, 2023: 3660-3669.
12.	Cai Z, Ding X, Shen Q, et al. Refconv: Re-parameterized refocusing convolution for powerful convnets. arXiv preprint arXiv, 2023: 2310.10563.
13.	Wang Z, Bovik A C, Sheikh H R, et al. Image quality assessment: from error visibility to structural similarity. IEEE T Image Process, 2004, 13(4): 600-612.
14.	Avanaki A N. Exact global histogram specification optimized for structural similarity. Opt Rev, 2009, 16: 613-621.
15.	Isola P, Zhu J-Y, Zhou T, et al. Image-to-image translation with conditional adversarial networks// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: CVPR, 2017: 1125-1134.
16.	Dalmaz O, Yurt M, Çukur T. ResViT: residual vision transformers for multimodal medical image synthesis. IEEE T Med Imaging, 2022, 41(10): 2598-2614.
17.	Zhu J-Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks// 2017 IEEE International Conference on Computer Vision. Venice: ICCV, 2017: 2223-2232.
18.	Luo Y, Zhou L, Zhan B, et al. Adaptive rectification based adversarial network with spectrum constraint for high-quality PET image synthesis. Med Image Anal, 2022, 77: 102335.
19.	Liu Y, Chen A, Shi H, et al. CT synthesis from MRI using multi-cycle GAN for head-and-neck radiation therapy. Comput Med Imag Grap, 2021, 91: 101953.
20.	De Bel T, Bokhorst J-M, Van Der Laak J, et al. Residual cyclegan for robust domain transformation of histopathological tissue slides. Med Image Anal, 2021, 70: 102004.
21.	Kalantar R, Messiou C, Winfield J M, et al. CT-based pelvic T1-weighted MR image synthesis using UNet, UNet++ and cycle-consistent generative adversarial network (Cycle-GAN). Front Oncol, 2021, 11: 665807.
22.	Feng E, Qin P, Chai R, et al. MRI generated from CT for acute ischemic stroke combining radiomics and generative adversarial networks. IEEE J Biomed Health, 2022, 26(12): 6047-6057.
23.	Nie D, Trullo R, Lian J, et al. Medical image synthesis with context-aware generative adversarial networks// Medical Image Computing and Computer Assisted Intervention− MICCAI 2017: 20th International Conference. Quebec City: MICCAI, 2017: 417-425.
24.	Yang H, Sun J, Carass A, et al. Unsupervised MR-to-CT synthesis using structure-constrained CycleGAN. IEEE T Med Imaging, 2020, 39(12): 4249-4261.
25.	Armanious K, Jiang C, Fischer M, et al. MedGAN: Medical image translation using GANs. Comput Med Imag Grap, 2020, 79: 101684.
26.	Wu H, Xiao B, Codella N, et al. Cvt: Introducing convolutions to vision transformers// 2021 IEEE/CVF International Conference on Computer Vision. Montreal: ICCV, 2021: 22-31.
27.	Guo J, Han K, Wu H, et al. CMT: Convolutional neural networks meet vision transformers// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: CVPR, 2022: 12175-12185.
28.	Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need// Advances in Neural Information Processing Systems. Long Beach: NeurlPS, 2017: 30.
29.	Chen J, Lu Y, Yu Q, et al. Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv, 2021: 2102.04306.
30.	Cao H, Wang Y, Chen J, et al. Swin-unet: Unet-like pure transformer for medical image segmentation// European Conference on Computer Vision. Tel Aviv: ECCV, 2022: 205-218.
31.	Nie D, Shen D. Adversarial confidence learning for medical image segmentation and synthesis. Int J Comput Vision, 2020, 128(10): 2494-2513.
32.	Augustin M, Bammer R, Simbrunner J, et al. Diffusion-weighted imaging of patients with subacute cerebral ischemia: comparison with conventional and contrast-enhanced MR imaging. Am J Neuroradiol, 2000, 21(9): 1596-1602.

1. 中华医学会神经病学分会, 中华医学会神经病学分会脑血管病学组. 中国急性缺血性脑卒中诊治指南2018. 中华神经科杂志, 2018, 51(9): 666-682.
2. Nagaraja N. Diffusion weighted imaging in acute ischemic stroke: A review of its interpretation pitfalls and advanced diffusion imaging application. J Neurol Sci, 2021, 425: 117435.
3. Dill T. Contraindications to magnetic resonance imaging. Heart, 2008, 94(7): 943-948.
4. Oliveri S, Pricolo P, Pizzoli S, et al. Investigating cancer patient acceptance of Whole Body MRI. Clin Imaging, 2018, 52: 246-251.
5. 靳志嘉, 李彦, 陆勇, 等. 医用磁共振成像设备临床需求调研及发展趋势分析. 磁共振成像, 2019, 10(2): 96-100.
6. Li W, Li Y, Qin W, et al. Magnetic resonance image (MRI) synthesis from brain computed tomography (CT) images based on deep learning methods for magnetic resonance (MR)-guided radiotherapy. Quant Imag Med Surg, 2020, 10(6): 1223.
7. Hu N, Zhang T, Wu Y, et al. Detecting brain lesions in suspected acute ischemic stroke with CT-based synthetic MRI using generative adversarial networks. Ann Transl Med, 2022, 10(2): 35.
8. Jiang J, Hu Y-C, Tyagi N, et al. Tumor-aware, adversarial domain adaptation from CT to MRI for lung cancer segmentation// Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference. Granada: MICCAI, 2018: 777-785.
9. Jin C-B, Kim H, Liu M, et al. Deep CT to MR synthesis using paired and unpaired data. Sensors, 2019, 19(10): 2361.
10. Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv, 2020: 2010.11929.
11. Tragakis A, Kaul C, Murray-Smith R, et al. The fully convolutional transformer for medical image segmentation// IEEE Workshop on Applications of Computer Vision (WACV). Waikoloa: WACV, 2023: 3660-3669.
12. Cai Z, Ding X, Shen Q, et al. Refconv: Re-parameterized refocusing convolution for powerful convnets. arXiv preprint arXiv, 2023: 2310.10563.
13. Wang Z, Bovik A C, Sheikh H R, et al. Image quality assessment: from error visibility to structural similarity. IEEE T Image Process, 2004, 13(4): 600-612.
14. Avanaki A N. Exact global histogram specification optimized for structural similarity. Opt Rev, 2009, 16: 613-621.
15. Isola P, Zhu J-Y, Zhou T, et al. Image-to-image translation with conditional adversarial networks// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: CVPR, 2017: 1125-1134.
16. Dalmaz O, Yurt M, Çukur T. ResViT: residual vision transformers for multimodal medical image synthesis. IEEE T Med Imaging, 2022, 41(10): 2598-2614.
17. Zhu J-Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks// 2017 IEEE International Conference on Computer Vision. Venice: ICCV, 2017: 2223-2232.
18. Luo Y, Zhou L, Zhan B, et al. Adaptive rectification based adversarial network with spectrum constraint for high-quality PET image synthesis. Med Image Anal, 2022, 77: 102335.
19. Liu Y, Chen A, Shi H, et al. CT synthesis from MRI using multi-cycle GAN for head-and-neck radiation therapy. Comput Med Imag Grap, 2021, 91: 101953.
20. De Bel T, Bokhorst J-M, Van Der Laak J, et al. Residual cyclegan for robust domain transformation of histopathological tissue slides. Med Image Anal, 2021, 70: 102004.
21. Kalantar R, Messiou C, Winfield J M, et al. CT-based pelvic T1-weighted MR image synthesis using UNet, UNet++ and cycle-consistent generative adversarial network (Cycle-GAN). Front Oncol, 2021, 11: 665807.
22. Feng E, Qin P, Chai R, et al. MRI generated from CT for acute ischemic stroke combining radiomics and generative adversarial networks. IEEE J Biomed Health, 2022, 26(12): 6047-6057.
23. Nie D, Trullo R, Lian J, et al. Medical image synthesis with context-aware generative adversarial networks// Medical Image Computing and Computer Assisted Intervention− MICCAI 2017: 20th International Conference. Quebec City: MICCAI, 2017: 417-425.
24. Yang H, Sun J, Carass A, et al. Unsupervised MR-to-CT synthesis using structure-constrained CycleGAN. IEEE T Med Imaging, 2020, 39(12): 4249-4261.
25. Armanious K, Jiang C, Fischer M, et al. MedGAN: Medical image translation using GANs. Comput Med Imag Grap, 2020, 79: 101684.
26. Wu H, Xiao B, Codella N, et al. Cvt: Introducing convolutions to vision transformers// 2021 IEEE/CVF International Conference on Computer Vision. Montreal: ICCV, 2021: 22-31.
27. Guo J, Han K, Wu H, et al. CMT: Convolutional neural networks meet vision transformers// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: CVPR, 2022: 12175-12185.
28. Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need// Advances in Neural Information Processing Systems. Long Beach: NeurlPS, 2017: 30.
29. Chen J, Lu Y, Yu Q, et al. Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv, 2021: 2102.04306.
30. Cao H, Wang Y, Chen J, et al. Swin-unet: Unet-like pure transformer for medical image segmentation// European Conference on Computer Vision. Tel Aviv: ECCV, 2022: 205-218.
31. Nie D, Shen D. Adversarial confidence learning for medical image segmentation and synthesis. Int J Comput Vision, 2020, 128(10): 2494-2513.
32. Augustin M, Bammer R, Simbrunner J, et al. Diffusion-weighted imaging of patients with subacute cerebral ischemia: comparison with conventional and contrast-enhanced MR imaging. Am J Neuroradiol, 2000, 21(9): 1596-1602.

Journal of Biomedical Engineering

Latest ArticlesStroke-p2pHD: Cross-modality Generation Model of Cerebral Infarction from CT to DWI images

Abstract Full text Figures/Tables Video References Cited by

Format

Content