A joint distillation model for the tumor segmentation using breast ultrasound images_Journal of Biomedical Engineering

Authors：

GUO Hongjiang ¹ , DING Youyou ¹ ,  DANG Hao ^1,2 , LIU Tongtong ¹ , YAO Shuo ¹ , HOU Daisen ¹ , ZHANG Lan ^2,3

1. School of Information Technology, Henan University of Chinese Medicine, Zhengzhou 450046, P. R. China;
2. Zhengzhou Key Laboratory of Intelligent Analysis and Utilization of Traditional Chinese Medicine Information, Zhengzhou 450046, P. R. China;
3. MRI Department, the First Affiliated Hospital of Henan University of Chinese Medicine, Zhengzhou 450046, P. R. China;

Corresponding author：

DANG Hao, Email: danglee@hactcm.edu.cn

Keywords：

Breast ultrasound images; Semantic segmentation; Attention mechanism; Joint distillation; U-Net

DOI：

10.7507/1001-5515.202311054

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

The accurate segmentation of breast ultrasound images is an important precondition for the lesion determination. The existing segmentation approaches embrace massive parameters, sluggish inference speed, and huge memory consumption. To tackle this problem, we propose T²KD Attention U-Net (dual-Teacher Knowledge Distillation Attention U-Net), a lightweight semantic segmentation method combined double-path joint distillation in breast ultrasound images. Primarily, we designed two teacher models to learn the fine-grained features from each class of images according to different feature representation and semantic information of benign and malignant breast lesions. Then we leveraged the joint distillation to train a lightweight student model. Finally, we constructed a novel weight balance loss to focus on the semantic feature of small objection, solving the unbalance problem of tumor and background. Specifically, the extensive experiments conducted on Dataset BUSI and Dataset B demonstrated that the T²KD Attention U-Net outperformed various knowledge distillation counterparts. Concretely, the accuracy, recall, Dice, and mIoU of proposed method were 95.26%, 86.23%, 85.09%, 83.59%and 77.78% on Dataset BUSI, respectively. And these performance indexes were 97.95%, 92.80%, 88.33%, 88.40% and 82.42% on Dataset B, respectively. Compared with other models, the performance of this model was significantly improved. Meanwhile, compared with the teacher model, the number, size, and complexity of student model were significantly reduced (2.2×10⁶ vs. 106.1×10⁶, 8.4 MB vs. 414 MB, 16.59 GFLOPs vs. 205.98 GFLOPs, respectively). Indeedy, the proposed model guarantees the performances while greatly decreasing the amount of computation, which provides a new method for the deployment of clinical medical scenarios.

1.	World Health Organization. World cancer report: Cancer research for cancer prevention. Geneva: World Health Organization, 2020.
2.	马国祥, 严传波, 杨凌菲, 等. 基于改进的多尺度深度残差网络肝包虫超声影像诊断方法. 东北师大学报(自然科学版), 2023, 55(1): 80-87.
3.	Medley D O, Santiago C, Nascimento J C. CyCoSeg: a cyclic collaborative framework for automated medical image segmentation. IEEE Trans Pattern Anal Mach Intell, 2021, 44(11): 8167-8182.
4.	孙玉波, 刘嘉男, 孙泽文, 等. 一种基于生成对抗网络的无监督域自适应磁共振图像分割方法. 生物医学工程学杂志, 2022, 39(6): 1181-1188.
5.	Zhang H, Burrows L, Meng Y, et al. Weakly supervised segmentation with point annotations for histopathology images via contrast-based variational model// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver: IEEE, 2023: 15630-15640.
6.	高威, 蒋慧, 焦一平, 等. 基于多任务和注意力的胰腺癌全切片图像多组织分割模型. 生物医学工程学杂志, 2023, 40(0): 70-78.
7.	Hu Q, Chen Y, Xiao J, et al. Label-free liver tumor segmentation// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver: IEEE, 2023: 7422-7432.
8.	刘云鹏, 李瑾, 王宇, 等. 图像增强下基于生成对抗网络和卷积神经网络的CT与MRI融合方法. 生物医学工程学杂志, 2023, 40(2): 208-216.
9.	Rahman A, Valanarasu J M J, Hacihaliloglu I, et al. Ambiguous medical image segmentation using diffusion models// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver: IEEE, 2023: 11536-11546.
10.	吴玉超, 林岚, 吴水才. 基于多尺度、多路注意力融合机制的多模态高等级脑胶质瘤语义分割网络. 生物医学工程学杂志, 2022, 39(3): 433-440.
11.	Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation// Medical Image Computing and Computer-Assisted Intervention (MICCAI). Munich: Springer, 2015, 9351: 234-241.
12.	Oktay O, Schlemper J, Folgoc L L, et al. Attention U-Net: learning where to look for the pancreas. arXiv preprints, 2018: 1804.03999.
13.	Zhang Y, Liu Y, Cheng H, et al. Fully multi-target segmentation for breast ultrasound image based on fully convolutional network. Med Biol Eng Comput, 2020, 58: 2049-2061.
14.	贡荣麟, 施俊, 王骏. 面向乳腺超声图像分割的混合监督双通道反馈U-Net. 中国图象图形学报, 2020, 25(10): 2206-2217.
15.	Wang K, Liang S, Zhong S, et al. Breast ultrasound image segmentation: A coarse-to-fine fusion convolutional neural network. Med Phys, 2021, 48(8): 4262-4278.
16.	Chen G, Liu Y, Dai Y, et al. BAGNet: bidirectional aware guidance network for malignant breast lesions segmentation// Asia-Pacific Conference on Intelligent Robot Systems (ACIRS). Tianjin: IEEE, 2022: 112-116.
17.	Chen G-P, Li L, Dai Y, et al. Rethinking the unpretentious U-net for medical ultrasound image segmentation. Pattern Recogn, 2023, 142: 109728.
18.	Vakanski A, Min X, Freer P E. Attention-enriched deep learning model for breast tumor segmentation in ultrasound images. Ultrasound Med Biol, 2020, 46(10): 2819-2833.
19.	Lei B, Huang S, Li H, et al. Self-co-attention neural network for anatomy segmentation in whole breast ultrasound. Med Image Anal, 2020, 64: 101753.
20.	Xue C, Zhu L, Fu H, et al. Global guidance network for breast lesion segmentation in ultrasound images. Med Image Anal, 2021, 70: 101989.
21.	Punn N S, Agarwal S. RCA-IUnet: a residual cross-spatial attention-guided inception U-Net model for tumor segmentation in breast ultrasound imaging. Mach Vis Appl, 2022, 33(2): 27.
22.	Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network. Comput Sci, 2015, 14(7): 38-39.
23.	邵仁荣, 刘宇昂, 张伟, 等. 深度学习中知识蒸馏研究综述. 计算机学报, 2022(8): 045.
24.	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 770-778.
25.	Al-Dhabyani W, Gomaa M, Khaled H, et al. Dataset of breast ultrasound images. Data in Brief, 2020, 28: 104863.
26.	Yap M H, Pons G, Martí J, et al. Automated breast ultrasound lesions detection using convolutional neural networks. IEEE J Biomed Health Inform, 2017, 22(4): 1218-1226.
27.	Zhou Z W, Siddiquee M M R, Tajbakhsh N, et al. Unet++: a nested U-Net architecture for medical image segmentation// Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop and 8th International Workshop, Granada, Spain. Cham: Springer International Publishing, 2018: 3-11.
28.	Huang H, Lin L, Tong R, et al. UNet 3+: a full-scale connected UNet for medical image segmentation// International Conference on Acoustics, Speech and Signal Processing (ICASSP). Barcelona: IEEE, 2020: 1055-1059.

1. World Health Organization. World cancer report: Cancer research for cancer prevention. Geneva: World Health Organization, 2020.
2. 马国祥, 严传波, 杨凌菲, 等. 基于改进的多尺度深度残差网络肝包虫超声影像诊断方法. 东北师大学报(自然科学版), 2023, 55(1): 80-87.
3. Medley D O, Santiago C, Nascimento J C. CyCoSeg: a cyclic collaborative framework for automated medical image segmentation. IEEE Trans Pattern Anal Mach Intell, 2021, 44(11): 8167-8182.
4. 孙玉波, 刘嘉男, 孙泽文, 等. 一种基于生成对抗网络的无监督域自适应磁共振图像分割方法. 生物医学工程学杂志, 2022, 39(6): 1181-1188.
5. Zhang H, Burrows L, Meng Y, et al. Weakly supervised segmentation with point annotations for histopathology images via contrast-based variational model// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver: IEEE, 2023: 15630-15640.
6. 高威, 蒋慧, 焦一平, 等. 基于多任务和注意力的胰腺癌全切片图像多组织分割模型. 生物医学工程学杂志, 2023, 40(0): 70-78.
7. Hu Q, Chen Y, Xiao J, et al. Label-free liver tumor segmentation// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver: IEEE, 2023: 7422-7432.
8. 刘云鹏, 李瑾, 王宇, 等. 图像增强下基于生成对抗网络和卷积神经网络的CT与MRI融合方法. 生物医学工程学杂志, 2023, 40(2): 208-216.
9. Rahman A, Valanarasu J M J, Hacihaliloglu I, et al. Ambiguous medical image segmentation using diffusion models// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver: IEEE, 2023: 11536-11546.
10. 吴玉超, 林岚, 吴水才. 基于多尺度、多路注意力融合机制的多模态高等级脑胶质瘤语义分割网络. 生物医学工程学杂志, 2022, 39(3): 433-440.
11. Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation// Medical Image Computing and Computer-Assisted Intervention (MICCAI). Munich: Springer, 2015, 9351: 234-241.
12. Oktay O, Schlemper J, Folgoc L L, et al. Attention U-Net: learning where to look for the pancreas. arXiv preprints, 2018: 1804.03999.
13. Zhang Y, Liu Y, Cheng H, et al. Fully multi-target segmentation for breast ultrasound image based on fully convolutional network. Med Biol Eng Comput, 2020, 58: 2049-2061.
14. 贡荣麟, 施俊, 王骏. 面向乳腺超声图像分割的混合监督双通道反馈U-Net. 中国图象图形学报, 2020, 25(10): 2206-2217.
15. Wang K, Liang S, Zhong S, et al. Breast ultrasound image segmentation: A coarse-to-fine fusion convolutional neural network. Med Phys, 2021, 48(8): 4262-4278.
16. Chen G, Liu Y, Dai Y, et al. BAGNet: bidirectional aware guidance network for malignant breast lesions segmentation// Asia-Pacific Conference on Intelligent Robot Systems (ACIRS). Tianjin: IEEE, 2022: 112-116.
17. Chen G-P, Li L, Dai Y, et al. Rethinking the unpretentious U-net for medical ultrasound image segmentation. Pattern Recogn, 2023, 142: 109728.
18. Vakanski A, Min X, Freer P E. Attention-enriched deep learning model for breast tumor segmentation in ultrasound images. Ultrasound Med Biol, 2020, 46(10): 2819-2833.
19. Lei B, Huang S, Li H, et al. Self-co-attention neural network for anatomy segmentation in whole breast ultrasound. Med Image Anal, 2020, 64: 101753.
20. Xue C, Zhu L, Fu H, et al. Global guidance network for breast lesion segmentation in ultrasound images. Med Image Anal, 2021, 70: 101989.
21. Punn N S, Agarwal S. RCA-IUnet: a residual cross-spatial attention-guided inception U-Net model for tumor segmentation in breast ultrasound imaging. Mach Vis Appl, 2022, 33(2): 27.
22. Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network. Comput Sci, 2015, 14(7): 38-39.
23. 邵仁荣, 刘宇昂, 张伟, 等. 深度学习中知识蒸馏研究综述. 计算机学报, 2022(8): 045.
24. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 770-778.
25. Al-Dhabyani W, Gomaa M, Khaled H, et al. Dataset of breast ultrasound images. Data in Brief, 2020, 28: 104863.
26. Yap M H, Pons G, Martí J, et al. Automated breast ultrasound lesions detection using convolutional neural networks. IEEE J Biomed Health Inform, 2017, 22(4): 1218-1226.
27. Zhou Z W, Siddiquee M M R, Tajbakhsh N, et al. Unet++: a nested U-Net architecture for medical image segmentation// Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop and 8th International Workshop, Granada, Spain. Cham: Springer International Publishing, 2018: 3-11.
28. Huang H, Lin L, Tong R, et al. UNet 3+: a full-scale connected UNet for medical image segmentation// International Conference on Acoustics, Speech and Signal Processing (ICASSP). Barcelona: IEEE, 2020: 1055-1059.

Journal of Biomedical Engineering

Latest ArticlesA joint distillation model for the tumor segmentation using breast ultrasound images

Abstract Full text Figures/Tables Video References Cited by

Format

Content