by Shanghai Jiao Tong University Journal Center

Visual comparison of cardiac segmentation results of different methods. Credit: Journal of Shanghai Jiao Tong University(2023). DOI: 10.16183/j.cnki.jsjtu.2022.088

Researchers from the University of Science and Technology Beijing have proposed a new method for cardiac image segmentation. The work, published in Journal of Shanghai Jiao Tong University, showcased a model called UConvTrans, which is a dual-flow cardiac image segmentation network characterized by global and local information integration.

Magnetic resonance imaging (MRI) is a widely-used medical imaging technique that helps examine organs and tissues in human bodies and assists in the diagnosis of various diseases, including cardiac diseases. Segmentation of magnetic resonance images is an essential step in the analysis of MRI results.

The traditional U-shaped convolutional neural network (CNN) structure is able to handle most cases of medical image segmentation. However, it can hardly perform well in cardiac magnetic resonance image segmentation as there is much noise, the target areas are indistinguishable from the background and the shape of the right ventricle is irregular.

"Due to its local operation characteristics, the ability of convolution for global feature extraction is limited," explained the first author, Professor Qing Li, from the School of Automation and Electrical Engineering, University of Science and Technology, who led the research. "That is the main problem we have to conquer when building a new model that aims to improve the accuracy of cardiac magnetic resonance image segmentation."

Yubin Huangfu, the second author of the article, shared more details of model building. "The Transformer structure can capture global representations, which may compensate for the defects of CNN."

"So, we decided to build a new model that combines CNN and Transformer. After fully investigating the model structure design ideas of image segmentation task based on Transformer architecture in the field of general natural images and the necessary factors required to realize precise medical image segmentation, we designed the preliminary baseline for testing," he said.

"According to the characteristics of medical image segmentation, we introduced a model design that captures global dependencies and local details to improve the model performance. Then, we introduced various effective, lightweight methods to improve the performance and efficiency of the model while ensuring that the accuracy remains unchanged as much as possible. That is how UConvTrans comes into being."

UConvTrans processes a strong ability to extract both local and global features owing to the introduction of Transformer into traditional CNN. The CNN branch extracts local features while the Transformer branch obtains global representations, which retains local detailed features while suppressing the interference of noise and background features.

A bidirectional fusion module called Fuse CNN and Transformer Block (FCTB) was proposed by the researchers to fuse the features extracted by CNN and Transformer with each other, which in turn enhances the feature expression capability and improves the segmentation accuracy of the right ventricle. Furthermore, the parameter setting of UConvTrans becomes flexible as the Transformer structure in the model does not require pre-trained weights.

UConvTrans was tested on the MICCAI 2017 ACDC dataset, including short-axis cardiac MRI of 150 patients, which proves that the model strikes a better balance between precision and efficiency compared with existing models based on CNN or Transformer.

Particularly, UConvTrans outperforms the traditional U-Net by 1.13% average dice coefficient while the parameter amount and the floating point operations are only 10% and 8% of the U-Net. UConvTrans eventually achieves a dice coefficient of 92.42% for the right ventricle, 91.64% for the myocardium, and 95.06% for the left ventricle, respectively, which ranks first in the myocardium left ventricle on the test.

"UConvTrans is able to conduct accurate segmentation of target areas in cardiac MRI, and it outperforms the existing models in terms of efficiency, suggesting a wide application prospect in cardiac image segmentation," said Professor Jiangyun Li, the corresponding author of the work.

"However, there is still much to do to tap the potential of the model fully. First, the current UConvTrans is a 2D network that handles 2D images only. Therefore, it needs further improvement to be capable of handling 3D medical images with more information. In addition, the efficiency of the model needs further improvement so that it can be deployed to terminal systems that require real-time performance."

"Furthermore, evaluation of the generalization of UConvTrans is required to know how the model performs on the medical image data of other organs and tissues."

More information: UConvTrans:A Dual-Flow Cardiac Image Segmentation Network by Global and Local Information Integration, Journal of Shanghai Jiao Tong University(2023). DOI: 10.16183/j.cnki.jsjtu.2022.088. xuebao.sjtu.edu.cn/EN/10.16183 … .cnki.jsjtu.2022.088

Provided by Shanghai Jiao Tong University Journal Center