标签归档:Medical Imaging

CheXseen: Unseen Disease Detection for Deep Learning Interpretation of Chest X-rays

We systematically evaluate the performance of deep learning models in the presence of diseases not labeled for or present during training. First, we evaluate whether deep learning models trained on a subset of diseases (seen diseases) can detect the presence of any one of a larger set of diseases. We find that models tend to falsely classify diseases outside of the subset (unseen diseases) as “no disease”. Second, we evaluate whether models trained on seen diseases can detect seen diseases when co-occurring with diseases outside the subset (unseen diseases). We find that models are still able to detect seen diseases even when co-occurring with unseen diseases. Third, we evaluate whether feature representations learned by models may be used to detect the presence of unseen diseases given a small labeled set of unseen diseases. We find that the penultimate layer of the deep neural network provides useful features for unseen disease detection. Our results can inform the safe clinical deployment of deep learning models trained on a non-exhaustive set of disease classes.

https://arxiv.org/abs/2103.04590

我们系统地评估了深度学习模型在未标注疾病上性能表现。首先,我们评估了深度学习模型在较小地数据集上预训练后在新的疾病种类上地测试表现。其次,我们评估了深度学习模型是否能判别已见过和未见过疾病的混合情况。我们发现在已见过和未见过病症同时出现的时候,深度学习模型依旧能检测到已见过的疾病。最后,我们评估了特征表示是否能检测到未见过的疾病在只有少量标签的情况下。我们发现深度学习模型的倒数第二层可以为未见过的疾病提供有用的特征。我们的结果展示了在不详尽的疾病种类上训练的深度学习模型部署是安全的。

TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation

Medical image segmentation is an essential prerequisite for developing healthcare systems, especially for disease diagnosis and treatment planning. On various medical image segmentation tasks, the u-shaped architecture, also known as U-Net, has become the de-facto standard and achieved tremendous success. However, due to the intrinsic locality of convolution operations, U-Net generally demonstrates limitations in explicitly modeling long-range dependency. Transformers, designed for sequence-to-sequence prediction, have emerged as alternative architectures with innate global self-attention mechanisms, but can result in limited localization abilities due to insufficient low-level details. In this paper, we propose TransUNet, which merits both Transformers and U-Net, as a strong alternative for medical image segmentation. On one hand, the Transformer encodes tokenized image patches from a convolution neural network (CNN) feature map as the input sequence for extracting global contexts. On the other hand, the decoder upsamples the encoded features which are then combined with the high-resolution CNN feature maps to enable precise localization. 
We argue that Transformers can serve as strong encoders for medical image segmentation tasks, with the combination of U-Net to enhance finer details by recovering localized spatial information. TransUNet achieves superior performances to various competing methods on different medical applications including multi-organ segmentation and cardiac segmentation.

https://arxiv.org/abs/2102.04306

医疗图像分割是医疗看护系统的一个基本任务,尤其是在疾病诊断和治疗计划领域。在多种医疗图像分割任务中,类U-Net的架构已经成为了一个基本的方法并且取得了巨大的成功。但是因为卷积操作固有的特点,UNet往往无法应对长距离的依赖。然而Transformers就是为了序列到序列预测任务设计的,它通过本身的自注意力机制实现架构融合但是它也受到定位的限制从而限制了其在低层级的表现。在本文中,我们提出TransUnet,一种融合了Transformer和Unet的有效的医疗图像分割方法。一方面,Transformer可以对已经序列化的卷积网络特征进行编码以提取全局上下文信息。另一方面,解码器对编码的特征进行上采样同时实现了多分辨率特征的融合,这使得网络在全局和局部的定位精确度得到了保证。TransUnet在多种器官和心脏病的分割任务上取得了良好的表现。

Convolution-Free Medical Image Segmentation using Transformers

Like other applications in computer vision, medical image segmentation has been most successfully addressed using deep learning models that rely on the convolution operation as their main building block. Convolutions enjoy important properties such as sparse interactions, weight sharing, and translation equivariance. These properties give convolutional neural networks (CNNs) a strong and useful inductive bias for vision tasks. In this work we show that a different method, based entirely on self-attention between neighboring image patches and without any convolution operations, can achieve competitive or better results. Given a 3D image block, our network divides it into n3 3D patches, where n=3 or 5 and computes a 1D embedding for each patch. The network predicts the segmentation map for the center patch of the block based on the self-attention between these patch embeddings. We show that the proposed model can achieve segmentation accuracies that are better than the state of the art CNNs on three datasets. We also propose methods for pre-training this model on large corpora of unlabeled images. Our experiments show that with pre-training the advantage of our proposed network over CNNs can be significant when labeled training data is small.

https://arxiv.org/abs/2102.13645

与其他的机器视觉任务类似,深度学习模型依赖卷积操作模块在医疗图像分割领域也取得了许多的成功。卷积操作有许多优势,例如稀疏交互,共享权重和翻译同异性。这些优势使得卷积神经网络变得强势且在许多视觉应用中获得广泛应用。在本文中我们提出一种不一样的方法,这种方法完全基于图像邻域patch之间的自注意力机制而不需要卷积操作,而且我们提出的方法能够获得与卷积模型相似甚至是更好的性能。我们的模型接受一个3D图像,然后会将它拆分成n^3个patches,这里n=3或5.之后我们会计算每一个patch的1D嵌入。网络会预测中心patch的分割结果依靠周围邻域的patch的自注意力。我们发现我们提出的模型在分割任务上优于CNNs模型。我们的模型可以在大型无标签的图像语料库中进行预训练,并使用预训练的优势在少样本测试中领先CNNs模型。

Learning domain-agnostic visual representation for computational pathology using medically-irrelevant style transfer augmentation

Suboptimal generalization of machine learning models on unseen data is a key challenge which hampers the clinical applicability of such models to medical imaging. Although various methods such as domain adaptation and domain generalization have evolved to combat this challenge, learning robust and generalizable representations is core to medical image understanding, and continues to be a problem. Here, we propose STRAP (Style TRansfer Augmentation for histoPathology), a form of data augmentation based on random style transfer from artistic paintings, for learning domain-agnostic visual representations in computational pathology. Style transfer replaces the low-level texture content of images with the uninformative style of randomly selected artistic paintings, while preserving high-level semantic content. This improves robustness to domain shift and can be used as a simple yet powerful tool for learning domain-agnostic representations. We demonstrate that STRAP leads to state-of-the-art performance, particularly in the presence of domain shifts, on a particular classification task of predicting microsatellite status in colorectal cancer using digitized histopathology images.

https://arxiv.org/abs/2102.01678

机器学习模型在未接触的数据上进行次优生成是一个具有挑战性的任务,这个任务执行的指令决定了医疗数据的可用性。尽管许多方法例如域适应以及域生成都可以解决上述挑战,然后学习一个鲁棒的且泛化能力强表示一直是医疗图像任务的核心。我们在本文中提出STRAP,一种基于随机风格迁移的数据扩增方法,它可以学习域不相关的计算病理学视觉表示。风格迁移用于将低层次纹理替换为统一的风格迁移图像,这样的风格迁移图像可以广泛地应用于域无关地表示中,我们展示了STRAP的SOTA性能在解决域飘逸的问题上,以及在大肠癌病理学图像的分类问题上。

MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysis

We present MedMNIST, a collection of 10 pre-processed medical open datasets. MedMNIST is standardized to perform classification tasks on lightweight 28×28 images, which requires no background knowledge. Covering the primary data modalities in medical image analysis, it is diverse on data scale (from 100 to 100,000) and tasks (binary/multi-class, ordinal regression and multi-label). MedMNIST could be used for educational purpose, rapid prototyping, multi-modal machine learning or AutoML in medical image analysis. Moreover, MedMNIST Classification Decathlon is designed to benchmark AutoML algorithms on all 10 datasets; We have compared several baseline methods, including open-source or commercial AutoML tools.

我们提出MedMNIST,这是一个10个经过预处理医疗图像数据集的集合。MedMNIST中的数据被标准化成适合于分类任务的轻量化的28X28尺寸,这样的尺寸可以剔除背景信息。这十个数据集覆盖了基本的医疗图像模态,并且照顾到了数据多样性(十万张数据)以及任务多样性(二分类/多分类,有序回归,多标签分类)。MedMNIST可以用于教育,快速原型设计,多域机器学习或者医疗图像AutoML领域。另外,MedMNISTClassification Decathlon是为AutoML设计的benchmark,我们在这个benchmark上比较了包括开源和商用的几个AutoML基线。

ivadomed: A Medical Imaging Deep Learning Toolbox

Alternative text

ivadomed is an open-source Python package for designing, end-to-end training, and evaluating deep learning models applied to medical imaging data. The package includes APIs, command-line tools, documentation, and tutorials. ivadomed also includes pre-trained models such as spinal tumor segmentation and vertebral labeling. Original features of ivadomed include a data loader that can parse image metadata (e.g., acquisition parameters, image contrast, resolution) and subject metadata (e.g., pathology, age, sex) for custom data splitting or extra information during training and evaluation. Any dataset following the Brain Imaging Data Structure (BIDS) convention will be compatible with ivadomed without the need to manually organize the data, which is typically a tedious task. Beyond the traditional deep learning methods, ivadomed features cutting-edge architectures, such as FiLM and HeMis, as well as various uncertainty estimation methods (aleatoric and epistemic), and losses adapted to imbalanced classes and non-binary predictions. Each step is conveniently configurable via a single file. At the same time, the code is highly modular to allow addition/modification of an architecture or pre/post-processing steps. Example applications of ivadomed include MRI object detection, segmentation, and labeling of anatomical and pathological structures. Overall, ivadomed enables easy and quick exploration of the latest advances in deep learning for medical imaging applications.

https://arxiv.org/pdf/2010.09984.pdf

ivadomed 是一个为了设计,端到端训练以及评估医疗图像深度学习模型的开源Python库。这个库包括API, 命令行工具,文档以及教程。ivadomed还包括例如脊柱肿瘤分割和脊椎标签的预训练模型。ivadomed的基本功能包括一个可以读取数据原始信息的data loader(包括参数,图像对比度和分辨率),案例原始信息 (病理、年龄和性别)用于数据分割或者丰富训练/评估数据。ivadomed可以兼容任何符合BIDS数据格式的数据集而不需要进行类似手动地组织数据等乏味的工作。除了提供传统的深度学习方法,ivadomed还提供先进的模型(任意的和认知的),以及适应于非平衡类别以及非二类判别的损失函数。每一步都可以轻松地使用单一文件进行配置。同时,代码是高度模块化的,可以允许增加或者修改模型或者前/后处理。样例应用包括MRI目标检测,分割,异常及病态结构标注。总的来说,ivadomed使得对于深度学习在医疗图像上的应用更为简化及便利。

Semantic Segmentation of Pathological Lung Tissue With Dilated Fully Convolutional Networks

PDF] Semantic Segmentation of Pathological Lung Tissue With Dilated Fully  Convolutional Networks | Semantic Scholar

Early and accurate diagnosis of interstitial lung diseases (ILDs) is crucial for making treatment decisions, but can be challenging even for experienced radiologists. The diagnostic procedure is based on the detection and recognition of the different ILD pathologies in thoracic CT scans, yet their manifestation often appears similar. In this study, we propose the use of a deep purely convolutional neural network for the semantic segmentation of ILD patterns, as the basic component of a computer aided diagnosis system for ILDs. The proposed CNN, which consists of convolutional layers with dilated filters, takes as input a lung CT image of arbitrary size and outputs the corresponding label map. We trained and tested the network on a data set of 172 sparsely annotated CT scans, within a cross-validation scheme. The training was performed in an end-to-end and semisupervised fashion, utilizing both labeled and nonlabeled image regions. The experimental results show significant performance improvement with respect to the state of the art.

间质性肺疾病(ILDs)的早期准确诊断对于做出治疗决策至关重要,但即使对有经验的放射科医生来说也是一项挑战。诊断程序基于在胸部CT扫描中对不同ILD病理的检测和识别,然而它们的表现通常看起来是相似的。在这项研究中,我们建议使用深度纯卷积神经网络进行ILD模式的语义分割,作为计算机辅助诊断系统的基本组成部分。所提出的由具有扩张滤波器的卷积层组成的CNN将任意大小的肺部CT图像作为输入,并输出相应的标签图。在交叉验证方案中,我们在172个稀疏注释的CT扫描数据集上训练和测试了该网络。训练以端对端和半监督的方式进行,利用标记和未标记的图像区域。实验结果表明,相对于现有技术水平,性能有显著提高。

项目地址:https://github.com/intact-project/LungNet

论文地址:https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8325482

ON SEGMENTATION OF PECTORAL MUSCLE IN DIGITAL MAMMOGRAMS BY MEANS OF DEEP LEARNING(利用深度学习对数字化乳腺X线照片中的胸大肌进行分割)

Upper row of subplots: (A1) input MLO mammogram, (A2) edge probability map OUT 1, (A3) edge probability map
OUT 2, (A4) binary mask B, (A5) modified binary mask, (A6) final edge probability map, and (A7) the result of graph-based
edge detection. Subplots B1-B7 are composed in an analogous manner, corresponding to a different input image shown in
Subplot B1.

Computer-aided diagnosis (CAD) has long become an integral part of radiological management of breast disease, facilitating a number of important clinical applications, including quantitative assessment of breast density and early detection of malignancies based on X-ray mammography. Common to such applications is the need to automatically discriminate between breast tissue and adjacent anatomy, with the latter being predominantly represented by pectoralis major (or pectoral muscle). Especially in the case of mammograms acquired in the mediolateral oblique (MLO) view, the muscle is easily confusable with some elements of breast anatomy due to their morphological and photometric similarity. As a result, the problem of automatic detection and segmentation of pectoral muscle in MLO mammograms remains a challenging task, innovative approaches to which are still required and constantly searched for. To address this problem, the present paper introduces a two-step segmentation strategy based on a combined use of data-driven prediction (deep learning) and graph-based image processing. In particular, the proposed method employs a convolutional neural network (CNN) which is designed to predict the location of breastpectoral boundary at different levels of spatial resolution. Subsequently, the predictions are used by the second stage of the algorithm, in which the desired boundary is recovered as a solution to the shortest path problem on a specially designed graph. The proposed algorithm has been tested on three different datasets (i.e., MIAS, CBIS-DDSm and InBreast) using a range of quantitative metrics. The results of comparative analysis show considerable improvement over state-of-the-art, while offering the possibility of model-free and fully automatic processing.

长期以来,计算机辅助诊断(CAD)已成为乳腺疾病放射治疗中不可或缺的一部分,CAD促进了很多重要的临床应用,包括对乳腺密度的定量评估和基于胸部X射线的恶性肿瘤的早期检测。这种应用的共同点是需要自动辨别乳房组织和相邻的解剖结构,后者主要由胸大肌(或胸肌)代表。特别是在从中外侧斜(MLO)角度获取的X光照片中,由于其形态和光度学相似性,肌肉很容易与乳房解剖的某些元素混淆。因此,在MLO乳房X线照片中自动检测和分割胸肌的问题仍然是一项艰巨的任务,仍然需要并不断寻求创新方法。为了解决这个问题,本论文介绍了一种基于数据驱动的预测(深度学习)和基于图的图像处理相结合的两步分割策略。特别地,所提出的方法采用了卷积神经网络(CNN),该卷积神经网络被设计为预测不同空间分辨率水平下的胸胸边界位置。随后,该预测被算法的第二阶段使用,在该预测中,将期望的边界作为特殊设计图上最短路径问题的解决方案进行恢复。所提出的算法已使用一系列定量指标在三个不同的数据集(即MIAS,CBIS-DDSm和InBreast)上进行了测试。

论文地址:https://arxiv.org/pdf/2008.12904.pdf

Learning Visual Context by Comparison

论文地址:https://arxiv.org/pdf/2007.07506.pdf

代码地址:https://github.com/mk-minchul/attend-and-compare

Finding diseases from an X-ray image is an important yet highly challenging task. Current methods for solving this task exploit various characteristics of the chest X-ray image, but one of the most important characteristics is still missing: the necessity of comparison between related regions in an image. In this paper, we present Attend-and Compare Module (ACM) for capturing the difference between an object of interest and its corresponding context. We show that explicit difference modeling can be very helpful in tasks that require direct comparison between locations from afar. This module can be plugged into existing deep learning models. For evaluation, we apply our module to three chest Xray recognition tasks and COCO object detection & segmentation tasks and observe consistent improvements across tasks.

在X-ray图像中寻找病变位置是一个非常有挑战的任务。当前的方法通过探索X-ray图像中各种特征来解决这个问题,但是最终要的特征仍然被忽视:对比一副图像中相关区域的必要性。在本文中,我们提出了一个注意-对比模型(ACM) 用于捕捉感兴趣区域和其对应的上下文之间的差异。我们展示了显式差异建模在需要直接对比位置远处的任务中是非常有用的。这个模块可以直接用于现有的深度学习模型。在验证过程中,我们应用我们的模型于胸部X-ray识别任务和COCO目标检测&分割任务,并在这些任务中观察到了一致的提升。

Data augmentation using learned transformations for one-shot medical image segmentation

PDF] Data Augmentation Using Learned Transformations for One-Shot Medical  Image Segmentation | Semantic Scholar

Image segmentation is an important task in many medical applications. Methods based on convolutional neural networks attain state-of-the-art accuracy; however, they typically rely on supervised training with large labeled datasets. Labeling medical images requires significant expertise and time, and typical hand-tuned approaches for data augmentation fail to capture the complex variations in such images.
We present an automated data augmentation method for synthesizing labeled medical images. We demonstrate our method on the task of segmenting magnetic resonance imaging (MRI) brain scans. Our method requires only a single segmented scan, and leverages other unlabeled scans in a semi-supervised approach. We learn a model of transformations from the images, and use the model along with the labeled example to synthesize additional labeled examples. Each transformation is comprised of a spatial deformation field and an intensity change, enabling the synthesis of complex effects such as variations in anatomy and image acquisition procedures. We show that training a supervised segmenter with these new examples provides significant improvements over state-of-the-art methods for one-shot biomedical image segmentation.

https://arxiv.org/pdf/1902.09383.pdf

长久以来,医疗图像分割一直都在医疗图像分析领域扮演着重要的角色。但是训练一个医疗怕图像分割网络往往需要监督学习和大量标记数据的支持,数据的标注的工作费时费力而且一般的数据扩增方法往往缺少多样性。因此,本文为脑部MRI图像分割任务提出了一种数据扩增方法,只需要一张标注过的脑部MRI图像,模型就可以通过空间和表征变换获得一系列带标注的合成脑部MRI图像。实验证明,使用本模型合成数据训练的分割模型可以在one-shot分割任务中获得state-f-the-art性能。