Model Diagram
We present a novel solution for accurate segmentation of pneumothorax from chest radiographs utilizing free-text radiology reports. Our solution employs text-guided attention to leverage the findings in the report to initially produce a low-dimensional region-localization map. These prior region maps are integrated at multiple scales in an encoder-decoder segmentation framework via dynamic affine feature map transform (DAFT). Extensive experiments on a public dataset CANDID-PTX, show that the integration of free-text reports significantly reduces the false positive predictions, while the DAFT-based fusion of localization maps improves the positive cases. In terms of Dice Similarity (DSC), our proposed approach achieves 0.60 and 0.95 for positive and negative cases, respectively, and 0.70 to 0.85 for medium and large pneumothoraces.
Results show that incorporating free-text reports reduces false positive predictions significantly, and the DAFT-based fusion of localization maps improves positive cases. A stratified analysis of performance on different-sized pneumothorax is presented, and the proposed method is seen to give the best results regardless of the size of the abnormality.
