ROCOv2: Radiology Objects in COntext Version 2, An Updated Multimodal Image Dataset

  1. Johannes Rückert 1
  2. Louise Bloch 1
  3. Raphael Brüngel 1
  4. Ahmad Idrissi-Yaghir 1
  5. Henning Schäfer 2
  6. Cynthia S. Schmidt 2
  7. Sven Koitka 3
  8. Obioma Pelka 1
  9. Asma Ben Abacha 4
  10. Alba Garcia Seco de Herrera 5
  11. Henning Müller 6
  12. Peter A. Horn 2
  13. Felix Nensa 3
  14. Christoph M. Friedrich 1
  1. 1 Department of Computer Science, University of Applied Sciences and Arts Dortmund, Dortmund, Germany
  2. 2 Institute for Transfusion Medicine, University Hospital Essen, Essen, Germany
  3. 3 Institute of Diagnostic and Interventional Radiology and Neuroradiology, University Hospital Essen, Essen, Germany
  4. 4 Microsoft, Redmond, Washington, USA
  5. 5 University of Essex, Wivenhoe Park, Colchester CO4 3SQ, UK
  6. 6 University of Applied Sciences Western Switzerland (HES-SO), Switzerland
Mostrar afiliaciones +

Editor: Zenodo

Año de publicación: 2023

Tipo: Dataset

CC BY-NC 4.0

DOI: 10.5281/ZENODO.8333645 lock_openAcceso abierto editor

Resumen

Recent advances in deep learning techniques have enabled the development of systems for automatic analysis of medical images. These systems often require large amounts of training data with high quality labels, which is difficult and time consuming to generate. Here, we introduce Radiology Object in COntext Version 2 (ROCOv2), a multimodal dataset consisting of radiological images and associated medical concepts and captions extracted from the PubMed Open Access subset. Concepts for clinical modality, anatomy (X-ray), and directionality (X-ray) were manually curated and additionally evaluated by a radiologist. Unlike MIMIC-CXR, ROCOv2 includes seven different clinical modalities. It is an updated version of the ROCO dataset published in 2018, and includes 35,852 new images added to PubMed since 2018, as well as manually curated medical concepts for modality, body region (X-ray) and directionality (X-ray). The dataset consists of 80,080 images and has been used, with minor modifications, in the concept detection and caption prediction tasks of ImageCLEFmedical 2023. The participants had access to the training and validation sets after signing a user agreement. The dataset is suitable for training image annotation models based on image-caption pairs, or for multi-label image classification using the UMLS concepts provided with each image, e.g., to build systems to support structured medical reporting. Additional possible use cases for the ROCOv2 dataset include the pre-training of models for the medical domain, and the evaluation evaluation of deep learning models for multi-task learning. Please do not use this version of the dataset, use the most recent version instead!