Overview of DIPROMATS 2023automatic detection and characterization of propaganda techniques in messages from diplomats and authorities of world powers

  1. Carrillo Albornoz, Jorge
  2. Gonzalo Verdugo, Iván
  3. Moral, Pablo
  4. Marco Remón, Guillermo
  5. Gonzalo Arroyo, Julio
Revista:
Procesamiento del lenguaje natural

ISSN: 1135-5948

Año de publicación: 2023

Número: 71

Páginas: 397-407

Tipo: Artículo

DIALNET GOOGLE SCHOLAR lock_openRUA editor

Otras publicaciones en: Procesamiento del lenguaje natural

Resumen

This paper presents the results of the DIPROMATS 2023 challenge, a shared task included at the Iberian Languages Evaluation Forum (IberLEF). DIPROMATS 2023 provides a dataset with 12012 annotated tweets in English and 9501 tweets in Spanish, posted by authorities of China, Russia, United States and the European Union. Three tasks are proposed for each language. The first one aims to distinguish if a tweet has propaganda techniques or not. The second task seeks to classify the tweet into four clusters of propaganda techniques, whereas the third one offers a fine-grained categorization of 15 techniques. For the three tasks we have received a total of 34 runs from 9 different teams.

Referencias bibliográficas

  • Amigo, E. and A. Delgado. 2022. Evaluating extreme hierarchical multi-label classification. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5809–5819, Dublin, Ireland, May. Association for Computational Linguistics.
  • Cañete, J., G. Chaperon, R. Fuentes, J.-H. Ho, H. Kang, and J. Pérez. 2020. Spanish pre-trained bert model and evaluation data. Pml4dc at iclr, 2020(2020):1–10.
  • Conneau, A., K. Khandelwal, N. Goyal, V. Chaudhary, G. Wenzek, F. Guzman, E. Grave, M. Ott, L. Zettlemoyer, and V. Stoyanov. 2020. Unsupervised Crosslingual Representation Learning at Scale, April. arXiv:1911.02116 [cs].
  • Da San Martino, G., A. Barrón-Cedeño, H. Wachsmuth, R. Petrov, and P. Nakov. 2020. SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 1377–1414, Barcelona (online). International Committee for Computational Linguistics.
  • Da San Martino, G., P. Nakov, J. Pirskoski, and N. Stefanovitch. 2022. SemEval2023 shared task on ”Detecting the Genre, the Framing, and the Persuasion Techniques in Online News in a Multi-lingual Setup”.
  • Da San Martino, G., S. Yu, A. Barrón Cedeño, R. Petrov, and P. Nakov. 2019. Fine-Grained Analysis of Propaganda in News Article. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5636–5646, Hong Kong, China, November. Association for Computational Linguistics.
  • De la Rosa, J., E. Ponferrada, M. Romero, P. Villegas, P. González de Prado Salas, and M. Grandury. 2022. BERTIN: Efficient Pre-Training of a Spanish Language Model using Perplexity Sampling. Procesamiento del Lenguaje Natural, pages 13– 23.
  • Devlin, J., M.-W. Chang, K. Lee, and K. Toutanova. 2018. BERT: Pretraining of Deep Bidirectional Transformers for Language Understanding. Publisher: arXiv Version Number: 2.
  • Hobbs, R. and S. McGee. 2014. Teaching about Propaganda: An Examination of the Historical Roots of Media Literacy. Journal of Media Literacy Education, 6(2).
  • Institute for Propaganda Analysis. 1938. Propaganda Analysis: Volume I of the Publications of the Institute for Propaganda Analysis. Institute for Propaganda Analysis, Inc., New York, 1 edition.
  • Johnson-Cartee, K. S. and G. Copeland. 2004. Strategic political communication: rethinking social influence, persuasion, and propaganda. Communication, media, and politics. Rowman & Littlefield, Lanham, Md.
  • Jowett, G. and V. O’Donnell. 2015. Propaganda & persuasion. SAGE, Thousand Oaks, Calif, sixth edition edition.
  • Liu, Y., M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach, July. arXiv:1907.11692 [cs].
  • Loureiro, D., F. Barbieri, L. Neves, L. Espinosa Anke, and J. Camacho-collados. 2022. TimeLMs: Diachronic Language Models from Twitter. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 251–260, Dublin, Ireland. Association for Computational Linguistics.
  • Nguyen, D. Q., T. Vu, and A. Tuan Nguyen. 2020. BERTweet: A pre-trained language model for English Tweets. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 9–14, Online. Association for Computational Linguistics.
  • Piskorski, J., Stefanovitch, N., Da San Martino, G., and Nakov, P. Semeval-2023 task 3: Detecting the category, the framing, and the persuasion techniques in online news in a multi-lingual setup. In Proceedings of the 17th International Workshop on Semantic Evaluation, Toronto, Canada.
  • Pérez, J. M., D. A. Furman, L. Alonso Alemany, and F. M. Luque. 2022. RoBERTuito: a pre-trained language model for social media text in Spanish. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 7235– 7243, Marseille, France, June. European Language Resources Association.
  • Sparkes-Vian, C. 2019. Digital Propaganda: The Tyranny of Ignorance. Critical Sociology, 45(3):393–409, May.
  • Spinde, T., L. Rudnitckaia, K. Sinha, F. Hamborg, B. Gipp, and K. Donnay. 2021a. MBIC – A Media Bias Annotation Dataset Including Annotator Characteristics, May. arXiv:2105.11910 [cs].
  • Spinde, T., M. Plank, J.-D. Krieger, T. Ruas, B. Gipp, and A. Aizawa. 2021b. Neu ral Media Bias Detection Using Distant Supervision With BABE – Bias Annotations By Experts. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 1166–1177. arXiv:2209.14557 [cs].
  • Teninbaum, G. H. 2009. Reductio ad Hitlerum: Trumping the judicial Nazi card. Mich. St. L. Rev., page 541. Publisher: HeinOnline.
  • Weston, A. 2017. A rulebook for arguments. Hackett Publishing Company, Inc, Indianapolis ; Cambridge, fifth edition edition.
  • Wolf, T., L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jernite, J. Plu, C. Xu, T. Le Scao, S. Gugger, M. Drame, Q. Lhoest, and A. Rush. 2020. Transformers: Stateofthe-Art Natural Language Processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.
Fuente de los datos: Dialnet