Overview of EXIST 2021:sEXism Identification in Social neTworks

  1. Rodríguez-Sánchez, Francisco
  2. Carrillo-de-Albornoz, Jorge
  3. Plaza Morales, Laura
  4. Gonzalo Arroyo, Julio
  5. Rosso, Paolo
  6. Comet, Miriam
  7. Donoso, Trinidad
Revista:
Procesamiento del lenguaje natural

ISSN: 1135-5948

Año de publicación: 2021

Número: 67

Páginas: 195-207

Tipo: Artículo

DIALNET GOOGLE SCHOLAR lock_openRUA editor

Otras publicaciones en: Procesamiento del lenguaje natural

Resumen

El presente artículo describe la organización, objetivos y resultados de la competición sEXism Identification in Social neTworks (EXIST), una tarea propuesta por primera vez en IberLEF 2021. EXIST 2021 propone dos tareas: la identificación y la categorización de sexismo en inglés y español. Se han recibido un total de 70 runs para la tarea de identificación de sexismo y 61 para la categorización de sexismo, enviadas por 31 equipos de 11 países. En este trabajo, se presentan el dataset, la metodología de evaluación, un análisis de los sistemas propuestos por los participantes y los resultados obtenidos. El dataset final está compuesto por más de 11,000 textos anotados procedentes de dos redes sociales (Twitter y Gab) y su elaboración ha sido supervisada por expertas en temas de género.

Referencias bibliográficas

  • Amigó, E., J. Carrillo-de Albornoz, M. Almagro-Cádiz, J. Gonzalo, J. Rodríguez-Vidal, and F. Verdejo. 2017. Evall: Open access evaluation for information access systems. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1301–1304.
  • Amigó, E., J. Gonzalo, S. Mizzaro, and J. Carrillo-de Albornoz. 2020. An effectiveness metric for ordinal classification: Formal properties and experimental results. arXiv preprint arXiv:2006.01245.
  • Amigó, E., D. Spina, and J. Carrillo-de Albornoz. 2018. An axiomatic analysis of diversity evaluation metrics: Introducing the rank-biased utility metric. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pages 625–634.
  • Barbieri, F., L. E. Anke, and J. CamachoCollados. 2021. Xlm-t: A multilingual language model toolkit for twitter. arXiv preprint arXiv:2104.12250.
  • Barbieri, F., J. Camacho-Collados, L. Espinosa Anke, and L. Neves. 2020. TweetEval: Unified benchmark and comparative evaluation for tweet classification. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1644–1650, Online, November. Association for Computational Linguistics.
  • Basile, V., C. Bosco, E. Fersini, N. Debora, V. Patti, F. M. R. Pardo, P. Rosso, M. Sanguinetti, et al. 2019. Semeval2019 task 5: Multilingual detection of hate speech against immigrants and women in twitter. In 13th International Workshop on Semantic Evaluation, pages 54–63. Association for Computational Linguistics.
  • Bassignana, E., V. Basile, and V. Patti. 2018. Hurtlex: A multilingual lexicon of words to hurt. In 5th Italian Conference on Computational Linguistics, CLiCit 2018, volume 2253, pages 1–6. CEURWS.
  • Baumgartner, J., S. Zannettou, B. Keegan, M. Squire, and J. Blackburn. 2020. The pushshift reddit dataset. In Proceedings of the international AAAI conference on web and social media, volume 14, pages 830–839.
  • Berg, S. H. 2006. Everyday sexism and posttraumatic stress disorder in women: A correlational study. Violence Against Women, 12(10):970–988. Canete, J., G. Chaperon, R. Fuentes, and J. Pérez. 2020. Spanish pre-trained bert model and evaluation data. PML4DC at ICLR, 2020.
  • Chiril, P., V. Moriceau, F. Benamara, A. Mari, G. Origgi, and M. CoulombGully. 2020. He said “who’s gonna take care of your children when you are at ACL?”: Reported sexist acts are not sexist. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4055–4066, Online, July. Association for Computational Linguistics.
  • Conneau, A., K. Khandelwal, N. Goyal, V. Chaudhary, G. Wenzek, F. Guzmán, E. Grave, M. Ott, L. Zettlemoyer, and V. Stoyanov. 2019. Unsupervised crosslingual representation learning at scale. arXiv preprint arXiv:1911.02116.
  • Devlin, J., M.-W. Chang, K. Lee, and K. Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota, June. Association for Computational Linguistics.
  • Donoso-Vázquez, T. and Rebollo-Catalán. 2018. Violencias de género en entornos virtuales. Ediciones Octaedro.
  • Fast, E., B. Chen, and M. S. Bernstein. 2016. Empath: Understanding topic signals in large-scale text. In Proceedings of the 2016 CHI conference on human factors in computing systems, pages 4647–4657.
  • Fersini, E., P. Rosso, and M. Anzovino. 2018. Overview of the task on automatic misogyny identification at ibereval 2018. IberEval@ SEPLN, 2150:214–228.
  • Fox, J., C. Cruz, and J. Y. Lee. 2015. Perpetuating online sexism offline: Anonymity, interactivity, and the effects of sexist hashtags on social media. Computers in Human Behavior, 52:436–442.
  • Frenda, S., B. Ghanem, M. Montes-y Gómez, and P. Rosso. 2019. Online hate speech against women: Automatic identification of misogyny and sexism on twitter. Journal of Intelligent & Fuzzy Systems, 36(5):4743–4752.
  • He, P., X. Liu, J. Gao, and W. Chen. 2020. Deberta: Decoding-enhanced bert with disentangled attention. arXiv preprint arXiv:2006.03654.
  • Joulin, A., E. Grave, P. Bojanowski, and T. Mikolov. 2017. Bag of tricks for efficient text classification. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 427–431. Association for Computational Linguistics, April.
  • Liu, Y., M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. Manne, K. 2017. Down girl: The logic of misogyny. Oxford University Press.
  • Martínez-Cámara, E., M. Díaz-Galiano, M. García-Cumbreras, M. García-Vega, and J. Villena-Román. 2017. Overview of tass 2017. IberEval@ SEPLN, 1896:13–21.
  • Mills, S. 2008. Language and sexism. Cambridge University Press.
  • Montes, M., P. Rosso, J. Gonzalo, E. Aragón, R. Agerri, M. Angel Alvarez Carmona, E. Alvarez Mellado, J. C. de Albornoz, L. Chiruzzo, L. Freitas, H. G. Adorno, Y. Gutiérrez, S. Lima, A. Montejo-Ráez, F. M. P. de Arco, and M. Taulé. 2021. Proceedings of the iberian languages evaluation forum (iberlef 2021). In CEUR Workshop Proceedings.
  • Parikh, P., H. Abburi, P. Badjatiya, R. Krishnan, N. Chhaya, M. Gupta, and V. Varma. 2019. Multi-label categorization of accounts of sexism using a neural framework. arXiv preprint arXiv:1910.04602.
  • Rodríguez-Sánchez, F., J. Carrillo-de Albornoz, and L. Plaza. 2020. Automatic classification of sexism in social networks: An empirical study on twitter data. IEEE Access, 8:219563–219576.
  • Swim, J., L. Hyers, L. Cohen, and M. Ferguson. 2001. Everyday sexism: Evidence for its incidence, nature, and psychological impact from three daily diary studies. Journal of Social Issues, 57:31 – 53.
  • Waseem, Z. 2016. Are you a racist or am I seeing things? annotator influence on hate speech detection on Twitter. In Proceedings of the First Workshop on NLP and Computational Social Science, pages 138–142, Austin, Texas, November. Association for Computational Linguistics.
  • Waseem, Z. and D. Hovy. 2016. Hateful symbols or hateful people? predictive features for hate speech detection on Twitter. In Proceedings of the NAACL Student Research Workshop, pages 88–93, San Diego, California, June. Association for Computational Linguistics.
Fuente de los datos: Dialnet