Overview of EXIST 2022:sEXism Identification in Social neTworks

  1. Rodríguez-Sánchez, Francisco
  2. Carrillo-de-Albornoz, Jorge
  3. Plaza Morales, Laura
  4. Mendieta-Aragón, Adrián
  5. Marco Remón, Guillermo
  6. Makeienko, Maryna
  7. Plaza, María
  8. Gonzalo Arroyo, Julio
  9. Spina, Damiano
  10. Rosso, Paolo
Revista:
Procesamiento del lenguaje natural

ISSN: 1135-5948

Año de publicación: 2022

Número: 69

Páginas: 229-240

Tipo: Artículo

DIALNET GOOGLE SCHOLAR lock_openRUA editor

Otras publicaciones en: Procesamiento del lenguaje natural

Resumen

El artículo describe la organización, objetivos y resultados de EXIST 2022 (sEXism Identification in Social neTworks), una competición que se celebra por segundo año consecutivo en el foro IberLEF. EXIST 2022 consta de dos tareas: detección de sexismo y categorización de sexismo de tweets y gabs, tanto en español como en inglés. Hemos recibido un total de 45 ejecuciones para la tarea de detección de sexismo y 29 ejecuciones para la tarea de categorización de sexismo, enviadas por 19 equipos diferentes. En el presente artículo, presentamos el conjunto de datos, la metodología de evaluación, una descripción general de los sistemas propuestos y los resultados obtenidos. El conjunto final de datos consta de más de 12.000 textos anotados de dos redes sociales (Twitter y Gab) etiquetados siguiendo dos procedimientos diferentes: colaboradores externos y expertos en el dominio.

Referencias bibliográficas

  • Amigo, E., J. Carrillo-de Albornoz, M. Almagro-Cadiz, J. Gonzalo, J. Rodrıguez-Vidal, and F. Verdejo. 2017. Evall: Open access evaluation for information access systems. In Proceedings of the 40th International
  • ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1301–1304.
  • Amigo, E., J. Gonzalo, S. Mizzaro, and J. Carrillo-de Albornoz. 2020. An effectiveness metric for ordinal classification: Formal properties and experimental results. arXiv preprint arXiv:2006.01245.
  • Amigo, E., D. Spina, and J. Carrillo-de Albornoz. 2018. An axiomatic analysis of diversity evaluation metrics: Introducing the rank-biased utility metric. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pages 625–634.
  • Arık, S. O. and T. Pfister. 2021. Tabnet: Attentive interpretable tabular learning. In AAAI, volume 35, pages 6679–6687.
  • Basile, V., C. Bosco, E. Fersini, N. Debora, V. Patti, F. M. R. Pardo, P. Rosso, M. Sanguinetti, et al. 2019. Semeval2019 task 5: Multilingual detection of hate speech against immigrants and women in twitter. In 13th International Workshop on Semantic Evaluation, pages 54–63. Association for Computational Linguistics.
  • Berg, S. H. 2006. Everyday sexism and posttraumatic stress disorder in women: A correlational study. Violence Against Women, 12(10):970–988.
  • Canete, J., G. Chaperon, R. Fuentes, and J. Perez. 2020. Spanish pre-trained bert model and evaluation data. PML4DC at ICLR, 2020.
  • Caselli, T., V. Basile, J. Mitrovic, and M. Granitzer. 2020. Hatebert: Retraining bert for abusive language detection in english. arXiv preprint arXiv:2010.12472.
  • Chiril, P., V. Moriceau, F. Benamara, A. Mari, G. Origgi, and M. CoulombGully. 2020. He said “who’s gonna take care of your children when you are at ACL?”: Reported sexist acts are not sexist. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4055–4066, Online, July. Association for Computational Linguistics.
  • Clark, K., M.-T. Luong, Q. V. Le, and C. D. Manning. 2020. Electra: Pretraining text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555.
  • Conneau, A., K. Khandelwal, N. Goyal, V. Chaudhary, G. Wenzek, F. Guzman, E. Grave, M. Ott, L. Zettlemoyer, and V. Stoyanov. 2019. Unsupervised crosslingual representation learning at scale.
  • arXiv preprint arXiv:1911.02116.
  • De la Rosa, J., E. G. Ponferrada, M. Romero, P. Villegas, P. G. de Prado Salas, and M. Grandury. 2022. Bertin: Efficient pretraining of a spanish language model using perplexity sampling. Procesamiento del Lenguaje Natural, 68:13–23.
  • Devlin, J., M.-W. Chang, K. Lee, and K. Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota, June. Association for Computational Linguistics.
  • Donoso-Vazquez, T. and Rebollo-Catalan. 2018. Violencias de genero en entornos virtuales. Ediciones Octaedro.
  • Feng, F., Y. Yang, D. Cer, N. Arivazhagan, and W. Wang. 2020. Language-agnostic bert sentence embedding. arXiv preprint arXiv:2007.01852.
  • Frenda, S., B. Ghanem, M. Montes-y Gomez, and P. Rosso. 2019. Online hate speech against women: Automatic identification of misogyny and sexism on twitter. Journal of Intelligent & Fuzzy Systems, 36(5):4743–4752.
  • Gutierrez-Fandin˜o, A., J. ArmengolEstape, M. Pa`mies, J. Llop-Palao, J. Silveira-Ocampo, C. P. Carrino, A. Gonzalez-Agirre, C. Armentano-Oller, C. Rodriguez-Penagos, and M. Villegas. 2021. Spanish language models. arXiv preprint arXiv:2107.07253.
  • Hartmann, J., M. Heitmann, C. Siebert, and C. Schamp. 2020. More than a feeling: Accuracy and application of sentiment analysis.
  • He, P., J. Gao, and W. Chen. 2021. Debertav3: Improving deberta using electra-style pre-training with gradientdisentangled embedding sharing. arXiv preprint arXiv:2111.09543.
  • Lamprinidis, S., F. Bianchi, D. Hardt, and D. Hovy. 2021. Universal joy: A data set and results for classifying emotions across languages. In The 16th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics.
  • Lan, Z., M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut. 2019. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942.
  • Liu, Y., M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  • Manne, K. 2017. Down girl: The logic of misogyny. Oxford University Press.
  • Mills, S. 2008. Language and sexism. Cambridge University Press.
  • Nguyen, D. Q., T. Vu, and A. T. Nguyen. 2020. Bertweet: A pre-trained language model for english tweets. arXiv preprint arXiv:2005.10200.
  • Perez, J. M., D. A. Furman, L. A. Alemany, and F. Luque. 2021. Robertuito: a pre-trained language model for social media text in spanish. arXiv preprint arXiv:2111.09453.
  • Radford, A., J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
  • Raffel, C., N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu. 2019. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683.
  • Rodrıguez-Sanchez, F., J. Carrillo-de Albornoz, L. Plaza, J. Gonzalo, P. Rosso, M. Comet, and T. Donoso. 2021. Overview of exist 2021: sexism identification in social networks. Procesamiento del Lenguaje Natural, 67:195–207.
  • Rodrıguez-Sanchez, F., J. Carrillo-de Albornoz, and L. Plaza. 2020. Automatic classification of sexism in social networks: An empirical study on twitter data. IEEE Access, 8:219563–219576.
  • Sanh, V., L. Debut, J. Chaumond, and T. Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108.
  • Swim, J., L. Hyers, L. Cohen, and M. Ferguson. 2001. Everyday sexism: Evidence for its incidence, nature, and psychological impact from three daily diary studies. Journal of Social Issues, 57:31 – 53.
  • Waseem, Z. 2016. Are you a racist or am I seeing things? annotator influence on hate speech detection on Twitter. In Proceedings of the First Workshop on NLP and Computational Social Science, pages 138–142, Austin, Texas, November. Association for Computational Linguistics.
  • Waseem, Z. and D. Hovy. 2016. Hateful symbols or hateful people? predictive features for hate speech detection on Twitter. In Proceedings of the NAACL Student Research Workshop, pages 88–93, San Diego, California, June. Association for Computational Linguistics.
  • Xue, L., A. Barua, N. Constant, R. AlRfou, S. Narang, M. Kale, A. Roberts, and C. Raffel. 2022. Byt5: Towards a token-free future with pre-trained byteto-byte models. Transactions of the Association for Computational Linguistics, 10:291–306.
  • Yang, Y., D. Cer, A. Ahmad, M. Guo, J. Law, N. Constant, G. H. Abrego, S. Yuan, C. Tar, Y.-H. Sung, et al. 2019. Multilingual universal sentence encoder for semantic retrieval. arXiv preprint arXiv:1907.04307.
Fuente de los datos: Dialnet