El meta-análisisuna metodología para la investigación en educación

  1. Botella, Juan 1
  2. Zamora, Ángela 2
  1. 1 Universidad Autónoma de Madrid
    info

    Universidad Autónoma de Madrid

    Madrid, España

    ROR https://ror.org/01cby8j38

  2. 2 Universidad Nacional de Educación a Distancia
    info

    Universidad Nacional de Educación a Distancia

    Madrid, España

    ROR https://ror.org/02msb5n36

Journal:
Educación XX1: Revista de la Facultad de Educación

ISSN: 1139-613X,  2174-5374

Year of publication: 2017

Volume: 20

Issue: 2

Pages: 17-38

Type: Article

DOI: 10.5944/EDUCXX1.19030 DIALNET GOOGLE SCHOLAR lock_openOpen access editor

More publications in: Educación XX1: Revista de la Facultad de Educación

Abstract

The methodology of meta-analysis is presented as a tool for educational research. A meta-analysis implies a quantitative synthesis of the cumulated evidence about a research question previously defined. The answer will be based on the information contained in the studies previously published (primary studies). Precision, objectivity and replicability are the main characteristics of this methodology. Its applications aim at obtaining a combined estimation of the effect size. The meta-analysis is also applied to explore the observed heterogeneity in a body of results. This allows the statement of new hypothesis that incorporate the role of new variables not taking into account before. A meta-analysis starts with the formulation of a problem. Then, a literature search is carried out. In the codification phase all the documents found are characterised. Finally, a statistical analysis leads to the results, which in turn will be published in a way that ensures the replicability of the meta-analysis. Among the available tools in the context of meta-analysis the so-called Forest Plot plays a remarkable role. The use of this type of graphics allows the joined plotting of both point and interval estimates of the studies selected for the meta-analysis. Even though the potential of the metaanalysis is obvious, it also exhibits some weaknesses derived from the lack of clear boundaries in the constructs studied or from the presence of bias in the publication of the results. The general appraisement of meta-analysis in the field of educational research is positive, although areas for improvement are also highlighted. A number of resources both bibliographic and softwarerelated are available for researchers who wish to venture into this fruitful tool for educational research.

Bibliographic References

  • Ahn, S., Ames, A. & Myers, N. (2012). A Review of Meta-Analyses in Education: Methodological Strengths and Weaknesses. Review of Educational Research, 82(4), 436-476.
  • American Psychological Association (2008). Reporting standards for research in psychology: Why do we need them? What might them be? American Psychologist, 63, 839-851.
  • American Psychological Association (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: American Psychological Association.
  • Anafarta A. & Irmak, S. (2014). A Reliability Generalization Study of The Service Quality Scale (Servqual) in The Clute Institute International Business Conference, München, Alemania, 534-541.
  • Baker, M. (2016). Is there a reproducibility crisis? Nature, 533(7604), 452-454.
  • Banks, G., Kepes, S. & Banks, K. (2012). Publication Bias: The Antagonist of Meta-Analytic Reviews and Effective Policymaking. Educational Evaluation and Policy Analysis, 34(3), 259-277.
  • Becker, B. J. & Schram, C. M. (1994). Examining explanatory models through research synthesis. En H. Cooper & L. V. Hedges (Eds.). The handbook of research synthesis (pp. 357-381). Nueva York: Russell Sage Foundation.
  • Biggs J., Kember D. & Leung D. Y. P. (2001). The revised two-factor Study Process Questionnaire: R-SPQ-2F. British Journal of Educational Psychology, 71(1), 133-149.
  • Blázquez, D., Botella, J. & Suero, M. (2017). The debate on the ego-depletion effect: Evidence from metaanalysis with the p-uniform method. Frontiers in Psychology, 8, 197.
  • Borenstein, M., Hedges, L. V., Higgins, J. P. T. & Rothstein, H. R. (2009). Introduction to meta-analysis. Chichester, UK: John Wiley and sons.
  • Borenstein, M., Hedges, L. V., Higgins, J. P. T. & Rothstein, H. R. (2010). A basic introduction to fixed-effects and random-effects models for meta-analysis. Research Synthesis Methods, 1, 97-111.
  • Borenstein, M., Hedges, L. V., Higgins, J. P. T. & Rothstein, H. R. (2013). Comprehensive Meta-analysis Version 3.0. Biostat Inc.
  • Botella, J. & Gambara, H. (2006). Doing and reporting a meta-analysis. International Journal of Clinical and Health Psychology, 6, 425-440.
  • Botella, J. y Huang, H. (2012). Procedimientos para realizar meta-análisis de la precisión de instrumentos de clasificación binaria. Psicothema, 24(1), 133-141.
  • Botella, J. y Sánchez-Meca, J. (2015). Meta-análisis en Ciencias Sociales y de la Salud. Madrid: Editorial Síntesis.
  • Botella, J., Suero, M. & Gambara, H. (2010). Psychometric inferences from a meta-analysis of reliability and internal consistency coefficients. Psychological Methods, 15, 386-397.
  • Cheung, M. W. L. (2015). Meta-analysis: A structural equation modeling approach. Chichester, Reino Unido: John Wiley & Sons.
  • Clarke, M. (2009). Reporting format. En H. Cooper, L. V. Hedges y J. C. Valentine (Eds.)(2009), The handbook of research synthesis and meta-analysis 2.ª ed. (pp. 521-534). Nueva York: Russell Sage Foundation.
  • Cooper, H., Hedges, L. V. & Valentine, J. C. (2009). The Handbook of Research Synthesis and Meta-analysis. Nueva York: Russell Sage Foundation.
  • Faul, F., Erdfelder, E., Lang, A. G. y Buchner, A. (2007). G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior research methods, 39(2), 175-191.
  • Gini, G. & Pozzoli, T. (2013). Measuring self-serving cognitive distortions: A meta-analysis of the psychometric properties of the How I Think Questionnaire (HIT). European Journal of Developmental Psychology, 10(4), 510-517.
  • Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5, 3-8.
  • Glass, G. V. (2016). One Hundred Years of Research Prudent Aspirations. Educational Researcher, 45(2), 69-72.
  • Glass, G. V. y Smith, M. K. (1979). Metaanalysis of research on class size and achievement. Educational Evaluation and Policy Analysis, 1, 2-16.
  • Hattie, J. (2009). Visible learning. A synthesis of over 800 meta-analyses relating to achievement. London: Routledge.
  • Hedges, L. V. & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.
  • Henson, R. K., Kogan, L. R., & VachaHaase, T. (2001). A reliability generalization study of the teacher efficacy scale and related instruments. Educational and Psychological Measurement, 61(3), 404-420.
  • Herrington, H. M., Smith, T. B., Feinauer, E. & Griner, D. (2016). Reliability generalization of the Multigroup Ethnic Identity Measure-Revised (MEIMR). Journal of counseling psychology, 63(5), 586-593.
  • Higgins, J. P. & Welton, N. J. (2015). Network meta-analysis: a norm for comparative effectiveness? The Lancet, 386(9994), 628-630.
  • Huedo-Medina, T., Sánchez-Meca, J., Marín-Martínez, F. & Botella, J. (2006). Assessing heterogeneity in meta-analysis: Q statistics or I2 index? Psychological Methods, 11, 193-206.
  • Klassen, R. M. & Tze, V. M. (2014). Teachers’ self-efficacy, personality, and teaching effectiveness: A meta-analysis. Educational Research Review, 12, 59-76.
  • Lipsey, M. W. (1994). Identifying potentially interesting variables and analysis opportunities. En H. M. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 111-123). New York: Sage Foundation.
  • Lipsey, M. W. y Wilson, D. B. (2001). Practical meta-analysis. Thousand oaks, CA: Sage Pub.
  • Neyeloff, J. L., Fuchs, S. C., & Moreira, L. B. (2012). Meta-analyses and Forest plots using a microsoft excel spreadsheet: step-by-step guide focusing on descriptive data analysis. BMC research notes, 5(1), 1.
  • Pashler, H. y Wagenmakers, E. J. (2012). Editors’ introduction to the special section on replicability in psychological science: A crisis of confidence? Perspectives on Psychological Science, 7(6), 528-530.
  • Patall, E. A., Cooper, H. y Robinson, J. C. (2008). Parent involvement in homework: A research synthesis. Review of educational research, 78(4), 1039-1101.
  • Pigott, T., Valentine, J., Polanin J., Williams, T. y Canada, D. (2013). Outcome-Reporting Bias in Education Research. Educational Researcher, 48(2), 424-432.
  • Polanin, J. R., Hennessy, E. A., & Tanner-Smith, E. E. (2016). A Review of Meta-Analysis Packages. R Journal of Educational and Behavioral Statistics. Prepublished November 30, 2016. Doi 1076998616674315.
  • R Development Core Team (2010). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0. URL http:// www.R-project.org/.
  • Reschly, A. L., Busch, T. W., Betts, J., Deno, S. L. & Long, J. D. (2009). Curriculum-based measurement oral reading as an indicator of reading achievement: A meta-analysis of the correlational evidence. Journal of School Psychology, 47(6), 427-469.
  • Rosenthal, R. (1995). Writing metaanalytic reviews. Psychological Bulletin, 18, 183-192.
  • Rothstein, H. R., Sutton, A. J. y Borenstein, M. (2005). Publication Bias in Meta-Analysis. Chichester: John Wiley & Sons.
  • Sánchez-Meca, J. y Botella, J. (2010). Revisiones sistemáticas y meta-análisis: herramientas para la práctica profesional. Papeles del Psicólogo, 31(1), 7-17.
  • Sirin, S. R. (2005). Socioeconomic status and academic achievement: A metaanalytic review of research. Review of educational research, 75(3), 417-453.
  • Smith, M. y Glass, G. V. (1977). Metaanalysis of psychotherapy outcome studies. American Psychologist, 32, 752-760.
  • Stanley, T. D. y Doucouliagos, H. (2014). Meta-regression approximations to reduce publication selection bias. Research Synthesis Methods, 5(1), 60-78.
  • Steffgen, G., Recchia, S. & Viechtbauer, W. (2013). The link between school climate and violence in school: A metaanalytic review. Aggression and Violent Behavior, 18(2), 300-309.
  • Schwinger, M., Wirthwein, L., Lemmer, G. & Steinmayr, R. (2014). Academic Self-Handicapping and Achievement: A Meta-Analysis. Journal of Educational Psychology, 106(3), 744-761.
  • Vacha-Haase, T. (1998). Reliability generalization: Exploring variance in measurement error affecting score reliability across studies. Educational and Psychological Measurement, 58(1), 6-20.
  • Van Assen, M. A., Van Aert, R. & Wicherts, J. M. (2015). Meta-analysis using effect size distributions of only statistically significant studies. Psychological methods, 20(3), 293.
  • Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1-48.
  • Zheng, B., Warschauer, M., Lin, C. H. & Chang, C. (2016). Learning in One-to-One Laptop Environments A Meta-Analysis and Research Synthesis. Review of Educational Research, 86(4), 1052-1084.
Data source: Dialnet