Evaluación de programas de formación continua en contextos no estandarizadoscomplementariedad entre Análisis Factorial y Multinivel para la obtención de evidencias de validez de constructo

  1. Holgado Tello, Francisco Pablo
  2. Chacón Moscoso, Salvador
  3. Vila Abad, Enrique
  4. Delgado Egido, Begoña
  5. Sanduvete-Chaves, Susana
  6. Barbero García, María Isabel
Revista:
Anales de psicología

ISSN: 0212-9728,  1695-2294

Ano de publicación: 2015

Volume: 31

Número: 2

Páxinas: 725-732

Tipo: Artigo

DOI: 10.6018/ANALESPS.31.2.172501 DIALNET GOOGLE SCHOLAR

Outras publicacións en: Anales de psicología

Obxectivos de Desenvolvemento Sustentable

Resumo

Program evaluation is usually applied to non-standardized inter-vention contexts. This implies, among others, deficiencies of: a) validated theoretical models; b) non-standard measurement instruments; c) reliable measures. In this work, we show that Factor Analysis with polychoric cor-relations and Multilevel Analysis could be an adequate procedure to gain construct validity evidence in non-standard evaluative contexts, where the measures are not quantitative and usually are nested. The empirical study is carried out on a sample of 2754 workers of the University of Seville. They have completed a satisfaction questionnaire about training courses aimed to prepare them for the correct performance of their jobs. We highlight the complementarities between both analytical techniques to study the differ-ential variability provided by explained variables nested in different hierar-chical level to predict the perceived satisfaction.

Referencias bibliográficas

  • Anguera, M.T. (2008). Evaluación de programas desde la metodología cuali-tativa. Acción Psicológica, 2(5), 87-101.
  • Anguera, M.T., Chacón-Moscoso, S. y Blanco, A. (2008). Evaluación de pro-gramas sociales y sanitarios: Un abordaje metodológico (pp. 173-224).  Madrid: Síntesis.
  • Barbero-García, M. I., Vila-Abad, E. y Holgado-Tello, F.P. (2011). Intro-ducción básica al Análisis Factorial. Madrid: UNED.
  • Bryk, A. y Raudenbush, S.W. (1992). Hierarchical linear models: Applications and data analysis methods. Newbury Park: Sage Publications.
  • Campbell, D. T. (1982). Can we be scientific in applied social science? Paper pre-sented at the Annual Meeting of the Educational Research Association. Reimpreso en R.F. Conner, D.G. Altman y C. Jackson (1984) Evaluation Studies Review Annual, 9, 26-48.
  • Chacón-Moscoso, S., Sanduvete-Chaves, S., Portell-Vidal, M. & Anguera, M. T. (2013). Reporting a program evaluation: Needs, program plan, inter-vention, and decisions. International Journal of Clinical and Health Psychology, 13(1), 58-66.
  • Cook, T. D. (1981). Dilemmas in evaluation of social programs. En M.B. Brewer y B.E. Collins (Eds.), Scientific inquiry and the social sciences. A vol-ume in honor of Donald T. Campbell (pp.257-287). San Francisco: Josey-Bass.
  • Cook, T. D, Leviton, L. C. y Shadish, W. R. (1985). Program evaluation. En G. Lindzey y E. Aronson (Eds.), Handbook of Social Psychology. (pp.699-777). Nueva York: Random House.
  • Du Toit, S., Du Toit, M. y Cudeck, R. (1999). Introduction to the analysis of mul-tilevel models with LISREL 8.30. Chicago: Scientific Software Internatio-nal.
  • Fernández, J. A., y Ovejero, A. (1994). Satisfacción laboral en un centro hospitalario: Un análisis del cuestionario de Porter. Psicología del Trabajo y de las Organizaciones,10, 39–61.
  • Ferrando, P. y Anguiano, C. (2010). El Análisis Factorial como técnica de investigación en Psicología. Papeles del Psicólogo, 31(1), 18-33.
  • Gadermann, A. M., Guhn, M. y Zumbo, B. D. (2011). Investigating the sub-stantive aspect of construct validity for the Satisfaction with Life Scale adapted for children: A focus on cognitive processes. Social Indicators Re-search, 100(1), 37-60.
  • Herman, J. L., Morris, L. L. y Fitz-Gibbon, C. T. (1987). Program evaluation kit (2ª edición) Beverly Hills: Sage Publications. (9 Vols.).
  • Holgado-Tello, F. P. y Barbero-García, I. (2008). Innovaciones metodológi-cas en evaluación psicológica: perspectivas de futuro. Acción Psicológica, 5, 5-6.
  • Holgado-Tello, F. P., Chacón-Moscoso, S., Barbero-García, I. y Vila-Abad, E. (2010). Polychoric versus Pearson correlations in exploratory and con-firmatory factor analysis of ordinal variables. Quality & Quantity, the Inter-national Journal of Methodology, 44(1), 153-166.
  • House, E. R. (1993). Professional Evaluation. Social impact and political consequenc-es. Londres: Sage.
  • House, E. R. (1994). Evaluación, ética y poder. Madrid: Morata.
  • House, E. R. y Shull, R. D. (1988). Rush to Policy. Using Analytic Techniques in Public Sector Decision-Making. New Brunswick: Transaction Publishers.
  • Jöreskog, K. G. (2001). Analysis of ordinal variables 2: Cross-Sectional Data. Documentación del Workshop Structural equation modelling with LISREL 8.51. Jena: Friedrich-Schiller-Universität Jena.
  • Jöreskog, K. G. (2003). Factor Analysis by MINRES. To the memory of Harry Harman and Henry Kaiser. Recuperado de http://www.ssicentral.com/lisrel/techdocs/minres.pdf
  • Jöreskog, K. G. y Sörbom, D. (1999). PRELIS 2. 30. Mooresville, IN: Scien-tific Software International.
  • Jöreskog, K. G. y Sörbom, D. (2003). LISREL 8.54. Chicago: Scientific Software International.
  • Jöreskog, K. G., Sörbom, D., Du Toit, S. y Du Toit, M. (1999). LISREL 8: New statistical features. Chicago: Scientific Software International.
  • Kirkpatrick, D. (1999). Evaluación de acciones formativas: Los cuatro niveles. Barce-lona: Training Club y Epise.
  • Kreft, I. y Leeuw, J. (1998). Introducing multilevel modeling. Londres: Sage Publi-cations.
  • Lee, C., Zhang, G. y Edwards, M. C. (2012). Ordinary least squares estima-tion of parameters in exploratory factor analysis with ordinal data. Mul-tivariate Behavioral Research, 47, 314-339.
  • Medina, M. (1996). Evaluation of the quality of assistance in social services. Intervención Psicosocial, 14, 23–42.
  • Mejías, S. y Huaccho, L. (2011). Macroergonomics intervention programs: recommendations for their design and implementation. Human Factors and Ergonomics in Manufacturing & Service Industries, 21(3), 227–243.
  • Morata, M. A. y Holgado-Tello, F. P. (2013). Construct validity of Likert scales through Confirmatory Factor Analysis: a simulation study com-paring different methods of estimation based on Pearson and poly-choric Correlations. International Journal of Social Science Studies, 1(1), 54-61.
  • Palumbo, D. J. y Nachmias, D. (1983). The preconditions for successful evaluation: is there an ideal paradigm? Policy Sciences, 16, 67-79. (Reim-preso en R. F. Conner, D.G. Altman y C. Jackson (Eds.), Evaluation Studies Review Annual, 9, 102-114).
  • Passmore, J. y Velez, M. (2012). SOAP-M: A training evaluation model for HR. Industrial and Commercial Training, 44(6), 315-325.
  • Pineda, P. (2010). Evaluation of training in organisations: A proposal for an integrated model. Journal of European Industrial Training, 34(7), 673-693.
  • Rajeev, P., Madan, M. S. y Jayarajan, K. (2009). Revisiting Kirkpatrick's model - an evaluation of an academic training course. Current Sci-ence, 96(2), 272-276.
  • Rossi, P. H. y Freeman, H. E. (1985). Evaluation: A systematic approach. Lon-dres: Sage Publications.
  • Sarkis, J., Gonzalez, P. y Adenso, B. (2010). Stakeholders pressure and the adoption of environmental practices: The mediating effect of training. Journal of Operations Management, 28, 163–176.
  • Sechrest, L. y Figueredo, A. J. (1993). Program evaluation. Annual Review of Psychology, 44, 645-674.
  • Thayer, P. W. (1991). A historical perspective on training. En I. L. Goldstein and Associates (Eds.), Training and development in organizations (pp. 457-468). San Francisco: Jossey–Bass.
  • Ventosa, P. (1998). Desde la evaluación de la formación al rendimiento de la inversión. Barcelona: Epise.
  • Wilkins, J. L. M. (2010). Modeling quantitative literacy. Educational and Psycho-logical Measurement, 70(2), 267-290.
  • Yang, F. Jöreskog, K. G. y Luo, H. (2010). Confirmatory Factor Analysis of ordinal variables with misspecified models. Structural Equation Modeling: A Multidisciplinary Journal, 17(3), 392-423.
  • Yin, H. y Schmeidler, P. J. (2009). Why do standardized ISO 14001 envi-ronmental management systems lead to heterogeneous environmental outcomes? Business Strategy and the Environment, 18, 469–486.
Fonte de datos: Dialnet