Construyendo Test adaptativos de elección forzosa “On The Fly” para la medición de la personalidad

  1. Francisco J. Abad 1
  2. Rodrigo S. Kreitchmann 1
  3. Sorrel, Miguel A. 1
  4. Nájera, Pablo 1
  5. García-Garzón, Eduardo 2
  6. Garrido, Luis Eduardo 3
  7. Jiménez, Marcos 1
  1. 1 Universidad Autónoma de Madrid, España
  2. 2 Universidad Camilo José Cela, España
  3. 3 Pontificia Universidad Católica Madre y Maestra, República Dominicana
Erakutsi afiliazioak +
Aldizkaria:
Papeles del psicólogo

ISSN: 0214-7823,  1886-1415

Argitalpen urtea: 2022

Alea: 43

Zenbakia: 1

Orrialdeak: 29-35

Mota: Artikulua

DIALNET GOOGLE SCHOLAR lock_openSarbide irekia editor

Beste argitalpen batzuk: Papeles del psicólogo

Laburpena

The new methodological and technological developments of the last decade make it possible to resolve or, at least, attenuate the psychometric problems of forced-choice (FC) tests for the measurement of personality. In these tests, the person being tested is shown blocks of two or more sentences of similar social desirability, from which he or she must choose which one best represents him or her. Thus, FC tests aim to reduce response bias in self-report questionnaires. However, their use is not without risks and complications if they are not created properly. Fortunately, new psychometric models make it possible to model responses in this type of test and to optimize their construction. Moreover, they allow the construction of “on the fly” computerized adaptive FC tests (CAT-FC), in which each item is constructed on the spot, optimally matching sentences from a previously calibrated bank.

Erreferentzia bibliografikoak

  • Abad, F. J., Sorrel, M. A., Garcia, L. F., & Aluja, A. (2018). Modeling general, specific, and method variance in personality measures: Results for ZKA-PQ and NEO-PI-R. Assessment, 25(8), 959-977. https://doi.org/10.1177/1073191116667547
  • Barrick, M. R., & Mount, M. K. (1991). The Big Five personality dimensions and job performance: A Meta-Analysis. Personnel Psychology, 44(1), 1- 26. https://doi.org/10.1111/j.1744-6570.1991.tb00688.x
  • Brown, A., & Maydeu-Olivares, A. (2011). Item response modeling of forced-choice questionnaires. Educational and Psychological Measurement , 7 1 ( 3 ) , 4 6 0 5 0 2 . https://doi.org/10.1177%2F0013164410375112
  • Brown, A., & Maydeu-Olivares, A. (2010). Issues that should not be overlooked in the dominance versus ideal point controversy. Industrial and Organizational Psychology, 3(4), 489-493. https://doi.org/10.1111/j.1754-9434.2010.01277.x
  • Brown, A., & Maydeu-Olivares, A. (2012). Fitting a Thurstonian IRT model to forced-choice data using Mplus. Behavior research methods, 44(4), 1135-1147. https://doi.org/10.3758/s13428- 012-0217-x
  • Brown, A., & Maydeu-Olivares, A. (2018). Modelling forced-choice response formats. En P. Irwing, T. Booth, & D. J. Hughes (Eds.), The Wiley handbook of psychometric testing (pp. 523-569). John Wiley & Sons, Ltd. https://doi.org/10.1002/9781118489772.ch18
  • Bürkner, P.-C., Schulte, N., & Holling, H. (2019). On the statistical and practical limitations of Thurstonian IRT models. Educational and Psychological Measurement , 79(5), 827-854. https://doi.org/10.1177/0013164419832063
  • Cao, M., & Drasgow, F. (2019). Does forcing reduce faking? A meta-analytic review of forced-choice personality measures in highstakes situations. Journal of Applied Psychology, 104(11), 1347-1368. https://doi.org/10.1037/apl0000414
  • Chen, C., Wang, W., Chiu, M. M., & Ro, S. (2020). Item selection and exposure control methods for computerized adaptive testing with multidimensional ranking items. Journal of Educational Measurement, 57(2), 343-369. https://doi.org/10.1111/jedm.12252
  • Cuadrado, D., Salgado, J. F., & Moscoso, S. (2021). Personality, intelligence, and counterproductive academic behaviors: A metaanalysis. Journal of Personality and Social Psychology, 120(2), 504-537. https://doi.org/10.1037/pspp0000285
  • Drasgow, F., Stark, S., Chernyshenko, O. S., Nye, C. D., Hulin, C. L., & White, L. A. (2012). Development of the tailored adaptive personality assessment system (TAPAS) to support army personnel selection and classification decisions. Drasgow Consulting Group Urbana IL.
  • Fisher, P., Robie, C., Christiansen, N., Speer, A., & Schneider, L. (2019). Criterion-related validity of forced-choice personality measures: A cautionary note regarding Thurstonian IRT versus classical test theory scoring. Personnel Assessment and Decisions, 5(1). https://doi.org/10.25035/pad.2019.01.003
  • Frick, S., Brown, A. & Eunike Wetzel (2021) Investigating the normativity of trait estimates from multidimensional forcedcho ice da ta , Mul t i va r ia te Behav io ra l Research , https://doi.org/10.1080/00273171.2021.1938960
  • Heggestad, E. D., Morrison, M., Reeve, C. L., & McCloy, R. A. (2006). Forced-choice assessments of personality for selection: Evaluating issues of normative assessment and faking resistance. Journal of Appl ied Psychology , 91(1), 9-24. https://doi.org/10.1037/0021-9010.91.1.9
  • Hicks, L. E. (1970). Some properties of ipsative, normative, and forced-choice normative measures. Psychological Bulletin, 74(3), 167-184. https://doi.org/10.1037/h0029780
  • Hontangas, P. M., de la Torre, J., Ponsoda, V., Leenen, I., Morillo, D., & Abad, F. J. (2015). Comparing traditional and IRT scoring of forced-choice tests. Applied Psychological Measurement, 39(8), 598-612. https://doi.org/10.1177/0146621615585851
  • Hontangas, P. M., Leenen, I., & de la Torre, J. (2016). Traditional scores versus IRT estimates on forced-choice tests based on a dominance model. Psicothema , 28 , 1, 76-82. https://doi.org/10.7334/psicothema2015.204
  • Judge, T. A., Rodell, J. B., Klinger, R. L., Simon, L. S., & Crawford, E. R. (2013). Hierarchical representations of the five-factor model of personality in predicting job performance: Integrating three organizing frameworks with two theoretical perspectives. Journal of Applied Psychology, 98(6), 875-925. https://doi.org/10.1037/a0033901
  • Kreitchmann, R. S., Abad, F. J., Ponsoda, V., Nieto, M. D., & Morillo, D. (2019). Controlling for response biases in self-report scales: Forcedchoice vs. psychometric modeling of Likert items. Frontiers in Psychology, 10, 2309. https://doi.org/10.3389/fpsyg.2019.02309
  • Kreitchmann, R. S., Abad, F. J., & Sorrel, M. A. (2021). A genetic algorithm for optimal assembly of pairwise forced-choice questionnaires. Behavior Research Methods. https://doi.org/10.3758/s13428-021-01677-4
  • Kreitchmann, R. S., Sorrel, M. A., & Abad, F. J. (enviado). On bank assembly and block selection in multidimensional forced-choice adaptive assessments.
  • Lee, P., & Joo, S.-H. (2021). A new investigation of fake resistance of a multidimensional forced-choice measure: An application of differential item/test functioning. Personnel Assessment and Decisions, 7(1). https://doi.org/10.25035/pad.2021.01.004
  • Lin, Y., & Brown, A. (2017). Influence of context on item parameters in forced-choice personality assessments. Educational and Psychologica l Measurement , 77 (3) , 389-414. https://doi.org/10.1177%2F0013164416646162
  • Makransky, G., Mortensen, E. L., & Glas, C. A. W. (2013). Improving personality facet scores with multidimensional computer adaptive testing: An illustration with the Neo Pi-R. Assessment, 20(1), 3-13. https://doi.org/10.1177/1073191112437756
  • Martínez, A., & Salgado, J. F. (2021). A meta-analysis of the faking resistance of forced-choice personality inventories. Frontiers in Psychology, 12, 732241. https://doi.org/10.3389/fpsyg.2021.732241
  • Morillo, D. (2018). Item response theory models for forced-choice questionnaires. Doctoral dissertation, Universidad Autónoma de, Madrid.
  • Morillo, D., Abad, F. J., Kreitchmann, R. S., Leenen, I., Hontangas, P., & Ponsoda, V. (2019). The journey from Likert to forced-choice questionnaires: Evidence of the invariance of item parameters. Journal of Work and Organizational Psychology, 35(2), 75-83. https://doi.org/10.5093/jwop2019a11
  • Morillo, D., Leenen, I., Abad, F. J., Hontangas, P., de la Torre, J., & Ponsoda, V. (2016). A dominance variant under the multi-unidimensional pairwise-preference framework: Model formulation and Markov Chain Monte Carlo estimation. Applied Psychological Measurement, 40(7), 500-516. https://doi.org/10.1177/0146621616662226
  • Mulder, J., & van der Linden, W. J. (2009). Multidimensional adaptive testing with optimal design criteria for item selection. Psychometrika, 74(2), 273-296. https://doi.org/10.1007/s11336-008-9097-5
  • Nieto, M. D., Abad, F. J., & Hernández-Camacho, A. (2017). Calibrating a new item pool to adaptively assess the Big Five. Psicothema, 29.3, 390-395. https://doi.org/10.7334/psicothema2016.391
  • Nieto, M. D., Abad, F. J., & Olea, J. (2018). Assessing the Big Five with bifactor computerized adaptive testing. Psychological Assessment, 30(12), 1678-1690. https://doi.org/10.1037/pas0000631
  • Olea, J., Abad, F. J., & Barrada, J. R. (2010). Tests informatizados y otros nuevos tipos de tests. Papeles del psicólogo, 31(1), 97- 107.
  • Otero, I., Cuadrado, D., & Martínez, A. (2020). Convergent and predictive validity of the Big Five factors assessed with single stimulus and cuasi-ipsative questionnaires. Journal of Work and Organizat ional Psychology , 36(3), 215-222. https://doi.org/10.5093/jwop2020a17
  • Pavlov, G., Shi, D., Maydeu-Olivares, A., & Fairchild, A. (2021). Item desirability matching in forced-choice test construction. Personal i ty and Individual Dif ferences , 183 , 111114. https://doi.org/10.1016/j.paid.2021.111114
  • Poropat, A. E. (2009). A meta-analysis of the five-factor model of personality and academic performance. Psychological Bulletin, 135(2), 322-338. https://doi.org/10.1037/a0014996
  • Reise, S. P., & Henson, J. M. (2000). Computerization and adaptive administration of the NEO PI-R. Assessment, 7(4), 347-364. https://doi.org/10.1177%2F107319110000700404
  • Richardson, M., Abraham, C., & Bond, R. (2012). Psychological correlates of university students’ academic performance: A systematic review and meta-analysis. Psychological Bulletin, 138(2), 353-387. https://doi.org/10.1037/a0026838
  • Salgado, J.F. (2005). Personalidad y deseabilidad social en contextos organizacionales: implicaciones para la práctica de la psicología del trabajo y las organizaciones. Papeles del psicólogo, 92, 115-128.
  • Salgado, J. F. (2016). A theoretical model of psychometric effects of faking on assessment procedures: Empirical findings and implications for personality at work: A Theoretical Model of faking psychometric effects. International Journal of Selection and Assessment, 24(3), 209-228. https://doi.org/10.1111/ijsa.12142
  • Salgado, J. F., Anderson, N., & Tauriz, G. (2015). The validity of ipsative and cuasi-ipsative forced-choice personality inventories for different occupational groups: A comprehensive meta-analysis. Journal of Occupational and Organizational Psychology, 88(4), 797-834. https://doi.org/10.1111/joop.12098
  • Salgado, J. F., & Táuriz, G. (2014). The Five-Factor Model, forced-choice personality inventories and performance: A comprehensive meta-analysis of academic and occupational validity studies. European Journal of Work and Organizational Psychology, 23(1), 3-30. https://doi.org/10.1080/1359432X.2012.716198
  • Sass, R., Frick, S., Reips, U.-D., & Wetzel, E. (2020). Taking the test taker’s perspective: Response process and test motivation in multidimensional forced-choice versus rating scale instruments. Assessment, 27(3), 572- 584. https://doi.org/10.1177/1073191118762049
  • Stark, S., Chernyshenko, O. S., & Drasgow, F. (2005). An IRT approach to constructing and scoring pairwise preference items involving stimuli on different dimensions: The multi-unidimensional pairwise-preference model. Applied Psychological Measurement, 29(3), 184-203. https://doi.org/10.1177/0146621604273988
  • Stark, S., Chernyshenko, O. S., Drasgow, F., Nye, C. D., White, L. A., Heffner, T., & Farmer, W. L. (2014). From ABLE to TAPAS: A new generation of personality tests to support military selection and classification secisions. Military Psychology, 26(3), 153-164. https://doi.org/10.1037/mil0000044
  • Trent, J. D., Barron, L. G., Rose, M. R., & Carretta, T. R. (2020). Tailored Adaptive Personality Assessment System (TAPAS) as an indicator for counterproductive work behavior: Comparing validity in applicant, honest, and directed faking conditions. Military Psychology, 32(1), 51-59. https://doi.org/10.1080/08995605.2019.1652481
  • Tsutsui, S. (2006). Node histogram vs. edge histogram: A comparison of probabilistic model-building genetic algorithms in permutation domains. 2006 IEEE International Conference on Evolutionary Computation, 1939-1946. https://doi.org/10.1109/CEC.2006.1688a44
  • Viswesvaran, C., & Ones, D. S. (1999). Meta-analyses of fakability estimates: Implications for personality measurement. Educational and Psychological Measurement , 59(2), 197-210.
Datuen iturria: Dialnet