Updates to the validity evidence based on the internal structure of the ASRS-18

Thiago Iamada Porto; Camélia Santina Murgo; Leonardo de Oliveira Barros; Andressa Pereira de Souza

doi:10.22235/cp.v17i2.3092

Authors

Thiago Iamada Porto Universidade do Oeste Paulista https://orcid.org/0000-0001-8984-2156
Camélia Santina Murgo Universidade do Oeste Paulista https://orcid.org/0000-0003-3932-7580
Leonardo de Oliveira Barros Universidade Federal da Bahia https://orcid.org/0000-0002-8406-0515
Andressa Pereira de Souza Universidade do Oeste Paulista https://orcid.org/0000-0003-3730-2056

DOI:

https://doi.org/10.22235/cp.v17i2.3092

Keywords:

ADHD, Adults, Psychological Assessment, Psychometrics

Abstract

The aim of this research was to present updated validity evidence based on the internal structure of the ASRS-18. Two studies were carried out, an exploratory factor analysis (EFA) and a confirmatory factor analysis (CFA). In the opposite direction to what the literature suggests for the instrument, the EFA indicated the adequacy of the one-factor model, which supported the decision regarding the number of factors to be retained. Once the structure of the scale had been determined, CFA and multi-group CFA analyses were carried out to assess the characteristics of the instrument. The results obtained in the CFA were initially unsatisfactory, with items having high residual correlations, explained by the redundancy of content between items. It was therefore decided to reduce the number of items in the instrument, and the analyses indicated that the scale was suitable for use in both clinical practice and research. Finally, the multi-group CFA showed that the gender variable influenced the scale's response pattern, with women needing a greater latent trait to score higher on the inattention items and a lower trait for the hyperactivity items. The proposed ASRS-18 presents an accurate and stable model for measuring attention deficit hyperactivity disorder (ADHD). It is suggested that a reduced version of the instrument be used, as well as further research to expand the evaluation of psychometric properties. In the clinical sphere, it is recommended to consider the suggestions presented in the discussions of this study for assessing ADHD symptoms for criterion A according to the DSM.

Downloads

Download data is not yet available.

Author Biographies

Thiago Iamada Porto, Universidade do Oeste Paulista

Graduando em Medicina (Famepp - Universidade do Oeste Paulista – UNOESE).

Camélia Santina Murgo, Universidade do Oeste Paulista

Programa de Pós-graduação em Educação (PPGE). Campus II- Doutorado em Educação.

Leonardo de Oliveira Barros, Universidade Federal da Bahia

Programa de Pós-Graduação Stricto Sensu em Psicologia da Universidade Federal da Bahia (UFBA).

References

Achenbach, T. M. (2020). Bottom-up and top-down paradigms for psychopathology: A half-century odyssey. Annual Review of Clinical Psychology, 16(1), 1-24. https://doi.org/10.1146/annurev-clinpsy-071119-115831

Achenbach, T. M., & Edelbrock, C. S. (1978). The classification of child psychopathology: A review and analysis of empirical efforts. Psychological Bulletin, 85(6), 1275-1301. https://doi.org/10.1037/0033-2909.85.6.1275

Achenbach, T. M., Conners, C. K., Quay, H. C., Verhulst, F. C., & Howell, C. T. (1989). Replication of empirically derived syndromes as a basis for taxonomy of child/adolescent psychopathology. Journal of Abnormal Child Psychology, 17(3), 299-323. https://doi.org/10.1007/BF00917401

Achenbach, T. M., Howell, C. T., Quay, H. C., & Conners, C. K. (1991). National survey of problems and competencies among four- to sixteen-year-olds: parents' reports for normative and clinical samples. Monographs of the Society for Research in Child Development, 56(3), 1-131. https://doi.org/10.2307/1166156

American Psychiatric Association. (1980). Diagnostic and Statistical Manual of Mental Disorders: DSM-III (3rd ed.).

American Psychiatric Association. (1987). Diagnostic and statistical manual of mental disorders: DSM-III-R (3rd ed., revised).

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders: DSM-IV (4th ed.).

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders: DSM-5 (5th ed.).

Arnett, A. B., Pennington, B. F., Willcutt, E. G., DeFries, J. C., & Olson, R. K. (2015). Sex differences in ADHD symptom severity. Journal of Child Psychology & Psychiatry, 56(6), 632-639. https://doi.org/10.1111/jcpp.12337

Barry, T. D., Lyman, R. D., & Klinger, L. G. (2002). Academic underachievement and Attention-Deficit/Hyperactivity Disorder: The negative impact of symptom severity on school performance. Journal of School Psychology, 40(3), 259-283. https://doi.org/https://doi.org/10.1016/S0022-4405(02)00100-0

Bornovalova, M. A., Choate, A. M., Fatimah, H., Petersen, K. J., & Wiernik, B. M. (2020). Appropriate use of bifactor analysis in psychopathology research: Appreciating benefits and limitations. Biological Psychiatry, 88(1), 18-27. https://doi.org/10.1016/j.biopsych.2020.01.013

Borsa, J. C., Damásio, B. F., & Bandeira, D. R. (2012). Adaptação e validação de instrumentos psicológicos entre culturas: algumas considerações. Paidéia (Ribeirão Preto), 22, 423-432. https://doi.org/10.1590/S0103-863X2012000300014

Brown, T. A. (2006). Confirmatory factor analysis for applied research. Guilford Press.

Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). The Guilford Press.

Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233-255. https://doi.org/10.1207/S15328007SEM0902_5

Clements, S. D., & Peters, J. E. (1962). Minimal brain dysfunctions in the school-age child. Diagnosis and treatment. Archives of General Psychiatry, 6, 185-197. https://doi.org/10.1001/archpsyc.1962.01710210001001

Conselho Federal de Psicologia. (2022). Cartilha Avaliação Psicológica. https://site.cfp.org.br/wpcontent/uploads/2022/08/cartilha_avaliacao_psicologica-2309.pdf

Corbisiero, S., Hartmann-Schorro, R. M., Riecher-Rössler, A., & Stieglitz, R. D. (2017). Screening for adult Attention-Deficit/Hyperactivity Disorder in a psychiatric outpatient population with specific focus on sex differences. Frontiers in Psychiatry, 8, 115. https://doi.org/10.3389/fpsyt.2017.00115

Costello, A. B., & Osborne, J. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research, and Evaluation, 10(1), 7. https://doi.org/10.7275/jyj1-4868

Damásio, B. F. (2012). Uso da análise fatorial exploratória em psicologia. Avaliação Psicológica, 11, 213-228.

DeMars, C. E. (2013). A tutorial on interpreting bifactor model scores. International Journal of Testing, 13(4), 354-378. https://doi.org/10.1080/15305058.2013.799067

Dharmaratne M. A., & Attygalle, M. D. T. (2018). Improving the Statistical Capacity Index: A Statistical Approach. American Journal of Applied Mathematics and Statistics, 6(3), 80-95. https://doi.org/10.12691/ajams-6-3-1

DiStefano, C., & Morgan, G. B. (2014). A comparison of Diagonal Weighted Least Squares Robust Estimation techniques for ordinal data. Structural Equation Modeling: A Multidisciplinary Journal, 21(3), 425-438. https://doi.org/10.1080/10705511.2014.915373

Ferrando, P. J., & Lorenzo-Seva, U. (2018). Assessing the quality and appropriateness of factor solutions and factor score estimates in exploratory item factor analysis. Educational and Psychological Measurement, 78(5), 762-780. https://doi.org/10.1177/0013164417719308

Frick, P. J., & Nigg, J. T. (2012). Current issues in the diagnosis of Attention Deficit Hyperactivity Disorder, Oppositional Defiant Disorder, and Conduct Disorder. Annual Review of Clinical Psychology, 8(1), 77-107. https://doi.org/10.1146/annurev-clinpsy-032511-143150

Giordano, C., & Waller, N. G. (2020). Recovering bifactor models: A comparison of seven methods. Psychological Methods, 25(2), 143-156. https://doi.org/10.1037/met0000227

Giordano, C., Ones, D. S., Waller, N. G., & Stanek, K. C. (2020). Exploratory bifactor measurement models in vocational behavior research. Journal of Vocational Behavior, 120. https://doi.org/10.1016/j.jvb.2020.103430

Goetz, C., Coste, J., Lemetayer, F., Rat, A. C., Montel, S., Recchia, S., Debouverie, M., Pouchot, J., Spitz, E., & Guillemin, F. (2013). Item reduction based on rigorous methodological guidelines is necessary to maintain validity when shortening composite measurement scales. Journal of Clinical Epidemiology, 66(7), 710-718. https://doi.org/10.1016/j.jclinepi.2012.12.015

Hancock, G. R., & Mueller, R. O. (2001). Rethinking construct reliability within latent variable systems. Structural equation modeling: Present and future, 195, 216.

Hinshaw, S. P. (2018). Attention Deficit Hyperactivity Disorder (ADHD): controversy, developmental mechanisms, and multiple levels of analysis. Annual Review of Clinical Psychology, 14(1), 291-316. https://doi.org/10.1146/annurev-clinpsy-050817-084917

International Test Commission. (2017). The ITC guidelines for translating and adapting tests. https://www.intestcom.org/

Kim, J.-H., Lee, E.-H., & Joung, Y.-S. (2013). The WHO Adult ADHD Self-Report Scale: Reliability and validity of the Korean version. Psychiatry Investigation, 10(1), 41-46. https://doi.org/10.4306/pi.2013.10.1.41

Kline, R. B. (2023). Principles and practice of structural equation modeling (4th ed). Todd D. Little.

Laher, S. (2010). Using exploratory factor analysis in personality research: Best-practice recommendations. SA Journal of Industrial Psychology, 36(1), 1-7. https://doi.org/10.4102/sajip.v36i1.873

Lambert, Z. V., Wildt, A. R., & Durand, R. M. (1991). Approximating confidence intervals for factor loadings. Multivariate Behavioral Research, 26(3), 421-434. https://doi.org/10.1207/s15327906mbr2603_3

Leahy, L. G. (2018). Diagnosis and treatment of ADHD in children vs adults: What nurses should know. Archives of Psychiatric Nursing, 32(6), 890-895. https://doi.org/10.1016/j.apnu.2018.06.013

Leite, W. B. (2011). Avaliação das propriedades psicométricas da escala de autorrelato de sintomas do transtorno do déficit de atenção e hiperatividade ASRS-18 [Master's dissertation]. Universidade Federal de Minas Gerais. http://hdl.handle.net/1843/BUOS-8NFFR5

Lewinsohn, P. M., Shankman, S. A., Gau, J. M., & Klein, D. N. (2004). The prevalence and co-morbidity of subthreshold psychiatric conditions. Psychological Medicine, 34(4), 613-622. https://doi.org/10.1017/s0033291703001466

Li, C. H. (2016). Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behavior Research Methods, 48(3), 936-949. https://doi.org/10.3758/s13428-015-0619-7

Lorenzo-Seva, U. (2021). SOLOMON: A method for splitting a sample into equivalent subsamples in factor analysis. Behavior Research Methods, 1-13. https://doi.org/10.3758/s13428-021-01750-y

Lorenzo-Seva, U., & Ferrando, P. J. (2019). Robust Promin: A method for diagonally weighted factor rotation. Liberabit: Revista Peruana de Psicología, 25(1), 99-106. https://doi.org/10.24265/liberabit.2019.v25n1.08

Luo, Y., Weibman, D., Halperin, J. M., & Li, X. (2019). A Review of Heterogeneity in Attention Deficit/Hyperactivity Disorder (ADHD). Frontiers in Human Neuroscience, 13, 42. https://doi.org/10.3389/fnhum.2019.00042

Marcus, D. K., & Barry, T. D. (2011). Does attention-deficit/hyperactivity disorder have a dimensional latent structure? A taxometric analysis. Journal of Abnormal Psychology, 120(2), 427-442. https://doi.org/10.1037/a0021405

Mattos, P., Segenreich, D., Saboya, E., Louzã, M., Dias, G., & Romano, M. (2006). Adaptação transcultural para o português da escala Adult Self-Report Scale para avaliação do transtorno de déficit de atenção/hiperatividade (TDAH) em adultos. Archives of Clinical Psychiatry (São Paulo), 33, 188-194. https://doi.org/10.1590/s0101-60832006000400004

Morin, A. J., Tran, A., & Caci, H. (2016). Factorial validity of the ADHD Adult Symptom Rating Scale in a French community sample: Results from the ChiP-ARD Study. Journal of Attention Disorders, 20(6), 530-541. https://doi.org/10.1177/1087054713488825

Narrow, W. E., & Kuhl, E. A. (2011). Dimensional approaches to psychiatric diagnosis in DSM-5. The Journal of Mental Health Policy and Economics, 14(4), 197-200.

Nigg, J. T., Karalunas, S. L., Feczko, E., & Fair, D. A. (2020). Toward a revised nosology for Attention-Deficit/Hyperactivity Disorder heterogeneity. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 5(8), 726-737. https://doi.org/https://doi.org/10.1016/j.bpsc.2020.02.005

Nigg, J. T., Sibley, M. H., Thapar, A., & Karalunas, S. L. (2020). Development of ADHD: etiology, heterogeneity, and early life course. Annual Review of Developmental Psychology, 2(1), 559-583. https://doi.org/10.1146/annurev-devpsych-060320-093413

Olorunniwo, F., Hsu, M. K., & Udo, G. J. (2006). Service quality, customer satisfaction, and behavioral intentions in the service factory. Journal of Services Marketing, 20(1), 59-72. http://dx.doi.org/10.1108/08876040610646581

Pérez, E. J. P., & García, C. P. (2007). El ASRS v. 1.1. como instrumento de cribado del trastorno por déficit de atención e hiperactividad en adultos tratados por conductas adictivas: propiedades psicométricas y prevalencia estimada. Adicciones, 19(4), 393-407. https://doi.org/10.20882/adicciones.298

Reckase, M. D. (1985). The difficulty of test items that measure more than one ability. Applied Psychological Measurement, 9(4), 401-412. https://doi.org/10.1177/014662168500900409

Regier, D. A., Kuhl, E. A., & Kupfer, D. J. (2013). The DSM‐5: Classification and criteria changes. World Psychiatry, 12(2), 92-98. https://doi.org/10.1002/wps.20050

Rios, J., & Wells, C. (2014). Validity evidence based on internal structure. Psicothema, 26(1), 108-116. https://doi.org/10.7334/psicothema2013.260

Rosler, M., Retz, W., & Stieglitz, R. D. (2010). Psychopathological rating scales as efficacy parameters in adult ADHD treatment investigations - benchmarking instruments for international multicentre trials. Pharmacopsychiatry, 43(3), 92-98. https://doi.org/10.1055/s-0029-1242819

Ruscio, J., Ruscio, A. M., & Carney, L. M. (2011). Performing taxometric analysis to distinguish categorical and dimensional variables. Journal of Experimental Psychopathology, 2(2), 170-196. https://doi.org/10.5127/jep.010910

Scandar, M. G. (2021). Validez y fiabilidad de las escalas ASRS y WURS-25 para el diagnóstico del trastorno por déficit de atención/hiperactividad en población argentina. Revista de Neurología, 72(3), 77-84. https://doi.org/10.33588/rn.7203.2019381

Slobodin, O., & Davidovitch, M. (2019). Gender differences in objective and subjective measures of ADHD among clinic-referred children. Frontiers in Human Neuroscience, 13, 441. https://doi.org/10.3389/fnhum.2019.00441

Somma, A., Borroni, S., & Fossati, A. (2019). Construct validity and diagnostic accuracy of the Italian translation of the 18-Item World Health Organization Adult ADHD Self-Report Scale (ASRS-18) Italian translation in a sample of community-dwelling adolescents. Psychiatry Research, 273, 753-758. https://doi.org/10.1016/j.psychres.2019.02.016

Swanson, J. M., Kinsbourne, M., Nigg, J., Lanphear, B., Stefanatos, G. A., Volkow, N., Taylor, E., Casey, B. J., Castellanos, F. X., & Wadhwa, P. D. (2007). Etiologic subtypes of Attention-Deficit/Hyperactivity Disorder: brain imaging, molecular genetic and environmental factors and the dopamine hypothesis. Neuropsychology Review, 17(1), 39-59. https://doi.org/10.1007/s11065-007-9019-9

Timmerman, M. E., & Lorenzo-Seva, U. (2011). Dimensionality assessment of ordered polytomous items with parallel analysis. Psychological Methods, 16(2), 209-220. https://doi.org/10.1037/a0023353

Ustun, B., Adler, L. A., Rudin, C., Faraone, S. V., Spencer, T. J., Berglund, P., Gruber, M. J., & Kessler, R. C. (2017). The World Health Organization Adult Attention-Deficit/Hyperactivity Disorder Self-Report Screening Scale for DSM-5. JAMA Psychiatry, 74(5), 520-527. https://doi.org/10.1001/jamapsychiatry.2017.0298

Valentini, F., & Damásio, B. F. (2016). Variância média extraída e confiabilidade composta: Indicadores de precisão. Psicologia: Teoria e Pesquisa, 32(2). https://doi.org/10.1590/0102-3772e322225

Vogel, S. W., Ten Have, M., Bijlenga, D., de Graaf, R., Beekman, A. T., & Kooij, J. S. (2018). Distribution of ADHD symptoms, and associated comorbidity, exposure to risk factors and disability: Results from a general population study. Psychiatry Research, 267, 256-265. https://doi.org/10.1016/j.psychres.2018.06.017

Willcutt, E. G. (2012). The prevalence of DSM-IV attention-deficit/hyperactivity disorder: a meta-analytic review. Neurotherapeutics, 9(3), 490-499. https://doi.org/10.1007/s13311-012-0135-8

Willcutt, E. G., Nigg, J. T., Pennington, B. F., Solanto, M. V., Rohde, L. A., Tannock, R., Loo, S. K., Carlson, C. L., McBurnett, K., & Lahey, B. B. (2012). Validity of DSM-IV attention deficit/hyperactivity disorder symptom dimensions and subtypes. Journal of Abnormal Psychology, 121(4), 991-1010. https://doi.org/10.1037/a0027347

Zohar, A. H., & Konfortes, H. (2010). Diagnosing ADHD in Israeli adults: the psychometric properties of the adult ADHD Self Report Scale (ASRS) in Hebrew. Israel Journal of Psychiatry and Related Sciences, 47(4), 308-315.