Psychometric properties of the online global symptoms scale (ESGO) Propiedades psicométricas de la Escala de Síntomas Globales Online (ESGO)

With the development of new technologies in the field of psychotherapy comes the need for valid and reliable tests that allow us to research the presence of symptoms in an online format. The aim of the following paper was to develop a scale to be used in the virtual format, which allows us to measure symptomatology for anxiety, depression, interpersonal relationships and psychoticism. 550 adults participated, who answered an online questionnaire that included four dimensions (anxiety, depression, interpersonal relationships, psychoticism). Results shows an adequate internal consistency and construct validity for the four dimensions tested. Also, for the external validity significant relationships were found whit the OQ-45 and SA-45. The test we constructed seems to be a valid tool for psychotherapeutic use. However, it is necessary to continue analyzing its psychometrics, taking into account patients who have the specific diagnoses evaluated by the test.


Introduction
Mental health in global terms is facing a significant crisis with respect to previous years, reflected in the still growing prevalence rates (World Health Organization, 2015). In this regard, the data pertaining to depression, anxiety, and other mental disorders are truly alarming (Baxter et al., 2014). For these reasons, psychotherapy, along with other practices such as meditation and religion, appear as a way to cope with this reality, which has grown in the last thirty years on a large scale (Weissman & Cuijpers, 2017). So, during the last century, the world has witnessed a significant increase in the pursuit of symptomatic relief, as well as in the range of offers (Fernandez Alvarez, Gomez, & Garcia, 2013).
In relation to psychotherapy as the more widely consumed treatment worldwide by those suffering from emotional disorders, Olfson and Pincus (1994) found that 80% of outpatient consultations in the United States provided by mental health professionals included a psychotherapeutic intervention. In this line, more recent research argues that the figures have remained the same (Mojtabai & Olfson, 2010).
According to the above, the follow-up of the psychological state of a consultant who is undergoing treatment is one of the most important and complex issues in a psychotherapeutic process (Feltham, Hanley, & Winter, 2017). Therefore, and in response to the challenges that have been raised regarding the validity of the evaluation of psychotherapy, emphasis has been placed on the consideration of the multiple variables that influence this process, on developing measurable operational criteria, and on having a sample of patients that allows us to obtain significant results (Valdivieso, 1994). In this sense, despite the considerable bias that comes with evaluating a patient, clinical observation has been one of the techniques most often used in the context of psychotherapy, along with psychological tests, which may be useful tools to detect mental changes in patients at the start or during the treatment process (Richards & Bergin, 2014). The latter provide the advantage of standardization, as well as the reduction of bias and the collection of information through a different method (Kline, 2013).
Receiving feedback from patients during the psychotherapeutic treatment is a real, but skewed and distorted process due to the cognitive heuristics and schemes displayed by the very same therapist (Blumenthal-Barby & Krieger, 2015). That is why, as several researchers of the psychotherapeutic process have argued (Gelo & Manzo, 2015;Hardy, & Llewelyn, 2015), the informal feedback that the therapist receives needs to be complemented by a standardized assessment system that is reliable. This leads to the measurement of the quality of the psychotherapy, as well as in the therapist's professional development (Kelley & Bickman, 2009).
The evaluation of the individual changes during a psychotherapeutic process has been a significant issue in the research into psychotherapy (Gelo & Salvatore, 2016). In this sense, different methods were developed over the years such as the Achievement of Therapeutic Objectives Scale, in which patients are evaluated at the end of the session. However, in relation to the inclusion of evaluation in psychotherapy, it was unusual for these evaluations to take place outside the scope of behavior therapists. According to Lambert (2013), assessment procedures over the course of a psychotherapeutic treatment could facilitate symptomatic reduction during the same. In this way, not only would the proposed objective be achieved, but it would also benefit those patients who see a constant improvement, as well as the positive bond with the therapist and the psychotherapeutic process. Additionally, the possibility to continuously evaluate the therapeutic process would offer mental health professionals a key tool to consider the patient's process, as well as also implement the changes they deem pertinent in view of the effectiveness of the treatment, which would be reflected in the symptomatic relief of the consultant (Levendosky & Hopwood, 2017).
Many of the current techniques (for example, the SCL-90, OQ-45, SA-45), investigate symptoms of anxiety, depression, interpersonal relations, and psychoticism, among others, which allow a first general screening of the consultant (Lara Muñoz, Espinosa de Santillana, Cárdenas, Fócil, & Cavazos, 2005). In this line, one of multidimensional self-report questionnaires that is most widely used for the overall assessment of mental health symptoms is the SCL-90 (Derogatis, & Cleary, 1977). This assessment was designed to allow the selfassessment of various dimensions of psychopathology, both in psychiatric patients and the general population. Although these instruments seem promising in the evaluation of the mental state of seriously ill patients, since they contain specific scales to measure psychotic and schizotypal symptoms, like the psychoticism or paranoid ideation scale, their validity has not yet been confirmed. For example, Wood (1982) found no evidence that patients with schizophrenia obtained higher scores on the psychoticism scale than patients without schizophrenia. For their part, Johnson, Chipp, Brems, and Neal (2008), found no difference in the psychoticism and paranoid ideation scales among patients diagnosed with schizophrenia and those without this diagnosis. In order to overcome these limitations, years later, Davison et al. (1997) developed the Symptom Assessment-45 Questionnaire (SA-45). The objective of this version was, while keeping the same dimensions of the SCL-90, to reduce the length of the questionnaire by half and ensure that all dimensions have the same number of items to obtain better psychometric indicators, while avoiding overlaps.
Another instrument used to evaluate symptoms in mental health is the Outcome Questionnaire -45 (OQ-45) (Lambert et al., 1996). It is a self-report created for patients undergoing psychotherapy, consisting of three subscales: general symptoms, interpersonal relations, and a subscale for social role, which measures fitness for work, study, and leisure. The OQ-45 evaluates the progress of the patient undergoing therapeutic treatment, designed to be applied during the therapeutic process and at the end of it. It is a brief scale that has shown good psychometric properties in terms of reliability and validity, applicable to different populations (Doerfler, Addis, & Moran, 2002). Although this instrument is not used to establish diagnoses, its design targets decision making (Von Berger & De la Parra, 2002).
From the development of new technologies in the field of psychotherapy arises the need for valid and reliable evidence in which to investigate the presence of symptoms from an online format (Clough, & Casey, 2015;Titov et al., 2015). For over three decades, there have been comparative studies between psychometric assessments through the use of both online and face-to-face formats, proving that the former was deemed to be as accurate and reliable as the latter (Lexcen, Hawk, Herrick, & Blank, 2006;Vallejo, Jordán, Díaz, Comeche & Ortega, 2007). Moreover, it has been evidenced that online assessments relating to clinical aspects allow an adequate diagnostic discrimination, with low rates of false positives (Hyler, Gangure, & Batchelder, 2005).
In this context, and due to the lack of free access to psychometric instruments with the adequate psychometric properties, the goal of the present study was to develop the Online Global Symptoms Scale (Escala de Síntomas Globales Online or ESGO), which would allow us to study four dimensions: depression, anxiety, psychoticism, and interpersonal relations. It is envisioned as an instrument of self-report, whose purpose is to become a psychological resource for gathering information about the internal states of the individual initiating treatment or for the professional who needs to assess patients.

Participants
The participants were selected through nonprobability purposive sampling. A total of 550 adults participated in the study, belonging to the general, non-clinical population, residents of the autonomous city of Buenos Aires, between 14 and 73 years of age (M = 36.6; DT = 13.08) and of both sexes. Of the total number of participants, 17.46% positioned themselves in a middle to high socio-economic level, 64,72% in a middle level, and 17.82% in a low level. Moreover, of the total number of participants, 24.91% indicated that they had undergone or were undergoing psychological treatment, while 75.09% stated they had never had any type of psychological treatment.

Data Collection Techniques
-Online Global Symptoms Scale (ESGO): To assess the construct, we proceeded to carry out the construction and validation of the ESGO, composed in its final version of 20 items that address the four dimensions that comprise the global assessment of symptoms presented by individuals. Each of the dimensions that comprise the construct were represented by a series of five items: for Depression (for example, "I often feel that there is no solution to my problems"; "I don't enjoy things as much as I used to"), Anxiety (for example, "At times I feel intense fear, and I think I'm going to die"; "I have sensations in my body that nobody can understand"), Interpersonal Relations (for example, "I tend to worry that I'll be judged negatively"; "I sometimes feel I'm less than my peers or friends"), and the fourth dimension, Psychoticism (for example, " I sometimes hear voices that others don't hear"; "I sometimes feel that an external force is trying to control me.") We propose to refer to it as a scale of global symptomatology, as the different items explored, although they were developed so as to represent a specific dimension, may also represent other pathologies. Moreover, the use of the term "global" allows us to show that the goal of the assessment is to approximate a description of clinical symptoms, but that this in no way results in a diagnostic presumption. The assessment is aimed at individuals over the age of 14 and of both sexes, its scope being the fields of clinical psychology and health. The format was a Likert scale questionnaire with five points, ranging from 1 = Strongly Disagree to 5 = Strongly Agree (the same answer format was used for the other scales used in this study).
-OQ-45 (Outcome Questionnaire-45): We used a version adapted for this study from the original version created by Lambert et al. (1996). The questionnaire is composed of 45 items grouped in three dimensions: Symptom Distress (SD), Interpersonal Relations (IR), and Social Role (SR), whose scores make up a total score for the scale. The instrument has shown a high internal consistency for use with patients in its original version, both for the totality of the scale (α = .93) and for each of its dimensions (SD = .91; IR = .74; SR = .71). However, for the present study, only the dimensions of Symptom Distress and Interpersonal Relations were considered in order to establish relationships with the ESGO.
-Socio-demographic Data Questionnaire: To collect this type of information, we inquired about sex, age, self-perceived socio-economic level, and if the individual has undergone or is currently under psychological treatment.

Procedure
For the construction of the ESGO and its validation within the Argentine context, we took into account the international methodological standards recommended by the International Test Commission (ITC) for the creation of new assessment instruments (Muñiz, Elosua, & Hambleton, 2013).
The initial items were purged until we reached a version with 60 reagents. Subsequently, it was presented to a panel of expert judges. After analysis from expert judges, regarding the correspondence of each item to the dimension being evaluated through Cohen's kappa coefficient (K = .73), a pilot test was conducted of the preliminary version of the ESGO with a small group in order to corroborate the degree of understanding of the reagents.
In a second phase, the items were edited based on the recommendations obtained from the judges.
Consent was requested from the participants. They were informed about the anonymity of their responses, the voluntary nature of their participation, the strictly scientific use of the information collected, and the protection of their personal data was guaranteed in compliance with the provisions set forth in the national law number 25.326 regarding the protection of personal data.

Data Analysis
The statistical analyses that guided the development of this study were carried out with the use of the SPSS software for Windows, version 19.0 (George & Mallery, 2010) and the EQS 6.1 program (Bentler, 2007) for the development of confirmatory factor analysis (CFA) of the structure of the ESGO.
Additionally, we analyzed the descriptive statistics for each of the items that make up the final version of the scale (mean, standard deviation, skewness, and kurtosis). Subsequently, two factor analyses were carried out to account for the construct validity of the assessment, one exploratory (EFA) and the other confirmatory (CFA), in the latter case using the maximum likelihood estimation to test the adjustment of the data to the model of four correlated dimensions. Next, we analyzed the internal consistency of the instrument through the use of Cronbach's alpha and, finally, with the goal of evaluating the validity of the criteria, we assessed the association between the ESGO, the SA-45 and the OQ-45 through the use of Pearson's r.

Results
First, the descriptive statistics for the scale were analyzed, mainly the mean, the deviation, the correlation between the item and the total, and the alpha if item deleted (table 1).
As seen in table 1, all descriptive statistics, as well as the indicators of internal consistency (.67 < < .83), turned out to be adequate, with the exception of the distribution of the items in the Psychoticism dimension, which is to be expected due to the low rate of occurrence of this symptomatology in the general population.
Next, a confirmatory factor analysis was conducted to analyze the construct validity of the scale. For this purpose, a model of four correlated dimensions was proposed, which turned out to be adequate (X 2 (164) = 574,916; p < .001; CFI = .91; AGFI = .90; RMSEA = .068 [.062 -.074]), thus corroborating the construct validity of the scale.
Subsequently, the relationships between the dimensions of the scale and other related variables included in the SA-45 and the OQ-45 were analyzed. Moreover, the relationships between all the variables and the participants' ages were studied (table 2).
As seen in table 2, the dimensions of our scale are largely related to the corresponding scales of the SA-45 and the OQ-45 in all cases. Additionally, low but significant relationships were observed with the participants' ages, except for interpersonal relations, whose strength was a bit higher.
Subsequently, the levels of the four dimensions of the assessment were analyzed with respect to the having undergone therapy and not having done it at all (table 3).
According to the results presented in table 3, statistically significant differences were seen in the dimensions of Depression and Anxiety, but not for Interpersonal Relations and Psychoticism.

Discussion
In order to address the needs identified by different authors on the subject (Clough, & Casey, 2015;Titov et al., 2015), the goal of the present research was to develop an online global symptoms scale that would allow us to assess symptoms of anxiety, depression, interpersonal relations, and psychoticism.
Firstly, the analyses of the reliability and the validity of the ESGO were carried out. As in most of the classic assessments considered for analysis of psychopathological symptomatology (Davison et al., 1997;Derogatis & Cleary, 1977;Doerfler et al., 2002), Doerfler et al., 2002, the adequate psychometric properties were obtained, and this led to a factor structure composed of four dimensions: depression, anxiety, relations, and psychoticism. For each of the dimensions, internal consistency was adequate, the subscale for psychoticism being the lowest (α = .67). The correlations between each of the items and its factor were adequate, with the criteria proposed by Hair, Black, Babin, Anderson, and Tatham (2010) as a reference. These results indicate that, unlike other forms of assessment of symptomatology, the ESGO presents itself as a valid and reliable tool that allows an adequate discrimination of different dimensions of symptoms.
Secondly, it was noted that the model of four correlated dimensions of symptomatology showed a suitable adjustment to the sample data. Moreover, after analyzing the associations between each of the dimensions of the ESGO, we observed that they are significant between each of them, although as the prior literature suggests (Johnson et al., 2008;Wood, 1982), they are lower for psychoticism.
There were relationships between the different dimensions of the ESGO, both with the comparable dimensions of the SA-45 and the OQ-45. For example, the strongest relationships were observed between the depression dimension of the ESGO and the depression dimension of the SA-45 (r = .79) and with the symptom distress dimension of the OQ-45 (r = .80). In this same vein, it was noted that the strongest relationships for the anxiety dimension of the ESGO occurred with the anxiety dimension of the SA-45 (r = .76) and with the symptom distress dimension of the OQ-45 (r = .74). Regarding the relations dimension, the ESGO presented strong relationships with the interpersonal relations dimension of the SA-45 (r = .70), as well as with the same dimension of the OQ-45 (r = -.54) (the relationship is negative because the OQ-45 scale items are inverted in this dimension).
Lastly, the psychoticism dimension of the ESGO had a stronger relationship with its correlates in the other two assessments, both with the psychoticism dimension of the SA-45 (r = .59), and with symptom distress of the OQ-45 (r = .41). Therefore, we may argue that the results obtained in the present study, as well as the construct validity, provide sufficient evidence of the ESGO's concurrent validity, as it is compared to similar dimensions present in two instruments that are widely used in the scientific literature for the assessment of symptomatology.
As for the participants' ages, the relationships were significant but low, except for Interpersonal Relations, whose value was the highest obtained.
In the Depression and Anxiety dimensions, statistically significant differences were seen with respect to whether the participant had previously undergone therapy or not, but not with respect to Interpersonal Relations and Psychoticism. More specifically, the subjects with previous experience with psychotherapy, presented higher levels of anxiety and depression than those who did not have previous experience.
The study contributes to the assessment of symptomatology through the creation of an instrument in Spanish, one that is valid and reliable, but which can also be used online and with open access (free). This work was carried out only with a sample of the general, nonclinical, population. Due to this limitation, it is recommended that studies be carried out considering different populations to achieve a greater generalization and representativeness of the results, including subjects with different psychopathological diagnoses. Finally, while there is ample evidence that online and faceto-face assessments are equally accurate and reliable (Lexcen et al., 2006;Vallejo et al., 2007), it is necessary to continue the evaluation of the psychometric properties of the ESGO in a paper format to compare the psychometric properties of both formats.