The present study was designed as a cross-sectional study that was conducted in two universities in Colombia (one public and one private). In these universities, residents complete full-time training during two to five years (depending on specialty) and subsequently apply to sub-specialties (usually taking two more years). Residents pay a fee to the university but usually do not receive a salary from hospitals, though a few residents receive a semi-annual governmental scholarship. The national directive of duty hours is 66 hours per week.

For the current study, residents from both universities were invited to complete the D-RECT questionnaire. A total of 245 residents from 17 different surgical and non-surgical specialties completed the D-RECT questionnaire. Twenty-five questionnaires were removed from further analysis due to missing data, leaving 220 D-RECT evaluations for further analysis. The mean age of the respondents was 28.8 years (SD = 2.8), of which 120 respondents were female (54.5%), and 97 were male (44.1%). The distribution of respondents according to the postgraduate year of residency (PG-Y) was: PG-Y1 (46.8%), PG-Y2 (26.8%), PG-Y3 (19.1%) and PG-Y4 (6.8%). Table 1 provides further details on the study population.

We used the D-RECT as well as PHEEM to evaluate the learning climate in PGME. The most recent version of the D-RECT questionnaire is validated in the Netherlands and consists of 35 items covering nine domains (educational atmosphere, teamwork, the role of specialty tutor, coaching and assessment, formal education, resident peer collaboration, work is adapted to residents’ competence, accessibility of supervisors and patient sign-out). 20 Respondents could rate items using a 5-point Likert scale (1 = totally disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = totally agree).²⁰

The validated Spanish PHEEM questionnaire consists of 40 items covering 5 domains (perceptions of teaching and learning, teaching and social support/counselling opportunities, fairness of the program, autonomy and accommodation and catering facilities).¹⁰ Respondents could rate items using a 5-point Likert scale (1 = totally disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = totally agree).^10,12,22

An in-depth description of the domains as well as an overview of the conceptual overlap between the two instruments is presented in Appendix A1.

We contacted one of the authors of the original 35-item D-RECT to get authorization for using the questionnaire. We translated the English items into Spanish following the recommendations of the International Quality of Life Assessment project (IQOLA), which were previously applied to translate the SF-36 Health Survey from English into other languages.²³ Initially, two natively Spanish speaking researchers performed two independent forward translations from English into Spanish (LD, AS). Translations were compared and unified into a single version. Once completed, two certified bilingual translators performed two independent backward translations of from Spanish into English and subsequently, a third certified translator unified these translations into one version. Hereafter, one of the authors of the original 35-item D-RECT compared the resulting backward translation with the original version (MS). To conclude, the main researchers (LD, MS, AS) organized various meetings to clarify any discrepancies and to generate the final Spanish D-RECT. The researchers then conducted a pilot with twenty residents from different specialties in the Faculty of Medicine (Universidad de la Sabana, Colombia) to test the Spanish D-RECT and to appraise whether they found any difficulty, upsetting, or confusing items. Simultaneously the authors assess the process of response. The Spanish version of the D-RECT is available in Appendix A2.

We prepared a paper-based questionnaire for data collection, which consisted of the Spanish D-RECT, as well as the Spanish PHEEM questionnaire.¹⁰ We collected data during the weekly meetings of the postgraduate programs in both Colombian universities, during which residents were asked to complete the questionnaire individually. The first author coordinated the admission of the questionnaire during these meetings and was also responsible for the collection of completed questionnaires. Also, the purpose of the study, voluntariness to participate, anonymity, confidentiality and further management of information was explained to residents during these meetings. The data were collected from October to December 2015. Once completed, the first author coordinated the transcription and organization of data in order to guarantee the quality and integrity of information. The ethical approval for the present study was provided by the Commission of Medical Education (Facultad de Medicina, Universidad de la Sabana, Colombia).

Frequencies and descriptive statistics were performed to gain insight in relevant study sample characteristics. Resident evaluations that exceeded 50% missing data for the D-RECT or PHEEM were excluded. For both the D-RECT and the PHEEM, subscale scores were computed by averaging item scores. We allowed one item per subscale to have missing data. Similarly, mean total scores were calculated for both instruments. Pearson correlation was used to examine the correlation between the domains as well as mean total scores of both instruments. For the D-RECT data, remaining missing data were imputed using expectation maximization (EM). The internal consistency of the 9-factor structure was assessed by calculating Cronbach’s α. A Cronbach’s α exceeding 0.70 was considered satisfactory.²⁴ Corrected item-total correlations were computed to examine the homogeneity of each subscale. An item-total correlation above 0.40 was considered satisfactorily.²⁵ Subscales scored satisfactorily on overlap when inter-scale correlations remained under 0.70.

To assess the fit of the 9-factor structure for the Spanish version of the D-RECT, a confirmatory factor analysis (CFA) was performed. Robust maximum likelihood was used to estimate the CFA model. The fit of the model was assessed by combining the following fit indices: the comparative fit index (CFI), the Tucker-Lewis index (TLI), the standardized root mean square residual (SRMR) and the root mean square error of approximation (RMSEA). We used pre-determined cut-off values to assess the fit (CFI and TLI >0.95 for good fit and >0.90 for acceptable fit; SRMR <0.08 for good fit and <0.12 for acceptable fit and RMSEA <0.06 for good fit and <0.10 for acceptable fit).²⁶ The Lavaan package in R statistical software version 3.3.1 was used to perform the CFA.²⁷ All other analyses were performed in SPSS version 23 (IBM Corp. 2015).

Cronbach’s α for the nine subscales of the D-RECT were as follows: 0.78 (subscale ‘work is adapted to residents’ competence’), 0.84 (subscale ‘teamwork’, ‘resident peer collaboration’, and ‘formal education’), 0.86 (subscale ‘educational atmosphere’ and ‘accessibility of supervisors’), and 0.88 (subscale ‘role of specialty tutor’, ‘coaching and assessment’, and ‘patient sign-out’). Corrected item-total correlations ranged from 0.45 (Item “It is possible to do follow up with patients”) to 0.82 (Item “Differences of opinion are not such that they have a negative impact on the work climate”) ( Appendix A2). Inter-scale correlations ranged from 0.21 (subscale ‘patient sign-out’ versus subscale ‘resident peer collaboration’) to 0.76 (subscale ‘coaching and assessment’ versus ‘subscale role specialty tutor’) ( Appendix A3). Table 2 shows the results for the CFA and the original fit indices for the Dutch D-RECT. Appendix A2 shows the standardized and unstandardized coefficients and the corresponding standard errors for observed and latent variables.

Mean scores for the subscales of the D-RECT varied from 3.6 (subscale ‘patient sign–out’; SD = 0.9) to 4.3 (subscale ‘resident peer collaboration’; SD = 0.7). Detailed summary statistics for the subscales of the D-RECT are provided in Appendix A2. The D-RECT had an overall mean score of 3.9 (SD = 0.6). Mean scores for the subscales of the PHEEM varied from 3.3 (subscale ‘teaching and social support/counselling opportunities’; SD=0.6) to 4.0 (subscale ‘teaching and learning’; SD= 0.6). The PHEEM had an overall mean score of 3.8 (SD= 0.5).  Pearson correlation showed all PHEEM and D-RECT domains were significantly correlated, except for the PHEEM domain ‘perceptions of teaching’ and the D-RECT domain ‘resident peer collaboration’ (Table 3). A significant correlation between the overall mean scores of the 35-item D-RECT and the PHEEM (r = 0.7, p = <0.01).

The present study adds support for the 35-item D-RECT in a new context, namely the Spanish language. We recruited medical residents from different universities, including public and private institutions, with a wide distribution of specialties that are representative of common conditions in Latin America, guaranteeing the generalizability of the results. Furthermore, we followed a rigorous process of translation following international recommendations and pilot tested the Spanish items in a small group of residents before the start of data collection.

The main limitation of the present study is that the D-RECT was tested in a non-aggregated form, providing information on the resident level only. As these limitations are due to the sample size achieved in the study, researching a wider number of residency programs would support the generalizability of results to the department level. Furthermore, a larger sample could offer information about differences between programs: information the present study is not able to provide. Another limitation of the present study is the absence of exploratory factor analysis. Following the example of the original validation of the D-RECT, we preferred splitting the sample and used part of the sample for exploratory factor analysis and the other part of the sample to confirm the identified structure with a CFA. However, as the original validation procedure of the D-RECT already identified a solid structure by using a CFA and as the content of the D-RECT was applicable in the Colombian context without alterations, we identified a CFA as the preferred method to assess the psychometric properties of the Spanish instrument. Finally, although the results of the present study provide a Spanish version of the D-RECT that is valid and reliable, other Spanish speaking countries should thoroughly assess the instrument to decide whether the instrument applies to other Spanish contexts in terms of content.

This study contributes to the lack of instruments for evaluating the learning climate in the Spanish language. The 35-item D-RECT can now be added to the toolbox of valid learning climate instruments in residency training. The validation of this instrument offers a standardized and objective method to evaluate the learning climate in Spanish language countries and to compare its different educational contexts, for example, compare differences in performance between specialties. Furthermore, it provides a reliable tool to evaluate improvement after curricular and educational changes are done.^3,30-32 Ultimately, the instrument can be used for research into the effect of the learning climate on patient outcomes and health care system cost-effectiveness. A good instrument to measure the learning climate in residency training offers an opportunity to understand the internal elements of climate and its relationship with patient outcomes.

When educators and researchers wish to evaluate the learning climate, it is important that they assess their goals in order to pick the right instrument from the existed toolbox. Whether the PHEEM or the D-RECT is the preferred instrument might depend on the goals of evaluation, and as such, cautiousness is advised for proper selection. Above all, it is important to realize that instruments such as the PHEEM and the D-RECT need continuous attention. Similarly, further studies are needed to establish the concurrent validity of the Spanish versions of D-RECT and PHEEM against, for example, ACLEEM.¹³ The practice is changing all the time, and so should these instruments to adapt to these changes. As we stated above, the D-RECT should be tested in other Spanish speaking countries were conditions for residency training, and wellbeing during training could be affected by different health and educative systems. We call for further research in this area.

The authors declare that they have no conflict of interest.