The relationship between study strategies and academic performance

Introduction

Why do some students perform well academically in medical school while others do not? This question has been explored and investigated in a variety of ways. Pre-matriculation data such as MCAT scores and Undergraduate GPA (UGPA) have been used to predict success in medical school,^1,2 and so have certain study/learning strategies.^3-20

Identifying relationships between academic performance and study/learning strategies is important because it offers the opportunity to provide students with specific feedback related to strengths and weaknesses that can be used to help scaffold students’ learning.  Many relevant studies of pre-admission aptitude and study skills dealt with broad concepts and exhibited mixed results. For example, Sleight and Mavis created a Study Aid questionnaire and found high performers were less reliant on study aids.²¹ Lobb, Wilkin, McCaffrey, Wilson, and Bentley applied three commercial instruments and found that none of them assessed abilities directly related to academic performance.²² However, one of those instruments, the Learning and Study Strategy Inventory (LASSI) was utilized in West and Sadoski’s research and some of the LASSI subscales appeared to predict preclinical academic performance.⁶

Due to these findings, several questions emerged.  First of all, how should study/learning strategies be defined? In the existing literature, the LASSI was frequently used to measure students’ study/learning strategies. However, researchers, including those who utilized the LASSI, may not agree that its' ten domains encompass all “study strategies”. For example, self-directed readiness does not appear to be directly measured by the LASSI.  However, helping students develop into self-directed learners has been a focus in medical education since the Liaison Committee on Medical Education (LCME)²³requires that programs “foster the skills necessary for lifelong learning.”

Based on Cook and West’s²⁴ suggestions, a systematic review was conducted utilizing the following databases: PubMed, EMBASE, SCOPUS, PsycINFO, Web of Science, CINAHL, and ERIC  and keywords “study/learning strategy (strategies)”, “academic performance” and “LASSI”. Even though there were hundreds of articles, only 19 studies were relevant and were kept for further analyses (a comprehensive summary of the 19 studies are provided in the Appendix 1). The excluded articles included qualitative studies, quantitative studies that used advanced methods to investigate more complicated scenarios, or studies that contained the same keywords but were irrelevant. Of the 19 studies, most^{3-11,13,14,16,17,19,20} concluded that study strategies have an influence on academic performance as indicated by GPA^{3,14,16,19,20} school performance,^{5-8,15,17,18,22} or standardized exams4,^5,8,9 (see Appendix 1). A retrospective review of those results was difficult due to several reasons. First, some studies’ analyses may have been inappropriate.  For example, converting a continuous variable into a categorical variable (e.g., defining academic performance as low, medium, or high) was common in the reviewed articles^{3,4,7,8,15,18,25}. Another approach used percentile scores in the multiple regression models.²² However, percentile scores are not interval or ratio scaled and should not be utilized. Many methodologists^26,27 have suggested caution when utilizing these types of practices because they may damage the nature of the relationship and weaken the conclusions. Second, the discrepancies of analytical methods and models between studies did not facilitate a direct comparison. Multiple regression was the most common method, but ANOVA^8,20, ANCOVA^16,25, correlational analysis,^9,19 and logistic regression¹² was also utilized. Even those who adopted the same methods like multiple regression, used different predictors,^5,6,17,18,22 or predictors with different scales (i.e.,continuous scales^{5,6,10,11,14,17,22,28} vs. categorical scales^4,18,29 or different forms (i.e., original subscale scores^{4-6,10,14,17,18} vs. latent scores).^11,13  The beta weights were also not comparable between studies. Third, most studies failed to report the key components such as correlation matrices and standard deviations as suggested by Zientek and Thompson.³⁰ This limited cross comparisons and retrospective thinking. Fourth, the narrow definition of learning strategies, such as those which only included the domains measured by LASSI, restricted generalizability of the results. Similarly, how academic performance was defined (such as course grade, GPA, or standardized test scores) may have also influenced the results. Last, but not least, most studies only addressed whether or not there was an effect of study strategies on academic performance, but did not address to what extent the study strategies may affect academic performance. Even if there were many positive conclusions (i.e., yes, there was an effect), this would not lead us closer to estimating the magnitude of the effect.  The size of the effect is a more important indicator of the potential importance of using the LASSI in medical school.

Due to the listed limitations, we redesigned the study and re-conducted the research project in 2013. There were two primary changes.  First, in addition to the LASSI, we added the Self-Directed Learning Readiness Scale (SDLRS)³¹ as another measure due to the increasing emphasis on self-directed learning in medical schools.  The SDLRS was selected because it is the most widely used assessment to measure self-directed learning readiness,³² and we wanted to assess how learning readiness influenced student performance on specific tasks.^33-35 We also added another measurement point at the beginning of the second year (i.e., 2014), so we could examine the difference between the two measurements and the effect related to timing.

The purpose of this study was to compare LASSI and SDLRS outcomes and to determine if the measures yield academic performance predictors.  The research questions are:

   •   Does the LASSI and the SDLRS measure the same construct?

   •   Are LASSI and SDLRS scores predictive of academic performance?  If so, is one measure more valuable than another?

   •   Assuming the LASSI or the SDLRS are good predictors, how much variance of academic performance can be explained by LASSI or SDLRS scores?

The authors integrated current study data with published relevant research^5,6,10,14,17 and conducted a secondary analysis of the two articles that contained a bivariate correlation matrix and related descriptive statistics.^5,6 The aim of the integration was to find a more generalizable index (i.e., R²) other than null hypothesis testing that may enable educational practitioners to estimate the size of the effect of study strategies on preclinical academic performance.

Methods

Participants

The Institutional Review Board at Texas A&M University approved the study. A sample of 168 (67 males, 74 females, and 27 not indicated) out of 203 medical students at Texas A&M Health Science Center College of Medicine voluntarily participated in the study and signed consent forms. Students received access information for the LASSI and SDLRS during orientation and were given time during the session to complete the two learning assessments.  The sessions were held at the beginning of their first year of medical school and again at the beginning of their second year. Seven students did not complete the first LASSI, and 13 students did not finish the first SDLRS assessment. The numbers of students who did not complete the second LASSI and SDLRS were 20 and 58, respectively.

The researchers also conducted secondary analyses on the only two studies^5,6 that contain sufficient information to  enable secondary analyses. Three other studies^10,14,17 that reported partial information are also included in the summary table.

Data collection

The LASSI³⁶ includes 80 items rated on a five-point Likert-type scale.  Immediately upon completion, each participant receives a performance profile including 10 subscales: Anxiety, Attitude, Concentration, Information Processing, Motivation, Selecting Main Idea, Self-Testing, Study Aids, Test Strategies, and Time Management. The SDLRS  is a valid assessment³⁷ consisting of 58 items related to attitudes, skills, and characteristics to determine one’s current level of self-directed learning readiness.³¹  The SDLRS scores range from 58 (below average) to 290 (above average) with an average score of 214.³¹

Academic Performance Data included Medical College Achievement Test (MCAT), UGPA, Year I average, and Year II average. The MCAT overall score was utilized as well as overall UGPA.  The Year I average included performance in integrated courses including Gross Anatomy, Histology, Biochemistry, Genetics, Cell Physiology, and Pharmacology, as well as Introduction to Disease and Neuroscience.  The Year II average included performance in Hematology/Oncology, Cardiovascular, Respiratory, Renal/Genitourinary, Gastrointestinal/Nutrition, Endocrinology/Reproductive Science/Human Sexuality, and Integument/Musculoskeletal organ system blocks.

Data analysis

SPSS 22.0 was used to conduct the statistical analyses. Descriptive statistics such as means and standard deviations were calculated. A correlational analysis including scatterplots was conducted to understand the relationship between the LASSI and SDLRS and the instruments’ predictive validity. Regression analyses were utilized to determine the predictive value of the LASSI and SDLRS. Pairwise deletion strategy was selected due to its’ ability to handle missing data. To investigate the change in LASSI and SDLRS over the course of the first year of medical school, we utilized paired t-tests to analyze mean changes.  Further exploration of the LASSI included comparing correlations and R² s for six studies. 

Results

Table 1 includes means, standard deviations, and bivariate corelation coefficients for all targeted variables. The SDLRS was significantly correlated with all LASSI subscales in both the first and the second measure: the correlations ranged from r₍₁₅₂₎ = .255, p = .001 to r₍₁₅₂₎ = .592, p = .000 for the first measure and from r₍₁₀₅₎ = .276, p=.004 to r₍₁₀₅₎ = .517, p = .000.

Scatterplots also confirmed the linear bivariate positive correlation between the SDLRS and each LASSI subscale. The subscales “Selecting Main Idea” and “Information Processing” had the highest correlation with SDLRS in both measures.  Academic performance was always statistically significantly correlated with the MCAT and UGPA.  For Year I overall performance, the association with MCAT was r₍₁₂₆₎ = .212, p < 0.05 and the association with UGPA was r₍₁₃₀₎ = .308, p < 0.01. For Year II overall performance, the association with MCAT was r₍₁₃₀₎ = .248, p < 0.01 and the association with UGPA was r₍₁₃₅₎ = .250, p < 0.01.

When comparing the first and the second measures, the authors discovered scores on the LASSI for all ten subscales and SDLRS decreased. To detect to what extent the two measures changed, ten paired t-tests on the first and second LASSI subscales and one paired t-test for the first and second SDLRS tests were conducted. Results indicated that for the LASSI, eight subscales significantly decreased. They were: Attitude (Cohen’s d₍₁₄₇₎ = .37, p < .001), Concentration (Cohen’s d₍₁₄₇₎ = .38, p < .001), Information Processing (Cohen’s d₍₁₄₇₎ = .32, p < 0.001), Motivation (Cohen’s d₍₁₄₇₎ = .75, p < .001), Self-Testing (Cohen’s d₍₁₄₇₎ = .18, p = .028), Study Aids (Cohen’s d₍₁₄₇₎ = .37, p < .001), and Test Strategies (Cohen’s d₍₁₄₇₎ = .31, p < .001). The score for the SDLRS also significantly decreased in the post measure (Cohen’s d₍₁₀₄₎ = .42, p = .000). After the drop, the LASSI subscales’ scores were more closely associated with academic performance.  However, the change in scores on the second test did not make the SDLRS a better predictor.

The correlation between the LASSI subscales and students’ overall performance was not consistent in the first and second measures. For the first LASSI measure, only “Test Strategies” was positively correlated with Year I academic performance (r₍₁₅₅₎ = .172, p < .05). With the second LASSI measure, five subscales were statistically significantly correlated with Year II academic performance. They included: Concentration (r₍₁₃₉₎ = .173, p < .05), Motivation (r ₍₁₄₀₎ = .218, p < .01), Selecting Main Ideas (r₍₁₃₉₎ = .278, p < .01), Time Management (r₍₁₃₉₎ = .213, p < .05), and Test Strategies (r₍₁₄₀₎ = .249, p < .01).

Table 2 summarized ten regression models’ results with pre-matriculation variable(s) (MCAT and undergraduate GPA) and/or study strategy (LASSI and SDLRS) as predictors. The model started with pre-matriculation variables as predictors and then added the ten LASSI subscales or SDLRS to the pre-matriculation variables. The pre-matriculation variables were then removed but the LASSI or SDLRS were still used as predictors. The results show that MCAT and undergraduate GPA were statistically significant predictors for Year I academic performance (R²₍₁₃₁₎ = .090, p = .003; see Model 1) and Year II academic performance (R²₍₁₂₆₎ = .119, p = .004; see Model 4). With the existence of MCAT and UGPA, adding the SDLRS or LASSI as predictor(s) did not significantly increase the explained variance for Year I (see Models 3 and 5) or Year II (see Models 4 and 6). If MCAT and UGPA were not included, in the first measure, neither the SDLRS nor the LASSI is good predictor(s) (the increased explained variance (R²) ranged from .000 to .096; see Model 5 and 7). However, in the second measure, when MCAT and UGPA were excluded, the LASSI became a statistically significant predictor of student academic performance (Model 10: R²₍₁₃₆₎ = .188, p = .003). Although factor analysis could neither replicate the factor structure proposed by Weinstein and Palmer³⁸ nor replicate the factor structure proposed by Cano,¹³ Olaussen and Braten,³⁹ and Olejnick and Nist,⁴⁰ the results suggest  that students’ motivation and components of strategic learning contribute to academic performance.

The authors compared the current study to the systematically reviewed studies.  Of those, only five provided information that enabled a cross-study comparison. The two that reported enough data for secondary analyses were coincidently conducted at the same institution. The comparable information for all six studies (including the present study) is summarized in Table 3. Even though the six studies have variations by nature, the correlation matrices were relatively robust across studies. Concentration, Motivation, Time Management and Test Strategies were likely to be significantly correlated with outcome variables that measured academic performance. Anxiety was correlated occasionally.

Table 3 also provided the R² and adjusted R² for the multiple regression models with the ten LASSI subscales as predictors and the academic performance variable as the outcome variable. Based on the R² reported in the six studies, 10-49% of the total variance could be explained by the LASSI subscales. This means that the effect of learning strategies on academic performance is at least small, but it could be substantial.

Discussion

The SDLRS was statistically significantly correlated with the LASSI subscales. Someone who scores high on the SDLRS  is more likely to be able to: distinguish important information from less important information, connect new knowledge to prior knowledge, master preparation and test strategies, and have a positive attitude.³¹ However, the predictability of academic performance between the two scales was not equivalent.  Some LASSI subscales have higher correlation coefficients with academic performance than the SDLRS. Students who scored high on the SDLRS did not always perform well academically. The finding is consistent with a prior study which concluded that readiness for self-directed learning may not be necessary for learning foundational knowledge.³³ However, the LASSI appears to be a good predictor when undergraduate GPA and MCAT are not controlled. This may indicate that students with a higher GPA and MCAT score tend to have better learning strategies. Therefore, in a multiple regression analysis with GPA, MCAT, and LASSI all as predictors, the unique contribution from learning skills cannot be identified.

The results for the two measures have statistically significant differences. All scores decreased in the second measure.  The drop resulted in the LASSI subscales being better academic performance predictors. The second measure may be a more accurate reflection of the real situation because the first LASSI was administered before students started taking courses in medical school. Therefore, the initial results are more likely to reflect their success prior to medical school. After students started their medical education, study strategies were likely adjusted based on the requirements and pacing in the medical school environment. The difference between the two measures indicates that study strategies may be dynamic. Timing could also be a moderator that affects the estimate of the entire effect. Since the factor analysis for the LASSI did not replicate the factor structure proposed by previous research, those who utilize it need to be aware that the scale may lack some stability and may be vulnerable to the testing environment. However, the LASSI performance profile does enable students to see how they did on the ten subscales as compared to a norm group and then target those areas if improvement is needed. The retrospective review of the five relevant studies identified that Concentration, Motivation, Time Management and Test Strategies were likely to be correlated with academic performance, although the R² explained by all LASSI subscales vary in different cases. Future work is required to investigate what causes the fluctuation of explained variance. Possible factors could be inconsistencies in how academic performance is defined, how study strategies are measured, timing, administration procedures, and the testing environment. We considered the possible administration effect since the first LASSI measure was administered during orientation which was consistent with the other two research projects conducted at the same institution.^5,6 However, the correlation with academic performance in the current study is smaller than in the two previous studies.  Perhaps this is due to the fact that the SDLRS was added in the current study requiring students to spend more time in the testing environment which may have resulted in testing fatigue.

This study has several limitations. How to define and measure study strategies still cannot be fully addressed even after adding the SDLRS as a predictor. The results indicate large differences for the two measures, but with the current research design, we were not able to address the factors that may have caused the differences. The retrospective review ended up with only five useful articles even though hundreds of articles with the same keywords were accessed. Therefore, additional work is needed in the area of study strategies and academic performance.

Conclusions

Based on the statistical analyses of current data, the review of the LASSI and SDLRS research, and the summary data for the six relevant studies, LASSI subscales were non-negligible predictors for medical student academic performance in the preclinical curriculum. This study not only addressed whether or not there is an effect of study strategies on academic performance, but also addressed to what extent study strategies affect academic performance. Since a sufficient number of previous studies were not available, a complete meta-analysis was not conducted. Even though this is a limitation, broadening the scope of this study has served to provide more generalizable conclusions. 

In summary, we suggest utilizing the LASSI in medical school in combination with pre-matriculation data since the LASSI consistently yields academic performance predictors.  Those who score low in Concentration, Time Management, Test Strategies, and Motivation should receive targeted support in these areas when they enter medical school.  Completing another LASSI at the beginning of the second year of medical school would enable students to continue to monitor their study strategies and focus on specific weaknesses related to those strategies which appear to be predictors. Instilling the practice of study strategy monitoring via the LASSI will likely improve students’ academic performance and may facilitate the development of self-directed learning skills that “foster life-long learning”.²³

Acknowledgements

This study was funded in part by the Southern Group on Educational Affairs (SGEA) region of the AAMC through the Research in Medical Education (RIME)/Medical Education Scholarship, Research, and Evaluation (MESRE) Grant.

Conflict of Interest

The authors declare that they have no conflict of interest. 

Supplementary materials

References

1. Donnon T, Paolucci EO and Violato C. The predictive validity of the MCAT for medical school performance and medical board licensing examinations: a meta-analysis of the published research. Acad Med. 2007; 82: 100-106.
   Full Text PubMed
2. Silver B and Hodgson CS. Evaluating GPAs and MCAT scores as predictors of NBME I and clerkship performances based on students' data from one undergraduate institution. Acad Med. 1997; 72: 394-396.
   Full Text PubMed
3. Schutz CM, Gallagher ML and Tepe RE. Differences in learning and study strategies inventory scores between chiropractic students with lower and higher grade point averages. J Chiropr Educ. 2011; 25: 5-10.
   Full Text PubMed
4. Schutz CM, Dalton L, Tepe RE. Learning and study strategies inventory subtests and factors as predictors of national board of chiropractic examiners part 1 examination performance. J Chiropr Educ. 2013;27:5-10.
5. West C, Kurz T, Smith S and Graham L. Are study strategies related to medical licensing exam performance? Int J Med Educ. 2014; 5: 199-204.
   Full Text PubMed
6. West C and Sadoski M. Do study strategies predict academic performance in medical school? Med Educ. 2011; 45: 696-703.
   Full Text PubMed
7. Yip MCW. Learning strategies and their relationships to academic performance of high school students in Hong Kong. Educational Psychology. 2013; 33: 817-827.
   Full Text
8. Yip MCW and Chung OLL. Relationship of Study Strategies and Academic Performance in Different Learning Phases of Higher Education in Hong Kong. Educational Research and Evaluation. 2005; 11: 61-70.
   Full Text
9. Albertini JA, Kelly RR and Matchett MK. Personal factors that influence deaf college students' academic success. J Deaf Stud Deaf Educ. 2012; 17: 85-8101.
   Full Text PubMed
10. Loong TE. Self-regulated learning strategies and pre-university math performance of international students in Malaysia. Journal of International Education Research. 2012;8:223-232.
11. Moliterni P, De Stasio S, Carboni M, Di Chiacchio C. Motivational and self-regulated learning components of academic performance. 39th EUCEN Conference; Rovaniemi, Finland; 2010.
12. Gatto SL. Learning and Study Strategies of Baccalaureate Nursing Students during First Semester Nursing Courses [Doctoral dissertation]. Little Rock, AR: University of Arkansas for Medical Sciences; 2010.
13. Cano F. An In-Depth Analysis of the Learning and Study Strategies Inventory (LASSI). Educational and Psychological Measurement. 2006; 66: 1023-1038.
    Full Text
14. Lipsky S. Correlating the LASSI with developmental students' academic performance. 1999 [cited 5 Oct 2016]; Available from: http://files.eric.ed. gov/fulltext/ED428632.pdf#page=51.
15. Gilles LT. The assessment of student study skills and learning strategies to prepare teachers for academic advising tasks or the prevalidation of motivated strategies for learning questionnaire (MSLQ), learning and study strategies inventory (LASSI), and test of reactions and adaptation to college (TRAC). 1994 [cited 5 Oct 2016]; Available from: http://files.eric.ed.gov/fulltext/ED372035.pdf.
16. Ickes C, Fraas JW. Study skills differences among high-risk college freshmen. The Annual Meeting of the Mid-Western Educational Research Association; Chicago, IL: Ashland University; 1990.
17. Kellogg SD, Durben D, Ayars-Junek S. Critical factors for success in an introductory astronomy class. 34th ASEE/IEEE Frontiers in Education Conference; Savannah, GA: IEEE; 2004.
18. Franciosi LP. Problem-solving appraisal, self-reported study strategies, and academic performance of first-year college students [Dissertation]. Milwaukee, WI: Marquette University; 1996.
19. Hanlon O'Connell KA. Predictive validity of the learning and study skills inventory on developmental community college student success [Doctoral dissertation]. San Diego, CA: Northcentral University; 2014.
20. Kus-Patena ST. The effects of learning strategy instruction on college students who are placed at risk of failure [Dissertation]. Akron, OH: University of Akron; 2003.
21. Sleight DA and Mavis BE. Study Skills and Academic Performance among Second-Year Medical Students in Problem-Based Learning. Medical Education Online. 2006; 11: .
    Full Text
22. Lobb WB, Wilkin NE, McCaffrey DJ, Wilson MC and Bentley JP. The predictive utility of nontraditional test scores for first-year pharmacy student academic performance. Am J Pharm Educ. 2006; 70: 128-97.
    Full Text PubMed
23. Liaison Committee on Medical Education. Functions and structure of a medical school: standards for accreditation of medical education programs leading to the MD Degree 2003 [cited 28 July 2013]; Available from: https://members.aamc.org/eweb/upload/LCME%20Standards%20May%202012.pdf.
24. Cook DA and West CP. Conducting systematic reviews in medical education: a stepwise approach. Med Educ. 2012; 46: 943-952.
    Full Text PubMed
25. Libutti DD. First-year academic success: Differences in pre-entry and learning and study skill characteristics for academically successful and unsuccessful students at a public flagship university in New England [Dissertation]. Providence, RI: Johnson & Wales University; 2005.
26. Altman DG. Categorizing continuous variables. Encyclopedia of Biostatistics. New York, NY: John Wiley & Sons Ltd; 2005.
27. Dinero TE. Seven reasons why you should not categorize continuous data. J Health Soc Policy. 1996; 8: 63-72.
    Full Text PubMed
28. Higgins DM, Peterson JB, Pihl RO and Lee AG. Prefrontal cognitive ability, intelligence, Big Five personality, and the prediction of advanced academic and workplace performance. J Pers Soc Psychol. 2007; 93: 298-319.
    Full Text PubMed
29. Ning HK and Downing K. Connections between learning experience, study behaviour and academic performance: a longitudinal study. Educational Research. 2010; 52: 457-468.
    Full Text
30. Zientek LR and Thompson B. Matrix Summaries Improve Research Reports: Secondary Analyses Using Published Literature. Educational Researcher. 2009; 38: 343-352.
    Full Text
31. Guglielmino LM. Development of the self-directed learning readiness scale [Doctoral dissertation]. Athens, GA: University of Georgia; 1977.
32. Merriam SB, Caffarella RS, Baumgartner LM. Learning in adulthood: a comprehensive guide. 3rd ed. San Francisco, CA: John Wiley & Sons; 2012.
33. Deyo ZM, Huynh D, Rochester C, Sturpe DA and Kiser K. Readiness for Self-directed Learning and Academic Performance in an Abilities Laboratory Course. Am J Pharm Educ. 2011; 75: 25.
    Full Text
34. Lai H. The influence of adult learners' self-directed learning readiness and network literacy on online learning effectiveness: a study of civil servants in taiwan. J Educ Techno Soc. 2011;14:98-106.
35. Abraham RR, Fisher M, Kamath A, Izzati TA, Nabila S and Atikah NN. Exploring first-year undergraduate medical students' self-directed learning readiness to physiology. Adv Physiol Educ. 2011; 35: 393-395.
    Full Text PubMed
36. Weinstein P and Gipple C. The relationship of study skills to achievement in the first two years of medical school. Academic Medicine. 1974; 49: 902-5.
    Full Text
37. Delahaye BL and Smith HE. The Validity of the Learning Preference Assessment. Adult Education Quarterly. 1995; 45: 159-173.
    Full Text
38. Weinstein CE, Palmer DR. LASSI user's manual: for those administering the learning and study strategies inventory. Clearwater, Florida: H & H; 2002 [cited 28 July 2013]; Available from: http://www.hhpublishing.com/_assessments/lassi/lassi_users_manual.pdf.
39. Olaussen BS and Bråten I. Identifying Latent Variables Measured by the Learning and Study Strategies Inventory (LASSI) in Norwegian College Students. The Journal of Experimental Education. 1998; 67: 82-96.
    Full Text
40. Olejnik S and Nist SL. Identifying Latent Variables Measured by the Learning and Study Strategies Inventory (LASSI). The Journal of Experimental Education. 2014; 60: 151-159.
    Full Text