Antimicrobial stewardship promotes the appropriate selection, dose, route, and duration of antimicrobial therapy.¹ A recent Cochrane review demonstrated that antimicrobial stewardship programs were associated with a reduction in the duration of antimicrobial therapy, hospital length of stay, and possibly Clostridium difficile infection.² Hospitals often develop institution-specific treatment algorithms in the context of their local antibiogram as part of an antimicrobial stewardship intervention. However, multiple studies have shown that clinicians are usually unaware these resources exist or cannot easily access this information.^3,4

Smartphone use amongst health-care professionals has rapidly increased over the past ten years.^5,6 Approximately 80% of doctors and 85% of medical trainees use smartphones.⁷ For this reason, they represent an innovative opportunity in the field of medical education.^7-9 For example, a smartphone application, or 'app', that provides point-of-care information about local antibiotic resistance patterns and treatment guidelines could provide trainees accurate and up to date information from the patient's bedside. A previous study demonstrated that access to an antibiotic decision management guide on the internet improved prescribing accuracy amongst critical care fellows.⁸ One limitation of this study, and similar studies,⁹ is that these tools are seldom evaluated from a medical education standpoint. When medical apps have been evaluated, the studies have been limited by inadequate, or non-existent, control groups.^10-12 Given these issues, we developed a smartphone app and prospectively evaluated it.

This study aimed to assess the change in knowledge of prescribing antimicrobials (e.g., antibiotics, antivirals) among medical trainees. Specifically, our research question was: Does access to a smartphone app improve medical trainees' knowledge of antimicrobials compared to medical trainees without access to the smartphone app?  We also evaluated whether the app improved confidence in prescribing, made antibiogram and treatment data more accessible, and was easy to use. We hypothesized that the smartphone app would be associated with enhanced antibiotic-related knowledge, improved confidence in prescribing and that the information contained in the app would be accessible and easy to use.

Of the 85 medical trainees approached, 62 (73%) participated in the study. Most participants were male, approximately half were residents, and most used an Apple smartphone (Table 1). Of the residents, 17 (55%) were in first year, 8 (26%) in second year, and 6 (19%) in third year. This included 20 (65%) internal medicine residents, five family medicine residents (16%), one surgical resident 1 (3%); the remainder were from other specialties. The remaining participants were senior medical students.

At the time of study enrollment only 35% of participants were aware that St Michael's Hospital had an antibiogram. Amongst trainees in the control group, 20% reported the use of an antibiogram in the 30-days prior to enrolling in the study, and by the end of the study, 31% had used the St Michael's Hospital antibiogram. Amongst trainees in the app group, 22% reported the use of an antibiogram in the 30 days prior to enrolling in the study, and by the end of the study, 81% had used the St Michael's Hospital antibiogram.

The average number of questions answered correctly on the knowledge assessment test for the app group significantly improved over the duration of the study (6.2 points, SD=2.1 vs. 8.1 points, SD=2.2, t(26)=4.6, p=0.0001) but did not improve in the control group (7.1 points, SD=1.7 vs. 7.5 points, SD=2.0, t(25)=-1.2, p=0.23) (Table 2). In the unadjusted linear regression analysis, use of the app was associated with a 1.5 point (95% CI: 0.46, 2.48) [β = 1.46, t(1)=2.86, p=0.006] higher change in knowledge score compared to the control group. In the adjusted multivariable linear regression analysis, app use was associated with a 1.1 point (95% CI: 0.10, 2.1) [β = 1.08, t(1) = 2.08, p = 0.04] higher change in knowledge score compared to the control group (Table 2). Self-reported confidence in prescribing antibiotics improved over the duration of the study for the app group, but the change in confidence between the two groups was not statistically different in the unadjusted and adjusted analyses (Table 2).

Amongst app users, nearly 90% found it easy to use and 85% agreed that it was useful (Table 3). One-quarter of participants used the app daily or multiple times a day and 41% used it weekly.

In this prospective, controlled, pre-post study, use of an antibiogram and treatment algorithm app was associated with higher use of hospital-specific antibiogram data and a greater improvement in knowledge scores compared to the control group. Self-reported confidence in prescribing, however, "was similar for both groups."

Multiple past studies have demonstrated that there is a strong interest in, and use of, smartphone apps among medical trainees.^11,14-21 An increasing number of medical apps are being developed, but many are never evaluated to see what, if any, educational impact they have. Instead, many are simply studied to assess patterns of use (e.g., number of downloads, screen views).^14,20,21 A recent single-arm case-study of an antimicrobial prescribing app found that 100% of the target population had downloaded the application within 12 months and that 71% of the participants experienced a self-reported improvement in antibiotic knowledge after accessing the application.¹⁴ A similar single-arm study assessed the impact of a smartphone app to aid family doctors (N=14) in treating depression.¹¹ Their study found that smartphone app use improved the doctors knowledge of treating depression, but their study was limited by a small sample size (N=14) and a lack of a comparator group that did not have access to the app.¹¹ Similar studies exist for apps related to other medical subspecialties (i.e., hematology, infectious diseases, plastic surgery).^8,11,13,18

The advantages of our study were its prospective design and the inclusion of a control group. Both are crucial when trying to assess the impact of an educational program.^15,16 Without an adequate control group and a method that accounts for the change in knowledge over time for other reasons, any observed findings might be due to unmeasured confounding factors. In our study, the inclusion of this control group was important since we expected that knowledge related to antimicrobial prescribing might increase over the course of an internal medicine rotation, even without access to an app.

One explicit limitation of our study is that clinically relevant endpoints such as appropriateness of antimicrobial therapy were not measured. This endpoint was a priori deemed to be beyond the scope of our study and would have required a substantially larger sample size to power the study adequately. Furthermore, assessing the impact of antimicrobial stewardship strategies is challenging because the definition of "appropriate therapy" is varied and controversial. One potential endpoint that has been suggested is frequency of inappropriate use of antimicrobials, but the definition of "inappropriate" can be subjective and may require consensus by expert reviewers.¹

Our study has several other limitations. First, the knowledge assessment test was developed for the purposes of our study and has not been previously validated. Second, our study took place at one hospital which limits the generalizability of our results. Third, the duration of follow-up in our study was relatively short, and thus it is unknown whether any gains in knowledge were maintained long-term. Finally, since our study was not randomized, our results may partially be explained by unmeasured confounding factors, such as differential educational opportunities or exposure to infectious diseases related cases during the study period for the two groups.

A theoretical concern of using apps is that people might become reliant on the app and not retain important basic facts since they know they can rely on their app.^17,18 The results of our study showed the opposite. The knowledge assessment scores were unchanged for the control group and improved in the app group. This might be because participants in the app group were exposed to the content more frequently since it was literally at their fingertips. Since medical trainees often use electronic clinical resources during their training, it is reassuring to know that these can directly improve their medical knowledge. This highlights the importance of these resources being credible and providing accurate and up to date information.

Another theoretical concern of practitioners using apps is that they might overstate or overestimate what they know since the knowledge is readily accessible.^17,18 Our study demonstrated that the app group and the control group expressed a similar level of confidence in their knowledge of various infectious disease topics.

Antimicrobial stewardship requires clinicians to be aware of both local antibiotic resistance rates and hospital-specific treatment guidelines.^2,19 Hospitals invest a lot of time, energy, and money into developing these documents, but if they are not readily accessible, they often go unused. Our study found that the use of a smartphone app improved trainee rates of accessing antibiogram data and improved their knowledge about antibiotic prescribing. These findings reinforce the notion that smartphone apps can be a useful and innovative means of delivering medical education. In the case of this project, we created our app, but this is not necessary if there are existing high-quality apps with accurate, up to date, and relevant information. Considering the busy schedules of medical trainees, a final benefit of using apps as an adjunct to traditional forms of teaching (e.g., lectures, tutorials) is that they can be used seamlessly during clinical rotations.