Introduction
The coronavirus disease 2019 (COVID-19) pandemic impacted all activities, including provision of medical education.1 Medical schools stopped clinical placements for students due to risk of COVID-19 exposure and junior doctors faced disruption to in-person teaching as a result of social distancing requirements.2 Consequently, the delivery of medical education largely transitioned from traditional face-to-face teaching methods to online learning.3 However, despite the educational value and broad range of available online resources, certain aspects of clinical skills are difficult to acquire without real-life practice. Pedagogical developments and innovations are needed to address this.
Recently, simulation-based learning (SBL) has been increasingly used to supplement real-life clinical experience. It is a useful teaching modality where learners are guided through the experience in a safe environment, thus improving clinicians’ competence without compromising patient care.4 While simulation has traditionally occurred via face-to-face role-play, many of its principles are adaptable to remote learning.
Simulation via Instant Messaging – Birmingham Advance (SIMBA) was our response to the sudden and complete disruption in medical education and training.5 Learners go through simulation via WhatsApp, followed by discussion of the simulated cases via Zoom. The discussions are chaired by experts who highlight key learning points and foster engagement and interactivity. This study aimed to explore the effectiveness of SIMBA to sustain medical education for various medical specialties during the COVID-19 pandemic.
Methods
SIMBA simulation
Five SIMBA sessions were conducted from May 2020 to August 2020. The sessions’ themes were adrenal, thyroid, and pituitary pathologies, inflammatory bowel disease, and acute medical presentations. For each session, consultants selected five real-life cases reflecting their routine clinical practice. Anonymised transcripts that included patient history, examination findings, investigation results, and multidisciplinary team (MDT) management plans were prepared. Specialist radiologists approved imaging findings for the transcripts where applicable. All transcripts were finally approved by an expert for the relevant session.
Participants were recruited via advertisements on social media (Twitter, Facebook) and mailing lists of endorsing societies. Two days before the session, participants were assigned their unique SIMBA IDs to keep personal information confidential and were matched with a moderator via WhatsApp.
‘The moderators’, mainly medical students, used transcripts to simulate the cases over WhatsApp. They responded to participants’ questions by copying and pasting the information from the transcripts into their chats with each participant. Prior to each session, the moderators were trained to run the simulation to ensure standardised replies were given to the participants. In case a participant requested information that was not provided in the transcript, moderators replied with ‘the information you have requested is unavailable’. In addition, each session commenced with a mock case, for participants to familiarise themselves with the SIMBA model and receive feedback on their performance.
For each case, participants went through a simulation reflecting routine clinical practice of eliciting patient’s medical history, followed by relevant clinical examinations. After this, they requested blood tests, imaging and other investigations via an online form. Moderators would then provide all available results for that case as done in real life. Results to investigations were provided either as images to interpret or as written reports, depending on the nature of the investigation in question. Upon receiving the necessary information, participants were asked to provide a differential diagnosis and initial treatment plan. The participants were then provided with the real-life MDT decisions on the diagnosis and further management, followed by a step-by-step development of a scenario until a definitive management was established. Following simulation, participants joined an interactive session via Zoom where an expert discussed the evidence-based management of the cases and highlighted the important learning points.
Data collection
Participants were asked to complete pre- and post-SIMBA surveys, which collected participants’ demographics including country of work and level of medical training. Participants rated their confidence levels in approaching each simulated clinical scenario before and after the session. Self-reported confidence levels were measured using a 7-point Likert scale, with answers ranging from ‘strongly agree’ to ‘strongly disagree’. They also provided feedback on the session, moderator’s performance, and the consultant’s discussion in the post-SIMBA survey. Data were anonymised prior to analyses.
Statistical analyses
Participants who completed both pre- and post-SIMBA evaluation forms were included in the analysis. Participants’ self-reported confidence levels were categorised into three groups: (i) confident: those who responded with ‘strongly agree’ or ‘agree’; (ii) unsure: those who responded with ‘agree somewhat’, ‘undecided’, or ‘disagree somewhat’; (iii) not confident: for those who responded with ‘strongly disagree’ or ‘disagree’. Statistical analysis was performed using Stata (Stata/SE 16.0). Wilcoxon signed-rank test was used to investigate statistical differences between confidence levels of matched pairs pre- and post-SIMBA. Statistical significance was accepted at 95% confidence level (p<0.05). Data from close-ended questions are reported in frequencies and percentages. Responses to open-ended questions were reviewed and an inductive thematic analysis was performed. Common themes were identified and presented in tables with examples. Additionally, data from all five sessions were combined, and the above-described analysis was performed.
Results
A total of 204 participants completed both pre- and post-SIMBA evaluation forms. Of these, 129 (63.2%) participants were UK based, and the rest international [Europe (n = 36), Asia (n = 23), Africa (n = 14), South America (n = 1), Oceania (n = 1)]. The participants consisted of consultants (n = 24), specialty trainees (n = 138), core medical or internal medicine trainees (n = 10), foundation doctors (n = 7) and medical students (n = 14). Nine participants were in a non-training job and two participants did not provide information on their level of training.
Significant improvement was observed in participants’ self-reported confidence post-SIMBA in their approach to the simulated scenarios (all sessions combined/overall n = 204, p<0.0001; adrenal n = 33, p<0.0001; thyroid n = 37, p<0.0001; pituitary n = 79, p<0.0001; inflammatory bowel disease n = 17, p<0.0001; acute medicine n = 38, p<0.0001) (Figure 1).
Figure 1 Illustration of improvement in participants’ confidence levels in their approach to the simulated cases, post-SIMBA, in all sessions individually and combined/overall (p<0.0001). SIMBA, Simulation via Instant Messaging – Birmingham Advance
A total of 94.6% (n = 193/204) strongly agreed/agreed that simulated topics were applicable to their practice. 92.6% (n = 189/204) found the session engaging, while 89.7% (n = 183/204) strongly agreed/agreed the session accommodated their personal learning style. 94.6% (n = 193/204) found the content impactful at a personal level, while 94.1% (n = 192/204) found the content impactful at a professional level, translating to patient care. With regards to the Accreditation Council for Graduate Medical Education (ACGME), improvements in all six domains of core competencies were observed: knowledge on patient management 84.8% (n = 173/204), practice-based learning 69.6% (n = 142/204), patient care 52.0% (n = 106/204), system-based learning 48.0% (n = 98/204), professionalism 30.9% (n = 63/204) and communication skills 25.5% (n = 52/204).
A total of 56.4% (n = 115/204) provided a response to the question ‘as a result of what I have learned today, I intend to make the following changes to my practice that I believe will impact my patients’ care in a positive way’, suggesting positive translation to patient care. Three main recurring themes were identified, namely professional competence, specific clinical practice changes, and personal behavioural changes (Table 1).
|
|
|
|
Professional competence
|
‘Evidence based practice to include latest guidelines…’
‘More reading on biologics’
‘Take proper detailed history’
‘Factual content’
‘Better understanding of pathophysiology’
‘Better understanding of holistic approach towards patient care’
|
|
Specific clinical practice changes
|
‘Increasing cabergoline every 1-2 weeks in pts with…’
‘Warn patients of possible CSF leak in management of macroprolactinoma’
‘COVID 19 and adrenal insufficiency treatment strategy…’
‘Requesting appropriate imaging for [incidentaloma]’
‘Drug history, low threshold for pituitary TSHoma’
‘Timing on transfusion, the use of tranexamic acid and PPI infusion in GI bleeding. Timely involvement of other teams…’
|
|
Personal behavioural changes
|
‘More confident about ordering the appropriate tests for work-up … and presenting the results to an MDT’
‘Need to read further around ectopic ACTH’
‘Think more practically…’
‘To proceed in a syste[mat]ic manner…’
‘Focus on going back to reading theoretical aspects and refamiliarise on guidelines’
‘...improve my analytical methods and work [on] improving clinical acumen’
|
Discussion
Our study included a large sample size and evaluated levels 1–3 of Kirkpatrick’s model. We used SIMBA across different specialties and included participants from different continents. We observed significant improvements in self-reported confidence in managing various conditions following participation in SIMBA. Participants felt the sessions were relevant, engaging, and accommodated their learning styles. Participation in SIMBA was also perceived to positively impact patient care; participants reported intentions to utilise knowledge and skills gained during the session to make positive changes to both clinical practice and personal behaviour.
SIMBA has previously proved effective in improving clinicians’ confidence in managing diabetes and pituitary cases.5 While our previous report focused solely on diabetes and endocrine pathologies, this study explored the suitability of SIMBA as a learning tool for a wider range of medical specialties in a larger patient cohort (n = 204 vs n = 24). Our study findings corroborate those of Melson et al.5 and highlight the versatility of SIMBA as a learning model for educating participants on the management of both acute and outpatient care. Another distinction between the two studies is that our study was conducted entirely virtually. This enabled international participation, demonstrating the accessibility and replicability of SIMBA as a pedagogical model transcending geographical boundaries.
In each SIMBA session, transcripts based on real-life clinical scenarios were used to provide relevance, whilst encouraging application of theoretical knowledge.6 Relevant and authentic learning materials are more likely to engage the learner and encourage connections with personal experiences, thus enhancing the learning process.7 Allocating participants individually to moderators, enabled learners to be actively involved in the simulation process, which has been shown to improve both engagement and academic performance.8 Participants then discussed the simulated cases with an expert clinician and peers, which provided an opportunity to reflect upon their learning. Reflection is a key component of simulation-based learning. It enables learners to consolidate knowledge and highlight areas of uncertainty for further exploration. Simulation involving active participation and reflection has shown to improve students’ confidence in managing similar cases in clinical settings.9
Previous studies have utilised standardised computer software to facilitate SBL. A mixed quantitative-qualitative study by Isaza-Restrepo et al.10 used a web-based tool, Virtual Patient (VP), to educate medical students over a 16-week period. Each student attended 2-hour weekly sessions on various case presentations of abdominal pain. In each session, students were asked to interact with a VP using a chatbot user interface. This was followed by a face-to-face small group discussion. Their study found significant improvements in medical students’ history-taking and clinical reasoning skills after the course. Learning was evaluated by observing the interaction between students and role-players. The use of live standardised patients to assess clinical skills has become increasingly popular due to its ability to better emulate real patient interaction.11 Other studies proved effectiveness of VP simulation using assessment of performance with mannequins.12,13 With regard to assessment of knowledge, most studies have used paper-based tests including multiple choice questions (MCQs). However, the restrictions due to COVID-19 raised the need for innovative virtual teaching methods. A study by Braun et al.14 used VP cases to assess diagnostic accuracy, in addition to performance in MCQs. This method has its limitations since it lacks evaluation of the transfer of knowledge or skills to daily practice. In contrast, our study used self-reported confidence levels obtained from pre- and post-SIMBA surveys to assess participants’ learning. We also explored participants’ perceptions towards SIMBA’s impact on their clinical behaviour, thus addressing three levels of Kirkpatrick’s evaluation model.15
Despite computer-based SBL demonstrating its effectiveness, several barriers to its implementation exist. Most of the existing VP SBL models use chatbot user interfaces to facilitate doctor–patient interaction.16 Our study involved human moderators to provide participants with a more authentic doctor–patient interaction. This was also beneficial for medical students who acted as moderators, to develop teamwork skills, gain medical knowledge and understanding of medical education through collaborating with peers around the world in a non-intimidating environment. Another obstacle to implementing computer-based SBL is the cost and time required for the software development.17 In comparison, SIMBA can be delivered with minimal resources. The SIMBA model uses WhatsApp and Zoom for simulation and discussions, respectively. Both platforms are familiar and freely accessible for the end user.
While we demonstrated that SIMBA can improve participants’ confidence in managing various medical conditions, no direct comparison was made with other learning strategies. Thus, we are unable to ascertain whether SIMBA is superior to other virtual teaching models, including computer-based simulation. Moreover, our study focused on differences in self-reported confidence levels and self-perceived changes in behaviour. Further work is warranted to determine whether our findings of increased self-reported confidence levels translate to an improvement in clinical performance (i.e. level 4
of Kirkpatrick’s model).
In conclusion, SIMBA was found to be an effective virtual learning model that improved clinicians’ confidence in managing conditions across various specialties during the COVID-19 pandemic. The model was well-received and accessible to clinicians worldwide. Further work is warranted to explore whether increased confidence levels translate to better real-life performance. 
Acknowledgements
We thank Health Education West Midlands Specialist trainee committee, Institute of Metabolism and Systems Research, University of Birmingham, and Institute of Applied Health Research, University of Birmingham, for their support to conduct this study.
References
1 Rose S. Medical student education in the time of COVID-19. JAMA 2020; 323: 2131–2.
2 Ahmed H, Allaf M, Elghazaly H. COVID-19 and medical education. Lancet Infect Dis 2020; 20: 777–8.
3 Dedeilia A, Sotiropoulos MG, Hanrahan JG et al. Medical and surgical education challenges and innovations in the COVID-19 era: a systematic review. In Vivo (Brooklyn) 2020; 34.
4 Aggarwal R, Mytton OT, Derbrew M et al. Training and simulation for patient safety. Qual Saf Heal Care 2010; 19.
5 Melson E, Davitadze M, Aftab M et al. Simulation via instant messaging - Birmingham advance (SIMBA) model helped improve clinicians’ confidence to manage cases in diabetes and endocrinology. BMC Med Educ 2020; 20: 274.
6 McLean SF. Case-based learning and its application in medical and health-care fields: a review of worldwide literature. J Med Educ Curric Dev 2016; 3.
7 Fan W, Williams CM. The effects of parental involvement on students’ academic self-efficacy, engagement and intrinsic motivation. Educ Psychol 2010; 30.
8 Starmer DJ, Duquette S, Howard L. Participation strategies and student performance: an undergraduate health science retrospective study. J Chiropr Educ 2015; 29.
9 Disler RT, Rochester SF, Kelly MA et al. Delivering a large cohort simulation - beginning nursing students’ experience: a pre-post survey. J Nurs Educ Pract 2013; 3.
10 Isaza-Restrepo A, Gómez MT, Cifuentes G et al. The virtual patient as a learning tool: a mixed quantitative qualitative study. BMC Med Educ 2018; 18.
11 Shahidullah JD, Kettlewell PW. Using standardized patients for training and evaluating medical trainees in behavioral health. Int J Heal Sci Educ 2017; 4.
12 Liaw SY, Chan SWC, Chen FG et al. Comparison of virtual patient simulation with mannequin-based simulation for improving clinical performances in assessing and managing clinical deterioration: randomized controlled trial. J Med Internet Res 2014; 16.
13 Lehmann R, Thiessen C, Frick B et al. Improving pediatric basic life support performance through blended learning with web-based virtual patients: randomized controlled trial. J Med Internet Res 2015; 17.
14 Braun LT, Zottmann JM, Adolf C et al. Representation scaffolds improve diagnostic efficiency in medical students. Med Educ 2017; 51.
15 Kirkpatrick DL, Kirkpatrick JD. Evaluating Training Programs: The Four Levels. San Francisco: Berrett-Koehler; 2006.
16 Kononowicz AA, Woodham LA, Edelbring S et al. Virtual patient simulations in health professions education: systematic review and meta-analysis by the digital health education collaboration. J Med Internet Res 2019; 21.
17 Huang G, Reynolds R, Candler C. Virtual patient simulation at U.S. and Canadian medical schools. Acad Med 2007; 82.
