Harvard Medical School Global Mental Health: Trauma and Recovery Course: What is the Global Impact? Three Year’s Results

Purpose : This paper describes and documents an innovative blended learning Global Mental Health: Trauma and Recovery certificate training course. This course combines a two-week face-to-face training in Orvieto, Italy with a five-month follow-up online virtual training as a learning experience for global health care practitioners. Continuing medical education (CME) accreditation is offered upon completion. This course utilized an innovative blended learning model with a community of practice approach, a combination of lectures and discussions, and online in-depth group case study discussions. Methodology : Data was collected by self-reported anonymous evaluation by participants of three continuous years of the CME Global Mental Health: Trauma and Recovery certificate training course sponsored by Harvard Medical School. One hundred fifty-five participants (n= 39 in 2011; n = 57 in 2012; n=59 in 2013) underwent a pre-and post-course evaluation to determine sustained confidence in performing medical and psychiatric care to traumatized patients and communities, as well as to determine their learning of the Global Mental Health Action Plan (GMHAP). Results : Over the course of three independent years, a total of 155 participants were evaluated. There was evidence for significant improvement in their confidence levels in all clinical areas (diagnosis; treatment of trauma; use of psychotropic medication) when comparing baseline to completion of the six-month course. All ten dimensions of the GMHAP and nine medical and psychiatric aspects of treatment revealed significant improvement in confidence levels. Regression analysis also indicated similar results after the adjustment of demographic covariates. Physicians and participants with mental health and social work background had significantly higher confidence. Participants who were MD’s or psychiatrists had higher confidence in most of the categories of confidence except for self-care, understanding culture, collaboration, and policy and financing. The model showed no difference in learning based upon gender and level of development of country of origin. Conclusion : The evaluation of this blended learning CME program provides evidence of significant enhancement of clinical practice and planning skills in health care practitioners working with highly traumatized patients and communities worldwide. This successful training over the past 18 years has gone far to achieve the health and mental health capacity building as requested by the Ministers of Health from post-conflict societies in the historic Rome meeting in 2004.


Introduction
Ministers of Health (MOHs) from the world's post-conflict affected countries met in Rome, Italy in December 2004, for the first time to endorse a science-based Global Mental Health Action Plan (GMHAP) and a Global Mental Health Textbook of Best Practices (GMHTBP) to be implemented and disseminated worldwide.This first-ever meeting was called Project One Billion (POB) for the more than one billion persons worldwide affected by violence from war, ethnic conflict, torture, and terrorism (Mollica & McDonald, 2003).Participants from the Ministries of Health came from thirty-five countries (n=35) including post-conflict societies such as Afghanistan, Uganda, Peru, Lebanon, Liberia, and Rwanda.A full list of MOH participants is available (Mollica, 2012).POB was sponsored by the Harvard Program in Refugee Trauma (HPRT), Caritas Rome, Istituto Superiore di Sanitá (ISS, Italian National Institute of Health), US Fulbright New Century Scholars Program, and the World Bank, with the support of the World Health Organization (WHO) and the Italian Ministry of Health and Foreign Affairs.
Project One Billion achieved its major goals and more.The MOHs requested that a global training of high quality be established at Harvard Medical School (HMS) through HPRT to build worldwide mental health capacity.At the time of this meeting, few if any, mental health practitioners existed in participant countries that could provide policy advice and consultation on the development and implementation of culturally effective mental health services.This challenge by the MOHs was taken up by HPRT through the HMS Division of Continuing Education through its accredited continuing medical education (CME) activities.Innovation was built into the first CME HMS/HPRT Certificate Training Course launched in November 2006.This course was designed as an innovative blended learning course combining face-to-face training with follow-up on-line virtual training.The goal of the Project One Billion CME course, Global Mental Health: Trauma and Recovery, was to implement scientifically effective, culturally valid mental health training to healthcare practitioners and professionals from other sectors throughout the world.Tuition was kept at a modest level and scholarships were offered to encourage participation from low-income post-conflict countries.At that time, published studies had suggested that didactic CME did not appear to produce effective change in physician performance.Most CME courses were low in interaction, especially meaningful interaction among peers, leading to limited behavioral change (Bloom, 2005;Davis et al, 1995;Rayburn, Regnier, McMahon, 2020;Cervero & Gaines, 2015;Davis & McMahon, 2018;Davis et al, 1999;Kanouse & Jacoby, 1988;Davis et al, 1995).In contrast, the GMH Course was located for two weeks onsite in Orvieto, Italy with intensive interaction between course participants and faculty; followed by weekly online learning experience for five months.The Frederico II Medical School and the ISS actively cosponsored this Harvard Medical School Course.
With the support of the Harvard Graduate School of Education (HGSE), HPRT/HMS introduced an innovative CME approach from the first training in November 2006.The Global Mental Health: Trauma and Recovery CME certificate course was built upon four elements.First, the course was built upon the GMHAP and GMHTBP (Mollica, 2012) Second, the teaching model was based upon the "community of practice" (COP) learning perspective of Wenger and colleagues (Wenger, McDermott, Snyder, 2002;Wenger, 1998).A COP is a "group of people who share a concern or a passion for something they do, and who learn how to do it better as they interact regularly" (McMahon, Asthagiri, Khalessi, 2019).Third, the course used a blended learning approach with face-to-face learning with a follow-up virtual learning model (Liu et al, 2016;Cook et al, 2008;Shaw et al, 2011).This blended learning model included two weeks on-site in Orvieto, Italy followed up by five months online in small groups which used case study discussions and interactive dialogue and conversations.including participants from healthcare and mental health backgrounds as well as humanitarian aid workers from the United Nations and international non-governmental organizations (NGOs), journalists, and human rights lawyers (Liu et al, 2016).
While each course training results were evaluated by HMS, HPRT conducted a comprehensive evaluation of a threeconsecutive year cohort from 2011 to 2013, (n=155 participants).Although completion of the GMH course has been almost universally successful with fewer than ten participants dropping out over thirteen years (primarily due to illness), an extensive evaluation to determine its impact on participants was undertaken.Mental health knowledge including learning the major dimensions of the GMHAP, confidence in performing medical and psychiatric procedures with highly traumatized patients, families and communities, self-care, and cultural competence were assessed.The major findings of this evaluation are presented in this report.By 2020, the GMH blended leaning course was in its 14th year with over 1,000 alumni working in over eighty-five countries, before pivoting to virtual only programming in Spring of 2021 due to the COVID-19 pandemic.Regardless, the present evaluation reassures us that the request of the World's Ministries of Health in 2004 was met through a six month culturally sensitive, evidencebased accredited CME blended learning COP model.In this study, we evaluated confidence level change before and after the GMH course among the 155 participants (Smith et al, 1998;Wickstrom, Kelley, Keyserling et al, 2000;Wickstrom, Kolar, Keyserling et al, 2000;Henderson et al, 2008;Henderson et al, 2005;Borba et al, 2015).

Purpose
This evaluation study contributes to the emerging evidence that CME activities can use innovative interactive approaches for training health care practitioners and humanitarian aid/human rights workers globally in the care of highly traumatized patients and communities.

Evaluation Approach
The participants' confidence levels were evaluated by a measure of competence on performance using the Smith, et al. approach (Smith et al, 1998;Wickstrom, Kelley, Keyserling et al, 2000;Wickstrom, Kolar, Keyserling et al, 2000;Henderson et al, 2008;Henderson et al, 2005;Borba et al, 2015).Considerations for the level of health practitioners' confidence is closely correlated with their actual performance, have been demonstrated.
Demographics (gender, age, occupation, and specialty) and confidence level were collected at the beginning of the training and end of the training.(See Table 1) First, participants' confidence was measured on implementing the GMHAP at the beginning of the training (baseline) and the end of the training(post-training).A six-point Likert scale (1 = not confident, 2 = slightly confident, 3 = somewhat confident, 4 = confident, 5 = very confident, 6 = extremely confident) for each question was used to measure their level of confidence.We measured the confidence level on nineteen aspects: policy/legislation, financing, science-based mental health services, multidisciplinary education, role of international agencies, linkages to economic development, human rights, research, evaluation, and ethics (Details about each category can be found in the Appendix) We asked sixty-four (64) questions about their confidence towards multiple aspects of medical and psychiatric treatment at the beginning of the training (baseline) and the end of the training (post-training).Similar to the above, a six-point Likert scale for each question was used to measure their level of confidence.The 64 confidence questions were summarized into 9 different categories: treating trauma (N = 15), psychiatric diagnosis (N = 6), assist patient care and social issue (N = 11), prescribe psychotropic med (N = 1), self-care (N = 3), understanding culture impact (N = 8), collaboration (N = 1), policy financing (N = 1), and teaching research evaluation (N = 11).Each category of confidence was measured by a set of questions from the questionnaire.We calculated the score of each category by summing the scores of questions in the category.Because the number of questions in different categories of confidence is not the same, the total confidence scores of the nine categories are different.The details about which questions are included in each category are in the Appendix.The total score for each category equals to six times the number of questions in the category.

Statistical analysis
The data from participants responses across 3 years were combined, and the summary statistics for the nine confidence categories was calculated.Table 2 includes the number of questions and total score for each category.The average score, standard deviation, standard error, average difference, and percentage of improvement of each confidence category at baseline and post-training was calculated, including the t-test statistics and p-value to compare the differences in average confidence score of each category at baseline and posttraining.

Results
The overall characteristics of the participants are described in table 1.Most of the participants were female (78.7%), the mean age was 42.9 (SD: 11.8).Most of them worked in North America (46.9%), some of them worked in Asia (9.5%), Africa (11.2%), and the Middle East (11.7%).They worked in diverse fields, including universities (30.3%), hospitals (21.9%), non-governmental organizations (NGOs) (31.6%).More than half of the participants had a professional specialty in mental health (54.8%), and many of them had a professional specialty in social work (47.1%), consulting (35.5%) and clinics (28.4%).Around 54% of participants have more than one professional specialty.Table 2 provides a summary of confidence scores for the nine categories of clinical outcomes.We measured confidence scores at baseline and post training and used paired t-test to compare confidence scores.Overall, there was a significant improvement in participants' confidence in all categories of confidence significant (p<0.001).The average improvements in confidence scores were all more than 14%.The largest improvement was in policy and financing (51.6%).There are also large improvements in treating trauma (27.3%), teaching research evaluation (27.1%), understanding culture impact (21.0%), and selfcare (20.5%).
Table 3 shows the regression results of the nine regression models with the nine medical and psychiatric aspects of treatment.Generally, the coefficient of post-training indicator in the adjusted models showed participants' confidence was significantly improved in the nine outcomes.From the results, participants with specialty in mental health or social work have higher confidence scores in most of the categories compared to people without specialty at mental health or social work.For example, compared to people who did not have specialty in mental health, people specialized in mental health had 0.37 standard deviation higher confidence score in treating trauma.Also, participants who are MDs or psychiatrists had higher confidence scores in most of the categories of confidence.For example, people who were psychiatrists had 1.02 standard deviation higher confidence score in prescribing psychotropic medication.Age was positively correlated with psychiatric diagnosis (Est: 0.02, p<0.001), self-care (Est: 0.02, p<0.001), and collaboration (Est:0.01,P = 0.003), and it is negatively correlated with policy financing (Est: -0.01, p = 0.012).Participants with specialty in consulting have higher confidence in policy financing (Est :0.32, P = 0.

Discussion
The MOHs from post-conflict countries (e.g., Afghanistan, Bosnia and Herzegovina, Haiti, etc.) were convened for the first time in Rome, Italy in December 2004, in order to generate together a Global Mental Health Action Plan.This was achieved.Unfortunately, at that time, most postconflict countries did not have the trained mental health personnel in their country who could assist in developing mental health policies for their civilian populations (Mollica et al, 2004).Thus, a request was made to the Harvard Program in Refugee Trauma (HPRT) to develop a training program to train healthcare providers in best practices for treating survivors of mass violence, such as, armed conflict.In response to this request, HPRT developed the Global Mental Health: Trauma and Recovery Course to address the gap in training as identified by the MOH in the Rome meeting.A curriculum was developed, and mental health experts were recruited worldwide to participate in HPRT's six-month training program through Harvard Medical School.Global trainees were assembled for two weeks of face-to-face intensive training on-site in Orvieto, Italy; followed by five months of continued training online with didactic lectures combined with supervised small groups (n=10) discussions of the lectures and intense case study discussion learning.The small groups were led by two faculty members; clinical cases and psychosocial projects were addressed.The global participants had an opportunity for a twoweek intensive in-depth discussion with each faculty member on their original on-site lectures in Italy.A healing environment exercise also took place in which each participant was taught how to design and implement a healing environment video.This video was presented and discussed within their small groups.In addition, an innovative blended learning model (two weeks onsite; five months online), and a Community of Practice (COP) model formed the foundation of the course.
A COP is a group of people or professionals who share a common interest and a desire to learn from and contribute to their communities with their variety of experiences and expertise.These people are intentionally committed to learning new skills, information, and knowledge within a model of dialogue and discussion.
• COP groups have a shared empathic horizon that aims to achieve support among groups members who share new ideas, policies, and plans in a professional environment.
• The ultimate aim is to foster greater goodness, beauty, and justice in the workplace, and the world.
• COP groups are focused on mutual learning through case-based discussions.Specifically, this model is based upon coconstructed learning -where everyone has something to share, and everyone has something to teach.
• The group process relies on the group members' willingness to reflect and exchange ideas.This process has demonstrated that new ideas and strategies emerge, as close relationships develop among participants.
An initial evaluation revealed the value of this innovative approach (Johnson, 2009) We found that the COP model was greatly appreciated by participants since all participants were highly competent clinicians, despite their limited knowledge and skills in caring for highly traumatized patients and communities.In addition, they were a vast reservoir of the cultural knowledge necessary to care for the patients within their local communities.
Seven years later, this evaluation of the GMH course revealed the significant improvement in GMH participants in learning the GMHAP and confidence in diagnosing and treating highly traumatized patients in culturally sensitive and scientifically valid ways.
Demonstrated results of the GMH course included: 1. Participants in this study worked at different locations all over the world, and they had diverse working backgrounds, many of them working in multiple fields.People also had various specialties, we had participants with specialty in mental health, also had participants with specialty in social work and consulting.
2. Participants had significant improvements in confidence level across all nine aspects of medical and psychiatric care, ranging from psychiatric diagnosis to policy and financing.
3. Participants had significant improvements in confidence level of all ten dimensions of the GMHAP.
There was no significant difference in learning across differences in gender, level of development of country of origin, and work specialties.Although older participants had less improvement in assisting with patient care, social issues, and self-care.

Figure 1
Figure 1 shows average confidence scores of the overall 10 Global Mental Health Action Plan (GMHAP) questions before and after the training.Figure 1 reveals statistically significant improvements in all of the questions after the training program.The average improvement score is about 0.7, which means on average, participants move to a higher confidence level in GMHAP after training.Participants generally had the largest improvement in "Linkage to economic development."Participants had relatively low average confidence scores in "Financing" before and after the training, they had relatively high average confidence scores in "Science-based mental health services" and "Evaluation" before and after the training.

Figure 1 .
Figure 1.Average confidence score of the overall 10 Mental Health Action Plan questions 005) and have lower confidence in psychiatric diagnosis (Est: -0.36, P<0.001) and prescribe psychotropic medicine (Est: -0.35, P<0.001).https://www.sciendo.com/journal/MHGCJISSN 2612-2138 Additionally, the model showed no significant difference in the change in confidence level before and after training based upon gender, level of development of country of origin, and work specialties.Participants with older age had less increase in confidence in assisting with patient care, social issues, and self-care.Detailed results are in the Appendix.

Table 2 .
Confidence comparing baseline to post-training.