Mental Health: Global Challenges Journal
https://www.sciendo.com/journal/MHGCJ
ISSN 2612-2138
Harvard Medical School Global Mental Health:
Trauma and Recovery Course: What is the Global
Impact? Three Year’s Results
Richard F. Mollica1, 2, Giovanni Muscettola3, Eugene F. Augusterfer1, Qiuyuan Qin4, Fanny Y. Cai1
1Massachusetts General Hospital, Boston, Massachusetts, USA
2Harvard Medical School, Boston, Massachusetts, USA
3University Medical School “Federico II,” Naples, Italy
4University of Rochester, Rochester, New York
Abstract
Purpose: This paper describes and documents an innovative blended learning Global Mental
Health: Trauma and Recovery certificate training course. This course combines a two-week
face-to-face training in Orvieto, Italy with a five-month follow-up online vir
tual training as a
learning experience for global health care practitioners. Continuing medical education (CME)
accreditation is offered upon completion. This course utilized an innovative blended learning
model with a community of practice approach, a combination of lectures and discussions, and
online in-depth group case study discussions.
Methodology: Data was collected by self-
reported anonymous evaluation by participants of
three continuous years of the CME Global Mental Health: Trauma and Recovery certificate
training course sponsored by Harvard Medical School. One hundred fifty-five participants (n= 39
in 2011; n = 57 in 2012; n=59 in 2013) underwent a pre- and post-
course evaluation to
determine sustained confidence in performing medical and psychiatric care to traumatized
patients and communities, as well as to determine their learning of the Global Mental Health
Action Plan (GMHAP).
Results: Over the course of three independent years, a total of 155 participants were evaluated.
There was evidence for significant improvement in their confidence levels in all clinical areas
(diagnosis; treatment of trauma; use of psychotropic medication) when comparing baseline to
completion of the six-
month course. All ten dimensions of the GMHAP and nine medical and
psychiatric aspects of treatment revealed significant improvement in confidence levels.
Regression an
alysis also indicated similar results after the adjustment of demographic
covariates. Physicians and participants with mental health and social work background had
significantly higher confidence. Participants who were MD’s or psychiatrists had higher
confidence in most of the categories of confidence except for self-care, understanding culture,
collaboration, and policy and financing. The model showed no difference in learning based
upon gender and level of development of country of origin.
Conclusion: The
evaluation of this blended learning CME program provides evidence of
significant enhancement of clinical practice and planning skills in health care practitioners
working with highly traumatized patients and communities worldwide. This successful training
over the past 18 years has gone far to achieve the health and mental health capacity building
as requested by the Ministers of Health from post-conflict societies in the historic Rome meeting
in 2004.
Keywords
Mental Health, psychiatry, medical education, trauma, recovery, training, international
3
14
Mental Health: Global Challenges Journal
https://www.sciendo.com/journal/MHGCJ
ISSN 2612-2138
Address for correspondence:
Richard F. Mollica, Harvard Program in Refugee Trauma, 2 Castle Drive,
Wilmington, MA 01887
E-mail: RMOLLICA@partners.org
This work is licensed under a Creative Commons Attribution-Non-Commercial 4.0 International
License (CC BY-NC 4.0).
©Copyright: Mollica, 2024
Publisher: Sciendo (De Gruyter)
DOI: https://doi.org/10.56508/mhgcj.v7i1.186
Submitted for publication: 04
January 2024
Revised: 27 February 2024
Accepted for publication: 04
March 2024
Introduction
Ministers of Health (MOHs) from the world’s
post-conflict affected countries met in Rome,
Italy in December 2004, for the first time to
endorse a science-based Global Mental Health
Action Plan (GMHAP) and a Global Mental Health
Textbook of Best Practices (GMHTBP) to be
implemented and disseminated worldwide. This
first-ever meeting was called Project One Billion
(POB) for the more than one billion persons world-
wide affected by violence from war, ethnic
conflict, torture, and terrorism (Mollica &
McDonald, 2003). Participants from the Ministries
of Health came from thirty-five countries (n=35)
including post-conflict societies such as
Afghanistan, Uganda, Peru, Lebanon, Liberia,
and Rwanda. A full list of MOH participants is
available (Mollica, 2012). POB was sponsored by
the Harvard Program in Refugee Trauma (HPRT),
Caritas Rome, Istituto Superiore di Sanitá (ISS,
Italian National Institute of Health), US Fulbright
New Century Scholars Program, and the World
Bank, with the support of the World Health
Organization (WHO) and the Italian Ministry of
Health and Foreign Affairs.
Project One Billion achieved its major goals
and more. The MOHs requested that a global
training of high quality be established at Harvard
Medical School (HMS) through HPRT to build
worldwide mental health capacity. At the time of
this meeting, few if any, mental health
practitioners existed in participant countries that
could provide policy advice and consultation on
the development and implementation of
culturally effective mental health services. This
challenge by the MOHs was taken up by HPRT
through the HMS Division of Continuing Education
through its accredited continuing medical
education (CME) activities. Innovation was built
into the first CME HMS/HPRT Certificate Training
Course launched in November 2006. This course
was designed as an innovative blended learning
course combining face-to-face training with
follow-up on-line virtual training. The goal of the
Project One Billion CME course, Global Mental
Health: Trauma and Recovery, was to implement
scientifically effective, culturally valid mental
health training to healthcare practitioners and
professionals from other sectors throughout the
world. Tuition was kept at a modest level and
scholarships were offered to encourage
participation from low-income post-conflict
countries. At that time, published studies had
suggested that didactic CME did not appear to
produce effective change in physician
performance. Most CME courses were low in
interaction, especially meaningful interaction
among peers, leading to limited behavioral
change (Bloom, 2005; Davis et al, 1995; Rayburn,
Regnier, McMahon, 2020; Cervero & Gaines,
2015; Davis & McMahon, 2018; Davis et al, 1999;
Kanouse & Jacoby, 1988; Davis et al, 1995). In
contrast, the GMH Course was located for two
weeks onsite in Orvieto, Italy with intensive
interaction between course participants and
faculty; followed by weekly online learning
experience for five months. The Frederico II
Medical School and the ISS actively co-
sponsored this Harvard Medical School Course.
With the support of the Harvard Graduate
School of Education (HGSE), HPRT/HMS introduced
an innovative CME approach from the first
training in November 2006. The Global Mental
Health: Trauma and Recovery CME certificate
course was built upon four elements. First, the
course was built upon the GMHAP and GMHTBP
(Mollica, 2012) Second, the teaching model was
based upon the “community of practice” (COP)
learning perspective of Wenger and colleagues
(Wenger, McDermott, Snyder, 2002; Wenger,
1998). A COP is a “group of people who share a
concern or a passion for something they do, and
who learn how to do it better as they interact
regularly” (McMahon, Asthagiri, Khalessi, 2019).
Third, the course used a blended learning
approach with face-to-face learning with a
follow-up virtual learning model (Liu et al, 2016;
Cook et al, 2008; Shaw et al, 2011). This blended
learning model included two weeks on-site in
Orvieto, Italy followed up by five months online in
small groups which used case study discussions
and interactive dialogue and conversations.
Fourth, participation was interdisciplinary,
15
Mental Health: Global Challenges Journal
https://www.sciendo.com/journal/MHGCJ
ISSN 2612-2138
including participants from healthcare and
mental health backgrounds as well as
humanitarian aid workers from the United Nations
and international non-governmental
organizations (NGOs), journalists, and human
rights lawyers (Liu et al, 2016).
While each course training results were
evaluated by HMS, HPRT conducted a
comprehensive evaluation of a three-
consecutive year cohort from 2011 to 2013,
(n=155 participants). Although completion of the
GMH course has been almost universally
successful with fewer than ten participants
dropping out over thirteen years (primarily due to
illness), an extensive evaluation to determine its
impact on participants was undertaken. Mental
health knowledge including learning the major
dimensions of the GMHAP, confidence in
performing medical and psychiatric procedures
with highly traumatized patients, families and
communities, self-care, and cultural
competence were assessed. The major findings
of this evaluation are presented in this report.
By 2020, the GMH blended leaning course was in
its 14th year with over 1,000 alumni working in
over eighty-five countries, before pivoting to
virtual only programming in Spring of 2021 due to
the COVID-19 pandemic. Regardless, the present
evaluation reassures us that the request of the
World’s Ministries of Health in 2004 was met
through a six month culturally sensitive, evidence-
based accredited CME blended learning COP
model. In this study, we evaluated confidence
level change before and after the GMH course
among the 155 participants (Smith et al, 1998;
Wickstrom, Kelley, Keyserling et al, 2000;
Wickstrom, Kolar, Keyserling et al, 2000;
Henderson et al, 2008; Henderson et al, 2005;
Borba et al, 2015).
Purpose
This evaluation study contributes to the
emerging evidence that CME activities can use
innovative interactive approaches for training
health care practitioners and humanitarian
aid/human rights workers globally in the care of
highly traumatized patients and communities.
Methodology
Study Sample
There were 155 participants in the training
program across the three years from 2011 to
2013 (N2011=39; N2012=57; N2013=59).
Evaluation Approach
The participants’ confidence levels were
evaluated by a measure of competence on
performance using the Smith, et al. approach
(Smith et al, 1998; Wickstrom, Kelley, Keyserling et
al, 2000; Wickstrom, Kolar, Keyserling et al, 2000;
Henderson et al, 2008; Henderson et al, 2005;
Borba et al, 2015). Considerations for the level of
health practitioners’ confidence is closely
correlated with their actual performance, have
been demonstrated.
Demographics (gender, age, occupation,
and specialty) and confidence level were
collected at the beginning of the training and
end of the training. (See Table 1)
First, participants’ confidence was measured
on implementing the GMHAP at the beginning of
the training (baseline) and the end of the
training(post-training). A six-point Likert scale (1 =
not confident, 2 = slightly confident, 3 =
somewhat confident, 4 = confident, 5 = very
confident, 6 = extremely confident) for each
question was used to measure their level of
confidence. We measured the confidence level
on nineteen aspects: policy/legislation, financing,
science-based mental health services,
multidisciplinary education, role of international
agencies, linkages to economic development,
human rights, research, evaluation, and ethics
(Details about each category can be found in
the Appendix)
We asked sixty-four (64) questions about their
confidence towards multiple aspects of medical
and psychiatric treatment at the beginning of the
training (baseline) and the end of the training
(post-training). Similar to the above, a six-point
Likert scale for each question was used to
measure their level of confidence. The 64
confidence questions were summarized into 9
different categories: treating trauma (N = 15),
psychiatric diagnosis (N = 6), assist patient care
and social issue (N = 11), prescribe psychotropic
med (N = 1), self-care (N = 3), understanding
culture impact (N = 8), collaboration (N = 1),
policy financing (N = 1), and teaching research
evaluation (N = 11). Each category of
confidence was measured by a set of questions
from the questionnaire. We calculated the score
of each category by summing the scores of
questions in the category. Because the number
of questions in different categories of confidence
is not the same, the total confidence scores of
the nine categories are different. The details
about which questions are included in each
category are in the Appendix. The total score for
each category equals to six times the number of
questions in the category.
16
Mental Health: Global Challenges Journal
https://www.sciendo.com/journal/MHGCJ
ISSN 2612-2138
Statistical analysis
The data from participants responses across 3
years were combined, and the summary statistics
for the nine confidence categories was
calculated. Table 2 includes the number of
questions and total score for each category. The
average score, standard deviation, standard
error, average difference, and percentage of
improvement of each confidence category at
baseline and post-training was calculated,
including the t-test statistics and p-value to
compare the differences in average confidence
score of each category at baseline and post-
training.
Table 1. Descriptive table combined 3 years (2011-2013)
Demography
N (%) mean (SD)
Age
42.9 (11.8)
Gender
33 (21.3%) Male
122 (78.7%) Female
Location of work
20 (11.2%) Africa
17 (9.5%) Asia
11 (6.1%) Australia
5 (2.8%) Caribbean
17 (9.5%) Europe
21 (11.7%) Middle East
84 (46.9%) North America
4 (2.2%) South America
Workplace
47 (30.3%) University
12 (7.7%) Field clinic
34 (21.9%) Hospital
49 (31.6%) NGO
20 (12.9%) Government
13 (8.4%) Inter-governmental agency
37 (23.9%) Private sector
16 (10.3%) Public sector
Professional
Specialty
44 (28.4%) Clinic
85 (54.8%) Mental health
73 (47.1%) Social work
55 (35.5%) Consulting
20 (12.9%) MD (not psychiatrist)
20 (12.9%) Psychiatrist
Multiple
specialty
(52.9%) Yes
(47.1%) No
Paired T-test
To compare the nine categories of
confidence score of participants before and after
the training program, we applied paired t-test on
confidence score at baseline and post training
for same participant. The null hypothesis assumes
the difference in average confidence score μ_d
is 0. The test statistic formula used is in the
Appendix.
Linear Regression
To evaluate the impact of the training
program on participants’ confidence, linear
regression analysis was used on the nine aspects
of medical and psychiatric treatment as
outcomes and adjusted demographics and work
background information about participants which
includes age, gender, work locations, and work
specialties. The outcomes were standardized due
to different scales of outcomes. Age was
included as a continuous variable, gender and
work location were as categorical variables.
Indicators were added about their specialty
including mental health, clinical work, social work,
consulting into the model, Also, indicators were
added about whether they are MD not
psychiatrist or psychiatrist. The indicators were
added for each category because those were
not mutually exclusive. To further examine
whether demographics or work backgrounds of
participants would impact their learning
outcomes, the same regression model was
applied with additional interaction terms of post-
training and each covariatel.
17
Mental Health: Global Challenges Journal
https://www.sciendo.com/journal/MHGCJ
ISSN 2612-2138
Results
The overall characteristics of the participants
are described in table 1. Most of the participants
were female (78.7%), the mean age was 42.9
(SD: 11.8). Most of them worked in North America
(46.9%), some of them worked in Asia (9.5%),
Africa (11.2%), and the Middle East (11.7%). They
worked in diverse fields, including universities
(30.3%), hospitals (21.9%), non-governmental
organizations (NGOs) (31.6%). More than half of
the participants had a professional specialty in
mental health (54.8%), and many of them had a
professional specialty in social work (47.1%),
consulting (35.5%) and clinics (28.4%). Around
54% of participants have more than one
professional specialty.
Figure 1 shows average confidence scores of
the overall 10 Global Mental Health Action Plan
(GMHAP) questions before and after the training.
Figure 1 reveals statistically significant
improvements in all of the questions after the
training program. The average improvement
score is about 0.7, which means on average,
participants move to a higher confidence level in
GMHAP after training. Participants generally had
the largest improvement in “Linkage to economic
development.” Participants had relatively low
average confidence scores in “Financing” before
and after the training, they had relatively high
average confidence scores in “Science-based
mental health services” and “Evaluation” before
and after the training.
Figure 1. Average confidence score of the overall 10 Mental Health Action Plan questions
Table 2 provides a summary of confidence
scores for the nine categories of clinical
outcomes. We measured confidence scores at
baseline and post training and used paired t-test
to compare confidence scores. Overall, there
was a significant improvement in participants’
confidence in all categories of confidence
significant (p<0.001). The average improvements
in confidence scores were all more than 14%.
The largest improvement was in policy and
financing (51.6%). There are also large
improvements in treating trauma (27.3%),
teaching research evaluation (27.1%),
understanding culture impact (21.0%), and self-
care (20.5%).
Table 3 shows the regression results of the nine
regression models with the nine medical and
psychiatric aspects of treatment. Generally, the
coefficient of post-training indicator in the
adjusted models showed participants’
confidence was significantly improved in the nine
outcomes. From the results, participants with
specialty in mental health or social work have
higher confidence scores in most of the
categories compared to people without specialty
at mental health or social work. For example,
compared to people who did not have specialty
in mental health, people specialized in mental
health had 0.37 standard deviation higher
confidence score in treating trauma. Also,
participants who are MDs or psychiatrists had
higher confidence scores in most of the
categories of confidence. For example, people
who were psychiatrists had 1.02 standard
deviation higher confidence score in prescribing
psychotropic medication. Age was positively
correlated with psychiatric diagnosis (Est: 0.02,
p<0.001), self-care (Est: 0.02, p<0.001), and
collaboration (Est:0.01, P = 0.003), and it is
negatively correlated with policy financing (Est: -
0.01, p = 0.012). Participants with specialty in
consulting have higher confidence in policy
financing (Est :0.32, P = 0.005) and have lower
confidence in psychiatric diagnosis (Est: -0.36,
P<0.001) and prescribe psychotropic medicine
(Est: -0.35, P<0.001).
18
Mental Health: Global Challenges Journal
https://www.sciendo.com/journal/MHGCJ
ISSN 2612-2138
Additionally, the model showed no significant
difference in the change in confidence level
before and after training based upon gender,
level of development of country of origin, and
work specialties. Participants with older age had
less increase in confidence in assisting with
patient care, social issues, and self-care.
Detailed results are in the Appendix.
Discussion
The MOHs from post-conflict countries (e.g.,
Afghanistan, Bosnia and Herzegovina, Haiti, etc.)
were convened for the first time in Rome, Italy in
December 2004, in order to generate together a
Global Mental Health Action Plan. This was
achieved. Unfortunately, at that time, most post-
conflict countries did not have the trained mental
health personnel in their country who could assist
in developing mental health policies for their
civilian populations (Mollica et al, 2004). Thus, a
request was made to the Harvard Program in
Refugee Trauma (HPRT) to develop a training
program to train healthcare providers in best
practices for treating survivors of mass violence,
such as, armed conflict. In response to this
request, HPRT developed the Global Mental
Health: Trauma and Recovery Course to address
the gap in training as identified by the MOH in the
Rome meeting. A curriculum was developed,
and mental health experts were recruited
worldwide to participate in HPRT’s six-month
training program through Harvard Medical
School. Global trainees were assembled for two
weeks of face-to-face intensive training on-site in
Orvieto, Italy; followed by five months of
continued training online with didactic lectures
combined with supervised small groups (n=10)
discussions of the lectures and intense case study
discussion learning. The small groups were led by
two faculty members; clinical cases and
psychosocial projects were addressed. The
global participants had an opportunity for a two-
week intensive in-depth discussion with each
faculty member on their original on-site lectures in
Italy. A healing environment exercise also took
place in which each participant was taught how
to design and implement a healing environment
video. This video was presented and discussed
within their small groups. In addition, an
innovative blended learning model (two weeks
onsite; five months online), and a Community of
Practice (COP) model formed the foundation of
the course.
A COP is a group of people or professionals
who share a common interest and a desire to
learn from and contribute to their communities
with their variety of experiences and expertise.
These people are intentionally committed to
learning new skills, information, and knowledge
within a model of dialogue and discussion.
COP groups have a shared empathic
horizon that aims to achieve support among
groups members who share new ideas, policies,
and plans in a professional environment.
The ultimate aim is to foster greater
goodness, beauty, and justice in the workplace,
and the world.
COP groups are focused on mutual
learning through case-based discussions.
Specifically, this model is based upon co-
constructed learning where everyone has
something to share, and everyone has something
to teach.
The group process relies on the group
members’ willingness to reflect and exchange
ideas. This process has demonstrated that new
ideas and strategies emerge, as close
relationships develop among participants.
An initial evaluation revealed the value of this
innovative approach (Johnson, 2009) We found
that the COP model was greatly appreciated by
participants since all participants were highly
competent clinicians, despite their limited
knowledge and skills in caring for highly
traumatized patients and communities. In
addition, they were a vast reservoir of the cultural
knowledge necessary to care for the patients
within their local communities.
Seven years later, this evaluation of the GMH
course revealed the significant improvement in
GMH participants in learning the GMHAP and
confidence in diagnosing and treating highly
traumatized patients in culturally sensitive and
scientifically valid ways.
Demonstrated results of the GMH course
included:
1. Participants in this study worked at
different locations all over the world, and they
had diverse working backgrounds, many of them
working in multiple fields. People also had various
specialties, we had participants with specialty in
mental health, also had participants with
specialty in social work and consulting.
2. Participants had significant improvements
in confidence level across all nine aspects of
medical and psychiatric care, ranging from
psychiatric diagnosis to policy and financing.
3. Participants had significant improvements
in confidence level of all ten dimensions of the
GMHAP.
There was no significant difference in learning
across differences in gender, level of
development of country of origin, and work
specialties. Although older participants had less
improvement in assisting with patient care, social
issues, and self-care.
19