By Sandra Basu
WASHINGTON — Between 2000 and 2011, almost 1 million servicemembers or veterans were diagnosed with at least one psychological disorder, either during or after deployment, with nearly half having multiple disorders, according to the Institute of Medicine (IoM).
In response, DoD has created programs to foster resilience and help prevent mental health issues resulting from military services.
The problem, according to a recent IoM report, is that few of DoD’s resilience, prevention and reintegration programs are appropriately based on evidence, and the interventions are not evaluated frequently enough for effectiveness.
The report, “Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs” concluded among other things that “a majority of DoD resilience, prevention, and reintegration programs are not consistently based on evidence and that programs are evaluated infrequently or inadequately.”
The IoM study was requested by the DoD as a follow-up to a 2013 report, “Returning Home from Iraq and Afghanistan: Assessment of Readjustment Needs of Veterans, Service Members, and Their Families.”
“The committee sought to assist the department in determining how it can use scarce resources to optimize its efforts, through prevention, to protect and restore the mental and behavioral health of its personnel and their families,” wrote Kenneth Warner, PhD, who chaired the committee, in the report preface.
For the study, the committee explained it “reviewed hundreds of publications, policy directives, and other documents pertaining to DoD’s many programs to identify and address mental health problems among military recruits, active duty personnel, and, to a lesser extent, military veterans.”
Among the committee’s recommendations is that DoD “employ only evidence-based resilience, prevention, and reintegration programs and policies and that it eliminate non-evidence-based programming.”
The report stated that, although the committee identified some programs that are based on evidence and that have demonstrated effectiveness, “comprehensive assessments of DoD programs show that a majority of the programs are not based on evidence and that programs are evaluated infrequently.”
“There is a lack of a process within DoD to systematically develop, track, and evaluate programs; among those that are evaluated, there is wide variability in the rigor of the evaluation. Similarly, although some of the evidence that DoD cites as justification for some of its programs may be statistically significant, it is not always clinically meaningful,” the report explained.
The report stated that preventive intervention programs “should be rigorously designed, and the programs and their components should be evaluated extensively.”
“This should occur as the program is being developed, while it is being conducted, and after it has been completed. Dedicated resources (e.g., funding, staffing, and logistical support) for data analysis and evaluation are essential to ongoing performance monitoring for quality improvement and accountability,” the committee noted.
The committee also pointed out that DoD “lacks a strategy, a framework, and a range of measures for monitoring performance that ultimately can be used to assess resilience, reintegration, and good psychological health, to determine program effectiveness.”
The committee recommends that, “when appropriate measures are not available, DoD should develop and test measures to assess the structure, process, and outcomes of prevention interventions across the phases of the military life cycle.”
Navy Capt. Richard Stoltz, Defense Centers of Excellence for Psychological Health and Traumatic Brain Injury (DCoE) director, told U.S. Medicine that he had a favorable impression of the report and that the findings are being reviewed.
“DoD did ask the Institute of Medicine to conduct the study. I feel that it was professionally done, and I believe the findings will be helpful to us as we learn from their findings and continue to make improvements to our psychological health prevention programs,” Stoltz said. “I also think the study clearly displays that that the Department of Defense had devoted considerable resources and efforts to improving resilience for servicemembers, families and veterans. It shows we are putting a lot on prevention and not simply waiting around for our beneficiaries to develop psychological disorders before we attempt to intervene.”
Stoltz emphasized that the report is on psychological prevention programs, not treatment programs. While there are good measures to determine the effectiveness of treatments, he said, finding good evidence-based measures to determine the effectiveness of psychological prevention programs can be challenging.
“I believe that [IOM’s] comment is valid, but I am concerned that some people may think that we neglected to utilize measures out there when, in fact, across the U.S., whether military or civilian, there is a lack of good measures to determine whether prevention programs achieve positive results or not. We are continuing to look at what measures are out there and continuing to figure out based upon what we have done to this point what lessons have we learned that we can do better next time in the way we measure our interventions aimed at prevention,” he said.
Stoltz also explained that prevention research is under way in DoD on resilience and psychological health.
“Part of the answer in developing good measures is that you have to do some research. We have a very active research portfolio focused on resilience issues for active duty servicemembers. We have another extensive amount of research going on for family members,” he said.
Meanwhile, DoD spokeswoman Lt. Col. Cathy Wilkinson said the IoM study did not take into account developments such as that oversight of suicide prevention and resilience policy now falls under the DoD Office of the Undersecretary of Defense for Personnel and Readiness as was directed by Congress.
She also said that the IoM study did not “acknowledge the work done by the U.S. Surgeon General and the National Action Alliance for Suicide Prevention in publishing the National Strategy for Suicide Prevention. Nor did it refer to the 2010 Final Report of the Defense Task Force on Prevention of Suicide by Members of the Armed Forces.”
Wilkinson also explained that DoD established the Defense Suicide Prevention Office in 2011 and that DSPO “believes it has a program that will address each of the recommendations found in this report.”
She added that strategies for both suicide prevention and resilience are under development, adding, “These comprehensive efforts are analyzing and mapping all programs, activities, and initiatives for suicide prevention and resilience to the goals and recommendations found in the NSSP and the Task Force report — supported by the RAND Corporation report on psychological resilience and many of the same reports and studies referenced in the IoM report. This mapping will be followed by evaluation for effectiveness, quality, and affordability to make recommendations for collaboration, consolidation, and improvement.”