In our reasonings concerning matter of fact, there are all imaginable degrees of assurance, from the highest certainty to the lowest species of moral evidence. A wise man, therefore, proportions his belief to the evidence

by U.S. Medicine

June 8, 2015
Editor-In-Chief, Chester "Trip" Buckenmaier III, MD,  COL, MC, USA

Editor-In-Chief,
Chester “Trip” Buckenmaier III, MD,
COL, MC, USA

“In our reasonings concerning matter of fact, there are all imaginable degrees of assurance, from the highest certainty to the lowest species of moral evidence. A wise man, therefore, proportions his belief to the evidence.” — David Hume (1711-1776)

Evidence based medicine has been defined as “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.”1 One would be hard-pressed to find a healthcare system that would admit conscious deviation from this definition. Personally, as a medical scientist, I would not have it any other way. Even when the evidence is overwhelming, the medical establishment can be slow to change established but erroneous behavior in the care of patients.

A favorite example I have used before in this column is the story of Dr. Ignaz Semmelweis who objectively demonstrated in 1847 that he could reduce maternal mortality from infection from 10-35% to less than 1% simply by washing his hands in a chlorinated lime solution between attending to laboring mothers. His book on the subject was rejected by the medical establishment, which ridiculed his work, and he was forced from his job at the Vienna General Hospital in Austria. His only “sin” was that his clinical results were unexplainable at that point in history (Louis Pasteur’s germ theory of disease was unavailable at the time), and his theories conflicted with established medical wisdom that was based on disease being an imbalance of the four humours. Semmelweis died tragically in an asylum long before his ideas were proven and accepted. Yet, today, no healthcare provider would dare not wash their hands between patients.

Modern medicine has the benefit of the peer-review process to determine what new medical ideas qualify as medical evidence and are worthy of publication for adoption by the wider medical community. Today, we have numerous peer-reviewed scientific journals on all manner of medical topics to provide the evidence base supporting our daily practice. At its most basic, the peer-review process is a method where recognized experts in a particular field of medicine or science are asked to review a submitted medical science paper. They determine if the researcher’s methods to study a hypothesis (explanation of some observed phenomena) meets appropriate standards in execution of the scientific method to justify the researcher’s conclusions. This process also provides a recommendation on the quality of the manuscript for publication as medical evidence. The goal of this process is to provide a system to validate the information being published to advance the field of medicine. Or, at least that is how it is supposed to function.

I know something about this process because I serve as a co-editor for a section of a nationally recognized pain journal. I am involved in selecting subject matter experts to review submitted medical science papers for their quality and suitability for publication. The process is rigorous, rarely pleasant for the authors or reviewers, time consuming and not often met with success (far more papers are rejected than accepted).

Because humans are involved in the process, it also is fair to say it is far from perfect. The briefest search on the Internet for information describing the flaws in the peer-review concept and others lamenting the value of the process will yield many articles. Furthermore, I have become concerned about what constitutes “peer review” in the information age. Too often, journals of questionable quality will claim they are peer-reviewed, although little if any attention is actually paid to the review process. A friend recently acquainted me with the “scientific” article, “Cuckoo for cocoa puffs? The surgical and neoplastic role of cacao extract in breakfast cereals,” authored by Harvard doctor Mark Shrime, which was produced by a computer random word generator. Shrime submitted the manuscript to 37 open-access journals, and 17 accepted the farcical study requiring only a $500 processing fee for the “study” to be published.

I receive almost weekly invites to serve on the editorial boards of new open access journals, many on topics I have no business calling myself an expert. As a medical researcher and writer of science, I find myself having to be ever more careful and suspect of the “evidence” I find on a particular topic from a science search engine. Understand, I am not against the open access journal movement, and there are plenty of traditional pay journals with dubious scientific review. My concern is that it appears we have (like in Semmelweis’ time) a broken medical information system standard with no clear pathway for fixing the problem. Although the Internet is unquestionably the best thing to happen to medical scholarly activity since books, searching for medical information has a Wild West feel these days with so many unscrupulous sources. There is even a list of likely predatory journals (medical articles can now stalk me I guess) published annually called Beall’s List of Predatory Publishers.3

Fact is, there is no clear accepted standard or guidance to follow and deal with this growing problem in medical information. In my own work, I have adopted the U.S. National Library of Medicine standard of MEDLINE, which is a bibliographic database of more than 21 million references going back to 1946. The MEDLINE system relies on the Literature Selection Technical Review Committee (LSTRC), which decides which journals are acceptable to index in this system.4 The LSTRC meets three times a year to evaluate the academic behavior of its indexed journals and journals seeking to be listed, and this review results in some journals being dropped and others added. Again, this process is run by humans and therefore is subject to human error, but at least it is a recognizable and published standard.

We in the medical field have some cleaning to do in our own house of medicine. We should be able to agree on what constitutes medical evidence and what encompassing standards will be adopted (to include open access journals) in the medical publishing community. Medical advancement depends on efficient access to quality medical information that is unimpeachable. Failure to establish and hold all medical research, researchers, and publishers to a common quality standard would be no less tragic a story than that of Dr. Semmelweis and his dying mothers.

1 Sackett DL, Rosenberg WMC, Muir Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMG 1996;312:71-2.

2 http://b.fastcompany.net/asset_files/-/2015/02/08/CocoaPuffsStudy2.pdf. Accessed May 4, 2015.

3 http://scholarlyoa.com/2015/01/02/bealls-list-of-predatory-publishers-2015/. Accessed May 4, 2015.

4http://www.nlm.nih.gov/pubs/factsheets/jsel.html. Accessed May 4, 2015.


2 Comments

  • Steven Zeitzew says:

    I am surprised to see you advocate evidence based medicine now after previously reading of your support for acupuncture, including battlefield auricular acupuncture, a distinctly non-evidence based practice. Have you truly changed your mind about integrating non-evidence based complementary therapies with evidence based medicine?

    Sometimes it is frustrating to face difficult problems without a great evidence based solution. Sometimes it is difficult to support evidence based medicine over politically popular non-evidence based complementary therapies.

    Supporters of the best scientific evidence based medical treatments are still often ostracized, but our veterans deserve the truth even when the truth is less appealing than wishful thinking based medicine.

  • Steven Zeitzew says:

    I am surprised to see you advocate evidence based medicine now after previously reading of your support for acupuncture, including battlefield auricular acupuncture, a distinctly non-evidence based practice. Have you truly changed your mind about integrating non-evidence based complementary therapies with evidence based medicine?

    Sometimes it is frustrating to face difficult problems without a great evidence based solution. Sometimes it is difficult to support evidence based medicine over politically popular non-evidence based complementary therapies.

    Supporters of the best scientific evidence based medical treatments are still often ostracized, but our veterans deserve the truth even when the truth is less appealing than wishful thinking based medicine.


Related Articles

In an Age of Incompetence

“I am, as I’ve said, merely competent. But in an age of incompetence, that makes me extraordinary.” ~Billy Joel Like many others who pursue a career in medicine, I invested my early years as a... View Article

“13. The delivery of good medical care is to do as much nothing as possible.” “Laws of the House of God,” —Samuel Shem

I have been a part of U.S. Medicine and this column for several years now. I am occasionally asked where my ideas come from for the editorials I produce. Many ideas, of course, are pulled... View Article


U.S. Medicine Recommends


More From editor in chief

Editor in Chief

In an Age of Incompetence

“I am, as I’ve said, merely competent. But in an age of incompetence, that makes me extraordinary.” ~Billy Joel Like many others who pursue a career in medicine, I invested my early years as a... View Article

Editor in Chief

“13. The delivery of good medical care is to do as much nothing as possible.” “Laws of the House of God,” —Samuel Shem

I have been a part of U.S. Medicine and this column for several years now. I am occasionally asked where my ideas come from for the editorials I produce. Many ideas, of course, are pulled... View Article

Editor in Chief

“When discussing collaboration, people often talk about ‘breaking down silos.’ ... ” —William D. Ramsey, MD

“When discussing collaboration, people often talk about ‘breaking down silos.’ I object to that terminology because ‘breaking down’ is a negative image that implies destruction or tearing down of silos. Academically, silos represent our areas... View Article

Editor in Chief

“God help us, if, the first time something fails—and something will fail—we crush whoever it was … whoever’s responsible,” —Gen. John “Mike” Murray

Gen. Mike Murray made this comment as he became the first commander of the Army Futures Command. The command’s website describes the mission of the organization with the following statement: “Army Futures Command leads a... View Article

Editor in Chief

Healthcare is vital to all of us some of the time, but public health is vital to all of us all of the time. —C. Everett Koop (1916-2013)

It has been with no small amount of dismay that I have watched national maps of the measles outbreak grow progressively redder on the national news as the epidemic continues to spread across the nation.... View Article

Subscribe to U.S. Medicine Print Magazine

U.S. Medicine is mailed free each month to physicians, pharmacists, nurse practitioners, physician assistants and administrators working for Veterans Affairs, Department of Defense and U.S. Public Health Service.

Subscribe Now

Receive Our Email Newsletter

Stay informed about federal medical news, clinical updates and reports on government topics for the federal healthcare professional.

Sign Up