“In our reasonings concerning matter of fact, there are all imaginable degrees of assurance, from the highest certainty to the lowest species of moral evidence. A wise man, therefore, proportions his belief to the evidence.” — David Hume (1711-1776)
Evidence based medicine has been defined as “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.”1 One would be hard-pressed to find a healthcare system that would admit conscious deviation from this definition. Personally, as a medical scientist, I would not have it any other way. Even when the evidence is overwhelming, the medical establishment can be slow to change established but erroneous behavior in the care of patients.
A favorite example I have used before in this column is the story of Dr. Ignaz Semmelweis who objectively demonstrated in 1847 that he could reduce maternal mortality from infection from 10-35% to less than 1% simply by washing his hands in a chlorinated lime solution between attending to laboring mothers. His book on the subject was rejected by the medical establishment, which ridiculed his work, and he was forced from his job at the Vienna General Hospital in Austria. His only “sin” was that his clinical results were unexplainable at that point in history (Louis Pasteur’s germ theory of disease was unavailable at the time), and his theories conflicted with established medical wisdom that was based on disease being an imbalance of the four humours. Semmelweis died tragically in an asylum long before his ideas were proven and accepted. Yet, today, no healthcare provider would dare not wash their hands between patients.
Modern medicine has the benefit of the peer-review process to determine what new medical ideas qualify as medical evidence and are worthy of publication for adoption by the wider medical community. Today, we have numerous peer-reviewed scientific journals on all manner of medical topics to provide the evidence base supporting our daily practice. At its most basic, the peer-review process is a method where recognized experts in a particular field of medicine or science are asked to review a submitted medical science paper. They determine if the researcher’s methods to study a hypothesis (explanation of some observed phenomena) meets appropriate standards in execution of the scientific method to justify the researcher’s conclusions. This process also provides a recommendation on the quality of the manuscript for publication as medical evidence. The goal of this process is to provide a system to validate the information being published to advance the field of medicine. Or, at least that is how it is supposed to function.
I know something about this process because I serve as a co-editor for a section of a nationally recognized pain journal. I am involved in selecting subject matter experts to review submitted medical science papers for their quality and suitability for publication. The process is rigorous, rarely pleasant for the authors or reviewers, time consuming and not often met with success (far more papers are rejected than accepted).
Because humans are involved in the process, it also is fair to say it is far from perfect. The briefest search on the Internet for information describing the flaws in the peer-review concept and others lamenting the value of the process will yield many articles. Furthermore, I have become concerned about what constitutes “peer review” in the information age. Too often, journals of questionable quality will claim they are peer-reviewed, although little if any attention is actually paid to the review process. A friend recently acquainted me with the “scientific” article, “Cuckoo for cocoa puffs? The surgical and neoplastic role of cacao extract in breakfast cereals,” authored by Harvard doctor Mark Shrime, which was produced by a computer random word generator. Shrime submitted the manuscript to 37 open-access journals, and 17 accepted the farcical study requiring only a $500 processing fee for the “study” to be published.
I receive almost weekly invites to serve on the editorial boards of new open access journals, many on topics I have no business calling myself an expert. As a medical researcher and writer of science, I find myself having to be ever more careful and suspect of the “evidence” I find on a particular topic from a science search engine. Understand, I am not against the open access journal movement, and there are plenty of traditional pay journals with dubious scientific review. My concern is that it appears we have (like in Semmelweis’ time) a broken medical information system standard with no clear pathway for fixing the problem. Although the Internet is unquestionably the best thing to happen to medical scholarly activity since books, searching for medical information has a Wild West feel these days with so many unscrupulous sources. There is even a list of likely predatory journals (medical articles can now stalk me I guess) published annually called Beall’s List of Predatory Publishers.3
Fact is, there is no clear accepted standard or guidance to follow and deal with this growing problem in medical information. In my own work, I have adopted the U.S. National Library of Medicine standard of MEDLINE, which is a bibliographic database of more than 21 million references going back to 1946. The MEDLINE system relies on the Literature Selection Technical Review Committee (LSTRC), which decides which journals are acceptable to index in this system.4 The LSTRC meets three times a year to evaluate the academic behavior of its indexed journals and journals seeking to be listed, and this review results in some journals being dropped and others added. Again, this process is run by humans and therefore is subject to human error, but at least it is a recognizable and published standard.
We in the medical field have some cleaning to do in our own house of medicine. We should be able to agree on what constitutes medical evidence and what encompassing standards will be adopted (to include open access journals) in the medical publishing community. Medical advancement depends on efficient access to quality medical information that is unimpeachable. Failure to establish and hold all medical research, researchers, and publishers to a common quality standard would be no less tragic a story than that of Dr. Semmelweis and his dying mothers.
1 Sackett DL, Rosenberg WMC, Muir Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMG 1996;312:71-2.
2 http://b.fastcompany.net/asset_files/-/2015/02/08/CocoaPuffsStudy2.pdf. Accessed May 4, 2015.
3 http://scholarlyoa.com/2015/01/02/bealls-list-of-predatory-publishers-2015/. Accessed May 4, 2015.
4http://www.nlm.nih.gov/pubs/factsheets/jsel.html. Accessed May 4, 2015.