LEUNG'S
(CHINESE) HERB NEWS
Number 37
March/April 2002

Herbs in this issue:
Hawthorn

What’s lacking in peer-reviewed publications?


     In a peer-reviewed journal, its papers are supposed to have been critically reviewed by a board of reviewers (experts who are the authors’ peers) before they are published. This will insure a high quality of the publication. That’s the premise. But it often doesn’t work in papers involving herb research, no matter how reputable the journal that carries them. I have written about this in previous issues of this Newsletter (Issue 18, pp. 1-2; Issue 23, pp. 1-2) but I didn’t offer reasons why that is so. Now I want to try to offer some clues to this problem based on personal experience.

 

     Normally a peer-reviewed journal has a board of reviewers who have expertise related to the areas in which the journal publishes. These reviewers are supposed to review articles in their areas of expertise.But there are 2 problems: (1) Scientific herb research is a relatively new field which only recently has been forced to address the issue of traditional usage.

 

Dr. Leung is author of the Encyclopedia of Common Natural Ingredients Used in Food, Drugs, and Cosmetics (Wiley-Interscience), which was published in 1980 and revised in 1996. He is also creator of PHYTOMED, a prototype computer database on Chinese herbal medicine developed under contract with the National Cancer Institute.


 

Previously, most research in medicinal plants focused only on isolation, characterization, biosynthesis, and pharmacology of active constituents or active principles. It was totally focused on an allopathic (erroneously touted as the only scientific) approach with the ultimate goal of obtaining active and effective pharmaceuticals from plant and other natural sources. No scientists were properly investigating the validity of the traditional use of these herbal medicines. And hardly any researchers who had knowledge of traditional herb use had training in modern scientific disciplines. Consequently, few or none of the reviewers of most journals know how to evaluate herb research that involves multiple active or unknown active components. This is exacerbated by the fact that some of these herbal journals picked reviewers more for political and financial reasons than for the true expertise of these reviewers. (2) Even if a particular journal’s board of reviewers does encompass all the appropriate fields of expertise, it doesn’t mean the papers published in it are necessarily of good quality. The reason is that most of the reviewers do their review free of charge as a professional courtesy. And these reviewers often do not have time to review the manuscripts. Since few, if any, of the editors are knowledgeable enough in all aspects of herb research, the result is the current mess of publications in scientific herb research (Issue 18, pp. 1-2; Issue 23, pp. 1-2).
          Examples of papers lacking clarity and basic information abound. They may meet rigorous scientific protocols in most aspects except the most important one, on which the whole research was based. This renders the publication basically useless because its research results can never be reproduced. I am talking about the most basic element in any biomedical research (especially in herb research) which is the object of our research. This is the material which we are investigating – the drug, the chemical, the device, or the herb. For a conventional drug or chemical, there is no problem defining its identity and quality, because it is chemically well defined, which can be readily analyzed. For a device, there shouldn’t be any problem either, because if it is not the same device with well-defined features, it is not the same device. But with an herb, it’s a totally different story. Unless the scientists who study it are trained and knowledgeable in both the traditional and scientific aspects of it, they won’t know what the ‘herb’ is that they are studying. And the results they produce could be interpreted many ways and are of little use to others. But when these results are submitted in the form of a manuscript to a ‘peer-reviewed’ journal for publication, unless the board of reviewers has an expert or experts knowledgeable in both the scientific and traditional aspects of herbs and these experts actually have time to review them, the lack of definition of this ‘herb’ will not be detected before the paper is published. There are many papers in the herbal scientific literature published in reputable herbal, natural products, and conventional medical journals which are of this nature. Naturally, since these papers have appeared in these well-known ‘peer-reviewed’ journals, the misinformation promptly entered major databases. Once there, it is not easy for us to expunge. I suspect a lot of the controversies, not only in herbal medicines/supplements but also in biomedical research in general, which seem to see-saw back and forth between positive and negative findings, are due to researchers’ basing their research unknowingly on ambiguous published data.

Are reviewers/editors falling asleep on the job?

          There is one well-known publisher, whose name shall remain anonymous, which has been putting out huge numbers of books in the natural products, food, drug, and medicines field for several decades. It is a prime example of how slick and persistent marketing can sell anything, because if you keep seeing the names of these books in mailings and in ads, it takes discipline not to buy them, especially if you are not very familiar with the subject. Since they are from such a well-known publisher, one would think that they must be good. But that assumption is wrong. I honestly have tried to give them a benefit of the doubt, but over the past 4 decades have only found a few that are decent or half decent! Even those I am not willing to spend my money to purchase, with the exception of one title that I actually bought a couple of years ago. Before that, I bought my last 2 books from this publisher in the mid-seventies, when those were the only books on the subject available. Their poor quality and limited information prompted me to write my Encyclopedia, which was published by Wiley-Interscience in 1980, since revised in 1996, and the 3rd edition currently in the works. My book is by no means without faults, but at least the errors are relatively minor and there are few bloopers, if any. Unfortunately, that is not the case with a sizable number of books in the herbal medicine field, which are marked by their scanty, outdated, and/or wrong information, as well as poor, illogical presentation. It appears that reviewers and editors may have fallen asleep on the job, otherwise how can one explain such poor-quality books emanating from such prolific publishers like the one mentioned above? One possible explanation is that these publishers may have been using experts in the wrong fields to review their book proposals and manuscripts (e.g., a chemist to evaluate the work of a botanist). The deplorable quality of such a sizable number of publications may also be the result of publishers’ desire and haste to capture a portion of the herbal medicine market before they are technically ready. Another reason is due to journal editors’ bias and lack of knowledge in the herbal medicine field, which is often a result of their conventional medical or pharmaceutical training. Here I want to quote 3 examples of what appears to be the result of “sleeping” reviewers or editors.
1. There is a recently published book on botanicals used in cosmetics by the same nameless but well-known publisher mentioned above, which appears to be another example of reviewers and editors falling asleep on the job. This reference volume has no substance whatsoever.
2. I don’t seem to be able to leave the “The Complete Guide to Herbal Medicines by Fetrow and Avila” alone, because it is such a typical example of editors and reviewers falling asleep on the job. If any of them were half awake, they would have caught perhaps 25% of the errors, most of which non-technical editors can easily detect. For those who have not read my previous reviews of this handbook (Issues 27, 28, & 29), it is one of the worst (if not the worst) books on herbs I have ever encountered. Yet it is efficiently marketed and sold in major bookstores, disseminating misinformation. The scary thing is that there are more than 1 well-known publisher putting out this kind of work!
3. Here is an example that is from another well-known (and venerable) publisher. It is an article on Panax notoginseng in a respectable journal, authored by Italian researchers [J. Ethnopharmacol., 73(2000): 387-391]. There are numerous typographical errors, including “hemorheological,” “notoginsenoides” (notoginsenosides), and “ginsenoides” (ginsenosides), the last 2 throughout the text. Then, there are also “damarrane” (dammarane) and “Arialiaceae” (Araliaceae) as well as “ginsenoide XVII” that does not exist. What they probably mean is gypenoside XVII that is found in the root of Panax notoginseng. Three other readily detected technical errors are: (1) wrong author citation of the botanical name, which requires a parenthesis for Burkill to be correct - Panax notoginseng (Burkill) F.H. Chen; (2) “notoginsenoides titolation” of which I have not the foggiest idea, and would appreciate enlightenment from my readers and colleagues; and (3) the authors have equated polysaccharides to “notoginsenoides” when describing their immunostimulating activities, which are due to the polysaccharides, not saponins. Where were the reviewers when they were so sorely needed?!
          Sometimes I feel rather frustrated and alone in this herbal information field. Why aren’t more of my colleagues concerned about the continuous publication of research in natural medicines which is often so egregious, and yet is being abstracted, entered into major databases, distributed worldwide, and indiscriminately quoted? To my academic colleagues, what good is your research if your findings are all mixed in with others that are dubious and downright worthless? Don’t you think it is about time to do something to arrest this tide of information pollution? Even as late as 6 or 7 years ago, I might have the same thinking as most other scientists in this field – not paying too much attention to the exact nature of the botanical materials we studied or wrote about. But I have since ‘matured’ in my evaluation of literature before incorporating it in my writings. I no longer consider abstracts worth incorporating without caveats, because, unless abstractors have been trained to recognize whether the original articles clearly identify the subject materials used in the studies, resulting abstracts don’t give you any useful information. So, let’s take hawthorn. If “hawthorn” (not specifically hawthorn leaf, fruit, etc.) or “Crataegus monogyna” were reported to be effective in, say, lowering serum cholesterol, what is this “hawthorn?” Is “it” the root, the leaf, the fruit, a water extract of the root, a water extract of the fruit, or maybe it could be the leaf, or maybe it was a 70% alcohol extraction, or maybe a pure alcohol extraction, etc., etc. It is no longer appropriate to simply say “hawthorn” is good for this and that unless one is specifically talking about hawthorn berry in its traditional context where it can be understood to be used in certain traditional ways (e.g., tea or decoction). Anything deviating from tradition must be clearly identified in modern scientific studies, otherwise no correlation can be made between scientific findings and the herb. The continued sloppy treatment of hawthorn (or any herb) as if it were a pure chemical drug must stop! This way, we can start eliminating poor-quality or dubious work from the new literature, which, in turn, will eventually reduce the confusion in the field of herbal medicines and supplements. As I have said it before, if we don’t do something meaningful to stamp the flow of the polluted information pool soon, the field of natural medicines/dietary supplements will be drowned in it [Issue 24, p. 3; Issue 27, p. 2].

How to improve the quality of scientific data on herbs

          In order to do this, we have to address the problem at 3 levels: (1) research; (2) publication; and (3) abstracting, indexing, and data input into databases.
I am not too concerned about basic scientific research technologies. We all learned those in college and then graduate school, and further honed our skill in our ‘real’ research jobs. What I am concerned about is that most researchers who have not been trained in natural medicines don’t seem to have a feel for the importance of the clear definition of these materials before using them as objects of research. Although we have all been trained in the use of good science in our research, we often ignore it when it comes to herb research. We all know the importance of applying the right quality control to the research material (e.g., a chemical or drug substance) and would never accept one that is not well defined. For instance, we would never think of investigating the effects of alcohol on, say, human cognitive function, by using a liquid called ‘alcohol.’ Based on our training, it is almost second nature for us to clearly define the alcohol (in this case, obviously ethanol, because it’s not ethical or permissible to use other alcohols on human experimentation), its concentration, purity (does it contain toxic adulterants?), etc. We would never accept just any ‘alcohol’ because it is an ‘alcohol’ or close enough, such as methyl alcohol or isopropyl alcohol. Yet many research studies on herbal medicines/supplements use test materials that are vaguely defined, if at all. Terms such as “ginseng,” “echinacea,” “kava kava,” and “St. John’s wort” have been routinely used alone as sole description of the test materials used in the studies, in at least 1 of the 3 levels mentioned above. In some of the better-defined (but still grossly inadequate) materials used, researchers may use the Latin binomials of the plant species (from which the material is derived), believing that that would be the definitive proof of identity. However, without providing which part of the plant supplies the test material, the scientific name of the plant is meaningless, and the study using this still-basically-undefined material will produce dubious and meaningless results. For example, again using hawthorn, how meaningful would the study of “Crataegus monogyna Jacq.” be? Here we have ‘scientifically identified’ English or one-seed hawthorn. But it has no relevance to the material under study. Is it the leaf, the seed, the fruit, the flowers, the root, or a combination of these parts? Also, is it the powdered crude material (whatever plant part) or is it an extract in water, ethanol (%?), or other organic solvent? And if so, how concentrated is it? Also, is it standardized to chemical markers, and which ones? These are just a few variables that must be clearly defined for resulting data to be reproducible and usable. Amazingly, many reports still simply use the name of the species (e.g., ‘hawthorn’ or ‘Crataegus monogyna’) to describe the materials used in the studies, at least in level 3 and sometimes also in level 2. No wonder we are suddenly being inundated with so much herb research data that are dubious or ambiguous and controversial! Which reminds me of how the use of Latin binomials to define a plant species is only as valid as the competence of the person who uses it. I used to know a flavor chemist who was in charge of the laboratories of an extraction company. He had the Latin binomials of most plant species associated with the materials being extracted at his plant memorized, though he had no clue as to how to identify the plant materials other than a few very common ones used in flavors, such as fenugreek seed, carob pod, chili pepper, coffee bean, etc. He was quick to put in the Latin binomials on his certificates of analysis, even though he might have no idea whether the materials extracted were actually from the plant species he put down on his certificates of analysis. Yet he was very proud of the fact that he knew the Latin binomials of plants and followed ‘standard scientific protocol’ when making his reports (certificates of analysis). To me, this is a typical case of a theoretical intention that does not always lead to a proper successful execution. Furthermore, in this case, it gives one a false sense of security by believing a correctly identified test material has been used. I suspect this situation is not unique. It probably applies to many companies involved in herb research and/or manufacture as well as to academia and other research institutions.
          The key to improving the quality of scientific herb research data lies in the clear definition of the test materials (crude botanicals and different forms of extracts) used in any research. Currently, there are no official or universally recognized guidelines (or criteria) for defining such materials. The late Dr. Varro (Tip) Tyler and I have independently addressed this issue (Issue 19, pp. 2-3; Issue 35, pp. 2-3).1 In my article, I have actually provided guidelines for defining these test materials, which can and should be used at all 3 levels. Those criteria were first published in the March/April 1999 issue of this Newsletter and later reprinted in the November/December 2001 issue. Yet to date, none of the journals in our fields (pharmacognosy, natural products, phytomedicine, herbal medicine, Chinese medicines, ethnomedicine, etc.) which should be the leaders in herb research, has set minimal criteria for defining test materials as conditions in accepting manuscripts for publication. I just don’t get it. How can my colleagues, who are publishers, editors, or reviewers of these journals and who are supposed to be experts in this field, continue to allow the publication of herb research data that are often not worth the paper on which they are printed? Furthermore, the longer these publications are allowed to continue to inundate us with ambiguous and dubious (some egregious) information, the more expensive and difficult it will be for us to rectify the problem. Without reliable scientific research data on which to base further research on botanical medicines or dietary supplements, we will continue to generate irreproducible, ambiguous, dubious, and, yes, controversial data. And we will be wasting our taxpayers’ money by continuing to support such research. The end result would be a declaration by the pharmaceutical and medical professions telling the world ‘I told you so,’ supported by scientific ‘evidence’ from research sponsored by our government.
          The only way to rectify this whole mess and stop wasting any more money and energy (first to produce dubious/ambiguous data and then try to deal with the controversy and more misinformation generated by these data) is to immediately institute criteria or guidelines for researchers, journal editors/reviewers, and data entry professionals (abstractors, indexers, database managers) to clearly define test materials before being accepted for research, publication, and/or incorporation into databases. These criteria should be instituted at all 3 levels – research, publication, and database. To start, they don’t need to be all-at-once comprehensive. The most urgent need is for these criteria to be there so that professionals at all 3 levels of the information generation and dissemination chain will be aware of the futility and wastefulness of dealing with undefined research materials. The key is to get them to stop equating a natural test material to a pure chemical drug or a plant species (common or scientific name), and to start thinking about what that test material actually is or should be. This will be the only way to ensure that the information generated from botanical medicine/supplement research has a universally acceptable level of quality, which scientific researchers from healthcare and related fields may use with confidence to develop new, credible, and useful data.
(1) Tyler, V.E., Scientific Review of Alt. Med. 4(2): 17-21(2000).
(2) C.A. Swanson, “Suggested guidelines for articles about botanical dietary supplements,” Am J Clin Nutr 2002; 75: 8-10.