(CHINESE) HERB NEWS
Herbs in this issue:
Peer review – where are the peers?
Over the past 5 years, I have written frequently about this subject, especially on the poor quality of peer-reviewed publications in the natural products and herbal medicine fields (Issues 18, 23, 27, 28, 29, & 37). In this issue, I want to plead guilty as one of the peer reviewers who has contributed to one of the major flaws in this peer-review system.
One of the most sacrosanct subjects in the advancement of scientific endeavors is the peer review process. It is the system on which all legitimate scientific publications, recognition, and awards (e.g., grants and contracts) are frequently based. If you are a scientist and want to publish your findings, you must let your peers review your work before publication, otherwise what you publish will not be considered scientifically legitimate. I don’t know too much about other scientific disciplines, but in the field of natural medicines and natural products in general, I have often noticed publications in peer-reviewed journals are of obviously poor
Dr. Leung is author of the Encyclopedia of Common Natural Ingredients Used in Food, Drugs, and Cosmetics (Wiley-Interscience), which was published in 1980 and revised in 1996. He is also creator of PHYTOMED, a prototype computer database on Chinese herbal medicine developed under contract with the National Cancer Institute.
quality. My first thoughts are, “Where are the peers?” and “Who are these peers?” The current system, although well intentioned and theoretically sound, is flawed in practice as described earlier (Issue 37). I know what is wrong in the peer review process in the herbal field, because I have been through it myself as a reviewer of both grant applications for the newly formed Office of Alternative Medicine (which eventually led to the establishment of NCCAM and ODS) and of manuscripts for journals. Unfortunately, politics and power plays exist within this benign-looking system.
Until about 18 months ago, I was one of those experts who agreed to be on the technical review board of the Herb Research Foundation which co-published the HerbalGram with the American Botanical Council. I used to spend maybe a couple of hours here and there checking out anything relating to TCM for the journal. Anything requiring more time I simply couldn’t afford to do. As some of the manuscripts in the TCM field started to get more and more verbose and ludicrous, I sometimes simply glanced over them and often returned them without serious comments. It was under such circumstances that a verbose and rambling article on ginseng (18 pages, approximately 10,000 words!) was published almost 2 years ago. Unintentionally, I had released a ‘monster’ that is full of untruths, hearsay, fabrications and wishful thinking. This incident just helps to show how easy it is to have poor-quality articles published by any peer-reviewed journal. I tried to rectify the situation immediately after I had a chance to scan the article by writing to the editor of HerbalGram. My concern, as always, is the truth in herbs. Since the article is extremely lengthy, with beautiful pictures and all, it looks very impressive on the surface. I was afraid its uninformed readers on ginseng would think that it held the ultimate truth on this subject. Hence its egregiousness needed to be revealed. However, the editor refused to publish my letter, and his rationale was that I approved it as a reviewer. The author (even privately) did not acknowledge the mistakes (I only pointed out a few of the many he made).
A major difference between the herbal field and other scientific fields is that much of its knowledge lies in industry and in a small segment of the federal and state governments. Unlike experts in academia and in government who draw salaries and perform pro bono work (such as reviewing manuscripts) as part of their paid duties, those in industry don’t have that luxury. This is especially true for a number of independent consultants and small-company employees who simply can’t afford to donate their otherwise billable or productive company time to shape up their colleagues’ poor manuscripts, often without getting due credit, or write chapters of books that financially benefit only the publishers.
Here comes the thorny issue for peer reviewers. Are they bound to review all the manuscripts or do they just do whatever they can or have time for, once they are listed on a publication’s review board? I am sure other reviewers sometimes just glance at a long manuscript and ‘approve’ it. I believe I am among the majority of such reviewers in the field of herbal medicine and natural products, otherwise there would not be so much misinformation, disinformation and untruth in publications on this subject. If the reviewers really took time (pro bono) to review long manuscripts, such as the ginseng article in question, they would have to spend at least 10 to 15 hours to do them justice. For myself, I probably would have to spend more time than that because I am a fussy and meticulous reviewer as well as a slow worker. Because of my continued concern regarding this ginseng article in question, I have decided to publish my letter to the editor in this issue, unchanged.
To: Editors of HerbalGram
From: Albert Y. Leung, Ph.D., Editor, Leung’s (Chinese) Herb News
Subject: Comments on ginseng article in HerbalGram 54 (pp. 34-51)
Date: April 17, 2002
I have always enjoyed your publication and considered it of high quality until the ginseng article in issue 54 written by Subhuti Dharmananda.
It is interesting to read about ginseng in an article like this which is written not based on original Chinese references but rather almost exclusively on translated or interpreted information. It appears that the author either does not read Chinese or has no access to first-hand Chinese literature, otherwise he would have discovered that the secondary references on which he based his information leave a lot to be desired. Nevertheless, he does seem to have some good insight in Chinese medicine and I wish to extend to him my compliments. However, like writing on Greek classics without being able to read the Greek language, one is bound to encounter misinterpretations and missed information. And it’s no exception with his ginseng article.
Much of the information in this article appears to be based on opinion or hearsay rather than on fact, because few of the statements (some seem to be from fairy-tale books) are substantiated by credible published Chinese data. This makes it extremely hard for anyone to determine the veracity of its contents without spending hours or days to research it. I am no expert in the practice of traditional Chinese medicine or Chinese medical history but I have always been interested in these subjects. Hence over the years I have accumulated a rather extensive library resource of Chinese books and journals in this field. I feel I have an obligation to the readers of HerbalGram at least to make a few comments on statements in the article which are obviously not substantiated by reliable and credible sources. My comments here are based on published articles written by Chinese experts in their respective fields in Chinese journals [here specifically, Zhongyi Yanjiu (TCM Research) and Zhonghua Yishi Zazhi (Chinese Journal of Medical History)] as well as monumental works like the Zhonghua Bencao (The Chinese Herbal) compiled by over 500 experts from 60 academic and research institutions in China, which I have reviewed in Issue 30 of my Newsletter.
1. I am rather puzzled by the author’s statement on p. 34, “The basic framework of the Chinese culture coalesced around a group of ideas and practices that were introduced during the period 500 B.C.E. to about 200 B.C.E. Essential contributions included development of the writing system based on ideographic characters;…” I assume B.C.E. is the same as BC. Does he mean Chinese culture with its written language started only around 500 to 200 BC? If so, how could the well-known Chinese classics, Shi Jing 诗经 (‘Book of Songs’) and Shan Hai Jing 山海经 (‘Mountain and Sea Classics’) have already been written during the period 1065 BC to 771 BC? And these are not simply primitive ideograms on cave walls; they are detailed works on their respective subjects. It is well known that the development of the Chinese written language dates back many centuries prior to that period. Also, unless the Chinese scholars are totally wrong, the author’s placement of Wu Shi Er Bing Fang’s origin at 300-200 BC is contrary to historical records. Based on comparisons of writing style and complexity of ideograms in the Wu Shi Er Bing Fang with those in the Shi Jing, the origin of the Wu Shi Er Bing Fang was placed in the same period as the Shi Jing, which is many centuries earlier than what the author states.1 Although the first Chinese herbal exclusively devoted to drugs, the Shen Nong Ben Cao Jing, is normally ascribed to the period between 200 BC and 100 AD, one recent publication places it at 140 BC to 87 BC while another puts it between 320 BC and 104 AD with a most probable date of 72 BC (at 90% confidence level).2,3
2. On p. 40, the author claims that the warm nature of Chinese ginseng was only first described in an herbal published in 1757 AD, again relying on a secondary reference. The fact is that its warm nature has been documented in various major herbals since around 500 AD (over a thousand years before the author’s claim) when Tao Hong-Jing first described Chinese ginseng as slightly warm (wei wen) in his Ming Yi Bie Lu, even though the Shen Nong Ben Cao Jing (circa 72 BC) describes it as “a little cold,” as correctly quoted by the author on p. 35. This information is summarized/abstracted in some of the major Chinese compilations on the subject, such as the Zhong Yi Da Ci Dian (Encyclopedia of Chinese Materia Medica; 3 volumes, 1977) and the Zhong Hua Ben Cao (The Chinese Herbal, 10 volumes, 1999). One does not need to learn to read the Chinese classics or to actually research the classic herbals to find this information, though one must be able to read substantial Chinese. Thus, as quoted in The Chinese Herbal, ginseng is described as warm in another early herbal, Yi Xue Qi Yuan (circa 1186 AD), almost six centuries earlier than the author’s date. This fact should cast doubt on the validity of his lengthy arguments leading to his incorrect statement.
3. On p. 41, in describing substitutes for Chinese ginseng, the author writes: “… In a few cases, the active constituents of the substitute herbs are similar. For example, modern investigations reveal that the saponin components of platycodon, or balloon flower, (Platycodon grandiflorus (Jacq.) A. DC., Campanulaceae) have a structure very similar to that of the ginsenosides….” As supporting authority he quotes two well-known references, but neither of which describes platycodon. This is the first time I have read about platycodon (Platycodon grandiflorum) containing saponins with structures “very similar” to those of the ginsenosides. Does the author mean that both saponins are “very similar” because they are both triterpene compounds? If so, why not include reishi (ling zhi), dandelion, licorice, astragalus, and many others that also contain these compounds?
4. My understanding is that HerbalGram articles are supposed to educate readers/consumers and provide them with guidance in their research and use of herbs. Although on first glance this ginseng article is visually quite impressive, it does not serve either purpose. It is first of all confusing. And, since most of the author’s statements are not referenced and he appears to rely exclusively on haphazardly available translated and interpreted materials (not systematic Chinese literature) whose veracity cannot be determined without comparing them with the original Chinese materials, it is a research project in itself just to try to verify whether they are fact or fancy. I would expect the Summary and Concluding Remarks to contain warning on Asian ginseng’s potential abuse and adverse effects. But instead, the author mixes and confuses therapeutic uses in Asia with commercial Western products containing Asian ginseng, many of which are manufactured with low levels of ginseng either knowingly (to be sure consumers don’t get hurt; benefits being a secondary concern) or unknowingly (with diluted or adulterated ginseng supplied by unscrupulous suppliers). I don’t see any mention of Asian ginseng’s contraindications and cautions, which I believe is extremely important, considering so many healthy young people, especially athletes, are using it to increase ‘energy.’ According to traditional usage, Asian ginseng is not to be used by people who have too much yang energy as in re zheng (‘heat syndrome’) and shi zheng (‘sthenia syndrome’ or ‘full of energy’). Also, among the traditional uses listed, there is no mention of two of Chinese ginseng’s major uses – to treat impending collapse due to exhaustion of vital energy marked by a faint pulse (ti xu yu tuo or qi xu yu tuo) and general weakness in chronic illness or after a chronic illness.4,5 Perhaps it is because of Asian ginseng’s ability to revive a collapsed person that it has been misinterpreted/mistranslated as good for boosting one’s energy in English writings. The truth is that a healthy vigorous person (e.g., athlete) is the last person who needs, or should take, Asian ginseng. It would be interesting to find out if any of the recent fatal cases of athletes had taken Asian ginseng along with other drugs or supplements.
5. The above are comments on only the most obvious errors and deficiencies in this article. I couldn’t spend any more time to verify other apparently incorrect statements. And I simply can’t take the author’s word for their accuracy without his revealing the sources on which he based his statements. I am sure my knowledgeable Chinese-reading colleagues can’t either.
(1) Sun Qi-Ming, “Deduction of the date of compilation of the silk scroll of Wu Shi Er Bing Fang based on comparison of its ideagrams with those of the Shi Jing,” Zhonghua Yishi Zazhi (Chinese Journal of Medical History), 16(4): 243-246 (1986).
(2) Ma Bo-Ying, “Research on the compilation date of the Shen Nong Ben Cao Jing using anthropological methods,” Zhongyi Yanjiu, 5(1): 44-46 (1992).
(3) Liang Mao-Xin, Li Dong-An, and Wang Pu-Min, “Determination of the compilation date of the Shen Nong Ben Cao Jing using quantitative analysis,” Zhonghua Yishi Zazhi (Chinese Journal of Medical History), 23(1): 60-63 (1993).
(4) Pharmacopoeia of the People’s Republic of China, 1995, pp. 4-5.
(5) Zhonghua Bencao Editiorial Committee, Chinese State Administration of TCM, Eds. Zhonghua Bencao (The Chinese Herbal), Abridged Version, Vol. 1, Shanghai Scientific and Technical Press, Shanghai, 1998, pp. 1269-1294.
In addition to the gratification derived from bringing the truth about herbs to my readers and exposing unethical practices in this field, a major satisfaction in writing this Newsletter is knowing that after putting a lot of serious thought into my writing, no one messes with it. I am totally responsible for this Newsletter’s contents because, unlike ‘peer-reviewed’ articles, I don’t have any peers but myself to blame for my mistakes. Hence, I am happy to stand behind my writings and to acknowledge any mistakes I make. However, with that ginseng article, I have a feeling its author and the editor might be blaming me for its mistakes because I ‘signed off’ on it. Did they actually expect me to spend a couple of days rewriting the ginseng article for them? If so, that’s real chutzpah! In any case, I am no longer reviewing manuscripts for HerbalGram – I resigned from their review board at the time of the above letter.
Here is my dilemma regarding peer review. On the one hand, it is such an important concept and tool for scientific advancement that we must preserve it. Yet, on the other hand, in its present form, it does not seem to work, at least in the natural medicines/products field. Nevertheless, we need to improve it. But how? I see at least 3 areas that should be improved: (1) We need to use more professionalism in selecting peer reviewers. Being the publisher’s or editor’s friend or a monetary contributor to his/her journal is not good enough as the major criteria for selection. (2) There must be some means we can find to compensate reviewers for their time and efforts spent in reviewing and correcting manuscripts. Maybe journals should charge authors of unsolicited manuscripts for reviewing and publishing their work. The money charged can be used to pay the reviewer(s). (3) To make reviewers responsible for their reviews, their names and affiliations should be published on the articles they have reviewed.
I welcome comments from my readers and colleagues regarding suggestions as to how to fix/improve our flawed peer- review system.