WWW.truthinlabeling.org -- Home page
Criticism of design and methodology of one glutamate-industry study
Letter to the Editor
Re: Monosodium L-glutamate: A double-blind study and review.
L. Tarasoff and M.F. Kelly. Food and Chemical Toxicology (1993) 31, 1019-1035.
Sir,--Tarasoff and Kelly have claimed to address the methodological problems inherent in previous investigations of the subjective effects of monosodium L-glutamate (MSG), resolve these problems, and demonstrate that MSG is "safe" (Tarasoff and Kelly, 1993).
The study, however, has sufficient methodological flaws to make its findings inconclusive. In addition, there are numerous examples of questionable data handling, and use of imprecise and/or questionable statements.
Following are the six most flagrant violations of sound methodology found in the Tarasoff/Kelly study. There are others. Any one of the following would be sufficient to cause one to question the results of the study.
1) Subject selection virtually assured the authors that subjects would not be sensitive to MSG.
A) Subjects were limited to "healthy volunteers." According to the authors, "Questions about pre-existing conditions such as pregnancy, asthma, general allergy syndromes, epilepsy (convulsion or fits) aspirin sensitivity, prescribed medication...and other conditions were...asked. Subjects were selected if they answered 'no' to all questions, were not taking prescription drugs and considered themselves to be in good health." This screening procedure would have effectively eliminated anyone who expressed the symptoms of MSG-toxicity and, therefore, would have effectively eliminated anyone who was sensitive to MSG. (See paragraph two of Point #3 below for a discussion of reported adverse reactions to MSG.)
B) Subjects participating in the study had to give informed consent. The study was described as including "treatments that might contain flavour enhancers." It is extremely unlikely that a person who knew he was sensitive to MSG would volunteer to participate in a study where he might be asked to ingest a flavour enhancer. MSG is advertised as a flavour enhancer.
2) The observation/reporting period following ingestion was inadequate. Subjects were queried about their reactions two hours after ingestion of test or placebo material, even though people who react adversely to MSG react anywhere from immediately up to 72 hours after ingestion.
3) The authors failed to consider all of the reported reactions to MSG. They apparently limited "acceptable" responses to "sensations" reported in response to the question, "Did you experience any sensations other than taste after breakfast this morning?" On this basis, had there been a sample of subjects representative of the general population, most of the adverse reactions experienced following ingestion of MSG would have been discounted.
Ingestion of MSG is known to produce a variety of adverse reactions in certain people, ranging from such things as simple skin rash, listlessness, bloating, nausea, vomiting, lightheadedness, and irritable bowel syndrome, to migraine headache, tachycardia, asthma, hyperactivity in children, convulsions, and depression. These reactions, although seemingly dissimilar, are no more diverse than the reactions found as side effects of certain neurological drugs.
4) The authors failed to control for MSG (free glutamic acid found in food as a consequence of manufacture) and/or other reactive substances in the diets of subjects during the course of testing. Even the Standard Breakfast provided for subjects following ingestion of test and placebo materials, which contained muesli bars and flavoured milks, was questionable. The ingredients of the Standard Breakfast were not elucidated. Had there been a sample of subjects representative of the general population, it would have been impossible to determine whether adverse reactions were reactions to test and/or placebo materials, per se, or to MSG or other reactive substances in the diet.
5) Even if the authors had controlled for reactive substances in the diet, results of the study would have been invalidated by failure to space test and placebo trials far enough apart to assure that reported reactions were reactions to the last test or placebo material ingested, as opposed to material ingested one, two, three, or four days previously; or to minimize the possibility that reported reactions were caused by accumulation of excess amounts of glutamic acid in the body. It is known, for example, that elevated levels of brain glutamate are found in persons who die with neurodegenerative diseases such as ALS, Parkinsonism, and Alzheimer's disease.
6) Had the authors failed to eliminate MSG-sensitive people from the study, the use of aspartame in the test and placebo materials in what the authors called "amounts sufficient to 'mask the taste' of MSG," would probably have guaranteed that approximately the same number of people who reacted to the MSG treatment would respond to the placebo. (The amount of aspartame used in test and placebo materials was not elucidated.) Aspartame contains aspartic acid, a structural analogue of the glutamic acid found in MSG. More important, both of these neurotoxic amino acids are known to load on the same receptors in the brain, kill brain cells, cause neuroendocrine disorders in laboratory animals, and work in an additive fashion; and adverse reactions reported to the FDA by consumers after ingesting MSG and aspartame are not only of the same kind, but essentially occur with the same relative frequency. There are presently over 6,000 reports of adverse reactions to aspartame on file with the Adverse Reactions Monitoring System at the FDA. Aspartame is not an inert substance.
If a subject reacted to ingestion of MSG and did not react to ingestion of an inert placebo, one would generally conclude that the subject was sensitive to MSG. If a subject reacted to ingestion of MSG and also reacted to the placebo, one would have no way of knowing whether the subject was sensitive to MSG; for no matter why the subject responded to the placebo, he might never-the-less be sensitive to MSG.
The use of aspartame in placebo materials is of particular importance because the International Glutamate Technical Committee argues that if a subject responds to a placebo, he can not possibly be sensitive to MSG. The logic is faulty. Moreover, in science, if a substance is even suspected of causing a reaction in a given person, it is inappropriate to use that substance, with that person, as a placebo. Tarasoff and Kelly used placebos containing aspartame in the drinks used in this study.
In addition to methodological flaws, the study suffers from questionable data handling. A sample follows:
1) Data of subjects who reported an after-taste, were excluded from some data analyses. It is most irregular that the data from all subjects were not reported.
After-taste is a reaction often reported following ingestion of aspartame, a substance used here in both test and placebo material of the drink portion of the experiment. However, since after-taste was reported following ingestion of both drink and capsules, one must consider that there might have been aspartame present in the capsules, or in one or more of the flavored muesli bars or flavoured drinks in the Standard Breakfast. The ingredients of the Standardized Breakfast were not elucidated. The authors maintained that "the presence of after-taste allowed the experimenter to deduce if the powerful taste of MSG had been detected. In such cases both the experimenter and the subject could not be considered blind and the results were not used."
2) Data were not reported and analyzed according to absence or presence of reactions to test and placebo materials, but were reported and analyzed by the "intensity" of the reactions as reported by subjects.
3) Inferences from statistical tests were defined by the authors, for purposes of this study, as being significant if the level of confidence was less than .05. On page 1031, however, a Spearman's rank-order correlation coefficient for "the difference in intensities of sensation between treatment and placebo against MSG dose," with P=0.33, is said to be statistically significant. Since .33 is greater than .05, the statement that the correlation is significant is not true. It is obvious that this is not a simple typographical error, for the first paragraph in the Discussion section reads, "The significant effect of food in negating the effects of MSG is illustrated by the small but significant negative correlation coefficient between MSG dose and after-effects."
4) The authors devised an index variously called "the difference in intensities of sensation between treatment and placebo" (page 1030); the "placebo adjusted response intensity" (page 1030); "after-effects" (page 1019 in the Summary and page 1031 in the Discussion); "responses" (page 1031); and "the effects of MSG" (page 1031 in the Discussion). The index is referred to on page 1030 of the report, but details of its development were not given. A close examination of Figure 1, however, suggests that the index was developed for each subject by subtracting the "intensity score" of the response to the placebo, from the "intensity score" of the response to the test material containing MSG.
A placebo intensity score of 5, for example, subtracted from an MSG treatment intensity score of 2, would yield an index of -3. Any negative index would indicate that on the day of the placebo treatment, there was a more intense reaction than that which occurred on the day of the MSG treatment. There would be nothing in the score, however that would explain why there had been a difference in intensity of reactions. If the placebo treatment followed the MSG treatment, one could argue that there had been a cumulative effect from all of the neurotoxic amino acids ingested up to, and including, the day of the placebo treatment. If the placebo treatment followed the day after the 3.0g or the 3.15g MSG treatment, it could be argued that the reaction was an MSG reaction, expressed 25 hours after MSG treatment, not a placebo reaction expressed one hour after placebo treatment. Any one of a number of hypotheses could be offered, none of which could be evaluated given the available data. The only certainty is that in this study, a reaction following ingestion of a placebo that is more intense than a reaction following MSG treatment, would not necessarily indicate that the subject was not sensitive to MSG.
5) The authors paired each subject's index score (which was discussed in Number 4 above) with the amount of MSG per kilogram of body weight ingested, reported two significant negative correlations, and concluded that they had thereby demonstrated that food canceled the effects of MSG. Even if the correlations were significant, the conclusion would not have follow from the data presented. Since the authors did not vary the amount of food given to subjects in this study, there is no logic that would allow one to conclude that food negates the effects of MSG.
The first paragraph of the discussion section reads, "The significant effect of food in negating the effects of MSG is illustrated by the small but significant negative correlation coefficient between MSG dose and after-effects." ("After-effects" is one of the names for the index.)
6) The authors have stated that "Sensations, previously attributed to MSG, did not occur at a significantly higher rate than did those elicited by placebo treatment." The authors concluded that failure to find a significant difference demonstrated that subjects were not sensitive to MSG.
But looking at the incidence of reactions to MSG (which are not discussed in the body of the paper, but will be found in Table 6, if the table is studied carefully), we find that considering drink and capsule treatments separately, 27% of the subjects responded to the MSG treatment without responding to the placebo. That fact was not reported.
7) A complete analysis of data would have included information on order effects. Readers should have been told whether not number of responses, and/or the "intensity" of responses, increased over time with repeated treatments. That information was not provided.
8) Wilcoxon's Signed Ranks Tests were used to evaluate much of the data. The Wilcoxon's Signed Ranks Test is used to compare two random samples of matched measurements. It is a non-parametric statistic inappropriate for use with repeated measures unless it is order effects that are being evaluated. In this study, treatments were not independent, but followed one another on consecutive days; and analyses were done without consideration of order effects. Therefore, any differences in the dependent variables (reactions to test and placebo materials) could just as well have been due to order effects as to treatments. Use of the Wilcoxon's Signed Ranks Test was inappropriate.
The Tarasoff/Kelly report also suffers from use of imprecise and/or questionable statements:
1) In the summary at the beginning of the paper, the authors stated that "An exhaustive review of previous methodologies identified the strong taste of MSG as the factor invalidating most 'blind' and 'double-blind' claims by previous researchers." It must be noted that there was no exhaustive review of the literature (the 19 studies of reactions to MSG cited and criticized being far from "exhaustive"); there was no evidence presented that there is a strong taste of MSG; and there was no evidence presented that suggested that a strong taste of MSG (if it existed) should serve to invalidate an otherwise properly executed double blind study.
2) The authors stated that "Ideally, a placebo treatment should match the MSG treatment in size, colour, texture and taste." That statement is not true.
The purpose of a placebo is to provide a means of identifying possible psychological responses to a test situation -- responses that might invalidate responses to test material. Therefore, the placebo must be made of material that would not, by virtue of its physical composition, cause a reaction in the subject taking it. In a clinical setting, a placebo to be used with a given patient should be tested to be certain that it does not cause the patient an adverse reaction in a non-threatening, non-experimental situation. If it passes that test, it can be used as a placebo in a double-blind study. Ideally, then, a placebo treatment should be one that will not, by virtue of its physical properties, cause the subject to react. Matching the size, colour, texture and taste of the placebo to the test materials would not be necessary in a well designed, well executed study. Pretending that a placebo that is matched with the test material for size, colour, texture, and taste is necessarily a proper placebo, is deceptive.
3) A chemical analysis of the placebo and monosodium glutamate (MSG) drinks was given in Table 5. There were seven factors accounted for; but there was no mention of aspartic acid. No analysis was given of the capsules, even though gelatin contains free glutamic acid.
4) On page 1031, a significant correlation was reported when there was none. The subject has been discussed in Item #3 on Page 4 above.
5) The authors claimed that "The profound effect of food in negating the effects of large MSG doses was demonstrated." That statement is untrue.
6) It is claimed that large doses of MSG were used, when the doses were only 1.5g, 3.0g and 3.15g. One can hardly consider 3.15g a large dose when a single eight ounce serving of Swanson clear chicken broth contains almost half a gram of MSG (Swanson, 1992) and it is common to find 5g or more MSG in a Chinese meal.
7) The statement was made that "...rigorous and realistic scientific evidence linking the [Chinese restaurant] syndrome to MSG could not be found." The authors made no attempt to look for it.
8) There was no mention made in the text of the paper, that 19 out of 71 subjects (27% of the subjects) responded to either the capsule or drink test material and not to the corresponding placebo. (See Table 6 of the Tarasoff and Kelly study.)
9) The Discussion section is replete with statements such as the following statement taken from paragraph 1, that are full of words, but have no meaning.
"Given that the subjects had fasted for more than 10 hr before treatment, one could rationalize this observation [the significant effect of food in negating the effects of MSG is illustrated by the small but significant negative correlation coefficient between MSG dose and after-effects] by consideration of the caloric content of MSG (13.14 kJ/g) and the ease with which it could be incorporated in the Krebs cycle by transanimation to a-ketoglutarate in the intestinal mucosa (Stegink et al. 1983). In other words, the effect may have been equivalent in energy to approximately 2.5 g sucrose before breakfast."
It is hard to believe that Drs Tarasoff and Kelly, from the
Faculty of Business and Technology of the University of Western Sydney,
Australia, Macarthur Campus, designed this study and understood the statistical
analyses used, or that they understood that they had effectively eliminated
MSG-sensitive people from their study and had, in addition, virtually
guaranteed that if there were people who responded to the MSG treatment,
there would be like numbers that responded to the placebos. It is
important to note that the International Glutamate Technical Committee (the
same group that supplies placebo materials laced with aspartame to researchers
engaged in double-blind studies on the safety of MSG) supported this study.
Adrienne Samuels, Ph.D
Food and Chemical Toxicology
(1995) 33:(1), 69-78..
Swanson clear Chicken Broth marked Dec 92 14:57 CXH33RHN was found to contain 59.2 mg MSG/oz.
Tarasoff L. and Kelly M.F. (1993) Monosodium L-glutmate:
a double-blind study and review. Food and chemical Toxicology 31,