Do scientists, engineers, or researchers deserve trust?
This question has been ringing in my head for the past week. I think through our discussion, we concluded that most people initially have trust for others unless they have acted in a way to lose that trust. I considered, “Do I deserve to be trusted?” The common misconception is that education and intellect has magically endowed those to be trustworthy but through my own experience, it seems that those who should be most trustworthy are not. I used to think that there were only a few bad apples in the orchard but maybe there’s a few bad trees in there too. With this weeks reading about published research, it brought light to the issue of valid findings that other professionals, such as doctors, use everyday.
When reading the article about Dr. Ioannidis, it offered a brief glimmer of hope [1]. The unfortunate truth is that most research is under pressure to deliver results (from funding, political pressures, etc.). This leads to researchers ignoring limitations on their study or not including past research or publishing data that may contradict their findings, such as in the case of lead in products and more specifically water. It seems that some researchers are very secretive about their data or methods, maybe they have something to hide. I would consider good science to include limitations and concerns about the conclusions that have been drawn and be open to criticism.
There’s no new “good” science to suggest that lead is anything but harmful to humans, however some have tried to hide its effects through botched research and false data [2]. Still years after the original MMWR, the CDC is still trying to cover up the fact that their original findings were inaccurate and misleading. In both articles, they mention some “limitations” or “misguiding statements” but fail to address the issues brought forth by other researchers and scientists dealing with greater flaws in their work [3,4].
The magic with statistics is that you can manipulate the data and perform various tests to show what you want to see. By including or excluding certain information, or taking just a subset of the whole, correlation or significance can be shown. But in the case of medical research specifically, it’s nearly impossible to relate the results to the true controlling variable given all the complexity of interactions and variability. Oftentimes correlation exists between two variables being tested but that does not always prove causation. Likewise, lack of correlation does not prove lack of causation. When the CDC came out and showed the 300 ppb study, they essentially said, “we see no correlation, therefore lead in water isn’t a problem”. This argument is not only false but was detrimental to the health of others facing elevated WLL. Specific care and caution should be exhibited when drawing conclusions from data and limitations and conflicts in results should be noted.
Also, go read this article, http://uncyclopedia.wikia.com/wiki/Random_Statistics, it brought joy to my day…
Excerpts from article (it has quite a bit of sarcasm in it, so reader beware):
“…pie charts DO NOT have to add up to 100%. That is a lie first embedded in our modern culture by communist soviet spies in the 1930’s.”
“99% of journalists use random statistics but only one in out of every ten of the 12.5% who responded yes to the question know why 68% of the quarter second half are wearing green jerseys. This is not to say that these reporters are lying 100% of the time. Although random, the statistics serve a purpouse. Which is to explain another thing, after saying you were going to explain the first thing, which is not statistically relevant, but your editor thinks its cool.”
Now you really want to go read it don’t you…
References:
[1] Freedman, D. H. 2010. Lies, Damned Lies, and Medical Science. The Atlantic (Nov.), pp. 1-12, http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/8269/.
[2] Markowitz, G. and R. Rosner. 2002. “Old Poisons, New Problems.” In Deceit and Denial: The Deadly Politics of Industrial Pollution, 108-138. Berkeley, CA: University of California Press.
[3] CDC. 2010. Notice to Readers: Examining the Effect of Previously Missing Blood Lead Surveillance Data on Results Reported in MMWR. MMWR 59(19):592, http://www.cdc.gov/mmwr/preview/mmwrhtml/mm5919a4.htm.
[4] CDC. 2010. Notice to Readers: Limitations Inherent to a Cross-Sectional Assessment of Blood Lead Levels Among Persons Living in Homes with High Levels of Lead in Drinking Water. MMWR 59(24):751, http://www.cdc.gov/mmwr/preview/mmwrhtml/mm5924a6.htm.