Meer ‘wetenschap via persbericht’
Tijdens een onderonsje van anti-rokers in Schotland werd afgelopen week een nieuw ‘onderzoek’ gepresenteerd dat zou aantonen dat na de invoering van het rookverbod het aantal opnames in ziekenhuizen wegens een hartaanval met 17% zou zijn verminderd.
Weer een geval van ‘shoddy science’ (krakkemikkige wetenschap) volgens Dr. Michael Siegel van de Universiteit van Boston. En het wordt zelfs steeds erger: werden soortgelijke onderzoeken zoals de Helena- en Pueblo-studies nog in een wetenschappelijk tijdschrift gepubliceerd, bij dit onderzoek ontbreekt alle controle op enige wetenschappelijke zorgvuldigheid via ‘peer-review’.
En de wetenschappelijke kwaliteit van dit onderzoek lijkt ook aan alle kanten te rammelen: er werd slechts in 9 van de 360 Schotse ziekenhuizen gekeken naar de hoeveelheid opnames voor hartaanvallen. De kans dat daarbij de krenten uit de pap zijn gehaald, om de positieve effecten van de invoering van een rookverbod in de horeca te onderstrepen, is metersgroot.
“Voor zover ik nu kan zeggen, is er onvoldoende bewijs om te kunnen concluderen dat de afname van het aantal opnames wegens hartaanvallen in Schotland aan het rookverbod te danken is. Die conclusie is voorbarig en zou niet openbaar gemaakt mogen worden door anti-rokengroepen zolang de studie zelf niet gepubliceerd is en kan worden beoordeeld door alle wetenschappers“, aldus Dr. Siegel.
I have now confirmed that the Scottish smoking ban study which I discussed earlier this week is indeed an example of science by press release. An article from the Times Online confirms that the study was presented at a conference in Edinburgh on Monday and that the study has not yet been published. Furthermore, the media coverage of the study was not a coincidence; it was the direct result of a press release that was issued by the University of Glasgow (home of the lead study researcher) on Sunday. Finally, I was unable to find the actual study, so it does not appear that the study has been made readily available for review.
According to the press release: “A University of Glasgow study has found a 17 per cent fall in admissions for heart attacks in the first year after the smoking ban came into force. The evaluation, led by Professor Jill Pell from the University’s BHF Cardiovascular Research Centre, found that after the legislation came into force there was a 17 per cent reduction in heart attack admissions to the hospitals. This compares with an annual reduction in Scottish admissions for heart attack of 3 per cent per year in the decade before the ban.”
Also of interest, data obtained from ISD (Information Services Division) Scotland reveal that between 1998/1999 and 1999/2000, total hospital discharges for acute myocardial infarction (heart attacks) in Scotland dropped from 19,216 to 17,208, a decline of 10.4%. This large single-year decline occurred during a period in which the smoking ban was not yet in effect.
These data also reveal that there was a steady decline in heart attack admissions in Scotland between January 2005 and September 2006 (the smoking ban was implemented in March 2006). From December 2005 to February 2006 (prior to the smoking ban), there was a 16.7% drop in heart attack admissions for the regions for which data were consistently reported throughout the study period.
The Rest of the Story
It does appear that the Scotland smoking ban/heart attack study is yet another example of science by press release. I view it as inappropriate and perhaps unethical to disseminate the results of a study widely to the media and the public before the research has been peer reviewed and published, unless one also releases the study itself so that the underlying methodology can be examined. Otherwise, we are all completely at the mercy of the researchers and there can be no independent review of the study findings and their validity.
In my career to this point, I have never released study findings to the public unless: (1) they had been peer reviewed and published; or (2) they were not yet published but I made the study readily available for review. I think that if you are going to release the results of a study, then it is your obligation to make the complete study (including the methodology and actual raw data) available for public review.
This science by press release approach is contributing to the decline in scientific integrity in the tobacco control movement.
To see why a thorough review of the actual study is necessary, just look at the hospital discharge data for heart attacks in Scotland. We find evidence of a large single-year decline in heart attacks of about 10% which occurred in the absence of any smoking ban. This indicates that there can be a large year-to-year variation in heart attacks in Scotland and makes it especially difficult to determine whether a 17% annual decline in heart attacks is due in part to random variation, and if so, to what extent. From the information that was provided to us, it is impossible to conclude that the smoking ban was responsible for the observed decline in heart attack admissions.
It is also important to consider the fact that the more rapid decline in heart attack admissions in Scotland appears to have begun prior to the smoking ban. A drop in heart attack admissions of 17% (equal to what was observed after the smoking ban) actually occurred during the three-month period immediately prior to the smoking ban. It would be very questionable to attribute the observed heart attack decline to the smoking ban in the face of this evidence that heart attacks were declining substantially prior to the ban. It is entirely possible that some other factor is responsible for the observed decline.
The point is – simply – that we don’t know. The evidence is not sufficient to support the researchers’ contention that the smoking ban in Scotland has caused a 17% decline in heart attacks. Without having the complete study in front of us to review, we cannot even judge the validity of the researchers’ contention (except to say that based on the data that are available and have been presented, there is not sufficient evidence to support their conclusion).
Does it really advance the cause of accurate scientific reporting to have the media widely report these conclusions to the public in the face of this complete lack of documentation and support and in the absence of any possible way to confirm the validity of the results?
From what I can tell at present, there is not sufficient evidence to conclude that the observed decline in heart attack admissions in Scotland is attributable to the smoking ban. That conclusion is premature and should not be disseminated further by anti-smoking groups until we have the actual research study in front of us and can review it.
Why the rush to release the results before they were published? One has to wonder.
And one final question: Why does it seem that it is the weakest studies that are disseminated by press release prior to their actual peer review and publication?
Scottish Smoking Ban Study is Example of Science by press release.
