Sense about Science ? equipping people to make sense of science and evidence
ArchiveView full archive
- Making the government's use of evidence more transparent
- Sense About Science at the METRICS conference
- Submission to the Independent Commission on Freedom of Information
- The vets are coming!
- The Times 10th October 2015
- Peer Review 101
- Peer review matters!
- John Maddox Prize - Guest post by Dr Thelma Lovick
- John Maddox Prize - Guest post by Professor Anthony David
- John Maddox Prize - Guest post by Sir Simon Wessely
Posted by Prateek Buch on 21 January 2016
Sense About Science has been asked by the Cabinet Office and the UK Open Government Network to lead a discussion on making the government's use of evidence more transparent. Civil society organisations and government are working in partnership to produce the UK's National Action Plan on Open Government, and I’m really pleased to be acting as civil society’s leader on proposals to improve the public’s access to information. The proposals include:
- Government departments committing to an evidence transparency framework
- A single register for all government research
- A single point of contact for public evidence requests in every public body
- Improving how freedom of information works to make policymaking more effective and accountable.
If you can help develop these ideas, please come to a workshop we’re holding on Tuesday 9th February where we’ll discuss the aims behind the proposals and what might be done to achieve them. Your input will inform the final Open Government commitments in the National Action Plan.
You can register here for the workshop, which is kindly hosted by the Royal Statistical Society, and co-organised with the Open Data Institute who will lead a discussion on the better use of open data.
To challenge the evidence behind public policy, the public needs to know the findings of government-commissioned research, the role that this evidence has played, and the extent to which contra-evidence, limitations in the evidence base and alternative policies have been considered. But before we can evaluate any of this, we need to know what's been looked at, which means making the evidence base more transparent as promised in the civil service reform plan in 2012. This would enable more scrutiny (by policy folk, researchers and the public) of where government is or isn't using the best available evidence.
This is broadly the conversation that Sense About Science and others promoting access to information and better policymaking have been having, leading to the proposals above. So join us at our workshop, or email me your suggestions, to take this excellent opportunity to improve how transparently government uses evidence and data.
Posted by on 07 December 2015
Hosted by the Meta-Research Innovation Center at Stanford University (METRICS), the conference gathered a vanguard of reformers around the problems of reproducibility and transparency, and why a lack of each is undermining trust in biomedical research. The goal of the conference was to move beyond describing the problems—although there was much of that—and find at least the outlines of a solution.
Much as the muckraking journalist Upton Sinclair observed that “It is difficult to get a man to understand something, when his salary depends upon his not understanding it,” the conference wrestled with the dissonance in scientists valuing transparency and reproducibility as core virtues of good science even as the academic and commercial world encourages them to assiduously avoid the former and ignore the latter. Openness and collegiality can be fatal to an academic career. Many academic researchers feel they must protect their highly valuable research as not to be “scooped” by competitors, but simultaneously, they realize the value of data sharing and how it can advance science for the common good. It’s a conundrum for any academic aspiring to tenure.
Brian Nosek, Executive Director of the Center for Open Science at the University of Virginia, argued that scientists needed to draw out these core values—quite literally in the form of badges signaling a commitment to data transparency. Such public symbols, while seeming “stupid,” as Nosek put it, actually work: In the journal Psychological Science, badges drove data sharing from 3% to 38%. Why? They signal community standards and enable social norming.
Stuart Buck, Vice President for Research Integrity at the Laura and John Arnold Foundation (which is a funder of both METRICS and Sense About Science USA) argued that regulators needed to set much higher community standards for data transparency: the Food and Drug Administration (FDA), for example, is sitting on a trove of data that doesn’t appear in academic study publication, and which is often vital to clinicians. He also detailed the absurdist lengths that Gilead Pharmaceuticals went to in order to avoid publishing data from its clinical trials: they claimed proprietorial confidentiality on the statistical methods they used. Ask any statistician and they’ll tell you that sharing your statistical methodology is essential to validating sound scientific findings and necessary for proper peer review; but that’s only half the point: how exactly can you justify proprietary claims on statistical methods? These processes are the building blocks of scientific research in the same way that addition and multiplication are the building blocks of math.
Tracey Brown, founder and director of the UK’s Sense About Science argued for the importance of including the public in this process of reform—as the AllTrials campaign has done by working with patients and patient groups to promote clinical trial transparency. Unless the public was engaged in “science about science” (to use Nosek’s phrase), the result would be cynicism: People would not discriminate in its mistrust of science, with disastrous consequences for public policy and individual behavior—along with funding for future research.
One commenter, who strongly advocates for reform of the current www.clinicaltrials.gov reporting system, went so far as to suggest that patient power and strike action—catalyzed by a “Patients Bill of Rights”—would be the most effective way to bring about change. Such a bill would advocate for non-participation in clinical trials if certain criteria were not met; other parties claimed that the standard of patient care was actually better when participating in a clinical trial.
While there are great strides being taken to promote clinical trial data sharing, there are still many hurdles to overcome—many of which involve the researchers themselves, their companies, academic journals, and academic institutions. METRICS showed that academics are ready to explore multifaceted solutions—and also that such change has to be in partnership with the public.
Posted by Tracey Brown on 03 December 2015
This is Sense About Science’s submission to the Commission on Freedom of Information. In it we raise concerns that researchers, research bodies, politicians and civil servants have shared with us, about making it harder to get hold of the research that underpins government policy.
It’s clear from our conversations with people who promote transparency on the use of evidence, whether they’re inside government or in civil society, that government hasn’t yet considered how changes to FoI might affect the public’s access to evidence. So if you value transparency on how government commissions, publishes and uses evidence, now is the time to speak up in whatever way you can.
Dear Lord Burns
Sense About Science is conducting an inquiry, led by the former appeal court judge the Rt. Hon. Sir Stephen Sedley, into concerns that government departments sometimes fail to publish the research they commission for policy promptly and in accordance with their own guidelines. The inquiry will report in Spring 2016, but I want to bring the points below, from initial scoping and evidence, to your attention, in case your commission is not aware of them.
The internal deliberations of public bodies: Should different protections apply to different kinds of information that are currently protected by sections 35 and 36? (Q1)
Removing the public interest test from Sections 35 and 36 suggests a blanket exemption of the deliberations for policy development, which would include related research commissioned by government.
- Independent research organisations and funders, such as the Wellcome Trust, are telling us that their rules require open publication of research, including research they undertake in partnership with others such as government. Restrictions on freedom of publication would prevent them entering into these contracts.
- You might also consider – in light of the public fuss that already occurs when government-commissioned research is withheld and in light of the research community’s support for open science – whether restrictions would make it hard to attract top researchers to government commissions. Our inquiry is already hearing from researchers whose experience of withheld research have made them reticent to work with government again.
- You might consider, too, whether it is defensible for government to insist on data sharing in other contexts
- Researchers and civil servants have shared with us good examples where complex, uncertain and potentially controversial research has been published promptly and in full, and communicated with the public. These show that exemption of research for policy development is unnecessary, and any desire for exemption may be better addressed by improving communication skills and learning across departments to communicate research better. (A matter for our inquiry recommendations rather than yours!)
Your commission should comment on how section 35 and 36 protections would distinguish between strategic advice for policy development and the underlying evidence. It is questionable whether the public interest would ever be served by such a blanket exemption, which would be over-inclusive and would result in government withholding information that could easily be published to improve public discussion.
Protection for information that involves candid assessment of risks (Q3)
It’s important that risks relating to matters of national security and defence are assessed candidly. Arguments that you have heard in favour of increased exemptions from FoI include the possibility of publication creating a chilling effect on this candour.
- However, researchers involved in Ministry of Defence and other security related agencies have told our inquiry that with carefully managed publication, redacting specific information but publishing the rest, even very sensitive risk analysis on things such as readiness for terror attacks can be communicated without compromising security.
- As some defence-related information is already in the public domain, or can easily be obtained, tighter exemptions around sensitive information could put government in a ridiculous position of withholding data that are openly available through other channels. Even in the highly sensitive example of FOI requests to release previous versions of a dossier on weapons of mass destruction in Iraq, the Information Tribunal concluded that “the ‘chilling effect’ would have been quite limited, given that the Hutton Report had not only put into the public domain a great deal of information on the subject but had also provided a detailed description of the circumstances in which the Dossier had been prepared, so that the public was in a good position to place the Williams draft into its correct context.”
- Government departments often find out about the research conducted in other parts of government through its release into the public domain.
Blanket exemptions on risk assessments relating to the delivery of major government projects would also harm public scrutiny. Such assessments are an essential part of the chain of reasoning behind government decisions, forming part of the case for or against a policy. As the Philips inquiry into the government’s handling of the BSE crisis showed, it is better to communicate candid assessments of risks, even if there is uncertainty, than to cover them up.
Fear of losing control of the way difficult topics are discussed is understandable, and may lie behind the desire to withhold sensitive information. But this risks undermining and trivialising the concept of information that is genuinely sensitive on grounds of national security. Exempting all sensitive information could encourage department personnel to behave as dolts who cannot distinguish between such information and that which they can readily publish without causing harm. An indistinct definition of national security is of no benefit to any party.
Executive veto over the release of information (Q4)
The Commission should clarify how any Ministerial veto would affect the publication of research commissioned by government, with reference to other protocols and the guidelines that cover the publication of government research and with reference to conditions of research contracts, which stipulate that findings should be published promptly and in full. Which arrangements would prevail – contractual agreements, ministerial and civil service codes, or a veto?
I hope your Commission reflects on the need to maintain and strengthen the public’s right to scrutinize the evidence underlying government decisions, as I know do many colleagues in the research community and in government who advocate openness on the use of evidence in public life.
Director, Sense About Science
Posted by Chris Peters on 23 November 2015
Last month I become an Ambassador for the Ask for Evidence campaign. I must have been mentioning the campaign around the office a fair bit as my colleagues suggested it would be a good idea for me to give them an Ask for Evidence talk at one of our regular meetings.
Working in the Centre for Evidence-based Veterinary Medicine (CEVM) it might seem that the presentation would be preaching to the converted (the CEVM has its own online resource for veterinary surgeons to find, and ask for, the evidence behind clinical questions). However, the Ask for Evidence campaign sparked an interest because of its huge reach and relevance to so many walks of life. People have asked about the evidence for, among other things, MET police claims on cracking down on scooter racing, a mobile phone app to prevent unwanted pregnancies, and even underpants to protect against electromagnetic radiation.
All of the CEVM team are researchers with a clinical veterinary or animal science background, therefore our evidence related focus is often somewhat narrow. Armed with tea and cake I gave the team a run through of the work that Sense About Science and the Voice of Young Science Network have done as well as how people can get involved in asking for evidence. By the end of the talk, and the lively discussion that followed, there were already lots of ideas for claims we were going to ‘ask’ about. Keep your eyes peeled - the vets are getting involved!
This blog was written by Imogen Richens, Ask for Evidence Ambassador.
Posted by Tracey Brown on 10 October 2015
To The Editor, The Times
Re Charity cast doubt on sugar deaths but failed to reveal Coca-Cola link, The Times 10th October 2015, p4
Sense About Science is concerned with accurate portrayal of evidence. Coca Cola gave us two donations to our annual fund three and four years ago, one of thousands of donations and not repeated since we closed general company donations in 2013. We are sorry to see Alexi Mostrous over-reach his investigation into Coca Cola and say that they have funded our charity to respond to stories about their products. Some people may not realize that many companies donate to many charities. This doesn’t mean they get something for it. In fact charity rules forbid that. Sense About Science is fortunate among our sector that in the last few years donations from members of the public have replaced company donations. However, in his effort to prove that something had been paid for with this earlier donation, Mostrous ignored material on our website criticizing a Coca Cola product and the many responses to news issues we’ve carried from dietitians who advocate reduction of fizzy drink consumption. Instead he alleges we…
… questioned whether children get sugar rushes. In fact he means by this that we retweeted a Guardian article about the research.
… stated that a no-sugar diet could have adverse health consequences. The only thing he could be referring to is a response from a biochemist to the ‘no-sugar diet’ fad. It said this would mean no fruit and vegetables, which would have health consequences.
… said that sugar does not cause cancer. In fact we retweeted a Cancer Research UK article saying it doesn’t, and drew it to the attention of a company CEO who had made this claim in the Metro.
… criticized research linking fizzy drink consumption to violent behaviour in teenagers. We did the opposite. We carried a comment from a professor of clinical psychology that headlines stating ‘Five fizzy drinks make teenagers more likely to carry a gun’ were wrong and the research didn’t find this. It didn’t criticize the research, it cited the authors.
… criticized research linking sugary drinks to 184,000 deaths. The headlines were ‘fizzy drinks kill 180,000 a year’. NHS Choices pointed out that the research covered all sweetened drinks, and we carried four sentences from a professor of nutrition stating a more precise figure for the increased risk of type II diabetes from measuring actual sugar intake.
Wow! In all my years of looking at media portrayal of evidence I have never encountered so much misrepresentation, exaggeration and innuendo in a few short sentences.
It is completely right that journalists should explore whether industry funding of research and donations to charities like ours has produced bias. But sadly Mostrous fails to do that through thorough investigation. Instead he lazily suggests that funding = bias without producing any evidence to prove it. That is smear not journalism. I could right now list all the Times advertisers and find hundreds of news articles that are positive about those companies. That would be a cheap move that would not prove anything about the editorial independence of Times journalists.
As Mostrous acknowledged, we published the Coca Cola donation in our funding list, which is accessible from every page on our website, including those he refers to and including those accessed via Twitter, and a donation stays on our website for at least two years after it is given. We are in the forefront of openness about donations and funding. Few, including The Times, would match us on that openness. Mostrous proposed to us in correspondence that we should also have declared donations in our retweets of the Guardian and Cancer Research UK articles. I find it hard to look at that suggestion seriously. In fact, the lack of seriousness in the whole piece about us undermines the very real meaning of conflict of interest. The more noise that is created with these kinds of constructed stories, where no evidence of inaccuracy or bias is demonstrated, the more genuine issues of conflict, such as data fabrication in papers and concealing research findings, get drowned out, while people rest back thinking job done.
In the end charities like Sense About Science have to demonstrate our independence through our transparency and integrity. And journalists have to prove our lack of independence with more than conjecture and guilt by association.
Sense About Science