Sense about Science ? equipping people to make sense of science and evidence
ArchiveView full archive
- The strange end of the Saatchi Bill
- Here's a plan to help the government to do better than its anti-lobbying clause
- Making the government's use of evidence more transparent
- Sense About Science at the METRICS conference
- Submission to the Independent Commission on Freedom of Information
- The vets are coming!
- The Times 10th October 2015
- Peer Review 101
- Peer review matters!
- John Maddox Prize - Guest post by Dr Thelma Lovick
Posted by Sile Lane on 29 March 2016
In quiet news last week, the "Saatchi Bill" was given Royal Assent on Wednesday 23rd March. It’s the Saatchi Act 2016 now. Except that it isn’t. This is not the legislation Lord Saatchi originally said he wanted. Everything has been stripped out of it except the provision for a database. All the Act does is give the Secretary of State provision to set up a database to capture information about innovative medical treatment, if he wants to. Doesn’t say he should. Doesn’t say anyone should use it if he does. This is the flimsiest Act I’ve seen for a long time. What a waste of time and resources.
Well done to all the people who drew attention to the dangers of earlier versions. The statute books are not a place for parliamentarians to seek therapy or consolation. Thank goodness most don’t raise pointless bills for personal reasons.
Posted by Tracey Brown on 18 March 2016
With its anti-lobbying clause, the government is going to restrict publicly funded individuals and bodies, including scientists and research groups, participating in public life. This is not what the government wants to restrict: it wants to restrict gross abuses of funds – councils spending government funding on making self-promotional political material and situations like the defunct charity Kids Company, who spent a lot of the funds on lobbying for more funds rather than actually helping kids.
Some hope the anti-lobbying clause will stop organisations that use public grants to pay for lobbying against the government. (There are, though, also cases of government grants to organisations that make cases that are helpful to government. I’m not sure the clause writers have thought this far.)
Regardless of the intention, the clause places a restriction on everybody who receives government funds, including research bodies. A lot of people, like me, and including some of the harshest critics of the misuses above, think this is ludicrous: society pays for research, and for people to become experts. We want them in public life. But let’s not get too exceptionalist about this: very useful contributions to policy making, and the evidence it draws on, come from advocates, users and providers of services. Why would we want anyone restricted? The government doesn’t really want this either.
So … the government should stop the gross abuses of funds instead of redefining the problem as one of trying to 'influence policy'. It should ask how it in some cases has ended up giving grant after grant to a body that doesn’t do what it says it will. And leave the question of influencing policy well alone.
The clause is fundamentally not the answer to this issue. So this isn't a question of clarifying or better defining it. There is no definition of lobbying that would work here: when even the most dispassionate of scientists submit material to a policy consultation, it’s called lobbying by one side and welcomed as information by the other. Even if you think we can just have an exemption for things we like, such as research, there is no easy definition of ‘research’ either and few government departments where that’s not going to present a challenge.
Let’s be totally clear: any restriction of people’s ability to influence parliamentary, government and legal matters will be over-inclusive and inhibit democratic participation. (It is under-inclusive too – this is science speak, for the policy folk – because it only targets one diversion of public funds. It has nothing to say about lobbying the public or a foreign government for example.) It is also practically counterproductive to what government actually wants. So instead of trying to control participation in public life, the government should simply seek to control whether it gives money to organisations that don’t use it for the intended purpose. I pay Jim for chips, Jim gives me a potato, I don’t pay Jim for chips anymore. I don’t try to stop everyone serving potato.
The government should drop the May implementation and consult more on what would really tackle the problem, without causing the collateral damage that will result from extending this clause beyond the services sponsored by the Department for Communities and Local Government.
A Big PS
It’s not okay to say, ah well the clause is dreadful but the government’s not targeting researchers and it will be used sensibly. Its effects won’t just be from formal enforcement. Think on these:
1. Sure, government is not going to come after researchers asking for its money back. But when a researcher’s contribution to a policy discussion is not on message with government’s position, do we imagine this clause won’t be brought into play?
2. When a researcher’s contribution to a policy discussion upsets an advocacy group – on energy, use of animals in research, drugs, NHS funding, IVF, badgers – do we really think this clause won’t be used as the basis for a complaint, which must then be investigated by the relevant government department and the research institution, and defended by the individual? (Think of the senior time one complaint will take up, producing accounts of the hours spent on the funded work versus the time taken on policy work, trying to parse the difference between the conduct of the work and how someone has used it to press a point!)
3. When a government-commissioned researcher provides a report with recommendations that reach further than anticipated, do we think the civil servants won’t have reference to the clause?
4. Participating in policy discussions can be hard, time consuming and intimidating. Finding a reason not to do it is often attractive. Do we think no-one will use this one?
5. Many organisations in receipt of government funding are already anxious not to misstep and provide a reason for that funding to be cut. Do we think they won’t err on the side of caution and apply the rule to their grantees and employees in any way at all? (“In order to protect ourselves from this being really enforced, we must enforce it a little bit ourselves” – we know this logic, it’s always in play.)
6. In every big organisation there are people who want more control over what others do and disagreements between different groups of people. If you’ve worked in one you may have encountered the way that the policy department complains about the activities of the communications department. The clause will be brought into play.
This article was amended on 21.03.16 for clarity.
Posted by Prateek Buch on 21 January 2016
Sense About Science has been asked by the Cabinet Office and the UK Open Government Network to lead a discussion on making the government's use of evidence more transparent. Civil society organisations and government are working in partnership to produce the UK's National Action Plan on Open Government, and I’m really pleased to be acting as civil society’s leader on proposals to improve the public’s access to information. The proposals include:
- Government departments committing to an evidence transparency framework
- A single register for all government research
- A single point of contact for public evidence requests in every public body
- Improving how freedom of information works to make policymaking more effective and accountable.
If you can help develop these ideas, please come to a workshop we’re holding on Tuesday 9th February where we’ll discuss the aims behind the proposals and what might be done to achieve them. Your input will inform the final Open Government commitments in the National Action Plan.
You can register here for the workshop, which is kindly hosted by the Royal Statistical Society, and co-organised with the Open Data Institute who will lead a discussion on the better use of open data.
To challenge the evidence behind public policy, the public needs to know the findings of government-commissioned research, the role that this evidence has played, and the extent to which contra-evidence, limitations in the evidence base and alternative policies have been considered. But before we can evaluate any of this, we need to know what's been looked at, which means making the evidence base more transparent as promised in the civil service reform plan in 2012. This would enable more scrutiny (by policy folk, researchers and the public) of where government is or isn't using the best available evidence.
This is broadly the conversation that Sense About Science and others promoting access to information and better policymaking have been having, leading to the proposals above. So join us at our workshop, or email me your suggestions, to take this excellent opportunity to improve how transparently government uses evidence and data.
Posted by on 07 December 2015
Hosted by the Meta-Research Innovation Center at Stanford University (METRICS), the conference gathered a vanguard of reformers around the problems of reproducibility and transparency, and why a lack of each is undermining trust in biomedical research. The goal of the conference was to move beyond describing the problems—although there was much of that—and find at least the outlines of a solution.
Much as the muckraking journalist Upton Sinclair observed that “It is difficult to get a man to understand something, when his salary depends upon his not understanding it,” the conference wrestled with the dissonance in scientists valuing transparency and reproducibility as core virtues of good science even as the academic and commercial world encourages them to assiduously avoid the former and ignore the latter. Openness and collegiality can be fatal to an academic career. Many academic researchers feel they must protect their highly valuable research as not to be “scooped” by competitors, but simultaneously, they realize the value of data sharing and how it can advance science for the common good. It’s a conundrum for any academic aspiring to tenure.
Brian Nosek, Executive Director of the Center for Open Science at the University of Virginia, argued that scientists needed to draw out these core values—quite literally in the form of badges signaling a commitment to data transparency. Such public symbols, while seeming “stupid,” as Nosek put it, actually work: In the journal Psychological Science, badges drove data sharing from 3% to 38%. Why? They signal community standards and enable social norming.
Stuart Buck, Vice President for Research Integrity at the Laura and John Arnold Foundation (which is a funder of both METRICS and Sense About Science USA) argued that regulators needed to set much higher community standards for data transparency: the Food and Drug Administration (FDA), for example, is sitting on a trove of data that doesn’t appear in academic study publication, and which is often vital to clinicians. He also detailed the absurdist lengths that Gilead Pharmaceuticals went to in order to avoid publishing data from its clinical trials: they claimed proprietorial confidentiality on the statistical methods they used. Ask any statistician and they’ll tell you that sharing your statistical methodology is essential to validating sound scientific findings and necessary for proper peer review; but that’s only half the point: how exactly can you justify proprietary claims on statistical methods? These processes are the building blocks of scientific research in the same way that addition and multiplication are the building blocks of math.
Tracey Brown, founder and director of the UK’s Sense About Science argued for the importance of including the public in this process of reform—as the AllTrials campaign has done by working with patients and patient groups to promote clinical trial transparency. Unless the public was engaged in “science about science” (to use Nosek’s phrase), the result would be cynicism: People would not discriminate in its mistrust of science, with disastrous consequences for public policy and individual behavior—along with funding for future research.
One commenter, who strongly advocates for reform of the current www.clinicaltrials.gov reporting system, went so far as to suggest that patient power and strike action—catalyzed by a “Patients Bill of Rights”—would be the most effective way to bring about change. Such a bill would advocate for non-participation in clinical trials if certain criteria were not met; other parties claimed that the standard of patient care was actually better when participating in a clinical trial.
While there are great strides being taken to promote clinical trial data sharing, there are still many hurdles to overcome—many of which involve the researchers themselves, their companies, academic journals, and academic institutions. METRICS showed that academics are ready to explore multifaceted solutions—and also that such change has to be in partnership with the public.
Posted by Tracey Brown on 03 December 2015
This is Sense About Science’s submission to the Commission on Freedom of Information. In it we raise concerns that researchers, research bodies, politicians and civil servants have shared with us, about making it harder to get hold of the research that underpins government policy.
It’s clear from our conversations with people who promote transparency on the use of evidence, whether they’re inside government or in civil society, that government hasn’t yet considered how changes to FoI might affect the public’s access to evidence. So if you value transparency on how government commissions, publishes and uses evidence, now is the time to speak up in whatever way you can.
Dear Lord Burns
Sense About Science is conducting an inquiry, led by the former appeal court judge the Rt. Hon. Sir Stephen Sedley, into concerns that government departments sometimes fail to publish the research they commission for policy promptly and in accordance with their own guidelines. The inquiry will report in Spring 2016, but I want to bring the points below, from initial scoping and evidence, to your attention, in case your commission is not aware of them.
The internal deliberations of public bodies: Should different protections apply to different kinds of information that are currently protected by sections 35 and 36? (Q1)
Removing the public interest test from Sections 35 and 36 suggests a blanket exemption of the deliberations for policy development, which would include related research commissioned by government.
- Independent research organisations and funders, such as the Wellcome Trust, are telling us that their rules require open publication of research, including research they undertake in partnership with others such as government. Restrictions on freedom of publication would prevent them entering into these contracts.
- You might also consider – in light of the public fuss that already occurs when government-commissioned research is withheld and in light of the research community’s support for open science – whether restrictions would make it hard to attract top researchers to government commissions. Our inquiry is already hearing from researchers whose experience of withheld research have made them reticent to work with government again.
- You might consider, too, whether it is defensible for government to insist on data sharing in other contexts
- Researchers and civil servants have shared with us good examples where complex, uncertain and potentially controversial research has been published promptly and in full, and communicated with the public. These show that exemption of research for policy development is unnecessary, and any desire for exemption may be better addressed by improving communication skills and learning across departments to communicate research better. (A matter for our inquiry recommendations rather than yours!)
Your commission should comment on how section 35 and 36 protections would distinguish between strategic advice for policy development and the underlying evidence. It is questionable whether the public interest would ever be served by such a blanket exemption, which would be over-inclusive and would result in government withholding information that could easily be published to improve public discussion.
Protection for information that involves candid assessment of risks (Q3)
It’s important that risks relating to matters of national security and defence are assessed candidly. Arguments that you have heard in favour of increased exemptions from FoI include the possibility of publication creating a chilling effect on this candour.
- However, researchers involved in Ministry of Defence and other security related agencies have told our inquiry that with carefully managed publication, redacting specific information but publishing the rest, even very sensitive risk analysis on things such as readiness for terror attacks can be communicated without compromising security.
- As some defence-related information is already in the public domain, or can easily be obtained, tighter exemptions around sensitive information could put government in a ridiculous position of withholding data that are openly available through other channels. Even in the highly sensitive example of FOI requests to release previous versions of a dossier on weapons of mass destruction in Iraq, the Information Tribunal concluded that “the ‘chilling effect’ would have been quite limited, given that the Hutton Report had not only put into the public domain a great deal of information on the subject but had also provided a detailed description of the circumstances in which the Dossier had been prepared, so that the public was in a good position to place the Williams draft into its correct context.”
- Government departments often find out about the research conducted in other parts of government through its release into the public domain.
Blanket exemptions on risk assessments relating to the delivery of major government projects would also harm public scrutiny. Such assessments are an essential part of the chain of reasoning behind government decisions, forming part of the case for or against a policy. As the Philips inquiry into the government’s handling of the BSE crisis showed, it is better to communicate candid assessments of risks, even if there is uncertainty, than to cover them up.
Fear of losing control of the way difficult topics are discussed is understandable, and may lie behind the desire to withhold sensitive information. But this risks undermining and trivialising the concept of information that is genuinely sensitive on grounds of national security. Exempting all sensitive information could encourage department personnel to behave as dolts who cannot distinguish between such information and that which they can readily publish without causing harm. An indistinct definition of national security is of no benefit to any party.
Executive veto over the release of information (Q4)
The Commission should clarify how any Ministerial veto would affect the publication of research commissioned by government, with reference to other protocols and the guidelines that cover the publication of government research and with reference to conditions of research contracts, which stipulate that findings should be published promptly and in full. Which arrangements would prevail – contractual agreements, ministerial and civil service codes, or a veto?
I hope your Commission reflects on the need to maintain and strengthen the public’s right to scrutinize the evidence underlying government decisions, as I know do many colleagues in the research community and in government who advocate openness on the use of evidence in public life.
Director, Sense About Science