Editorial: What Works? The Role of Evidence in Public Sector Policy and Practice 社论：什么有效?证据在公共部门政策和实践中的作用
Abstract 抽象
There is nothing a politician likes so little as to be well informed; it makes decision-making so complex and difficult. (John Maynard Keynes.) 政客最不喜欢的就是被充分了解;这让决策变得如此复杂和困难。(约翰·梅纳德·凯恩斯)
When Tony Blair announced that ‘what counts is what works’, in the run-up to the General Election in May 1997, the intention was to signal a new ‘post-ideological’ approach to public policy making-an approach where evidence would take centre stage in the decision-making process. It is easy to be sceptical about the likelihood that public policies and practices will become more evidence-based. To date, research evidence on what works has been just one, relatively minor, ingredient in the process from which policy decisions emerge. In practice there are enormous forces of inertia which operate to preserve the status quo, influenced not only by party ideology, but also by the policy preferences of the bureaucracy and professional groupings, and by the demands of the public and client-based pressure groups. 当托尼·布莱尔在 1997 年 5 月大选前夕宣布"关键在于效果"时,其意图是传达一种新的"后意识形态"公共政策制定方法,在这种方法中,证据将在决策过程中处于核心地位。对于公共政策和实践变得更加以证据为基础的可能性,人们很容易持怀疑态度。迄今为止,关于"什么有效"的研究证据只是政策决策过程中相对较小的一个因素。实际上,有着巨大的惯性力量在起作用,不仅受政党意识形态的影响,还受官僚体系和专业团体政策偏好以及公众和施压团体需求的影响,从而保持着现状。
Nevertheless, despite the scepticism arising from past experience, there is a growing rhetoric about the use of evidence to determine policy and practice. As several of the articles in this theme issue report, such rhetoric is supported by changing attitudes and occasional concrete actions in some parts of the public sector. Therefore there is an unusual opportunity for research evidence to have substantial impact on policy and practice. This opportunity arises as a result of a number of factors, including: new Labour’s declaration of government as an ideology-free zone, the apparent political will to give privileged status to research evidence, and the formation of the new Scottish parliament and regional assemblies which will be seeking to create their own public policy agendas. 尽管过去经验造成了怀疑,但越来越多的人呼吁根据证据确定政策和实践。正如本专刊中几篇文章报道的那样,这种呼吁得到了公共部门某些地方态度的改变和偶尔的具体行动的支持。因此,研究证据有机会对政策和实践产生重大影响。这一机会源于若干因素,包括:新工党宣布政府是无意识形态的区域,显然有政治意愿赋予研究证据优先地位,以及新建立的苏格兰议会和地区议会将寻求创造自己的公共政策议程。
This theme issue of Public Money E\mathcal{E} Management reviews progress so far and assesses potential future developments in the use of evidence in the public sector. Each article seeks to provide insight into a number of questions: 这期公共资金与管理专题审视了到目前为止的进展,并评估了在公共部门使用证据的潜在未来发展。每篇文章都力求就以下一些问题提供洞见:
What constitutes evidence of service effectiveness? 什么构成服务效能的证据?
In what ways does research evidence on service 在服务上的研究证据的方式
effectiveness assist in shaping national and local policies (if at all)? 是否有效地参与形塑国家和地方政策?
How do practitioners use evidence to improve professional practice (if at all)? 从事相关工作的人如何利用证据来改善专业实践（如果有的话）?
What are the practical, methodological and organizational barriers to a more evidencebased approach? 实践、方法和组织上的障碍是什么?
Learning from Diversity 从多元性中学习
Even a passing acquaintance with the public sector indicates that different service areas adopt diverse approaches towards identifying what works. At one extreme, the health sector has in general adopted a research culture in which it is accepted that the services provided should in principle be exposed to rigorous scientific evaluation. Central to this culture is the notion of experimentation, usually in the form of randomized controlled trials. Of course, in practice, many medical procedures have not been properly evaluated, or the studies that have been undertaken fall some way short of providing incontrovertible guidance. Much activity remains an act of faith. Moreover, strategic policies, such as the introduction of fundholding in general practice (and its subsequent abandonment), have not hitherto received the same careful evaluative attention as individual medical procedures. Yet the principle is enshrined: interventions should be tested before widespread use. 与公共部门有一些接触就可以发现,不同的服务领域采取不同的方法来确定什么是有效的。在极端情况下,医疗部门通常已经采用了研究文化,认为所提供的服务原则上应该接受严格的科学评估。这种文化的核心是实验的概念,通常采取随机对照试验的形式。当然,在实践中,许多医疗操作并未得到适当的评估,或者已经进行的研究在提供无可争议的指导方面还存在不足。很多活动仍然依赖于信任。此外,像全科医生持基金的引入(以及随后的放弃)这样的战略政策,到目前为止还没有得到与个别医疗操作一样谨慎的评估关注。然而,这个原则已经确立:在大规模使用之前,应该对干预措施进行测试。
In contrast, research on effectiveness is rather less visible in other parts of the public sector. In areas such as education, social services and criminal justice, there has been considerable research activity over several decades. However, coverage is patchy, there is less consensus regarding appropriate methodology, and there is little agreement as to how to use research evidence to inform policy and practice. In particular, apart from health care, rigorous largescale randomized controlled trials are used much less widely to answer questions of compelling public interest. 相比之下，在公共部门的其他领域,有关效率方面的研究较不为人所知。在教育、社会服务和刑事司法等领域,几十年来一直有大量的研究活动。然而,覆盖范围不一,对适当方法论存在较少共识,以及如何利用研究证据来为政策和实践提供信息方面也鲜有共识。特别是除了医疗保健领域外,大规模严格的随机对照试验在回答引人注目的公众利益问题方面使用的程度要小得多。
In part, this neglect of experimentation suggests missed opportunities. On the face of it, there are numerous issues in, say, education, which could be resolved by such methods-for example, identifying the most effective approach to teaching reading; or assessing the impact of calculators on the child’s mathematical development; or evaluating the introduction of a new home-work policy. Such questions may be amenable to experimental study, but adoption of such an approach seems to have been lacking. In contrast, there are many other public policy questions which cannot readily be answered by the use of randomized controlled trials, such as whether legalizing soft drugs decreases or increases the use of hard drugs; or whether a ‘right-to-buy’ scheme, leading to extensive council 这种忽视实验的情况暗示了错过了许多机会。表面上看,在教育等领域有许多问题可以通过这种方法来解决,例如,确定最有效的教读方法,或评估计算器对儿童数学发展的影响,或评估新的家庭作业政策。这些问题都可以通过实验研究来解决,但似乎并未采取这种方法。相比之下,也有许多其他公共政策问题无法通过随机对照实验来解答,例如:是否合法化软毒品会减少还是增加硬毒品的使用;或者是否实施"购买权"计划,导致大规模的公有房地产私有化,会产生什么后果。
house sales, would be effective in providing decent homes for all at affordable cost. There is, moreover, a vast ‘middle ground’ where experimentation may be feasible in principle, but where the costs of implementing such methodologies renders them practically infeasible. Even in these situations there are nevertheless some satisfactory study methodologies and analytic techniques which can yield useful evidence from a policy perspective. 房屋销售,将有效地为所有人提供负担得起的体面住房。此外,还有一个广阔的"中间地带",在原则上可能进行实验,但实施这些方法的成本使其在实际上不可行。即使在这些情况下,也有一些令人满意的研究方法和分析技术,可以从政策角度提供有用的证据。
Irreconcilable Differences? 不可调和的分歧?
The most striking feature to emerge from the articles which follow is confirmation of the divergence of approach in various public sector areas. In some areas (most notably health care) the need for evidence and the nature of convincing evidence is a given. In other areas (most strikingly, social care), the very nature of evidence is hotly disputed and there is strong resistance to assigning privileged status to one research method over another. Such divergent attitudes arise from deep-rooted ontological and epistemological assumptions. Furthermore, where post-modern perspectives are prevalent there is a general distrust of any notion ofobjective evidence. 从随后的文章中显现出的最显著特征是,在各种公共部门领域,方法论的分歧得到确认。在某些领域(最引人注目的是医疗保健),证据的需求和说服力证据的性质是一个既定事实。在其他领域(最引人注目的是社会福利),证据的本质本身就是一个备受争议的话题,人们强烈抵制将某一研究方法置于更有优越地位的做法。这种分歧的态度源于根深蒂固的本体论和认识论假设。此外,在后现代主义视角占主导地位的地方,人们普遍不信任任何形式的客观证据的概念。
Methodological differences are not confined to choice of research design. The different sectors also vary in their use of theory to unravel questions of what works. In health care, much of the emphasis of intervention assessment is on the very pragmatic question of whether or not the intervention offers overall benefits in aggregate. In criminal justice and social care however there is a much greater concern to ‘unpack the box’ of the intervention, to seek understanding as to why it works. A recognition that an intervention will not work for all persons under all circumstances, or that not all parts of an intervention necessarily contribute to its effectiveness, leads to a desire to tease-out the effective elements from the ineffectual in a highly contextualized manner. In this, theories of human behaviour and qualitative methods play a prominent role, counterbalancing the empiricism of pure experimentation. 研究方法的差异不仅限于研究设计的选择。不同领域在使用理论来解开"什么有效"的问题时也存在差异。在医疗保健领域,干预措施评估的重点主要是总体利益是否达到。但在刑事司法和社会福利领域,人们更关注于"揭开干预措施的黑箱",去理解其为何有效。认识到干预措施在不同情况下并不能适用于所有人,或干预措施的各个部分并不都会对其有效性产生贡献,这种认识会引发人们以高度情境化的方式剥离有效和无效的要素。在这一过程中,人类行为理论和定性研究方法发挥了重要作用,平衡了纯粹实验主义的经验主义。
In some fields, the aims of policy are relatively clear: in health care, both policy and practice are aimed at increasing longevity and decreasing morbidity (adding years to life and life to years). With such clarity, assessing ‘what works’ is simplified. In other policy areas, objectives may be multiple and competing, and the relative balance between them may change over time. This complicates and politicises the evaluation of what works. For example, an intervention aimed at young offenders may be very successful at punishing but may also be an abject failure at reducing future offending behaviour. Under these circumstances, reconciling multiple competing objectives is essentially a political task (although research evidence clearly has great potential to inform). What is more, the diverse and changing ways in which the criteria for judging success are 在某些领域,政策的目标相对较为清晰:在医疗保健领域,政策和实践都旨在提高寿命和降低发病率(增加生命年数和生活质量)。有了这样的清晰性,评估"什么有效"就变得简单了。在其他政策领域,目标可能多种多样、互相竞争,而它们之间的相对平衡也可能随时间而变化。这使得评估什么有效变得复杂化和政治化。例如,针对年轻罪犯的一项干预措施可能非常成功地实施了惩罚,但却可能在减少未来犯罪行为方面彻底失败。在这种情况下,协调多重竞争目标本质上是一项政治任务(尽管研究证据显然有很大潜力来为此提供信息)。此外,评判成功的标准也存在多样化和不断变化的方式,
defined in areas such as criminal justice and transport makes it difficult to provide a cumulative research base from which to draw general lessons about what works. 在刑事司法和运输等领域定义的内容,使得很难提供一个累积的研究基础,以从中得出关于什么有效的一般性经验教训。
The relative weight accorded to different inputs into the policy process also varies between policy areas. Ideology, expediency and public preferences compete with scientific evidence for the ears of Ministers. Development of an evidencebased approach in some public policy areas (such as education or criminal justice) may be constrained because key stakeholders (parents, victims) have their own intuitive and strongly held views about what constitutes effective intervention. Such views may be in conflict with rigorously obtained research evidence but nonetheless they will influence both policy and practice. Areas where the nature of the interventions makes user-knowledge less secure (for example medicine) tend to be less influenced by these extraneous factors. But even here, client perspectives are assuming a greater prominence fuelled by a growing public expertise, ready access to technical information (for example via the Internet) and increasingly assertive public attitudes. 不同政策领域中给予不同输入因素的相对权重也存在差异。理念、权宜之计和公众偏好会与科学证据进行竞争,以争取部长的关注。在某些公共政策领域(如教育或刑事司法)发展以证据为基础的方法可能受到限制,因为关键利益相关方(家长、受害者)对什么构成有效干预拥有自己的直觉和牢固的观点。这些观点可能与通过严谨研究获得的证据相冲突,但仍然会影响政策和实践。干预措施的性质使用户知识较不确定的领域(例如医疗)往往受这些外部因素的影响较小。但即使在这里,客户的观点也越来越重要,这是由于公众专业知识的增长、可轻松获取的技术信息(例如通过互联网)以及日益强烈的公众态度。
Many other points of divergence are highlighted when the ways in which evidence is gathered and used across different public services are compared. Policy areas differ according to whether interventions are primarily at an individual level (education) or at the community level (housing). Methodologies for assessing ‘what works’ at an individual level are not necessarily appropriate or feasible at the community level. There is also a differential emphasis given to the issue of cost-effectiveness: ranging from a tradition of cost-benefit analysis in transport to a situation in criminal justice where even the basic cost of interventions is not readily available. 在不同公共服务中收集和使用证据的方式被比较时,许多其他分歧点都得到了强调。政策领域因干预是主要在个人层面(教育)还是在社区层面(住房)而有所不同。评估"什么有效"的个人层面方法论不一定适当或可行于社区层面。成本效益问题也存在不同的强调:从交通领域的成本收益分析传统到刑事司法领域连干预措施的基本成本都无法轻易获得的情况。
The different ontological and epistemological starting points in different professional traditions undoubtedly colour the methods and enthusiasm with which professionals engage with evidence. However, what is clear is that there remains in all of the areas examined great potential for research evidence to be vastly more influential than hitherto. 不同专业传统中不同的本体论和认识论起点无疑影响了专业人士处理证据的方法和热情。然而,很明确的是,在所有研究的领域,研究证据仍然有巨大的潜力可以比以前更有影响力。
Concluding Remarks 总结语
Those concerned with increasing the influence of research evidence in public policy and practice are faced with a formidable task. Improving the supply of evidence is a necessary but not sufficient means of getting evidence of what works into practice. Even in health care, for all the richness of the research base, it remains unclear how best to bring about changes in professional behaviour that are congruent with the evidence. 那些关注增加研究证据在公共政策和实践中的影响的人面临着艰巨的任务。提高证据供给是必要的,但不足以将可行的证据纳入实践。即使在医疗保健领域,尽管研究基础丰富,如何最好地推动专业行为变革以符合证据仍然不明确。
Assuming (as we do) the desirability of both improving the evidence base and increasing its influence on policy and practice in the public services, a number of important goals emerge 假设(正如我们所做的那样)在改善证据基础并增加其对公共服务中的政策和实践的影响方面是可取的,会出现一些重要的目标
which would foster an enhanced role for evidence: 这将培养证据在决策过程中发挥更大作用
Agreement as to what constitutes legitimate evidence on service effectiveness (within specific policy areas). This may be more than simply a methodological question: it may also cover such questions as the source of the research findings, and their political and public acceptability. 关于什么构成服务效果合法证据(在特定政策领域内)的协议。这可能不仅仅是一个方法论问题:它也可能涉及研究结果的来源以及它们的政治和公众可接受性等问题。
Recognition of the policy and practice questions which arebest answered using experimentation, and an equal acceptance that many policy questions are unsuited to such an approach (by dint of ethics, expense, practicability or complexity). Criteria to distinguish between the two would be helpful. 识别最适合使用实验来回答的政策和实践问题,同时也要接受许多政策问题不适合这种方法(由于伦理、费用、可行性或复杂性的原因)。区分这两种情况的标准会很有帮助。
Establishment and continued development of a valid, reliable and relevant evidence base on what works. This would recognise the importance of systematic reviews and metaanalysis as a means of drawing together the results and implications of existing evaluation studies. 建立和持续发展一个有效、可靠和相关的证据库,了解什么是有效的。这将承认系统评审和元分析作为整合现有评估研究结果和影响的一种方式的重要性。
Development of a more strategic approach to the commissioning of research on what works to help ensure coverage of key policy questions using appropriate methodologies. 对关键政策问题使用适当方法开展研究委托的更加战略性方法的开发。
Development of sufficient research capacity in each of the public policy areas, academically rooted but also in tune with practice realities. 在每个公共政策领域都发展足够的研究能力,理论与实践并重。
Development of effective mechanisms for ensuring that the available evidence influences policy and practice. 确保可利用证据影响政策和实践的有效机制的开发。
Simultaneous consideration of the costs as well as the effectiveness of policy and practice interventions (given the inevitability of tight constraints placed on public expenditure in all policy areas). 同时考虑政策和实践干预的成本及其效果(考虑到各政策领域公共支出将面临不可避免的严格约束)。
Because of the variable rates of progress of different public sector areas in meeting these requirements, there is much to be learnt from a cross-sector analysis. If evidence is to play a more central role in the future there are many challenges ahead, but help is also at hand in the form of understanding and learning from the experience of others. 由于不同公共部门在满足这些要求方面取得进展的速度不同，从跨行业分析中可以学习到很多东西。如果证据在未来要发挥更中心的作用,前面还有很多挑战,但也可以从了解和学习他人的经验中获得帮助。
Acknowledgements 致谢
We would like to thank all the participants of a two-day seminar on ‘Evidence-based policy and practice’ held in St Andrews on 10-1 1 September 1998. The presentations, workshops and discussions at this seminar were extremely useful in enabling us to clarify the ideas expressed in this editorial. Funding for the seminar was provided by the Russell Trust to whom we are extremely grateful. 我们要感谢在 1998 年 9 月 10-11 日在圣安德鲁斯举办的"基于证据的政策和实践"两天研讨会的所有与会者。此次研讨会上的演讲、工作坊和讨论非常有助于我们澄清本社论中表达的观点。研讨会的资金由罗素信托基金提供,我们对此深表感谢。
Huw T. O. Davies, Sandra M. Nutley 休·T·O·戴维斯，桑德拉·M·纳特利University of St Andrews 圣安德鲁斯大学and Peter C. Smith 和彼得·C·斯密斯University of York 约克大学