Implementing Information Science in Policing: Mapping the Evidence Base
In many disciplines, there is a wealth of primary evaluation research on what works, and systematic reviews that synthesize that evidence. This is, of course, extremely positive. However, the sheer scale of the information and the way in which it is indexed and presented can mean that it is difficult for practitioners to locate the best available evidence. For this reason, in health, education, and other disciplines, using techniques from information science, which bring together the most reliable evidence. Hitherto, no such database has existed for crime and criminal justice interventions. This article sets out some of challenges and early findings of one exercise which aims to produce such a database, being completed as part of the What Works Centre for Crime Reduction initiative in collaboration with the College of Policing.
In terms of policy evaluation, what the ‘best available evidence’ actually is, is a widely debated issue. Primary research studies can be conceived of, and executed, along a continuum of rigour (e.g. Campbell and Stanley, 1963 ; Sherman et al. , 1997 ) and can answer questions of different kinds—how much crime was reduced, how was a reduction actually achieved and so on. Studies concerning the impact on crime of interventions may produce misleading results due to chance (i.e. statistical fluke) or through the use of a poorly conducted evaluation.
In evidence synthesis, searching for primary study evidence is akin to the data collection phase of a research project, and this needs to be completed in a way that minimizes the possibility that the identified set of studies on which conclusions are to be based is in some way biased. The explicit and systematic methods used by evidence synthesists for this purpose draw heavily from the field of ‘information science’ ( Rubin, 1998 ), which has become a keystone in high-quality research. Information science is concerned with the properties and flow of information, and studies the means of processing it for ‘optimum accessibility and usability’ ( Borko, 1968 , p. 3). Although allied to computer science and library studies, information science is considered a discipline in its own right, with a broad field of study and a multi-disciplinary focus.
With a view to highlighting some of the important principles, in this article, we begin by considering some of the empirically validated methods used by Information Specialists to search and retrieve information relating to evaluation research. We then discuss how these methods were applied to identify existing systematic reviews concerned with crime reduction, and then provide some summary statistics about the reviews identified. We also discuss the degree to which the role of information synthesist should be taken on by academics and by the police themselves.
Electronic reference databases (e.g. PsychINFO, MEDLINE) that hold millions of research article details are typically the main source of ‘leads’ to potentially relevant studies. These databases can be searched using search strings that combine keywords, index terms, and other characteristics of the documents (i.e. date, publication outlet, language, document type to name a few). Prior to using these, researchers engaged in systematic reviews typically consult with experts to ensure that the best terms are used and that important ones are not omitted. For transparency, researchers make these syntaxes publically available so that they might be scrutinized and used in replication studies if desired.
A related concern is language bias, whereby studies with larger effects are more likely to be published in English (e.g. Egger et al. , 1997 ). Ideally, the assessment of studies that are published in languages other than English is, of course, important for other reasons, as different effects may be observed in different countries. However, the costs (time and financial) associated with translating texts written in languages other than English may realistically preclude their consideration in search exercises.