Improving methodology for place and community-based public health natural experiments and interventions

Public health researchers often study events, policies or initiatives that they do not have any direct control over. For example, the introduction of the smoking ban in Scotland in 2006 and subsequently in Wales, Northern Ireland and England in 2007. These evaluations can provide important evidence in for public health, but they can be hard to evaluate. In particular, it can be hard for researchers to find a good quality comparison group when they have no control over where and when an initiative is implemented and who receives it. Without a good comparison group, studies often struggle establish whether observed effects are truly the result of, say, the change in policy, or are the result of other (unknown) factors. Because the information gained from natural experiments is vital for evidence-based policy making, it is important to obtain the most robust evidence of their impact. The aim of this work package (WP3) therefore is (1) scientifically, to provide and improve on the methodological underpinning for natural experiments conducted in the Places & communities workstream and in SPHR more broadly, (2) to develop a guide for natural experiments at the local level that focuses on advice for local practitioners and evaluators.

The new practical guide we aim to develop will be targeted at local and national public health practitioners and will discuss when (and when not) to conduct natural experiments, how they could be designed, what different methods are available for analyses, and how the results of these place-based natural experiments should be interpreted. To help develop the guidance, we will conduct a literature review to obtain an overview of the many different methodologies that have been used in natural experiments and assess their strengths and limitations.

We will use the lessons learned from this review to analyse several natural experiment case studies conducted by colleagues across SPHR to explore if and when the optimized methods improve what we can learn from these studies.

The guidance will be co-produced with practitioners and will be made available as open access.