Scoping review of UK school-based, health and wellbeing screening tools.
July 2020- Aug 2020
University of Bristol
Commentary on summer internship
Can grey literature add colour to public health research?
Grey literature: a bleak sounding name but is it a necessary addition to reviews of literature in public health research?
The project I worked on is an evaluation of a health and wellbeing questionnaire, the Digital Health Contact, which is delivered to school children in Leicester for early identification of unmet needs. I worked with teams at University of Bristol, University of Sheffield and Leicester City Council. As a part of the review, I produced a search protocol and carried out a grey literature search. The approach I took is quite novel as grey literature searches are usually mentioned in passing. In contrast, I developed a detailed plan that improves the possibility that the searches can be replicated, and the methods are transparent.
What is Grey Literature?
Grey literature is not peer reviewed or formally published in academic journals. Some examples are PhD and MSc theses, conference proceedings, webpages, reports and government publications. When doing a review of literature, solely searching peer reviewed publications could mean that relevant grey literature resources are not included.
In the context of my project, the audience of relevant resources may be teachers, head teachers and school nurses who might not read or have access to academic journals. This means that grey literature material could be relevant to find instances where mental health screening tools like the Digital Health Contact have been used in schools.
Researchers should be able to reproduce reviews of literature by following clear systematic search methods outlined in a protocol. However, the characteristics of grey literature and how it is published make it challenging to search systematically in this way. For example, the vast amount of information and lack of archiving make reproducibility difficult. Therefore, I had to work through challenges.
The Reviewing Process
I developed my search plan using publications that outlined existing grey literature reviewing processes (Godin et al., 2015; Adams et al., 2016). Using case studies as examples, both papers highlighted the importance of replicability and used systematic approaches to ensure that searches were transparent. Therefore, I developed a robust method, documenting the results to ensure that searches can be replicated. However, I also recognised that replicating the search plan may yield different results, for example, due to unique search engine algorithms and rating schemes, and this was unavoidable. Another issue that I encountered was that abstracts were not available for all sources and it was not always possible to export results to document the searches. To overcome this, using Google search as an example, I reviewed the webpage title and short text description and had to screenshot the result pages for documentation purposes.
As grey literature covers a vast amount of resources, it was important that my plan outlined searches for all types. These fell into three broad categories: grey literature databases, Google searching and targeted websites. Grey literature databases included Web of Science Conference Proceedings Index, ProQuest Dissertations & Theses Global and the National Institute for Health and Care Excellence (NICE) Evidence search. I structured Google searches using different combinations of the search terms so that varying relevant results could be captured. Targeted websites, mostly including health and education organisations, were suggested by team members and consultations with professionals and searched accordingly.
Reviewing the grey literature was an important part of my project so information that was not in peer-reviewed literature could be identified For example, based on my preliminary findings, a resource that came up frequently was a highly relevant Public Health England document developed with UCL which reports validated mental wellbeing measures used in schools, using UK case studies to evaluate their effectiveness (Deighton et al., 2016). This may not have been identified if it was not for the grey literature search.
However, I also reflected on why grey literature material might not be published academically. For example, it may be more opinion based and not robust enough. These factors could potentially mean the inclusion of grey literature could have a negative impact on the project, however, to reduce publication bias, I believe that grey literature was important in this project.
By adapting systematic search methods, I was able to produce a clear and detailed search plan that captured relevant public health resources. My work provides a valuable contribution to the overall review which explores the effectiveness of screening tools used in the UK. This will be used in the evaluation of the screening tool and help Leicestershire City Council to understand its benefit, informing its future delivery.
Adams, J., Hillier-Brown, F.C., Moore, H.J., Lake, A.A., Araujo-Soares, V., White, M. and Summerbell, C., 2016. Searching and synthesising ‘grey literature’ and ‘grey information’ in public health: critical reflections on three case studies. Systematic reviews, 5(1), p.164.
Godin, K., Stapleton, J., Kirkpatrick, S.I., Hanning, R.M. and Leatherdale, S.T., 2015. Applying systematic review search methods to the grey literature: a case study examining guidelines for school-based breakfast programs in Canada. Systematic reviews, 4(1), p.138.
Deighton, J., Lereya, S.T., Morgan, E., Breedvelt, H., Martin, K., Feltham, A. and Robson, C., 2016. Measuring and monitoring children and young people’s mental wellbeing: A toolkit for schools and colleges. Public Health England and the Evidence Based Practice Unit.