Page 71 of 76
4 Are there any aspect of the execution of the study methods
that undermine your confidence in the trustworthiness of the
claims being made? Indicate any concerns below for all the
method specific questions below (for systematic reviews)
(i) Please specify under the relevant methods
specific question and summarize here:
(ii) Please state whether these undermine the evidence claim:
Yes
No
Partly
Unclear
DFE SAGE QA SYSTEMATIC REVIEWS
Domain 1. What are the methods SRs?
SR1. Did the review address a clearly focused question?
SR2. Did the authors look for the right kinds of studies?
Were the study designs suitable for answering the review's question?
SR3. Was there a comprehensive search for all relevant material?
SR4. Risk of selection bias
taking into account the answers to questions SR.1 - SR3 what is the likelihood of the
reviewers' decisions and methods of how to find relevant studies impacting on the findings
of the study?
SR5. Did the authors do enough to assess the quality of the included studies?
Did the authors assessed the quality of the included studies? including the internal and
external validity of the execution and relevance to their review?
SR6. Were the methods of data extraction, synthesis valid?
this includes measures to minimise rater bias, interrater reliability exercises, double coding
qualitative studies for thematic analysis etc
SR7 Risk of researcher bias
taking into account the answers to the questions in domain 1, what is the likelihood of
researcher bias impacting on the findings of this review?
SR8. If sponsored by industry, was the involvement of the sponsor described?
all conflicts of interest must be disclosed by the authors of a systematic review, including
who sponsored the study and what role the sponsor had in its design, conduct and reporting
SR9, Risk of Conflicts of interest bias
Taking into account the question on industry involvement, what is the risk of conflicts of
interest bias?
Domain 2. Are the results valid? SRs
Adapted from CASP- the results themselves are not an indicator of quality, but instead look
for the choice of methods and consideration of variation, any sensitivity analysis and
discussions of heterogeneity or any consideration of bias and how this might impact on
findings