Guidance and resources available to support review authors

Review authors should first check with the editors of the CRG under which they propose to register their protocol whether there is a CRG-specific policy in place about the inclusion of NRS in a review. Authors should also discuss with the editors the extent of methodological advice available in the CRG since they are likely to require more support than with a review that includes randomized trials only, and attempt to recruit informed methodologists to their review team. Regrettably, the NRSMG is not currently in a position to collaborate with authors on particular reviews, but encourages authors who include NRS in their reviews to feedback their experiences to the NRSMG, particularly where their experiences support, or contradict, the experiences described in this chapter.

Review authors intending to review the adverse effects (harms) of an intervention should read Chapter 14, which has been prepared by the Adverse Effects Methods Group.

We recommend that review authors use explicit study design features (NB: not study design labels) when deciding which types of NRS to include in a review. Members of the NRSMG have developed two lists that can be used for this purpose, although experience using them is limited. Tables 13.2.a and 13.2.b describe separate lists for individually-allocated and cluster-allocated studies. Sixteen (or fifteen) items are grouped under four headings:

1. Was there a comparison?

2. How were groups created?

3. Which parts of the study were prospective?

4. On which variables was comparability [between groups receiving different interventions] assessed?

The items are designed to characterize key features of studies which, on the basis of the experiences of NRSMG members and 'first principles' (rather than evidence), are suspected to define the major study design categories or to be associated with susceptibility to bias. The tables indicate which features are associated with different NRS designs, identified by labels that are more specific than those in Box 13.1.a. There is not total consensus about the use of these (column) labels. This disagreement does not mean that the row items are inappropriate or poorly described; the value of the lists depends on the agreement between review authors when classifying primary studies. We will also propose that these lists be used as checklists in the processes of data collection and as part of the critical assessment of the studies (Section 13.4.2 and Section 13.5.2). Instructions for using the items as checklists in Box 13.4.a provide further explanation of the terms.

A number of organizations are carrying out systematic reviews of NRS where there are no, or very few, randomized trials. Reviews are often commissioned on behalf of organizations responsible for issuing policy or guidance to healthcare professionals, e.g. the National Institute for Health and Clinical Excellence (NICE), the Canadian Agency for Drugs and Technologies in Health (CADTH), and carried out by teams of systematic reviewers in university departments of health sciences. In general, reviewers in these

Table 13.2.a List of study design features (studies with allocation to interventions at the individual level)

RCT

Q-RCT

NRCT

CBA

PCS

RCS

HCT

NCC

CC

XS

BA

CR/C

Was there a comparison:

Between two or more groups of participants receiving different

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

N

N

interventions?

Within the same group of participants over time?

P

P

N

Y

N

N

N

N

N

N

Y

N

Were participants allocated to groups by:

Concealed randomization?

Y

N

N

N

N

N

N

N

N

N

na

na

Quasi-randomization?

N

Y

N

N

N

N

N

N

N

N

na

na

By other action of researchers?

N

N

Y

P

N

N

N

N

N

N

na

na

Time differences?

N

N

N

N

N

N

Y

N

N

N

na

na

Location differences?

N

N

P

P

P

P

P

na

na

na

na

na

Treatment decisions?

N

N

N

P

P

P

N

N

N

P

na

na

Participants' preferences?

N

N

N

P

P

P

N

N

N

P

na

na

On the basis of outcome?

N

N

N

N

N

N

N

Y

Y

P

na

na

Some other process? (specify)

Which parts of the study were prospective:

Identification of participants?

Y

Y

Y

P

Y

N

p*

Y

N

N

P

P

Assessment of baseline and allocation to intervention?

Y

Y

Y

P

Y

N

p*

Y

N

N

na

na

Assessment of outcomes?

Y

Y

Y

P

Y

P

P

Y

N

N

P

P

Generation of hypotheses?

Y

Y

Y

Y

Y

Y

Y

Y

P

P

P

na

On what variables was comparability between groups assessed:

Potential confounders?

P

P

P

P

P

P

P

P

P

P

N

na

Baseline assessment of outcome variables?

P

P

P

Y

P

P

P

N

N

N

N

na

Y = Yes; P = Possibly; P* = Possible for one group only; N = No; na = not applicable. NB: Note that 'possibly' is used in the table to indicate cells where either 'Y' or 'N' may be the case. It should not be used as a response option when applying the checklist; if uncertain, the response should be 'can't tell' (see Box 13.4.a). RCT = Randomized controlled trial; Q-RCT = Quasi-randomized controlled trial; NRCT = Non-randomized controlled trial; CBA = Controlled before-and-after study; PCS = Prospective cohort study; RCS = Retrospective cohort study; HCT = Historically controlled trial; NCC = Nested case-control study; CC = Case-control study; XS = Cross-sectional study; BA = Before-and-after comparison; CR/CS = Case report/Case series.

Y = Yes; P = Possibly; P* = Possible for one group only; N = No; na = not applicable. NB: Note that 'possibly' is used in the table to indicate cells where either 'Y' or 'N' may be the case. It should not be used as a response option when applying the checklist; if uncertain, the response should be 'can't tell' (see Box 13.4.a). RCT = Randomized controlled trial; Q-RCT = Quasi-randomized controlled trial; NRCT = Non-randomized controlled trial; CBA = Controlled before-and-after study; PCS = Prospective cohort study; RCS = Retrospective cohort study; HCT = Historically controlled trial; NCC = Nested case-control study; CC = Case-control study; XS = Cross-sectional study; BA = Before-and-after comparison; CR/CS = Case report/Case series.

Table 13.2.b List of study design features (studies with allocation to interventions at the group level)

C1RCT C1Q-RCT C1NRT CITS CChBA ITS ChBA EcoXS

Table 13.2.b List of study design features (studies with allocation to interventions at the group level)

Was there a comparison:

Between two or more groups of clusters receiving different

Y

Y

Y

Y

Y

Y

Y

Y

interventions?

Within the same group of clusters over time?

P

P

N

Y

N

N

N

N

Were clusters allocated to groups by:

Concealed randomization?

Y

N

N

N

N

N

N

N

Quasi-randomization?

N

Y

N

N

N

N

N

N

By other action of researchers?

N

N

Y

P

P

N

N

N

Time differences?

N

N

N

Y

Y

Y

Y

N

Location differences?

N

N

P

P

P

N

N

P

Policy/public health decisions?

Na

na

P

P

P

P

na

na

Cluster preferences?

Na

na

P

P

P

P

na

na

Some other process? (specify)

Which parts of the study were prospective:

Identification of participating clusters?

Y

Y

Y

P

P

P

P

N

Assessment of baseline and allocation to intervention?

Y

Y

Y

P

P

P

P

N

Assessment of outcomes?

Y

Y

Y

P

P

P

P

N

Generation of hypotheses?

Y

Y

Y

Y

Y

Y

Y

P

On what variables was comparability between groups assessed:

Potential confounders?

P

P

P

P

P

P

P

P

Baseline assessment of outcome variables?

P

P

P

Y

Y

Y

Y

N

Note that 'cluster' refers to an entity (e.g. an organization), not necessarily to a group of participants; 'group' refers to one or more clusters; see Box 13.4.a.

Note that 'possibly' is used in the table to indicate cells where either 'Y' or 'N' may be the case. It should not be used as a response option when applying the checklist; if uncertain, 'can't tell' should be used (see Box 13.4.a).

Y = Yes; P = Possibly; P* = Possible for one group only; N = No; NR = Not required. C1RCT = Cluster randomized controlled trial; C1Q-RCT = Cluster quasi-randomized controlled trial; C1NRT = Cluster non-randomized controlled trial; CITS = Controlled interrupted time series (Shadish 2002); CChBA = Controlled cohort before-and-after study (Shadish 2002); ITS = Interrupted time series; ChBA = Cohort before-and-after study (Shadish 2002); EcoXS = Ecological cross-sectional study.

teams have sought to apply methods developed for systematic reviews of randomized trials to NRS. These groups include:

• Effective Practice and Organisation of Care (EPOC) Group (www.epoc.cochrane. org);

• The Centre for Reviews and Dissemination (www.york.ac.uk/inst/crd);

• EPPI centre, Institute of Education, University of London (eppi.ioe.ac.uk);

• The Effective Public Health Practice Project (EPHPP), Canadian Ministry of Health, Long-Term Care and the City of Hamilton, Public Health Services (link to list of EPHPP reviews: old.hamilton.ca/phcs/ephpp).

CRGs and Cochrane review authors have tended to limit inclusion of NRS by study design or methodological quality, acknowledging that NRS design influences susceptibility to bias. For example, the EPOC CRG accepts protocols that include interrupted time series and controlled before-and-after studies, but not other NRS designs. Other reviews have limited inclusion to studies with 'adequate methodological quality' (Taggart 2001).

Was this article helpful?

0 0

Post a comment