Statistical issues in evaluating an intervention to improve interpretation of CT colonography images

The NIHR Statistics Group Imaging section meeting was held on 20th July 2017, in the University of Birmingham which brought together a range of statisticians, clinicians and other methodologists to discuss common statistical issues in imaging studies. This meeting met to discuss “Statistical issues in evaluating an intervention to improve interpretation of CT colonography images”.

After a short welcome and explaining the aim of the NIHR Statistics group by Dr Sue Mallett (Statistician, Biostatistics group, Institute of Applied Health Research). Professor Steve Halligan (Radiologist, Centre for Medical Imaging, University College London) provided an overview of the clinical challenges in the area of CT colonography. Dr Sue Mallett then provided an overview of the statistical challenges in this area. She then outlined the research scenario of a trial currently in set-up: “Can a training/monitoring schedule improve diagnostic accuracy of radiologists interpreting CT colonography in the NHS bowel cancer screening programme (BCSP)” with the following PICO:

Primary Hypothesis: No difference in test set sensitivity (for cancer or polyps) at 1 month between intervention and controls.
Typical screening centre: 110 hospitals scanning patients, 58 screening centres interpreting scans, 3-4 radiologists per hospital.
Participants: (a) Stored colonoscopy images from patients in BSCP (b) Radiologists normally reading in BCSP.
Intervention: One-day focused training (with prior baseline assessment) followed by 6-monthly test sets with individualised feedback.
Comparator: Current practice
Outcomes: Sensitivity (proportion of those with abnormality detected)

The members then split into three groups to discuss different aspects of this primary scenario:

1. Format of training intervention; time-point for measuring primary outcome; choice of patient images for diagnostic accuracy – including choice of types of patients, number of scans (each one takes approx. 15minutes to read), patients with positive or negative results, patients with more than one polyp …
2. Definition of endpoint, how to score a correct reader identification, should stratification be used in the randomisation (e.g. more experienced vs less experienced radiologists, screening centres vs hospitals, size of workload), paired or unpaired analysis, how to ensure blinding, training for control arm at end of study
3. Pros and cons of a cluster randomised trial, sources of clustering, alternative study design such as stepped wedge designs, contamination between arms. Estimating the parameters for the cluster study design.

The groups then reconvened and presented some of the discussion points raised to the whole group. Several interesting points were raised which were taken away for consideration with the potential to include in the ongoing trial in set-up, of for developing the statistical analysis plan.

The NIHR Statistics Imaging subgroup meets bi-annually. For more information and to join the mailing list please go to: https://statistics-group.nihr.ac.uk/research/imaging-studies/ where you can also find full reports of previous meetings.

Seid Mohammed & Sue Dutton