Core concepts in assessing statistics

P-values and confidence interval

Along with p-values, the 95% confidence interval (CI) is one of the most frequently used statistical interpretations of data. The p-value reflects the probability that there is no difference between the groups being compared (null hypothesis). So, the smaller p-value is the less likely it is that any differences are due to chance. However, CI informs us of the lower and upper limits surrounding a parameter such as mean effect or odds ratio. In palliative care variability between patients or study participants can be considerable. Therefore, in many cases the CI can be more useful, and it may be large. You will need to decide whether the lower end of effect is acceptable. The video from NCCMT explains this from a pragmatic viewpoint of CI.

Clinical significance

Statistical confidence (p value) tells you if an effect is real but it does not inform you whether it is sufficient clinically to prompt a change in practice. Clinical significance (CS) is about the practical impact of an intervention in the real world, and it requires clinical judgement based on the context of care. Analysis of the CI can help you to determine CS.

Forest plot

A meta-analysis is used to statistically compare and quantitatively combine data from a number of studies (often RCTs). In this way outcomes from palliative care studies with very few participants might be combined to increase the cohort size and hence study power to arrive at a conclusion. This comparison is generally represented as a forest plot that graphically summarises the findings and any uncertainty surrounding the outcome. To help you interpret a forest plot the Cochrane blog Reading a Forest plot or the NCCMT video provide a starting point. If the studies are very different e.g. study design, participants, outcomes reported, then it may not be possible to conduct a meta-analysis.

Attrition bias

The high mortality rates and symptom burden among people with palliative care needs is challenging for research. [2] Because of this, patient attrition, or loss to follow up is highly likely and can affect study outcomes. The MORECare statement was developed in response to this to ensure best practice reporting of the reason for study attrition (death, illness, or random loss). [3] Generally, an attrition rate of 20% is considered acceptable. Attrition rates in palliative care trials vary but average 29%. [2] To account for this as well as reporting the reasons you should check that the study authors use a suitable imputation method such as Area under the curve analysis, last observation carried forward, or intention to treat analysis.

AI for bias

RobotReviewer is an artificial intelligence approach to extracting PICO information and assessing bias within RCT reports. This free to use software is now utilised by TRIPdatabase as part of their article description and is being trialled by Cochrane for use in their systemic review process. You can add pdfs of your RCT articles to the Demo RobotReviewer to gain a first impression of potential bias within a paper or series of papers and then read the paper to compare your judgment.

The bias outputs for RobotReviewer are:

• Random sequence generation
• Allocation concealment
• Blinding of participants and personnel
• Blinding of outcome assessment