As part of our Journal Club summaries our JC Chairs (Drs. Lisa Calder and Ian Stiell @EMO_Daddy) have been tasked with explaining Epidemiological concepts so that everyone in our department can analyze the literature and appraise articles on their own. For this Blog post we have all the “Epi Lessons” as they relate to “Decision Rules Articles”. More to follow in the coming weeks.

Approach to Decision Rule Development                                                       

By: Dr. Venkatesh Thiruganasambandamoorthy
In this era of clinical decision tools being developed for almost anything, we need to think will a clinical decision tool be helpful and how to use them. There are methodological standards that have been developed (i.e. when should one try to develop a tool, how to do it and the stages towards developing a robust tool).

  1. There must be a need due to prevalence of the clinical condition and current practice. Such a need must be a belief among physicians practicing in that area.
  2. The outcome or diagnosis to be predicted must be clearly defined. Assessment of the outcome should be made without knowledge of the predictor variables (Blinded outcome assessment).
  3. The clinical findings to be used as predictors must be clearly defined, standardized, and clinically sensible and their assessment must be done without the knowledge of the outcome (Blinding for the outcome, blinded variable collection).
  4. The reliability or reproducibility of the predictor findings must be clearly demonstrated (usually reported as kappa for the predictors).
  5. To increase generalizability, the subjects in the study should be selected without bias and should represent a wide spectrum of patients with and without the outcome.
  6. Sound mathematical techniques must be used for deriving the tools and must be clearly explained.
  7. Decision tools should be clinically sensible and their accuracy must be demonstrated.
  8. Prospective validation is an essential step in the evolution of this form of decision support. Implementation phase (to demonstrate the true effect on patient care) is the ultimate test of a decision tool.
Not all clinical decision tools are developed for simple decisions (x-ray or not), some are developed to aid complex decision making process. As physicians, we must use our clinical acumen to incorporate these tools in our practice (e.g. using the risk factors identified among syncope patients to ensure that no serious conditions already exist, among patients with multiple comorbidities using these tools to make shared decisions with patients/families or identification of patients for further outpatient consultations/testing).

Methodological Standards for Clinical Decision Rules

Dr. Lisa Calder 
January 2013As more clinical decision rules are created, this will lead to further systematic literature reviews of such rules. This raises the challenge of evaluating methodological quality of clinical decision rules. There are standards published in the literature for emergency medicine – these include: well defined and prospectively collected predictor variables, well defined clinically important outcomes and prospective validation. 

Nested Case Control Study

In case control study two groups of patients one with the outcome (cases) and the other without the outcome of interest (controls) are studied to identify the factors that contributed to the outcome. Controls can be anyone who is at risk for the outcome but did not develop the outcome. Nested case control study is a modification to the above study design where both the cases and controls are derived (nested) within a larger well-defined cohort.

Stages of Clinical Decision Rule Development

Dr. Lisa Calder & Ian Stiell        October-November 2014
Clinical Decision Rules require 4 key stages of development prior to adoption in to clinical practice: derivation, prospective validation, evaluation of implementation and knowledge translation. The first step entails a derivation study that ideally is conducted prospectively and has a large number of outcome cases. The second step is a prospective validation study that explicitly evaluates the new rule for accuracy, physician acceptability and potential impact. The third step is an implementation trial to evaluate the actual impact of the rule on patient outcomes in real clinical practice.  Be very cautious incorporating any decision rule into your practice which has not been through at least the first two steps.  Examples of such rigorous decision rules include the Canadian CT head rule, Canadian C-spine rule and Ottawa Ankle rule. 

Validation of clinical decision rules

Dr. Ian Stiell        
November 2012 Critical appraisal criteria for a paper that validates an existing decision rule are different than those for a study that derives or creates the rule. Most important is that the study evaluates the existing rule accurately and explicitly such that the physicians using it are adequately instructed. Some studies do a validation from an existing database but we believe that it is far better to conduct a prospective real-time validation by clinicians. 

What is a Clinically Sensible Clinical Decision Rule? 

Dr. Ian Stiell          March 2015
Clinical decision rules for emergency medicine should be “clinically sensible.” This means the rules should be easy to use and comprised of as few variables as possible. Emergency physicians prefer rules that give a simple yes/no answer or use a basic scoring system that can be quickly calculated. The component variables should make have good face validity for clinicians.

What is Collinearity? Why does it Matter? How do you Measure it?

Dr. Christian Vaillancourt
Collinearity means that two of the predictors entered in a regression analysis model correlate with each other (they measure almost the same thing, e.g. %body fat and total body weight). When more than two predictors interact with each other, it is called multicollinearity.Collinearity can be a problem, especially when very high, since the software will simply not be able to perform the regression analyses, or will provide unreliable results. The degree of collinearity can be estimated using the Variance Inflation Factor (VIF) which should be <5-10.