Module 1 – Lesson 5 – Appendix B
Sample Critical Appraisal of a Clinical Practice Guideline
Critical Appraisal: Clinical Practice Guideline
Note: This sample is intended to highlight the most important features of this type of critical appraisal. It is not all-inclusive and does not represent or reflect any specific currently-available critical appraisal tool.
Title:
The Treatment of Restless Legs Syndrome (RLS) and Periodic Limb Movement Disorder (PLMD) in Adults: Update for 2012: Practice parameters with an Evidence-based Systematic Review and Meta-Analysis
Authors:
- Aurora, R. N., Kristo, D. A., Bista, S. R., Rowley, J. A., Zak, R. S., Casey, K. R., Lam, C., Tracy, S. and Rosenberg, R. S. (2012). (American Academy of Sleep Medicine CPG Group)
Reference:
- Sleep, 35(8), 1039-1062
| QUESTION | RESPONSE | COMMENTS |
|---|---|---|
| Is the group, committee or organization that developed the guidelines identified? | Yes | Task force members of the American Academy of Sleep Medicine were identified on Page 1 with their detailed affiliations. |
| -Have the guidelines been sponsored or funded by an external group with potential conflict of interest? -Have the guidelines been endorsed or supported by an external organization? |
Yes | Page1055 (just before references); disclosure statement re: not an industry supported study and no financial conflicts of interest. Critique was undertaken by an external source. Endorsed by American Academy of Sleep Medicine). |
| Does the document indicate WHY they undertook this guideline development work, or is it a revision of previous work? | Yes | While two previous (limited) works had been conducted on these rare disease entities, authors noted that an update of 2003 and 2004 was needed to support consistency and to include literature (page 1) and a need was expressed for consistency of screening tools (page 2). |
| Is the overall Objective, Purpose and Question (PICOs) for guideline stated? | Yes | Yes, Table 1, page 1041 outlines the multiple PICO questions undertaken/formulated and defined clearly that guidelines for adults only, and specific drug classes vs. control. |
| -Is a systematic literature search described? -Is there mention of how relevant articles were selected or excluded for review? | Yes | Yes, processes for literature search were described on page 1040 with MeSH terms and limitations defined. Exclusions included studies with <5 subjects and treatments <1 week. English only adults only, randomized controlled trials only, no editorials, letters, or case reports (bottom of 1040). Of note: this document is both the comprehensive systematic review and also a CPG; the group opted to create the same thing all at once. They could do so because their clinical topic area was very finite/limited, and it was a somewhat less-common condition. Other topics such as cardiovascular or diabetic topics could never be effectively handled this way. |
| Did authors declare conflicts of interest among all parties involved in guideline preparation and consensus? | Yes | Last page: declaration of no conflicts of interest and no industry funding. |
| Is it clear how consensus for evidence rating and recommendation preparation was reached? | Yes | GRADE process was used as a process for rating quality of evidence; Page 1041: all studies were assessed by two task force members for design and limitations to validate bias; Box 1 reflects the final assessments of evidence. Tables 1-3 outline processes for same. |
| Have patient viewpoints (or those who will be most affected by these guidelines) been sought? | No | This is not specified and anticipated to be “no” although one “flag” that should always be kept in mind with less-common clinical topics or conditions is that there’s always that risk for expert viewpoints to find their way into guidelines anyway, even when steps have been taken to mitigate this). The reason is that there just isn’t ample research available on all entities of care for less-common conditions (such as restless legs syndrome, as one example), so there will always be aspects were viewpoints of experts will just filter in. |
| Were evidence ratings or indicators of value indicated for each recommendation or guidance statement? | Yes | Table 4 captures in simplified language the key results with strength of recommendation and body of evidence level noted for each practice parameter. |
| Are there hints of bias or untoward influence in the process? | No | There does not appear to be. Therapies are summarized based on ‘classes’ presentation and included pharmacologic and non-pharmacologic options (CBT, exercise and placebo effect) |
| Is the overall Objective, Purpose and Question (PICOTs) for guideline stated? | Yes | Yes, Table 1, page 1041 outlines the multiple PICO questions undertaken/formulated and defined clearly that guidelines for ADULTS only, and specific drug classes vs control. |
| -Is a systematic literature search described? -Is there mention of how relevant articles were selected or excluded for review? | Yes | Yes, processes for literature search were described on page 1040 with MeSH terms and limitations defined. Exclusions included studies with <5 subjects and treatments <1 week. English only adults only, RCTs only, no editorials, letters or case reports (bottom of 1040) Of note….this document is both the comprehensive systematic review (SR) and also a CPG…the group opted to “kill 2 birds with 1 stone” in creating same all at once. They could do so because their clinical topic area was very finite/limited and it was a somewhat less-common condition. Other topics such as CV or diabetic topics could never be effectively handled this way. |
| Did authors declare conflicts of interest among all parties involved in guideline preparation and consensus? | Yes | Last page declaration of no conflicts of interest and no industry funding. |
| Is it clear how consensus for evidence rating and recommendation preparation was reached? | Yes | GRADE process was used as a process for rating quality of evidence; Page 1041: all studies were assessed by 2 task force members for design and limitations to validate bias; Box 1 reflects the final assessments of evidence. Tables 1-3 outline processes for same. |
| Have patient viewpoints (or those who will be most impacted by these guidelines) been sought? | No | This is not specified and anticipated to be “no” although one “flag” that we should always keep in mind with less-common clinical topics or conditions is that there’s always that risk for expert viewpoints to find their way into guidelines anyway, even when steps have been taken to mitigate this). Reason is that there just isn’t ample research available on all entities of care for less-common conditions (such as restless leg, as one example), so there will always be aspects were viewpoints of experts will just filter in. |
| Were evidence ratings or indicators of value indicated for each recommendation or guidance statement? | Yes | Table 4 captures in simplified language the key results with strength of recommendation and body of evidence level noted for each practice parameter. |
| Are there hints of bias or untoward influence in the process? | No | There does not appear to be. Therapies are summarized based on classes presentation and included pharmacologic and non-pharmacologic options (cognitive behavioural therapy, exercise and placebo effect). |
| What are the results? | ||
|---|---|---|
| QUESTION | RESPONSE | COMMENTS |
| Are the messages from guidelines clear and easily understandable to the end-user (policy-maker, clinician, patient)? | Yes | Table 4 presents all recommendations as a 1-page reference guide for clinicians. Harm/burden assessment ratings are included and FDA status on drug classes are discussed. |
| Are there direct and easy to follow linkages between guideline ratings and raw data/evidence used to create them? | Yes | Yes, as above. |
| Is there discussion of benefits, harm, risks and cost impacts? | Yes | Yes, harms and benefits included in Table 4 and are highlighted along with individual study results in forest plots for individual drug classes/therapies. There is no discussion of cost impact, however. |
| In sections whereby expert consensus was required or no evidence was found, are processes to arrive at ratings outlined? | Can’t tell | This is not clearly defined and expected/assumed to be through committee discussions. There was mention of several areas where no evidence was found, thus no CPG was prepared in most cases. |
| Will the results help me in my decision-making? | ||
|---|---|---|
| QUESTION | RESPONSE | COMMENTS |
| Are the guidelines presented in a user-friendly and easy to follow format? (Are they ACTIONABLE?) | Yes | Table 4 supports a user-friendly format for clinicians to use as a reference. Not every CPG has this spelled out in this simplistic way. It is because the topic is limited/finite and not interconnected with other conditions that this can occur with relative ease in this guideline. One should not expect all guidelines to be this simple or straightforward. |
| Do the guidelines offer next-steps for practical implementation; is there recognition of implementation barriers? | No | This is not completely clear within the recommendations, and there are no suggestions for knowledge transfer or implementation strategies. |
| Is there advice presented on how to audit for compliance or quality improvement impact? | No | Not noted in this document. |
| Is there supporting patient and family (plain language) versions or education materials available? | No | Does not appear to be included. |
| Is there discussion of clinical flexibility for application in multiple or diverse clinical settings? | Yes | Options were presented based on evidence presented and summarized within Table 4 |
| Is there a timeline or plan for updates to these guidelines? | Can’t tell | Not stipulated. The authors described the reason for this update was due to the large amount of literature on dopaminergic medications for RLS and newer therapies. |