The educational impact of Mini-Clinical Evaluation Exercise (Mini-CEX) and Direct Observation of Procedural Skills (DOPS) and its association with implementation: A systematic review and meta-analysis.
- Lörwald AC Department of Assessment and Evaluation, Institute of Medical Education, University of Bern, Bern, Switzerland.
- Lahner FM Department of Assessment and Evaluation, Institute of Medical Education, University of Bern, Bern, Switzerland.
- Nouns ZM Department of Assessment and Evaluation, Institute of Medical Education, University of Bern, Bern, Switzerland.
- Berendonk C Department of Assessment and Evaluation, Institute of Medical Education, University of Bern, Bern, Switzerland.
- Norcini J FAIMER, Philadelphia, Pennsylvania, United States of America.
- Greif R Department of Anaesthesiology and Pain Therapy, Bern University Hospital, University of Bern, Bern, Switzerland.
- Huwendiek S Department of Assessment and Evaluation, Institute of Medical Education, University of Bern, Bern, Switzerland.
- 2018-06-05
Published in:
- PloS one. - 2018
Clinical Competence
Education, Medical
Educational Measurement
Humans
Outcome Assessment, Health Care
English
INTRODUCTION
Mini Clinical Evaluation Exercise (Mini-CEX) and Direct Observation of Procedural Skills (DOPS) are used as formative assessments worldwide. Since an up-to-date comprehensive synthesis of the educational impact of Mini-CEX and DOPS is lacking, we performed a systematic review. Moreover, as the educational impact might be influenced by characteristics of the setting in which Mini-CEX and DOPS take place or their implementation status, we additionally investigated these potential influences.
METHODS
We searched Scopus, Web of Science, and Ovid, including All Ovid Journals, Embase, ERIC, Ovid MEDLINE(R), and PsycINFO, for original research articles investigating the educational impact of Mini-CEX and DOPS on undergraduate and postgraduate trainees from all health professions, published in English or German from 1995 to 2016. Educational impact was operationalized and classified using Barr's adaptation of Kirkpatrick's four-level model. Where applicable, outcomes were pooled in meta-analyses, separately for Mini-CEX and DOPS. To examine potential influences, we used Fisher's exact test for count data.
RESULTS
We identified 26 articles demonstrating heterogeneous effects of Mini-CEX and DOPS on learners' reactions (Kirkpatrick Level 1) and positive effects of Mini-CEX and DOPS on trainees' performance (Kirkpatrick Level 2b; Mini-CEX: standardized mean difference (SMD) = 0.26, p = 0.014; DOPS: SMD = 3.33, p<0.001). No studies were found on higher Kirkpatrick levels. Regarding potential influences, we found two implementation characteristics, "quality" and "participant responsiveness", to be associated with the educational impact.
CONCLUSIONS
Despite the limited evidence, the meta-analyses demonstrated positive effects of Mini-CEX and DOPS on trainee performance. Additionally, we revealed implementation characteristics to be associated with the educational impact. Hence, we assume that considering implementation characteristics could increase the educational impact of Mini-CEX and DOPS.
Mini Clinical Evaluation Exercise (Mini-CEX) and Direct Observation of Procedural Skills (DOPS) are used as formative assessments worldwide. Since an up-to-date comprehensive synthesis of the educational impact of Mini-CEX and DOPS is lacking, we performed a systematic review. Moreover, as the educational impact might be influenced by characteristics of the setting in which Mini-CEX and DOPS take place or their implementation status, we additionally investigated these potential influences.
METHODS
We searched Scopus, Web of Science, and Ovid, including All Ovid Journals, Embase, ERIC, Ovid MEDLINE(R), and PsycINFO, for original research articles investigating the educational impact of Mini-CEX and DOPS on undergraduate and postgraduate trainees from all health professions, published in English or German from 1995 to 2016. Educational impact was operationalized and classified using Barr's adaptation of Kirkpatrick's four-level model. Where applicable, outcomes were pooled in meta-analyses, separately for Mini-CEX and DOPS. To examine potential influences, we used Fisher's exact test for count data.
RESULTS
We identified 26 articles demonstrating heterogeneous effects of Mini-CEX and DOPS on learners' reactions (Kirkpatrick Level 1) and positive effects of Mini-CEX and DOPS on trainees' performance (Kirkpatrick Level 2b; Mini-CEX: standardized mean difference (SMD) = 0.26, p = 0.014; DOPS: SMD = 3.33, p<0.001). No studies were found on higher Kirkpatrick levels. Regarding potential influences, we found two implementation characteristics, "quality" and "participant responsiveness", to be associated with the educational impact.
CONCLUSIONS
Despite the limited evidence, the meta-analyses demonstrated positive effects of Mini-CEX and DOPS on trainee performance. Additionally, we revealed implementation characteristics to be associated with the educational impact. Hence, we assume that considering implementation characteristics could increase the educational impact of Mini-CEX and DOPS.
- Language
-
- English
- Open access status
- gold
- Identifiers
-
- DOI 10.1371/journal.pone.0198009
- PMID 29864130
- Persistent URL
- https://folia.unifr.ch/global/documents/269922
Statistics
Document views: 21
File downloads:
- fulltext.pdf: 0