Programmatic Assessment of General Practitioners’ medical competences

During a doctoral project, the ePortfolio proof-of-concepts will be validated in the context of General Practitioner training programs.

This work package has different aims. Firstly, the proof-of-concept ePortfolio will be validated in view of van der Vleuten’s programmatic assessment model. This model sequences assessment tools (from workplace-based assessment to multiple-choice tests) combined with feedback (peer feedback, multi-source feedback, patient satisfaction surveys) and self-reflection. Secondly, the researchers will investigate how formative and summative assessment can be combined in a programmatic ePortfolio assessment. Thirdly, this work package will explore the use of a programmatic lifelong learning ePortfolio.

Current research

This work package focuses on the assessment of competences within the ePortfolio in the context of GP training. Last summer, we investigated with three focus groups how competence assessment proceeds within general practitioner training. This showed that competence assessment at the workplace is part of the daily life of a general practitioner in training.

We also looked at how the ePortfolio can support and facilitate assessment in practice. The results of that study are ready for publication and was presented on ICME 2021. You can read the abstract here.

Important lessons from Factors Influencing Users’ Perceptions on Workplace-Based Formative Assessment: A Grounded Theory Study:

  1. Workplace-based formative assessment (WBA) has met with mixed reactions from users in postgraduate medical education. Despite its usefulness for workplace learning, users’ perceptions about WBA tend to be negative.
  2. To successfully introduce a new ePortfolio that supports workplace learning, exploring factors influencing users’ perceptions about formative assessment in the workplace deemed necessary.
  3. Three predominant categories of factors were identified in the data:
    • Users’ engagement and agency in the assessment progress
    • Users’ conceptualization about their own role in the curriculum
    • Users’ trust and their relationship with each other

A Delphi study was carried out to validate the CanMEDS core competencies within the GP training. The purpose of this study was to validate the CanMEDS competences based on the criteria of test feasibility and test consistency within the Flemish General Practitioner program (for more information about the study design, do not hesitate to contact the responsible researcher).

The results showed discrepancies between these two criteria. A CanMEDS competence that is considered "feasible" does not automatically mean that it can be evaluated in a consistent manner. While the CanMEDS competencies provide a comprehensive framework, they remain vague and too generic to support workplace assessments. Therefore, there is a need for additional specific competences and / or partial competences that more clearly identify and define the competences to be achieved for students.

Future research

We are going to explore the literature on programmatic assessment in an ePortfolio in various healthcare training courses. Program-specific assessment criteria based on the CanMEDS framework for general practitioner training will be validated. Finally, a pilot study must test a competency-based way of evaluation in practice.

The study "Implementing competency-based assessment in the workplace: where theory meets practice" will be presented at the AMEE conference that takes place from 27-31/08. You can read the abstract below.