What exactly is an EPA, really?
EPAs, or Entrustable Professional Activities, are the basic building blocks of assessment within many models of Competency-Based Medical Education, such as Competency By Design. In theory, EPA assessments are supposed to serve as a collection of observations that put together a cohesive high-resolution picture from many little data points or pixels, much like a mosaic.
Photo by Jean-Philippe Delberghe on Unsplash
But in real life, the EPA experience often generates a scattered, incomplete mosaic of learner performance that is not as accurate, or as authentic as what actually happened. This is important because when it comes to resident perspectives on EPAs, we learned through our qualitative study that, really, authenticity is everything. Depending on your role with EPAs, here are some factors to consider to improve the teaching and learning experience.
To Program Directors and Competence Committees:
1. How are you communicating these EPAs to your residents?
Nearly all the residents who didn’t believe the EPA system to be working felt that EPAs were being communicated to them by their program as a numbers game, or a checklist that needed all its boxes ticked to be able to pass to the next stage of training. Meetings with their program directors and/or competence committees often consisted of what they felt amounted to pep talks, or at times, threats, to “get their numbers up.” This perception led to a highly selective collection pattern of EPAs, i.e. the ‘bad kind of cherry picking’, where residents would only seek EPAs retroactively once they felt they had already done well on a task.
ACTION: Residents believe an accurate, high-resolution picture of their lived experiences can be generated through the use of EPAs as low-stakes, formative rather than summative assessments that aim to elicit actionable feedback. This is the original intent of EPAs – they were never designed to be a numbers game, nor a checklist. The program must take the lead in ensuring the messaging to both learners and assessors is appropriate and uniform.
2. Are you establishing safe and reliable channels of feedback from your residents?
Many residents expressed a sort of catharsis, or relief, towards the end of the interviews during our study, at the opportunity to express their thoughts, air grievances, and offer suggestions. Most told us they didn’t have a satisfactory (or safe!) method of providing upstream feedback to their respective programs.
ACTION: Establish a regular and reliable method of collecting feedback from the primary stakeholders themselves, the learners. These items must be perceived to be taken into serious consideration and whenever possible, acted upon. Townhall format, or feedback-via-intermediary (e.g. academic coach as liaison to competency committee) can provide safer methods to encouraging feedback.
To Faculty Assessors
3. What kind of training and buy-in do you have yourself?
Residents were much more prone to “play the EPA numbers game” if they felt their faculty hadn’t fully bought into the system. This was especially true of programs with a pervasive culture of “service over learning,” where teaching, and by extension assessment and feedback, took a second seat to the daily duties of the profession. What ended up happening for many residents in this group was that the EPA became not much more than a self-assessment tool, that was asked for, triggered by, and filled out by the learner, with the faculty tending to sign off or make cursory edits at the most.
ACTION: Seek out training or training from the academic program at your site, or through the Royal College CPD program. As physicians we carefully and rigorously evaluate new clinical practices with scrutiny and study, and EPAs should be no different. Residents can tell, and are very discouraged, when you don’t engage with what is an important, feedback-generating tool for them.
4. How easy is it for a resident to get an EPA from you, really?
Residents felt that asking for an EPA often felt like asking for a favor. However, with more training and more buy-in, staff would often pre-emptively ask about and offer EPA assessments, even before the resident brought the issue up. They furthermore did so in a more timely manner, which was universally felt to be more valuable. In turn, the workplace culture seemed to pivot away from “service over learning,” as faculty were felt to consider daily EPA assessments an expectation and a duty.
ACTION: EPAs, just like any other assessment, can and should be weaved into the daily routine when working with learners. The value of the feedback documented in the EPA diminishes with time – the gold standard is to do them in-the-moment. The best response to “can I just send it to you via email?” should really be “Let’s do it now.” Excellence is to preempt and complete EPAs based on observed behavior without being prompted by the learner. It’s already hard to ask for something when there is a power differential. It’s even harder if they don’t feel you believe in the system.
To Resident Learners
5. How much responsibility are you taking for your own learning?
Without a doubt, the systems-based issues above can be a major hindrance to feeling like EPAs are actually here to help. Residents in our study who saw the most utility in EPAs shared the belief in the need for reciprocity of responsibility, meaning that for the EPA system to work, this would require a fully engaged and goal-oriented learner. These residents would often participate in the ‘good kind of cherry-picking,’ i.e. actively seeking out EPAs in which they felt they needed improvement, rather than putting on a show to pass.
ACTION: Take EPAs by the horns, and be aggressive in seeking out opportunities to improve, rather than put on a show. For sure, this is harder when you don’t feel supported – you can start with staff that you feel safer with, and eventually this will become the rule rather than the exception. Your role in this is as important as everybody else’s. Like it or not, EPAs are here to stay – what you put into them is what you will get out of them.
6. Keeping the peace is over-rated, say your piece
There were many suggestions for improvement from residents, unique to each learning environment (e.g. surgical/operating room versus ward) and to each service culture. Some were technological, such as the digital interface and how things could be streamlined for more efficient EPA collection. Some were scheduling-related, to increase exposure to rare clinical presentations. The unfortunate truth is many of these fantastic ideas never make it to the ears of those with the power to do anything about it.
ACTION: Speak up, and do so regularly and consistently in a united voice through your leadership (e.g. chief residents). Townhall format, or feedback-via-intermediary (e.g. academic coach as liaison to competency committee) can also be safe ways of providing upstream feedback to your program. “They” can only change what “they” know to be a problem.
Outro: Reclaiming Authenticity
The concern in much of the literature before the implementation of EPAs had largely been about a loss of authenticity through reductionism, or the oversimplification of complex professions into unrealistically small bite-size pieces that wouldn’t accurately reflect the whole. However, the authenticity problem we uncovered was, and still is, much more complex than a mere degree of granularity. The authenticity problem at hand is in fact driven by the dynamic interrelationships between the 3 key stakeholders (program, faculty and learner), and their relative function (or dysfunction!).
The problem is not with the EPA itself, but rather with the difficulty faced by the 3 key stakeholders tasked with collecting and putting the pieces together, to form a cohesive, authentic mosaic.
And so with that, I am calling upon all of you, learners, assessors and administrators alike, to reclaim authenticity in your own programs by recognizing, valuing and re-assessing the interdependent relationships between the EPA stakeholders on an ongoing basis. The EPA system is only as strong as the weakest link in the chain, and the system must be firing on all 3 cylinders to not only just work, but to provide true added value to the teaching and learning experience.
“Are you a program director? A medical educator? Read more here”
