Dispel the myths and see the benefits of workplace-based assessments

The current imperative to deliver more for less applies as much to ensuring a competent and capable pharmacy workforce as it does to direct service delivery. The cost-effectiveness of work-based learning (WBL) and workplace-based assessment (WPBA) will therefore be under scrutiny.

The Academy of Medical Royal Colleges published a report entitled “Improving assessment” in 2009,¹ which explored a UK-wide approach to assessment in medicine. The present article explores some key themes from the report that apply to pharmacy, drawing on additional insights from “Time for training”, a review of the impact of the European Working Time Directive on the quality of training.² In this article, we use the term “trainee” as used in “Improving assessment”.

WPBA in medicine

“Time for training” states that medical training and the delivery of patient care are inextricably linked, and acknowledges that most training should occur in a workplace environment. Good quality training develops professionals who deliver high standards of safe patient care. Medicine has traditionally employed an experiential model of training, relying on trainees spending time delivering the service while developing their skills and knowledge. WPBA has until recently been largely informal, with formal assessments mainly testing knowledge, and clinical and practical skills only assessed as a component of formal exams. The move from predominantly experiential to competencybased training has advanced significantly in the past decade, with an increased focus on assessing doctors in the workplace.

The introduction of WPBA into postgraduate medical training in the UK occurred for foundation trainees under “Modernising medical careers”.³ “Improving assessment” asserts that rushed implementation and inadequate training and resources lead to cynicism and undesirable practice. WPBA has attracted other criticism within medicine, particularly that it is a tickbox exercise that “dumbs down” training, partly due to the association with competency-based training. Moreover, what has added to this negativity has been the requirement for a minimum number of assessments, the shortcomings of assessors and workplace assessments being undervalued or being viewed as simplistic or mundane.

ENDOSTOCK/DREAMSTIME.COM

WPBA in pharmacy

Post-registration pharmacy developments in WPBA demonstrate some interesting parallels with medicine.

An early introduction of formal postregistration WPBA occurred with the postgraduate diploma in general pharmacy practice developed in London and the South of England by the Joint Programmes Board collaborative in 2006.⁴ The WBL and WPBA methodologies used have attracted similar criticisms to medicine, including concerns about philosophy and inconsistent local educational infrastructures, including time to train and assess.

Pharmacists’ attitudes towards WPBA may depend on their skills and knowledge, personal experience of WPBA, local support and resources. As with medicine, WPBA is seen by some junior pharmacists as an unwelcome hurdle as they seek the next career step, rather than an opportunity to develop competence and capability to deliver a safe, effective service.

Key features of WPBA for medicine

“Improving assessment” indicates that most doctors reported to the General Medical Council demonstrate communication or interpersonal problems — domains that are best assessed in the workplace — rather than a lack of knowledge. Clinical competence is also acknowledged as covering more than knowledge and decision-making. Yet traditional knowledge-based examinations do not assess other domains such as interpersonal skills, professionalism, self-appraisal and the ability to improve. WPBA allows assessment of practitioners’ strengths and weaknesses in these areas, and demonstration of trainee progression. It also provides evidence for summative assessment decisions, and identifies practitioners in difficulty and patterns of behaviour that may help to define a “good” trainee.

As such, the main purpose of WPBA is to help trainees identify areas for improvement within the context in which they are being assessed. WPBA can provide valuable feedback to motivate and encourage aspirations to mastery. It can promote learning and inform learning objectives, provide evidence for review and demonstrate progression. WPBA is thus formative, not summative. Educational supervisors⁵ and trainees need to grasp this important distinction, particularly to overcome trainee reluctance to select challenging clinical situations for assessments.

Ideally, outcomes from a range of workplace assessments should be one component of the evidence on which a judgement is made about progress. Multiple observations are needed, preferably from a range of different assessors, to improve reliability. We recognise that the latter ideal may present challenges for training in community pharmacy. Nevertheless increasing the number of observations will have greater impact on improving reliability than increasing the number of items assessed. Assessment results should avoid numerical values but, instead, outline standards or expectations at the end of a period of training or those required for independent practice.

The relationship between trainee and educational supervisor should be balanced, establishing early on the expectations and responsibilities of both. The process of WPBA is intended to be trainee-led, requiring a level of maturity in the trainee that needs adequate facilitation⁶ and development. Educational supervisors should also ensure that WPBA sampling is appropriate, intervening to ensure that assessments are undertaken and are not last-minute. Educational supervisors should use WPBA to formulate development plans with their trainees, feeding them into appraisals and facilitating meaningful feedback to improve performance.

Crucially, WPBA relies on the notion that by deciding on the level of supervision required for a particular procedure, doctors regularly (and comfortably) make judgements on the ability of their trainees. These expert judgements are based on previous experience of trainees and will change over time. Each experience of a trainee contributes to the development of the observer’s own absolute standard of what is expected of a trainee at that particular point.

Application of key features to pharmacy

Pharmacy employers and educationalists may endorse many of these features, for example the need to address domains that are best assessed in the workplace and for senior staff to invest in the development of junior staff. Yet experience from the diploma in general pharmacy practice programme suggests that pharmacy may struggle with facilitating trainee-led development, developing appropriate self-directed learning6 and the overall benefits of formative WPBA. Some assessors doubt their ability to judge performance in a junior practitioner, although face-to-face and DVD assessor training has improved confidence in this area.

Barriers and problems in medicine

It is tempting to suggest that because WPBA requires the provision of feedback, which can improve learning and performance, the implementation of such assessment strategies will have a demonstrable positive impact. However, despite the considerable weight placed on them in postgraduate training, there is little in the medical education literature to support this claim.⁷

“Improving assessment” acknowledges that the tension between using WPBA tools for formative assessment (assessment for learning) versus summative assessment (assessment of learning) has yet to be resolved. The most vital aspect of WPBA is to detect and then inform development needs. However, this is often overlooked by trainees and trainers given the use of WPBA to contribute to academic progress decisions. Traditional mindsets regarding assessment, including trainee fears about “getting it wrong” and a psychological need to score highly throughout a programme, can reduce the developmental benefits of WPBA or foster reluctance to use more challenging cases from which they may learn more.

Unfortunately, WPBA often continues to be erroneously seen as separate from and additional to service delivery. Critics of WPBA are (sometimes rightly) suspicious of a perceived simplistic “tick box” approach to assessing complexities of professional behaviour when there is a list of competencies to assess.

There is evidence in medicine that supervisors are not always good at assessing the clinical skills of trainees. Training in the use of WPBA tools is critical, since even the most meticulously designed and validated assessment system or instrument loses its value if it is not delivered appropriately. The formative impact is lost if no feedback is given and if assessments are undertaken all at once, they cannot demonstrate progression. These issues are still encountered with WPBA in medicine.

Barriers and problems in pharmacy

Similar post-registration challenges are faced in pharmacy, and centre on attitudes, culture and the role of WPBA. These are some key experiences from the diploma in general pharmacy practice programme:

• Introducing such profound changes take time, resources and culture change
• Existing educational infrastructures⁸ essential for work-based learning and WPBA are inconsistent
• Some assessors lack the skills to assess trainees’ clinical skills and provide developmental feedback⁶
• Awareness is lacking that WPBA can aid performance management processes
• Pharmacists assessed using WPBA tools often fear “failing” assessments, due to misunderstandings about formative and summative assessment
• The “tick-box” view remains, namely, that WPBA is about getting assessments achieved for the portfolio rather than an opportunity to learn
• Broadening and developing access to assessor training is a key issue

Recommendations

We have tried to contextualise some recommendations from “Improving assessment” for pharmacy stakeholders (see Panel)

Conclusions

“Improving assessment” correctly asserts that ensuring high quality patient care tomorrow means providing excellent training for health professionals today. Training across the sectors of pharmacy practice cannot be divorced from service provision and so training has to be a fundamental part of the design and delivery of patient care. It is recognised that an institution that trains well also delivers high quality care.²

Pharmacy, like medicine, experiences many conflicting demands and limited time to spend on educational activities. It is more important than ever in the current economic climate that training programmes deliver real value for organisations.² Therefore, methods that fit into normal routines and work patterns are more likely to be successfully implemented. As WPBA becomes accepted as an essential part of the culture of quality training, the role of the facilitator and educational supervisor will require further development.

Moreover it is crucial that pharmacy employers value and develop infrastructure that encourages best and imaginative use of WPBA methods, and a risk-free opportunity for staff to develop the best possible clinical and professional skills and knowledge.

Our hope is that WPBA will be increasingly and effectively used in routine pharmacy practice. We have some early indications in the diploma in general pharmacy practice programme that this is the case. For example, diploma graduates are now becoming supervisors and report particular familiarity with the role and delivery of WPBA.

References

1. Academy of Royal Medical Colleges. (2009) Improving assessment. 2009. Available at: www.aomrc.org.uk (accessed 10 June 2011).
2. Temple J Time for training: a review of the impact of the European Working Time Directive on the quality of Training. 2010. Available at: www.mee.nhs.uk (accessed 10 June 2010).
3. The Foundation Programme: Training and Assessment Available at: www.foundation programme.nhs.uk(accessed 11 June 2011).
4. Connelly D. Post registration education and training: a potential model for future revalidation. The Pharmaceutical Journal 2008;280:223–4.
5. Jubraj B, Fleming G, Wright E, Jones S, Cook S, Morris K. Say goodbye to clinical tutors: standardising the terminology in education. The Pharmaceutical Journal 2010;285:191–2.
6. Jubraj B. Developing a culture of self-directed workplace learning in pharmacy. The Pharmaceutical Journal 2009;283:47–8.
7. Sandars J. Continuing medical education across Europe. BMJ 2010;341:c5064.
8. Jones S, Safdar A, Jubraj B. Educational Infrastructure: teach a man to fish and you feed him for life. The Pharmaceutical Journal 2010;284:45.