What could the NHS appraisal system contribute to revalidation in pharmacy?

Independent review by Donaldson¹ and Foster² into the regulation of doctors and other non-medical healthcare professionals was followed by the publication of the White Paper “Trust, assurance and safety — the regulation of health professionals in the 21st century”.

This White Paper tasked healthcare regulators, including the pre-September 2010 Royal Pharmaceutical Society — and now the General Pharmaceutical Council — with a requirement to implement a system of revalidation that has not previously existed.³

Revalidation aims to reassure the public and is generally described as the process that allows healthcare professionals to demonstrate to their regulator that they remain up to date and fit to practise.^4,5 Possible components of a model for revalidation in pharmacy include the use of continuing professional development (CPD) portfolios and their monitoring and assessment, appraisals, or other sources of evidence, for the purpose of revalidation.

The undertaking and recording of CPD is embedded in the Code of Ethics⁶ and now the Standards of Conduct, Ethics and Performance⁷ and is a requirement for pharmacists and pharmacy technician registrants.⁸ There is some evidence that this form of revalidation may be acceptable and workable for the profession. However, further research will need to establish the possibility of effectively linking CPD records for revalidation purposes and that CPD can indeed assure professional competence, which has not been shown as yet.^9–11

A devolved model of revalidation, with appraisals as a cornerstone, is the model adopted by the General Medical Council (GMC) in the form of a licence to practise for doctors. This was introduced in November 2009.^12,13

All doctors wanting to practise in the UK have to hold such a licence that will need to be renewed periodically. The renewal will be underpinned by a system of revalidation and will have three main elements: participation in annual appraisal in the workplace, participation in an independent process for obtaining feedback from patients (where applicable) and colleagues, and confirmation from the responsible officer in their local healthcare organisation.

The GMC is in the process of developing a framework (for agreed generic standards of practice), based on ‘Good medical practice’,¹⁴ against which doctors’ practice can be appraised and assessed objectively. The framework will cover the domains of knowledge, skills and performance, safety and quality, communication, partnership and teamwork, and maintaining trust, and will be a revised system of NHS appraisal.¹⁵

Annual appraisals for all doctors working in the NHS were introduced by the Department of Health in 2001.¹⁶ The appraisals were a formative process that aimed to identify development needs and to provide doctors with regular feedback on performance.^17,18 They were normally conducted by a trained appraiser familiar with the work of the doctor and included clinical and non-clinical achievements, audit, training and education, and complaints.

The use of appraisals has been considered in a report by the pre-September 2010 Society revalidation advisory group to the Department of Health — “A draft model for revalidation in pharmacy”¹⁹ — as one possible component of a model for revalidation in pharmacy. Their workability depends on the engagement of employers.

More than two thirds of pharmacists work in community pharmacy,²⁰ where there is little knowledge about appraisals. Appraisals for NHS employees, such as pharmacists and pharmacy technicians who work in NHS hospitals or primary care organisations, follow the principles of the NHS Knowledge and Skills Framework (KSF).²¹ Indeed, Foster,² when reviewing the regulation of non-medical healthcare professionals, proposed that a system of revalidation for NHS staff be based on the KSF.²¹ This framework defines the knowledge and skills all staff need to provide quality services in the NHS, and includes a development review to ensure staff are given the opportunity to learn and develop.

The aim of our study was to explore the existence and content of current appraisal systems for pharmacists and pharmacy technicians employed by NHS hospitals or primary care organisations, and whether elements of them might lend themselves to a potential use for revalidation, possibly following some adaptation.

Method

A brief questionnaire was designed to explore whether appraisal systems are in place in primary care trusts and hospitals trusts in England; how frequently they are conducted; who conducts them and what is contained in them. Other questions explored whether respondents thought current appraisal systems could be adapted for the purpose of revalidation.

Space for open comments on the use of appraisal systems and revalidation was provided. The questionnaire was designed after a series of telephone interviews that were commissioned by the National Clinical Assessment Service.²² Evidence from the literature on the use of appraisals and revalidation also informed questionnaire design.

Comments on the content and clarity of question and answer categories were sought, initially, from pharmacist colleagues. The questionnaires were then piloted with a small number of pharmacists working in PCTs and hospital trusts, following which some minor changes were made.

The study was viewed as service evaluation and development by the National Research Ethics Service and therefore did not require ethics committee approval. This view was endorsed by the university ethics committee.

The questionnaire was initially sent by post in October 2009 and addressed to clinical governance leads in all 152 PCTs in England.

In November 2009, the postal questionnaire was sent to clinical governance leads in a 50 per cent random sample (n=85) of acute hospital trusts in England. Community and mental health trusts were excluded to ensure that a “true” secondary care perspective was captured.

Clinical governance leads were contacted initially, because they are responsible for continuously improving the quality of services and safeguarding high standards of care. Their remit includes appraisals, in particular dealing with issues of (poor) performance.²³ The idea of the questionnaire was to provide an employer/appraiser perspective, rather than that of individual appraisees.

A postal reminder survey was sent four weeks after the initial mailing to all non-responders in both groups.

Non-responding PCTs were contacted by telephone to obtain the name and contact details of the medicines management leads or the clinical governance leads. Subsequently, these named individuals were emailed or mailed another survey for completion 10 weeks after the initial survey mail-out.

The covering letter accompanying this first reminder to hospital trusts recommended that clinical governance leads contact their chief pharmacists if they were more closely involved with pharmacists’ performance and appraisals. The hospital trust survey was mailed to chief pharmacists of all the non-responding hospital trusts 10 weeks after the initial survey mailing.

Medicines management pharmacists in PCTs and hospital chief pharmacists were approached for reminders following feedback that responsibility for pharmacy professionals’ appraisals and performance was often delegated to them.

Data analysis

Data from all questionnaires were entered onto, and analysed, using SPSS software. No statistical comparisons between PCTs and NHS hospital trusts were carried out owing to small sample sizes, especially from hospital trusts. However, findings from PCTs and NHS hospital trusts are presented alongside each other, to facilitate comparison across these types of organisations. When presenting quantitative findings, frequencies and valid percentages are given.

Open comments from respondents were transcribed verbatim and grouped under similar themes.

Results

Following all reminders, 67 responses were received from PCTs and 28 from hospital trusts, giving response rates of 44 per cent and 33 per cent, respectively.

Appraisal systems currently in place

Respondents were asked to select whether appraisal systems were in place within their organisation. All respondents from hospital trusts stated that they had put in place appraisal systems for pharmacists and pharmacy technicians. Most PCTs (80 per cent) said they had an appraisal system in place, though in some it was only for pharmacists (30 per cent), while in others it was for both pharmacists and pharmacy technicians (50 per cent).

The following analyses of questions that refer to the use of appraisals will only examine those respondents from PCTs that reported that they had an appraisal system in place (n=53), either for pharmacists or for pharmacists and pharmacy technicians.

Respondents were asked who is responsible for conducting appraisals. The question gave the options of “line manager”, “clinical governance lead” or “other”, but all respondents from PCTs and hospitals trusts chose “line manager”.

Most respondents from PCTs (n=51; 96 per cent) and hospital trusts (n=21; 75 per cent) reported that appraisals were conducted about every 12 months. In some of the remaining PCTs (n=1) and hospital trusts (n=6), appraisals were conducted about every six months.

Other responses provided by both groups of respondents indicated that appraisals were conducted less frequently in PCTs (n=1, 2 per cent) and hospital trusts (n=1; 4 per cent), sometimes every 18 to 24 months. A small number of respondents from PCTs (n=1) and NHS hospital trusts (n=2) reported that in addition to an annual appraisal, a review was conducted every six months. No respondent from either PCTs or hospital trusts said appraisals were conducted in response to an incident or complaint.

Respondents were asked what was contained or discussed in appraisals: options and responses are listed in Table 1. Respondents were asked to tick all options that applied. They also had the option of providing a brief description of other details discussed within the appraisal system.

A review of progress on the personal development plans of pharmacists/pharmacy technicians was almost always contained in the appraisals, as was identification of learning and development needs, an agreement on a personal development plan, and a performance review. A review of the appraisee’s CPD log/record was a part of the appraisals in most (86 per cent) NHS hospital trusts. However, this was not routine practice among responding PCTs, where less than half (47 per cent) stated that a review of the CPD log/record was part of the appraisal. In NHS hospital trusts, the KSF was usually incorporated in an appraisal, but this option had not been included in the PCT survey.

Other responses were provided by five PCT and four hospital trust respondents. Some commented that objectives were set in the appraisal (PCT n=3; hospital trusts n=2). One PCT respondent and one hospital trust respondent said that meeting the requirements of the job description was contained in the appraisal. One further PCT respondent said that non-work issues were discussed and one hospital trust respondent stated that only the KSF was used as evidence.

Adaptability of current appraisal for revalidation purposes
Respondents were asked whether they thought their current appraisal system could be adapted for the purpose of revalidation, that is the positive affirmation of a pharmacist’s or pharmacy technician’s fitness to practise. A five-point scale ranging from “not at all adaptable” (1) to “very adaptable” (5) was used, and responses can be found in Table 2.

Almost a quarter of respondents from PCTs and NHS hospital trusts rated the adaptability of their current appraisal for revalidation purposes mid-range (3). In PCTs, more respondents (46 per cent) responded with a 4 or 5 rating, suggesting their current appraisal system was adaptable to some extent or very adaptable. Approximately 29 per cent of PCT respondents did not see their current appraisal system as adaptable (responding with 1 or 2).

Most (65 per cent) respondents from NHS hospital trusts indicated that their current appraisal system was adaptable, providing responses of either 4 or 5. Only 12 per cent of respondents from this sector reported that their current appraisal system was not very adaptable for revalidation purposes (2), but nobody responded that they were not at all adaptable (1).

A further question asked respondents to consider how they would feel about incorporating additional questions into appraisals for the purpose of revalidation. Respondents could rate their concerns, ranging from “no concerns” to “major concerns”. The data obtained from this section of the questionnaire are in Table 3.

The response type that received the highest percentage of respondents’ selections from both PCTs and NHS hospital trusts was “no concerns” (40 per cent and 50 per cent, respectively). The smallest proportion of respondents noted that they had minor or moderate concerns about these proposals rather than major concerns. A smaller proportion of respondents from PCTs (12 per cent) indicated that they had “major concerns” with this proposal, while 15 per cent of hospital trust respondents had “major concerns”.

Open comments about the use of appraisal systems for the purpose of revalidation

The final section of the questionnaire allowed respondents to make open comments about the use of appraisal systems for the purpose of revalidation. Nearly half of PCT respondents (n=24; 45 per cent) and half of hospital trust respondents (n=14; 50 per cent) provided additional comments. These are summarised in Panel 1, where the numbers of respondents for each point are given in parentheses.

Responses from PCTs and hospital trusts covered similar ground with regard to the use of appraisal for the purpose of revalidation. Although some respondents thought it made sense to link appraisals with revalidation, others thought that they should be kept separate. Some qualified this by explaining that appraisals and revalidation had different purposes: that one was about a supportive and open performance review, while the other was about performance monitoring.

Other comments concerned the issue of time taken to conduct appraisals. Further comments were made about who would be responsible for conducting appraisals for revalidation purposes, that sometimes pharmacy staff were appraised by non- pharmacy professionals, and that appraisers ought probably to be familiar with work sector and context. To meet these objectives and to ensure a consistent and unbiased approach, training for appraisers may be needed.

Some more general comments were made about revalidation, most of which came from hospital trust respondents. Some were positive and supported the need for revalidation. Others suggested that revalidation ought to draw on a number of sources and that CPD should be part of revalidation.

Discussion

Appraisals are being considered as a potential source of evidence for the purpose of revalidation in pharmacy.¹⁹ The findings from this questionnaire study provide useful and novel insights into appraisal systems used in NHS organisations, and views on their adaptability to revalidation. They are reported from an employer perspective: that of clinical governance leads (and lead pharmacists).

The results of this study have illustrated some of the similarities and differences between appraisal systems in PCTs and NHS hospital trusts in England. First, it appears that a framework for appraisals exists across hospital trusts, as all respondents from these organisations had an appraisal system in place for pharmacists and pharmacy technicians.

There seemed to be slightly less consistency among PCTs, as 20 per cent of respondents stated that no appraisal system was in place for pharmacists or pharmacy technicians in their organisation. One explanation for the finding that 30 per cent reported that an appraisal system was in place, but only for pharmacists, may be because these organisations did not employ any pharmacy technicians. However, this would need to be verified.

The most common practice was for appraisals to be carried out annually. All respondents from the PCTs and hospital trusts surveyed stated that the line manager was responsible for conducting the appraisal of pharmacists and pharmacy technicians as opposed to other people, such as the clinical governance lead.²⁴ It is not known if these line managers were pharmacy professionals, or if it was considered important that they were, and this merits further exploration.

When examining what is contained or discussed in the appraisal systems for employed pharmacy professionals, there were a number of similarities between PCTs and hospital trusts. All (or almost all) contained a review of personal development since the previous appraisal, the identification of future learning and development needs, and an agreement on a personal development plan for the coming appraisal period.

The KSF also featured strongly as an element in hospital trusts. (This answer category was not given as an option in the PCT questionnaire.) One notable difference between appraisal systems in PCTs and hospital trusts emerged with regard to whether a review of an appraisee’s CPD log/record was covered. This appeared to be the case in most responding hospital trusts, while less than half of PCT respondents reported that this was contained in an appraisal. This may benefit from further qualitative exploration.

Respondents from PCTs and hospital trusts had similar views on the adaptability of their organisations’ appraisal systems for the purpose of revalidation. In general more respondents than not thought their appraisal system was adaptable. This was particularly true for respondents from hospital trusts. Most respondents from both types of organisation had no or only minor concerns with a proposal of incorporating additional questions into an appraisal for the purpose of revalidation.

Open comments provided useful insights into other aspects of appraisals not covered in the questionnaire. In particular, concerns were raised over who would conduct revalidation appraisals and what their background and training might need to be.

Respondents had conflicting views over whether they thought the usual appraisal system could be combined with those undertaken for revalidation, or whether these would need to remain separate. Issues of time, who would conduct appraisals for revalidation, and where ultimate responsibility would lie were raised.

The importance of appropriate training was noted, and that an appraiser might need to understand a pharmacy professional’s sector and work, role and context. Many of these concerns have been raised in relation to the revalidation of doctors.^25–29

Finally, some comments suggested that appraisals should form only one source of revalidation evidence, where a portfolio of evidence, and CPD records in particular, would feature. This is already the case in revalidation developments in the medical profession, where doctors will need to collect a wide variety of evidence, including written documents (such as qualifications, examples of patient records, and CPD), and observational materials (through peer review and patient satisfaction surveys).³⁰

One limitation of this study is that responses reported here only cover NHS organisations in England. Even though the response rate from PCTs (44 per cent) was acceptable, a 33 per cent response rate from hospital trusts was disappointing. As revalidation under the GPhC will also apply in Scotland and Wales, it will be important to gain insights into appraisal systems in place there, as well as employers’ experiences of their current use and views on their potential adaptability for revalidation.

The questionnaire was distributed to relevant NHS organisations in England, Scotland and Wales, but the response rates from Scotland and Wales were so low (only five responses were received from Scotland and none from Wales), that it was decided to exclude these from analysis.

Conclusion

From the responses gathered from NHS PCTs and hospital trusts, it appears that there may be some scope to consider the use of appraisals for revalidation, but some questions would need to be explored further. These include, for example, any specific requirements appraisers would need to satisfy, such as their background and experience of the particular sector of practice, as well as training to ensure consistency.

Further questions will concern issues of objectivity of assessment and who will have ultimate responsibility for a revalidation assessment and feeding this back to the GPhC. The latter is the responsibility of the responsible officer for doctors,³¹ but no such structure is in place or being considered for pharmacy.

Finally, it is worth noting that these findings and conclusions only apply to pharmacy professionals employed in the NHS managed sector. Further pieces of work commissioned by the pre-September 2010 Royal Pharmaceutical Society are now managed by the GPhC. They were undertaken by the authors in other sectors employing pharmacy professionals, in particular community pharmacy, the pharmaceutical industry and academia.

References

1. Department of Health. Good doctors, safer patients: proposals to strengthen the system to assure and improve the performance of doctors and to protect the safety of patients. A report by The Chief Medical Officer Sir Liam Donaldson. London: Department of Health, 2006.
2. Department of Health. The regulation of the non-medical healthcare professionals. A review by the Department of Health, chaired by Andrew Foster. London: Department of Health, 2006.
3. Department of Health. Trust, assurance and safety — the regulation of health professionals in the 21st century. London: Stationery Office, 2007.
4. General Medical Council. Revalidation. GMC, 2010. Available at: www.gmc-uk.org/doctors/7330.asp (accessed 11 January 2011).
5. Hasman A. Get ready for revalidation in pharmacy. Pharmaceutical Journal 2009;283:485.
6. Royal Pharmaceutical Society. Code of Ethics for Pharmacists and Pharmacy Technicians. London: Royal Pharmaceutical Society, 2007.
7. General Pharmaceutical Council. Standards of Concust, Ethics and Performance. London: GPhC, 2010.J1J1
8. Royal Pharmaceutical Society. Continuing professional development. 2010. Available at: www.rpsgb.org/registrationandsupport/ continuingprofessionaldevelopment/ (accessed 3 September 2010).
9. Rouse MJ. Continuing professional development in pharmacy. American Journal of Health System Pharmacy 2004;61:2069–76.
10. Maguire T, Bell H. CPD is about learning for life. Pharmaceutical Journal 2001;267:594.
11. Winslade NE, Tamblyn RM, Taylor LK, Schuwirth LWT, Van der Vleuten CPM. Integrating performance assessment, maintenance of competence and continuing professional development of community pharmacists. American Journal of Pharmaceutical Education 2007;71:15.
12. General Medical Council. Licensing. GMC, 2010. Available at: www.gmc- uk.org/doctors/ licensing.asp (accessed 11 January 2011).
13. NHS Revalidation Support Team. Strengthening NHS medical appraisal to support revalidation in England. A proposal paper for piloting from the NHS Revalidation Support Team. London, 2009. London.
14. General Medical Council. Good Medical Practice. London: GMC, 2009.
15. General Medical Council. Revalidation — GMP framework for appraisal andassessment. GMC, 2010. Available at: www.gmc-uk.org/doctors/revalidation/revalidation_gmp_framework.asp (accessed 11 January 2011).
16. Department of Health. Supporting doctors, protecting patients: aconsultation paper. London: DoH, 1999.
17. Department of Health. Appraisal. London: DoH. 2007.
18. Department of Health. Appraisal for GPs. London: DoH, 2007.
19. Royal Pharmaceutical Society. A draft model for revalidation in pharmacy. Report to the Department of Health. London, RPSGB. 2009.
20. Seston E, Hassell K. An overview of the main findings from the 2008 pharmacy workforce census. Pharmaceutical Journal 2009;283:419–20.
21. Department of Health. The NHS Knowledge and Skills Framework (NHS KSF) and the Development Review Process. London: Department of Health, 2004.
22. Hassell K, Jacobs S, Potter H, Prescott J, Schafheutle E, Seston L. Managing performance concerns about pharmacists. A report for the National Clinical Assessment Service. 2010. NCAS, CPWS — University of Manchester.
23. NHS Clinical Governance Support Team. Assuring the quality of medical appraisal. NHS CGST, 2005. Available at: www.appraisalsupport.nhs.uk/files2/Assuring_ the_Quality_of_Medical_Appraisal.pdf (accessed 11 January 2011).
24. Fletcher C. Performance appraisal and management: The developing research agenda. Journal of Occupational and Organizational Psychology 2001;74:473–87.
25. Finlay K, McLaren S. Does appraisal enhance learning, improve practice and encourage continuing professional development? A survey of general practitioners’ experiences of appraisal. Quality in Primary Care 2009;17:387–95.
26. McKinstry B, Peacock H, Shaw J. GP experiences of partner and external peer appraisal: A qualitative study. British Journal of General Practice 2005;55:539–43.
27. Nayar V. A qualitative study of general practitioners’ views of the appraisal process. Education for Primary Care 2005;16:672–9.
28. Boylan O, Bradley T, McKnight A. GP perceptions of appraisal: professional development, performance management, or both? British Journal of General Practice 2005;55:544–5.
29. Jelley D, Cheek B, Van Zwanenberg, T Quality assurance of general practitioner appraisal: successes and challenges in the Northern Deanery. Education for Primary Care 2007;18:180–91.
30. Conlon M, Lyons N, Warner L, Anthony J, Judkins K, Pringle M et al. Using appraisal as the vehicle for revalidation: Squeezing a quart into a pint pot? Education for Primary Care 2009;20:77–80.
31. General Medical Council. Revalidation — Responsible Officer. 2010. Available at: www.gmc-uk.org/doctors/revalidation/ responsible_officer.asp (accessed 11 January 2011).