Development and reliability of a clinical pharmacy triage tool in the emergency department

Abstract

Introduction: Rising patient numbers and increasing medical complexities have necessitated the clinical pharmacy service utilising staff effectively, while targeting patients with the highest pharmaceutical care needs. A triage tool was developed to assist with the provision of a clinical pharmacy service tailored to meet the needs of such patients. 

Aim: To develop and adapt a definitive clinical pharmacy triage tool (CPTT) to identify patients by their pharmaceutical needs and to assess how reliable a CPTT is when used in practice by a team of emergency department (ED) pharmacists.

Method: A mixed-methods design comprising three phases allowed for appropriate data triangulation and evaluation. Phase 1 used key informant interviews to explore the factors that were important in the initial design and adaptation of a CPTT. Phase 2 utilised action research methodology to rapidly adapt the CPTT into a definitive version. Phase 3 determined inter- and intra-rater reliability data of the definitive CPTT when used in practice within an ED setting by clinical pharmacists.

Results: Phase 1 data indicated several themes important in the development of a CPTT, including medication, disease and patient-related factors. These were adapted into the design of the CPTT, which was further modified in phase 2. When the definitive CPTT was used in practice by ED pharmacists (phase 3), reliability data demonstrated substantial and almost perfect levels of agreement (Cohen’s kappa 0.65–0.86).

Discussion and conclusion: The study demonstrated that a CPTT could be adapted, refined and used locally within an ED setting in an acute hospital environment. Potential barriers and limitations may restrict the application of the CPTT in other clinical areas, until further testing can demonstrate appropriate reliability results. Further work is required to demonstrate the benefit of using the CPTT in terms of patient-related outcomes and whether its use has any effects on staff satisfaction or staff utilisation.

Keywords: Emergency department; pharmacy triage; pharmacy risk assessment; clinical pharmacy triage tool; triage tool; emergency department pharmacist; emergency medicine pharmacist; pharmacist.

Original submitted: 3 May 2019; Revised submitted: 12 March 2021; Accepted for publication: 18 March 2021.

Introduction

An investigation between 2009 and 2017 revealed that targets for emergency department (ED) performance in Northern Ireland (NI) have never been met​[1]​. Acknowledging the need for improvement, the NI Department of Health commissioned service-wide reviews of the health and social care system in 2014 and 2016, focusing on local service delivery​[2,3]​. The reviews identified implementation of a more patient-centred model of care as an important focus area​[2,3]​. Value for money and better distribution of resources were also highlighted as major areas for development​[2,3]​.

The role of pharmacists in benefiting patients and healthcare systems has been examined by Health Education England and there is emerging evidence to support pharmacy teams as an essential component of smooth and safe ED operation​[4–7]​. UK pilot studies have shown that one-third of ED attendees can be independently managed by experienced and appropriately trained clinical pharmacists​[6]​. Benefits of ED pharmacist intervention during a patient’s hospital journey include: increased safety from medication-related harm; a reduction in omitted medication doses; enhanced medicines reconciliation; and an improved patient experience​[8]​.

Ulster Hospital (UHD), which forms part of South Eastern Health and Social Care Trust (SET), has one of the busiest EDs in NI with almost 88,000 ED attendances recorded in 2015​[9]​.

These patients often require a pharmacy or medication review. However, limited resource means clinical pharmacy services occasionally struggle to provide consistent cover to all inpatient beds and clinical areas.

In 2016, the ED clinical pharmacy team at UHD began operating seven days per week to help facilitate patient flow and timely review of medication-related issues. The team comprised 4 whole-time equivalent (WTE) clinical pharmacists and 1.5 WTE pharmacy technicians ranging in experience level. Daily duties included medicines reconciliation and optimisation, discharge planning, rationalisation of current treatment and provision of prescribing and administration advice. This was in addition to performing core pharmacist duties, such as education and training, governance and research and development. Senior ED clinicians and pharmacists have previously discussed the potential for further clinical pharmacy input with higher complexity patients within the ED; however, with no increase in pharmacy resource, this needed to be weighed against the risk to patient safety in existing clinical areas.

The NI Clinical Pharmacy Standards state that clinical pharmacy services should facilitate the safe, effective and economic use of medicines at all stages of the patient’s hospital journey​[10]​. Performance against such targets may be internally audited using key performance indicators (such as medicines reconciliation within 24 hours of admission)​[10]​.

In order to comply with these standards and to allow for further development of ED pharmacist roles, senior pharmacy management at SET recognised the need for a patient risk assessment and prioritisation system. In 2015, a triage-based tool was considered as an option to help clinical pharmacists identify and target patients who require clinical pharmacy input.

Triage in healthcare is not novel, and the use of risk assessments in clinical areas to determine individual patient needs and the required level of care is often outlined​[11–17]​. Within the ED environment, triage systems are critical for safe and effective ED management and are used routinely by nursing and medical staff to prioritise competing demands​[14]​.

Such a system could help ED pharmacists balance their workload more effectively, allow time for other responsibilities and facilitate the onward communication and handover of information via the triage tool. This would allow for optimisation of care and resources as patients transfer between clinical areas.

In Scotland, pharmacists use triage-based systems to help target their pharmacy services to patients at high risk of medication-related harm​[15–17]​. PharmacyView is a real-time software system that assigns a priority code to patients based on predictors of medication-related harm, including medicines prescribed and individual patient needs. This allows all clinical pharmacy staff at NHS Scotland Health Boards to visualise prioritisation of patients on an ‘electronic whiteboard’​[17]​.

The use of clinical pharmacy triage assessment tools in other clinical environments has also been emerging​[18,19]​. Clinical pharmacists working within NHS Scotland obstetric services assign patients into two risk categories, so only patients who require an intervention, or those for whom pharmacist input is specifically requested, are reviewed​[18]​. In Manchester, pharmacist attitudes towards using and developing a pharmaceutical assessment screening tool (PAST) across different clinical areas have been studied​[19]​. The authors concluded that participating pharmacists were confident in using assessment tools to assign a grade or code as a predictor of future involvement, further supporting the idea of clinical pharmacy triage. However, it was acknowledged that differences in pharmacist experience can potentially lead to differences in the application of the PAST, indicating the need for reliability testing of locally developed triage tools​[19]​.

The need for such testing is again highlighted in one study, where use of a locally developed PAST was trialled in a 900-bed teaching hospital in Manchester​[13]​. Feedback from participating pharmacists revealed that in 57% of patients (n=20), pharmacists disagreed with the risk category assigned by the PAST​[13]​.

Internationally, a pharmacy team in New Zealand has identified 38 clinically related characteristics that correspond to a priority risk assessment score, enabling their team to undertake clinical review and medicines reconciliation using a more targeted approach​[20]​. Combining technology, risk assessment and triage has also been undertaken and evaluated in the United States, where a different healthcare model and approach has shown benefit from prioritising medication-related monitoring needs and targeting those patients with the highest risk-related requirements​[21]​.

While there appear to be triage and risk assessment tools already employed in a range of pharmacy settings, a lack of ED-specific models presented an opportunity for research in this area. Consequently, in 2015, the clinical leads within SET began development of an ED pharmacist triage tool, building on existing UK examples​[15–17]​. Further development and modification were identified as requirements for robust application at a local level. 

Aim

To explore the factors that are important in the local development and modification of a clinical pharmacy triage tool (CPTT) for use by clinical pharmacists within the ED at the Ulster Hospital. To test the reliability of using a CPTT to undertake a risk assessment of a patient’s clinical pharmacy needs within the ED.

Method

To meet requirements, the study was split into three discrete phases (see Figure 1). A mixed-methods approach was chosen as the optimal way to obtain a balance in understanding relative issues and the problems associated with research within the healthcare setting​[22]​.

Although this study was deemed to be a service evaluation project, appropriate ethics approval was submitted and obtained from the local NHS research and ethics team and Cardiff University ethics team.

Phase 1 predominantly explored what key informants determined to be the main factors and themes important for the development of an initial CPTT. Phase 2 followed the initial evaluation: feedback was obtained, examined and utilised, using action research methodology, to further modify and develop the CPTT into a definitive tool that could be tested for reliability. Phase 3 tested the reliability of using the definitive tool in practice within the ED of the UHD. Throughout all phases of the study, usual care was continued irrespective of CPTT.

Phase 1

Key informant interviews were undertaken with participants trialling the triage tool to obtain information related to important factors in the design of a CPTT. Participants were pharmacy and medical professionals actively working in the ED and acute medical environment at UHD.

Owing to the exploratory nature of the data required to develop the CPTT, it was decided that qualitative data collection would be most appropriate​[23]​. Several approaches were considered; however, the interview method was chosen owing to the benefits of a face-to-face approach and it being the commanding method for data collection for novel approaches, such as the CPTT​[23,24]​.

A semi-structured interview technique was chosen in preference to a structured one. All participants were given the same topics to consider, which is important owing to the individual nature of the participants and their potential different experiences of clinical pharmacy triage at the point of interview​[25]​. Criterion purposive sampling was used to identify and recruit participants, the main criteria being pharmacy or medical professionals actively working in the ED and acute medical environment at UHD​[26]​. This method was chosen over other sampling methods, owing to the exploratory nature of identifying key factors related to clinical pharmacy triage in phase 1 of the study​[26]​. Given the reliance on the close working partnership between medical and pharmacy staff, both were included in the sample. Furthermore, it was acknowledged that medical staff would have practical experience in using triage tools and this would bring suitable benefits to the data during this phase.

Letters of invitation and a participant information sheet were sent to potential participants. Those who agreed to take part were subsequently invited to complete a written consent form in line with best NHS research practice​[27]​.

During interviews, an interview schedule and prompt sheet were used to help outline key questions and prompts in order to guide the interviews and ensure any potential bias was kept to a minimum​[25,28]​. Question topics included: identifying previous experiences of triage; ideas surrounding design of the triage tool; important factors to consider; and any perceived benefits and limitations of using the triage tools in practice. The interview schedule followed a sieve-based approach, initially exploring general views, before exploring more specific topics​[25]​.

Following completion of the semi-structured interviews, thematic analysis was undertaken to develop a coding tree using a blend of inductive and deductive approaches​[29]​. Data were coded and the key factors were identified. From this, an initial CPTT was further developed.

Phase 2

The second phase focused on pilot use of the CPTT by clinical pharmacists within the ED setting.

Action research method was utilised so that a systematic and cyclical approach of ongoing CPTT development was adopted at the same time any changes were being undertaken and evaluated​[30]​​[31]​. For such research to be effective, the author — rather than outsiders to the team — undertook the test and evaluation process themselves​[32]​. This approach meant the author, an experienced ED pharmacist, used and iteratively developed the CPTT in practice, alongside a team of experienced ED pharmacist colleagues.

A simple paper-based feedback form was chosen to obtain improvement data. The form was designed to be succinct and to only obtain data that were relevant for the continued development of the CPTT​[33,34]​. A mixture of closed and open questions were chosen to ensure relevant data were captured for further development​[33]​. There were five questions in total, which related to: whether the CPTT could be used to categorise all patients reviewed (yes or no); whether there were factors that prevented or assisted in triage (open question); what participants liked about the CPTT (open); ideas for development (open) and other general suggestions (open).

In order to ensure there were adequate opportunities for each version of the CPTT to be used in practice and to allow for subsequent feedback, no amendments were made until a time period of at least one week had passed, unless there were critical failings (such as not being able to categorise a patient). This ensured the CPTT was used on a variety of patient types. Throughout phase 2, the CPTT was used in routine practice, identifying and risk-assessing patients as had been done prior to the CPTT introduction. Participants were asked to complete the CPTT feedback form, and these data were analysed within 48 hours so that a new CPTT could be developed in a timely manner. Using the cyclical approach allowed the changes to be implemented as suggested by the participants, increasing motivation for inclusion in the study and setting a shared agenda for change​[32]​.

Phase 3

The aim of the final phase was to test the intra- and inter-reliability of the definitive CPTT when used in practice within the ED to ensure that the tool could be used by the whole team consistently. Reliability was assessed using intra-rater and inter-rater reliability measures​[35]​. These measures demonstrated the extent to which there was agreement between individuals and groups of pharmacists, respectively, using the CPTT to risk-assess patients based on their clinical pharmacy needs.

Given the requirement to test the reliability of an assessment tool being used in clinical practice, it was determined that reliability would be examined in terms of agreement of triage categorisation using the tool in practice with quantitative data. A similar study assessing agreement of measuring patient acuity to prioritise pharmaceutical care also used reliability testing to determine agreement between pharmacists with a risk assessment tool, providing merit for this approach​[13]​.

Procedure

Participants were asked to use the definitive CPTT in practice for each patient they reviewed during the entirety of phase 3. The triage category of every patient reviewed using the CPTT was recorded in a table and a computer-generated random sample of patients was chosen for further reliability testing. Medical notes were obtained for these assigned patients. The notes and a copy of the medication chart were shown to the same pharmacist at a minimum of a two-week interval to test for intra-rater reliability. The same process was undertaken by a different pharmacist for inter-rater data. A two-week interval was chosen as the authors believed it would provide enough time to reduce the bias associated with the participant recognising the details of the patient they had previously reviewed or potentially discussed with the multidisciplinary team.

A total of 15 patients were chosen for each step of phase 3 reliability testing: initial CPTT patient categorisation, intra-rater test and inter-rater test. This equated to a total of 60 different patients, or 15 patients per pharmacist per step, giving a total of 45 different patient assessments for each pharmacist. Each pharmacist assessed a total of 30 patients individually as described in the Box below:

Intra-rater reliability testing was undertaken using the same 15 patient details previously triaged by the same pharmacist. Randomly allocating ED clinical pharmacists into two sets of pairs and assessing the extent to which their assessments matched the relevant initial assessments, using the Cohen’s kappa test, explored inter-rater reliability.

It was determined that undertaking reliability testing for consistency using this method enhanced the reliability of the definitive CPTT, as it evaluated the extent to which the tool produced the same outcome with the same patient data for each individual pharmacist and team of clinical pharmacists using the tool.

Intra-rater reliability

A test-retest method was chosen to evaluate intra-rater reliability, given the simplicity of the approach and the fact that subsequent assessments are based on similar information, reducing the chances of bias​[35,36]​.

Percentage of absolute agreement, whereby the total number of agreements is divided by the total number of ratings, was chosen​[37]​. While this method does not consider the probability of chance, given the relatively small number of nominal categories expected, it was determined to be the most appropriate test.

Inter-rater reliability

A Cohen’s kappa statistical test was chosen to determine the extent of inter-rater reliability, given its application for agreement between two raters (clinical pharmacists) using the CPTT; that data were nominal; and agreement was corrected for chance​[35,38]​. Furthermore, there is published evidence suggesting the use of Cohen’s kappa in the evaluation of triage system reliability testing and in previous studies evaluating pharmaceutical assessment tools​[13,39]​.

Results

Data illustrated several themes that are important in the development of a CPTT, including medication, disease and patient-related factors. These themes were adapted into the design of the initial CPTT, which was further modified in phase 2. When the definitive CPTT was used in practice by a team of pharmacists working in the ED (phase 3), reliability data demonstrated reassuring levels of agreement. Below is a breakdown of detailed results by each phase of the development process.

Phase 1

A total of 11 participants were included in phase 1. Table 1 illustrates key demographics, including job role/status at the time of interview. The average length of interview time was around 29 minutes and 30 seconds. Interviews took around six weeks to complete, from 29 May to 10 July 2017.

Based on the interview data and the field notes made during the interviews, key themes and subcategories were analysed using a combination of deductive and inductive approaches. A coding framework was developed based on the key themes and subcategories, as shown in Figure 2.

The data determined several key factors related to the design, content specifics and potential applications of a CPTT. The key factors in the development of an initial CPTT could be divided into three main themes and further sub-factors, as outlined below.

Medication-related factors:

• Critical medicines;
• Monitoring of high-risk/critical medicines and therapeutic drug monitoring (TDM);
• Adverse effects owing to medicines;
• Drug interactions;
• Polypharmacy.

Individual disease patient-related factors:

• Renal insufficiency or disease;
• Hepatic insufficiency or disease;
• Disease-specific requirements (e.g. swallowing difficulties or nil by mouth).

Other specific patient-related factors, including social and discharge requirements:

• Designated for priority discharge or current discharge status;
• Non-medical-related admission (e.g. social-related admission).

Two main options for categorisation were discussed. These included a definitive categorisation (use of red, amber, and green) approach or a continuous scale. Given the ease of a nominal scale, previous work within SET and that most participants suggested this format, a traffic-light system was chosen for the initial CPTT design.

Factors were grouped into subsequent triage categories and review periods and developed into the initial CPTT for use in phase 2. Categorisation into groups was based on a mixture of previous work, interview data and author discretion. It was anticipated that phase 2 feedback would provide necessary data related to whether this categorisation was appropriate.

Phase 2

Data were collected over a seven-week period from 7 August to 22 September 2017, during which participants provided feedback using the feedback form. Amendments were made every one to two weeks following a suitable trial period. In total, four versions were modified, based on feedback from 12 feedback forms. Data were collected on all versions of the CPTT following phase 1. Common feedback themes included formatting issues; patients not fitting into discrete categories; patients fitting into multiple categories; capturing priority discharges; and what constitutes complex regimes and polypharmacy. Critical feedback included inability to use the tool to determine a grading with version one. This prompted urgent review and development. The final CPTT version was approved once no additional feedback was provided and no further issues were encountered.

The finalised CPTT is illustrated in Figure 3.

Phase 3

Data were collected over an approximately nine-week period from 2 October 2017 to 1 December 2017. A total of 15 patients for each pharmacist were chosen to be appropriate for each component of reliability testing, based on the total number of patients triaged during this time. This meant a total of 60 different patients were assessed for appropriate clinical pharmacy triage during phase 3. Each patient was assessed initially, then again for intra-rater reliability, and then finally for inter-rater reliability, giving a total of 180 assessments during phase 3.

Intra-rater reliability

Table 2 illustrates the percentage of absolute agreement between each ED pharmacist between initial CPTT categorisation and subsequent re-evaluation following an approximately two-week interval.

The average percentage of absolute agreement across the team of ED pharmacists was 87%.

Inter-rater reliability

Pharmacists were randomly allocated into pairs and comparisons with initial triage data were undertaken for each pharmacist in that pair. A percentage of absolute agreement and a Cohen’s kappa result were calculated for each pair​[38]​. As each clinical pharmacist initially assessed 15 patients, a total of 30 matches were possible for each pair.

ED pharmacist pair 1 (pharmacists 1 and 2)

Percentage of absolute agreement = 93%

Cohen’s kappa = 0.86

Of the 28 total agreements, 13 were red and 15 were amber. One disagreement related to pharmacist 1 assigning red and pharmacist 2 assigning amber. The remaining disagreement related to pharmacist 2 assigning red and pharmacist 1 assigning amber.

ED pharmacist pair 2 (pharmacists 3 and 4)

Percentage of absolute agreement = 83%

Cohen’s kappa = 0.65

Of the 25 total agreements, 18 were red and 7 were amber, indicating a higher number of red patients when compared with ED pharmacist pair 1. With respect to the disagreements, pharmacist 3 assigned 3 green categories to patients, who were assigned as amber twice and red once by pharmacist 4. The other two disagreements were similar to ED pharmacist pair 1, where one assigned amber, the other red and vice versa. The discrepancy of green to amber/red was determined to be owing to individual interpretation of the patient, rather than use of the CPTT.

Summary

Overall results for reliability have demonstrated that while there is a small level of inconsistency among the team of ED pharmacists when using the CPTT, reliability has been demonstrated to be substantial in most cases. A Cohen’s kappa level of agreement between 0.61–0.80 would be considered as ‘substantial agreement’, with levels between 0.81–1.0 considered as ‘almost perfect agreement’​[35]​.

Discussion

This study achieved its aims of developing a local CPTT for use by clinical pharmacists within the ED and assessing its reliability. While much of the existing literature regarding triage omits details regarding the initial designs of tools, one study outlined that the PAST developed was initially designed by an individual consultant pharmacist working in a medication safety role​[13]​. This indicates that such tools can be developed by specialist individuals, as opposed to a group of key informers, as was undertaken in this study. Interestingly, when exploring the attitudes towards using the PAST in practice, it was found there was great disparity in the application of the tool between different grades of staff based on their years of experience​[19]​. It could be argued that disparity in use of the tool could be owing to the limited input from a variety of team members, although this is not a definitive conclusion. Therefore, it was determined that using a group of key informers from a range of pharmacy and medical roles in the development process for the CPTT was a strength, and perhaps accounts for the improved reliability data.

Careful consideration was given to the sampling method chosen for phase 1. Table 1 demonstrates the wide range of participants included, and it was determined to be an advantage to gain the input of a wide range of grades of staff in order to reduce disparity in application of the tool as found previously​[19]​. The decision to include medical staff in phase 1 was made after observing how closely pharmacists work with medical staff on a day-to-day basis. Medical staff involvement was not described explicitly in any previous literature, and it was therefore important to use this opportunity to ascertain if any additional information relating to triage development could be identified in comparison with other literature in this area. Medical staff focusing solely on patient-specific clinical/social factors was considered as a potential limitation; however, in practice, medical input complemented the data in phase 1. Reviews of the NI health and social care systems have outlined the importance of developing relationships between different healthcare professions within the wider team, and it was concluded that involving medical staff in the initial design would add strength when adopting any pharmacy-related triage into the trust​[2,3]​.

Given the complexities associated with the design and undertaking of interviews, it was concluded that data obtained in phase 1 were largely successful in that they provided the necessary information for the initial design of the CPTT​[28,33]​. Scrutiny of the interview data was undertaken by thematic analysis, allowing for identification of key factors.

In terms of specific themes, most participants stated that similar factors (e.g. medication-related, disease/physiology-related and social/medication use-related factors) were important in the design of the initial CPTT, and this perhaps demonstrates the consistent approach of clinical pharmacy services and input in the ED and clinical areas in general. All participants were able to discuss triage without further explanation by the interviewer, providing more evidence that staff already had a good general insight into this topic. In addition, all participants provided opinions when pressed on individual components regarding triage and its application in practice, demonstrating their insight into its potential application within SET.

In terms of identifying the main factors important for the tool, interview data provided the required evidence to satisfy the objective. All participants outlined medication-related factors as important for the tool, and this fits with literature that considers prioritising patients based solely on medication-related factors and the importance of medication-related factors when risk-assessing patients​[11,21,40,41]​.

In comparison with previous tools developed, findings from this study indicate that several different factors are important in the design of a CPTT​[13]​. Previous work has concluded that tools need to be considered locally to be effective, and larger-scale studies using validated tools are required before results can be generalised and used reliably to prioritise pharmaceutical care​[13]​. While this study adds strength to the design of the tool — given wider pharmacist and medical involvement at a local level — the generalisability of the tool is still debatable in wider applications, and requires further appraisal before it can be reliably implemented into alternative clinical areas for decision-making within the trust.

As with the PAST, disease- and patient-related factors were also identified within phase 1 as important factors in the risk assessment of patients, highlighting that other factors, as well as medication, need to be considered​[13]​. The results indicate that these are of equal importance to medication-related factors, and therefore suggest that tools based solely on medication-related factors may be limited in terms of their application to a clinical environment. This allowed the initial design of the CPTT to include information related to patients’ renal and hepatic function, comorbidities, critical disease states and whether patients should be highlighted for urgent or priority discharge. Social factors, including issues related to medication adherence, were also included, as they were considered to have an impact on the holistic care of the patient.

For the design of the CPTT, most participants suggested a categorical proposal in favour of a continuous score, given the ease with which this could be adopted and applied. In comparison to the previous literature, attitudes towards the application of such triage tools suggested that pharmacists use a mixture of professional judgement and the tool itself to categorise patient risk​[19]​. Therefore, it was considered most prudent to adopt a simple traffic light system with which most users would be comfortable. This approach was strengthened by medical opinion, which suggested a traffic light system was appropriate for visualising more easily which patients required being seen urgently. Given that pharmacists are already making difficult decisions in challenging environments, the use of a traffic light system was proposed as a relatively uncomplicated scoring system to interpret, making the process of triage more straightforward at ward level and when used by other pharmacists when patients are transferred. This was important, as a further objective was to use the tool throughout the trust in all areas, so that it could assist staff in using pharmaceutical prioritisation as part of a developing seven-day pharmacy service.

Most participants discussed the platform for the CPTT at length, and while most stated they would prefer an electronic system, it was acknowledged that technology limitations may preclude this from initial implementation and that a paper format might be appropriate for initial designs and roll-out. In 2015, clinical leads within SET were involved in the development of the pharmacy section of the electronic whiteboard used within the trust, in preparation for a tool that could be used as part of a pilot for pharmacists attending post-take ward rounds. Electronic systems have been proven to allow better use of resources, and careful consideration was given to the initial design of the tool in light of these findings​[17]​. Given the preference for electronic systems identified by the interviews in phase 1 and the previous literature, it was decided to continue using a combination of paper and electronic platforms in the developed design and application of the CPTT, satisfying this approach as much as possible within the limits of the study​[17]​. This led to the development of a paper-based CPTT, the results of which could be uploaded on to an already available electronic whiteboard, allowing for visualisation of triage electronically. The risk of transcribing errors between paper and electronic systems was considered; however, given the resources available and on weighing up different options, this paper–electronic approach seemed the most feasible at the time.

All participants highlighted the importance of the timing of triage, referral and follow-up, leading to the conclusion that this must be considered in any wider roll-out. The difficulties associated with using triage in practice indicated that users needed to be confident that any patient they triaged would be followed up in a timely manner, otherwise there was the risk of assigning higher categorisation inappropriately​[19]​. Therefore, time periods for review were altered in comparison with other triage tools, to ensure inappropriate ‘up-triage’ categorisation would be kept to a minimum​[15–17]​.

The main objectives of phase 2 were to use the refined triage tool in practice, and harness action research approaches to develop the tool into a ‘conclusive tool’ that could be tested for reliability in the final phase of the study. Given the barriers associated with implementing change in practice, it was prudent to consider developing any successive versions of the CPTT as part of routine practice within the ED environment by the ED pharmacy team​[41]​. Utilising action research methodology allowed the benefits of questionnaire approaches, including the ability to readily obtain relevant information into a systematic and cyclical process​[30]​. In practice, this approach reduced barriers and time constraints associated with developing the CPTT, and helped engage relevant staff in its subsequent development that was able to be shared with the wider pharmacy team and colleagues.

Initial findings from version 1 of the CPTT suggested that it could not be used to triage an individual patient, and this prompted an urgent review to ensure patient safety was not being compromised in applying the tool in practice. ED pharmacists were in agreement with the triage categories chosen as the outcome measures for the triage tool, and this was in keeping with the findings from other studies previously discussed in the literature ​[13,15–17,19]​. Other findings suggested that the tool content needed to be made more succinct in order to be used more easily at ward level, which was also found to be crucial in previous studies that examined the views of some junior pharmacists using the PAST​[19]​. Given the time and resource constraints, a limitation of phase 2 was that the total number of patients reviewed using each version of the tool was not captured. This decision was made to focus data collection on the qualitative feedback on the use of the CPTT itself, rather than focus on the quantitative nature of total numbers reviewed. However, on reflection, this data would have provided a baseline comparison when using the latest version of the CPTT prior to phase 3 testing.

The main objective of phase 3 was to determine a reliability measure of the definitive CPTT when used in practice by ED pharmacists. Using pharmacists in the ED meant that only four pharmacists could be included in this final phase of the study, which limits the reliability testing when compared with the evaluation of the PAST in Manchester, where the tool was tested across seven different wards​[13]​. However, a strength of this study is that overall reliability data were collected from a total of 60 different patients, compared with 35 patients in the previous study​[13]​. While the exact number was not recorded, the actual number of patients reviewed using the final version of the tool was greater than 60. Reliability was assessed in terms of intra-rater and inter-rater measures, providing a more detailed insight into the evaluation of the tool when used in practice by the team it was ultimately designed for. Previous literature had demonstrated the benefits of using a quasi-experimental evaluative method with a retest period of six months; however, given the timing and logistical limitations, this was not deemed feasible for this project​[13]​.

The test-retest method allowed the researchers to determine the extent to which individual pharmacist triage categorisation on day zero matched the triage categorisation after a two-week period​[36]​. This design differed to the quasi-experimental design used in previous studies, where expected patient acuity levels (PAL) were compared with the outcome PAL in terms of a difference in scores of agreement​[13]​. On reflection, a method linking outcome data with expected data gives more detailed insight into the use of the tool in practice, which is perhaps limited in the reporting of crude intra-rater reliability absolute agreement calculations. A more appropriate design to test for intra-rater measurement might have been to introduce a similar scoring system that demonstrates not only whether there is agreement, but how disagreements correspond to the tool when used in practice.

Unlike in intra-rater testing, participants were randomly assigned into two sets of pairs, where inter-rater comparisons were made between pair members only. Given further time, a crossover design would have been more appropriate. However, given the time pressures associated with the study and the demand on individual pharmacists to undertake triage from notes and drug charts away from ward level during inter-rater testing, this was not deemed feasible. Furthermore, given that the pairs of pharmacists examined the details of different patients, the ability to cross-compare the results from each pair with the other’s are also limited. While the pairings were assigned randomly, one could argue that this introduced bias into the results, as true inter-rater reliability was only determined for two individual pairs of pharmacists, not the whole cohort of ED pharmacists.

Cohen’s kappa results (κ=0.65 and κ=0.86) from the two pairs of ED pharmacists demonstrate ‘substantial’ and ‘almost perfect’ levels of agreement respectively (terms as referenced in the literature), illustrating the potential effect the CPTT could have in redesigning the way the triage is used in practice​[42]​. This contrasts with a previous study, where agreement was found to be ‘only slight’ (κ=0.344)​[13]​. The implications of these findings demonstrate that, with appropriate design and development of a CPTT, the ability to attain higher levels of reliability when used in practice is possible. Given the limitations in the results of the PAST, it was acknowledged that greater input is required in the design and development of triage tools in order for them to be used appropriately in NHS hospitals, potentially adding strength in the application of this CPTT locally​[19]​.

Limitations

There are some limitations with this study. The CPTT was developed for use within a specific hospital ED and therefore its use may not be generalisable to other settings without further review and local reliability testing. If more time was available, inter-rater testing between each of the ED pharmacists, rather than between two pairs of pharmacists, could have been undertaken, which would have contributed to further reliability testing.

The study did not explore the onward review of patients following triage. It would be prudent to consider whether these patients were being reviewed on the wards in the designated time periods as per the CPTT assessment, and to reflect on whether the appropriate prioritisation assessment was made in the ED. Furthermore, the total number of patients reviewed by the pharmacy team during the period was not calculated, which meant that the total number of patients reviewed using the CPTT could not be calculated.

Conclusion

This study identified the main factors important in the design of a CPTT for use by clinical pharmacists in the ED to assist with prioritising patient review. To establish the reliability of using the CPTT, intra-rater and inter-rater reliability measures were undertaken, and these illustrated the reliability of the tool when used by a team of ED pharmacists. Since completion of this project, the CPTT has continued to be rolled out beyond the ED pharmacy team, and is now used routinely by clinical pharmacy teams across the trust to assist with prioritising clinical workloads.

Further work

Further work to consider the impact the CPTT has on patient-orientated outcomes and any effect on length of stay is required to improve the strength in the application of the CPTT in practice. Further testing of reliability could be explored in alternative areas for increasing the generalisability of the CPTT application locally and more widely. Other additional areas of study planned include the rapid review of patients who have undergone triage and the potential de-escalation of triage status to help facilitate more prompt clinical pharmaceutical care without the requirement for a full review.