Using critical incident data to promote paediatric medication safety: a regional quality improvement initiative in Northern Ireland

Abstract

Background: Medication errors in children are a common cause of serious avoidable harm. Existing quality improvement (QI) measures, hampered by lack of evidence about errors’ characteristics and causes, have had limited impact. Critical incident data can provide this information but remain an underused resource in guiding QI.

Aim: To describe a programme of service evaluation and research, driven by mixed methods analysis of critical incident data, used to underpin paediatric medication safety QI in Northern Ireland (NI). Objectives were to:
(i) Analyse a large dataset of reported paediatric medication incidents in children, describing their characteristics and underlying causes, to provide an evidence base for QI;
(ii) Investigate how systematic analysis of aggregated critical incident data can inform improvement;
(iii) Draw attention to the need to improve paediatric medication safety in NI;
(iv) Demonstrate a transparent, learning focused approach to reported medication error.

Methods: Mixed methods analysis of reported medication errors in children in secondary care: quantitative analysis of error characteristics; and qualitative analysis of factors contributing to these errors; dissemination strategy comprising publication of regional report, concurrent research articles, and direct engagement of NI healthcare community; measures of success to include early evidence of impact on practice.

Results: Incident analysis included 1,522 errors. Quantitative analysis identified neonatal care, medication dosing and omission, and treatment with antimicrobials, paracetamol and vaccines as high-risk areas of practice. Qualitative analysis revealed that multiple interacting causes, including inadequate communication, distractions, and not using information sources, led to errors. This information fed into wide-reaching recommendations. Outputs included the proposed regional report, two research articles and multi-level stakeholder engagement. QI initiatives guided by findings; formation of a group to enact recommendations; and development of a business case to increase paediatric pharmacist provision represented indicators of impact.

Conclusion: This QI initiative shows how critical incident data can guide paediatric medication safety and provides a model that could be applied in other settings such as adult secondary care.

Original submitted: 2 October 2020; Revised submitted: 7 December 2020; Accepted for publication: 7 January 2021; Published online: 2 March 2021; doi: 10.1211/PJ.2021.1.28114

Introduction

A highly publicised report in 2018 estimated that 237 million medication errors occur annually at a cost of £98m to the NHS, causing the then UK health secretary to refer to “appalling levels of harm and death that are totally preventable”​[1,2]​. In 2017, the World Health Organization (WHO) made medication safety the focus of their third global patient safety challenge, ‘Medication without harm’, vowing to reduce severe, avoidable medication-related harm by 50% over a five-year period​[3]​.

Medication errors in children make up a significant proportion of the burden of avoidable harm: errors are particularly common in children and are more likely to result in harm than in adults​[4,5]​. Serious error types, such as tenfold dosing errors, disproportionately affect children, sometimes with lethal consequences​[6–8]​. Yet, despite increasing recognition of the problem, there has been a lack of improvement in paediatric medication safety to date, calling into question whether the WHO’s goal is achievable. Experts have cited a lack of specific, underpinning evidence — particularly information about the underlying causes of errors, without which solutions are unlikely to be effective — as a main reason​[9,10]​. This represents a problem in paediatrics because the specific nature of medication use in children limits direct translation of solutions from other areas of practice​[11]​.

Northern Ireland (NI), as with other areas, was tasked with addressing the challenge set by the WHO, which made specific reference to ‘high-risk situations’. Within patient safety, risk has been defined as the probability of an event’s occurrence multiplied by its severity​[12]​. For the reasons outlined, paediatric medication use therefore represents an important example of a high-risk situation. This added impetus to a pre-existing desire to promote paediatric medication safety within NI, as well as an opportunity to inform the regional response to the WHO challenge. In particular, this desire was precipitated by specific issues that had led to harm within the region (e.g. errors involving intravenous [IV] paracetamol and fluids)​[13]​. While specific examples of excellent medication safety improvements were noted, NI would benefit from a regional, evidence-based approach to paediatric medication safety — an approach that would help to address the wider issue within quality improvement (QI) that initiatives are often conducted locally and not based on underpinning evidence​[14,15]​.

These issues played out in the context of an influential report by Sir Liam Donaldson, which encouraged the NI healthcare community to learn from adverse incidents systemically and openly ​[16]​. Donaldson drew specific attention to the need to ‘make [critical] incident[s]… really count’, highlighting that they represent a significant pre-existing resource, of which the potential to support learning and improvement has not been realised​[16]​. As well as benefitting paediatric medication safety, this improvement-driven strategy would exemplify a transparent approach, as advocated by Donaldson. In contrast to the UK mainland’s UK National Reporting and Learning System (NRLS), there is not an equivalent national reporting system within NI. Notably, the NI Medicines Governance Team (MGT), a regional body established in 2002 and comprising pharmacists in roles equivalent to UK medication safety officers, collaborates to prevent and minimise medication-related harm, which includes collating and sharing learning from incidents​[17]​. Nevertheless, an important opportunity existed to extend the team’s work, and the use of critical incident data, by collaborating with academia to systematically collate and analyse incident data within a population group, at regional level, over an extended reporting period.

Aim

This article describes a programme of service evaluation and research, driven by mixed methods analysis of critical incident data, and how this was used to underpin QI in NI paediatric medication safety.

The objectives of this initiative were to:

• Analyse a large dataset of reported paediatric medication incidents in children, describing their characteristics and underlying causes to provide an evidence base for QI work;
• Investigate how systematic analysis of aggregated critical incident data can inform improvement;
• Draw attention to the need to improve paediatric medication safety in NI;
• Demonstrate a transparent, learning-focused approach to reported medication error.

The purpose of this article is to describe the programme of work and reflect on its methodology, process, impact and limitations, thereby offering a transferable approach to the use of critical incident data in QI that may be useful in other settings. The article structure is based on QI reporting underpinned by Standards for Quality Improvement Reporting Excellence guidelines; however, because this was preliminary improvement work, it does not describe elements such as improvement cycles​[18,19]​.

The article first describes a background review of existing literature on paediatric medication error and use of critical incident data conducted to inform the initiative. It then describes the methods used within the programme, with particular focus on the critical incident analysis, and provides results, summarising the critical incident analysis findings, and describing dissemination and indicators of impact. Finally, it reflects on transferable lessons from the NI experience, and limitations of the approach, with the aim of supporting others who wish to apply it elsewhere.

Background

Error characteristics

To underpin analysis of critical incidents, background literature was reviewed; first, studies using reported incidents to analyse characteristics of paediatric medication errors were explored. Many concerned specific areas of practice (e.g. neonatal intensive care unit settings​[20,21]​, emergency departments​[22]​), specific medications (e.g. vaccines​[23]​, anaesthesia​[24]​), or involved relatively small numbers of incidents​[25]​. Moreover, many studies used critical incident data to estimate error rates​[20,26]​, an approach that has been criticised owing to issues with underreporting and bias​[27]​. Nevertheless, several studies did provide information about characteristics of reported errors analogous to those that would be obtained from the NI analysis.

A 2009 NRLS report analysed more than 60,000 paediatric patient safety incidents, of which medication errors made up 17%​[28]​. Medication errors in children aged 0–4 years accounted for almost 10% of the total number of incidents reported in patients of any age, with incident numbers similar to those reported in the 75–79 years and 80–84 years age categories​[28]​. This implies that children aged 0–4 years are disproportionately involved in medication incidents — a finding that is difficult to explain by frequency of healthcare encounters or differential reporting rates. Dosing errors were the most common type of error, making up 23% of medication incidents in children, with medication omissions the second most frequent​[28]​. The authors of the report drew attention to errors involving preparation and administration of IV medications, and prescribing and administration of gentamicin​[28]​.

Shaw et al. identified 597 reported incidents involving medication across a network of paediatric emergency departments​[22]​. Wrong dose errors comprised 39% of incidents. Of these, 51% resulted from calculation errors and 12% were tenfold errors. Medications most commonly involved were anti-infective agents (25%), analgesics (21%) and IV fluids (21%).

Rees et al. analysed patient safety incidents occurring in sick children in primary care, including 674 medication errors​[29]​. Incidents were most commonly reported in children aged under one year and in conjunction with treatment for epilepsy, asthma and infections. However, most incidents (57%) were related to dispensing in community pharmacies, limiting the relevance of this study to this work, of which the scope was secondary care.

Together, existing evidence suggested that medication errors are among the most commonly reported incidents in children, frequently involve medication dosing, and disproportionately affect young children and neonates. The types of medication involved appear to reflect those often used in children, such as antimicrobials and analgesics. However, no large-scale analysis was identified that fully described characteristics of paediatric medication incidents (or used incident data to shed light on underlying causes). This, coupled with the fact that QI is best guided by local data, confirmed the potential value of the approach proposed.

Contributing factors to errors

Evidence about contributing factors, and underlying individual and systemic issues that shape practitioners’ actions, within medication errors in children was then reviewed​[30]​. Little primary research addressed this issue, with most available information stemming from review articles and expert opinion​[11]​. The majority of evidence concerned specialist paediatric settings, such as children’s wards and critical care units, with little focus on other areas of practice that often encounter children, such as community settings​[31]​. Authors frequently proposed specific issues that complicate medication use in children, such as the requirement to provide doses based on weight and the use of different medication formulations​[11]​. Limited evidence also suggested that the same sort of social, environmental and contextual factors that have been identified as causing errors in adults, such as distractions, interruptions and miscommunication, may apply in children​[32–34]​. This review underlined the need for further study into the underlying causes of errors. It also informed the choice of a theoretical framework (the Swiss cheese model) that enables identification of systemic causes of errors, and sensitised us to some of the factors that might be identified​[34]​. Findings of this review (in relation to prescribing errors) were later published​[11]​.

Critical incident methodology

Evidence was also sought to guide the use of critical incident data. Historically, concerns about validity of reported incidents have limited their use in QI​[27]​. These concerns centre around biased and incomplete reporting, undermining incidents’ representativeness​[35,36]​. More recently, however, authors have called for a data-driven approach to QI, supported by use of incident data​[37]​. They argue that the value of critical incident reports lies in their ability to identify safety threats and shed light on the underlying causes of errors​[27]​. These attributes align well with QI’s ultimate goal of promoting safe, high-quality care, rather than, for example, producing research-level data about rates of disease.

To achieve this goal, practitioners have often used large numbers of reports in aggregate, enabling identification of priority areas and yielding information about uncommon events​[27]​. They have also made use of narrative descriptions of events to shed light on errors’ underlying causes, an approach that has been applied to paediatric medication errors​[29,38]​. For example, Shaw et al. found that factors such as miscommunication, failure to comply with procedures and miscalculations contributed to errors​[22]​; Rees et al. found that issues with communication and documentation often led to errors in the administration of vaccines​[23]​.

This emerging body of evidence supported the decision to make use of incident data and shaped the choice of methodology. Specifically, it led to the selection of a ‘mixed methods’ approach, that quantitatively analysed the categorical variables (e.g. error type, medication involved) within incidents to establish their characteristics, then qualitatively analysed narrative descriptions to explore their underlying causes. It also led to focus on areas of risk within practice, rather than trying to estimate error rates or use numbers of reported incidents to measure change over time.

Methods

Design

Figure 1 provides an overview of the programme of work. A multi-professional team, with links to health and social care trusts (HSCs), academia and the NI regional QI body, sought NHS ethics and governance permission to conduct a mixed methods analysis of a large dataset of reported medication incidents occurring in children in secondary care. Quantitative analysis would enable identification of error characteristics, while qualitative analysis would reveal underlying contributing factors. Literature review and stakeholder engagement would contribute to validation of findings.

Planned dissemination included publication of a regional report, direct stakeholder engagement, parallel publication of research articles, and communication of findings to the NI healthcare and QI community through presentations at relevant conferences and meetings. The resulting evidence base could be used to support policy change and QI work; achievement of dissemination outputs, and early indicators of changes in policy and practice, constituted measures of success.

Context

NI has a population of around 1.9 million, including 400,000 children​[39]​. Healthcare is delivered as part of the UK NHS, and administrated by five HSC trusts under the governance of the HSC Board​[40]​. Children are cared for in a wide range of primary and secondary care settings. Secondary care settings include: a regional children’s hospital, several district general hospitals providing inpatient paediatric medical and surgical care, maternity services and neonatal units. Other local hospitals also provide ambulatory and outpatient services for children. Beyond these dedicated paediatric settings, children also receive care in general settings, such as emergency departments and specialist services that look after patients of all ages. The age at which children transfer to adult services varies from 14–16 years​[41]​.

Within NI, QI work is initiated locally or coordinated under the remit of a regional body, HSC QI​[42]​. Specific clinical networks (e.g. Neonatal Network Northern Ireland) coordinate and share improvement as part of their remit​[43]​. The NI MGT also facilitates QI through incident reporting, risk management and dissemination of learning​[17]​. Finally, QI is supported by strong links with the academic community, specifically via NI’s (at the time of study) single medical school​[44]​. Despite this wide-ranging activity, paediatric medication safety was not a regionally agreed priority area.

In NI, adverse events are reported electronically, using the Datix system (at the time of study, one trust used paper reports, which were subsequently inputted to Datix)​[45]​. Each HSC trust investigates incidents using local risk management procedures. As mentioned, the NI MGT supports local analysis of incidents and sharing of learning identified. For example, medication safety briefings, disseminated across the region, summarise salient incidents and recommended practice changes​[46]​.

Target audience, purpose and scope

The work was intended to provide information useful to both policymakers and frontline clinicians. Its purpose was to direct and support QI activities, to influence organisational and policy-level decisions (for example, around allocation of resources to enable QI), and to supply a granular level of detail directly useful to practitioners working towards safer medication use.

Based on the literature review, which suggested that children outside specialist paediatric settings may also be at risk of error, all children in secondary care were included within the scope. Critical incident data from primary care were not readily accessible and so this setting was excluded.

Quality improvement team

This initiative was carried out by a collaboration between the NI MGT and the Centre for Medical Education, Queen’s University Belfast, and in conjunction with the paediatric group of the regional QI body (now called HSC QI). The project was intended to complement and extend the NI MGT’s existing work in using reported incidents to support medication safety. Alongside the QI initiative, associated research activity contributed to Richard Conn’s PhD and Vincent McLarnon’s Master’s degree.

Critical incident analysis

An overview of the critical incident methodology is given here to support its transfer to other settings; further details are available within the report and research publications​[31,47,48]​.

Ethics and governance

As the incident analysis was intended to be used for purposes of both research and QI/service evaluation, a full NHS ethics and research governance application was submitted. Ethical approval was granted by the first available proportionate review committee, National Research Ethics Service Committee East Midlands – Nottingham 2, reference 15/EM/0353; all trusts in NI accepted the study protocol.

Data extraction and processing

Medicines governance pharmacists (MGPs) in each HSC trust obtained all reported incidents involving children aged 0–16 years within the date range 1 July 2011–31 July 2015 in all secondary care (hospital and community) settings. Accuracy of data is routinely checked after reporting; MGPs applied a second check during data extraction. Data were inserted into a Microsoft Excel spreadsheet and anonymised before transfer to the study team. An administrator held the pseudonyms applied to each HSC Trust, in case data specific to an individual unit were requested. 1,552 medication incidents were obtained; after initial review, 85 of these were deemed ineligible (based on: not relating to individual patients; adverse drug reactions without error; incidents occurring in primary care; incidents relating to medical devices) and removed. If a single incident mentioned more than one error that occurred at different stages of medication delivery (e.g. an error in both prescribing and administration), then it was duplicated and included in each applicable category. This led to a final dataset of 1,522 incidents for analysis.

Analytic approach

Incidents were quantitatively analysed using descriptive statistics, reporting stage of medication delivery, level of harm, where the incident occurred, medication involved, reporter job role etc. These findings formed part of a regional report and also formed the basis of a published article, which specifically focused on identification of areas of risk as QI priorities​[31]​.

Qualitative thematic analysis of narrative descriptions was then used to establish underlying themes in the occurrence of prescribing and administration incidents​[49]​. Owing to time and resource constraints, dispensing incidents were not analysed and the administration incident analysis was confined to the 369 incidents that occurred in hospital paediatric wards (excluding other settings, such as community care and children managed on non-paediatric wards). Narrative descriptions of incidents were reviewed and, within Microsoft Excel, potential contributing factors were coded. Codes were both inductive (factors identified from the errors described in the dataset) and deductive (pre-established factors suggested by our prior understanding of medication error and existing literature). Rigour was maintained during coding through reflexive discussions between team members with different backgrounds and perspectives, and member checking through stakeholder discussion and reference to previous literature. Coding decisions that team members felt were contentious were discussed and resolved by consensus.

Because initial literature searches suggested that underlying causes of errors were multifactorial and systemic, Reason’s Model of Human Error was used to guide analysis, which provided four overarching categories​[34]​. Latent conditions represented organisational processes and management decisions, such as those affecting staffing; error-producing conditions were ward-level issues, such as workload, and patient and task complexity; active failures were practitioners’ actions within errors, such as a lack of knowledge or forgetting; and defences were the barriers by which errors were intercepted​[50]​. The issues that were coded were grouped together into potential themes. As the analysis progressed, themes and sub-themes were refined and classified within the four broad categories suggested by Reason’s model. The final analysis was agreed upon by all authors and formed the basis of the published report​[47]​. Stakeholder discussion was conducted based on quantitative findings (to identify areas of risk) and, subsequently, full findings within a draft report​[31]​. Discussions were conducted informally (i.e. not recorded or transcribed) for resource reasons, and with the intention of mirroring a level of time commitment that might be reasonably expected of clinicians engaged in QI. Recommendations were agreed by the research team based on the quantitative and qualitative findings, and informed by the stakeholder discussion and reference to published literature.

Dissemination and measures of success

To maximise impact, multiple modes of dissemination were planned:

• A regionally published report based on the critical incident analysis, widely disseminated to key stakeholders, including both policymakers (including trust chief executives) and frontline clinicians;
• Direct engagement with stakeholders through attendance at meetings and working groups;
• Provision of support or data to existing or new QI initiatives;
• Engagement with the regional quality improvement body;
• Parallel academic outputs, including research publications and conference presentations.

As this programme of work was intended to provide underpinning evidence to support paediatric medication safety QI, measures of success were determined to be:

• Publication of the regional report; presentations of findings and engagement with stakeholders; academic publications and presentations;
• Preliminary evidence of changes in practice (e.g. formation of working groups, use of data to support new QI initiatives, material changes to strategy or policy);
• Further understanding of how critical incidents can better support medication safety and recommendations relating to their future use.

Results

Summary of critical incident analysis findings

This section summarises findings and recommendations from the critical incident analysis to demonstrate the insights that were gained from incident data.

Quantitative findings and high-risk aspects of practice

Overall, 1,522 incidents occurring between 2011 and 2015 in NI paediatric secondary care were analysed. Most occurred during prescribing and administration. The majority did not lead to harm, but reporters, guided by a regional risk matrix, judged that most had the potential to do so. Incidents were reported across the full range of ages and settings, but occurred most in neonates and infants, and on paediatric wards, neonatal units and in community paediatric settings. Full quantitative findings are available with the published report and an associated research article​[31,47]​.

Detailed quantitative findings identified high-risk aspects of medication use in children, based on their frequency of occurrence and the potential severity of resultant harm (published separately as a research article)​[31]​. These included neonates, particularly in intensive care settings, medications including antimicrobials, paracetamol, vaccines and IV fluids, and medication dosing and omissions. These aspects of practice were recommended as priorities for improvement efforts.

Qualitative findings/causes of errors

Table 1 summarises findings from qualitative analysis of contributing factors to prescribing and administration errors, based on in-depth analysis of 886 incidents (517 prescribing; 369 administration). Medication errors usually resulted from multiple interacting, situation-dependent causes, involving individual actions, workplace pressures, relationships with colleagues and the complexity of the task. While lack of knowledge contributed to errors, the knowledge required was often context-specific rather than the more generic knowledge taught in classroom settings. Errors were commonly reported during processes, such as medicines reconciliation, transcribing, and when patients moved from one clinical area to another. Features specific to medication use in children also caused errors: weight-based dosing, different medicine units and calculations. While defences prevented many errors from reaching patients, they worked unpredictably and relied heavily on the availability and alertness of staff. Together, these factors pointed to the complexity of medication use in children, and suggested that improvements would require further specific study and packages of solutions that work together to address different elements of the problem.

Summary of recommendations based on incident analysis

Based on the complex causes of medication errors identified, multi-level solutions were recommended, summarised in Table 2.

Dissemination and impact on practice

Findings were made available in a full report, published online and widely disseminated within HSC trusts​[47]​. The authors engaged with a wide range of stakeholders, both during and subsequent to the analysis. These included: paediatric teams within HSC trusts; drug and therapeutics committees; the regional medications safety body; the NI MGT; and the paediatric sub-group of the regional QI network. In parallel, two academic papers were published: a specific analysis of IV fluid prescribing errors and an analysis presenting a methodology for identifying high-risk areas of practice using quantitative data​[31,48]​. Findings were also presented at regional meetings, including Ulster Paediatric Society conferences, the NI Simulation and Human Factors Network and the inaugural NI Medicines Safety Conference.

Early evidence of impact was identified at different levels. The report was used to provide evidence to aid in development of a regional business case to support medication safety by increasing paediatric pharmacist provision. In emphasising the significance of medication safety issues in children, it also influenced NI’s response to the WHO challenge, which plans to empower young people to become ‘medication safety-wise’​[51]​. The regional children’s hospital formed a task and finish group to evaluate and implement the report’s recommendations. The regional QI body assembled a paediatric medication safety sub-group, in which members of the authorship team were represented. Following the work, it was agreed that paediatric medication safety would become a specific priority for the QI body’s paediatric group; however, subsequent restructuring and plans to develop a regional paediatric network have meant that further activity is postponed at present. Furthermore, direct support was given to ongoing QI activities: for example, an analysis of IV paracetamol prescribing and administration errors was conducted and shared with an existing QI team, and a specific data set of antimicrobial errors was shared with a specialist antimicrobial pharmacist.

Recommendations were made within the report about expanding the role of critical incident data to support medication safety and a transferable methodology relating to the use of incidents in guiding QI priorities was published​[31]​. Critical incident data were also used on a trial basis to support reflective discussions about causes of errors and potential approaches to safer practice within a multidisciplinary group of paediatric healthcare professionals. While the effect of our programme of work on the culture around learning from error was not directly measured, the strategy adopted, in openly publishing findings and using error to drive improvement, demonstrably promoted transparency.

Lessons and limitations

This programme of work, designed to guide and stimulate QI in NI paediatric medication safety, was based on an in-depth, mixed methods analysis of a large dataset of critical incidents occurring over a four-year period across an entire geographic region. This process went beyond the routine use of critical incident data in risk management by: incorporating clinical and academic stakeholders from a range of professions; using research-level systematicity in its analysis; emphasising investigation of causes and contributing factors, guided by a systemic framework; aggregating a large number of incidents and sharing findings at regional level; and reinforcing findings with stakeholder discussion and reference to published literature​[52]​. The use of regional-level analysis, in particular, facilitated engagement with policy-level stakeholders — an essential part of supporting the culture change that is needed to enable effective bottom-up improvement​[53]​. This approach provides a model for learning from incident data consistent with that advocated by NI’s recently published WHO ‘Medication without harm’ strategy​[51]​.

Lessons from this project can inform both future work within NI and that of others elsewhere. Firstly, a data-driven approach was used that placed a strong emphasis on defining the problem and identifying its underlying causes as a basis for improvement efforts. This aligns well with calls that QI should be firmly rooted in evidence and not “intuition and anecdotal accounts of successful strategies for changing provider behaviour”​[14]​. It is noted that recommendations did not arise directly from findings, but resulted from the authors’ interpretations alongside stakeholder input and insights from existing evidence. For example, findings that errors were frequently reported during medicines reconciliation and transitions of care occurred owing to prescribers’ difficulties with individualised dosing, and were frequently intercepted by pharmacists, indicated the potential benefit of increased clinical pharmacist provision and informed this recommendation. This was reinforced by stakeholder perceptions of need in this area, and systematic review evidence underlying their potential benefits​[54]​. Similarly, that dosing errors were frequently reported — often related to calculating doses based on weight — guided the recommendation that introduction of electronic prescribing should be expedited because this offers the potential to support prescribers and reduce errors​[55]​. While it can be argued that this process introduces subjectivity into recommendations, the in-depth, rigorous analysis and validation steps taken make this less likely to be the case than if recommendations were based on expert opinion alone, as has often been the case.

Undertaking this initiative offered insights into use of critical incident data within quality improvement. By focusing on their ability to provide authentic evidence directly drawn from — and relevant to — practice, concerns about their lack of representativeness could be offset. A method to identify QI targets from quantitative analysis of reported incidents was developed, which could be applied elsewhere​[31]​. Moreover, the feasibility of using narrative data to illuminate incidents’ underlying causes was established. While the amount of detail present varied greatly — ranging from 10 to 200 words — incidents were found to contain sufficient detail to enable meaningful analysis, as reflected in previous studies​[22,38,56]​. This process was facilitated by the authors being experienced clinicians with different professional backgrounds, with direct experience of the sort of incidents reported. Steps were taken to maintain rigour, as described above, although the potential for bias to affect qualitative analysis is nevertheless acknowledged.

Second, the critical incident analysis was supported by close engagement with the user community. Choosing an entire region for study enabled the formulation of a rich dataset to facilitate in-depth study, while also giving stakeholders a sense of ownership over incidents and the resultant findings. Engagement steps were taken both during analysis, as a means of informing the process, and afterwards, to disseminate findings and shape recommendations. Undertaking study to underpin QI within a community of practice will make practitioners more likely to pursue ongoing work based on its findings.

Third, by working with academic partners, additional expertise and resources were leveraged. This supported a rigorous, systematic approach to analysis and production of outputs — such as published research articles — that went beyond what is usual within local QI activities. This approach also encourages the sharing of findings and demonstrates a progressive approach to using error as a stimulus for improvement.

The work also had limitations. While critical incident data are informative, findings cannot be considered exhaustive. Other types of errors, and underlying causes of errors, may exist that are not brought to light by incident analysis. In particular, because reported errors do not often lead to harm, it has been argued that they may not represent the most important, potentially serious errors​[57]​. Moreover, recommendations were produced through consensus within the QI team, informed by, but not directly drawn from, critical incident findings. This is necessary when attempting to address a complex problem such as medication error but, nevertheless, introduces a subjective element to the process. Consulting with experienced stakeholders and reviewing existing literature can add validity; it is also recommended that incident analysis be used alongside other methods (e.g. drug chart review) in data collection.

The work set out to make paediatric medication safety become a greater priority and stimulate downstream QI initiatives. This meant that it was inappropriate to directly measure its success by, for example, evaluating error rates. Instead, evaluation focused on achieving widespread dissemination and preliminary evidence that the work influenced changes in practice. Outputs were widely shared and read, and resonated with stakeholders. Evidence also suggested that the work had direct effects on clinical practice; for example, that it was used to provide evidence in support of business cases to appoint new clinical pharmacists. However, it is recognised that underpinning work of this nature represents only the first part of the improvement process. Ensuring that its recommendations are carried forward into frontline QI initiatives is crucial and will require sustained effort and engagement. It is important that downstream initiatives such as these should be implemented using ‘Plan, D, Study, Act’ methodology and evaluated systematically to assess their effect on clinically important outcomes, including error rates​[58]​.

QI work of this nature demands significant time and resources. This limited the scope of the work, in which only prescribing and administration errors were qualitatively analysed. Primary care settings, in which children receive a significant proportion of their healthcare, were not included. It is also recognised that, while conducting QI and research in parallel may bring benefits, this may also present additional logistical challenges. For example, obtaining research ethics and governance approval can be an involved and time-consuming process. On the other hand, engaging with practitioners, who are completing research degrees, may enable quality improvers to work alongside motivated colleagues with dedicated time to pursue QI activity alongside their research objectives.

Conclusion

This regional QI initiative was based on a large-scale analysis of reported medication errors in children across a wide range of secondary care settings. It makes an important addition to emerging literature on how critical incident data can be used to inform QI, offering an approach to identify high-risk areas of practice, and further exploring the ability of incidents to shed light on the underlying causes of errors​[31]​. The methods are transferable to other contexts and could be used on an ongoing basis within healthcare organisations to support cyclical QI activity.

Reflecting on the initiative’s other objectives, evidence was identified to suggest that the work drew attention to the importance of paediatric medication safety and impacted practice. The goal of establishing paediatric medication safety as a strategic priority for the paediatric subset of NI’s regional QI body was also achieved, but it is recognised that, for capacity reasons, ongoing work is currently postponed. This reflects the fact that establishing an evidence base is necessary to underpin change, but that sustained effort is essential to carry this through to on-the-ground impact.

In conclusion, this regional QI initiative responds to a call to “make incident reports really count”, presenting a case study in how critical incident data can guide paediatric medication safety initiatives and providing a model that can be used to improve other aspects of medication safety​[16]​.