The impact of a medicines administration pharmacist on patient safety and medicines management: a service evaluation

This study considers the benefits of the introduction of a medicines administration pharmacist on an inpatient ward at St George’s University Hospitals NHS Foundation Trust.
Abstract
Introduction: As a strategy to address recruitment and retention of nursing staff and the need for wards to meet patient safety standards in medication management and administration, pharmacists — as medicines experts — were identified as healthcare professionals that could administer medicines to inpatients, alongside nurses. Full-time medicines administration pharmacists (MAPs) at St George’s University Hospitals NHS Foundation Trust, London, received training to administer medication and began work on inpatient wards. The impact of the introduction of MAPs was measured to determine the validity of this approach.
Methodology: Data relating to the administration of general and critical medication doses were collected between 1 November 2017 and 5 January 2018 from a six-bay older person’s care ward following introduction of the MAP role. The MAP was placed in one bay per day (the intervention group), to allow direct comparisons to be made with traditional nursing bays (the control group). All patients on the ward who had medication prescribed were eligible for data collection. Patients had the opportunity to opt out from inclusion in the study.
Results: Over a two-month study period, data were collected for 236 patient days (a patient day can be defined as one patient’s data collected for one working day) for the intervention group and 228 for the control group. More medications were administered in the intervention group versus the control group (n=1,374 vs. n=1,112, respectively). In addition, there were 84 missed doses in the control group (of which five were critical medicines) versus zero in the intervention group. In the control group, 90.8% (n=932) of medicines were given within a two-hour window of intended prescribing time, in comparison with 99.7% (n=1,281) in the intervention group. Additionally, it was shown that almost 94 hours of nursing time was released by implementing the MAP.
Discussion and conclusion: The MAP improved patient safety through medicines management and optimisation, providing medicines in a timely manner and eliminating missed doses. This expedited discharge planning and released nursing time for patient care. The embedded pharmacist seamlessly added medicines expertise to frontline care. Further discussion, exploration and development of this role are warranted. As the study was retrospective and relied solely on data collection from drug charts, it was difficult to know if the results are a true reflection of practice (e.g. a nurse may give a medication but not sign for it). Additionally, the study was limited to one ward with one type of setting; it may not be representative of other areas.

Key points

  • The medicines administration pharmacist (MAP) role was created to address recruitment and retention of nursing staff and the need for wards to meet patient safety standards in medication management and administration;
  • The day-to-day role of the MAP included not only administering medicines but medicines optimisation, governance work, discharge planning, and education and training;
  • Direct comparisons made between MAP and nursing staff over a period of two months showed benefits for patients and staff where medicines administration was concerned, with fewer missed doses and timelier dosing;
  • Further benefits were identified with a reduction in medication errors, reduced discharge times and hours of nursing time released to complete core nursing duties.

Introduction

A medicines administration pharmacist (MAP) is a pharmacist who has specific training in medicines administration, which is historically carried out by nurses. The MAP concept is a relatively new one and has not previously been trialled in this way within St George’s University Hospitals NHS Foundation Trust. Successes and limitations have been shown in paediatrics and administering oral medications to adults in rehabilitation, where improvements in discharge times were seen​[1]​. Until now, this programme had not been tested on an adult inpatient ward where oral and intravenous (IV) medicines are administered by a pharmacist.

St George’s University Hospitals NHS Foundation Trust is a large teaching hospital in south west London serving a population of 1.3 million people, providing around 650,000 outpatient appointments per year​[2]​. The trust has 1,083 inpatient beds, with around 135,000 patients being admitted in 2018/2019​[1]​.

In 2015, the authors observed that several wards at St George’s University Hospitals NHS Foundation Trust were struggling to meet trust patient safety standards in medication management.

At the same time, the trust was struggling to recruit and retain permanent nursing staff and therefore relied on temporary agency staff, at considerable cost. As medicines experts, pharmacists were identified by the trust as complementary healthcare professionals that could support the nursing workforce in administering oral and intravenous medicines and, because of their skills and knowledge, also contribute positively to improve patient care. 

In 2015, a pharmacist received standard trust nurse medication administration training to deliver IV and oral medication. The pharmacist worked full time on a paediatric surgery ward and recorded their day-to-day activities, including the time spent checking medication and facilitating discharge. The MAP made particular interventions on oral and IV doses/prescriptions (which may have gone unnoticed before administration) and improved staff presence at ward rounds and discharge planning meetings. Moreover, there was a reduction in the time patients waited for their medicines (unpublished in-house observations). With the pharmacist undertaking these activities, the nursing staff had more time to perform core duties, leading to improvements in patient safety and experience. Previous research had shown that similar schemes resulted in improved nursing and medical staff satisfaction​[3]​.

More wards, where patient safety relating to medicines management could be improved, were identified during 2015, 2016 and 2017. The MAP programme was rolled out in a flexible and bespoke way to meet the individual needs of each ward. The MAPs were deployed on wards that were using paper-based prescribing and discharge, and wards using electronic prescribing systems. During this time pressures — including increased nursing vacancies, subsequent spending on agency nurses and patient demand on the trust — continued to grow. In response to these pressures, the MAP role was subsequently rolled out across further paediatric wards in 2015 and 2016 and, later, an adult rehabilitation ward in 2017 (where only oral medication is administered).

This article describes an evaluation of the Mary Seattle Ward — a mixed-sex older person’s care ward within St George’s University Hospitals NHS Foundation Trust — which has 42 beds and where self-administration of medicines was not allowed. The ward had struggled to meet set standards in medicines management, administration and patient safety. This was partially attributed to the high reliance on agency nursing staff that had not received formal trust training (unpublished in-house observations). Furthermore, the ward was operating with substantial nursing vacancies and found it difficult to recruit and retain nurses. The MAP programme was seen as a potential way of combating this. A three-month pilot of the MAP programme was put in place, following the model used on other wards but allowing for flexibility and an adaptive approach. This differs from any other current MAP service as it will be the only ward where a MAP administers both oral and IV medication to adult patients, prompting the requirement for evaluation.

Aims

This service evaluation aimed to:

  1. Assess several patient safety standards in medicines management;
  2. Assess the number of nursing hours released for other duties through the implementation of a MAP.

Objectives and standards

The main objective was to collect a representative sample using data from patients on the intervention and control bays. By doing so, the following secondary objectives can be achieved:

  • Comparison of the number of missed doses between control and intervention, and to trust standards. The trust states that “all reasonable steps are taken to administer all prescribed medication”. This means 100% of medications should be given unless a valid reason is documented​[4]​. This is further analysed by splitting into critical (e.g. antibiotics, anticoagulants, anti-epileptics, insulin etc.) and non-critical medications (e.g. vitamins, mild pain relief etc.);
  • Comparison of ‘on-time’ dosing on each ward and to trust standards. To measure if the timing of a patient receiving a dose matches the intended administration time. This can be classed as early or late dosing. Trust standards for this currently state: “Most medicines should be administered plus or minus two hours from the time prescribed.” However, critical medications (e.g. insulins and antibiotics) must be administered “much closer to the prescribed time”. For the purposes of the service evaluation, the acceptable limit will be 30 minutes, allowing for some flexibility for the day-to-day demands on an NHS ward​[4]​. If a valid reason is documented, this is completely acceptable. Further analysed into:
    • Non-critical medications
      • Within two hours, between two and four hours, and over four hours from the intended administration time, and
    • Critical medications
      • Within 30 minutes, between 30 and 60 minutes, and over 60 minutes from the intended administration time​[4]​;
  • Quantifying the number of interventions made (e.g. incorrect dosing/prescribing/timing and checking errors);
  • Measuring the number of nursing hours saved by having a MAP in place (i.e. time spent carrying out shared duties to be recorded on a day-to-day basis by the MAP).

Method

Overview

Data were collected between 1 November 2017 and 5 January 2018, on the Mary Seacole ward, within St George’s University Hospitals NHS Foundation Trust. Information was collected on a purpose-made data collection and recording sheet, developed during a pilot audit (see below and Supplementary information). Data were extracted from inpatient drug charts, once a week.

All eligible intervention patients were audited and therefore no sample size calculation is necessary. As all bays on the ward are the same size, the intervention group’s number of patients was matched closely to the control group’s number of patients without manipulation.

Pilot audit and data collection tools

A data collection spreadsheet (see ‘Supplementary information: Data collection sheet 1′ [collected by author]) was produced in Microsoft Excel, which outlined the data fields required to measure performance against the aforementioned standards and to meet the aims. The MAP (AR) recorded data on interventions made and the number of medications administered when necessary (e.g. after each intervention or medication round). This was completed prospectively and reported back to the auditor (MH) on a weekly basis and the information was also included as part of a weekly report from the MAP to show service provision.

The spreadsheet and data collection technique was piloted in ten patients. Aesthetic amendments were made to allow more space to record the results and enable revisiting of patient charts (see Supplementary information: Data collection sheet 2 [collected by MAP]).

The plan for the initial pilot audit in October 2017 was to audit the whole of the Mary Seacole Ward, which contains six bays, where the MAP would be responsible for one bay from 07:30–15:20 on a daily basis, as decided by the head of nursing. This time period was chosen as this is when the majority of medications are given and also when the majority of medication-related problems arise. During the pilot, it was deemed inappropriate to use five bays for control data to one bay of intervention data, as this was not comparable. Therefore, data were only collected from a random control bay and the intervention bay for medication given between these hours, on the days the MAP worked. The control bay allowed for comparison and served as the baseline data. The bay that the MAP worked in was determined by the nursing staff and the control bay was picked at random, using a random number generator.

Inclusion and exclusion criteria

Patients that were not prescribed any medication and those explicitly opting out of the study were excluded. Rectal and vaginal preparations were not administered by the MAP, owing to the additional training needs required for their administration. All patients on the ward who had a medication prescribed were eligible for inclusion data collection.

Data collection and analysis

Data were obtained and collated once a week, collecting all new data from the previous week retrospectively. If a patient was discharged, their data up to the point of discharge were collected and any new patients had their data collected from the time of admission. The MAP fed back to the auditor regarding number of medications given and number of interventions made each week. It was originally planned that data would be collected for one month; however, because of illness and annual leave, this was extended to two months.

Once obtained, this data was scrutinised for anomalies by the authors, of which none were found. If necessary, it was possible to visit the patient’s previous drug charts, filling any gaps in information that had been accidentally omitted.

Ethical considerations

The confidentiality of both patients and participants was maintained throughout the study in line with trust standards and the General Pharmaceutical Council’s guidelines​[5]​.

Results

All patients (236 patient days) that had medicines administered by the MAP had data collected for a total of 34 days from their inpatient drug charts. During this time, the MAP was present on the ward for seven hours per day, five days a week (apart from during illness and annual leave — in which case the same strategy for providing nursing cover was used). Critical medications are defined by the trust and general medications are all other medications​[4]​. The usual pharmacy service was supplied to the ward throughout the evaluation and comprised of one band 7 pharmacist with no further support.

General and critical medication doses administered

Both cohorts had data collected for 34 days. The total number of patient days (collated number of patients who were administered medicines over the course of data collection — NB: the same patients were often used multiple times, on separate days) collected was 228 and 236 for the control and intervention group, respectively (see Table 1).

Table 1: Collated results totals for number of days, total patient days (patients administered to * days audited), general doses given and critical medication doses given

From the data obtained, there is a clear difference between the number of general medications given by the control and intervention group (n=1,026 vs. 1,285 respectively). The numbers of critical medications given was very similar between each cohort (n=86 vs. 89 respectively).

Missed doses

Table 2 shows that the MAP delivered 100% (n=1,374) of medications or provided valid reasons for not doing so; there were no missed doses. This meets the trust standard of 100% of all medications being given (unless a valid reason is recorded)​[4]​. The control group missed 79 general and 5 critical medications or did not provide valid reasons for medication omission. There are many reasons why a patient may intentionally have their medication withheld or not administered, such as the medication being unavailable, the patient refusing or a clinical reason. Missed doses were recorded when administration records did not have a signature or coded number in the appropriate box and was therefore identified as a potential training need for nursing staff.

Table 2: Total missed doses by each cohort

On-time doses

General medications

The intervention group administered 99.7% (n=1,281) of doses within two hours, compared with 90.8% (n=932) in the control group. The intervention group administered 100% of medications within four hours, compared with 98.5% for the control group. The control group administered 1.5% of medications four hours or more from the intended time (see Table 3).

Table 3: Timing of the administration of general medication doses, in relation to intended prescribed time

Critical medications

Table 4 presents the timings of critical medication administration. The intervention group administered 98.9% (n=88) of critical medications within 30 minutes and 100% within 60 minutes. This is in comparison with the control group, who administered 87.2% (n=75) of critical medications within 30 minutes and 89.5% (n=77) within 60 minutes. This means that 10.5% (n=9) of medications were given over 60 minutes outside of the prescribed intended time.

Table 4: Timing of the administration of critical medication doses, in relation to intended prescribed time

Nursing time saved/replaced by the medical administration pharmacist’s activity

An average of the time taken to complete these tasks allowed the number of tasks completed and the total time released to be calculated. The majority (58.4%) of the MAP’s time was spent administering medications. As shown in Table 5, several other tasks can be completed by the MAP, which further released valuable nursing time. The MAP spent 32.5% of their time doing patient’s own drug checks; 8.9% of their time making interventions; and 3.1% of their time checking discharge medications. Almost 94 hours of nursing time was released by the implementation of the MAP.

Table 5: Total number of tasks completed by the medical administration pharmacist and the total time taken to complete these nursing activities

Discussion

Total number of medications administered

In regard to the significant difference between the numbers of medications administered by both cohorts, analysis showed that the MAP was often placed in the bay with the heaviest workload relating to medicines administration. This relieved the nurse who was in the bay to complete nursing tasks that could not be shared with the MAP, such as pressure sore checking, feeding and administration duties. This may have been the nurse in charge (NIC; band 6/7) or a nurse performing duties solely for patients in the bay (band 5/6). If the nurse had not been relieved, core nursing duties may not have been completed. Therefore, the addition of the MAP to allow these tasks to be completed could be interpreted as a potential increase in patient safety. It is difficult to say if it would have been possible, otherwise, for a nurse to have completed all the medication administration safely and then thoroughly complete patient safety checks.

Missed doses

In the control group, a total of 1,196 medications should have been administered. This corresponds to 7% (n=84) of medications being missed or having no code for omission recorded, meaning 93% (n=1,112) of medications were administered or had a valid reason for their omission, missing the trust standard of 100%​[4]​. On further analysis, the missed doses were generally low-risk medications that are given frequently (e.g. paracetamol). On some occasions, the patient refused medication; however, the nurse failed to document or code this, hence a missed dose was recorded.

More concerning was the fact that five critical medications were missed in the control group during the study. All incidents were immediately brought to the attention of the nurse responsible and medical team, for patient safety reasons. On these occasions, a critical medication was missed and no acute adverse effects were noted; however, there is a potential risk that length of stay and patient satisfaction were adversely affected.

The MAP met the 100% standard set by the trust with regard to documentation and administration within the required time period​[4]​. This may be because the MAP can focus mostly on medication-related activities as per their expertise, contributing positively and potentially improving patient safety. It is important that medications are not missed for a variety of reasons, including increased length of stay and increased morbidity and mortality, with clinical and patient outcomes often decreasing or worsening as a result​[6]​. A practice educator involved in the programme commented: “Having the medicines administration pharmacist on the ward with us makes a real difference to patient care and patient safety.”

On-time doses

The MAP came very close to meeting these standards, being 0.3% and 1.1% from meeting the general and critical medication standards, respectively​[4]​. In the control group, around 10% (n=94) of general and critical medications were administered outside of the required timing. There was a clear improvement and movement towards the trust standards using a MAP​[4]​.

This improvement could be attributed to the fact that the MAP can provide additional focus on the administration of medication. Furthermore, the impact on patient safety here is clear (e.g. if an antibiotic is not given to a patient on time you run the risk of treatment failure, sepsis, resistance and potentially death). In addition, it is widely known that the time of administration of Parkinson’s disease medications are critical to prevent patients experiencing motor fluctuations — severe, uncomfortable and unnecessary lack of control — owing to an ill-timed dose​[7,8]​. In this instance, there were no acute untoward outcomes reported to the author. However, because of the retrospective nature of data collection, this cannot be confirmed, and this does not rule out a negative event in the future. The MAP was praised for timeliness and this could have attributed to an increase in patient satisfaction scores over this time period. Patient satisfaction scores were not collected and analysed as part of this study, but were routinely collected at discharge.

Nursing time released by the medical administration pharmacist

The ‘time released’ was calculated by observing the MAP in day-to-day work, for each of the tasks mentioned in Table 5.

This is important because of the challenge nursing staff face when completing tasks for all the patients they are responsible for, while very often being understaffed​[1]​. By releasing this time, nurses can focus on tasks that only a nurse is qualified and competent to complete. This has a benefit on patient safety, which is corroborated by the results found by this study.

On several occasions, the MAP was relied on to cover staff shortages and did so to good effect. There were several instances when the NIC was required to look after a bay of patients because of the team being short. When this occurred, the MAP stepped into the same bay and gave medications, sharing responsibilities within this bay, allowing the NIC to focus on the duties that can only be completed by a senior nurse. This allowed patient safety to be maintained to a high standard.

Interventions

The MAP made a total of 23 medication-related interventions during the study. These varied in the level of potential/actual harm. It is impossible to say if a nurse would have intervened in the same way, or if the medication would have been administered.

It is possible to say that patient safety was maintained at the very least and problems were rectified in a very timely manner, owing to the MAP having the skillset to amend prescriptions independently or after liaising with the medical team.

Further benefits

The nursing, pharmacy and medical teams have, overall, made positive comments on the pilot and implementation of the MAP.

Discharges were facilitated well, and the MAP stepped in on several occasions to ensure patients who would have had delays in discharge (owing to pharmacy and/or nursing) were discharged on time. One patient, while completing the discharge patient satisfaction questionnaire, commented: “To have someone take on board my concerns and really listen means a great deal to people in my position.”

The MAP designed and implemented an on-the-ward medicines management training and educational programme that was delivered to all nursing staff, with an opportunity for revalidation should mistakes be made. This covered dosing, calculations, common interactions and prescribing errors. The success of this was reflected in a reduction of DATIX incidents and a vast improvement in governance measures, such as the controlled drug audit and the safety of medicines storage and handling audit. DATIX incident reduction and controlled drug audit results could not be accessed retrospectively because of governance and accessibility issues.

Communication between pharmacy and the nursing team improved; this was mentioned several times by the nursing team, ad hoc.

Nursing staff sought the MAP for clarification of medication-related questions, improving patient safety. The MAP acted as a ‘safety net’ and was approachable at all times.

It became clear immediately that the added value of having a medicines expert seamlessly embedded in the core patient-facing team added value to the staffing structure and the quality of care patients received, alongside many other benefits that have not been measured during this study, such as job satisfaction and financial savings.

It is prudent to note that there is a risk of de-skilling nurses where medicines administration is concerned. It is also difficult to plan for MAP absence, particularly when it is unplanned, and this could negatively affect the skill mix within the ward.

There is also a potential risk of a MAP leading a bay during an unforeseen event (e.g. cardiac arrest, collapse etc.); however, training was given to combat this and would be no different to a healthcare assistant overseeing a bay. There is also the option to train a MAP in emergency care, equivalent to a nurse.

Recommendations and future work

In April 2018, results were fed back to the senior management team on the ward and the directorate. Senior management in pharmacy were continually updated and offered support. The trust’s medicines optimisation committee requested an overview of the findings and results. The following recommendations were made:

  1. Continue the MAP programme on the Mary Seacole Ward, supported by the results found throughout this study;
  2. Continue to roll out the MAP programme across the trust in areas identified with poor medication safety and/or high usage of agency nurses​[9]​;
  3. Continue to offer high levels of support from pharmacy management to the wards with a MAP; as this programme is new, problems may arise that need someone with authority and experience to tackle them;
  4. Continually adapt the service provided by the MAP, to meet changing needs. Explore shift patterns and the viability of the MAP during these different shifts;
  5. Re-audit the area in one year.

As of August 2021, the recommendations have not yet been undertaken. This is owing to recent changes in workforce skill mix and other pressing priorities, such as the COVID-19 pandemic. There is continued work to develop the MAP role in the trust.

The following were identified as areas that would benefit from further research:

  • Investigate deeper patient safety/wellbeing impacts, such as mortality/morbidity, length of stay and polypharmacy;
  • Investigate the financial impact of the MAP, including cost savings made in terms of medicines optimisation and potential reduced use of bank/agency staff;
  • Measure the significance of the MAP on core nursing duties, such as pressure sore checks/early warning scoring and see if this has an effect on patient safety;
  • Assess near misses, which are incidents where a medication error almost occurs (for example, this can be a missed dose that is remembered or a double dose that is almost given). A knowledge base could be built, and training could be designed to help tackle these issues, should patterns become apparent;
  • Investigate the potential risk of de-skilling nurses.

Limitations

As the study was retrospective and relied solely on data collection from drug charts, it was difficult to know if the results are a true reflection of practice (e.g. a nurse may give a medication but not sign for it, resulting in a false missed dose).

The study was limited to one ward, one specific MAP with one type of setting; it may not be representative of each area with a MAP. The small sample size means further research is needed to predict how the programme could be expanded to a whole ward, or the whole hospital. Additionally, missed doses and dose timings were not compared to other similar areas of the trust and comparisons of intervention numbers with traditional ward-based services were not made.

As the nurses’ banding was not recorded, it was not possible to assess the financial impact of the MAP.

There was no night shift cover; therefore, data were not collected at this time and this may be an interesting area of future research (as mentioned in recommendations). There is scope for a MAP to follow a nurse’s shift pattern, but currently this is not the case and would require proper planning to implement.

Ward pharmacist duties on the same ward were not timed for the control group. Therefore, we do not have an accurate idea of how these tasks were affected. As the MAP was often placed in the bay with the heaviest medication workload, it was thought this would not be a true reflection of a normal nurse workload. I would suggest analysis in this area be added to further work.

Supplementary information: Data collection sheet 1 (collected by author)
Supplementary information: Data collection sheet 2 (collected by MAP)

Financial and conflicts of interest disclosure

The authors have no relevant affiliations or financial involvement with any organisation or entity with a financial interest in or financial conflict with the subject matter or materials discussed in this manuscript. No writing assistance was used in the production of this manuscript.

  1. 1
    Hospital Admitted Patient Care Activity 2018-19. NHS Digital. 2019.https://digital.nhs.uk/data-and-information/publications/statistical/hospital-admitted-patient-care-activity/2018-19 (accessed Sep 2021).
  2. 2
    Annual Report and Accounts 2019/20. St George’s University Hospitals NHS Foundation Trust. 2020.https://www.stgeorges.nhs.uk/wp-content/uploads/2020/09/STGEORGE-ANNUAL-REPORT-2020-internal-printing-compressed.pdf (accessed Sep 2021).
  3. 3
    South West London Academic, Health and Social Care System. Ward Embedded Pharmacist Relieves Pressure on Nurses. 2017.https://healthinnovationnetwork.com/volume-2-issue-4-ward-embedded-pharmacist-relieves-pressure-on-nurses (accessed Sep 2021).
  4. 4
    Spiteri K. Medicines Management Policy, 7th edition. London: St George’s University Hospitals NHS Foundation Trust 2016.
  5. 5
    Standards for Pharmacy Professionals. General Pharmaceutical Council. 2017.https://www.pharmacyregulation.org/sites/default/files/standards_for_pharmacy_professionals_may_2017_0.pdf (accessed Sep 2021).
  6. 6
    O’Grady I, Gerrett D. Minimising harm from missed drug doses. Nursing Times 2015;111:12–5.https://www.nursingtimes.net/clinical-archive/medicine-management/minimising-harm-from-missed-drug-doses-26-10-2015/ (accessed Sep 2021).
  7. 7
    Barnes J. Parkinson’s disease: management and guidance. Clinical Pharmacist 2018;10.https://pharmaceutical-journal.com/article/ld/parkinsons-disease-management-and-guidance
  8. 8
    Nawaz M, Close O. Considerations for the inpatient management of Parkinson’s disease. The Pharmaceutical Journal 2020;304:N/A.https://pharmaceutical-journal.com/article/ld/considerations-for-the-inpatient-management-of-parkinsons-disease
  9. 9
    Bae S-H, Mark B, Fried B. Use of temporary nurses and nurse and patient safety outcomes in acute care hospital units. Health Care Management Review 2010;35:333–44. doi:10.1097/hmr.0b013e3181dac01c

Last updated
Citation
The Pharmaceutical Journal, PJ, September 2021, Vol 307, No 7953;307(7953)::DOI:10.1211/PJ.2021.1.102600
Peer reviewed article
This article has been peer reviewed by relevant subject experts prior to acceptance for publication. The reviewers declared no relevant affiliations or financial involvement with any organisation or entity with a financial involvement with any organisation or entity with a financial interest in or in financial conflict with the subject matter or materials discussed in this article.