Abstract
Aim
To ascertain what medicines managers in English primary care trusts were doing to promote antimicrobial stewardship.
Design
Online survey
Subjects and setting
All medicines managers of PCTs in England.
Results
108 medicines managers representing 79% of PCTs responded. 54% of PCTs had an antimicrobial committee, 61% had an antimicrobial audit strategy, 29% had an antimicrobial training strategy and 23% had a substantive primary care antimicrobial prescribing adviser.
All PCTs had evidence-based antimicrobial community guidance suggesting choice, dose and duration; 73% gave guidance on delayed prescribing. 95% of PCTs reported antimicrobial use back to practices and 32% of practices undertook annual audit. 90 PCTs used the HPA guidance to develop their local guidance; but only 6 respondents reported the web-based HPA guidance was easy to find. 63 referred to NICE, 57 Clinical Knowledge Summaries and 52 NPC resources to develop their guidance, but they were reportedly less useful than HPA guidance.
Conclusion
These results are encouraging, as all PCTs have guidance. There is an opportunity to improve antimicrobial use in primary care, as 73% of PCTs do not have an antimicrobial training strategy for community clinicians and a third have no defined audit strategy. As medicines managers valued resources that were national, comprehensive, evidence-based, concise, easy to navigate and read; these characteristics should be a feature in any resources. Short, concise guidance should be supported, as most PCTs use a national template to develop their local guidance. There should be easy access to national templates.
Antimicrobial use and resistance in primary healthcare in England continues to rise despite ongoing public campaigns for the past 10 years.1,2 There is a greater than two-fold risk of isolation of antimicrobial-resistant organisms from an individual if that individual had been prescribed antimicrobials within the past two months,3–5 and reducing community antimicrobial use has, in some instances, reduced resistance rates.6,7
The English Standing Medical Advisory Committee’s (SMAC) 1998 report “The path of least resistance” found that community antimicrobial guidelines were highly variable.9 SMAC suggested that guidelines should be based upon local microbiological and epidemiological advice and should take their cue from national guidelines. A 1999 questionnaire survey of GPs confirmed the need for antimicrobial guidelines.9 National sources of community guidance that cover the use of antimicrobials include the Health Protection Agency Primary Care Unit web-based management of infection guide,10,11
National Institute for Health and Clinical Excellence guidance (limited to acute respiratory tract infections), Clinical Knowledge Summaries,12 National Prescribing Centre resources (which are a primary resource for medicines managers), the Map of Medicine (www.mapofmedicine. com) and Infection Specialist Library (www.library.nhs.uk/guidance).
Although the “Atlas of variation in healthcare” in 201013 showed a 15-fold variation in cephalosporin usage across England,14 we do not have detailed information on how antimicrobial stewardship is being encouraged in the community. In England, GPs provide general medical services to all of the 50 million resident population from centralised local practices.13 These practices are currently administered by primary care trusts. PCTs in England employ pharmacists in the role of community medicines managers, with responsibility for surveillance of trends in GP and non-medical prescribing and promotion of cost-effective, high-quality prescribing through academic detailing and feedback of prescribing data.
The aim of our survey was to determine how community medicines management teams were facilitating antimicrobial stewardship in the general practices they served and what resources they used to develop their local antimicrobial guidance. The information obtained should inform commissioners and providers of primary care services how they can improve provision of antimicrobial guidance and facilitate stewardship in primary care.
Methods
Questionnaire development
The questionnaire was modified from an English hospital antimicrobial stewardship audit tool15 — by a multidisciplinary team including a GP, pharmacists and microbiologists — and piloted. The questionnaire asked about antimicrobial stewardship in the PCT, including: number of staff involved in antimicrobial stewardship, presence of an antimicrobial committee, antimicrobial audits, antimicrobial education strategies, how antimicrobials guidance was developed, and the usefulness, strengths and weakness of the national website resources listed above.
Participants
We obtained a list of all PCTs in England in 2010 from the NHS PCT website. Each PCT was telephoned to ask for the contact details of the individual responsible for antimicrobial stewardship. An identified individual responsible for antimicrobial stewardship in each English PCTs was emailed an invitation to participate in the survey, with a web link to the questionnaire on SurveyMonkey,16 in July 2010. Contact details of non-responders were checked and potential participants were sent up to two email reminders.
Ethics
Since this survey was considered to be a service evaluation of currently available antimicrobial guidance and antimicrobial stewardship in the community, ethical approval was not required.
Data analysis
SurveyMonkey data were imported into an Excel spreadsheet. We counted responses for each of the categorical questions, determining the percentage of PCTs answering for each response when the question asked about a service or behaviour in the PCT. For questions that asked about a respondent’s personal behavior, we determined the number and percentage of respondents that answered each question. For the open-ended questions, two researchers agreed categorical responses. Descriptive statistics were calculated as simple tabulations of frequencies and percentages.
Results
A total of 126 questionnaires were completed online. Several respondents completed the survey for more than one PCT, or we received more than one response from a single PCT; thus we analysed 108 questionnaires, representing antimicrobial stewardship in 116 PCTs (79 per cent). Some respondents did not answer every question and therefore the number of respondents and PCTs in each question may vary.
Antimicrobial guidance
Almost all respondents were involved in the development or review of their local PCT antimicrobial guidance (109/116 PCTs, 94 per cent). Two respondents were also antimicrobial prescribers. All responding PCTs (110) reported that they had peer-reviewed, evidence-based guidelines available for the treatment of common infections in the community. Most PCTs reported that they updated guidance every two years (61/106, 58 per cent), 35 (33 per cent) annually and 4 (4 per cent) every three years. However only 38 respondents (36 per cent) reported that their guidance was dated.
All respondents indicated that their PCT guidance stated the antimicrobial choice and duration. One set of guidance was reported not to state antimicrobial dose, 73 per cent (80/110) reported providing guidance on delayed prescribing strategies and 61 per cent (67/109) reported that local microbiology laboratories used selective antimicrobial susceptibility reporting on reports to GPs in line with the antimicrobial guidance.
An antimicrobial committee or equivalent was present in 54 per cent of PCTs (60/112) and 22.5 per cent (25/111) had a substantive prescribing adviser in place. In 13 per cent of PCTs (14/102), staff spent more than 0.4 whole time equivalents on antimicrobial stewardship; 75 per cent (83/110) spent less than 0.4 WTE and 12 per cent (13/102) did not know how much time was spent on this activity. Respondents reported that in 72 per cent of PCTs (78/109) all primary care prescribing advisers received printed information about antimicrobial prescribing and guidelines at their induction, but only 21 per cent of respondents (23/108) reported that over 80 per cent of medicines managers received specific training on safe and effective antimicrobial prescribing.
An antimicrobial training or education strategy was present in 29 per cent of PCTs (32/111). Four PCTs commented that their educational sessions covered delayed prescribing and three PCTs had developed delayed prescribing leaflets for clinicians to use within the patient consultation. Antimicrobial consumption was reported back to practices by 95.5 per cent of PCTs (107/112), with 88 per cent (99/112) monitoring practice antimicrobial use more often than annually.
An antimicrobial audit strategy was in place in 61 per cent of PCTs (68/111) and 57 per cent (64/112) reported that general practices took part in antimicrobial audits, 32 per cent (36/112) at least annually. When reporting antimicrobial use back to practices 91 per cent of PCTs (102/112) compared practices’ prescribing to national data, and 77 per cent (86/112) compared both antimicrobial prescribing in similar local general practices to each other and prescribing of different antimicrobial groups. Eight respondents commented that their PCT monitored use of cephalosporins, quinolones and co-amoxiclav, six PCTs concentrated their activities in high prescribing practices and six PCTs reported that they had local antimicrobial incentive schemes or linked audits to the Quality and Outcomes Framework (QOF) 6 target.
Development of local guidance
Of 96 responding PCTs, 22 (23 per cent) indicated that the PCT developed their antimicrobial guidance “from scratch” themselves; 73 (76 per cent) used a national template, mostly the HPA guidance (70, 73 per cent); and 30 (31 per cent) modified other community antimicrobial guidance. Of the total 116 PCTs, 86 (74 per cent) reported that they provided antimicrobial guidance electronically, either on the intranet (74, 64 per cent) or internet (30, 26 per cent); and 73 (63 per cent) provided it as a hard copy A4 format (44, 38 per cent), A5 book or spiral (24, 21 per cent), desktop version (12, 10 per cent) and 14 (12 per cent) as a small card, and 70 (60 per cent) provided guidance in both electronic and hard copy formats.
Some 83 per cent of respondents (90/108) reported that they were aware of the HPA antimicrobial guidance for primary care and 90 of the 116 PCTs (78 per cent) reported using it to develop their PCT guidance. Fifty-seven PCTs (49 per cent) reported using CKS, 63 (54 per cent) used NICE, 52 (45 per cent) used the National Prescribing Centre, five (4 per cent) the Infection Specialist Library and four (3 per cent) used the Map of Medicine to develop their own PCT antimicrobial guidance, but respondents found them less useful than the HPA guidance (Table 1).
Of the 90 PCTs that reported using the HPA guidance to develop local guidance, 21 (23 per cent) changed it very little and only added local contact details, 53 (59 per cent) changed some antimicrobial recommendations, 43 (48 per cent) added extra sections, and 18 (20 per cent) removed sections. Almost half of respondents (41, 46 per cent) used the web links, references and rationale sections of the HPA guidance. Fourteen respondents (13 per cent) specifically commented that they used the references to support changing recommendations and in training sessions. Half the respondents (40/80, 50 per cent) had difficulty finding the web-based HPA guidance; only six respondents reported finding it very easily. The greatest reported strengths of the HPA guidance were that it was authoritative, peer-reviewed and evidence based with references. The majority reported that the HPA guidance format was clear and concise; the ability to adapt the guidance for local use was specifically mentioned as a strength. The most common conditions respondents suggested could be added were dental infection, acne and breast infections.
The strengths of CKS most commonly reported by respondents were the in-depth information included, its strong evidence base, relevance to general practitioners, coverage of most infections seen in primary care, the easy accessibility and the fact that it was very clear. Many respondents reported that CKS was difficult to navigate. Thirty respondents (28 per cent) commented on the strengths of the NPC guidance; those most commonly reported were that it was national, evidence-based and respected, very comprehensive, containing a range of resources, and was easy to use. Only 13 (12 per cent) reported weaknesses, which included the limited number of infections covered, the fact that it was not known by prescribers and that at times there was too much information.
NICE’s greatest strengths were that it was national evidence-based guidance that covered when to treat respiratory tract infections, including the use of delayed antimicrobials. The NICE quick respiratory tract infections guide was considered useful; however, the full NICE guidance was considered too long and open to several interpretations, and did not cover specific antimicrobial recommendations and doses. Several thought the NICE guidance was not reviewed often enough and the website was difficult to navigate.
Only three (3 per cent) and one (1 per cent) respondents, respectively, commented on the strengths and weaknesses of the Infection Specialist Library and Map of Medicine.
Discussion
Our results are encouraging. This survey of over 80 per cent of PCT medicines managers in England showed that all responding PCTs had antimicrobial guidance for their primary care practitioners and 73 per cent gave advice on delayed prescribing. Antimicrobial prescribing was reported back to general practices in 95.5 per cent of PCTs with 57 per cent of practices taking part in antimicrobial audits. There is still room for improvement in community antimicrobial stewardship in England since only 54 per cent of PCTs had an antimicrobial committee and less than a quarter had a substantive primary care antimicrobial prescribing adviser. Furthermore, 75 per cent of PCTs spent less than 0.4 WTE staff time on antimicrobial stewardship activities.
Although antimicrobial use was fed back to GPs annually in 88 per cent of PCTs, 43 per cent reported that general practices did not take part in annual antimicrobial audits and 39 per cent did not have an antimicrobial audit strategy; furthermore, 71 per cent of PCTs did not have an antimicrobial education strategy.
Characteristics of antimicrobial resources that were considered strengths were that they were national, well respected sources that were comprehensive, evidence-based, concise, and easy to navigate and read. Weaknesses were that some were too long, difficult to find or navigate or not updated frequently. About half of PCTs used NICE, CKS and the NPC to facilitate development of their local guidance and most used the short and concise HPA guidance, finding it the most useful resource, with two-thirds finding it very useful.
Strengths and weaknesses
The near 80 per cent questionnaire completion rate indicates that this is likely to be an accurate reflection of PCT antimicrobial stewardship in England. The 20 per cent failure to respond may reflect a lack of dedicated antimicrobial stewardship resources in these PCTs. We asked respondents to comment on the strengths and weaknesses of the national resources available and responses give valuable insight to resource providers and commissioners about the factors that strengthen and weaken resources. However, few respondents made any comments on the value, strengths and weaknesses of the Map of Medicine or Infection Specialist Library. This may reflect that they did not use these resources and we could infer that they are not considered useful, or may reflect a lack of knowledge of these resources. This survey reflects antimicrobial stewardship within PCTs during 2010 and although we would hope that these activities will continue, we cannot be certain that they will reflect activity within the GP commissioning framework that will come into play in England through 2012 and 2013. However, it does demonstrate the value of the resources and current service provision that they may wish to at least replicate. This study was not designed to assess how effectively the antimicrobial stewardship initiatives were implemented and received by their target audience, clinicians. It would be useful to follow up this research with a questionnaire to GPs about local and national resources, what works well and how they can be incorporated into the workflow of a busy GP.
Where this work fits into current knowledge
Although all PCTs reported having community antimicrobial guidance, this has not led to a reduction in total antimicrobial items prescribed. Prescribing analysis and cost tabulation (PACT) and IMS Health Mediplus data show a 10–20 per cent increase in overall prescribing of systemic antimicrobials between 2000 and 2010.17,18 Although prescribing of cephalosporins and quinolones has decreased over the past two years, total prescribing, and that of amoxicillin and macrolides, continues to increase.13,17 This suggests that guidance and feedback alone will not lead to a reduction in overall prescribing volume.
In the past, medicines managers may have encouraged guideline adherence with regard to choice of agent rather than reducing the overall burden of prescribing, which is traditionally considered a medical/diagnostic issue. The NICE guidance suggests particular patient groups, with signs that should be considered for immediate antimicrobial treatment, and that clinicians should discuss the rationale for prescribing choices and the likely duration of symptoms, with and without antimicrobial treatment.19 We suggest these communication strategies20,21 should be included in any education programme for antibiotic prescribers in primary care.
In response to the Atlas of Variation publication showing the great variation in prescribing, the NPC recommended that PCTs should review and, where appropriate, revise current prescribing practice, and use implementation techniques to ensure prescribing meets Health Protection Agency guidance.13,22–24 When clinicians were interviewed following a complex intervention to improve antimicrobial prescribing they reported that frequent, brief reminders to be careful about antimicrobial use and patient education materials, specifically brochures and office posters, were the most useful parts of the intervention.25 The patient materials saved time, stimulated discussion with parents and decreased pressure to prescribe.25
Feeding back data without an audit strategy, as was the case in 75 per cent of PCTs, is likely to have less effect than using indicators within a structured audit strategy. Coenen et al have proposed 12 indicators of prudent antimicrobial prescribing for primary care that could be used in future general practice audits.26 They considered yearly consumption of all antimicrobials for systemic use to be the most useful measure of the pressure driving antimicrobial resistance.26 Other indicators, for example, a low use of penicillins would represent good practice — but not if it were accompanied by a high or marked seasonal use of cephalosporins or quinolones, which suggest higher use for respiratory tract infections, which has a poor evidence base.26 The ESAC group has also developed evidence-based disease-specific quality indicators for acute uncomplicated respiratory tract infections.27
It would be useful to establish whether the antimicrobial prescribing in those PCTs with an educational strategy or audit in place for several years have changed in comparison with their counterparts without such a strategy in place.
Several of the medicines managers in our survey reported that they had developed prompts for the GPs’ computer screens to remind them about prudent antimicrobial prescribing. With the pivotal role played by computers in primary care consultations, computerised decision support or reminders may provide a useful additional strategy to improve antimicrobial prescribing. Computerised decision support systems have led to an improvement in antimicrobial prescribing practices for community-acquired pneumonia in the hospital emergency medical unit setting, over and above that obtained with academic detailing alone.28
We know of no previous published process evaluation of antimicrobial guidance within the national resources about which we questioned the medicines managers. NICE produces an implementation strategy for each set of guidance and specifies implementation reports, which measure the uptake of specific recommendations taken from selected pieces of NICE guidance; however there are no reports on the implementation or value of the respiratory tract infections guidance.29
Implications for resource developers
This survey shows that the national HPA, NICE, NPC and CKS resources are greatly valued, save duplication of effort and promote evidence-based antibiotic prescribing. It is essential that central funding for these resources continues. As medicines managers valued resources that were national, comprehensive, evidence-based, concise, and easy to navigate and read, these features should be considered in any future resources. Resource developers should address the weaknesses highlighted by medicines managers, including considering the length of resources, ease of finding the resources and navigation within the sites; furthermore, regular updates are appreciated by users. Current guidelines do not specifically aim to reduce use of antimicrobials; resource developers should include more information (and evidence) on recognising cases where antimicrobials can be safely withheld or prescribing delayed. When guidance is long, resource developers should provide a “quick reference guide” with every guideline, to allow quick consultation, with web-links to the full guideline for rationale and evidence. Across Europe Antibiotic Awareness Day facilitates antibiotic awareness and promotes European educational materials for clinicians and the public.30 However a European primary care antibiotic template is not available and would be a useful addition to the EU materials.
Implications for primary care commissioners and medicines managers
Primary care commissioners should be reassured that antimicrobial guidance was available to all primary care clinicians; this provision should be continued and backed up with an antimicrobial audit and educational strategy to improve antimicrobial stewardship. A collaboration of the primary care, infection and pharmacy professional groups called Antimicrobial Stewardship in Primary Care (ASPIC) is working to develop a strategy and toolkit to improve antimicrobial stewardship within primary care and the Royal College of General Practitioners in England has highlighted antimicrobial prescribing as a clinical priority. ASPIC is lobbying the Care Quality Commission to develop and publish indicators of appropriate antimicrobial use that will highlight the importance of antimicrobial resistance and encourage prudent antimicrobial use in primary care through an antimicrobial prescribing strategy in all primary care commissioning groups.
Acknowledgements
Thanks to the medicines managers who completed the survey; to the BSAC, which facilitated the survey; to Kim Turner, who encouraged survey completion and helped with development of the data tables; and to Jill Whiting and Gemma Lasseter, who gave ongoing support throughout.
Declarations
The Health Protection Agency Primary Care Unit did not receive specific funding for this survey. Staff time to compile the survey and analyse the results was provided by the HPA and the British Society for Antimicrobial Chemotherapy.
Potential conflicts of interest
Cliodna McNulty leads the writing and review of the HPA “Management of infection guide for primary care” and is a member of the Department of Health Advisory Committee on Antimicrobial Resistance and Healthcare-Associated Infections and the prescribing subgroup, and chairman of the public education subgroups of this Committee She was also a member of the NICE self-limiting respiratory tract infection guideline development group.
Kieran Hand is a member of council and a director of the British Society for Antimicrobial Chemotherapy.
Matthew Dryden has accepted honoraria for speaking at pharmaceutical industry sponsored symposia and has been on advisory boards for the development and clinical use of a number of antibiotics.
Jonathan Cooke is chairman of the antimicrobial stewardship subgroup of the Department of Health’s Advisory Committee on Antimicrobial Resistance and Healthcare Associated Infection. He is a member of the MHRA’s Regulation of Medicines Review Panel and is external examiner for the MSc in infection management for pharmacists at Imperial College, London.
This paper was accepted for publication on 30 March 2012.
About the authors
Cliodna A. M. McNulty, FRCPath, is head of the primary care unit at the Health Protection Agency.
Tracey Guise is chief executive officer at the British Society for Antimicrobial Therapy.
Kieran Hand, PhD, MRPharmS, is consultant pharmacist, anti-infectives, at University Hospital Southampton NHS Foundation Trust.
Philip Howard, BPharm, MRPharmS, is consultant antimicrobial pharmacist at Leeds General Infirmary.
Matthew S. Dryden, FRCPath, is director of infection prevention and control at the Royal Hampshire County Hospital, Winchester.
Jonathan Cooke, PhD, MRPharmS, is honorary professor at the School of Pharmacy and Pharmaceutical Sciences, University of Manchester.
Correspondence to: Cliodna McNulty (email cliodna.mcnulty@hpa.org.uk)
References
- Charani E, Cooke J, Holmes A. Antibiotic stewardship programmes — what’s missing? Journal of Antimicrobial Chemotherapy 2010;65:2275–77.
- Stockley JM. European antibiotic awareness day 2010: Why doesn’t promoting antibiotic awareness always work? Journal of Infection 2010;61:361–3.
- Costelloe C, Metcalfe C, Lovering A et al. Effect of antibiotic prescribing in primary care on antimicrobial resistance in individual patients: systematic review and meta-analysis. BMJ 2010;340:c2096.
- Del Mar C. Prescribing antibiotics in primary care. BMJ 2007;335:407–8.
- Chung A, Perera R, Brueggemann AB, et al. Effect of antibiotic prescribing on antibiotic resistance in individual children in primary care: prospective cohort study. BMJ 2007;335:429.
- Enne VI. Reducing antimicrobial resistance in the community by restricting prescribing: can it be done? Journal of Antimicrobial Chemotherapy 2010;65:179–82.
- Butler CC, Dunstan F, Heginbothom M, et al. Containing antibiotic resistance: decreased antibiotic-resistant coliform urinary tract infections with reduction in antibiotic prescribing by general practices. British Journal of General Practice 2007;57:785–92.
- Department of Health Standing Medical Advisory Committee, Subgroup on Antimicrobial Resistance Occasional Report. The path of least resistance. London: Stationery Office, 1998.
- McNulty CA, Smith, G. on behalf of the PHLS Primary Care Co-ordinators. PHLS works towards effective interaction with primary care. Review. Communicable Disease and Public Health 2001;3:234–9.
- McNulty CAM, Gammie SM, Weinberg JR. Antibiotic guidance on the web: an opportunity for open review. Journal of Antimicrobial Chemotherapy 2002;49:667–9.
- Health Protection Agency. Management of infection guidance for primary care for consultation and local adaptation. Available at: www.hpa.org.uk (accessed 25 February 2011).
- NHS Clinical Knowledge Summaries. Available at: www.cks.nhs.uk/home (accessed 23 May 2011).
- Cooke J, Mason AR, Drummond MF et al. Medication management in English National Health Service Hospitals. American Journal of Health-System Pharmacy 2005;62:189–95.
- NHS Atlas of Variation in Healthcare. Reducing unwarranted variation to increase value and improve quality. 2010. Available at: www.rightcare.nhs.uk/atlas/index.html and www.sepho.org.uk/extras/maps/NHSatlas/ atlas.html (accessed 25 February 2011).
- Cooke J, Alexander K, Charani E, et al. Antimicrobial stewardship: an evidence-based antimicrobial self-assessment toolkit (ASAT) for acute hospitals. Journal of Antimicrobial Chemotherapy 2010;64:2669–73.
- SurveyMonkey. Online survey software and questionnaire tool. Available at: www.surveymonkey.com (accessed 23 May 2011).
- NHS Prescription services. Antibiotic prescribing excluding topical antibiotic preparations. 2009. NHS Business Services Authority. Available at: www.nhsbsa.nhs.uk (accessed 5 April 2011).
- Thompson PL, Spyridis N, Sharland M, et al. Changes in clinical indications for community antibiotic prescribing for children in the UK from 1996 to 2006: will the new NICE prescribing guidance on upper respiratory tract infections just be ignored? Archives of Disease in Childhood 2009;94:337–40.
- NHS National Institute for Health and Clinical Excellence. Respiratory tract infections: Prescribing of antibiotics for self-limiting respiratory tract infections in adults and children in primary care. http://guidance.nice.org.uk/CG69 Published 2008. Accessed September 20, 2011.
- Cals JW, Butler CC, Hopstaken RM et al. Effect of point of care testing for C reactive protein and training in communication skills on antibiotic use in lower respiratory tract infections: cluster randomised trial. BMJ 2009;338:b1374.
- Francis NA, Butler CC, Hood K et al. Effect of using an interactive booklet about childhood respiratory infections in primary care consultations on reconsulting and antibiotic prescribing: a cluster randomised controlled trial. BMJ 2009;339:b2885.
- National Prescribing Centre. Key therapeutic topics 2010/11 — Medicines management options for local implementation. Available at: www.npc.nhs.uk (accessed 19 May 2011).
- Mol PG, Wieringa JE, Nannanpanday PV et al. Improving compliance with hospital antibiotic guidelines: a time-series intervention analysis. Journal of Antimicrobial Chemotherapy 2005;55:550–7.
- Deuster S, Roten I, Muehlebach S. Implementation of treatment guidelines to support judicious use of antibiotic therapy. Journal of Clinical Pharmacy and Therapeutics 2010;35:71–8.
- Stille CJ, Rifas-Shiman SL, Kleinman K et al. Physician responses to a community-level trial promoting judicious antibiotic use. Annals of Family Medicine 2008;6:206–12.
- Coenen S, Ferech M, Haaijer-Ruskamp FM et al, and the ESAC Project Group. European Surveillance of Antimicrobial Consumption (ESAC): quality indicators for outpatient antibiotic use in Europe. Qual Saf Health Care 2007;16:440-445.
- Adriaenssens N, Coenen S, Tonkin-Crine S, Verheij TJ, Little P, Goossens H on behalf of the ESAC Project Group. European Surveillance of Antimicrobial Consumption (ESAC): disease-specific quality indicators for outpatient antibiotic prescribing. BMJ Quality and Safety 2011;20:764–72.
- Buising KL, Thursky KA, Black JF et al. Improving antibiotic prescribing for adults with community acquired pneumonia: does a computerised decision support system achieve more than academic detailing alone? A time series analysis. BMC Medical Informatics and Decision Making 2008;8:35.
- NICE. Measuring the use of NICE guidance. 2011. Available at: www.nice.org.uk (accessed 5 April 2011).
- Earnshaw S, Monnet DL, Duncan B et al, the European Antibiotic Awareness Day Technical Advisory Committee, the European Antibiotic Awareness Day Collaborative Group. European Antibiotic Awareness Day 2008. The first Europe-wide information campaign on prudent antibiotic use: methods and survey of activities in participating countries. Available at: www.eurosurveillance.org (accessed December 2011).