Science reproducibility crisis: better measurement is required to tackle global health challenges

Every time you put petrol in your car, use GPS on your phone, fly by plane or receive a medical diagnosis, you are putting your trust in measurement. While often invisible, measurement plays a vital role in underpinning UK trade and regulation. It helps accelerate innovation and the adoption of new technologies. But it also plays an important role in responding to new challenges, such as antimicrobial resistance (AMR).

For example, the lack of an appropriate diagnostic test means that antibiotics are seldom prescribed based on a definitive diagnosis and up to half of antibiotic prescriptions have been deemed to be inappropriate. This not only carries the risk of increasing antimicrobial resistance, but also wastes money and does not deliver patient benefit. An increased use of in vitro diagnostics would be expected to lead to a reduction in unnecessary prescription of antibiotics and more targeted use of therapies.

The emergence of antimicrobial resistance is relentless and time is of the essence — diseases we previously considered conquered are returning. Without action, the O’Neill report^[1]
predicted the global cost of antimicrobial resistant infections could rise to 10 million lives lost per year by 2050, costing the global economy $100 trillion annually. To combat this growing issue, various organisations and commentators have called for a reinvigoration of the drug discovery pipeline and the implementation of rapid, low-cost diagnostics.

National Physical Laboratory (NPL), the UK’s National Measurement Institute, is one of the organisations responsible for delivering the National Measurement System. This system provides the framework, facilities and expertise to enable measurements to be reproduced with confidence, and with quan­tified uncertainty in the UK, this helps to ensure confidence in new innovation and treatments, assurance of consistency of care, and turn data into decisions^[2]
. However, the measurement system is not a static system, and adapts to ensure that we can respond to our end users’ needs, and support innovation and government policy.

Reproducibility crisis in science

The complexity of biological systems and processes can make it difficult to apply measurement methods and standards, so this is a significant challenging arena for measurement researchers. These challenges have led to poor reproducibility and a subsequent lack of confidence in results.

Indeed, many are now calling out this reproducibility crisis in science and in the life sciences in particular, it is becoming a barrier to market for new drugs and medical devices. There are numerous examples of figures across the scientific community highlighting the crisis, including Professor Sir Mark Walport (designated chief executive of UK Research and Innovation), who spoke recently on Radio 4’s Today programme about the problems of reproducibility in biological-related research^{[2]
,[3]
,[4]
,[5]}
.

But the issue goes beyond academic interest and also has real economic ramifications. In a study from Leonard Freedman and colleagues on the reproducibility problem, and its implications in the United States (US), the authors’ analysis of past studies concluded that in the US alone, an incredible $28 billion per year is spent on pre-clinical research that is not reproducible^[4]
.

“Low reproducibility rates within life science research undermine cumulative knowledge production and contribute to both delays and costs of therapeutic drug development,” the study authors concluded.

Ultimately, a lack of reproducibility often leads to wasted time, resources and money. But it also leads to false starts in product development, as in the case of developing new drugs. In fact, our stakeholders, whether they are in industry, government or the wider public sector such as the NHS, are telling us that the reproducibility issue is a major problem.

Therefore for the UK to try to overcome some of these issues, meet the recommendations of the Accelerated Access Review^[6]
and deliver against the Life Sciences Industrial Strategy^[7]
, better measurement needs to play a key part. Better measurements will help overcome some of the challenges in developing diagnostics and drugs by enabling quicker and more effective translation from the laboratory to patient end use.

Measurement challenges in antimicrobial resistance

Antimicrobial resistance is a fast-growing, and changing, global challenge and will require innovation on many fronts to overcome effectively. One front includes advances in measurement science and standardisation to provide an improved understanding of antimicrobial resistance, through which existing drugs, therapies and diagnostics can be optimised and new versions developed. However, currently, there is no single measurement technique that can deliver all the measurement answers needed by scientists. So to try to meet these needs and based on consultation with industry, regulators, researchers and healthcare providers, we have identified three main areas where we are focusing activities:

• Increasing end users’ understanding of antimicrobial resistance and microbial systems and processes by improving standardised measurements and reproducibility from the molecular through to the tissue level, while in different environments.
• Bolstering confidence in a range of diagnostics interventions, from the ability to diagnose and detect the health of large groups of people, non-invasively and in real time, through to the tailored personal monitoring of an individual at the point of care to prevent unnecessary treatment. This could also be translated eventually to animals.
• Providing validated evidence and data that is fit for purpose and enables users to meet regulatory requirements.

Specifically, NPL is already active in providing measurements for new antimicrobial drug discovery and development (with the aim of reducing late-stage drug attrition), for the development of rapid and validated diagnostics, and for data collection and comprehension.

We are currently leading various European-wide projects, including one linked to the Innovative Medicines Initiative’s New Drugs for Bad Bugs programme. In this case the challenge given to us by the pharmaceutical industry was: “How can you measure or quantify the effective penetration of drugs into Gram-negative bacteria?” Based on that challenge, we have brought together a consortium with a range of advanced analytical techniques and reference protocols, which will be standardised so that no matter where they are used in Europe, harmonisation will be achieved.

Additionally, we will be able to quantify the image analysis produced using these analytical tools (for example, mass spectrometry imaging) in terms of not only the penetration of the drug down to the subcellular level, but also its localisation, its accumulation and even its removal, if the bacteria cell is able to provide efflux processes.

In the areas of data and the associated challenges with collection, storage, collation and analysis, NPL has recently created a new Data Science Department specifically to support the growing use of data analytics and the integration of large datasets. So one can imagine that if we are able to collect better quality, preclinical, biomedical and clinical data, with improved traceability, then we will be able to stratify patients better and achieve improved patient outcomes.

We are developing reference cell membranes that can be used to screen antibiotics, based on their ability to traverse different biological membranes. In addition, NPL is also investigating the use of cognitive computing approaches to screen the chemical white space, to identify new and promising antimicrobial peptide sequences and then use computer modelling to understand how these sequences interact with microbial cell walls and membranes^[8]
,[9]
.

Opportunities to make better use of the UK’s measurement infrastructure

The UK is facing many challenges, be that the ageing population and expansion of chronic diseases including diabetes and neurodegenerative diseases, the rapid acceleration in antimicrobial resistance, and the uncertainties regarding research, regulation and access to skills following Brexit. However, the UK’s measurement system is world leading and underpins not only regulation and trade for the UK, but also improves the quality of our lives.

It will be a vital component of successfully implementing the recently published Life Sciences Industrial Strategy^[7]
, including the recommendation to address areas of market failures and encourage AMR research. Through increasing our engagement we will be better able to support the UK leading the way in world-class reproducible research and collaborating internationally to tackle upcoming challenges.

We are building momentum and recognise that we have to work more closely with the life sciences and health community. It is vital that we pool our resources effectively, align ourselves, and create and deliver the measurement infrastructure to underpin new antimicrobial therapies, accelerate medical innovation and improve patient outcomes.