Charlotte Gurr
By the end of this article, you should be able to:
- Briefly describe the main types of study designs, including qualitative and quantitative studies;
- Describe the features, advantages and disadvantages of the quantitative observational study designs;
- Explain why the overall study design is important when evaluating studies and applying their findings to practice.
This article is the part of a comprehensive series exploring how to evaluate clinical studies when addressing information needs using a five-step process:
- Identifying study type or design;
- Appraising the journal, authors and study purpose;
- Critiquing the methods used;
- Understanding basic data or statistical analysis;
- Analysing the study results, interpretation and conclusions.
This article explores step one: identifying the study type or design.
Types of research study
There are two broad types of published studies: studies which employ qualitative research methods and studies which employ quantitative research methods. Each of these types might include some features of the other on occasion. A mixed methods study is a formal design involving both qualitative and quantitative approaches. Although we will focus on quantitative studies — since these are used most for drug and therapeutic clinical research — a brief description of qualitative studies follows.
Qualitative studies
Qualitative studies tend to describe or explore social or behavioural issues, problems or questions. They gather descriptive data and evidence to understand and characterise situations and problems to be able to formulate conclusions. The focus of qualitative research is on the descriptive process to gather the desired data. In contrast, as the name implies, quantitative studies focus on collecting (i.e. quantifying) numerical data from instruments and other tools that can be more concretely measured. Both qualitative and quantitative studies are valuable in healthcare research and can complement each other, with qualitative research providing important insights into behaviours or social factors that might affect individuals’ actions or therapy responses.
The focus of qualitative research is generally on the why or how of social and human/patient issues or problems (see Box 1). To obtain the desired information, qualitative studies often use methods such as focus groups, direct or indirect observations (e.g. using notes, recordings) of study participants, interviews or surveys. Detailed discussion about qualitative research designs and methods used are beyond the scope of this series and readers are encouraged to refer to many other comprehensive resources focused on this specific type of research.
Box 1: Qualitative studies
There are several types of qualitative research study designs, which can be quite complex.
Qualitative studies involving healthcare or medications might be conducted, for example, to:
- Explore why certain patients might be very adherent to a therapeutic intervention while others are not;
- Examine how patients make decisions involving their healthcare;
- Observe workflow in a pharmacy or other practice setting (e.g. emergency department) to determine ways to enhance communication and reduce medication errors;
- Determine reasons why some patients select healthier diets than others and what influences these selections;
- Identify the social and behavioural factors that affect opioid or other substance addiction and therapy success.
Quantitative studies
Quantitative studies focus on obtaining and using measurable, quantifiable or numerical data to resolve specific problems and answer clinical questions. There are two main types of quantitative study designs: experimental and observational designs. A third type, which we will discuss, is the ‘N-of-1’ design. Descriptive reports that obtain measurable, numerical data are also frequently seen in the literature, although they are generally not considered studies. These include case reports (i.e. reporting data from observations in one or a small number of individual patients) and case series (i.e. reporting data from observations in a small group ‘series’ of patients).
Experimental studies
Experimental studies — controlled or noncontrolled — gather quantifiable data from actual intervention by investigators. Subjects are assigned at random to a treatment/intervention group and, importantly, are given treatments by the investigators. These types of studies have also been referred to as explanatory studies, since they can determine cause-and-effect relationships — that is, whether an exposure, intervention or treatment actually caused an outcome or condition. Controlled experimental designs are best (i.e. ‘gold standard’), since they use a treatment group(s) and a control (i.e. comparison) group(s).
Types of control groups include placebo, active (i.e. the use of another treatment with established efficacy for the condition studied), no treatment or historical (i.e. a comparison with a treatment previously studied, which is not commonly used and only when it is not possible to use a different type of control). The control group helps account for factors — other than the treatment — that might affect the study results. Investigators compare the effects seen in the control-group patients with those in the treated patients to determine if there is a difference between them.
Quasi-experimental studies
There are studies that seem to be experimental in design but lack a control group to which participants were randomly assigned. These are referred to as having a quasi-experimental design. For example, in educational research, investigators might want to determine how a new teaching method affects learning, but it is not possible, or practical, to split the students in one class into two groups and randomly use different teaching methods for each. Even though investigators might compare measurable course exam grades after using the new method to past students’ exam grades obtained with other teaching approaches, the student groups being compared in the study did not receive their teaching method at random. Rather, new students in the class received the new method, while students in prior classes were taught using previous teaching methods.
A limitation of not using random assignment to the intervention — in this case teaching method — group is that there could be important differences between groups that could affect the findings obtained, independent of the intervention. For example, past students might differ from current students in levels of motivation, grade point averages or outside influences affecting their education (e.g. marital status, work schedules, extracurricular activities). Another example of a quasi-experimental design often encountered in practice is the pre-test/post-test design (see Box 2).
Box 2: Pre-test/post-test quasi-experimental design
With this design, measurements are repeated in the same group of subjects before and after a treatment or intervention. Suppose investigators in a pulmonary medicine clinic wanted to determine if patient education about using asthma inhalers increased patients’ knowledge about the proper use of inhalers. The investigators developed a pre-test and post-test designed to evaluate patients’ knowledge about inhalers and prepared patient education brochures describing appropriate inhaler use. The investigators then administered the pre-test to their patients with asthma, provided the educational brochures to patients, and followed up by giving patients the post-test two weeks later. The post-test scores were found to be significantly higher than the pre-test scores; the investigators concluded that the educational brochures improved the patients’ inhaler knowledge.
Is it appropriate to conclude that the patient education provided by the investigators was an effective method for improving inhaler knowledge?
Not necessarily. A concern with a quasi-experimental study design is the lack of a control or comparison group. Without such a group, it cannot be said with certainty that it was the education delivered by the investigators that produced the increase in patients’ knowledge of inhalers. While it appears that the education was effective, there might have been other reasons for the increase in patients’ knowledge — perhaps they read about the inhalers on their own, spoke with others about inhaler use, were educated about inhaler use by the pharmacist when they had their prescription filled, etc. It is best whenever possible to use a study design that includes a control/comparison group.
N-of-1 design
One unique type of experimental study is the ‘N-of-1’ or single-patient/subject research design. With this type of study, the researcher, often a primary care practitioner, identifies a specific patient to study. The researcher conducts a baseline assessment of the patient’s condition, followed by therapy initiation. The researcher then measures changes in the condition during and after therapy. The researcher might decide, after stopping therapy and re-assessing baseline measures, to repeat the therapy to determine if the same effects are again observed. If there is another therapy to study, after stopping the first therapy, the researcher re-assesses the baseline measures and then initiates the next treatment, repeating the same measurements during/after therapy. The N-of-1 trial can be beneficial for studying individuals with rare or chronic conditions that require unique or individualised therapy, although its disadvantages do not allow it to replace other experimental designs:
- Unable to generalise results to others;
- Difficult-to-impossible to perform statistical analyses;
- Difficult to validate findings.
They are most applicable to chronic, stable conditions not subject to rapid fluctuations.
Observational studies
Case–control, cohort, and cross-sectional studies are observational designs, meaning the treatment(s) taken or other exposures studied were not given by the study investigators (i.e. the patients/subjects were exposed or already taking the medication studied). The investigators carefully observe and evaluate patients/subjects to achieve their study objectives. Although the controlled experimental study design is best, observational designs are generally used when it is not possible, feasible (e.g. for rare conditions or those that require a long time to develop), or ethical to use an experimental design (e.g. exposing subjects to substances likely to cause harm without benefits). In these situations, they can provide helpful and extremely valuable information. An example is outlined in Box 3.
Box 3: Studying the relationship between coffee consumption and pancreatic cancer
Investigators are interested in studying whether coffee intake is associated with an increased risk of pancreatic cancer development. They suspect that coffee might be a risk factor for pancreatic cancer.
Would it be appropriate for the investigators to use an experimental design to test their hypothesis?
No. It is unethical for investigators to expose subjects to a substance (e.g. coffee) or to administer a therapy for the primary objective to see if the exposure or therapy increases the risk of an adverse outcome (e.g. pancreatic cancer, usually fatal), without any possible offsetting advantages. Since cancer generally takes a long time to develop, an experimental study would also not be practical. An observational study design enrolling subjects already drinking coffee or who have been diagnosed with pancreatic cancer — in order to study their history of coffee intake — would be appropriate here.
Figure 1 provides a summary of each of the observational designs. The case–control design is used to determine possible factors (e.g. exposures, drugs) influencing or causing an event or outcome. It is always retrospective (i.e. looking backward). Why? This design begins with patients who already have the event or outcome (i.e. cases) and also enrol another group of similar patients who lack the event or outcome (i.e. controls). The investigators then need to look back in time to compare drug use or the extent of exposure in both groups, which would be prior to outcome development in the cases and back a specified amount of time in the controls. If the cases are found to have significantly greater drug use or extent of exposure than the controls, a possible association exists between the drug/exposure and outcome development.
Figure 1: Observational study features, advantages, and disadvantages
Box 4: Prospective versus retrospective cohort studies
A cohort study can be prospective (i.e. concurrent) or retrospective (i.e. non-concurrent, historical) in nature. The basic design of each is the same:
- First identify groups (i.e. cohorts) with and without the drug use/exposures of interest — no one has the outcome at the start;
- Next, follow the groups forward over time and measure differences in outcome development.
The retrospective design differs from the prospective cohort study in that all information (i.e. drug use/exposures and outcomes) is obtained from already existing medical records or databases. The start of a retrospective cohort study occurs at a designated point in the past. The investigators initially select the cohorts for inclusion in either the study or control groups based on whether they have, or do not have, the exposure(s) of interest. No one in either group has the outcome at the start, and investigators at the start have no knowledge of whether the outcome will develop later. Once all subjects are included, the investigators examine the existing data going forward in time from the starting point, to determine the extent to which the subjects in each group developed the outcome of interest. The important distinction with the retrospective design is that all data examined going forward are still from the past.
Which cohort design, prospective or retrospective, is strongest?
The prospective or concurrent design is best because it is less subject to bias and inaccuracies. The non-concurrent or retrospective design is dependent upon existing records or databases that might be incomplete or incorrect.
The cohort design follows a study ‘cohort’ (i.e. a group of individuals/subjects who share a common characteristic) over time, longitudinally, to determine if a drug or other exposure might lead to the development of an outcome of interest. Unlike the case–control design, none of the subjects in a cohort study have the outcome at the start of the study. Instead, investigators identify subjects who are taking the drug or have the exposure of interest (i.e. study subjects), as well as similar subjects who are not taking the drug or who lack the exposure (i.e. control/comparison subjects). The investigators then follow the subjects in both groups (e.g. through scheduled visits, by examining medical records) over a certain time period to compare the extent to which they develop the outcome. If significantly more subjects in the study exposed group develop the outcome compared to the control subjects, it is concluded that the drug or other exposure might contribute to outcome development.
The subjects/patients (e.g. those with the exposure or without the exposure) enrolled in a cohort study can be identified from inpatient, outpatient or other settings. In addition, population-based cohort studies can be conducted. In a population cohort study, an entire population, or segment of the population, is studied. For example, suppose a population cohort study wishes to examine the relationship between dietary sodium intake and heart failure development. The population might consist of all patients with medical and dietary information available in a national managed care organisation’s database. The dietary histories can be searched in the database to identify and divide all patients into groups based upon their dietary sodium intake (e.g. high, moderate, low). The different dietary sodium population groups would then be followed over time to determine, and compare, rates of heart failure development.
In a cross-sectional design, the study sample is selected from a targeted population of interest and information about both the extent of drug use or other exposures, as well as the presence of the outcome is obtained from the sample at the same time. Thus, the cross-sectional study provides a ‘cross-section’ snapshot of the prevalence or existence of specific conditions, characteristics and outcomes at one point in time. The investigators obtain all the exposure and outcome data from the study sample through questionnaires, surveys, interviews or other records. The data from subjects within the sample can then be separated and compared based on the presence or absence of these factors. Since a cross-sectional study collects data about any past exposures or drug use from subjects’ recollections or records, it is subject to similar limitations as the case–control study. The cross-sectional study also lacks a separate control/comparison group at the start.
Worked examples
Worked example 1: Aspirin and Reye’s syndrome in children
Investigators wish to study if aspirin might lead to Reye’s syndrome development in small children who take aspirin for a viral illness. Children with a viral illness who later developed Reye’s syndrome were identified, along with other children with a viral illness who did not develop Reye’s syndrome. The use of aspirin during the viral illness, including dose and duration of therapy, was analysed and compared in both groups through interviews with parents.
What study design was used?
Case–control. The children with Reye’s syndrome (i.e. the condition/outcome present) were the cases, and the children without Reye’s syndrome were the controls.
What was compared/analysed?
Use of aspirin in both groups. Since the exposure — in this case, aspirin use — must precede development of the condition/outcome (i.e. Reye’s syndrome), information about the exposure is obtained by looking in the past (i.e. retrospective). Greater aspirin use in the cases might indicate a link between aspirin use and development of Reye’s syndrome. However, even if aspirin use was much greater in the cases, this study design cannot prove that aspirin causes Reye’s syndrome.
Was this a good design to use here?
Yes. Reye’s syndrome is a relatively rare, serious condition that could be fatal. A case–control design is an excellent choice to fairly rapidly explore possible causes of uncommon adverse outcomes, such as a suspected association of Reye’s syndrome with aspirin therapy in children. A cohort design would be more time consuming for studying rare/uncommon conditions since a very large number of children either taking aspirin or not taking aspirin for a viral illness would need to be enrolled and followed, with most not developing the outcome. An experimental design is impossible owing to ethical issues — if aspirin was suspected to possibly cause Reye’s syndrome, deliberately administering aspirin to children to see if the drug increased their risk of developing this serious illness is not justifiable.
Worked example 2: Vitamin D and depression in children
Previous studies in adults have indicated a possible link between low vitamin D levels and depression symptoms. Since children can also develop depression, investigators studied whether low vitamin D levels might be associated with depression in children. Several hundred children were identified and their vitamin D concentrations were measured. The children were then divided into groups based upon their vitamin D level: low, normal, or high. The children were followed over the next three to four years and were periodically tested to determine if they developed symptoms of depression.
What study design was used?
Cohort. The children were initially divided into groups based upon the extent to which they had the exposure/factor of interest (i.e. their vitamin D level); the study subjects had low vitamin D levels and subjects in the comparison groups had normal or high levels. There was no therapy intervention. The investigators did not administer vitamin D to the children but rather measured existing concentrations, so the design was not experimental.
What was compared/analysed?
Development of depressive symptoms (i.e. the outcome of interest) in all children. Children were followed prospectively over time to determine the extent to which depression occurred. Greater depressive symptoms in children with low vitamin D levels compared to those with normal to high levels could indicate a link between vitamin D deficiency and depression. However, even if depression developed to a much larger degree in children with low vitamin D concentrations, this study design cannot prove that a vitamin D deficiency causes depression.
Summary
Key points
- In general, the order of the quantitative study designs from strongest (i.e. best) to weakest (i.e. most limitations/disadvantages) is: controlled experimental; prospective cohort; and case–control/cross-sectional/retrospective cohort;
- Owing to their possible disadvantages, observational studies (e.g. case–control, cohort, cross-sectional) cannot prove that a drug or exposure caused a certain outcome; only well designed controlled experimental studies can do this;
- Observational studies can still provide very useful information when it is not possible, feasible, or ethical to conduct an experimental study — for example, to study whether a drug or other exposure might cause an adverse outcome or to study the factors that might predispose to development of a rare or infrequently occurring condition.
How to apply to practice
- If a news report claims that a drug or other exposure causes a certain adverse outcome based upon findings from an observational study, this might not be accurate. Further investigation is generally needed for confirmation;
- For the weakest study designs (e.g. case–control, cross-sectional, retrospective cohort), confirmation of their results by further study is even more important.
It is important to know the overall design used in a study since each design has inherent strengths and limitations. This chapter reviewed the basic types of study designs, with a focus on the structure of quantitative observational and experimental study designs, along with their advantages and limitations in practice.
To test your understanding of the key points, work through the self-assessment questions below. Answer guidance is provided at the end of the article.
The next step in the process of evaluating a clinical study is covered in the article ‘Evaluating clinical studies: journals, authors and study purpose’.
Self-assessment questions
Question 1
A study was conducted to determine whether daily users of non-steroidal anti-inflammatory drugs (NSAIDs) were at lower risk of developing benign prostatic hyperplasia (BPH) than non-daily NSAID users. The medical records from a large health centre were used to identify men who were either daily (n=536) or non-daily (n=659) NSAID users. The patients then received twice-yearly examinations for the next five years for signs and symptoms of BPH.
After adjusting for age differences between groups, greater daily NSAID use was associated with less BPH development compared to non-daily NSAID use. The authors concluded that regular NSAID use might prevent or delay the development of BPH.
Which type of design was used in this study?
Can this study be used to prove that daily NSAIDs can decrease the risk of BPH development?
Question 2
Investigators wish to conduct a study with the following objective: to determine whether statins used to reduce cholesterol might increase the likelihood of developing type 2 diabetes mellitus (T2DM).
Briefly describe how this study could be conducted using:
- A prospective cohort design; and
- A case–control design.
Question 3
A study was performed to determine the effects of garlic powder tablets on blood glucose levels and plasma lipids in patients with T2DM. In total, 56 patients with T2DM were randomised to receive either two garlic tablets BID or a placebo control for four weeks. Fasting blood glucose was measured daily, and plasma cholesterol and triglycerides were measured at baseline and after two and four weeks. At the end of four weeks, blood glucose and cholesterol levels were found to be significantly reduced in patients receiving the garlic tablets compared to placebo. It was concluded that garlic tablets might be a useful supplement to reduce cardiovascular risk in patients with T2DM.
Was this an experimental or observational study? Explain.
Question 4
Acetaminophen is often responsible for poisonings that can result in acute liver failure1. The investigators wanted to study the extent to which adults are knowledgeable about acetaminophen and its potential toxicity and whether this knowledge increased their likelihood of recognising available acetaminophen-containing medications. Subjects aged at least 19 years who were being seen for a variety of reasons in a large outpatient clinic in Boston were given a survey to assess their knowledge about acetaminophen dosing and toxicity, and whether they could identify commonly used non-prescription combination drug products that contained acetaminophen.
Results showed that many patients had difficulty recognising acetaminophen-containing products by name and were lacking knowledge of acetaminophen dosing and toxicity. No association was found between having greater knowledge of acetaminophen dosing and toxicity and the ability to recognise products that contained acetaminophen.
What type of design was used in this study?
Is this considered a strong study design?
Question 5
Which of the following is least likely to be affected by selection bias?
A: Case–control study
B: Cohort study
C: Controlled experimental study
Question 6
A study was conducted to determine if glitazones — a drug class used to treat diabetes — might protect against stroke development compared to other antidiabetic drugs (e.g. sulfonylureas, metformin) in patients with diabetes. The investigators identified 4,100 adult patients with T2DM who had experienced a stroke and 3,900 patients with T2DM of similar age and sex who did not have a stroke. The investigators examined the types of anti-diabetic drugs the patients in both groups had been taking prior to stroke development through medication records and interviews. It was found that a glitazone was taken by 43% of patients who did not have a stroke compared to only 32% of patients who had a stroke. It was concluded that glitazone use in patients with T2DM might result in a reduction in stroke risk compared to other anti-diabetic therapy.
What type of design was used in this study?
Can this study be used to prove that taking a glitazone is superior to other anti-diabetes therapy for reducing the risk of stroke?
Answer guidance
Question 1: answer
Cohort (i.e. prospective, concurrent) design. Subjects were initially enrolled based upon whether or not they had the exposure (i.e. daily NSAID use). Medical records were used to identify these individuals at the start, but the men in both groups were actively followed prospectively for the next five years to compare the extent to which they developed the outcome (i.e. BPH).
Since this is an observational study design, subject to more potential weaknesses and biases than an experimental study, the results from this study cannot definitely prove that regular NSAID use would delay BPH development.
Question 2: answer
Prospective cohort design: a group of patients is identified with the drug use/exposure of interest (i.e. taking statin drugs for elevated cholesterol concentrations), along with another group of patients with similar characteristics (e.g. comparable ages, gender proportions, cholesterol concentrations) but without the drug use/exposure of interest (i.e. not taking statin drugs). Both groups of patients would then be followed into the future for a specified amount of time with their blood glucose concentrations periodically monitored. At the end of the follow-up period, the number of patients who developed the outcome (i.e. type 2 diabetes) during the study would be compared in both groups to determine if there is a significant difference.
Case–control design: a group of patients with the outcome (i.e. cases: T2DM) is identified along with another group of similar patients (e.g. comparable ages, gender proportions, cholesterol concentrations) but without the outcome (i.e. controls: without T2DM). Patients in both groups would be interviewed and/or medical records examined to determine their history of statin use. The number of cases and controls who used statins would be compared to determine if there is a significant difference.
Question 3: answer
This is an experimental design because actual intervention was used by the investigators (i.e. the administration to patients of garlic tablets or placebos).
Question 4: answer
Cross-sectional design. All study subjects were given a survey to determine their knowledge of both acetaminophen and its toxicity (i.e. the ‘exposure’) and their ability to identify acetaminophen-containing products (i.e. the outcome) at one point in time. The survey results were analysed to identify individuals with greater knowledge and those with less knowledge about acetaminophen, and to determine if there was an association between this knowledge and the ability to identify acetaminophen-containing products.
The cross-sectional design is not strong; it is one of the weakest designs. With this design, it is difficult to determine what came first, the ability to identify the products or the knowledge about acetaminophen. It also cannot be determined if the subjects recruited from the Boston clinic are truly representative of the rest of the population.
Question 5: answer
C: controlled experimental study. A controlled experimental study is less subject to selection bias than observational case–control and cohort studies. With a controlled experimental design, patients are enrolled based upon specified criteria and the patients are then assigned to receive either the control or treatment groups. Thus, the patients in these groups would likely be similar to each other. In contrast, the patients in a case–control study (i.e. cases versus controls) or cohort study (i.e. study group versus comparison group) differ from each other from the start in that they either have or do not have the outcome (i.e. case–control design) or have or do not have the exposure (i.e. cohort design). The investigators have to select (i.e. enrol) patients in the study groups who are similar with regard to other important characteristics that might affect the findings; this is difficult to accomplish and could lead to selection bias (i.e. differences in the patients resulting from how they were selected for study inclusion).
Question 6: answer
Case–control design. The case patients had T2DM and the outcome of interest (i.e. a stroke). The control patients had T2DM and were similar to the cases with the exception that they did not have the outcome (i.e. a stroke). At the start of the study, the investigators did not know the type of therapy the patients were taking for their T2DM. Following enrolment of the patient cases and controls, the investigators then determined the extent of exposure (i.e. the types of anti-diabetic medications the patients in each group took) through interviews and reviews of medication records.
A case–control study design is one of the weakest designs and cannot prove a cause–effect relationship. Thus, in this case, it is not known for sure whether glitazones might be more efficacious in preventing stroke development compared to other types of drugs used to treat diabetes. Based on this study’s findings, further study using an experimental design appears justified to determine if glitazones truly have an advantage over the other drug classes in preventing strokes.
- 1.Athbhaiya G, Tiwari A, Choudhary R, Samal PK. Pharmacological and toxicological effects of paracetamol: current knowledge and a review. Drug and Chemical Toxicology. 2026;49(2):404-427. doi:10.1080/01480545.2025.2604674
Acknowledgements
This article was adapted from Drug Information and Literature Evaluation, Second Edition, previously published by Pharmaceutical Press.
A full list of resources and materials used to prepare the book can be accessed from the bibliography page.
