By Scarlett Barge
Before collecting and analyzing data, we must first ensure the design of the experiment is plausible and valid. Certain hypotheses can be easily tested using a typical lab setting, an experimental group, and a control. But what happens when this is simply not possible?
Enter John Snow. Long before dragons and direwolves, the 19th-century physician John Snow faced a deadly cholera outbreak in London with no randomized controlled trials, no IRB, and definitely no statistical software. What he did have was a keen eye, a map, and a natural experiment hiding in plain sight.
Snow suspected cholera was waterborne, differing from the widely held belief that it was airborne or caused my moral failings. Instead of randomly assigning Londoners to drink contaminated water (for obvious ethical reasons), he compared neighborhoods supplied by different water companies. One company drew water upstream before the sewage of London, the other did not. Cholera deaths clustered accordingly. This was a classic quasi-experiment, no random assignment, but a comparison structured by real-world conditions which provide a desired level of randomness.
Today, Snow’s work reads like an early case study in what the TREND (Transparent Reporting of Evaluations with Nonrandomized Designs) framework later formalized. TREND emphasizes clarity: explain how participants were selected, why groups differ, what biases might exist, and how conclusions should be interpreted. Snow did exactly that, even without the acronym. He documented his assumptions, acknowledged limitations, and made his reasoning explicit. Transparency turned an imperfect design into persuasive evidence.
When grappling with quasi-experiments, it helps to remember what to look for and what to question. The necessary elements can be summed up by CABIN:
Comparison group Assignment mechanism Baseline balance Interference Noncompliance
This simple word can anchor experiment structure and analysis with these natural experiments. The comparison group acts as the untreated(control) group. The assignment mechanism provides the mode of ‘randomness.’ Baseline balance is the measure of groups before being treated; in a good experiment these measures should be very similar. Interference measures the difference between the treated and untreated groups after the treatment occurs. And finally noncompliance requires us to acknowledge the possible interference that comes from it being a natural experiment and not happening in a controlled environment.
John Snow’s legacy reminds us that good research design is not about perfection; it’s about plausibility, transparency, and disciplined reasoning. When randomization is impossible, clarity becomes the next best thing. And sometimes, a well drawn map and an argument can change the course of public health.
Leave a Reply