RCT and Quasi-Experimental Design
A randomized controlled trial assigns participants to a treatment or comparison group at random, so any difference in outcomes can be attributed to the program. A quasi-experimental design compares groups without randomizing. Praxia Insights designs and runs both, for funders and organizations that need evidence a program actually caused the results.
Most evaluation tells you what happened. These designs are how you find out whether your program caused it, and that's a much harder question to answer.
It matters because the alternative explanations are always available. The students who chose your program might have been the motivated ones anyway. The clinics that adopted your model might have already been the better-run ones. Without a comparison group, you can't rule any of that out, and a reviewer will notice.
Our team served as Project Lead on a $3.75 million Education Innovation and Research randomized controlled trial conducted in partnership with the American Institutes for Research, so we've done this at scale in real schools and not only on a design document.
Discuss your study design
When You Need This Level of Evidence
Your funder requires it: many federal programs specify an evidence standard that only these designs can meet.
You want to claim impact: saying your program works is a causal claim, and it needs a design that supports one.
Buyers are asking: districts, health systems, and government purchasers increasingly want to see rigorous evidence before they buy.
You're scaling: before you expand a program significantly, it's worth knowing whether it works or whether you've been measuring self-selection.
You're publishing: peer-reviewed journals expect a design that can support the conclusions you're drawing.
The Designs We Use
Randomized controlled trial
Participants, classrooms, sites, or organizations are assigned to treatment or comparison at random. Randomization is what removes the alternative explanations, and it's the reason most evidence standards rank this design highest. It's also the most demanding to run, and it isn't possible everywhere.
Difference-in-differences
Comparing how outcomes changed over time in the treatment group against how they changed in a comparison group, which handles the fact that both groups might have improved anyway.
Cluster randomized trial
Randomization happens at the group level, so whole schools, clinics, or sites get assigned instead of individuals. This is usually the practical option when a program is delivered to a whole group and you can't have some people in a classroom getting it and others not.
Regression discontinuity
Useful when participation is decided by a cutoff score or threshold, since the people just above and just below the line are similar in everything except whether they got the program.
Matched comparison design
When randomizing isn't possible, comparison sites or participants are matched to treatment ones on the characteristics that matter, so the two groups start out as similar as the data allows.
Interrupted time series
Tracking an outcome across many points before and after a program starts, so you can see whether the trend actually changed when the program did.
What Makes These Studies Succeed or Fail
The design is the easy part. What decides whether the study produces usable evidence is everything around it.
Sample size has to be large enough to detect an effect the size you actually expect, and underpowered studies are the single most common way a well-run trial ends up unable to say anything. Attrition has to be tracked and kept low, because a study that loses a third of its comparison group won't pass review. Fidelity matters too, since a program delivered inconsistently across sites is a different program at each one.
We plan for all of that up front, and we'll tell you honestly if the design you want can't be run with the sites and timeline you have.
Project Examples
A NIH-funded study examining neuropsychological predictors of adaptation after trauma.
A five-year randomized control trial (RCT), funded by the U.S. Department of Ed, to study individualized math instruction.
Led a national survey study in partnership with Berklee BEATL & Adobe to study AI use in the music industry.
How We Work
1. Initial consultation and plan
We meet to discuss your specific needs and confirm this is the right approach for the decision you're facing.
2. Project scoping and budget
You receive a custom scope of work with tasks, due dates, and budget before any work begins. Fixed scope, no hourly surprises.
3. Project execution
You receive updates on progress throughout, including recruitment status and fielding milestones.
4. A true partnership approach
We share insights with you as they come up throughout the project, so your team can start acting on what we learn long before the final report is finished.
5. Project completion
We deliver final insights in the format your team will actually use, whether that's a written report, an executive presentation, or a working session.
Frequently Asked Questions
-
It's a study where participants or sites are assigned at random to receive a program or not. Because assignment is random, the two groups should be similar in everything except the program, so a difference in outcomes can reasonably be attributed to it. That's what makes it the strongest design for causal claims.
-
It's a study that compares a treatment group to a comparison group without randomizing. Matching, difference-in-differences, and regression discontinuity are all quasi-experimental approaches. They're weaker than randomization because you can't fully rule out that the groups differed to begin with, and they're often the only practical option.
-
Often not. RCTs are expensive, slow, and impossible in a great many real settings, and a well-designed quasi-experimental study answers many questions well. You need an RCT when your funder specifies that standard, or when you intend to make a strong causal claim that will be scrutinized, and we'll tell you which situation you're in.
-
Longer than most people plan for. A study needing a full program cycle plus analysis and reporting runs at least eighteen months, and multi-year designs are common. Recruiting sites and getting approvals adds months before any data is collected.
-
Substantially more than other evaluation, because of the sample size, the data collection across sites, and the length of the study. For funded work it's built into the grant budget, and we'll help you scope a defensible figure before you submit.
-
It depends on how big an effect you expect and how much variation there is between sites. A power analysis answers this properly, and we run one before the design is finalized. Studies that skip this step often end up unable to detect the effect they were looking for.
-
Many programs can't, for practical or ethical reasons, and that's normal. We'll design the strongest quasi-experimental alternative your setting allows, and tell you plainly what claims the resulting evidence will and won't support.
-
It can, and the design has to be built for it from the start. Standards care about things like attrition, baseline equivalence, and how the comparison group was formed, and a study that ignores those can be perfectly sensible and still fail review. Tell us which standard you need to meet and we'll design to it.
-
Yes. Our team has run research with K-12 schools and districts across all 50 states and served as Project Lead on a $3.75 million EIR randomized controlled trial in schools. School-based trials need long lead times for district approval and consent, and a design built to fit the school calendar instead of fighting it.
-
It's a real possibility and you should plan for it before you start. A null result is still evidence, and it's frequently more useful than people expect, because it tells you where to look next. We'll also design the study so you learn about implementation along the way, so you'll know whether the program didn't work or simply wasn't delivered the way it was meant to be.
Our Experience
More than 60 studies designed, including case study, quasi-experimental, RCT, and mixed-method designs
More than 100 focus groups and in-depth interviews conducted
More than $4 million in grant-funded research managed
Clients spanning Fortune 500 companies, startups, nonprofits, universities, and federally funded programs
Our researchers hold doctorates in educational neuroscience, psychology, and related fields
Need evidence that will hold up?
Tell us about the program, the funder, and what claim you need to be able to make, and we'll advise on the design that would support it.