What research really looks like

Bruce Fuller, professor emeritus of education and public policy at UC Berkeley, on describing versus explaining, confounds, and turning beliefs into questions you can test.

Recorded for Atlas fellows in summer 2026. Fuller is trained in sociology, has also taught at Harvard, and worked at the World Bank for several years. Atlas is not affiliated with or endorsed by these universities.

Summary

Fuller walks through how a researcher gets from a problem they care about to evidence that can actually answer a question. He describes research as shared work: each study adds a small piece to a large body of evidence and is judged by whether it confirms or disconfirms what others have found. Using examples from preschool and from AI in classrooms, he separates three kinds of questions (describing, correlating and explaining) and shows why a correlation is not yet a cause. He ends on how to present findings honestly when the people around you hold strong beliefs about the topic.

Key takeaways

  1. Describing, correlating and explaining are different jobs. Describing says what is there: how preschools vary in class size or teacher experience. Correlating says what goes with what: do preschools that spend more money have smaller classes? Explaining makes a causal claim, and that needs much stronger evidence. Fuller says even his PhD students mix these up.
  2. Watch for confounds. Children in smaller preschool classes may learn more, but children from better-off families are also more likely to be in smaller classes. Their background could be driving both. A randomized experiment, or a quasi-experimental design when random assignment isn't possible, is how researchers pull the effect of class size apart from who ends up in small classes.
  3. Ask for whom, and under what conditions. An effect can hold in one setting and not another (scope conditions), and it can help some groups more than others (heterogeneous effects). Preschool may lift learning for some children and do little for others; AI may help some students' writing and not others'.
  4. Separate beliefs from testable questions. "Everyone benefits from preschool" and "AI is a waste of time" are positions people hold with conviction. Research respects them, but breaks them into small questions that evidence can answer.
Show full transcript

Lightly edited for reading: filler words and repeated words are removed, transcription errors are corrected, and a few passages are left out.

Hello everybody. If you're awake and presentable, do turn on your camera. It's a little more meaningful to see everybody's friendly faces. You all come from 18 different time zones, so thank you for staying up late or getting up early. And we're going to record for the other students.

I'm Bruce Fuller. I'm trained in sociology. I most recently taught at the University of California, Berkeley. I recently retired from teaching, but I'm still doing research and some writing. I spent part of my career in policy organizations. I worked for the World Bank for several years, mainly in East Africa and Southern Africa. I taught at Harvard for a few years in comparative public policy. And I worked for a state legislature many, many moons ago, many years ago. So my career has been sort of in policy groups and in academia.

I sent you a couple of things to look at, which were kind of applied readings in sociology and education policy. I wanted to go a little bit more in depth today on some methods issues, and kind of how in the social sciences we approach empirical research. So I'm going to talk at you for a few minutes, and then hopefully we'll have 20, 25 minutes for questions and discussion, points of clarification, things I haven't been sufficiently precise on. Feel free to raise those questions.

So I say here that you all are future researchers contributing to a community of evidence builders. The research world is a little bit contradictory in the sense that, at least in the Western world, we get rewarded in the university for being creative, kind of individualistic, original. There's sort of this Western bias to be your own thinker in your head, being creative and trying to make original contributions. On the other hand, we're contributing to several centuries of research and philosophy. So personally, I'm a big believer in thinking about my role as contributing a little chunk to a very vast, voluminous collection of research, whether it's in sociology or educational policy.

So my opening point is really that you all are going to be part of a community, whether it's in STEM research or physics or chemistry or medical research or social sciences. You're going to quickly find out how much you don't know, and how over time you're going to know a lot more, and you're going to contribute to these different communities, these different fields of research, as a community member. I think that's an important thing to think about as you're building your career and building networks of fellow researchers. And I think the Atlas program is a great way to start building that sort of network.

The other opening point is that a lot of my work is reviewing the empirical evidence of other scholars. I just finished this book called Debating Childhood and Preschool. I do report on some of my own original research in it, but most of this book is trying to make sense of the work of other people. The book looks at five empirical questions, like: what's the quality of preschool in the US and other countries? Does preschool contribute to the early learning of certain kids, and is it not very effective in contributing to the learning of other children?

So my second point is that part of what we do is make our own little contribution to this big field, whether it's in physics or medicine or chemistry. But we're also building on the shoulders of other people, and we're trying to locate our work in this community, in this field. A lot of what we do is trying to see how our findings compare to other findings. Do we confirm the work of other people showing that preschool helps kids from poor families, or do we disconfirm that? I am somewhat of a positivist, in the sense that I'm all about disconfirming findings, and I'm all about trying to get to the truth by disconfirming rival hypotheses. And as our colleagues do that, we're talking to each other, and there's a conversation going on that's informed by empirical evidence.

Okay, with that, let me walk through some of these methods. First of all, in starting a study, my groups spend time trying to figure out what the problem actually is. What do we care about in the world? Are we worried? A lot of my research has been on problems facing young children, whether it's health issues, or stress and emotional anxiety in the home, or how different ethnic groups raise children differently. Usually there's a problem or something that catches our curiosity.

I really want to emphasize that this first step is crucial, because you're going to be bounced around a little bit, and people are going to say, "Why don't you do work on this?" or "Why don't you research that?" I think it's important to have a passion about the question you're asking, whether that's a problem or a challenge or a potential breakthrough and discovery. You really want to feel passionate about the issue you're tackling, because a lot of our research may take a year, it might take five years. I did a book on the history of school improvement in LA that took about 12 years with my students. So you want to start out with a pretty heartfelt passion, because you need that motivation, and that's going to power you through your work.

So in the sociology of education, these are some of the questions that my group has tackled. These are just examples from my particular field. Do preschools lift the early learning of young children? There's a lot of faith, and even in Uzbekistan, in Vietnam, in India, in Mexico, there's a lot of advocacy to expand preschools or child care subsidies. And there's an empirical question there, which is: what is the evidence about preschool lifting the early learning of young children? Or maybe preschools lift the early learning of certain children but not other children. We call these scope conditions. You might have an effect under a certain scope but not have an effect under another scope.

Another question is: do certain types of preschools or quality elements lift children? If governments are investing a lot of public monies in preschool, we sort of pray that they're putting some of the money into elements of quality that actually do elevate children's learning. Does that mean smaller classes? Does it mean longer periods of teacher training? Does it mean full-day preschool versus half-day preschool? So there's an efficiency, cost-effectiveness question here, which is: where do we invest our dollars, in what elements of quality?

And then there's this heterogeneity of effects question, which I referred to before: which kids may benefit, and which kids maybe don't benefit from preschool? Maybe we're wasting public investment thinking that preschool pays off, when maybe it doesn't pay off for wealthy kids or upper-middle-class kids. We call this heterogeneity of effects. In medical research (my wife is a medical doctor, and she's always talking about findings from the medical field): does penicillin affect certain patients more effectively than other patients? Do certain surgical procedures pay off for younger patients versus older patients? These are all heterogeneous effects that we need to understand before we put public money or energy into certain treatments, whether it's medicine or education.

You can also think about these questions from a STEM perspective. There's a lot of interest now in whether AI improves the writing skills of high school students. That's an empirical question. Can AI be used in specific ways to advance student writing? Scope conditions: how can AI be used effectively, and when is AI a waste of time if our goal is to increase students' writing skills? And again, heterogeneous effects: do certain students benefit from AI, whereas other students may not benefit from AI in their education, in elementary school or in high school? So these are different kinds of empirical questions that try to pinpoint effects.

The other thing to notice is that we try to distinguish between philosophy and empirical questions. In my field of early education, in the United States and in Uzbekistan and in Vietnam, there are advocates who just have faith that if we expand preschool, everybody will benefit. That's a philosophical position. But you've got to break that apart into empirically testable, bite-sized pieces, and then go after those empirical questions. There's a lot of advocacy around AI around the world, of course: a lot of people arguing that AI is a waste of time, or that it's going to slow down learning in high school. Those are philosophical advocacy positions that we have to respect, but at the same time, in the research world, we're trying to tease out and identify empirically testable questions.

Now, the other thing I wanted to emphasize from a method standpoint, and this applies to social science as well as STEM work or medical research, is that we think about three ways to dig into an empirical question. The first way is describing what's going on. The second is correlating variables. The third way is explaining: trying to come up with causal explanations.

A lot of the work that you're probably reading already is correlational. It's saying that the children who attend smaller classes learn more. That's a correlational statement. We have a big sample of 80 preschool classrooms. They vary in the number of kids in those classrooms, and the number of kids is correlated with the rate of learning among kids in preschool. That's a correlational statement.

I want to be clear here about the differences between describing, correlating and explaining. This might be old hat, this might not be new information for some of you, but even my PhD students make mistakes and confuse one for the other.

For example, in sociology we might have a descriptive question like: how does the quality of preschools vary? What are the elements or constructs that matter for teachers or children? So first of all, how does the quality of preschools vary? Preschools might vary in terms of class size. Preschools might vary in terms of the experience of preschool teachers. Preschools might vary in terms of the social class background of the children who attend them. These are all descriptive questions. In your own work, as you develop a sample, whether it's kids involved with AI or whether it's a chemistry problem, you're going to have a sample of experiments, a sample of trials, and you want to explain features of that sample. These are descriptive exercises.

Now, the second clause here: what elements or constructs matter for teachers or children? In the social sciences we run into real-life human beings, and in the case of early education, teachers have really strong philosophical positions about what quality means. Preschool teachers might on balance say smaller classes equal higher quality, or they might say more colorful classrooms equal higher quality. Again, we're back to philosophy or ideology. There are advocacy positions about how you describe quality in a preschool, or how you describe the utility of oxycodone in the medical field. These are heartfelt feelings about some phenomenon. But what we're trying to do empirically is really get to these constructs, get to these variables, and more carefully describe these different elements of quality, in this case.

The second stage is: how do levels of quality covary with, say, preschool spending? When we have a big sample of preschools, do we see that preschools that spend more money have higher quality? Do preschools that spend more money have smaller classes? Do preschools that spend more money attract stronger teachers? These are correlational or associational statements and findings. Usually social sciences, medical researchers, chemists, when you have a sample or a set of trials, you have six or seven variables, or 25 variables, and you correlate all those variables with each other to get a sense of what goes with what. Where are there correlations?

But this is still very far from developing a causal argument or causal evidence. At the end of the day, we'd like to know what elements of preschool quality causally drive higher learning. What is the causal story here? It may be that we find that, overall, preschools with smaller class sizes have kids with higher learning. But it turns out that kids from higher social classes go to preschools with smaller classes. That is, we have confounding variables. The prior background characteristics of kids may be driving their learning, and also driving their selection into preschools with fewer kids. So we've got this confound problem.

So a lot of what we do in social science, and now in biomedical research, is try to use advanced statistics to come up with causal arguments. A simple way to do this is to run a real experiment: have a control group, and randomly assign people into a treatment group. Even better is to have a placebo group, and this is what medical researchers do. In clinical trials, they assign patients randomly to the treatment, to the control, or to a group that thinks they're getting a treatment but in fact isn't getting the treatment that's being studied.

But in the social sciences, or in classrooms, or in real-life situations, sometimes you can't randomly assign people. There are ethical concerns. There's suspicion about what the scientist is doing. So we have to come up with quasi-experimental statistical models. You're probably starting to read a little bit about this, and I won't go into the weeds on it today, but it's important, because this is moving from correlational analysis, just looking at associations among variables, to actually trying to identify the factors that are driving the outcome we're trying to achieve. Whether it's healthier children or better AI use, whatever the outcome is, we want to know the causal factors that are driving it. And that takes a couple of extra steps beyond correlational analysis.

This is the same sort of breakdown if you were looking at AI in classrooms. How does the quality of AI applications vary? What features matter to teachers or children? Teachers might have strong feelings about different AI software programs, but we want to describe the different quality elements of AI packages more systematically. We want to know how elements of AI packages might covary with, say, student engagement or student motivation. But even once we've established these associations, we still can't mount a causal argument, because there might be prior confounds. Maybe kids who are more nerdy, who spend more time on the computer, select more effective AI packages, but it's their prior characteristics, their prior motivation, that's leading to stronger engagement, not necessarily the AI package itself. So to deal with this confounding problem, again, we've got to get to causal arguments and quasi-experimental designs with more advanced statistics.

And then lastly, I wanted to make the point that even if you're in a STEM field, or looking at AI, or an aspiring physicist, you're going to run into competing ideologies and competing social values, in the surrounding environment and also in the organizations where you're doing research. In universities, in the West and other parts of the world, faculty and professors take all sorts of philosophical positions. There are all sorts of empirical findings floating around that people feel they need to back and support. In research and development organizations, you have different social values, different political currents running through them. And you've got to figure out a way to address your empirical findings while respecting the competing social values and ideologies that surround you.

In sociology, labor unions in the West support certain kinds of child care and oppose other kinds of child care. I've been doing some work in Vietnam recently, and in the industrial zones where there are big factory manufacturing operations, a lot of firms want very inexpensive child care, because they don't want to spend a lot of money on it. They're trying to make a higher profit. Whereas the funders and researchers are saying: we shouldn't invest in cheap child care, we should invest in high-quality preschool and child care, so the kids are elevated and advanced in their learning. But again, in the case of Vietnam, we have to present our findings in a somewhat politicized environment.

In AI, of course, you're probably reading about this already. There's a conflicting science of cognitive development. You've got cognitive psychologists saying that when it comes to writing skills, AI is eroding the writing skills of high school students, AI is a crutch, and it's slowing down cognitive development for adolescents. And you have other scientists saying no, no, no, AI can actually escalate and advance the rate at which cognitive skills are developed. Funders prefer competing lines of research in many fields. There's the controversy over doing research on genetic material. Medical research is controversial. Renewable energy research. I don't need to go through all these examples.

My point here is that as you get better at methods, you also want to get better at understanding the surrounding social and political environment, because you're going to run into it, and you don't want to be scared by it. You don't want to be put off by it. You just need to deal with it without compromising your scientific values and your commitment to empirical science. So it's a tricky walk. It's a balancing act, but you do want to pursue science with respect for, and an understanding of, this broader societal environment.

So I'm going to stop there, and I'm happy to dig into anything more, answer questions or get into a discussion.

Want more? Join the Atlas community

Members get every session live, plus recordings, guides and feedback.

Join our Discord

Ready to write your own paper? Apply to the Winter Fellowship

Atlas Research Institute is a nonprofit that introduces students to research. In our free six-week online fellowship, students pick an issue they care about, learn to find and test evidence, and finish their first research paper.

Applications close December 21, 2026.

Apply

All resources