How to Read an Econometrics Paper
Reading and understanding econometrics papers can be hard work. Most published articles, even review articles, are written by specialists for specialists. Unless you’re already familiar with the literature, it can be a real uphill battle to make it through a recent paper. In grad school I remember our professors repeatedly admonishing me and the rest of the cohort to “read the papers!” But when I did my best to follow this advice, I nearly always felt like I was banging my head against a wall.
Effective reading is a skill that can be learned, and the only way to learn is through practice. But you can learn the easy way or the hard way. The hard way is to keep trying and hope for the best; the easy way is to adjust your approach based on the experiences of others. With that in mind, this post offers some tips and tricks that I’ve picked up through the years for reading technical material efficiently and effectively. My target audience is PhD students in Economics, especially students in the Econometrics Reading Group at Oxford, but I hope that some of the following tips will be helpful for others as well.
If you have any tips of your own, or if you violently agree or disagree with any of mine, I hope to hear from you in the comments section below!
Read Something Else Instead
The first question to ask yourself is whether you should even be reading this paper in the first place. Just because White’s (1980) paper on heteroskedasticity-robust standard errors is a “classic” in econometrics, that doesn’t mean that you should read it. In fact, as a graduate student just starting out, you probably shouldn’t! The paper that introduces a new idea or procedure is rarely the paper that gives the clearest explanation. Reading a good textbook explanation is a much more effective way to get to grips with a new idea. You might, for example, try reading the relevant chapters in White’s textbook Asymptotic Theory for Econometricians instead.
But sometimes you have to read a particular paper. Maybe it’s the paper you’ve been assigned to present in a reading group, or maybe it’s highly relevant to your own research. In that case you may still want to start by reading something else. For example, there might be a more recent paper or review article that gives a good summary of the idea or method in question. Reading this paper first can make it much easier for you to tackle the original paper.
So to all those professors out there who keep telling their students to “read the papers!” I say: “read the papers, but only after you’ve read something else first!”
Don’t Assume You Have to Understand the Whole Thing
As a general rule you should not expect to understand everything when you read a paper. You may only get 10% on the first read, but that’s fine! Besides papers I’ve written myself, there are relatively few articles that I’ve checked line-by-line from start to finish. Even if you’ve been assigned to present a paper that doesn’t mean that you need to understand every detail of every lemma in the online technical appendix. Instead your goal should be to understand the key ideas and contributions of the paper. Like anything in life, there are diminishing returns to effort in reading a paper. When reading papers to support your own research, you can be even more selective. The key question becomes: “how is this relevant for what I’m doing?” It may be that you only need to understand a small part of the paper to get what you need.
Don’t Assume You’re Stupid
If you’re confused, don’t assume that it’s your fault. Notice your confusion and try to get to the bottom of it without taking things for granted or engaging in negative self-talk. The only way to learn is by getting confused and then unconfusing yourself!
You may be confused because the authors assume you know something that you don’t. They are likely experts in the field who have spent years thinking about this particular question. You, on the other hand, are just starting out. As you gain a bit more context, things may fall rapidly into place. (See my next tip below.)
You may be confused because the paper is confusingly written. Writing is hard, and technical writing is especially hard. The referee process can even make papers more confusing, since our present system for evaluating research involves multiple rounds of revisions in which the authors must try to satisfy referees with differing views. The result is that published papers often contain a substantial element of “cruft” that distracts from the main message.
You may even be confused because the paper is wrong! As a good Bayesian, you shouldn’t immediately jump to the conclusion that you, a newcomer to this field, have stumbled upon a crucial error that everyone else has missed. On the other hand, you definitely shouldn’t believe everything that you see in print! All papers are wrong in some way, and some papers are wrong in serious and important ways. If you’re confused, it’s worth considering whether the authors were confused too!
Spread Yourself Thin
Let’s say you really need to get to grips with paper X on topic Y. You’ve read the relevant textbook material, you’ve tried a review article, and you’re still struggling. What now? Strange though it may sound, one helpful answer is to read more papers on topic Y in an extremely shallow way. Skim the abstracts, introductions, and conclusions. Note any terms or concepts that keep appearing, especially ones that you don’t understand.
I can think of many occasions when I skimmed nine papers and didn’t understand any of them, but then read a tenth and suddenly everything clicked. The key here is context. When you’re new to topic Y, there will be lots of little things that you’ve never thought of before but that the literature takes for granted. Since most papers are written for specialists by specialists, crucial details are often left out or glossed over as if they were obvious. Just as fish don’t realize that they’re in water, specialists often fail to realize that they’re taking a lot of things for granted. The reason that reading many papers can help is that different specialists will leave out different details. The key that you need to understand paper X might be a seemingly throwaway comment in paper Z!
Explain It to Someone Else
The best way to understand something is by trying to explain it to someone else. This holds true even when the “someone else” in question is just a figment of your imagination. As you read, start by trying to explain the paper to yourself in your own words. I find it helpful to write in the margins of the paper as I go, summarizing the key ideas with less jargon and simpler terminology and notation. When you’re confused about something, try to put your confusion into words; make it concrete and write it down.
Talking to a real person can be even more helpful. If you’re in a reading group, try discussing the paper informally with one your peers who has also read it. You may be surprised at how much two people, neither of whom understands something on their own, can learn from each other. In this brave new world of LLMs like Claude and GPT4o, you could even try uploading your paper and discussing it with an AI. You cannot assume that the AI will necessarily give you reliable information about the paper, but just like a peer who only partially understands it, an AI can be a useful sounding board for your own ideas and confusions. Noticing mistakes in the AI’s understanding, pointing them out and continuing the conversation can also be a great way to clarify your own thinking.
Head Straight for the Simulation / Empirical Example
Ideally every paper would have a fantastic introduction that makes it clear what the paper is about and why it’s important. In real life, introductions can be hit-or-miss. So after reading the introduction, you might consider heading straight for the simulation study and/or empirical example. Most econometrics papers propose a method that solves a particular problem. What is the problem, and why does the particular data generating process (DGP) in the simulation (or the real data in the empirical example) exhibit it? What parameters of the simulation DGP control the extent of the problem? What is the “old” method on which the paper improves? This is likely to be something familiar such as a “textbook” method. How exactly is the new method implemented? In other words, how exactly is it computed from real or simulated data? Try to write down all the steps in the implementation in a sufficiently precise way that you could code it yourself.
Once you know how to answer these questions you’re in a much better position to understand the rest of the paper. As you read through the assumptions and theorems, refer back to the simulation study. Why does the DGP satisfy the assumptions? Can you think of a different DGP in which the assumptions fail? Is there anything “fishy” about the simulation example? Does it seem like the authors have cooked the books in some way, e.g. by introducing a very “mild” version of the central problem, or something else that would be unrealistic in practice? Answering these questions will help you to evaluate the paper, understand its limitations and possibly think about how to improve upon it.
Make Things Simpler
Many econometrics papers present results at an extremely high level of generality. On the one hand this is a good thing. Much of the power of mathematics comes from abstraction and general results are more widely-applicable. But from an expositional standpoint, this is terrible. This history of mathematics is a history of concrete problems to specific problems that were progressively generalized and expanded over time. The history of ideas mirrors the way that the average person learns most effectively: by starting with concrete examples and then generalizing.
With this in mind, try to simplify the theorems and examples in the paper. Getting rid of covariates often cuts down on both algebra and notation, so start with this. Try re-writing the assumptions and theorems in this simpler notation. Are some of the assumptions confusing? Try strengthening them or try to see if you can find a concrete example in which they hold, possibly taken from the simulation DGP.
Don’t Get Hung Up on Technicalities
Some parts of a paper are “core material” and some parts are “technicalities”. Keeping these separate in your mind will make it much easier to understand a paper. One helpful approach is to make a dependency tree of the assumptions, lemmas, and theorems before trying to understand them. Once you see how things fit together you may notice, for example, that the only role of Proposition 3 is to establish that an appropriate Central Limit Theorem holds and the only role of Assumptions 2-6 is to prove Proposition 3. Fantastic! In this case, just assume the conclusion of Proposition 3 and move on to see where this is needed in the core results. Even when you’re reading assumptions, lemmas, propositions, theorems, and proofs, you should be aiming to get the “big picture” rather than to assimilate every tiny detail.
Be Appropriately Skeptical of Asymptotics
Asymptotics are a crucial tool in econometrics but remember that it is finite sample properties that we actually care about. The “asymptotic distribution” of an estimator is just a thought experiment, not something you can take to the bank. An asymptotic argument is a kind of approximation that in effect supposes that certain things are “negligible.” This approximation could be fantastic or it could be terrible. It’s only through simulation studies that we can really know which is the case. Or, to quote van der Vaart (1998),
strictly speaking, most asymptotic results that are currently available are logically useless. This is because most asymptotic results are limit results, rather than approximations consisting of an approximating formula plus an accurate error bound … This is why there is good asymptotics and bad asymptotics and why two types of asymptotics sometimes lead to conflicting claims … Because it may be theoretically very hard to ascertain that approximation errors are small, one often takes recourse to simulation studies
For an example of “good” versus “bad” asymptotics applied to power analysis, see this post.