How to get started with academic literature
When I started my PhD back in 2003, one of the first things my supervisor asked me to do was write a literature review.
Wanting to impress my new boss, I decided that I was going to write the best literature review the world had ever seen. I even thought that if it was good enough I might be able to get it published; and what an amazing start that would have been! So I started reading and writing, but very quickly got overwhelmed by the amount of material that was available, much of which I didn’t really understand…
So I didn’t end up writing the best literature review the world had ever seen, and in fact that particular review never even got finished. This was an early knock to my confidence… it didn’t crush me, but it just fed that little voice at the back of my head that said I wasn’t good enough.
Much later I did write a very good literature review, but it was only after figuring out how to work with work with the literature in a more efficient and effective way.
So in this video I’d like to talk about why working with the literature is so stressful, and introduce you to some other ways of thinking about it.
So why is academic literature so hard to work with?
The most obvious reason is the sheer scale. Depending on your project, there might be 10s of thousands of sources, maybe even 100s of thousands or more, that are in some way related to what you do.
If there are 10,000 papers, reading at a rate of 10 per day, every day, it’ll take you nearly 3 years to get through them all. And that’s assuming you only read each one once and it’s not accounting for new work being published in that time (because in a research area where there are already 10,000 papers, there are probably more than 10 new papers being published every day).
And then of course you have the fact that each source you read references many others, and if you try to follow up on those the the reading list grows even faster.
Academic papers aren’t written to teach
So you might have to work with a lot of papers, but then each individual paper could be a nightmare to read. This might be because it’s badly written, or it could simply be because academic papers are written for an expert audience and they assume the reader has a lot of pre-existing knowledge.
A lot of students fall into the trap of taking notes and making summaries of papers, treating the literature as if it were an undergraduate course where you just have to absorb all the information.
This doesn’t really work because academic papers are not written to teach, and you need a certain level of knowledge to be able to engage with them effectively. The way to gain that knowledge isn’t by simply reading more papers—that just piles confusion on top of more confusion.
I’ll talk a bit about how to start building that knowledge in a few minutes, but understand that even if you do have a really good level of foundational knowledge, some papers are just really difficult to understand. Take any world-leading expert in any field and I can guarantee that there will be some papers they have to take time to fully grasp, and there’s no way to shortcut this.
Working on the edge of knowledge
Another reason why the literature is so difficult is that it exists on the edge of knowledge.
In an undergraduate degree, generally speaking, you will have been dealing with well-established knowledge; ideas that have been tested and refined and largely accepted by the field. In most areas of study, this core of established knowledge doesn’t change very fast, and you can just accept what the professors tell you.
But when you look at the academic literature, each individual paper is working at the boundary between knowledge and doubt (at least at the time of publication). One paper in isolation usually isn’t enough to prove something, and you might find different papers presenting contradictory results or arguments.
So you can’t just accept what any given paper says as true.
Not all published research is good
To make this more complicated, not all published research is any good, and you need to be able to judge the quality of the papers you read.
Even though every published paper has to go through peer-review, the system is far from perfect. Different journals have different standards, and some will accept almost anything, but even good journals can publish sub-standard work.
So much depends on who reviews the paper, and occasionally an over-worked or under-qualified referee will miss important mistakes.
Then, of course, there’s the issue of academic fraud, where some researchers manipulate or fake their results, which can be difficult to detect, especially if you’re reading too fast. Just last week I heard about allegations of data fixing against a high profile researcher at Harvard. There’s a great video breakdown of this by Pete Judo.
What this highlights to me is the importance of reading and understanding every paper you cite, and certainly anything that you use as the basis of your own work, because from what I know of this particular case the effect sizes being reported really should have raised some red flags much sooner.
So just because a paper is published, that doesn’t mean it’s any good.
The psychological factor
Finally, we have the psychological factor.
Most people who end up doing PhDs have done well throughout their whole education. And for many of us, doing well in education became a hard-wired part of our identity because it was rewarded from such a young age.
When you start a PhD and you’re faced with the tangled mess of literature, it might be the first time you’ve really struggled academically. The natural temptation might be to work harder, read more, take more notes… basically do more of the kinds of things that got you through your education up to this point, but when that doesn’t work it can have quite a profound effect on your confidence and self-esteem, especially when combined with all the other stresses of PhD life.
But if you’re struggling with the literature, it’s not a reflection on your intelligence, your ability, or your worth… you’re most likely struggling because, as I’ve said elsewhere, the skills that got you into a PhD are not the same skills you need to complete one, and the way you’ve approached studying in the past won’t work when applied to academic literature.
So what can we do differently?
Before I answer, it’s important to point out that there’s some nuance here, and your approach to the literature should be different depending on the stage you’re at, depending on your level of knowledge, and on what you’re trying to achieve.
For example, if you’re already very familiar with the literature and you just want to keep up with the latest developments, that requires a different approach to when you’re first starting.
Or, if you’re writing a literature review, that requires a different approach to searching the literature for a solution to a specific problem to help in your research.
So the first thing is knowing what you’re trying to achieve right now. Then it’s about selecting the best sources to help you achieve your immediate goal.
This idea of selection is crucial. Because your brain has limited bandwidth, instead of trying to read as much as possible, we want to identify a small number of sources that are most useful to you right now.
A lot of papers won’t be relevant, but then there will also be a lot of papers that may be relevant to your work, but aren’t helpful to you in this moment, either because of the current goal, or maybe because you aren’t ready for them yet. It’s OK to put these to one side and come back to them another time. You don’t have to read everything right now.
Getting started with Academic Literature
I’d like to give just one example here of how you might select papers based on a specific goal.
Let’s say you’re starting out in a new field and you really don’t know any of the literature. If you just download a stack of recent papers you’ll get lost very quickly. So we need identify a identify a small number of sources that are most useful to help you get started.
When you’re selecting papers, it’s important to understand that most research is incremental. Collectively that incremental work is important, but the vast majority of individual papers have little impact on their own. But, occasionally, there are papers that make major new discoveries or propose new theories or techniques that change the way that others in the field think or conduct their research.
If you understand even a just a few of these groundbreaking, paradigm-shifting papers, if you can understand what problem they solved, or what they did that was so innovative, or what discovery was so surprising, or what technical challenge they overcame and why it was important, it gives you a foundation for understanding the incremental work that followed.
But those groundbreaking papers might be quite difficult to read. They may assume a lot of prior knowledge, or they might not be clearly written.
Reading them again might help, but not if something’s just not explained.
In that case, we can look for other sources that explain the groundbreaking work a bit more clearly. These sources could be textbooks, review articles, even Wikipedia or Youtube videos in some cases, or you can just ask people.
Now you obviously wouldn’t cite Wikipedia or YouTube or a conversation in your writing. But if the goal is to build your knowledge, use whatever sources are most helpful to you right now.
Once you’ve got the foundation, you can start to look at the trends in the literature, the kinds of problems people are working on and the open questions being debated.
And then you can start looking at specifics, really diving into the details of some of that incremental work (although of course you’ll probably go back to the groundbreaking work many times, this isn’t a strict sequence)
Writing as a way to build your knowledge
There’s a common assumption that writing a literature review helps you develop your expertise, but I don’t think it’s the best way to go about it.
Coming back to what I said at the start about the literature review I tried to write at the start of my PhD… I think one of the big problems (other than my wild over-ambition to write the best literature review the world had ever seen) was that I was trying to write about the literature before I understood it, putting too much pressure on showing knowledge rather than developing it.
i was able to write a very good literature review at the end of my PhD when I had the knowledge to base it on, and when I understood the sources I was citing.
But this understanding didn’t just come from reading. I also had a lot of practical experience in the lab, using the techniques I read about in the literature (so the literature helped my practical work, but the practical work helped me understand the literature, too).
For example, by my third year I could quickly spot mistakes in some papers in the interpretation of data, because I’d done that kind of analysis (and made those kinds of mistakes) myself.
The other crucial factor that helped build my knowledge of the literature was that I was talking to other people in my research group all the time. I heard about their work and I heard about other things a little outside my area of focus that I wouldn’t have learned about just by reading. This gave extra breadth to my knowledge that fed into my writing.
