If you’re writing a term paper, a literature survey, or chapter 2 of your thesis and have carefully followed the instructions in my note about reference material, you’re now quite probably drowning in papers1. Which leads us to a problem statement:
Given a large collection of possibly-interesting papers, and limited time, how can you efficiently find the valuable ones?
The most efficient strategy I know is repeated rounds of triage. The idea is simple: you quickly evaluate each paper (without reading it in its entirety) and slot it into one of three bins:
Must read: Papers that are definitely relevant and useful.
Might read: Papers that may be useful.
Won’t read: Papers that are almost certainly not useful.
If you have enough papers in the must read bin, you’re done—get on with reading them in detail. If not, do another round of triage on the might read bin, moving the papers into must read, won’t read, and still don’t know. Repeat as necessary.
Simple, right?
Of course, this explanation is useless, because it glosses over the most important question: how do you “quickly evaluate” a paper?
You can efficiently evaluate a paper’s probable relevance and utility by skimming the parts most likely to contain useful hints. Ordered by the ratio of the number of hints you’ll likely find to the amount of effort you’ll have to expend reading them, these are:
Depending how big a triage problem you have on your hands, you may want to look at one of these in each round and do four rounds, or you may want to look at all of them in a single round until you have enough information to come to a decision.
Most technical papers include an abstract, which is supposed to be a short summary of the contents of the paper. If the abstract is well-written, it should give you a very strong hint regarding the paper’s probable relevance.
However.
Abstracts are often very poorly written—at least from your point of view. There are a few reasons for this:
Abstracts are short, short writing is hard, and many people simply don’t have the knack.
When the abstract is written, its primary intended audience consists of the journal or conference reviewers who will decide whether or not the paper should be published. So, abstracts tend to stress a paper’s novel contributions, sometimes at the expense of truly providing an overview of its contents.
You are reading the paper with a particular intent. Your intent is almost certainly not the same as the purpose the papers’ authors had in mind when they wrote it. For example, early in your reference search, you may actually be more interested in the authors’ discussion of related work than you are in the particular contributions of their paper—but the abstract will almost certainly say nothing about the related work.
Even taking all those caveats into account, the abstract is still the best place to start. It’s short, it’s supposed to be a summary, and besides, it’s right there at the front.
“Now hang on,” I hear you say, “surely after the abstract the next thing to look at is the introduction!” Well, no. The conclusion of a paper normally provides a summary of what has already been presented, but in a particularly compact form. Often, what’s in the conclusion is what really should have been in the abstract in the first place. And, since a conclusion comes at the end of the paper (hence the name), the authors have no incentive to over-claim or mislead; after all, anyone reading the conclusion can be assumed to have already read the rest of the paper.
Except you, of course, since it’s the second thing you looked at. Aren’t you clever?
A well written introduction will motivate the problem to be addressed, briefly situate the paper in the context of the related literature, outline the research reported and the paper’s main contributions, and provide an overview of the paper’s structure.
Unfortunately, well-written introductions are almost as rare as well-written abstracts, for many of the same reasons. But almost every introduction does include a section on the structure of the paper, normally in its last paragraph. Give the introduction a quick skim, and pay particular attention to this last paragraph, which is a bit like a mini-outline.
If you’ve managed to make it through the abstract, conclusion and introduction without being able to decide whether the paper is worth reading, there’s still one thing you can do to save you the effort of reading the whole thing.
In most papers, the first paragraph of each section explains what that section will contain. (Sometimes the last paragraph of the preceding section does this, so you have to be a bit careful.) If the paper’s authors have followed this convention, skimming these paragraphs will give you a very good overview of the paper’s contents, in considerably more detail than the abstract, conclusion and introduction. Obviously, it’s a lot more work to read all of these, but still a lot less work than reading the whole paper.
If you get through all of these and still can’t decide whether or not the paper is valuable, you can put it at the bottom of your “must read” pile and read the entire thing as and when you have the time. Sometimes this is the only way.
This web site is not an official publication of the Royal Military College of Canada nor of the Department of National Defence. Ce site web n’est pas une publication officielle du Collége militaire royal du Canada ni du Ministère de la défense nationale.