Militantly Ignorant

Feb 15 2011 Published by under reviews and reviewing

Consider these two examples of a certain type of reviewer:

A few years ago, I wrote a paper that added some new information and discussed new ideas about a phenomenon that was discovered by others decades ago and that has been much discussed in the literature. This phenomenon is related to the observation that purple kangaroos can leap extremely high. Before the initial discovery, it was thought that only green kangaroos could leap extremely high, but now it is well established that both types of kangaroos can do this. The early inferences, which were quite compelling, have been confirmed by observation.

In my paper, I wrote a few context-establishing sentences in the introduction, mentioning the high-leaping by purple kangaroos [citation] before moving on to set up the particular focus of the paper. One reviewer of the paper wrote in their review "What is the evidence for high-leaping purple kangaroos?" and went on to express great doubt that this ever occurred.

The reviewer was unable to get over his/her shock and disbelief about the purple kangaroo phenomenon and recommended rejection. The paper was initially rejected, but was ultimately published.

Another example: A proposal involving a recently developed but well-known (and trendy!) method -- the kind that you could only not know about if you had not read any journals and not gone to any conferences in the past 5 years -- got this review comment: "I have never heard of [that method] so I am not sure if this research is [doable/worthwhile]." The grant was awarded anyway; lucky for us the other reviewers were up on the topic and liked our ideas.

Such comments are not rare, although I thought these particular incidents were extreme. This post is not, however, a rant about how some editors and program directors must look under rocks to find certain reviewers (perhaps that is what it takes to find enough reviewers in some cases). Instead I want to muse about other aspects of the phenomenon of Hard-Core Ignorant Reviewers.

I know the answer to the obvious question:

Don't these reviewers know they are ignorant? No, they don't. Anything they don't know is not worth knowing, or doesn't exist.

but

Why don't these reviewers know they are ignorant? This is a rhetorical question. Nevertheless, I wonder if these people are never told that they are ignorant by anyone, or whether they have repeated evidence (director or indirect) of this but ignore this, as they do many other things. Both are likely. In the examples described above, the reviewers did not hesitate to admit their ignorance in their reviews, and they interpreted their lack of knowledge as a problem with my work. These people are very comfortable in their ignorance.

Which leads me to my real question: Can someone become like these reviewers, or is it an inherent trait that is evident early-on, or at least by mid-career?

Worrying about this would have been unimaginable to me in my academic youth, but as I get older and more established, I see more examples of situations in which I previously would have been held to a higher -- perhaps even an impossible or unfair -- standard of proof for statements I make or ideas that I propose. So perhaps encountering reviews from the hard-core ignorant serves a useful purpose of keeping me from becoming one of them. Maybe it keeps me on my toes and prevents complacency (?). This is a hypothesis. Feel free to reject it.

For me, the prospect of becoming like these militantly ignorant reviewers is one of those "Shoot me if I ever get like this" kinds of things. Or at least tell me. But would I listen?

19 responses so far

  • moom says:

    This seems related to the reviewers who seem like they didn't read your paper. We just got reviews back for a paper with this thesis:

    "Our previous research showed no evidence for purple kangaroos, therefore, erosion in the area couldn't be due to trampling by purple kangaroos. Others found evidence for purple kangaroos. We have now revisited the data and showed that there were no purple kangaroos in the period we examined before but there are now as the others have found. But now there is no erosion either".

    The referee said "the authors claim there are no purple kangaroos despite others finding them".

    We are going to appeak.

  • GEARS says:

    The ignorant reviewers hypothesis is the same as the ignorant authors argument. Most ignorant authors *forget* what it's like to review articles when it comes time to submit their own. Reviewers failed to do their literature search on the topic when asked to review, just like a good author should do before publishing.

    There are times when I know a decent amount about an article's topic when submitted for review but not enough to make a "yea" or "nay" call. That means, to do a proper review, the reviewer should search around and do their homework on the subject as well.

    Maybe after those instances there needs to be a mechanism where the authors can talk to the journal editor/proposal committee about reviewers not being qualified to perform a review. It's probably a long shot but I think most editors would take a smaller pool of reviewers who are all qualified than more reviewers who are only going to screw it up anyway.

  • GradStudentAbroad says:

    It reminds me of a colleague who once told me directly and in almost exactly these words: "I don't think much of the scientific quality of X's work. Of course, I admit I know almost nothing about the topic X is working on."

    My first though, though, was that these militantly ignorant reviewers must be fairly senior (or in other words old) and technology-averse, since for me the automatic response to such situations is use a scientific search engine to do a quick background check.

    • Bagelsan says:

      As a grad student, my first response is somewhat like yours -- do a quick background check -- fueled by a (healthy?) dose of terror that omgisthatathingIshouldalreadyknow?? My default assumption is that I missed something, and at the very least I like to check that possibility first before I embarass myself by "nuh-UH"ing in ignorance.

      In the age of Google and Wikipedia and Pubmed it's pretty easy to at least type in a new term and see if you get any hits before you start claiming X thing doesn't exist. I wonder if that impulse to google everything does break down along age lines somewhat, or if there are always going to be people of all ages who do think to wiki and just don't because screw you I know everything worth knowing!

    • GradStudentAbroad says:

      (Possibly not incidentally, the colleague I quoted above was a native born man, while the X he was talking about was a foreign-born woman, who was also lower on the academic totem pole. The comment was completely unprovoked and offered for no apparent reason. I spent some time trying to figure out whether it was simple all-around arrogance, or a particular prejudice against X related to her gender and/or nationality, but in this case I finally came to the conclusion that it was equal-rights arrogance, although the status difference may have made him feel freer about expressing it.)

  • EngineeringProf says:

    Wow. I have never experienced anything like these stories. Never. Really. Do I just live a sheltered, happy-go-lucky life? I guess so.

    My sympathies to you for such nonsense. I'd be pretty upset. Glad to hear you seem able to take it in such stride.

  • Anonymous says:

    I have a militantly ignorant colleague in my department. Now that I have tenure, I take great pleasure in pointing out his ignorance in seminars, discussion groups, etc. I doubt it sinks in - he probably just thinks I'm a raving bitch. But I suspect he has gone for years without having any kind of push-back for his woefully out-of-date ideas, so I am just doing my duty. πŸ™‚

  • Jim Thomerson says:

    Best review I ever received. " I am sorry to say I lost the manuscript you sent me for review. it was well written, an important contribution, and ideal for your journal. I recommend publication without revision." πŸ™‚

  • Han Aiwen says:

    I was once asked to review a paper in which I was one of the authors.

  • Anon says:

    My very first paper was savaged by an anonymous reviewer who had no idea that 'he' was highly ignorant about my topic. I was applying a well-established technique to a new problem, but he didn't believe that this technique was valid despite the 15+ references I had included in the background to document it. He was so sloppy in the reading of my paper that he believed that my point was to propose this 'new' method of measurement! It was completely insane. I was a grad student at the time, and I think this reviewer thought I was an easy target. I got the distinct impression that he was trying to teach me a lesson, somehow. Luckily for me, the other reviewer made no mention of these supposed issues and gave me a very positive review, the editor agreed with the positive review, and the paper was ultimately published.

    Having had this experience, I am now very careful to write reviews in a highly respectful tone regardless of what my final recommendation is. I find that this also helps me to slow down and give the author's ideas some serious thought, rather than just dismissing something I don't understand out of hand. I don't know what my reviews would be like without this experience--while I certainly wouldn't wish it on anyone else, I think it has probably helped me to be a more compassionate (and technically better) reviewer.

  • BugDoc says:

    Given the comments here and elsewhere that document at least anecdotally the existence of multiple militantly ignorant reviewers, I have to wonder why it is that (1) many graduate programs have no formal training for manuscript reviews (good mentors provide this for their students, but there is no mandate to do so in my program), and (2) there doesn't seem to be sufficient quality control to prevent such militantly ignorant reviews from happening on a regular basis. Of course there are also many excellent reviewers out there, and I'm profoundly thankful for them, but it would be nice if all reviewers were held to a certain standard.

  • DrDoyenne says:

    I unfortunately experienced militantly ignorant reviewers earlier in my career (not so much now that I'm more established). I always suspected that their comments were partly a reflection of a belief that they (probably senior, male) had to know more than I (junior, female). Thus, anything in the paper that they did not understand or had never heard of was due to my ignorance, not theirs.

    It would never occur to me, then or now, to question a statement in a paper I was reviewing, if I was not familiar with the specific method or topic. I would conduct a literature search and acquaint myself with the subject in question before criticizing (or just explain to the editor that I was not qualified to comment on purple kangaroos).

    One of my favorite quotes: β€œIgnorance begets confidence more frequently than does knowledge.” Charles Darwin

  • atmos_prof says:

    The only remedy for this is editors who know how to and are willing to filter reviews for reasonableness. Sounds from the original post and some of the comments (as well as my own experience) that this is what happens, maybe more often than not... so it creates headaches for authors but doesn't prevent the system from working, in general. It is a good skill for authors to learn how to politely but assertively refute such ignorant reviews. It is more of a problem with proposal reviews where one never gets the chance to do that directly.

  • Recently got a review from multiple militantly ignorant reviewers who kept on ranting about how solving problem B is easy; whereas we happened to give a solution for problem A.

    Funny part is that "we solved problem A" was the title of the paper πŸ˜€

  • bouncetwice says:

    I just had a paper rejected because a reviewer stated that the (very well-known) purple kangaroo equation did not include any physics about the "purple" bit. I didn't really understand how to reply other than to state that this equation was well-known for involving both purpleness and kangaroos. The reviewer replied with "There is nothing to do with purple in this equation. Therefore I reject this paper". And the associate editor said that he had no alternative but to reject it. A senior colleague in the department said that there would be no point in appealing the decision (just because apparently journal editors don't react well to appeals). So I did not appeal. Should I have? Neither I or the reviewer provided any supporting evidence for our point of view (although I will next time). Surely the right thing to do would have been to ask for either more information or another independent opinion.

    It was bad enough, but it was made worse by the condescending tone of the review and by the continual reference to me as "she", even though only my initial was given on the title page. I was the sole author. It was hard to not wonder whether the paper would have had a different response with a senior male name at the top, instead of my (junior, female) one.

    • GMP says:

      It was bad enough, but it was made worse by the condescending tone of the review and by the continual reference to me as β€œshe”, even though only my initial was given on the title page. I was the sole author. It was hard to not wonder whether the paper would have had a different response with a senior male name at the top, instead of my (junior, female) one.

      Oh, I have soooooo been there myself. A couple of the most dismissive and disrespectful reviews I received were like that, with the referee savoring the female gender pronoun, for papers where I was the sole (& female) author. I think your gut feeling is spot on -- while such reviewer may be a jerk to everyone, he is a particulary unyielding and dismissive jerk to a woman, especially a junior one (who then by definition could not possibly know anything).

  • David Gaba says:

    As a journal Editor-in-Chief I can agree with the comment that the "system" of review by multiple reviewers and editor(s) can nearly always cope with an aberrant reviewer, even one who "ought to know better." A few things for people to remember: a) most journals shoot for 2-3 reviewers PLUS the Editor in Chief or Section Editor. Occasionally we have to make do with fewer reviews (you can't force reviewers to actually do the reviews they agree to, and there are only so many alternates you can bring into the process). Thus a single reviewer rarely wins the day with an aberrant review; b) in many cases, the other reviewers and editor(s) will know the field well enough to make the aberrant person's opinion stick out as "ignorant" or "misguided". The editor may well make a comment in the decision letter like: "...Reviewer 1's belief that purple leaping kangaroos do not exist can be ignored, although other comments by Reviewer 1 about the XXXXXX are still relevant........"; c) if someone is ignorant but willing to learn, a common way to do this is to read the reviews by the other reviewers and editors and/or the editor's decision letter to the author --- good reviewers welcome reading these to see what they picked up that the others didn't and whether the editor agreed with them and used their pickups in the letter --- conversely they like to see what the other reviewers picked up that they missed. So someone who naively raised an issue about no leaping purple kangaroos might see that other people didn't have a problem with that and they MIGHT be able to learn from that they they are ignorant or wrong. But, you can lead a horse to water but can't make them drink so you never know; d) many journals score the reviews by reviewers, and periodically cull out poor performing reviewers (or reward with special accolades the best reviewers). Making a big issue about something that is flat out wrong would get a reviewer a bad score; e) in the event that a paper is rejected on the basis of a review that is flat-out wrong, I would advise the author to write directly to the Editor-in-Chief, marshalling the evidence and arguments that demonstrate why the reviewer was wrong about the literature, and ask for a re-review. Editors are usually pretty savvy, and if they see that a reviewer was just misguided on something like that that affected a decision on publication, the editor is likely to be more than happy to get a re-review by some other knowledgeable parties.

    So, the system does have means to deal with such aberrant reviews but admittedly nothing is perfect.

  • As to your question- Can someone become like these reviewers, or is it an inherent trait that is evident early-on, or at least by mid-career?-

    I hypothesize that the ignorant mindset is one that would develop early on. Even as a postdoc, I can see my fellow postdoc colleagues developing along the open-minded vs. I-know-it-if-it's-important lines. Some people readily embrace the fact that there is much to know in this world, and most of us know but a small slice. And others really do think they know everything worth knowing, because they are delusional.

    So, perhaps delusion ages and matures like a fine wine, but I think it's almost always there to start with.