Going in for the Kill

Nov 09 2011 Published by under academic etiquette

A reader wonders (original e-mail shortened/edited):

I am interested on your take on the etiquette of Q&A sessions during talks: who, if anyone, should ask critical questions? By critical I mean any question with a clear orientation of "I don't buy your results much, if at all, and I'm going to ask about a deficiency in your work to see if you will give in and agree with me."

I've seen undergrads ask these types of questions (direct quote: "I don't understand the overall point of your research") and it be considered a major gaffe, in part because the critique was unsophisticated; I've seen post docs hone in on a methodological weakness and be perceived as too aggressive and outspoken for doing so in a direct manner; and I've seen senior, tenured faculty really go in for the jugular and everyone just thinks they are being mean like always but no one really tries to call them on it or rein them in.

At a talk yesterday, there was a potentially major flaw to the results presented. The speaker did not come across as credible, and at the end of the talk a senior faculty member went right in for the kill.

The thing is, I agreed with him, but as a 2nd year Ph.D. student I don't feel like I could phrase a question so directly. This made me wonder how I COULD phrase it if I wanted to politely but directly inquire. My question is, how would you phrase this type of pointed, critical question and do you think it's appropriate for a graduate student to do so (considering they have more on the line than tenured faculty).

As a spectator at a talk, I enjoy a well-posed killer question, no matter who delivers it, but I think that everyone, from first-year students to ancient professors, can be most effective at asking these questions if the questions are simple and polite. These questions are most satisfying if delivered to a worthy recipient -- that is, someone who enjoys questions, who isn't vulnerable (e.g., an interviewee), and who might be able to provide an interesting response.

It's not so great seeing someone destroyed in an aggressive way by piranhas in the audience.

I want to mention here that I think it is great when students and postdocs ask questions after a talk (or during, if that is the culture of a department), so the question is not whether early-career academic people should ask questions, it's specifically about how to ask killer questions.

Although I don't think I have an inflated view of the awesome brilliance and cosmic knowledge of professors relative to students, I think the person who wrote the letter is right to wonder whether it is somehow different for students than for others to ask these questions.

I admit that I am bothered if a student asks an apparently rude or aggressive question that seems to be based on the assumption that the student has the necessary knowledge to tell someone their work is pointless or flawed. Maybe they do (in which case, I am less bothered), but if they clearly don't, they come off as jerks.

Of course, faculty can be jerks as well, particularly if the faculty member doesn't know much (or anything) about the research they are attacking. I am not so bothered if someone (professor, student, postdoc, anyone) with relevant expertise is a bit aggressive and asks a really good, probing question. The best questions of this sort, though, are politely and simply expressed.

You don't have to bend over backwards to be polite. I also find it annoying when someone has a really long, self-deprecating preface to try to soften the blow of what might be a killer question. You can briefly say "Maybe I missed your explanation of this, but..", but then go for it. Or just ask your question, but focus on the material, not your opinion of it.

I understand that, even though some visitors to departments are told that they will be speaking to a general audience that includes students and people from a variety of sub-fields, some speakers make no effort to provide the necessary information for most people in the audience to understand the talk. It's fine to call them on this, and students should ask what questions they want to ask (politely).

If, however, a student's intent is to be aggressive and tear down someone's work, rather than their presentation style, they should be quite sure that they know what they are talking about.

Is there a polite way for anyone to say "I don't understand the overall point of your research"? Perhaps. First of all, it might be better to phrase it as a question (but not "What is the point of your research?"). What did the student mean by that statement? That they were confused or that they thought the research was pointless? It's not clear.

For example, if the student was trying to say that the speaker did a lousy job of explaining the context of the research and wants to know why the research was done, it's perfectly reasonable to ask "Could you take a step back and explain the overall motivation for this work?" (or ask for specific questions being addressed, or ask if this work has anything to do with [something you think is relevant and more interesting, without saying that]).

If, however, the purpose of the question was to say "I think your research is not worth doing", then, as I said, the asker of that question should be quite sure that they know what they are talking about.

If one of my advisees asked what I thought was a rude question, I would talk to them to see if they knew how their question sounded. Some apparently rude questions are asked without any intention of being rude, and it's just a matter of some friendly, constructive advice to fix the problem.

Does this post have a point? Maybe, maybe not, but I hope others will leave comments and weigh in on the topic.

25 responses so far

  • GradStudentAbroad says:

    When I notice a speaker has left something important out / made a major error / overstated their claims / completely failed to explain what the point of their talk is, I often respond by asking a polite leading question to gently notify them that more / better explanation is needed and give them a chance to elaborate.

    In my experience, most speakers totally miss the point of my questions, and give meaningless answers. I find this sad, since I have often gone way out of my way to gently set them up to say something interesting (and maybe think more carefully about what they are doing). If I phrased my questions more directly, speakers would presumably get the point. For example, instead of saying "is it a problem that so many variables in your model are poorly constrained by measurements?", I could say "your model has too many experimentally unobservable and unconstrainable variables to be falsifiable or have any predictive validity". I usually avoid the direct approach, under the assumption that (a) it will not make me any friends, and (b) the speaker will get defensive and it will therefore not lead to any productive discussion. Also, (c) I am aware that I might have missed something important and my question might simply reveal my ignorance. Sometimes, I approach the speaker again after the talk, for a one-on-one discussion. Sometimes, however, I wonder whether I should speak up, especially when everyone seems to be totally buying into research that to me appears deeply flawed. Sometimes the emperor really has no clothes.

  • Here is how you politely state that the conclusions of a presentation do not follow from the data presented:

    If I am understanding you correctly, then blah, blah, blah. However, have you considered, bleh, bleh, bleh?

  • another anon says:

    I asked one of those simple polite questions AFTER a talk in the privacy of the empty room and had a grad student tell me that it was outside of my field so I should trust that he knows what he is talking about and I am wrong. It cannot be helped if someone refuses to be corrected on a basic/fundamental mistake, so I should have left it. Instead I came at it from a different tack (perhaps you could answer this other question instead and then I will see where my reasoning is wrong...). I think he eventually got the message without losing face. I think...

  • In my profession we're just jerks. In fact, it is impolite NOT to go for the jugular, and speakers are very good at taking those attacks and addressing them in ways that please the questioner. We terrify people from other disciplines, but that's just our culture. The focus is on making research bullet-proof so it gets published and it's nothing personal. We also like to be efficient so we don't tend to use many nice padding words or phrases just for the sake of politeness. But that's ok.

  • drdoyenne says:

    Aggressive, means-spirited questions make the questioner, not the speaker, look like a jerk. The audience will likely sympathize with the speaker. They recognize the difference between a question designed to destroy and one that is intended to initiate a discussion or clarify a point. I once saw a professor go after a student at a national conference; she was giving her first talk and was clearly devastated. The professor was not a regular member of our field. His point was very minor (statistical terminology), but he made it out to be a major flaw. Several of us in the audience quickly spoke up to stop this professor's tirade. I also made a point to talk to the student later to assure her that she did nothing wrong.

  • Pat says:

    Some of those who go in for the kill already know the answer. They just want the speaker to feel like shit. If the speaker is a jerk and overconfident with his/her data notwithstanding major methodological flaws, fair enough.

    But if he/she is inexperienced and you have a criticism, don't be mean. If he/she is not able to answer your first "polite" question in front of the audience, just meet the speaker after the talk to continue the discussion. He/she will definitely be more receptive to the criticism.

  • Anonymous says:

    Well, perhaps I have a different take on this than others, but I only support productive discourse and I am not going to ask a question for the sole point of destroying someone else. If the speaker is 'a mess' then I'm just going to leave it alone. If the speaker seems competent but has made an oversight, then I might say 'A limitation of what you have done is X. Some approaches to get around that could be Y and Z. Do you have any thoughts on this?' I'm not going to bring up a problem, without also having in mind a solution to the problem. And I do think that people who bring up problems and are not willing to elaborate further on what the next steps should be are not being constructive.

  • userj says:

    In my view, talks are not the place for to go for the jugular. Questions can perhaps probe at potential weak points but they should be short and simple so they are easily understood and replied to. In particular, detailed (rather than general) methods questions are out of place. There is simply not enough time to give the amount of methods that would be required to make everything absolutely clear in a talk without making the talk entirely about methods and therefore incredibly booooring. Talks are a way to describe in a general way the sort of work the person/lab does and then to summarize recent results.

    The other problem with detailed questions about complicated methods (or even the more convoluted questions about results) is that it is too easy to mishear-misspeak-misunderstand the question. Then it becomes a 5 minute duel of trying to clarify what is meant by the question and the answer between two people, both of which invariably come out unsatisfied. Meanwhile the other 300 people in the room are twiddling their thumbs and rolling their eyes because A) they already know the answer or B) the amount of detail being discussed is irrelevant to them. These types of discussions should happen 1-on-1 after the talk where it is easier to take the time to think, ponder, and reply, or even better in writing.

    As for undergrads wondering "what's the point of this research" they really should be asking their professors that question, not the speaker. I've done this myself. "So... prof, I understand that the speaker was saying Y, but how does that actually tell us anything about Z? What am I missing?" Asking a guest this in public is both rude and also irrelevant for 99% of the audience (just like detailed too-specific methods questions). It's common courtesy.

  • Stephanie says:

    Does every physical science department have one old crazy bastard who always gives the colloquium speakers a really hard time? That has been the case at the two departments I have been in, but those statistics aren't good enough for me.

    Science is hard. I wish there was some way to unwrap our egos from our science. It's ok for science to be hard---if it was easy, someone else would have already done it and therefore it wouldn't be science! If we were just science robots we would be all objective about it and not have our egos all wrapped up in the success of our research. Would science be better or more productive if done by Data-like androids? I'm not sure? Maybe the human element is actually very important to come up with the crazy ideas or make those mistakes that often end up leading to cool discoveries? Maybe we'd still be trying to figure out the Ultraviolet catastrophe if we were cylons?

    You may wonder, what is the point of this comment, how is it related to this post? If you ask me a question about that I will insult you to try and get you to shut up because this comment is all I've spent the last 4 years doing and I barely know my 3 year old because of my hard work on this comment so it's REALLY IMPORTANT that I be correct since it is my life.

    Hey, maybe making science more friendly to non-singular type scientists (ie, their whole life isn't science) will help somewhat extract the egos, since if you have lots going on in your life, being wrong in part of your science won't destroy your ego? Maybe not? Would that be good for science?

  • postdoc says:

    What userj said. I almost never find that enough detail is presented on models in talks for me to ask what I'm confident is a "killer" question. Another problem is the way my brain works: it invariably gets excited about a point, goes off on a 15-s, ponderous tangent, and then wonders what riveting and critical details the speaker might have imparted in the meantime. Because there's almost never enough detail and I'm never sure I heard everything perfectly, I do the "I might've missed this, but did you consider the potential effects of blah...?" To be honest, I probably more often say, "I'm sorry if I missed this, but..."

    The deeper question of "Why are you really doing this at all?" is so profound, common, and subjective that I never ask it. I mostly try not to go to those talks. When I'm in them, I usually have trouble paying attention after 10 min anyway and just want to leave.

  • BugDoc says:

    There is never a good reason/situation where "going for the jugular" is appropriate - this attitude is inhibitory to open scientific discourse. It is perfectly fine for anyone, including students to ask "can you comment on whether this alternative interpretation might be possible?" or "do you know of any evidence that excludes this other possibility?", etc.

  • Simjockey says:

    On a related note, what's the right way for a junior person to say, "you should've done this.", or "this comparison is not fair, you should be comparing with this instead."

    • Abby says:

      > On a related note, what's the right way for a junior person to say, "you should've done this.", or "this comparison is not fair, you should be comparing with this instead."

      I'd generally go with "Did you consider making this other comparison?" or "Why did you choose to compare with this and not $thisothercomparison?" Possibly even with "It seems to me that comparing with foo is more appropriate because ____."

      But I'm a lowly grad student, so I don't have good perspective to how that sounds to someone more senior.

  • Eli Rabett says:

    Why is anyone funding this nonsense?

  • anon says:

    Simjockey -

    For ANY person, junior or not, instead of saying "you should've-- ", or "you're wrong..", etc, goes against the advice of not sharing your opinion during the discussion. Instead, you could be more polite by saying "Have you thought of doing it this way?" or "Have you considered doing this comparison, instead, because...?"

    In many cases, speakers are invited to be there to present their work, and the audience serves as the host. Out of civility and respect, I agree with userj, who says that talks are not the place to go for the jugular. It's one thing to initiate an interesting discussion, but it's no fun for anyone to knock the wind out of the speaker who has presumably made a huge effort to be there.

  • Math postdoc says:

    Some people are more confident or quicker on their feet than others (without necessarily being better researchers). A very aggressive question can be hard to answer on the spot, and unfairly make the presenter look stupid or unqualified. For instance: "wouldn't it be much simpler to just do X, rather than all this complicated stuff you just explained?" may have the very good answer "we thought of that, but then we realized that Y, so no it wouldn't". But if X was considered and discarded a long time ago in the project, the speaker may have temporarily forgotten about the possibility and not be able to respond right away.

    So I think a little humility on the part of the questioner is always appropriate -- the speaker deserves benefit of the doubt. Say instead: "Have you thought about or tried X instead of the method you described? It seems at first glance like it might be simpler?"

    I also like what "anon" above said about being a good host/audience and being polite to the speaker, who has (hopefully) worked hard putting together a show on stage for everyone.

  • Jim Thomerson says:

    We had a situation where our chairman brought in a series of postdoc level speakers. He would let the presentation proceed along for fifteen minutes or so and then interrupt to ask, "Tell us, what hypothesis are you testing?" I don't think any of his invited speakers were able to answer his question. I never understood what that was all about. The chair and I were not on good terms at the time so I did not inquire into the matter.

  • In my profession we're just jerks.


  • Anonymous says:

    I posted above... there were several of us saying that "it is never appropriate to go for the jugular" however, now I remember a story, a story so awesome it is legend in my field. It was at the biggest conference there is: where everyone goes to meet-up. A postdoc is giving a talk. He's super over-confident and speaking way beyond his results. You can tell that the audience is getting a bit uncomfortable with his presentation style. The jist of the talk is that general-body-of-theory by famous person X is deficient and far better is his new-idea-Y. The second person to ask a question is famous person X. The question is so elegant and goes something like 'in your method Y, have you thought about a, b and c? and if you had, might you then want to modify your new idea, so much so that it would turn into general-body-of-theory? OMG. Classic 🙂

  • Everyone here seems to think that presenters are people with fragile egos who the audience have to be polite to, even if they are spouting the stupidest garbage ever.

    If a person has a basically sound idea, but has some minor flaws in the presentation, or if a slightly better test could be done, then gentle, polite questions of the from others propose are appropriate.

    If a speaker has just managed to waste the time of an audience of 50 by not having done the basic research to discover that the question they are addressing was answered by better experiments than theirs 30 years ago, then rather brutal "how does you work differ from so-and-sos?" questions are appropriate.

    Asking tough questions of people interviewing for jobs is fair game, as long as the questions are fair ones. So asking a postdoc halfway through their talk "what is your hypothesis?" seems perfectly reasonable to me—if someone has gotten that far into their talk without managing to state the problem they are working on clearly, they are going to be pretty hopeless as teachers or presenters of research.

  • Patricia says:

    Mean, go-for-the-jugular questions are never appropriate, but as a speaker my research has benefitted greatly from politely-phrased questions about limitations and alternative interpretations of our results. This is part of the reason why I like giving talks: to 'try out' new results and get feedback and additional suggestions for future work.

  • Tor Bertin says:

    As an undergraduate, I'm fairly well known for asking questions of speakers, though I've certainly never asked something aggressively. During a conference that I attended last February, one of the speakers gave a presentation on the use of occupancy modeling to understand wolf population dynamics in western Montana. However, because sightings were based on hunter interviews, there was a key covariate (hunter density) that would have affected the probability of detection in each locality that he did not mention in the formulation of their models. I asked, probably over cautiously, "is there a way to include hunter density in your model?" The speaker then said that they did include the covariate, but had neglected to mention it. He thanked me after the presentation, as he suspected that others were thinking the same thing, but did not ask. So long as you understand your limitations, I encourage more people to ask questions, as it's not just your understanding that's being improved by the answers.

  • assistant professor says:

    I disagree with the many people who think going for the kill is inappropriate. I think the best thing to do is for seminar participants to think of the most major _potential_ flaw of any research project. It is usually a productive exercise for both the questioners and the speaker. This is for two reasons: one, this usually gets to the core of the research in question, two it is a way for the speaker to prepare a defense when they will have to handle referee criticism (I am in a field where people rarely present published papers). Of course, you should still phrase your question politely enough, and you should not insist too hard when the speaker does not give a satisfactory answer. Still, it is appropriate to insist a bit in order to get an answer. Also, if you are a PhD student, it is appropriate to be more polite and insist less than faculty members, but you should still most definitely ask your question! The obvious reason is that as a PhD student, you have more to lose if you are in fact wrong, so you want to be more careful. On the other hand, asking an intelligent and relevant question as a student will improve your standing in the eyes of seminar participants (including the speaker, at least most of the time).

    • anon says:

      Using a tactful approach to point out a "major potential flaw" in a talk is not the same as going for the kill. To me, the latter is simply making a statement (instead of posing a question) using a dismissive tone. I've actually seen assholes do this, and it doesn't make them look smart or insightful, and definitely does not improve their standing, not to my eyes at least. It makes them look like assholes.