Research Group Feedback

Jan 05 2011 Published by under advising, graduate school

A reader wonders how to get feedback from research group members; that is, to find out from one's grad students and postdocs how things are going for the group or for specific group members. Is there a good, systematic way to do this?: Have a group discussion? Ask each person for comments in individual meetings? Get written, anonymous comments? Set up a suggestion box?

Do any readers who are advisers/mentors have a particular approach they find constructive?

My own opinion is that the group dynamics as a whole should be open and friendly enough to encourage discussion  in a group or in an individual meeting of any problems that arise, but I have no system for acquiring feedback.

The types of information that would be useful can be divided into two categories: everyday kinds of issues and serious issues. Ideally, the research group is functioning well enough that the everyday issues can be openly discussed, or one or more group members could bring such issues to the attention of the faculty.

Serious issues are of course more complicated to discuss and to deal with. Serious issues may be ones that affect the entire group or might be specific to an individual. Even in the latter case, the entire group might be affected.

At various times over the years, I have learned, typically indirectly, that someone in the group is unhappy. For example, years ago, I kept hearing that one advisee was complaining all the time about me, about the research, about everything. I therefore asked this student in our individual weekly meeting, without saying anything about having heard rumors that s/he was unhappy: "Do you want to talk about how things are going? Do you want to change anything? Are there things that I can do to help you more?" The answer to everything was no, no, no.

The first time this happened, I assumed that the rumors were untrue and that this student was actually fine and had been misunderstood; perhaps routine venting was interpreted as deep distress. But the reports kept coming: so-and-so is really unhappy, s/he blames you for not providing enough help etc. Nothing I could said or did elicited any direct indication from the student that anything was wrong; s/he continued to maintain (to me) that everything was fine, nothing needed to change. Yet the unhappy student continued to complain (to others) right up to the unhappy end.

If I had had an anonymous system by which advisees could complain, suggest, or criticize, would this have helped? I don't think so, in part because my group is not so huge that someone could be really anonymous. In being specific enough to complain or criticize in any useful way, the person's identity would become obvious.

This particular situation bothered me a lot at the time, but ultimately I decided that some people are determined to be unhappy, at least in certain circumstances. Some adviser-student relationships just aren't going to work out, even if the adviser is well-meaning. Perhaps the student will do well with another adviser and another research project or perhaps the problem is graduate school (or life) in general.

One thing that can be difficult for an adviser is to know how much to tell the group as a whole about attempts to solve a problem involving a particular group member. This relates to my point above that even issues that are related to one particular individual can affect the entire group.

In the situation described above, for example, I didn't want the rest of the group to think I didn't care or wasn't doing anything to help the unhappy student, so I told some of my other students about my efforts to discuss any problems with the unhappy student, but that all my efforts had failed. Perhaps some of these students (reasonably) concluded that I was ineffectual and, if I were a better adviser, I could have found some way to get through to the unhappy student and make things right, but it was important to me to show that I cared and had tried.

Over the years, I have at times marveled at the fact that some of the most successful research groups in my field are led by professor who really don't care whether their advisees are *happy* or not. These advisers somehow consistently produce successful students who go on to do well with their subsequent scientific careers. Other groups headed by professors who devote much time to devising individualized advising strategies for each student end up mired in complexity, drama, and woe.

At times, some colleagues and I have wondered whether the "factory" approach is somehow more effective because the focus is on the work, whereas the more "sensitive" approach encourages a focus on problems. According to this totally unsupported hypothesis, the range of personalities, backgrounds, learning approaches, life issues, interests, priorities, and sanity level of graduate students is so great that trying to adjust advising style for each advisee is impossible and causes more problems than it solves. (Discuss)

I prefer to think that it is possible to have a high-functioning research group somewhere between those extremes; that is, a group in which the adviser is not an unfeeling person who only cares about the "product" (data, papers, grants) and has a sink-or-swim advising philosophy, but instead is one who encourages independence, self-reliance, problem-solving, and a healthy amount of communication among group members.

Whether that goal can be realized depends in part on the advising abilities of the professor(s) leading the group, but also is affected by the other members of the research group. Research groups change with time, and therefore so do research group dynamics. Perhaps this variability is why having a system for obtaining feedback is a good idea, and I hope readers will share information about how they approach this in their own research groups: as advisers, students, postdocs, or other group members.

27 responses so far

  • rknop says:

    It's possible that the "factory approach" research groups succeed because only the goal-oriented, driven types go into and stay in those groups. They may not actually be soulless monsters *themselves*, but they probably don't need a lot of nurturing. They might bitch, but that's all part of the process to them. The people who have actual human feelings scatter out of those groups-- either because "they aren't serious enough", or because they realize that this is not a great place to be.

    On the other hand, if an advisor shows some sympathy, it may be harder to realize that it's a cruel group and it's time to get out. What's more, everybody will probably tell you that "factory professor" X is a hardass who brooks no humanity and drives you, so you'll feel justified if you find him that way, and realize that it's OK to switch. If your advisor isn't somebody thought to be a hardass by word on the street, then there may be perceived social pressure against "not being able to fit in".

    I do have to say that my observation is that a majority of graduate students complain, perhaps a lot, about their advisor by the end. I haven't done a systematic study on this, but anecdotally, a lot of folks (even some who continue to work with their advisor) seem to have perceived their relationships with their advisors as dysfunctional by the end. (I can think of some exceptions, though.)

  • GradStudentAbroad says:

    rknop writes: "I do have to say that my observation is that a majority of graduate students complain, perhaps a lot, about their advisor by the end. I haven’t done a systematic study on this ..."

    Well, he's in luck, because I and some fellow grad students have done a systematic study on this.

    We did a survey of graduate students at my institution and found that the following factors are significantly related to, and apparently causal drivers of, graduate student satisfaction with advising (roughly in order from most important to least important):

    1) Students who perceived that they "chose" their advisor (or primary research supervisor) were more satisfied (86% satisfied) than those who perceived that their advisor/supervisor was "assigned" to them (65% satisfied).

    2) Higher frequency of meetings with advisor/supervisor is better (daily: 91% satisfied; weekly: 84% satisfied; monthly: 70% satisfied; "rarely or never": 30% satisfied).

    3) The longer students have been in school the more dissatisfied they are. This was actually a pretty big effect, with 85% of first-year students satisfied, decreasing to 72% by the third year, with the decrease almost entirely in the "very high" satisfaction category. This effect seems to be largely independent of other, possibly covariate factors (such as the frequency of meetings).

    4) We found minor minor effects of gender and nationality (as a group, women were very slightly -- but statistically significantly -- less satisfied than men -- 74% vs. 78% -- and certain groups of foreigners were slightly less satisfied.)

    The first two factors are, to a great extent, within the control of the advisor and/or department -- and have large, statistically significant relationships to graduate student satisfaction for a large sample of students from my institution.

    This is not to say these are the only factors that make students more or less satisfied with their supervision; they're just the ones for which we were able to measure statistically significant effects using our survey instrument.

  • AL says:

    In answer to your question about how to get feedback from everyone in the group, a pair of computer science professors use a method described here. The gist of it is that they have frequent (but very short) all-hands status meetings, which gets them regular updates. These are supplemented by on-demand research meetings with individual students, which enable them to quickly intervene when problems arise on a particular project. It seems to work for them. Although the origin (and some of the jargon) they use in this approach originate from CS, it seems like it would be generally applicable in a lot of scientific and engineering settings.

  • Anonymous says:

    I am all in favor of the factory approach. Does that make me seem callous, soulless, un-nurturing...perhaps even male? Well so be it. But I have seen some of my colleagues get mired down in some unbelievably messy relationships with their students. We not their friends, parents, or therapists. We are their mentors, so let's keep it professional.

    I'm sure some of your readers will think the factory approach is rough for female or untraditional students (I would have thought so before I became a prof). But my lab is full of students with untraditional backgrounds and personal lives, who would have been excluded from some of my colleagues' labs. They can have five sets of twiblings as far as I'm concerned, as long as they do what they need to do to be successful.

  • BGS says:

    Perhaps students who are unhappy in a "factory" quickly leave out of pure misery, whereas a nurturing advisor makes the environment somewhat bearable, albeit still unproductive?

  • We meet once a week individually with our boss and then once a week for lab meeting. He will sort of probe us in the personal meetings to survey the scenery of the lab and using that information we chat about issues in our lab meeting. Also the lab manager serves as an extension of this non-covert intelligence program and she reports back issues to him. System works pretty well for us.

  • SALC prof says:

    I think a professional approach as suggested by Anon 8:55 is appropriate. In the case of Sci Prof's situation, I think I would have called the student out on the complaining at some point by saying something along the lines of "I have asked you what I can do to make sure that you are satisfied/happy with your graduate experience and each time you have said that you are happy/satisfied with the project and our interactions yet I have heard repeatedly for the last year from several other students that you are complaining about me and our project, could you please explain?" I have found that "Negative Nancys or Negative Neds" can poison a research group causing people to be less satisfied but more importantly less engaged in the work and therefore less productive. My job is to mentor students and do good science. When a student's issues and attitude contribute to a less productive environment then eventually they get to hear about it. But generally after I have tried a gentle approach first.

  • atmos_prof says:

    I have historically been, I think, a relatively nurturing advisor, or at least very tolerant of students' many issues. I have had mostly unproductive students whom I think I should have better kicked out (or at least changed advisors) earlier. I would like to move towards a more sink-or-swim approach with more serious early evaluation of whether it's going to work out. Don't know if I have the stomach to really do it. I am at least trying to be more careful before taking a student on, and overall taking fewer of them (spending more $ on postdocs).

    When I was a grad student I had a hands-off advisor who *never* discussed anything personal (in either my life or his, nor our working relationship, nothing) and whom I met with pretty infrequently. I was happy basically all the time - he gave me good advice when I really needed it, and the rest of the time didn't get in my way. There were certainly many moments when I'd have wanted more of this time and communication, but the fact that I didn't get it made me grow up faster.

    I would like to see a study of the relationship between student satisfaction with advisors and other measures of student performance. It's hard to isolate cause and effect but I think some weak students may blame their advisors for their own failings or at least expect more hand-holding than they should. I believe a PhD should be an indication of maturity but that we (I accept my share of blame) are prolonging the period of academic adolescence so that this is often not the case. Not to say that we can't all be better advisors, but I think the possibility of immature student whining should not be neglected in any discussion like this.

  • strigiformes says:

    I agree with rknop that the factory styled labs only attract a certain group of people, while the more sympathetic ones attract another. And it's really hard to do if the lab is relatively small.

    I also believe, partly from my own experience, that the longer you spend interacting with your PI on a regular basis, the more possible that a student would complain and bitch about the PI, no matter how in the end the student appreciate the PI. It could also just be a way which the student vent about their grad life, and PIs are easy subject for venting. I used to have long hours of discussions about my previous advisor with a labmate, but we never thought that the advisor is a bad person or a bad advisor, or that we were planning to leave the lab because of that.

  • My group has varied in size from 0 to 5 students, so there is no way that any sort of anonymous feedback could work. I meet with the group weekly (currently we meet together with another faculty member's lab group, since she also has a small number of students and our research interests overlap) and with each student individually weekly (and on demand, which often results in an additional meeting each week).

    I've had students with various problems over the years that have interfered with their work (depression, bipolar disorder, ADHD, … ). I'm willing to talk with the students about how to try to get work done while coping with their problems, but I direct them to competent professionals for more direct intervention. I don't have the skills or the personality type to do counseling myself. I am willing to share with them my own difficulties (with remembering names, with writer's block, … ), which makes it somewhat easier for them to tell me about the reasons behind their problems.

  • GMP says:

    It is certainly possible to err on the side of too much involvement and allowing for students who eventually won't work out to linger on for way too long.

    In my experience, it has happened only once that a student who initially had problems adjusting to the requirements in my group had a change of pace and attitude after a serious conversation and is now productive (albeit he's nowhere near being the star of the group). In all the other cases (more than 10) a student who needed a lot of hand holding ( tasks chewed down into trivial bits in order to get stuff done) or regularly substituted excuses for resutls and suffered from inexplicable continuous "bad luck" (a series of ailments, lost car keys, lost pets, lots assignments, dead relatives, broken alarm clock/cell phone/computer, ailments/lost pets/lost relatives of significant other...) and therefore required too much involvement in their personal life turned out simply not to be PhD material. The statistics that I can draw from my experience says that the signs are there right away -- it becomes obvious pretty early on who will work out and who won't, and additional hand holding and extra accommodation for all sorts of personal requirements don't do anything but just drain your energy and financial resources.

    I have a weekly group meeting and weekly 1-on-1 meetings. Additional face time as needed. I will do everything in my power to assure that a smart, motivated, and hard working student gets the techical help when needed, gets all the funds and equipment needed, is visible at conferences and through high-profile publications. I think these are my duties to the good students and this is the domain of happiness/ satisfaction of a grad student that I should be concerned about. I will ask a student explicitly, in 1-on-1 meetings, if they want to meet more or less, if they are doing well in their classes, if they want to go to certain conferences or workshops, if they need feedback on job application materials etc.

    However, it is NOT my duty to help a graduate student find the motivation they never really had or to instill the work habits in them (which should have been done in grade school) or to teach them about time management or, worse yet, about managing their personal lives. Being a grownup should be a prerequisite for graduate school. If a good and productive student is really sick or needs family leave, that should certainly be brought up so appropriate arrangements can be made. But I really really don't want to know about anyone's girlfriend/boyfriend problems, or issues with parents or roommates or siblings or pets or whatnot. As someone said above, we are not the students' parents or friends, the relationship should be professional.

  • It is important not to conflate a management style that nurtures the professional development of trainees with a management style that nurtures the personal development of trainees. As professors, our job is to nurture our trainees professionally. If they need personal nurturing, it is not my job to provide it, nor am I qualified to do so.

    In my experience, the most dysfunctional labs are those in which the PI behaves like a cheerleader/friend hybrid, and the most functional labs are those in which the PI behaves like an NFL head coach.

  • [...] Science Professor has an interesting post up in which she muses about how professorial management styles that might, on the surface, seem less [...]

  • Sarah says:

    I have a different theory about the "factories." The most functional research groups I've observed have a balance of first year students through senior grad students and post-docs. When that kind of group exists, everyone has both a mentor and a mentee. This removes some of the burden from the advisor and gives everyone in the group a range of mentor styles to chose from. If the person doesn't quite click with the advisor, there are others in the group to pick up the slack. And there is also always someone in the group who has recently gone through what you are going through to give you advice, suggestions, and who is intimately familiar with exactly what needs to be done to succeed. The two most dysfunction research groups in my department are completely unbalanced (mine with 3 post-docs and 1 senior grad student, the other with 4 senior grad students). The advisors of both of these groups are known to be difficult but I think there would be significantly fewer problems if these groups were more balanced.

    Every advisor will have the occasional person that they don't get along with, but every department has faculty that are known to be terrible advisors and it is speaks poorly of the way science is currently structure that these people continue to advise students.

    FSP- it doesn't surprise me that students who are having problems will talk to everyone else but you about it. Students are terrified of the prospect of a bad recommendation letter and in my experience will avoid talking to their advisor about problems until those problems simply cannot be ignored anymore.

  • I think this is a situation where the other members of the student's supervisory committee should be brought into play.

  • Anon says:

    How interesting to read all of these responses from PI’s about “keeping it professional.” I agree that the PI is not supposed to be your BFF, shoulder to cry on, or therapist. Yet I can’t help, while reading this, thinking of other posts written by some of the same people here about how scientists aren’t “brains on a stick,” about how the workplace needs to be more accommodating of people’s personal needs/lives. I guess those needs were obviously not those of the students….

    In my experience, grad students usually know a helluva lot more about the personal lives of their advisors than vice versa. So you don’t want to hear about my boyfriend? Fine with me. But please spare me the details of the latest virus your kid picked up at daycare, the tree house you just built for your daughter, or your frustration with the contractor who is handling your kitchen remodel.

    As for the topic of this post, a prerequisite for collecting honest feedback from your students is first convincing them that that’s what you’re truly interested in. It never ceases to amaze me how many people request “feedback” when all they really want is for you to tell them what a great job they’re doing.

    • GMP says:

      I agree there can be oversharing on both ends, but I also think there is a difference between small talk and too much personal information.

      For instance, a student mentioning she is going to see her parents for Christmas falls under small talk. Talking at length about the relationship problems with boyfriend is too much information . The advisor mentioning a tree house for daughter falls under small talk; discussing own marital problems or parents' illnesses with students is too much. Some people enjoy small talk more than others, I don't particularly, but I have some collaborators who do it with students at the beginning of every meeting to make the atmosphere more relaxed.

      And people can be accommodating of the lives of students without too much personal involvment. Student is having a child? Congratulations and let's see what can be done with the funding and how much time off you can get and how it will influence your progress towards your degree. Isn't that what you would expect your advisor to be focused on - your professional well-being?

      I think a good professional relationship between advisors and students is similar to the one between collagues -- the vast majority of my faculty colleagues know nothing about my personal life beyond the level of chit-chat, if that. My close collaborators probably know my kid count and perhaps their approximate ages, but we never talk about personal issues and we chit-chat only minimally.

      It never ceases to amaze me how many people request “feedback” when all they really want is for you to tell them what a great job they’re doing.

      This is human and holds for advisors and advisees alike. I think most students are not really prepared to hear what their advisors really think of them, just like the advisors probably aren't too keen on hearing everything that their students think of them. Any successful long term professional relationship necessary involves sugar coating and a lot of tact in how facts and opinions are communicated.

      • Anon says:

        The advisor mentioning a tree house for daughter falls under small talk….

        I guess that depends on how you define “mention.” I really shouldn’t have to use up part of my meeting time with my advisor listening to him drone on and on about a tree house or golf or whatever else he finds particularly absorbing about his life at the moment. My time is valuable, too, and if you don’t know enough about me to know that I share your outside interests, just keep it to yourself.

        I think most students are not really prepared to hear what their advisors really think of them, just like the advisors probably aren’t too keen on hearing everything that their students think of them.

        Agreed. The important difference here, of course, is that students are highly dependent upon the good will of their advisor, while the reverse is not the case. In this way the student-advisor relationship is unlike that of colleagues. But I’m sure that this fact is easy to lose track of when you are in the dominant position. Some—perhaps most—students cannot afford the luxury of treating their advisors like a colleague.

        • Charles says:

          The important difference here, of course, is that students are highly dependent upon the good will of their advisor, while the reverse is not the case.

          Yes and no. It is of course true that the advisor has much more power in the relationship than the student. But faculty are judged by the performance of their students. If none of the students in a lab do well, this is noticed, and hurts the advisor in many ways down the line.

          • Anon says:

            Charles, the point is that my individual opinion and/or experience with my advisor has almost zero impact on his future. Thus, he has little to lose by offending me with honest feedback. I, on the other hand, will depend on him for quite a bit long after I’ve graduated – I have a lot to lose and little to gain by telling him what I really think. Professors who think that considerations like these don’t heavily influence what they hear from students are horribly naïve.

  • GradStudentAbroad says:

    atmos_prof wrote: "I would like to see a study of the relationship between student satisfaction with advisors and other measures of student performance."

    In our survey mentioned in my previous comment, we actually did find positive correlations between student satisfaction with supervision and measures of student productivity (e.g. self-reported number of hours worked and papers published). However, as you point out it is extremely difficult (maybe impossible) to isolate cause and effect here.

    The Sigma Xi Postdoc survey found that a professional approach to supervision, similar to what is outlined above by several commenters (clearly defined goals and criteria for evaluation, accompanied by regular performance reviews and opportunities for professional training) was associated with both higher levels of performance and higher satisfaction with supervision among postdocs.

  • Pharm Sci Grad says:

    I think more than "factory" vs "not" you have to look at students on a continuum. What I needed as a 2nd year student and what I want and need now as a 5th year student are polar opposites. The key is to meet each person where they are and tell them where they need to get to. My labmates and I laughed at my boss' January email which said, "How was your holiday at home? About the paper..." but that's all I want from PI. I NEED PI to guide me on my scientific life, and my personal life I can handle on my own just fine, thankyouverymuch.

    The way we've always handled things in PI's lab is that the senior grad student is responsible for keeping the PI in the loop about lab group dynamics. I personally have gone to PI and said "The tech is causing problems and I can't deal with it on my own anymore." When I was a jr grad student, PI did nothing until tech left and then complained to me how tech worked for one year and "produced nothing." I avoided saying "I told you so" somehow. 🙂 The second time, I was a sr grad student and PI fired the tech the next month and had a new one in within 30 days. My jr. grad student thanked me, and I was less stressed even with more to do, so it worked out for the best I think. Positive feedback goes back the same way, through the senior student. It's a system that was strange to me at first, but it does seem to work for us.

    As for the "dislike" of the PI, let's be honest here: after working with someone for YEARS, you get to know them pretty freaking well. I know my boss, I know my boss' strengths and weaknesses. It is now a running joke between us how I will bring in my other "mentors" (i.e. "outsiders") to get what I need that I know my adviser isn't good at providing. It's not personal. But it only works if everyone understands that's just what I have to do for me, and it's not a reflection on PI.

    It's not a CareBears Picnic/Tea Party, but a little civility and empathy can go a long way...

  • Another thing to recognize is that the ratio of professional:personal interactions naturally decreases over time. When I began my graduate career, my advisor and I did not know each other well, and science and schooling dominated our meetings and conversations. Five years latter, we knew each other much better, and it was comfortable for me, for example, to talk to him about my career goals, goals of having a family, and how it all might mesh together.

    I can't say I would have liked it if he would have refused to engage me in any conversation that veered away from the scientific. After a multi-year advising relationship, I wouldn't call that "professional"- I'd just call it "cold".

  • AnonForNow says:

    I can’t say I would have liked it if he would have refused to engage me in any conversation that veered away from the scientific. After a multi-year advising relationship, I wouldn’t call that “professional”- I’d just call it “cold”.

    This exactly!

    I was part of a factory-type lab. In my fourth year, my mom’s cancer came back with a vengeance. She moved in with me and my husband, and we became her primary caretakers during the last 10 mo. of her life. This definitely had an impact on my work: I went from being one of the most productive people in the lab to barely squeaking by. In the beginning, I was in denial, but eventually I realized that I had to say something to my advisor about the situation. I tried several times to have that conversation with him, but it soon became clear to me that he knew something was wrong but just didn’t want to hear it / deal with it. I took a semester off after my mom’s death, returned to the lab, and finished my degree 1.5 yrs later. These days, I try my best to have as little contact with my PhD advisor as possible. I suppose I am one of the “successes” of my lab, but if being like my advisor is what it takes to run a successful lab, count me out!

    I agree with the other Anon above that you don’t have to be your students’ friend or therapist. But it’s ridiculous to pretend that one’s personal happiness does not affect one’s professional happiness and vice versa. The problem with factory labs is that they dehumanize people, and I am not a cog in a machine!!! Even big business has come to the realization that treating people like this doesn’t make sense. Maybe academia could join the rest of the world in the 21st C.

  • Han Aiwen says:

    As a grad student, my advisor was a mentor, a friend, and in many ways, almost like family. He was very supportive when my son was born, and with other problems in my personal life. Having this trust between us made our relationship more open, and he never hesitated to point out that I needed to work harder and get things done. Now I'm a successful independent post-doc. So "keeping it professional" is not necessarily the solution.

  • Chloe Lewis says:

    As a middle philosophy, one could be an advisor who is focussed on getting work done, but believes that reasonably healthy sane people do the best work. You don't have to paint pinstripes on the dynamo, but you do have to oil and balance it.

  • Grad Student says:

    As a grad student myself, I can personally say that the reason why a student would insist nothing is wrong is because they're simply scared of pissing off the adviser. It puts you in the position of the "needy student" and simply makes you more of a pain in the ass as someone who doesn't complain.

    That being said, that approach during my MASTERS degree has left me with a PhD sized thesis and 3.5 years of study to get it simply because I never said "no". Looks good on a resume having 8 publications out of an M.Sc, but honestly I'd have been happier getting it done in standard time to move on to a PhD program almost 2 years ago.

    Sadly, my institution doesn't grant PhDs.