In the belly of the beast. NIH Early Career Reviewer Part 2: meeting

In a recent post I went through some of the benefits of the NIH Early Career Reviewer program, which was developed by the Center for Scientific Review (CSR) to train scientist with no previous NIH experience in the NIH review process. I was accepted into the program last year and just participated in my first study section meeting.

Detail from Raffaello's The School of Athens with the artist (in white)
eavesdropping on reviewers...

To preserve full confidentiality the NIH doesn't allow reviewers to speak of the proposals or the discussion outside of the grant discussion itself, even among each other, so there will be not details divulged or even specific references. The CSR has some excellent videos and tutorials in their applicant resource page and the discussion process is pretty much as shown in this video. Briefly, each application is assigned to three reviewers. Reviewers state their initial scores, then reviewer 1, the primary reviewer, sets the tone by summarizing the application and stating their critiques, then reviewers 2 and 3 can add their comments. If there is variation in the scores, as shown in the video, the discussion can focus on specific points and all members of the study section can chime in and ask questions or add comments. At the end of the discussion the chair summarizes the comments and the primary reviewers can confirm or change their scores. Their scores set the range for all other reviewers to vote, but anyone can choose to vote outside the range (above or below) as long as their declare their intention to do so. And that's how an NIH score is born.

It was a fascinating and eye-opening process and what I can say will reiterate what has been said to me. But these concepts were really driven home by the experience.

1) Study sections can be VERY DIFFERENT from each other: they have a specific focus and specific expertise, which fluctuates depending on the members and whether there are power groups. Part of the SROs job is to balance things out since study sections change or merge depending on how research progresses, but it may not always work. Knowing your study sections can go a long way for finding the right fit. I spoke to several people about their experience in different study sections and they knew that there were places where they wouldn't get funded.
- Look up the standing members. Do you know who they are? Are they people in your field?
- Go to NIH Reporter and look up the grants they funded in the past few cycles. READ the summaries. Do not just assume that because there was one grant that kind of sounds like yours, it would be a good fit.

2) If you truly think you are in the wrong place, move heaven and earth to MOVE YOUR GRANT. Make a request and justify it. Ask for advice from senior people. You can literally be Not Discussed from one group and funded from another. It is again a question of fit because the right people will have the expertise to understanding why your work in important and how it fits in the field. Excitement about the problem can go a long way to rescue a proposal with grantsmanship issues.
- Call the SRO, call the CSR. Get out.

3) Take a STEP BACK and look at your grant from the prospective of an educated tax payer or a science journalist. Why is this exciting? How will giving you money make the world better? Why should people be reading your work? Your reviewers will be good scientists, but may not have direct expertise in your specific project. You need to make them see the relevance (Significance), the novelty (Innovation), the sound hypothesis (Premise) and the feasibility (Rigor) of your work. If you hit all the right notes your impact will be high (or in reality as close to 1 as possible). In a funding climate where only 10-15% make it, it's not the question of whether your project is worth it, but rather whether you can make multiple people believe it is WORTHIER than others.
- It's a mantra I wrote down multiple times in my weekly planner for the month "Go big picture! Be clear! Be specific!". Counterintuitive, I know.

4) BE CLEAR! I'll double down on this. By "Be clear" I mean...If it's 5pm, people have been discussing grants for several hours and your Specific aim page is 30% acronyms, therefore not readable, you are at the mercy of the lowest scoring reviewer.  If it's 11pm and your reviewer cannot understand how the different molecules in your proposal are connected and how your experiments flow, your impact score may be lower.
- Leave the acronyms and abbreviations for the Approach and even there use them in moderation. Also, don't make new acronyms up.
- Generate as many diagrams and visual aids as possible to drive the message home.
- BE CLEAR! Be so clear that the undergrad in your lab gets it.

5) BE RESPONSIVE, DON'T BE A JERK. This seems like a given, but I think people have gotten used to pushing back to editors and reviewers on paper submissions. Suddenly, they'll believe that if you say "I know you told be X, but I don't want to do it and I'll do Y". Unless Y is groundbreaking new work, it's not going to fly. Be responsive to reviewers' comments, you only have one shot at resubmission. As part of the discussion of resubmissions the assigned reviewers have to comment on the steps taken to improve the application. Do not behave like you're too cool for school, because people will not buy it, if the grant does not hold up to scrutiny.
- Do as much as you can to satisfy reviewers' comments and justify things that you couldn't do in the Introduction. It will not necessarily work, but at least your review will not start with people who are annoyed.

As I wrote in the last post, I knew all these things, but for some reason I feel like I didn't. I rushed home to take a long hard look at my grant and make sure that everything was properly framed and explained. At the end you never know. My own grant was reviewed at the same time at a different study section and while I'm usually a basket-case on "study section day", I was much calmer. There was a definite sense of control because I could tell myself "This is what is happening right now to my grant." "This is how people are looking at things." There was also a sense of resignation because of the callousness of the whole process. Not that the reviewers were mean or dismissive, it was a great group and I felt review was deep and fair. It's just that the paylines are so low that you cannot get attached...even if it feels deeply personal for the applicant, whose academic survival may depend on it. If your goal as a reviewer is to be fair and make sure the best grants get funded, it cannot be personal. And that is the topic for a whole new post...