In 2019 U-M ranked 4th nationally in the number of NSF Graduate Research Fellowships received. This is a huge deal. Each NSF-GRF brings in $138,000 to the University: $102,000 to the graduate student over three years and $36,000 to the student’s graduate department/program. The NSF uses two broad criteria in evaluating NSF-GRF applications: Intellectual Merit and Broader Impacts. Intellectual Merit is the more intuitive to most applicants, so we spend more time talking about Broader Impacts in NSF-GRF advising. What I’ve learned from our efforts to make NSF-GRF applicants more competitive has much wider applicability than just NSF-GRF or even STEM scholarships. In particular, thinking about the NSF’s Broader Impacts criterion has given me a way to think of the values of diversity, equity, and inclusivity as integral to our scholarship selection and advising practices rather than ‘bonus’ categories. For those of you who don’t work on NSF-GRF, if you’ll stick with me through a somewhat narrow introduction, I’ll try to draw out these wider applications in my conclusions.
The NSF’s Intellectual Merit criterion essentially asks:
- Is this good science? (i.e. will it advance knowledge in the field in important ways?)
- Is this a good scientist? (i.e. well-trained and positioned to successfully complete the proposed research)
The Broader Impacts criterion asks:
- What is the broader relevance or utility of the research? (e.g. does it advance the national security, economy, or well-being of individuals in society?)
- What impact will this research have on the infrastructure of science as a social institution?
In fact, between 1992 and 1996 these were the four criteria used to evaluate NSF proposals. In 1996 these four were consolidated into Intellectual Merit (a and b) and Broader Impacts (c and d). Part of the confusion surrounding Broader Impacts remains it’s Janus-faced nature, looking both to the scientific and technological impacts of potential knowledge gained from research and the societal impacts of the research process itself. Examples of scientific and technological impacts might include results that can be applied in other fields of science and technology, that improve commercial technology, enhance national security, inform public policy, inform the broader public, or create collaborations between academic institutions, government laboratories, and industry research. Examples of societal impacts might include the training of graduate students and postdoctoral researchers, involving undergraduates in research, outreach and mentoring of future STEM students in K-12 schools, and especially broadening the participation of women and under-represented groups in STEM in each of the above categories.
For NSF-GRF applicants, the confusion surrounding these criteria is further compounded by the fact that they are phrased for the evaluation of research project grant proposals, whereas NSF-GRFs are given to individuals. For students already enrolled in a PhD program, the gap between researcher and research project may be somewhat narrower. For undergraduate seniors applying for the NSF-GRF along with their PhD applications - in other words, applicants who don’t yet know at which institutions, much less in which labs, they will be conducting their PhD research - the criteria require further elaboration and adaptation. Senior undergraduates must demonstrate the potential to fulfill these criteria in their future PhD research. In the case of Intellectual Merit this means demonstrating that the applicant is well positioned to flourish in her chosen graduate program in terms of preparatory coursework and research experience/skills, and likely to make significant research contributions to the field in her future research career. In the case of Broader Impacts, this means demonstrating commitment to the values of public engagement and increasing the diversity, equity, and inclusivity of STEM fields. If applicants have devoted considerable time and effort to public outreach and diversity efforts as part of their undergraduate training, NSF evaluators can reasonably conclude that the applicant ‘gets it’ and is likely to do so in her future research career.
When it comes to demonstrating commitment to increasing the diversity, equity, and inclusivity of STEM fields, applicants can do so in at least two ways. If they come from a background that is traditionally underrepresented in STEM, they may embody these diversity goals by their own participation in the field. Or, even if they do not come from an underrepresented background themselves, they may devote time and energy to outreach, mentoring, and support activities that are aimed at increasing scientific literacy and the diversity of STEM. Students who come from an underrepresented background can, of course, demonstrate both. They can articulate how they have participated and benefitted from such outreach programs, and how they will ‘pay it forward’ in their future research careers by mentoring and supporting future generations of STEM students. Conversely, they might articulate how they have struggled with and overcome barriers in their field, and how they will use their experiences to mentor and support others.
For applicants who do not come from underrepresented background, it is vital that they demonstrate understanding of and commitment to these values, and articulate how they fit into their future research career plans alongside their coursework and research activities - not as unfortunately necessary ‘baggage,’ but as important contributions to the ongoing vitality and health of science as a social institution.
Fortunately, there are many opportunities for STEM undergraduates and graduate students at U-M to demonstrate this understanding and commitment. The University has opportunities for public engagement, outreach to area schools, mentoring, tutoring, and support and DEI initiatives. If an applicant’s department or program does not already have such programs, there are numerous templates for creating new programs, or for revitalizing old ones through better program assessment and design.
The trick is that once the NSF-GRF application cycle begins, it is too late to engage in these activities. I came to this realization as we were discussing Broader Impacts in our September info session. With just two months before the deadline, all we can do is help students articulate and highlight the activities that they have done. This is where advisors play a vital role. We need to communicate the message to STEM students early on that their career preparations are a three-legged stool: coursework, research, and Broader Impacts activities. We can also help them come to a more articulate understanding of how these activities fit into their preparations to be future scientists and leaders in their field.
Thinking about the NSF’s Broader Impacts criterion has influenced how I think of these issues in other fields in at least two ways. First, it has led me to ask: “What would a Broader Impacts criterion look like for non-STEM fields?” The answer is fairly straightforward for other types of research in the social sciences or humanities. “What is the relevance or utility of this research to people outside your field?” In asking this question, I don’t intend to reduce the value of humanistic research to instrumental utility alone. “If the justification for the research is intrinsic and non-instrumental, how will you communicate this value to a broader audience and invite a more diverse and inclusive set of participants?” These are vital questions for future researchers in humanistic fields to answer effectively and convincingly. Struggling to articulate NSF’s Broader Impacts criterion has made me more articulate in talking with students about their answers in these fields.
Second, thinking about how the NSF explains diversity, equity, and inclusion as contributions to the ongoing vitality and health of science as a social institution has helped me articulate their value as integral to the scholarship selection and advising process. This counteracts the often unspoken assumption that DEI considerations are ‘add-ons’ to the values we ‘really care about’ in higher education. Several national scholarship rubrics that ONSF works with allow reviewers to assign bonus points to applicants who are first-generation students, come from underrepresented backgrounds, or have overcome significant socio-economic or personal challenges. However, this scoring system reflects and institutionalizes these considerations as separate and secondary to the scholarship’s primary selection criteria, goals, and values. How can we confront and change this classification of values?
In 2018 ONSF launched a new STEM Research Career Award for sophomores and juniors. The goal was to provide an inclusive alternative to the Goldwater and Astronaut Scholarships, which are the premier awards for undergraduates planning to pursue STEM research careers, but which require US citizenship. In developing our selection criteria, we included Broader Impacts alongside Academic Excellence and Research Experience. We then provided students with two ways to fulfill the Broader Impacts criterion along the lines of what I sketched above.
Here is the question on our application:
- Choose one of the following prompts: (1) As a leading public research institution the University of Michigan is committed to the public dissemination of scientific knowledge and to increasing STEM opportunities for students from diverse economic, ethnic, and educational backgrounds. Describe how you have contributed to these goals either on campus or through community outreach programs. (2) Describe any social and/or economic impacts you have encountered that have influenced your STEM education – either positively or negatively – and how you have dealt with them. (Limit 2000 characters)
Changing the U-M STEM Research Career Award to include Broader Impacts as a primary rubric category has changed the way that I advise potential applicants. Having specified their future career goals, I talk with students about their preparation in relevant STEM courses, research, and Broader Impacts activities. This has also pushed me to think about how we can institutionalize these values in other selection processes on campus. The most ubiquitous criterion for national scholarships and fellowships, beyond even academic excellence, is leadership. How can we use the NSF’s conception of Broader Impacts as care for the infrastructure of science as a social institution as a model for thinking about how DEI activities prepare students for future leadership and stewardship roles in other fields? How can we build this understanding into the rubrics we use for evaluation and the questions we ask in applications and interviews?