The NSF Graduate Research Fellowship:

A Missed Opportunity for Many

 

Daniel M. Boye, Davidson College, Davidson, NC

Shila Garg, The College of Wooster, Wooster, OH

Gerald A. Goldin, Rutgers University, New Brunswick, NJ

 

1. Introduction

 

This year the National Science Foundation (NSF) celebrates a half-century of awarding the Graduate Research Fellowship (GRF).  Through the years, the criteria for these fellowships have undergone significant changes, primarily through mandates from Congress.  Fine-tuning of the program literature has come through feedback to the NSF from the panels of judges.  But our experience on the panel that judges the Physics and Astronomy (P/A) applications leads us to think that the current criteria are not well understood by many applicants and their mentors.  It seems that some students who could be highly competitive fail to apply in the belief that the traditional measures of grades and Graduate Record Examination (GRE) scores are of overriding importance.  As a result, these individuals miss a career-defining opportunity.  A related concern is that, despite a talented and increasing number of applicants, P/A’s share of awards is actually declining.

 

For the past nine years, the Oak Ridge Associated Universities has effectively administered the fellowship competition including the panel process.  The P/A judging panel is comprised of 20-25 physicists and astronomers from a wide range of higher education institutions.  Each of us has participated on the P/A panel in different ways: Daniel Boye has been a member of this panel for the past 5 years and chair for the past two years.  Shila Garg has been a panel member for the past two years.  Gerald Goldin held an NSF-GRF as a graduate student during the 1960s, and served on the panel during the early 1990s as well as for the past two years.  After the 2002 panel meeting, all three of us came away feeling a strong need to communicate some of our observations to the P/A academic community.

 

We hope the information and advice in this article will encourage some students who might not otherwise do so to apply for GRFs.  And we hope it will aid candidates, and their mentors, in effectively describing strengths with regard to the newer as well as the more traditional judging criteria.

 

2. Overview of the NSF Graduate Research Fellowship: Some Statistics and Trends

 

The GRF is awarded competitively to the nation's best and brightest students to help support the early stages of their graduate study.  As stated in the program description, NSF "seeks to ensure the vitality of the human resource base of science, mathematics, and engineering in the United States and to reinforce its diversity."  With over 6,600 applicants this year, it is by far the largest competition of its kind.  The award is for three years of support and is to be used within a five-year period. The stipend accompanying the GRF for the 2002-03 fellowship year is $21,500.  US citizens or nationals, permanent resident aliens who are entering or have completed no more than a full year of graduate study are eligible. The high stature of the GRF results from the size of the award, the tough competition, the impact on graduate programs where fellows enroll, and the contributions that past fellows have made to the advancement of science and technology.  In many ways, the program sets the standards for other fellowship programs.

 

For the 2002 competition, there were a total of 376 applications in P/A, up 25% from two years before.  Of the total applications in P/A, 233 were from students who were in the process of completing a bachelor’s degree with plans to enter graduate school in the fall of 2002.  These 233 are roughly 9% of all (US and foreign) students entering their first year of graduate school in P/A in the United States, and around 15% of the entering students who meet the eligibility requirements for a GRF.  The number of applicants from Primarily Undergraduate Institutions (PUIs) was 18% of the P/A total, an increase from 12% from the previous year.  Nearly half of all applicants intend to enroll (or are currently enrolled for graduate study) in just seven institutions (listed in Table 1).  But the number of applicants attending these universities at the time of application was actually only 20% of the total.  Women comprised just 22% of the P/A applicants.  The question of representation from ethnic and racial minorities was an area of great concern to the panel.  Since 1998, NSF has not held a separate competition for Minority Graduate Fellowships, and we do not have the total count on minority applicants.  This year, however, there were only three applicants in P/A from Historically Black Colleges and Universities (HBCUs) and one from a Hispanic Serving Institution (HSI). We doubt that the total number of minority applicants could have exceeded 5%.

Institution

GRF Applicants in P/A

 Berkeley

35

 Harvard

29

 Stanford

28

 Cornell

25

 Princeton

23

 CIT

20

 MIT

19

 

Table 1. Graduate institutions most frequently proposed by applicants for attendance

 

Table 2 presents some data regarding the awards made by NSF over the past seven years.  Within the P/A panel there is a nominal distinction made between the subfields of physics and astrophysics/astronomy (denoted by Ast in Table 2).  The P/A panel does not typically judge applications in the subfields of biophysics and geophysics.  There is a disappointing trend in these data.  The number of fellowship awards in all disciplines has increased by nearly 18% during this period but the number of P/A awards actually dropped by 13%. The percentage of all awards allotted to P/A dropped significantly.

 

The total number of awards for the program each year is directly determined by the congressional budget for NSF.  This year there were 900 awards overall, with 46 of them in the P/A field.  Representatives from NSF indicate that the number of applicants in a field of specialization determines the number of awards that are given to that field.  Engineering fields of study receive a somewhat higher proportion of awards in relation to applicant numbers than do other fields.  The Division of Graduate Education of NSF has reported statistics for the GRF program for the past six years at the website www.ehr.nsf.gov/dge/programs/grf/grfstat.asp.

 

The panel in each discipline chooses a subset of applicants who will definitely receive awards.  A second subset, designated and rank-ordered by the panel, will either receive awards or will be distinguished with honorable mention.  For this subset, the panel’s ranking as well as the geographical and demographic distribution of the entire applicant pool of the program plays a role in NSF’s final assignment of awards.

Year

 

Program

Awards

Physics

Awards

Ast

Awards

Phys/Ast Total

Awards

% Program

Awards

96

765

39

14

53

6.9

97

850

40

13

53

6.2

98

766

35

13

48

6.3

99

900

42

10

52

5.8

00

850

37

12

49

5.8

01

903

38

12

50

5.5

02

900

34

12

46

5.1

Table 2.  Trends in the Physics/Astronomy Award

 

The decrease in the number of P/A awards suggests that qualified students should be more aggressively encouraged to apply.  Other fields of study have seen tremendous growth (greater than the 25% shown by P/A) in the numbers of applications due to proactive strategies - some research groups in other areas even require that all of their incoming graduate students apply!  We do not suggest or encourage such extreme, undiscriminating tactics for obvious reasons of educational responsibility and program integrity.  However, we do think that P/A faculty should be seriously engaged in finding and encouraging viable candidates for the GRF.

 

As noted, women and minorities remain seriously underrepresented in our field.  It is well known that women’s representation in physics decreases with each step up the academic ladder and that physics is not attracting from the increasing pool of women degree holders (AIP Report, Women in Physics, 2000).  We were able to gather some data on the successful women applicants in the competition.  Table 3 shows a slight increase in the number of female awardees over the past six years, but the overall representation of women in physics is quite low.

 

Year

 

Physics Awards

Women Awardees

Ast

Awards

Women

Awardees

Total P/A Awards

Total Women Awardees

97

40

5

13

2

53

7

98

35

5

13

3

48

8

99

42

9

10

2

52

11

00

37

6

12

2

49

8

01

38

8

12

7

50

15

02

34

7

12

6

46

13

Table 3. Statistics of Women Awardees

 

3. Merit criteria for the GRF

 

NSF has two main criteria for selection in this competition as stated in the program description: (i) intellectual merit and (ii) broader impact. The former criterion includes traditional measures that are probably familiar to most students and faculty: the applicant’s proposed plan of research, previous research experience, academic record, reference reports and GRE test scores.  The latter criterion includes three explicit components: effective integration of research and education, potential contributions to diversity, and contributions to community. 

 

Early in the work of the panel, different aspects of the criteria are presented and discussed.  A short “calibration” exercise and the resulting discussion are used to illustrate and make concrete the myriad of factors to be considered.  There are no specific directives to the panelists as to how to weigh these factors, or how to balance the two main criteria with each other.  Individual panelists must do this, developing in their minds and applying a consistent and graduated system of professional and personal values.   Scoring by individual panelists and deliberations of the panel at important break points narrows the field to those recommended for awards and honorable mentions.

 

i. Intellectual merit criterion

 

Panelists are asked to evaluate the merits of each applicant based on intellectual achievement and ability.  While the most easily quantified measures such as grades and GRE scores remain important to panel members, it is clear that these are not the sole or even the determining factors.  Panelists look also for evidence of original thought, creativity, and depth of commitment to advancing science through research.  The applicant should demonstrate ability in other accepted requisites for scholarly scientific work such as communicating effectively through writing and formulating careful statements of research plans.  A student with modest GRE scores, but who is highly distinguished in other ways, has an excellent chance for a fellowship.

 

The proposed plan of research reveals what the “hot topics” are in today’s young scientists’ minds.  This year many wanted to work in materials science research areas including carbon nanotubes and quantum dots.  Several proposed to work in particle physics, including CP-violation related research.  String theory drew some of the prospective theorists, and quantum computing was a recurring topic.  In astronomy, gravity wave detection and cosmology led the list.  The identification and formulation of a scientific question and the description of how the applicant would go about addressing the question are important features of the proposed research plan that reveal a lot about the applicant’s thought process.  Thus, the applicants should try to focus on one, or at most two, research questions, going into depth about how he or she would address them.  Last year NSF rephrased the instructions regarding the proposed plan of research and removed language that suggested applicants needed to write about more than one project.

 

Under the category of ‘previous research experience,’ it is a good idea for applicants not only to state what was accomplished but what they learned personally - even if their experiment or project was unsuccessful.  All else being equal, panelists sometimes looked favorably at applicants with research experiences extending beyond their own campuses.  This can be an indicator of the self-motivation to gain exposure to the broader P/A community and to experience a variety of practices and research tools in areas of scientific interest through guided research.  Some Research Experiences for Undergraduates (REU) sites provide training in scientific writing, scientific ethics, career planning, or other topics, in addition to the actual research projects.

 

At this level of competition, it is of the greatest importance that every bit of writing by the applicant be clear, concise and free of grammatical errors.  Applications that were not proofread carefully were looked on more negatively.  Some students had apparently not done the research to learn what projects were being pursued at their intended institutions – an obvious, important component of a good plan.  A valuable practice can be for a department to recommend a mentor to the applicant, someone who can make suggestions for presenting the applicant’s material in the most effective way.  The application process itself can be an important component of professional training and warrants guidance at this level.  A mentor can give some editorial assistance but more importantly can discuss what it takes to give sensible direction to the student’s ideas.  Of course, there should not be an undue level of substantive influence.  Ultimately, the application must represent the student’s own best efforts.

 

ii. Broader impact criterion

 

For a large number of applicants, the broader impact criterion was the decisive factor.  The panel sought not only to distinguish among students of equally demonstrated intellectual merit but also to find students who truly distinguished themselves through educational activity, service, social commitment, and outreach to the wider community.

 

But the questions related to this criterion were the most misunderstood by applicants and their mentors. The “effective integration of research and education“ does not mean studying a particular topic from the book, and then seeing it “in action” in the lab, nor does serving as a grader or teaching assistant in one’s department demonstrate excellence in this regard. Panel members looked for original, self-motivated contributions by applicants to science education, such as the development of new or innovative teaching materials, significant volunteer work with science in local schools or an extraordinary level of departmental service.  “Potential contributions to diversity” refer to increasing the diversity of the US population entering science or knowledgeable about it, not to increasing the diversity of the applicant’s scientific or other interests (an unfortunate but recurring misunderstanding).  Helping one or two minority or female fellow-students after class hardly constitutes real distinction here, and being a minority applicant does not automatically fulfill this criterion.  The panel looks for impact—e.g., taking science to underrepresented groups in the population through work with public or independent schools, club activity, college- or university-based programs, or summer work. Initiating science activity and effective advocacy for science education are highly valued.  A minority applicant might not only engage in such activity but also, through it, serve as a role model to attract others toward scientific interests.  Likewise, “contributions to community” may include organizing or working with department-based initiatives, with science museums, or with students through independent programs.  Applicants and their mentors should think in terms of making a real difference in the lives of others.

 

Some university Physics and Astronomy departments now offer the option of specializing in Physics Education Research (PER) for doctoral work. The P/A panel and NSF value and encourage applications by students who wish to pursue PER. Until now, there have been extremely few.  Here we would expect great strength on the "impact" criterion, demonstrated intellectual excellence, and an appropriate, carefully developed plan of research that indicates exceptional knowledge of the field.

 

 

 

4. Reference reports

 

The reference reports are crucial!  Panelists take both the letters and the rating sheet very seriously in determining the rank of an applicant.  But, referring faculty members, being perhaps overworked, do not always take the time to write letters that contribute meaningfully to this very important part of the application.  The panelists are looking for information that would tip the balance one way or the other.  A panelist needs specific information that gives a personal sense of the applicant—more than the fact that he or she is an excellent student, or did very well in one’s advanced E&M course.  The most helpful letters provide real insight into what makes the student stand out among the best, what the student has been able to accomplish as a researcher, how the student has gone beyond just being enrolled in the class, and especially how the student thinks about physics.  The least helpful letters were those that rated the student as outstanding or “1” in all the categories but provided just a few sentences that conveyed little in the narrative portion of the letter.  Sometimes the letter was very revealing but the ratings did not correlate with the letter.  We urge faculty members to think about what is useful to communicate in such a letter and to be consistent in the rating and the narration.  The focus should be on an evaluation of the individual applicant relative to the criteria for the award and not on justifying or promoting a particular area of research.

 

Most applicants had two or three research experiences at or away from their home institutions but some applicants did not get letters from their research mentors.  If a student works with a faculty member on a research project, it is very important for the research advisor to write about what the students strengths were—scientific potential, independence, creativity, thoroughness, experimental skills, perseverance, etc. As students go off campus to do research (such as participating in an REU program), they should be advised to get to know their faculty mentors well, to the extent that they can rely on their providing truly useful letters of recommendation.

 

5. Conclusions

It is gratifying to see that there are so many excellent P/A students and that the stewardship of the future of physics and astronomy in this country is in no danger.  But our experience as judges leads us to believe that more students, with more diverse strengths, should be encouraged to apply for these prestigious fellowships.  Faculty mentors can provide better guidance to students in the preparation of their applications.  There are no fixed recipes in which a certain combination of the ingredients insures success.  But, when we identify the strongest students, encourage them to put themselves forward, and help them present themselves most effectively, we not only benefit the students but strengthen the wider physics and astronomy community.

 

 

End Note:  For the 2003 competition, application material will become available at the NSF GRF website (www.ehr.nsf.gov/dge/programs/grf/) around the first of August and the application deadline will be November 7, 2002.  In addition, the pool of potential panelists always needs refreshing.  Scientists who would like to become involved should either call toll free 866-353-0905 or email nsfgrfp@orau.gov.