Surprisingly there is very little information in the science education literature regarding efforts to define and assess the undergraduate research (UR) experience. Therefore, we have carried out a study of biology/chemistry undergraduates at universities and 4-year colleges across the country to examine from the student perspective what current UR research practices are and what constitute to student minds meaningful UR experiences. 320 past and present UR students were asked to complete a 68-question www-survey exploring characteristics defining the typical lab environment, student beliefs about scientists and science, self-perceptions about skills, abilities, and personal traits, and the desirable traits in an effective research advisor. 126 students (63.5 % women) from 21 states and Montreal, Canada completed the anonymous www-based survey tool. The majority of students surveyed worked in relatively small research groups (< 10 people) usually alongside other undergraduate students (< 15 % worked in labs with no undergraduate colleagues). Current UR practices appear to be largely effective from a student perspective - 98 % of UR students surveyed indicate that they would recommend UR to a friend. Students undertake UR primarily (74 %) in order learn on their own and because they want to help others not because they want to author papers, build a resume, etc. Overall, undergraduates feel that the UR experience provides them with technical experience, the ability to develop good problem solving skills, and acquire professional self-confidence.
Not surprisingly, the faculty advisor was found to play a key role in the UR experience. Student comments allowed us to identify several characteristics students believe faculty mentors need to have in order for the UR experience to be successful. In particular, a research advisor should be knowledgeable about the student's research project, enthusiastic, available, and patient. Students felt that these characteristics were of much greater significance than whether an advisor was intellectually brilliant, had external research funding, whether he/she worked at the frontiers of knowledge, or whether their advisor was famous. The relative importance of these characteristics was found to be influenced by gender and the size of the student's research group. Difficulties were noted to arise due to: 1) differences between faculty and student perceptions of the necessary time commitment; 2) difficulties students have in prioritizing the competing demands of classes, work, the lab, and life in general; 3) inadequate support (emotional, advisory, etc.) from the research advisor; 4) self-motivation; and 5) student fear of failure were also found to be significant impediments to student success in UR.
Introduction
The purpose of this paper is to share the first fruits of a www-based study aimed at capturing a snapshot of what undergraduate research looks like today from a student perspective in the fields of chemistry and biology at colleges and universities across the United States. My reasons for conducting the study were two-fold. First, I had a long-standing interest in undergraduate research having successfully involved twenty-five undergraduate and seven high school researchers in my graduate research group at Northeastern University over the past ten years. My interest has recently led me to consider developing a pedagogical framework for effective undergraduate research practices. Some of my own pedagogical strategies have recently been outlined in The Chemical Educator.(1) Secondly, I am actively involved in efforts to develop and implement a college and university-wide undergraduate research program on my own university campus. These interests led me to examine the education literature to learn what is already known about the undergraduate research experience. The Council on Undergraduate Research (CUR) publishes an excellent journal, The CUR Quarterly, that appears quarterly and describes various aspects of undergraduate research such as funding, initiating institutional efforts, faculty development programs, etc. This is an invaluable resource for any institution, department, and/or faculty member considering undertaking a quality UR effort at any level. This said however there is surprisingly very little information in the science education literature regarding efforts to define and assess the undergraduate research (UR) experience. The most significant work is a three-school survey that was recently briefly described in a National Research Council Report.(2)Therefore, I have carried out the present study in order to learn what current UR research practices are and what characteristics define from a student perspective meaningful UR experiences. My own interests and limitations on time and resources have regretfully necessitated limiting the study to an investigation of current practices in the fields of chemistry and biology.
It is also worth mentioning at the outset that this paper is itself is the product of an undergraduate research experience. Kristen Peters, a biology major, worked with me on this project last spring while she was an undergraduate cooperative education intern at the Northeastern University Educational Technology Center. The Center, led by Alicia Russell, teams talented undergraduates like Kristen with faculty who have an interest in adapting computer technology to their specific teaching needs. Kristen "translated" my paper-version of a 68-question survey-tool into a www-based survey tool that allowed students to anonymously complete my survey on the www. When a student respondent successfully completed the survey on-line, Kristen's program generated an anonymous e-mail message containing the student's responses that was readily assimilated into an SPSS data file for analysis.
Method
First, 59 faculty at institutions active in undergraduate research representing twenty-five states and Canada were identified. The majority of these were selected either because the individual faculty member's efforts in UR were well-known to me or because the institution's involvement in a successful NSF REU program was widely established. Next, e-mail cover letters were sent to these 59 faculty mentors describing our intent and requesting the faculty mentor's assistance in identifying students who either in the past or at present were involved in undergraduate research experiences in the faculty member's labs. Whenever possible we requested e-mail addresses as a means of contacting these students. Once faculty responded, e-mail cover letters went out to students identifying the project and requesting their assistance. Out of 59 faculty, 27 faculty (46 %) responded to my request and provided student names and addresses. 33 % of the faculty providing student names represented 4-year colleges and/or non-Ph.D. granting institutions. Cover-letters requesting student participation were e-mailed to 320 students. Students were provided a www-URL and were asked to complete an anonymous 68-question www-based survey that consisted of 68 questions and was divided into three main sections:
When a student completed the survey, an anonymous e-mail message containing the student's responses was sent to the PI. 126 students successfully completed the survey producing an e-mail file of their survey responses. This corresponds to an overall 39 % student response rate.
Both fixed- and free-format (narrative) responses were built into the survey. Items for which we were seeking general information on the respondents and the programmatic nature of the respondents' UR experiences were generally assessed using fixed format items. Opinion items were generally presented using a standard fixed-format five-point response scale (Likert) ranging from "strongly agree"(1) to "strongly disagree"(5).
Data were analyzed using SPSS for Windows v. 8.0. The chi-squared statistic was used to examine data for the presence of significant correlations between different variables. Following the usual convention, differences below the 0.05 level were taken to be statistically significant.
General Characteristics of Survey Respondents
Self-Described Ethnic Group |
Percent Representation in Bachelor's Degrees Awarded in Physical Sciences in 1995(3) |
Percent Representation in Survey |
Caucasian |
|
87.3 |
Asian |
3 |
8.7 |
African American |
7 |
1.6 |
Hispanic |
6 |
1.6 |
General Characteristics of the Undergraduate Research Experience
Percent of Respondents |
Length of Involvement |
19 |
1 quarter/semester |
27 |
1 year |
20 |
1- 2 years |
33 |
more than 2 years |
Percent of Respondents |
Academic Rank of Advisor: |
18 |
Assistant Professor |
36 |
Associate Professor |
36 |
Professor |
10 |
Rank Unknown |
Table 4. Ethical Dilemnas Experienced by UR Student Respondents
Type of Ethical Challenge |
Percent of Respondents |
assignment of credit |
3.2 |
confidentiality |
2.4 |
plagiarism |
1.6 |
fabrication/falsification of data |
0.8 |
Experiencing ethical challenges does not appear to correlate with student satisfaction of their undergraduate research experience. Students were equally likely to recommend undergraduate research independent of whether or not they experienced an ethical challenge during their project.
A Student Perspective on the Research Advisor
Students were queried regarding the characteristics that make for a good UR advisor. The traits we selected for study and student rankings of these characteristics are summarized in Table 5. Overall, student respondents felt that all of the traits we identified are important traits in a good research advisor. The most valuable characteristics from the student perspective appear to be the advisor's knowledge about the project, enthusiasm, availability and patience. Students felt that these characteristics were of much greater value than whether an advisor is intellectually brilliant, has external research funding, works at the frontiers of knowledge, or is well-known professionally. We believe this is important information for all current and prospective faculty to know. Students want an advisor who is interested in their professional development not on the faculty advisor's own career advancement.
Table 5. Ranking of Most Valuable Traits in an UR Advisor According to Student Survey Respondents
Trait |
Mean Scorea |
Standard Deviation |
Knowledgeable about project |
1.33 |
0.49 |
Charismatic (enthusiasm) |
1.41 |
0.56 |
Availability (time) |
1.66 |
0.67 |
Patience |
1.67 |
0.60 |
Previous experience in working with undergraduates |
1.79 |
0.69 |
Provides opportunities for presenting and publishing |
1.82 |
0.76 |
Good sense of humor |
1.86 |
0.79 |
Research funding |
1.90 |
0.73 |
Intellectually brilliance |
2.05 |
0.66 |
Nurturer |
2.31 |
0.79 |
Works on the frontiers of knowledge |
2.37 |
0.86 |
Professional reputation |
2.82 |
0.92 |
aScoring: 1 - strongly agree; 2 - agree; 3 - neutral; 4 - disagree; and 5 - strongly disagree
We examined these characteristics for gender differences to determine whether young men and women have different ideas regarding what makes for a good research advisor. Statistically significant differences were observed for only one characteristic, professional reputation (see Table 6). Differences were also observed for two other characteristics, specifically, whether the advisor works on the frontiers of knowledge and whether the advisor provides opportunities for publishing and presenting, though these did not rise to the level of being statistically significant (a = 0.1). In each case, women appear to value these characteristics more strongly than men. Perhaps this is because young women believe more strongly that the advisor's professional reputation will be of greater value to them in their own career advancement. We hope to learn more about this finding once we have completed our analysis of student data for self-perceptions about skills, abilities, and personal traits.
Table 6. Percentage of Males and Females Who Feel the Research Advisor's Professional Reputation is Important
Score |
Respondent's Gender |
|
Percent Female |
Percent Male |
|
Strongly Agree |
5 |
4 |
Agree |
41 |
24 |
Neutral |
43 |
30 |
Disagree |
9 |
35 |
Strongly Disagree |
2 |
7 |
The presence or absence of other undergraduate peers in the same laboratory also appears to strongly influence student views concerning the nature of the role of the research advisor. The presence of several undergraduates in the same lab was found to correlate positively with student perceptions of availability as being an extremely important characteristic in a good research mentor. With several undergraduates competing for the faculty advisor's time and attention, it is easy to see why students working in a lab with several undergraduate colleagues might place a much higher premium on an advisor's availability.
Some Observations Pertaining to the Peculiarities of the Small (1-5 Person) Research Group
Since the majority of students participating in undergraduate research appear to work in relatively small research groups (containing fewer than five people), it is worthwhile considering/discussing some of the differences we observed for this cohort of student researchers. First, though it did not reach statistical significance due to the very small number of students involved it is interesting to note that all of the very small number of students who indicated that they would not recommend undergraduate research to a friend worked in small research groups. One possible explanation is that this dissatisfaction is due to the difference in infrastructure in small vs. large research groups. Students working in very small labs are more likely to have to learn everything on their own and may have to make more of an effort to reach out for information than in larger labs where a student may more readily turn to his/her colleagues for assistance. This may translate into a greater likelihood of a student working in a small lab experiencing difficulties and consequently greater student dissatisfaction for this student cohort. Second, 50 % of those students experiencing ethical challenges were also observed to work in small research groups. Again, infrastructure issues may again play a significant role here. In addition, we found a statistically significant correlation between salary dissatisfaction and small group size where students appear to receive on average lower salaries. Most likely the lower salaries are due to funding issues with smaller labs likely having smaller and less well funded research programs.
The size of the research group appears to strongly influence student views regarding what characteristics make for a good research advisor. Two characteristics, the advisor as nurturing and available (time), were found to be strongly correlated with group size. Students working in small groups consisting of 1-5 people had significantly stronger views about the necessity for these two characteristics than students working in larger groups. Likely the correlation reflects the difference in student needs for students working in small labs versus in larger ones. Students working in small labs are less likely to have other people in the lab to turn to on a daily basis when experiments fail or when equipment becomes fussy. Consequently, students in smaller labs are more likely to be much more dependent on their advisors for assistance and to need to have more free (time and proximity) access to their advisors than students working in larger labs.
Student View of the Benefits of Undergraduate Research
Student respondents were queried regarding how useful the undergraduate research experience was in developing the variety of skills and benefiting from the opportunities listed in Table 7. Students consistently highly ranked all of the traits listed suggesting that undergraduate research students feel they benefit strongly from the UR experience. Students felt that the most valuable skills they derived were technical skills, problem-solving skills, and development of a healthy professional self-confidence. Gender played a significant role in student views. Women consistently ranked the development of professional self-confidence more highly than men suggesting that for women self-confidence is a more significant issue than it is for men considering a career in science.
Table 7. Most Valuable Skills Derived from UR Experience According to Student Survey Respondents
Trait |
Mean Scorea |
Standard Deviation |
Technical skills |
1.63 |
0.59 |
Problem-solving skills |
1.72 |
0.74 |
Professional self-confidence |
1.78 |
0.69 |
Realistic sense of career options |
2.09 |
0.89 |
Employability |
2.17 |
0.78 |
Networking opportunities |
2.26 |
0.93 |
Communication skills |
2.31 |
0.78 |
Interpersonal skills |
2.36 |
0.79 |
Prestige during college |
2.39 |
1.00 |
Publication opportunities |
2.44 |
1.10 |
Leadership experience |
2.69 |
0.96 |
aScoring: 1 - strongly agree; 2 - agree; 3 - neutral; 4 - disagree; and 5 - strongly disagree
UR Student Views of Science.
Previously, Michele Trankina investigated the effect of race(5) and gender(6) on attitudes toward science. To compare our findings to hers, we queried student respondents using the same four statements and a 5-point Likert scale. The overall results are summarized in Table 8.
Table 8. UR Student Responses to Questions About Confidence in Science
Statement |
Mean Scorea |
Standard Deviation |
Science will solve our social problems like crime and mental illness |
3.60 |
1.00 |
One of the bad effects of science is that it breaks down people's ideas of right and wrong |
4.17 |
0.79 |
One trouble with science is that it makes our way of life change too fast |
4.15 |
0.90 |
Scientists always seem to be prying into things that they really ought to stay out of |
4.41 |
0.74 |
aScoring: 1 - strongly agree; 2 - agree; 3 - neutral; 4 - disagree; and 5 - strongly disagree
Overall, undergraduate researchers were found to have fairly positive views of science. Students were most ambivalent concerning the first statement suggesting it is not clear to them if/how science participates in the resolution of social problems. This may actually be evidence of a greater problem and opportunity. In the narrative responses, a repeated theme voiced by the students was that UR is unique in that because it is the first time students have the opportunity to see applications. Given this prevalent view, it is not surprising that students have not yet made a contextual connection between science and its role in the solution of social problems.
Michele Trankina's study(6) demonstrated the existence of gender differences between men and women at all educational levels in their fundamental attitudes toward science. Women were found to generally have less favorable views of science overall. However, these differences were also observed to decrease at higher educational levels. Therefore, we examined the data to determine whether there were any gender-related differences in student attitudes toward science. We found no statistically significant gender differences for the cohort investigated here. Clearly, all gender differences are erased by the time that students have made a decision to pursue a career in science.
The Narrative Record
The final, narrative section of the survey consisted of five questions:
The narrative data supported the principal findings we derived from the quantitative data. In general, participants generally voiced comments highly supportive of the undergraduate research experience. The majority experienced a strongly positive UR experience as evidenced by their identification of a memorable UR experience. The single most frequently cited type of memorable experience was the opportunity to present at a regional, national, or international conference. Over 20 % of the students identified specific opportunities for presenting their research publicly as being their most memorable experiences. Several students indicated that they had presented at international meetings. The majority indicted that they had presented their work at national or regional ACS meetings or NCUR. Public approbation by an advisor, obtaining their first results were also frequently cited most memorable experiences. One student wrote:
"The second week into my research, one of my advisors set up a meeting with the director of the entire research center to discuss my project and the results I was getting. About a half-an-hour into the meeting, the director looked over at the postdoc and said "She'll be fine to handle this project on her own after you leave next week." My advisor was leaving, he had just gotten a job at a pharmaceutical agency and even though I had absolutely no research experience they were handing me the project. It showed that they had enough confidence in my abilities to handle this on my own, which was invaluable to my confidence as a scientist."
Consistent with these offerings, 36 % of the student respondents in the narrative section cited self-confidence as being the single most important benefit of their UR experience. Another 9 % identified their ability to develop and exercise good communication skills as being the most important benefit of undergraduate research.
Not all of the responses were however positive ones. A significant number of students used the narrative section as an opportunity to share their "war" stories and express concerns regarding difficulties they encountered as UR students. Several students described horrific experiences that included, for example, having an honors thesis rejected, chemical explosions, and inadvertently damaging expensive equipment. One student recounted her first day as an UR student as her most memorable experience. She was left alone working alone with a radionuclide (32P). Having no guidance, the student disposed of her contaminated gloves in the regular trash container. A safety inspector visiting the lab found high activity in the waste container and rebuked the student who felt unjustly blamed since she had received no instruction concerning the proper disposal of radioactive waste and had been left to work unsupervised.
Students also used this section to express a wide variety of concerns regarding what they believe to be the most difficult aspects of their UR experience. The most frequently cited concerns were 1) dealing with frustration when experiments did not go as planned; 2) time management; and 3) learning to do things on one's own. The first and last concerns can be attributed to the disconnect that students clearly felt between their previous experiences of science largely derived from the traditional classroom and the realities of doing research. One student wrote:
"The grad students were clueless and the professors were too busy with administration and other crap to get anything done. I was expected to learn too much on my own. I have a 3.8 GPA but they treat me like an idiot just because I can't figure out everything by myself. They tell me I don't think enough but they don't give me enough facts to think about!!!"
Another student wrote:
"I learned that experiments do not always work out. In the traditional academic classroom, experiments are designed to work; however, in a research environment, in which new theories are being tested, [they] do not always work out. An[d] that it takes a lot of knowledge of the subject to figure out what went wrong and where you should go next."
Frequently students expressed a concern that faculty expectations for time commitment were greater than students were willing or able to commit to. One student wrote:
"time - advisor worked like a madwoman 7 days a week 12 hours a day and seemed like she expected us to do the same"
More than half of the student respondents offered suggestions for choosing an UR advisor when answering the last question on the survey. Students were highly vocal regarding the characteristics that make for a good versus poor UR research advisor. One student wrote:
"You have to have a research advisor that makes sure you do everything from the beginning. Training is half the research experience. I did not get trained to write in my notebook and I tended to leave important info out. You also need an advisor that is looking to get you published. It shows that the advisor has enthusiasm about your research."
Another student wrote:
"In my opinion, undergraduate research advisors should be carefully selected for their teaching skills and availability and should be monitored while advising... For instance, I worked completely alone even on a different floor of the building than my advisor. I saw him perhaps once a week. He had me working on a daily basis with large quantities of acetonitrile in a lab that did not have a fume hood without ever once mentioning how hazardous it is to the liver when breathed or adsorbed through the skin. I practically bathed in the stuff!"
Student respondents working in labs alongside graduate students also expressed concern that graduate students need to foster an attitude of respect for their young undergraduate colleagues. One student wrote:
"I hope that grad students and post docs will learn to treat undergrads better because although they do not always understand everything that is going on, they are there to learn and should be respected for their will and determination."
Another student wrote:
"I would often be treated as someone who was not capable of working in a laboratory because I was very young when I started. People were disrespectful to me and did not treat me equally."
Clearly, we, faculty, are doing a great deal very well when it comes to undergraduate research. However, it is also clear that we also have much more to learn when it comes to being effective research mentors for undergraduate students. Based on our study, we offer the following suggestions for those who are or are considering becoming UR advisors:
Suggestions for UR Advisors
References
Acknowledgements
We wish to thank the faculty and students of Auburn University, Bates College, Boise State University, Brandeis University, Bucknell University, Eastern Illinois University, Holy Cross College, James Madison University, Kansas State University, M.I.T., Northeastern University, SUNY Binghamton, Rider University, Southern Illinois University, University of Kansas, University of Montreal, University of North Carolina at Greensboro, University of the Pacific, University of Southern Missouri, University of Washington, Wellesley College, and Williams College for partnering with us in this effort. We also wish to thank the National Science Foundation (MCB-9600847), the Northeastern University Center for Experiential Education, and the Northeastern University Educational Technology Center for their support of this work. This study was examined by the Northeastern University Human Subjects Research Review Committee (HSRRC #99-06-15) and approved as exempt, category 2 on June 23, 1999.
Copyright © 2000 by Patricia Ann Mabrouk, all rights reserved.