|
The Crucial Task of Training the Readers by Cathy Cockrell and Jesus Mena, Public Affairs The applications of some 30,000 high school students have arrived at Sproul Hall. Now comes one of the campus’s most critical and complex tasks: choosing which 8,250 individuals will be offered freshman admission for fall 1998. This year, from Jan. 9 to March 31, a team of 52 highly trained readers is using a new admissions policy, developed by a special faculty committee, to tackle this challenge. But who are these readers, and how do they prepare for – and carry out – this difficult assignment? The group includes 31 professional admissions or outreach officers in the Office of Undergraduate Admission and Relations With Schools (OUARS), 10 part-time readers, six volunteers from other units on the campus, and five Bay Area high school counselors who read applications as paid interns. Rigorous reader training – continually reinforced by quality control – as well as group “norming” sessions, guidance by the most experienced readers (who lead and give support to five- or six-member teams) and, especially, faculty review are central to the process. “The faculty must participate in training readers, and by training I mean communicating to them our understanding of the values embedded in the new system, the kinds of criteria we believe are legitimate and those that are not,” said Jack Citrin, professor of political science and a member of the Academic Senate’s Committee on Admissions, Enrollment and Preparatory Education (AE&PE). “This interaction between faculty and readers is critical to the new process and will be an ongoing feature of the new system,” added Jenny Franchot, English associate professor and AE&PE chair. New Reader OrientationReaders began preparation for the admissions process months before the applications deadline. Those readers recruited to temporarily augment the OUARS staff attended an intensive two-day orientation, at which they studied the policy developed by AE&PE and the faculty admissions committee. Designed to give readers both essential procedural information and a perspective on their role, the training included topics such as UC systemwide eligibility standards and admission requirements as well as background on the admissions policies established by the UC Board of Regents and the Academic Senate. What followed was a 10-hour introduction to evaluating applications, including how to assign each applicant an Academic Score and a Comprehensive Score. The new admissions policy requires each application to be reviewed independently by two readers. Each reader gives a score for academic performance and another comprehensive score that includes academic and non-academic achievement. Readers learn how to assign an Academic Score, which ranges from 1 to 7 (with 1 being the highest), by evaluating standardized test scores, grade point averages, academic records, college preparatory courses, and the rigor of the applicant’s high school course load. But the faculty also recognizes that grades and test scores alone don’t completely capture “vital things about a high schooler’s true experience or merit,” said Franchot. “Consideration of school and family context is crucial to evaluating performance and potential.” “If the student has won at the state science fair,” said reader Mary Dubitzky, OUARS assistant director, “that’s academic information that I want to consider.” Training leaders discuss topics such as the impact family or social circumstances might have on a high school student’s academic performance and how to evaluate the academic records of students from very dissimilar schools. “We don’t penalize a person for not taking more honors courses or AP courses if those courses weren’t available,” said Dubitzky. Similarly, readers are taught what to look for when assigning a Comprehensive Score of 1 to 5 (with 1 being the highest). These factors include non-academic achievements, leadership ability, motivation, tenacity and initiative, as well as likely contributions to the intellectual and cultural vitality of Berkeley. “We need students at Berkeley who are interesting to teach and who are interested in one another, students who have intellectual curiosity, achievement and talent,” said Franchot. Having absorbed these key points, readers then test and refine their understanding of the scoring system, using actual applications from past years provided by Pam Burnett, OUARS associate director, who organizes and conducts the reader training. She selects a group of applications that represent the entire range of academic and comprehensive scores. By the end of this meeting, called a norming session, each reader will have rated applications deserving an Academic Score of 1 and an Academic Score of 7 – and all the scores in between – as well as rated applications for the full range of comprehensive scores. Readers are also exposed to a variety of files that merit the same score. Said Dubitzky, “We’re building a class of different kinds of individuals, including the number one oboist, the student with the highest grades and the student with outstanding athletic skills.” Those three students’ files might look quite different, she said, and yet each might deserve high scores. After readers finish scoring sample applications, their scores are tallied and written on a grid. The pattern that results is often significant and encouraging: closely clustered scores indicate that readers share a common understanding of the scoring criteria. For high school students hoping to get into Berkeley, the results assure that, although admission is competitive, it is not arbitrary – all applications are held to the same standards. “I find it remarkable that, even from the very first scoring by outside readers, the scores are very closely clustered,” said reader Nina Robinson, policy manager for Admissions and Enrollments. Readers in training discuss individual applications and their strengths, said Robinson, “and how those strengths translate into scores. They justify why they assigned one score and not another.” Said Burnett of the reader training, “With practice, we gain experience and competence in applying the new set of guidelines.” With this information fresh in their minds, readers start evaluating applications shortly before spring semester begins. Weekly, even daily print-outs of the scores assigned by each reader allow admissions officers to monitor whether all readers “stay true to the norm scores and are implementing the policies consistently,” said Dubitzky. Statistics compiled in recent admissions cycles show that for 93 out of 100 applications, the two readers’ scores are within a point of each other. In the remaining seven cases, the file goes to a senior reader for a third opinion. Team leaders meet frequently to cull applications that illustrate problems or important issues. “I’m always seeking instructive examples,” said Burnett. Group NormingOnce actual reading has begun, readers and faculty representatives tackle Burnett’s instructive examples at the norming sessions which occur weekly throughout the reading period. Weekly attendance is required “as part and parcel of the job,” said Dubitzky. “The process is too important to the students and the campus to make them optional.” Readers find the sessions valuable. “I sit at home reading application after application in a necessarily solitary way,” said a retired campus administrator who has read admissions applications for the past three years. “Becoming reacquainted with the norming and other readers’ perspectives brings me back to center. When scores are tallied and they appear on the board, closely clustered, it reassures me that I’m on the right track. And that this daunting task is do-able.” This reality check, she said, “makes the process much more satisfying.” Faculty members take part in the lively dialogue. Their input “keeps readers in touch with the underlying philosophy, intentions and aims of the Faculty Senate,” said Franchot. Participating faculty also experience first-hand the complexity of the admissions process. Complexities of the TaskBy sharing questions and perceptions, readers hone their ability to think and talk about the evaluation process. Such skills are critical when evaluating applications in which the information presented is contradictory. For example, readers discussed the application of a student referred to as “N” at a recent norming session. When the scores were tallied on the grid, three readers assigned N an academic score of 3, 16 gave her a 4 and three voted for a 5. The complexity of N’s record accounted for the three-point spread in scores. Her 3.75 GPA at an elite high school consisted of lots of B’s and A’s. The A’s were primarily in art and debate, not “solid courses,” readers said, while her math SATs were low. On the other hand, she did well on the SAT verbal exam and was a very effective debater – accomplishments that readers felt would serve her well at Berkeley. Applications like N’s are among the most challenging for the team, which finds itself searching for distinctions among an increasingly qualified group of students. “Back in the ’80s,” Dubitzky said, “we had half as many qualified students as we do now. As the application pool has become more competitive, we’ve become more sophisticated in our approach. Now we pay more attention to subtle aspects of the application.” Each year, “literally thousands of students who have done wonderfully in high school” are turned away, she said. “They’re everything you’d like your kid to be. It’s frustrating and hard to have to turn away so many outstanding students.” Yet readers also use words like “honor” and “privilege” to describe their role in Berkeley’s undergraduate admissions process. Said Robinson, “It’s one of the most important things we as a
campus do – deciding which of these really talented, prepared
students get to come to Berkeley.”
[ Back to top ] |
||||
