The Crucial Task of Training the Readers
by Cathy Cockrell and Jesus Mena, Public Affairs
posted Mar. 4, 1998
The applications of some 30,000 high school students have arrived
at Sproul Hall. Now comes one of the campuss most critical and
complex tasks: choosing which 8,250 individuals will be offered
freshman admission for fall 1998.
This year, from Jan. 9 to March 31, a team of 52 highly trained
readers is using a new admissions policy, developed by a special
faculty committee, to tackle this challenge. But who are these
readers, and how do they prepare for and carry out this difficult
The group includes 31 professional admissions or outreach officers
in the Office of Undergraduate Admission and Relations With Schools
(OUARS), 10 part-time readers, six volunteers from other units
on the campus, and five Bay Area high school counselors who read
applications as paid interns.
Rigorous reader training continually reinforced by quality control
as well as group norming sessions, guidance by the most experienced
readers (who lead and give support to five- or six-member teams)
and, especially, faculty review are central to the process.
The faculty must participate in training readers, and by training
I mean communicating to them our understanding of the values embedded
in the new system, the kinds of criteria we believe are legitimate
and those that are not, said Jack Citrin, professor of political
science and a member of the Academic Senates Committee on Admissions,
Enrollment and Preparatory Education (AE&PE).
This interaction between faculty and readers is critical to the
new process and will be an ongoing feature of the new system,
added Jenny Franchot, English associate professor and AE&PE chair.
New Reader Orientation
Readers began preparation for the admissions process months before
the applications deadline. Those readers recruited to temporarily
augment the OUARS staff attended an intensive two-day orientation,
at which they studied the policy developed by AE&PE and the faculty
Designed to give readers both essential procedural information
and a perspective on their role, the training included topics
such as UC systemwide eligibility standards and admission requirements
as well as background on the admissions policies established by
the UC Board of Regents and the Academic Senate. What followed
was a 10-hour introduction to evaluating applications, including
how to assign each applicant an Academic Score and a Comprehensive
The new admissions policy requires each application to be reviewed
independently by two readers. Each reader gives a score for academic
performance and another comprehensive score that includes academic
and non-academic achievement.
Readers learn how to assign an Academic Score, which ranges from
1 to 7 (with 1 being the highest), by evaluating standardized
test scores, grade point averages, academic records, college preparatory
courses, and the rigor of the applicants high school course load.
But the faculty also recognizes that grades and test scores alone
dont completely capture vital things about a high schoolers
true experience or merit, said Franchot. Consideration of school
and family context is crucial to evaluating performance and potential.
If the student has won at the state science fair, said reader
Mary Dubitzky, OUARS assistant director, thats academic information
that I want to consider.
Training leaders discuss topics such as the impact family or social
circumstances might have on a high school students academic performance
and how to evaluate the academic records of students from very
We dont penalize a person for not taking more honors courses
or AP courses if those courses werent available, said Dubitzky.
Similarly, readers are taught what to look for when assigning
a Comprehensive Score of 1 to 5 (with 1 being the highest). These
factors include non-academic achievements, leadership ability,
motivation, tenacity and initiative, as well as likely contributions
to the intellectual and cultural vitality of Berkeley.
We need students at Berkeley who are interesting to teach and
who are interested in one another, students who have intellectual
curiosity, achievement and talent, said Franchot.
Having absorbed these key points, readers then test and refine
their understanding of the scoring system, using actual applications
from past years provided by Pam Burnett, OUARS associate director,
who organizes and conducts the reader training. She selects a
group of applications that represent the entire range of academic
and comprehensive scores. By the end of this meeting, called a
norming session, each reader will have rated applications deserving
an Academic Score of 1 and an Academic Score of 7 and all the
scores in between as well as rated applications for the full
range of comprehensive scores.
Readers are also exposed to a variety of files that merit the
Said Dubitzky, Were building a class of different kinds of individuals,
including the number one oboist, the student with the highest
grades and the student with outstanding athletic skills.
Those three students files might look quite different, she said,
and yet each might deserve high scores.
After readers finish scoring sample applications, their scores
are tallied and written on a grid. The pattern that results is
often significant and encouraging: closely clustered scores indicate
that readers share a common understanding of the scoring criteria.
For high school students hoping to get into Berkeley, the results
assure that, although admission is competitive, it is not arbitrary
all applications are held to the same standards.
I find it remarkable that, even from the very first scoring by
outside readers, the scores are very closely clustered, said
reader Nina Robinson, policy manager for Admissions and Enrollments.
Readers in training discuss individual applications and their
strengths, said Robinson, and how those strengths translate into
scores. They justify why they assigned one score and not another.
Said Burnett of the reader training, With practice, we gain experience
and competence in applying the new set of guidelines.
With this information fresh in their minds, readers start evaluating
applications shortly before spring semester begins.
Weekly, even daily print-outs of the scores assigned by each reader
allow admissions officers to monitor whether all readers stay
true to the norm scores and are implementing the policies consistently,
Statistics compiled in recent admissions cycles show that for
93 out of 100 applications, the two readers scores are within
a point of each other. In the remaining seven cases, the file
goes to a senior reader for a third opinion.
Team leaders meet frequently to cull applications that illustrate
problems or important issues. Im always seeking instructive
examples, said Burnett.
Once actual reading has begun, readers and faculty representatives
tackle Burnetts instructive examples at the norming sessions
which occur weekly throughout the reading period. Weekly attendance
is required as part and parcel of the job, said Dubitzky. The
process is too important to the students and the campus to make
Readers find the sessions valuable.
I sit at home reading application after application in a necessarily
solitary way, said a retired campus administrator who has read
admissions applications for the past three years. Becoming reacquainted
with the norming and other readers perspectives brings me back
to center. When scores are tallied and they appear on the board,
closely clustered, it reassures me that Im on the right track.
And that this daunting task is do-able.
This reality check, she said, makes the process much more satisfying.
Faculty members take part in the lively dialogue. Their input
keeps readers in touch with the underlying philosophy, intentions
and aims of the Faculty Senate, said Franchot. Participating
faculty also experience first-hand the complexity of the admissions
Complexities of the Task
By sharing questions and perceptions, readers hone their ability
to think and talk about the evaluation process. Such skills are
critical when evaluating applications in which the information
presented is contradictory.
For example, readers discussed the application of a student referred
to as N at a recent norming session. When the scores were tallied
on the grid, three readers assigned N an academic score of 3,
16 gave her a 4 and three voted for a 5.
The complexity of Ns record accounted for the three-point spread
in scores. Her 3.75 GPA at an elite high school consisted of lots
of Bs and As. The As were primarily in art and debate, not
solid courses, readers said, while her math SATs were low. On
the other hand, she did well on the SAT verbal exam and was a
very effective debater accomplishments that readers felt would
serve her well at Berkeley.
Applications like Ns are among the most challenging for the team,
which finds itself searching for distinctions among an increasingly
qualified group of students.
Back in the 80s, Dubitzky said, we had half as many qualified
students as we do now. As the application pool has become more
competitive, weve become more sophisticated in our approach.
Now we pay more attention to subtle aspects of the application.
Each year, literally thousands of students who have done wonderfully
in high school are turned away, she said. Theyre everything
youd like your kid to be. Its frustrating and hard to have to
turn away so many outstanding students.
Yet readers also use words like honor and privilege to describe
their role in Berkeleys undergraduate admissions process.
Said Robinson, Its one of the most important things we as a
campus do deciding which of these really talented, prepared
students get to come to Berkeley.
[ Back to top ]