||Volume 20, Number
3, November/December 2007
Confessions of two FSA markers
Much of the attention on Foundation Skills Assessments (FSAs) has
been on the enormous waste of time involved, the stress on students
who are unaccustomed to this kind of testing, and the abuse of the
tests by organizations like the Fraser Institute who use the numbers
to "rank" schools. But as if these were not reasons enough
to call for the end to these tests, it turns out that the marking of
them is just as questionable as the tests themselves. Here are two
storiesconfessions of FSA markersor rather, as one points
Speed was a must
I have kept this story to myself for quite a while, but having
finally given up my dream of being appointed a superintendent of
achievement, I suppose it cant do much harm to tell it.
In the summer of 2001, I agreed to be a marker for the Grade 7
Writing component. After our training, it was clear that speed was an
absolute must. As the white boxes were wheeled in, it was explained
that, on no account was there any budget to work longer than the
allotted five days. We had to get finished by Friday. We were also
promised we could leave early on the last day if we were finished. We
were constantly told to go faster; indeed we were specifically told
NOT to read the whole essay! This was told to us not once but several
times. "Read the first part, get a sense of the quality, and put
down a number from one to five." As for going back to reconsider
a mark, or pausing to confernot a chance. The sheets whizzed by
in a blur. By the way, they dont even call it
"marking." Apparently, the term for what we were doing is
But even more surreal was my later role as a member of the
Provincial Standards Setting Committee. We were directly supervised
by two ministry staff and it was our job to decide where to place the
cut-off grade for Meeting Expectations. The process (Im totally
serious here) was that we FIRST decided what percentage of the
province should meet expectations, THEN we looked at the raw scores
and calculated what threshold would make it "come out
right." Well, we were getting paid, so... I cant remember
the exact numbers now, and as usual we had to leave every scrap of
paper behind, but I swear its the truth.
Patrick Truelove is a Delta teacher.
Behind the scenes in an FSA marking mill
Do FSA scores provide a reliable measure of the abilities of
students in BC schools? My experience as an FSA marker does not
inspire confidence. In 2002, needing the money, I volunteered to mark
Grade 7 English FSA essays. A room full of teachers worked for five
days at SFU under the supervision of facilitators from the
The first days orientation included about an hour devoted to
explaining the marking criteria. The purpose of our marking was
explained to us, and we were given a rubric for judging whether the
essays would be classified as 1 (the lowest), 2, 3, or 4. This was
followed by a practice session. Markers would work in pairs with a
bundle of 20 or 30 papers at a time.
Once we started, the tone gradually changed. We soon realized that
the top priority was speed. The marking used to be done in seven
days, so the reduction of time to five days greatly increased the
pressure to work quickly. Each daythe facilitators told us our group
was doing fine but that we still needed to speed up.
On the third day, they started writing on a chalk board the number
of boxes of essays that had been completed and how many still had to
be done. The supervisors also began announcing the range between the
pair that had completed the largest number of bundles and the pair
that had done the least. People who complained that there was no way
to mark faster and still read the whole essay were humiliated by the
rest of the group. During the week, at the urging of the
facilitators, the overall pace constantly increased. The fastest
markers were openly praised for their speed and accuracy. The
facilitators paired faster markers with slower ones in hopes of
speeding them up. Fast markers were told they would be hired back
Eventually it became clear that markers were not reading the
entire essay. One of the faster markers admitted that he only looked
at the last paragraph and only skimmed it at that, spending no more
than 10 seconds per booklet to assess, mark, and initial. At no time
did the facilitators ever tell markers not to read the whole
composition, but they did have the speedy ones demonstrate and even
give verbal directions on how to mark quickly by skipping over almost
all of each essay.
Some discovered an even faster method. Given that 95% of the
papers were arguably 2s, one could achieve 95% accuracy just by
marking every paper with a 2.
I marked bundle after bundle with 2s and was told my work was accurate.
The only apparent accountability was the inclusion of a
reliability paper in some of the bundles. However, this paper was not
only obviously photocopied but was actually labelled
"Reliability Paper." Of course, the partners examined and
discussed this type of paper meticulously and made sure each of them
recorded the same mark for it.
On the last day, our group was rewarded for its speed by being
dismissed early after a 90-minute discussion session about FSAs.
Source: FSA Examiner, Special Edition, April 24, 2003,
Surrey Teachers Association.