Print      |       Back
------------------------------------------------------------

Measuring What Matters Least
  
Review by Jonathan Pollard

In his book The Case Against Standardized Testing: Raising the Scores, Ruining the Schools (2000), Alfie Kohn - one of the nation's most outspoken educational critics - concisely explains the inherent harms of a test-driven curriculum.  Additionally, he suggests ways in which teachers, parents, and students can successfully fight against both the ignorant politicians who parade their notions of accountability for media sound-bites, and the wave of test-obsession that our schools have embraced with something akin to religious fervor.

Many years ago, tests were administered mostly to decide placement of students in their classes, or to ascertain which students needed additional help.  Today, test scores are quoted by newspapers; they are used as the primary criteria for judging the success or failure of students, teachers, and school.  Furthermore, they are used by public officials to impose their will upon the educational system.  From an international perspective, our situation must seem entirely unusual.  Few countries administer exams to children so young, or with such a frequency as we do.  Our children are tested to an extent that is unmatched in the history of our society.  There is no more discussion of learning or of new educational methods.  Kohn states that the educational discourse in our nation has been limited to the following statement: "Test scores are too low.  Make them go up."

Over the past few decades, testing has increasingly become a decidedly political issue.  Testing allows politicians to display their concern for the school system.  Test scores offer a simple means of gratification.  Demanding increased test scores fits nicely with political buzzwords such as 'accountability' and 'tougher standards'.  Some people might argue that such accountability is necessary and that we need an objective means of measuring students' achievements.  But do standardized tests truly provide an objective measure of achievement, and if not, then what do they measure? Kohn argues that they do not, making the following points:

   • First and foremost, we must ask ourselves if we are truly measuring something that is important.  Are we measuring intelligence and practical ability, or are we simply measuring test-taking ability?

   • Though standardized testing may seem to be something of a scientific nature, they are nothing at all similar to, for example, the process of measuring the size and weight of an object.  Though they are objective, in the sense that they are sometimes scored by machines, they are decidedly subjective, in that they are created by human beings.  People write the questions, which may be confusing, biased, or even stupid.  Furthermore, people decided which questions to include, and which ones to exclude.

   • Many proponents of standardized testing argue that it is not 'realistic' to think that we could eliminate such exams.  People who are worried about reality and the 'real-world' ought to realize that artificial exercises such as standardized tests are unrealistic, and do nothing whatsoever toward preparing students for life outside of the classroom. 

   • Test results don't necessarily indicate achievement, but rather, tend to be much more accurate indicators of the size of a student's house or the income of the student's parents.  Research has indicated that the amount of poverty found in a community, and other factors that have absolutely nothing to do with what happens in the classroom, account for the great majority of differences in test scores from one area to another. 

   • Rather than providing the opportunity for students to demonstrate a higher level of reasoning ability, or carry out any form of extended analysis, standardized tests stress a more superficial level of reasoning, and are most typically extensive exercises in short term memory. 

------------------------------------------------------------

The Worst Tests
  
Review by Jonathan Pollard

Though Kohn finds most standardized tests to be objectionable, he comments that some tests are even worse than others; proving even more damaging to the development of the student's mind, and measuring even less.  The most damaging testing programs can be characterized by certain readily identifiable features:

   • Multiple choice examinations.  Quoting Roger Farr, a professor of education at Indiana University, "I don't think there's any way to build a multiple-choice question that allows students to show what they can do with what they know."

   • Even standardized tests that include some amount of open ended or free-response questions are equally ineffective measures of achievement.  The essays written on these tests are frequently not scored by educators, but by temp workers, who are paid minimum wage, and who generally spend no more than two minutes on each exam.  According to one former scorer, "There were times I'd be reading a paper every ten seconds.  I know this sounds very bizarre, but you could put a number on these things without actually reading the paper." Furthermore, the scorer added that he and his coworkers were offered a "two hundred dollar bonus that kicked in after eight thousand papers."

   • Timed exams.  The ability to work quickly, and perform under extreme pressure, is valued above all else. 

   • Tests are given far too frequently, and at every grade level.  This is simply a manifestation of the school system's obsession with speed.  Grade specific standards are simply another way of measuring how fast children can learn, the only difference is that, rather than minutes and hours, the time is measured in years. 

   • We must be weary of norm-referenced tests.  Unlike tests that are "criterion-referenced", meaning that they compare the scores of each student to a given standard, norm-referenced tests compare the performance of the students to each other.  No matter how well or how poorly students do on norm-referenced tests, there will always be a top 10% and a bottom 10%; there will always be 50% of the students who have test scores that fall below the median.  This is not an indication that our schools are performing poorly or failing, this is simply a necessary product of the definition of "median".  Norm-referenced tests don't tell us how much a student has learned, but rather, how much more or less than other students he has learned.  Perhaps everyone - even those who had scores below the median - did reasonably well.  Unfortunately, we will never know. 

   • Norm-referenced tests do not assess how well children are learning, but rather, are used to compute who is better than whom.  Such tests are used to create a sharp division between the winners and the losers.  Why is that we could never accept a system in which everyone could succeed? Why is it that our society so deeply values this selection process, in which some students are labeled smart, and others are labeled stupid? The psychological damage caused by such a system is simply ignored by those who support standardized testing. 

   • Tests are most damaging when given to younger students.  Increasingly, students in primary school are being frequently subjected to timed examinations. 

   • According to educator Bill Ayers, standardized tests ignore the most important characteristics of being a good learner or a good person.  "What they can measure and count," he says, "are isolated skills, specific facts and functions, the least interesting and least significant aspects of learning." Knowing a lot of facts does not necessarily equate with being intelligent or possessing any practical knowledge.  Additionally, teamwork, consulting with classmates, or any other form of cooperative learning is explicitly forbidden during the completion of standardized tests.  Doesn't our society - and for that matter, the vision statement of every corporation - express the notion that the ability to work as part of a team is a desirable quality?

Burnt at the High Stakes
  
Review by Jonathan Pollard

As we move away from the classroom, and through the various levels of educational bureaucracy - first principals, then progressing to administrators, school board members, state board members, state legislators, governors - support for standardized testing grows.  For the politicians who use classroom visits as occasional photo opportunities, supporting standardized testing and other means of producing 'accountability' make it appear as though they hold the future of our children in the highest priority.  It couldn't be further from the truth: most politicians don't care about real education, they care about reelection.  We are fooled into believing the argument that, if we truly cared about the education of our children, we would 'raise the bar' and make testing an even more aspect of the curriculum.  This has given rise to the implementation of what Kohn terms "high stakes testing".  This method is advocated by government officials as a means of forcing students and teachers! t! o care about testing.  The author describes it as follows:

   • High stakes testing often involves the use of bribery and coercion.  Sometimes teachers and schools can receive bonuses for increased test scores.  Students might receive food, tickets to theme parks or sporting events, and other such perks.  Threats may include loss of funding or accreditation for schools, or failure of a grade and even denial of a high school diploma for students. 

   • The idea of high stakes testing goes hand in hand with the idea of 'accountability'.  But can teachers and schools be fairly held accountable for the scores of their students? Low scores are often, to a large extent, due to social and economic factors over which even the best teachers and the best schools have no control.  These factors include the amount of resources available to a given school and the affluence of the community.  But even to the extent that scores do reflect classroom experiences, those experiences are hardly limited to the current year.  It seems unjust to hold a fourth-grade teacher accountable for her students' poor test scores when those scores reflect not only what has happened in her classroom, but also what the children have learned (or failed to learn) in their prior years schooling. 

   • No test is perfect, and there is always the possibility that errors may be present on the physical test, or that errors may occur in the grading process.  Every so often, the big test publishers make some major mistake in the correction of their exams.  In one such episode, New York City officials ordered nearly 9,000 students to attends remedial summer school on the basis of a scoring error. 

   • The idea of giving people an incentive - or threatening them - to improve is a flawed one.  In order to produce temporary compliance, the proponents of standardized testing often advocate - through their actions - the creation of a climate of fear, in which people only cooperate to avoid being punished.  The system of punishments and rewards (external motivations) serves to strip children of their innate desire to learn (internal motivation).

High stakes testing has resulted in the following:

   • Exaggerated reports of success.  In Texas, many proponents of standardized testing argue that the high stakes movement along with other means of holding people and schools accountable has been an integral part the state's improvement in test scores.  Others will argue that any improvement in the Texas educational system is not the result of high stakes testing, but rather, a product of smaller class sizes, overall increases in educational spending, and a court ordered equalization of resources between schools that serve the rich and schools that serve the poor. 

   • Rather than learning anything of consequence, students are taught how to beat the test.  Consider this passage taken directly from Kohn's book:

When tests are first administered, the scores are distressingly low.  (And the headlines read: Our schools are failing! Our students are ignorant!) After a year or two, the scores begin to rise as students and teachers get used to the test.  (And the headlines read: Our schools are improving! Tougher standards are working!) Then the scores level off or begin to drop, or, if a new test is substituted for the original one, even plummet.  (We've gown complacent! Even tougher standards are needed!)

This is not due to a change in the competency of teachers, or level of instruction.  This is simply the process of students and teachers acclimating to the tests. 

   • Good teachers and principals leave the profession.  Those individuals who realize that the priorities of the educational system are entirely skewed, and are often not able to remain in an environment where they are forced to implement a curriculum that they are ethically opposed to.  Those teachers who truly value education often cannot tolerate a system that is centered on test scores rather than actual learning. 

   • Those educators who do not leave the system often become defensive and competitive.  In the high stakes environment, they are forced to do all that they can to raise the test scores of their students.  They are forced to defend themselves and their teaching methods when their students score low.

   • Cheating has become widespread.  And not only is cheating occurring on the part of the students, teachers - under similar pressure - have been caught giving inappropriate help to students during the exams, or even going so far as to alter the answer sheets after the exams are completed.  Such behavior is condemned by the media, the politicians, and the school boards.  But what about the system that produces this behavior?

   • Rather than focusing on true understanding or higher level reasoning, educational curriculum becomes more and more fact oriented.  As the tests become more fact based, teachers - in order to prepare their students for the exams, and to avoid a condemnation of their teaching methods or punitive actions - tailor their classes to the tests.  Education becomes dry, boring, and predictable, and concerns itself with nothing but the barest of facts. 

   • The high stakes movement is centered around the notion of quantification.  But education is not necessarily a quantifiable entity.  Since when are only the things that are measurable by standardized tests considered important? The best and most innovative educational programs that presently exist in our country are being eliminated, because their results cannot be readily reduced to a series of meaningless numbers. 

------------------------------------------------------------

Poor Teaching for Poor Kids
  
Review by Jonathan Pollard

"The movement toward high stakes testing marks a major retreat from fairness, accuracy, and educational equity.  The misuse of tests for high stakes purposes has subverted the benefits these tests can bring.  We cannot close the achievement gap until we close the gap in investment between poor and rich schools.  Instead of taking responsibility as policy makers to invest in improving students' lives, we place the responsibility squarely on children.  It is simply negligent to force children to pass a test and expect that the poorest children, who face every disadvantage, will be able to do as well as those who have every advantage." - Senator Paul Wellstone, in a speech delivered to a group of Minnesota educators on April 4, 2000

 A lot of people who advocate the high stakes movement and the system's excessive use of standardized testing claim that they do so in the name of promoting educational equality.  Instead of turning promises of 'bridging the gap' into realities, the high stakes movement has resulted in poor children falling even further behind their more economically fortunate counter-parts.  Kohn argues that this statement is true for a number of reasons:

   • Tests may be biased.  Many standardized tests include questions that are more likely to be answered correctly by students who come from privileged backgrounds.  Its crazy for anyone to suggest that such tests measure what children learn in the classroom, while neglecting to even consider the possibility that knowledge gained outside of school is equally relevant on these exams.  Children who come from more affluent families, have educated parents, attend quality daycare or preschool programs, own computers, travel, or attend cultural events are far more likely than less affluent children to possess such outside knowledge.

   • Test preparation.  When the stakes rise, people will do anything to make sure that their kids succeed.  Scholarships, college admission, and reputation are all on the line.  If the students don't profit from high stakes testing, many companies certainly do.  They have turned the desperation of parents and students into a money-making industry.  Test prep courses, vocabulary boosting audio tapes, exam-specific tutors.  Guess who can afford these luxuries? Affluent families, schools, and school districts.  The poorer schools are too concerned with paying for basic educational supplies, like books and other necessities. 

   • Standardized tests are great at ensuring that poor kids become poor adults.  Standardized tests don't really measure anything important.  They tend to focus on short term memory and actual test-taking ability rather than genuine understanding.  In short, they stunt children's academic potential by 'dumbing them down'.  The more frequently poor children mindlessly fill in worksheets on command (in an effort to raise test scores), the more likely they are to fall behind affluent kids who actually understand ideas and concepts. 

   • Funding issues.  Rather than spending money for better teachers, funding goes to test preparation materials.  Plus, affluent schools that are already successful are likely to get financial bonuses for performing well on exams, while poor schools that truly need the money sometimes lose funding due to low test scores. 

   • Demanding tougher standards, through offering rewards or threatening punishments, isn't the remedy.  Some very real factors that effect our schools - such as racism, poverty, inadequate resources, language barriers, and crime - are being dismissed as 'excuses'. 

   • The teachers that really want to help poor students learn will be pressured to leave the profession.  This system of punishments and rewards which stresses 'teaching to the test' will drive some of the most intelligent and idealistic young teachers out of the profession.  Many of the teachers who do remain will begin to look at low scoring children as detriments, rather than challenges.  The quality of teaching at poor schools will go from second to third rate. 

------------------------------------------------------------

If Not Standardized Tests, Then What?

  
Review by Jonathan Pollard

A lot of people will argue that we need standardized tests, if only because we have no alternative method of measuring student achievement, and making sure that all kids are getting a decent education.  This is completely false.  The only thing that standardized tests do is promote competition and a winner/loser environment by ranking one school, state, or student against another.  So how can parents be certain that their students are learning? Kohn offers the following alternatives:

   • Parents could receive written descriptions of their child's performance from the teacher. 

   • Parents could attend a conference with the teacher, or even maintain regular communication. 

   • The most skillful teachers don't rely very heavily on standardized tests.  They observe their students, and communicate with them on a daily basis.  Good teachers can often tell, without using exams, how well a student is understanding things.  Parents might worry about whether or not a teacher's personal and non-test oriented evaluation of his or her students is accurate.  But how can we assume that tests are any more credible?

   • Performance assessments.  These are opportunities for children to actually do something; maybe the conducting of an experiment and the presentation of its results, or even writing a play.  Another version of the performance assessment is the "portfolio".  Students can collect examples of work that they have done over the course of the year, or over the course of multiple years.  These types of assessments are far better than standardized or conventional tests at providing data about what students can do, and areas where they might need additional help. 

   • Not all parents think that tests are the best way of evaluating their children's performance.  In a 1999 Phi Delta Kappa/Gallup poll of the general public, respondents were asked which of four methods would provide the most accurate measure of a public school student's academic progress.  Only 27% of the respondents chose standardized test scores.  Examples of the students work was the first choice, receiving 33% of the votes.  The remainder of responses were split between letter grades and teacher-written observations. 

  • If we continue to use standardized tests, we should do as much as possible to make them less damaging to children.  This includes making sure that tests aren't timed and don't include multiple choice questions.  It also includes making sure that the results of such tests are not norm-referenced.  Test results should be considered in absolute terms, meaning with reference to a given standard of achievement, rather than with reference to the scores of other students.  Additionally, reports of test scores should be evaluated with consideration given to special challenges faced by certain schools or districts: very low income community, lack of resources, language barriers, etc.  (Please see section 1 for a more detailed explanation of the term "norm-referenced"). 

   • Not so frequently.  We need to realize that tests - in the traditional sense of the word - don't have to play such a huge role in our schools.  They can be used infrequently, as a means of obtaining some basic, but limited, information. 

------------------------------------------------------------

Fighting the Tests
  
Review by Jonathan Pollard

Learning to live with it.  Even more disturbing than someone defending the tough standards movement is someone who agrees that standardized tests are unnecessary and damaging, but refuses to do anything about it.  Standardized tests aren't an inherent part of our educational system.  People decided to incorporate them into our school curriculum, and likewise, people can decide to remove them.  This is not another educational fad that will simply run its course.  People need to fight back, and can do so in a number of ways. 

Short ways for teachers to fight the standardized testing movement:  

   • You can prepare your students for the tests, but then get back to concentrating on real learning.  Keep it clear in your mind that these are two different objectives.  Whenever possible, help others to see the difference (in letters to parents, for example). 

   • Don't devote all of your classroom time to test prep.  A basic introduction to the test format and a review of test content is good enough.  Some teachers spend most of the year doing exciting classroom activities that focus on real learning followed by a two week 'cram-session' that focuses on the tests.  Studies have shown that these students do just as well as the kids who have had an entire year of test preparation. 

   • As far as possible, make preparing for the test fun.  Use the test questions in some type of stimulating game or puzzle that the whole class can have fun playing or solving.

But the course of action listed above is hardly a solution.  It is more like a way of damage control.  Rather than just working around the tests, or attempting to accommodate them, we must do something to fight back.  We must do something that will make our views heard by the public and move our schools in the direction of eliminating standardized tests.  We need to organize.  Find people in our own areas who share our beliefs, and work together so that we can collectively have a more powerful impact.  Find friends, coworkers, and neighbors who share similar beliefs, and form an organization.  Give yourselves a name, for example, [name of your area] Educators Opposing Excessive Testing.  You'll instantly gain some credibility, and be able to recruit more members.  Whether you work alone or as part of a group, you should begin by learning all that you can about the tests that are used in your area.  Once you taken these steps, the real action begins:

   • Discuss the situation with your acquaintances at every possible opportunity.  In the grocery store, at the doctors office, or whenever the chance to do so arises. 

   • Attend your local school board meetings and other local events that deal with education.  Voice your concerns. 

   • Write to school administrators, public officials, and newspapers. 

   • Form a delegation of concerned citizens, and visit your state legislators or other elected public officials.  Politicians will be much more likely to take your concerns seriously if you speak to them in person. 

   • Sponsor a forum on testing.  Invite the media.  Sign up new members for your organization. 

   • Make bumper stickers with slogans like "STANDARDIZED TESTING IS DUMBING DOWN OUR SCHOOLS!"

   • Protest.  Organize, participate in, and ensure press coverage for some form of protest.  This can include marches, demonstrations, and other activities. 

   • Invite researchers in your area to commission a survey.  Questions could include: Do the tests improve students motivation? Do teachers think that tests measure the curriculum fairly? How much money is spent on assessment and related services? How do administrators use the test results?

   • Challenge politicians, corporate executives, public officials, and other advocates of the 'tough standards' movement to take the tests themselves.  Do this especially if your district uses high stakes exit exams, which are increasingly being used to deny diplomas to students.  You can go about issuing the challenge in two ways.  First, you can describe it as a private invitation for these individuals to learn more about the tests.  The second approach is a bit less thinly veiled: set forth an outright challenge.  In one such instance, several top elected officials in Florida were challenged to take the Florida Comprehensive Assessment Test along with 735,000 students.  All of these officials declined the invitation.  If using the second approach, you might want to consider holding a press conference to publicly issue the challenge.  Imagine the public's response if top elected officials won't agree to take the tests that they advocate!

   • Consider filing a lawsuit against the tests on the grounds that they are inherently discriminatory or statistically invalid measuring instruments. 

   • Opt out.  Some states have a clause that allows parents to exempt their children from testing just by notifying the authorities.  Not many people are even aware that these clauses exist, so do some investigating.  If you find out that such a clause does exist in your state, do your best to make this information public knowledge.  Though this may sound extreme, in the case of their being no opt-out provision, boycott the tests altogether. 

Desperate times call for drastic measures.  Not only does Alfie Kohn tell us why standardized tests are harmful to children, he also describes a concrete approach that we can take to help fight this detrimental system.  The author makes it clear that it is not enough to simply read this book, and consider yourself 'well-informed' on the dangers of schools' excessive use of exams.  He advocates taking a proactive approach to the problem, and making yourself a part of the fight against standardized testing. 

Learn more by visiting http://www.alfiekohn.org/

------------------------------------------------------------
Beware of Movements

James Alexander, Ph.D.
Asst. Prof. of Elem. Educ.
Kentucky Wesleyan College


A middle grades science teacher was describing a recent planning meeting.  Someone mentioned an activity they had done in the past, which was highly successful and much enjoyed by students. However, it was decided that the activity could not be done because it did not reflect the content of THE TEST.

I was introducing a new student teacher to the cooperating teacher. Both were concerned about expectations for student teaching.  I explained that we liked to see a few well thought out thematic units.  The cooperating teacher agreed that that would be nice, but that THE TEST left little room for such “frills.”

THE TEST has become the conservative’s answer as to how to “fix” the educational system. That is a frightening thought. A clergyperson wrote the following to me:  "I personally have an aversion to THE TEST!  When I was rector of the parish, I officiated at the funeral of 9 year old who blew his brains out, because of the TEST!  The last words anyone heard him speak were to his teacher as he left school, "I hope I did well on the test."  He got off the bus went into the house and got a shot gun and mortally wounded himself.  He had been anxious about the test all week.  Who wouldn't be with teachers telling young children that the whole school, salaries, etc. depends on the TEST?
James, I am still angry about the loss of his life some 11 years later.  I keep his picture on my bookshelf in the office.

Diane Ravitch’s popular book, Left Back: A Century of Failed School Reform, is a cry against what she sees as wishy-washy, touchy-feely nonsense and a beckoning to, what she sees as higher standards, traditional approaches, and accountability.  Ravitch especially warns us to beware of anything resembling a movement.  She views most (non-conservative) movements as “warmed over” liberalism in disguise.

The book is clearly written from a conservative, “Back to Basics” perspective.  And it is here that the problem lies.  Ravitch merely pits her conservative philosophy (she has become a real darling of the conservatives) against the philosophy of others.  And, philosophy is pretty difficult to prove, since it deals with opinion and values.  Educational movements come and go.  However, we are now on the verge of something entirely different — if we will only recognize it for it truly is.  Not a movement, but a revolution.  It is a revolution based on science that has great promise of offering true lasting reform.

This revolution is a revolution based on how people learn.  This revolution is coming from the field of neuroscience, based on standard scientific procedures and biology.  Although all of the educational implications of new scientific discoveries are not clear, they are becoming increasingly more so.  Within the last several years it has been discovered actual physical changes take place in the brains of organisms placed in enriched environments.  Environments where children are free to explore and discover knowledge are far superior to hours of worksheets, lectures, and useless rote knowledge.

New knowledge is emerging on the importance of biology and environment that help us understand the roots of violence and aggression in our society and schools and the effect of these influences on children.  The implications of such concepts as relaxed alertness and the effects of fear and anxiety—such as associated with THE TEST-- are becoming increasingly clear.  Even formerly mysterious processes such as personal identity, motivation, and memory are yielding to investigation.

We now know the importance of a print-rich, experience rich environment for preschoolers.  Windows of opportunity abound for various connections to be made and learning to occur, but these windows get increasing harder to open if they are missed.  School environments must be structured for enrichment, and children must be encouraged to investigate.

It turns out that emotions may be the key to the whole educational endeavor.  Emotional stress “hijacks” the higher order thinking processes.  Even bearing in mind there is a very great deal about neurological functioning and emotions that scientist have yet to uncover, much practical advice concerning the learning climate has been implied by research.  Research in the social sciences and cognitive psychology has produced an entire literature concerning “best practice.”  Best practice has been described by Marzano (2001) and others and is the fruit of meta-analyses of studies with conclusions so definite that they are undeniable.

All of this information has been accumulating for years.  But the conservatives simply seem to ignore it and recommend THE TEST.  We need a new field that might be called “neuro-education” that spends its effort in exploring the learning process as opposed to devoting countless hours creating and revising core content and perfecting THE TEST.  As an education professor who must deal with state structures that promote THE TEST, I assure the reader that there is a certain fear involved in even mentioning another way.  It appears now that the philosophy of THE TEST has won the day. Teachers, and many of them good ones, are afraid to speak out.  It is not a popular position to “buck the system that holds all of the cards.  So, they sigh, remember their love for children, and go about trying to be true to themselves and their students and live in the face of the pressure of THE TEST.