Tag Archives: SAT

How Standardized Testing Created, then Destroyed, an Empire

Candidates looking at scrolls where the results of the examinations are announced. (Circa 1540)

This is a story about one of the greatest civilizations on Earth. It rose to power and dominated a continent for almost a millennium.

Its decline was swift and complicated.  By the dawn of the 20th Century, it was a shell of its former self…

and it was all, at least partly, due to standardized testing.

China has had standardized testing longer than any other society on the planet.  For an almost unbroken string of 1,300 years, the imperial examination system attracted candidates from all over the country, studying and hoping for a chance to rise to positions of power and influence in the imperial government.

Along with gunpowder, paper, the compass and the printing press, a civil service based on meritocratic competitive exams is one of China’s great contributions to the world.  The abuse and corruption of that system, unfortunately, helped signal its downfall in the late 19th-early 20th century.

Although earlier attempts were made in the 3rd Century BCE, the Chinese government began their famous examination system in the Sui Dynasty, around 605 CE.  Under the T’ang period (618-907), the exam system would spread across China, prompting a cottage industry of schools and tutors designed to prepare candidates for the tests.

The examination system, at least until the Ming Dynasty (1368-1644), consisted of a comprehensive examination of the “Five Studies” (military strategy, civil law, revenue and taxation, agriculture, and geography) as well as the Four Books and Five Classics, a set of philosophical works written by Confucius and some of his disciples.

There were multiple levels of exams based on the position one sought in the government.  Each degree level demanded a more complex understanding of the content material.  At the district level, the exam focused on knowledge of the Classics and composing poetry using proscribed forms.  At the provincial level, the exam expanded to also test the breadth of knowledge of the content material.  At the highest level, the national examination or palace exams, the candidates were required to apply the Confucian classics to analysis of contemporary political problems, along with all the tests needed in the previous levels.

Examination Hall with 7500 cells, 1873

Candidates would often take 24 to 72 hours to complete the exam.  They were locked in bare, isolated rooms or in cubicles with other candidates.  They had to bring their papers, brushes, ink, food, and other materials for the duration of the test.  In the room were two slabs of wood, that could be put together as a bed, or aligned at different heights to create a desk and chair.  To prevent bias in scoring, all candidates identified their work with a number instead of their name.  After the exam was completed, another person would rewrite the examination before evaluation, so that the candidate’s handwriting couldn’t be recognized.

More or less, the exams were open to all male candidates in China, regardless of social position.  In truth, however, the preparations for the exam, as well as the exams themselves, were time-consuming and expensive.  Often a village would funnel all their resources so that one lucky boy can prepare for exams and rise to high office.  Yet the system remained remarkably free of social or geographic bias, according to Justin Crozier in a 2002 article for China in Focus magazine:

“During the Qing period, over a third of jinshi (national) degree holders came from families with little or no educational background. Nor was the system biased towards the inhabitants of the capital. Degrees were awarded to scholars from throughout China; indeed the provinces of Jiangsu and Zhejiang boasted the greatest number of jinshi graduates.”

Furthermore, the exams’ emphasis on uniform standards of content and skills formed a unifying force that united China, a country with dozens of ethnic groups.  No matter where the candidate came from, they had to learn the same Confucian classics and content knowledge, and helped spread the Mandarin Chinese dialect—the dialect used in examinations—towards its status as the standard for Chinese language today.

Yet with any system, the examination system would decay and decline—and its decline can serve a lesson to those who cling to standardized tests as the only standard in measuring student progress.

That decline came with an essay.

Around 1370, the so-called “eight-legged essay” developed in the tutoring houses and academies of exam preparation.  700 characters long, this essay form had 8 specifically proscribed sections that demanded an extreme rote knowledge of the Confucian classics and certain idioms and figurative language used in Confucius’ time.   By this time, the other subjects needed for the exams were largely abandoned in favor of an extremely intense knowledge of only the Confucian texts.   Each text averaged over 400,000 characters that had to be carefully memorized and interpreted.

"Damn! I'm two legs short on this essay and I gotta use the john."

By the 1500s, the eight-legged essay became the standard for imperial examinations throughout the empire. Scholars at the time praised its rigor, structure and ease of scoring, since essays of the same format could be scored uniformly.  Thanks to the change in format, the best examination papers were published at the behest of the emperor, who wanted to give candidates anchor papers of what was considered exemplary work.  So along with the exams came the first exam guides, first published in 1587.

This push towards memorizing texts led to various methods to cheat on the test, as well as multiple methods to thwart cheaters.  Crozier mentions that:

“The sheer volume of knowledge required to succeed in the Imperial examinations elevated cheating to something of an art form in China. Miniature books were devised to be concealed in the palm of a hand; shirts had important passages from the Confucian Classics sewn, in miniscule lettering, to their insides; fans were constructed with pass-notes on their obverse.”

Other forms of corruption would also proliferate.  Proctors were often bribed for various reasons: to give the benefit of the doubt on an essay, or simply to have the candidate skip the early degrees to go straight to the national exams.  Imposters, usually exam-takers themselves, would often take tests in others’ stead.

The biggest disadvantage of the new system, however, was also seen as an asset: its uniformity.

The emphasis on the Confucian texts and proscribed methods such as the eight-legged essay produced government officials with no practical knowledge of government service or political problems.  The rote method of learning the classics meant that the deep morals of the Confucian philosophy were lost on candidates who memorized the texts in order to pass the exams, yet used their government posts to corrupt their office and enrich themselves at the expense of the people.

Ranked list of results of palace examinations, Southern Song Dynasty (1127-1279)

As the Western universities moved towards natural and social sciences in the 19th century, the Chinese system seemed outdated and obsolete.  Western powers would exert more influence on China through pressuring increasingly corrupt and incompetent government leaders—leaders whose only qualification was the completion of an essay they probably memorized.

By 1850, the examinations would indirectly lead to the second-most bloody conflict in world history.  Hong Xiuquan had failed the provincial examinations in four attempts.  The stress of the exams forced Hong into bed, where he had a mental breakdown and claimed he was the younger brother of Jesus sworn to overthrow the imperial government.  Hong’s movement would create a rival kingdom in southern China and result in the Taiping Rebellion, a civil war that lasted 14 years and cost over 20 million lives.

Attempts were made to reform the exams in 1898, but it was too late.  The Chinese empire officially ended the examinations in 1905, only to be finally overthrown by nationalist warlords in 1911.

The imperial examination system is an important historical example of the triumphs and limitations of standardized testing.  The Chinese system, in its original inception, tested candidates in a basket of knowledge that was applied to political and social problems, thereby creating a civil service based on merit, not on personal connections.  It was open to all levels of Chinese society, and it provided a unifying spirit to a vast empire of disparate peoples.

Yet the system’s greatest strength, its uniformity, would lead to its downfall.  The overemphasis on a simplified—albeit still complex—answer form and the de-emphasis of critical thinking at the expense of rote knowledge created an artificial test.  It became an exam that created industries and crimes designed not to find the best candidates, but rather to do best on the exam.

It led to corruption, foreign influence, stagnation, rebellion, death and disaster.

The Chinese imperial examination system is truly a warning from history.

6 Comments

Filed under Uncategorized

The Complex Legacy of Stanley H. Kaplan

Stanley H. Kaplan (Photo from Kaplan, Inc.)

Stanley H. Kaplan (Photo from Kaplan, Inc.)

Teachers today, for good or ill, work in a world that was shaped, in part, by Stanley H. Kaplan.

As much as teachers carp about the emphasis on standardized tests, they are an unavoidable reality.  Yet there was—and still is—a good, thorough and efficient way to prepare for those bubble monsters.

Stanley H. Kaplan, test prep pioneer and founder of what is today Kaplan, Inc., was proof that the right preparation was the driving factor in great test scores.  His passing on Sunday at age 90 is a milestone in the standardized testing world.

It’s a shame, then, that his most important lessons remained unlearned.

A little disclosure is in order.  I was a Kaplan instructor and tutor for a few years, teaching SAT preparation classes to high school students.  Through my teaching, I became involved in curriculum development, writing and editing instruction material for Kaplan’s new programs for SAT and the Specialized High School Admission Tests.  In fact, I was a contributing editor for the first major overhaul of the SAT program in 2004, leaving to start my teaching career.   It was a lot of fun and the people there were the best.  So no, I’m not exactly unbiased.

Yet five years removed from the Kaplan universe has shown me where Stanley Kaplan’s vision has gone and, more importantly, where it went wrong.

Kaplan’s basic tenet changed the way we look at tests.  Tests, according to Kaplan, follow certain patterns and methods.  Therefore, preparing for a test was more than simply reviewing the content, but also learning the strategies embedded in the natural patterns of a test.  Test writers are human, thus tests are not inhuman monstrous machines.  Every test is beatable.  It simply takes a review (emphasis on review) of the content followed by useful tricks and methods that help counter the traps often found in testing material.

The legacy of Kaplan’s work extends beyond his company, which grew from a few students in his Brooklyn basement to a company with at least $250 million in revenues and over 100,000 students over 120 teaching centers worldwide.   The test prep course has become a rite of passage for students ranging from middle school to graduate school.  The current educational landscape is littered with test prep companies, methodologies, books, instructors, and software that seek to emulate Kaplan’s results, if not his techniques outright.

Even the makers of tests, including schools, education departments, and government agencies, have provided test prep for their own material.  It’s amazing considering the fierce opposition Kaplan received from the College Board and the Federal Trade Commission, which questioned his claims of student success and the need for test preparation in the first place.

However, in the wake of Kaplan’s success comes the seed of its own perversion.

One thing that Kaplan insisted was that test prep is no substitute for learning the material.  Test prep courses are not meant to TEACH any content.  Rather, they are designed to reinforce content already learned in school using effective testing strategies.   He even de-emphasized the test’s importance, stressing that in most cases, a test is but one factor in a basket of variables that determine acceptance, promotion or graduation.

The education establishment, particularly the architects of No Child Left Behind (NCLB) did not get the message.

Accountability means measured, scientific data, in the NCLB world.  Standardized tests are that instrument to measure student progress.  In most instances, it became the ONLY instrument to measure a child’s achievement.  Since the entire emphasis for federal funding, teacher rating, and school rank centered on these tests, every waking moment is spent preparing for these tests.  Instead of test prep supporting or augmenting the curriculum, it replaced the curriculum.

The results will really show in a few years, when the NCLB youngsters begin high school and college.  Students will be dumbfounded at research, debate, analysis and exploration—things not easily quantified on a scan-tron sheet with a # 2 pencil.  I personally know of many students who “rise to the occasion” on test day, yet could barely function in a classroom setting under more rigorous circumstances.

Furthermore, through the NCLB lens, Kaplan-esque techniques and methods are driving, rather than abating, stress levels on tests where the stakes are ever higher.  Kaplan himself was once questioned that his methods caused more anxiety at test time.  He replied that it was the test administrators, not he, that established the stress level.  The Kaplan methods were designed to ease stress, to make the test more straightforward and manageable.  Yet the quantity and stakes of these tests now trump any relief found in test prep methodology.

NCLB has corrupted Kaplan’s vision.  It made test prep, inadvertently, the driving method of content instruction, flying in the face of everything Kaplan stood for.   The higher stakes of these tests has added to an anxiety level that was never meant to exist in the Kaplan universe.

Stanley Kaplan stood for giving students the tools to succeed in a world with roadblocks made by others.  Yet Kaplan also understood that education is more than a series of roadblocks–it is training the mind to reconceive the world, and the roadblocks themselves.  Anyone can learn how to take a test.  No course in the world, however, can teach someone how to think.

Let’s hope the world shaped by Stanley Kaplan does not choke on the perversion of its ideals.  By the looks of things, though, it may be a foregone conclusion.

Leave a comment

Filed under Uncategorized