Tag Archives: test preparation

How Standardized Testing Created, then Destroyed, an Empire

Candidates looking at scrolls where the results of the examinations are announced. (Circa 1540)

This is a story about one of the greatest civilizations on Earth. It rose to power and dominated a continent for almost a millennium.

Its decline was swift and complicated.  By the dawn of the 20th Century, it was a shell of its former self…

and it was all, at least partly, due to standardized testing.

China has had standardized testing longer than any other society on the planet.  For an almost unbroken string of 1,300 years, the imperial examination system attracted candidates from all over the country, studying and hoping for a chance to rise to positions of power and influence in the imperial government.

Along with gunpowder, paper, the compass and the printing press, a civil service based on meritocratic competitive exams is one of China’s great contributions to the world.  The abuse and corruption of that system, unfortunately, helped signal its downfall in the late 19th-early 20th century.

Although earlier attempts were made in the 3rd Century BCE, the Chinese government began their famous examination system in the Sui Dynasty, around 605 CE.  Under the T’ang period (618-907), the exam system would spread across China, prompting a cottage industry of schools and tutors designed to prepare candidates for the tests.

The examination system, at least until the Ming Dynasty (1368-1644), consisted of a comprehensive examination of the “Five Studies” (military strategy, civil law, revenue and taxation, agriculture, and geography) as well as the Four Books and Five Classics, a set of philosophical works written by Confucius and some of his disciples.

There were multiple levels of exams based on the position one sought in the government.  Each degree level demanded a more complex understanding of the content material.  At the district level, the exam focused on knowledge of the Classics and composing poetry using proscribed forms.  At the provincial level, the exam expanded to also test the breadth of knowledge of the content material.  At the highest level, the national examination or palace exams, the candidates were required to apply the Confucian classics to analysis of contemporary political problems, along with all the tests needed in the previous levels.

Examination Hall with 7500 cells, 1873

Candidates would often take 24 to 72 hours to complete the exam.  They were locked in bare, isolated rooms or in cubicles with other candidates.  They had to bring their papers, brushes, ink, food, and other materials for the duration of the test.  In the room were two slabs of wood, that could be put together as a bed, or aligned at different heights to create a desk and chair.  To prevent bias in scoring, all candidates identified their work with a number instead of their name.  After the exam was completed, another person would rewrite the examination before evaluation, so that the candidate’s handwriting couldn’t be recognized.

More or less, the exams were open to all male candidates in China, regardless of social position.  In truth, however, the preparations for the exam, as well as the exams themselves, were time-consuming and expensive.  Often a village would funnel all their resources so that one lucky boy can prepare for exams and rise to high office.  Yet the system remained remarkably free of social or geographic bias, according to Justin Crozier in a 2002 article for China in Focus magazine:

“During the Qing period, over a third of jinshi (national) degree holders came from families with little or no educational background. Nor was the system biased towards the inhabitants of the capital. Degrees were awarded to scholars from throughout China; indeed the provinces of Jiangsu and Zhejiang boasted the greatest number of jinshi graduates.”

Furthermore, the exams’ emphasis on uniform standards of content and skills formed a unifying force that united China, a country with dozens of ethnic groups.  No matter where the candidate came from, they had to learn the same Confucian classics and content knowledge, and helped spread the Mandarin Chinese dialect—the dialect used in examinations—towards its status as the standard for Chinese language today.

Yet with any system, the examination system would decay and decline—and its decline can serve a lesson to those who cling to standardized tests as the only standard in measuring student progress.

That decline came with an essay.

Around 1370, the so-called “eight-legged essay” developed in the tutoring houses and academies of exam preparation.  700 characters long, this essay form had 8 specifically proscribed sections that demanded an extreme rote knowledge of the Confucian classics and certain idioms and figurative language used in Confucius’ time.   By this time, the other subjects needed for the exams were largely abandoned in favor of an extremely intense knowledge of only the Confucian texts.   Each text averaged over 400,000 characters that had to be carefully memorized and interpreted.

"Damn! I'm two legs short on this essay and I gotta use the john."

By the 1500s, the eight-legged essay became the standard for imperial examinations throughout the empire. Scholars at the time praised its rigor, structure and ease of scoring, since essays of the same format could be scored uniformly.  Thanks to the change in format, the best examination papers were published at the behest of the emperor, who wanted to give candidates anchor papers of what was considered exemplary work.  So along with the exams came the first exam guides, first published in 1587.

This push towards memorizing texts led to various methods to cheat on the test, as well as multiple methods to thwart cheaters.  Crozier mentions that:

“The sheer volume of knowledge required to succeed in the Imperial examinations elevated cheating to something of an art form in China. Miniature books were devised to be concealed in the palm of a hand; shirts had important passages from the Confucian Classics sewn, in miniscule lettering, to their insides; fans were constructed with pass-notes on their obverse.”

Other forms of corruption would also proliferate.  Proctors were often bribed for various reasons: to give the benefit of the doubt on an essay, or simply to have the candidate skip the early degrees to go straight to the national exams.  Imposters, usually exam-takers themselves, would often take tests in others’ stead.

The biggest disadvantage of the new system, however, was also seen as an asset: its uniformity.

The emphasis on the Confucian texts and proscribed methods such as the eight-legged essay produced government officials with no practical knowledge of government service or political problems.  The rote method of learning the classics meant that the deep morals of the Confucian philosophy were lost on candidates who memorized the texts in order to pass the exams, yet used their government posts to corrupt their office and enrich themselves at the expense of the people.

Ranked list of results of palace examinations, Southern Song Dynasty (1127-1279)

As the Western universities moved towards natural and social sciences in the 19th century, the Chinese system seemed outdated and obsolete.  Western powers would exert more influence on China through pressuring increasingly corrupt and incompetent government leaders—leaders whose only qualification was the completion of an essay they probably memorized.

By 1850, the examinations would indirectly lead to the second-most bloody conflict in world history.  Hong Xiuquan had failed the provincial examinations in four attempts.  The stress of the exams forced Hong into bed, where he had a mental breakdown and claimed he was the younger brother of Jesus sworn to overthrow the imperial government.  Hong’s movement would create a rival kingdom in southern China and result in the Taiping Rebellion, a civil war that lasted 14 years and cost over 20 million lives.

Attempts were made to reform the exams in 1898, but it was too late.  The Chinese empire officially ended the examinations in 1905, only to be finally overthrown by nationalist warlords in 1911.

The imperial examination system is an important historical example of the triumphs and limitations of standardized testing.  The Chinese system, in its original inception, tested candidates in a basket of knowledge that was applied to political and social problems, thereby creating a civil service based on merit, not on personal connections.  It was open to all levels of Chinese society, and it provided a unifying spirit to a vast empire of disparate peoples.

Yet the system’s greatest strength, its uniformity, would lead to its downfall.  The overemphasis on a simplified—albeit still complex—answer form and the de-emphasis of critical thinking at the expense of rote knowledge created an artificial test.  It became an exam that created industries and crimes designed not to find the best candidates, but rather to do best on the exam.

It led to corruption, foreign influence, stagnation, rebellion, death and disaster.

The Chinese imperial examination system is truly a warning from history.


Filed under Uncategorized

New York State’s 5th Grade Social Studies Test: A Race to the Bottom!

I’ve been knee deep in short answers and essay questions since before Thanksgiving break.  I really need to vent on this–fast!

If there was ever a situation where standardized test scores were meaningless, it is with the 5th grade social studies test in New York State.  This test makes a court eunuch look like a stud horse.  I won’t go into particulars, since I cannot by law, but I will fill you in on some trends:

(1) You only need a minimal amount of knowledge to pass this test.  Most of the questions are charts, graphs, maps, quotations, pictures, etc.  While these skills are useful, and come in handing in reading and math tests, it doesn’t necessarily test what the kid knows.  And I don’t buy into the nonsense that kids don’t need to know facts.  The next brat that asks me if George Washington is still alive will be thrown out a window.  At this rate, in a few years you can safely pass the state test without knowing ANY history, geography (beyond the basics), or rudimentary civics. 

(2) The parts that require more time–and skill–count little.  The parts that require minimal brain function (like multiple choice) count the most.  This leads to a skewed scale wherein a student can ace the multiple choice, do okay on the short answer questions, and tank the essay completely.  This genius not only passes, but almost achieves superior proficiency, according to New York State.  Makes a lot of sense, doesn’t it?

(3) The rubrics for the short answers and essay make it next to impossible to fail, unless you really want to do so.  For some questions, the acceptable right answers take up half the page, while the strict, non-bending wrong aswers (I’m sorry–incorrect.  Can’t use the “w” word) occupy a measly one or two lines.  I’m all for giving a kid a break when he knows the material but can’t express it in perfect Oxbridge quality.  Yet this system rewards the saps who copy from documents and score high marks using sheer dumb luck.

I’d like to hear if other New York teachers have had similar experiences with this test in particular, or tests in general.  Comments are always welcome.

There.  I feel better now.

1 Comment

Filed under Uncategorized