-JIM FARRELL, Hartford Courant

A sometimes reluctant writer, Robin Squirrell was nevertheless excited to compose an essay asserting that Friar Lawrence was the “Romeo and Juliet” character most to blame for the deaths of the star-crossed lovers.

Robin’s enthusiasm was heightened because she knew she wouldn’t have to wait long for feedback.

“It takes, like, two seconds,” she said, beaming when she recalled that the automated essay scoring system used at East Haven High School had given her some of the highest marks in the class for the content and organization of her work.

For generations, students have nervously turned in essays only to wait days or weeks before getting their papers back – and even then the papers often would be decorated with red scribblings pointing out “run-ons” and “faulty parallelism” and other writing sins.

A few schools in Connecticut and around the nation, though, are starting to use sophisticated Internet-based systems that quickly and thoroughly provide feedback on student writing.

The technology doesn’t replace teachers, in part because computers are not fully adept at providing feedback on higher order skills such as analysis and interpretation. But experts say educators now have an invaluable tool that provides the kind of individual attention that is impossible when teachers see 100 or more students each day.

“Students now are more attuned to a digital environment,” said Don Knezek, who runs the Washington-based International Society for Technology in Education. “We know that if they have access to technology, they write more, they revise more. And you have kids getting competitive with the assessment tool. They take it as a challenge, as a kind of gaming activity.”

Only about 1.5 million of the almost 50 million public school students in the country have access to online assessments, but industry officials say the business is steadily growing.

In Los Angeles, for example, more than 80,000 students are using the technology, and a Phoenix district purchased about 25,000 subscriptions.

In Connecticut, schools in Bristol, Glastonbury, Windsor, New Fairfield, Killingly and Stonington were awarded state grants last year to try out the technology.

While many educators and students are embracing the new technology, critics warn that it squelches creativity.

“It appears to encourage formulaic writing,” said Robert Schaeffer of the National Center for Fair & Open Testing advocacy group. “It drills the basics and tends to ignore the nuances of good writing.”

Better Than Humans?

Proponents of automated essay scoring say efficiency is the key.

“Students still need guidance, but the fact that they can get immediate feedback is amazing,” said Brooke Unger, who has been piloting the program in East Haven with teacher Christine Bauer.

Both teachers said the program is especially useful in heterogeneous classes – those with students of vastly different ability levels.

Bauer said she is able to instantaneously sort and filter data to determine which students are having the same kinds of problems.

“You can pull them aside for a quick mini-lesson while the others keep writing,” she said, adding that the program also automatically gives instructions linked to each student’s particular assessment.

For example, Robin was asked to reread her essay and temporarily highlight in green specific passages, quotes and details she had cited from “Romeo and Juliet.” The computer then asked Robin if all of the highlighted information supported her thesis.

“If not, remove them now,” the computer instructed. “Add more details related to your topic.” East Haven uses a program called My Access from Pennsylvania-based Vantage Learning, which relies on an artificial scoring engine called IntelliMetric.

Another major player in the field is ETS, which is perhaps best known for administering the SAT college admissions examination and has an automated essay assessment program called Criterion.

The systems essentially emulate the process carried out by human scorers – such as those who grade the tens of thousands of essays written each year as part of the Connecticut Mastery Test.

The Vantage system is able to examine more than 400 features of text using a 500,000 unique-word library organized into “a 16 million word concept net that retains an understanding of the relationships between and among words.”

It examines the structure of language, from basic grammar and punctuation to sentence complexity. It also can focus on content matters, evaluating word choice and evidence of elaboration, for example, to determine whether the writers have supported their conclusions.

“It is the systemic interaction, or the way in which these features relate to each other, that produces meaning,” reads a passage in a 16-page text that explains how the system works.

Company officials say hundreds of tests have been done that confirm the reliability and accuracy of its product. “We’re more consistent than a group of expert humans,” said Harry Barfoot, a Vantage Learning spokesman.

As the explanatory packet points out: “A cup of coffee or a rest break can lead to a drift in criteria and standards. It is very difficult for a human scorer to score the first and last paper in a set exactly the same way.”

IntelliMetric, however, “can maintain the exact same standards throughout the process.”

Pros And Cons

In addition to concerns about inhibiting student creativity, some observers have suggested that the technology might displace teachers.

Phil Apruzzese, the president of the Connecticut Education Association, said he has no immediate trepidation.

“Long-term, however, we are concerned about possible implications,” Apruzzese said. “Student-teacher interaction and dialogue is fundamental to high-quality teaching and learning.”

Another issue is cost.

Vantage generally charges schools $18 to $24 annually for each student subscription. “Right now it’s cost-prohibitive because it comes down to, `Should we fix the roof on the elementary school or do we buy this program?'” said Art Skerker, a consultant at the state Department of Education who works on technology issues.

Skerker said he hopes the grant-funded projects will produce evidence of the value of online assessments. Ultimately, he said, the state might make subscriptions available to students and schools through the Connecticut Education Network, a fiber-optic connection that links all districts.

In East Haven, Unger and Bauer have completed two essay assignments with about 70 ninth-grade students. Those students, dubbed the “experimental group,” appear to have done a better job organizing their essays than their “control group” classmates, who did not get to use the automated system.

Unger also noted that one of her “experimental group” classes last year was a particularly rowdy bunch.

“Sometimes I’d have to say to them, `You can’t start to work on your essays until you quiet down,’ and they would,” she said, marveling at the irony of writing as a reward.

Bauer said she still reads all of her students’ essays and provides feedback, but noted that having automated assistance makes things move much more quickly. Grading papers is exhausting, she said, adding that under the traditional approach it can takes weeks for students to move from a rough draft to a polished final version.

Peer editing exercises – which call for students to swap papers and give one another feedback – are well-intentioned but often only reveal obvious spelling and grammatical errors.

Experts note that the federal No Child Left Behind law has increased the emphasis on standardized tests that focus on the three R’s – “reading, writing and ‘rithmetic.” Writing is considered by many to be the most challenging discipline to teach.

Many experts say that some technology applications popular in English classes – such as PowerPoint presentations – have done little to develop writing skills.

And English teachers everywhere bemoan the deterioration of writing skills by students who butcher and abbreviate their way through instant messages and emails.

For Robin, online grading simply makes sense. “I’m pretty competitive,” she said. “And now I’ve gotten into the habit of wanting feedback right away. I’ll be like: `What did I get? How’d I do?’

“I think we got spoiled.”

Story PDF