- What is IntelliMetric®?
- How long has IntelliMetric® been used?
- How does the IntelliMetric® artificial intelligence scoring engine work?
- How do we know that IntelliMetric® works?
- How is IntelliMetric® trained?
- How long does it take to train IntelliMetric®?
- Is IntelliMetric® similar to other automated essay scoring products available?
- Does IntelliMetric® score the same as an expert rater?
- Can IntelliMetric® score on only one rubric?
- Can IntelliMetric® provide domain scores?
- Can IntelliMetric® score essays written in languages other than English?
- How accurate is IntelliMetric®?
- Can IntelliMetric® be tricked?
- Why use IntelliMetric®?
What is IntelliMetric®?
How long has IntelliMetric® been used?
How does the IntelliMetric® artificial intelligence scoring engine work?
How do we know that IntelliMetric® works?
In order to evaluate whether IntelliMetric® is able to accurately score essays, we put IntelliMetric® to the same tests that we would an expert human rater. After a human rater is trained, he/she is asked to score a set of essays that have previously been scored by experts. The agreement rate between the new scorer and the “known” scores are compared. If the human rater meets the criteria for acceptable agreement, the human is allowed to score new essays. Similarly, after IntelliMetric® is trained, it is asked to score a set of essays that were previously scored by experts. Just as in the human scoring process, we look at the agreement between IntelliMetric® and the expert scores. If IntelliMetric® meets or exceeds the standards for a human rater, IntelliMetric® is able to be put into use to score new essays.
In short, IntelliMetric® is treated much like any expert scorer on the team when evaluating for consistency and accuracy. It must meet or exceed the same high benchmarks of quality that any human expert must meet.
How is IntelliMetric® trained?
How long does it take to train IntelliMetric®?
Is IntelliMetric® similar to other automated essay scoring products available?
IntelliMetric® is a unique automated scoring engine. Its leverage of Artificial Intelligence, Natural Language Processing, and its close modeling of the human rating process make it distinctive from other scoring engines. Research has shown that the use of two or more raters provides a more accurate final score than the use of a single rater, and as such, IntelliMetric® was developed to include the equivalent of a panel of raters. Specifically, within IntelliMetric®, there are in essence multiple automated scoring systems at work, each using a different approach to scoring. With the resulting scores from each “judge,” a final IntelliMetric® score is provided.
Another key difference between IntelliMetric® and other solutions is in its inductive approach to learning how to score essays. Since IntelliMetric® is not rule based or driven upon a set list of features, the IntelliMetric® engine is able to score submissions that range from as short as one word all the way through to very long pieces of writing. IntelliMetric® does not require a special solution for short answers compared to long answers or persuasive essays compared to narrative essays. One engine can do it all!
The most important differentiation between IntelliMetric® and other automated essay scoring is the accuracy of the engine. IntelliMetric® provides unsurpassed essay scoring accuracy.
Does IntelliMetric® score the same as an expert rater?
Can IntelliMetric® score on only one rubric?
Can IntelliMetric® provide domain scores?
Can IntelliMetric® score essays written in languages other than English?
How accurate is IntelliMetric®?
Can IntelliMetric® be tricked?
Why use IntelliMetric®?
IntelliMetric® is a highly accurate scoring engine. It provides accuracy that is superior to that of other scoring engines currently available. Its combination of artificial intelligence, natural language processing, and its similarity in training to human expert raters allows IntelliMetric® to achieve an agreement rate that is often higher than the rate achieved between two human raters.
After IntelliMetric® has been trained to score responses written to a specific prompt, it can be used to successfully and consistently score essays. Unlike human scorers, which require considerable time to score each response, IntelliMetric® requires minimal time to score a large number of essays.
The ability to score both short-answer and extended response items using a multiple-scoring systems approach, provide holistic and domain scores on any rubric, detect non-legitimate responses, and score essays written in a variety of languages make IntelliMetric® the most comprehensive scoring engine available.