Ranking robots
WSU researchers create a way to test AI smarts, hope method leads to robots that can do more
January 15, 2020
Facial recognition may be one of the uses for artificial intelligence, but researchers at WSU are hoping to further diversify the use of AI.
“Most AI systems are designed to solve a specific task,” said Larry Holder, professor in WSU’s School of Electrical Engineering and Computer Science. “If you ask those AI systems to solve a different problem, they would fail because they’re designed to do one thing and one thing only.”
Holder said they are developing an IQ test with a collection of different problems to determine an AI’s ability to perform multiple tasks. The test would then provide an AI IQ score, which they can compare to others.
One of the tasks in the test includes something similar to the Raven Progressive Matrices test, he said, which is when a sequence of shapes or different colors are presented to the AI system.
Christopher Pereyda, doctoral student in computer science, said the system should be able to come up with the next item in the pattern.
Holder said another task involves a shooter game, in which the AI system must accomplish a series of tasks and mini-scenarios.
Creating an AI system that can do both the Raven Progressive Matrices test and a shooter game, he said, is more difficult than creating a system that just does one task or the other.
“Nobody had really come up with a measure like this before,” Holder said.
Researchers also thought about the ways people measure human intelligence and how that could help them develop a system for evaluating AI, he said.
One of the obstacles they encountered was determining how to keep a score of certain tasks, Holder said. An example of this is a game of chess.
“What number do you use?” he said. “Number of games? How many moves did it take [to win]? How long did it take to think about the next move?”
Pereyda said they hope to determine a way to balance the level of difficulty of AI IQ tests.
Holder said they plan to add more tasks to test different facets of intelligence.
“We’re not convinced that the set of tests we have in there now covered all aspects of intelligence that we would like to cover,” he said.
Pereyda said they could also have human participants take the IQ test to compare the score with AI systems.
Holder said their ultimate goal is to push the research community away from creating systems that can only fulfill one task.
“If you want a robot that’s going to help you around the house, you don’t want to have 10 different robots,” he said. “You’d like one robot that can do all those things, and possibly even some things that it wasn’t originally designed to do.”
Holder said exposure to a lot of sci-fi movies like “Star Trek” plays a factor in his research.
“I’ve always been intrigued by designing a system that isn’t just good at one thing,” he said, “but good at a lot of things that ultimately can help humans get past some really tough problems.”