ANGELES — When a person’s intelligence is tested,
there are exams. IQ tests, general knowledge quizzes,
artificial intelligence is tested, there are games.
Checkers, chess, Go.
what happens when computer programs beat humans at all
of those games? This is the question AI experts must ask
after a Google-developed program called AlphaGo defeated
a world champion Go player in four out of five matches
in a series that concluded Tuesday.
a yardstick for advances in AI, the era of board game
testing has come to an end, said Murray Campbell, an IBM
research scientist who was part of the team that
developed Deep Blue, the first computer program to beat
a world chess champion.
are fun and they’re easy to measure," said
Campbell. "It’s clear who won and who lost, and
you always have the human benchmark," he said.
"Can you do better than a human?"
checkers, chess, and now Go, it seems the answer is now
a resounding yes. Computer algorithms beat world
champion-level human players in checkers and chess in
— an ancient board game developed in China that is
more complex than chess — was seen as one of the last
board game hurdles.
games, Campbell said, were perfect tests because they
have clear rules and nothing is hidden from players. The
real world is much messier and full of unknowns. What’s
next, it seems, is for AI to get messy.
AI having conquered what experts call "complete
information" games — the kind in which players
can see what their opponents are doing — Tuomas
Sandholm, a professor at Carnegie Mellon University who
studies artificial intelligence, said the next step is
"incomplete information games" like poker.
game of two-player-limit Texas hold ‘em poker has
almost been solved," said Sandholm, who described
"solving" a game as finding the optimal way of
playing it. "In the larger game of two-player
no-limit Texas hold ‘em poker, we’re right at the
cusp of it. We currently have the world’s best
computer program, but we are still not better than the
very best dozen or so humans."
are typically chosen for the specific challenges that
researchers want their AI to be able to overcome. With
board games such as chess and Go, computer programs are
put through the ringer to see if they can learn from
past matches and determine the best next move. With
Texas hold ‘em, it’s about interpreting actions as
signals, and figuring out the next best move without
knowing what the opponent has in their hand.
these things have real-world applications, Sandholm
said. In complete information games, AI can help people
search through large databases and do calculations and
modeling. In incomplete information games, it can be
used in situations where there are lots of unknown
factors, such as negotiations, cybersecurity and
auctions, and even in planning medical treatment.
robotics experts believe AI will one day get messier
than that, taking algorithms out of controlled game
environments and into the open world.
next," said Thomas Johnson, founder of
MotionFigures, a startup that is bringing robotics and
AI to toys. "The challenge will be putting AI
algorithms into practice in open environments where the
rules are not all given to it upfront, and adaptation is
required to be successful."
points to DARPA challenges — competitions run by the
U.S. Department of Defense — as an example of robots
and AI being put to the test in the real world. Past
DARPA challenges forced researchers to build robots that
can walk up and down hills; many fell over or staggered
like toddlers learning to walk.
take for granted things like balance and vision,"
he said. "But for a robot, to walk up and down
hills requires so many complicated decisions to be made
in real time, and it’s really difficult to do."
still more that can be done in controlled environments,
such as AlphaGo only know how to play Go. Oren Etzioni,
executive director of the Allen Institute for Artificial
Intelligence, believes the next step could be for AI to
learn to play (and beat world champions) at any game.
Or, as his institute is doing, putting AI through
standardizing testing. As in, the SATs. Or eighth grade
scientific part of it isn’t complicated," Etzioni
all, it’s not hard to get a computer program to
remember and regurgitate facts. What is hard is getting
computers to apply their knowledge to everyday
question in the test doesn’t require the computer to
give a definition of gravity or recite an equation, but
to describe a real world situation," he said.
"For example, ‘There’s a ball rolling down a
hill.’ This is the paradox: The hard part for the
machine is easy for the human. The machine is struggling
to figure out what does it mean when it says ‘the ball
is rolling down the hill’?"
are no shortages of benchmarks and tests that come after
Go. And that doesn’t even get into benchmarks for
different types of artificial intelligence, such as
emotionally intelligent AI, speech recognizing robots,
or computers designed to understand language.
an exciting field to be in," said Johnson. "I
think it’s incredible and it makes me stop and think
what the next 10, 20 years is gonna bring in AI."