Robots may seem a few steps behind most humans when it comes to learning new tricks. But efforts by researchers at the U.S. Army Research Laboratory (ARL) and the University of Texas at Austin are showing that it is possible for robots to learn from humans, according to results of their studies shared at the recent Association for the Advancement of Artificial Intelligence Conference in New Orleans, and published in the conference proceedings. The training and learning algorithms are an extension of Training an Agent Manually via Evaluative Reinforcement (TAMER) techniques in which a robot can learn how to perform tasks by viewing video streams with a human trainer. The researchers refer to the new training approach as Deep Tamer.
U.S. Army researcher Dr. Garrett Warnell explained that the human trainers provide critique to gauge a robot’s progress during training, with phrases such as “good job” or “bad job” as a human might do when training a canine companion. Warnell said the researchers extended earlier work in this field to enable this type of training for robots or computer programs that currently see the world through images, which is an important first step in designing learning agents that can operate in the real world.
Dr. Garrett Warnell (left) of the U.S. ARL and Dr. Peter Stone (right) of the University of Texas at Austin have been developing new methods for teaching robots new skills and functions, by interacting with a human instructor. (Photo courtesy of the U.S. Army)
Robots are often required to interact with their environments for extended periods of time to gain the optimum artificial intelligence (AI) on how to perform a task. Quite simply, the robot may learn improper actions from a companion field agent; the human trainer is there to make sure that the robot doesn’t pick up knowledge that can be damaging to the robot and/or detrimental to the task at hand.
One of the examples used to demonstrate the effectiveness of the Deep TAMER training technique was to train a robot, in 15 minutes, to perform well at the Atari game of bowling. Such training has proven difficult for many types of robots and many different AI methods. But in this case, after the 15-minute training session, the robot was capable of beating its trainers and even expert Atari bowling players at the game. The researchers hope to apply this new training approach to applications far beyond this bowling game, including to many other video games.
Robots and other autonomous systems, including unmanned aerial vehicles (UAVs), will play key roles in the armies of the future. “The Army of the future will consist of soldiers and autonomous teammates working side-by-side,” Warnell noted. “While both humans and autonomous agents can be trained in advance, the team will inevitably be asked to perform tasks—for example, search and rescue or surveillance—in new environments they have not seen before. In these situations, humans are remarkably good at generalizing their training, but current artificially intelligent agents are not.”
The U.S. Army is hoping that Deep TAMER is an enabling technology for more successful human-robotic-autonomous-system teams in the future. An ultimate goal is to create autonomous agents capable of quickly learning new skills and functions from human teammates, using training methods that may be based on different actions, including graphics and sign language.
