Methodologies for Learning Behaviors

In contrast to the technical nature of the previous sections, the next few paragraphs take a practical perspective on learning. We discuss the different ways that designers interact with the AI to get it to learn.

Training

Training involves providing examples from which the AI can learn. In most cases, experts analyze the problem and design a solution. After the knowledge is expressed with a convenient representation, it's possible for the animats to learn those examples, as well as provide suitable interpretations of the cases where no examples are provided.

Imitation

Imitation is similar to training in that data samples are required for the animat to learn. The difference is that examples are gathered from observing any other player in the game. In this case, there is no longer any need for experts; random players can be used as a reference for the learning.

Trial and Error

Trial and error is another way for animats to acquire the desired behavior. The idea is to provide no guidance about what to do, and instead rate the quality of actions (at every step) or behaviors (in larger intervals). Thus, the designer is involved in a much higher-level fashion, usually designing the way the feedback is given to the animat. Learning is achieved by attempting to maximize the reward received—regardless of the granularity.

Shaping

Shaping is a way for designers to provide their knowledge of the task to help the animat find the solution [Perkins96]. Generally, medium-level insights are provided by breaking the problem up so that it's easier to solve. In practice, the developer sets up a series of trials, incrementally revealing different aspects of the problem. Generally, these trials are spaced out so that the fundamental concepts are presented first and the more difficult ones later.

Similar principles to shaping can be combined with the three previous approaches. When training, the important cases can be learned first. With imitation, the teacher can make a point of demonstrating simple issues first. Finally, learning based on trial and error can focus on essential concepts first.

Critical Review

In games, training is used the most often because it provides the most predictable results. It's advisable to use this as a default approach. Imitation is becoming increasingly common for small problems, but usually takes more time and effort to set up. Trial and error is extremely powerful, but presents the most technical challenges. A form of shaping is present throughout AI development generally, notably in the iterative nature of the design process (with significant involvement from the engineer).