Robust Design

Although engineering provides a robust platform for learning and tools to analyze the result, it's the design of the system that solves most of the problems.

Modeling

Successful learning systems are based on strong understanding of both the problem and the solution. Chapter 21, "Knowledge of the Problem," and Chapter 28, "Understanding the Solution," cover this issue in depth. When it comes to adaptation, there are particular ways to deal with problems by modeling the system differently:

• The learning can be isolated to small aspects of the behaviors, which are easier to develop and debug.

• The search spaces can be left simple and smooth, which increases the speed and effectiveness of learning.

Assisting learning algorithms with expert features helps tremendously from this point of view.

Fitness and Reward

When the learning is not supervised, fitness functions and reward signals define the problem—but can be a surprisingly large source of grief. Learning based on feedback is particularly difficult to achieve realistically and reliably. When done right, however, the design of such feedback mechanisms can assist the learning in many ways:

• Smooth functions help learning algorithms by providing continuous and consistent feedback. The noisier these functions, the harder the learning task.

• To increase smoothness and provide more hints to the algorithm, reward signals and fitness functions can be set up using levels of magnitude. Essentially, multiple factors are involved in the feedback, and each is scaled according to its importance. Important features will have high weight but will be harder to reach, so simpler features with lower-weight coefficients are used to guide the learning in early stages. This is discussed more in the next chapter, because it benefits AI systems in general—and not only adaptive systems.

These tricks certainly minimize the disadvantages of learning by feedback over supervised techniques.

Architectures

An AI architecture is capable of combining learning components—or fixed components—to provide the desired behaviors. For example, the learning component can be overridden using a subsumption architecture. The essential actions are identified and provided by fixed components while the rest is learned.

Another useful architectural pattern is to provide a fixed component to supervise the adaptation. This can be understood as a learning manager (sometimes referred to as a critic); it can adapt the learning rate, verify that key learning examples are not forgotten, or it can determine when the learning algorithm needs to process new patterns.