Discussion

Decision trees are extremely efficient at computing estimates for unknown samples. The data structure is simple and small, and requires little effort to traverse. DTs are particularly suited to data-mining problems, so mass processing of information in real time is not a problem.

In addition, due to the small amounts of information needed, the footprint for storing DTs in memory is very small. After the knowledge has been encoded in the tree, there is often little need to keep the data samples.

Finally, DTs are relatively flexible. They can be used on continuous and symbolic predictor variables. The response variables can also be used for regression or classification, which makes them easy to apply to a wide variety of problems.

However, some disadvantages are worth noting. Decision trees are well suited to batch processing data sets. When it comes to learning them online, however, existing algorithms can be somewhat clumsy (both memory intensive and tricky to implement). Everything considered, gathering the samples incrementally and then applying batch learning seems the easiest approach—which usually results in the same behavior.

The recursive partitioning algorithm is greedy in nature. This performs relatively well, but it is usually suboptimal in both the quality of the results and the size of the tree. This is the case for surprisingly many trees, although the quality is good enough for game development.

Last but not least, there can be problems dealing with overfitting. Secondary pruning phases can be used to remedy the problem, but this requires additional computation and additional data to validate the result. Integrated algorithms, sadly, loose the benefit of simplicity.