Learning AI with Rule-Based Systems

Artificial intelligence in video games is usually implemented statically. As soon as you think about developing a "learning" AI, you automatically think of neural networks, reinforcement learning, etc. All of these are variants that are comparatively costly to implement and considerably more complex to train. Rule-based systems are a simple, straightforward way to implement a learning AI that can react to, and anticipate, players' strategies. In this short post, I'd like to introduce the theory behind rule-based systems.

Rule-based systems make it possible to create a set of predefined rules so that the artificial intelligence can make decisions based on them. However, this isn't simply a list of boolean statements. These rules can reference other rules, forming a graph model and thereby offering a wide range of possible combinations.

The system is structured around two modules: the Working Memory and the Rules Memory.

• The Working Memory stores facts about the game state. For example, information about whether the game has been won, or how many hit points the player character has left. The Working Memory holds the facts on which decisions are based.

• The Rules Memory defines rules by way of boolean statements. Each statement contains a consequence that gets triggered when the statement holds true. That consequence, in turn, can modify the Working Memory and/or call other functions of the application, thereby kicking off new processes. In the first case, the artificial intelligence would have expanded its knowledge. In the second case, it would have actively reacted. Both at once is also possible.

To give an example: say you want to develop an artificial intelligence for a soccer video game. The Working Memory is used to collect facts giving information about the number of goals, yellow and red cards, fouls, the number of duels won and lost, shots on goal, and much more.

Suppose you want to introduce a rule to react to a looming defeat. The condition for this rule could be that your own team is behind and less than ten minutes of playtime remain. All that's left is to define a consequence appropriate for a looming, narrow defeat. The consequence could be that the team switches to a more aggressive tactic - a concrete action taken by the AI.

So far, this system doesn't differ much from a chain of simple if-statements. But the advantage of rule-based systems over plain if-statements in procedural languages lies, for one, in their modular structure and the resulting improvements to maintainability and management. Each rule can be implemented as a class. With a class, you can build relationships between rules. But what happens when several rules share the same precondition? Which rule gets picked? You could let chance decide. But this is exactly where the strength of rule-based systems comes into play.

The core aspect of rule-based systems is adjusting the weighting of rules at runtime. Each rule has an internal value, initialized with a neutral value at the start. At regular intervals, a heuristic evaluates whether a previously executed rule's consequence brought the AI closer to victory or not. For example, if it turns out that the more aggressive tactic led to the team conceding another goal, the weighting of the rule that triggered the tactical shift could be reduced. These weightings are taken into account when selecting rules. With this principle, it's possible to give the AI the ability to learn from its mistakes. Only when several rules share the same weighting does chance decide which rule gets selected.

In the next iteration, a different rule could react to the change in the Working Memory and pass its information on to AI subsystems, such as the agents controlling the soccer players. How frequently the system's iterations run depends on the available performance and the number of rules that need to be checked on every iteration.