Tuning Game Parameters

Tuning Game Parameters

As well as tuning the parameters of a game-playing agent, TAG can also be used to tune the parameters of a game with the ParameterSearch class.

Three things are required:

TunableParameters

A set of parameters to be optimised must implement the ITunableParameters interface with a no-argument constructor. To make this as easy as possible, the following implementation hierarchy is useful:

  • ITunableParameters The main interface.
    • TunableParameters This is an abstract class.
      • TicTacToeParameters is an example of a Game Parameters class that supports ITunableParameters.
      • DiceMonasteryParams is a slightly more complex example.

The section on tuning an AI agent has full details on the necessary code additions on top of TunableParameters.

JSON Search Space

A JSON file can be used to define the search space used by ParameterSearch (as the first argument). Two simple examples are shown below.

{
	"class" : "games.tictactoe.TicTacToeGameParameters",
	"gridSize" : [3, 4, 5, 6, 7]
}

{
	"class" : "games.dicemonastery.DiceMonasteryParams",
	"mandateTreasureLoss" : [false, true],
	"calfSkinsRotInWinter" : [false, true],
	"brewBeerCost" : [1, 2, 3],
	"bakeBreadCost" : [1, 2, 3],
        "brewMeadCost" : 2
}

The most important entry is the class attribute, that defines a sub-class of TunableParameters.

Each further entry in the main object is for one tunable parameter. It must have the same name as the parameter (and any unknown names will be reported by ParameterSearch to try and spot unwitting typing errors). The value is either a single value - which is fixed for the optimisation run - or an array of all the values to be considered. An array can hold values of one of the core JSON types: Integer, Double, Boolean, String. This copes with parameters that are an enum by entering the different values as Strings. These are converted to the enum of the same exact name (using Enum.valueOf()).

Objective Functions

When tuning an AI agent to play a game, we have some natural objective functions in terms of Win rate or score. Such natural choices do not exist when tuning a game, and will depend on what the designer is trying to achieve. Generally this means a function needs to be implemented to take a finished Game, and report how well it achieved the objective. The IGameHeuristic interface is used for this, with a single abstract method with a signature of double evaluateGame(Game g).

A simple example is:

public class TargetScoreDelta implements IGameHeuristic {

    public final int target;

    public TargetScoreDelta(int target) {
        this.target = target;
    }


    @Override
    public double evaluateGame(Game game) {
        AbstractGameState state = game.getGameState();
        DoubleSummaryStatistics stats = IntStream.range(0, state.getNPlayers())
                .mapToDouble(state::getGameScore)
                .summaryStatistics();
        return -Math.abs(stats.getMax() - stats.getMin() - target);
    }
}

TargetScoreDelta sets an objective function of having a target gap in victory points (game score) between the winner and the person in last place. The potential design objective is to have a game that feels quite ‘close’, without the syndrome of a runaway winner who exploits an early lead to build an unassailable position. (Other more nuanced objectives are possible for this design goal; see the game balancing literature for ideas.) The evaluation.heuristics package contains a couple of other simple examples.

The choice of valuing a Game, and not an AbstractGameState, is deliberate. This enables objective functions to take account of the individual agents, which are only stored at the Game level, for example if we want to optimise game parameters to allow MCTS to beat RHEA, or vice versa.

This example takes an input parameter (the desired size of the point gap). The value that such parameters should take can be specified differently for each optimisation run. This is done via the eval argument of ParameterSearch.

ParameterSearch

The ParameterSearch class provides the main access point for optimising a game (or agent). It is designed to be usable via the command line, and requires a minimum of five arguments. The first three are in fixed order:

  1. A JSON file defining the search space (see JSON section). For example:
    • config\NTBEA\TicTacToeSearchSpace.json
  2. Number of NTBEA iterations to run.
  3. The game to optimise for. This is a String that matches a GameType enumeration. For example ‘TicTacToe’.
  4. tuneGame must be one of the arguments to indicate we are not optimising an AI agent.
  5. eval=objectiveFn.json provides a link to a file defining the objective function to use.

There are then a multitude of other options available - full details can be found here. When run this will log the best set of parameters found for the agent to maximise the specified objective function on the game.

For the fifth requirement, an example of a json file to tie in with the earlier example is:

{
	"class" : "evaluation.heuristics.TargetScoreDelta",
	"args" : [10]
}

This requires two attribute-value pairs:

  • Firstly class with the full name of a class that implements the IGameHeuristic interface.
  • Secondly args with the list of the constructor arguments to use. In this case we have just one for the desired point gap.
    • If there is a no-argument constructor, then args can be dropped.
    • Parameters of type Integer, Double, Boolean and String can be mixed in as needed, as long as they are specified in the same order as in the constructor.