Creating graphs

This recipe explains how yaplf can be used in order to generate graphical output. A basic knowledge of the python programming language is required, as well as the comprehension of the basic concepts related to perceptrons, multilayer perceptrons and their corresponding learning algorithms.

A light gray cell denotes one or more python statements, while a subsequent darw gray cell contains the expected output of the above statements. Statements can either be executed in a python or in a sage shell. For sake of visualization this document assumes that statements are executed in a sage notebook, so that graphics are shown right after the cell generating them. The execution in a pure python environment works in the same way, the only difference being that graphic functions return a matplotlib object that can be dealt with as usual.

Decision function graphs

Each model in yaplf has its own decision function whose graph can be obtained, when the model has two or three inputs, through invocation of the plot function. This function has a specific behaviour for the various subclasses of Model, although they all share a common functionality. Precisely, the only mandatory argument of plot is a sequence of two or three tuples describing the range within which the independent variables should be analyized and the corresponding values for the decision function should be included in the graphic.

Thus for instance a SV classifier decision function is obtained as follows:

from yaplf.models.svm import SVMClassifier
from yaplf.models.kernel import GaussianKernel
from yaplf.data import LabeledExample
s = SVMClassifier([6.2, 6.2, 6.7, 6.7], -0.65,
  [LabeledExample((1.00000000000000, 1.00000000000000), -1.0),
  LabeledExample((0.000000000000000, 0.000000000000000), -1.0),
  LabeledExample((0, 1), 1.0), LabeledExample((1, 0), 1.0)],
  kernel = GaussianKernel(1))
s.plot((-1, 2), (-1, 2))
sage-plot-0

The default setting for the SVMClassifier class plots the decision function boundary. Additional graphical elements can be added through specification of named argument as in the following cell:

from matplotlib.cm import Greens
s.plot((-1, 2), (-1, 2), shading=True, margin=True, margin_color='yellow',
  shading_color=Greens, separator_color='red')

The procedure for getting, for instance, the decision function graph of a perceptron is analogous, although the default setting does not produce any graph:

from yaplf.models.neural import Perceptron
from yaplf.utility import SigmoidActivationFunction
p = Perceptron( ((.3, 9.56),), threshold=(1.7,),
  activation=SigmoidActivationFunction(beta=.1))
p.plot((-5, 5), (-5, 5))
0

This is due to the fact that no predefined boundary value for the decision function is available for a perceptron. It is therefore necessary to explicitly add specific named argument in order to actually getting a meaningful output:

p = Perceptron( ((.3, 9.56),), threshold=(1.7,),
  activation=SigmoidActivationFunction(beta=.1))
p.plot((-5, 5), (-5, 5), plot_points = 100, contours=(0.1, 0.5, 0.9),
  contour_color=('red', 'green', 'blue'), shading = True)
sage-plot-2

See the documentation for plot in the specific subclasses of Model for a full list of possible customizations.

Observing iterative learning algorithms

A significant number of learning algorithms performs an incremental inference, for it starts considering a model randomly chosen which is modified as time evolves until some convergence criterion is met. Sometimes this criterion should be checked through an external intervention; in other cases one simply wants to watch how the learning process is evolving through time. In such cases yaplf allows these special learning algorithms, tagged as subclasses of IterativeLearningAlgorithm, to be observed by an external class which eventually produces a graphic form of the observation process.

The typical form of observation is that focusing on some error metric to be minimized under a given threshold in order to declare a successful learning. This is accomplished through the ErrorTrajectory class. For instance, in the following cell the error backpropagation algorithm is used to learn the binary XOR function for a thousand iterations, and the corresponding square error averaged through all examples is plotted versus the iteration number:

from yaplf.algorithms.neural import BackpropagationAlgorithm
from yaplf.utility import ErrorTrajectory, FixedIterationsStoppingCriterion
xor_sample = [LabeledExample((0.1, 0.1), (0.1,)),
  LabeledExample((0.1, 0.9), (0.9,)), LabeledExample((0.9, 0.1), (0.9,)),
  LabeledExample((0.9, 0.9), (0.1,))]
alg = BackpropagationAlgorithm(xor_sample, (2, 2, 1), threshold = True,
  learning_rate = .3, momentum_term = .8, min_error = 0.1,
  activations = SigmoidActivationFunction(10))

errObs = ErrorTrajectory(alg)
alg.run(stopping_criterion = FixedIterationsStoppingCriterion(1000))
errObs.get_trajectory(color='red', joined = True)
sage-plot-3

The metric used in order to measure error can be changed through the error_measure named argument, to be assigned to an instance of any sublcass of ErrorModel. For instance, the next cell performs again the previous learning task, now focusing on the maximum square error over all patterns rather than on the error average:

alg.reset()
from yaplf.utility import MaxError
errObs = ErrorTrajectory(alg, error_measure=MaxError())
alg.run(stopping_criterion = FixedIterationsStoppingCriterion(1000))
errObs.get_trajectory(color='red', joined = True)
sage-plot-4

There are also other ways of observing a learning algorithm run: for instance, the class PerceptronWeightTrajectory outputs the trajectory described by the perceptron weights (seen as a point in 2D or 3D space) during the learning phase:

from yaplf.utility import PerceptronWeightTrajectory
from yaplf.algorithms.neural import GradientPerceptronAlgorithm
and_sample = [LabeledExample((0.1, 0.1), (0.1,)),
  LabeledExample((0.1, 0.9), (0.1,)), LabeledExample((0.9, 0.1), (0.1,)),
  LabeledExample((0.9, 0.9), (0.9,))]
alg = GradientPerceptronAlgorithm(and_sample, threshold = True,
  weight_bound = 0.1, beta = 0.8)
weightObs = PerceptronWeightTrajectory(alg)
alg.run(stopping_criterion = FixedIterationsStoppingCriterion(5000),
  batch = False, learning_rate = .1)
weightObs.get_trajectory(joined = True)
sage-output-5