A Wolfram Mathematica package for support vector learning

The svMathematica package implements some support vector-based algorithms for machine learning. The package is described in a paper presented at the 2009 Mathematica Italian User Group Meeting.

Prerequisites

The package requires a working installation of Wolfram Mathematica. Moreover, several of the learning algorithms can be executed exploiting different implementations, and some of the latter rely on external computing components, which shall be previously installed. The following table lists these components and the implementation requiring them.

External component svMathematica functionality
ampl and snopt svmClassificationAMPL, svmRegressionAMPL
Svmlight svmClassificationSVMLight
python svmClassificationPython, svmRegressionPython

In turn, the python-based components depend on the python packages CvxOpt and NumPy.

Note the external components can be installed in other directories. Their actual location will affect the symlink creation on step 3 of the Install instructions below.

Install instructions

The software is available on GitHub, and is installed as follows.

  1. Clone the xmlRPC repository locally.
  2. Create a svMathematica directory in a place chosen according to the following table, depending on the operating system you use and your preference for a user-based or a system-wide installation.
    OS User-based installation System-wide installation
    Mac OS X ~/Library/Mathematica/Applications /Library/Mathematica/Applications
    Linux ~/.Mathematica/Applications R/AddOns/Applications
    Windows ~/Applications Data/Mathematica/Applications R/AddOns/Applications
    In the table ~ denotes the user's home directory in Mac OS X and Linux and the user's directory inside Documents and Settings in Windows, while R denotes the Mathematica root directory.
    Note If you like the non-standard way, saving the package file in any directory contained in Mathematica's $Path variable will also work.
  3. Copy the file init.m in the directory created on step 2.
  4. Create symlinks from the directory created on step 2 to the executables of ampl, snopt, python, and svm_learn (the latter from SvmLight), according to the actually installed external components.
  5. In order to use the python implementation, copy the file classificationDefs.py in the same directory of previous steps. Moreover, the file test.py can be executed in order to verify that the NumPy and CvxOpt packages are correctly installed.
  6. Read the user manual notebook svMathematica-usage.nb.

Using the package

The package is loaded as usual.

In[1]:= <<svMathematica`

The starting point of a learning algorithm is a labelled sample, which svMathematica encodes in two lists, referred to as patterns and labes henceforth, containing patterns (lists of numeric values) and labels (lists of -1 or 1 for classification problems and lists of numerical values for regression problems).

Besides the functions actually running learning algorithms, a set of graphics routines is provided: svmClassificationSamplePlot and svmRegressionSamplePlot plot classification and regression samples, respectively.

In[2]:= grClassPoints = svmClassificationSamplePlot[pattern, label,Frame -> True]
svMathematica classification plot
In[3]:= grRegrPoints = svmRegressionSamplePlot[pattern, label, Frame ->True]
svMathematica regression plot

These functions accept two arguments, specifying respectively patterns and labels, as well as standard options for graphical routines.

The package provides two families of learning algorithms, solving classification and regression problems. The first family is executed by svmClassification, accepting patterns and labels as arguments and outputting a classification function:

In[4]:= classifier = svmClassification[pattern, label] 

The returned value is a function which can be queried for classification of new patterns or for plotting purposes, using the svmDecisionFunctionPlot:

In[5]:= grClassifier = svmDecisionFunctionPlot[classifier, {x, 0, 1}, {y, 0, 1}, margin -> False];
In[6]:= Show[grClassPoints,grClassifier]
svMathematica decision function plot

The function svmClassification, besides patterns and labels, accepts several options allowing to specify specific kernels or trade-off constant values, as well as a specific implementation of the algorithm and other parameters:

In[7]:= Options[svmClassification] 
Out[7]= {c -> \[Infinity], kernel -> "linear", bVarThreshold -> 0.0001, implementation -> svmClassificationPython, verbose -> False,  classifierOutput -> "real", classifierInput -> "pattern"}

Regression problems are solved similarly, through svmRegression:

In[8]:= regressor = svmRegression[pattern, label, kernel -> {"Gaussian", 0.4}]; 
In[9]:= grRegressor = Plot[regressor, {x, 0, 1}, PlotRange -> All, AspectRatio -> 1/GoldenRatio, Frame -> True];
In[10]:= Show[grRegrPoints,grRegressor]
svMathematica regression plot

License

svMathematica is licensed uder the LGPL license.