Package: libsvm-official
------------------------------------- Python interface of LIBSVM -------------------------------------Table of Contents=================- Introduction- Installation via PyPI- Installation via Sources- Quick Start- Quick Start with Scipy- Design Description- Data Structures- Utility Functions- Additional InformationIntroduction============Python (http://www.python.org/) is a programming language suitable for rapiddevelopment. This tool provides a simple Python interface to LIBSVM, a libraryfor support vector machines (http://www.csie.ntu.edu.tw/~cjlin/libsvm). Theinterface is very easy to use as the usage is the same as that of LIBSVM. Theinterface is developed with the built-in Python library "ctypes."Installation via PyPI=====================To install the interface from PyPI, execute the following command:> pip install -U libsvm-officialInstallation via Sources========================Alternatively, you may install the interface from sources bygenerating the LIBSVM shared library.Depending on your use cases, you can choose between local-directoryand system-wide installation.- Local-directory installation:On Unix systems, type> makeThis generates a .so file in the LIBSVM main directory and youcan run the interface in the current python directory.For Windows, the shared library libsvm.dll is ready in thedirectory `..\windows' and you can directly run the interface inthe current python directory. You can copy libsvm.dll to thesystem directory (e.g., `C:\WINDOWS\system32\') to make itsystem-widely available. To regenerate libsvm.dll, pleasefollow the instruction of building Windows binaries in LIBSVMREADME.- System-wide installation:Type> pip install -e .or> pip install --user -e .The option --user would install the package in the home directoryinstead of the system directory, and thus does not require theroot privilege.Please note that you must keep the sources after the installation.For Windows, to run the above command, Microsoft Visual C++ andother tools are needed.In addition, DON'T use the following FAILED commands> python setup.py install (failed to run at the python directory)> pip install .Quick Start==========="Quick Start with Scipy" is in the next section.There are two levels of usage. The high-level one uses utilityfunctions in svmutil.py and commonutil.py (shared with LIBLINEAR andimported by svmutil.py). The usage is the same as the LIBSVM MATLABinterface.>>> from libsvm.svmutil import *# Read data in LIBSVM format>>> y, x = svm_read_problem('../heart_scale')>>> m = svm_train(y[:200], x[:200], '-c 4')>>> p_label, p_acc, p_val = svm_predict(y[200:], x[200:], m)# Construct problem in python format# Dense data>>> y, x = [1,-1], [[1,0,1], [-1,0,-1]]# Sparse data>>> y, x = [1,-1], [{1:1, 3:1}, {1:-1,3:-1}]>>> prob  = svm_problem(y, x)>>> param = svm_parameter('-t 0 -c 4 -b 1')>>> m = svm_train(prob, param)# Precomputed kernel data (-t 4)# Dense data>>> y, x = [1,-1], [[1, 2, -2], [2, -2, 2]]# Sparse data>>> y, x = [1,-1], [{0:1, 1:2, 2:-2}, {0:2, 1:-2, 2:2}]# isKernel=True must be set for precomputed kernel>>> prob  = svm_problem(y, x, isKernel=True)>>> param = svm_parameter('-t 4 -c 4 -b 1')>>> m = svm_train(prob, param)# For the format of precomputed kernel, please read LIBSVM README.# Other utility functions>>> svm_save_model('heart_scale.model', m)>>> m = svm_load_model('heart_scale.model')>>> p_label, p_acc, p_val = svm_predict(y, x, m, '-b 1')>>> ACC, MSE, SCC = evaluations(y, p_label)# Getting online help>>> help(svm_train)The low-level use directly calls C interfaces imported by svm.py. Note thatall arguments and return values are in ctypes format. You need to handle themcarefully.>>> from libsvm.svm import *>>> prob = svm_problem([1,-1], [{1:1, 3:1}, {1:-1,3:-1}])>>> param = svm_parameter('-c 4')>>> m = libsvm.svm_train(prob, param) # m is a ctype pointer to an svm_model# Convert a Python-format instance to svm_nodearray, a ctypes structure>>> x0, max_idx = gen_svm_nodearray({1:1, 3:1})>>> label = libsvm.svm_predict(m, x0)Quick Start with Scipy======================Make sure you have Scipy installed to proceed in this section.If numba (http://numba.pydata.org) is installed, some operations will be much faster.There are two levels of usage. The high-level one uses utility functionsin svmutil.py and the usage is the same as the LIBSVM MATLAB interface.>>> import numpy as np>>> import scipy>>> from libsvm.svmutil import *# Read data in LIBSVM format>>> y, x = svm_read_problem('../heart_scale', return_scipy = True) # y: ndarray, x: csr_matrix>>> m = svm_train(y[:200], x[:200, :], '-c 4')>>> p_label, p_acc, p_val = svm_predict(y[200:], x[200:, :], m)# Construct problem in Scipy format# Dense data: numpy ndarray>>> y, x = np.asarray([1,-1]), np.asarray([[1,0,1], [-1,0,-1]])# Sparse data: scipy csr_matrix((data, (row_ind, col_ind))>>> y, x = np.asarray([1,-1]), scipy.sparse.csr_matrix(([1, 1, -1, -1], ([0, 0, 1, 1], [0, 2, 0, 2])))>>> prob  = svm_problem(y, x)>>> param = svm_parameter('-t 0 -c 4 -b 1')>>> m = svm_train(prob, param)# Precomputed kernel data (-t 4)# Dense data: numpy ndarray>>> y, x = np.asarray([1,-1]), np.asarray([[1,2,-2], [2,-2,2]])# Sparse data: scipy csr_matrix((data, (row_ind, col_ind))>>> y, x = np.asarray([1,-1]), scipy.sparse.csr_matrix(([1, 2, -2, 2, -2, 2], ([0, 0, 0, 1, 1, 1], [0, 1, 2, 0, 1, 2])))# isKernel=True must be set for precomputed kernel>>> prob  = svm_problem(y, x, isKernel=True)>>> param = svm_parameter('-t 4 -c 4 -b 1')>>> m = svm_train(prob, param)# For the format of precomputed kernel, please read LIBSVM README.# Apply data scaling in Scipy format>>> y, x = svm_read_problem('../heart_scale', return_scipy=True)>>> scale_param = csr_find_scale_param(x, lower=0)>>> scaled_x = csr_scale(x, scale_param)# Other utility functions>>> svm_save_model('heart_scale.model', m)>>> m = svm_load_model('heart_scale.model')>>> p_label, p_acc, p_val = svm_predict(y, x, m, '-b 1')>>> ACC, MSE, SCC = evaluations(y, p_label)# Getting online help>>> help(svm_train)The low-level use directly calls C interfaces imported by svm.py. Note thatall arguments and return values are in ctypes format. You need to handle themcarefully.>>> from libsvm.svm import *>>> prob = svm_problem(np.asarray([1,-1]), scipy.sparse.csr_matrix(([1, 1, -1, -1], ([0, 0, 1, 1], [0, 2, 0, 2]))))>>> param = svm_parameter('-c 4')>>> m = libsvm.svm_train(prob, param) # m is a ctype pointer to an svm_model# Convert a tuple of ndarray (index, data) to feature_nodearray, a ctypes structure# Note that index starts from 0, though the following example will be changed to 1:1, 3:1 internally>>> x0, max_idx = gen_svm_nodearray((np.asarray([0,2]), np.asarray([1,1])))>>> label = libsvm.svm_predict(m, x0)Design Description==================There are two files svm.py and svmutil.py, which respectively correspond tolow-level and high-level use of the interface.In svm.py, we adopt the Python built-in library "ctypes," so thatPython can directly access C structures and interface functions definedin svm.h.While advanced users can use structures/functions in svm.py, toavoid handling ctypes structures, in svmutil.py we provide some easy-to-usefunctions. The usage is similar to LIBSVM MATLAB interface.Data Structures===============Four data structures derived from svm.h are svm_node, svm_problem, svm_parameter,and svm_model. They all contain fields with the same names in svm.h. Accessthese fields carefully because you directly use a C structure instead of aPython object. For svm_model, accessing the field directly is not recommanded.Programmers should use the interface functions or methods of svm_model classin Python to get the values. The following description introduces additionalfields and methods.Before using the data structures, execute the following command to load theLIBSVM shared library:>>> from libsvm.svm import *- class svm_node:Construct an svm_node.>>> node = svm_node(idx, val)idx: an integer indicates the feature index.val: a float indicates the feature value.Show the index and the value of a node.>>> print(node)- Function: gen_svm_nodearray(xi [,feature_max=None [,isKernel=False]])Generate a feature vector from a Python list/tuple/dictionary, numpy ndarray or tuple of (index, data):>>> xi_ctype, max_idx = gen_svm_nodearray({1:1, 3:1, 5:-2})xi_ctype: the returned svm_nodearray (a ctypes structure)max_idx: the maximal feature index of xifeature_max: if feature_max is assigned, features with indices larger thanfeature_max are removed.isKernel: if isKernel == True, the list index starts from 0 for precomputedkernel. Otherwise, the list index starts from 1. The defaultvalue is False.- class svm_problem:Construct an svm_problem instance>>> prob = svm_problem(y, x)y: a Python list/tuple/ndarray of l labels (type must be int/double).x: 1. a list/tuple of l training instances. Feature vector ofeach training instance is a list/tuple or dictionary.2. an l * n numpy ndarray or scipy spmatrix (n: number of features).Note that if your x contains sparse data (i.e., dictionary), the internalctypes data format is still sparse.For pre-computed kernel, the isKernel flag should be set to True:>>> prob = svm_problem(y, x, isKernel=True)Please read LIBSVM README for more details of pre-computed kernel.- class svm_parameter:Construct an svm_parameter instance>>> param = svm_parameter('training_options')If 'training_options' is empty, LIBSVM default values are applied.Set param to LIBSVM default values.>>> param.set_to_default_values()Parse a string of options.>>> param.parse_options('training_options')Show values of parameters.>>> print(param)- class svm_model:There are two ways to obtain an instance of svm_model:>>> model = svm_train(y, x)>>> model = svm_load_model('model_file_name')Note that the returned structure of interface functionslibsvm.svm_train and libsvm.svm_load_model is a ctypes pointer ofsvm_model, which is different from the svm_model object returnedby svm_train and svm_load_model in svmutil.py. We provide afunction toPyModel for the conversion:>>> model_ptr = libsvm.svm_train(prob, param)>>> model = toPyModel(model_ptr)If you obtain a model in a way other than the above approaches,handle it carefully to avoid memory leak or segmentation fault.Some interface functions to access LIBSVM models are wrapped asmembers of the class svm_model:>>> svm_type = model.get_svm_type()>>> nr_class = model.get_nr_class()>>> svr_probability = model.get_svr_probability()>>> class_labels = model.get_labels()>>> sv_indices = model.get_sv_indices()>>> nr_sv = model.get_nr_sv()>>> is_prob_model = model.is_probability_model()>>> support_vector_coefficients = model.get_sv_coef()>>> support_vectors = model.get_SV()Utility Functions=================To use utility functions, type>>> from libsvm.svmutil import *The above command loadssvm_train()            : train an SVM modelsvm_predict()          : predict testing datasvm_read_problem()     : read the data from a LIBSVM-format file.svm_load_model()       : load a LIBSVM model.svm_save_model()       : save model to a file.evaluations()          : evaluate prediction results.csr_find_scale_param() : find scaling parameter for data in csr format.csr_scale()            : apply data scaling to data in csr format.- Function: svm_trainThere are three ways to call svm_train()>>> model = svm_train(y, x [, 'training_options'])>>> model = svm_train(prob [, 'training_options'])>>> model = svm_train(prob, param)y: a list/tuple/ndarray of l training labels (type must be int/double).x: 1. a list/tuple of l training instances. Feature vector ofeach training instance is a list/tuple or dictionary.2. an l * n numpy ndarray or scipy spmatrix (n: number of features).training_options: a string in the same form as that for LIBSVM commandmode.prob: an svm_problem instance generated by callingsvm_problem(y, x).For pre-computed kernel, you should usesvm_problem(y, x, isKernel=True)param: an svm_parameter instance generated by callingsvm_parameter('training_options')model: the returned svm_model instance. See svm.h for details of thisstructure. If '-v' is specified, cross validation isconducted and the returned model is just a scalar: cross-validationaccuracy for classification and mean-squared error for regression.To train the same data many times with differentparameters, the second and the third ways should be faster..Examples:>>> y, x = svm_read_problem('../heart_scale')>>> prob = svm_problem(y, x)>>> param = svm_parameter('-s 3 -c 5 -h 0')>>> m = svm_train(y, x, '-c 5')>>> m = svm_train(prob, '-t 2 -c 5')>>> m = svm_train(prob, param)>>> CV_ACC = svm_train(y, x, '-v 3')- Function: svm_predictTo predict testing data with a model, use>>> p_labs, p_acc, p_vals = svm_predict(y, x, model [,'predicting_options'])y: a list/tuple/ndarray of l true labels (type must be int/double).It is used for calculating the accuracy. Use [] if true labels areunavailable.x: 1. a list/tuple of l training instances. Feature vector ofeach training instance is a list/tuple or dictionary.2. an l * n numpy ndarray or scipy spmatrix (n: number of features).predicting_options: a string of predicting options in the same format asthat of LIBSVM.model: an svm_model instance.p_labels: a list of predicted labelsp_acc: a tuple including accuracy (for classification), meansquared error, and squared correlation coefficient (forregression).p_vals: a list of decision values or probability estimates (if '-b 1'is specified). If k is the number of classes in training data,for decision values, each element includes results of predictingk(k-1)/2 binary-class SVMs. For classification, k = 1 is aspecial case. Decision value [+1] is returned for each testinginstance, instead of an empty list.For probabilities, each element contains k values indicatingthe probability that the testing instance is in each class.For one-class SVM, the list has two elements indicating theprobabilities of normal instance/outlier.Note that the order of classes is the same as the 'model.label'field in the model structure.Example:>>> m = svm_train(y, x, '-c 5')>>> p_labels, p_acc, p_vals = svm_predict(y, x, m)- Functions: svm_read_problem/svm_load_model/svm_save_modelSee the usage by examples:>>> y, x = svm_read_problem('data.txt')>>> m = svm_load_model('model_file')>>> svm_save_model('model_file', m)- Function: evaluationsCalculate some evaluations using the true values (ty) and the predictedvalues (pv):>>> (ACC, MSE, SCC) = evaluations(ty, pv, useScipy)ty: a list/tuple/ndarray of true values.pv: a list/tuple/ndarray of predicted values.useScipy: convert ty, pv to ndarray, and use scipy functions to do the evaluationACC: accuracy.MSE: mean squared error.SCC: squared correlation coefficient.- Function: csr_find_scale_parameter/csr_scaleScale data in csr format.>>> param = csr_find_scale_param(x [, lower=l, upper=u])>>> x = csr_scale(x, param)x: a csr_matrix of data.l: x scaling lower limit; default -1.u: x scaling upper limit; default 1.The scaling process is: x * diag(coef) + ones(l, 1) * offset'param: a dictionary of scaling parameters, where param['coef'] = coef and param['offset'] = offset.coef: a scipy array of scaling coefficients.offset: a scipy array of scaling offsets.Additional Information======================This interface was originally written by Hsiang-Fu Yu from Department of ComputerScience, National Taiwan University. If you find this tool useful, pleasecite LIBSVM as followsChih-Chung Chang and Chih-Jen Lin, LIBSVM : a library for supportvector machines. ACM Transactions on Intelligent Systems andTechnology, 2:27:1--27:27, 2011. Software available athttp://www.csie.ntu.edu.tw/~cjlin/libsvmFor any question, please contact Chih-Jen Lin <cjlin@csie.ntu.edu.tw>,or check the FAQ page:http://www.csie.ntu.edu.tw/~cjlin/libsvm/faq.html