Package: scikinC
scikinCscikinCis a simple tool intended for deployment of simple Machine Learning
algorithms as shared objects.
We consider as a target scikit-learn and keras neural networks.There are many other options to deploy machine learning algorithms in C and C++
environments, but they usually involve either specific compilation environments
or require complicated threading structures that may make it difficult to
integrate the developed models into existing frameworks.Besides, in large distributed-computing environments it may be interestring to
distribute new models without the need to recompile the entiere software stack.
Some libraries (e.g. TMVA or PMML or LWTNN) allow to export trained models
into portable formats, that can then be converted at run-time in a sequence
of function calls providing the expected results.
While very effective, these libraries add a bit of overhead to function calls
and requires specific compilation environment that may be uneasy to reproduce
in the target environment.ThescikinCproject aims at replacing these intermediate file formats, with
C files, and the run-time interpretation of these files with a ahead-of-time
compilation into dynamically linked shared objects.Using C instead of C++ allows to deploy machine learning function as plugin
function which can be easily binded to other languages and invoked with minimal
overhead. The compiled shared object do not make use of multithreading letting
the larger code infrastructure to deal with parallelization without introducing
overhead.Finally, the portable C files can be included as header files in other
programs and statically compiled for less-conventional architectures such
as microcontroller and FPGAs.As in many other circumstances, distributing binaries hinder software security,
exposing clients to more severe risks than dedicated ML format. Users should be
aware that plugging untrusted shared objects to their program may result in
severe security breachs.LogicscikinCis a transpiler for scikit-learn and keras models generating
C files withextern "C"functions sharing the same signature:FLOAT_T* <function_name> (FLOAT_T* output, const FLOAT_T* input);Everything which is not either the input or the output is hardcoded in
the C function, including:the shape of the input and output tensors;the structure of the ML method (number of trees in a forest o number of
layers in a DNN);the weights of the ML method.The generated C function is inteded for immediate compilation withgcc,
but most C/C++ compiler should be supported.Once compiled, the binary file contains everything that is needed to
evaluate the ML function and with no external dependency beyond standard
C libraries.scikinCis designed to be as modular as possible in order to make it
easy to extend it by adding converters for additional scikit-learn
models and keras layers.Command Line InterfaceThe easiest way to use scikinC is through its Command Line Interface (CLI).
To provide an example, let's consider the following simple python script
that train a preprocessing step from scikit learn and dumps it into a
pickle file.importnumpyasnpimportpicklefromsklearn.preprocessingimportMinMaxScalerminmax=MinMaxScaler()minmax.fit(np.random.normal(0,5,(2,1000))withopen("example_scaler.pkl",'wb')asf:pickle.dump(minmax,f)Once the file is created, one can convert the scaler
into a C file, asscikinCexample_scaler.pkl>Cfile.CFinally you can compile the C file for dynamic loadinggcc-odeployed_scaler.soCfile.C-shared-fPIC-OfastPython InterfaceSometimes it may be useful to include the conversion in C
directly in the Python script where the training procedure
is defined. This is made possible by importing thescikinCfunction and calling the convert method.
For example,importnumpyasnpfromsklearn.preprocessingimportMinMaxScalerimportscikinCminmax=MinMaxScaler()minmax.fit(np.random.normal(0,5,(2,1000))c_string=scikinC.convert({'myMinMaxScaler':minmax})c_stringcontains the text that describeMinMaxScalertransform in C. It can be stored in a text file and compiled
withgccand outlined above.Using the compiled models in C/C++ applicationsConsidering the example producing the shared objectdeployed_object.soabove, one can easily evaluate
it from a C program, linking the shared object at
run-time and then pointing to the function:// C Library for dynamic linking#include<dlfcn.h>// Define the type for generic machine learning functionstypedeffloat*(*mlfunc)(float*,constfloat*);voidsomewhere_in_your_code(void){// Open the shared object libraryvoid*handle=dlopen("./deployed_scaler.so",RTLD_LAZY);if(!handle)exit(1);// Load the scaler by name (by default, the pickle file name is used as name)mlfuncminmax=mlfunc(dlsym(handle,"example_scaler"));// Prepares the input and output buffer and evaluate the functionfloat*inp[]={/* your input goes here */};float*out[/*output n_features goes here*/];minmax(out,inp);// Optionally, closes the linked library filedlclose(handle);}A few notes:the function prototype (FLOAT_T* <name> (FLOAT_T* output, const FLOAT_T*))
is the same for all the models converted by scikinC. This is basically the
only strict requirement on what models can be converted.The floating point type,floatby default, can be updgraded for
numerically instable models (scikinC --float_t doubleor scikinC --float_t "long double"`)the symbol to load through dlsym is the name of the pickle file,
stripped of its extension, if any. In this casesome_model.pklgets compiled
in the symbolsome_model. The compiled function name can be specified asscikinCdesired_name=example_scaler.pkl>Cfile.Cthis is especially useful when the pickle name contains non alphanumeric
characters which would break the C compilation (consider for example a
pickle file named "example-scaler.pkl"More than one model can be compiled in a single shared objectgcc-odeployed_scaler.soCfile1.CCfile2.CCfile3.C-shared-fPIC-Ofastand this considered good practice for bundling together preprocessing
and machine learning steps.Implemented convertersScikit-Learn preprocessingModelImplementationTestNotesMinMaxScalerAvailableAvailableStandardScalerAvailableAvailableQuantileTransformerAvailableAvailableFunctionTransformerAvailableAvailableOnly functions in math.hColumnTransformerAvailableAvailableOnly integer column indicesPipelineAvailablePartialPipelines of pipelines breakScikit-Learn modelsModelImplementationTestNotesGradientBoostingClassifierAvailableAvailableKeras ModelsModelImplementationTestNotesSequentialAvailableAvailableKeras LayersModelImplementationTestNotesDenseAvailableAvailablePReLUAvailableAvailableLeakyReLUAvailableAvailableKeras Activation functionsModelImplementationTestNotestanhAvailableAvailablesigmoidAvailableAvailablereluAvailableAvailableRunning testsIn order to install the full dependencies needed to test the whole package,
install with the tagfql.python3 setup.py bdist_wheel 
pip install dist/scikinC*.whl[fql]Then run the tests withpytest testRelated projectsLWTNNSimpleNNTensorFlow C APIGaudiTensorFlow