Repository: onnx/onnxmltools

Issue:
XGBoost support
see https://github.com/KOLANICH/pyxgboost for xgboost parser and how to use it.
Feel free to split that project into two: xgboost -> onnx and onnx -> python

Discussion:
KOLANICH: You may find [my parser for XGBoost binary format](https://gitlab.com/KOLANICH/pyxgboost) useful
xadupre: Thanks for the offer, I saw you called xgboost executable and then parse the output to retrieve the coefficient. I chose a different option by using the API to jump the model into json format. It was introduced in XGBoost 0.7. I prefer to use this way even if it forces the user to use a recent version of xgboost. What do you think?
KOLANICH: For storage I prefer to use the binary format because JSON one has larger overhead, but IDK what people prefer (except the obvious leader of pickle used by the one who don't care about security).
xadupre: Just to clarify, I use JSON to retrieve the coefficients from the xgboost model but the model is then stored into ONNX format which is a binary format based on protobuf. I did not compare the sizes of both ONNX and xgboost format but they should be pretty similar.
KOLANICH: >I use JSON to retrieve the coefficients from the xgboost model but the model is then stored into ONNX format which is a binary format based on protobuf.

I know, it is interchange format to import the model into GPU computation network frameworks.

I mean that it may be useful to have the converter workable without xgboost itself, since we read serialized models, we don't need xgboost library itself.
youngkim93: Hi Xavier, I'm curious to know what new support we have in this PR from previous xgboost converter in winmltools. Thanks!
xadupre: The converter is now tested against onnxruntime and follows the same API as every converter in onnxmltools. That's easier to maintain in this module.
youngkim93: Yeah I agree that all the converter logic should live in onnxmltools. Thanks for doing this! The last converter that is still in winmltools is libsvm. Any plan to migrate that as well?
xadupre: That's PR #184. I have a couple of PRs which are not merged yet :)
wenbingl: Please note that the travis yaml was broken in this PR so the travis check was skipped actually.
youngkim93: Can we update the readme.md to reflect dependencies on xgboost and libsvm? For libsvm, I think there is no distribution in pypi so it would be good to provide a way to install one from different sources (maybe https://www.lfd.uci.edu/~gohlke/pythonlibs/#libsvm)?
xadupre: I updated readme.md to add xgboost and libsvm. I did not add a link on https://www.lfd.uci.edu/~gohlke/pythonlibs/#libsvm as it is built from its sources everytime it is tested (see appveyor.yml). Users can check how it is installed on Windows and Linux by looking at CI files.

Patch:
diff --git a/README.md b/README.md
index 50662c2..97e8e1b 100644
--- a/README.md
+++ b/README.md
@@ -32,6 +32,8 @@ This package relies on ONNX, NumPy, and ProtoBuf. If you are converting a model
 2. CoreMLTools
 3. Keras (version 2.0.8 or higher) with the corresponding Tensorflow version
 4. LightGBM (scikit-learn interface)
+5. XGBoost (scikit-learn interface)
+6. libsvm
 
 # Examples
 If you want the converted ONNX model to be compatible with a certain ONNX version, please specify the target_opset parameter upon invoking the convert function. The following Keras model conversion example demonstrates this below. You can identify the mapping from ONNX Operator Sets (referred to as opsets) to ONNX releases in the [versioning documentation](https://github.com/onnx/onnx/blob/master/docs/Versioning.md#released-versions). 
diff --git a/onnxmltools/convert/__init__.py b/onnxmltools/convert/__init__.py
index 830bcdf..07ccbb9 100644
--- a/onnxmltools/convert/__init__.py
+++ b/onnxmltools/convert/__init__.py
@@ -9,3 +9,5 @@ from .main import convert_keras
 from .main import convert_libsvm
 from .main import convert_lightgbm
 from .main import convert_sklearn
+from .main import convert_xgboost
+
diff --git a/onnxmltools/convert/common/_container.py b/onnxmltools/convert/common/_container.py
index 480df3d..d65f726 100644
--- a/onnxmltools/convert/common/_container.py
+++ b/onnxmltools/convert/common/_container.py
@@ -92,6 +92,10 @@ class LightGbmModelContainer(CommonSklearnModelContainer):
     pass
 
 
+class XGBoostModelContainer(CommonSklearnModelContainer):
+    pass
+
+
 class KerasModelContainer(RawModelContainer):
 
     def __init__(self, keras_model):
diff --git a/onnxmltools/convert/common/data_types.py b/onnxmltools/convert/common/data_types.py
index deeaaab..698cac1 100644
--- a/onnxmltools/convert/common/data_types.py
+++ b/onnxmltools/convert/common/data_types.py
@@ -15,6 +15,10 @@ class DataType(object):
 
     def to_onnx_type(self):
         raise NotImplementedError()
+    
+    def __repr__(self):
+        name = self.__class__.__name__
+        return "{}({}, '{}')".format(name, self.shape, self.doc_string)
 
 
 class Int64Type(DataType):
diff --git a/onnxmltools/convert/common/interface.py b/onnxmltools/convert/common/interface.py
index f4934c6..8ac076f 100644
--- a/onnxmltools/convert/common/interface.py
+++ b/onnxmltools/convert/common/interface.py
@@ -10,6 +10,7 @@
 import abc
 import six
 
+
 @six.add_metaclass(abc.ABCMeta)
 class ModelContainer:
     __metaclass = abc.ABCMeta
@@ -41,6 +42,7 @@ class ModelContainer:
         """
         return
 
+
 @six.add_metaclass(abc.ABCMeta)
 class OperatorBase:
     __metaclass__ = abc.ABCMeta
@@ -77,6 +79,7 @@ class OperatorBase:
         """
         pass
 
+
 @six.add_metaclass(abc.ABCMeta)
 class ScopeBase:
     __metaclass__ = abc.ABCMeta
diff --git a/onnxmltools/convert/common/optimizer.py b/onnxmltools/convert/common/optimizer.py
index 08f1d24..551f5f6 100644
--- a/onnxmltools/convert/common/optimizer.py
+++ b/onnxmltools/convert/common/optimizer.py
@@ -117,7 +117,8 @@ class LinkedNode(object):
             ln = LinkedNode(o_)
             view.append(ln)
             for var_ in o_.output:
-                assert var_map.get(var_) is None
+                if var_map.get(var_) is not None:
+                    raise RuntimeError("Duplicated output name (accross all nodes) '{0}'".format(var_))
                 var_map[var_] = ln
 
         additional_nodes = []
diff --git a/onnxmltools/convert/common/utils.py b/onnxmltools/convert/common/utils.py
index 4dca6c0..bbd498c 100644
--- a/onnxmltools/convert/common/utils.py
+++ b/onnxmltools/convert/common/utils.py
@@ -120,7 +120,7 @@ def xgboost_installed():
     try:
         _LIB.XGBoosterDumpModelEx
     except AttributeError:
-        # The version is now recent enough even though it is version 0.6.
+        # The version is not recent enough even though it is version 0.6.
         # You need to install xgboost from github and not from pypi.
         return False
     from xgboost import __version__
@@ -290,7 +290,7 @@ def check_input_and_output_numbers(operator, input_count_range=None, output_coun
     if max_output_count is not None and len(operator.outputs) > max_output_count:
         raise RuntimeError(
             'For operator %s (type: %s), at most %s outputs(s) is(are) supported but we got %s output(s) which are %s' \
-            % (operator.full_name, operator.type, max_output_count, len(operator.outputs), operator.outputs_full_names))
+            % (operator.full_name, operator.type, max_output_count, len(operator.outputs), operator.output_full_names))
 
 
 def check_input_and_output_types(operator, good_input_types=None, good_output_types=None):
diff --git a/onnxmltools/convert/lightgbm/_parse.py b/onnxmltools/convert/lightgbm/_parse.py
index 12dd708..5ff32c6 100644
--- a/onnxmltools/convert/lightgbm/_parse.py
+++ b/onnxmltools/convert/lightgbm/_parse.py
@@ -21,7 +21,7 @@ def _get_lightgbm_operator_name(model_type):
     '''
     Get operator name of the input argument
 
-    :param model_type:  A scikit-learn object (e.g., SGDClassifier and Binarizer)
+    :param model_type:  A lightgbm object.
     :return: A string which stands for the type of the input model in our conversion framework
     '''
     if model_type not in lightgbm_operator_name_map:
@@ -60,7 +60,7 @@ def _parse_lightgbm(scope, model, inputs):
     This is a delegate function. It doesn't nothing but invoke the correct parsing function according to the input
     model's type.
     :param scope: Scope object
-    :param model: A scikit-learn object (e.g., OneHotEncoder and LogisticRegression)
+    :param model: A lightgbm object
     :param inputs: A list of variables
     :return: The output variables produced by the input model
     '''
diff --git a/onnxmltools/convert/main.py b/onnxmltools/convert/main.py
index fda6c41..7809e29 100644
--- a/onnxmltools/convert/main.py
+++ b/onnxmltools/convert/main.py
@@ -63,3 +63,13 @@ def convert_sklearn(model, name=None, initial_types=None, doc_string='', target_
     from skl2onnx.convert import convert_sklearn as convert_skl2onnx
     return convert_skl2onnx(model, name, initial_types, doc_string, target_opset,
                    custom_conversion_functions, custom_shape_calculators)
+
+
+def convert_xgboost(*args, **kwargs):
+    if not utils.xgboost_installed():
+        raise RuntimeError('xgboost is not installed. Please install xgboost to use this feature.')
+
+    from .xgboost.convert import convert
+    return convert(*args, **kwargs)
+
+
diff --git a/onnxmltools/convert/xgboost/__init__.py b/onnxmltools/convert/xgboost/__init__.py
new file mode 100644
index 0000000..e4f7e92
--- /dev/null
+++ b/onnxmltools/convert/xgboost/__init__.py
@@ -0,0 +1,7 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for
+# license information.
+# --------------------------------------------------------------------------
+
+from .convert import convert
diff --git a/onnxmltools/convert/xgboost/_parse.py b/onnxmltools/convert/xgboost/_parse.py
new file mode 100644
index 0000000..283a960
--- /dev/null
+++ b/onnxmltools/convert/xgboost/_parse.py
@@ -0,0 +1,91 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for
+# license information.
+# --------------------------------------------------------------------------
+
+from ..common._container import XGBoostModelContainer
+from ..common._topology import *
+
+from xgboost import XGBRegressor, XGBClassifier
+
+xgboost_classifier_list = [XGBClassifier]
+
+# Associate types with our operator names.
+xgboost_operator_name_map = {XGBClassifier: 'XGBClassifier',
+                              XGBRegressor: 'XGBRegressor'}
+
+
+def _get_xgboost_operator_name(model_type):
+    '''
+    Get operator name of the input argument
+
+    :param model_type:  A xgboost object.
+    :return: A string which stands for the type of the input model in our conversion framework
+    '''
+    if model_type not in xgboost_operator_name_map:
+        raise ValueError("No proper operator name found for '%s'" % model_type)
+    return xgboost_operator_name_map[model_type]
+
+
+def _parse_xgboost_simple_model(scope, model, inputs):
+    '''
+    This function handles all non-pipeline models.
+
+    :param scope: Scope object
+    :param model: A xgboost object
+    :param inputs: A list of variables
+    :return: A list of output variables which will be passed to next stage
+    '''
+    this_operator = scope.declare_local_operator(_get_xgboost_operator_name(type(model)), model)
+    this_operator.inputs = inputs
+
+    if type(model) in xgboost_classifier_list:
+        # For classifiers, we may have two outputs, one for label and the other one for probabilities of all classes.
+        # Notice that their types here are not necessarily correct and they will be fixed in shape inference phase
+        label_variable = scope.declare_local_variable('label', FloatTensorType())
+        probability_map_variable = scope.declare_local_variable('probabilities', FloatTensorType())
+        this_operator.outputs.append(label_variable)
+        this_operator.outputs.append(probability_map_variable)
+    else:
+        # We assume that all scikit-learn operator can only produce a single float tensor.
+        variable = scope.declare_local_variable('variable', FloatTensorType())
+        this_operator.outputs.append(variable)
+    return this_operator.outputs
+
+
+def _parse_xgboost(scope, model, inputs):
+    '''
+    This is a delegate function. It doesn't nothing but invoke the correct parsing function according to the input
+    model's type.
+    :param scope: Scope object
+    :param model: A xgboost object
+    :param inputs: A list of variables
+    :return: The output variables produced by the input model
+    '''
+    return _parse_xgboost_simple_model(scope, model, inputs)
+
+
+def parse_xgboost(model, initial_types=None, target_opset=None,
+                   custom_conversion_functions=None, custom_shape_calculators=None):
+
+    raw_model_container = XGBoostModelContainer(model)
+    topology = Topology(raw_model_container,
+                        initial_types=initial_types, target_opset=target_opset,
+                        custom_conversion_functions=custom_conversion_functions,
+                        custom_shape_calculators=custom_shape_calculators)
+    scope = topology.declare_scope('__root__')
+
+    inputs = []
+    for var_name, initial_type in initial_types:
+        inputs.append(scope.declare_local_variable(var_name, initial_type))
+
+    for variable in inputs:
+        raw_model_container.add_input(variable)
+
+    outputs = _parse_xgboost(scope, model, inputs)
+
+    for variable in outputs:
+        raw_model_container.add_output(variable)
+
+    return topology
diff --git a/onnxmltools/convert/xgboost/common.py b/onnxmltools/convert/xgboost/common.py
new file mode 100644
index 0000000..f914bab
--- /dev/null
+++ b/onnxmltools/convert/xgboost/common.py
@@ -0,0 +1,16 @@
+"""
+Common function to converters and shape calculators.
+"""
+
+def get_xgb_params(xgb_node):
+    """
+    Retrieves parameters of a model.
+    """
+    if hasattr(xgb_node, 'kwargs'):
+        # XGBoost >= 0.7
+        params = xgb_node.get_xgb_params()
+    else:
+        # XGBoost < 0.7
+        params = xgb_node.__dict__
+        
+    return params        
diff --git a/onnxmltools/convert/xgboost/convert.py b/onnxmltools/convert/xgboost/convert.py
new file mode 100644
index 0000000..4a85657
--- /dev/null
+++ b/onnxmltools/convert/xgboost/convert.py
@@ -0,0 +1,44 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for
+# license information.
+# --------------------------------------------------------------------------
+
+from uuid import uuid4
+from ...proto import onnx, get_opset_number_from_onnx
+from ..common._topology import convert_topology
+from ._parse import parse_xgboost
+
+# Invoke the registration of all our converters and shape calculators
+# from . import shape_calculators
+from . import operator_converters, shape_calculators
+
+
+def convert(model, name=None, initial_types=None, doc_string='', target_opset=None,
+            targeted_onnx=onnx.__version__, custom_conversion_functions=None,
+            custom_shape_calculators=None):
+    '''
+    This function produces an equivalent ONNX model of the given xgboost model.
+
+    :param model: A xgboost model
+    :param initial_types: a python list. Each element is a tuple of a variable name and a type defined in data_types.py
+    :param name: The name of the graph (type: GraphProto) in the produced ONNX model (type: ModelProto)
+    :param doc_string: A string attached onto the produced ONNX model
+    :param target_opset: number, for example, 7 for ONNX 1.2, and 8 for ONNX 1.3.
+    :param targeted_onnx: A string (for example, '1.1.2' and '1.2') used to specify the targeted ONNX version of the
+        produced model. If ONNXMLTools cannot find a compatible ONNX python package, an error may be thrown.
+    :param custom_conversion_functions: a dictionary for specifying the user customized conversion function
+    :param custom_shape_calculators: a dictionary for specifying the user customized shape calculator
+    :return: An ONNX model (type: ModelProto) which is equivalent to the input xgboost model
+    '''
+    if initial_types is None:
+        raise ValueError('Initial types are required. See usage of convert(...) in \
+                           onnxmltools.convert.xgboost.convert for details')
+    if name is None:
+        name = str(uuid4().hex)
+
+    target_opset = target_opset if target_opset else get_opset_number_from_onnx()
+    topology = parse_xgboost(model, initial_types, target_opset, custom_conversion_functions, custom_shape_calculators)
+    topology.compile()
+    onnx_model = convert_topology(topology, name, doc_string, target_opset, targeted_onnx)
+    return onnx_model
diff --git a/onnxmltools/convert/xgboost/operator_converters/XGBoost.py b/onnxmltools/convert/xgboost/operator_converters/XGBoost.py
new file mode 100644
index 0000000..d361b2d
--- /dev/null
+++ b/onnxmltools/convert/xgboost/operator_converters/XGBoost.py
@@ -0,0 +1,269 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for
+# license information.
+# --------------------------------------------------------------------------
+
+import ctypes
+import numbers
+import numpy
+import json
+from xgboost import XGBRegressor, XGBClassifier
+from xgboost.core import _LIB, _check_call, from_cstr_to_pystr
+from ...common.tree_ensemble import get_default_tree_classifier_attribute_pairs
+from ...common._registration import register_converter
+from ...common import utils
+from ..common import get_xgb_params
+
+
+class XGBConverter:
+    
+    @staticmethod
+    def get_xgb_params(xgb_node):
+        """
+        Retrieves parameters of a model.
+        """
+        return get_xgb_params(xgb_node)
+    
+    @staticmethod
+    def validate(xgb_node):
+        params = XGBConverter.get_xgb_params(xgb_node)
+        try:
+            if not "objective" in params:
+                raise AttributeError('ojective')
+        except AttributeError as e:
+            raise RuntimeError('Missing attribute in XGBoost model ' + str(e))
+
+    @staticmethod
+    def common_members(xgb_node, inputs):
+        params = XGBConverter.get_xgb_params(xgb_node)
+        objective = params["objective"]
+        base_score = params["base_score"]
+        booster = xgb_node.get_booster()
+        # The json format was available in October 2017.
+        # XGBoost 0.7 was the first version released with it.
+        js_tree_list = booster.get_dump(with_stats=True, dump_format = 'json')
+        js_trees = [json.loads(s) for s in js_tree_list]
+        return objective, base_score, js_trees
+        
+    @staticmethod
+    def _get_default_tree_attribute_pairs(is_classifier):        
+        attrs = {}
+        for k in {'nodes_treeids',  'nodes_nodeids',
+                  'nodes_featureids', 'nodes_modes', 'nodes_values',
+                  'nodes_truenodeids', 'nodes_falsenodeids', 'nodes_missing_value_tracks_true'}:
+            attrs[k] = []
+        if is_classifier:
+            for k in {'class_treeids', 'class_nodeids', 'class_ids', 'class_weights'}:
+                attrs[k] = []
+        else:
+            for k in {'target_treeids', 'target_nodeids', 'target_ids', 'target_weights'}:
+                attrs[k] = []
+        return attrs
+
+    @staticmethod
+    def _add_node(attr_pairs, is_classifier, tree_id, tree_weight, node_id, 
+                  feature_id, mode, value, true_child_id, false_child_id, weights, weight_id_bias,
+                  missing, hitrate):
+        if isinstance(feature_id, str):
+            # Something like f0, f1...
+            if feature_id[0] == "f":
+                try:
+                    feature_id = int(feature_id[1:])
+                except ValueError:
+                    raise RuntimeError("Unable to interpret '{0}'".format(feature_id))
+            else:
+                try:
+                    feature_id = int(feature_id)
+                except ValueError:
+                    raise RuntimeError("Unable to interpret '{0}'".format(feature_id))
+                    
+        # Split condition for sklearn
+        # * if X_ptr[X_sample_stride * i + X_fx_stride * node.feature] <= node.threshold: 
+        # * https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/tree/_tree.pyx#L946 
+        # Split condition for xgboost 
+        # * if (fvalue < split_value) 
+        # * https://github.com/dmlc/xgboost/blob/master/include/xgboost/tree_model.h#L804             
+    
+        attr_pairs['nodes_treeids'].append(tree_id)
+        attr_pairs['nodes_nodeids'].append(node_id)
+        attr_pairs['nodes_featureids'].append(feature_id)
+        attr_pairs['nodes_modes'].append(mode)
+        attr_pairs['nodes_values'].append(float(value))
+        attr_pairs['nodes_truenodeids'].append(true_child_id)
+        attr_pairs['nodes_falsenodeids'].append(false_child_id)
+        attr_pairs['nodes_missing_value_tracks_true'].append(missing)
+        if 'nodes_hitrates' in attr_pairs:
+            attr_pairs['nodes_hitrates'].append(hitrate)
+        if mode == 'LEAF':
+            if is_classifier:
+                for i, w in enumerate(weights):
+                    attr_pairs['class_treeids'].append(tree_id)
+                    attr_pairs['class_nodeids'].append(node_id)
+                    attr_pairs['class_ids'].append(i + weight_id_bias)
+                    attr_pairs['class_weights'].append(float(tree_weight * w))
+            else:
+                for i, w in enumerate(weights):
+                    attr_pairs['target_treeids'].append(tree_id)
+                    attr_pairs['target_nodeids'].append(node_id)
+                    attr_pairs['target_ids'].append(i + weight_id_bias)
+                    attr_pairs['target_weights'].append(float(tree_weight * w))
+        
+    @staticmethod
+    def _fill_node_attributes(treeid, tree_weight, jsnode, attr_pairs, is_classifier, remap):
+        if 'children' in jsnode:
+            XGBConverter._add_node(attr_pairs=attr_pairs, is_classifier=is_classifier, 
+                        tree_id=treeid, tree_weight=tree_weight, 
+                        value=jsnode['split_condition'], node_id=remap[jsnode['nodeid']], 
+                        feature_id=jsnode['split'], 
+                        mode='BRANCH_LT', # 'BRANCH_LEQ' --> is for sklearn
+                        true_child_id=remap[jsnode['yes']], # ['children'][0]['nodeid'], 
+                        false_child_id=remap[jsnode['no']], # ['children'][1]['nodeid'], 
+                        weights=None, weight_id_bias=None,
+                        missing=jsnode.get('missing', -1) == jsnode['yes'], # ['children'][0]['nodeid'],
+                        hitrate=jsnode.get('cover', 0))
+
+            for ch in jsnode['children']:
+                if 'children' in ch or 'leaf' in ch:
+                    XGBConverter._fill_node_attributes(treeid, tree_weight, ch, attr_pairs, is_classifier, remap)
+                else:
+                    raise RuntimeError("Unable to convert this node {0}".format(ch))                        
+                
+        else:
+            weights = [jsnode['leaf']]
+            weights_id_bias = 0
+            XGBConverter._add_node(attr_pairs=attr_pairs, is_classifier=is_classifier, 
+                        tree_id=treeid, tree_weight=tree_weight, 
+                        value=0., node_id=remap[jsnode['nodeid']], 
+                        feature_id=0, mode='LEAF',
+                        true_child_id=0, false_child_id=0, 
+                        weights=weights, weight_id_bias=weights_id_bias,
+                        missing=False, hitrate=jsnode.get('cover', 0))
+
+    @staticmethod
+    def _remap_nodeid(jsnode, remap=None):
+        if remap is None:
+            remap = {}
+        nid = jsnode['nodeid']
+        remap[nid] = len(remap)
+        if 'children' in jsnode:
+            for ch in jsnode['children']:
+                XGBConverter._remap_nodeid(ch, remap)
+        return remap
+            
+    @staticmethod
+    def fill_tree_attributes(js_xgb_node, attr_pairs, tree_weights, is_classifier):
+        if not isinstance(js_xgb_node, list):
+            raise TypeError("js_xgb_node must be a list")
+        for treeid, (jstree, w) in enumerate(zip(js_xgb_node, tree_weights)):
+            remap = XGBConverter._remap_nodeid(jstree)
+            XGBConverter._fill_node_attributes(treeid, w, jstree, attr_pairs, is_classifier, remap)
+
+
+class XGBRegressorConverter(XGBConverter):
+
+    @staticmethod
+    def validate(xgb_node):
+        return XGBConverter.validate(xgb_node)
+
+    @staticmethod
+    def _get_default_tree_attribute_pairs():
+        attrs = XGBConverter._get_default_tree_attribute_pairs(False)
+        attrs['post_transform'] = 'NONE'
+        attrs['n_targets'] = 1
+        return attrs
+
+    @staticmethod
+    def convert(cscope, operator, container):
+        xgb_node = operator.raw_operator
+        inputs = operator.inputs
+        objective, base_score, js_trees = XGBConverter.common_members(xgb_node, inputs)
+        
+        if objective in ["reg:gamma", "reg:tweedie"]:
+            raise RuntimeError("Objective '{}' not supported.".format(objective))
+        
+        booster = xgb_node.get_booster()
+        
+        attr_pairs = XGBRegressorConverter._get_default_tree_attribute_pairs()
+        attr_pairs['base_values'] = [base_score]
+        XGBConverter.fill_tree_attributes(js_trees, attr_pairs, [1 for _ in js_trees], False)
+        
+        # add nodes
+        container.add_node('TreeEnsembleRegressor', operator.input_full_names,
+                           operator.output_full_names, op_domain='ai.onnx.ml', **attr_pairs)
+        #try:
+        #    if len(inputs[0].type.tensor_type.shape.dim) > 0:
+        #        output_dim = [inputs[0].type.tensor_type.shape.dim[0].dim_value, 1]
+        #except Exception:
+        #    raise ValueError('Invalid/missing input dimension.')
+
+
+class XGBClassifierConverter(XGBConverter):
+
+    @staticmethod
+    def validate(xgb_node):
+        return XGBConverter.validate(xgb_node)
+
+    @staticmethod
+    def _get_default_tree_attribute_pairs():
+        attrs = XGBConverter._get_default_tree_attribute_pairs(True)
+        # TODO: check it is implemented. The model cannot be loaded when they are present.
+        #attrs['nodes_hitrates'] = []
+        return attrs
+
+    @staticmethod
+    def convert(scope, operator, container):
+        xgb_node = operator.raw_operator
+        inputs = operator.inputs
+        
+        objective, base_score, js_trees = XGBConverter.common_members(xgb_node, inputs)
+        
+        params = XGBConverter.get_xgb_params(xgb_node)
+        
+        attr_pairs = XGBClassifierConverter._get_default_tree_attribute_pairs()
+        XGBConverter.fill_tree_attributes(js_trees, attr_pairs, [1 for _ in js_trees], True)
+        
+        if len(attr_pairs['class_treeids']) == 0:
+            raise RuntimeError("XGBoost model is empty.")
+        ncl = (max(attr_pairs['class_treeids']) + 1) // params['n_estimators']
+        if ncl <= 1:
+            ncl = 2
+            # See https://github.com/dmlc/xgboost/blob/master/src/common/math.h#L23.
+            attr_pairs['post_transform'] = "LOGISTIC"
+            attr_pairs['class_ids'] = [0 for v in attr_pairs['class_treeids']]
+        else:
+            # See https://github.com/dmlc/xgboost/blob/master/src/common/math.h#L35.
+            attr_pairs['post_transform'] = "SOFTMAX"
+            # attr_pairs['base_values'] = [base_score for n in range(ncl)]
+            attr_pairs['class_ids'] = [v % ncl for v in attr_pairs['class_treeids']]
+        class_labels = list(range(ncl))
+
+        attr_pairs['classlabels_int64s'] = class_labels 
+
+        # add nodes
+        if objective == "binary:logistic":
+            ncl = 2
+            container.add_node('TreeEnsembleClassifier', operator.input_full_names,
+                               operator.output_full_names,
+                               op_domain='ai.onnx.ml', **attr_pairs)
+        elif objective == "multi:softprob":
+            ncl = len(js_trees) // params['n_estimators']
+            container.add_node('TreeEnsembleClassifier', operator.input_full_names,
+                               operator.output_full_names,
+                               op_domain='ai.onnx.ml', **attr_pairs)
+        else:
+            raise RuntimeError("Unexpected objective: {0}".format(objective))            
+
+
+
+def convert_xgboost(scope, operator, container):
+    xgb_node = operator.raw_operator
+    if isinstance(xgb_node, XGBClassifier):
+        cls = XGBClassifierConverter
+    else:
+        cls = XGBRegressorConverter
+    cls.convert(scope, operator, container)
+
+
+register_converter('XGBClassifier', convert_xgboost)
+register_converter('XGBRegressor', convert_xgboost)
diff --git a/onnxmltools/convert/xgboost/operator_converters/__init__.py b/onnxmltools/convert/xgboost/operator_converters/__init__.py
new file mode 100644
index 0000000..c4b94d7
--- /dev/null
+++ b/onnxmltools/convert/xgboost/operator_converters/__init__.py
@@ -0,0 +1,8 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for
+# license information.
+# --------------------------------------------------------------------------
+
+# To register converter for scikit-learn operators, import associated modules here.
+from . import XGBoost
diff --git a/onnxmltools/convert/xgboost/shape_calculators/Classifier.py b/onnxmltools/convert/xgboost/shape_calculators/Classifier.py
new file mode 100644
index 0000000..86bd6e9
--- /dev/null
+++ b/onnxmltools/convert/xgboost/shape_calculators/Classifier.py
@@ -0,0 +1,33 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for
+# license information.
+# --------------------------------------------------------------------------
+
+from ...common._registration import register_shape_calculator
+from ...common.utils import check_input_and_output_numbers, check_input_and_output_types
+from ...common.data_types import Int64TensorType, FloatTensorType, DictionaryType, SequenceType
+from ..common import get_xgb_params
+
+
+def calculate_xgboost_classifier_output_shapes(operator):
+    check_input_and_output_numbers(operator, input_count_range=1, output_count_range=2)
+    check_input_and_output_types(operator, good_input_types=[FloatTensorType, Int64TensorType])
+    N = operator.inputs[0].type.shape[0]
+
+    xgb_node = operator.raw_operator
+    params = get_xgb_params(xgb_node)
+    booster = xgb_node.get_booster()
+    atts = booster.attributes()
+    ntrees = len(booster.get_dump(with_stats=True, dump_format = 'json'))
+    objective = params["objective"]
+            
+    if objective == "binary:logistic":
+        ncl = 2
+    else:
+        ncl = ntrees // params['n_estimators']
+    operator.outputs[0].type = Int64TensorType(shape=[N])
+    operator.outputs[1].type = operator.outputs[1].type = FloatTensorType([N, ncl])
+
+
+register_shape_calculator('XGBClassifier', calculate_xgboost_classifier_output_shapes)
diff --git a/onnxmltools/convert/xgboost/shape_calculators/Regressor.py b/onnxmltools/convert/xgboost/shape_calculators/Regressor.py
new file mode 100644
index 0000000..1e39fb2
--- /dev/null
+++ b/onnxmltools/convert/xgboost/shape_calculators/Regressor.py
@@ -0,0 +1,10 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for
+# license information.
+# --------------------------------------------------------------------------
+
+from ...common._registration import register_shape_calculator
+from ...common.shape_calculator import calculate_linear_regressor_output_shapes
+
+register_shape_calculator('XGBRegressor', calculate_linear_regressor_output_shapes)
diff --git a/onnxmltools/convert/xgboost/shape_calculators/__init__.py b/onnxmltools/convert/xgboost/shape_calculators/__init__.py
new file mode 100644
index 0000000..6ef1bde
--- /dev/null
+++ b/onnxmltools/convert/xgboost/shape_calculators/__init__.py
@@ -0,0 +1,9 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for
+# license information.
+# --------------------------------------------------------------------------
+
+# To register shape calculators for lightgbm operators, import associated modules here.
+from . import Classifier
+from . import Regressor
diff --git a/onnxmltools/utils/utils_backend_onnxruntime.py b/onnxmltools/utils/utils_backend_onnxruntime.py
index 27a9bf5..b4f1d45 100644
--- a/onnxmltools/utils/utils_backend_onnxruntime.py
+++ b/onnxmltools/utils/utils_backend_onnxruntime.py
@@ -237,7 +237,7 @@ def _compare_expected(expected, output, sess, onnx, decimal=5, onnx_shape=None,
             if len(expected) != len(output):
                 raise OnnxRuntimeAssertionError("Unexpected number of outputs '{0}', expected={1}, got={2}".format(onnx, len(expected), len(output)))
             for exp, out, osh in zip(expected, output, onnx_shapes):
-                _compare_expected(exp, out, sess, onnx, decimal=5, onnx_shape=osh, **kwargs)
+                _compare_expected(exp, out, sess, onnx, decimal=decimal, onnx_shape=osh, **kwargs)
                 tested += 1
         else:
             raise OnnxRuntimeAssertionError("Type mismatch for '{0}', output type is {1}".format(onnx, type(output)))


Test patch:
diff --git a/onnxmltools/utils/tests_helper.py b/onnxmltools/utils/tests_helper.py
index 19451c6..a08b7a5 100644
--- a/onnxmltools/utils/tests_helper.py
+++ b/onnxmltools/utils/tests_helper.py
@@ -39,7 +39,7 @@ def dump_data_and_model(data, model, onnx=None, basename="model", folder=None,
         for the backends, otherwise a string which is then evaluated to check
         whether or not the test can fail, example:
         ``"StrictVersion(onnx.__version__) < StrictVersion('1.3.0')"``
-    :param verbose: additional information
+    :param verbose: prints more information when it fails
     :return: the created files
 
     Some convention for the name,
@@ -169,6 +169,9 @@ def convert_model(model, name, input_types):
     if model.__class__.__name__.startswith("LGBM"):
         from onnxmltools.convert import convert_lightgbm
         model, prefix = convert_lightgbm(model, name, input_types), "LightGbm"
+    elif model.__class__.__name__.startswith("XGB"):
+        from onnxmltools.convert import convert_xgboost
+        model, prefix = convert_xgboost(model, name, input_types), "XGB"
     elif isinstance(model, BaseEstimator):
         from onnxmltools.convert import convert_sklearn
         model, prefix = convert_sklearn(model, name, input_types), "Sklearn"
@@ -207,7 +210,7 @@ def dump_one_class_classification(model, suffix="", folder=None, allow_failure=N
                                basename=prefix + "One" + model.__class__.__name__ + suffix)
 
 
-def dump_binary_classification(model, suffix="", folder=None, allow_failure=None):
+def dump_binary_classification(model, suffix="", folder=None, allow_failure=None, verbose=False):
     """
     Trains and dumps a model for a binary classification problem.
     
@@ -218,6 +221,7 @@ def dump_binary_classification(model, suffix="", folder=None, allow_failure=None
         for the backends, otherwise a string which is then evaluated to check
         whether or not the test can fail, example:
         ``"StrictVersion(onnx.__version__) < StrictVersion('1.3.0')"``
+    :param verbose: prints more information when it fails
     :return: output of :func:`dump_data_and_model`
     
     Every created filename will follow the pattern:
@@ -225,11 +229,12 @@ def dump_binary_classification(model, suffix="", folder=None, allow_failure=None
     """
     X = [[0, 1], [1, 1], [2, 0]]
     X = numpy.array(X, dtype=numpy.float32)
-    y = ['A', 'B', 'A']
+    y = [0, 1, 0]
     model.fit(X, y)
     model_onnx, prefix = convert_model(model, 'tree-based binary classifier', [('input', FloatTensorType([1, 2]))])
     dump_data_and_model(X, model, model_onnx, folder=folder, allow_failure=allow_failure,
-                        basename=prefix + "Bin" + model.__class__.__name__ + suffix)
+                        basename=prefix + "Bin" + model.__class__.__name__ + suffix,
+                        verbose=verbose)
 
 def dump_multiple_classification(model, suffix="", folder=None, allow_failure=None):
     """
diff --git a/tests/lightgbm/test_LightGbmTreeEnsembleConverters.py b/tests/lightgbm/test_LightGbmTreeEnsembleConverters.py
index d6f79a4..41f526e 100644
--- a/tests/lightgbm/test_LightGbmTreeEnsembleConverters.py
+++ b/tests/lightgbm/test_LightGbmTreeEnsembleConverters.py
@@ -10,7 +10,7 @@ from lightgbm import LGBMClassifier, LGBMRegressor
 from onnxmltools import convert_lightgbm
 from onnxmltools.convert.common.data_types import FloatTensorType
 from onnxmltools.utils import dump_data_and_model
-from onnxmltools.utils import dump_one_class_classification, dump_binary_classification, dump_multiple_classification
+from onnxmltools.utils import dump_binary_classification, dump_multiple_classification
 from onnxmltools.utils import dump_multiple_regression, dump_single_regression
 
 
diff --git a/tests/xgboost/test_xgboost_converters.py b/tests/xgboost/test_xgboost_converters.py
new file mode 100644
index 0000000..ceba5c1
--- /dev/null
+++ b/tests/xgboost/test_xgboost_converters.py
@@ -0,0 +1,54 @@
+"""
+Tests scilit-learn's tree-based methods' converters.
+"""
+import sys
+import unittest
+from sklearn.datasets import load_iris
+from xgboost import XGBRegressor, XGBClassifier
+from onnxmltools.convert import convert_xgboost
+from onnxmltools.convert.common.data_types import FloatTensorType
+from onnxmltools.utils import dump_multiple_classification, dump_single_regression, dump_binary_classification
+
+
+class TestXGBoostModels(unittest.TestCase):
+
+    @unittest.skipIf(sys.version_info[0] == 2, reason="xgboost converted not tested on python 2")
+    def test_xgb_regressor(self):
+        iris = load_iris()
+        X = iris.data[:, :2]
+        y = iris.target
+
+        xgb = XGBRegressor()
+        xgb.fit(X, y)
+        conv_model = convert_xgboost(xgb, initial_types=[('input', FloatTensorType(shape=[1, 'None']))])
+        self.assertTrue(conv_model is not None)
+        dump_single_regression(xgb, suffix="-Dec4")
+
+    @unittest.skipIf(sys.version_info[0] == 2, reason="xgboost converted not tested on python 2")
+    def test_xgb_classifier(self):
+        iris = load_iris()
+        X = iris.data[:, :2]
+        y = iris.target
+        y[y == 2] = 0
+
+        xgb = XGBClassifier()
+        xgb.fit(X, y)
+        conv_model = convert_xgboost(xgb, initial_types=[('input', FloatTensorType(shape=[1, 'None']))])
+        self.assertTrue(conv_model is not None)
+        dump_binary_classification(xgb, verbose=True)
+
+    @unittest.skipIf(sys.version_info[0] == 2, reason="xgboost converted not tested on python 2")
+    def test_xgb_classifier_multi(self):
+        iris = load_iris()
+        X = iris.data[:, :2]
+        y = iris.target
+
+        xgb = XGBClassifier()
+        xgb.fit(X, y)
+        conv_model = convert_xgboost(xgb, initial_types=[('input', FloatTensorType(shape=[1, 'None']))])
+        self.assertTrue(conv_model is not None)
+        dump_multiple_classification(xgb, allow_failure="StrictVersion(onnx.__version__) < StrictVersion('1.3.0')")
+
+
+if __name__ == "__main__":
+    unittest.main()
diff --git a/tests/xgboost/test_xgboost_unpickle_06.py b/tests/xgboost/test_xgboost_unpickle_06.py
new file mode 100644
index 0000000..43febfa
--- /dev/null
+++ b/tests/xgboost/test_xgboost_unpickle_06.py
@@ -0,0 +1,27 @@
+"""
+Tests scikit-linear converter.
+"""
+import sys
+import os
+from onnxmltools.convert.xgboost import convert as convert_xgboost
+import unittest
+import pickle
+from onnxmltools.convert.common.data_types import FloatTensorType
+
+
+class TestXGBoostUnpickle06(unittest.TestCase):
+
+    @unittest.skipIf(sys.version_info[0] == 2, reason="xgboost converted not tested on python 2")
+    def test_xgboost_unpickle_06(self):
+        # Unpickle a model trained with an old version of xgboost.
+        this = os.path.dirname(__file__)
+        with open(os.path.join(this, "xgboost10day.pickle.dat"), "rb") as f:
+            xgb = pickle.load(f)
+
+        conv_model = convert_xgboost(xgb, initial_types=[('features', FloatTensorType([1, 10000]))])
+        assert conv_model is not None
+
+
+if __name__ == "__main__":
+    unittest.main()
+