Repository: onnx/onnx

Issue:
Proposal: Op Registry
Once we start getting into more complicated passes having knowledge about operators becomes incredibly useful. What I propose is to maintain a registry of operators mapping to a set of descriptors. 

For example let's take a look at the [eliminate_nop_monotone_argmax](https://github.com/onnx/onnx/blob/master/onnx/optimizer/passes/eliminate_nop_monotone_argmax.h) pass. Within the pass we have to maintain an unordered set of ops that are monotone. The correct design would be to have a registry which we can ask whether or not if a certain op is monotone. Once such a registry is implemented, passes can be generalized through descriptors, therefore as we add new ops and their respective descriptors our passes naturally scale out.

Some possible descriptors we could have are:
- monotonic
- elementwise independent (op works on an individual element without knowledge of other elements, such as log, but not softmax since it requires calculating a sum over an axis)
- elementwise dependent (softmax, logsoftmax, softmin, softplus)
- reduction
- shape transform (e.g. transpose, view)

@houseroad @bddppq What do you guys think?


Discussion:
@houseroad Any comments on this? I have a couple of passes in mind that could benefit from this.
We can maintain our own mapping, also we can extend existing operator schema to specify such attributes: https://github.com/onnx/onnx/blob/master/onnx/defs/schema.h

I prefer the latter, since it's like a descriptor of the operator.
@houseroad I think adding this to the operator schema is a great idea. What should we call such attributes (i.e. OpAnnotation, OptimAnnotation)? Also if we went this way, is there a easy way to view these attributes through the IR?

I can start working on this once we've cleared up the scope.
@houseroad So I was taking a look at `ir_pb_converter.cc` and it seems that the IR is constructed from a proto object. So in order to have these annotations visible to the IR wouldn't we need to change onnx.proto?
Either OpAnnotation or OptimAnnotation sounds good to me. We can add some helper functions to provide such access in our operator schema (not IR). We should not change IR, because this is purely about the operator spec.

Resolving patch:
diff --git a/onnx/optimizer.py b/onnx/optimizer.py
--- a/onnx/optimizer.py
+++ b/onnx/optimizer.py
@@ -26,13 +26,23 @@
 
 Supported pass names:
     -- nop
+    -- eliminate_deadend
     -- eliminate_identity
-    -- eliminate_nop_transpose
+    -- eliminate_nop_dropout
+    -- eliminate_nop_monotone_argmax
     -- eliminate_nop_pad
+    -- eliminate_nop_transpose
     -- eliminate_unused_initializer
+    -- extract_constant_to_initializer
+    -- fuse_add_bias_into_conv
+    -- fuse_bn_into_conv
+    -- fuse_consecutive_concats
+    -- fuse_consecutive_log_softmax
+    -- fuse_consecutive_reduce_unsqueeze
     -- fuse_consecutive_squeezes
     -- fuse_consecutive_transposes
-    -- fuse_add_bias_into_conv
+    -- fuse_matmul_add_bias_into_gemm
+    -- fuse_pad_into_conv
     -- fuse_transpose_into_gemm
 """
 
@@ -45,8 +55,10 @@ def optimize(model, passes=None, fixed_point=False):  # type: (ModelProto, Optio
                   'eliminate_nop_pad',
                   'fuse_consecutive_transposes',
                   'fuse_transpose_into_gemm']
+
     if not isinstance(model, ModelProto):
-        raise ValueError('Optimizer only accepts ModelProto, incorrect type: {}'.format(type(model)))
+        raise ValueError(
+            'Optimizer only accepts ModelProto, incorrect type: {}'.format(type(model)))
 
     model_str = model.SerializeToString()
     if fixed_point:


Test patch: