Repository: pytorch/glow

Issue #1641: Optimization pipeline design with more backend control
Created: 2018-09-13T11:50:59Z
Right now we have target-independent optimizations that may or may not benefit the backend executing the graph, and may even degrade performance. We need to allow the backend we are targeting have more control over what optimizations are performed.
For example, the GraphOptimizer's small matrix multiplication/batched add merging optimizations (mergeMatMul()/mergeBatchedAdd()) may not be beneficial (#1494).
Another example is the GraphOptimizer's optimizePool(), where we swap RELU -> MaxPool to MaxPool -> RELU. If the original subgraph is Conv -> RELU -> MaxPool, backends may prefer to vertically fuse Conv -> RELU into a single ConvRELU instead of  swapping the order of the RELU and MaxPool. So this optimization might be counterproductive/may prevent fusion from occurring.
Also related to #1633.

Duplicate issue #1839: [GraphOptimizer] Single use requirement for constant node in optimizeTranspose
Created: 2018-10-11T00:37:28Z
In optimizeTranspose we require for constant node to have a single use, however for the optimization to be correct it's not required because we create a new constant node with transposed data.
So why do we have this requirement? To avoid increasing memory footprint, similar to constant folding in optimizeReshape? Does it make sense, because as opposed to folding reshape, folding transpose saves cycles needed for data movement during runtime?

  
    
      glow/lib/Optimizer/GraphOptimizer.cpp
    
    
        Lines 1301 to 1323
      in
      f71543d
    
  
  
    

        
          
           /// Statically transpose Constants. 
        

        
          
           static void optimizeTranspose(Function *F) { 
        

        
          
             auto &nodes = F->getNodes(); 
        

        
          
            
        

        
          
             for (auto &node : nodes) { 
        

        
          
               auto *TN = dyn_cast<TransposeNode>(&node); 
        

        
          
               if (!TN) { 
        

        
          
                 continue; 
        

        
          
               } 
        

        
          
               auto *C = dyn_cast<Constant>(TN->getInput()); 
        

        
          
               // C must have a single use. 
        

        
          
               if (!C || !C->hasOneUse()) { 
        

        
          
                 continue; 
        

        
          
               } 
        

        
          
               // Create a new Constant NC to hold the transposed result. 
        

        
          
               auto *NC = 
        

        
          
                   F->getParent()->createConstant(TN->getResult().getType(), C->getName()); 
        

        
          
               // Transpose the value of C into NC. 
        

        
          
               genericTranspose(&C->getPayload(), &NC->getPayload(), TN->getShuffle()); 
        

        
          
               // Rewrite uses of TN to reference NC. 
        

        
          
               TN->getResult().replaceAllUsesOfWith(NC); 
        

        
          
             } 
        

        
          
           }

Closing comment (2018-10-11T01:31:32Z): To avoid increasing memory footprint, similar to constant folding in optimizeReshape?

Yes, I believe that is the motivation. There is of course a tradeoff here, and it's absolutely possible some backends may prefer to duplicate the Constant and transpose it instead, but for now there is no way for the backend to control this optimization. This is related to #1641