I'm good with keeping this like that for now. But most of what is done here, can be done as a graph of Theano op, except numpy.tril and numpy.triu.  When we have ops for them, it would be better to remove this op and return a graph of existing Theano ops instead of creating a specific op for this grad.