This second loop is only their as you miss state_below[mask] as on the cpu? Using that way would call gemm instead of gemv and so could be much faster. On this gpu back-end we have the AdvancedSubtensor1 Op that do that. Look into its perform to find how to do it in python. In your case, it won't need a loop in python.