Improve performance of aggregate

By dispatching manually and falling back to the generic interface testing
shows 2x-5x performance increase.