Lazy init the allocator