Build: Fix testswarm task, use API correctly

(cherry-picked from 22c515c6717efe46685b43215208aa4e5648fbc8)