Turns out you need to pass the format as oembed instead of URL -- I definitely need more API surrounding the oembed stuff.