changes to support pulling by url correctly