everything before creating D3 json files