files are now fully added
computed shard_key added