I think you need to tell `json.dumps` to dump keys in order. Otherwise this could generate a string of various orders and that'll fail the equality check.