twitter functions + avro + code refactor