added preprocess and build_dict functions