Module ImportFrom alias alias Import alias identifier Import alias identifier Import alias Import alias identifier Import alias Expr Call AttributeLoad NameLoad attr Assign NameStore Num Assign NameStore Num Assign NameStore NameLoad Assign NameStore Str Assign NameStore Str If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Import alias Assign NameStore Call AttributeLoad NameLoad attr NameLoad keyword NameLoad keyword Num With Call NameLoad NameLoad Str NameStore body For NameStore Call AttributeLoad NameLoad attr BinOpPow Num Num body Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore ListLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad keyword Str keyword NameLoad For TupleStore NameStore NameStore Call NameLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr body If CompareIs NameLoad Call AttributeLoad NameLoad attr Str body Assign SubscriptStore NameLoad Index NameLoad Call AttributeLoad SubscriptLoad NameLoad Index NameLoad attr Str Assign NameStore AttributeLoad Call AttributeLoad NameLoad attr Str attr Assign TupleStore NameStore NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad keyword Num keyword NameLoad Delete NameDel Assign NameStore Call NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad keyword Num Assign NameStore Call AttributeLoad NameLoad attr NameLoad Print Str Call AttributeLoad Call AttributeLoad NameLoad attr NameLoad attr Assign NameStore Call AttributeLoad SubscriptLoad NameLoad Index Str attr Assign NameStore AttributeLoad SubscriptLoad NameLoad Index CompareLt NameLoad NameLoad attr Assign NameStore AttributeLoad Call AttributeLoad SubscriptLoad NameLoad Index Str attr NameLoad attr Assign NameStore Call AttributeLoad AttributeLoad SubscriptLoad NameLoad Index Str attr attr Assign SubscriptStore NameLoad Index NameLoad Str Assign SubscriptStore NameLoad Index Str NameLoad Assign NameStore Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index Str Assign NameStore AttributeLoad NameLoad attr Assign NameStore AttributeLoad NameLoad attr Assign NameStore AttributeLoad Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index Str attr Assign NameStore Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index Str keyword Str Assign NameStore BinOpDiv BinOpSub NameLoad Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Str Assign NameStore Call AttributeLoad NameLoad attr Str Assign SubscriptStore NameLoad Index Str NameLoad Assign SubscriptStore NameLoad Index Str NameLoad Assign SubscriptStore NameLoad Index Str NameLoad Print Str Call AttributeLoad NameLoad attr Print Str Call AttributeLoad NameLoad attr Print Str Call AttributeLoad NameLoad attr Assign NameStore AttributeLoad SubscriptLoad NameLoad Index Str attr Assign NameStore AttributeLoad SubscriptLoad NameLoad Index Str attr Assign NameStore TupleLoad NameLoad NameLoad NameLoad NameLoad NameLoad Assign TupleStore NameStore NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad Assign TupleStore NameStore NameStore NameStore NameStore NameStore NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Call NameLoad Str Str keyword Num Expr Call AttributeLoad NameLoad attr NameLoad Call NameLoad Str Str keyword Num Expr Call AttributeLoad NameLoad attr Str Assign NameStore Call NameLoad keyword NameLoad keyword NameLoad keyword NameLoad keyword NameLoad keyword NameLoad keyword NameLoad keyword NameLoad keyword NameLoad Expr Call AttributeLoad NameLoad attr Str NameLoad Expr Call AttributeLoad NameLoad attr Call NameLoad Str Str NameLoad
N/A lda2vec preprocess Corpus N/A numpy np N/A pandas pd N/A logging N/A cPickle pickle N/A os.path N/A N/A N/A logging basicConfig N/A max_length 250 N/A min_author_comments 50 N/A nrows None N/A fn hacker_news_comments.csv N/A url https://zenodo.org/record/45901/files/hacker_news_comments.csv N/A N/A N/A N/A N/A os path exists fn N/A N/A requests N/A response N/A N/A requests get url stream True timeout 2400 N/A N/A open fn w fh N/A N/A data N/A N/A response iter_content N/A 1024 2 N/A N/A N/A N/A fh write data N/A features N/A N/A features N/A N/A pd read_csv fn encoding utf8 nrows nrows N/A N/A col dtype N/A zip N/A features columns N/A features dtypes N/A N/A N/A dtype N/A N/A np dtype int64 N/A N/A N/A features N/A col N/A N/A N/A features N/A col astype int32 N/A texts N/A N/A N/A features pop comment_text values N/A N/A tokens vocab N/A N/A preprocess tokenize texts max_length n_threads 4 merge True N/A texts N/A corpus N/A Corpus N/A N/A N/A corpus update_word_count tokens N/A N/A N/A corpus finalize N/A compact N/A N/A corpus to_compact tokens N/A pruned N/A N/A corpus filter_count compact min_count 10 N/A clean N/A N/A corpus subsample_frequent pruned N/A n_words N/A N/A N/A N/A np unique clean max N/A author_counts N/A N/A N/A features N/A comment_author value_counts N/A to_remove N/A N/A author_counts N/A N/A author_counts min_author_comments index N/A mask N/A N/A N/A N/A features N/A comment_author isin to_remove values N/A author_name N/A N/A N/A N/A features N/A comment_author values copy N/A N/A author_name N/A mask infrequent_author N/A N/A features N/A comment_author author_name N/A authors N/A N/A pd Categorical N/A features N/A comment_author N/A author_id N/A authors codes N/A author_name N/A authors categories N/A story_id N/A N/A N/A pd Categorical N/A features N/A story_id codes N/A story_time N/A N/A pd to_datetime N/A features N/A story_time unit s N/A days_since N/A N/A story_time N/A N/A story_time min N/A N/A pd Timedelta 1 day N/A time_id N/A N/A days_since astype int32 N/A N/A features N/A story_id_codes story_id N/A N/A features N/A author_id_codes story_id N/A N/A features N/A time_id_codes time_id N/A n_authors N/A N/A author_id max N/A n_stories N/A N/A story_id max N/A n_times N/A N/A time_id max N/A ranking N/A N/A features N/A comment_ranking values N/A score N/A N/A features N/A story_comment_count values N/A feature_arrs N/A story_id author_id time_id ranking score N/A N/A flattened features_flat N/A N/A corpus compact_to_flat pruned feature_arrs N/A N/A story_id_f author_id_f time_id_f ranking_f score_f features_flat N/A N/A N/A pickle dump corpus N/A open corpus w protocol 2 N/A N/A N/A pickle dump vocab N/A open vocab w protocol 2 N/A N/A N/A features to_pickle features.pd N/A data N/A dict flattened flattened story_id story_id_f author_id author_id_f time_id time_id_f ranking ranking_f score score_f author_name author_name author_index author_id N/A N/A N/A np savez data data N/A N/A N/A np save N/A open tokens w tokens