Module Import alias ImportFrom alias Import alias Import alias Import alias ImportFrom alias Import alias Import alias Import alias Import alias ImportFrom alias ClassDef bases AttributeLoad NameLoad attr body FunctionDef arguments args NameParam NameParam defaults body Assign AttributeStore NameLoad attr NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Str Assign NameStore Call AttributeLoad NameLoad attr Str Str Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr Str Str Assign NameStore Call AttributeLoad NameLoad attr Str Str Assign NameStore Call AttributeLoad NameLoad attr Str Str Assign NameStore Call AttributeLoad NameLoad attr Str Str Assign NameStore Call AttributeLoad NameLoad attr Str Str Assign NameStore Call AttributeLoad NameLoad attr Str Str Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr keyword NameLoad keyword NameLoad keyword NameLoad keyword NameLoad keyword NameLoad keyword NameLoad keyword NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr decorator_list FunctionDef arguments args NameParam defaults body While UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore SubscriptLoad NameLoad Index Num Assign NameStore SubscriptLoad NameLoad Index Num Expr Call AttributeLoad NameLoad attr NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Assign NameStore Call NameLoad NameLoad NameLoad If CompareEq NameLoad Str body Return Num Assign NameStore Call NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str keyword Dict Str Str Assign NameStore Num Assign NameStore Str Assign NameStore ListLoad Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr For NameStore NameLoad body Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Str Assign NameStore Call AttributeLoad NameLoad attr Str Str Assign NameStore BinOpAdd NameLoad ListLoad TupleLoad NameLoad Call NameLoad NameLoad Num Num Num NameLoad Num Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad Return AttributeLoad AttributeLoad NameLoad attr attr decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Assign NameStore Num For NameStore Call NameLoad Num Num body Assign NameStore BinOpAdd BinOpAdd BinOpAdd Str NameLoad Str Call NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad Assign NameStore BinOpAdd NameLoad NameLoad If CompareLtE NameLoad Num body Break If CompareEq BinOpMod NameLoad Num Num body Print BinOpAdd BinOpAdd BinOpAdd BinOpAdd BinOpAdd BinOpAdd BinOpAdd BinOpAdd BinOpAdd Call NameLoad NameLoad Str Call AttributeLoad NameLoad attr Str NameLoad Str Call NameLoad NameLoad Str Call NameLoad NameLoad Str Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr Assign NameStore Str Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad TupleLoad NameLoad NameLoad decorator_list decorator_list ClassDef bases body FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Str Assign NameStore Call AttributeLoad NameLoad attr Str Str Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr Str Str Assign NameStore Call AttributeLoad NameLoad attr Str Str Assign NameStore Call AttributeLoad NameLoad attr Str Str Assign NameStore Call AttributeLoad NameLoad attr Str Str Assign NameStore Call AttributeLoad NameLoad attr Str Str Assign NameStore Call AttributeLoad NameLoad attr Str Str Assign AttributeStore NameLoad attr Call NameLoad Call AttributeLoad NameLoad attr Str Str Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr keyword NameLoad keyword NameLoad keyword NameLoad keyword NameLoad keyword NameLoad keyword NameLoad keyword NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr Assign NameStore BinOpSub NameLoad Num Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore ListLoad Assign NameStore Num Assign NameStore Str Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad TupleLoad NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr For NameStore NameLoad body Assign NameStore Call NameLoad SubscriptLoad NameLoad Index Num Expr Call AttributeLoad NameLoad attr ListLoad NameLoad NameLoad Assign NameStore BinOpAdd NameLoad Num For NameStore Call NameLoad AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr Call NameLoad NameLoad For NameStore Call NameLoad AttributeLoad NameLoad attr body Expr Call AttributeLoad SubscriptLoad NameLoad Index NameLoad attr For NameStore Call NameLoad AttributeLoad NameLoad attr body Expr Call AttributeLoad SubscriptLoad NameLoad Index NameLoad attr Expr Call AttributeLoad AttributeLoad NameLoad attr attr Print Str decorator_list decorator_list If CompareEq NameLoad Str body Assign NameStore Call NameLoad Expr Call AttributeLoad NameLoad attr
N/A N/A MySQLdb bs4 BeautifulSoup N/A json N/A re N/A time math ceil N/A logging N/A threading N/A Queue N/A ConfigParser util get_content UpdateOneTopic N/A N/A threading Thread N/A __init__ N/A N/A self queue N/A N/A N/A N/A self queue queue N/A N/A N/A N/A threading Thread __init__ self N/A cf N/A N/A ConfigParser ConfigParser N/A N/A N/A cf read config.ini N/A host N/A N/A cf get db host N/A port N/A int N/A N/A cf get db port N/A user N/A N/A cf get db user N/A passwd N/A N/A cf get db passwd N/A db_name N/A N/A cf get db db N/A charset N/A N/A cf get db charset N/A use_unicode N/A N/A cf get db use_unicode N/A N/A self db N/A N/A MySQLdb connect host host port port user user passwd passwd db db_name charset charset use_unicode use_unicode N/A N/A self cursor N/A N/A N/A self db cursor N/A run N/A N/A self N/A N/A N/A N/A N/A N/A N/A self queue empty N/A N/A t N/A N/A N/A self queue get N/A link_id N/A t N/A 0 N/A count_id N/A t N/A 1 N/A N/A N/A self find_new_question_by_topic link_id count_id N/A find_question_by_link N/A N/A self topic_url count_id N/A N/A N/A content N/A get_content topic_url count_id N/A N/A content FAIL N/A N/A 0 N/A soup N/A BeautifulSoup content N/A questions N/A N/A soup findAll a attrs N/A class question_link N/A i 0 N/A p_str INSERT IGNORE INTO QUESTION (NAME, LINK_ID, FOCUS, ANSWER, LAST_VISIT, ADD_TIME, TOP_ANSWER_NUMBER) VALUES (%s, %s, %s, %s, %s, %s, %s) N/A anser_list N/A N/A time_now N/A int N/A N/A time time N/A question questions N/A N/A tem_text N/A N/A question get_text N/A tem_id N/A N/A question get href N/A tem_id N/A N/A tem_id replace /question/ N/A anser_list N/A anser_list N/A N/A tem_text N/A int tem_id 0 0 0 time_now 0 N/A N/A N/A N/A self cursor executemany p_str anser_list N/A N/A N/A self cursor rowcount N/A find_new_question_by_topic N/A N/A self link_id count_id N/A N/A N/A new_question_amount_total 0 N/A i N/A range 1 7 N/A N/A topic_url N/A N/A N/A http://www.zhihu.com/topic/ link_id /questions?page= N/A str i N/A new_question_amount_one_page N/A N/A self find_question_by_link topic_url count_id N/A new_question_amount_total N/A new_question_amount_total new_question_amount_one_page N/A N/A new_question_amount_one_page 2 N/A N/A N/A N/A N/A count_id 2 0 N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A str count_id  ,  N/A N/A self getName  Finshed TOPIC  link_id , page  N/A str i  ; Add  N/A str new_question_amount_total  questions. N/A time_now N/A int N/A N/A time time N/A sql UPDATE TOPIC SET LAST_VISIT = %s WHERE LINK_ID = %s N/A N/A N/A N/A self cursor execute sql N/A time_now link_id N/A N/A UpdateTopics N/A N/A __init__ N/A N/A self N/A N/A N/A cf N/A N/A ConfigParser ConfigParser N/A N/A N/A cf read config.ini N/A host N/A N/A cf get db host N/A port N/A int N/A N/A cf get db port N/A user N/A N/A cf get db user N/A passwd N/A N/A cf get db passwd N/A db_name N/A N/A cf get db db N/A charset N/A N/A cf get db charset N/A use_unicode N/A N/A cf get db use_unicode N/A N/A self topic_thread_amount N/A int N/A N/A cf get topic_thread_amount topic_thread_amount N/A N/A self db N/A N/A MySQLdb connect host host port port user user passwd passwd db db_name charset charset use_unicode use_unicode N/A N/A self cursor N/A N/A N/A self db cursor N/A run N/A N/A self N/A N/A N/A time_now N/A int N/A N/A time time N/A before_last_vist_time N/A time_now 10 N/A queue N/A N/A Queue Queue N/A threads N/A N/A i 0 N/A sql SELECT LINK_ID FROM TOPIC WHERE LAST_VISIT < %s ORDER BY LAST_VISIT N/A N/A N/A N/A self cursor execute sql N/A before_last_vist_time N/A results N/A N/A N/A self cursor fetchall N/A row results N/A N/A link_id N/A str N/A row N/A 0 N/A N/A N/A queue put N/A link_id i N/A i N/A i 1 N/A i N/A range N/A self topic_thread_amount N/A N/A N/A N/A threads append N/A UpdateOneTopic queue N/A i N/A range N/A self topic_thread_amount N/A N/A N/A N/A N/A threads N/A i start N/A i N/A range N/A self topic_thread_amount N/A N/A N/A N/A N/A threads N/A i join N/A N/A N/A N/A self db close N/A All task done N/A N/A N/A N/A __name__ __main__ N/A N/A topic_spider N/A UpdateTopics N/A N/A N/A topic_spider run