Package: kml-analysis-parasKumarSahu
Knolml-Analysis-PackageThe aim of this project is to do various types of analysis on knolml which can be used by a reseracher who is working on wikipedia data.Analysis1: Controversy Analysis using wiki-linksTo measure the relative controversy level of various wiki-links present in a wikipedia article.Module: from kml_analysis_pkg import controversyInput Format: controversy.run(input_file_name)Example: controversy.run("sample.knolml")Analysis2: Contributions of an author over a given period of time in a wikipedia articleTo find contributions of an author in terms of words, sentences, bytes etc over a given period of time (given starting and ending dates)Module: from kml_analysis_pkg import author_contributionInput Format: author_contribution.run("sample.knolml", start_date, end_date, measure_option)Date Format: YYYY-MM-DDMeasure Options: sentences/bytes/wikilinks/wordsExample: author_contribution.run("sample.knolml", "2000-01-01", "2010-01-01", "words")Analysis3: Ranking all the authors based on their contribution to a given paragraphTo rank all the authors of a wikipedia article based on their contribution to a particular paragraph present in the article. The paragraph will be given as input to the program.Module: from kml_analysis_pkg import author_para_contributionInput Format: author_para_contribution.run(input_file_name)Example: author_para_contribution.run("sample.knolml")Analysis4: Finding external knowledge gaps in a wikipedia articleA wikipedia article represents knowledge about some related topics, like a wikipedia article on IIT Ropar may be talking about placements of IIT Ropar in a particular section. But, in this section there was no information regarding a new branch say Biotechnology which was newly introduced. So, can we write a Python program that can tell that the information regarding placements of Biotechnology is missing from the IIT Ropar wikipedia page? Or in general can we tell that there is a knowledge gap in a wikipedia article?Module: from kml_analysis_pkg import external_gapsInput Format: external_gaps.run(input_file_name, word_vector_model)Word Vector Model: Pretrained model avaiable athttps://github.com/parasKumarSahu/Knolml-Analysis/blob/master/Text-Segmentation/wrdvecs-text8.binExample: external_gaps.run("GeneralScience.txt", "wrdvecs-text8.bin")Analysis5: Finding internal knowledge gap in a wikipedia articleInternal Knowledge Gap arise due to difficulty in understanding of an article. We define various readability prameter and compute them for thousands of articles to study how these parameters affect the overall readability of a wikipedia article.To download featured articles(sorted by page views):-Module: from kml_analysis_pkg import downloader_featuredInput Format: downloader_featured.run(path_of_html_file)html_file avaiable athttps://github.com/parasKumarSahu/Knolml-Analysis/tree/master/Wiki%20Featured%20Articles%20DownloaderExample: downloader_featured.run("Wikipedia Featured articles - Wikipedia.htm")To download B, C, GA, Start and Stub articles:-Module: from kml_analysis_pkg import downloader_from_titlesInput Format: downloader_from_titles.run(path_of_file_containing_list_of_articles)Example: downloader_from_titles.run("B")To calaculate readability parameters:-Module: kml_analysis_pkg import internal_gapsInput Format: internal_gaps.run(path_of_folder_containing_articles, word_vector_model)Word Vector Model: Pretrained model avaiable athttps://github.com/parasKumarSahu/Knolml-Analysis/blob/master/Text-Segmentation/wrdvecs-text8.binExample: internal_gaps.run("test", "wrdvecs-text8.bin")