Package: logdiv
LogDiv: A Python Module for Computing Diversity in Transaction LogsLogDiv is a Python module for the computation of the diversity of items requested by users in transaction logs.It takes two inputs:A log file with transactions.A file with item atributes.Computing the diversity of items requested by users is a task of interest in many fields, such as sociology, recommender systems, e-commerce, and media studies. Check the example below.Getting StartedPrerequisitesLogDiv requires:PythonNumpy - EssentialPandas - EssentialMatplotlib - Essentialpyyaml - Essentialscikit-learn - Essentialtqdm - Optional: progression bar, only one function requires itGraph-tool - Optional: only one function requires it$python3-mpipinstallnumpy
$python3-mpipinstallpanda
$python3-mpipinstallmatplotlib$python3-mpipinstallpyyaml
$python3-mpipinstallscikit-learn
$python3-mpipinstalltqdmInstalling Graph-tool is more complicated:https://git.skewed.de/count0/graph-tool/wikis/installation-instructionsInstallingTo install LogDiv, you need to execute:$pipinstalllogdivSpecificationInputs formatLogDiv needs a specific format of inputs to run:A file describing all requests under a table format, whose fields are:user IDtimestamprequested item IDreferrer item IDA file describing all pages visited under a table format, whose fields are:item IDclassification 1classification 2...YAML fileCodes that use LogDiv are directed by a YAML file: if you want to modify input files, or the features you want to compute,
you just need to modify the YAML file, not the code itself.YAML file are similar to JSON file, once you load them, they take the form of a dictionnary. In your codes, you have for instance
a function that take a parameter that need to be changed often. You can give to your function the key of the dictionnary, and then
change the value in the YAML file. This allows to make less mistakes and take less time when you want to change parameters in your code.DocumentationIf you want precision on a function of LogDiv:what is the purpose of the functionwhat these functions take in inputwhat they returnyou need to run in a Console Python:>>>help(function)ExamplesYou dispose of two examples to familiarize yourself with LogDiv:Example 1 uses a short dataset to show how to use LogDivExample 2 uses a dataset of more than 100 thousands of requests to show what kind of results can be obtainedThese examples (dataset, script and yaml file) can be found indatasetsdirectory. These YAML files are self-explanatory.Example 1The following example illustrates the inputs format of the package.usertimestamprequested_itemreferrer_itemuser12019-07-03 00:00:00v1v4user12019-07-03 00:01:00v4v2user12019-07-03 00:01:10v4v6user12019-07-03 00:01:20v4v6user12019-07-03 00:02:00v6v9user12019-07-03 03:00:00v8v10user12019-07-03 03:01:00v8v5user22019-07-05 12:00:00v3v5user22019-07-05 12:00:30v5v7user22019-07-05 12:00:45v7v9user22019-07-05 12:01:00v9v6user32019-07-05 18:00:00v10v5user32019-07-05 18:01:15v10v7user32019-07-05 18:03:35v10v9user32019-07-05 18:06:00v7v4user32019-07-05 18:07:22v5v2itemclass1class2class3v1x\alphahv2y\betahv3y\betafv4x\betahv5z\gammmafv6y\alphahv7z\alphafv8x\gammmafv9y\alphafv10z\gammmahIf you want to run example 1, you need to be in the directory datasets/example1, and run:$python3example_1.pyExample 2This figure is the Gephi graph of dataset 2, where each color correspond to a different media.The file describing requests has the same structure than the one in example 1.The file describing pages is more concrete than the one in example 1:itemmediacontinentitem0PoliticsEuropeitem1HealthAsiaitem2PoliticsNorth AmericaIf you want to run example 2, you need to be in the directory datasets/example2, and run:$python3example_2.py