Package: capexplain
CapeAs an initial contribution we have developed a novel system called Cape (Counterbalancing with Aggregation Patterns for Explanations) for explaining outliers in aggregation queries through counterbalancing. That is, explanations are outliers in the opposite direction of the outlier of interest.Cape(Counterbalancing withAggregationPatterns forExplanations) is a system that explains outliers (surprisingly low or high) aggregation function results for group-by queries in SQL. The user provides the system with a query and a surprising outcome for this query. Cape then uses patterns discovered over the input data of the query in an offline pattern mining step to explain the outlier by finding anrelatedoutlier in the opposite direction thatcounterbalancesthe outlier of interest.InstallationPrerequisitesCape requires python 3 and uses python'stkinterfor its graphical UI. For example, on ubuntu you can install the prerequisites with:sudoapt-getinstallpython3python3-pippython3-tkInstall with pipMake sure you have pip installed (see previous step).pip3installcapexplainInstall from githubAlternatively, you can directly install from source. For that you need python3 and the setuptools package. You probably would want to use avirtual environmentfor that. If you havepython3andpipinstalled you can install/update thesetuptoolspackage by running:</code></pre>
<p>ToinstallCape,clonethegithubrepositoryanduse<code>setup.py</code>toinstall.</p>
<prelang="shell"><code>gitclonegit@github.com:IITDBGroup/cape.gitcapexplaincdcapexplain
python3setup.pyinstallUsageCape provides a single binarycapexplainthat support multiple subcommands. The general form is:capexplainCOMMAND[OPTIONS]Options are specific to each subcommand. Usecapexplain helpto see a list of supported commands andcapexplain help COMMANDget more detailed help for a subcommand.OverviewCape currently only supports PostgreSQL as a backend database (version 9 or higher). To use Cape to explain an aggregation outlier, you first have to let cape find patterns for the table over which you are aggregating. This an offline step that only has to be executed only once for each table (unless you want to re-run pattern mining with different parameter settings). Afterwards, you can either use the commandline or Cape's UI to request explanations for an outlier in an aggregation query result.Mining PatternsUsecapexplain mine [OPTIONS]to mine patterns. Cape will store the discovered patterns in the database. The "mined" patterns will be stored in a created schema calledpattern, and the pattern tables generated after runningminecommand arepattern.{target_table}_globalandpattern.{target_table}_local. At the minimum you have to tell Cape how to connect to the database you want to use and which table it should generate patterns for. Runcapexplain help mineto get a list of all supported options for the mine command. The options needed to specify the target table and database connection are:-h,--host<arg>-databaseconnectionhostIPaddress(DEFAULT:127.0.0.1)-u,--user<arg>-databaseconnectionuser(DEFAULT:postgres)-p,--password<arg>-databaseconnectionpassword
-d,--db<arg>-databasename(DEFAULT:postgres)-P,--port<arg>-databaseconnectionport(DEFAULT:5432)-t,--target-table<arg>-minepatternsforthistableFor instance, if you run a postgres server locally (default) with userpostgres(default), passwordtest, and want to mine patterns for a tableemployeesin databasemydb, then run:capexplainmine-ptest-dmydb-temployeesMining algorithm parametersCape's mining algorithm takes the following arguments:--gof-const<arg>-goodness-of-fitthresholdforconstantregression(DEFAULT:0.1)--gof-linear<arg>-goodness-of-fitthresholdforlinearregression(DEFAULT:0.1)--confidence<arg>-globalconfidencethreshold
-r,--regpackage<arg>-regressionanalysispackagetouse{'statsmodels','sklearn'}(DEFAULT:statsmodels)--local-support<arg>-localsupportthreshold(DEFAULT:10)--global-support<arg>-globalsupportthresh(DEFAULT:100)-f,--fd-optimizations<arg>-activatefunctionaldependencydetectionandoptimizations(DEFAULT:False)-a,--algorithm<arg>-algorithmtouseforpatternmining{'naive','optimized','naive_alternative'}(DEFAULT:optimized)--show-progress<arg>-showprogressmeters(DEFAULT:True)--manual-config-manuallyconfigurenumeric-likestringfields(treatfieldsasstringornumeric?)(DEFAULT:False)Running our "crime" data exampleWe included a subset of the "Chicago Crime" dataset (https://data.cityofchicago.org/Public-Safety/Crimes-2001-to-present/)
in our repository for user to play with. To import this dataset in your postgres databse, under/testdbdirectory, run the following command template:psql-h<host>-U<username>-d<localdatabasenamewhereyouwanttostoreourexampletable><~/cape/testdb/crime_demonstration.sqlthen run thecapexplaincommands accordingly to explore this example.Explaining OutliersTo explain an aggregation outlier usecapexplain explain [OPTIONS].-l,--log<arg>-selectloglevel{DEBUG,INFO,WARNING,ERROR}(DEFAULT:ERROR)--help-showthishelpmessage
-h,--host<arg>-databaseconnectionhostIPaddress(DEFAULT:127.0.0.1)-u,--user<arg>-databaseconnectionuser(DEFAULT:postgres)-p,--password<arg>-databaseconnectionpassword
-d,--db<arg>-databasename(DEFAULT:postgres)-P,--port<arg>-databaseconnectionport(DEFAULT:5432)--ptable<arg>-tablestoringaggregateregressionpatterns
--qtable<arg>-tablestoringaggregationqueryresult
--ufile<arg>-filestoringuserquestion
-o,--ofile<arg>-filetowriteoutputto
-a,--aggcolumn<arg>-columnthatwasinputtotheaggregationfunctionforexplainoption, besides the common options, user should give--ptable,thepattern.{target_table}and--qtable, thetarget_table. Also, we currently only allow user pass question through a.txtfile, user need to put the question in the following format:attribute1,attribute2,attribute3....,direction
value1,value2,value3....,high/lowplease refer toinput.txtto look at an example.Starting the Explanation Explorer GUICape comes with a graphical UI for running queries, selecting outliers of interest, and exploring patterns that are relevant for an outlier and browsing explanations generated by the system. You need to specify the Postgres server to connect to. The explorer can only generate explanations for queries over tables for which patterns have mined beforehand usingcapexplain mine.
Here is our demo video : (https://www.youtube.com/watch?v=gWqhIUrcwz8)$capexplainhelpgui
capexplaingui[OPTIONS]:OpentheCapegraphicalexplanationexplorer.

SUPPORTEDOPTIONS:
-l,--log<arg>-selectloglevel{DEBUG,INFO,WARNING,ERROR}(DEFAULT:ERROR)--help-showthishelpmessage
-h,--host<arg>-databaseconnectionhostIPaddress(DEFAULT:127.0.0.1)-u,--user<arg>-databaseconnectionuser(DEFAULT:postgres)-p,--password<arg>-databaseconnectionpassword
-d,--db<arg>-databasename(DEFAULT:postgres)-P,--port<arg>-databaseconnectionport(DEFAULT:5432)For instance, if you run a postgres server locally (default) with userpostgres(default), passwordtest, and databasemydb, then run:capexplaingui-ptest-dmydbLinksCape is developed by researchers at Illinois Institute of Technology and Duke University. For more information and publications see the Cape project pagehttp://www.cs.iit.edu/~dbgroup/projects/cape.html.