pycanvasdata 0.0.3
Python utilities for working with Canvas Data
License: MIT
Keywords: database canvasdata development

CanvasData Utilities
=======================

.. image:: https://travis-ci.org/kajigga/py_canvas_data.svg

Full documentation 

This python module is designed to make it easy to access Canvas Data files.

Currently, this module makes it possible to:

  - Convert downloaded Canvas Data files to CSV files with headers
  - export SQL table creation statements 
  - list files for a table
  - download all files or files for a specific table
  - view the Canvas Data schema (fields, field types, etc)
  - connect to a database (uses sqlalchemy) to create tables, import data, run SQL queries


----

Module Usage
------------

This module can be used programatically in other scripts and software. An
example of creating a canvas_data object is found below.::
  
  from canvas_data_utils.canvas_data_auth import CanvasData

  canvas_data_object = CanvasData(
      API_KEY=YOUR_API_KEY ,
      API_SECRET=YOUR_API_SECRET, 
      base_folder = YOUR_BASE_DIR,
      data_folder = YOUR_DATA_DIR)

  
Once you have that object created, you can...

generate mysql table creation statements

::

  mysql_table_creation_statement = canvas_data_object.table_creation_statement('mysql')

generate sqlite table creation statements

::

  sqlite_table_creation_statement = canvas_data_object.table_creation_statement('sqlite')

generate postgres table creation statements

::

  postgres_table_creation_statement = canvas_data_object.table_creation_statement('postgres')

create tables in a database given by a connection string

::

  canvas_data_object.create_tables('sqlite:///{}'.format(db_filename))

fetch the current schema (as json)

::

  schema = canvas_data_object.fetch_schema()

get a list of columns in a table

::

  user_dim_columns = canvas_data_object.get_schema_columns( 'user_dim')

convert an text file download from TSV (Tab Separated Values) to CSV

::

  canvas_data_object.convert_tsv_to_csv(tsv_filepath)

list all the tables in the schema

::

  table_list = canvas_data_object.table_list()

download and convert all files to CSV

::

  canvas_data_object.convert_all_to_csv()

list all downloadable files for a table

::

  file_list = canvas_data_object.list_all_files('user_dim')

----

Config File
------------
You need to create a config file somewhere. This config file is a typical .INI
file. It should look something like the following example.

::

  [config]
  API_SECRET = replace_with_api_secret_from_canvas_data
  API_KEY = replace_with_api_key_from_canvas_data

  base_folder = /path/to/base/folder/for/downloads/
  data_folder = %(base_folder)s/test2

  connection_string = sqlite:///%(base_folder)s/sample.db


Note: The connection_string configuration follows the connection pattern needed
by SQLAlchemy at http://docs.sqlalchemy.org/en/rel_1_0/core/engines.html.

This library supports any database type than SQLAlchemy does.

----

Command-line Tool
-----------------

This library includes a command line utility called `canvasdata`.

Usage
-----

::

  canvasdata [-h] [--config CONFIG] [-t TABLE] [--offline OFFLINE]
                  {convert_to_csv,import,create_tables,reset,sql_create_statement,list_files,download,sample_queries,schema}

  optional arguments:
    -h, --help            show this help message and exit
    --config CONFIG       path to the configuration file
    -t TABLE              specify a specific table
    --offline OFFLINE     run in offline mode