azure-batch-apps 0.5.2
Batch Apps Python Client
License: MIT License

===============================
Azure Batch Apps Python Client
===============================

The package is to enable Azure Batch Apps customers to interact with the
Management API using Python.

This client module is designed to work with the applications set up within an 
existing Batch Apps service.
You can upload your Application Image and Cloud Assembly via the `Batch Apps Portal <https://manage.batchapps.windows.net/>`_.
For more information on setting this up, `check out this article <http://azure.microsoft.com/en-us/documentation/articles/batch-dotnet-get-started/#tutorial2>`_.


License
========

This project is licensed under the MIT License.
For details see LICENSE.txt or visit `<http://opensource.org/licenses/MIT>`_

Installation
============

This package has been tested with Python 2.6, 2.7, 3.2, 3.3 and 3.4

>> pip install azure-batch-apps

Required packages:

* `Requests <http://docs.python-requests.org/en/latest/>`_

* `Keyring <https://bitbucket.org/kang/python-keyring-lib>`_

* `Requests-OAuthlib <http://requests-oauthlib.readthedocs.org/en/latest/>`_


Documentation
=============

The documentation is generated by Sphinx and can be found zipped up in the project 
root. It is also hosted online `here <http://dl.windowsazure.com/batchapps/pythondocs/>`_.


Release History
================

For full summary of changes, see CHANGES.txt

* 2015-07-15	- 0.5.2
	- Fixed bug in download streaming
* 2015-07-09	- 0.5.1
	- Added auto-refresh for unattended tokens
	- Exposed configuration of block size to vary upload and download callback frequency
* 2015-07-01	- 0.5.0
	- Added progress callbacks for file upload and download
	- Removed 3 instance minimum for pool creation
* 2015-05-11	- 0.4.0
	- Added optional auth config validation
	- Added JobSubmission.settings attribute
* 2015-01-13	- 0.3.0
	- Added preliminary support for Batch Apps pool management
* 2014-11-26	- 0.2.0	
	- Changed file upload format
	- Changed Authentication config format
	- Changed terminology: application to jobtype
	- Changed terminology: service principal to unattended account
	- Added FileCollection.index method
	- Added better handling for missing auth values in Configuration
* 2014-11-03	- 0.1.1 
	- Authentication bug fixes
* 2014-10-28	- 0.1.0	
	- Beta Release


Usage
============

Application Configuration
--------------------------

In order to interact with the applications set up in your services in your Batch Apps 
account, you will need to configure the python client.

When you instantiate a Configuration object for the first time, the configuration 
file will be created by default as::

	$HOME/BatchAppsData/batch_apps.ini

A single configuration object represent a single service in your Batch Apps account.
This means that each configuration needs an endpoint and client ID.

To set up a new job type reference you can add it to the configuration file, 
along with any custom parameters you want associated with it.

You can edit the file directly, or via the Configuration class::

	from batchapps import Configuration

	# These can be retrieved when creating an unattended account in the Batch Apps portal.
	# See the authentication section below for more details.
	endpoint = 'myservice.batchapps.core.windows.net'
	account_details = 'ClientID=xxxxxxxx;TenantID=abcdefg'
	account_key = '12345'

	cfg = Configuration(log_level='debug', default=True)
	cfg.aad_config(account=account_details, key=account_key,
		endpoint=endpoint, unattended=True)

	cfg.add_jobtype('my_job_type')

	# Set this job type as the current job type
	cfg.current_jobtype('my_job_type')

	# Set the current job type as the default job type for future jobs
	cfg.set_default_jobtype()

	# Set up some default parameter values for the current job type
	cfg.set('quality', 10)
	cfg.set('timeout', 500)

	# Save updated configuration to file
	cfg.save_config()

Authentication
---------------

The module authenticates with Azure Active Directory (an implementation of OAuth2).
The batchapps module provides a helper class to assist in retrieving an AAD token 
using Requests-OAuthlib. However if you have a preferred OAuth implementation, you 
can authenticate with this instead.

You can create a set of "Unattended Account" credentials in your 
`Batch Apps account <https://manage.batchapps.windows.net/>`_. These will be in the 
format::

	Account Id = ClientId=abc;TenantId=xyz
	Account Key = ***********************

Once you have these credentials, you can authenticate the python client by adding 
them to the batch_apps.ini configuration either with Python, as described above, 
or by editing the file directly::

	[Authentication]
	endpoint = myservice.batchapps.core.windows.net
	unattended_account = ClientID=abc;TenantID=xyz
	unattended_key = ***********************

Then you can authenticate with these credentials::

	from batchapps import AzureOAuth

	creds = AzureOAuth.get_unattended_session()


Or alternatively, if you use a different AAD implementation to retrieve a token::

	from batchapps import Credentials, Configuration
	import my_oauth

	client_id = "abc"
	cfg = Configuration()

	aad_token = my_oauth.get_token(client_id)
	creds = Credentials(cfg, client_id, token=aad_token)

Authentication via logging into a Web UI will be supported soon.


Job Management
---------------

Job management, including submission, monitoring, and accessing outputs is done 
through the JobManager class::

	from batchapps import AzureOAuth, JobManager
	import time

	creds = AzureOAuth.get_unattended_session()
	mgr = JobManager(creds)

	my_job = mgr.create_job("First Job")
	
	# Apply any custom parameters and source files here
	my_job.example_parameter = "test123"

	# Then submit the job
	new_job = my_job.submit()

	job_progress = mgr.get_job(url=new_job['link'])
	
	# Let's allow up to 30 minutes for the job to complete
	timeout = time.time() + 1800

	while time.time() < timeout:

		if job_progress.status is 'Complete':
			job_progress.get_output('c:\\my_download_dir')
			break

		if job_progress.status is 'Error':
			break

		time.sleep(30)
		job_progress.update()
	
	else:
		job_progress.cancel()


File Management
----------------

File management, including syncing job source files and dependencies to 
the cloud can be done using the FileManager class::

	from batchapps import AzureOAuth, FileManager

	creds = AzureOAuth.get_unattended_session()
	mgr = FileManager(creds)

	file_collection = mgr.files_from_dir('c:\\my_job_assets')
	job_source = mgr.file_from_path('C:\\start_job.bat')
	file_collection.add(job_source)

	file_collection.upload()

	# Check files previously uploaded matching a certain name
	mgr.find_files('start_job.bat')

	# Retrieve a list of all uploaded files
	mgr.list_files()


Pool Management
----------------

Pool management, including creating, resizing and deleting pools can
be done using the PoolManager class.

Once a pool has been created, jobs can be submitted to it. By default, 
when a job has been submitted without referencing an existing pool, it will 
use an auto-pool which will be created for the running of the job, then 
deleted on completion::

	from batchapps import AzureOAuth, PoolManager

	creds = AzureOAuth.get_unattended_session()
	mgr = PoolManager(creds)

	new_pool = mgr.create_pool(target_size=5)

	# Create new job submission, then submit to pool
	my_job.pool = new_pool
	my_job.submit()

	# After job has completed, and we no longer need the pool
	pool.delete()