hasadna/Open-Knesset — pull request #20 by astupidog (2012-01-22 20:43:08+00:00)
Title: Committees video

There are 3 major changes:

1. Fixes to the update committee videos process - this process parses the knesset website and finds out mms videos we can download. The fixes are to better handle exceptions and to make sure all committees are parsed properly (I checked with production data). This process should already be running as a cronjob on the production server.

2. Added the download committees videos process - this process goes over the videos that were parsed in the update phase and downloads them to the data directory.
Need to set a cronjob with the following command:
bin/django update_videos --download --download-mb-quota=DOWNLOAD-MB-QUOTA --time-limit=TIME-LIMIT
DOWNLOAD-MB-QUOTA should be set according to the available disk space, bear in mind that each video is around 250MB and there are quite a lot of them.
TIME-LIMIT is the number of minutes this process will run.
For debugging you can add -v2 to see what the process is doing.

3. Added the upload committees videos process - this process goes over the videos that were downloaded in the download phase and uploads them to youtube.
* This process is using the gdata egg - I couldn't get it to install using buildout, so I did apt-get install python-gdata.
Prior to running this phase you need to create a youtube user that the videos will be downloaded under. Right now the limit for uploading is 64GB per video, which should not be a problem. Follow the instructions in settings.py regarding the paramters YOUTUBE_AUTHSUB_TOKEN and YOUTUBE_DEVELOPER_KEY.
Once that is setup you need to set a cronjob with the following command:
bin/django update_videos --upload --time-limit=TIME-LIMIT
TIME-LIMIT - the number of minutes this process will run (bear in mind that the timer check is not done while uploading, only between uploads)
For debugging you can add -v2 to see what the process is doing.
** both the download and upload phases can run at the same time, there shouldn't be a problem with that.

4. there are unit tests for all the processes and can be run using bin/django test knesset.video