Package: download-manager
download-managerDownload Manager is a single python tool dedicated to help to download products from LTA and AIP.Download manager featuresThe tool has been implemented with the objective to implements the following features:Manage download in a configurable local folder,Manage partial downloads and recovery downloads,Manage parallelized downloadsManage bulk downloadManagement of connections error/retryMonitoring of downloads (bandwidth/progress) and errorsRun in command line (GUI is a nice to have)Support of multiple sourcesManage checksum validation of downloadsLocal storage management (identification of incomplete downloads to be resume, evictions...)Manage/anticipate quota limitation:bandwidth limitationparallel transfers numbertransfer volume per time ..Download list issued from an OData filterManage OData endpoint notifications/action when new product matching filter is up to allow performing routine
downloads.Install the download managerInstalling download-manager with execute the following in a terminal:pip install download-managerOptionsOptionTypeExplanationExample-s, --serviceTEXTService to requests data  [required]download_manager --service odata://my_service.com-f, --filtersTEXTFilter to apply to requests products by default only online products are requested.download_manager --service odata://my_service.com -f 'filter to apply'-O, --orderTEXTSort query (ASC or DESC).download_manager --service odata://my_service.com -O ASC-u, --usernameTEXTService connection username.download_manager --service odata://my_service.com -u user-p, --passwordTEXTService connection password.download_manager --service odata://my_service.com -p pwd-P, --process_numberINTEGERNumber of parallel download threads (default:2).download_manager --service odata://my_service.com -P 16-l, --limitINTEGERLimit the number matching products (default: 10)download_manager --service odata://my_service.com -l 100-o, --outputTEXTThe directory to store the downloaded files.download_manager --service odata://my_service.com -o /path/to/products-q, --quietBOOLEANSilent mode: only errors are reported.download_manager --service odata://my_service.com -q-c, --chunk-sizeINTEGERThe size of downloaded chunks (default: 4194304).download_manager --service odata://my_service.com -c 16777216-v, --verifyBOOLEANCheck file integrity by hashes.download_manager --service odata://my_service.com -v-r, --resumeBOOLEANResume downloading a partially downloaded file.download_manager --service odata://my_service.com -r-d, --databaseTEXTFolder to store the database if not present the database will be in a folder /.download_manager in the home directory..download_manager --service odata://my_service.com -d /path/to/your/db-S, --storage_limit_sizeINTEGERThe size max of file to download.download_manager --service odata://my_service.com -S 20000000-C, --continuousBOOLEANWhen present the download keep going download package by package.download_manager --service odata://my_service.com -C-b, --bulkTEXTPath to a csv file containing name of products to downloads.download_manager --service odata://my_service.com --bulk <Path_to_csv>--helpShow this message and exit.download_manager --helpGetting startedDownload one product, with one thread:download_manager --service odata://service.com/ -P 1 -u user -p password -l 1Download 10 products:download_manager --service odata://service.com/ -u user -p password -l 10Use the silent option:download_manager --service odata://service.com/ -u user -p password --quietUse filter to download specific products:download_manager --service odata://service.com/ -u user -p password -f "Online eq true and startswith(Name,'S1')"Use the resume option:download_manager --service odata://service.com/ -u user -p password --db_folder 'database_folder' --resumeUse the storage management option:download_manager --service odata://service.com/ -u user -p password -f "Online eq true and startswith(Name,'S2')" -l 25 -S 278209392 -o tests -db resourcesUse python code:fromdownload_manager.download_managerimportDownloadManagerfromrequests.authimportHTTPBasicAuthservice='https://odata.service.com'auth=HTTPBasicAuth("user","pwd")dm=DownloadManager(service=service,auth=auth)dm.start()nodes=dm.find_nodes(filter="startswith(Name, 'S2B')")fornodeinnodes:ifnode['Attributes']['cloudCover'].value==0:dm.submit(node)dm.join()dm.stop()LimitationFor now only odata implementation is available, quota still not supported.The error management is implemented but cannot be parametrized in command line.Offline product are not yet supported.