Package: oarepo-upload-cli
oarepo-upload-cliPackage that synchronizes documents between the student system and repository up to some date.CLI UsageTo use the upload CLI tool, you first have to install the package somewhere.Installing upload CLI in a separate virtualenvCreate a separate virtualenv and install upload CLI into it:python3.10 -m venv .venv-upload-cli
(source .venv-upload-cli/bin/activate; pip install -U pip setuptools wheel; pip install oarepo-upload-cli)ConfigurationIni fileIn order for the configuration file to be parsed correctly, create the file following these rules:name -~/.repository-uploader.ini(or anywhere else using --config param)content template[authentication]token=enter-token-here[repository]collection_url=url_of_the_collection# use dot notation to select the metadata field. Note that this should not be the repository's# modified but a different field that will reflect the modified attribute of your source recordrecord_modified_field=metadata.dateModifiedfile_modified_field=metadata.dateModified# field inside the record that holds the id from the sourceid_query_field=metadata.originalId# to be able to upload only non-modified records, provide a max(..) aggregation# inside the repository and set the path within the aggs element herelast_modification_date_agg=dateModifiedAgg.value[entrypoints]# name of the entrypoint inside oarepo_upload_cli.dependencies# that gives implementation of RecordSourcesource=# name of the entrypoint inside oarepo_upload_cli.dependencies that gives implementation# of RepositoryClient. In most cases, not need to specify itrepository=Environment variablesValues in the configuration can be overriden by these environment variables:REPOSITORY_UPLOADER_BEARER_TOKEN

REPOSITORY_UPLOADER_COLLECTION_URL
REPOSITORY_UPLOADER_FILE_MODIFIED_FIELD_NAME
REPOSITORY_UPLOADER_RECORD_MODIFIED_FIELD_NAME
REPOSITORY_UPLOADER_ID_QUERY_FIELD
REPOSITORY_UPLOADER_LAST_MODIFICATION_DATE_AGG

REPOSITORY_UPLOADER_SOURCE
REPOSITORY_UPLOADER_REPOSITORYCommand-line optionsCommandline options take the highest priority:oarepo_upload--configconfig-file-location--tokenbearer_token--collection-urlcollection-url--file-modified-fieldfile-modified-field--record-modified-fieldrecord-modified-field--sourcesource-entrypoint--repositoryrepository-entrypointThe following options handle which records should be uploaded:--modified_after- Timestamp that represents date after modification. If not specified, the last updated timestamp from repository will be used.--modified_before- Timestamp that represents date before modification.Implementing Source of recordsStep 1:Before calling the client, you have to implement your own source of records. To do so, inherit fromRecordSource:fromoarepo_upload_cli.sourceimportRecordSource,SourceRecord,SourceRecordFilefromoarepo_upload_cli.configimportConfigclassMySource(RecordSource):"""Describes a source that is used to generate records."""def__init__(self,config:Config)->None:self._config=configdefget_records(self,modified_after,modified_before):forrecinbackend.records(modified_after,modified_before):yieldSourceRecord(record_id=rec.id,datetime_modified=rec.modified,deleted=rec.deleted,# make sure that the metadata include datetimeModified# or whatever the modified record name is in configmetadata=rec.metadata.as_json(),files=[SourceRecordFile(key=f.filename,content_type='application/octet-stream',# use always thisdatetime_modified=f.modified,metadata={# always include the datetimeModified inside the metadata# or whatever the modified field name is in config'datetimeModified':f.modified.isoformat()},reader=lambda:f.open_for_read())forfinrec.files])defget_records_count(self,modified_after,modified_before):returnbackend.records_changed_count(modified_after,modified_before)Step 1:Register the implementation to entrypoints:[options.entry_points]oarepo_upload_cli.dependencies=my_source="my_source_module:MySource"Step 3:Run the client with--source=my_sourceor put then source to configuration.