Package: kinto-attachment
Attach files toKinto records.Installpip install kinto-attachmentSetupIn the Kinto project settingskinto.includes=kinto_attachmentkinto.attachment.base_url=http://cdn.service.org/files/Local File storageStore files locally:kinto.attachment.base_path=/tmpS3 File StorageStore on Amazon S3:kinto.attachment.aws.access_key=<AWS access key>kinto.attachment.aws.secret_key=<AWS secret key>kinto.attachment.aws.bucket_name=<bucket name>kinto.attachment.aws.acl=<AWS ACL permissions|public-read>Noteaccess_keyandsecret_keymay be omitted when using AWS Identity
and Access Management (IAM).SeePyramid Storage.Google Cloud Storagekinto.attachment.gcloud.credentials=<Path to the Service Accounts credentials JSON file>kinto.attachment.gcloud.bucket_name=<bucket name>kinto.attachment.gcloud.acl=publicReadSeeGoogle Cloud ACL permissionsThefolderoptionWith this option, the files will be stored in sub-folders.Use the{bucket_id}and{collection_id}placeholders to organize the files
by bucket or collection.kinto.attachment.folder={bucket_id}/{collection_id}Or only for a particular bucket:kinto.attachment.resources.blog.folder=blog-assetsOr a specific collection:kinto.attachment.resources.blog.articles.folder=articles-imagesThekeep_old_filesoptionWhen set totrue, the files won’t be deleted from disk/S3 when the associated record
is deleted or when the attachment replaced.kinto.attachment.keep_old_files=trueOr only for a particular bucket:kinto.attachment.resources.blog.keep_old_files=falseOr a specific collection:kinto.attachment.resources.blog.articles.keep_old_files=trueThegzippedoptionIf you want uploaded files to get gzipped when stored (default: False):kinto.attachment.gzipped=trueOr only for a particular bucket:kinto.attachment.resources.blog.gzipped=trueOr a specific collection:kinto.attachment.resources.blog.articles.gzipped=trueTherandomizeoptionIf you want uploaded files to be stored with a random name (default: True):kinto.attachment.randomize=trueOr only for a particular bucket:kinto.attachment.resources.blog.randomize=trueOr a specific collection:kinto.attachment.resources.blog.articles.randomize=trueTheextensionsoptionIf you want to upload files which are not in the default allowed extensions (seePyramid extensions groups(default:default):kinto.attachment.extensions=default+videoThemimetypesoptionBy default, the mimetype is guessed from the filename using Python standard mimetypes module.If you want to add or override mimetypes, use the following setting and the associated syntax:kinto.attachment.mimetypes=.ftl:application/vnd.fluent;.db:application/vnd.sqlite3Default bucketIn order to upload files on thedefaultbucket, the built-in default bucket
plugin should be enabled before thekinto_attachmentplugin.In the configuration, this means adding it explicitly to includes:kinto.includes=kinto.plugins.default_bucketkinto_attachmentProductionMake sure thebase_urlcan be reached (and points tobase_pathif
files are stored locally)Adjust the max size for uploaded files (e.g.client_max_body_size 10m;for NGinx)For example, with NGinxserver {
    listen 80;

    location /v1 {
        ...
    }

    location /files {
        root /var/www/kinto;
    }
}APIPOST /{record-url}/attachmentIt will create the underlying record if it does not exist.Requiredattachment: a single multipart-encoded fileOptionaldata: attributes to set on record (serialized JSON)permissions: permissions to set on record (serialized JSON)DELETE /{record-url}/attachmentDeletes the attachement from the record.AttributesWhen a file is attached, the related record is given anattachmentattribute
with the following fields:filename: the original filenamehash: a SHA-256 hex digestlocation: the URL of the attachmentmimetype: themedia typeof
the filesize: size in bytes{"data":{"attachment":{"filename":"IMG_20150219_174559.jpg","hash":"ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad","location":"http://cdn.service.org/files/ffa9c7b9-7561-406b-b7f9-e00ac94644ff.jpg","mimetype":"image/jpeg","size":1481798},"id":"c2ce1975-0e52-4b2f-a5db-80166aeca688","last_modified":1447834938251,"theme":"orange","type":"wallpaper"},"permissions":{"write":["basicauth:6de355038fd943a2dc91405063b91018bb5dd97a08d1beb95713d23c2909748f"]}}If the file is gzipped by the server, anoriginalkey is added in theattachmentkey, containing the file infobeforeit’s gzipped. Theattachmentkeys are
in that case referring to the gzipped file:{"data":{"attachment":{"filename":"IMG_20150219_174559.jpg.gz","hash":"ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad","location":"http://cdn.service.org/files/ffa9c7b9-7561-406b-b7f9-e00ac94644ff.jpg.gz","mimetype":"application/x-gzip","size":14818,"original":{"filename":"IMG_20150219_174559.jpg","hash":"hPME6i9avCf/LFaznYr+sHtwQEX7mXYHSu+vgtygpM8=","mimetype":"image/jpeg","size":1481798}},"id":"c2ce1975-0e52-4b2f-a5db-80166aeca688","last_modified":1447834938251,"theme":"orange","type":"wallpaper"},"permissions":{"write":["basicauth:6de355038fd943a2dc91405063b91018bb5dd97a08d1beb95713d23c2909748f"]}}UsageUsing HTTPiehttp--authalice:passwd--formPOSThttp://localhost:8888/v1/buckets/website/collections/assets/records/c2ce1975-0e52-4b2f-a5db-80166aeca689/attachmentdata='{"type": "wallpaper", "theme": "orange"}'"attachment@~/Pictures/background.jpg"HTTP/1.1201CreatedAccess-Control-Expose-Headers:Retry-After, Content-Length, Alert, BackoffContent-Length:209Content-Type:application/json; charset=UTF-8Date:Wed, 18 Nov 2015 08:22:18 GMTEtag:"1447834938251"Last-Modified:Wed, 18 Nov 2015 08:22:18 GMTLocation:http://localhost:8888/v1/buckets/website/collections/font/assets/c2ce1975-0e52-4b2f-a5db-80166aeca689Server:waitress{"filename":"IMG_20150219_174559.jpg","hash":"ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad","location":"http://cdn.service.org/files/ffa9c7b9-7561-406b-b7f9-e00ac94644ff.jpg","mimetype":"image/jpeg","size":1481798}Using Python requestsauth=("alice","passwd")attributes={"type":"wallpaper","theme":"orange"}perms={"read":["system.Everyone"]}files=[("attachment",("background.jpg",open("Pictures/background.jpg","rb"),"image/jpeg"))]payload={"data":json.dumps(attributes),"permissions":json.dumps(perms)}response=requests.post(SERVER_URL+endpoint,data=payload,files=files,auth=auth)response.raise_for_status()Using JavaScriptvarheaders={Authorization:"Basic "+btoa("alice:passwd")};varattributes={"type":"wallpaper","theme":"orange"};varperms={"read":["system.Everyone"]};// File object from input fieldvarfile=form.elements.attachment.files[0];// Build form datavarpayload=newFormData();// Multipart attachmentpayload.append('attachment',file,"background.jpg");// Record attributes and permissions JSON encodedpayload.append('data',JSON.stringify(attributes));payload.append('permissions',JSON.stringify(perms));// Post form using GlobalFetch APIvarurl=`${server}/buckets/${bucket}/collections/${collection}/records/${record}/attachment`;fetch(url,{method:"POST",body:payload,headers:headers}).then(function(result){console.log(result);});ScriptsTwo scripts are provided in this repository.They rely on thekinto-clientPython package, which can be installed in a
virtualenv:$ virtualenv env --python=python3
$ source env/bin/activate
$ pip install kinto-clientOr globally on your system (not recommended):$ sudo pip install kinto-clientUpload filesupload.pytakes a list of files and posts them on the specified server,
bucket and collection:$ python3 scripts/upload.py --server=$SERVER --bucket=$BUCKET --collection=$COLLECTION --auth "token:mysecret" README.rst pictures/*If the--gzipoption is passed, the files are gzipped before upload.
Since theattachmentattribute contains metadata of the compressed file
the original file metadata are stored in aoriginalattribute.Seepython3 scripts/upload.py--helpfor more details about options.Download filesdownload.pydownloads the attachments from the specified server, bucket and
collection and store them on disk:$ python3 scripts/download.py --server=$SERVER --bucket=$BUCKET --collection=$COLLECTION --auth "token:mysecret"If the record has anoriginalattribute, the script decompresses the attachment
after downloading it.Files are stored in the current folder by default.
Seepython3 scripts/download.py--helpfor more details about options.Known limitationsNo support for chunk upload (#10)Files are not removed when server is purged withPOST /v1/__flush__Relative URL in records (workaround)Currently the full URL is returned in records. This is very convenient for API consumers
which can access the attached file just using the value in thelocationattribute.However, the way it is implemented has a limitation: the full URL is stored in each record
directly. This is annoying because changing thebase_urlsetting
won’t actually change thelocationattributes on existing records.As workaround, it is possible to set thekinto.attachment.base_urlto an empty
value. Thelocationattribute in records will now contain arelativeURL.Using another settingkinto.attachment.extra.base_url, it is possible to advertise
the base URL that can be preprended by clients to obtain the full attachment URL.
If specified, it is going to be exposed in the capabilities of the root URL endpoint.Run testsRun a fake Amazon S3 server in a separate terminal:make run-motoRun the tests suite:make testsReleasingCreate a release on Github onhttps://github.com/Kinto/kinto-attachment/releases/newCreate a new tagX.Y.Z(This tag will be created from the target when you publish this release.)Generate release notesPublish releaseNotesAPI design discussionabout mixing upattachmentand record fields.