Package: odoo-addon-fs-attachment
In some cases, you need to store attachment in another system that the Odoo’s
filestore. For example, when your deployment is based on a multi-server
architecture to ensure redundancy and scalability, your attachments must
be stored in a way that they are accessible from all the servers. In this
way, you can use a shared storage system like NFS or a cloud storage like
S3 compliant storage, or….This addon extend the storage mechanism of Odoo’s attachments to allow
you to store them in any storage filesystem supported by the Python
libraryfsspecand made
available via thefs_storageaddon.In contrast to Odoo, when a file is stored into an external storage, this
addon ensures that the filename keeps its meaning (In odoo the filename
into the filestore is the file content checksum). Concretely the filename
is based on the pattern:
‘<name-without-extension>-<attachment-id>-<version>.<extension>’This addon also adds on the attachments 2 new fields to use
to retrieve the file content from a URL:Internal URL: URL to retrieve the file content from the Odoo’s
filestore.Filesystem URL: URL to retrieve the file content from the external
storage.NoteThe internal URL is always available, but the filesystem URL is only
available when the attachment is stored in an external storage.
Particular attention has been paid to limit as much as possible the consumption
of resources necessary to serve via Odoo the content stored in an external
filesystem. The implementation is based on an end-to-end streaming of content
between the external filesystem and the Odoo client application by default.
Nevertheless, if your content is available via a URL on the external filesystem,
you can configure the storage to use the x-sendfile mechanism to serve the
content if it’s activated on your Odoo instance. In this case, the content
served by Odoo at the internal URL will be proxied to the filesystem URL
by nginx.Last but not least, the addon adds a new methodopenon the attachment. This
method allows you to open the attachment as a file. For attachments stored into
the filestore or in an external filesystem, it allows you to directly read from
and write to the file and therefore minimize the memory consumption since data
are not kept into memory before being written into the database.Table of contentsUsageConfigurationServer EnvironmentAdvanced usage: Using attachment as a fileTips & TricksChangelog16.0.1.0.8 (2023-12-20)16.0.1.0.6 (2023-12-02)16.0.1.0.5 (2023-11-29)16.0.1.0.4 (2023-11-22)16.0.1.0.3 (2023-10-17)16.0.1.0.2 (2023-10-09)Bug TrackerCreditsAuthorsContributorsMaintainersUsageConfigurationThe configuration is done through the creation of a filesytem storage record
into odoo. To create a new storage, go to the menuSettings > Technical > FS Storageand click onCreate.In addition to the common fields available to configure a storage, specifics
fields are available under the section ‘Attachment’ to configure the way
attachments will be stored in the filesystem.Optimizes Directory Path: This option is useful if you need to prevent
having too many files in a single directory. It will create a directory
structure based on the attachment’s checksum (with 2 levels of depth)
For example, if the checksum is123456789, the file will be stored in the
directory/path/to/storage/12/34/my_file-1-0.txt.Autovacuum GC: This is used to automatically remove files from the filesystem
when it’s no longer referenced in Odoo. Some storage backends (like S3) may
charge you for the storage of files, so it’s important to remove them when
they’re no longer needed. In some cases, this option is not desirable, for
example if you’re using a storage backend to store images shared with others
systems (like your website) and you don’t want to remove the files from the
storage while they’re still referenced into the others systems.
This mechanism is based on afs.file.gcmodel used to collect the files
to remove. This model is automatically populated by their.attachmentmodel when a file is removed from the database. If you disable this option,
you’ll have to manually take care of the records in thefs.file.gcfor
your filesystem storage.Use As Default For Attachment: This options allows you to declare the storage
as the default one for attachments. If you have multiple filesystem storage
configured, you can choose which one will be used by default for attachments.
Once activated, attachments created without specifying a storage will be
stored in this default storage.Force DB For Default Attachment Rules: This option is useful if you want to
force the storage of some attachments in the database, even if you have a
default filesystem storage configured. This is specially useful when you’re
using a storage backend like S3, where the latency of the network can be
high. This option is a JSON field that allows you to define the mimetypes and
the size limit below which the attachments will be stored in the database.Small images (128, 256) are used in Odoo in list / kanban views. We
want them to be fast to read.
They are generally < 50KB (default configuration) so they don’t take
that much space in database, but they’ll be read much faster than from
the object storage.The assets (application/javascript, text/css) are stored in database
as well whatever their size is:a database doesn’t have thousands of themof course better for performancebetter portability of a database: when replicating a production
instance for dev, the assets are includedThe default configuration is:{“image/”: 51200, “application/javascript”: 0, “text/css”: 0}Where the key is the beginning of the mimetype to configure and the
value is the limit in size below which attachments are kept in DB.
0 means no limit.Default configuration means:images mimetypes (image/png, image/jpeg, …) below 50KB are
stored in databaseapplication/javascript are stored in database whatever their sizetext/css are stored in database whatever their sizeThis option is only available on the filesystem storage that is used
as default for attachments.It is also possible to use different FS storages for attachments linked to
different resource fields/models. You can configure it either on thefs.storagedirectly, or in a server environment file:From thefs.storage: Fieldsmodel_idsandfield_idswill encode for which
models/fields use this storage as default storage for attachments having these resource
model/field. Note that if an attachment has both resource model and field, it will
first take the FS storage where the field is explicitely linked, then is not found,
the one where the model is explicitely linked.From a server environment file: In this case you just have to provide a comma-
separated list of models (under themodel_xmlidskey) or fields (under thefield_xmlidskey). To do so, use the model/field XML ids provided by Odoo.
See the Server Environment section for a concrete example.Another key feature of this module is the ability to get access to the attachments
from URLs.Base URL: This is the base URL used to access the attachments from the
filesystem storage itself. If your storage doesn’t provide a way to access
the files from a URL, you can leave this field empty.Is Directory Path In URL: Normally the directory patch configured on the storage
is not included in the URL. If you want to include it, you can activate this option.UseX-SendfileTo Serve Internal Url: If checked and odoo is behind a proxy
that supports x-sendfile, the content served by the attachment’s internal URL
will be served by the proxy using the filesystem url path if defined (This field
is available on the attachment if the storage is configured with a base URL)
If not, the file will be served by odoo that will stream the content read from
the filesystem storage. This option is useful to avoid to serve files from odoo
and therefore to avoid to load the odoo process.To be fully functional, this option requires the proxy to support x-sendfile
(apache) or x-accel-redirect (nginx). You must also configure your proxy by
adding for each storage a rule to redirect the url rooted at the ‘storagge code’
to the server serving the files. For example, if you have a storage with the
code ‘my_storage’ and a server serving the files at the url ‘http://myserver.com’,
you must add the following rule in your proxy configuration:location/my_storage/{internal;proxy_passhttp://myserver.com;}With this configuration a call to ‘/web/content/<att.id>/<att.name><att.extension>”
for a file stored in the ‘my_storage’ storage will generate a response by odoo
with the URI/my_storage/<paht_in_storage>/<att.name>-<att.id>-<version><att.extension>in the headersX-Accel-RedirectandX-Sendfileand the proxy will redirect tohttp://myserver.com/<paht_in_storage>/<att.name>-<att.id>-<version><att.extension>.seehttps://www.nginx.com/resources/wiki/start/topics/examples/x-accel/for more
information.Use Filename Obfuscation: If checked, the filename used to store the content
into the filesystem storage will be obfuscated. This is useful to avoid to
expose the real filename of the attachments outside of the Odoo database.
The filename will be obfuscated by using the checksum of the content. This option
is to avoid when the content of your filestore is shared with other systems
(like your website) and you want to keep a meaningful filename to ensure
SEO. This option is disabled by default.Server EnvironmentWhen you configure a storage through the use of server environment file, you can
provide values for the following keys:optimizes_directory_pathautovacuum_gcbase_urlis_directory_path_in_urluse_x_sendfile_to_serve_internal_urluse_as_default_for_attachmentsforce_db_for_default_attachment_rulesuse_filename_obfuscationmodel_xmlidsfield_xmlidsFor example, the configuration of my storage with codefsprodused to store
the attachments by default could be:[fs_storage.fsprod]protocol=s3options={"endpoint_url": "https://my_s3_server/", "key": "KEY", "secret": "SECRET"}directory_path=my_bucketuse_as_default_for_attachments=Trueuse_filename_obfuscation=Truemodel_xmlids=base.model_res_lang,base.model_res_countryfield_xmlids=base.field_res_partner__image_128Advanced usage: Using attachment as a fileTheopenmethod on the attachment can be used to open manipulate the attachment
as a file object. The object returned by the call to the method implements
methods fromio.IOBase.  The method can ba called as any other python method.
In such a case, it’s your responsibility to close the file at the end of your
process.attachment=self.env.create({"name":"test.txt"})the_file=attachment.open("wb")try:the_file.write(b"content")finally:the_file.close()The result of the call toopenalso works in a contextwithblock. In such
a case, when the code exit the block, the file is automatically closed.attachment=self.env.create({"name":"test.txt"})withattachment.open("wb")asthe_file:the_file.write(b"content")It’s always safer to prefer the second approach.When your attachment is stored into the odoo filestore or into an external
filesystem storage, each time you call the open method, a new file is created.
This way of doing ensures that if the transaction is rolled back the original content
is preserved. Nevertheless you could have use cases where you would like to write
to the existing file directly. For example you could create an empty attachment
to store a csv report and then use theopenmethod to write your content directly
into the new file. To support this kind a use cases, the parameternew_versioncan be passed asFalseto avoid the creation of a new file.attachment=self.env.create({"name":"test.txt"})withattachment.open("w",new_version=False)asf:writer=csv.writer(f,delimiter=";")....Tips & TricksWhen working in multi staging environments, the management of the attachments
can be tricky. For example, if you have a production instance and a staging
instance based on a backup of the production environment, you may want to have
the attachments shared between the two instances BUT you don’t want to have
one instance removing or modifying the attachments of the other instance.To do so, you can add on your staging instances a new storage and declare it
as the default storage to use for attachments. This way, all the new attachments
will be stored in this new storage but the attachments created on the production
instance will still be read from the production storage. Be careful to adapt the
configuration of your storage to the production environment to make it read only.
(The use of server environment files is a good way to do so).Changelog16.0.1.0.8 (2023-12-20)BugfixesFix the error retrieving attachment files when the storage is set to optimize directory paths. (#312)16.0.1.0.6 (2023-12-02)BugfixesImprove performance at creation of an attachment or when the attachment is updated.Before this change, when the fs_url was computed the computed value was always
reassigned to the fs_url attribute even if the value was the same. In a lot of
cases the value was the same and the reassignment was not necessary. Unfortunately
this reassignment has as side effect to mark the record as dirty and generate a
SQL update statement at the end of the transaction. (#307)16.0.1.0.5 (2023-11-29)BugfixesWhen manipulating the file system api through a local variable namedfs,
we observed some strange behavior when it was wrongly redefined in an
enclosing scope as in the following example:with fs.open(…) as fs.
This commit fixes this issue by renaming the local variable and therefore
avoiding the name clash. (#306)16.0.1.0.4 (2023-11-22)BugfixesFix error when an url is computed for an attachment in a storage configure wihtout directory path. (#302)16.0.1.0.3 (2023-10-17)BugfixesFix access to technical models to be able to upload attachments for users with basic access (#289)16.0.1.0.2 (2023-10-09)BugfixesEnsures python 3.9 compatibility. (#285)If a storage is not used to store all the attachments by default, the call to theget_force_db_for_default_attachment_rulesmethod must return an empty dictionary. (#286)Bug TrackerBugs are tracked onGitHub Issues.
In case of trouble, please check there if your issue has already been reported.
If you spotted it first, help us to smash it by providing a detailed and welcomedfeedback.Do not contact contributors directly about support or help with technical issues.CreditsAuthorsCamptocampACSONE SA/NVContributorsThierry Ducrest <thierry.ducrest@camptocamp.com>
Guewen Baconnier <guewen.baconnier@camptocamp.com>
Julien Coux <julien.coux@camptocamp.com>
Akim Juillerat <akim.juillerat@camptocamp.com>
Thomas Nowicki <thomas.nowicki@camptocamp.com>
Vincent Renaville <vincent.renaville@camptocamp.com>
Denis Leemann <denis.leemann@camptocamp.com>
Patrick Tombez <patrick.tombez@camptocamp.com>
Don Kendall <kendall@donkendall.com>
Stephane Mangin <stephane.mangin@camptocamp.com>
Laurent Mignon <laurent.mignon@acsone.eu>
Marie Lejeune <marie.lejeune@acsone.eu>
Wolfgang Pichler <wpichler@callino.at>
Nans Lefebvre <len@lambdao.dev>MaintainersThis module is maintained by the OCA.OCA, or the Odoo Community Association, is a nonprofit organization whose
mission is to support the collaborative development of Odoo features and
promote its widespread use.Currentmaintainer:This module is part of theOCA/storageproject on GitHub.You are welcome to contribute. To learn how please visithttps://odoo-community.org/page/Contribute.