Package: cytomine-pims
cytomine-pimsCytomine Python Image Management ServerRun development server with Dockerdocker build -f docker/backend.dockerfile -t pims .
docker run -p 5000:5000 pimsThe server is running athttp://127.0.0.1:5000and API documentation is available
athttp://127.0.0.1:5000/docsAt this stage, it is hard to use Docker for development because hot-reload is not enabled and
PIMS plugin system is not yet easily manageable in a Docker container for development. However,
as PIMS requires a lot of low-level dependencies, developing using Docker would be a benefit.Run development server locallyDependenciesFirst, dependencies must be installedDependencies indocker/backend.dockerfilemust be installed first. For plugins, prerequisites
have to be
installed manually, especially forbefore_vipsandbefore_python. Seeinstall_prerequisites.shin respective plugins.pip install -r requirements.txtCacheTo run PIMS with cache, the cache must be configured in the settings. The cache uses an
external Redis in-memory database. To launch a Redis instance using default values in PIMS
settings, run:dockerrun-d--namepims-cache-p6379:6379redisIf the PIMS cache cannot be reached at PIMS startup, cache is automatically disabled.Task queueHeavy computation like image imports are run in a task queue to prevent server flooding or
blocking. The task queue uses Celery workers and RabbitMQ to communicate, but a fallback is
possible when unavailable.Celery and RabbitMQPIMS is pre-configured to run with the Cytomine RabbitMQ configuration (seeCytomine-bootstrap for PIMS).
RabbitMQ broker can be launched without Cytomine, by changing username and default password for
task queue in PIMS settings byguest/guest, and then run:dockerrun-d--namerabbitmq-p5672:5672-p15672:15672--hostnamerabbitmqrabbitmq:3.9Then, you have to run a Celery worker:CONFIG_FILE="/path/to/my/config.env"celery-Apims.tasks.workerworker-linfo-Qpims-import-queue-c1where-cis the concurrency level; 1 is enough for development.
See below for environment variables.Fallback task queueIf task queue is disabled in PIMS configuration of if RabbitMQ is unreachable, heavy
computations are still done in an asynchronous way but task queue is not mo running in separate
processes and there is no guarantee that all tasks will be able to fallback on this.RunTo run the development server, run:CONFIG_FILE="/path/to/my/config.env"python-mpims.mainThe server is running athttp://127.0.0.1:5000and API documentation is available
athttp://127.0.0.1:5000/docsIn order to test PIMS without Cytomine, you can import images locally usingCONFIG_FILE=/path/to/config.envpythonpims/importer/import_local_images.py--path/my/folderEnvironment variablesCONFIG_FILE: path to a.envconfiguration file. Default topims-config.env(but some required configuration
settings need to be filled)LOG_CONFIG_FILE: path to a.ymlPython logging configuration file. Default tologging.ymlDEBUG: When set, enable debug tools and uselogging-debug.ymlas default logging configuration if not other
log config file is specified.Configuration settings can be also given as environment variables and override values fromCONFIG_FILE.PIMS pluginsPIMS plugins are quite new and plugin API is subject to change. To add a new format plugin, the
best is to adapt the existingpims-plugin-format-openslide.A plugin can add one or several formats.Create a directorypims-plugin-format-{name}and copy/paste content from Openslide pluginInsetup.py, rename theNAMEvariableRename the source directory topims_plugin_format_{name}Adaptpims_plugin_format_{name}/__version__.pycontentIf needed, adapt dependencies prerequisites ininstall-prerequisites.shImplement new formats by defining classes namedXYZFormat(ending thus withFormat) and
extendingAbstractFormat.Develop with pluginsAs the core serverpimsis a dependency of every PIMS plugins,pimshas to be installed in
the plugin virtual environment. At this stage,pimsand plugins will probably be developed at
same time, and the easiest way is toinstallpimsin editable mode in the plugin Python virtual env. Activate the plugin
environment, and runpip install -e /local/path/to/pims(you may need to add other options
like--extra-index-url)install the plugin in editable mode in PIMS Python virtual env. Activate the PIMS
environment (e.g. in another terminal) and runpip install -e /local/path/to/pims-plugin-format-{name}When you start PIMS, the logs should list your plugin.Docker image with pluginsThepims/scriptsfolder has a script to build Docker images of PIMS with or without some
plugins. Adapt theplugin-list.csvto build an image with the plugins (and their versions)
you want.Available pluginsOne can find below the listing of the PIMS plugins that have already been implemented.Plugin nameFormat(s)URLRemarksOpenslideBIF, MRXS, NDPi, Philips TIFF, SCN, SVS, VMShttps://github.com/cytomine/pims-plugin-format-openslide/tree/main/pims_plugin_format_openslideDepends on VIPS and Openslide.Example/https://github.com/cytomine/pims-plugin-format-exampleThis is just a example plugin to explain how to implement a PIMS plugin.WSI DicomWSIDICOMhttps://github.com/cytomine/pims-plugin-format-dicomPIMS plugin based on the WSI Dicom format implementedhere. Annotations management not implemented yet.Docker image with plugins resolution orderDuring the upload of an image, a check is made to ensure the existence of the image format in accordance with the plugins installed with the Cytomine instance (these pulgins are specified in the CSV file namedplugin-list.csv). To ensure that the uploaded file is handle with the right format resolver, one must define the resolution order of the plugins such that the most conservative checker happens before the less conservative one.One will find a resolution order column in theplugin-list.csv: the values must be assigned either an integer value or left unset. This resolution order is used to arrange plugins in relation to PIMS's native formats, which are given a default order of zero (0).The plugin that has the smallest, or most negative, integer for its resolution order will be checked before any other. This means that if a plugin's resolution order is more negative than all others, its formats will be evaluated first, even ahead of PIMS's native formats. In situations where the resolution order isn't specified for a plugin, it automatically inherits the same order as the native formats in PIMS, which is 0. This means it will be checked alongside, but not before, the native formats.Run development server locally with plugins resolution orderIn development mode, one can now create a newcheckerResolution.csvfile (name and path of this file can be defined inpims-dev-config.env) in order to specify format checkers resolution order.The CSV file must apply the following convention:nameresolution_orderpims_plugin_format_{name}INToremptypims_plugin_format_{name}must be the string referring to the name of the plugin as specified in the source directory.the term 'resolution order' must be assigned either an integer value or left unset. This resolution order is used to arrange plugins in relation to PIMS's native formats, which are given a default order of zero (0).
The plugin that has the smallest, or most negative, integer for its resolution order will be checked before any other. This means that if a plugin's resolution order is more negative than all others, its formats will be evaluated first, even ahead of PIMS's native formats.
In situations where the resolution order isn't specified for a plugin, it automatically inherits the same order as the native formats in PIMS, which is 0. This means it will be checked alongside, but not before, the native formats.To create the necessary file, you can modify the plugin-list.csv file to include your desired resolution order. After making these changes, execute the following command within the Docker directory:python plugins.py --plugin_csv_path=/pims-ce/scripts/plugin-list.csv --checkerResolution_file_path=/pims-ce/checkerResolution.csv --method=checker_resolution_file