spectral-data-converter 0.1.0
Python3 library for converting (and filtering) spectral data in various formats.
License: MIT License

Python library for converting (and filtering) spectral data in various formats.

Supported spectral formats:

* ADAMS (read/write)
* ARFF row-wise (read/write)
* ASC (read/write)
* ASCII XY (read/write)
* CAL FOSS (read/write)
* CSV (read/write)
* DPT (read/write)
* MPS (read)
* NIR FOSS (read/write)
* Opus Bruker (read)
* Opus Ext Bruker (read)
* SPA Thermo Fisher (read)

Supported sample data formats:

* ADAMS report (read/write)
* CSV row-wise (read/write)
* JSON (read/write)

Examples can be found here:

https://github.com/waikato-datamining/spectral-data-converter-examples


Changelog
=========

0.1.0 (2025-10-31)
------------------

- `split-records` filter now allows specifying the meta-data field in which to store the split name
- the `tee` meta-filter can now forward or drop the incoming data based on a meta-data evaluation
- the `sub-process` filter can be used for processing data with sub-flow of filters, can be conditional based on meta-data evaluation
- the `metadata-from-name` filter can work on the path now as well (must be present)
- switched to `kasperl` library for base API and generic pipeline plugins
- requiring seppl>=0.3.0 now
- added `@abc.abstractmethod` decorator where appropriate
- the `sdc-exec` tool now uses all remaining parameters as the pipeline components rather than having
  to specify them via the `-p/--pipeline` parameter, making it easy to simply prefix the `sdc-exec`
  command to an existing `sdc-convert` command-line
- added the `text-file` and `csv-file` generators that work off files to populate the variable(s)
- `sdc-exec` can load pipelines from file now as well, useful when dealing with large pipelines
- added `--load_pipeline` option to `sdc-convert`
- added `from-text-file` reader and `to-text-file` writer
- readers now locate files the first time the `read()` method gets called rather than in the
  `initialized()`, to allow more dynamic placeholders
- added `from-text-file` reader and `to-text-file` writer
- added `block`, `stop` filters for controlling the flow of data (via meta-data conditions)
- added email support with `get-email` reader and `send-email` writer
- added `list-files` reader for listing files in a directory
- added `list-to-sequence` stream filter that forwards list items one by one
- added `console` writer for outputting the data on stdout that is coming through
- added `watch-dir` meta-reader that uses the watchdog library to react to file-system events
  rather than using fixed-interval polling like `poll-dir`
- added `delete-files` writer
- added `copy-files` filter
- added support for caching plugins via `SDC_CLASS_CACHE` environment variable
- added `to-metadata` writer that outputs the meta-data of an image
- added `attach-metadata` filter that loads meta-data from a directory and attaches it to the data passing through


0.0.3 (2025-07-15)
------------------

- requiring seppl>=0.2.20 now for improved help requests in `sdc-convert` tool


0.0.2 (2025-07-11)
------------------

- wai.spectralio-based readers now instantiate the wai.spectralio reader in the `initialize` method
- wai.spectralio-based writers now instantiate the wai.spectralio writer in the `initialize` method
- introduced `SpectralIOBased`, `SpectralIOReader` and `SpectralIOWriter` mixins to wai.spectralio-based
  readers/writers for a cleaner class hierarchy
- requiring wai-spectralio>=0.0.5 now
- requiring seppl>=0.2.19 now
- added experimental support for direct read/write operations using file-like objects
- fixed initialization of sample ID and sample data prefix in `CSVSampleDataWriter`
- fixed initialization of None values of `OPUSExtReader`, aligning it with the command-line args
- added `from-zip` meta-reader for reading spectra and sample data from zip files
- added `to-zip` meta-writer for writing spectra and sample data to zip files


0.0.1 (2025-06-27)
------------------

- initial release