Package: dcclab
CERVO-dcclab Python moduleThis simple module is meant to simplify the loading and treatment of images at CERVO (or in any lab) and to provide access to databases for data. The ultimate goal of this module is to rapidly be able to extract useful and pertinent information about microscopy images and spectra. It consists of several "sub modules"Installationpip install dcclab --usershould work, but if by any chance you get an error for a missing module when using it, thenpip install modulenameshould fix it.
Required modules should be installed automatically. If anything is missing,let us know.You should then be able to simply import the module in your own scripts:from dcclab import *

# ... you script
img = Image('yourFile.tiff')
img.display()You are a lab member and you want to do image AnalysisDocumentationThe documentation so far is pretty minimal. I recommend using (help(classname)). This needs to be worked on.There are many image libraries, and this is one more to work with.  Every module has a purpose, and the present module aims to beeasy to use.  This means clarity of the module for users is key, and to a certain extent, clarity for developers is also important: the module was developed by trainees in the laboratory ofDaniel Côtéat theCERVO Brain Research Centerin Québec city, and new trainees every year contribute to the module. Therefore, it is the primary design consideration for the module to be readable, not to be high-performance.  That said, the library offers a very impressive performance nevertheless. Below you will find definitions and conventions for the code, classes and files.The module makes heavy use of thenumpymodule, because every package manipulating images goes back to this high-performance module for storing arrays. We are no different, and we use it like everyone else.This module is a task-oriented module for image analysis: it provides simple tools (classes) to easily read image files, inspect them and manipulate them. For instance, the following classes:Image: can read most image formats, including Zeiss microscope files (.czi).Channel: each image has one or several channels.  The channels, which correspond to specific fluorophores, can be manipulated with filters, threshold, segmentation and other operations. More complex methods like  watershed are also available to use.ImageCollection: can read a collection of image files (e.g., a directory, a z-stack, a map, etc...)DefinitionsChannel: a channel is what most people would consider a grayscale image.  It represents a collection of pixels in 2D for a single contrast mechanism (for instance, GFP, DAPI, Raman, wide field, etc…). When need as an array, it is anumpy.ndarraythat has 2 dimensions: width ⨉ height.It is never a 3D array because it never has more than one contrast agent.The Channel contains also any information that was recovered from reading a file (if it came from a file: objective, magnification, scale, etc…) if that information was available.Image: an image is what anyone would consider "an image": it represents a collection channels. For instance, an image from a microscope may return three colours in red, green and blue (commonly stored as an RGB image such as TIFF for instance). When needed, it is anumpy.ndarraythat has 3 dimensions: width ⨉ height ⨉ channels. Note that the is always a channel dimension, even with only one channel.ImageCollection: as the name implies, it is a collection ofImages. We often deal with collection of images, and we often want to operate on a group of images (e.g., segment many images, strip a channel from images, obtain the average of a given colour, etc…).  The images may or may not be stored as separate files.  They may be stored as separate frames in a movie.  The may be stored in some proprietary format.  Regardless, to the scientist, it is a collection of images that may or may not have more than oneChannel. The images may or may not have the same format, therefore  it is not always possible to obtain anumpy.ndarrayrepresenting the entire collection. When all images are the same format (width, height and number of channels), then one can obtain a 4Dnumpy.ndarraywith width ⨉ height ⨉ channels ⨉ collection. Note that theImageCollectionalways has a channel dimension, even with only one channel, and there is always a collection dimension, even with only one image in the collection.ZStack: of allImageCollections, the z-satck is the most common.  It represents a one-dimensional series of images that were all taken at different z depths.  It is of course a special type ofImageCollectionbecause all images are the same contrast, essentially at the same position (x,y) but different z, and all have the same "properties" (size, laser, etc…), therefore it is always possible to obtain a 4Dnumpy.ndarrayrepresenting the z-stack with width ⨉ height ⨉ channels ⨉ collection. Note that the stack always has a channel dimension, even with only one channel, and there is always a collection dimension, even with only one image in the collection.OperationsWith images, we want to filter noise, segment images, find cells, threshold them, mask them, blur them, etc… All these operations are defined in the module with a language that is "task-oriented": the function for removing noise is calledfilterNoise, the function to threshold is calledthreshold, etc… That way, code will read like a sentence:image.filterNoise()orimage.threshold(), or evenimage.filterNoise().maskWithThreshold().labelMaskComponents().Operations to manipulate "images" as we say, really are operations that operate onChannels, notImages. Indeed, when a scientist wants to segment an "image", he or she really wants to segment either a single channel, or all channels separately.  Hence most (but not all) operations are defined at the level ofChannelwhere all the work takes place. Before going any further, we can already hear people taking offense:"But I want to segment my images! I don't care about this abstract separation of channels and images defined in your module. And examples above use images, not channels!".  And you are right.  This is whyImageandImageCollectionalso define most operations, butImagewill loop through itsChannelsto operate, andImageCollectionwill loop through itsImages, which will loop through theirChannelsto actually get the work done. 99% of the time, this is what people expect. If one had the following script:fromdcclabimport*coll=ImageCollection(filePath='somefiles-\d+.tiff')#details on loading patterns latercoll.filterNoise()coll.applyGaussianFilter()one would expect that the noise be removed from all images in the collection, in each channel. If one knows that the collection has several unnecessary channels (say we know GFP is in the Green channel (2) and Red (1) and Blue (3) are not used), then we can remove them from the images before filtering out the noise:fromdcclabimport*coll=ImageCollection(filePath='somefiles-\d+.tiff')#details on loading patterns latercoll.removeChannels(1,3)coll.filterNoise()coll.applyGaussianFilter()coll.align()You are a lab member and you want to use the databasesWe have a growing database of spectra, for different projects and for different modalities.  The data is organized like this:We havedatasets: a dataset is typically a series of measurements in the lab, typically done in a single day.  For instance, we haveDRS-001,DRS-002WINE-001, etc. You can obtain a description of the datasets withdb.describeDatasets()Each dataset contains many spectra, each identified with auniquespectrumId.  It looks likedatasetId-id1-id2-id3-id4.  What id1, id2,id3 and id4 are depends on the dataset. It is very important to understand that each spectrumId is unique. Theidxrepresents a parameter of the dataset.  It can be a region (for SHAVASANA-001, they can be STN, RPG, etc...) but also just an acquisition (1,2,3,) representing multiple measurements.The classdb = SpectraDB()can return a Panda Dataframe with all the spectra you need, which happens to be very convenientA large number of spectra or a small one depending on its parameters when it is called.db.getSpctralDataframe(datasetId="WINE-001")will return all the spectra in that dataset.The data is retrieved from the sewrver at CERVO.  You will need to have the appropriate passwords, instructions will appear the first time you try to connect.  They will be securely stored in a keychain on your computer, you will not have to reenter them later. You need one to access the server, and one to access the database. Most people will usedcclabfor the username, who can read the data but not delete it.Because the spectra can be quite large, a cache is created on your disk and subsequent calls will not go back to the server, they will get the data from the disk, which will be much faster. You still need to do it once though.If you know MySQL, then you canexecuteSQL statements on the database and usefetchAllorfetchOneto retrieve the results.  This is very general and will allow you to perform any task even if it was not programmed inSpectraDBorLabdataDB.Example:fromdcclab.databaseimport*db=SpectraDB()# to access the database of spectraprint(db.describeDatasets())# Shows the datasetprint(db.getSpectralDataFrame(datasetId="WINE-001"))db.execute("select * from wines")# wines is a separate table with information about the wines project. It is specific to this project.forrowindb.fectchAll():print(row)You are a programmer and want to programm the databaseADatabaseclass allows one to manage files that may be spread over different fileservers. A local example at CERVO — thePlateforme d'Outils Moléculaires— is supported, but the DCCLab, PDK, and Martin Levesque groups will be supported in the near future.For each specific database, a new class inheriting from theDatabaseobject can be queried through a general SQL API but also specific-task-oriented API. MySQL is supported (SQLite support has been dropped for now, too complicated to manage both).  MySQL over ssh is also supported.  For example, the database allow requests such as:all images using the viral vector AAV-173,all images of microglia,all images of neurons from the subthalamic nucleus.The database is ready to use (i.e.connected) upon creation.  To begin using theDatabase, making queries or inserting into it, use the exposed API (e.g.,select(table, columns, condition) -> Row:) or execute an explicit SQL command (e.g.,execute(statement)). To create a new database, aDatabaseobject has to be created withwritePermission=True(sqlite3) or created on the MySQL server directly. If it does not exist yet, the database will be created at theDatabase.pathlocation (inURI).DisclaimerCopyrights DCC/M Lab Members (2019-).