Package: mcl.sickbay
👩‍⚕️ MCL Sickbay"MCL Sickbay" is the data model and object-relational mapping for the clinical data application of theConsortium for Molecular and Cellular Characterization of Screen-Detected Lesions.🏃‍♀️ Getting StartedThe "Sickbay" software provides aPythonbased API into a data model (a series of related classes) and takes advantage ofSQLAlchemyas the object-relational mapper. This section will help you get started.📀 The DatabaseFor this project, we're usingPostgreSQL. You can create a PostgreSQL database to use with this software as follows:dropdb --if-exists clinical_datadropuser --if-exists mclcreateuser \--createdb \--inherit \--login \--no-createrole \--no-superuser \mclcreatedb --encoding=UTF8 --owner=mcl clinical_data🖥 The SoftwareTo use this software, simply addmcl.sickbayas a dependency to your project or install it into your Python virtual environment.You can develop, build, and test the package locally as follows:python3 -m venv venvvenv/bin/pip install --quiet --upgrade setuptools pip wheel buildvenv/bin/pip install --editable .You can runvenv/bin/create-clinical-dbto populate a PostgreSQL database with the schema of the Sickbay data model. Add--add-test-datato include some test data or--add-sample-datato add some sample data (or use both!).To build and publish this software, trybuildandTwine.🔢 VersioningWe use theSemVerphilosophy for versioning this software. For versions available, see therelease history.📦 Additional ResourcesSome resources that provide further context for this software are as follows:Sample data, presentations, mockups, etc.Common Data Elements👥 ContributingWell it's wide open right now, but later you might look atopen issues, forking the project, and submitting a pull request.📃 LicenseThe project is licensed under theApache version 2license.📜 ChangelogThis documents the changes from release to release.1.2.4This release updates pancreas common data elements as follows:exocrine_clinical_M_AJCC_8is now optionalexocrine_clinical_N_AJCC_8is now optionalexocrine_clinical_T_AJCC_8is now optionalexocrine_group_stage_AJCC_8is now optionalexocrine_pathologic_M_AJCC_8is now optionalexocrine_pathologic_N_AJCC_8is now optionalexocrine_pathologic_T_AJCC_8is now optionalfinal_path_duct_communicationis now a string and is optionalhistological_subtypes_ipmnnow has a new permissible value:Mixedlesion_sizeis now optionalneuroendocrine_clinical_M_AJCC_8is now optionalneuroendocrine_clinical_N_AJCC_8is now optionalneuroendocrine_clinical_T_AJCC_8is now optionalneuroendocrine_group_stageis now optionalneuroendocrine_pathologic_M_AJCC_8is now optionalneuroendocrine_pathologic_N_AJCC_8is now optionalneuroendocrine_pathologic_T_AJCC_8is now optionalnumber_lesionsis now optionalpath_acc_num_diag_biopsyis now optionalpath_management_recommendationhas a new permissible value:Pancreaticoduodenectomypath_number_of_tumorsis now optionalpath_tumor_size_largest_lesionis now a string and is optionaltumor_pathology_locationis now a string; the TumorPathologyLocation enumeration no longer exists1.2.3This release simply addsunknownas a possible value to theModeenumeration, thus satisfyingissue 21.1.2.2This release increases the limits on sizes inBiospecimenfor the following fields:specimen_ID_local25 → 200specimen_parent_ID50 → 2001.2.1This release accommodateslistings in the CDE changelogfrom 2022-03-15–2022-04-11, notably:pancreaticoduodenectomyis a new enumerated value inRulesOfAcquisition.higpinis a new enumerated value inPrecancers.sarcomais a new enumerated value inLesion.In classBiospecimen, these fields are no longer enumerations or floats, but nullable strings:storage_methodsection_thicknessshipping_destinationIn classGenomics,sequencing_platformis now a nullable string.The CDE changelog also mentions thatlocation_extent_extraprostatic_extensionandseminal_vesicle_invasion(although it misspells it "vessicle") have two new enumerated values, however in Sickbay these are simply strings, so there's no need to change anything.1.2.0This release contains some incompatible changes to accommodate CDE updates from 2022-01-20 through 2022-01-31 Seethe CDE changelogfor details. However, given the moribund nature of the Consortium for Molecular and Cellular Characterization of Screen-Detected Lesions, these changes are simplified for the following reasons:The project has concluded and we must intake any form of outstanding data presented (asKristen Antonsaid: "MCL is ended, we should take what data we can.")No one actually writes queries for the relational database or uses the query interface of the ORM.The user interface just wants plain text over JSON anyway.Given that, this and all future changes no longer models 1-to-many relationships as traditional relational database structures of 1-table-row-to-multiple-other-table-rows. Instead, we just use plain text and expect pipe (|) separated strings. This is what the input to this database is (pipe-separated cells in a spreadsheet) and what the UI expects, so the Herculean overhead of making the tables and ORM models to support this is ultimately fatuous.The changes in this release include:The enumerationGroupStage8now has an additional value,unknown. This is not currently reflected in theLung CDEsat this time, but we're not concerned.In theLungOrganclass:The following values are now string types instead of enumerations:primary_adenocarcinoma_differentiation_typeprior_treatmentTheajcc_8_lung_pathologic_mhas an error; the enumeration name was duplicated asmetastasis_enum; it's nowmetastasis_enum8.The changelog for CDEs says that all 14 AJCC fields are now optional; but this was already the case for Sickbay since 1.1.0.In theProstateOrganclass, the following values are now string types instead of enumerations:location_dominant_nodulelocation_secondary_nodulelocation_extent_extraprostatic_extensionlocation_nature_positive_marginssummed_length_positive_marginseminal_vesicle_invasion1.1.0This release contains some incompatible changes in order to accommodate CDE updates from 2021-08-26 through 2021-11-18. Pleasesee the CDE changelogfor highly pedantic details of these updates. The changes to the software include:On classOrgan:histopathology_precancer_typewas a 1-to-many attribute ofLungOrganonly; now it belongs toallorgans as 1-to-many.This base class now has the following optional attributes:ajcc_clinical_majcc_clinical_najcc_clinical_tajcc_clinical_stageajcc_pathologic_majcc_pathologic_najcc_pathologic_tajcc_pathologic_stagelymph_nodes_testedlymph_node_locationOn the classLungOrgan:There arenumerouschanges. For one, theajcc_staging_system_editionindicates whether the entire record uses the AJCC Staging edition 7 or 8, and depending on this, it tells which set of attributes to use.The attributes are:ajcc_7_lung_clinical_majcc_7_lung_clinical_najcc_7_lung_clinical_tajcc_7_lung_disease_stageajcc_7_lung_pathologic_majcc_7_lung_pathologic_najcc_7_lung_pathologic_tajcc_8_lung_clinical_majcc_8_lung_clinical_najcc_8_lung_clinical_tajcc_8_lung_disease_stageajcc_8_lung_pathologic_majcc_8_lung_pathologic_najcc_8_lung_pathologic_tNote that all of these attributes areoptional; this is because it's also possible thatajcc_staging_system_editionisunknownornot_reported, in which case we can't enforce that a specific set of the above attributes are actually used.Lungs also have a new attribute:lymph_nodes_positive, an optional integer.On the classProstateOrgan, these attributes have moved "up" into the superclassOrgan:lymph_nodes_testedlymph_node_locationajcc_clinical_majcc_clinical_najcc_clinical_tajcc_clinical_stageajcc_pathologic_majcc_pathologic_najcc_pathologic_tajcc_pathologic_stageIn classBiospecimen, these attributes were required and are now optional:
-days_to_collection-time_excision_to_processing-days_to_storageThe following enumerated types have changed:TStage7no longer includes the termst1cort1miClinicalMStage7has dropped the termsM1candpM1ForGroupStage7, the following permissible values are no longer permissible:ia1ia2ia3ivaivbPrecancersnow includes anormalkindFixativesnow supports anot_applicablevalueWhen it comes toStorageyou now have two new optionsroom_temperature_then_refrigeratedfrozen_at__20cSlideChargeshas made these values impermissible:cm0,cm1,pm1,pm1a,pm1b,pm1cWe now finally have a blessed description forTreatmentinstead of the kind contrived by a mere software developerAt long last an expert has realized thatcannot_be_determineshould becannot_be_determinedinNecrosisThe following new enumerations are ready for use:ClinicalMStage8with 8 valuesClinicalNStage8with 7 valuesGroupStage8with 17 valuesAJCCMetastasisStage8with 8 valuesRemoval ofzc.buildout. We cannot recommend this tool less. Just use virtual environments like everyone else.1.0.2For issuehttps://github.com/EDRN/MCL-metadata/issues/22Additional permissible value onsequencing_platform(enumGenomicAnalyzer), namelyillumina_hiseq_1500.Changed theread_lengthfrom numeric to a string (10)Note that we do not have schema migrations set up so these steps must be run manually:ALTER TABLE "genomics" ALTER "read_length" SET DATA TYPE CHARACTER VARYING(10)ALTER TYPE "genomic_analyzier_enum" ADD VALUE 'illumina_hiseq_1500' AFTER 'illumina_genome_analyzer_iix'1.0.1This version adds the human-readable label plus the token value to all enumerations over the JSON; seehttps://github.com/MCLConsortium/mcl.sickbay/issues/16for more information.1.0.0A "more official" release.0.0.10For issuehttps://github.com/MCLConsortium/mcl.sickbay/issues/1:OnClinicalCore:Theraceattribute is now a 1-to-many mapping toCoreRaceviacore_racesThetype_tobacco_usedis now a 1-to-many mapping toCoreTobaccoviacore_tobaccosThe attributedays_to_birthis now requiredOnBiospecimen:The enumeration forPrecancershas a whole bunch of new permitted valuesOnBreastOrgan:The enumeration forPrecancerousHistopathologycontains values for "unknown" and "data not available"The enumeration forBreastSitenow has anunknownvalueA new valuependingis available forGeneticTestingAnswer,TestResults,EstrogenTestResultsThe enumerationHER2ResultsaddspendingandunknownvaluesThe enumerationBreastImagingWorkupadds anunknownvalueThe enumerationBIRADSTissuesadds values for "unknown" and "data not available"NewLungOrganplus (bogus) test data for itNewPancreasOrganplus (bogus) test data for itUpdatedProstateOrganPreviously, this was just a placeholder to test multiple inheritance from the commonOrganclass in terms of both Python class hierachy and database hierarchyNow it's completely filled out with thev0prostate common data elements with its numerous controlled vocabulariesExpanded enumerations:ClinicalMStage7,TStage7,ClinicalNStage7,GroupStage7,MarginalStatusNew enumerations, far too many to enumerate 😏For issuehttps://github.com/MCLConsortium/mcl.sickbay/issues/4:All fields inLabCASMetadataare nowString.For issuehttps://github.com/MCLConsortium/mcl.sickbay/issues/3:inscribed_clinicalCore_participant_IDis a new field onPriorLesion,CoreRace, andCoreTobaccoinscribed_biospecimen_identifieris a new field onAdjacentSpecimenFor issuehttps://github.com/MCLConsortium/mcl.sickbay/issues/5:The following updates diverge from thedata dictionariesof the common data elements:participant_IDis now 50 characters (along with foreign keys andinscribedfields), up from 14specimen_IDis now 50 characters (along with foreign keys andinscribedfields), up from 16And finally, for issuehttps://github.com/MCLConsortium/mcl.sickbay/issues/6… we addunknownto all enumerations that didn't have it already.0.0.9Renameinscribed_participant_ID→inscribed_clinicalCore_participant_IDRenameinscribed_specimen_ID→inscribed_biospecimen_specimen_ID0.0.8Addresseshttps://github.com/MCLConsortium/mcl.sickbay/issues/2by:Addinginscribed_participant_IDandinscribed_specimen_IDtoGenomicsAddinginscribed_participant_IDandinscribed_specimen_IDtoImagingAddinginscribed_participant_IDtoBiospecimen(It also adds some test data to these fields.)0.0.7In this release:ThelabcasFileURLfield is now justlabcasID; everything else is the same except the name (and the semantics; it no longer is used to hold URLs)TheOrganclass now has aninscribed_participant_IDfield you can use to note a future participant ID association with aClinicalCoreAll enumerations now useadvanced enumerationsfor their base class.All enumerations now have a case-insensitive lookup.The implications of that last bullet mean:>>>frommcl.sickbay.model.enumsimportRace>>>Race.black_or_african_american==Race('Black or African American')True>>>Race.black_or_african_american==Race['Black or African American']True>>>Race.black_or_african_american==Race['black or african american']True>>>Race('black or african american')Traceback(mostrecentcalllast):...ValueError:'black or african american'isnotavalidRaceSo if you want case-insensitive lookups, use brackets, not parentheses.0.0.6In this release:Base metadata for all classes now includes:consortium, a nullable string that can be used to contain an RDF URI to the consortium that originated the data, such ashttps://mcl.nci.nih.gov/for the Consortium for Molecular and Cellular Characterization of Screen-Detected Lesions.protocolID, a nullable integer that tells the research protocol that generated the data.Kristen's sample data (--add-sample-data) includes these consortium and protocol IDs0.0.5This release fixes:InBreastOrgan, the fieldher2_in_situ_hybridizationwas the wrong enumerated type. It should've beenHER2InSituHybridization.In the enums, add the typeHER2InSituHybridization.Add test data from12_78_BreastCore_20200625_0.Removed foreign key constraint fromBiospecimen.specimen_parent_IDbecause the parent ID may be either another biospecimen or could be a participant (clinical core) object.New classAdjacentSpecimento work around circular dependency problem of having adjacent specimens directly onBiospeciment.New JSON serialization foradjacent_specimensonBiospecimenMisspelled enumerationAnatomicalSite:pancrease→pancreasChangecreate-demo-dbtocreate-clinical-dbsince this is no longer a demo but the real dealTransition from old stylesetup.pyto everything insetup.cfgIn this release, 0.0.5, we also finally start keeping a changelog 😮