Package: osemosys2iamc
osemosys2iamcUse this Python package to convert OSeMOSYS results to the IAMC formatAcknowledgementsThis work was financially supported by:TheEuropean Climate and Energy Modelling Forum (ECEMF)has received funding from the European Union’s Horizon 2020 Research and Innovation programme under the grant agreement No 101022622TheIAM COMPACTproject has received funding from the European Union’s HORIZON EUROPE Research and Innovation Programme under grant agreement No 101056306Install from Python Packaging Index (PyPI)pip install osemosys2iamcRun the package$ osemosys2iamc --help
osemosys2iamc <inputs_path> <results_path> <config_path> <output_path>inputs_path: Path to a folder of csv files (OSeMOSYS inputs). File names should correspond to OSeMOSYS parameter names.results_path: Path to a folder of csv files (OSeMOSYS results). File names should correspond to OSeMOSYS variable names.config_path: Path to the configuration file (see below)output_path: Path to the .xlsx file you wish to write outThe IAMC formatThe IAMC format was developed by theIntegrated Assessment Modeling Consortium (IAMC)and is used in many model comparison projects at the global and national level.
It can be used for integrated-assessment models, energy-systems scenarios
and analysis of specific sectors like transport, industry or buildings.The format is a tabular structure with the columnsmodel,scenario,region,variable,unit, and a time domain. Each project defines "codelists"
to be used across modelling teams for comparison and analysis of results.The most recent high-profile application of the IAMC format is theAR6 Scenario Explorerhosting the scenario ensemble supporting the quantitative assessment
in the contribution by Working Group III to the IPCC's Sixth Assessment Report (AR6).Please refer to the Horizon 2020 projectopenENTRANCEfor more information about the format and its usage in that project.Writing a configuration fileWrite a configuration file in YAML format. A simple configuration file with two result variables looks like this:model:OSeMBE v1.0.0scenario:DIAG-C400-lin-ResidualFossilregion:iso2_x, iso3_x, from_csv, or a name of a country/region [substitute x with start, end, or a positive number]results:-iamc_variable:'CarbonCapture|Biomass'tech_emi:['(?=^.{2}(BM))^.{4}(CS)']emissions:[CO2]unit:kt CO2/yrtransform:absosemosys_param:AnnualTechnologyEmission-iamc_variable:'CapitalInvestment'capacity:['^.*$']unit:MUSDosemosys_param:CapitalInvestmentThe first section of the configuration file with the keysmodel,scenario, andregionare used to define the metadata for
the IAMC template.modelandscenarioare user-defined, taking any word or phrase and having no set format.regionmay be defined using 4 methods:iso2_x- using the ISO 3166-1 alpha-2 country codes located in the names of technologies/fuels/emissions (depending on the file) at the location given by the user.*iso3_x- using the ISO 3166-1 alpha-3 country codes located in the names of technologies/fuels/emissions (depending on the file) at the location given by the user.*from_csv- using the region as defined in the REGION column of the CSV fileAny other word or phrase the user defines (other than an invalid iso option) will be used as the region for all variables. eg. Austria, Nepal, Ethiopia, Guyana, etc.Thexin the above ISO options is to be replaced by:start, if the codes are at the beginning of the namesend, if the codes are at the end of the names, ora positive number indicating the position of the first letter of the code in the name. eg. iso2_5 will target the 'GH' in 'POWRGHSOL'The second section,results, is where you describe each of the IAMC variables and provide instructions to osemosys2iamc on how
to compute the values.iamc_variable- this should match one of the IAMC variable namesunit- provide the units of the OSeMOSYS resultstransform- onlyabsis currently available. This returns the absolute value of the resultsosemosys_param- provide the name of the result file from which the script should extract the result dataOne or more of the following filter keys. These filter the
results by one or more columns found in OSeMOSYS results.
Following the fitering, the remaining columns except region
and year are discarded and rows are summed.tech_emi- filter the results by TECHNOLOGY and EMISSION columns using the provide regular expression and anemissionsentryemissions- a list of emissions to filter the results by the EMISSION columnfuel- filter by the FUEL columncapacity- filter the TECHNOLOGY columnprimary_technology- filter the TECHNOLOGY column (can be replaced bycapacitykey)excluded_prod_tech- filter the TECHNOLOGY column (can be replaced bycapacitykey)el_prod_technology- filter the TECHNOLOGY column (can be replaced bycapacitykey)demand- filters by the FUEL column (final energy)The value for each of these keys is a list of regular expressions. These regular expressions are used to filter the rows of data in the chosen column to those that match the
regular expression.Writing regular expressions can be tricky, but there areuseful toolsto help.
Below we provide some examples:^.{2}(WI)match rows with any two characters followed byWI(?=^.{2}(HF))^((?!00).)*$match rows withHFin 3rd and 4th position which do not include00(?=^.{2}(NG))^((?!(00)|(CS)).)*$match rows withNGin 3rd and 4th position that do not include00orCS^.{6}(I0)match rows which contain any 6 characters followed byIOin the 7th and 8th positionPutting this together, the following entry extracts results from the result fileProductionByTechnologyAnnual.csv, filters out the rows
by matching values in the TECHNOLOGY column with a list of 6 regular expressions (this is an OR operation)
and assigns the unitPJ/yrand adds the aggregated (summing over region and year) total underPrimary Energyin the IAMC template format.- iamc_variable: 'Primary Energy'
  primary_technology: ['^.{6}(I0)','^.{6}(X0)','^.{2}(HY)','^.{2}(OC)','^.{2}(SO)','^.{2}(WI)']
  unit: PJ/yr
  osemosys_param: ProductionByTechnologyAnnualList of relevant IAMC variables for OSeMOSYSPrimary EnergyLevelCommodityPrimary EnergyPrimary EnergyBiomassPrimary EnergyCoalPrimary EnergyFossilPrimary EnergyGasPrimary EnergyGeothermalPrimary EnergyHydroPrimary EnergyNon-Biomass RenewablesPrimary EnergyNuclearPrimary EnergyOceanPrimary EnergyOilPrimary EnergyOtherPrimary EnergySecondary Energy TradePrimary EnergySolarPrimary EnergyWindPriority VariablesLevelCommodityFuelSecondary EnergyElectricityCoal, w/ and w/o CCSSecondary EnergyLiquidsCoalSecondary EnergyGasesCoalSecondary EnergySolidsCoalSecondary EnergyhydrogenCoal, w/ and w/o CCSLevelCommodityFuelFinal EnergySolidsCoalFinal EnergyIndustrySolidsFinal EnergyResidential and CommercialSolidsFinal EnergyOther SectorSolidsLevelCommoditySectorSectorSectorEmissionsCO2Energy and Industrial ProcessesEmissionsCO2EnergySupplyEmissionsCO2EnergySupplyElectricityEmissionsCO2EnergySupplyOther SectorEmissionsCO2EnergyDemandEmissionsCO2EnergyDemandIndustryEmissionsCO2EnergyDemandResidential and CommercialEmissionsCO2EnergyDemandTransportationEmissionsCO2EnergyDemandAgricultureEmissionsCO2EnergyDemandOther SectorEmissionsCO2Industrial ProcessesEmissionsCO2AFOLULevelCommodityTechnologyTypeCapacityElectricitySolarPVCapacityElectricitySolarCSPCapacityElectricityWindCapacityElectricityWindOffshoreCapacityElectricityWindOnshoreCarbon SequestrationCCSCarbon SequestrationCCSBiomassCarbon SequestrationCCSFossilCarbon SequestrationDirect Air CaptureEmissionsN2OEmissionsCH4