Repository: NREL/OpenStudio (language: python)

Issue:
running osw when measure not found, instead of stopping it keeps running the osw and E+.
Issue overview
--------------
Issue with how workflows where one or more measures can't be found. It skips over those measure and runs the measure it can find. In cases where some measure are found and others are not it really makes mess out.osw putting results from one measure over the wrong position.

## Current Behavior
Test Case A (running bar_typical from osw2osa repo): https://github.com/NREL/osw2osa/blob/master/workflows/bar_typical/in.osw 
- The workflow fails with no measure running. E+ runs with an error. Below is relevant parts of out.osw. Note current step shows 0. When all measure are run at end current step shows as 7. None of the measure in out.osw have any results.
Relevant elements of out.osw
   "completed_at" : "20240806T065546Z",
   "completed_status" : "Fail",
   "current_step" : 0,
   "eplusout_err" : ". . . . . . ** Severe  ** GetNextEnvironment: Weather Environment(s) requested, but no weather file found\n   **  Fatal  ** Due to previous error condition, simulation terminated\n  . . . . . ",
   "started_at" : "20240806T065545Z",

Test Case B (running floorspace_typical from osw2osa repo) https://github.com/NREL/osw2osa/blob/master/workflows/floorspace_typical/in.osw
- Similar to Test case a no measures are found but and E+ runs, but because in this case the seed model had enough to run the E+ simulation was successful and even produced an eplusout.html, although no reporting measure were run or produced, but I think only because the measures were not found. When all measure are run current step would show as 7.
Relevant elements of out.osw
   "completed_at" : "20240806T065546Z",
   "completed_status" : "Success",
   "current_step" : 0,
   "eplusout_err" : " . . . . ******* EnergyPlus Sizing Error Summary. During Sizing: 2 Warning; 0 Severe Errors.\n   ************* EnergyPlus Completed Successfully-- 85 Warning; 0 Severe Errors; Elapsed Time=00hr 00min  0.32sec\n"
   "started_at" : "20240806T065546Z",

Test Case C (running ze_retail from osw2osa repo) https://github.com/NREL/osw2osa/blob/master/workflows/ze_retail/in.osw
- Ok, os here is where things get weird. There are 16 measure in this repo, and only 2 of them would be found although the workflow failed and stoped on the first of two measures. This results in an odd out.osw where results are shown for the 11th measure but on top of the first measure (that wasn't found. I'll show in.osw relevant parts of in.osw here first for clarity. 

Relevant elements of in.osw for test case C
      {
   "file_paths" : 
   [
      "../../../weather",
      "../../../seeds",
      "../../../files"
   ],
   "hash" : "4EDC1148",
   "measure_paths" : 
   [
      "../../../measures"
   ],
   "seed_file" : "aedg_retail.osm",
   "steps" : 
   [
      {
         "arguments" : 
         {
            "set_year" : "2019",
            "weather_file_name" : "USA_GA_Atlanta-Hartsfield-Jackson.Intl.AP.722190_TMY3.epw"
         },
         "measure_dir_name" : "ChangeBuildingLocation"
      },
      {
         "arguments" : 
         {
            "__SKIP__" : false,
            "facade" : "North",
            "wwr" : "0.0"
         },
         "measure_dir_name" : "SetWindowToWallRatioByFacade"
      }, (14 more measures follow this)

Relevant elements of out.osw for test case C. Note that results for the 11th measure `ze_retail_typical` are written on top of the first measure `ChangeBuildingLocation`. This was the most confusing of the test cases because it looked like ChangeBuildingLocation ran, but it really didn't. When measure are found current step shows at 11.

{
   "completed_at" : "20240806T065547Z",
   "completed_status" : "Fail",
   "current_step" : 1,
   "file_paths" : 
   [
      "/Users/dgoldwas/Documents/github/nrel/osw2osa/run/workflows/ze_retail/generated_files",
      "../../../weather",
      "../../../seeds",
      "../../../files",
      "./files",
      "./weather",
      "../../files",
      "../../weather",
      "./"
   ],
   "hash" : "8CD3DD8F",
   "measure_paths" : 
   [
      "../../../measures"
   ],
   "run_options" : null,
   "seed_file" : "aedg_retail.osm",
   "started_at" : "20240806T065545Z",
   "steps" : 
   [
      {
         "arguments" : 
         {
            "set_year" : "2019",
            "weather_file_name" : "USA_GA_Atlanta-Hartsfield-Jackson.Intl.AP.722190_TMY3.epw"
         },
         "measure_dir_name" : "ChangeBuildingLocation",
         "result" : 
         {
            "completed_at" : "20240806T065547Z",
            "measure_class_name" : "ZeRetailCreateTypicalBuildingFromModel",
            "measure_display_name" : "ZE Retail Create Typical Building from Model",
            "measure_name" : "ze_retail_create_typical_building_from_model",
            "measure_taxonomy" : "Whole Building.Space Types",
            "measure_type" : "ModelMeasure",
            "measure_uid" : "05fcdf1f-8266-4428-9ba2-da9fa94dceec",
            "measure_version_id" : "07aa88f1-ec5d-4442-8d50-db3c3228abb8",
            "measure_version_modified" : "20201118T053018Z",
            "measure_xml_checksum" : "EE13F09F",
            "started_at" : "20240806T065547Z",
            "stderr" : "",
            "stdout" : "[openstudio.model.YearDescription] <Error> 'UseWeatherFile' is selected in YearDescription, . . . . "
           "step_errors" : 
            [
               "SWIG director method error. NoMethodError: undefined metho model_get_building_climate_zone_and_building_type for #<NRELZNEReady2017:0x000000012abd6070 @template=\"NREL ZNE Ready 2017\", @standards_data={\"hvac_inference\"=>[{\"template\"=>\"NREL ZNE Ready 2017\", \"size_categor . . . . . ."
            "step_files" : [],
            "step_info" : 
            [
               "Adding loads to space type named Office_Restroom - NREL ZNE Ready 2017",
               "Adding loads to space type named Conference - NREL ZNE Ready 2017",
               "Adding loads to space type named Break_Room - NREL ZNE Ready 2017",
               "Adding loads to space type named Back_Space - NREL ZNE Ready 2017",
               "Adding loads to space type named Retail - NREL ZNE Ready 2017",
               "Adding loads to space type named Point_of_Sale - NREL ZNE Ready 2017",
               "Adding loads to space type named Front_Retail - NREL ZNE Ready 2017",
               "Adding loads to space type named Closed_Office - NREL ZNE Ready 2017",
               "Adding loads to space type named Entry - NREL ZNE Ready 2017",
               "Adding loads to space type named Mechanical_Room - NREL ZNE Ready 2017",
               "Adding loads to space type named Core_Retail - NREL ZNE Ready 2017"
            ],
            "step_initial_condition" : "The building started with 220 objects.",
            "step_result" : "Fail",
            "step_values" : 
            [
               {
                  "name" : "template",
                  "value" : "NREL ZNE Ready 2017"
               },
               {
                  "name" : "template_space_loads",
                  "value" : "ZE AEDG Multifamily"
               },
               {
                  "name" : "system_type",
                  "value" : "PSZ-HP"
               },
               {
                  "name" : "hvac_delivery_type",
                  "value" : "Forced Air"
               },
               {
                  "name" : "htg_src",
                  "value" : "NaturalGas"
               },
               {
                  "name" : "clg_src",
                  "value" : "Electricity"
               },
               {
                  "name" : "swh_src",
                  "value" : "Electricity"
               },
               {
                  "name" : "kitchen_makeup",
                  "value" : "Adjacent"
               },
               {
                  "name" : "exterior_lighting_zone",
                  "value" : "3 - All Other Areas"
               },
               {
                  "name" : "add_constructions",
                  "value" : true
               },
               {
                  "name" : "add_space_type_loads",
                  "value" : true
               },
               {
                  "name" : "add_elevators",
                  "value" : true
               },
               {
                  "name" : "add_internal_mass",
                  "value" : true
               },
               {
                  "name" : "add_exterior_lights",
                  "value" : true
               },
               {
                  "name" : "onsite_parking_fraction",
                  "value" : 1.0
               },
               {
                  "name" : "add_exhaust",
                  "value" : true
               },
               {
                  "name" : "add_swh",
                  "value" : true
               },
               {
                  "name" : "add_thermostat",
                  "value" : true
               },
               {
                  "name" : "add_hvac",
                  "value" : true
               },
               {
                  "name" : "add_refrigeration",
                  "value" : false
               },
               {
                  "name" : "modify_wkdy_op_hrs",
                  "value" : false
               },
               {
                  "name" : "wkdy_op_hrs_start_time",
                  "value" : 8.0
               },
               {
                  "name" : "wkdy_op_hrs_duration",
                  "value" : 8.0
               },
               {
                  "name" : "modify_wknd_op_hrs",
                  "value" : false
               },
               {
                  "name" : "wknd_op_hrs_start_time",
                  "value" : 8.0
               },
               {
                  "name" : "wknd_op_hrs_duration",
                  "value" : 8.0
               },
               {
                  "name" : "unmet_hours_tolerance",
                  "value" : 1.0
               },
               {
                  "name" : "remove_objects",
                  "value" : false
               },
               {
                  "name" : "use_upstream_args",
                  "value" : false
               },
               {
                  "name" : "enable_dst",
                  "value" : true
               },
               {
                  "name" : "climate_zone",
                  "value" : "Lookup From Model"
               }
            ],
            "step_warnings" : []
         }
      },
      {
         "arguments" : 
         {
            "__SKIP__" : false,
            "facade" : "North",
            "wwr" : "0.0"
         },
         "measure_dir_name" : "SetWindowToWallRatioByFacade"
      },

Still for Test Case C here is part of the out.osw if all measure were found. Note that the results for `ChangeBuildingLocation` are actually from `ChangeBuildingLocation`.

{
   "completed_at" : "20240806T065132Z",
   "completed_status" : "Fail",
   "current_step" : 11,
   "file_paths" : 
   [
      "/Users/dgoldwas/Documents/github/nrel/osw2osa/run/workflows/ze_retail/generated_files",
      "../../../weather",
      "../../../seeds",
      "../../../files",
      "./files",
      "./weather",
      "../../files",
      "../../weather",
      "./"
   ],
   "hash" : "CE6AE42F",
   "measure_paths" : 
   [
      "../../../.bundle/install/ruby/3.2.0/gems/openstudio-calibration-0.10.0/lib/measures",
      "../../../.bundle/install/ruby/3.2.0/gems/openstudio-common-measures-0.10.0/lib/measures",
      "../../../.bundle/install/ruby/3.2.0/gems/openstudio-ee-0.10.0/lib/measures",
      "../../../.bundle/install/ruby/3.2.0/gems/openstudio-model-articulation-0.10.0/lib/measures",
      "../../../.bundle/install/ruby/3.2.0/gems/tbd-3.4.2/lib/measures",
      "../../../measures"
   ],
   "run_options" : null,
   "seed_file" : "aedg_retail.osm",
   "started_at" : "20240806T065043Z",
   "steps" : 
   [
      {
         "arguments" : 
         {
            "set_year" : "2019",
            "weather_file_name" : "USA_IL_Chicago-OHare.Intl.AP.725300_TMY3.epw"
         },
         "measure_dir_name" : "ChangeBuildingLocation",
         "result" : 
         {
            "completed_at" : "20240806T065045Z",
            "measure_class_name" : "ChangeBuildingLocation",
            "measure_display_name" : "ChangeBuildingLocation",
            "measure_name" : "change_building_location",
            "measure_taxonomy" : "Whole Building.Space Types",
            "measure_type" : "ModelMeasure",
            "measure_uid" : "d4db4971-f5ba-11e3-a3ac-0800200c9a66",
            "measure_version_id" : "746afe91-d190-49c5-9534-26416d0bc55a",
            "measure_version_modified" : "20240711T143210Z",
            "measure_xml_checksum" : "057E8D9D",
            "started_at" : "20240806T065045Z",
            "stderr" : "",
            "stdout" : "[openstudio.model.YearDescription] <Error> 'UseWeatherFile' is selected in YearDescription, but there are no weather file set for the model.\n[openstudio.model.YearDescription] <Error> 'UseWeatherFile' is selected in YearDescription, but there are no weather file set for the model.\n",
            "step_errors" : [],
            "step_files" : [],
            "step_final_condition" : "The final weather file is Chicago Ohare Intl Ap and the model has 3 design day objects.",
            "step_info" : 
            [
               "city is Chicago Ohare Intl Ap. State is IL",
               "Changing Calendar Year to 2019,",
               "mean dry bulb is 9.924999999999999",
               "Adding object Chicago Ohare Intl Ap Ann Clg .4% Condns DB=>MWB",
               "Adding object Chicago Ohare Intl Ap Ann Clg .4% Condns WB=>MDB",
               "Adding object Chicago Ohare Intl Ap Ann Htg 99.6% Condns DB",
               "Setting ASHRAE Climate Zone to 5A"
            ],
            "step_initial_condition" : "No weather file is set. The model has 0 design day objects",
            "step_result" : "Success",
            "step_values" : 
            [
               {
                  "name" : "weather_file_name",
                  "value" : "USA_IL_Chicago-OHare.Intl.AP.725300_TMY3.epw"
               },
               {
                  "name" : "climate_zone",
                  "value" : "Lookup From Stat File"
               },
               {
                  "name" : "set_year",
                  "value" : 2019
               },
               {
                  "name" : "use_upstream_args",
                  "value" : true
               },
               {
                  "name" : "epw_gsub",
                  "value" : "Do Nothing"
               },
               {
                  "name" : "reported_climate_zone",
                  "value" : "5A"
               }
            ],
            "step_warnings" : []
         }
      },

## Expected Behavior
In the past when a measure in a workflow could not be found the workflow would fail and there would be a warning that a measure could not be found or loaded. This info didn't make it into out.osw. I believe the workflow would fail to start to run even if missing measure was not first measure in the workflow.

I have not confirmed but this may have changed from Ruby based Workflow gem switch to C++ workflow. I more minor changes to investigate is workflows with out of order measures (where they are not model, E+, then reporting measure). This used to also fail but now will be more flexible which would have it run measure out of order if a model measure is listed after an E+ measure, or an E+ measure after a reporting measure.

## Steps to Reproduce
1. Setup a workflow with one or more measures that can't be found in measure paths
2. Run workflow
3. Look at out.osw and resulting files

## Possible Solution
Check for missing measures either up front or as each measure is run and have it halt the workflow. An enhancement over prior behavior woudl be to  make message for measure that can't be found or measure that can't be loaded into the out OSW at the proper step of the workflow.

## Details

### Environment

Tested OpenStudio 3.8.0 Mac

### Context
This isn't a blocker but is problematic because it's possible that a workflow may appear to run, even to show as being successful but one or more measures are skipped without user being made aware of this.
running osw when measure not found, instead of stopping it keeps running the osw and E+.
Issue overview
--------------
Issue with how workflows where one or more measures can't be found. It skips over those measure and runs the measure it can find. In cases where some measure are found and others are not it really makes mess out.osw putting results from one measure over the wrong position.

## Current Behavior
Test Case A (running bar_typical from osw2osa repo): https://github.com/NREL/osw2osa/blob/master/workflows/bar_typical/in.osw 
- The workflow fails with no measure running. E+ runs with an error. Below is relevant parts of out.osw. Note current step shows 0. When all measure are run at end current step shows as 7. None of the measure in out.osw have any results.
Relevant elements of out.osw
   "completed_at" : "20240806T065546Z",
   "completed_status" : "Fail",
   "current_step" : 0,
   "eplusout_err" : ". . . . . . ** Severe  ** GetNextEnvironment: Weather Environment(s) requested, but no weather file found\n   **  Fatal  ** Due to previous error condition, simulation terminated\n  . . . . . ",
   "started_at" : "20240806T065545Z",

Test Case B (running floorspace_typical from osw2osa repo) https://github.com/NREL/osw2osa/blob/master/workflows/floorspace_typical/in.osw
- Similar to Test case a no measures are found but and E+ runs, but because in this case the seed model had enough to run the E+ simulation was successful and even produced an eplusout.html, although no reporting measure were run or produced, but I think only because the measures were not found. When all measure are run current step would show as 7.
Relevant elements of out.osw
   "completed_at" : "20240806T065546Z",
   "completed_status" : "Success",
   "current_step" : 0,
   "eplusout_err" : " . . . . ******* EnergyPlus Sizing Error Summary. During Sizing: 2 Warning; 0 Severe Errors.\n   ************* EnergyPlus Completed Successfully-- 85 Warning; 0 Severe Errors; Elapsed Time=00hr 00min  0.32sec\n"
   "started_at" : "20240806T065546Z",

Test Case C (running ze_retail from osw2osa repo) https://github.com/NREL/osw2osa/blob/master/workflows/ze_retail/in.osw
- Ok, os here is where things get weird. There are 16 measure in this repo, and only 2 of them would be found although the workflow failed and stoped on the first of two measures. This results in an odd out.osw where results are shown for the 11th measure but on top of the first measure (that wasn't found. I'll show in.osw relevant parts of in.osw here first for clarity. 

Relevant elements of in.osw for test case C
      {
   "file_paths" : 
   [
      "../../../weather",
      "../../../seeds",
      "../../../files"
   ],
   "hash" : "4EDC1148",
   "measure_paths" : 
   [
      "../../../measures"
   ],
   "seed_file" : "aedg_retail.osm",
   "steps" : 
   [
      {
         "arguments" : 
         {
            "set_year" : "2019",
            "weather_file_name" : "USA_GA_Atlanta-Hartsfield-Jackson.Intl.AP.722190_TMY3.epw"
         },
         "measure_dir_name" : "ChangeBuildingLocation"
      },
      {
         "arguments" : 
         {
            "__SKIP__" : false,
            "facade" : "North",
            "wwr" : "0.0"
         },
         "measure_dir_name" : "SetWindowToWallRatioByFacade"
      }, (14 more measures follow this)

Relevant elements of out.osw for test case C. Note that results for the 11th measure `ze_retail_typical` are written on top of the first measure `ChangeBuildingLocation`. This was the most confusing of the test cases because it looked like ChangeBuildingLocation ran, but it really didn't. When measure are found current step shows at 11.

{
   "completed_at" : "20240806T065547Z",
   "completed_status" : "Fail",
   "current_step" : 1,
   "file_paths" : 
   [
      "/Users/dgoldwas/Documents/github/nrel/osw2osa/run/workflows/ze_retail/generated_files",
      "../../../weather",
      "../../../seeds",
      "../../../files",
      "./files",
      "./weather",
      "../../files",
      "../../weather",
      "./"
   ],
   "hash" : "8CD3DD8F",
   "measure_paths" : 
   [
      "../../../measures"
   ],
   "run_options" : null,
   "seed_file" : "aedg_retail.osm",
   "started_at" : "20240806T065545Z",
   "steps" : 
   [
      {
         "arguments" : 
         {
            "set_year" : "2019",
            "weather_file_name" : "USA_GA_Atlanta-Hartsfield-Jackson.Intl.AP.722190_TMY3.epw"
         },
         "measure_dir_name" : "ChangeBuildingLocation",
         "result" : 
         {
            "completed_at" : "20240806T065547Z",
            "measure_class_name" : "ZeRetailCreateTypicalBuildingFromModel",
            "measure_display_name" : "ZE Retail Create Typical Building from Model",
            "measure_name" : "ze_retail_create_typical_building_from_model",
            "measure_taxonomy" : "Whole Building.Space Types",
            "measure_type" : "ModelMeasure",
            "measure_uid" : "05fcdf1f-8266-4428-9ba2-da9fa94dceec",
            "measure_version_id" : "07aa88f1-ec5d-4442-8d50-db3c3228abb8",
            "measure_version_modified" : "20201118T053018Z",
            "measure_xml_checksum" : "EE13F09F",
            "started_at" : "20240806T065547Z",
            "stderr" : "",
            "stdout" : "[openstudio.model.YearDescription] <Error> 'UseWeatherFile' is selected in YearDescription, . . . . "
           "step_errors" : 
            [
               "SWIG director method error. NoMethodError: undefined metho model_get_building_climate_zone_and_building_type for #<NRELZNEReady2017:0x000000012abd6070 @template=\"NREL ZNE Ready 2017\", @standards_data={\"hvac_inference\"=>[{\"template\"=>\"NREL ZNE Ready 2017\", \"size_categor . . . . . ."
            "step_files" : [],
            "step_info" : 
            [
               "Adding loads to space type named Office_Restroom - NREL ZNE Ready 2017",
               "Adding loads to space type named Conference - NREL ZNE Ready 2017",
               "Adding loads to space type named Break_Room - NREL ZNE Ready 2017",
               "Adding loads to space type named Back_Space - NREL ZNE Ready 2017",
               "Adding loads to space type named Retail - NREL ZNE Ready 2017",
               "Adding loads to space type named Point_of_Sale - NREL ZNE Ready 2017",
               "Adding loads to space type named Front_Retail - NREL ZNE Ready 2017",
               "Adding loads to space type named Closed_Office - NREL ZNE Ready 2017",
               "Adding loads to space type named Entry - NREL ZNE Ready 2017",
               "Adding loads to space type named Mechanical_Room - NREL ZNE Ready 2017",
               "Adding loads to space type named Core_Retail - NREL ZNE Ready 2017"
            ],
            "step_initial_condition" : "The building started with 220 objects.",
            "step_result" : "Fail",
            "step_values" : 
            [
               {
                  "name" : "template",
                  "value" : "NREL ZNE Ready 2017"
               },
               {
                  "name" : "template_space_loads",
                  "value" : "ZE AEDG Multifamily"
               },
               {
                  "name" : "system_type",
                  "value" : "PSZ-HP"
               },
               {
                  "name" : "hvac_delivery_type",
                  "value" : "Forced Air"
               },
               {
                  "name" : "htg_src",
                  "value" : "NaturalGas"
               },
               {
                  "name" : "clg_src",
                  "value" : "Electricity"
               },
               {
                  "name" : "swh_src",
                  "value" : "Electricity"
               },
               {
                  "name" : "kitchen_makeup",
                  "value" : "Adjacent"
               },
               {
                  "name" : "exterior_lighting_zone",
                  "value" : "3 - All Other Areas"
               },
               {
                  "name" : "add_constructions",
                  "value" : true
               },
               {
                  "name" : "add_space_type_loads",
                  "value" : true
               },
               {
                  "name" : "add_elevators",
                  "value" : true
               },
               {
                  "name" : "add_internal_mass",
                  "value" : true
               },
               {
                  "name" : "add_exterior_lights",
                  "value" : true
               },
               {
                  "name" : "onsite_parking_fraction",
                  "value" : 1.0
               },
               {
                  "name" : "add_exhaust",
                  "value" : true
               },
               {
                  "name" : "add_swh",
                  "value" : true
               },
               {
                  "name" : "add_thermostat",
                  "value" : true
               },
               {
                  "name" : "add_hvac",
                  "value" : true
               },
               {
                  "name" : "add_refrigeration",
                  "value" : false
               },
               {
                  "name" : "modify_wkdy_op_hrs",
                  "value" : false
               },
               {
                  "name" : "wkdy_op_hrs_start_time",
                  "value" : 8.0
               },
               {
                  "name" : "wkdy_op_hrs_duration",
                  "value" : 8.0
               },
               {
                  "name" : "modify_wknd_op_hrs",
                  "value" : false
               },
               {
                  "name" : "wknd_op_hrs_start_time",
                  "value" : 8.0
               },
               {
                  "name" : "wknd_op_hrs_duration",
                  "value" : 8.0
               },
               {
                  "name" : "unmet_hours_tolerance",
                  "value" : 1.0
               },
               {
                  "name" : "remove_objects",
                  "value" : false
               },
               {
                  "name" : "use_upstream_args",
                  "value" : false
               },
               {
                  "name" : "enable_dst",
                  "value" : true
               },
               {
                  "name" : "climate_zone",
                  "value" : "Lookup From Model"
               }
            ],
            "step_warnings" : []
         }
      },
      {
         "arguments" : 
         {
            "__SKIP__" : false,
            "facade" : "North",
            "wwr" : "0.0"
         },
         "measure_dir_name" : "SetWindowToWallRatioByFacade"
      },

Still for Test Case C here is part of the out.osw if all measure were found. Note that the results for `ChangeBuildingLocation` are actually from `ChangeBuildingLocation`.

{
   "completed_at" : "20240806T065132Z",
   "completed_status" : "Fail",
   "current_step" : 11,
   "file_paths" : 
   [
      "/Users/dgoldwas/Documents/github/nrel/osw2osa/run/workflows/ze_retail/generated_files",
      "../../../weather",
      "../../../seeds",
      "../../../files",
      "./files",
      "./weather",
      "../../files",
      "../../weather",
      "./"
   ],
   "hash" : "CE6AE42F",
   "measure_paths" : 
   [
      "../../../.bundle/install/ruby/3.2.0/gems/openstudio-calibration-0.10.0/lib/measures",
      "../../../.bundle/install/ruby/3.2.0/gems/openstudio-common-measures-0.10.0/lib/measures",
      "../../../.bundle/install/ruby/3.2.0/gems/openstudio-ee-0.10.0/lib/measures",
      "../../../.bundle/install/ruby/3.2.0/gems/openstudio-model-articulation-0.10.0/lib/measures",
      "../../../.bundle/install/ruby/3.2.0/gems/tbd-3.4.2/lib/measures",
      "../../../measures"
   ],
   "run_options" : null,
   "seed_file" : "aedg_retail.osm",
   "started_at" : "20240806T065043Z",
   "steps" : 
   [
      {
         "arguments" : 
         {
            "set_year" : "2019",
            "weather_file_name" : "USA_IL_Chicago-OHare.Intl.AP.725300_TMY3.epw"
         },
         "measure_dir_name" : "ChangeBuildingLocation",
         "result" : 
         {
            "completed_at" : "20240806T065045Z",
            "measure_class_name" : "ChangeBuildingLocation",
            "measure_display_name" : "ChangeBuildingLocation",
            "measure_name" : "change_building_location",
            "measure_taxonomy" : "Whole Building.Space Types",
            "measure_type" : "ModelMeasure",
            "measure_uid" : "d4db4971-f5ba-11e3-a3ac-0800200c9a66",
            "measure_version_id" : "746afe91-d190-49c5-9534-26416d0bc55a",
            "measure_version_modified" : "20240711T143210Z",
            "measure_xml_checksum" : "057E8D9D",
            "started_at" : "20240806T065045Z",
            "stderr" : "",
            "stdout" : "[openstudio.model.YearDescription] <Error> 'UseWeatherFile' is selected in YearDescription, but there are no weather file set for the model.\n[openstudio.model.YearDescription] <Error> 'UseWeatherFile' is selected in YearDescription, but there are no weather file set for the model.\n",
            "step_errors" : [],
            "step_files" : [],
            "step_final_condition" : "The final weather file is Chicago Ohare Intl Ap and the model has 3 design day objects.",
            "step_info" : 
            [
               "city is Chicago Ohare Intl Ap. State is IL",
               "Changing Calendar Year to 2019,",
               "mean dry bulb is 9.924999999999999",
               "Adding object Chicago Ohare Intl Ap Ann Clg .4% Condns DB=>MWB",
               "Adding object Chicago Ohare Intl Ap Ann Clg .4% Condns WB=>MDB",
               "Adding object Chicago Ohare Intl Ap Ann Htg 99.6% Condns DB",
               "Setting ASHRAE Climate Zone to 5A"
            ],
            "step_initial_condition" : "No weather file is set. The model has 0 design day objects",
            "step_result" : "Success",
            "step_values" : 
            [
               {
                  "name" : "weather_file_name",
                  "value" : "USA_IL_Chicago-OHare.Intl.AP.725300_TMY3.epw"
               },
               {
                  "name" : "climate_zone",
                  "value" : "Lookup From Stat File"
               },
               {
                  "name" : "set_year",
                  "value" : 2019
               },
               {
                  "name" : "use_upstream_args",
                  "value" : true
               },
               {
                  "name" : "epw_gsub",
                  "value" : "Do Nothing"
               },
               {
                  "name" : "reported_climate_zone",
                  "value" : "5A"
               }
            ],
            "step_warnings" : []
         }
      },

## Expected Behavior
In the past when a measure in a workflow could not be found the workflow would fail and there would be a warning that a measure could not be found or loaded. This info didn't make it into out.osw. I believe the workflow would fail to start to run even if missing measure was not first measure in the workflow.

I have not confirmed but this may have changed from Ruby based Workflow gem switch to C++ workflow. I more minor changes to investigate is workflows with out of order measures (where they are not model, E+, then reporting measure). This used to also fail but now will be more flexible which would have it run measure out of order if a model measure is listed after an E+ measure, or an E+ measure after a reporting measure.

## Steps to Reproduce
1. Setup a workflow with one or more measures that can't be found in measure paths
2. Run workflow
3. Look at out.osw and resulting files

## Possible Solution
Check for missing measures either up front or as each measure is run and have it halt the workflow. An enhancement over prior behavior woudl be to  make message for measure that can't be found or measure that can't be loaded into the out OSW at the proper step of the workflow.

## Details

### Environment

Tested OpenStudio 3.8.0 Mac

### Context
This isn't a blocker but is problematic because it's possible that a workflow may appear to run, even to show as being successful but one or more measures are skipped without user being made aware of this.


Discussion / hints:
A related issue is trying to run an OSW where measures are out of order. For example:
* ModelMeasureA
* ModelMeasureB
* ReportingMeasureA
* ModelMeasureC
* ReportingMeasureB

This should stop the workflow and finish as failed.
I have seen this on Ubuntu with OpenStudio 3.7, I just didn't realize what was happening. My "measure_paths" value was messed up so none of the measures were being found, but I had a configuration issue that masked what was happening. When fixing the configuration, I must have fixed my measure path. This came up recently when I went back to rerun things that had the original wrong measure path in the OSW but I set it up properly this time. The workflow ran to completion without errors but none of the measures were applied and the results were very much not what was expected.
In case it isn't documented above this also happens on OSW where a measure is a measure with "__SKIP__": true,. This made issues for workflows.

The specific use case of skipped measure was fixed sometime between 3.7 and 3.8. It still happens in 3.7. It doesn't happen in 3.8 and 3.9. I still believe this issue for missing measure paths is still valid. I can make a zip file replicating this after 3.9 is out and will target this for 3.10 instead. It is more of an edge case that would be with mall-formed OSW files or resource paths.
@jmarrec I'll re assign you after I can setup a good example. 
A related issue is trying to run an OSW where measures are out of order. For example:
* ModelMeasureA
* ModelMeasureB
* ReportingMeasureA
* ModelMeasureC
* ReportingMeasureB

This should stop the workflow and finish as failed.
I have seen this on Ubuntu with OpenStudio 3.7, I just didn't realize what was happening. My "measure_paths" value was messed up so none of the measures were being found, but I had a configuration issue that masked what was happening. When fixing the configuration, I must have fixed my measure path. This came up recently when I went back to rerun things that had the original wrong measure path in the OSW but I set it up properly this time. The workflow ran to completion without errors but none of the measures were applied and the results were very much not what was expected.
In case it isn't documented above this also happens on OSW where a measure is a measure with "__SKIP__": true,. This made issues for workflows.

The specific use case of skipped measure was fixed sometime between 3.7 and 3.8. It still happens in 3.7. It doesn't happen in 3.8 and 3.9. I still believe this issue for missing measure paths is still valid. I can make a zip file replicating this after 3.9 is out and will target this for 3.10 instead. It is more of an edge case that would be with mall-formed OSW files or resource paths.
@jmarrec I'll re assign you after I can setup a good example. 

Fix patch:
diff --git a/resources/workflow/invalid_measures/missing_a_measure.osw b/resources/workflow/invalid_measures/missing_a_measure.osw
new file mode 100644
index 00000000000..2f78c0633ba
--- /dev/null
+++ b/resources/workflow/invalid_measures/missing_a_measure.osw
@@ -0,0 +1,19 @@
+{
+	"weather_file": "../../Examples/compact_osw/files/srrl_2013_amy.epw",
+	"seed_file": "../example_model.osm",
+	"measure_paths": ["../measures/"],
+	"steps": [
+		{
+			"measure_dir_name": "FakeModelMeasure",
+			"arguments": {}
+		},
+		{
+			"measure_dir_name": "NON_EXISTING_MEASURE_THIS_SHOULD_BE_CAUGHT",
+			"arguments": {}
+		},
+		{
+			"measure_dir_name": "FakeReport",
+			"arguments": {}
+		}
+	]
+}
diff --git a/resources/workflow/invalid_measures/unloadable_measure.osw b/resources/workflow/invalid_measures/unloadable_measure.osw
new file mode 100644
index 00000000000..cb5d1c88178
--- /dev/null
+++ b/resources/workflow/invalid_measures/unloadable_measure.osw
@@ -0,0 +1,19 @@
+{
+	"weather_file": "../../Examples/compact_osw/files/srrl_2013_amy.epw",
+	"seed_file": "../example_model.osm",
+	"measure_paths": ["../measures/"],
+	"steps": [
+		{
+			"measure_dir_name": "FakeModelMeasure",
+			"arguments": {}
+		},
+		{
+			"measure_dir_name": "UnloadableMeasure",
+			"arguments": {}
+		},
+		{
+			"measure_dir_name": "FakeReport",
+			"arguments": {}
+		}
+	]
+}
diff --git a/resources/workflow/invalid_measures/wrong_measure_type_order.osw b/resources/workflow/invalid_measures/wrong_measure_type_order.osw
new file mode 100644
index 00000000000..01b28e49177
--- /dev/null
+++ b/resources/workflow/invalid_measures/wrong_measure_type_order.osw
@@ -0,0 +1,15 @@
+{
+	"weather_file": "../../Examples/compact_osw/files/srrl_2013_amy.epw",
+	"seed_file": "../example_model.osm",
+	"measure_paths": ["../measures/"],
+	"steps": [
+		{
+			"measure_dir_name": "FakeReport",
+			"arguments": {}
+		},
+		{
+			"measure_dir_name": "FakeModelMeasure",
+			"arguments": {}
+		}
+	]
+}
diff --git a/resources/workflow/measures/FakeModelMeasure/measure.rb b/resources/workflow/measures/FakeModelMeasure/measure.rb
new file mode 100644
index 00000000000..68a978a9096
--- /dev/null
+++ b/resources/workflow/measures/FakeModelMeasure/measure.rb
@@ -0,0 +1,42 @@
+class FakeModelMeasure < OpenStudio::Measure::ModelMeasure
+  # human readable name
+  def name
+    # Measure name should be the title case of the class name.
+    return 'A dumb ModelMeasure'
+  end
+
+  # human readable description
+  def description
+    return 'Does nothing'
+  end
+
+  # human readable description of modeling approach
+  def modeler_description
+    return 'Just for testing'
+  end
+
+  # define the arguments that the user will input
+  def arguments(model)
+    args = OpenStudio::Measure::OSArgumentVector.new
+
+    return args
+  end
+
+  # define what happens when the measure is run
+  def run(model, runner, user_arguments)
+    super(model, runner, user_arguments)  # Do **NOT** remove this line
+
+    # use the built-in error checking
+    if !runner.validateUserArguments(arguments(model), user_arguments)
+      return false
+    end
+
+    # report final condition of model
+    runner.registerFinalCondition("The FakeModelMeasure run.")
+
+    return true
+  end
+end
+
+# register the measure to be used by the application
+FakeModelMeasure.new.registerWithApplication
diff --git a/resources/workflow/measures/FakeModelMeasure/measure.xml b/resources/workflow/measures/FakeModelMeasure/measure.xml
new file mode 100644
index 00000000000..a1fd30f66f0
--- /dev/null
+++ b/resources/workflow/measures/FakeModelMeasure/measure.xml
@@ -0,0 +1,44 @@
+<?xml version="1.0"?>
+<measure>
+  <schema_version>3.1</schema_version>
+  <name>fake_model_measure</name>
+  <uid>677b8fd3-2627-4516-b090-f6e47dc99fea</uid>
+  <version_id>162465e5-b6f6-419f-ab5f-c2fc4bd549e0</version_id>
+  <version_modified>2024-11-07T11:55:10Z</version_modified>
+  <xml_checksum>82D8F881</xml_checksum>
+  <class_name>FakeModelMeasure</class_name>
+  <display_name>A dumb ModelMeasure</display_name>
+  <description>Does nothing</description>
+  <modeler_description>Just for testing</modeler_description>
+  <arguments />
+  <outputs />
+  <provenances />
+  <tags>
+    <tag>Envelope.Form</tag>
+  </tags>
+  <attributes>
+    <attribute>
+      <name>Measure Type</name>
+      <value>ModelMeasure</value>
+      <datatype>string</datatype>
+    </attribute>
+    <attribute>
+      <name>Measure Language</name>
+      <value>Ruby</value>
+      <datatype>string</datatype>
+    </attribute>
+  </attributes>
+  <files>
+    <file>
+      <version>
+        <software_program>OpenStudio</software_program>
+        <identifier>3.9.0</identifier>
+        <min_compatible>3.9.0</min_compatible>
+      </version>
+      <filename>measure.rb</filename>
+      <filetype>rb</filetype>
+      <usage_type>script</usage_type>
+      <checksum>DDA977B0</checksum>
+    </file>
+  </files>
+</measure>
diff --git a/resources/workflow/measures/UnloadableMeasure/README.md b/resources/workflow/measures/UnloadableMeasure/README.md
new file mode 100644
index 00000000000..4459d7e1aa3
--- /dev/null
+++ b/resources/workflow/measures/UnloadableMeasure/README.md
@@ -0,0 +1,1 @@
+This doesnt have a xml
diff --git a/resources/workflow/measures/UnloadableMeasure/measure.rb b/resources/workflow/measures/UnloadableMeasure/measure.rb
new file mode 100644
index 00000000000..e69de29bb2d
diff --git a/src/cli/CMakeLists.txt b/src/cli/CMakeLists.txt
index 179c35f7a04..e79684fcdfc 100644
--- a/src/cli/CMakeLists.txt
+++ b/src/cli/CMakeLists.txt
@@ -270,6 +270,63 @@ if(BUILD_TESTING)
     PASS_REGULAR_EXPRESSION "HI FROM ERB PYTHON PLUGIN[\r\n\t ]*HI FROM JINJA PYTHON PLUGIN"
   )
 
+  # ======================== Workflows should fail ========================
+  add_test(NAME OpenStudioCLI.Run_Validate.MissingAMeasure
+    COMMAND $<TARGET_FILE:openstudio> run --show-stdout -w missing_a_measure.osw
+    WORKING_DIRECTORY "${PROJECT_BINARY_DIR}/resources/workflow/invalid_measures/"
+  )
+  set_tests_properties(OpenStudioCLI.Run_Validate.MissingAMeasure PROPERTIES
+    WILL_FAIL TRUE
+    RESOURCE_LOCK "invalid_measures"
+  )
+
+  add_test(NAME OpenStudioCLI.Run_Validate.UnloadableMeasure
+    COMMAND $<TARGET_FILE:openstudio> run --show-stdout -w unloadable_measure.osw
+    WORKING_DIRECTORY "${PROJECT_BINARY_DIR}/resources/workflow/invalid_measures/"
+  )
+  set_tests_properties(OpenStudioCLI.Run_Validate.UnloadableMeasure PROPERTIES
+    WILL_FAIL TRUE
+    RESOURCE_LOCK "invalid_measures"
+  )
+
+  add_test(NAME OpenStudioCLI.Run_Validate.WrongMeasureTypeOrder
+    COMMAND $<TARGET_FILE:openstudio> run --show-stdout -w wrong_measure_type_order.osw
+    WORKING_DIRECTORY "${PROJECT_BINARY_DIR}/resources/workflow/invalid_measures/"
+  )
+  set_tests_properties(OpenStudioCLI.Run_Validate.WrongMeasureTypeOrder PROPERTIES
+    WILL_FAIL TRUE
+    RESOURCE_LOCK "invalid_measures"
+  )
+
+  # Classic
+  add_test(NAME OpenStudioCLI.Classic.Run_Validate.MissingAMeasure
+    COMMAND $<TARGET_FILE:openstudio> classic run --show-stdout -w missing_a_measure.osw
+    WORKING_DIRECTORY "${PROJECT_BINARY_DIR}/resources/workflow/invalid_measures/"
+  )
+  set_tests_properties(OpenStudioCLI.Classic.Run_Validate.MissingAMeasure PROPERTIES
+    WILL_FAIL TRUE
+    RESOURCE_LOCK "invalid_measures"
+  )
+
+  add_test(NAME OpenStudioCLI.Classic.Run_Validate.UnloadableMeasure
+    COMMAND $<TARGET_FILE:openstudio> classic run --show-stdout -w unloadable_measure.osw
+    WORKING_DIRECTORY "${PROJECT_BINARY_DIR}/resources/workflow/invalid_measures/"
+  )
+  set_tests_properties(OpenStudioCLI.Classic.Run_Validate.UnloadableMeasure PROPERTIES
+    WILL_FAIL TRUE
+    RESOURCE_LOCK "invalid_measures"
+  )
+
+  add_test(NAME OpenStudioCLI.Classic.Run_Validate.WrongMeasureTypeOrder
+    COMMAND $<TARGET_FILE:openstudio> classic run --show-stdout -w wrong_measure_type_order.osw
+    WORKING_DIRECTORY "${PROJECT_BINARY_DIR}/resources/workflow/invalid_measures/"
+  )
+  set_tests_properties(OpenStudioCLI.Classic.Run_Validate.WrongMeasureTypeOrder PROPERTIES
+    WILL_FAIL TRUE
+    RESOURCE_LOCK "invalid_measures"
+  )
+  # ====================== End Workflows should fail ======================
+
   if (Pytest_AVAILABLE)
     add_test(NAME OpenStudioCLI.test_loglevel
       COMMAND ${Python_EXECUTABLE} -m pytest --verbose ${Pytest_XDIST_OPTS} --os-cli-path $<TARGET_FILE:openstudio> "${CMAKE_CURRENT_SOURCE_DIR}/test/test_loglevel.py"
diff --git a/src/utilities/filetypes/WorkflowJSON.cpp b/src/utilities/filetypes/WorkflowJSON.cpp
index 1ba116ceea1..4bbc8a60393 100644
--- a/src/utilities/filetypes/WorkflowJSON.cpp
+++ b/src/utilities/filetypes/WorkflowJSON.cpp
@@ -46,10 +46,12 @@ namespace detail {
     std::string formattedErrors;
     bool parsingSuccessful = Json::parseFromStream(rbuilder, ss, &m_value, &formattedErrors);
 
+    openstudio::path p;
+
     if (!parsingSuccessful) {
 
       // see if this is a path
-      openstudio::path p = toPath(s);
+      p = toPath(s);
       if (boost::filesystem::exists(p) && boost::filesystem::is_regular_file(p)) {
         // open file
         std::ifstream ifs(openstudio::toSystemFilename(p));
@@ -65,6 +67,10 @@ namespace detail {
 
     parseSteps();
     parseRunOptions();
+
+    if (!p.empty()) {
+      setOswPath(p, false);
+    }
   }
 
   WorkflowJSON_Impl::WorkflowJSON_Impl(const openstudio::path& p) {
@@ -851,6 +857,65 @@ namespace detail {
     }
   }
 
+  bool WorkflowJSON_Impl::validateMeasures() const {
+    // TODO: should we exit early, or return all problems found?
+
+    bool result = true;
+    MeasureType state = MeasureType::ModelMeasure;
+
+    for (size_t i = 0; const auto& step : m_steps) {
+      LOG(Debug, "Validating step " << i);
+      if (auto step_ = step.optionalCast<MeasureStep>()) {
+        // Not calling getBCLMeasure because I want to mimic workflow-gem and be as explicit as possible about what went wrong
+        const auto measureDirName = step_->measureDirName();
+        auto measurePath_ = findMeasure(measureDirName);
+        if (!measurePath_) {
+          LOG(Error, "Cannot find measure '" << measureDirName << "'");
+          result = false;
+          continue;
+        }
+        auto bclMeasure_ = BCLMeasure::load(*measurePath_);
+        if (!bclMeasure_) {
+          LOG(Error, "Cannot load measure '" << measureDirName << "' at '" << *measurePath_ << "'");
+          result = false;
+          continue;
+        }
+
+        // Ensure that measures are in order, i.e. no OS after E+, E+ or OS after Reporting
+        const auto measureType = bclMeasure_->measureType();
+
+        if (measureType == MeasureType::ModelMeasure) {
+          if (state == MeasureType::EnergyPlusMeasure) {
+            LOG(Error, "OpenStudio measure '" << measureDirName << "' called after transition to EnergyPlus.");
+            result = false;
+          }
+          if (state == MeasureType::ReportingMeasure) {
+            LOG(Error, "OpenStudio measure '" << measureDirName << "' called after Energyplus simulation.");
+            result = false;
+          }
+
+        } else if (measureType == MeasureType::EnergyPlusMeasure) {
+          if (state == MeasureType::ReportingMeasure) {
+            LOG(Error, "EnergyPlus measure '" << measureDirName << "' called after Energyplus simulation.");
+            result = false;
+          }
+          if (state == MeasureType::ModelMeasure) {
+            state = MeasureType::EnergyPlusMeasure;
+          }
+
+        } else if (measureType == MeasureType::ReportingMeasure) {
+          state = MeasureType::ReportingMeasure;
+
+        } else {
+          LOG(Error, "MeasureType " << measureType.valueName() << " of measure '" << measureDirName << "' is not supported");
+          result = false;
+        }
+      }
+      ++i;
+    }
+
+    return result;
+  }
 }  // namespace detail
 
 WorkflowJSON::WorkflowJSON() : m_impl(std::shared_ptr<detail::WorkflowJSON_Impl>(new detail::WorkflowJSON_Impl())) {}
@@ -1123,6 +1188,10 @@ void WorkflowJSON::resetRunOptions() {
   getImpl<detail::WorkflowJSON_Impl>()->resetRunOptions();
 }
 
+bool WorkflowJSON::validateMeasures() const {
+  return getImpl<detail::WorkflowJSON_Impl>()->validateMeasures();
+}
+
 std::ostream& operator<<(std::ostream& os, const WorkflowJSON& workflowJSON) {
   os << workflowJSON.string();
   return os;
diff --git a/src/utilities/filetypes/WorkflowJSON.hpp b/src/utilities/filetypes/WorkflowJSON.hpp
index 1ac097baa7f..5d6c4dd693b 100644
--- a/src/utilities/filetypes/WorkflowJSON.hpp
+++ b/src/utilities/filetypes/WorkflowJSON.hpp
@@ -233,6 +233,9 @@ class UTILITIES_API WorkflowJSON
   /** Reset RunOptions for this workflow. */
   void resetRunOptions();
 
+  /** Checks that all measures in the Workflow can be found, and are in the correct order (ModelMeasure > EnergyPlusMeasure > ReportingMeasure) */
+  bool validateMeasures() const;
+
  protected:
   // get the impl
   template <typename T>
diff --git a/src/utilities/filetypes/WorkflowJSON_Impl.hpp b/src/utilities/filetypes/WorkflowJSON_Impl.hpp
index 65d65c52e83..03be7a07339 100644
--- a/src/utilities/filetypes/WorkflowJSON_Impl.hpp
+++ b/src/utilities/filetypes/WorkflowJSON_Impl.hpp
@@ -153,6 +153,8 @@ namespace detail {
     // Emitted on any change
     Nano::Signal<void()> onChange;
 
+    bool validateMeasures() const;
+
    private:
     REGISTER_LOGGER("openstudio.WorkflowJSON");
 
diff --git a/src/workflow/RunInitialization.cpp b/src/workflow/RunInitialization.cpp
index a35ba499a64..0d691b2dba5 100644
--- a/src/workflow/RunInitialization.cpp
+++ b/src/workflow/RunInitialization.cpp
@@ -54,10 +54,17 @@ void OSWorkflow::runInitialization() {
     }
   });
 
-  // TODO: create the runner with our WorkflowJSON (workflow gem uses datapoint/analysis too?!)
-
   // TODO: Validate the OSW measures if the flag is set to true, (the default state)
-  // Note JM 2022-11-07: Is it better to try and load all measures once, instead of crashing later?
+  // There isn't a 'verify_osw' key in the RunOptions, so always do it for now. Maybe don't if `fast`?
+  {
+    LOG(Info, "Attempting to validate the measure workflow");
+
+    if (!workflowJSON.validateMeasures()) {
+      LOG_AND_THROW("Workflow is invalid");
+    }
+
+    LOG(Info, "Validated the measure workflow");
+  }
 
   LOG(Debug, "Finding and loading the seed file");
   auto seedPath_ = workflowJSON.seedFile();


Test patch:
diff --git a/src/utilities/filetypes/test/WorkflowJSON_GTest.cpp b/src/utilities/filetypes/test/WorkflowJSON_GTest.cpp
index d88b5b8247a..48e2eb5a070 100644
--- a/src/utilities/filetypes/test/WorkflowJSON_GTest.cpp
+++ b/src/utilities/filetypes/test/WorkflowJSON_GTest.cpp
@@ -15,9 +15,11 @@
 
 #include "../../time/DateTime.hpp"
 
+#include "../../core/Filesystem.hpp"
 #include "../../core/Exception.hpp"
 #include "../../core/System.hpp"
 #include "../../core/Checksum.hpp"
+#include "../../core/StringStreamLogSink.hpp"
 
 #include <resources.hxx>
 
@@ -1436,3 +1438,48 @@ TEST(Filetypes, RunOptions_overrideValuesWith) {
   ASSERT_FALSE(ftOptions.excludeSpaceTranslation());
   ASSERT_TRUE(ftOptions.isExcludeSpaceTranslationDefaulted());
 }
+
+TEST(Filetypes, WorkflowJSON_ValidateMeasures_Ok) {
+  auto p = resourcesPath() / toPath("utilities/Filetypes/full.osw");
+  WorkflowJSON w(p);
+  EXPECT_TRUE(w.validateMeasures());
+}
+
+TEST(Filetypes, WorkflowJSON_ValidateMeasures_Missing) {
+  auto p = resourcesPath() / toPath("workflow/invalid_measures/missing_a_measure.osw");
+  ASSERT_TRUE(boost::filesystem::is_regular_file(p));
+  WorkflowJSON w(p);
+  StringStreamLogSink sink;
+  sink.setLogLevel(Error);
+  EXPECT_FALSE(w.validateMeasures());
+  ASSERT_EQ(1, sink.logMessages().size());
+  EXPECT_EQ("Cannot find measure 'NON_EXISTING_MEASURE_THIS_SHOULD_BE_CAUGHT'", sink.logMessages()[0].logMessage());
+}
+
+TEST(Filetypes, WorkflowJSON_ValidateMeasures_Unloadable) {
+  auto p = resourcesPath() / toPath("workflow/invalid_measures/unloadable_measure.osw");
+  ASSERT_TRUE(boost::filesystem::is_regular_file(p));
+  WorkflowJSON w(p);
+  StringStreamLogSink sink;
+  sink.setLogLevel(Error);
+  EXPECT_FALSE(w.validateMeasures());
+  auto logMessages = sink.logMessages();
+  ASSERT_EQ(3, logMessages.size());
+  EXPECT_EQ("utilities.bcl.BCLXML", logMessages.at(0).logChannel());
+  EXPECT_EQ("utilities.bcl.BCLMeasure", logMessages.at(1).logChannel());
+  EXPECT_EQ("openstudio.WorkflowJSON", logMessages.at(2).logChannel());
+  auto logMessage = sink.logMessages()[2].logMessage();
+  EXPECT_TRUE(logMessage.find("Cannot load measure 'UnloadableMeasure' at '") != std::string::npos) << logMessage;
+}
+
+TEST(Filetypes, WorkflowJSON_ValidateMeasures_WrongOrder) {
+  auto p = resourcesPath() / toPath("workflow/invalid_measures/wrong_measure_type_order.osw");
+  ASSERT_TRUE(boost::filesystem::is_regular_file(p));
+  WorkflowJSON w(p);
+  StringStreamLogSink sink;
+  sink.setLogLevel(Error);
+  EXPECT_FALSE(w.validateMeasures());
+  ASSERT_EQ(1, sink.logMessages().size());
+
+  EXPECT_EQ("OpenStudio measure 'FakeModelMeasure' called after Energyplus simulation.", sink.logMessages()[0].logMessage());
+}