alphagov/trade-tariff-backend — pull request #37 by sauliusg (2012-11-29 12:10:59+00:00)
Title: Chief transformer refactor

Okay this fix was long due. It is related to the email I sent a while ago. Two major things here:

1. Certain TARIC updates may invalidate national measures apparently. GoodsNomenclature associated to Measure should span its validity dates (ME8 conformance test). So say we create Measure for GoodsNomenclature that has open ended validity end date. Later Taric updates that GoodsNomenclature to have certain validity date. Some of the national Measures can become invalid. On top of that, CHIEF may try to update those invalid measures later on (just what happened actually). So to prevent it we check for national measure validity after GoodsNomeclature gets updated and do a soft delete (set invalidated by to Taric transaction id) for the measure. Later on, we ignore validations for such measures in case CHIEF will try to update them, so we just allow that. I found that there were 56 national measures invalidated up to 29th of Nov.

2. Consolidate update application and transformation, so that transformation is invoked after CHIEF updates get applied. This would eventually become an issue for snapshot recreation (in case of need) process. We used to apply all Taric updates, then all CHIEF updates and then do transformation in one go for all CHIEF records. Imagine a scenario where Taric introduces Commodity on 1st of July, CHIEF adds national measures on say the 2nd of July and Taric removes Commodity on 1st of August. If we would apply all Taric's updates then we wouldn't even create CHIEF measure as if that Commodity was never there.