## 4 Proposal for Data Transformation to Resolve Incompatibilities between Medical Formats
### NLP Parser Details  
Due to the scarcity of trained medical NER models, a decision was made to move away from simply analysing the text with NER, in favour of deploying several approaches at once. The NLP implemented in the prototype in particular combines NER, word vectors and rule-based matching in an attempt to maximize the recognition of important details, with the techniques being applied in the following order:  
Let there be hyperparameters \(S\) that indicate how synonyms for the program load, coefficients \(k_{1},k_{2},\ldots,k_{N}\), lower and upper thresholds \(l_{1},l_{2},\ldots,l_{N}\) and \(u_{1},u_{2},\ldots,u_{N}\), where \(i=1,2,\ldots,N\) is the unique number for each NLP model, then:  
1. Load the program's schema stored in file and NLP models, as well as second instances of these models with NER disabled. Look through the feature labels and their units in the schema;
2. If all labels have been processed, proceed to _step 5_. Otherwise, load the next label and proceed to _step 3_;
3. Check the label name and assigned unit in the file. If a recognized feature is a collection or a Boolean, load \(S\) label synonyms from the word vector model, else check a numerical value's unit. If there is one, select the names of all the units with compatible dimensionality;
4. Create extra entities and entity recognition rules based on the findings in _step 3_, replace the missing NER component in the previously loaded model instances with the entity rulers based on our newly created rules;
5. Perform NER, group each recognized phrase by entity. Have the models that discovered respective entities compare it to the existing schema labels. If similarity reaches the given lower threshold value \(l_{i}\), its values are passed on for further selection as potential entries to fill the missing values;
6. Selected values are now sorted by their similarity score to a certain feature label. The similarity score is also modified by a coefficient \(k_{i}\) that is unique to the model that discovered the value. Values with scores of \"1\" and above are automatically assigned to the features. If there is more than one value like that, the one with maximum score is always assigned;
7. If all the labels have assigned values, proceed to _step 10_. Otherwise, proceed to _step 9_;
8. Process values with the score below \"1\" by comparing the highest score among a label's assigned values to the upper threshold numbers. If the score is greater or equal than \(u_{i}\), where \(i\) is the number of the model that discovered the value, proceed to _step 9_. Otherwise, repeat with the next label;
9. Parse the selected value depending on whether it is a numerical value, Boolean, or a connection of values. Numerical values (both with assigned units and without them)  
Figure 5: Three features end up in one table, with application of polynomials for dependent trait calculation.  
are processed using a special function that parses the string, collections are processed by finding a collection value most similar to the discovered one, and Booleans are parsed by discovering negation in the sentences;
* Assign the parsed result to the feature currently being processed;
* Return to _step 8_ if there are more feature labels to be analysed, otherwise finish parsing.