Patent ID: 11886461
Assignee: SALESFORCE, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 14:
15. A method for machine-learnt field-specific standardization, the method comprising:
training a machine-learning model to tokenize raw values and corresponding standardized values into raw token sequences and corresponding standardized token sequences;
training the machine-learning model to learn standardization of each one of a plurality of specific data fields from token insertions and token substitutions that modify the raw token sequences to match the corresponding standardized token sequences;
tokenizing an input value into an input token sequence;
determining, by the machine-learning model, a probability of inserting an insertion token after an insertion markable token in the input token sequence, based on learning a standardization of one of the plurality of specific data fields which comprises inserting the insertion token after a class of the insertion markable token in the raw token sequences;
determining whether the probability of inserting the insertion token exceeds a threshold;
inserting the insertion token, which is punctuation inserted after the insertion markable token in the input token sequence, in response to a determination that the probability of inserting the insertion token exceeds the threshold;
determining, by the machine-learning model, a probability of substituting a substitution token, for a substitutable token that differs in location from the insertion markable token, in the input token sequence, based on learning a standardization of one of the plurality of specific data fields which comprises substituting the substitution token for a class of the substitutable token in the raw token sequences;
determining whether the probability of substituting the substitution token exceeds another threshold; and
substituting the substitution token, which is one of nil, a letter that is part of a word, or a digit that is part of a number, for the substitutable token in the input token sequence, in response to a determination that the probability of substituting the substitution token exceeds the other threshold.