Patent Document ID: 9875736
Application ID: 14625828
Patent Status: 1

Claim One:
1. A sequence tagging system that provides for transfer learning, the sequence tagging system comprising: a computing device including a processing unit and a memory, the processing unit implementing a first hidden layered conditional random field (HCRF) model, the first HCRF model comprises a pre-training system and a first training system, the pre-training system is operable to: obtain unlabeled data; run a word clustering algorithm on the unlabeled data to generate word clusters; determine a pseudo-label for each input of the unlabeled data based on the word clusters to form pseudo-labeled data; extract pre-training features from the pseudo-labeled data, and estimate pre-training model parameters for the pre-training features utilizing a first training algorithm, wherein the pre-training model parameters are stored in a first hidden layer of the first HCRF model, wherein the first hidden layer of the first HCRF model further includes a first upper layer and a first lower layer, wherein task-shared pre-training model parameters are stored in the first lower layer, and wherein first task specific model parameters and task specific pre-training model parameters are stored in the first upper layer; the first training system is operable to: obtain a first set of labeled data for a first specific task; estimate the first task specific model parameters based on a second training algorithm that is initialized utilizing the pre-training model parameters; send the first lower layer of the first hidden layer of the first HCRF model to a second HCRF model; and applying the second HCRF model to a tagging task to determine a tag and outputting by the second HCRF model the determined tag as a result.