Cross-domain multi-task learning for text classification

A method includes providing input text to a plurality of multi-task learning (MTL) models corresponding to a plurality of domains. Each MTL model is trained to generate an embedding vector based on the input text. The method further includes providing the input text to a domain identifier that is trained to generate a weight vector based on the input text. The weight vector indicates a classification weight for each domain of the plurality of domains. The method further includes scaling each embedding vector based on a corresponding classification weight of the weight vector to generate a plurality of scaled embedding vectors, generating a feature vector based on the plurality of scaled embedding vectors, and providing the feature vector to an intent classifier that is trained to generate, based on the feature vector, an intent classification result associated with the input text.

This disclosure is generally related to cross-domain multi-task learning for text classification.

In a particular example, a computer-implemented method of cross-domain multi-task learning (MTL) for text classification includes obtaining input text and providing the input text to a plurality of MTL models corresponding to a plurality of domains. Each MTL model is trained, based on text samples corresponding to a respective domain of the plurality of domains, to generate an embedding vector based on the input text. The computer-implemented method further includes providing the input text to a domain identifier that is trained, based on the text samples associated with the plurality of domains, to generate a weight vector based on the input text. The weight vector indicates a classification weight for each domain of the plurality of domains, where the classification weight for a particular domain is associated with a probability that the input text is associated with the particular domain. The computer-implemented method further includes scaling each embedding vector based on a corresponding classification weight of the weight vector to generate a plurality of scaled embedding vectors, generating a feature vector based on the plurality of scaled embedding vectors, and providing the feature vector to an intent classifier that is trained to generate, based on the feature vector, an intent classification result associated with the input text.

In another example, an apparatus includes a memory and a processor coupled to the memory. The processor is configured to obtain input text and to provide the input text to a plurality of MTL models corresponding to a plurality of domains. Each MTL model is trained, based on text samples corresponding to a respective domain of the plurality of domains, to generate an embedding vector based on the input text. The processor is further configured to provide the input text to a domain identifier that is trained, based on the text samples associated with the plurality of domains, to generate a weight vector based on the input text. The weight vector indicates a classification weight for each domain of the plurality of domains, where the classification weight for a particular domain is associated with a probability that the input text is associated with the particular domain. The processor is further configured to scale each embedding vector based on a corresponding classification weight of the weight vector to generate a plurality of scaled embedding vectors, to generate a feature vector based on the plurality of scaled embedding vectors, and to provide the feature vector to an intent classifier that is configured to generate, based on the feature vector, an intent classification result associated with the input text.

In another example, a computer-readable medium stores instructions executable by a processor to perform, initiate, or control operations. The operations include obtaining input text and providing the input text to a plurality of MTL models corresponding to a plurality of domains. Each MTL model is trained, based on text samples corresponding to a respective domain of the plurality of domains, to generate an embedding vector based on the input text. The operations further include providing the input text to a domain identifier that is trained, based on the text samples associated with the plurality of domains, to generate a weight vector based on the input text. The weight vector indicates a classification weight for each domain of the plurality of domains, where the classification weight for a particular domain is associated with a probability that the input text is associated with the particular domain. The operations further include scaling each embedding vector based on a corresponding classification weight of the weight vector to generate a plurality of scaled embedding vectors, generating a feature vector based on the plurality of scaled embedding vectors, and providing the feature vector to an intent classifier that is trained to generate, based on the feature vector, an intent classification result associated with the input text.

IV. DETAILED DESCRIPTION

Text classification plays an important role in certain services, such as enterprise-level natural language services that include interactive text response. An interactive text response system can be trained using labeled data (e.g., data that includes labeled examples to train an artificial intelligence system, such as a neural network).

Due to language diversity, data assigned to the same label can share a relatively small amount of lexical information. As a result, effectiveness of classifiers that are based on lexical information from sentences (other than semantic similarity) is reduced.

Some techniques use deep learning training to improve accuracy of text classification. A deep learning approach may involve learning sentence representations by recognizing semantic relations between the examples in the labeled data.

In some cases, deep learning training is inadequate for certain applications. For example, certain commercial systems (e.g., enterprise conversational service systems) may call for low training latency, such as where training sets are edited and tested in real-time or near real-time. As a result, some deep learning training is performed offline.

Offline deep learning training techniques may use different data sets associated with different domains (e.g., different sets of labeled data that feature different training examples). For example, a first data set corresponding to a first domain may be used to train a deep learning model, followed by retraining using a second data set corresponding to a second domain, etc. In some cases, separately training a deep learning model using multiple data sets can reduce training effectiveness and increase training latency. For example, in some circumstances, retraining using the second data set may “undo” certain aspects of training based on the first data set.

In accordance with one aspect of the disclosure, a multi-task learning (MTL) framework uses offline cross-domain training. For example, multiple MTL models may each be trained based on a corresponding domain of multiple domains, and a domain identifier may be trained based on each of the multiple domains. At runtime, each of the multiple MTL models may process input text to generate a respective embedding vector, and the domain identifier may generate a weight vector indicating, for each of the multiple domains, a probability that the input text is associated with the domain. The embedding vectors may be scaled based on the weight vector (e.g., by increasing weight assigned to embedding vectors associated with a greater probability, by reducing weight assigned to embedding vectors associated with a lower probability, or both). As a result, latency associated with training and/or runtime operation may be reduced as compared to deep learning techniques that sequentially train based on multiple domains.

Alternatively or in addition, use of multiple MTL models enables effective processing of a particular data input that includes information associated with multiple domains. As a particular illustrative example, particular input data may have content that semantically “overlaps” domains, such as text having a 90 percent association (or “match”) with one domain and a 10 percent association (or match) with another domain. Instead of assigning the input data to a single domain as in certain conventional techniques, processing can be weighted, such as by weighting data associated with one domain with a value of 0.9 and by weighting data associated with another domain with a value of 0.1. As a result, accuracy of certain classification operations is increased by accounting for semantic overlap between domains (e.g., instead of selecting a single domain as a “best match” for input data).

FIG. 1Aillustrates an example of a system100for cross-domain multi-task learning for text classification in accordance with an example of the disclosure. In some implementations, the system100corresponds to or is included in an interactive text response system.

In the example ofFIG. 1A, the system100includes a plurality of multi-task learning (MTL) models108. To illustrate, inFIG. 1A, the plurality of MTL models108includes a first MTL model110and a second MTL model112. In some implementations, the plurality of MTL models108includes a different number of MTL models than illustrated in the example ofFIG. 1A(e.g., three or more MTL models).

Each MTL model of the plurality of MTL models108corresponds to a respective domain of a plurality of domains. To illustrate, the plurality of domains may include different text response applications having different text response characteristics. In some examples, the plurality of domains each are based on different enterprise networks, different business applications, or different business entities. As a particular example, the first MTL model110may correspond to a banking domain of the plurality of domains, and the second MTL model112may correspond to an insurance domain of the plurality of domains. In this particular example, operation of the first MTL model110may differ from operation of the second MTL model112due to differences between text associated with banking and text associated with insurance.

In a particular example, the system100further includes a domain identifier122. In some examples, the domain identifier122is configured to classify text according to the plurality of domains (e.g., by determining, for each domain of the plurality of domains, a probability that text is associated with the domain).

The example ofFIG. 1Aalso illustrates that the system100includes a scaler132. In one example, the scaler132is coupled to the plurality of MTL models108and to the domain identifier122. In a particular example, the system100further includes a feature vector generator140and an intent classifier152. In some implementations, the feature vector generator140is coupled to the scaler132, and the intent classifier152is coupled to the feature vector generator140. Depending on the particular implementation, any aspect depicted inFIG. 1Acan be implemented using hardware, instructions executed by a processor, or a combination thereof.

By using the plurality of MTL models108, certain operations can be performed concurrently instead of sequentially. For example, in some implementations, concurrently training the plurality of MTL models108based on a plurality of domains reduces training time as compared to sequential training. Further, concurrent training of the plurality of MTL models108may reduce or prevent instances of “undoing” (or overriding) training, which may occur in the case of sequentially training a single model.

Alternatively or in addition, in some implementations, the system100is configured to process input data that includes information associated with multiple domains. As a particular example, particular input data may include text having a 90 percent association (or “match”) with one domain and a 10 percent association (or match) with another domain. Instead of assigning the input data to a single domain as in certain conventional techniques, processing operations by the system100can be weighted (e.g., by weighting data associated with one domain with a value of 0.9 and by weighting data associated with another domain with a value of 0.1). For example, outputs generated by the plurality of MTL models108can be weighted by the scaler132based on weights generated by the domain identifier122, as described further with reference toFIG. 1B.

Referring toFIG. 1B, a particular example of the system100includes a processor102and a memory162that is coupled to the processor102. In a particular example, the memory162is configured to store instructions176that are executable by the processor102to initiate, control, or perform operations described herein. InFIG. 1B, certain features described with reference toFIG. 1Aare included in (or implemented using) the processor102.

During operation, a first process (e.g., a training process) may be performed to train certain operations of the processor102. To illustrate, the training process may include training the plurality of MTL models108based on text samples164corresponding to the plurality of domains. As a particular example, the text samples164may include first text samples166corresponding to the first domain associated with the first MTL model110and may further include second text samples168corresponding to the second domain associated with the second MTL model112. In this example, the training process may include training the first MTL model110based on the first text samples166(e.g., by inputting the first text samples166to the first MTL model110) and may further include training the second MTL model112based on the second text samples168(e.g., by inputting the second text samples168to the second MTL model112). In some examples of the training process, the first MTL model110is trained irrespective of the second text samples168(e.g., where the second text samples168are not provided to the first MTL model110), and the second MTL model112is trained irrespective of the first text samples166(e.g., where the first text samples166are not provided to the second MTL model112).

The training process may further include training the domain identifier122based on the text samples164(e.g., by inputting the text samples164to the domain identifier122). In a particular example, the domain identifier122is trained using the first text samples166and the second text samples168.

In some implementations, the plurality of MTL models108and the domain identifier122are trained further based on labeled training data170. To illustrate, the first MTL model110may be trained (e.g., according to a training process) using first labeled training data172associated with the first domain of the plurality of domains, and the second MTL model112may be trained using second labeled training data174associated with the second domain of the plurality of domains, where the second labeled training data174is distinct from the first labeled training data172. In some examples, the labeled training data170includes data that is pre-verified as corresponding to different domains of the plurality of domains.

In one example, the domain identifier122is trained using both the first labeled training data172and the second labeled training data174. In another example, the domain identifier122is trained using cross-domain training data178that is independent of the first labeled training data172and the second labeled training data174.

The training process may further include training the intent classifier152. In some implementations, training the intent classifier152includes configuring the intent classifier152to determine (or estimate) intent associated with text or speech (e.g., by determining whether the text or speech includes a command or a question, as an illustrative example).

The processor102is configured to obtain input text106. To illustrate, in one example, the processor102is configured to operate based on a runtime mode of operation after completion of the training process. In some examples, the input text106corresponds to audio speech input104. For example, the audio speech input104may be recorded (e.g., using a microphone), digitized (e.g., using an analog-to-digital converter), and converted to the input text106(e.g., using a speech recognition technique).

The processor102is configured to provide the input text106to the plurality of MTL models108. The plurality of MTL models108is configured to generate, based on the input text106, embedding vectors126. Each embedding vector of the embedding vectors126corresponds to a respective domain of the plurality of domains associated with the plurality of MTL models108. For example, inFIG. 1B, the embedding vectors126include a first embedding vector128that is associated with the first domain and that is generated by the first MTL model110. As another example,FIG. 1Balso illustrates that the embedding vectors126include a second embedding vector130that is associated with the second domain and that is generated by the second MTL model112.

As used herein, an embedding vector may refer to a vector having values that indicate whether certain words, phrases, sentences, or other language features are present (or detected) in the input text106. Each domain of the plurality of domains may be associated with a particular set of words, phrases, sentences, or other language features. Thus, an embedding vector associated with one domain may differ from an embedding vector associated with another domain (e.g., where the first embedding vector128differs from the second embedding vector130due to differences in features associated with the first domain and the second domain).

In some examples, each MTL model of the plurality of MTL models108includes (or operates according to) a convolutional neural network (CNN). To illustrate, in the example ofFIG. 1B, the first MTL model110includes a first CNN114, and the second MTL model112includes a second CNN116. In some implementations, each CNN includes one or more max pooling layers. For example, inFIG. 1B, the first CNN114includes one or more max pooling layers118, and the second CNN116includes one or more max pooling layers120.

To further illustrate, in one example, the first MTL model110is configured to generate the first embedding vector128by providing the input text106as input to an upper convolutional layer of the first CNN114. The first MTL model110may build a max-pooling layer of the one or more max pooling layers118on a convolutional output of the first CNN114to generate the first embedding vector128.

The processor102is further configured to provide the input text106to the domain identifier122. The domain identifier122is configured to generate, based on the input text106, a weight vector124. The weight vector124includes a classification weight for each domain of the plurality of domains, and the classification weight for a particular domain is associated with a probability that the input text106is associated with the particular domain. To illustrate, inFIG. 1B, the weight vector124includes a first classification weight W1indicating a first probability that the input text106is associated with the first domain associated with the first MTL model110. As another example,FIG. 1Balso depicts that the weight vector124includes a second classification weight W2indicating a second probability that the input text106is associated with the second domain associated with the second MTL model112. To illustrate, in one example, the domain identifier122determines that the input text106is more likely to be classified in the first domain (e.g., based on a first number of keywords in the input text106that are associated with the first domain) and is less likely to be classified in the second domain (e.g., based on a second number of keywords in the input text106that are associated with the second domain, where the second number is less than the first number). In this example, the domain identifier122can assign a probability to the first classification weight W1that is greater than a probability assigned to the second classification weight W2. Alternatively or in addition, the domain identifier122can be configured to determine the classification weights W1and W2using one or more other techniques, such as using a support vector machine (SVM) technique, a linear regression technique, or a neural network model, as illustrative examples.

In some implementations, domains of the plurality of domains are non-exclusive (e.g., overlapping), and probabilities of the weight vector124can add up to more than one. For example, if the first domain overlaps the second domain (e.g., where subject matter is shared by the first domain and the second domain), then the first classification weight W1and the second classification weight W2can add up to more than one (e.g., where the classification weights W1, W2indicate that the input text is likely to correspond to both the first domain and the second domain). In other implementations, domains of the plurality of domains are exclusive (e.g., non-overlapping), and probabilities of the weight vector124do not add up to more than one.

In some implementations, the domain identifier122is configured to determine each weight of the weight vector124based on a threshold123. For example, the domain identifier122may be configured to “round” a weight of the weight vector to a particular probability (e.g., zero or one) based on whether the weight satisfies (e.g., is greater than or equal to) the threshold123. As an illustrative example, if the threshold corresponds to 0.5 and if the probability that the input text106corresponds to the first domain is 0.6, then the domain identifier122may be configured to round the first weight W1up to one to generate a probability of one. As another example, if the probability that the input text106corresponds to the second domain is 0.4, then the domain identifier122may be configured to round the second weight W2down to zero to generate a probability of zero.

The processor102is configured to scale each embedding vector of the plurality of embedding vectors126based on a corresponding classification weight of the weight vector124to generate a plurality of scaled embedding vectors134. In a particular example, the scaler132is configured to receive the weight vector124and the plurality of embedding vectors126and to scale the plurality of embedding vectors126based on the weight vector124to generate the plurality of scaled embedding vectors134. For example, inFIG. 1B, the scaler132is configured to scale the first embedding vector128based on the first classification weight W1to generate a first scaled embedding vector (SEV)136. As another example, inFIG. 1B, the scaler132is configured to scale the second embedding vector130based on the second classification weight W2to generate a second scaled embedding vector (SEV)138. In some implementations, scaling an embedding vector includes modifying (e.g., increasing or decreasing) a magnitude of the embedding vector based on a corresponding classification weight (e.g., by increasing the magnitude in response to a greater probability indicated by the classification weight, or by decreasing the magnitude in response to a lower probability indicated by the classification weight). In some examples, scaling the embedding vector128accounts for semantic overlap of the input text106among multiple domains (e.g., where the input text106is a 90 percent match to the first domain and a 10 percent match to the second domain, as an illustrative example).

The processor102is configured to generate a feature vector146based on the plurality of scaled embedding vectors134. In a particular example, the feature vector generator140is configured to receive the plurality of scaled embedding vectors134from the scaler132and to generate the feature vector146based on the plurality of scaled embedding vectors134, such as by concatenating the plurality of scaled embedding vectors134.

In some examples, the processor102is further configured to generate one or more natural language processing (NLP) features142based on the input text106, and the feature vector generator140is further configured to generate the feature vector146by concatenating the plurality of scaled embedding vectors134and the one or more NLP features142. For example, in some implementations, the one or more NLP features142include an n-gram144associated with the input text106.

The processor102is further configured to provide the feature vector146to the intent classifier152. The intent classifier152is configured to generate, based on the feature vector146, an intent classification result154associated with the input text106. As a particular example, in some implementations, the intent classification result154indicates whether the input text includes a command or a question.

In some examples, the intent classification result154is provided to a component of an interactive text response system. For example, the intent classification result154may be used by the interactive text response system to determine whether the input text106indicates that a user is asking a question or giving a command. In some examples, the interactive text response system uses the intent classification result154to answer a question indicated by the audio speech input104(e.g., by querying a server for data responsive to the question and by providing the data to the user in the form of audio, graphical information, or a combination thereof). Alternatively or in addition, in some examples, the interactive text response system uses the intent classification result154to perform an operation responsive to a command indicated by the audio speech input104(e.g., by providing a control signal to a particular device, by initiating an Internet communication, or by performing another operation).

Although certain examples are provided for illustration, it is noted that other examples are within the scope of the disclosure. For example, in some implementations, one or both of the MTL models110,112include or operate based on another network (e.g., a recurrent neural network) alternatively or in addition to the CNNs114,116and the max pooling layers118,120. Examples of recurrent neural networks include a gated recurrent unit (GRU) neural network or a long short-term memory (LSTM) neural network. To further illustrate, in other examples, the MTL models110,112are configured to perform sentence encoding to generate the embedding vectors128,130using one or more other techniques, such as using a “last hidden vector” technique, as an illustrative example.

One or more aspects ofFIGS. 1A and 1Bincrease efficiency of training and/or runtime operation associated with a text classification system, such as the system100. For example, by using the plurality of MTL models108corresponding to a plurality of domains, certain operations can be performed concurrently instead of sequentially. In some implementations, concurrently training the plurality of MTL models108based on a plurality of domains reduces training time as compared to sequential training. Further, concurrent training of the plurality of MTL models108may reduce or prevent instances of “undoing” (or overriding) training, which may occur in the case of sequentially training a model.

Alternatively or in addition, in some implementations, the system100is configured to process input data that includes information associated with multiple domains. As a particular example, the input text106may include text having a 90 percent association (or “match”) with one domain and a 10 percent association (or match) with another domain. Instead of assigning the input text106to a single domain as in certain conventional techniques, processing operations by the system100can be weighted (e.g., by weighting data associated with one domain with a value of 0.9 and by weighting data associated with another domain with a value of 0.1). As a result, accuracy of certain classification operations is increased by accounting for semantic overlap between domains (e.g., instead of selecting a single domain as a “best match” for input data).

Referring toFIG. 2, an example of a computer-implemented method of cross-domain multi-task learning for text classification is depicted and generally designated200. In a particular example, the method200is performed by the processor102ofFIG. 1B.

The method200includes obtaining input text, at202. For example, the processor102may be configured to receive the input text106.

The method200further includes providing the input text to a plurality of MTL models corresponding to a plurality of domains, at204. Each MTL model is trained, based on text samples corresponding to a respective domain of the plurality of domains, to generate an embedding vector based on the input text. To illustrate, in one example, the input text106is provided to the plurality of MTL models108, and each of the plurality of MTL models108is trained, based on respective text samples of the text samples164, to generate an embedding vector of the plurality of embedding vectors126.

The method200further includes providing the input text to a domain identifier, at206. The domain identifier is trained, based on the text samples associated with the plurality of domains, to generate a weight vector based on the input text. The weight vector indicates a classification weight for each domain of the plurality of domains, and the classification weight for a particular domain associated with a probability that the input text is associated with the particular domain. To illustrate, in one example, the input text106is provided to the domain identifier122, and the domain identifier122is trained, based on the text samples164, to generate the weight vector124based on the input text106. In some implementations, the domain identifier122is trained to determine, for each particular domain of the plurality of domains, the probability that the input text106is associated with the particular domain and to generate the weight vector124based on whether the probability satisfies the threshold123(e.g., by rounding weights of the weight vector124, as described with reference toFIG. 1B).

The method200further includes scaling each embedding vector based on a corresponding classification weight of the weight vector to generate a plurality of scaled embedding vectors, at208. In one example, the scaler132is configured to scale the plurality of embedding vectors126based on the weight vector124to generate the plurality of scaled embedding vectors134.

The method200further includes generating a feature vector based on the plurality of scaled embedding vectors, at210. In some implementations, generating the feature vector includes concatenating the plurality of scaled embedding vectors. To illustrate, in one example, the feature vector generator140is configured to generate the feature vector146based on the plurality of scaled embedding vectors134(e.g., by concatenating the plurality of scaled embedding vectors134). In some examples, the processor102is configured to generate the one or more NLP features142based on the input text106, and the feature vector generator140is configured to generate the feature vector146by concatenating the plurality of scaled embedding vectors and the one or more NLP features142.

The method200further includes providing the feature vector to an intent classifier that is trained to generate, based on the feature vector, an intent classification result associated with the input text, at212. To illustrate, in some implementations, the intent classifier152is configured to receive the feature vector146and to generate the intent classification result154based on the feature vector146.

One or more aspects ofFIG. 2improve accuracy or efficiency of a machine learning device. For example, by weighting embedding vectors associated with a plurality of domains, accuracy of intent classification can be improved as compared to intent classification that is based on a single domain.

To further illustrate aspects of the disclosure, certain conventional techniques involve retraining for each new model, increasing training time. In some circumstances, the increased training time results in latency that is unacceptable for certain applications, such as certain applications of a real-time cloud deployed machine learning service. One aspect of the disclosure uses hot-plugging into a faster classification model for each new task, speeding the transfer learning process. In addition, the framework is suitable for multiple conversational workspaces belonging to a semantically similar domain or a specific customer.

In some implementations, the domain identifier122includes or corresponds to an independent domain classifier trained by a large-scale corpus. A workspace may be assigned to an MTL model for a specific top-level domain. In some cases, multiple workspaces belong to the same domain. In some cases, data may include multiple significantly different domains.

The domain classifier may assign a “confidence” associated with each domain. The confidence of the domain classifier may be used to rescale the sentence embedding vectors from each domain, resulting in flexibility when the target task subsumes multiple domains.

In one aspect of the disclosure, a domain-specific offline-trained model of multi-task learning for intent short-text classification is enabled by defining a set of top-level domains, by training an independent top-level domain classifier (e.g., the domain identifier122), and by constructing an MTL model of sentence representation for each top-level domain (e.g., the plurality of embedding vectors126). During runtime, a confidence distribution (e.g., the weight vector124) is obtained for the top-level domains and the sentence representation for each domain. The representations of all the domains may be concatenated with the confidences as weights can be used as features for another independent intent classifier (e.g., the intent classifier152). Thus, a higher dimensional semantic feature vector from domain-level MTL models can be combined with other types of usual text features, such as a bag-of-words n-gram or character-level n-gram features, which may be included in the one or more NLP features142.

Further, certain aspects of the disclosure enable simpler and faster training of classification models (e.g., a support vector machine (SVM) or logistic regression) for dataset training while also enabling background deep-learning MTL models in a runtime manner to generate sentence representations. Thus, dataset training turn-around is quick and accurate.

For training, a set of top-level domains may be defined, such as telecom, health, sports, politics, finance, etc. A domain classifier may be built using a large amount of training data. For example, documents belonging to each top-level domain may be randomly selected to train the domain classifier. The domain classifier can use a particular machine-learning algorithm. Features and properties may be extracted from the whole training set. In some examples, training data is provided from a particular source (e.g., a business entity) as a set of labeled documents to accurately train the top-level domain classifier.

For each domain, a multi-task learning model can be built in order to determine the sentence representation at runtime. In one example, the domain classifier is operated based on a sampled set of utterances from each training workspace to simplify the training.

An input sentence can be represented by a concatenation of embedding vectors for each word, which can be pre-trained in an unsupervised manner using a large corpus of data in a target language. An upper convolutional layer may receive the word embedding vectors as input, and the model may obtain the sentence representation by building a max-pooling layer on the top of the convolutional output. This model can be shared by all tasks/workspaces with the same top-level domain. The sentence representations can be connected with a regression layer for each workspace to get the final distribution across labels.

Thus, a general knowledge representation may be learned from existing datasets to help improve the accuracy of a new task. In one implementation, offline training includes training a sentence encoder by applying multi-task learning on a large amount of existing classification datasets. Alternatively or in addition, in another implementation, online serving is performed using the sentence encoder by applying multi-task learning on a large amount of existing classification datasets.

In connection with online serving, the sentence encoder may be used to provide extra features to train SVM classifiers (or another machine learning technique) on new datasets. A technical benefit of this transfer learning is natural resolution of knowledge conflicts among datasets while enabling learning of a shared knowledge representation (e.g., via the sentence encoder) from the existing datasets. Another benefit is that the sentence encoder can be hot-plugged for each new classification task without retraining on a new dataset, increasing speed of the transfer learning for a new task.

At runtime, given an input text example S, the domain classifier can be run to obtain domain distributions D(S). The multi-task learning model for each domain dkcan be run to obtain the sentence representations Rdk(S). The concatenated representation vectors Rdk(S) can be combined with the weights Dk(S) for the final feature representation of the sentences as:
Fk(S)=Concat{Dk(S)Rdk(S)},k=1,2 . . .K.

The features can be fed into an independent classifier, such as an SVM. Further, the MTL-based features can be combined with other more traditional text-classification features, such as a unigram, a bigram, a character n-gram, etc.

One or more aspects described herein can be implemented in a cloud computing environment. Although this disclosure includes a description on cloud computing, implementation of the teachings recited herein are not limited to a cloud computing environment. Rather, aspects of the present disclosure are capable of being implemented in conjunction with any other type of computing environment.

Referring toFIG. 3, an illustrative cloud computing environment50is depicted. As shown, cloud computing environment50includes one or more cloud computing nodes10with which local computing devices used by cloud consumers, such as, for example, personal digital assistant (PDA) or cellular telephone54A, desktop computer54B, laptop computer54C, and/or automobile computer system54N, may communicate. One or more of the nodes10may include a cross-domain MTL and text classifier302(e.g., the system100ofFIG. 1A). Aspects of the cross-domain MTL and text classifier302may be implemented using infrastructure, platforms, and/or software provided as services by the cloud computing environment50.

Nodes10may communicate with one another. They may be grouped (not shown) physically or virtually, in one or more networks, such as Private, Community, Public, or Hybrid clouds as described hereinabove, or a combination thereof. This allows the cloud computing environment50to offer infrastructure, platforms and/or software as services for which a cloud consumer does not need to maintain resources on a local computing device. The types of computing devices54A-N shown inFIG. 3are intended to be illustrative only and that computing nodes10and cloud computing environment50can communicate with any type of computerized device over any type of network and/or network addressable connection (e.g., using a web browser).

Referring toFIG. 4, a set of functional abstraction layers provided by cloud computing environment50ofFIG. 3is shown. One or more of the abstraction layers provide cross-domain multi-task learning for text classification of the system100. It should be understood in advance that the components, layers, and functions shown inFIG. 2are intended to be illustrative only and aspects of the disclosure are not limited thereto. As depicted, the following layers and corresponding functions are provided:

Hardware and software layer60includes hardware and software components. Examples of hardware components include: mainframes61; RISC (Reduced Instruction Set Computer) architecture based servers62; servers63; blade servers64; storage devices65; and networks and networking components66. In some aspects, software components include network application server software67and database software68. In some examples, the cross-domain MTL and text classifier302ofFIG. 3is included in a device of the hardware and software layer60.

FIG. 5is a block diagram of an example of a computing environment500that includes electronic components through which the described system may be implemented. The components inFIG. 5support aspects of computer-implemented methods and computer-executable program instructions or code according to the present disclosure. For example, a computing device510, or portions thereof, may execute instructions to perform cross-domain multi-task learning for text classification.

InFIG. 5, the computing device510may include the processor102ofFIG. 1B, a main memory514, an input/output (I/O) adapter546, a non-volatile memory518, a memory controller520, a bus adapter524, a display adapter554, a communications adapter550, and a disk drive adapter542. The I/O adapter546may be configured to interface with one or more user input devices548. For example, the I/O adapter546may communicate via serial interfaces (e.g., universal serial bus (USB) interfaces or Institute of Electrical and Electronics Engineers (IEEE) 1394 interfaces), parallel interfaces, display adapters, audio adapters, and other interfaces. The user input devices548may include keyboards, pointing devices, displays, speakers, microphones, touch screens, magnetic field generation devices, magnetic field detection devices, and other devices. The processor102may detect interaction events based on user input received via the I/O adapter546. Additionally, the processor102may send a graphical user interface (GUI) and related elements to a display device via the I/O adapter546.

The main memory514may include volatile memory devices (e.g., random access memory (RAM) devices), nonvolatile memory devices (e.g., read-only memory (ROM) devices, programmable read-only memory, and flash memory), or both. The main memory514of the computing device510includes software, such as an operating system532. The operating system532may include a basic/input output system for booting the computing device510as well as a full operating system to enable the computing device510to interact with users, other programs, and other devices.

The display adapter554may be configured to interface with a display device556. The communications adapter550may be configured to interface with the one or more networks552. The disk drive adapter542may be configured to interface with one or more data storage devices540. The data storage devices540may include nonvolatile storage devices, such as magnetic disks, optical disks, or flash memory devices. The data storage devices540may include both removable and non-removable memory devices. The data storage devices540may be configured to store an operating system, images of operating systems, applications, and program data. One or more buses544or other communication circuitry may enable the various components of the computing device510to communicate with one another.

The descriptions of the various aspects of the present disclosure have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the aspects disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described aspects. The terminology used herein was chosen to best explain the principles of the aspects, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the aspects disclosed herein.