Machine Learning Model Based Embedding for Adaptable Content Evaluation

A system includes a computing platform having processing hardware, and a system memory storing software code and one or more machine learning (ML) model(s) trained using contrastive learning based on a similarity metric. The processing hardware is configured to execute the software code to receive input data including a plurality of content segments, map, using the ML model(s), each of the plurality of content segments to a respective embedding in a continuous vector space to provide a plurality of mapped embeddings, and perform one of a classification or a regression of the content segments using the plurality of mapped embeddings. The processing hardware is also configured to execute the software code to discover, based on the classification or the regression, at least one new label for characterizing the plurality of content segments.

BACKGROUND

Due to its nearly universal popularity as a content medium, ever more visual media content is being produced and grade available to consumers. As a result, the efficiency with which visual images can be analyzed, classified, and processed has become increasingly important to the producers, owners, and distributors of that visual media content.

One significant challenge to the efficient classification and processing of visual media content is that entertainment and media studios produce many different types of content having differing features, such as different visual textures and movement. In the case of audio-video (AV) film and television content, for example the content produced may include live action content with realistic computer-generated imagery (CGI) elements, high complexity three-dimensional (3D) animation, and even two-dimensional (2D) hand-drawn animation. Moreover, each different type of content produced may require different treatment in pre-production, post-production, or both.

Consider, for example, the post-production treatment of AV or video content. Different types of AV or video content may benefit from different encoding schemes for streaming, or different workflows for localization. In the conventional art, the classification of content as being of a particular type is typically done manually, through human inspection, and in the example use case of video encoding, the most appropriate workflow may not be identifiable even after manual inspection, but may require trial and error to determine how to classify the content for encoding purposes. This classification process can be particularly challenging for mixed content types, such as animation embedded in otherwise live action content, or for visually complex 3D animation Inch may be better suited post-processing using live action content workflows than traditional animation workflows.

DETAILED DESCRIPTION

As noted above, entertainment and media studios produce many different types of content having differing features, such as different visual textures and movement. In the case of audio-video (AV) or video content, for example the content produced may include live action content with realistic computer-generated imagery (CGI) elements, high complexity three-dimensional (3D) animation, and even two-dimensional (2D) hand-drawn animation. Each different type of content produced may require different treatment in pre-production, post-production, or both.

As further noted above, in the post-production treatment of AV or video content, different types of video content may benefit from different encoding schemes for streaming, or different workflows for localization. In the conventional art, the classification of content as being of a particular type is done manually, through human inspection, and in the example use case of video encoding, the most appropriate workflow may not be identifiable even after manual inspection, but may require trial and error to determine how to categorize the content for encoding purposes. This classification process can be particularly challenging for mixed content types, such as animation embedded in otherwise live action content, or for visually complex 3D animation which may be better suited for post-processing using live action content workflows than traditional animation workflows.

The present application discloses systems and methods for performing machine learning (ML) model based embedding for adaptable content evaluation. It is noted that the disclosure provided in the present application focuses on optimizations within the encoding pipeline for video streaming. Examples of tasks under consideration include 1) selection of pre- and post-processing algorithms or algorithm parameters, 2) automatic encoding parameter selection per title or per segment, and 3) automatic bitrate ladder selection for adaptive streaming per title or per segment. However, the present ML model based adaptable evaluation solution is task-independent and can be used in contexts that are different from those specifically described herein.

Thus, although the present adaptable content evaluation solution is described below in detail by reference to the exemplary use case of video encoding in the interests of conceptual clarity, the present novel and inventive principles may more generally be utilized in a variety of other content post-production processes, such as colorization, color correction, content restoration, mastering, and audio cleanup or synching, to name a few examples, as well as in pre-production processing. Moreover, the adaptable content evaluation solution disclosed in the present application may advantageously be implemented as an automated process.

As defined in the present application, the terms “automation,” “automated,” and “automating” refer to systems and processes that do not require human intervention. Although in some implementations a human editor may review the content evaluations performed by the systems and using the methods described herein, that human involvement is optional. Thus, the methods described in the present application may be performed under the control of hardware processing components of the disclosed automated systems.

Moreover, as defined in the present application, the expression “ML model” may refer to a mathematical model for making future predictions based on patterns learned from samples of data or “training data.” Various learning algorithms can be used to map correlations between input data and output data. These correlations form the mathematical model that can be used to make future predictions on new input data. Such a predictive model gray include one or more logistic regression models, Bayesian models, or neural networks (NNs).

A “deep neural network,” in the context of deep learning, may refer to an NN that utilizes a plurality of hidden layers between input and output layers, which may allow for learning based on features not explicitly defined in raw data. As used in the present application, a feature identified as an NN refers to a deep neural network.

FIG. 1shows system100for performing machine learning model based embedding for adaptable content evaluation, according to one exemplary implementation. As shown inFIG. 1, system100includes computing platform102having processing hardware104and system memory106implemented as a computer-readable non-transitory storage medium. According to the present exemplary implementation, system memory106stores software code110, one or more ML models120(hereinafter “ML model(s)120”), and content and classification database112storing category assignments determined by ML model(s)120.

As further shown inFIG. 1, system100is implemented within a use environment including communication network114providing network communication links116, training database122, user system130including display132, and user118of user system130. Also shown inFIG. 1are training data124, input data128, and content classification134for input data128, determined by system100. It is noted that although the exemplary use case for system100described below refers to system110as performing a classification of input data128, in some implementations system100may perform a regression on content100rather than a classification, as those processes are known to be distinguishable in the art.

Although the present application refers to one or more of software code110, ML model(s)120, and content and classification database112as being stored in system memory106for conceptual clarity, more generally, system memory106may take the form of any computer-readable non-transitory storage medium. The expression “computer-readable non-transitory storage medium,” as defined in the present application, refers to any medium, excluding a carrier wave or other transitory signal that provides instructions to processing hardware104of computing platform102. Thus, a computer-readable non-transitory storage medium may correspond to various types of media, such as volatile media and non-volatile media, for example. Volatile media may include dynamic memory, such as dynamic random access memory (dynamic RAM) while non-volatile memory may include optical, magnetic, or electrostatic storage devices. Common forms of computer-readable non-transitory storage media include, for example, optical discs, RAM, programmable read-only memory (PROM), erasable PROM (EPROM), and FLASH memory.

Moreover, althoughFIG. 1depicts software code110, ML model(s)120, and content and classification database112as being co-located in system memory106, that representation is also merely provided as an aid to conceptual clarity. More generally, system100may include one or more computing platforms102, such as computer servers for example, which may be co-located, or may form an interactively linked but distributed system, such as a cloud-based system, for instance. As a result, processing hardware104and system memory106may correspond to distributed processor and memory resources within system100. Consequently, in some implementations, one or more of software code110, ML model(s)120, and content and classification database112may be stored remotely from one another on the distributed memory resources of system100. It is also noted that, in sonic implementations, ML model(s)120may take the form of one or more software modules included in software code110.

Furthermore. althoughFIG. 1shows training database122to be remote from system100, that representation is also merely by way of example. In some implementations, training database122may be included as a feature of system100and may be stored in system memory106.

Processing hardware104may include a plurality of hardware processing units, such as one or more central processing units, one or more graphics processing units, and one or more tensor processing units one or more field-programmable gate arrays (FPGAs), and an application programming interface (API) server, for example. By way of definition, as used in the present application, the terms “central processing unit” (CPU), “graphics processing unit” (GPU), and “tensor processing unit” (TPU) have their customary meaning in the art. That is to say, a CPU includes an Arithmetic Logic Unit (ALU) for carrying out the arithmetic and logical operations of computing platform102, as well as a Control Unit (CU) for retrieving programs, such as software code110, from system memory106, while a GPU may be implemented to reduce the processing overhead of the CPU by performing computationally intensive graphics or other processing tasks. A TPU is an application-specific integrated circuit (ASIC) configured specifically for artificial intelligence (AI) processes such as machine learning.

In some implementations, computing platform102may correspond to one or more web servers, accessible over communication network114in the form of a packet-switched network such as the Internet, for example. Moreover, in some implementations, communication network114may be a high-speed network suitable for high performance computing (UPC), for example a 10 GigE network or an Infiniband network. In some implementations, computing platform102may correspond to one or more computer servers supporting a private wide area network (WAN), local area network (LAN), or included in another type of limited distribution or private network. As yet another alternative, in some implementations, system100may be implemented virtually, such as in a data center. For example, in some implementations, system100may be implemented in software, or as virtual machines.

Although user system130is shown as a desktop computer inFIG. 1, that representation is provided merely as an example as well. More generally, user system130may be any suitable mobile or stationary computing device or system that includes display132and implements data processing capabilities sufficient to provide a user interface, support connections to communication network114, and implement the functionality ascribed to user system130herein. For example, in other implementations, user system130may take the form of a laptop computer, tablet computer, or smartphone, for example.

With respect to display132of user system130, display132may be implemented as a liquid crystal display (LCD), light-emitting diode (LED) display, organic light-emitting diode (OLED) display, quantum dot (QD) display, or any other suitable display screen that perform. a physical transformation of signals to light. Furthermore, display132may be physically integrated with user system130or may be communicatively coupled to but physically separate from user system130. For example, where user system130is implemented as a smartphone, laptop computer, or tablet computer, display132will typically be integrated with user system130. By contrast, where user system130is implemented as a desktop computer, display132may take the form of a monitor separate from user system130in the form of a computer tower.

Input data128, as well as training data124, may include segmented content in the form of video snippets (e.g., sampling of frames), including raw frames, encoded frames, or both. In addition, in some implementations, input data128, training data124, or both, may be augmented with additional data, such as one or more of encoding statistics, distortion maps or metrics, pre-computed features such as per-pixel noise or texture information, for example, or any combination thereof. Thus, in various implementations, input data128and training content124may be 3D (e.g., in the case of video), 2D (e.g., in case of individual frames), 1D (e.g., in case of per-frame values), or even single variable for a segment.

In the case of AV or video content, input data128and training data124may include content segmented by shot, scene, timecode interval, or as individual video frames. Regarding the term “shot,” as defined for the purposes of the present application, a “shot” refers to a continuous series of video frames that are captured from a u C camera perspective without cuts and other cinematic transitions, while a scene typically includes a plurality of shots. Alternatively, input data128and training data124may include content segmented using the techniques described in U.S. Patent Application Publication Number 2021/0076045, published on Mar. 11, 2021, and titled “Content Adaptive Boundary Placement for Distributed Encodes,” which is hereby incorporated fully by reference into the present application. It is noted that, in various implementations, input data128and training data124may include video content without audio, audio content without video, AV content, text, or content having any other format.

ML model(s)120include an ML model based embedder trained using training data124that is selected based on one or more similarity metrics. Such similarity metrics may be metrics that are quantitatively similar, i.e., objectively similar, or may be metrics that are perceptually similar, i.e., subjectively similar under human inspection. Examples of perceptual similarity metrics for AV and video content may include texture, motion, and perceived encoding quality, to name a few. Examples of quantitative similarity metrics for AV and video content may include rate distortion curves, pixel density, computed optical flow, and computed encoding quality, also to name merely a few.

Referring toFIGS. 2A and 2B,FIG. 2Ashows diagram200A illustrating an exemplary contrastive learning process for training ML model based embedder226included among ML model(s)120, according to one implementation, whileFIG. 2Bshows diagram200B illustrating an exemplary contrastive learning process for training ML model based embedder226, according to another implementation.

As shown inFIG. 2A, in addition to ML model based embedder226, diagram200A includes training content segment224a,training content segment224bhaving a similarity metric with a value comparable to the value of the same similarity metric for training content segment224a,and training content segment224dhaving a similarity metric with a value that is different than the value of the same similarity metric for training content segment224a.In addition,FIG. 2Ashows embedding240aof training content segment224a,embedding2406of training content segment2246, and embedding240dof training content segment224d.Also shown inFIG. 2Ais distance function242comparing embeddings240aand240b,and comparing embeddings240aand240d.It is noted that training content. segments224a,224b,and224dcorrespond in general to training data124, inFIG. 1.

As show inFIG. 2A, ML model based embedder226is trained using a contrastive learning process to identify training content segments224aand224bas similar while minimizing distance function242. As further shown inFIG. 2A, ML model based embedder226is also trained using the contrastive learning process to identify training content segments224aand224das dissimilar bile maximizing distance function242.

As shown inFIG. 2B, in addition to ML model based embedder226, diagram200B includes training content segment224e,training content segment224fhaving a similarity metric with a value comparable to the value of the same similarity metric for training content segment224e,training content segment224ghaving a similarity metric different than the values of the same similarity metric for training content segments224eand224f,and training content segment224hhaving a similarity metric with a value that is comparable to the value of the sate similarity metric for training content segment224gbut different than the values of the same similarity metric for training content segments224eand224f.In addition,FIG. 2Bshows embedding240eof training content segment224e,embedding240fof training content segment224f,embedding240gof training content segment224g, and embedding240hof training content segment224h.Also shown inFIG. 2Bis classification or regression block260configured to perform one of a classification or a regression, respectively, of embeddings240e,740f,240g,and240hreceived from ML model based embedder226. It is noted that training content segments224e,224f,224g,and224hcorrespond in general to training data124, inFIG. 1.

ML model based embedder226is responsible for mapping content segments to embeddings. Example implementations for ML model based embedder226include but are not restricted to one or more of a 1D, 2D, or 3D convolutional neural network (CNN) with early or late fusion, that is/are trained from scratch or is/are pre-trained to leverage transfer learning. Depending on the target task, features extracted from different layers of a pre-trained CNN, such as the last layer of a Visual Geometry Group (VGG) CNN or Residual Network (ResNet) might be used to shape the embedding.

Classification or regression block260is responsible for performing classification or regression tasks, such as selection of pre-processing or post-processing algorithms or parameters, rate-distortion prediction where distortion can be Measured with various quality metrics, per title or per segment automatic encoding parameter selection, and per title or per segment automatic bitrate ladder selection for adaptive streaming prediction of the highest bitrate a given title or segment needs to be encoded in to reach a certain perceptual quality, to name a few examples.

Classification or regression block260can be implemented as, for instance a similarity metric plus threshold to determine what cluster of embeddings a particular embedding belongs to among different cluster groups available in a continuous vector space into which the embeddings are mapped. Alternatively, classification or regression block260can be implemented as a neural network (NN) or other ML model architecture included among ML model(s)120that is trained to classify or regress the embedding to the ground truth result. Moreover, in some implementations, classification or regression block260may be integrated with ML, model based embedder226, and may serve as one or more layers of ML model based embedder226, for example.

The contrastive learning processes depicted inFIGS. 2A and 2Bmay be repeated on a corpus of similar and dissimilar training content segment pairs. In the use case in which the content being categorized is AV or video content, for example, similar training content segments may both be live action segments or hand-painted animations, while each of dissimilar training content segments may be a different one of those two video content types. In the specific use case of video encoding, content segments can be labeled as similar for training purposes based on both having performed well (according to a performance threshold) on a particular encoding schema, regardless of whether one segment includes live action content and the other includes animation. In that use case, training content segments may be labeled dissimilar if they perform well on different encoding schemas even though they share the same content type, e.g., live action or animation.

The training of ML model based embedder226, classification or regression block260, or both ML model based embedder226and classification or regression block260, may be performed by software code110, executed by processing hardware104of computing platform102. In some use cases, ML model based embedder226and classification or regression block260may be trained independently of one another.

Alternatively, in some implementations, ML model based embedder226may be trained first, and then the embeddings provided by ML model based embedder226may be used for different downstream classification or regression tasks. In such a case, ML model based embedder226may be trained by a) identifying content segments that are deemed similar, and feed them in as training data while minimizing a distance function between them, b) identifying two content segments that are deemed dissimilar, and feed them in as training while maximizing a distance function between them, and c) repeating steps a) and b) for the length of the training.

As yet another alternative in some implementations ML based embedder226and classification or regression block260may be trained together. Moreover, in some implementations in which ML model based embedder226and classification or regression block260are trained together, such as where classification or regression block260is integrated with ML model based embedder226as one or more layers of ML, model based embedder226, for example, ML model based embedder226including classification or regression block260may be trained using end-to-end learning.

After training of ML model based embedder226, classification or regression block228, or both ML model based embedder226and classification or regression block260, is complete, processing hardware104may execute software code111to receive input data128from user system130, and to use ML model based embedder226to transform input data128or segments thereof to a vector representation (hereinafter “embedding”) of the content mapped into a continuous one dimensional or multi-dimensional vector space, resulting in an embedded vector representation of the content in that vector space.

FIG. 3Ashows exemplary 2D subspace300of continuous multi-dimensional vector space350, according to one implementation. It is noted that continuous multi-dimensional vector space350may be a relatively low dimension space, such as a sixty-four (64), one hundred and twenty-eight (128), or two hundred and fifty-six (256) dimension space, for example. Alternatively, in some implementations, continuous multi-dimensional vector space350may be a relatively high dimension space having tens of thousands of dimensions, such as twenty thousand (20k) dimensions, for example. Also shown inFIG. 3Aare mapped embeddings352a,352b,352c,352d,552e,352f,352g,352h,352i,and352j(hereinafter “embeddings352a-352j”) in continuous multi-dimensional vector space350and each corresponding to a different sample of content included in input data128, inFIG. 1.

In addition to using ML model based embedder226to map embeddings352a-352jonto continuous multi-dimensional vector space350, software code110, when executed by processing hardware104, may further perform an unsupervised clustering process to identify clusters each corresponding respectively to a different content category with respect to the similarity metric being used to compare content.FIG. 3Bshows subspace300of continuous multi-dimensional vector space350including embeddings352a-352j.FIG. 3Balso shows distinct clusters354a,354b,354c,and354d(hereinafter “clusters354a-354d”), each of which identifies a different category of content with respect to a particular similarity metric.

In the case of AV and video content, for example embeddings352aand352bof live action content are mapped to a region of multi-dimensional vector space350identified by cluster354c,while embeddings352c,352d,352f,and352hof low complexity animation content such as hand-drawn and other two-dimensional (2D) animation are mapped to different regions of continuous multi-dimensional vector space350identified by clusters354aand354d.Embeddings352e,352i,and352jof high complexity animation content such as 3D animation is shown to be mapped to yet a different region of continuous multi-dimensional vector space350identified by cluster354b.It is noted that embedding352gof a mixed content type, such as animation mixed with live action, for example, may be mapped to the border of either an animation cluster, a live action cluster, or between such clusters.

In use cases in which the content corresponding to embeddings352a-352jis AV or video content and the process for which that content is being categorized is video encoding, for example, each of clusters354a-354dmay correspond to a different codec. For instance cluster354cmay identify content for which a high bit-rate codec is required, while cluster354amay identify content for which a low bit-rate codec is sufficient. Clusters354band354dmay identify content with other specific codecs. In one such implementation in which a new codec is introduced or an existing codec is retired, system100may be configured to automatically re-evaluate embeddings352a-352jrelative to the changed group of available codecs. An analogous re-evaluation may be performed for any other process to which the present concepts are applied.

It is noted that the continuity of multi-dimensional vector space350advantageously enables adjustment of the way in which embedding corresponding to content are clustered into categories per individual use case through utilization of different clustering algorithms and thresholds. In contrast to conventional classifications methods, which depend on a priori knowledge of the number of classification labels to be trained for, the present novel and inventive embedding approach can be adaptably used for a plurality of classification schemes. Moreover, due to the unsupervised nature of the clustering performed as part of the present adaptable content evaluation solution, the approach disclosed in the present application can yield unanticipated insights into similarities among items of content that may appear superficially to be different.

The functionality of system100will be further described by reference toFIG. 4.FIG. 4shows flowchart470presenting an exemplary method for performing ML model based embedding for adaptable content evaluation, according to one implementation. With respect to the method outlined inFIG. 4, it is noted that certain details and features have been left out of flowchart470in order not to obscure the discussion of the inventive features in the present application.

Referring now toFIG. 4in combination withFIG. 1, flowchart470includes receiving input data128including a plurality of content segments (action471). As shown inFIG. 1, input data128may be received by system100from user system130, via communication network114and network communication links116. As discussed above, input data128may include video content without audio, where the video content includes raw frames, encoded frames, or both. Alternatively, input data128may include audio content without video, AV content, text, or content having any other format. As further discussed above, in some implementations, input data128may also include one or more of encoding statistics, distortion maps or metrics, pre-computed features such as per-pixel noise or texture information, or any combination thereof. Input data128may be received in action471by software code110, executed by processing hardware104of computing platform102.

Referring toFIGS. 2A, 2B, and 3Ain combination withFIGS. 1 and 4, flowchart470further includes mapping, using an ML model based embedder included among ML model(s)120, each of the plurality of content segments received in action471to a respective embedding in continuous vector space350to provide a plurality of mapped embeddings (e.g., embeddings352a-352j) corresponding respective to the plurality of content segments (action472). The mapping of the plurality of content segments received in action471to embeddings in continuous vector space350may be performed in action472by software code110, executed by processing hardware104of computing platform102, and using ML model based embedder226.

As discussed above, ML model based embedder226may be trained using contrastive learning based on one or more similarity metrics. As also discussed above, the one or more similarity metrics on which contrastive learning by ML model based embedder226may include a quantitative similarity metric, a perceptual similarity metric, or both. In some implementations, as noted above, ML model based embedder226may include one or more of a ID CNN, a 2D CNN, or a 3D CNN, for example, with early or late fusion. Furthermore, with respect to continuous vector space350, it is noted that in some implementations continuous vector space350may be multi-dimensional, as shown inFIG. 3A, as well as inFIG. 3B.

Flowchart470further includes performing one of a classification or a regression of the content segments using mapped embeddings (e.g., embeddings352a-352j) (action473). In some implementations, as described above by reference toFIG. 313, a classification performed in action473may include grouping each of at least one of mapped embeddings embeddings352a-352j) into one or more clusters (e.g., clusters354a-354d), each corresponding respectively to a distinct category of the similarity metric upon which the contrastive learning by ML model based embedder226is based. As further described above, such clustering may be performed as an unsupervised process.

As also discussed above, in some implementations, the classification or regression performed in action473may be performed using classification or regression block260, which may take the form of a trained neural network (NN) or other ML model architecture included among ML model(s)120. Moreover and as further discussed above, in some implementations, classification or regression block260may be integrated with ML model based embedder226, and may serve as one or more layers of ML model based embedder226, for example. Action473may be performed by software code110, executed by processing hardware104of computing platform102, and in some implementations, using classification or regression block260.

Flowchart470further includes discovering, based on the one of the classification or the regression performed in action473, at least one new label for characterizing the plurality of content segments received in action471(action474). For example, and as described above by reference toFIG. 3B, due to the unsupervised nature of the clustering performed as part of the present adaptable content evaluation solution, the method outlined by flowchart470can yield unanticipated insights into similarities among items of content that may appear superficially to be different. The at least one new label discovered in action474may be stored in content and classification database112. in addition, or alternatively, that at least label discovered in action474may be transferred to training database122for storage and inclusion in training data124.

For example, the at least one new label discovered in action474may advantageously result in implementation of new, and more effective, AV or video encoding parameters. In addition, or alternatively, information discovered as part of action474may be used to selectively enable one or more presently unused encoding parameters, as well as to selectively disable one or more encoding parameters presently in use. As yet another example, the information discovered as part of action474may be used to selectively turn-on or turn-off certain preprocessing elements in the transcoding pipeline, such as denoising or debanding, for example, based on content characteristics. Action474may be performed by software code110, executed by processing hardware104of computing platform102.

In some implementations, the method outlined by flowchart470may conclude with action474described above. However, in other implementations, the method outlined by flowchart470may also include further training ML model based embedder226using contrastive learning and the at least one new label discovered in action474(action475). That is to say, action475is optional. When included in the method outlined by flowchart470, action475may be performed by software code110, executed by processing hardware104of computing platform102, and may advantageously result in refining and improving the future. classification or regression performance of system100. With respect to the actions included in flowchart470, it is noted that actions471,472,473, and474(hereinafter “actions471-474”), or actions471-474and475, may be performed as an automated process from which human involvement may be omitted.

Thus, the present application discloses systems and methods for performing ML model based embedding for adaptable content evaluation. The solution disclosed in the present application advances the state-of-the-art by addressing and providing solutions for situations in which a downstream task lacks annotated clear labeling, but it is desirable to discover how and why different things are responding differently to this task. The novel and inventive concepts disclosed in the present application can be used to automate the process of discovering appropriate labels for these differences (auto-discovery), and to be able to train a model so that a prediction can be generated about which one of those different groupings something is going to fall into, given a particular set of parameters for that downstream task.

As described above, the present application discloses two approaches to addressing the problem of auto-discovery of labels that can build on each other. In the first approach, as described by reference toFIGS. 2A, 3A, and 3B, a contrastive learning approach is adopted. Embeddings are mapped to a continuous vector space, and then an unsupervised clustering analysis is performed to discover what different groupings there may be and what additional unanticipated groupings there may be. This first approach may be refined by adding more signals to the data used to create the continuous vector space to which the embeddings are mapped, to provide a more complex vector space that is sensitive to more factors and that will do more different kinds of groupings.

In the second approach, the first approach described above may be further supplemented by adding a classification or regression block for different types of downstream tasks, as described above by reference toFIG. 2B. In that implementation, the ML model based embedder and the classification or regression block, which can be implemented as a neural network (NN), may be trained together using end-to-end learning while preserving the auto-discovery of labels that is so advantageously enabled by the present systems and methods.