Patent ID: 12242538

The diagrams depicted herein are illustrative. There can be many variations to the diagram or the operations described therein without departing from the spirit of the invention. For instance, the actions can be performed in a differing order or actions can be added, deleted or modified.

In the accompanying figures and following detailed description of the described embodiments of the invention, the various elements illustrated in the figures are provided with two or three-digit reference numbers. With minor exceptions, the leftmost digit(s) of each reference number corresponds to the figure in which its element is first illustrated.

DETAILED DESCRIPTION

Designing a document (or other written and/or visual project) involves making intentional decisions about the visual elements of the document to ensure that it is visually appealing, easy to read, and effectively communicates its message. One important aspect of document design is font selection. Fonts, or typefaces, are sets of characters with a consistent design that can be used to display text. Font selection is important for a wide range of products, particularly those involving written communication and/or visual design, such as, for example, in printed materials (e.g., books, magazines, brochures, flyers, posters, etc.), websites (e.g., web design), product packaging, logos, presentations, and advertisements.

The right font can make these materials more visually appealing and easier to read. For example, a font can help to create a consistent visual style, making it easier for users to read and navigate a website. A font can help to communicate the brand identity and intended message of a product. Similarly, fonts can help to communicate the personality and tone of a brand. In short, by selecting the right fonts, designers can enhance the overall effectiveness and appeal of a product as well as the readability of a document.

When choosing a font, it is important to consider the purpose and tone of the product, as well as the intended audience. For example, a formal document may require a more traditional font, while a more creative or casual document could benefit from a more playful or unique font. Unfortunately, users currently follow a somewhat tedious approach for font selection, typically searching through an exhaustive list of available fonts and trying those fonts, one by one, until ultimately selecting a font(s) for the document, design, project, etc.

This disclosure involves the use of large language models and compact models for font selection. As described previously, large language models have been adapted across a number of domains, including question answering, also referred to as query-and-response. Large language models are trained over relatively huge datasets (millions of words) and can include a large number of internal features (e.g., 12 hidden layers, 24 hidden layers, etc.). While extremely powerful, these large language models suffer from native latency constraints and are typically domain-agnostic.

Aspects of the present disclosure introduce a way to leverage a pre-trained large language model to build a compact model to provide a theme based font search, bypassing the latency limitations inherent to the large language model and ensuring domain-relevant answers. In some embodiments, a pre-trained large language model is fed domain-specific data including scenario-specific sentences. In some embodiments, the scenario-specific sentences are extracted from domain-specific product templates, such as, for example, existing product designs, documents/documentation, product brochures, flyers, webpages, packaging, logos, presentations, advertisements, etc. (collectively, “documents” and/or “content items”). In some embodiments, the large language model outputs an embedding for each sentence. Pairs of sentence embeddings are selected and scored (soft labeled).

In some embodiments, the soft labels are used to build a compact model using contrastive learning. Notably, the compact model is of lower architectural complexity than the large language model. In essence, contrastive learning involves training the compact model to match the soft labels of the sentence pair embeddings derived using the large language model. In this manner, a compact model can be built to efficiently provide domain-relevant answers (e.g., font category recommendations) without the overhead (latency limitations, model complexity, etc.) of the large language model.

Advantageously, compact models built for a theme based font search can provide users with a seamless, natural language based font selection process without the latency limitations inherent to large language models. Moreover, providing a theme based font search as described herein enables users to efficiently search for any kind of font, even those they do not know to look for, based on the font name, font style, and/or merely an intent/theme of the font/product of interest.

Compact models trained for theme based font search in accordance with one or more embodiments offer several technical advantages over other font selection processes. As described previously, the compact models can provide relevant font suggestions to a user at latencies that are not achievable using large language models. Notably, compact models trained using contrastive learning in accordance with one or more embodiments need not be trained to understand how a particular font visually “looks”, such as the font's text height, style, etc.-instead the compact model can be trained directly on the soft labels and embeddings derived from the large language model-further improving latencies by greatly reducing model complexity. In addition, systems designed according to one or more embodiments offer notable improvements in user interaction, including miniaturization (e.g., less required space for full font selection GUI functionality), improved usability (moving beyond simple lists of available fonts), and improved user efficiency (e.g., system can populate font options in response to a user query even for fonts that are unknown to the user-reducing the time it takes a user to find and select a font in a straightforward way using natural language prompts).

FIG.1depicts a block diagram of a workflow100for leveraging a large language model102to build a compact model104for a theme based font search in accordance with one or more embodiments of the present invention. In some embodiments, the large language model102is a pretrained large language model. In some embodiments, the large language model102is pretrained to output an embedding for an input sentence. In some embodiments, the large language model102is pretrained prior to the workflow100. In some embodiments, the workflow100optionally includes pretraining106the large language model.

Overall, training a large language model involves significant computing resources and can take several days, weeks, or even longer to complete, depending on the size of the dataset(s) and the complexity of the model. Processes for training large language models are known and are not meant to be particularly limited. For example, training a transformer-based generative language model typically involves data preparation, model initialization, training, and model evaluation.

Data preparation involves sourcing and preprocessing a large corpus of text for training. In many cases, the corpus is preprocessed by tokenizing the text into individual words or subwords and mapping them to numerical embeddings, although other preprocessing techniques are possible. The model parameters can be initialized randomly or using pre-trained weights taken, for example, from a similar and/or prior model. The model can be trained using one or more of a variety of techniques, such as, for example, by a stochastic gradient descent (SGD) approach in combination with the use of an objective function. The objective function can include, for example, a so-called maximum likelihood estimation (MLE), which involves maximizing the probability of generating a next word in a sequence given the previous words, although other objective functions are possible. A trained model can be validated using a validation data set to monitor the model's performance and to prevent overfitting. Model validation can include some combination of a variety of metrics, such as perplexity and accuracy, to measure the quality of the generated text.

In some embodiments, domain-specific training data108is input to the pretrained large language model102to generate domain-specific embeddings. In some embodiments, the domain-specific training data108includes a library of existing product designs (e.g., documents, product brochures, flyers, webpages, packaging, logos, presentations, adverts, etc.), also referred to herein as templates. In some embodiments, the library (or template library) includes a large number of such product designs (e.g., thousands, tens of thousands, hundreds of thousands, or even larger data sets). For example, a template library might contain 40,000 to 50,000 product designs. The size of the domain-specific training data108is not meant to be particularly limited and template libraries having any number of product designs are within the contemplated scope of this disclosure.

In some embodiments, the product designs are sourced from subject matter experts. In some embodiments, the product designs are scrapped from publicly and/or privately available data sources, such as, for example, the Internet. In some embodiments, the product designs are sourced from a combination of subject matter expert creations as well as public and private data sources.

In some embodiments, each design includes some combination of written and visual data. Written data can include, for example, a title, a subtitle, and text. Visual data can include, for example, graphics and colors. In some embodiments, the domain-specific training data108is built by scrapping the templates for the written data, resulting in large “sentences” that include, for example, the title, subtitle, and text of a respective template. In some embodiments, these sentences (e.g., thousands, tens of thousands, or more) are fed to the pretrained large language model102to generate the domain-specific embeddings (e.g., thousands, tens of thousands, or more).

In some embodiments, the workflow100includes generating a labeled dataset110. In some embodiments, the labeled dataset110includes scored sentence pairs. In some embodiments, sentence pairs (e.g., S1, S2) are selected from the domain-specific embeddings of the sentences and scored (e.g., soft labeled). In some embodiments, the sentence pairs are selected randomly. In some embodiments, the sentence pairs are selected manually, for example, by a subject matter expert labeler(s). In some embodiments, all possible sentence pairs are selected. In some embodiments, only a subset of all possible sentence pairs are selected.

In some embodiments, each selected sentence pair is scored (or labeled) as one of a positive sentence pair (e.g., “1”) or a negative sentence pair (e.g., “0”). For example, a first sentence S1and a second sentence S2can be scored 0 or 1. In other words, soft labels having a value of 0 or 1 can be generated for each of the selected pairs of sentences.

In some embodiments, a positive score denotes a passing similarity threshold between the sentences in the sentence pair, whereas a negative score denotes a failing similarity threshold between the sentences (i.e., that the sentences are dissimilar). As used herein, “similarity” means that the two sentences share a same contextual theme. For example, a first sentence encoding a birthday card and a second sentence encoding a birthday party invitation share a same contextual theme—birthdays—and will be scored positively. Conversely, a first sentence encoding a birthday card and a second sentence encoding a construction notice do not share a same contextual theme and will be scored negatively. Observe that, in many contexts, whether two sentences are similar (and to what degree) includes a measure of subjectivity. In some embodiments, a sentence pair is scored manually, for example, by a subject matter expert labeler(s) using domain-specific knowledge, personal knowledge, and/or general knowledge.

In some embodiments, a 3-tuple <S1, S2, SCORE> is generated for each selected sentence pair, where “SCORE” denotes the soft label value. In some embodiments, the generated 3-tuples for each selected sentence pair define the labeled dataset110.

In some embodiments, the labeled dataset110is used, at block112, to build (sometimes referred to as fine-tune) the compact model104. As described previously, the compact model104is of lower architectural (model) complexity than the pretrained large language model102. For example, while the pretrained large language model102might be a large transformer-based generative model having 78 transformer layers, a hidden size of 4256, and 28 attention heads, the compact model104might be a neural network having only 2 hidden layers. It should be understood that the provided examples for the pretrained large language model102and the compact model104are provided only for ease of discussion and to illustrate the difference in complexity between the respective models. The exact configuration (e.g., transformer, RNN, etc.) and complexity (e.g., number of layers) of the pretrained large language model102and the compact model104are not meant to be particularly limited, and all such configurations of a first, relatively complex model and a second, relatively less complex model are within the contemplated scope of this disclosure.

In some embodiments, the compact model104is built using less than 25 percent the number of layers of the pretrained large language model102(e.g., for a large transformer-based generative model having 78 transformer layers the compact model104will include 19 or fewer hidden layers). In some embodiments, the compact model104is built using less than 10 percent the number of layers of the pretrained large language model102(e.g., for a large transformer-based generative model having 78 transformer layers the compact model104will include 7 or fewer hidden layers). In some embodiments, the compact model104is built using less than 5 percent the number of layers of the pretrained large language model102(e.g., for a large transformer-based generative model having 78 transformer layers the compact model104will include 3 or fewer hidden layers). In some embodiments, the compact model104is built using less than 3 percent the number of layers of the pretrained large language model102(e.g., for a large transformer-based generative model having 78 transformer layers the compact model104will include 1 or 2 hidden layers).

In some embodiments, the generated 3-tuples (sentence pairs and respective scores) are used to build the compact model104using a contrastive learning approach. In essence, contrastive learning involves training the compact model104to match the soft labels (scores) of the sentence pair embeddings derived using the pretrained large language model102. The contrastive learning approach will be illustrated by way of an example compact model104: a two layer Turing Natural Language Representation (T-NLR) model, as shown inFIG.2.

As shown inFIG.2, the compact model104includes one or more input nodes202, an output node204, and a plurality of hidden nodes206distributed amongst one or more hidden layers208. In some embodiments, the compact model104includes three input nodes202, one output node204, and two hidden layers208, each hidden layer208having four hidden nodes206(as shown). In some embodiments, the compact model104includes three input nodes202encoding, respectively, a first sentence S1, a second sentence S2, and a score SN, corresponding, for example, to an input 3-tuple configured as described herein. Again, other configurations are possible, and all such configurations are within the contemplated scope of this disclosure.

In some embodiments, the compact model104is initialized by setting, manually or randomly, the weights of the various hidden nodes206of the hidden layers208to some initial value(s). In some embodiments, the weights of the various hidden nodes206are initialized using pre-trained weights taken, for example, from a similar and/or prior model. In some embodiments, the weights of the various hidden nodes206are initialized to the same value, or alternatively, to different values.

In some embodiments, the compact model104is trained (i.e., the weights of the hidden nodes206are determined) by successively inputting a plurality of 3-tuples into the input nodes202and, for each 3-tuple, adjusting one or more weights of the hidden nodes206until a value of the output node204matches the known score of the respective 3-tuple. In short, the known scores for each 3-tuple serve as a ground truth against which the internal weights of the compact model104can be trained.

To illustrate, consider a scenario where, prior to training completion, the compact model104, responsive to an input first sentence S1and a second sentence S2, indicates that the sentences are not similar, when it is known (via the respective score SN) that the sentences are in fact similar. Continuing with this scenario, one or more (possibly all) weights of the hidden nodes206can be adjusted until the output node204agrees with the ground truth (i.e., that the sentences are similar). For example, the weights of the hidden nodes206can be adjusted until the output node204has a value of “1”, or a value within a predetermined threshold of “1” (i.e., a value sufficient to indicate that the sentences are similar).

The compact model104can be trained in this manner using one or more of a variety of techniques, such as, for example, by a stochastic gradient descent (SGD) approach in combination with the use of an objective function over a set of 3-tuples to achieve a desired level of accuracy. That is, the compact model104can be trained on an arbitrarily large set of training data (3-tuples) until agreement with the known ground truth of an input 3-tuple reaches a predetermined degree of accuracy. The particular degree of accuracy enforced during training is not meant to be particularly limited, but may include, for example, a prediction accuracy of at least 50 percent, at least 75 percent, at least 90 percent, at least 95 percent, at least 98 percent, at least 99 percent, etc.

In some embodiments, the compact model104, once fine-tuned on the soft labels (i.e., trained on sentence pairs and known scores using contrastive learning as described previously), is leveraged to provide domain-relevant answers (e.g., font category recommendations) without the overhead (latency limitations, model complexity, etc.) of the pretrained large language model102. In other words, the compact model104can be leveraged to provide a theme based font search.

FIG.3depicts a block diagram of a workflow300for leveraging the compact model104for a theme based font search in accordance with one or more embodiments of the present invention. In some embodiments, workflow300includes an offline component302and an online component304. As used in this context, the term “offline” does not necessarily mean off-network (e.g., disconnected from the Internet) but rather that this portion of the workflow300occurs separately (and prior) to any user interactions. Conversely, as used in this context, the term “online” does not necessarily mean on-network (e.g., connected to the Internet) but rather that this portion of the workflow300occurs actively (and responsively) to one or more user interactions (e.g., the inputting of a search query as described in greater detail below).

In some embodiments, the offline component302includes inputting one or more font categories, sometimes referred to as font tags, to the compact model104. In some embodiments, each font tag is a single word token. For example, a font tag can include “holiday”, “anniversary”, “business”, “education”, “cartoon”, “comic”, “fun”, “wedding”, “spooky”, “sports”, “school”, “friendly”, “sprightly”, “serious”, etc. The particular font tags employed are not meant to be particularly limited and are provided only to illustrate the variety of possible font tags. In short, a font tag is meant to be a simple token (often only one or two words) describing an aspect(s) of a font.

In some embodiment, one of more font tags are created for each of a plurality of available fonts. In some embodiments, each font is “tagged” with one or more font tags manually by a human labeler. In some embodiments, each font is tagged with one or more font tags by a subject matter expert.

To illustrate, consider the font “Comic SANS MS”. Comic SANS MS has a distinctive, informal appearance that is intended to resemble handwriting. The letters are rounded and have uneven edges, which gives the typeface a playful, casual feel. Comic SANS MS is often used to convey a lighthearted and/or humorous tone and is widely used in informal settings where a less formal and more approachable, fun tone is desired. For example, Comic SANS MS is the most popular font for Halloween invitations. A labeler might tag Comic SANS MS with the following font tags: “Greeting card”, “Invitation”, “Children's Materials”, “Spooky”, and “Halloween”. Other tags are certainly possible.

In some embodiments, the compact model104generates an output, referred to herein as a font tag embedding, for each input font tag. For example, inputting the font tag “Halloween” into the compact model104will result in an output—this output can be stored as the font tag embedding of the font tag “Halloween”. In this manner, a data set of font tag embeddings306can be generated for any number of desired font tags. Observe that a particular font tag embedding for a respective font tag need only be generated once during the offline component302of the workflow300.

In some embodiments, the compact model104and the font tag embeddings306are stored in a data warehouse308. In some embodiments, each of the font tag embeddings306are stored as a 2-tuple (or other mapping data structure) with their respective input token (font tag). For example, the font tag embedding for “Beach” can be stored alongside the token “Beach” itself in the data warehouse308. The scale and configuration of the data warehouse308is not meant to be particularly limited. In some embodiments, the data warehouse308is accessed during the online component304of the workflow300.

In some embodiments, the online component304includes a user interface310having a search box312(also referred to as a search bar). In some embodiments, the user interface310is presented to a user during a session (i.e., a user interaction with the user interface310). In some embodiments, the search box312is configured such that a user can input (e.g., by typing, dictation, or otherwise) a font recommendation query (referred to herein as search query314) into the search box312. In some embodiments, the search query314includes a text string. For example, the search query314might include the text string, “Halloween” (as shown). The search query314need not be a single word string. For example, the search query314might include the text string, “A scary invitation with bats and ghosts” (not separately shown).

In some embodiments, the search query314is passed to a Designer Application Service (DAS)316. In some embodiments, the DAS316is configured to parse and/or otherwise analyze the search query314to determine a type of data structure contained in the search query314. In some embodiments, the DAS316determines that the search query314contains text data. In some embodiments, the DAS316is configured to pass the search query314to a Text Analysis Service (TAS)318responsive to the search query314containing text data.

In some embodiments, the TAS318is configured to parse and/or otherwise analyze the search query314to determine that the search query314includes a font recommendation query. In some embodiments, the TAS318is configured to pass the search query314to the compact model104responsive to the search query314containing a font recommendation query. In some embodiments, the TAS318is configured to pass the search query314to the data warehouse308responsive to the search query314containing a font recommendation query and one or more internal components (not separately shown) of the data warehouse308are configured to route the search query314to the compact model104.

In some embodiments, the compact model104generates, as output, a query embedding320responsive to receiving, as input, the search query314. The query embedding320can be generated in substantially the same manner as the font tag embeddings306described previously herein.

In some embodiments, the query embedding320and the font tag embeddings306are passed to a matching module322. The matching module322can be stored within, or separately from, the data warehouse308. In some embodiments, the matching module322is configured to complete a one-to-many similarity mapping between the query embedding320and the font tag embeddings306. That is, the matching module322is configured to determine a similarity score between the query embedding320and each of the font tag embeddings306.

In some embodiments, the similarity score between the query embedding320and each respective embedding of the font tag embeddings320is quantified using a distance measure. A distance measure (also referred to as a distance metric) measures the degree of separation between two entities in a given space. In most constructions, a smaller distance means that the values are relatively similar to each other, while a larger distance implies they are relatively dissimilar.

The specific distance measure used to determine the similarity scores between the query embedding320and the font tag embeddings320is not meant to be particularly limited. A number of distance measures are known and all such distance measures are within the contemplated scope of this disclosure. For example, one type of distance measure used to quantify similarity between two values is the Cosine distance (or cosine similarity). The Cosine distance metric calculates the cosine of the angle between two vectors, which is a measure of their similarity. The Cosine distance ranges from 0 to 1, with 0 indicating that the vectors are completely dissimilar, while 1 means they are identical. Other distance measures include, for example, Euclidean distance, Manhattan distance, Minkowski distance, and Hamming distance.

In some embodiments, the matching module322is configured to select the top N most similar embeddings of the font tag embeddings320to the query embedding320. The value of N is not meant to be particularly limited and can be, for example, a predetermined percentage and/or number of the font tag embeddings320. For example, the top “3” most similar embeddings can be selected. In some embodiments, the value N is allowed to float to allow for any number of embeddings to be selected, subject to a predetermined threshold similarity score. For example, N can be allowed to dynamically float such that all embeddings having a similarity score of greater than or equal to 0.8, 0.9, 0.95, 0.97, 0.99, etc., are selected as the top N most similar embeddings. For example, the top “5” most similar embeddings can be selected in a scenario where 5 of the embeddings have a similarity score above 0.9.

In some embodiments, the matching module322is configured to fetch and/or otherwise determine the respective font category (font tag) for each of the top N most similar embeddings. As described previously, each of the font tag embeddings306can be stored as a 2-tuple with their respective input token (font tag). In some embodiments, the matching module322determines the N respective font tags by reading the font tag portion of the 2-tuples. In some embodiments, the matching module322passes each of the top N most similar embeddings back to the data warehouse308and the data warehouse308is configured to return the N respective font tags.

In some embodiments, the matching module322is configured to pass the top N font categories324(i.e., the N respective font tags) to the TAS318. In some embodiments, the TAS318is configured to pass the top N font categories324to the DAS316. In some embodiments, the DAS316is configured to pass the top N font categories324to the user interface310.

In some embodiments, the user interface310includes a font category suggestion frame326configured to populate (display) the top N font categories324. For example, the font category suggestion frame326might include the “Suggested” font categories “Spooky”, “Gothic”, and “Holiday” responsive to a search query314for “Halloween” (as shown). In some embodiments, one or more of the top N font categories324are visually emphasized relative to the other ones of the top N font categories324. For example, the suggestion for “Spooky” might be put in bold typeface (as shown). In other embodiments, one or more suggestions can be highlighted, underlined, italicized, and/or made of a larger font, etc., to distinguish the emphasized suggestion from the remaining suggestions. In some embodiments, the one or more visually emphasized suggestion(s) are the font tags of the most similar embeddings determined via the matching module322.

In some embodiments, the user interface310is configured to display an example text328depicting a font of one or more of the top N font categories324. In some embodiments, the user interface310is configured to display an example text328responsive to a selection by the user of one top N font categories324in the font category suggestion frame326. In some embodiments, the user interface310is configured to display an example text328responsive to an indication of interest from the user, such as, for example, on mouseover, of one the top N font categories324in the font category suggestion frame326.

FIG.4depicts a block diagram of a workflow400for suggesting one or more fonts in accordance with one or more embodiments of the present invention. As described previously herein, the user interface310can include a font category suggestion frame326for displaying the top N font categories324. In some embodiments, the font category suggestion frame326and/or the user interface310is configured to receive a user selection of a font category of the top N font categories324. For example, the user interface310can be configured to allow a user to select, via clicking or otherwise, a font category of the top N font categories324.

In some embodiments, the user interface310is configured to display a font suggestion frame402. In some embodiments, the user interface310is configured to display the font suggestion frame402responsive to a selection (via clicking or otherwise) of a font category of the top N font categories324in the font category suggestion frame326.

In some embodiments, the font suggestion frame402includes one or more suggested fonts identified via respective font labels404. For example, upon selection of the “Spooky” font category in the font category suggestion frame326, the font suggestion frame402might suggest the font labels: “Comic SANS MS”, “Blackadder ITC”, “Curlz MT”, “Gigi”, and “Snap ITC” (as shown). As described previously, in some embodiments, each font is “tagged” with one or more of the font tags manually by a human labeler. In some embodiments, the font suggestion frame402is configured to suggest, via the font labels404, one or more of the fonts having a font tag matching the selected font category.

In some embodiments, each of the font labels404is displayed in a stylized manner corresponding to their respective font typeface. For example, the font label “Comic SANS MS” can be displayed using the comic SANS MS font, while the font label “Blackadder ITC” can be displayed using the Blackadder ITC font. In this manner, a user can quickly evaluate the one or more suggested font labels404.

FIG.5illustrates aspects of an embodiment of a computer system500that can perform various aspects of embodiments described herein. In some embodiments, the computer system(s)500can implement and/or otherwise be incorporated within or in combination with any of the workflows100,300,400described previously herein. In some embodiments, a computer system500can be implemented client-side. For example, a computer system500can be configured to display and carry out the functionality of the user interface310. In some embodiments, a computer system500can be implemented server-side. For example, a computer system500can be configured to build the compact model104using a contrastive learning approach. In another server-side example, a computer system500can be configured to receive a search query314from a user (remote) system and to, responsive to receiving the search query314, provide the top N font categories324(including, e.g., some or all aspects of the DAS316, the TAS318, the matching module322, and/or the data warehouse308).

The computer system500includes at least one processing device502, which generally includes one or more processors or processing units for performing a variety of functions, such as, for example, completing any portion of the workflows100,300,400described previously herein. Components of the computer system500also include a system memory504, and a bus506that couples various system components including the system memory504to the processing device502. The system memory504may include a variety of computer system readable media. Such media can be any available media that is accessible by the processing device502, and includes both volatile and non-volatile media, and removable and non-removable media. For example, the system memory504includes a non-volatile memory508such as a hard drive, and may also include a volatile memory510, such as random access memory (RAM) and/or cache memory. The computer system500can further include other removable/non-removable, volatile/non-volatile computer system storage media.

The system memory504can include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out functions of the embodiments described herein. For example, the system memory504stores various program modules that generally carry out the functions and/or methodologies of embodiments described herein. A module or modules512,514may be included to perform functions related to the workflow100,300,400as described previously herein. The computer system500is not so limited, as other modules may be included depending on the desired functionality of the computer system500. As used herein, the term “module” refers to processing circuitry that may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or group) and memory that executes one or more software or firmware programs, a combinational logic circuit, and/or other suitable components that provide the described functionality.

The processing device502can also be configured to communicate with one or more external devices516such as, for example, a keyboard, a pointing device, and/or any devices (e.g., a network card, a modem, etc.) that enable the processing device502to communicate with one or more other computing devices. Communication with various devices can occur via Input/Output (I/O) interfaces518and520.

The processing device502may also communicate with one or more networks522such as a local area network (LAN), a general wide area network (WAN), a bus network and/or a public network (e.g., the Internet) via a network adapter524. In some embodiments, the network adapter524is or includes an optical network adaptor for communication over an optical network. It should be understood that although not shown, other hardware and/or software components may be used in conjunction with the computer system500. Examples include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, and data archival storage systems, etc.

Referring now toFIG.6, a flowchart600for a theme based font search method is generally shown according to an embodiment. The flowchart600is described with reference toFIGS.1to5and may include additional steps not depicted inFIG.6. Although depicted in a particular order, the blocks depicted inFIG.6can be, in some embodiments, rearranged, subdivided, and/or combined.

At block602, a font category is input to a compact model to generate a font tag embedding. The font category can include, for example, the token “family”.

At block604, a search query is input to the compact model to generate a query embedding. The search query can include, for example, the text string “a wedding invitation”.

At block606, a distance measure between the query embedding and the font tag embedding is determined. In some embodiments, the distance measure is defined according to a cosine distance between the respective embeddings.

At block608, the font category for the font tag embedding is provided as a suggested font category responsive to the distance measure satisfying a predetermined threshold. In some embodiments, the top N font categories (font tags) are provided as suggested font categories.

In some embodiments, the method includes receiving the search query in a user interface and, responsive to receiving the search query, displaying the suggested font category in the user interface.

In some embodiments, the method includes receiving a selection of the suggested font category within the user interface. In some embodiments, the method includes, responsive to receiving the selection of the suggested font category, providing one or more font recommendations for fonts labeled with the respective suggested font category.

In some embodiments, the method includes training the compact model from a large language model using contrastive learning. In some embodiments, the compact model includes a first model complexity having a first number of hidden layers and the large language model includes a second model complexity having a second number of layers larger than the first number of hidden layers.

In some embodiments, domain-specific training data including sentences extracted from design templates are input to the large language model to generate, for each extracted sentence, a domain-specific sentence embedding. In some embodiments, one or more pairs of sentence embeddings are labeled with a soft label. In some embodiments, each pair of sentence embeddings is labeled as one of a positive sentence pair and a negative sentence pair. In some embodiments, a positive soft label denotes a passing similarity threshold between the sentences in the respective pair of sentence embeddings and a negative soft label denotes a failing similarity threshold between the sentences in the respective pair of sentence embeddings.

In some embodiments, the compact model is trained to output scores that agree with the soft labels of the pairs of sentence embeddings derived using the large language model.

While the disclosure has been described with reference to various embodiments, it will be understood by those skilled in the art that changes may be made and equivalents may be substituted for elements thereof without departing from its scope. The various tasks and process steps described herein can be incorporated into a more comprehensive procedure or process having additional steps or functionality not described in detail herein. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the disclosure without departing from the essential scope thereof. Therefore, it is intended that the present disclosure not be limited to the particular embodiments disclosed, but will include all embodiments falling within the scope thereof.

Unless defined otherwise, technical and scientific terms used herein have the same meaning as is commonly understood by one of skill in the art to which this disclosure belongs.

Various embodiments of the invention are described herein with reference to the related drawings. The drawings depicted herein are illustrative. There can be many variations to the diagrams and/or the steps (or operations) described therein without departing from the spirit of the disclosure. For instance, the actions can be performed in a differing order or actions can be added, deleted or modified. All of these variations are considered a part of the present disclosure.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, element components, and/or groups thereof. The term “or” means “and/or” unless clearly indicated otherwise by context.

The terms “received from”, “receiving from”, “passed to”, “passing to”, etc. describe a communication path between two elements and does not imply a direct connection between the elements with no intervening elements/connections therebetween unless specified. A respective communication path can be a direct or indirect communication path.

The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed.

For the sake of brevity, conventional techniques related to making and using aspects of the invention may or may not be described in detail herein. In particular, various aspects of computing systems and specific computer programs to implement the various technical features described herein are well known. Accordingly, in the interest of brevity, many conventional implementation details are only mentioned briefly herein or are omitted entirely without providing the well-known system and/or process details.

The present invention may be a system, a method, and/or a computer program product at any possible technical detail level of integration. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.

Various embodiments are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.

The descriptions of the various embodiments described herein have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the form(s) disclosed. The embodiments were chosen and described in order to best explain the principles of the disclosure. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the various embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments described herein.