Systems and methods for training of multi-objective machine learning algorithms

Systems and methods including one or more processors and one or more non-transitory storage devices storing computing instructions configured to run on the one or more processors and perform: receiving one or more objectives and one or more constraints from a user electronic device of a user; generating a combined objective using the one or more objectives; calculating, using the combined objective, a return per click for an advertisement campaign; determining one or more bids based on (a) the return per click for the advertisement campaign and (b) a return on advertising spend for the advertisement campaign; estimating, using a first predictive algorithm and the one or more bids, an average cost per click; estimating an expected number of clicks for the advertisement campaign based on the average cost per click; determining a total amount used of the one or more constraints for the one or more bids based on the expected number of clicks for the advertisement campaign; and when the total amount used of the one or more constraints for the one or more bids exceeds at least one of the one or more constraints, lowering the one or more bids. Other embodiments are disclosed herein.

TECHNICAL FIELD

This disclosure relates generally to machine learning algorithms, and more specifically training multi-objective machine learning algorithms.

BACKGROUND

Machine learning algorithms (e.g., unsupervised learning, deep learning, supervised learning, etc.) are becoming more commonplace in today's computer systems, but many data scientists and software engineers continue to encounter problems while training novel algorithms (e.g., a multi-objective algorithm). One problem encountered when training machine learning algorithms is a lack of adequate amounts of representative training data. Machine learned algorithms trained on problematic training data suffer from a number of flaws. For example, machine learned algorithms trained on an insufficient amount of data can be inaccurate and, depending on the content of the training data, can overpredict or underpredict outcomes. Further, machine learned algorithms trained on non-representative training data can be skewed due to a unique event in the training data. These inaccuracies can also pose problems for machine learned algorithms used for automatic actions (e.g., automated trading algorithms, search engine marketing algorithms, search engine keyword bidding algorithms, etc.), as a severely overpredicted outcome can lead to recurring overpayments, among other problems.

In the past, solutions to this problem of poor or insufficient amounts of training data have been simply to (1) gather more training data, (2) purchase higher quality training data sets from a vendor, or (3) use a pre-trained model. Each of these past solutions had their own limitations. In many instances, gathering more training data can be time consuming due to the large corpus of training data needed to accurately train a machine learning model. Purchasing training data can also pose problems, as these training datasets can be expensive and can become outdated quickly. The disadvantages of pre-trained models are similar to those seen with purchased training data, as pre-trained models can also be expensive when they are bespoke and can become outdated quickly without updating or re-training. Further, embeddings that have not been seen before by a model or are new can be misclassified by a model (pre-trained or not) due to a lack of representation in the training data (either gathered or purchased). Therefore, there is a need for a system and method to bootstrap training of a machine learning algorithm without obtaining new training data or purchasing a pre-trained model.

DESCRIPTION OF EXAMPLES OF EMBODIMENTS

A number of embodiments can include a system. The system can include one or more processors and one or more non-transitory computer-readable storage devices storing computing instructions. The computing instructions can be configured to run on the one or more processors and perform receiving one or more objectives and one or more constraints from a user electronic device of a user; generating a combined objective using the one or more objectives; calculating, using the combined objective, a return per click for an advertisement campaign; determining one or more bids based on (a) the return per click for the advertisement campaign and (b) a return on advertising spend for the advertisement campaign; estimating, using a first predictive algorithm and the one or more bids, an average cost per click; estimating an expected number of clicks for the advertisement campaign based on the average cost per click; determining a total amount used of the one or more constraints for the one or more bids based on the expected number of clicks for the advertisement campaign; and when the total amount used of the one or more constraints for the one or more bids exceeds at least one of the one or more constraints, lowering the one or more bids.

Various embodiments include a method. The method can be implemented via execution of computing instructions configured to run at one or more processors and configured to be stored at non-transitory computer-readable media The method can comprise receiving one or more objectives and one or more constraints from a user electronic device of a user; generating a combined objective using the one or more objectives; calculating, using the combined objective, a return per click for an advertisement campaign; determining one or more bids based on (a) the return per click for the advertisement campaign and (b) a return on advertising spend for the advertisement campaign; estimating, using a first predictive algorithm and the one or more bids, an average cost per click; estimating an expected number of clicks for the advertisement campaign based on the average cost per click; determining a total amount used of the one or more constraints for the one or more bids based on the expected number of clicks for the advertisement campaign; and when the total amount used of the one or more constraints for the one or more bids exceeds at least one of the one or more constraints, lowering the one or more bids.

Turning to the drawings,FIG.1illustrates an exemplary embodiment of a computer system100, all of which or a portion of which can be suitable for (i) implementing part or all of one or more embodiments of the techniques, methods, and systems and/or (ii) implementing and/or operating part or all of one or more embodiments of the memory storage modules described herein. As an example, a different or separate one of a chassis102(and its internal components) can be suitable for implementing part or all of one or more embodiments of the techniques, methods, and/or systems described herein. Furthermore, one or more elements of computer system100(e.g., a monitor106, a keyboard104, and/or a mouse110, etc.) also can be appropriate for implementing part or all of one or more embodiments of the techniques, methods, and/or systems described herein. Computer system100can comprise chassis102containing one or more circuit boards (not shown), a Universal Serial Bus (USB) port112, a Compact Disc Read-Only Memory (CD-ROM) and/or Digital Video Disc (DVD) drive116, and a hard drive114. A representative block diagram of the elements included on the circuit boards inside chassis102is shown inFIG.2. A central processing unit (CPU)210inFIG.2is coupled to a system bus214inFIG.2. In various embodiments, the architecture of CPU210can be compliant with any of a variety of commercially distributed architecture families.

In many embodiments, all or a portion of memory storage unit208can be referred to as memory storage module(s) and/or memory storage device(s). In various examples, portions of the memory storage module(s) of the various embodiments disclosed herein (e.g., portions of the non-volatile memory storage module(s)) can be encoded with a boot code sequence suitable for restoring computer system100(FIG.1) to a functional state after a system reset. In addition, portions of the memory storage module(s) of the various embodiments disclosed herein (e.g., portions of the non-volatile memory storage module(s)) can comprise microcode such as a Basic Input-Output System (BIOS) operable with computer system100(FIG.1). In the same or different examples, portions of the memory storage module(s) of the various embodiments disclosed herein (e.g., portions of the non-volatile memory storage module(s)) can comprise an operating system, which can be a software program that manages the hardware and software resources of a computer and/or a computer network. The BIOS can initialize and test components of computer system100(FIG.1) and load the operating system. Meanwhile, the operating system can perform basic tasks such as, for example, controlling and allocating memory, prioritizing the processing of instructions, controlling input and output devices, facilitating networking, and managing files. Exemplary operating systems can comprise one of the following: (i) Microsoft® Windows® operating system (OS) by Microsoft Corp. of Redmond, Wash., United States of America, (ii) Mac® OS X by Apple Inc. of Cupertino, Calif., United States of America, (iii) UNIX® OS, and (iv) Linux® OS. Further exemplary operating systems can comprise one of the following: (i) the iOS® operating system by Apple Inc. of Cupertino, Calif., United States of America, (ii) the Blackberry® operating system by Research In Motion (RIM) of Waterloo, Ontario, Canada, (iii) the WebOS operating system by LG Electronics of Seoul, South Korea, (iv) the Android™ operating system developed by Google, of Mountain View, Calif., United States of America, (v) the Windows Mobile™ operating system by Microsoft Corp. of Redmond, Wash., United States of America, or (vi) the Symbian™ operating system by Accenture PLC of Dublin, Ireland.

Alternatively, or in addition to, the systems and procedures described herein can be implemented in hardware, or a combination of hardware, software, and/or firmware. For example, one or more application specific integrated circuits (ASICs) can be programmed to carry out one or more of the systems and procedures described herein. For example, one or more of the programs and/or executable program components described herein can be implemented in one or more ASICs. In many embodiments, an application specific integrated circuit (ASIC) can comprise one or more processors or microprocessors and/or memory blocks or memory storage.

In the depicted embodiment ofFIG.2, various I/O devices such as a disk controller204, a graphics adapter224, a video controller202, a keyboard adapter226, a mouse adapter206, a network adapter220, and other I/O devices222can be coupled to system bus214. Keyboard adapter226and mouse adapter206are coupled to keyboard104(FIGS.1-2) and mouse110(FIGS.1-2), respectively, of computer system100(FIG.1). While graphics adapter224and video controller202are indicated as distinct units inFIG.2, video controller202can be integrated into graphics adapter224, or vice versa in other embodiments. Video controller202is suitable for monitor106(FIGS.1-2) to display images on a screen108(FIG.1) of computer system100(FIG.1). Disk controller204can control hard drive114(FIGS.1-2), USB port112(FIGS.1-2), and CD-ROM drive116(FIGS.1-2). In other embodiments, distinct units can be used to control each of these devices separately.

Network adapter220can be suitable to connect computer system100(FIG.1) to a computer network by wired communication (e.g., a wired network adapter) and/or wireless communication (e.g., a wireless network adapter). In some embodiments, network adapter220can be plugged or coupled to an expansion port (not shown) in computer system100(FIG.1). In other embodiments, network adapter220can be built into computer system100(FIG.1). For example, network adapter220can be built into computer system100(FIG.1) by being integrated into the motherboard chipset (not shown), or implemented via one or more dedicated communication chips (not shown), connected through a PCI (peripheral component interconnector) or a PCI express bus of computer system100(FIG.1) or USB port112(FIG.1).

Returning now toFIG.1, although many other components of computer system100are not shown, such components and their interconnection are well known to those of ordinary skill in the art. Accordingly, further details concerning the construction and composition of computer system100and the circuit boards inside chassis102are not discussed herein.

Meanwhile, when computer system100is running, program instructions (e.g., computer instructions) stored on one or more of the memory storage module(s) of the various embodiments disclosed herein can be executed by CPU210(FIG.2). At least a portion of the program instructions, stored on these devices, can be suitable for carrying out at least part of the techniques and methods described herein.

Turning ahead in the drawings,FIG.3illustrates a block diagram of a system300that can be employed for training multi-objective machine learning algorithms, as described in greater detail below. System300is merely exemplary and embodiments of the system are not limited to the embodiments presented herein. System300can be employed in many different embodiments or examples not specifically depicted or described herein. In some embodiments, certain elements or modules of system300can perform various procedures, processes, and/or activities. In these or other embodiments, the procedures, processes, and/or activities can be performed by other suitable elements or modules of system300.

In some embodiments, system300can include a web server310and/or user computers330,331. Web server310and/or user computers330,331can each be a computer system, such as computer system100(FIG.1), as described above, and can each be a single computer, a single server, or a cluster or collection of computers or servers, or a cloud of computers or servers. In another embodiment, a single computer system can host each of two or more of web server310and/or user computers330,331. Additional details regarding web server310and/or user computers330,331are described herein.

In many embodiments, system300can comprise user computers340,341. In other embodiments, user computers340,341are external to system300. User computers340,341can comprise any of the elements described in relation to computer system100. In some embodiments, user computers340,341can be mobile devices. A mobile electronic device can refer to a portable electronic device (e.g., an electronic device easily conveyable by hand by a person of average size) with the capability to present audio and/or visual data (e.g., text, images, videos, music, etc.). For example, a mobile electronic device can comprise at least one of a digital media player, a cellular telephone (e.g., a smartphone), a personal digital assistant, a handheld digital computer device (e.g., a tablet personal computer device), a laptop computer device (e.g., a notebook computer device, a netbook computer device), a wearable user computer device, or another portable computer device with the capability to present audio and/or visual data (e.g., images, videos, music, etc.). Thus, in many examples, a mobile electronic device can comprise a volume and/or weight sufficiently small as to permit the mobile electronic device to be easily conveyable by hand. For examples, in some embodiments, a mobile electronic device can occupy a volume of less than or equal to approximately 1790 cubic centimeters, 2434 cubic centimeters, 2876 cubic centimeters, 4056 cubic centimeters, and/or 5752 cubic centimeters. Further, in these embodiments, a mobile electronic device can weigh less than or equal to 15.6 Newtons, 17.8 Newtons, 22.3 Newtons, 31.2 Newtons, and/or 44.5 Newtons.

Exemplary mobile electronic devices can comprise (i) an iPod®, iPhone®, iTouch®, iPad®, MacBook® or similar product by Apple Inc. of Cupertino, Calif., United States of America, (ii) a Blackberry® or similar product by Research in Motion (RIM) of Waterloo, Ontario, Canada, (iii) a Lumia® or similar product by the Nokia Corporation of Keilaniemi, Espoo, Finland, and/or (iv) a Galaxy™ or similar product by the Samsung Group of Samsung Town, Seoul, South Korea. Further, in the same or different embodiments, a mobile electronic device can comprise an electronic device configured to implement one or more of (i) the iPhone® operating system by Apple Inc. of Cupertino, Calif., United States of America, (ii) the Blackberry® operating system by Research In Motion (RIM) of Waterloo, Ontario, Canada, (iii) the Palm® operating system by Palm, Inc. of Sunnyvale, Calif., United States, (iv) the Android™ operating system developed by the Open Handset Alliance, (v) the Windows Mobile™ operating system by Microsoft Corp. of Redmond, Wash., United States of America, or (vi) the Symbian™ operating system by Nokia Corp. of Keilaniemi, Espoo, Finland.

Further still, the term “wearable user computer device” as used herein can refer to an electronic device with the capability to present audio and/or visual data (e.g., text, images, videos, music, etc.) that is configured to be worn by a user and/or mountable (e.g., fixed) on the user of the wearable user computer device (e.g., sometimes under or over clothing; and/or sometimes integrated with and/or as clothing and/or another accessory, such as, for example, a hat, eyeglasses, a wrist watch, shoes, etc.). In many examples, a wearable user computer device can comprise a mobile electronic device, and vice versa. However, a wearable user computer device does not necessarily comprise a mobile electronic device, and vice versa.

In specific examples, a wearable user computer device can comprise a head mountable wearable user computer device (e.g., one or more head mountable displays, one or more eyeglasses, one or more contact lenses, one or more retinal displays, etc.) or a limb mountable wearable user computer device (e.g., a smart watch). In these examples, a head mountable wearable user computer device can be mountable in close proximity to one or both eyes of a user of the head mountable wearable user computer device and/or vectored in alignment with a field of view of the user.

In more specific examples, a head mountable wearable user computer device can comprise (i) Google Glass™ product or a similar product by Google Inc. of Menlo Park, Calif., United States of America; (ii) the Eye Tap™ product, the Laser Eye Tap™ product, or a similar product by ePI Lab of Toronto, Ontario, Canada, and/or (iii) the Raptyr™ product, the STAR 1200™ product, the Vuzix Smart Glasses M100™ product, or a similar product by Vuzix Corporation of Rochester, N.Y., United States of America. In other specific examples, a head mountable wearable user computer device can comprise the Virtual Retinal Display™ product, or similar product by the University of Washington of Seattle, Wash., United States of America. Meanwhile, in further specific examples, a limb mountable wearable user computer device can comprise the iWatch™ product, or similar product by Apple Inc. of Cupertino, Calif., United States of America, the Galaxy Gear or similar product of Samsung Group of Samsung Town, Seoul, South Korea, the Moto360product or similar product of Motorola of Schaumburg, Ill., United States of America, and/or the Zip™ product, One™ product, Flex™ product, Charge™ product, Surge™ product, or similar product by Fitbit Inc. of San Francisco, Calif., United States of America.

In many embodiments, system300can comprise graphical user interface (“GUI”)350,351. In the same or different embodiments, GUI350,351can be part of and/or displayed by user computers340,341, which also can be part of system300. In some embodiments, GUI350,351can comprise text and/or graphics (image) based user interfaces. In the same or different embodiments, GUI350,351can comprise a heads up display (“HUD”). When GUI350,351comprises a HUD, GUI350,351can be projected onto glass or plastic, displayed in midair as a hologram, or displayed on monitor106(FIG.1). In various embodiments, GUI350,351can be color or black and white. In many embodiments, GUI350,351can comprise an application running on a computer system, such as computer system100, user computers330,331, and/or web server310. In the same or different embodiments, GUI350,351can comprise a website accessed through internet320. In some embodiments, GUI350,351can comprise an eCommerce website. In these or other embodiments, GUI350,351can comprise an administrative (e.g., back end) GUI allowing an administrator to modify and/or change one or more settings in system300. In the same or different embodiments, GUI350,351can be displayed as or on a virtual reality (VR) and/or augmented reality (AR) system or display. In some embodiments, an interaction with a GUI can comprise a click, a look, a selection, a grab, a view, a purchase, a bid, a swipe, a pinch, a reverse pinch, etc.

In some embodiments, web server310can be in data communication through Internet320with user computers330,331. In certain embodiments, user computers340,341can be desktop computers, laptop computers, smart phones, tablet devices, and/or other endpoint devices. Web server310can host one or more websites. For example, web server310can host website allowing users340,341to generate and train multi-objective machine learning algorithms.

In many embodiments, web server310and/or user computers330,331can each comprise one or more input devices (e.g., one or more keyboards, one or more keypads, one or more pointing devices such as a computer mouse or computer mice, one or more touchscreen displays, a microphone, etc.), and/or can each comprise one or more display devices (e.g., one or more monitors, one or more touch screen displays, projectors, etc.). In these or other embodiments, one or more of the input device(s) can be similar or identical to keyboard104(FIG.1) and/or a mouse110(FIG.1). Further, one or more of the display device(s) can be similar or identical to monitor106(FIG.1) and/or screen108(FIG.1). The input device(s) and the display device(s) can be coupled to the processing module(s) and/or the memory storage module(s) of web server310and/or user computers330,331in a wired manner and/or a wireless manner, and the coupling can be direct and/or indirect, as well as locally and/or remotely. As an example of an indirect manner (which may or may not also be a remote manner), a keyboard-video-mouse (KVM) switch can be used to couple the input device(s) and the display device(s) to the processing module(s) and/or the memory storage module(s). In some embodiments, the KVM switch also can be part of web server310and/or user computers330,331. In a similar manner, the processing module(s) and the memory storage module(s) can be local and/or remote to each other.

In Many embodiments, web server310and/or user computers330,331can communicate or interface (e.g., interact) with one another through a network or internet320. Internet320can be an intranet that is not open to the public. In further embodiments, internet320can be a mesh network of individual systems. Accordingly, in many embodiments, web server310(and/or the software used by such systems) can refer to a back end of system300operated by an operator and/or administrator of system300, and user computers330,331(and/or the software used by such systems) can refer to a front end of system300used by one or more users340,341, respectively. In some embodiments, users340,341can also be referred to as customers, in which case, user computers330,331can be referred to as customer computers. In these or other embodiments, the operator and/or administrator of system300can manage system300, the processing module(s) of system300, and/or the memory storage module(s) of system300using the input device(s) and/or display device(s) of system300.

Meanwhile, in many embodiments, web server310and/or user computers330,331also can be configured to communicate with one or more databases. The one or more databases can comprise a product database that contains information about products, items, or SKUs (stock keeping units) sold by a retailer. The one or more databases can be stored on one or more memory storage modules (e.g., non-transitory memory storage module(s)), which can be similar or identical to the one or more memory storage module(s) (e.g., non-transitory memory storage module(s)) described above with respect to computer system100(FIG.1). Also, in some embodiments, for any particular database of the one or more databases, that particular database can be stored on a single memory storage module of the memory storage module(s), and/or the non-transitory memory storage module(s) storing the one or more databases or the contents of that particular database can be spread across multiple ones of the memory storage module(s) and/or non-transitory memory storage module(s) storing the one or more databases, depending on the size of the particular database and/or the storage capacity of the memory storage module(s) and/or non-transitory memory storage module(s).

The one or more databases can each comprise a structured (e.g., indexed) collection of data and can be managed by any suitable database management systems configured to define, create, query, organize, update, and manage database(s). Exemplary database management systems can include MySQL (Structured Query Language) Database, PostgreSQL Database, Microsoft SQL Server Database, Oracle Database, SAP (Systems, Applications, & Products) Database, IBM DB2 Database, and/or NoSQL Database.

In many embodiments, the techniques described herein can provide a practical application and several technological improvements. In some embodiments, the techniques described herein can provide for an improved machine learning algorithm. These techniques described herein can provide a significant improvement over conventional machine learning algorithms by allowing multiple objectives to be specified by an individual implementing the algorithm. In many embodiments, the techniques described herein can beneficially create a feedback loop that allows a machine learning algorithm to converge on a most accurate bid amount. In this way, the techniques described herein can avoid problems with stale and/or outdated machine learned models by continually updating based on feedback from the loop.

In a number of embodiments, the techniques described herein can solve a technical problem that arises only within the realm of computer networks, as bidding on search engine advertisements does not exist outside the realm of computer networks.

Turning ahead in the drawings,FIG.4illustrates a flow chart for a method400, according to an embodiment. Method400is merely exemplary and is not limited to the embodiments presented herein. Method400can be employed in many different embodiments or examples not specifically depicted or described herein. In some embodiments, the activities of method400can be performed in the order presented. In other embodiments, the activities of method400can be performed in any suitable order. In still other embodiments, one or more of the activities of method400can be combined or skipped. In many embodiments, system300(FIG.3) can be suitable to perform method400and/or one or more of the activities of method400. In these or other embodiments, one or more of the activities of method400can be implemented as one or more computer instructions configured to run at one or more processing modules and configured to be stored at one or more non-transitory memory storage modules. Such non-transitory memory storage modules can be part of a computer system such as web server310, and/or user computers330,331(FIG.3). The processing module(s) can be similar or identical to the processing module(s) described above with respect to computer system100(FIG.1).

In many embodiments, method400can comprise an activity401of receiving one or more objectives and one or more constraints. In these or other embodiments, one or more objectives can comprise one or more objectives to optimize in a multi-objective algorithm. In some embodiments, one or more objectives can be for one or more advertisements, one or more keywords, and/or one or more advertisement campaigns. For example, the one or more objectives can be to optimize revenue generated from the campaign, profit generated from the campaign, orders generated from the campaign, clicks generated from the campaign, and/or new customers acquired from an advertisement/advertising campaign. In many embodiments, an objective can also be weighted based on an importance in the optimization. In some embodiments, a weight for an objective can be set by a user (e.g., users340,341(FIG.4)). In these or other embodiments, one or more objectives and one or more constraints can be received from a user electronic device (e.g., user computers330,331(FIG.3)). In many embodiments, one or more constraints can comprise one or more limits put on a multi-objective algorithm described herein. For example, when a multi-objective algorithm is used to optimize an advertising campaign, one or more constraints can comprise a total budget for implementing an advertising campaign.

In many embodiments, method400can comprise an activity402of generating a combined objective. In these or other embodiments, a combined objective can be generated using one or more objectives, as described above. A combined objective can be considered an amalgamation and/or average of its constituent part objectives. In many embodiments, a combined objective can be generated using weighted objectives, as described above. In this way, a user setting objectives can ensure that one or more objectives can have a larger influence on a multi-objective algorithm than other objectives, based on the respective weights of the objectives.

In some embodiments, method400can optionally comprise activity403of merging one or more objectives. In many embodiments, activity403can be performed at the same time or as a part of activity402. In these or other embodiments, merging one or more objectives can comprise combining one or more objectives in proportion to their weights. In many embodiments, a weighted objective can be calculated by taking an average value of one or more objectives. In various embodiments, a weighted objective can be calculated after normalizing and scaling one or more objectives to a unitless value. In various embodiments, scaling one or more objective can ensure that median values within the one or more objectives are approximately identical.

In many embodiments, method400can comprise an activity404of calculating a return per click. In some embodiments, a return per click can be calculated using one or more objectives and/or a combined objective, as described above. In these or other embodiments, a return per click can comprise a numerical value indicating a return on an objective for every click on an advertisement in an advertising campaign. For example, when profit is an objective, a reward per click can comprise an amount of profit gained for each click on an advertisement. In these or other embodiments, a return per click can comprise an average value calculated over all clicks on an advertisement during a period of time. In many embodiments, a reward per click can be calculated for a combined objective and/or a weighted combined objective. In various embodiments, a reward per click can be calculated by summing one or more return per clicks of one or more objectives. In these or other embodiments, a return per click can be calculated after an objective has been normalized and/or scaled as described above.

In many embodiments, method400can comprise an activity405determining one or more bids. In these or other embodiments, one or more bids can be determined using a return per click and/or a return on advertising spend. In various embodiments, one or more bids can comprise an estimated maximum bid (or estimated maximum value) for an impression slot showing an advertisement, keyword, or advertising campaign. In various embodiments, a bid can be determined by dividing the return per click by the return on advertising spend. In these or other embodiments, a bit can be determined by dividing a weighted return per click by a weighted return on advertising spend (“ROAS”). In many embodiments, (e.g., before activities409and/or410are performed) an initial value for ROAS can be calculated. In these embodiments, a predictive model (e.g., a machine learning model) can be used to calculate ROAS. For example, a log-log regression model can be built to predict return given one or more constraints (e.g. an advertising budget). In these or other embodiments, a training dataset (as described in further detail below) for a predictive model calculating ROAS can comprise historical values for one or more constraints and/or historical values for ROAS.

In many embodiments, method400can comprise an activity406of estimating an average cost per click. In these or other embodiments, an average cost per click can be generated, at least in part, using one or more bids. In various embodiments, a predictive algorithm can be used to estimate an average cost per click. In these or other embodiments, a predictive algorithm can comprise a hierarchical linear regression algorithm. In many embodiments, a predictive algorithm (e.g., a hierarchical linear regression algorithm) can be trained.

In some embodiments, training a predictive algorithm can comprise estimating internal parameters of a model configured to estimate an average cost per click. In various embodiments, a machine learning algorithm can be trained using labeled training data, otherwise known as a training dataset. In many embodiments, a training dataset can comprise historical data for a category in a hierarchical categorization. In these or other embodiments, historical data for a category can comprise measurements of bids and corresponding average cost per click for the category. In various embodiments, an additional, non-historical feature can be added to historical data to enhance its accuracy as a training dataset. For example, specifications (e.g., descriptions) of an item can be added to historical data. In various embodiments, features quantifying a historical competitiveness of an advertisement or advertising campaign can be added to a training data set. For example, search click ratio and/or an impression share ratio can be added to show whether an advertisement was competitive after an advertisement auction has been completed. In many embodiments, a training dataset can be aggregated over a predetermined time period. For example, one day's worth of historical data for items in a category of an advertised item can be used to train a predictive algorithm. In this way, a predictive algorithm can be trained when there is insufficient labeled training data for a specific advertisement, keyword, or campaign. By training a predictive algorithm on a related item's data instead of no or too little data about the item, problems associated with inaccuracies and over/under trained algorithms can be avoided and a more accurate multi-objective algorithm can be executed.

In the same or different embodiments, a pre-trained predictive algorithm can be used, and the pre-trained algorithm can be re-trained on the labeled training data. In some embodiments, the predictive model can also consider both historical and dynamic input from items in a category of an advertised item. In this way, a predictive algorithm can be trained iteratively as data from the category is added to a training data set. In many embodiments, a predictive algorithm can be iteratively trained in real time as data is added to a training data set. In various embodiments, a predictive algorithm can be trained, at least in part, on a single user's (e.g., user350) click data, or the single user's click data can be weighted in a training data set. In this way, a predictive algorithm tailored to a single user can be generated. In the same or different embodiments, a predictive algorithm tailored to a single user can be used as a pre-trained algorithm for a similar user. In several embodiments, due to a large amount of data needed to create and maintain a training data set, a predictive model can use extensive data inputs. Due to these extensive data inputs, in many embodiments, creating, training, and/or using a predictive algorithm as described herein cannot practically be performed in a mind of a human being.

In many embodiments, method400can comprise an activity407of estimating an expected number of clicks. In these or other embodiments, an expected number of clicks can be generated, at least in part, using an average cost per click. In the same or different embodiments, the expected number of clicks can be generated using one or more of the average cost per click, the return per click, or the return on advertising spend for the advertising campaign. In various embodiments, a predictive algorithm can be used to estimate an expected number of clicks. In these or other embodiments, a predictive algorithm in activity407can be similar to a predictive algorithm in activity406, but can be trained on different training data. For example, a predictive algorithm in activity407can be trained on historical data describing an average cost per click and a number of clicks for a different category during a different time period. In embodiments where the predictive algorithm is used to estimate the expected number of clicks, the estimation process does not need to be generated using the return per click or the return on advertising spend for the advertising campaign.

In many embodiments, method400can comprise an activity408of determining a total amount used of one or more constraints. In various embodiments, a total amount used can be determined based on one or more bids and an expected number of clicks. In these or other embodiments, a total amount used of one or more constraints can comprise a total cost of an advertising campaign (e.g., an amount of a budget used). In various embodiments, a total amount can be calculated by multiplying an estimated number of clicks by a cost per click (e.g., an average cost per click or a bid amount). A number of things can happen depending on whether a total amount used exceeds one or more constraints. For example, in some embodiments a total amount used does not exceed one or more constraints, and method400can terminate.

In many embodiments, method400can comprise an activity409of lowering one or more bids. In some embodiments, one or more bids can be lowered until a total amount used is within one or more constraints. In various embodiments, due to the structure of system300(FIG.3) and method400, one or more bids cannot be directly lowered by a user of the method. Therefore, in some embodiments, method400can optionally comprise activity410of reducing a return on advertising spend. In many embodiments, activity410can be performed at the same time or as a part of activity409. In these or other embodiments, a return on advertising spend can be lowered in an iterative process that uses a feedback loop to find an optimal value of a return on advertising spend. For example, a reduced return on advertising spend can be shunted back into activity405, as described above, and a new total amount used can be determined for one or more bids in activity405. Activity405, activity406, activity407, etc. can be repeated one or more times such that method400continues until the new total amount used is within one or more constraints. In many embodiments, a return on advertising spend can be lowered by half in activity410before it is shunted back into activity405. In other embodiments, return on advertising spend can be decreased by different amounts and/or increased. In various embodiments, an algorithmic approach can be used to lower a bid and/or a ROAS. For example, a one-dimensional Brent-Dekker method to lower a ROAS.

In some embodiments, method400can optionally comprise activity411of determining an estimated performance. Activity411can occur after or part of activity407, and activity411also can occur before activity409and/or410. In many embodiments, an estimated performance can be determined using a weighted return per click and an expected number of clicks. In these or other embodiments, an estimated performance can be determined by multiplying a weighted return per click and an expected number of clicks together. In many embodiments, an estimated performance can be displayed to a user (e.g., on user computer330,331). In these or other embodiments, various visualizations can be created to model the return per click against other metrics. For example, a future amount of an objective (e.g., can be predicted total revenue, profit, number of newly acquired customers, total number of orders, and/or advertising spending) can be predicted. A user of method400can then check whether this prediction matches with their objectives and adjust their bids accordingly.

Turning ahead in the drawings,FIG.5illustrates a block diagram of a system500that can be employed for training multi-objective machine learning algorithms. System500is merely exemplary and embodiments of the system are not limited to the embodiments presented herein. System500can be employed in many different embodiments or examples not specifically depicted or described herein. In some embodiments, certain elements or modules of system500can perform various procedures, processes, and/or activities. In these or other embodiments, the procedures, processes, and/or activities can be performed by other suitable elements or modules of system500. In many embodiments, elements of system500can be implemented in one or more of web server310(FIG.3) and/or user computers330,331(FIG.3).

In many embodiments, system500can comprise non-transitory memory storage module501. Memory storage module501can be referred to as objective receiving module501. In many embodiments, objective receiving module501can store computing instructions configured to run on one or more processing modules and perform one or more acts of method400(FIG.4) (e.g., activity401(FIG.4)).

In many embodiments, system500can comprise non-transitory memory storage module502. Memory storage module502can be referred to as combined objective generating module502. In many embodiments, combined objective generating module502can store computing instructions configured to run on one or more processing modules and perform one or more acts of method400(FIG.4) (e.g., activity402(FIG.4)).

In many embodiments, system500can comprise non-transitory memory storage module503. Memory storage module503can be referred to as objective merging module503. In many embodiments, objective merging module503can store computing instructions configured to run on one or more processing modules and perform one or more acts of method400(FIG.4) (e.g., activity403(FIG.4)).

In many embodiments, system500can comprise non-transitory memory storage module504. Memory storage module504can be referred to as return per click calculating module504. In many embodiments, return per click calculating module504can store computing instructions configured to run on one or more processing modules and perform one or more acts of method400(FIG.4) (e.g., activity404(FIG.4)).

In many embodiments, system500can comprise non-transitory memory storage module505. Memory storage module505can be referred to as bid determining module505. In many embodiments, bid determining module505can store computing instructions configured to run on one or more processing modules and perform one or more acts of method400(FIG.4) (e.g., activity405(FIG.4)).

In many embodiments, system500can comprise non-transitory memory storage module506. Memory storage module506can be referred to as average cost per click estimating module506. In many embodiments, average cost per click estimating module506can store computing instructions configured to run on one or more processing modules and perform one or more acts of method400(FIG.4) (e.g., activity406(FIG.4)).

In many embodiments, system500can comprise non-transitory memory storage module507. Memory storage module507can be referred to as expected clicks estimating module507. In many embodiments, expected clicks estimating module507can store computing instructions configured to run on one or more processing modules and perform one or more acts of method400(FIG.4) (e.g., activity407(FIG.4)).

In many embodiments, system500can comprise non-transitory memory storage module508. Memory storage module508can be referred to as total amount used determining module508. In many embodiments, total amount determining module508can store computing instructions configured to run on one or more processing modules and perform one or more acts of method400(FIG.4) (e.g., activity408(FIG.4)).

In many embodiments, system500can comprise non-transitory memory storage module509. Memory storage module509can be referred to as bid lowering module509. In many embodiments, bid lowering module509can store computing instructions configured to run on one or more processing modules and perform one or more acts of method400(FIG.4) (e.g., activity409(FIG.4)).

In many embodiments, system500can comprise non-transitory memory storage module510. Memory storage module510can be referred to as return on advertising spend reducing module510. In many embodiments, return on advertising spend reducing module510can store computing instructions configured to run on one or more processing modules and perform one or more acts of method400(FIG.4) (e.g., activity410(FIG.4)).

In many embodiments, system500can comprise non-transitory memory storage module511. Memory storage module511can be referred to as estimated performance determining module511. In many embodiments, estimated performance determining module510can store computing instructions configured to run on one or more processing modules and perform one or more acts of method400(FIG.4) (e.g., activity411(FIG.4)).

Although systems and methods for training multi-objective machine learning algorithms have been described with reference to specific embodiments, it will be understood by those skilled in the art that various changes may be made without departing from the spirit or scope of the disclosure. Accordingly, the disclosure of embodiments is intended to be illustrative of the scope of the disclosure and is not intended to be limiting. It is intended that the scope of the disclosure shall be limited only to the extent required by the appended claims. For example, to one of ordinary skill in the art, it will be readily apparent that any element ofFIGS.1-5may be modified, and that the foregoing discussion of certain of these embodiments does not necessarily represent a complete description of all possible embodiments. For example, one or more of the procedures, processes, or activities ofFIG.4may include different procedures, processes, and/or activities and be performed by many different modules, in many different orders.