Patent Description:
Computing devices (e.g., desktop computers, laptop computers, tablet computers, smartphones, wearable computing devices, and/or the like) are ubiquitous in modern society. They can support communications between their users, provide their users with information about their environments, current events, the world at large, and/or the like. A popular use of such devices is generating, manipulating, communicating, and/or the like digital imagery (e.g., pictures, video, and/or the like). <CIT> discloses a computer-implemented method for selecting representative frames for videos.

One aspect of the present disclosure is directed to a computer-implemented method according to claim <NUM>.

Another aspect of the present disclosure is directed to a system according to claim <NUM>.

Detailed discussion of embodiments directed to one of ordinary skill in the art is set forth in the specification, which refers to the appended figures, in which:.

Example aspects of the present disclosure are directed to processing imagery using one or more machine learning (ML) models. In particular, one or more computing devices can receive data describing imagery comprising a plurality of different and distinct frames (e.g., individual still images, component frames of one or more videos, and/or the like). Based at least in part on one or more ML models and the data describing the imagery, the computing device(s) can determine, for each frame of the plurality of different and distinct frames, one or more scores for the frame. Each score of the score(s) can indicate a determined measure of suitability of the frame with respect to one or more of various different and distinct uses for which the ML model(s) are configured to determine suitability of imagery.

Such uses can include, for example, use with a particular application (e.g., an email application, a messaging application, a social-media application, and/or the like); use with a particular audience (e.g., the public at large, a messaging group, a list of particular recipients, friends, family members, work colleagues, and/or the like); inclusion in a series of frames for sequential presentation (e.g., a slideshow, montage, and/or the like); use as a representative, thumbnail, cover, and/or the like for an album, set, series, and/or the like of related frames; and/or the like.

In some embodiments, the computing device(s) can provide an application programming interface (API) for processing imagery using the ML model(s), for example, to identify imagery determined to be suitable for one or more of the various different and distinct uses. In some of such embodiments, the computing device(s) can receive the data describing the imagery via the API (e.g., from a requesting application, and/or the like). In some embodiments, the computing device(s) can communicate (e.g., via the API, to the requesting application, and/or the like) data indicating, for each frame of one or more frames of the plurality of different and distinct frames, at least one of the score(s) for the frame. Based at least in part on such data, in some of such embodiments, the computing device(s) can generate data describing an interface comprising a curation (e.g., for presentation, browsing, manipulation, and/or the like) of at least a portion of such frame(s).

In some embodiments, the computing device(s) can determine, for one or more frames in a set of frames included in the plurality of different and distinct frames, one or more scores indicating the frame(s) in the set are determined to be better suited for one or more particular uses of the various different and distinct uses than each other frame in the set. In some of such embodiments, the computing device(s) can receive (e.g., via the API, from the requesting application, and/or the like) data indicating the particular use(s) (e.g., in association with a request to identify one or more frames suitable for such use(s), and/or the like).

In some embodiments, the set of frames can include a series of contemporaneously generated similar frames (e.g., a burst, stream, and/or the like). Each frame in the series can include one or more subjects (e.g., people, animals, objects of focus, and/or the like) shared with each other frame in the series, and a contextual background (e.g., associated with the setting, surrounding environment, and/or the like), of the one or more subjects, shared with each other frame in the series. Additionally or alternatively, the set of frames can include a set of different and distinct frames including frames from multiple different and distinct series of frames. Each frame in such set can include one or more subjects different and distinct from each other frame in such set, and/or a contextual background, of one or more subjects of the frame in such set, different and distinct from each other frame in such set.

In some embodiments, the computing device(s) can receive (e.g., via the API, from the requesting application, in association with a request to identify one or more frames, and/or the like) data indicating one or more particular subjects of the plurality of different and distinct frames, one or more particular expressions (e.g., smiling, open eyes, laughter, and/or the like) exhibited by subjects of the plurality of different and distinct frames, one or more particular events (e.g., scoring a point in a sporting event, and/or the like) depicted by the plurality of different and distinct frames, and/or the like. In some of such embodiments, determining the score(s) for each of the plurality of different and distinct frames can include identifying, from amongst the plurality of different and distinct frames, a set of frames that each include the particular subject(s), expression(s), event(s), and/or the like.

In some embodiments, the computing device(s) can (e.g., in association with a request to identify one or more frames for inclusion in a slideshow, montage, and/or the like) determine, for each of one or more frames (e.g., based at least in part on the score(s) for such frame(s), and/or the like), a position of the frame in a series of frames for sequential presentation. In some of such embodiments, the computing device(s) can communicate (e.g., via the API, to the requesting application, and/or the like) data indicating, for each frame of such frame(s), the position of the frame in the series of frames for sequential presentation.

In some embodiments, the data describing the imagery can be generated based at least in part on one or more arrangements of pixels included in the imagery (e.g., the visual content of the imagery, and/or the like) but not be sufficient to enable reconstruction of such arrangement(s) (e.g., to mitigate privacy concerns, and/or the like).

In some embodiments, the computing device(s) can generate the ML model(s) based at least in part on data (e.g., training data, and/or the like) describing other imagery and user input with respect to such imagery. In some embodiments, the data describing such other imagery and user input can be received from a particular user device from which the data describing the imagery comprising the plurality of different and distinct frames is received. Additionally or alternatively, the data describing the other imagery and user input can be received from one or more user devices different and distinct from the particular user device from which the data describing the imagery comprising the plurality of different and distinct frames is received.

In some embodiments, the ML model(s) can be, otherwise include, and/or the like various ML models, such as neural networks (e.g., deep neural networks, and/or the like), other types of ML models, including non-linear models, linear models, multilayer models, and/or the like. Neural networks can include feed-forward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks, and/or the like), convolutional neural networks, other forms of neural networks, and/or the like. In some embodiments, the ML model(s) can be trained using supervised techniques, semi-supervised techniques, unsupervised techniques, and/or the like.

The technology described herein can provide a number of technical effects and benefits. For example, the technology described herein can enable efficient identification, organization, generation, and/or the like of imagery (e.g., one or more frames, sets of frames, series of frames, and/or the like) suitable for one or more particular uses, thus obviating or reducing otherwise required user effort and device interaction, and thereby conserving computing resources (e.g., energy, processing cycles, network bandwidth, and/or the like).

With reference now to the figures, example embodiments of the present disclosure will be discussed in further detail.

<FIG> depicts an example computing environment according to example embodiments of the present disclosure.

Referring to <FIG>, environment <NUM> can include one or more computing devices (e.g., one or more desktop computers, laptop computers, tablet computers, mobile devices, smartphones, wearable devices, servers, and/or the like). For example, environment <NUM> can include computing devices <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> and computing system <NUM>, which can include one or more computing devices. Environment <NUM> can also include one or more networks <NUM> (e.g., one or more wired networks, wireless networks, and/or the like). Network(s) <NUM> can interface computing device(s) <NUM>, <NUM>, <NUM>, <NUM>, and/or <NUM> with one another and/or with computing system <NUM>.

Computing device <NUM> can include one or more processor(s) <NUM>, one or more communication interfaces <NUM>, and memory <NUM> (e.g., one or more hardware components for storing executable instructions, data, and/or the like). Communication interface(s) <NUM> can enable computing device <NUM> to communicate with computing device(s) <NUM>, <NUM>, <NUM>, and/or <NUM> and/or with computing system <NUM>. Memory <NUM> can include (e.g., store, and/or the like) instructions <NUM>. When executed by processor(s) <NUM>, instructions <NUM> can cause computing device <NUM> to perform one or more operations, functions, and/or the like described herein. It will be appreciated that computing device(s) <NUM>, <NUM>, <NUM>, and/or <NUM> can include one or more of the components described above with respect to computing device <NUM>.

Computing system <NUM> can include one or more processor(s) <NUM>, one or more communication interfaces <NUM>, and memory <NUM> (e.g., one or more hardware components for storing executable instructions, data, and/or the like). Communication interface(s) <NUM> can enable computing system <NUM> to communicate with computing device(s) <NUM>, <NUM>, <NUM>, <NUM>, and/or <NUM>. Memory <NUM> can include (e.g., store, and/or the like) instructions <NUM>. When executed by processor(s) <NUM>, instructions <NUM> can cause computing system <NUM> to perform one or more operations, functions, and/or the like described herein.

Unless explicitly indicated otherwise, the operations, functions, and/or the like described herein can be performed by computing device(s) <NUM>, <NUM>, <NUM>, <NUM>, and/or <NUM> and/or computing system <NUM> (e.g., by computing device <NUM>, <NUM>, <NUM>, <NUM>, or <NUM>, by computing system <NUM>, by a combination of one or more of computing device(s) <NUM>, <NUM>, <NUM>, <NUM>, and/or <NUM> and/or computing system <NUM>, and/or the like).

<FIG> and <FIG> depict an example event sequence according to example embodiments of the present disclosure.

Referring to <FIG>, at (<NUM>), computing device <NUM> can generate data describing imagery, one or more aspects of the imagery, one or more uses for which the imagery is well suited, and/or the like and can communicate (e.g., via network(s) <NUM> (as indicated by the pattern-filled box over the line extending downward from network(s) <NUM>), and/or the like) such data to computing system <NUM>, which can receive the data. Similarly, at (<NUM>), computing device <NUM> can generate data describing imagery, one or more aspects of the imagery, one or more uses for which the imagery is well suited, and/or the like and can communicate such data to computing system <NUM>, which can receive the data.

At (<NUM>), user input with respect to imagery can be provided via computing device <NUM> (e.g., the imagery can be utilized for one or more particular purposes for which it is well suited, and/or the like), and, at (<NUM>), computing device <NUM> can generate (e.g., based at least in part on the user input, and/or the like) data describing the imagery, one or more aspects of the imagery, the user input, one or more uses for which the imagery is well suited, and/or the like and can communicate such data to computing system <NUM>, which can receive the data. Similarly, at (<NUM>), user input with respect to imagery can be provided via computing device <NUM>, and, at (<NUM>), computing device <NUM> can generate (e.g., based at least in part on the user input, and/or the like) data describing the imagery, one or more aspects of the imagery, the user input, one or more uses for which the imagery is well suited, and/or the like and can communicate such data to computing system <NUM>, which can receive the data; and, at (<NUM>), user input with respect to imagery can be provided via computing device <NUM>, and, at (<NUM>), computing device <NUM> can generate (e.g., based at least in part on the user input, and/or the like) data describing the imagery, one or more aspects of the imagery, the user input, one or more uses for which the imagery is well suited, and/or the like and can communicate such data to computing system <NUM>, which can receive the data.

For example, computing device <NUM> can be a user device (e.g., one or more associated computing devices at least temporarily associated with a particular user, user account, and/or the like). Similarly, computing device <NUM> can be a user device (e.g., one or more associated computing devices at least temporarily associated with a different and distinct user, user account, and/or the like) different and distinct from computing device <NUM>; and computing device <NUM> can be a user device (e.g., one or more associated computing devices at least temporarily associated with a further different and distinct user, user account, and/or the like) different and distinct from computing devices <NUM> and <NUM>.

Computing device(s) <NUM>, <NUM>, and/or <NUM> can execute one or more different and distinct applications. For example, referring to <FIG>, computing device(s) <NUM>, <NUM>, and/or <NUM> can execute an application (e.g., a photo-management application, and/or the like) associated with graphical user interface (GUI) element <NUM>, and/or the like.

Referring to <FIG>, in some embodiments (e.g., to mitigate potential privacy concerns, and/or the like), one or more users of computing device(s) <NUM>, <NUM>, and/or <NUM> can be provided (e.g., via element <NUM>, and/or the like) with information regarding collection of certain data, and/or the like, one or more controls (e.g., via element(s) <NUM> and/or <NUM>, and/or the like) for allowing the user(s) to make one or more elections as to if and/or when the methods, systems, functions, operations, and/or the like described herein can enable collection of certain data, and/or the like (e.g., data describing imagery, one or more aspects of imagery, user input, one or more uses for which imagery is well suited, and/or the like). Additionally or alternatively, certain data (e.g., data describing imagery, one or more aspects of imagery, user input, one or more uses for which imagery is well suited, and/or the like) can be treated in one or more ways before being stored, utilized, and/or the like (e.g., so personally identifiable information can be removed, underlying visual content of imagery can be removed, abstracted, and/or the like). For example, a user's identity, imagery, data associated therewith, and/or the like can be treated so that no personally identifiable information can be determined for the user, the underlying visual content of imagery cannot be discerned, reconstructed, and/or the like. Thus, the user(s) can have control over what data is collected about them, how that information is used, what information they are provided, and/or the like.

In accordance with one or more aspects of the disclosure, computing system <NUM> and/or computing device(s) <NUM>, <NUM>, and/or <NUM> can provide an application programming interface (API) for processing imagery using one or more machine learning (ML) models, for example, to identify imagery determined to be suitable for one or more of various different and distinct uses for which the ML model(s) are configured to determine suitability of imagery, and/or the like. In some embodiments, the data (e.g., communicated at (<NUM>), (<NUM>), (<NUM>), and/or the like) describing the imagery, one or more aspects of the imagery, the user input, one or more uses for which the imagery is well suited, and/or the like can be communicated via such API, and/or the like. The various different and distinct uses can include, for example, use with a particular application (e.g., an email application, a messaging application, a social-media application, and/or the like); use with a particular audience (e.g., the public at large, a messaging group, a list of particular recipients, friends, family members, work colleagues, and/or the like); inclusion in a series of frames for sequential presentation (e.g., a slideshow, montage, and/or the like); use as a representative, thumbnail, cover, and/or the like for an album, set, series, and/or the like of related frames; and/or the like.

Referring to <FIG>, at (<NUM>), computing system <NUM> can generate (e.g., based at least in part on the data communicated at (<NUM>), (<NUM>), (<NUM>), (<NUM>), (<NUM>), and/or the like) data and can communicate such data to computing device <NUM>, which can receive the data.

At (<NUM>), the ML model(s) configured to determine suitability of imagery for the various different and distinct uses can be generated. For example, at (220A), computing device <NUM> can generate (e.g., based at least in part on the data communicated at (<NUM>), (<NUM>), and/or the like) one or more of such ML model(s). Additionally or alternatively, at (220B), computing system <NUM> can generate (e.g., based at least in part on the data communicated at (<NUM>), (<NUM>), (<NUM>), (<NUM>), (<NUM>), and/or the like) one or more of such ML model(s).

<FIG> depicts an example computing platform according to example embodiments of the present disclosure.

Referring to <FIG>, computing device <NUM> and/or computing system <NUM> can execute one or more different and distinct applications <NUM>, <NUM>, and/or <NUM>. For example, application <NUM> can be the application (e.g., the photo-management application, and/or the like) associated with element <NUM>, and/or the like. Computing device <NUM> and/or computing system <NUM> can also execute ML platform <NUM>, which can provide the API for processing imagery using the machine learning (ML) model(s). Such API can interface, enable communications between, and/or the like (e.g., via one or more calls, returns, and/or the like) ML platform <NUM> and application(s) <NUM>, <NUM>, and/or <NUM>. It will be appreciated that computing device(s) <NUM> and/or <NUM> can provide functionality (e.g., associated with ML platform <NUM>, and/or the like) similar to that described with respect to computing device <NUM> and/or computing system <NUM>.

ML platform <NUM> can interface with model repository <NUM>, which can include one or more ML models <NUM>, <NUM>, and/or <NUM> (e.g., the ML model(s) configured to determine suitability of imagery for the various different and distinct uses, and/or the like) generated by one or more ML engines <NUM>, for example, based at least in part on one or more examples (e.g., training data, and/or the like) included in example database <NUM>. Such example(s) can, for example, include one or more examples managed, generated, and/or the like by ML platform <NUM>, for example, based at least in part on the data communicated at (<NUM>), (<NUM>), (<NUM>), and/or the like (e.g., from application(s) <NUM>, <NUM>, and/or <NUM>, via the API, and/or the like), the data communicated at (<NUM>), (<NUM>), (<NUM>), and/or the like.

In some embodiments, ML models <NUM>, <NUM>, and/or <NUM> can be, otherwise include, and/or the like various ML models, such as neural networks (e.g., deep neural networks, and/or the like), other types of ML models, including non-linear models, linear models, multilayer models, and/or the like. Neural networks can include feed-forward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks, and/or the like), convolutional neural networks, other forms of neural networks, and/or the like. In some embodiments, ML models <NUM>, <NUM>, and/or <NUM> can be trained using supervised techniques, semi-supervised techniques, unsupervised techniques, and/or the like.

Returning to <FIG>, at (<NUM>), user input with respect to imagery can be provided via computing device <NUM>, and, at (<NUM>), computing device <NUM> can generate (e.g., based at least in part on the user input, and/or the like) data describing the imagery, one or more aspects of the imagery, the user input, and/or the like and can communicate (e.g., via the API provided by ML platform <NUM>, and/or the like) such data to computing system <NUM>, which can receive the data. The imagery can comprise a plurality of different and distinct frames (e.g., individual still images, component frames of one or more videos, and/or the like).

For example, referring to <FIG>, portion <NUM> of the illustrated GUI can be associated with application <NUM> (e.g., the photo-management application, and/or the like) and can include one or more portions <NUM>, <NUM>, and/or <NUM>, as well as element <NUM>. Imagery can be selected via application <NUM>, and the selected imagery can comprise a plurality of different and distinct frames. For example, the imagery can include one or more frames from imagery included in one or more selected albums identified within portion <NUM>, one or more frames from selected imagery identified within portion <NUM>, one or more frames from imagery included in one or more selected videos identified within portion <NUM>, and/or the like. Responsive to element <NUM> being invoked (e.g., in association with a request via application <NUM> to identify one or more frames within the selected imagery determined to be suitable for one or more of the various different and distinct uses for which ML models <NUM>, <NUM>, and/or <NUM> are configured to determine suitability of imagery, and/or the like), computing device <NUM> can generate data describing the selected imagery, one or more aspects of the selected imagery, and/or the like and can communicate (e.g., via the API provided by ML platform <NUM>, and/or the like) such data to computing system <NUM>, which can receive the data.

In some embodiments, one or more portions of the data described herein (e.g., the data communicated at (<NUM>), (<NUM>), (<NUM>), (<NUM>), (<NUM>), (<NUM>), (<NUM>), and/or the like) can be generated based at least in part on one or more arrangements of pixels included in imagery (e.g., the visual content of the imagery, and/or the like) but not be sufficient to enable reconstruction of such arrangement(s) (e.g., to mitigate privacy concerns, and/or the like).

Returning to <FIG>, at (<NUM>), based at least in part on ML models <NUM>, <NUM>, and/or <NUM> and the data describing the selected imagery, one or more scores can be determined for each frame of the plurality of different and distinct frames of the selected imagery. Each of the score(s) can indicate a determined measure of suitability of the frame with respect to one or more of the various different and distinct uses for which ML models <NUM>, <NUM>, and/or <NUM> are configured to determine suitability of imagery. For example, at (226A), computing device <NUM> can utilize ML models <NUM>, <NUM>, and/or <NUM> to determine one or more such scores for one or more of the plurality of different and distinct frames of the selected imagery. Additionally or alternatively, at (226B), computing system <NUM> can utilize ML models <NUM>, <NUM>, and/or <NUM> to determine one or more such scores for one or more of the plurality of different and distinct frames of the selected imagery.

In some embodiments, which of ML models <NUM>, <NUM>, and/or <NUM> are utilized to determine such scores can be specified (e.g., by data received via the API provided by ML platform <NUM>, from application <NUM>, in association with the request to identify one or more frames within the selected imagery determined to be suitable for one or more of the various different and distinct uses for which ML models <NUM>, <NUM>, and/or <NUM> are configured to determine suitability of imagery, and/or the like).

In some embodiments, computing device <NUM> and/or computing system <NUM> can determine, for one or more frames in a set of frames included in the plurality of different and distinct frames, one or more scores indicating the frame(s) in the set are determined to be better suited for one or more particular uses of the various different and distinct uses than each other frame in the set. In some of such embodiments, computing device <NUM> and/or computing system <NUM> can receive (e.g., via the API provided by ML platform <NUM>, from application <NUM>, and/or the like) data indicating the particular use(s) (e.g., in association with the request to identify one or more frames within the selected imagery determined to be suitable for one or more of the various different and distinct uses for which ML models <NUM>, <NUM>, and/or <NUM> are configured to determine suitability of imagery, and/or the like).

In some embodiments, the set of frames can include a series of contemporaneously generated similar frames (e.g., a burst, stream, and/or the like). Each frame in the series can include one or more subjects (e.g., people, animals, objects of focus, and/or the like) shared with each other frame in the series, and a contextual background (e.g., associated with the setting, surrounding environment, and/or the like), of the one or more subjects, shared with each other frame in the series. In some of such embodiments, the score(s) for the frames can be determined in real time, near real time, and/or the like, for example, enabling selection of one or more frames (e.g., determined to be suitable for one or more particular uses, and/or the like) from a buffer of contemporaneously generated frames, for example, thereby assisting a user in capturing one or more particular frames (e.g., a shot with all the subjects eyes simultaneously open, and/or the like). Additionally or alternatively, the set of frames can include a set of different and distinct frames including frames from multiple different and distinct series of frames. Each frame in such set can include one or more subjects different and distinct from each other frame in such set, and/or a contextual background, of one or more subjects of the frame in such set, different and distinct from each other frame in such set.

In some embodiments, computing device <NUM> and/or computing system <NUM> can receive (e.g., via the API provided by ML platform <NUM>, from application <NUM>, in association with the request to identify one or more frames within the selected imagery determined to be suitable for one or more of the various different and distinct uses for which ML models <NUM>, <NUM>, and/or <NUM> are configured to determine suitability of imagery, and/or the like) data indicating one or more particular subjects of the plurality of different and distinct frames, one or more particular expressions (e.g., smiling, open eyes, laughter, and/or the like) exhibited by subjects of the plurality of different and distinct frames, one or more particular events (e.g., scoring a point in a sporting event, and/or the like) depicted by the plurality of different and distinct frames, and/or the like. In some of such embodiments, determining the score(s) for each frame of the plurality of different and distinct frames can include identifying, from amongst the plurality of different and distinct frames, a set of frames that each include the particular subject(s), expression(s), event(s), and/or the like.

In some embodiments, determining the score(s) for each frame of the plurality of different and distinct frames can include determining the score(s) based at least in part on one or more quality metrics determined for the frame (e.g., regarding image quality, focus, blur, exposure, and/or the like). Additionally or alternatively, determining the score(s) can include identifying, counting, and/or the like incidences, occurrences, and/or the like of one or more particular subjects, groups of subjects, and/or the like.

In some embodiments, computing device <NUM> and/or computing system <NUM> can (e.g., in association with a request to identify one or more frames for inclusion in a slideshow, montage, and/or the like) determine, for each frame of one or more frames (e.g., based at least in part on the score(s) for such frame(s), and/or the like), a position of the frame in a series of frames for sequential presentation.

In some embodiments, computing device <NUM> and/or computing system <NUM> can utilize scene-analysis technology to determine, for each frame of one or more frames, one or more of the score(s) for the frame. For example, to determine one or more of such score(s), computing device <NUM> and/or computing system <NUM> can utilize one or more aspects of the technology described in <CIT>, and entitled "SYSTEMS AND METHODS FOR SELECTIVE RETENTION AND EDITING OF IMAGES CAPTURED BY MOBILE IMAGE CAPTURE DEVICE,".

At (<NUM>), computing system <NUM> can generate data indicating, for each frame of one or more of the plurality of different and distinct frames, at least one of the score(s) for the frame, one or more positions of the frame in one or more series of frames for sequential presentation, and/or the like and can communicate (e.g., via the API provided by ML platform <NUM>, to application <NUM>, and/or the like) such data to computing device <NUM>, which can receive the data.

Based at least in part on such data, at (<NUM>), computing device <NUM> can generate output, for example, data describing an interface comprising a curation (e.g., for presentation, browsing, manipulation, and/or the like) of at least a portion of such frame(s). For example, referring to <FIG>, portion <NUM> of the illustrated GUI can identify one or more of the plurality of different and distinct frames determined to be suitable for a particular application (e.g., "App_1," and/or the like); portion <NUM> can identify one or more of the plurality of different and distinct frames determined to be suitable for a different and distinct application (e.g., "App_N," and/or the like); portion <NUM> can identify one or more of the plurality of different and distinct frames determined to be suitable for a particular audience (e.g., "Friends," and/or the like); portion <NUM> can identify one or more of the plurality of different and distinct frames determined to be suitable for a different and distinct audience (e.g., "Public," and/or the like); portion <NUM> can identify one or more series of frames for sequential presentation (e.g., "Slideshows," and/or the like) comprising one or more of the plurality of different and distinct frames at one or more particular positions for which they were determined to be suitable, and/or the like; and portion <NUM> can identify one or more different and distinct series of frames for sequential presentation (e.g., "Montages," and/or the like) comprising one or more of the plurality of different and distinct frames at one or more particular positions for which they were determined to be suitable, and/or the like.

<FIG> depicts an example method according to example embodiments of the present disclosure.

Referring to <FIG>, at (<NUM>), one or more computing devices can provide an API for processing imagery using one or more ML models to identify imagery determined to be suitable for one or more different and distinct uses for which the ML model(s) are configured to determine suitability of imagery. For example, computing device <NUM> and/or computing system <NUM> can provide an API (e.g., the API provided by ML platform <NUM>, and/or the like) for processing imagery using ML models <NUM>, <NUM>, and/or <NUM>.

At (<NUM>), the computing device(s) can receive, via the API, data describing imagery comprising a plurality of different and distinct frames for processing using the ML model(s). For example, computing device <NUM> and/or computing system <NUM> can receive (e.g., via the API provided by ML platform <NUM>, and/or the like) data describing the imagery selected via application <NUM>.

At (<NUM>), the computing device(s) can determine, for each frame of the plurality of different and distinct frames, one or more scores for the frame, each score of the score(s) indicating a determined measure of suitability of the frame with respect to one or more of the different and distinct uses for which the ML model(s) are configured to determine suitability of imagery. For example, computing device <NUM> and/or computing system <NUM> can utilize ML models <NUM>, <NUM>, and/or <NUM> to determine, for each frame of the plurality of different and distinct frames of the imagery selected via application <NUM>, one or more scores for the frame.

At (<NUM>), the computing device(s) can communicate, via the API, data indicating, for each frame of one or more frames of the plurality of different and distinct frames, at least one of the score(s) for the frame. For example, computing device <NUM> and/or computing system <NUM> can communicate (e.g., via the API provided by ML platform <NUM>, and/or the like) data indicating, for each frame of one or more frames of the plurality of different and distinct frames of the imagery selected via application <NUM>, at least one of the score(s) for the frame.

The technology discussed herein refers to servers, databases, software applications, and/or other computer-based systems, as well as actions taken and information sent to and/or from such systems. The inherent flexibility of computer-based systems allows for a great variety of possible configurations, combinations, and/or divisions of tasks and/or functionality between and/or among components. For instance, processes discussed herein can be implemented using a single device or component and/or multiple devices or components working in combination. Databases and/or applications can be implemented on a single system and/or distributed across multiple systems. Distributed components can operate sequentially and/or in parallel.

Various connections between elements are discussed in the above description. These connections are general and, unless specified otherwise, can be direct and/or indirect, wired and/or wireless. In this respect, the specification is not intended to be limiting.

The depicted and/or described steps are merely illustrative and can be omitted, combined, and/or performed in an order other than that depicted and/or described; the numbering of depicted steps is merely for ease of reference and does not imply any particular ordering is necessary or preferred.

The functions and/or steps described herein can be embodied in computer-usable data and/or computer-executable instructions, executed by one or more computers and/or other devices to perform one or more functions described herein. Generally, such data and/or instructions include routines, programs, objects, components, data structures, or the like that perform particular tasks and/or implement particular data types when executed by one or more processors in a computer and/or other data-processing device. The computer-executable instructions can be stored on a computer-readable medium such as a hard disk, optical disk, removable storage media, solid-state memory, read-only memory (ROM), random-access memory (RAM), and/or the like. As will be appreciated, the functionality of such instructions can be combined and/or distributed as desired. In addition, the functionality can be embodied in whole or in part in firmware and/or hardware equivalents, such as integrated circuits, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and/or the like. Particular data structures can be used to more effectively implement one or more aspects of the disclosure, and such data structures are contemplated to be within the scope of computer-executable instructions and/or computer-usable data described herein.

Although not required, one of ordinary skill in the art will appreciate that various aspects described herein can be embodied as a method, system, apparatus, and/or one or more computer-readable media storing computer-executable instructions. Accordingly, aspects can take the form of an entirely hardware embodiment, an entirely software embodiment, an entirely firmware embodiment, and/or an embodiment combining software, hardware, and/or firmware aspects in any combination.

As described herein, the various methods and acts can be operative across one or more computing devices and/or networks. The functionality can be distributed in any manner or can be located in a single computing device (e.g., server, client computer, user device, and/or the like).

Claim 1:
A computer-implemented method comprising:
providing, by one or more computing devices (<NUM>, <NUM>, <NUM>, <NUM>), one or more machine learning (ML) models (<NUM>, <NUM>, <NUM>) configured to process imagery to identify images determined to be suitable for one or more different and distinct uses;
receiving, by the one or more computing devices (<NUM>, <NUM>, <NUM>, <NUM>), data describing imagery comprising a plurality of different and distinct frames for processing using the one or more ML models (<NUM>, <NUM>, <NUM>);
determining, by the one or more computing devices (<NUM>, <NUM>, <NUM>, <NUM>), based at least in part on the one or more ML models (<NUM>, <NUM>, <NUM>) and the data describing the imagery, and for each frame of the plurality of different and distinct frames, one or more scores for the frame, each score of the one or more scores indicating a determined measure of suitability of the frame with respect to a use of the one or more different and distinct uses; and
communicating, by the one or more computing devices (<NUM>, <NUM>, <NUM>, <NUM>), data indicating, for each frame of one or more frames of the plurality of different and distinct frames, at least one of the one or more scores for the frame;
wherein the plurality of different and distinct frames comprises a series of contemporaneously generated similar frames captured from a camera burst shot;
wherein the one or more scores are determined in real-time or near real-time and the one or more scores are determined based upon one or more quality metrics;
wherein communicating, by the one or more computing devices (<NUM>, <NUM>, <NUM>, <NUM>), data indicating, for each frame of one or more frames of the plurality of different and distinct frames, at least one of the one or more scores for the frame comprises selecting the frame determined to be suitable for the use from a buffer of the series of contemporaneously generated similar frames; and
providing a graphical user interface within a photograph management application (<NUM>), wherein the graphical user interface identifies the frame determined to be suitable for the use from the buffer of the series of contemporaneously generated similar frames.