Patent ID: 12211078

DETAILED DESCRIPTION

An e-commerce marketplace framework has been developed to synchronize app metadata (data about apps) across data centers (DCs) within an e-commerce marketplace. The e-commerce marketplace framework is, in one embodiment, built on an Asynchronous Web Server Framework (AWSF) of Zoho Corporation Pvt. Ltd. The AWSF provides a NIO-based HTTP Server with GET/POST support along with streaming of data (NIO stands for Non-blocking Input/Output). This framework supports heavy file uploads/downloads and both HTTPS and web socket protocols. The framework can handle asynchronous data flow across communication networks interconnecting the DCs.

FIG.1represents an implementation of a framework100that supports the synchronization of marketplace apps details across multiple marketplace DCs105. A key at upper left identifies symbols that illustrate network features and functions for storing and communicating information. There are multiple DCs105in different locations to service user requests across the globe, a primary DC105pin the US, at left inFIG.1, and secondary DCs105sin Europe (EU DC), China (CN DC), and India (IN DC). The “p” for “primary” and “s” for secondary are used throughout to distinguish elements associated with primary and secondary DCs105pand105s. A user, interacting with any of DCs105to download or interact with an app, might request to leave a review of the app. Reviews and ratings are treated as app metadata in association with the respective app and distributed across DCs105to be made available to global users of the e-commerce marketplace represented by framework100.

Each DC105, primary or secondary, includes a marketplace module110that employs a software agent115, or “agent module,” for synchronization of e.g. app metadata in real-time or near real-time to meet customer needs in a timely manner. In this context, a software agent, or just “agent,” is a computer program or program module that acts on behalf of another program or module in a relationship of agency. Software agents can transfer data between DCs105as needed. A module is a portion of a computer program that encapsulates code and data to implement a particular functionality in service of the larger program. A module has a logical interface that conveys messages to and from other modules.

Each marketplace DC105also includes computing resources to execute programs or program modules. In one scenario each marketplace DC105includes a server with one or more processors (hereafter just “processor”) with access to memory running an operating system (OS) that supports the server's basic functions, such as the scheduling of tasks, executing programs, and controlling peripherals. Programs or program modules execute on the computing resources of the local marketplace DC105. The OS can allocate logical address spaces in main memory to different programs or program modules executing on the processor or processors. In the example ofFIG.1, for each DC105a marketplace module110executing on a local processor or processors is allocated first logical addresses in main memory, while an agent module115is allocated second logical addresses distinct from the first. In another scenario a marketplace module110and its agent module115may execute on different server or host machines, in which case they may be allocated different physical address spaces and therefore execute in different physical address spaces. Each DC105also includes a database (DB)120with a database management system (DBMS) coupled to the associated marketplace module110. The DBMS, using well-known DBMS protocols, allows the marketplace module110to communicate with the local database120and manages synchronization of the collection of databases120across the distributed network of DCs105. Each database120may be physically remote from the computing resources employed in executing the corresponding agent module115and employ storage that is orders of magnitude slower than main memory. DBMS scheduling further reduces the relative speed performance of each database120. Communication and synchronization of data between marketplace modules110via corresponding agent modules115can thus be much faster and more efficient than via corresponding databases120.

Users of a marketplace DC105include product developers, sellers, and buyers. Developers and sellers can upload product descriptions and other product metadata. Buyers, e.g. via an Internet connection, can peruse product offerings (e.g. items for sale or apps on offer via the marketplace data centers105). Buyers interact with one of the marketplace modules110to purchase products and can read and submit feedback for distribution and storage as additional product metadata. This form of metadata is for publication and so can be synchronized among DCs105without consideration of privacy restrictions that may be in place for other information stored in databases120. The marketplace module110in one DC105can share product metadata with other DCs105in real-time (or near real-time) via the corresponding agent module115without having to involve the local DBMS or wait for the DBMS to perform its relatively slow and delayed synchronization process. Each agent module115supports fast, efficient, and asynchronous data flow using the software framework noted previously. The inclusion of agent modules115in the various DCs105thus makes product metadata, such as user reviews, rapidly available to users across the network of DCs105.

An agent module115of a particular DC105can receive requests from the co-located marketplace module110, or from an agent115of other DCs105. In the embodiment ofFIG.1, user metadata directed to primary DC105pis handled by primary agent115p, which messages the metadata to secondary DCs105sand ensures those secondary DCs105report successful reception of the updates. User metadata directed to one of secondary DCs105sis handled by that secondary DC105s. In one embodiment, the recipient secondary DC105ssends the metadata to primary DC105p, which then manages synchronization with the remaining secondary DCs105s.

User requests from different regions will reach agents115of the respective DCs105through their respective marketplace modules110. Primary agent115pperforms either a propagation or broadcast of data (e.g. app metadata) based on the source of the request. Each secondary agent115sreceives the data and also notifies primary agent115p. These functionalities of agents115help in the synchronization of metadata across DCs105in real-time or near real-time (for example when there are network transmission delays) to meet/serve customer needs in a timely manner. The flows of metadata across DCs105for synchronization can be secured using an RSA encrypted MD5 hash of the metadata. In some scenarios, for faster and more efficient data transfer, an agent115in coordination with its marketplace module110may directly access an associated DBMS/database or distributed file system (DFS).

The agent module in each DC can be implemented as a service in Zoho Marketplace grid. The details of each app that is listed in Zoho Marketplace persist in the structured data format and any file related to it persists in unstructured data format in all the DCs. The information about each app available in one DC should be available in all DCs.

The location where a given agent module115is deployed can determine the roles and actions associated with the agent. In framework100, primary agent module115psupports the following actions:Receives data related to users as well as user events from marketplace module110p.Forwards app metadata to marketplace module110p, though in one embodiment this step may await reports of successful completion from secondary agents115s.Checks the source of app metadata and:If from marketplace module110p(of primary DC105p), then primary agent115pinitiates a metadata broadcast to secondary DCs105s.If from a marketplace module110s(of one of the secondary DCs105s), then primary agent115pinitiates a metadata broadcast to all secondary DCs105sexcept the one from which the metadata arrived.

There can be more than one secondary agent, the number generally depending upon the number of DCs105. Secondary agents115sreceive metadata from local users, transfer such received metadata to primary agent115p, and receive metadata updates from primary agent115p. In framework100, secondary agents115ssupport the following functionalities.Receive data related to users as well as events from the marketplace module110sof the respective secondary DC105s.The details about a user's event will be treated as metadata and the metadata will be messaged to primary agent115p.Receive and store app metadata when primary agent115pperforms propagation.

FIG.2depicts framework100, including the same four DCs105, to illustrate metadata broadcast functionality of primary DC105pin accordance with one embodiment. Once a request with app metadata is made by a user in a region served by primary DC105p, primary agent115pinitiates a broadcast of the metadata to secondary DCs105sas an https/host request. Primary agent115puses a private key to encrypt the metadata, creates a message with the encrypted metadata as payload, and sends the message to secondary agents115s. Each secondary agent115suses a public key to decrypt the metadata and forwards the metadata to the marketplace module in a respective secondary DC. Successful decryption is part of a handshake that authenticates messages and verifies message sources.

Secondary agents115scan be updated sequentially, in parallel, or in a combination of the two. In some embodiments, secondary agents115sare arranged in a daisy-chain configuration, with each passing on a message to the next in some preset or communicated order. Messages reporting success or failure can similarly be communicated back to primary agent115pin various ways.

Successive requests with metadata for update are queued and processed sequentially to ensure synchronization across all DCs105. The agent115, primary or secondary, that first receives app metadata from a user assigns a unique identifier (UID) to the metadata. The assigned UID is generated based on time stamp, date, and marketplace-specific code assigned within the e-commerce marketplace. For example, a marketplace data center number/identifier can be assigned to a given marketplace DC105to identify and differentiate it from other marketplace data centers. The UID that is assigned to metadata allows the agent modules collectively to distinguish metadata updates. A UID accompanies each instance of metadata communications.

FIG.3depicts framework100, including the same primary and secondary DCs105pand105s, to illustrate the flow of metadata when a user event is reported from a region served by one of secondary DCs105s, a European DC in this example. Secondary agent115sof the EU DC105sgenerates and conveys an encrypted message notifying primary agent115pabout the event. Primary agent115pdecrypts the data using the public key to obtain the message payload (e.g. app metadata). Primary agent115palso initiates a propagation process to synchronize with secondary DCs115s.

FIG.4illustrates how primary agent115pverifies synchronization with a secondary agent115sand handles failure events. Messages that reach primary agent115p, e.g. via marketplace110p, are queued before they are acted upon. When a failure occurs during the synchronization process, the failed event is captured and cached in primary DC105p. A scheduler400in e.g. marketplace module fetches the details of failed events from a local cache410e.g. every 10 minutes and retries synchronization until data integrity is confirmed.

Example: User rating provided for an app.

FIG.5depicts four web pages500,505,510, and515summarizing ratings and reviews, and seeking user feedback for four respective and geographically distinct DCs, page500a user interface for a master (primary) DC in the US and pages505,510, and515user interfaces for three respective secondary DCs in each of China, Europe, and India. A user provides a rating and a review for an app (e.g. a four-star rating and a comment relating to a browser plug-in) available in a cloud marketplace distributed across multiple DCs. This user feedback is received via one DC but is made available to other users via all DCs so that it impacts in ratings and rating statistics520that appear on each of pages500,505,510, and515.

FIG.6graphically depicts a cloud marketplace600with a source agent605pof a primary DC610pconnected to destination agents of secondary DCs610svia a network615(e.g., the Internet) that employs the hypertext transfer protocol (HTTP) and public-key/private-key data security. User requests that update metadata can be conveyed to and between DCs610as URL-encoded Hypertext Transfer Protocol Secure (HTTPS) messages with metadata payloads formatted in JavaScript Object Notation (JSON). HTTPS is an extension of HTTP commonly used for secure communications over the Internet. The JSON format is commonly used for serializing and transmitting structured data over a network connection.

In general, the receiving agent605decides how to manage a given request. As the request is received at the primary DC610p, primary agent605pbroadcasts the metadata of the request to secondary agents605svia network615. To perform a secured data flow, the primary agent605pencrypts the metadata before broadcasting the encrypted JSON-format metadata to the secondary DCs610s. The secondary DCs610sparse their respective messages from the primary DC610pand decrypt the message payload to obtain the JSON-formatted metadata. Typically, image files are fetched by the marketplace modules from the respective DFS servers present in each DC610.

If a request for a metadata update arrives via a secondary DC610s, then the respective secondary agent605smessages the primary agent605p, which in turn broadcasts messages to the remaining secondary agents605s. In this example, the secondary EU DC610s, the middle of the three DCs610sat bottom, messages primary US DC610p, at top, encrypted metadata. Primary agent605pdecrypts the metadata and propagates the encrypted metadata to the remaining secondary DCs (all but the EU DC). If a request fails while syncing to another DC, the failure event is captured and stored in cache for scheduled reprocessing via the associated agents. Primary DC610pcan also assume failure for a secondary DC610suntil receipt of a message indicative of success.

FIG.7is a diagram depicting data flow for user requests broadcast to DCs within a cloud marketplace. A user interacting with a primary DC requests an update to be made to the metadata, the update comprising data700communicated in JSON format. The metadata is encrypted and URL encoded in the primary DC marketplace (702) before being sent to the primary DC agent. This action is depicted as a decision704because the primary DC marketplace awaits a response706from the DC agent reporting successful message communication or otherwise caches the data that failed to send (708) for a subsequent attempt. URL encoding is a well-known mechanism for encoding information in a Uniform Resource Identifier (URI).

The URL-encoded message received by the primary agent is decoded (710) and the JSON data extracted (715) at the primary DC agent. Either or both of the decoding and storing can be delayed until the primary agent receives confirmation of a successful update from all of the secondary agents. If the message payload has been decoded, the primary agent can sign (720), re-encode (725), and broadcast the message (730). When broadcast, the message is stored locally in a message queue awaiting reported success from the secondary DCs.

Communication infrastructure for broadcasting, including network components, is depicted as a cloud735. The action of directing the message to a first DC agent is depicted as a decision730because the primary DC agent awaits a response from each of the DC agents reporting successful message communication before reporting success to the primary DC marketplace. Taking just one DC agent (DC1 Agent) for illustrative purposes, the agent received the message seeking a metadata update (740), decodes the URL associated therewith (745), verifies message authenticity via a public/private key process (750), recovers the JSON data from the message (755), and updates the database associated with the local DC1 marketplace (760). The broadcast message should reach all the secondary DC agents, other than a secondary agent that is the source of initial message. When all of the secondary agents signal success, the primary DC removes that entry from the message queue. Data-integrity checks can be performed periodically to verify that the metadata matches among the distributed DCs.

FIG.8is a diagram depicting data flow for user requests and messages conveyed between a primary DC agent and a collection of secondary DC agents, including secondary DC agents DC1 and DC2. The steps are similar to those ofFIG.7, with like-labeled elements being the same or similar, for updates based on interactions with the DC1 marketplace. Likewise, a user interacting with secondary DC2 can issue a request that updates metadata. Secondary agent DC2 encrypts and encodes the message before propagating the message to the primary DC agent via e.g. cloud communication. The primary DC agent decrypts and decodes the message and notifies the primary DC marketplace to update. Once the primary DC marketplace is updated, the message is again encrypted, encoded, and broadcast to the remaining secondary DC agents as detailed previously until all the DCs, primary and secondary, are updated to include the new metadata. The secondary DC2 that initiated the metadata update can perform the update locally before or after awaiting direction from the primary DC agent.

Variations of these embodiments, and variations in usage of these embodiments, including separate or combined embodiments in which features are used separately or in any combination, will be obvious to those of ordinary skill in the art. Therefore, the spirit and scope of the appended claims should not be limited to the foregoing description. In U.S. applications, only those claims specifically reciting “means for” or “step for” should be construed in the manner required under 35 U.S.C. § 112(f).