Patent Description:
Various types of devices (e.g., routers, servers, and so forth) may operate in various types of environments (e.g., communication networks, datacenters, enterprises, home networks, and so forth).

<CIT> concerns a method and a system for centralized cluster management.

<CIT> concerns configuration of networks with a server cluster device.

Device management clustering is configured to enable a set of devices to be managed as a group. Device management clustering is configured to enable a set of devices to be managed as a group based on access to only one of the devices in the set of devices to be managed rather than based on access to all of the devices in the set of devices to be managed. Device management clustering is configured to enable a set of devices to be managed by a user (e.g., a human user, or a non-human user such as a device, a program, an application, or the like) as a group based on access by the user to only one of the devices in the set of devices to be managed (e.g., via login by the user to the device via an interface, such as a command-line interface (CLI) or other suitable interface, of the device) rather than based on access by the user to all of the devices in the set of devices to be managed (e.g., without a need for the user to login to any of the other devices in the set of devices to be managed). Device management clustering includes creating a device management cluster including a set of devices and performing device management for the set of devices based on the device management cluster.

Various possibilities for supporting device management clustering for routers (which also may be referred to as router management clustering) are presented. Router management clustering is configured to enable a set of routers to be managed by a user (e.g., a human user, or a non-human user such as a device, a program, an application, or the like) as a group. Router management clustering is configured to enable a set of routers to be managed by a user as a group based on login by the user to only one of the routers in the set of routers rather than to all of the routers in the set of routers to be managed. Router management clustering is configured to enable a set of routers to be managed by a user as a group via access to an interface (e.g., a CLI or other suitable interface) of only one of the routers in the set of routers rather than to all of the routers in the set of routers to be managed. Router management clustering includes creating a router management cluster including a set of routers and performing router management for the set of routers based on the router management cluster. Router management cluster creation may be performed dynamically (e.g., based on sharing of router cluster membership information between routers, such as by using extensions of existing protocols, using new protocols, or the like, as well as various combinations thereof), statically (e.g., via static configuration by a user via a management device), or the like, as well as various combinations thereof. Router management for the set of routers of the router management cluster may be performed via an interface (e.g., a CLI or other suitable interface) of one of the routers in the set of routers of the router management cluster. Router management for the set of routers of the router management cluster may be performed via an interface (e.g., a CLI or other suitable interface)of one of the routers in the set of routers of the router management cluster based on activation of a cluster management mode for the user in the interface for causing commands entered by the user to be applied to some or all of the other routers in the set of routers of the router management cluster (e.g., some or all of the routers based on one or more of explicit specification by the user via the interface, reachability of the other routers from the router being accessed, or the like, as well as various combinations thereof). In this manner, a user is able to control a set of routers based on direct access to only a single router in the set of routers without a need to explicitly access (e.g., log into or otherwise be authenticated to access) each of the routers in the set of routers. It will be appreciated, although primarily presented herein within the context of example embodiments for supporting device management clustering for routers, various example embodiments presented herein may be used or adapted for use in supporting device management clustering for various other types of devices (e.g., switches, servers, or the like). It will be appreciated that these and various other example embodiments and advantages or potential advantages of supporting router management clustering may be further understood by way of reference to the various figures, which are discussed further below.

<FIG> depicts an example embodiment of a communication system including a communication network and a management device configured to manage the communication network.

The communication system <NUM>, as indicated above, includes a communication network <NUM> and a management device <NUM> configured to manage the communication network <NUM>.

The communication network <NUM> may include any network which may be managed by a management device <NUM>. For example, the communication network <NUM> may be an operator network, an enterprise network, or the like. For example, the communication network <NUM> may be an access network, a core network, a backhaul network, a datacenter network, or the like. For example, the communication network <NUM> may be a physical communication network, a virtualized communication network (e.g., virtualized based on network function virtualization (NFV) techniques or other suitable virtualization techniques), or the like. It will be appreciated that, although primarily presented herein within the context of specific types of communication networks, various other types of communication networks may be supported.

The communication network <NUM> includes a set of routers <NUM>-<NUM> to <NUM>-N (collectively, routers <NUM>). The routers <NUM> may be configured to provide various communication services for supporting communications within communication network <NUM>. The routers <NUM> may be arranged in various network topologies, which may depend on the type of communication network <NUM> in which the routers <NUM> are disposed. For example, the routers <NUM> may be arranged in a mesh topology (e.g., in a network operator backhaul network), in a spine-and-leaf topology (e.g., in a datacenter network), or the like. The routers <NUM> may be configured to be managed by the management device <NUM>. It will be appreciated that, although primarily presented herein as including specific types devices (illustratively, routers <NUM>), communication network <NUM> may include various other types of devices (e.g., switches, hubs, bridges, or the like, as well as various combinations thereof).

The management device <NUM> may be configured to support management of the communication network <NUM>. The management device <NUM> may be configured to support management of the communication network <NUM> for various management purposes, at various management scales, or the like, as well as various combinations thereof. For example, the management device <NUM> may be configured to support configuration functions, monitoring functions, maintenance functions, or the like, as well as various combinations thereof. For example, the management device <NUM> may be configured to support service-level management (e.g., as a service management system or other similar system), network-level management (e.g., as a network management system (NMS) or other similar system), element-level management (e.g., as an element management system (EMS) or other similar system), or the like, as well as various combinations thereof. The management device <NUM> may be one of various types of devices which may be used to support management functions for communication network <NUM>, such as a workstation of a management system configured to support management of the routers <NUM>, a user device (e.g., a laptop, a tablet, a smartphone, or the like) configured to provide remote management access to the routers <NUM>, a local console connected to one or more of the routers <NUM>, or the like. It will be appreciated that the management device <NUM> may be configured to support various other management capabilities for managing the communication network <NUM>.

The management device <NUM> is configured to manage the routers <NUM> of communication network <NUM>. The management device <NUM> may support various capabilities which enable the management system <NUM> to access and manage routers <NUM> and, similarly, routers <NUM> may support various capabilities which enable access and management by the management device <NUM>. For example, such capabilities may include management interfaces (e.g., command line interfaces (CLIs), graphical user interfaces (GUIs), or the like), management applications (e.g., router configuration applications, router status retrieval applications, or the like), management elements (e.g., devices, modules, systems, sub-systems or the like), communications capabilities (e.g., inter-process communications capabilities such as remote procedure calls (RPCs), protocols such as telnet or secure shell (SSH), or the like), or the like, as well as various combinations thereof. It will be appreciated that these capabilities may be considered to be represented in <FIG> by the management elements <NUM>-<NUM> - <NUM>-N on routers <NUM>-<NUM> - <NUM>-N, respectively, and the management element <NUM> on management device <NUM>.

For example, the communication network <NUM> may be configured to support router management clustering. For example, a router <NUM> of the communication network <NUM> and the management device <NUM> may be configured to, and may be configured to cooperate in order to, provide router management clustering. It is noted that various possibilities for creation of a router management cluster for use in router management are presented with respect to <FIG> and that various example embodiments for use of a router management cluster in router management are presented with respect <FIG>.

It will be appreciated that, although primarily presented with respect to specific types, numbers, and arrangements of elements, communication system <NUM> may include various other types, numbers and arrangements of elements.

<FIG> depicts a communication system configured to support creation of a router management cluster for use in router management.

The communication system <NUM> includes a communication network <NUM> including a set of routers <NUM>-<NUM> - <NUM>-<NUM> (collectively, routers <NUM>) and also includes a management device <NUM>. It will be appreciated that the communication system <NUM> may be based on and, thus, similar to the communication system <NUM> of <FIG>.

The communication system <NUM>, as indicated above, is configured to support creation of a router management cluster for use in router management. As illustrated in <FIG>, a cluster of routers <NUM>, which is denoted as router cluster <NUM>, is created. The router cluster <NUM> includes routers <NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>, and <NUM>-<NUM>. The router cluster <NUM> may be defined in various ways and may be created within the communication network <NUM> in various ways.

The router cluster <NUM>, as indicated above, may be defined in various ways. The router cluster <NUM> may be defined by the router operator (e.g., the network operator of communication network <NUM>). The router cluster <NUM> may be defined based on various types of information, such as based on the context of the communication network <NUM> (e.g., Enterprise network, datacenter network, or the like), based on the network topology of the communication network <NUM> (e.g., mesh, spine-and-leaf, or the like), based on the roles of the routers <NUM>, based on the types of management functions to be supported, or the like, as well as various combinations thereof). The router cluster <NUM> may be defined manually or automatically. The router cluster <NUM>, after being defined, may be identified using a router cluster identity, which may be in the form of a cluster identifier for the router cluster <NUM>, a cluster domain for the router cluster <NUM> (e.g., a domain name, such as a fully qualified domain name (FQDN), or a hostname), or the like, as well as various combinations thereof.

The router cluster <NUM>, as indicated above, may be created within the communication network <NUM> in various ways. The creation of the router cluster <NUM> includes configuration of each of the routers <NUM> in the router cluster <NUM> with router cluster membership information so that each router <NUM> of the router cluster <NUM> knows that it belongs to the router cluster <NUM> (referred to herein as cluster identity information) and knows the router cluster membership of the router cluster <NUM> to which it belongs (referred to herein as cluster membership information). The routers <NUM> that belong to the router cluster <NUM> may be configured to be members of the router cluster <NUM> (e.g., configured with their cluster identity information) in various ways, may obtain the router cluster membership information for the router cluster <NUM> in various ways (e.g., based on various rules, protocols, or the like), may maintain the router cluster membership information for the router cluster <NUM> in various ways (e.g., using various formats, tables, or the like), and so forth. The creation of the router cluster <NUM>, as indicated further below, may be performed dynamically (e.g., using various protocols, such as Zero Touch Provisioning (ZTP), Link Layer Discovery Protocol (LLDP), or the like), statically (e.g., via CLI without using dynamic protocols), or using a combination thereof.

The creation of the router cluster <NUM>, as indicated above, includes configuration of each of the routers <NUM> in the router cluster <NUM> with cluster identity information so that each router <NUM> of the router cluster <NUM> knows that it belongs to the router cluster <NUM>. The configuration of each of the routers <NUM> in the router cluster <NUM> with cluster identity information so that each router <NUM> of the router cluster <NUM> knows that it belongs to the router cluster <NUM> may be performed dynamically (e.g., using ZTP or other suitable automated provisioning protocols or mechanisms), statically (e.g., via CLI without using dynamic protocols), or a combination thereof.

The creation of the router cluster <NUM>, as indicated above, includes configuration of each of the routers <NUM> in the router cluster <NUM> with cluster membership information so that each router <NUM> of the router cluster <NUM> knows the router cluster membership of the router cluster <NUM> to which it belongs. The configuration of each of the routers <NUM> in the router cluster <NUM> with cluster membership information so that each router <NUM> of the router cluster <NUM> knows the router cluster membership of the router cluster <NUM> to which it belongs may be performed dynamically (e.g., using LLDP or other suitable automated discovery protocols or mechanisms), statically (e.g., via CLI without using dynamic protocols), or a combination thereof.

Configuration of each of the routers <NUM> in the router cluster <NUM> with router cluster membership information of the router cluster <NUM> may be based on use of a dynamic protocol to distribute the router cluster membership information of the router cluster <NUM> to routers <NUM> of the router cluster <NUM>. A router <NUM>, after learning that it belongs to router cluster <NUM>, can start advertising its membership within the router cluster <NUM> to neighboring routers <NUM> of the communication network <NUM> (e.g., to each of its links). As a result, the routers <NUM> learn the cluster identity information of their adjacent neighbors. As routers <NUM> receive cluster identity information from their adjacent neighbors, the routers <NUM> identify which of the adjacent neighbors belong to the router cluster <NUM> and store cluster membership information received from adjacent neighbors that belong to the router cluster <NUM> while ignoring cluster membership information received from adjacent neighbors that do not belong to the router cluster <NUM> (e.g., without storing or further propagating it). The routers <NUM> also continue to advertise router cluster membership information to their adjacent neighbors, advertising not only their own cluster identity information but the cluster membership information received from their adjacent neighbors that belong to the router cluster <NUM> (i.e., the full set of cluster membership information of the router cluster <NUM> that has been discovered by that router <NUM> up to that point). In this manner, the full cluster membership information of the router cluster <NUM> is eventually propagated to and stored by each of the routers <NUM> that are members of the router cluster <NUM>. The dynamic protocol may be a link layer protocol that is configured to support advertising and discovery of link layer information, such as LLDP (e.g., LLDP may be extended to support advertisement and discovery of router cluster membership information by the routers <NUM>) or the like.

Configuration of each of the routers <NUM> in the router cluster <NUM> with router cluster membership information of the router cluster <NUM> may be based on use of static configuration capabilities to configure the router cluster membership information of the router cluster <NUM> onto the routers <NUM> that belong to the router cluster <NUM>. The configuration of the router cluster membership information of the router cluster <NUM> onto the routers <NUM> that belong to the router cluster <NUM> may be performed via the CLIs of the routers <NUM>. The configuration of the router cluster membership information of the router cluster <NUM> onto the routers <NUM> that belong to the router cluster <NUM> may be performed via the CLIs of the routers <NUM> based on access to the CLIs of the routers <NUM> from the management device <NUM>. It will be appreciated that configuration of each of the routers <NUM> in the router cluster <NUM> with router cluster membership information of the router cluster <NUM> may be based on use of various other types of static configuration capabilities capable of supporting configuration of the router cluster membership information of the router cluster <NUM> onto the routers <NUM> that belong to the router cluster <NUM>.

The router cluster membership information that is maintained by the routers <NUM> of the router cluster <NUM> may include various types of information which may be used by the routers <NUM> of the router cluster <NUM> in supporting cluster level management of the routers <NUM> of the router cluster <NUM>. The router cluster membership information that is maintained by the routers <NUM> of the router cluster <NUM> may include, for each of the routers <NUM> that is a member of the router cluster <NUM>, router identification information (e.g., a router name, a router identifier, or the like), router reachability information (e.g., a reachable IP address, a port to be used, or the like), or the like, as well as various combinations thereof. It will be appreciated that the router cluster membership information that is maintained by the routers <NUM> of the router cluster <NUM> may include various other types of information which may be used by the routers <NUM> of the router cluster <NUM> in supporting cluster level management of the routers <NUM> of the router cluster <NUM>.

The router cluster membership information that is maintained by the routers <NUM> of the router cluster <NUM> may be maintained and managed by the routers <NUM> in various ways. The router cluster membership information that is maintained by the routers <NUM> of the router cluster <NUM> may be maintained using one or more database tables or other storage devices. The router cluster membership information that is maintained by the routers <NUM> of the router cluster <NUM> may be maintained based on use of a management server subsystem, a cluster management subsystem, and an inter-process communication subsystem that is configured to support communications between the management server subsystem and the cluster management subsystem. The management server subsystem may be configured to hold configuration information for other subsystems and to provide a YANG model for operation of the router. The management server subsystem may be configured to read the router cluster management information and provide it to the cluster management subsystem for storage and management of the router cluster management information (e.g., the cluster management subsystem may hold the cluster domain name of the router cluster <NUM> and members of the router cluster <NUM> (e.g., router names of the routers, reachable IP addresses of the routers, or the like, as well as various combinations thereof) for use in performing router management for the router cluster <NUM>. It will be appreciated that the router cluster membership information that is maintained by the routers <NUM> of the router cluster <NUM> may be maintained and managed by the routers <NUM> in various other ways.

It will be appreciated that the communication system <NUM> may be configured to provide various other functions supporting creation of a router management cluster for use in router management. A method for use by a router in supporting creation of a router management cluster for use in router management is presented with respect to <FIG>.

<FIG> depicts a method for use by a router in supporting creation of a router management cluster for use in router management. It will be appreciated that, although primarily presented as being performed serially, at least a portion of the functions of method <NUM> may be performed contemporaneously or in a different order than as presented with respect to <FIG>. At block <NUM>, method <NUM> begins. At block <NUM>, maintain, by a router of a router cluster including a set of routers, router cluster membership information indicative of router membership of the router cluster. At block <NUM>, support, by the router, communication with one or more neighboring routers to support distribution of the router cluster membership information indicative of router membership of the router cluster. At block <NUM>, method <NUM> ends. It will be appreciated that various functions presented with respect to <FIG> may be implemented within the context of method <NUM> of <FIG>.

<FIG> depicts a communication system configured to support use of a router management cluster in router management.

The communication system <NUM> includes a set of routers <NUM>-<NUM> - <NUM>-<NUM> (collectively, routers <NUM>) and an Authentication, Authorization, and Accounting (AAA) server <NUM>. The routers <NUM> are organized as a router cluster and AAA server <NUM> is configured to perform AAA functions for the routers <NUM> of the router cluster. It will be appreciated that the routers <NUM> of the router cluster may have a topology different than that depicted in <FIG>, but that the routers <NUM> of the router cluster are depicted in this arrangement, because a user (e.g., a human user, or a non-human user such as a device, a program, an application, or the like) is using the router <NUM>-<NUM> to control each of the routers <NUM>-<NUM> - <NUM>-<NUM> of the router cluster, to illustrate management of the router cluster by the user based on access by the user only to router <NUM>-<NUM> without a need for the user to directly access routers <NUM>-<NUM> - <NUM>-<NUM> in order to control routers <NUM>-<NUM> - <NUM>-<NUM>. It will be appreciated that the router cluster may be created as presented with respect to <FIG>. It will be appreciated that the router cluster may include fewer or more routers <NUM> and that any of the routers <NUM> of the router cluster could be accessed by the user in order to control each of the routers <NUM> of the router cluster. It also will be appreciated that the communication network within which the routers <NUM> are deployed may include other routers which are not part of the router cluster (which have been omitted for purposes of clarity).

As presented in <FIG>, the user is able to manage the routers <NUM> of the router cluster based on access to only router <NUM>-<NUM> of the router cluster without a need to directly access any of routers <NUM>-<NUM> - <NUM>-<NUM> of the router cluster. The router <NUM>-<NUM> supports a number of router access capabilities <NUM> via which the user may access the router <NUM> locally (illustratively, denoted as console <NUM>-C) or remotely (illustratively, denoted as telnet <NUM>-T and SSH <NUM>-S) and, thus, access various router service functions of the router <NUM>-<NUM> (e.g., traffic routing functions, traffic monitoring functions, or the like). The router <NUM>-<NUM> supports a command-line interface (CLI) <NUM> via which the user may interact with the router <NUM>-<NUM> for performing various functions on the router <NUM>-<NUM> (including high level functions at the router system level as well as router service functions at the router service level). The user may access the CLI <NUM> via any of the router access capabilities <NUM>. The router <NUM>-<NUM> also supports a cluster management element <NUM> configured to provide various cluster management functions presented herein as being supported by the router <NUM>-<NUM> (e.g., detecting that the CLI <NUM> is operating in a cluster management mode, sending commands to other routers <NUM>-<NUM>-<NUM>-<NUM> of the router cluster, collecting command responses from other routers <NUM>-<NUM>-<NUM>-<NUM> of the router cluster, and so forth, as discussed further below). It will be appreciated that the router <NUM>-<NUM> may be configured to run a Linux system supporting router access capabilities <NUM> and supporting a router Linux system which is configured to provide the router service functions based on access to the router service functions via the CLI <NUM>. The AAA server <NUM> is configured to perform authentication, authorization, and accounting functions for router <NUM>-<NUM> (as well as routers <NUM>-<NUM> - <NUM>-<NUM> based on control of these routers via router <NUM>-<NUM> based on router management clustering). The communication between the router <NUM>-<NUM> and the AAA server <NUM> may be based on use of any suitable AAA protocol (e.g., Remote Authentication Dial-In User Service (RADIUS), Terminal Access Controller Access-Control System (TACACS), or the like).

As presented in <FIG>, the routers <NUM> are configured to support router cluster management functions enabling the user to manage the routers <NUM> of the router cluster based on access by the user to only router <NUM>-<NUM> of the router cluster without a need for the user to directly access any of routers <NUM>-<NUM> - <NUM>-<NUM> of the router cluster. It will be appreciated that each of the routers <NUM> also could be controlled if the user accessed any of the routers <NUM>-<NUM> - <NUM>-<NUM> of the router cluster.

As presented in <FIG>, the user is able to manage the router <NUM>-<NUM> without use of router cluster management. The user accesses the CLI <NUM> of the router <NUM> via console <NUM>-C, telnet <NUM>-T, or SSH <NUM>-S. The user is authenticated by the AAA server <NUM> before being permitted to execute commands on router <NUM>-<NUM> via the CLI <NUM>. The user enters user authentication credentials (e.g., a user ID and associated password) via the CLI <NUM> and the user authentication credentials are sent to AAA server <NUM>, which determines whether the user should be given access to the router <NUM>-<NUM>. If the AAA server <NUM> indicates to the router <NUM>-<NUM> that the user authentication credentials are invalid, the user is denied access to the router <NUM>-<NUM>. If the AAA server <NUM> indicates to the router <NUM>-<NUM> that the user authentication credentials are valid, the user is given access to the router <NUM>-<NUM> (including access to operate on the router service functions of the router <NUM>-<NUM>). If the user is successfully authenticated, a user profile may be created for the user, where the user profile of the user includes the authorization level of the user to operate on the router cluster. The user, after being authenticated by the AAA server <NUM> for operating on the router <NUM>-<NUM> via the CLI <NUM>, may enter commands via the CLI <NUM>. The commands entered by the user via the CLI <NUM> are authorized and accounted by the AAA server <NUM>. The router <NUM>-<NUM> sends a command entered by the user via the CLI <NUM> to the AAA server <NUM> for authorization and accounting. The AAA server <NUM> receives the command from the router <NUM>-<NUM> and performs authorization and accounting for the command. For authorization, the AAA server <NUM> determines whether execution of the command on the router <NUM>-<NUM> is authorized and provides to the router <NUM>-<NUM> a response indicative as to whether execution of the command on the router <NUM>-<NUM> is authorized. The authorization of the command entered by the user via the CLI <NUM> may be based on command authorization information provided by the router <NUM> to the AAA server <NUM> for use in determining whether execution of the command on the router <NUM> is authorized (e.g., the command, a user identifier of user, a user session identifier of a user session of user, a terminal type via which the user accessed the CLI <NUM> (e.g., console, telnet, SSH, or the like), or the like, as well as various combinations thereof). The router <NUM>-<NUM>, based on a response from the AAA server <NUM> that is indicative that the user is authorized to execute the command, executes the command on the router <NUM> and may provide an indication to the user via the CLI <NUM> that the command was executed (e.g., a result of execution of the command). The router <NUM>-<NUM>, based on a response from the AAA server <NUM> that is indicative that the user is not authorized to execute a command, prevents execution of the command on the router <NUM>-<NUM> and may provide an indication to the user via the CLI <NUM> that the command was not executed due to lack of authorization. The AAA server <NUM> also performs accounting for the command entered by the user via the CLI <NUM> for maintaining a record of user activity on the router <NUM>-<NUM> (e.g., commands entered, commands executed after being authorized, commands not execute after authorization was denied, and so forth), which may be used for various activity tracking purposes (e.g., auditing, analysis, or the like). The router <NUM>-<NUM> also may perform accounting for the command entered by the user via the CLI <NUM> for maintaining a local record of user activity on the router <NUM>-<NUM>.

As presented in <FIG>, the user, after accessing router <NUM>-<NUM> of the router cluster, may attempt to manage the routers <NUM> of the router cluster as a group based on use of router cluster management. The user, after being authenticated to access and operate on the router <NUM>-<NUM> (e.g., based on interaction between router <NUM>-<NUM> and AAA server <NUM> for authentication of the user, as discussed above), may attempt to put the CLI <NUM> of router <NUM>-<NUM> into a router cluster management mode. The user may attempt to put the CLI <NUM> of the router <NUM>-<NUM> into the router cluster management mode by entering a command via the CLI <NUM>. For example, the user may put the CLI <NUM> of the router <NUM>-<NUM> into the router cluster management mode by typing "cluster management" via the CLI <NUM>. The router <NUM>-<NUM>, based on a detection of an attempt by the user to put the CLI <NUM> in the cluster management mode, checks the authorization of the user to put the CLI <NUM> into the cluster management mode. The authorization of the user to put the CLI <NUM> into the cluster management mode may be checked based on a user profile of the user (e.g., a user profile created for the user when the user was authenticated to access the router <NUM>-<NUM>). The authorization of the user to put the CLI <NUM> into the cluster management mode may be checked locally at the router <NUM>-<NUM> or may be checked at the AAA server <NUM> based on interaction between the router <NUM>-<NUM> and the AAA server <NUM>. If the user is not authorized to put the CLI <NUM> into the cluster management mode, then the CLI <NUM> is prevented from entering the cluster management mode and any commands entered by the user via the CLI <NUM> are only executed locally on the router <NUM>-<NUM> that the user has accessed. If the user is authorized to put the CLI <NUM> into the cluster management mode, then the CLI <NUM> enters the cluster management mode such that the user may control the routers <NUM> of the router cluster as a group (e.g., commands entered by the user via the CLI <NUM> may be executed locally on router <NUM>-<NUM> as well as on some or all of the other routers <NUM>-<NUM> - <NUM>-<NUM> of the router cluster as discussed further below).

As presented in <FIG>, the user, after accessing router <NUM>-<NUM> of the router cluster and putting the CLI <NUM> of the router <NUM>-<NUM> into cluster management mode, is able to manage routers <NUM> of the router cluster as a group based on use of router cluster management.

The router <NUM>-<NUM>, based on a determination that the user has put the CLI <NUM> of the router <NUM>-<NUM> into the router cluster management mode, activates router cluster management for the router cluster to which router <NUM>-<NUM> belongs. The router <NUM>-<NUM> based on a determination that the user enters a command via the CLI <NUM> while the CLI <NUM> is in the router cluster management mode, executes the command on the router <NUM>-<NUM> and sends the command to at least a portion of the other routers <NUM>-<NUM> - <NUM>-<NUM> of the router cluster for execution (illustratively, in the example of <FIG>, the command is sent to each of the other routers <NUM>-<NUM> - <NUM>-<NUM> of the router cluster), determines a command result based on a local command result determined by the router <NUM>-<NUM> based on execution of the command on the router <NUM>-<NUM> and based on command responses received by the router <NUM>-<NUM> based on execution of the command on ones of the other routers <NUM>-<NUM>-<NUM>-<NUM> of the router cluster, and presents the command result to the user via the CLI <NUM> of the router <NUM>-<NUM>.

The router <NUM>-<NUM>, based on a determination that the user enters a command via the CLI <NUM> while the CLI <NUM> is in the router cluster management mode, executes the command on the router <NUM>-<NUM> and determines a local command result based on execution of the command on the router <NUM>-<NUM>.

The router <NUM>-<NUM>, based on a determination that the user enters a command via the CLI <NUM> while the CLI <NUM> is in the router cluster management mode, sends the command to at least a portion of the other routers <NUM>-<NUM> - <NUM>-<NUM> of the router cluster for execution. The router <NUM>-<NUM>, based on a determination that the user enters a command via the CLI <NUM> while the CLI <NUM> is in the router cluster management mode, identifies the other routers <NUM>-<NUM>-<NUM>-<NUM> that are part of the router cluster, determines which of the other routers <NUM>-<NUM> - <NUM>-<NUM> of the router cluster to which the command is to be sent, and sends the command to those other routers <NUM>-<NUM> - <NUM>-<NUM> of the router cluster to which the command is to be sent. It will be appreciated that the command may be sent to all of the other routers <NUM>-<NUM> - <NUM>-<NUM> of the router cluster or to a subset of the other routers <NUM>-<NUM> - <NUM>-<NUM> of the router cluster.

The router <NUM>-<NUM> may identify the other routers <NUM>-<NUM>-<NUM>-<NUM> of the router cluster in various ways. For example, the router <NUM>-<NUM> may identify the other routers <NUM>-<NUM>-<NUM>-<NUM> of the router cluster based on router cluster membership information maintained at the router <NUM>-<NUM> (e.g., as presented with respect to <FIG>).

The router <NUM>-<NUM> may determine which of the other routers <NUM>-<NUM>-<NUM>-<NUM> of the router cluster to which the command is to be sent in various ways. For example, the router <NUM>-<NUM> may determine which of the other routers <NUM>-<NUM>-<NUM>-<NUM> of the router cluster to which the command is to be sent based on the command entered by the user (e.g., the user may specify that the command is intended for all of the routers <NUM> of the router cluster, may identify specific ones of the routers <NUM> of the router cluster for which the command is intended, or the like), based on status information associated with the routers <NUM> of the router cluster (e.g., the command is only provided to other routers <NUM> of the router cluster that are reachable at the time that the command is entered via the CLI <NUM>), or the like, as well as various combinations thereof.

The router <NUM>-<NUM> may send the command to those other routers <NUM>-<NUM> - <NUM>-<NUM> of the router cluster to which the command is to be sent in various ways. For example, router <NUM>-<NUM> may send the command to those other routers <NUM>-<NUM> - <NUM>-<NUM> of the router cluster to which the command is to be sent based on use of remote procedure calls from the router <NUM>-<NUM> to each of the other routers <NUM>-<NUM> - <NUM>-<NUM> of the router cluster to which the command is to be sent. It will be appreciated that the remote procedure calls may be based on gRPC or other suitable remote procedure call protocols.

The other routers <NUM>-<NUM> - <NUM>-<NUM> of the router cluster to which the command is sent by the router <NUM>-<NUM> each receive the command from the router <NUM>-<NUM>, execute the command, determine a local command result based on execution of the command, and send the local command result to the router <NUM>-<NUM> in response to the command.

The router <NUM>-<NUM> determines a command result for the command and presents the command result to the user via the CLI <NUM> of the router <NUM>-<NUM>. The router <NUM>-<NUM> determines the command result based on the local command result determined at the router <NUM>-<NUM> based on execution of the command at the router <NUM>-<NUM> and based on the local command results received from the other routers <NUM>-<NUM> - <NUM>-<NUM> of the router cluster to which the command was sent based on execution of the command on the other routers <NUM>-<NUM> - <NUM>-<NUM> of the router cluster to which the command was sent. The command result may be determined based on a concatenation of the local command results of the routers <NUM> of the router cluster on which the command was executed. It will be appreciated that the command result for the command may depend on the type of command that was executed (e.g., concatenated query results where the command was a query, concatenated route configuration results where the command was a route configuration command, or the like).

The router <NUM>-<NUM> may perform authorization and accounting for commands entered via the CLI <NUM> both while the CLI is not operating in the router cluster management mode as well as while the CLI <NUM> is operating in the router cluster management mode. In this manner, the commands entered by the user via the CLI <NUM>, including commands operating at the router cluster level, are authorized and accounted by the AAA server <NUM>. The router <NUM>-<NUM> sends a command entered by the user via the CLI <NUM> to the AAA server <NUM> for authorization and accounting. The AAA server <NUM> receives the command from the router <NUM>-<NUM> and performs authorization and accounting for the command. For authorization of a command entered via the CLI <NUM> while the CLI <NUM> is operating in the cluster management mode, the AAA server <NUM> determines whether execution of the command by the user is authorized and provides to the router <NUM>-<NUM> a response indicative as to whether execution of the command by the user is authorized. The authorization of the command entered by the user via the CLI <NUM> may be based on command authorization information provided by the router <NUM> to the AAA server <NUM> for use in determining whether execution of the command on the router <NUM> is authorized (e.g., the command, a user identifier of user, a user session identifier of a user session of user, a terminal type via which the user accessed the CLI <NUM> (e.g., console, telnet, SSH, or the like), or the like, as well as various combinations thereof). The router <NUM>-<NUM>, based on a response from the AAA server <NUM> that is indicative that the user is authorized to execute the command, initiates executes the command for the router cluster. The router <NUM>-<NUM>, based on a response from the AAA server <NUM> that is indicative that the user is not authorized to execute a command, prevents execution of the command for the router cluster and may provide an indication to the user via the CLI <NUM> that the command was not executed due to lack of authorization. The AAA server <NUM> also performs accounting for the command entered by the user via the CLI <NUM> for maintaining a record of user activity on the router <NUM>-<NUM> and user activity in operating on the router cluster via the router <NUM>-<NUM> (e.g., commands entered, commands executed after being authorized, commands not execute after authorization was denied, and so forth), which may be used for various activity tracking purposes (e.g., auditing, analysis, or the like). The router <NUM>-<NUM> also may perform accounting for the command entered by the user via the CLI <NUM> for maintaining a local record of user activity on the router <NUM>-<NUM> and user activity in operating on the router cluster via the router <NUM>-<NUM>.

It will be appreciated that the routers <NUM> of the router cluster may be configured to support various other functions for supporting router cluster management.

It will be appreciated that, although primarily presented herein within the context of supporting device management clustering for a particular type of device (namely, a router), various possibilities for supporting device management clustering for a router that are presented herein may be used or adapted for use in supporting device management clustering for other types of devices (e.g., switches, servers, or any other devices which may be grouped for various purposes). As such, as discussed further below, various more general embodiments are further presented with respect to <FIG>.

<FIG> depicts a method for use by a router in supporting router management for a router management cluster. It will be appreciated that, although primarily presented as being performed serially, at least a portion of the functions of method <NUM> may be performed contemporaneously or in a different order than as presented with respect to <FIG>. At block <NUM>, method <NUM> begins. At block <NUM>, execute, at a router of a router cluster, a command entered via an interfaceof the router. At block <NUM>, send, by the router toward one or more other routers of the router cluster based on the interface of the router operating in a cluster management mode, the command entered via the interface of the router. At block <NUM>, method <NUM> ends. It will be appreciated that various functions presented with respect to <FIG> may be implemented within the context of method <NUM> of <FIG>.

<FIG> depicts a method for use by a router in supporting router management for a router management cluster. It will be appreciated that, although primarily presented as being performed serially, at least a portion of the functions of method <NUM> may be performed contemporaneously or in a different order than as presented with respect to <FIG>. At block <NUM>, method <NUM> begins. At block <NUM>, detect, by a first router of a router cluster, a command received via an interface of the first router of the router cluster. At block <NUM>, send, by the first router of the router cluster toward a second router of the router cluster, the command received via the interface of the first router of the router cluster. At block <NUM>, method <NUM> ends. It will be appreciated that various functions presented with respect to <FIG> may be implemented within the context of method <NUM> of <FIG>.

<FIG> depicts a method for use by a router in supporting router management for a router management cluster. It will be appreciated that, although primarily presented as being performed serially, at least a portion of the functions of method <NUM> may be performed contemporaneously or in a different order than as presented with respect to <FIG>. At block <NUM>, method <NUM> begins. At block <NUM>, receive, at a first router of a router cluster from a second router of the router cluster, a command received via an interface of the second router of the router cluster. At block <NUM>, execute, at the first router of the router cluster, the command received via the interface of the second router of the router cluster. At block <NUM>, method <NUM> ends. It will be appreciated that various functions presented with respect to <FIG> may be implemented within the context of method <NUM> of <FIG>.

It will be appreciated that, although primarily presented herein within the context of use of embodiments of router management clustering within a particular type of communication network (e.g., a network operator network) having a particular network topology (e.g., mesh), various embodiments of router management clustering may be used within various other types of communication networks (e.g., Enterprise networks, datacenter networks, or the like) having various other network topologies (e.g., tree, spine-and-leaf, or the like). An example illustrating use of router management clustering within a datacenter communication network using a spine-and-leaf topology is presented with respect to <FIG>.

<FIG> depicts a communication network configured to use router management clustering for router management. The communication network <NUM> is a datacenter communication network having a hierarchical spine-and-leaf router topology supporting communications of a set of underlying servers. The spine-and-leaf router topology of includes three layers of routers, including a first layer (denoted as servers T0-<NUM> through T0-<NUM>) connected to the servers, a second layer (denoted as servers T1-<NUM> through T1-<NUM>) in full mesh connectivity with the first layer, and a third layer (denoted as servers T2-<NUM>-<NUM> through T2-<NUM>-<NUM>) in full mesh connectivity with the second layer. The spine-and-leaf router topology has been configured such that the routers at the second layer of the spine-and-leaf router topology are associated as part of a first router cluster (and, thus, may be controlled as a group based on access to only one of those routers) and, similarly, such that the routers at the third layer of the spine-and-leaf router topology are associated as part of a second router cluster (and, thus, may be controlled as a group based on access to only one of those routers). It will be appreciated that use of router clustering to control a group of routers at a particular hierarchical level of such a spine-and-leaf topology may provide significant advantages given that operations performed on the routers at any given layer of such a such spine-and-leaf topology may be similar given the similar contexts of the routers at any given layer of the such spine-and-leaf topology. It also will be appreciated that such advantages may be realized for other topologies and other types of communication networks based on organization of routers into router clusters based on similarity of routers (e.g., in terms of network context, functions supported or performed, or the like, as well as various combinations thereof).

It will be appreciated, although primarily presented herein within the context of example embodiments for supporting device management clustering for routers, various example embodiments presented herein may be used or adapted for use in supporting device management clustering for various other types of devices (e.g., switches, servers, or the like). Accordingly, various references herein to routers and associated router-related terms may be read more generally, for at least some example embodiments, as being references to devices (e.g., routers, switches, servers, or the like) and associated device-related terms.

The device management clustering may provide various advantages or potential advantages. For example, the device management clustering may enable a set of multiple devices to be controlled by a user based on authentication of the user to access only one of the devices in the set of multiple devices, thereby enabling the user to control other devices in the set of multiple devices without a need to explicitly log into those other devices in order to be authenticated to control those other devices. For example, the device management clustering may be considered to provide a virtual device concept at the device interface level (e.g., text-based interface level, such as at a CLI level) since access to a single interface of a single device enables operations on a set of multiple devices including that single device that is accessed and other devices that are part of the same device management cluster. The device management clustering may provide various other advantages or potential advantages.

<FIG> depicts a computer suitable for use in performing various functions described herein.

The computer <NUM> includes a processor <NUM> (e.g., a central processing unit, a processor having a set of processor cores, a processor core of a processor, or the like) and a memory <NUM> (e.g., a random access memory, a read only memory, or the like). The processor <NUM> and the memory <NUM> may be communicatively connected. The computer <NUM> may include at least one processor and at least one memory including program code, wherein the at least one memory and the program code are configured to, with the at least one processor, cause the computer to perform various functions presented herein.

The computer <NUM> also may include a cooperating element <NUM>. The cooperating element <NUM> may be a hardware device. The cooperating element <NUM> may be a process that can be loaded into the memory <NUM> and executed by the processor <NUM> to implement functions as discussed herein (in which case, for example, the cooperating element <NUM> (including associated data structures) can be stored on a non-transitory computer-readable storage medium, such as a storage device or other storage element (e.g., a magnetic drive, an optical drive, or the like)).

It will be appreciated that computer <NUM> may represent a general architecture and functionality suitable for implementing functional elements described herein, portions of functional elements described herein, or the like, as well as various combinations thereof. For example, computer <NUM> may provide a general architecture and functionality that is suitable for implementing one or more elements presented herein, such as a router <NUM> or a portion thereof, a management element <NUM> or a portion thereof, a management device <NUM> or a portion thereof, a management element <NUM> or a portion thereof, a router <NUM> or a portion thereof, a management device <NUM> or a portion thereof, a router <NUM> or a portion thereof, a cluster management element.

<NUM> or a portion thereof, a device or a portion thereof configured to execute method <NUM>, a device or a portion thereof configured to execute method <NUM>, a device or a portion thereof configured to execute method <NUM>, a device or a portion thereof configured to execute method <NUM>, or the like, as well as various combinations thereof.

It will be appreciated that at least some of the functions presented herein may be implemented in software (e.g., via implementation of software on one or more processors, for executing on a general purpose computer (e.g., via execution by one or more processors) so as to provide a special purpose computer, and the like) and/or may be implemented in hardware (e.g., using a general purpose computer, one or more application specific integrated circuits, and/or any other hardware equivalents).

It will be appreciated that at least some of the functions presented herein may be implemented within hardware, for example, as circuitry that cooperates with the processor to perform various functions. Portions of the functions/elements described herein may be implemented as a computer program product wherein computer instructions, when processed by a computer, adapt the operation of the computer such that the methods and/or techniques described herein are invoked or otherwise provided. Instructions for invoking the various methods may be stored in fixed or removable media (e.g., non-transitory computer-readable media), transmitted via a data stream in a broadcast or other signal bearing medium, and/or stored within a memory within a computing device operating according to the instructions.

Claim 1:
An apparatus (<NUM>, <NUM>, <NUM>), comprising:
at least one processor (<NUM>); and
at least one memory (<NUM>) including program code;
wherein the at least one memory and the program code are configured to, with the at least one processor, cause the apparatus to at least:
maintain (<NUM>), by a device of a device cluster (<NUM>) including a set of devices, device cluster membership information indicative of device membership of the device cluster; said apparatus being characterized in that it is further caused to
support (<NUM>), by the device, communication with one or more neighboring devices to support distribution of the device cluster membership information indicative of device membership of the device cluster, comprising receiving, by the device from one of the one or more neighboring devices, an indication that the one of the one or more neighboring devices is a member of the device cluster and updating, by the device, the device cluster membership information to indicate that the one of the one or more neighboring devices is a member of the device cluster.