Secure remote bootstrapping of network device

Techniques are disclosed for performing secure remote bootstrapping operations of a network device such that sensitive configuration resides in volatile memory or is inaccessible upon power loss. In one example, a network device performs a first request for onboarding information. In response to determining that a first initialization of the network device has not occurred, the network device performs the first initialization by configuring, with the onboarding information, the network device to mount a portion of a file system to a volatile memory and not a non-volatile memory. After rebooting, the network device performs a second request for the onboarding information. In response to determining that the first initialization of the network device has occurred, the network device performs a bootstrapping operation of the network device. The bootstrapping operation may configure the network device for remote management such that any subsequent configuration obtained remotely is not retained on power loss.

TECHNICAL FIELD

This disclosure generally relates to network devices and, more particularly, to deployment and activation of network devices

BACKGROUND

Large enterprises often face the challenge of mass distribution and deployment of network infrastructure. For example, an enterprise may operate a number of geographically distributed facilities (e.g., offices, retail outlets, and the like) that require network connectivity to a central or main office of the enterprise and optionally to each other. In such cases, a challenge arises when the enterprise desires to install or upgrade network devices with each of the many remote facilities. For example, a central information technology (IT) administrative group of the enterprise may coordinate an effort to upgrade computers, firewalls, gateways, routers, VPN appliances, switches or other network equipment in each of the remote facilities. Such operations may require deployment and activation of hundreds or sometimes thousands of devices.

To simplify the process, the enterprise may purchase similar network devices for deployment at the remote facilities in a single mass rollout. By purchasing similar if not the same network devices, the enterprise may ease administrative burdens with respect to deploying and operating these network devices. In such cases, it is common for the enterprise to contract with the manufacturer to ship the units directly to the remote facilities. This saves shipping costs and offers the advantage of alleviating the central IT group from having the burden of physically receiving and reshipping the devices. However, when devices are shipped directly from the manufacturer to the final location at which the devices are to be deployed, it may not always be possible for a trained network administrator to physically manipulate the devices to ensure proper installation and activation. As a result, the person who configures the devices is typically a store manager or other person who does not have experience in configuring network devices. In such cases, it may be difficult to ensure that the devices are correctly deployed and activated in a manner so as to match a centralized device management system often maintained by the IT group for remotely monitoring and managing devices in the enterprise.

SUMMARY

In general, the disclosure describes techniques for performing a secure remote bootstrapping operation of a network device. To reduce the technical complexity of configuring a network device to an unsophisticated user, conventional devices may use the Zero-touch Provisioning (ZTP) to remotely and automatically perform a first-time configuration of a network device for use within an enterprise network. Additional information with respect to the ZTP protocol is described in “Zero Touch Provisioning for Networking Devices,” Internet-Draft, Internet Engineering Task Force (IETF), available at https://tools.ietf.org/html/draft-ietf-netconf-zerotouch, the entire contents of which is incorporated herein by reference.

However, network devices that are deployed to public places (e.g., kiosks, shopping malls, offices, apartment basements) may still be vulnerable to attack by a malicious actor with physical access to the device. For example, a conventional network device may store sensitive information, such as user data and credentials, network traffic, or the address and identity of other devices within the enterprise network, within a non-volatile memory of the network device. A malicious actor with physical access to the network device potentially could remove the non-volatile memory and glean valuable information from analysis of data stored on the non-volatile memory. One potential solution would be to configure the network device to store such sensitive information within a volatile memory of the network device, such that the sensitive information is lost upon loss of power to the device. However, this may be beyond the ability of an end-user that relies upon ZTP to perform first-time configuration of the network device. Furthermore, some security configurations should be performed in non-volatile memory to prevent loss of power from removing the security configurations, further increasing the technical burden on the end-user.

Accordingly, techniques are disclosed for performing a secure, remote bootstrapping operation of a network device such that sensitive information is stored only on a volatile memory of the network device. For example, a network device as described herein may, during an initial boot cycle, perform a first touchless provisioning operation to retrieve onboarding information. The network device may process the onboarding information to determine whether a first initialization of the network device has occurred. If the first initialization has not yet occurred, the network device performs the first initialization. For example, the network device may perform the first initialization by configuring local user access permissions and configuring itself to mount at least a portion of a file system to a volatile memory of the network device and not a non-volatile memory of the network device. Further, the network device reboots itself.

For each subsequent boot cycle, the network device mounts at least a portion of the file system to the volatile memory and/or encrypts at least a portion of the non-volatile memory. Further, the network device performs a second touchless provisioning operation to request the onboarding information. The network device processes the onboarding information to determine whether the first initialization of the network device has occurred. In response to determining that the first initialization has occurred, the network device performs a bootstrapping operation. As examples of bootstrapping operations, the network device may configure itself for remote management by another network device.

Thus, by using the techniques of the disclosure, the network device may perform a secure remote bootstrapping operation. Further, upon loss of power to the network device, the network device loses information stored on the volatile memory or access to encrypted information stored on the non-volatile memory. Thus, the network device does not retain any information that may be of use to a malicious actor that has physical access to the network device. Accordingly, the techniques of the disclosure provide specific technical improvements to the computer-related field of network device deployment and configuration. For example, the techniques of the disclosure may allow for the deployment to public spaces of network devices that are secured from physical or local attack by a malicious actor. Further, the techniques of the disclosure may allow for the use of touchless provisioning to securely and remotely configure a network device. Additionally, the techniques of the disclosure may reduce the cost and technical burden on the end user of deploying and configuring network devices to operate within an enterprise network.

In one example, this disclosure describes a method comprising: performing, by one or more processors of a network device, a first request for onboarding information for the network device; processing, by the one or more processors, the onboarding information to determine that a first initialization of the network device has not occurred; in response to determining that the first initialization of the network device has not occurred, performing the first initialization by: configuring, by the one or more processors and with the onboarding information, the network device to mount at least a portion of a file system to a volatile memory of the network device and not a non-volatile memory of the network device; and rebooting, by the one or more processors, the network device; after rebooting the network device: performing, by the one or more processors, a second request for the onboarding information for the network device; processing, by the one or more processors, the onboarding information to determine that the first initialization of the network device has occurred; and in response to determining that the first initialization of the network device has occurred, performing, by the one or more processors, a bootstrapping operation of the network device.

In another example, this disclosure describes a network device comprising: a non-volatile memory; a volatile memory; and one or more processors configured to: perform a first request for onboarding information for the network device; process the onboarding, information to determine that a first initialization of the network device has not occurred; in response to determining that the first initialization of the network device has not occurred, perform the first initialization by: configuring, with the onboarding information, the network device to mount at least a portion of a file system to the volatile memory and not the non-volatile memory; and rebooting the network device; after rebooting the network device: perform a second request for the onboarding information for the network device; process the onboarding information to determine that the first initialization of the network device has occurred; and in response to determining that the first initialization of the network device has occurred, perform a bootstrapping operation of the network device.

In another example, this disclosure describes a non-transitory computer-readable medium comprising instructions configured to cause one or more processors of a network device to: perform a first request for onboarding information for the network device; process the onboarding information to determine that a first initialization of the network device has not occurred; in response to determining that the first initialization of the network device has not occurred, perform the first initialization by: configuring, with the onboarding information, the network device to mount at least a portion of a file system to a volatile memory of the network device and not a non-volatile memory of the network device; and rebooting the network device; after rebooting the network device: perform a second request for the onboarding information for the network device; process the onboarding information to determine that the first initialization of the network device has occurred; and in response to determining that the first initialization of the network device has occurred, perform a bootstrapping operation of the network device.

DETAILED DESCRIPTION

FIG. 1is a block diagram illustrating example system2for performing a secure remote bootstrapping operation in accordance with the techniques of the disclosure. In general, this disclosure describes techniques for securely and remotely configuring network device14. To deploy and manage network device14with a centralized management system (e.g., orchestrator server20and voucher server10), network device14solicits network settings, management configurations, and security configurations automatically upon power-up. In accordance with the techniques of this disclosure, network device14receives configuration information and a boot image from, e.g., bootstrap server18.

System2, in the example ofFIG. 1, may include a plurality of sub-networks, e.g., service provider network32and customer network30. Service provider network32provides to customer network30network services that are available for request and use by customer devices36within customer network30. As will be discussed in greater detail below, service provider network32provides, via voucher server10, authentication and verification services to customer network30. In some examples, service provider network32is an Internet Service Provider (ISP) that provides customer network30with access to one or more external networks such as, for example, the Internet.

Customer network30may correspond to, for example, a retail outlet or a corporate division (e.g., legal, engineering, marketing, sales, accounting, etc.). Customer network30may correspond to a new sub-network for which new network device14is to be enabled. Customer network30includes orchestrator server20that manages each network element (e.g., router, switch, gateway, VPN appliance, firewall, and the like) within customer network30, DHCP server16that provides DHCP services to devices within customer network30, bootstrap server18that provides configuration and bootstrap services to devices within customer network30, one or more customer devices36, and network device14that provides network traffic routing and forwarding services to customer devices36. For ease of discussion, customer network30is depicted as including a single network device14, a single DHCP server16, and a single bootstrap server18. However, in other examples of the techniques of the disclosure, customer network30includes a plurality of network devices14, a plurality of DHCP servers16, a plurality of bootstrap servers18, or some combination thereof.

Orchestrator server20is communicatively coupled to network device14of customer network30. Orchestrator server20may obtain ownership vouchers33from voucher server10of service provider network32, as described in greater detail below. Orchestrator server20may provide DHCP configuration27to DHCP server16to enable DHCP server16to provide DHCP services to devices within customer network30, such as network device14. Additionally, orchestrator server20may provide bootstrap configuration information29to bootstrap server18to enable bootstrap server18to provide bootstrapping services to network device14. Once network device14is deployed and activated, orchestrator server20may manage network device14using a communications protocol, such as NETCONF. Managed network device14is also referred to herein as a network “element.” In common practice, orchestrator server20and network device14managed by orchestrator server20are centrally maintained by an group of the customer and are collective referred to as an element management system (EMS) or a network management system (NMS). An administrator may interact with orchestrator server20to remotely monitor and configure network device14. For example, the administrator may receive alerts from orchestrator server20regarding network device14, view configurations or management data of network device14, modify the configurations or management data of network device14, add new network devices to customer network30, remove existing network device14from customer network30, or otherwise manipulate customer network30and network device14. In some examples, orchestrator server20provides a device management interface (DMI) for an administrator to manage network device14once network device14becomes active. The DMI may comprise an interface, such as a graphical user interface (GUI) or command line interface, by which the administrator dynamically adjusts configuration data for network device14or other devices managed by orchestrator server20.

DHCP server16provides DHCP services to devices within customer network30, such as network device14, bootstrap server18, and customer devices36. As described herein, network device14may request bootstrapping data from DHCP server16via a touchless provisioning operation to perform automatic configuration of network device14. For example, network device14may request a network assignment from DHCP server16as well as redirect information that redirects network device14to bootstrap server18for onboarding information. In some examples, the touchless provisioning operation is a ZTP operation.

Bootstrap server18may be used as a source of onboarding information for network device14. As described herein, network device14may request onboarding information from bootstrap server18via a touchless provisioning operation to perform automatic configuration of network device14. In some examples, the touchless provisioning operation is a ZTP operation. In some examples, Bootstrap server18is a RESTCONF server implementing a YANG module.

Network device14generally is a network device that may be difficult for inexperienced, non-technical users to operate. Network device14, for example, typically does not include a keyboard, a monitor, or other conventional user interfaces. Accordingly, a console is not generally accessible when starting network device14. Network device14may correspond, for example, to a router, switch, gateway, hub, server, computing device, computing terminal, printer, firewall, intrusion detection and/or prevention device, wireless Access Point (AP), or other type of network device. An inexperienced user, such as a store manager, may have difficulty manually configuring network device14. Accordingly, the techniques of this disclosure may simplify the process of configuring network devices14through the use of a secure touchless provisioning operation, which is used by network device14to configure itself during a boot cycle, at which time an input console is typically not available.

A boot cycle is generally a process during which a processor of a device, such as one of network devices14, “bootstraps” loading of an operating system kernel. Generally, the processor includes hard-coded instructions to retrieve a boot loader from a defined memory address following an initial receipt of power, that is, after the device is initially turned on. The boot loader includes bootstrapping instructions for loading the kernel, as well as for initializing variables, such as various register values. In some examples, the boot loader may include instructions for performing a touchless provisioning operation to retrieve onboarding information used for loading the kernel and/or mounting a file system for network device4.

Customer devices36may be, for example, personal computers, laptop computers or other types of computing devices associated with users of customer network30. Additional examples of customer devices36include mobile telephones, laptop or desktop computers having, e.g., a 3G or 4G wireless card, wireless-capable netbooks, video game devices, pagers, smart phones, personal data assistants (PDAs), Internet of Things (IoT) devices such as televisions, refrigerators, light bulbs, thermostats, home security systems, baby monitors, or the like. Each of customer devices36may run a variety of software applications, such as word processing and other office support software, web browsing software, software to support voice calls, video games, videoconferencing, and email, among others. Customer devices36connect to customer network30via wired and/or wireless communication links. The term “communication link,” as used herein, comprises any form of transport medium, wired or wireless, and can include intermediate nodes such as network devices, such as network device14.

In accordance with the techniques of the disclosure, a secure, remote bootstrapping operation of a network device is disclosed such that sensitive information is stored only on a volatile memory of network device14. To implement the techniques of the disclosure, network device14perform a touchless provisioning operation to retrieve onboarding information during each boot cycle of network device14. Network device14processes the onboarding information to determine whether a first initialization of network device14has occurred. If the first initialization has not yet occurred, network device14performs the first initialization by configuring itself to mount at least a portion of a file system to a volatile memory of the network device and not a non-volatile memory of the network device and reboots itself.

For each subsequent boot cycle, network device14mounts at least a portion of the file system to the volatile memory and/or encrypts at least a portion of the non-volatile memory, such as, e.g., a swap portion of the non-volatile memory. Further, network device14performs another touchless provisioning operation to request the onboarding information. Network device14processes the onboarding information to determine whether the first initialization has occurred. In response to determining that the first initialization has occurred, network device14performs a bootstrapping operation. As examples of bootstrapping operations, the network device may configure itself for remote management by another network device, e.g., by Obtaining an IP address for a remote network device or security credentials for authenticating orchestrator server20or by establishing a local administrator account with which orchestrator server20can log into network device14, etc.).

An example operation in accordance with the techniques of the disclosure is set forth in the algorithm below:# check if running first timeif [running first time]; thendo first-time initialization:set non-volatile configuration (e.g., disable console port or set root password).download and install a package that includes rc scripts to initialize a memory-based Master File System (MFS) to a volatile memory with encrypted swap, mount filesystems, and initialize filesystems, causing the MFS to mount to the volatile memory on boot.reboot the network device (this restarts the network device and also the secure touchless provisioning process)elsedo normal configuration operations, per user's discretionactions performed here, depending on what they are, may or may not survive a power loss, depending on whether the actions affect files in filesystems moved to the MFS in the volatile memory.fi

In one example of the techniques of the disclosure, upon startup, network device14performs first touchless provisioning operation21to request, from DHCP server16, address information for bootstrap server18. Typically, network device14makes this request using an unsecured protocol and blindly trusts the response. In some examples, the address information is a list of one or more bootstrap servers18from which network device14may obtain configuration information. In some examples, the list is a tuple data structure that specifies a hostname and a port for each bootstrap server18. The address information is redirect information which redirects a request for configuration information from network device14to bootstrap server18.

Upon receiving the address information for bootstrap server18, network device14performs second touchless provisioning operation23to request, from bootstrap server18, onboarding information for network device14. In some examples, network device14makes the request using an unsecured connection. Network device14may blindly trust a TLS certificate of bootstrap server18. Bootstrap server18may authenticate network device14via a TLS-level client certificate such as IDevID. Network device14receives the onboarding information from bootstrap server18. Typically, the configuration information received from bootstrap server18is signed.

Network device14verifies the signature of bootstrap server18in the configuration information. After verifying the signature, network device14processes the onboarding information to determine whether a first initialization of the network device has occurred. If the first initialization has not yet occurred, network device14performs the first initialization. For example, network device14may perform the first initialization by configuring local user access permissions to limit local user access and configuring itself to mount at least a portion of a file system to a volatile memory of network device14and not a non-volatile memory of network device14. Further, network device14reboots itself.

The onboarding information may include instructions to perform the configuring of the local user access permissions. For example, network device14may configure, based on the onboarding information, the local user access permissions by performing a secure hardening of network device14so as to limit local user access to network device14. For example, network device14may disable one or more console ports and/or open management ports of network device14or set a root access password to network device14. Typically, network device14stores the local user access permissions in a non-volatile memory of network device14such that the local user access permissions are not lost upon power loss to network device14.

In some examples, the onboarding information specifies a particular boot image that network device14is to use, an initial configuration that network device14should use, and one or more scripts for execution by network device14. In some examples, the onboarding information specifies a particular operating system type and version. In some examples, network device14uses the onboarding information to configure one or more remote management protocols, such as NETCONF over SSH, and to configure whether network device14initiates an outbound SSH connection, or opens a port enabling inbound SSH connections. In some examples, network device14uses the onboarding information to configure whether orchestrator server20or another user may access network device14via a root or other login. In some examples, network device14uses the onboarding information to configure how SSH authentication may be performed (e.g., via password, public-key encryption, RADIUS, tacplus, etc.).

For each subsequent boot cycle, network device14mounts at least a portion of the file system to the volatile memory and/or encrypts at least a portion of the non-volatile memory. Further, network device14performs third touchless provisioning operation25to request the onboarding information. Network device14processes the onboarding information to determine whether the first initialization of network device14has occurred. In response to determining that the first initialization has occurred, network device14performs a bootstrapping operation. As examples of bootstrapping operations, network device14may configure itself for remote management by another network device (e.g., orchestrator server20). Further, network device14may perform volatile configuration of network device14. As examples of volatile configuration, network device14may disable the console on system ports, enable access of system services via SSH, disable the use of passwords to network device14via SSH, allow root login via SSH, configure a root account for SSH public-key authentication, and configure network device14to initiate an outbound SSH session. These configurations may be lost upon loss of power to or reboot of network device14. After completing the bootstrapping operation, network device14may establish secure management connections (e.g., NETCONF) to orchestrator server20. Further, network device14begins normal operation. For example, in the case where network device14is a router, network device14may begin the processing and routing of network traffic31.

Typically all control plane and management traffic is encrypted. While secure zero touch provisioning (e.g., secure ZTP) and NETCONF over SSH are built on top of secure transport layers, not all protocols are such (e.g., syslog). Thus, a VPN may be configured between network device14and bootstrap server18as an aspect of the bootstrapping process. Different technologies may be used to implement these VPNs. For instance, they may be implemented in software or hardware. In one example, an MS-MPC line card may be used to perform hardware encryption of all control plane and management plane traffic. In some examples, the secure touchless provisioning operation described herein may be used to configure the MS-MPC line cards. In other examples, a user configures the MS-MPC line cards as an interactive step that occurs over a management connection (e.g., NETCONF over SSH), either through access via network device14or orchestrator server20.

In some examples, the bootstrapping process may be further split into two operations: the first being an operation where network device14is configured with onboarding information and the second being an operation where orchestrator server20or another network management system configures network device14with additional software applications after orchestrator server20establishes a connection to network device14for the first time. In some examples, each touchless provisioning operation concludes with network device14sending a “bootstrap-complete” progress report to bootstrap server18. Bootstrap server18may propagate this report to orchestrator server20, thus providing a signal for when orchestrator server20may perform first-time operations on network device14.

In some examples, orchestrator server20may obtain ownership vouchers33from voucher server10of service provider network32. For example, voucher server10may provide a REST-based API that authenticates operator credentials. Voucher server10verifies that a device within customer network30, such as network device14, is owned by an operator of customer network30. Voucher server10encodes an owner certificate of orchestrator server20into the ownership voucher; network device14may use the owner certificate to verify onboarding information signed by the owner during a touchless provisioning operation. Network device14may, for example, use a preconfigured trust anchor for verifying ownership vouchers were generated by a trusted authority. Network device14may also, for example, examine the ownership voucher to ensure that it contains the network device's serial number, and therefore knows that the ownership voucher applies to the network device14. Voucher server10may issue a voucher signed by a signing authority trusted by a manufacturer of network device14(or the manufacturer itself) to each authorized device accessing customer network30.

As one example, orchestrator20generates a PKI for owner certificates. Orchestrator20requests, from voucher server10to provide vouchers, containing a supplied domain certificate, for a list of devices (identified by their serial numbers) authorized to access customer network30. Voucher server10provides a list of ownership vouchers for the devices authorized to access customer network30. Orchestrator20stores the device-specific ownership vouchers and owner certificates for subsequent use. Orchestrator20may use the stored device-specific ownership vouchers and owner certificates to sign bootstrapping data for network device14during touchless provisioning operations to configure network device as described above.

Thus, by using the techniques of the disclosure, the network device may maintain configuration information obtained during the bootstrapping process within a volatile memory of the network device. Upon loss of power to the network device, the network device loses any information that may be of use to a malicious actor that has physical access to the network device, such as user data, configuration information, or boot images. Each boot cycle, network device14may retrieve a new onboarding information upon startup using a touchless provisioning operation. Further, after the first initialization, local access to network device14, such as via console ports, is disabled.

Accordingly, the techniques of the disclosure provide specific improvements to the computer-related field of network device deployment and configuration. For example, the techniques of the disclosure may allow for the deployment to public spaces of network devices that are secured from physical or local attack by a malicious actor. Further, the techniques of the disclosure may allow for the use of touchless provisioning to securely and remotely configure a network device. Additionally, the techniques of the disclosure may reduce the cost and technical burden on the end user of deploying and configuring network devices to operate within an enterprise network. Further, the techniques of the disclosure may reduce the complexity of deployment, maintenance, and upgrading of network devices.

FIG. 2is a block diagram illustrating an example network device200in accordance with the techniques of the disclosure. In some examples, network device200is an example of network device14ofFIG. 1. For example, network device200may correspond to a router, a bridge, a hub, a switch, a server, a printer, a gateway, a firewall, an IDP device, or other network device. In the example ofFIG. 2, network device200includes user interface module202, control unit208, and network interface220.

User interface202is configured to send/receive data to/from a user, such as a network administrator. Typically, user interface202comprises a console port that enables local access by a user, such as an administrator. However, in some examples, user interface202is or otherwise includes a workstation, a keyboard, pointing device, voice responsive system, video camera, biometric detection/response system, button, sensor, mobile device, control pad, microphone, presence-sensitive screen, network, or any other type of device for detecting input from a human or machine. In some examples, user interface202further includes a display for displaying an output to the user. The display may function as an output device using technologies including liquid crystal displays (LCD), quantum dot display, dot matrix displays, light emitting diode (LED) displays, organic light-emitting diode (OLED) displays, cathode ray tube (CRT) displays, e-ink, or monochrome, color, or any other type of display capable of generating tactile, audio, and/or visual output. In other examples, user interface202may produce an output to a user in another fashion, such as via a sound card, video graphics adapter card, speaker, presence-sensitive screen, one or more USB interfaces, video and/or audio output interfaces, or any other type of device capable of generating tactile, audio, video, or other output. In some examples, user interface202may include a presence-sensitive display that may serve as a user interface device that operates both as one or more input devices and one or more output devices.

Control unit208comprises hardware for performing the techniques of this disclosure. Processors204may include any one or more of a microprocessor, a controller, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or equivalent discrete or integrated logic circuitry. Network device further includes non-volatile memory230and volatile memory240. As described herein, “non-volatile memory” refers to a storage device which retains data even if power to the non-volatile memory is lost. In contrast, as described herein, “volatile memory” refers to a storage device which loses data if power to the volatile memory is lost. Alternatively, control unit208may comprise dedicated hardware, such as one or more integrated circuits, one or more Application Specific Integrated Circuits (ASICs), one or more Application Specific Special Processors (ASSPs), one or more Field Programmable Gate Arrays (FPGAs), or any combination of the foregoing examples of dedicated hardware, for performing the techniques described herein.

Non-volatile memory230may include a disk drive, an optical drive, memory, such as random-access memory (RAM), read only memory (ROM), programmable read only memory (PROM), erasable programmable read only memory (EPROM), electronically erasable programmable read only memory (EEPROM), flash memory, etc., comprising executable instructions for causing processors204to perform the actions attributed to them.

Volatile memory100may include memory, such as random-access memory (RAM) or flash memory, etc., comprising executable instructions for causing processors204to perform the actions attributed to them.

In the example ofFIG. 2, control unit208comprises device modules212and protocols214, which may comprise software modules executed by control unit208or discrete, independent hardware units of control unit208. When any or all of device modules212and protocols214comprise software, instructions executable by a processor for device modules212and protocols214may be encoded in a computer-readable medium of network device200, such as non-volatile memory230or volatile memory240.

Network interface220may comprise any interface for connecting to devices of a computer network, such as DHCP server16, bootstrap server18, or orchestrator server20ofFIG. 1. For example, network interface220may comprise an Ethernet interface, a gigabit Ethernet interface, a telephone modem, a cable modem, a satellite modem, or other network interface. In some examples, network interface220comprises one or more network interface cards.

Device modules212generally correspond to components specific to network device200. For example, when network device200comprises a router, device modules212may comprise a control plane that maintains a routing information base, a forwarding engine that maintains a forwarding information base, one or more routing protocols, or other modules required to route packets through a network. As another example, when network device200comprises a security device, device modules212may comprise a protocol decoder module, an application identification module, and an attack detection module, or other network security modules. Protocols214comprise one or more communication protocols for communicating with management device10and/or other network devices. For example, protocols214may comprise a touchless provisioning protocol such as ZTP216. Protocols214may also comprise one or more routing protocols, security protocols, or other protocols, depending upon the type of device to which network device200corresponds.

In accordance with the techniques of the disclosure, network device200performs a secure, remote bootstrapping operation. Upon startup, processors204perform a first touchless provisioning operation to request, from DHCP server16, address information for bootstrap server18. Upon receiving the address information for bootstrap server18, processors204perform a second touchless provisioning operation to request, from bootstrap server18, onboarding information for network device200.

Processors204process the onboarding information to determine whether a first initialization of network device200has occurred. If the first initialization has not yet occurred, processors204perform the first initialization. For example, processors204may perform the first initialization by configuring local user access permissions270of network device200and configuring network device200to mount at least a portion of file system250to volatile memory240and not non-volatile memory230. Further, processors204cause network device200to reboot.

To configure local user access permissions270, processors204may perform a secure hardening of network device200by disabling one or more console ports and/or open management ports of network device200or by setting a root access password to network device200. Typically, processors204store local user access permissions270in non-volatile memory230such that local user access permissions270are not lost upon power loss to network device200

For each subsequent boot cycle, processors204perform an operating-system level initialization. For example, processors204mount at least a portion of file system250to volatile memory240. As another example, processors204encrypt at least a portion of non-volatile memory230, such as swap portion260. Further, processors204perform a touchless provisioning-level initialization. As an example, processors204perform a touchless provisioning operation to retrieve onboarding information from bootstrap server18. Processors204store the onboarding information in volatile memory240such that the onboarding information is not retained upon loss of power to or reboot of network device200. Processors204process the onboarding information to determine whether the first initialization of network device200has occurred. In response to determining that the first initialization has occurred, processors204perform a bootstrapping operation. As examples of bootstrapping operations, processors204may configure network device200for remote management by another network device (e.g., orchestrator server20). For example, processors204may assign a hostname, IP address, or port for communication with orchestrator server20, processors204may define a trust anchor certificate to authenticate orchestrator server20, and processors204may establish an admin account with which orchestrator server20may access network device200. As further examples, processors204may disable the console on system ports, enable access of system services via SSH, disable the use of passwords to network device200via SSH, allow root login via SSH, configure a root account for SSH public-key authentication, and configure network device200to initiate an outbound SSH session (e.g., with orchestrator20). Typically, processors204store any configuration that is performed at this stage (e.g., after the first initialization has occurred) in volatile memory240such that the configuration is not retained upon loss of power to or reboot of network device200. After completing the bootstrapping operation, processors204may establish secure management connections (e.g., NETCONF) to orchestrator server20and commence operation of network device200. For example, in the case where network device200is a router, processors204may begin the processing and routing of network traffic.

Thus, by using the techniques of the disclosure, network device200may maintain at least a portion of filesystem250within volatile memory240. Upon loss of power to network device200, volatile memory240loses the portion of filesystem250. Further, network device200may lose a cryptographic cypher for accessing encrypted swap260. Thus, network device200does not retain any information in filesystem250or swap260that may be of use to a malicious actor that has physical access to network device200. Each boot cycle, network device200may perform a new touchless provisioning operation to perform bootstrapping of network device200and so remount the portion of filesystem250to volatile memory240. Further, after the first initialization, local access to network device200, such as via console ports of user interface202, is disabled.

FIG. 3is a block diagram illustrating example server300in accordance with the techniques of the disclosure. Server300is a computing device that may implement, for example, one or more of DHCP server16, bootstrap server18, orchestrator server20, or voucher server10ofFIG. 1. In the example ofFIG. 3, server300includes user interface302, control unit308, and network interface320.

User interface302is configured to send/receive data to/from a user, such as a network administrator. In some examples, user interface302is or otherwise includes a workstation, a keyboard, pointing device, voice responsive system, video camera, biometric detection/response system, button, sensor, mobile device, control pad, microphone, presence-sensitive screen, network, or any other type of device for detecting input from a human or machine. In some examples, user interface302further includes a display for displaying an output to the user. The display may function as an output device using technologies including liquid crystal displays (LCD), quantum dot display, dot matrix displays, light emitting diode (LED) displays, organic light-emitting diode (OLED) displays, cathode ray tube (CRT) displays, e-ink, or monochrome, color, or any other type of display capable of generating tactile, audio, and/or visual output. In other examples, user interface302may produce an output to a user in another fashion, such as via a sound card, video graphics adapter card, speaker, presence-sensitive screen, one or more USB interfaces, video and/or audio output interfaces, or any other type of device capable of generating tactile, audio, video, or other output. In some examples, user interface302may include a presence-sensitive display that may serve as a user interface device that operates both as one or more input devices and one or more output devices.

Network interface320may comprise any interface for connecting to devices of a computer network, such as network device14of customer network30. For example, network interface320may comprise an Ethernet interface, a gigabit Ethernet interface, a telephone modem, a cable modem, a satellite modem, or other network interface. In some examples, network interface320comprises one or more network interface cards.

Control unit308comprises hardware for performing the techniques of this disclosure. Processors304may include any one or more of a microprocessor, a controller, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or equivalent discrete or integrated logic circuitry. Storage device306may include a disk drive, an optical drive, memory, such as random-access memory (RAM), read only memory (ROM), programmable read only memory (PROM), erasable programmable read only memory (EPROM), electronically erasable programmable read only memory (EEPROM), flash memory, comprising executable instructions for causing the processors304to perform the actions attributed to them. Alternatively, control unit308may comprise dedicated hardware, such as one or more integrated circuits, one or more Application Specific Integrated Circuits (ASICs), one or more Application Specific Special Processors (ASSPs), one or more Field Programmable Gate Arrays (FPGAs), or any combination of the foregoing examples of dedicated hardware, for performing the techniques described herein.

In the example ofFIG. 3, control unit308comprises device manager310, device modules312, and protocols314, which may comprise software modules executed by control unit308or discrete, independent hardware units of control unit308. When any or all of device manager310, device modules312, and protocols314comprise software, e.g., executable software modules, instructions executable by a processor for device manager310, device modules312, and protocols314may be encoded in a computer-readable medium, such as storage device306.

Device manager310interacts with one or more managed devices, e.g., network device14, to manage the network devices. In one example, device manager310executes an implementation of NETCONF. Device manager310sends electrical signals to managed network device14via network interface320. Therefore, device manager310sends and receives packets comprising data for managing the managed network devices14indirectly via a network, such as the Internet, to, e.g., network devices14.

Device modules312generally correspond to components specific to server300. For example, device modules312may comprise a control plane that maintains a routing information base, a forwarding engine that maintains a forwarding information base, one or more routing protocols, or other modules required to route packets through a network.

In the example where server300is an example of DHCP server16ofFIG. 1, device modules312include a DHCP module for providing DHCP services to devices within customer network30, such as network device14, bootstrap server18, and customer devices36. In some examples, server300may provide redirect information in response to a touchless provisioning operation by network device14. For example, in response to a first touchless provision operation in which network device14requests a network assignment and requests redirect information for onboarding information, server300provides address information for reaching bootstrap server18. Typically, server300provides this information using an unsecured protocol. In some examples, the address information is a list of one or more bootstrap servers18from which network device14may obtain configuration information. In some examples, the list is a tuple data structure that specifies a hostname and a port for bootstrap server18. In some examples, the address information is redirect information which redirects a request for configuration information from network device14to bootstrap server18.

In the example where server300is an example of bootstrap server18ofFIG. 1, server300may be used as a source of onboarding information for network device14. As described herein, network device14may request onboarding information from bootstrap server18via a touchless provisioning operation to perform automatic configuration of network device14. In some examples, the touchless provisioning operation is a ZTP operation. For example, in response to a second touchless provisioning request from network device14, server300provides onboarding information for network device14. In some examples, the onboarding information specifies a particular boot image that network device14is to use, an initial configuration that network device14should use, and one or more scripts for execution by network device14. In some examples, the onboarding information specifies a particular operating system type and version. In some examples, network device14uses the onboarding information to configure one or more remote management protocols, such as NETCONF over SSH, and to configure whether network device14initiates an outbound SSH connection, or opens a port enabling inbound SSH connections. In some examples, network device14uses the onboarding information to configure whether orchestrator server20or another user may access network device14via a root or other login. In some examples, network device14uses the onboarding information to configure how SSH authentication may be performed (e.g., via password, public-key encryption, RADIUS, tacplus, etc.).

In the example where server300is an example of orchestrator server20ofFIG. 1, server300may obtain ownership vouchers from voucher server10of service provider network32, provide DHCP configuration to DHCP server16to enable DHCP server16to provide DHCP services to devices within customer network30, or provide bootstrap configuration information to bootstrap server18to enable bootstrap server18to provide bootstrap services to network device14. Additionally, once network device14is deployed and activated, server300may establish secure management connections to manage network device14using a communications protocol, such as NETCONF. Further, orchestrator server20may forward network traffic to network device14for processing and routing.

In the example where server300is an example of voucher server10ofFIG. 1, server300may provide a REST-based API that authenticates operator credentials. For example, server300verifies that a device within customer network30, such as network device14, is owned by an operator of customer network30. Server300encodes an owner certificate of orchestrator server20into the ownership voucher; network device14may use the owner certificate to verify onboarding information signed by the owner during a touchless provisioning operation. Server300may issue a voucher signed by a signing authority trusted by a manufacturer of network device14(or the manufacturer itself) to each authorized device accessing customer network30.

Protocols314comprise one or more network communication protocols for communicating over a network. For example, protocols314may comprise one or more routing protocols, security protocols, or other protocols, e.g., a touchless provisioning protocol such as ZTP316, DHCP318, or other network protocols for communicating over a network not expressly depicted inFIG. 3, such as SSH, PPP, PPPoE, PPPoA, MPLS, BGP, SNMP, NETCONF, etc.

FIG. 4is a flowchart illustrating an example secure remote bootstrapping operation in accordance with the techniques of the disclosure. For convenience,FIG. 4is described with respect toFIG. 1.

In one example, upon startup, network device14performs a first touchless provisioning operation to request, from DHCP server16, address information for bootstrap server18(404). In response, DHCP server16provides the address information as a list of one or more bootstrap servers18from which network device14may obtain configuration information (406). In some examples, the list is a tuple data structure that specifies a hostname and a port for bootstrap server18. In some examples, the address information is redirect information which redirects a request for configuration information from network device14to bootstrap server18.

Upon receiving the address information for bootstrap server18, network device14performs second touchless provisioning operation23to request, from bootstrap server18, onboarding information for network device14(408). Bootstrap server18provides the boot configuration information to network device14(410). Network device14receives the onboarding information from bootstrap server18. Typically, the configuration information received from bootstrap server18is signed.

Network device14processes the onboarding information to determine whether a first initialization of the network device has occurred. In response to determining that the first initialization has not yet occurred, network device14performs the first initialization (412). For example, network device14may perform the first initialization by configuring local user access permissions and configuring itself to mount at least a portion of a file system to a volatile memory of network device14and not a non-volatile memory of network device14. For example, network device14may perform a secure hardening of network device14by disabling one or more console ports and/or open management ports of network device14or by setting a root access password to network device14. Typically, network device14stores the local user access permissions in a non-volatile memory of network device14such that the local user access permissions are not lost upon power loss to network device14. Further, network device14reboots itself.

For each subsequent boot cycle, network device14mounts at least a portion of the file system to the volatile memory. In some examples, network device14encrypt at least a portion of the non-volatile memory, such as a swap portion of the file system. Further, network device14performs a third touchless provisioning operation to request the onboarding information (414). Network device14processes the onboarding information to determine whether the first initialization of network device14has occurred. In response to determining that the first initialization has occurred, network device14performs a bootstrapping operation (418). As examples of bootstrapping operations, network device14configure itself for remote management by another network device (e.g., orchestrator server20). Further, network device14may perform volatile configuration of network device14. As examples of volatile configuration, network device14may disable the console on system ports, enable access of system services via SSH, disable the use of passwords to network device14via SSH, allow root login via SSH, configure a root account for SSH public-key authentication, and configure network device14to initiate an outbound SSH session.

After completing the bootstrapping operation, orchestrator server20and network device14may establish secure management connections (e.g., NETCONF) with one another (420). Further, network device14begins normal operation (422). For example, in the case where network device14is a router, network device14commences the processing and routing of network traffic31.