Patent Description:
A device such as a storage device, an accelerator device, and/or the like, may include an embedded operating system to run one or more programs that may be used, for example, to perform computations that may be offloaded from a host. The device may be connected to the host with an interconnect that may be used to exchange input and/or output (I/O) data for the program between the host and the device. The embedded operating system may be accessible using a system terminal.

The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive principles and therefore it may contain information that does not constitute prior art.

<CIT> discloses: A method of operating a storage device controller which controls a storage device includes receiving a debugging data request command through a peripheral component interconnect express (PCIe) interface of the storage device controller from outside of the storage device controller, and storing debugging data in a register included in the PCIe interface.

], discusses debugging techniques.

Embodiments of the invention are defined in the appended claims.

The figures are not necessarily drawn to scale and elements of similar structures or functions may generally be represented by like reference numerals or portions thereof for illustrative purposes throughout the figures. The figures are only intended to facilitate the description of the various embodiments described herein. The figures do not describe every aspect of the teachings disclosed herein and do not limit the scope of the claims. To prevent the drawings from becoming obscured, not all of the components, connections, and the like may be shown, and not all of the components may have reference numbers. However, patterns of component configurations may be readily apparent from the drawings. The accompanying drawings, together with the specification, illustrate example embodiments of the present disclosure, and, together with the description, serve to explain the principles of the present disclosure.

A device such as a storage device, an accelerator device, and/or the like, may include an embedded operating system to run one or more programs that may be used, for example, to perform computations that may be offloaded from a host. The device may be connected to the host with an interconnect such as Peripheral Component Interconnect Express (PCIe). The host and the device may use the interconnect to exchange input and/or output (I/O) data for the program. The embedded operating system may support a terminal that may enable a user to run one or more programs for monitoring and/or troubleshooting the program such as a tracing program, a debugging program, a profiling program, and/or the like. However, the terminal may not be able to access the operating system through the interconnect. Thus, the user may only be able to access these features by connecting additional hardware to the device.

A device in accordance with example embodiments of the disclosure may enable a user to access an embedded operating system through an interconnect. In some embodiments, a device may expose one or more features of an operating system through an interconnect by using an additional function of the interconnect. For example, a storage device may have a storage device controller configured to use a first function of an interconnect. The storage device may use the first function to send and receive data related to its normal operation as a storage device through the interconnect. An embedded operating system running on the storage device may be configured to use a second function of the interconnect. Thus, a user (e.g., one or more programs running on a host) may access the operating system using the same interconnect used to send storage data to and/or from the storage device. Depending on the implementation details, this may enable a user to access the embedded operating system with little or no additional hardware. Depending on the implementation details, this may also enable a user to access any type of feature made available by the operating system using the second function such as programs for debugging, tracing, profiling, file transfers, software updates, firmware updates, and/or the like.

In some embodiments, a device may include communication logic that may enable an embedded operating system running on the device to be accessed through an interconnect. In some embodiments, a host may use a terminal application to access the embedded operating system through an interconnect. For example, the device may include communication logic (that may include, e.g., at least one terminal support driver) that may enable a terminal (e.g., a virtual terminal) to establish a connection over an interconnect such as PCIe and/or associated protocols. The host may include a driver that may enable the terminal application to connect to the embedded operating system through the interconnect. In some embodiments, the communication logic may enable a user to access one or more features of the operating system based on a privilege level of the user.

In some embodiments, a device and a host may be configured to exchange data using a producer-consumer scheme. In some embodiments of a producer-consumer scheme, a producer may generate data that may be used by a consumer. For example, a program controlled using communication logic may operate as a producer and store output data in a buffer (e.g., a ring buffer). The user (e.g., an application running on a host) may operate as a consumer and retrieve the output data from the buffer.

<FIG> illustrates an embodiment of a system in accordance with example embodiments of the disclosure. The system illustrated in <FIG> includes a host <NUM> and a device <NUM>. The host <NUM> may include one or more processors <NUM> and an interconnect interface <NUM> that may enable the host <NUM> to communicate with the device <NUM> through an interconnect fabric <NUM>. The host <NUM> may be configured to run an application <NUM> that may relate to a primary functionality of the device <NUM>. For example, if the device <NUM> is a storage device, the application <NUM> may be a storage application. As another example, if the device <NUM> is a network interface card (NIC), the application <NUM> may be a networking application.

The device <NUM> may be implemented as any type of device such as a storage device, an accelerator, a NIC, and/or the like or any combination thereof. The device <NUM> may include one or more processors <NUM> and an interconnect interface <NUM>. The processor <NUM> may be configured as a function Function A of the interconnect implemented by the interconnect interface <NUM>. For example, if the interconnect interface <NUM> is implemented with PCIe, the processor <NUM> may be configured as function <NUM>. The processor <NUM> may be configured to run an embedded operating system <NUM> having one or more terminal support drivers <NUM> that may enable the embedded operating system <NUM> to be accessed using a terminal and/or terminal application. For example, if the operating system <NUM> is implemented with embedded Linux, the one or more terminal support drivers <NUM> may enable a Linux console to access (e.g., provide input to and/or output from) the Linux kernel.

In some embodiments, the host <NUM> may not be able to access the embedded operating system <NUM> through the interconnect interface <NUM>. Thus, to access the embedded operating system <NUM> (e.g., to collect debug logs of firmware in the device <NUM>), the user may connect additional hardware to the device <NUM>, for example, using a communication port <NUM> (e.g., a universal asynchronous receiver-transmitter (UART)) that may be accessed through a communication connection <NUM> (e.g., a serial connection), using other debug tools and/or software hooks with debugging through ports such as a Joint Test Action Group (JTAG) and/or Serial Wire Debug (SWD) connection. However, the use of additional hardware may be expensive and/or difficult to implement, especially for example, with a device at a remote location.

<FIG> illustrates an example embodiment of a software stack for implementing a hardware connection for a terminal in accordance with example embodiments of the disclosure. The embodiment illustrated in <FIG> may be used, for example, by any apparatus (e.g., a host) to connect to the communication port <NUM> shown in <FIG>. For purposes of illustration, the embodiment illustrated in <FIG> may be described in the context of a Linux operating system, but a similar stack may be implemented with any operating system.

Referring to <FIG>, the software stack may include a terminal application <NUM> configured to access a communication port <NUM> that may be represented as a file descriptor <NUM>. The terminal application <NUM> may access the communication port <NUM> through a system call <NUM> to a virtual file system (VFS) <NUM>. The virtual file system <NUM> may handle the system call by calling an associated function in a character device driver <NUM>. The character device driver <NUM> may include a communication port driver <NUM> that may read and/or write data to and/or from a hardware buffer register in the communication port <NUM>.

<FIG> illustrates an embodiment of a system that may provide a user with access to an embedded operating system in accordance with example embodiments of the disclosure. The system illustrated in <FIG> may include components similar to those illustrated in <FIG> in which references numerals ending in the same digits may indicate components that may perform similar functions. However, in the embodiment illustrated in <FIG>, communication logic <NUM> is configured to expose one or more features of an operating system <NUM> using a second function Function B of the interconnect implemented by the interconnect interface <NUM>. The for interconnect interface <NUM> is implemented with PCIe, a device controller (which may be implemented, for example, by the processor <NUM>) is configured to use function <NUM>, and the communication logic <NUM> is configured to expose another controller (e.g., a debug controller) using function <NUM>.

In the embodiment illustrated in <FIG>, the host processor <NUM> may be configured to run a terminal application <NUM> that may enable a user (e.g., an application running on the host <NUM>) to access the operating system <NUM> through the interconnect interfaces <NUM> and <NUM>. For example, the terminal application <NUM> may be used to monitor and/or troubleshoot a user application such as a computational storage (CS) program running on the operating system <NUM> by launching, through the communication logic <NUM>, a program (e.g., a utility) for debugging, tracing, profiling, and/or the like, the user application. Depending on the implementation details, this may enable a user to access some or all of the features provided by the embedded operating system <NUM> without using additional hardware such as the communication port shown in <FIG>.

<FIG> illustrates an example embodiment of a driver configuration in a system having access to a device operating system through an interconnect in accordance with example embodiments of the disclosure. The driver configuration illustrated in <FIG> may be implemented, for example, to support data flow in any of the embodiments having access to a device operating system through an interconnect disclosed herein including those described with respect to <FIG>, <FIG>, <FIG>, <FIG>, and/or <FIG>.

Referring to <FIG>, a host <NUM> and a device <NUM> may be connected through interconnect fabric <NUM>. The host <NUM> may include a terminal application <NUM>, a virtual file system <NUM>, and one or more drivers <NUM>. The one or more drivers <NUM> may include a hardware driver <NUM> for the interconnect fabric <NUM>, line discipline <NUM>, and/or a teletype (TTY) driver <NUM>. The terminal application <NUM> may communicate with the one or more drivers <NUM> through the virtual file system <NUM> using a file descriptor <NUM>. The one or more drivers <NUM> may enable the terminal application <NUM> to communicate over the interconnect fabric <NUM>, for example, through an interconnect interface such as interconnect interface <NUM> illustrated in <FIG>. In some embodiments, the one or more drivers <NUM> may be implemented by adding the functionality of the drivers <NUM> to an existing device driver for the device <NUM>. In such an embodiment, the hardware driver <NUM> may already be present in the existing device driver for the device <NUM>.

The device <NUM> may include one or more processors <NUM>, one or more drivers <NUM>, and device functionality <NUM>. The device functionality <NUM> may implement one or more primary functionalities of the device <NUM> such as a storage controller and storage media for a storage device, one or more compute resources for a computational storage device, one or more processing resources for an accelerator, network interface hardware for a NIC, and/or the like. In some embodiments, the device functionality <NUM> may be accessed using a first function of an interconnect communicating over the interconnect fabric <NUM>.

The one or more drivers <NUM> may enable an operating system running on the one or more processors <NUM> to communicate over the interconnect fabric <NUM>, for example, through an interconnect interface such as interconnect interface <NUM> illustrated in <FIG>. The one or more drivers <NUM> may include communication logic <NUM> that may be configured to provide access to an operating system running on the processor <NUM> using a second function of an interconnect communicating over the interconnect fabric <NUM>. The one or more drivers <NUM> may also include a hardware driver <NUM> for the interconnect fabric <NUM>, line discipline <NUM>, and/or a teletype (TTY) driver <NUM>. Although shown as a separate component in <FIG>, in some embodiments, the communication logic <NUM> may be partially or entirely integrated with one or more of the hardware driver <NUM>, the line discipline <NUM>, and/or the teletype (TTY) driver <NUM>. In some embodiments, the communication logic <NUM> may include one or more terminal support drivers.

In some embodiments, the one or more drivers <NUM> may implement one or more devices (e.g., one or more controllers) such as a debug device, debug communication (dbgCOMM) device, a USB debug device (dbgUSB), a computational storage Nonvolatile Memory Express (NVMe) debug (dbgCSNVMe) device, and/or the like. In some embodiments, the one or more drivers <NUM> may implement a software communication (software COMM) port that may expose (e.g., make accessible to a user such as an application running on the host <NUM>) one or more features of an operating system running on the device <NUM>.

An example operation in which the host <NUM> may access a device operating system in accordance with example embodiments of the disclosure using the system the system illustrated in <FIG> may proceed as follows. The host <NUM> may receive a command from a user through the terminal application <NUM>. The command may be processed by the host stack illustrated in <FIG>, e.g., using a virtual file descriptor <NUM> through the virtual file system <NUM>. The command may be transferred, using one or more of the drivers <NUM>, from the host <NUM> to the device <NUM> through the interconnect fabric <NUM> using a function of the interconnect other than a function used for normal operation of the device <NUM>. At the device <NUM>, the command may be processed by the device stack, for example, using one or more of the drivers <NUM> (e.g., communication logic <NUM> implemented with one or more terminal support drivers) which may pass the command to the operating system kernel. The operating system kernel may then execute the command. One or more outputs of the command (e.g., debugging logs, performance profiles, calculations, and/or the like) may be transferred back to the user through the terminal application <NUM> using a process that may essentially be the reverse of sending the command. Thus, the one or more outputs may be processed by the device stack using one or more of the drivers <NUM> (e.g., a terminal support driver) which may transfer the one or more outputs through the interconnect fabric <NUM> using a function of the interconnect other than a function used for normal operation of the device <NUM>. At the host <NUM>, the one or more outputs of the command may be processed by the host stack using one or more of the drivers <NUM>. The one or more outputs may then be passed to the terminal application <NUM> by the virtual file system <NUM> using a file descriptor <NUM>.

In some embodiments, one or more of the drivers <NUM> may be implemented by adding the functionality of the driver to an existing driver for an interconnect interface in device <NUM>. In such an embodiment, the hardware driver <NUM> may already be present in an existing driver for the device <NUM>. In some embodiments, the one or more drivers <NUM> may be configured to enable a user to access one or more features of the operating system based on a privilege level of the user.

<FIG> illustrates a more detailed example embodiment of a device that may provide a user with access to an embedded operating system of a device through an interconnect in accordance with example embodiments of the disclosure. The device illustrated in <FIG> may be used, for example, to implement any of the devices illustrated herein, including those illustrated in <FIG>, <FIG>, <FIG>, and/or <FIG>.

Referring to <FIG>, the device <NUM> may include an interconnect interface <NUM>, a device controller <NUM>, device functionality <NUM>, and one or more processors <NUM>.

In some embodiments, the device controller <NUM> is configured to use a first function Function A of the interconnect interface <NUM>, and the one or more processors <NUM> run an operating system <NUM> having communication logic <NUM> that may expose one or more features of the operating system <NUM> using a second function Function B of the interconnect interface <NUM>. The interconnect interface <NUM> is implemented with PCIe, the device controller <NUM> implements function <NUM>, and the communication logic implements function <NUM>. In some embodiments (e.g., a PCIe implementation), the first and second functions Function A and Function B may be handled by the interconnect as separate (e.g., logical) devices even though they may be implemented with the same physical device. In some embodiments, the communication logic <NUM> may include one or more terminal support drivers.

Some embodiments may further include a protocol interface <NUM> which, for example, may implement a communication protocol on top of, or integral with, the interface implemented by the interconnect interface <NUM>. The interconnect interface <NUM> and/or protocol interface <NUM> may implement any interconnects and/or storage protocols including Peripheral Component PCIe, NVMe, NVMe-over-fabric (NVMe-oF), Ethernet, Transmission Control Protocol/Internet Protocol (TCP/IP), remote direct memory access (RDMA), RDMA over Converged Ethernet (ROCE), FibreChannel, InfiniBand, Serial ATA (SATA), Small Computer Systems Interface (SCSI), Serial Attached SCSI (SAS), iWARP, and/or the like, or any combination thereof. For example, in an embodiment in which the device <NUM> is implemented as a storage device, the interconnect interface <NUM> may implement a PCIe interconnect, and the protocol interface <NUM> may implement an NVMe protocol.

In some embodiments, the interconnect interface <NUM>, protocol interface <NUM>, and/or other components of the device <NUM> may implement a coherent (e.g., memory coherent, cache coherent, and/or the like) or memory semantic interface such as Compute Express Link (CXL), and/or a coherent protocol such as CXL. cache, and/or CXL. Other examples of coherent and/or memory semantic interfaces and/or protocols may include Gen-Z, Coherent Accelerator Processor Interface (CAPI), Cache Coherent Interconnect for Accelerators (CCIX), and/or the like.

The device functionality <NUM> may implement one or more primary functionalities of the device <NUM> such as storage media for a storage device, one or more compute resources for a computational storage device, one or more processing resources for an accelerator, network interface hardware for a NIC, and/or the like. The device controller <NUM> may control access to, and/or the operation of, the device functionality <NUM>. For example, the device controller may be implemented as a storage device controller if the device <NUM> is implemented as a storage device. Additionally, or alternatively, the device controller <NUM> may be implemented as a compute resource controller if the device <NUM> is implemented as a computational storage device. As another example, the device controller <NUM> may be implemented as a media access controller (MAC) if the device <NUM> is implemented as a NIC.

In some embodiments, the communication logic <NUM> may enable a virtual terminal to access the operating system <NUM>. For example, if the operating system <NUM> is implemented with embedded Linux, a Linux console may provide access to the Linux kernel, e.g., using the virtual terminal (VT) subsystem of the Linux kernel.

In some embodiments, some or all of the communication logic <NUM> may be implemented as one or more kernel modules. For example, in an embodiment in which the operating system <NUM> is implemented with embedded Linux, the communication logic <NUM> may include one or more terminal support drivers that may be implemented as Linux kernel modules, which may be one or more pieces of code that may be loaded on demand and extend the functionality of the Linux kernel.

Some embodiments may further include one or more buffers <NUM> that may be used to buffer data exchanged between a host and the operating system <NUM>, or a program that may be controlled (e.g., started, monitored, and/or the like) using the communication logic <NUM>. In some embodiments, the host and device <NUM> may be configured in a producer-consumer model. For example, a process controlled using the communication logic <NUM>, may operate as a producer and store output data in the one or more buffers (e.g., a ring buffer) <NUM>. A process such as a user process (e.g., an application running on a host) may operate as a consumer and retrieve output data from the one or more buffers <NUM>.

Any of the components illustrated in <FIG> including the interconnect interface <NUM>, device controller <NUM>, device functionality <NUM>, and one or more processors <NUM>, communication logic <NUM>, protocol interface <NUM>, and/or buffer <NUM> may be implemented with software, hardware, or any combination thereof including combinational logic, sequential logic, one or more timers, counters, registers, and/or state machines, any type of memory including volatile memories such as dynamic random access memory (DRAM) and/or static random access memory (SRAM), any type of nonvolatile memory including nonvolatile random access memory (NVRAM), flash memory, persistent memory, and/or the like or any combination thereof, one or more complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), central processing units (CPUs) such as complex instruction set computer (CISC) processors such as x86 processors and/or reduced instruction set computer (RISC) processors such as ARM processors, graphics processing units (GPUs), neural processing units (NPUs), tensor processing units (TPUs) and/or the like, executing instructions stored in any type of memory, or any combination thereof. In some embodiments, one or more components may be implemented as a system-on-chip (SOC).

In some embodiments, the interconnect interface <NUM> may include circuitry such as one or more transmitters, receivers, buffers, and/or the like to communicate with one or more hosts and/or other devices through the interconnect fabric <NUM>. In some embodiments, any of the device functionality <NUM> may include fixed and/or programmable functionality to perform any functions such as compression and/or decompression, encryption and/or decryption, microservices, erasure coding, video encoding and/or decoding, database acceleration, search, machine learning, graph processing, and/or the like.

In some embodiments, some or all of the functionality of the interconnect interface <NUM>, the protocol interface <NUM>, the device controller <NUM>, the one or more processors <NUM>, the buffer <NUM>, and/or the device functionality <NUM> may be integrated into a single component. For example, if the device <NUM> is implemented as a storage device, the device controller <NUM>, and at least one of the one or more processors may be integrated in a single component. As another example, if the device <NUM> is implemented as a computational storage device, one or more compute resources in the device functionality <NUM> (e.g., one or more processing elements that may be configured to perform computations on data stored in storage media at the device <NUM>) may be implemented as an FPGA, ASIC, and/or the like on which the operating system <NUM> may run.

In some embodiments, any of the terminal applications and/or terminals disclosed herein may implement connection-based and/or connectionless communications.

In some embodiments, accessing an operating system on a device through an interconnect may enable a user to some or all of the features provided by the operating system through a terminal including any or all of the following: a command line interface to a kernel of the operating system, resources to trace, debug, profile, and/or the like, a program running on the operating system (e.g., GNU debugger (GDB), Itrace, strace, dmesg, top, cpuinfo, ebpf tools, and/or the like), access to log messages (e.g., debug logs) from programs, management of the device (e.g., resources, configuration, and/or operation), and/or the like. Depending on the implementation details, one or more of these features may be accessed through an interconnect without the use of additional hardware such as a cable.

In some embodiments, accessing an operating system on a device through an interconnect may be especially beneficial with a device that may run microservices. For example, accessing the device operating system in accordance with example embodiments of the disclosure may enable a user to monitor and/or debug each microservice individually. Moreover, accessing the device operating system in accordance with example embodiments of the disclosure may enable a user to manage and/or control the microservices, for example, to start and/or halt one or more microservices (individually and/or in specific groups), configure microservices (individually and/or in specific groups), and/or the like.

<FIG> illustrates an example embodiment of a ring buffer scheme in accordance with example embodiments of the disclosure. The embodiment illustrated in <FIG> may be used, for example, to implement the one or more buffers <NUM> illustrated in <FIG>.

Referring to <FIG>, the ring buffer <NUM> may be implemented with memory locations (e.g., contiguous memory locations) that may be arranged as a ring (e.g., a bounded buffer) by looping the addressing of the beginning memory location to the ending memory address. In some embodiments, a host and device may be configured in a producer-consumer model. For example, a program controlled using communication logic may operate as a producer and store output data in the ring <NUM> buffer. The user (e.g., an application running on a host) may operate as a consumer and retrieve output data from the ring buffer <NUM>.

The ring buffer <NUM> may operate by placing new entries (e.g., output data such as logs and/or debug messages) into the ring buffer <NUM> at a memory location pointed to by a producer index <NUM>. A dot may indicate an occupied memory location. The producer index <NUM> may be incremented to point to the next available memory location each time a new entry is placed in the ring buffer <NUM>. Entries may be read from the ring buffer <NUM> at a memory location pointed to by a consumer index <NUM>. In some embodiments, the producer (e.g., a host) may ensure that it reads data from the location pointed to by the consumer index <NUM> before that location is overwritten with a new entry. If the producer index <NUM> and the consumer index <NUM> point to the same location, it may indicate that the ring buffer <NUM> is empty. In some embodiments, the ring buffer <NUM> may be located at a shared memory location that may be accessible to the host and the device. In some embodiments, the ring buffer <NUM> may be located at the host, at some other location, or distributed between multiple locations.

In some embodiments, the ring buffer <NUM> may notify a user (e.g., an application running on a host) that output data is available, for example, through a doorbell function which may be implemented with an interrupt (e.g., a message signaled interrupt (MSI)), a status bit in a status register, and/or the like.

<FIG> illustrates an example embodiment of a device register for a device in accordance with example embodiments of the disclosure. For purposes of illustration, the embodiment illustrated in <FIG> may be described in the context of a PCIe register, but the principles may be applied to any type of interconnect. In the embodiment illustrated in <FIG>, the three bits identified as Function Number may identify the function associated with a specific feature of a device. For example, if the embodiment of a device <NUM> illustrated <FIG> is illustrated with PCIe, a first register for the first function Function A configured for the storage device controller <NUM> may have bits <NUM> (function <NUM>) at bit locations <NUM>-<NUM>, and a second register for the second function Function B configured for the communication logic <NUM> may have bits <NUM> (function <NUM>) at bit locations <NUM>-<NUM>.

<FIG> illustrates an example embodiment of a query command and resulting output for a system having a device and device operating system that may be accessed using functions of an interconnect in accordance with example embodiments of the disclosure. For purposes of illustration, the embodiment illustrated in <FIG> illustrates a command line interface in which a command Ispci is used to determine the PCIe configuration of a system such as that illustrated in <FIG> running a Linux operating system, but other interconnects and/or operating systems may be used.

Referring to <FIG>, the first line illustrates the command Ispci used with a pipeline through a filter (e.g., grep) to narrow the results to entries from Manufacturer A. The output on the following two lines may have the following format: [bus]:[slot]. [function] [class]:[vendor and product]. Thus, the second line shows that a device controller (e.g., device controller <NUM> illustrated in <FIG>) may be configured for bus <NUM>, slot <NUM>, function <NUM> as a class of Non-Volatile memory controller from Manufacturer A as an NVMe SSD Controller. The third line shows that a debug controller (e.g., implemented by processor <NUM> using communication logic <NUM> illustrated in <FIG>) may be configured for bus <NUM>, slot <NUM>, function <NUM> as a class of Non-Volatile memory controller from Manufacturer A as a debug controller.

<FIG> illustrates an example embodiment of a user interface for a terminal program (which may also be referred to as a terminal application) that may be used to access a device operating system through an interconnect in accordance with example embodiments of the disclosure. For purposes of illustration, the embodiment illustrated in <FIG> may be illustrated in the context of a PCIe interconnect and implemented by modifying an existing terminal program such as PuTTY, but a terminal program for accessing a device operating system through an interconnect may be implemented with any other modified and/or custom software for any type of interconnect. Referring to <FIG>, the terminal program may provide a user with an option to connect to a device operating system using PCIe as shown by the radio button <NUM>.

<FIG> illustrates an example embodiment of a system in which a management interface may be used to access a device embedded operating system through an interconnect in accordance with example embodiments of the disclosure. The embodiment illustrated in <FIG> may include a first host <NUM>, a second host <NUM>, and a device <NUM>. The first host <NUM> and the device <NUM> may be connected through an interconnect fabric <NUM>. The second host <NUM> and the device <NUM> may be connected through an interconnect fabric <NUM>, which, in some embodiments, may be an extension of interconnect fabric <NUM>.

The first host <NUM> may include some components similar to those illustrated in <FIG> in which references numerals ending in the same digits may indicate components that may perform similar functions. However, the first host <NUM> illustrated in <FIG> may include an NVMe driver <NUM> configured to implement an NVMe protocol on top of a PCIe interconnect implemented by the interconnect interface <NUM>. The NVMe driver <NUM> may include a portion <NUM> that may enable the terminal application <NUM> to access the operating system <NUM> running on device <NUM> as a second function Function B through the (PCIe) interconnect interface <NUM>. The second function Function B may be implemented, for example, by communication logic <NUM> of the operating system <NUM>. In some embodiments, the portion <NUM> of the NVMe driver <NUM> may be integral with, or implemented in a similar manner as, the one or more drivers <NUM> illustrated in <FIG>.

The device <NUM> may include some components similar to those illustrated in <FIG> in which references numerals ending in the same digits may indicate components that may perform similar functions. However, the device <NUM> illustrated in <FIG> may include a bus interface <NUM>. Moreover, in the device <NUM> illustrated in <FIG>, the protocol interface <NUM> may implement an NVMe and NVMe Management Interface (NVMe-MI) protocol on top of a PCIe interconnect implemented by the interconnect interface <NUM>. In some embodiments, the NVMe and NVMe-MI interface <NUM> may have a driver that is integral with, or implemented in a similar manner as, the one or more drivers <NUM> illustrated in <FIG>.

The second host <NUM> may be implemented as a management controller such as a baseboard management controller (BMC). The second host <NUM> may include one or more processors <NUM>, an interconnect interface <NUM>, a remote access interface <NUM>, and a bus interface <NUM>. The one or more processors <NUM> may be configured to run a management application <NUM>, a terminal application <NUM>, and an NVMe-MI driver <NUM>. The interconnect interface <NUM> may be configured to implement a PCIe interconnect.

The bus interface <NUM> at the second host <NUM> may be configured to communicate with the bus interface <NUM> at the device through a management bus such as System Management Bus (SMBus) and/or Inter-Integrated Circuit (I2C) bus. This management bus connection may enable the management application <NUM> to manage one or more administrative aspects of the device <NUM> such as configuration, system startup, system reset, firmware updates and/or upgrades, power monitoring and/or management, and/or the like. The remote access interface <NUM> may enable a user to access the one or more processors <NUM> through a remote access connection <NUM> using an interface and/or protocol such as Ethernet, TCP/IP, RDMA, ROCE, FibreChannel, InfiniBand, and/or the like.

The NVMe-MI driver <NUM> at the second host <NUM> may include a portion <NUM> that may be integral with, or implemented in a similar manner as, the driver <NUM> illustrated in <FIG>. Thus, in addition to, or as an alternative to, the management bus <NUM>, the one or more processors <NUM> in the second host <NUM> may access the device operating system <NUM> through the (PCIe) interconnect fabric <NUM> and/or <NUM> using the NVMe-MI protocol.

The terminal application <NUM> may also access the device operating system <NUM> through the (PCIe) interconnect fabric <NUM> and/or <NUM> using the NVMe-MI protocol. Moreover, if the remote access interface <NUM> is configured to access the terminal application <NUM>, a user may remotely access the device operating system <NUM> through the (PCIe) interconnect fabric using the NVMe-MI protocol. Thus, the user may access some or all of the features enabled by the operating system <NUM> as described above.

In some embodiments, a switch may be located at the intersection of the interconnect fabric <NUM> and <NUM>. Alternatively, the interconnect fabric <NUM> may connect to a separate interconnect port of the device <NUM>. Although the embodiment illustrated in <FIG> is shown with two hosts, in other embodiments, only one of the hosts may be included, or additional hosts may be included.

<FIG> illustrates an embodiment of a system that may provide a user with remote access to an embedded operating system of a device in accordance with example embodiments of the disclosure. The system illustrated in <FIG> may include components similar to those illustrated in <FIG> in which references numerals ending in the same digits may indicate components that may perform similar functions. However, in the embodiment illustrated in <FIG>, the host <NUM> may include a remote access interface <NUM>. The remote access interface <NUM> may enable a user to access the one or more processors <NUM> through a remote access connection <NUM> using an interface and/or protocol such as Ethernet, TCP/IP, RDMA, ROCE, FibreChannel, InfiniBand, and/or the like. Moreover, if the remote access interface <NUM> is configured to access the terminal application <NUM>, a user may remotely access the operating system <NUM> of the device <NUM> through the (PCIe) interconnect fabric using the NVMe-MI protocol. Thus, the user may remotely access some or all of the features exposed by the communication logic <NUM> as described above.

In some embodiments, one or more of the devices disclosed herein may be implemented as a storage device. Any such storage devices may be implemented in any form factor such as <NUM> inch, <NUM> inch, <NUM> inch, M. <NUM>, Enterprise and Data Center SSD Form Factor (EDSFF), NF1, and/or the like, using any connector configuration such as Serial ATA (SATA), Small Computer System Interface (SCSI), Serial Attached SCSI (SAS), U. <NUM>, and/or the like.

Any of the devices disclosed herein may be implemented entirely or partially with, and/or used in connection with, a server chassis, server rack, dataroom, datacenter, edge datacenter, mobile edge datacenter, and/or any combinations thereof.

<FIG> illustrates an example embodiment of a host apparatus that may be used to provide a user with access to a device operating system through an interconnect in accordance with example embodiments of the disclosure. The host apparatus <NUM> illustrated in <FIG> may include a processor <NUM>, which may include a memory controller <NUM>, a system memory <NUM>, access logic <NUM>, and/or an interconnect interface <NUM>, which may be implemented, for example using CXL. Any or all of the components illustrated in <FIG> may communicate through one or more system buses <NUM>. In some embodiments, one or more of the components illustrated in <FIG> may be implemented using other components. For example, in some embodiments, the access logic <NUM> may be implemented by the processor <NUM> executing instructions stored in the system memory <NUM> or other memory. In some embodiments, the access logic <NUM> may implement a terminal application, a device driver, and/or the like to enable a user to access a device operating system through an interconnect.

<FIG> illustrates an example embodiment of a storage device that may be used to provide a user with access to a device program in accordance with example embodiments of the disclosure. The storage device <NUM> may include a device controller <NUM>, a media translation layer <NUM>, a storage media <NUM>, access logic <NUM>, and an interconnect interface <NUM>. The components illustrated in <FIG> may communicate through one or more device buses <NUM>. In some embodiments that may use flash memory for some or all of the storage media <NUM>, the media translation layer <NUM> may be implemented partially or entirely as a flash translation layer (FTL). In some embodiments, the storage device <NUM> illustrated in <FIG> may be used to implement any of the device-side functionality relating to providing a user with access to a device terminal through an interconnect disclosed herein. For example, in some embodiments, the access logic <NUM> may implement communication logic (e.g., one or more terminal support drivers) to enable a user to access an operating system running on the storage device <NUM> through an interconnect. In some other embodiments, the storage device <NUM> may alternatively or additionally be implemented as any other type of device such as an accelerator, NIC, and/or the like.

Any of the functionality described herein, including any of the host functionality, device functionally, and/or the like described above may be implemented with hardware, software, or any combination thereof including combinational logic, sequential logic, one or more timers, counters, registers, state machines, volatile memories such as DRAM and/or SRAM, nonvolatile memory and/or any combination thereof, CPLDs, FPGAs, ASICs, CPUs including CISC processors such as x86 processors and/or RISC processors such as ARM processors, GPUs, NPUs, and/or the like, executing instructions stored in any type of memory. In some embodiments, one or more components may be implemented as a system-on-chip (SOC).

<FIG> illustrates an embodiment of a method for providing a user with access to a device terminal in accordance with example embodiments of the disclosure. The method may begin at operation <NUM>. At operation <NUM>, the method may run, at a device, an operating system. For example, a device such as a storage device, an accelerator, a NIC, and/or the like may run an embedded operating system. At operation <NUM>, the method may communicate, using a first function of an interconnect, with the device. For example, a device controller may be configured as function <NUM> of a PCIe interconnect. At operation <NUM>, the method may communicate, using a second function of the interconnect, with the operating system. For example, the terminal may be configured as function <NUM> of a PCIe interconnect. The method may end at operation <NUM>.

The embodiment illustrated in <FIG>, as well as all of the other embodiments described herein, are example operations and/or components. In some embodiments, some operations and/or components may be omitted and/or other operations and/or components may be included. Moreover, in some embodiments, the temporal and/or spatial order of the operations and/or components may be varied. Although some components and/or operations may be illustrated as individual components, in some embodiments, some components and/or operations shown separately may be integrated into single components and/or operations, and/or some components and/or operations shown as single components and/or operations may be implemented with multiple components and/or operations.

Claim 1:
A method (<NUM> - <NUM>) for communicating with a device (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>), the method comprising:
running (<NUM>), at a device (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>), an operating system (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>) comprising a communication logic (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>);
communicating (<NUM>), by a host (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>), using a first function of a PCIE, Peripheral Component Interconnect Express, interface (<NUM>-<NUM>), with a device controller (<NUM>, <NUM>, <NUM>) of the device (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>); and
communicating (<NUM>), by the host (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>), using a second function of the PCIe interface (<NUM>-<NUM>), with the communication logic (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>),
wherein the communicating (<NUM>) with the communication logic (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>) comprises storing, by the device (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>), in a ring buffer (<NUM>, <NUM>), output data associated with the second function of the interconnect (<NUM>-<NUM>).