Patent Publication Number: US-2023145162-A1

Title: Queue protection using a shared global memory reserve

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     This application is a continuation of U.S. patent application Ser. No. 15/250,860, filed on Aug. 29, 2016, of which is hereby expressly incorporated by reference in its entirety. 
    
    
     TECHNICAL FIELD 
     The subject technology pertains to managing memory resources in a network switch and in particular, for managing a shared buffer memory amongst multiple queues in a shared memory network switch. 
     BACKGROUND 
     Several different architectures are commonly used to build packet switches (e.g., IP routers, ATM switches and Ethernet switches). One architecture is the output queue (OQ) switch, which places received packets in various queues that are dedicated to outgoing ports. The packets are stored in their respective queues until it is their turn to depart (e.g. to be “popped”). While various types of OQ switches have different pros and cons, a shared memory architecture is one of the simplest techniques for building an OQ switch. In some implementations, a shared memory switch functions by storing packets that arrive at various input ports of the switch into a centralized shared buffer memory. When the time arrives for the packets to depart, they are read from the shared buffer memory and sent to an egress line. 
     There are various techniques for managing a shared memory buffer. In some memory management solutions, the network switch prevents any single output queue from taking more than a specified share of the buffer memory when the buffer is oversubscribed, and permits a single queue to take more than its share to handle incoming packet bursts if the buffer is undersubscribed. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       In order to describe the manner in which the above-recited and other advantages and features of the disclosure can be obtained, a more particular description of the principles briefly described above will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings. Understanding that these drawings depict only exemplary embodiments of the disclosure and are not therefore to be considered to be limiting of its scope, the principles herein are described and explained with additional specificity and detail through the use of the accompanying drawings in which: 
         FIG.  1 A  graphically illustrates an example of queue occupancy relative to free buffer memory in a shared memory network switch. 
         FIGS.  1 B and  1 C  illustrate examples of memory allocation tables that indicate occupancy for various queues, as well as a total available free memory resource for a shared buffer memory. 
         FIG.  2    illustrates an example flow chart for implementing a shared buffer memory allocation algorithm utilizing a global shared reserve, according to some aspects of the technology. 
         FIG.  3 A  illustrates an example table of queue occupancy levels for multiple queues implementing a global shared reserve memory management technique, according to some aspects of the technology. 
         FIG.  3 B  graphically illustrates an example of memory allocated to a shared buffer memory by various queues using a global shared reserve memory management technique, according to some aspects of the technology. 
         FIG.  4    illustrates an example network device. 
         FIGS.  5 A and  5 B  illustrate example system embodiments. 
     
    
    
     DESCRIPTION OF EXAMPLE EMBODIMENTS 
     Various embodiments of the disclosure are discussed in detail below. While specific implementations are discussed, it should be understood that this is done for illustration purposes only. A person skilled in the relevant art will recognize that other components and configurations can be used without parting from the spirit and scope of the disclosure. 
     Overview 
     One problem with managing shared memory space amongst multiple queues is to ensure that active queues (i.e., “aggressor queues”) do not occupy the entire memory and thereby impede buffer access by other queues. Queues that are prevented from enqueue due to limited buffer space are referred to herein as “victim queues.” In a shared memory switch, an algorithm is required to prevent any single queue from taking more than its fair allocation of shared memory. In some memory management solutions, the algorithm calculates a dynamic maximum threshold by multiplying the amount of unallocated/free memory in the shared memory by a parameter (e.g., “alpha”). Typically values of alpha range between 0.5 and 2.0. 
     With alpha set to 1.0 consider a single oversubscribed queue: the system stabilizes with the queue and the free memory both being the same size, i.e., the queue can consume only half of memory. With  2  oversubscribed queues the queues can each have ⅓ rd  of the memory and ⅓ rd  remains unallocated, and so on up to N oversubscribed queues, where each queue will have 1/(N+1) of the memory and 1/(N+1) will remain unallocated. An example of the relative memory allocation amongst multiple queues is discussed in further detail with respect to  FIG.  1 A , below. 
     In some data center deployments, the buffer is required to be able to absorb large data bursts into a single queue (e.g., incast burst absorption). So the “alpha” parameter (which is programmable), is usually set to greater than 1, typically 9 (e.g., 90% of the buffer). With this setting, few aggressor queues/concurrent bursts could consume the entire buffer, and any new incoming traffic is dropped (e.g. a tail-drop), affecting throughput for victim queues. 
     Another solution is to provide a dedicated memory allocation for each queue (e.g., a minimum reserve), and reduce the total shareable buffer space by the sum of all minimum reserves. Depending on implementation, this can result in carving out a section of the buffer memory that isn&#39;t efficiently used. Additionally, the amount of reserved buffer space is a function of the number of ports and classes of service required, so as the number of ports/services scale, dedicated memory allocations become increasingly likely to deplete available memory. 
     DESCRIPTION 
     Aspects of the subject technology address the foregoing problem by providing memory management systems, methods and computer-executable instructions to facilitate packet storage using a shared buffer memory. In particular, the disclosed technology provides a packet enqueuing method which requires certain preconditions before a received packet can be enqueued. In some aspects, the decision of whether or not to enqueue a packet is first based on a fill level of the shared buffer memory. That is, if an occupancy of the queue in the shared buffer memory is below a pre-determined dynamic queue threshold (e.g., a “dynamic queue maximum” or “dynamic queue MAX”), then the packet is enqueued. 
     Alternatively, in instances where the queue occupancy in the shared buffer exceeds the dynamic queue max threshold, then further conditions may be verified before the packet is enqueued (or dropped). As discussed in further detail below, if the fill level of the queue in the shared buffer memory exceeds the dynamic queue max threshold, then an occupancy of the referring queue may be compared to static queue threshold (e.g., a “static queue minimum” or “static queue MIN”), to determine if the packet can still be enqueued. 
     As used herein, the dynamic queue maximum refers to a measure of shared buffer occupancy for the entire shared buffer memory. Thus, the dynamic queue max can be understood as a function of total free/available memory in the buffer. As discussed in further detail below, the static queue minimum threshold is a threshold that relates to a minimum amount of memory in the shared buffer that is allocated for use by victim queues. 
       FIG.  1 A  graphically illustrates an example of queue occupancy levels relative to a free shared buffer memory allocation in a network switch. For example, shared buffer  102  illustrates an example in which a shared buffer occupancy is maximally allocated at ½ of the total memory capacity, i.e., for a single queue wherein alpha=1.0. Shared buffer  104  illustrates an example of a total buffer allocation for two total queues, wherein the respective shared buffer memory allocation for each is ⅓ rd  of the of the total buffer size. Similarly, shared buffer  106  illustrates an example of a shared buffer allocation amongst N total queues. 
       FIG.  1 B  illustrates an example memory allocation table  108  that indicates occupancy for multiple queues, as well as a total free memory for a shared buffer. 
     In particular, the example of  FIG.  1 B  illustrates a memory management scheme in which any queue is permitted to occupy the entirety of shared buffer memory. As table  108  illustrates, this memory management method can be problematic due to the fact that aggressor queues can rapidly utilize the entirety of shared memory space, consequently halting the ability for victim queues to enqueue additional incoming packets. 
     By way of example, table  108  illustrates various occupancy levels for multiple queues (i.e., Q 0 , Q 1 , and Q 2 ), such that any individual queue is permitted to utilize all available free memory. This scenario is demonstrated, for example, at time=T 5  where Q 0 , and Q 1  occupy 90% and 9% of the total memory, respectively (leaving a total free memory of 1%). 
       FIG.  1 C  illustrates an example memory allocation table  110  that indicates occupancy levels for multiple queues using a memory management technique that employs a dedicated “minimum reserve” for each respective queue. As discussed above, such solutions can also be sub-optimal due to the fact that some amount of shared buffer memory can be persistently reserved for inactive queues, even when memory resources are needed elsewhere. For example, using a per-queue memory reservation technique depicted by  FIG.  1 C  (e.g., with alpha=9) a total of 25% of the total buffer memory is reserved for various queues. 
     By way of example, table  110  illustrates this scenario at time=T 7 , where Q 0  occupancy is at 69 (e.g., 69% of the shared buffer size), and Q 1  occupancy is at 8 (e.g., 8% of the shared buffer size), however, dynamic queue max=0, indicating that free memory (e.g., total free=23) is no longer available to other aggressor queues. Therefore, in this scenario, a total of 23% of the shared buffer memory is unallocated if all victim queues are unutilized. 
     As discussed above, aspects of the subject technology address the foregoing limitations of conventional buffer memory management techniques, by providing a shared buffer memory in which packet enqueuing is dependent upon the verification of various conditions, for example, relating to a fill level of the shared buffer (e.g., a dynamic queue max threshold), as well as comparisons between a fill level of a referring queue and a threshold related to a reserved apportionment of buffer resources (e.g., a static queue min threshold). 
       FIG.  2    illustrates an example flow chart for implementing a method  200  for allocating shared buffer memory in a network switch. Method  200  begins at step  202 , in which a determination is made as to whether there is any available (unallocated) memory in the shared buffer of a network switch. If it is determined that no free memory is available, method  200  proceeds to step  204 , and any newly arriving packets are dropped. 
     Alternatively, if it is determined that the shared buffer memory contains unallocated space, method  200  proceeds to step  206 , in which a determination is made as to whether any shared buffer space is available in the shared buffer memory. 
     If it is determined in step  206  that no memory in the shared buffer is available, then method  200  proceeds to step  208 , in which a determination is made as to whether or not the occupancy of the referring queue is below a predetermined static queue minimum, e.g., a “static queue MIN” threshold, as discussed above. In some aspects, the static queue MIN threshold is a predetermined threshold used to define a minimum threshold, above which the received data/packets from a referring queue cannot be accepted into the shared buffer memory. As such, if in step  208  it is determined that the referring queue occupancy is not less than the static queue minimum, then method  200  proceeds to step  204  and incoming packet/s are dropped. 
     Alternatively, if in step  208  it is determined that the referring queue occupancy is less than the static queue MIN threshold, method  200  proceeds to step  212 , and data from the referring queue is stored in a “reserved portion” of the shared buffer memory. It is understood herein that the reserved portion of buffer memory (or “global reserve”) refers to a logical allotment of memory space in the shared buffer. However, it is not necessary that the global reserve portions of memory be physically distinct memory spaces that are separate, for example, from various other regions in the shared memory buffer. 
     Referring back to step  206 , if it is determined that shared memory space is available, then method  200  proceeds to step  210  in which a determination is made as to whether the queue occupancy is less than a dynamic queue threshold (e.g., “dynamic queue MAX”). As used herein, the dynamic queue max is a threshold that defines a cutoff, above which data from an aggressor queue cannot be admitted into the shared buffer memory. Because the dynamic queue max is a function of unallocated memory space in the shared buffer memory, in some aspects the dynamic queue max threshold may be conceptualized as a function of queue activity for each associated queue in the network switch. 
     If in step  210  it is determined that the queue occupancy is less than the dynamic queue max, then method  200  proceeds to step  212  and the packet/s are stored in the buffer memory. Alternatively, if it is determined that the queue occupancy (e.g., queue allocation) is greater than the dynamic queue max threshold, then method  200  proceeds to step  208 , where it is determined if the referring queue occupancy is less than the static queue minimum (see above). 
     By providing a global reserve buffer available to any referring queue that has less than a specified occupancy level, the subject memory management techniques permit data storage in the shared buffer by less active (victim) queues, even in instances where the majority of buffer storage space has been filed by aggressor queues. 
       FIG.  3 A  illustrates an example table  301  of queue occupancy levels for multiple queues implementing a shared memory management technique, as discussed above. By implementing the memory management scheme discussed with respect to method  200  above, victim queues with relatively low throughput (as compared to aggressor queues) can access portions of shared buffer memory that would otherwise be unavailable in other shared memory schemes. As illustrated in table  301 , for example, at time=T 5 , victim queue (Q 1 ) is able to occupy some amount of the total buffer memory (e.g., 9%), although aggressor queue Q 1  has occupied most of the shared buffer. A similar example is graphically illustrated with respect to  FIG.  3 B . 
     Specifically,  FIG.  3 B  illustrates an example of the apportionment of a memory  303  amongst multiple queues (e.g., first queue  309 , second queue  311 , and third queue  313 ), using a global shared reserve management technique, according to aspects of the subject technology. As illustrated, memory  303  is logically apportioned into a dynamic allocation  305  and a global reserve  307 . As discussed above, dynamic allocation  305  can be a shared resource available to any aggressor queue until an occupancy level of that queue has reached a pre-determined threshold (e.g., a dynamic queue max). However, the global reserve  307  of memory  303  remains reserved for low volume or victim queues, so long as the occupancy of the referring queue does not exceed a pre-determined threshold governing storage to the global reserve (e.g., a static queue min threshold), as discussed above with respect to step  208  of method  200 . 
     By way of example, the occupancy of buffer memory  303  in the example of  FIG.  3 B  illustrates storage by three different queues. The storage of data pertaining to first queue  309  is managed solely within dynamic allocation  305 . The storage of data pertaining to second queue  311  is shared amongst dynamic allocation  305  and global reserve  307 , and data associated with third queue  313  is stored exclusively into global reserve  307 . 
     As discussed above, storage of data from second queue  311  first began by storing data to dynamic allocation  305 , until occupancy of dynamic allocation  305  was complete. After dynamic allocation  305  reached capacity, a determination was made as to whether the remaining data in second queue  311  was smaller than a static queue threshold, necessary to admit the data into the global reserve. Lastly, data from third queue  313 , which could not have been stored to dynamic allocation  305  (due to its fill state), was exclusively stored into global reserve  307 . 
     By maintaining global reserve  307  portion of buffer memory  303 , the disclosed memory management technique provides for a minimal apportionment of shared buffer space that is continuously available to victim queues. 
     Example Devices 
       FIG.  4    illustrates an example network device  410  suitable for high availability and failover. Network device  410  includes a master central processing unit (CPU)  462 , interfaces  468 , and a bus  415  (e.g., a PCI bus). When acting under the control of appropriate software or firmware, the CPU  462  is responsible for executing packet management, error detection, and/or routing functions. The CPU  462  preferably accomplishes all these functions under the control of software including an operating system and any appropriate applications software. CPU  462  may include one or more processors  463  such as a processor from the Motorola family of microprocessors or the MIPS family of microprocessors. In an alternative embodiment, processor  463  is specially designed hardware for controlling the operations of router  410 . In a specific embodiment, a memory  461  (such as non-volatile RAM and/or ROM) also forms part of CPU  462 . However, there are many different ways in which memory could be coupled to the system. 
     The interfaces  468  are typically provided as interface cards (sometimes referred to as “line cards”). Generally, they control the sending and receiving of data packets over the network and sometimes support other peripherals used with the router  410 . Among the interfaces that may be provided are Ethernet interfaces, frame relay interfaces, cable interfaces, DSL interfaces, token ring interfaces, and the like. In addition, various very high-speed interfaces may be provided such as fast token ring interfaces, wireless interfaces, Ethernet interfaces, Gigabit Ethernet interfaces, ATM interfaces, HSSI interfaces, POS interfaces, FDDI interfaces and the like. Generally, these interfaces may include ports appropriate for communication with the appropriate media. In some cases, they may also include an independent processor and, in some instances, volatile RAM. The independent processors may control such communications intensive tasks as packet switching, media control and management. By providing separate processors for the communications intensive tasks, these interfaces allow the master microprocessor  462  to efficiently perform routing computations, network diagnostics, security functions, etc. 
     Although the system shown in  FIG.  4    is one specific network device of the present invention, it is by no means the only network device architecture on which the present invention can be implemented. For example, an architecture having a single processor that handles communications as well as routing computations, etc. is often used. Further, other types of interfaces and media could also be used with the router. 
     Regardless of the network device&#39;s configuration, it may employ one or more memories or memory modules (including memory  461 ) configured to store program instructions for the general-purpose network operations and mechanisms for roaming, route optimization and routing functions described herein. The program instructions may control the operation of an operating system and/or one or more applications, for example. The memory or memories may also be configured to store tables such as mobility binding, registration, and association tables, etc. 
       FIG.  5 A  and  FIG.  5 B  illustrate example system embodiments. The more appropriate embodiment will be apparent to those of ordinary skill in the art when practicing the present technology. Persons of ordinary skill in the art will also readily appreciate that other system embodiments are possible. 
       FIG.  5 A  illustrates a conventional system bus computing system architecture  500  wherein the components of the system are in electrical communication with each other using a bus  505 . Exemplary system  500  includes a processing unit (CPU or processor)  510  and a system bus  505  that couples various system components including the system memory  515 , such as read only memory (ROM)  520  and random access memory (RAM)  525 , to the processor  510 . The system  500  can include a cache of high-speed memory connected directly with, in close proximity to, or integrated as part of the processor  510 . The system  500  can copy data from the memory  515  and/or the storage device  530  to the cache  512  for quick access by the processor  510 . In this way, the cache can provide a performance boost that avoids processor  510  delays while waiting for data. These and other modules can control or be configured to control the processor  510  to perform various actions. Other system memory  515  may be available for use as well. The memory  515  can include multiple different types of memory with different performance characteristics. The processor  510  can include any general purpose processor and a hardware module or software module, such as module  1   532 , module  2   534 , and module  3   536  stored in storage device  530 , configured to control the processor  510  as well as a special-purpose processor where software instructions are incorporated into the actual processor design. The processor  510  may essentially be a completely self-contained computing system, containing multiple cores or processors, a bus, memory controller, cache, etc. A multi-core processor may be symmetric or asymmetric. 
     To enable user interaction with the computing device  500 , an input device  545  can represent any number of input mechanisms, such as a microphone for speech, a touch-sensitive screen for gesture or graphical input, keyboard, mouse, motion input, speech and so forth. An output device  535  can also be one or more of a number of output mechanisms known to those of skill in the art. In some instances, multimodal systems can enable a user to provide multiple types of input to communicate with the computing device  500 . The communications interface  540  can generally govern and manage the user input and system output. There is no restriction on operating on any particular hardware arrangement and therefore the basic features here may easily be substituted for improved hardware or firmware arrangements as they are developed. 
     Storage device  530  is a non-volatile memory and can be a hard disk or other types of computer readable media which can store data that are accessible by a computer, such as magnetic cassettes, flash memory cards, solid state memory devices, digital versatile disks, cartridges, random access memories (RAMs)  525 , read only memory (ROM)  520 , and hybrids thereof. 
     The storage device  530  can include software modules  532 ,  534 ,  536  for controlling the processor  510 . Other hardware or software modules are contemplated. The storage device  530  can be connected to the system bus  505 . In one aspect, a hardware module that performs a particular function can include the software component stored in a computer-readable medium in connection with the necessary hardware components, such as the processor  510 , bus  505 , display  535 , and so forth, to carry out the function. 
       FIG.  5 B  illustrates an example computer system  550  having a chipset architecture that can be used in executing the described method and generating and displaying a graphical user interface (GUI). Computer system  550  is an example of computer hardware, software, and firmware that can be used to implement the disclosed technology. System  550  can include a processor  555 , representative of any number of physically and/or logically distinct resources capable of executing software, firmware, and hardware configured to perform identified computations. Processor  555  can communicate with a chipset  560  that can control input to and output from processor  555 . In this example, chipset  560  outputs information to output device  565 , such as a display, and can read and write information to storage device  570 , which can include magnetic media, and solid state media, for example. Chipset  560  can also read data from and write data to RAM  575 . A bridge  580  for interfacing with a variety of user interface components  585  can be provided for interfacing with chipset  560 . Such user interface components  585  can include a keyboard, a microphone, touch detection and processing circuitry, a pointing device, such as a mouse, and so on. In general, inputs to system  550  can come from any of a variety of sources, machine generated and/or human generated. 
     Chipset  560  can also interface with one or more communication interfaces  590  that can have different physical interfaces. Such communication interfaces can include interfaces for wired and wireless local area networks, for broadband wireless networks, as well as personal area networks. Some applications of the methods for generating, displaying, and using the GUI disclosed herein can include receiving ordered datasets over the physical interface or be generated by the machine itself by processor  555  analyzing data stored in storage  570  or  575 . Further, the machine can receive inputs from a user via user interface components  585  and execute appropriate functions, such as browsing functions by interpreting these inputs using processor  555 . 
     It can be appreciated that example systems  500  and  550  can have more than one processor  510  or be part of a group or cluster of computing devices networked together to provide greater processing capability. 
     For clarity of explanation, in some instances the present technology may be presented as including individual functional blocks including functional blocks comprising devices, device components, steps or routines in a method embodied in software, or combinations of hardware and software. 
     In some embodiments the computer-readable storage devices, mediums, and memories can include a cable or wireless signal containing a bit stream and the like. However, when mentioned, non-transitory computer-readable storage media expressly exclude media such as energy, carrier signals, electromagnetic waves, and signals per se. 
     Methods according to the above-described examples can be implemented using computer-executable instructions that are stored or otherwise available from computer readable media. Such instructions can comprise, for example, instructions and data which cause or otherwise configure a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions. Portions of computer resources used can be accessible over a network. The computer executable instructions may be, for example, binaries, intermediate format instructions such as assembly language, firmware, or source code. Examples of computer-readable media that may be used to store instructions, information used, and/or information created during methods according to described examples include magnetic or optical disks, flash memory, USB devices provided with non-volatile memory, networked storage devices, and so on. 
     Devices implementing methods according to these disclosures can comprise hardware, firmware and/or software, and can take any of a variety of form factors. Typical examples of such form factors include laptops, smart phones, small form factor personal computers, personal digital assistants, rackmount devices, standalone devices, and so on. Functionality described herein also can be embodied in peripherals or add-in cards. Such functionality can also be implemented on a circuit board among different chips or different processes executing in a single device, by way of further example. 
     The instructions, media for conveying such instructions, computing resources for executing them, and other structures for supporting such computing resources are means for providing the functions described in these disclosures. 
     Although a variety of examples and other information was used to explain aspects within the scope of the appended claims, no limitation of the claims should be implied based on particular features or arrangements in such examples, as one of ordinary skill would be able to use these examples to derive a wide variety of implementations. Further and although some subject matter may have been described in language specific to examples of structural features and/or method steps, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to these described features or acts. For example, such functionality can be distributed differently or performed in components other than those identified herein. Rather, the described features and steps are disclosed as examples of components of systems and methods within the scope of the appended claims. Moreover, claim language reciting “at least one of” a set indicates that one member of the set or multiple members of the set satisfy the claim.