Patent Publication Number: US-7724654-B2

Title: Method for synchronized trunk failover and failback in a FC-AL switching environment

Description:
FIELD OF THE INVENTION 
   This invention relates to trunking in Fibre Channel Arbitrated Loop (FC-AL) switches, and more particularly, to synchronizing trunk failover and failback to ensure that only one loop initialization occurs, and that loop initialization completes properly and promptly. 
   BACKGROUND OF THE INVENTION 
     FIG. 1  illustrates an exemplary Storage Area Network (SAN)  100  comprised of two FC-AL switches S 1  and S 2 , each of which may be a Switch On a Chip (SOC). A number of drives  102  and initiators I 1  and I 2  are attached to each FC-AL switch S 1  and S 2  through node ports  104 . Each FC-AL switch S 1  and S 2 , when connected to drives and initiators, may form a Switched Bunch Of Disks (SBOD)  120 . A processor  108  is also coupled to each switch for executing firmware  122  to perform operations on the switch, among other things. 
   In addition, cascade ports  106  on the FC-AL switches S 1  and S 2  enable the switches to be interconnected via two trunks T 1  and T 2 . Trunking involves the use of multiple inter-switch connections for increased throughput and redundancy. In the example of  FIG. 1 , trunk T 1  is designated as the primary trunk, and trunk T 2  is designated as the duplicate trunk. Note that the designation of trunks as primary and duplicate does not merely represent a difference in names, but represents substantive differences between the two trunks. In the exemplary configuration of  FIG. 1 , the initiators I 1  and I 2  connected to S 1  are able to communicate with devices connected to S 2  (e.g. drive D 1 ) through the trunks T 1  and T 2 . 
   However, primary trunk T 1  may fail (see  110  in  FIG. 1 ) for a number of reasons. In conventional FC-AL switches, each FC-AL switch (switch S 1  and S 2  in the example of  FIG. 1 ) responds to the failure independently. When hardware in S 1  recognizes the failure, it enters a failover mode and initiates a failover event. Trunk failover and failback is the ability to redirect traffic from a failed trunk to a working trunk and/or back to the original primary trunk once it is re-established. Failover and fail back is managed and driven by an Application Programming Interface (API)  124 , which is used to manage each of the switches independently in separate enclosures. 
   When the failover event is initiated, a bypass event occurs in which the cascade port  106  coupled to the trunk T 1  is changed from an “inserted” state (i.e. device connected) to a “bypass” state (i.e. device not connected) according to FC protocols well-understood by those skilled in the art. The hardware in S 1  also generates and sends an interrupt to the processor  108  connected to S 1 , which recognizes that T 1  is no longer available but that duplicate trunk T 2  exists, and reconfigures the switch to establish T 2  as the primary trunk. The processor  108  then initiates a Loop Initialization process to all devices in the SAN  100  to initialize them. When devices receive the Loop Initialization Primitive (LIP) ordered sets at the start of the Loop Initialization cycle, they cease normal communications and enter an initialization state by sending specific ordered sets and frames out over the loop and actively reserving an address, which is necessary to establish subsequent normal communications with other devices on the loop. Normal communications may be resumed only when the Loop Initialization has completed and the participating devices have reserved a new address. 
   Because each switch is managed by a separate instance of the API, installed in separate enclosures, trunk failures and re-inserts between two enclosures may not be detected simultaneously by the APIs. As a result, failover and fail back may not take place simultaneously; instead, these often occur sequentially, separated by up to approximately 40 ms of time, depending on the rate at which firmware accesses the API. These sequential failover or fail back events cause multiple loop initializations to be triggered which can lead to multiple system issues. 
   If S 1  recognizes the failure before S 2 , it is possible that the LIP ordered sets generated by S 1  may cause all devices attached to S 1  to be initialized before switch S 2  recognizes the failure and enters a failover mode. However, because S 2  has not yet recognized the failure and reconfigured itself to establish T 2  as the primary trunk, when S 1  tries to send LIP ordered sets down the duplicate trunk T 2  (see  112  in  FIG. 1 ), they will be blocked. (Note that duplicate trunks are not allowed to participate in loop initialization, and therefore block LIP primitives and loop initialization frames.) When S 2  finally recognizes the failure and enters the failover mode, it reconfigures itself to establish T 2  as the primary trunk, and may generate its own LIP ordered sets which causes all devices attached to S 2  to be initialized. Furthermore, because S 1  had already recognized the failure and had established T 2  as the primary trunk, S 2  is able to send LIP ordered sets up T 2  (see  114  in  FIG. 1 ), which results in the devices connected to S 1  to be initialized a second time, a duplication of effort with respect to the devices connected to S 1 . 
   In the scenario described above, two independent failover events including multiple LIP ordered sets are generated. During a failover event, no data can be communicated between devices attached to the switches S 1  and S 2 . This time period where no regular communication is possible is approximately the time difference between when the first switch recognized the failure and when the second switch recognized the failure (which is somewhat random and non-deterministic), plus the time it takes for Loop Initialization to complete in both switches, and may represent a potentially severe disruption to data communications. 
   Note that the possibility of multiple failover events caused by a trunk down event only occurs when the switches are FC-AL switches that act like hubs to connect the devices as though they were in a regular shared arbitrated loop, yet allow multiple switches to be cascaded using trunks. Multiple failover events are a side effect of the capability of the FC-AL switches, and occur because the devices connected to the switches were not designed to handle non-deterministic failover modes. 
   Therefore, there is a need to be able to synchronize and coordinate trunk failover and failback between the two FC-AL switches when a trunk failure occurs and the primary trunk designation is changed in order to minimize the disruption to data communications caused by multiple unnecessary loop initialization cycles. 
   SUMMARY OF THE INVENTION 
   Embodiments of the present invention synchronize and coordinate trunk failover and failback between the two FC-AL switches when a trunk failure occurs and the primary trunk designation is changed in order to minimize the disruption to data communications caused by multiple unnecessary loop initialization cycles. Failover and fail back may be managed and driven by an API, which is used to manage each of the switches independently. 
   In a system comprised of FC-AL switches S 1  and S 2  connected together via primary trunk T 1  and duplicate trunk T 2 , if T 1  should fail and the T 1  failure is detected by S 1 , S 1  initiates a failover event by performing a port bypass event to change the cascade port coupled to T 1  to a bypass state and bypass the cascade port. S 1  then causes a MaRK (MRK) ordered set to be sent out over T 2  to S 2 . After S 1  sends the MRK ordered set, it waits to receive an acknowledgement of the MRK ordered set from S 2 . 
   If S 2  receives MRK (00, EF), it acknowledges receipt by sending an acknowledgement MRK ordered set over T 2  back to S 1 . If S 1  receives the acknowledgement MRK ordered set, S 1  reconfigures the switch to establish T 2  as the primary trunk as seen from the perspective of both S 1  and S 2 . The processor connected to S 1  then acts as a master in the failover process and initiates LIP (F 7 , F 7 ) ordered sets which are communicated to all devices in the system (all devices connected to S 1  and S 2 ) to initialize them. When devices receive the LIP ordered sets, they reserve an address, which is necessary to establish subsequent communications with the devices. Once the devices have been initialized and the Loop Initialization cycle is complete, S 1  can return to a normal operation state. 
   Note that when S 2  receives a MRK ordered set and acknowledges it by sending an acknowledgement MRK ordered set back to S 1 , it acts as a slave in the failover process and does not attempt to initiate LIPs, thereby eliminating the possibility of multiple Loop Initialization cycles and reducing the time in which data cannot be transmitted. 

   
     BRIEF DESCRIPTION OF THE DRAWINGS 
       FIG. 1  is an exemplary block diagram of a SAN comprised of two FC-AL switches interconnected using trunking. 
       FIG. 2  is an exemplary block diagram of a SAN comprised of two FC-AL switches interconnected using trunking and implementing synchronized trunk failover according to embodiments of the present invention. 
       FIG. 3  is an exemplary state diagram of a FC-AL switch implementing synchronized trunk failover and failback according to embodiments of the present invention. 
       FIG. 4  is an exemplary block diagram of a SAN comprised of two FC-AL switches interconnected using trunking and implementing synchronized trunk failover according to embodiments of the present invention. 
   

   DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 
   In the following description Of preferred embodiments, reference is made to the accompanying drawings which form a part hereof, and in which it is shown by way of illustration specific embodiments in which the invention may be practiced. It is to be understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the preferred embodiments of the present invention. 
   Embodiments of the present invention synchronize and coordinate trunk failover and failback between the two FC-AL switches when a trunk failure occurs and the primary trunk designation is changed in order to minimize the disruption to data communications caused by multiple unnecessary loop initialization cycles. 
     FIG. 2  illustrates an exemplary SAN  200  comprised of two FC-AL switches S 1  and S 2 . A number of drives  202  and initiators I 1  and I 2  are attached to each FC-AL switch S 1  and S 2  through node ports  204 . In addition, cascade ports  206  on the FC-AL switches S 1  and S 2  enable the switches to be interconnected via two trunks T 1  and T 2 . A processor is also coupled to each switch (see  208  and  216  in  FIG. 2 ) for executing firmware  222  to perform operations on the switch, among other things. In the example of  FIG. 2 , trunk T 1  is designated as the primary trunk, and trunk T 2  is designated as the duplicate trunk. 
     FIG. 3  illustrates a state diagram  300  for the switch S 1  of  FIG. 2  operating in a normal state, in a synchronized trunk failover state, and in a synchronized trunk failback state according to embodiments of the present invention. Failover and fail back may be managed and driven by an API  224 , which is used to manage each of the switches independently in separate enclosures  220 . In alternative embodiments, failover and failback could be implemented in hardware implementing a state machine. 
   Normal operation, both trunks operational. When S 1  of  FIG. 2  is in a “Normal Operation State” (see  302  in  FIG. 3 ), data is being communicated between devices connected to S 1  and to S 2 , and both trunks T 1  (primary) and T 2  (duplicate) in  FIG. 2  are up and operating normally. 
   Failover. If primary trunk T 1  should fail as indicated at  210  and the T 1  failure is detected by S 1 , hardware in S 1  initiates a failover event by performing a port bypass event to change the cascade port  206  coupled to T 1  to a bypass state and bypass the cascade port. The cascade port  206  contains hardware such as a state machine that may detect either in-band indications of a T 1  failure such as a complete loss of synchronization due to a loss of signal, a particular ordered set such as a LIP F 8 , or a particular frame, or out-of-band indications of a T 1  failure such as dedicated pins that are asserted when a T 1  failure occurs. Examples of out-of-band indications include an asserted Receive Loss (RX_LOS) pin or an asserted Transmit Fault (TX_FAULT) pin when a Small Form factor Pluggable (SFP) connection is used on a switch (see the “INS-8074I Specification for SFP (Small Form factor Pluggable) Transceiver” by the SFF Committee). The hardware in S 1  generates and sends an interrupt referred to as “Primary Trunk Down Event” to firmware being executed by processor  208 . The firmware may call the API  224  every 30 ms or so to determine whether any “Primary Trunk Down Events” have occurred. When the firmware receives the Primary Trunk Down Event interrupt (see  304  in  FIG. 3 ), S 1  enters a “Received Primary Trunk Event State” (see  306  in  FIG. 3 ), and the processor connected to S 1  causes a MaRK ordered set (e.g. MRK (00, EF)) to be sent out over the duplicate trunk T 2  to S 2  (see  214  in  FIG. 2 and 308  in  FIG. 3 ). The MRK ordered set indicates that S 1  received a Primary Trunk Down Event, and indicates to S 1  that it should also enter a failover mode. Note that the ordered set MRK (00, EF) will be used herein for purposes of illustration only, a different ordered set could also be used. After S  1  sends MRK (00, EF), it enters a wait state (see  310  in  FIG. 3 ), where it starts a timer and waits to receive an acknowledgement of the MRK (00, EF) ordered set from S 2 . 
   Failover, S 2  acknowledges. If S 2  receives MRK (00, EF), it acknowledges receipt by sending an acknowledgement MRK ordered set (e.g. MRK (EF, 00)) over duplicate trunk T 2  back to S 1  (see  212  in  FIG. 2 ). If S 1  receives the acknowledgement MRK (EF, 00) and the timer has not yet timed out (see  312  in  FIG. 3 ), S 1  enters a “Received MRK (EF, 00) State” (see  314  in  FIG. 3 ). The processor  208  connected to S 1  recognizes that T 1  is no longer available but that duplicate trunk T 2  exists, and reconfigures the switch to establish T 2  as a temporary primary trunk (see  316  in  FIG. 3 ) as seen from the perspective of both S 1  and S 2 . The processor  208  connected to S 1  then acts as a master in the failover process and initiates LIP (F 7 , F 7 ) ordered sets  228  (see  318  in  FIG. 3 ) which are communicated to all devices in the system (connected to S 1  and S 2  in the present example) to initialize them. When devices receive the LIP ordered sets, they reserve an address, which is necessary to establish subsequent communications with the devices. Once the devices have been initialized and Loop Initialization is complete, S 1  can return to a normal operation state (see  302  in  FIG. 3 ). 
   Note that when S 2  receives MRK (00, EF) and acknowledges it by sending a MRK (EF, 00) back to S 1 , it acts as a slave in the failover process and does not attempt to initiate LIPs, thereby eliminating the possibility of multiple Loop Initialization cycles and reducing the time in which data cannot be transmitted. 
   Failover, S 2  does not acknowledge. On the other hand, while S 1  is still in the wait state (see  310  in  FIG. 3 ), the timer may time out (see  322  in  FIG. 3 ). In such a case, S 1  enters a “Timed Out Waiting for MRK State” (see  324  in  FIG. 3 ). This may occur, for example, if S 2  does not support synchronized trunk failover. In embodiments of the present invention, the selected approach is to have S 1  act as though synchronized trunk failover was not being implemented. Accordingly, the hardware in S 1  reconfigures the switch to establish T 2  as a temporary primary trunk (see  326  in  FIG. 3 ). The processor  208  connected to S 1  then initiates LIP ordered sets (see  328  in  FIG. 3 ) which are communicated to all devices connected to S 1  and S 2  in an attempt to initialize them. Once the devices have been initialized, S 1  can return to a normal operation state (see  302  in  FIG.3 ). 
   Note that because in the present example S 2  does not support synchronized trunk failover, S 2  will independently detect the T 1  failure and initiate its own failover process, wherein a bypass event occurs in which the cascade port coupled to the trunk T 1  is changed from an “inserted” state (i.e. device connected) to a “bypass” state (i.e. device not connected), and S 2  is reconfigured to establish T 2  as a temporary primary trunk. After S 2  is reconfigured to establish T 2  as the temporary primary trunk, the LIP ordered sets from S 1  may be sent down T 2  to initialize the devices connected to S 2 . Of course, because S 2  is acting independently, the processor  216  connected to S 2  may also initiate LIP ordered sets that may re-initialize all devices connected to S 1  and S 2 , resulting in a duplication of effort and added disruptions to normal data communications. 
   Failback.  FIG. 4  is a continuation of the exemplary SAN  200  of  FIG. 2 , wherein T 1  has failed, the failover process has run to completion, T 2  is now the temporary primary trunk, LIPs have caused the devices connected to be reinitialized, and both S 1  and S 2  have been operating in a normal operation state  302  for some period of time. Now suppose trunk T 1  is properly connected back into cascade ports  206  on both S 1  and S 2  at  214 . When T 1  is plugged back in, hardware in S 1  will perform a port insert event or a trunk up event to re-insert the cascade port  206 , and generate and send an interrupt referred to as an Original Primary Up Event (see  320  in  FIG. 3 ) to the processor  208 . When the processor  208  receives the Original Primary Up Event interrupt (see  320  in  FIG. 3 ), S 1  once again enters the “Received Primary Trunk Event State” (see  306  in  FIG. 3 ), and the processor connected to S 1  causes a MRK ordered set (e.g. MRK (00, EF)) to be sent out over the temporary primary trunk T 2  to S 2  (see  210  in  FIG. 4 and 308  in  FIG. 3 ). After S 1  sends MRK (00, EF), it enters a wait state (see  310  in  FIG. 3 ), where it starts a timer and waits to receive an acknowledgement of the MRK (00, EF) ordered set from S 2 . 
   Failback, S 2  acknowledges. If S 2  receives MRK (00, EF), it acknowledges receipt by sending an acknowledgement MRK ordered set (e.g. MRK (EF, 00)) over temporary primary trunk T 2  back to S 1  (see  212  in  FIG. 2 ). Note that when S 2  receives MRK (00, EF), which may be the same MRK ordered set received for either a Primary Trunk Down Event or a Original Primary Up Event, the current designation of the primary trunk will enable S 2  to distinguish and recognize which event is occurring at the time. 
   If S 1  receives the acknowledgement MRK (EF, 00) and the timer has not yet timed out (see  312  in  FIG. 3 ), S 1  enters the “Received MRK (EF, 00) State” (see  314  in  FIG. 3 ). The hardware in S 1  then reconfigures the switch to once again establish T 1  as the primary trunk (see  316  in  FIG. 3 ). The processor  208  connected to S 1  then acts as the master in the failback process and initiates LIP ordered sets (see  318  in  FIG. 3 ) which are communicated to all devices connected to S 1  and S 2  to initialize them. When devices receive the LIP ordered sets, they reserve an address, which is necessary to establish subsequent communications with the devices. Once the devices have been initialized, S 1  can return to a normal operation state (see  302  in  FIG. 3 ). 
   Note that when S 2  receives MRK (00, EF) and acknowledges it by sending a MRK (EF, 00) back to S 1 , it acts as the slave in the failback process and does not attempt to initiate LIPs thereby eliminating the possibility of multiple Loop Initialization cycles and reducing the time in which data cannot be transmitted. 
   Failback, S 2  does not acknowledge. On the other hand, if S 1  is still in the wait state (see  310  in  FIG. 3 ) and the timer times out (see  322  in  FIG. 3 ), S 1  enters a “Timed Out Waiting for MRK State” (see  324  in  FIG. 3 ). The hardware in S 1  also reconfigures the switch to establish T 2  as a temporary primary trunk (see  326  in  FIG. 3 ). The processor  208  connected to S 1  then initiates LIP ordered sets (see  328  in  FIG. 3 ) which are communicated to all devices connected to S 1  and S 2  in an attempt to initialize them. Once the devices have been initialized, S 1  can return to a normal operation state (see  302  in  FIG. 3 ). However, because S 2  never acknowledged receipt of the MRK ordered set, it is possible that the LIP ordered set generated by S 1  may cause all devices attached only to S 1  to be initialized before switch S 2  recognizes the failure and enters a failover mode. When S 2  finally recognizes the failure and enters the failover mode, it may generate its own LIP ordered set which again causes all devices attached to both S 1  and S 2  to be initialized. In other scenarios, S 1  may become blocked and unable to complete the LIP until S 2  recognizes the failure and also enters a failover mode. The LIP ordered set generated by S 2  is then able to complete its initialization of all devices attached to S 1  and S 2 . 
   Failover, simultaneous MRKs. The situation in which both switches S 1  and S 2  simultaneously recognize a failure and enter a failover mode will now be discussed. As noted above, if primary trunk T 1  should fail, hardware in S 1  will perform a port bypass event to change the cascade port coupled to T 1  to a bypass state and bypass the cascade port, generate and send an interrupt referred to as Primary Trunk Down to the processor  208 . When the processor  208  receives the Primary Trunk Down interrupt (see  304  in  FIG. 3 ), which is generally referred to as a Primary Trunk Event, S 1  enters a “Received Primary Trunk Event State” (see  306  in  FIG. 3 ), and the processor connected to S 1  causes a MRK ordered set (e.g. MRK (00, EF)) to be sent out over the duplicate trunk T 2  to S 2  (see  210  in  FIG. 2 and 308  in  FIG. 3 ). After S 1  sends MRK (00, EF), it enters a “Sent MRK (00, EF) State” (see  310  in  FIG. 3 ), where it starts a timer and waits to receive an acknowledgement of the MRK (00, EF) ordered set from S 2 . 
   If S 2  recognizes the failure at about the same time as S 1 , then hardware in S 2  will also perform a port bypass event to change the cascade port coupled to T 1  to a bypass state and bypass the cascade port, generate and send an interrupt referred to as Primary Trunk Down to the processor  216  connected to S 2 . When the processor  216  receives the Primary Trunk Down interrupt, S 2  enters a “Received Primary Trunk Event State,” and the processor connected to S 2  causes a MRK ordered set (e.g. MRK (00, EF)) to be sent out over the duplicate trunk T 2  to S 1  at  218 . After S 2  sends MRK (00, EF), it enters a “Sent MRK (00, EF) State,” where it starts a timer and waits to receive an acknowledgement of the MRK (00, EF) ordered set from S 1 . 
   At this point in time, both S 1  and S 2  are in the “Sent MRK (00, EF) State” (see  310  in  FIG. 3 ), each waiting for an acknowledgement MRK (EF, 00) ordered set to be received from the other switch. In other words, because both S 1  and S 2  detected the T 1  failure at about the same time, and both initiated the failover process at about the same time, a collision has occurred. The scenario is handled by a random back off time and resend algorithm described below. This mechanism is used to quickly resolve which of the two connected switches will be responsible for commencing the subsequent loop initialization. 
   If S 1  receives an MRK (00, EF) from S 2  instead of the expected acknowledgement MRK (EF, 00) (see  330  in  FIG. 3 ), which indicates that both S 1  and S 2  have recognized the T 1  failure and have initiated a failover process, both S 1  and S 2  enter a “Wait Random Time State” (see  332  in  FIG. 3 ), where each switch waits a random period of time that is different for each switch. When the random time period for S 1  expires (see  334  in  FIG. 3 ), if S 1  has not received another MRK (00, EF) from S 2  (see  336  in  FIG. 3 ), this indicates that the random time period for Si was shorter than the random time period for S 2 , and that S 1  can act as the master in the failover process and be first in re-sending the MRK (00, EF). Accordingly, S 1  re-sends a MRK (00, EF) to S 2  across T 2  (see  338  in  FIG. 2 ), and re-enters the “Sent MRK (00, EF) State” (see  310  in  FIG. 3 ). Because S 2  waits a random time before re-sending the MRK (00, EF), a collision with a MRK (00, EF) re-sent from S 2  should be avoided. 
   When the random time period for S 2  expires (see  342  in  FIG. 3 ), S 1  will already have sent out a MRK (00, EF) to S 2 , so S 2  will have received a MRK (00, EF). S 2  then enters a “Received MRK (00, EF) State” in which S 2  acts as a slave in the failover process, and sends an acknowledgement MRK (EF, 00) back to S 1  over T 2 . The hardware in S 2  reconfigures the switch to once again establish T 1  as the primary trunk, and S 2  returns to the “Normal Operation State.” 
   When S 1 , which was waiting in the “Sent MRK (00, EF) State”  310 , receives the acknowledgement MRK (EF, 00) back from S 2  (see  312  in  FIG. 3 ), it acts as the master in completing the failover process, as described above. 
   If, on the other hand, S 1  is in the “Wait Random Time State”  332  and the random time period for S 1  expires (see  342  in  FIG. 3 ) and S 1  has received a MRK (00, EF), this indicates that the random time period for S 1  was longer than the random time period for S 2 , and that S 1  should act as the slave in the failover process. S 1  then enters a “Received MRK (00, EF) State” (see  344  in  FIG. 3 ) and sends an acknowledgement MRK (EF, 00) back to S 2  over T 2  (see  346  in  FIG. 3 ). The hardware in S 1  reconfigures the switch to once again establish T 1  as the primary trunk (see  348  in  FIG. 3 ), and S 1  returns to the “Normal Operation State” (see  302  in  FIG. 3 ). 
   The previous discussion of  FIG. 3  presumed that S 1  either detected the T 1  failure first, or at least detected the T 1  failure simultaneously with S 2 , so that in either case S 1  generated a MRK (00, EF). However, in the case where S 1  does not detect the T 1  failure first, or does not detect the T 1  failure simultaneously with T 2 , S 1  may be in the “Normal Operation State” (see  302  in  FIG. 3 ) when it receives a MRK (00, EF) from S 2  (see  350  in  FIG. 3 ). In this case, S 1  enters the “Received MRK (00, EF) State” (see  344  in  FIG. 3 ), in which S 1  acts as a slave in the failover process, and sends an acknowledgement MRK (EF, 00) back to S 2  over T 2  (see  346  in  FIG. 3 ). The hardware in S 1  reconfigures the switch to establish T 2  as the primary trunk (see  348  in  FIG. 3 ), and S 1  returns to the “Normal Operation State” (see  302  in  FIG. 3 ). 
   Although the present invention has been fully described in connection with embodiments thereof with reference to the accompanying drawings, it is to be noted that various changes and modifications will become apparent to those skilled in the art. Such changes and modifications are to be understood as being included within the scope of the present invention as defined by the appended claims.