# pcilmr

## NAME



           pcilmr - margin PCIe Links
    


## SYNOPSIS



           **pcilmr**[**--margin**] [_< common options>_] _< link port>_ [_< link options>_]
           [_< link port>_ [_< link options>_] _..._]
           **pcilmr --full**[_< common options>_]
           **pcilmr --scan**
    


## CONFIGURATION



           List of the requirements for links and system settings to run the
           margining test.
    
           **BIOS settings**(depends on the system, relevant for server
           baseboards with Xeon CPUs):
    
           •  Turn off PCIe Leaky Bucket Feature, Re-Equalization and Link
              Degradation;
    
           •  Set Error Thresholds to 0;
    
           •  Intel VMD for NVMe SSDs - in case of strange behavior of the
              **pcilmr,** try to run it with the VMD turned off.
    
           **Device (link) requirements:**
    
                  _Configured by the user before running the utility, the_
                  _utility does not change them:_
    
                  •  The current Link data rate must be 16.0 GT/s or higher
                     (right now utility supports 16 GT/s and 32 GT/s Links);
    
                  •  Link Downstream Component must be at D0 Power Management
                     State.
    
                  _Configured by the utility during operation, utility set_
                  _them to their original state after receiving the results:_
    
                  •  The ASPM must be disabled in both the Downstream Port
                     and Upstream Port;
    
                  •  The Hardware Autonomous Speed Disable bit of the Link
                     Control 2 register must be Set in both the Downstream
                     Port and Upstream Port;
    
                  •  The Hardware Autonomous Width Disable bit of the Link
                     Control register must be Set in both the Downstream Port
                     and Upstream Port.
    


## DESCRIPTION



           **pcilmr** utility allows you to take advantage of the PCIe Lane
           Margining at the Receiver capability which is mandatory for all
           Ports supporting a data rate of 16.0 GT/s or higher, including
           Pseudo Ports (Retimers). Lane Margining at Receiver enables system
           software to obtain the margin information of a given Receiver
           while the Link is in the L0 state. The margin information includes
           both voltage and time, in either direction from the current
           Receiver position. Margining support for timing is required, while
           support for voltage is optional at 16.0 GT/s and required at 32.0
           GT/s and higher data rates. Also, independent time margining and
           independent voltage margining is optional.
    
           Utility allows to get an approximation of the eye margin diagram
           in the form of a rhombus (by four points). Lane Margining at the
           Receiver capability enables users to margin PCIe links without a
           hardware debugger and without the need to stop the target system.
           Utility can be useful to debug link issues due to receiver
           margins.
    
           **pcilmr** requires root privileges (to access Extended Configuration
           Space), but during our testing there were no problems with the
           devices and they successfully returned to their normal initial
           state after the end of testing.
    


## RESULTS GRADING



           The PCIe specification provides reference values for the eye
           diagram, which are also used by the **pcilmr** to evaluate the
           results, but it seems that it makes sense to contact the
           manufacturer of a particular device for references.
    
           The utility uses values set in PCIe Base Spec Rev. 5.0 Section
           8.4.2 as the default eye width and height minimum references.
           Recommended values were taken from the PCIe Architecture PHY Test
           Spec Rev 5.0 (Transmitter Electrical Compliance).
    
           Reference grading values currently used by the utility are
           presented in the table below:
    
           ┌─────┬─────────────────────┬──────────────────────┐
           │     │  **16 GT/s** │**(Gen 4)** │   **32 GT/s** │**(Gen 5)** │
           │     │    **EW** │   **EH** │   **EW** │   **EH** │
           ├─────┼──────────┼──────────┼───────────┼──────────┤
           │ **Min** │ 18.75 ps │  15 mV   │ 9.375 ps  │  15 mV   │
           │     │ 30% UI   │          │ 30% UI    │          │
           ├─────┼──────────┼──────────┼───────────┼──────────┤
           │ **Rec** │ 23.75 ps │  21 mV   │10.157 ps  │19.75 mV  │
           │     │ 38% UI   │          │33% UI     │          │
           └─────┴──────────┴──────────┴───────────┴──────────┘
    
           **pcilmr** uses full eye width and height values to grade lanes.
           However, it is possible that device supports only one side
           margining. In such cases by default utility will calculate EW or
           EH as a double one side result.
    
           If info for specific device is available, you can configure
           grading criteria and tweak utility behavior in one-side only cases
           for your device using _-g_ link specific option (see below).
    


## HARDWARE QUIRKS SUPPORT



           Thanks to testing or directly from the manufacturer's
           documentation, we know that some devices require special treatment
           during the margining.  Utility detects such devices based on their
           Vendor ID - Device ID pair.  Right now the list of special devices
           is hardcoded in _margin_hw_ file. For such devices utility can
           automatically adjust port margining parameters or grading options.
    
           For example, for Ice Lake CPUs RC ports **pcilmr** will change device
           MaxVoltageOffset value and will force the use of '_one side is the_
           _whole_ ' grading mode.
    


## OPTIONS



       **Device Specifier**
           **You can specify Downstream or Upstream Port of the Link.**
    
           **< device/component>**
                  [_< domain>_:]_< bus>_:_< dev>_._< func>_ (see lspci(8))
    
       **Utility Modes**
           **--margin** _< downstream component> ..._
                  Margin selected Links.
    
           **--full** Margin all ready for testing (in a meaning similar to the
                  **--scan** option) Links in the system (one by one).
    
           **--scan** Scan for Links with negotiated speed 16 GT/s or higher.
                  Mark "Ready" those of them in which at least one of the
                  Link sides have Margining Ready bit set meaning that these
                  Links are ready for testing and you can run utility on
                  them.
    
       **Margining Common (for all specified links) options**
           **-c** Print Device Lane Margining Capabilities only. Do not run
           margining.
    
           **-e** _< errors>_
                  Specify Error Count Limit for margining.
                  Default: 4.
    
           **-o** _< directory>_
                  Save margining results in csv form into the specified
                  directory. Utility will generate file with the name in form
                  of "lmr__< port>__Rx _#_ __< timestamp>_.csv" for each successfully
                  tested receiver.
    
           **-d** _< time>_
                  Specify dwell time in seconds for the margining step.
                  Default: 1 s
    
       **Margining Link specific options**
           **-l** _< lane>_**[**_, <lane>..._**]**
                  ®.br Remember that Device may use Lane Reversal for Lane
                  numbering. However, utility uses logical lane numbers in
                  arguments and for logging. Utility will automatically
                  determine Lane Reversal and tune its calls.
                  Default: all link lanes.
    
           **-r** _< recvn>_**[**_, <recvn>..._**]**
                  Specify Receivers to select margining targets.
                  Default: all available Receivers (including Retimers).
    
           **-p** _< parallel_lanes>_
                  Specify number of lanes to margin simultaneously.
                  According to spec it's possible for Receiver to margin up
                  to MaxLanes + 1 lanes simultaneously, but during testing,
                  performing margining on several lanes simultaneously led to
                  results that were different from sequential margining, so
                  this feature requires additional verification and _-p_ option
                  right now is for experiments mostly.
                  Default: 1.
    
           **Use only one of -T/-t options at the same time (same for -V/-v).**
           **Without these options utility will use MaxSteps from Device**
           **capabilities as test limit.**
    
           **-T** Time Margining will continue until the Error Count is no
                  more than an Error Count Limit. Use this option to find
                  Link limit.
    
           **-t** _< steps>_
                  Specify maximum number of steps for Time Margining.
    
           **-V** Same as _-T_ option, but for Voltage.
    
           **-v** _< steps>_
                  Specify maximum number of steps for Voltage Margining.
    
           **-g** _< recvn>_**t=**_< criteria>_**{%|ps}[,f]**
    
              _< recvn>_**t=f[,**_< criteria>_**{%|ps}]**
    
              _< recvn>_**v=**_< criteria>_**[,f]**
    
              _< recvn>_**v=f[,**_< criteria>_**]**
                  Specify pass/fail grading criteria for eye width (timing -
                  t) or height (voltage - v) for one of the link receivers.
                  For EW you must choose one of the units (% UI or ps), for
                  EH mV is used.
                  Additional flag _f_ is for situations when port doesn't
                  support two side independent margining. In such cases by
                  default utility will calculate EW or EH as a double one
                  side result. You can add _f_ flag for _-g_ option to tell the
                  utility that the result in one direction is actually the
                  measurement of the full eye and it does not need to be
                  multiplied. This is so called '_one side is the whole_ '
                  grading mode.
    


## EXAMPLES



           Utility syntax example:
                  **pcilmr -o** _csv ab:0.0_ **-r** _1,6_ **-g** _1t=20%_ **-g** _1v=f,30 52:0.0_ **-l**
                  _0,1,2_ **-TV**
    
           Examples of collected results on different systems.  
           ⟨https://gist.github.com/bombanya/f2b15263712757ffba1a11eea011c419⟩
    


* * *