OPTICAL AND ELECTRICAL MODULES WITH ENHANCED FEATURES

Optical and electrical modules with enhanced features and associated apparatus and methods. The optical modules are configured to implement one or more features that are offloaded from Ethernet devices to which the optical modules are configured to be attached. The features include support for timestamping packets and preamble using IEEE 1588 Precision Time Protocol (PTP) profiles, support for implementing IEEE 1588 one-step operations, support for implementing IEEE 1588 Ethernet Synchronous Clocks (SyncE) profiles, support for In-Band Network Telemetry (INT), and support for implementing a MACsec security protocol defined by IEEE standard 802.1AD. The enhanced features provided by the optical modules enable Ethernet devices to be upgraded to support the enhanced features by replacing conventional optical modules with the optical modules disclosed herein. Support for White Rabbit IEEE PTP and SyncE profiles is also provided.

BACKGROUND INFORMATION

Computer networks span the range for Local Area Network (LANs) to Wide Area Networks (WANs), to collections of interconnected networks (such as the Internet). The build-out of these networks includes a mixture of switches and networking equipment such a network interface controllers (NICs), some with more recent advancements and features, while others are older and have limited features. In some instances, it is desired to add features to such older switches and networking equipment. Some of these features include Re-timers, In-band Network Telemetry (INT), Precise Time features, and security features.

CERN's “White Rabbit” was designed for high precision and accuracy. In some cases, the precision and/or accuracy are in the single digit picosecond or less (e.g., femtoseconds). Some of these concepts are included in IEEE 1588-2019 (Institute of Electrical Engineers and Electronics Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems) as a profile, such as Ethernet Synchronous Clocks (SyncE).

DETAILED DESCRIPTION

For clarity, individual components in the Figures herein may also be referred to by their labels in the Figures, rather than by a particular reference number. Additionally, reference numbers referring to a particular type of component (as opposed to a particular component) may be shown with a reference number followed by “(typ)” meaning “typical.” It will be understood that the configuration of these components will be typical of similar components that may exist but are not shown in the drawing Figures for simplicity and clarity or otherwise similar components that are not labeled with separate reference numbers. Conversely, “(typ)” is not to be construed as meaning the component, element, etc. is typically used for its disclosed function, implement, purpose, etc.

In accordance with aspects of the embodiments disclosed herein, circuitry and logic are added to and/or modified in networking modules, such as optical modules to support extended functionality and features. The optical modules are configured to implement one or more features that are offloaded from Ethernet devices to which the optical modules are configured to be attached. The features include support for timestamping packets and/or preambles using IEEE 1588 Precision Time Protocol (PTP) profiles, support for implementing IEEE 1588 one-step operations, support for implementing IEEE 1588 Ethernet Synchronous Clocks (SyncE) profiles, support for In-Band Network Telemetry (INT), and support for implementing a MACsec security protocol defined by IEEE standard 802.1AD. In some embodiments, White Rabbit IEEE PTP and SyncE profiles are supported. The enhanced features provided by the optical modules enable Ethernet devices to be upgraded to support the enhanced features by replacing conventional optical modules with the optical modules described and illustrated herein.

In addition to optical modules, the principles and teachings may be applied to wired networking modules such as used for Ethernet. For example, copper or metal Ethernet connections are common in 1G, 10G and 25G. Likewise, a 10G or 25G device could use pluggable devices that receive and transmit optical or electrical signals. Hence the concepts in this application should not be limited to optical or optical pluggable devices, but may be implemented in electrical modules such as the pluggable electrical modules described and illustrated herein.

Optical Module with IEEE 1588 PTP Support

FIG. 1shows circuitry for an optical module100the supports IEEE 1588 Precision Time Protocol (PTP), according to one embodiment. Optical module100includes an optics block102, an optical logic block104, and a White Rabbit logic block106. Generally, optical block102and optical logic block104may comprise conventional blocks in optical modules (noting that the optical logic blocks may be modified/augmented to support the features described herein in some embodiments). They are used to process inbound received signals (received at an input port from a network) that are processed and forwarded along a receive (Rx) path108to an Ethernet device109and outbound signals originating from Ethernet device109that are received by optical logic104and forwarded and processed along a transmit (Tx) path110. In some embodiments, optics block102and optical logic block104are configured to facilitate communication using optical signals in accordance with one or more Ethernet standards. Example Ethernet standards include but are not limited to the following IEEE standards and proposals: 802.3bs, 802.3bm, 802.3by, 802.3ca, 802.3cp, 802.3cs, and 802.3df.

White Rabbit is a sub-nanosecond synchronization technology that uses IEEE 1588 Precision Time Protocol (PTP) and (optionally) Ethernet Synchronous Clocks (SyncE). There are White Rabbit profiles included in the IEEE1588-2019 Standard that include IEEE 1588 PTP and IEEE 1588+SyncE. The IEEE 1588 PTP profile is implemented in White Rabbit logic block106using circuitry and logic comprising a PHY (Physical Layer) block112, a MAC block114with timestamping, a management unit116, and a Pulse Per Second (PPS) block118. Management unit116is coupled to MAC block114and memory120. In one embodiment, the circuitry and logic illustrated in White Rabbit logic block106is implemented in optical logic block104.

PPS logic in PPS block118is used, in part, to determine the current precise time. It may include a pulse per second pin, a 10 KHz pin, or some other sync signal. It may also include a reference clock to allow for the best accuracy.

The circuitry and logic implemented in White Rabbit logic block106may comprise some form of embedded logic (such as a Field Programmable Gate Array (FPGA) or other programmable logic, an Application Specific Integrated Circuit (ASIC), or a structure ASIC (e.g., an Intel® eASIC™) or a combination thereof that can synchronize its clock to an incoming sync signal or signals such as a 1 Pulse Per Second, a 10 KHz clock, and/or a recovered external clock. White Rabbit logic block106is further configured to employ this circuitry and logic to timestamp arriving and departing packets, and to perform associated timing measurements. The timestamps for arriving and departing packets may be sent to external circuitry (not shown) through an MDIO interface or management interface of optical module100. In one embodiment a mechanism is used to identify which packets need to be time stamped, as not every packet arrival and departure time needs to be known.

As shown inFIG. 1, White Rabbit logic block106“listens” to both the receive and transmit paths108and110. PHY block112(e.g., PCS Physical Coding Sublayer (PCS) and Forward Error Correction (FEC) sublayer in the PHY) and MAC block114are configured to enable timestamping of received and transmitted packets. In one embodiment, the packets have a simple parser to determine which packets need to be timestamped. Details of those packets and their arrival or departure time are stored in memory120for retrieval using the management unit116.

FIG. 1ashows an optical module100aillustrating a variant of optical module100aunder which listening of signal occurs within optical logic104.FIG. 1aalso show further details of optical logic104, which includes an optical-electrical block122, a gearbox/clock data recovery (CDR) block124, and a SERDES (serializer/deserializer) block126.

Optical-electrical block122includes circuitry that converts signals between an optical domain and an electrical domain. Optics block102is configured to receive and transmit optical signals in accordance with one or more optical network standards. On the receive side, these signals are converted to an electrical domain by circuitry and physical components in optical-electrical block122. On the transmit side, electrical signals are converted to optical signals by optical-electrical block122. In some embodiments optics block102and optical-electrical block122are combined or otherwise their functionality is effectively combined. The operations and structure of optics block102and optical-electrical block122are generally known in the art and outside the scope of this disclosure.

Gearbox/CDR block124is used to perform clock data recovery. The “gearbox” aspect also may be used to change the data rate used for some of the electrical circuitry such that the data rate is different than the line rate used by the optical network. The operations and structure of Gearbox/CDR block124are generally known in the art, except for the modified versions described and illustrated herein.

SERDES block126is used to serialize and deserialize signals forwarded to and received from Ethernet device109. Depending on the communication standard used, the signal may employ one or more “lanes.” The circuitry used in SERDES block126are generally known in the art and are outside the scope of this disclosure.

As further shown inFIG. 1a, signals along Rx path108and Tx path110are listened to and processed by circuitry in White Rabbit logic block106in a manner similar to that described above inFIG. 1. The primary difference between the embodiments shown inFIGS. 1 and 1aare where the signals are listened to. It is further noted that optical logic block104inFIG. 1may comprise the same or similar circuitry to that illustrated inFIG. 1a.

Optical Module with IEEE 1588 PTP+SyncE

FIGS. 2 and 2arespectively show optical modules200and200awhich include a White Rabbit logic block206that is configured to implement IEEE 1588 PTP and SyncE according to a White Rabbit profile in the IEEE1588-2019 standard. InFIGS. 1, 1a,2and2a, components and blocks with like-numbered references perform similar functions. For optical module200, these components and blocks include optics block102, optical logic104, PHY block112, MAC block114, management unit116and memory120. As before, signals are forward to and received from Ethernet device109using Rx path108and Tx path110.

White Rabbit logic block206further includes a SyncE clock (CLK) recover block208and an SyncE clock filtering block210. The SyncE clock recovery block208and SyncE clock filtering block210recovers and cleans up the recovered clock and outputs the recovered clock to the rest of the system. In one embodiment, SyncE recovered clock block208is also used by the internal logic of optical module200. For example, the recovered clock could be used as the receive clock associated with the receive data. As another option, another optical module could recover the clock and share it with this module, where this module uses it for its IEEE 1588 related logic.

Under the embodiment of optical module200ashown inFIG. 2a, optical logic104has a similar structure and circuitry to that shown inFIG. 1aand discussed above. Also as above, in this embodiment the listening by circuitry in White Rabbit logic block206is performed within Gearbox/CDR block124.

Optical Module with IEEE 1588 One-Step Support

IEEE 1588 supports one-step and two-step clocks. Under one-step, for a PTP event message the timestamp is placed in the message/packet on-the-fly using hardware. Under two-step, the timestamp is added to a separate message/packet. The following embodiments add support for IEEE 1588 one-step.

FIG. 3ashows an optical module300aincluding an optics block302and an optical logic block304awith two MACs and two PHYs. Optical logic block304aincludes an original optical logic block306(e.g., a conventional optical logic block used in today's optical modules), a first PHY block308, a first MAC block310with timestamping, a second MAC block312, a second PHY block314, a management unit316, and memory318. The receive path320is from original optical logic306to first PHY block308to first MAC block310to second MAC block312to second PHY block314. The transmit path322is from second PHY block314to second MAC block312to first MAC block310to first PHY block308to original optical logic306. As packets traverse receive path320and transmit path322a timestamp may be added (for all or selected packets) in MAC block310. Optionally, timestamping may be added elsewhere along the receive and transmit paths.

Under one implementation of optical module300a, a timestamp is added by MAC block312to the preamble going to Ethernet device109such that the preamble contains the arrival time of each packet, allowing the Ethernet device to know the exact arrival time of each packet. Likewise details about the packet transmission could be sent in the preamble (like timestamp this packet and/or insert a timestamp at a location for an IEEE1588 1-step departing packet). Under one embodiment, MAC block312includes IEEE 1588 one-step logic324a.

Optical Modules with IEEE 1588+SyncE+INT Support

FIG. 4ashows an embodiment of an optical module400aproviding IEEE 1588+SyncE support using two PHYs and two MACs. Optical module400aincludes an optics block402, optical logic404, and a White Rabbit logic block406asupporting IEEE 1588+SyncE White Rabbit profiles. White Rabbit logic block406aincludes a first PHY block408a first MAC block410with timestamping support, a second MAC block412, and a second PHY block414. White Rabbit logic block406aalso includes a SyncE clock recover logic416, SyncE clock filtering logic418, and a management unit420which may include or be coupled to memory (not shown). As shown, the receive path422is from optical logic404to PHY block408to MAC block410to MAC block412to PHY block414. The transmit path424is the reverse of this, beginning at PHY block414to MAC block412to MAC block410to PHY block408to optical logic404.

Under one implementation of optical module400a, a timestamp is added by MAC block410to the preamble going to Ethernet device109such that the preamble contains the arrival time of each packet, allowing the Ethernet device to know the exact arrival time of each packet. Likewise details about the packet transmission could be sent in the preamble (such as by adding or updating a timestamp in the packet preamble somewhere along transmit path424).

FIG. 4bshows an optical module400bthat further includes support for In-Band Network Telemetry (INT). As shown, most of the logic and blocks inFIGS. 4aand 4bhave the same reference numbers and, accordingly, have similar structures and perform similar functions. In addition to this logic and blocks, a White Rabbit logic block406bfurther includes an INT packet buffer update block426interposed between MAC blocks410and412and an INT statistics (stats) block428.

Packet buffer426may be used to store the packet, update the packet, pace the transmission/reception of the packet, etc. In the case of INT, the arrival or departure time could be updated in the packet which may or may not change the packet size depending on implementation. This may cause the CRC, checksums and/or other packet contents to be modified, either in the buffer or on the fly as the packet passes through the update logic.

Generally, INT stats block428may be configured to collect INT statistics that may be used for various purposes. Examples could be arrival and departure time through the switch or latency from the SERVER to the Optical module attached to a NIC or optical module. Other states could include packet counts, byte counts, etc. This could be based on traffic class, VLANs, or data parsed from the packet as exemplary and non-limiting examples. INT stats block428may employ various counters based on one or more of VSI, SWITCH ID, per subscriber (e.g. identified using unique ID for host driver), VLAN, traffic class, packet type (e.g., Unicast/multicast/mirror), errors, recirculation etc. INT Stats block428can update packet metadata fields based on various events and send the periodic counter notifications to software.

Optical Module with IEEE 1588+SyncE+MACsec Support

FIG. 4cshows an optical module400cthat further includes support for MACsec. MACsec is a security protocol defined by the IEEE 802.1AD standard. MACsec employs integrity check data for Ethernet frames by appending 8-byte header and the 16-byte tail to the Ethernet frame and adding/storing integrity check data derived from the Ethernet frame content. MACsec devices are configured to implement various aspects of IEEE 802.1AD, including setup and teardown of MACsec connections.

As before, most of the logic and blocks inFIG. 4chave the same reference numbers as shown inFIGS. 4aand 4band, accordingly, have similar structures and perform similar functions. In addition to this logic and blocks, a White Rabbit logic block406cfurther includes a frame buffer430interposed between MAC blocks410and412and an a MACsec block432configured to implement MACsec operations in accordance with IEEE 802.1AD.

In some embodiments, a packet buffer may be needed if the packet size changes. For example, when a security header is added or removed. Likewise, security information may need to be parsed from the packet data or metadata (which could be in the preamble) to operate correctly.

For outbound Ethernet frames, the frame is buffered in frame buffer430, where an 8-byte header and 16-byte tail are added. Integrity check data is calculated for the frame and written to the 8-byte header and 16-byte tail, with the modified Ethernet frame being forwarded to MAC block410. For inbound Ethernet frames, the integrity check data are read from the 8-byte header and 16-byte tail and are used to confirm the integrity of the Ethernet frames. Frames that pass the integrity check are then forwarded to MAC block412. As an option, the 8-byte header and 16-byte tail are stripped from the forwarded Ethernet frame.

In addition to MACsec, a similar approach may be used to apply to various security, encryption, decryption, etc. protocol.

Generally, in the embodiments illustrated inFIGS. 3a, 3b, and 4a-4cthe PHY and MAC blocks may be combined into a PHY/MAC block or a MAC/PHY block. In addition, other circuitry may be combined or modified to support additional features and/or functionality. For example, a packet or frame buffer may be used to support other features as packets or frames are being processed and forwarded via optical modules. For instance, a Flexible Packet Parser (FXP) type block could be added to perform statistics and other tasks. Packet encapsulation/decapsulation could also be performed. In addition to packet parsing, other FXP tasks like metering could be performed. As yet another option, logic for implementing packet pacing may be implemented. For example, an F×P block may be used to indicate the pacing or departure time of packets.

Another feature that could be added is circuitry to the phase detection method that White Rabbit uses in the Rx direction to measure the phase difference between the recovered clock and the TIME REF clock. For example, White Rabbit uses a DDMTD (digital dual mixer time difference).

FIG. 5shows circuitry500for implementing SyncE operations, according to one embodiment. Circuitry500includes a re-timer block502, and a Digital Phase Lock Loop (DPLL)508. Re-timer block502includes a MAC and PHY blocks (or a combined MAC/PHY block) along with a MAC/PHY interface510and a MAC/PHY interface512. MAC/PHY interface512is connected to a NIC chip506including a MAC/PHY via a Management Data Input/Output (MDIO) link514, also known as Serial Management Interface (SMI) or Media Independent Interface Management (MIIM) link. NIC chip506is connected to DPLL508via multiple signal paths including an I2C bus516. In the case where the optics are part of a switch SOC or multichip module, NIC chip506and MAC/PHY interface512may be combined to reduce latency, power, area, pin count, etc.

Input signals from optical logic504are received over a bidirectional link518by re-timer block502at MAC/PHY interface510. As its name implies, circuitry in re-timer block502is used to re-time the received signals. The re-timer circuitry outputs received clocks A and B (depicted as RCLK_A, _B signals520, which are received on the input side of DPPL508. MC Chip506outputs an 1PPS reference signal522that is also received as an input to DPLL508. Meanwhile, DPLL508outputs two clock signals: a CLK signal524that is received by re-timer block502and a reference clock (REFCLK) signal526that is received by MC Chip506. In addition to the inputs and outputs shown and described above, DPLL508may also receive other inputs528and output other outputs530.

As depicted by the dashed box inFIG. 5, circuitry500is part of an optical module501, which is generally representative of any of the optical module embodiments, described herein that includes the SyncE feature. As an alternative, a DPLL may be separate from the optical module, such as shown inFIG. 5a. In this embodiment, the circuitry within dashed box500ais implemented in optical module501a, while DPLL508ais a separate chip that is installed on a circuit board to which optical module501is coupled. In one embodiment, DPLL508ais a Microchip Technology® ZL30632 chip, which is a two-channel SyncE network synchronizer. In one embodiment, a single DPLL chip is used for two optical modules.

Quad Small Form Factor Pluggable (QSFP) and QSFP+Modules

Quad Small Form Factor Pluggable (QSFP) modules are hot-pluggable optical modules that employ four transceiver channels rather than one. QSFP+modules similarly support four transceiver channels, but generally operate at high transmission speeds than QSPF modules. They are designed to support multi-channel and or multi-“lane” high speed Ethernet links, such as 40 GB links employing 4×10 GB lanes or channels. The principles and teachings disclosed in the embodiments discussed and illustrated herein can be extended to QSFP and QSFP+modules (or any other existing or future multi-port and or multi-lane optical modules). In the case of multiple lanes, there would be replicated circuitry for the optical block and optical logic blocks along the receive and transmit paths described and illustrated herein, as would be recognized by those skilled in the art.

Generally, features such as IEEE 1588 PTP, SyncE, one-step, INT stats, etc. are applicable for flows of packets and related processing/timing/statistics etc. Thus, a single set of circuitry would be employed for both single lane and multi-lane links (rather than employing replicated circuitry for each lane). In cases where the White Rabbit logic “listens” to one or both of the receive and transmit paths, the listening could be for a single lane or may be done after the optical logic block (such as shown inFIG. 1). Similarly in the multiport module cases, a single set of White Rabbit logic could listen to more than one port, or separate sets of white rabbit logic could be on a per port bases.

An advantage of implementation the enhanced circuitry in a multi-port module is that it can save power, and area. This is because the circuitry may be implemented in the same silicon device, removing the need for redundant circuitry.

Generally, features such as IEEE 1588 PTP, SyncE, one-step, INT stats, etc. are applicable for flows of packets and related processing/timing/statistics etc. Thus, a single set of circuitry would be employed for both single lane and multi-lane links (rather than employing replicated circuitry for each lane). In cases where the White Rabbit logic “listens” to one or both of the receive and transmit paths, the listening could be for a single lane or may be done after the optical logic block (such as shown inFIG. 1).

Pluggable Electrical Module with IEEE 1588 PTP

The White Rabbit logic and related functionality may also be implemented in pluggable electrical modules. A primary difference between the pluggable electrical modules and the optical modules (which also may be pluggable) is the signal processing in the pluggable electrical modules remains entirely within the electrical domain, whereas optical modules employ signal processing in both the optical domain and the electrical domain, as discussed above.

FIG. 6shows an embodiment of an pluggable electrical module600, which includes an electrical TX driver and RX driver block602coupled to electrical TX/RX logic604. Electrical TX/RX logic604includes an equalizers/decoder/encoder block606, a gearbox/CDR block624, and a SERDES block626. Pluggable electrical module600further includes an Rx datapath608and a Tx datapath610coupled between SERDES626and an Ethernet device609. Similar to described and illustrated for the optical module embodiments above, the Rx and Tx datapaths also pass through electrical TX/RX logic604including gearbox/CDR block624.

As further shown, White Rabbit logic block106comprises the same circuitry shown inFIGS. 1 and 1aand discussed above. In a manner similar to that shown inFIG. 1aand discussed above, circuitry in White Rabbit logic block106would listen to signal transmitted over Rx datapath608and/or Tx datapath610. Those signals would be processed by PHY block112, and MAC block114using PPS block118in the manner described above to add timestamps to inbound and/or outbound packets, packet preambles, or Ethernet frames.

Pluggable Electrical Module with IEEE 1588 PTP+SyncE

FIG. 7show a pluggable electrical module700which includes White Rabbit logic block206that is configured to implement IEEE 1588 PTP and SyncE according to a White Rabbit profile in the IEEE1588-2019 standard. As illustrated by like-numbers blocks inFIGS. 6 and 7, the electrical components in the upper portion of both diagrams is the same. White Rabbit logic block206employs the same or similar components to that shown inFIGS. 2 and 2aand described above. In this case, White Rabbit logic block206listens to signals on one or both of Rx datapath608and Tx datapath610.

Pluggable Electrical Modules with IEEE 1588 One-Step Support

IEEE 1588 supports one-step and two-step clocks. Under one-step, for a PTP event message the timestamp is placed in the message/packet on-the-fly using hardware. Under two-step, the timestamp is added to a separate message/packet. The following embodiments add support for IEEE 1588 one-step.

FIGS. 8aand 8bshows pluggable electrical modules800aand800bthat are configured to provide IEEE 1588 one-step support. Pluggable electrical module800aincludes an electrical TX drive and RX buffer block802coupled to electrical logic with MACs and PHYs804a, which includes electrical TX/RX logic804that is similar to TX/RX logic604shown inFIGS. 6 and 7and employs a Rx datapath820and a Tx datapath822. The remainder of the circuitry in electrical logic with MACs and PHYs804ais similar to circuitry shown in optical logic block304aofFIG. 3a, where like-numbered blocks and components are similar for both embodiments.

Pluggable electrical module800binFIG. 8bincludes an electrical logic block804bthat supports IEEE 1588 one-step. Electrical logic block804bis similar to optical logic block304bshown inFIG. 3b, except original optical logic306is replaced with electrical TX/RX logic804. As before, IEEE 1588 one-step block324bis interposed between MAC blocks310and312, and is configured to perform timestamping and associated IEEE 1588 one-step operations.

Pluggable Electrical Modules with IEEE 1588+SyncE+Support

FIGS. 9a, 9b, and 9crespectively show pluggable electrical modules900a,900b, and900cwhich respectively include IEEE 1588+SyncE support, IEEE 1588+SyncE+INT support, and IEEE 1588+SyncE+MACsec support. As shown by like-numbered blocks and components inFIGS. 9a, 9b, and 9cand respectiveFIGS. 4a, 4b, and 4c, the same White Rabbit logic blocks406a,406band406care used in both the pluggable electrical modules and the optical modules. For the pluggable electrical module embodiments, these White Rabbit logic blocks are connected to an electrical TX driver RX buffer block902and electrical TX/RX logic904via signals transmitted over an Rx datapath922and a Tx datapath924.

Exemplary Use Cases

FIGS. 10 and 10ashows respective infrastructure processor units (IPUs)1000and1000a, each including two enhanced optical modules1002and1004. In the illustrated embodiment ofFIG. 10, IPU1000comprises a Peripheral Component Interconnect Express (PCIe) card including a circuit board1006having a PCIe edge connector to which various integrated circuit (IC) chips and enhanced optical modules1002and1004are mounted. The IC chips include an FPGA1008, a CPU/SoC (System on a Chip)1010, a pair of Ethernet NICs1012and1014, and memory chips1016and1018. Programmed logic in FPGA1008and/or execution of software on CPU/SoC1010may be used to implement various IPU functions. FPGA1008may include logic that is pre-programmed (e.g., by a manufacturing) and/or logic that is programmed in the field. For example, logic in FPGA1008may be programmed by a host CPU for a platform in which IPU1000is installed. IPU1000may also include other interfaces (not shown) that may be used to program logic in FPGA1008. Under an optional configuration, a DPLL chip1022is used for re-timer and SyncE operations, such as illustrated inFIG. 5aand discussed above. Under an IPU1000ashown inFIG. 10a, the functionality associated with Ethernet NICs1012and1014is implemented in FPGA1008by (pre-) programming associated logic in the FPGA. Optionally, similar functionality may be implemented using an ASIC or an SOC.

CPU/SoC1010employs a System on a Chip including multiple processor cores. Various CPU/processor architectures may be used, including x86 and ARM architectures. In one non-limiting example, CPU/SoC1006comprises an Intel® Xeon® processor. Software executed on the processor cores may be loaded into memory1018, either from a storage device (not shown), for a host, or received over a network coupled to enhanced optical module1002and1004.

FIG. 11shows a SmartNIC1100including a pair of enhanced optical modules1102and1104. SmartNIC1100comprises a Peripheral Component Interconnect Express (PCIe) card including a circuit board1106having a PCIe edge connector to which various integrated circuit (IC) chips and enhanced optical modules1102and1104are mounted. The IC chips include an SmartNIC chip1108, an embedded processor1110and memory chips1116and1118. SmartNIC chip1108is a multi-port Ethernet NIC that is configured to perform various Ethernet NIC functions, as is known in the art. In some embodiments, SmartNIC chip1108is an FPGA and/or includes FPGA circuitry. SmartNIC chip1108may include embedded logic for performing various packet processing operations, such as but not limited to packet classification, flow control, RDMA (Remote Direct Memory Access) operations, an Access Gateway Function (AGF), Virtual Network Functions (VNFs), a User Plane Function (UPF), and other functions.

In addition to IPUs and SmartNICs, the optical modules described and illustrated herein generally may be used with various devices that have one or more Ethernet ports. Examples of such devices include but are not limited to line cards, switches routers, cellular equipment (like nano-cells, picocells, ethernet connected radios), WiFi equipment, network appliances, storage devices, security devices, servers with Ethernet ports, telecom equipment, and test equipment,

An example of a switch1200including enhanced optical modules1202and1204is shown inFIG. 12. Switch1200includes multiple switch ports1206and switching circuitry, logic, and buffers1208, which is generally representative of circuitry implemented in switches, such as but not limited to Top of Rack (ToR) switches and other types of switches. In one embodiment, optical modules1202and1204, which also comprise two of the switch ports, are used for uplink connections to an optical network. Generally, ports1206may employ either wired or optical communication. Optical ports may employ conventional optical modules, in addition to one or more optical modules with enhanced features.

Generally, enhanced optical modules1002and1004,1102,1104,1202and1204represent any of the embodiments of optical modules described and illustrated herein. Depending on the requirements of a system, an optical module may include one or more of the features and enhancements described and illustrated herein.

The embodiments of advanced optical modules and pluggable electrical modules disclosed herein provide several advantages over existing approaches. Significantly, they provide a mechanism for adding precise time measurements and associated time parameters (e.g., using SyncE) to existing equipment by simply replacing the existing optical modules with new advanced optical modules. This applies to both equipment in the field and new equipment being manufactured— these capabilities can be added without changes to silicon and with generally minor changes to firmware (to take advantage of the new features and capabilities). A vendor or end-user may also update capabilities over time, if so desired, or mix and match capabilities. For example, a given end-user or customer may want to add MACsec to some of its equipment, while adding White Rabbit IEEE 1588 PTP with or without SyncE to other equipment, while add the combination of capabilities to yet other equipment. From a vendor standpoint, the same board hardware could be deployed to support multiple use cases, again, by simply providing different advanced optical modules with the boards. This is in comparison to current approaches, where adding these capabilities requires new silicon (e.g., new NIC chips). In addition, unlike many optical modules, which are pluggable, most NIC chips are fixedly bounded to their circuit boards (e.g., using solder balls or the like).

In addition to the circuitry and apparatus illustrated and described herein, a 1PPS reference single(s) may be supplied by a separate component not separately shown in the Figures or provided by one or the components illustrated in the drawings. Moreover, other frequencies may be used, such as but not limited to 10 KHz. For example, such a reference signal may be implemented in FPGA1008inFIGS. 10 and 10A, SmartNIC chip1108inFIG. 11, and switching circuitry logic inFIG. 12.

In the foregoing embodiments the terms “PHY” and “MAC” are used; however, this terminology is not meant to limiting, as circuitry performing similar functions that are not called PHY and MAC may also be used. For example, such functions include converting between a physical analog signal domain to a digital domain under which data (such as data in packets, preamble, frames, etc.) may be manipulated. Thus, timestamps may be added/updated in the digital domain.

In the description and claims, the terms “coupled” and “connected,” along with their derivatives, may be used. It should be understood that these terms are not intended as synonyms for each other. Rather, in particular embodiments, “connected” may be used to indicate that two or more elements are in direct physical or electrical contact with each other. “Coupled” may mean that two or more elements are in direct physical or electrical contact. However, “coupled” may also mean that two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other. Additionally, “communicatively coupled” means that two or more elements that may or may not be in direct contact with each other, are enabled to communicate with each other. For example, if component A is connected to component B, which in turn is connected to component C, component A may be communicatively coupled to component C using component B as an intermediary component.

As used herein, a list of items joined by the term “at least one of” can mean any combination of the listed terms. For example, the phrase “at least one of A, B or C” can mean A; B; C; A and B; A and C; B and C; or A, B and C.