MFC r241388-241393:
r241388:
  If the maximum payload size is 256 bytes or more, set the DMA write
  water mark to 256 bytes.  Otherwise controller will encounter DMA
  write under run errors and would result in RX DMA hang. If the
  maximum payload size is 128 bytes, the water mark is set to 128
  bytes as usual.
  While here, set maximum read request size to 2048 for BCM5719/BCM5720.
  For other PCIe devices, use 4096. And reprogram the maximum read
  request size whenever device reset is performed.

r241389:
 On PHY write error use hex number to show the value.
 Add more comments.

r241390:
  Honor PHY type fiber for BCM5717/BCM5718/BCM5719/BCM5720.

r241391:
  Do not force PCIe 1.0a mode in device reset on BCM5717 and newer
  controllers. BCM5785 does not require PCI 1.0a mode as well during
  reset.

r241392:
  Fix a long standing VCPU reset sequence bug on BCM5906.
  The VCPU(Virtual CPU) of BCM5906 is used to provide a mechanism to
  control the bootcode execution and to pick up configuration data
  stored inside the EEPROM.
  The bootcode of BCM5906 will check the BGE_VCPU_STATUS_DRV_RESET
  bit to decide which booting procedure to choose.
  Data sheet indicates the VCPU of BCM5906 should set
  BGE_VCPU_STATUS_DRV_RESET bit *before* VCPU reset or global reset.

r241393:
  Remove unnecessary delay. I don't see any comments in data sheet
  that requires 10ms delay after device reset.  Because that code was
  there from day 1, I guess it was added to give enough settlement
  time after updating BGE_MAC_MODE register.
  The recommended delay time for BGE_MAC_MODE after updating is 40us
  and it was already done in r241219.
MFC r241436:
  Rework controller reset procedure. Previously driver saved
  BGE_PCI_PCISTATE register before issuing global reset. After
  issuing reset, it reads BGE_PCI_PCISTATE register again and
  compares the saved register value and current value. It was used to
  know whether the global reset operation was completed or not.
  Unfortunately, this logic caused several issues on recent BCM5717/
  5718/5719 and BCM5720 controllers. It seems APE firmware accesses
  some registers while global reset is in progress such that reading
  BGE_PCI_PCISTATE register after reset does not yield old pre-reset
  state value. This resulted in consuming too much time in global
  reset and sometimes it couldn't successfully complete reset.

  The BGE_MISCCFG_RESET_CORE_CLOCKS of BGE_MISC_CFG register is
  self-clearing bit so driver is able to know the reset completion.
  But the core-lock reset will disable indirect/flat/standard access
  modes such that driver cannot poll BGE_MISCCFG_RESET_CORE_CLOCKS
  bit of BGE_MISC_CFG register. So just wait enough time for
  core-clock reset to complete.
  Data sheet says driver should wait 100us for PCI/PCI-X devices and
  100ms for PCIe devices. I chose 1ms for PCI/PCI-X since this value
  was used for many years in bge(4). For PCIe devices, use 100ms as
  recommended by data sheet.

  bge_chipinit() also cleared BGE_MAC_MODE register which shall clear
  firmware configured mode information. I think this will result in
  losing ASF/IPMI link in device attachment. Let bge_reset() honor
  firmware configured BGE_MAC_MODE register and don't announce driver
  is UP in bge_reset(). Firmware should have control over driver until
  it's fully initialized by driver.

  While I'm here, enable workaround for PCI-X BCM5704 A0 in
  bge_reset(). This will prevent internal arbitration logic from
  switching to the other DMA engine after a retry cycle.
MFC r241438:
  Add APE firmware support and improve firmware handshake procedure.
  This change will enable IPMI access on 5717/5718/5719/5720 and 5761
  controllers. Because ASF is not available when APE firmware is
  present, bge_allow_asf tunable is ignored when driver detects APE
  firmware.  Also bge(4) no longer performs two resets(one blind
  reset and the other reset with firmware in mind) in device attach.
  Now bge(4) performs a reset with enough information in bge_reset().
  The APE firmware also needs special handling to make suspend/resume
  work but it was not implemented yet.

  With this change, bge(4) should work on any 5717/5718/5719/5720
  controllers. Special thanks to Mike Hibler at Emulab who setup
  remote debugging on Dell R820. Without his help I couldn't be able
  to address several issues happened on Dell Rx20 systems. And many
  thanks to Broadcom for continuing to support FreeBSD!

  Submitted by:	davidch (initial version)
  H/W donated by:	Broadcom
  Tested by:	many
  Tested on:	Del R820/R720/R620/R420/R320 and HP Proliant DL 360 G8
MFC r241983-241985:
r241983:
  Do not hardcode phy address. Multi-port controllers use different phy
  address.

r241984:
  Ethernet@WireSpeed is defined for 1000baseT adapter to establish a
  link at a lower speed so enabling it for fiber adapters is wrong.
  Fix the issue by setting BGE_PHY_NO_WIRESPEED such that brgphy(4)
  wouldn't enable the feature.
  While I'm here move PHY specific feature/bug configuration to new
  location(just before mii attach) for readability.

r241985:
  For fast ethernet controllers, Ethernet@WireSpeed is not defined so
  explicitly set BGE_PHY_NO_WIRESPEED flag.
MFC r242426:
  TCP/UDP checksum offloading feature for IP fragmented datagram was
  removed in r99417.  bge(4) controllers can do TCP checksum offload
  for IP fragmented datagrams but unlike ti(4), it lacks UDP checksum
  offloading for IP fragmented datagrams. The problem was bge(4)
  blindly requested TCP/UDP checksum for IP fragmented datagrams such
  that it resulted in corrupted UDP datagrams before r99417.
  Remove remaining code for TCP checksum offloading for IP fragmented
  datagrams which should have been removed in r99417.