Renewing my lab LAN - Page 1
Forum section: electronics / general

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 15, 2024, 12:20:36 am
content: I'm going to upgrade my lab LAN. Currently, it's still 1Gbe with about 10 equipments connected to LAN. And then WiFi.I'd like to upgrade to at least 2.5Gbe, possibly 10Gbe if it's not too expensive (which it looks to be still). So, that would be one router and 10 ports. The router can have fewer ports, and I'd add a switch if needed. 2 sub-lans would be enough, and ideally 2 WAN ports, so a 4-port router, and one or two switche should do.Also, I'm considering fiber rather than copper. Again if the cost isn't prohibitive.What kind of equipment would you suggest? (Again, not expensive stuff.)

author: Halcyon
author_rank: Unknown
timestamp: July 15, 2024, 12:59:18 am
content: Define "expensive". How much are you looking to spend in total?To give you some idea, I'm a huge fan of pfSense and Netgate appliances. You can pick up something like the Netgate 2100 for USD$349, which will give you a very decent, commercial router/firewall product. If you have a very fast internet connection (say 1 Gbps or more), you might want to go up to the Netgate 4200. Of course, pfSense is still available for free, which you can run on any x64-based* machine with a few NICs.As for switches and wireless, it's hard to beat Ubiquiti on price, performance and features, however some people don't like the idea of having to run a seperate network controller. I prefer it (I just run a dedicated VM, but you can host it on your desktop PC. It doesn't have to be running all the time, just when you need to monitor or reconfigure anything). If you're looking to build a 10 Gigabit backbone to your network, look at the Ubiquiti Pro switches. They have 10 Gig SPF+ ports and come in various models depending on how many copper ports you want, and whether you want PoE or not. The Pro 24 is around USD$399.As for fibre vs copper, that will depend on your requirements. If you're in an area prone to storms, fibre on the WAN side, absolutely, even if it's just a short length with media converters. That will act as an air gap between the outside infrastructure and your network. For everything inside, I just use Cat 6. It's good up to 10 Gigabits. I run DAC cables between the switches and servers (these are cheap).10 gig NICs can be expensive, but you might find some at a good price on ebay or similar. I picked up a bunch of Intel X520-DA2 NICs for about AUD$75 each a while ago. Being SFP, you can decide whether you want to use fibre or direct attach copper cables. Just watch out for Chinese fakes. If they are "too cheap", they might not be genuine cards.You can have as many VLANs as you want on the same equipment. You just need to configure your router and switch(es) accordingly and decide whether you want Tagged vs Untagged VLANs. (I use port-based untagged VLANs for my devices.)If all this is sounding too expensive, some of the "budget" brands I would go for are:Switches: Netgear business-series; MikroTik; Second-hand Cisco switches.Wireless: MikroTik; Second-hand Cisco or HP Enterprise.Routing: pfSense on your own hardware; MikroTik.Don't buy into this whole wireless "mesh" bullshit the consumer brands love to push. Meshing becomes entirely irrelevant if you're connecting your wireless access points via an ethernet cable. The ability to mesh only becomes useful if you aren't able to run a network cable to a radio, and even then, has its limitations.* I know pfSense will also currently run on x86 hardware, however talk of dropping 32-bit support has been around for a while. It'll happen eventually.

author: Psi
author_rank: Super Contributor
timestamp: July 15, 2024, 01:19:21 am
content: I wouldn't bother with fiber unless you have a reason to

author: Postal2
author_rank: Frequent Contributor
timestamp: July 15, 2024, 01:42:23 am
content: Am I right that optical communication gives a high ping?

author: Halcyon
author_rank: Unknown
timestamp: July 15, 2024, 02:07:03 am
content: Quote from: Postal2 on July 15, 2024, 01:42:23 amWith all things being equal, ever so slightly, but realistically not enough for most people to notice or even care about. You'd be talking nanoseconds.However with copper, other issues could impact overall latency, such as crosstalk (which may lead to delays due to re-transmission). Considering most of the internet is built on fibre, your last little bit of cabling in your house is going to make zero difference.

author: Postal2
author_rank: Frequent Contributor
timestamp: July 15, 2024, 02:12:18 am
content: Quote from: Halcyon on July 15, 2024, 02:07:03 amA friend of mine could not get less than 10 ms locally on optical fiber. That's why I asked.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 15, 2024, 02:47:49 am
content: I now have 2Gbps fiber for Internet.You're right that fiber indoors would probably make little sense for now. One reason would be redoing (or doing from scratch) a complete installation and replacing all cables. Fiber takes significantly less space, so it's easy to fit almost anywhere.Price range, that would depend on wether I redo all the cabling or not. I was thinking under $1000 if so, and probably about $500 for just a new router/switch and possibly some new cables. Which sounds doable.For the router, I'd want something running Pfsense or OpenWrt - definitely not one with a proprietary firmware. I've seen some mini-PCs with 4 2.5Gbe ports, but I don't know how reliable these are, nor what the idle power consumption is. My impression is that it's a bit of a gamble with these mini PCs.

author: Smokey
author_rank: Super Contributor
timestamp: July 15, 2024, 03:16:53 am
content: What are you doing now that maxes out gigabit ethernet?

author: Smokey
author_rank: Super Contributor
timestamp: July 15, 2024, 03:23:09 am
content: Quote from: SiliconWizard on July 15, 2024, 02:47:49 amWhat about AsusWRT?

author: Halcyon
author_rank: Unknown
timestamp: July 15, 2024, 04:18:13 am
content: Quote from: Postal2 on July 15, 2024, 02:12:18 amThat could be due to any number of things. I'm getting sub-10ms ping times out to the internet, which is fibre all the way to the house. On a local network, it should be far less than that (even on WiFi).

author: Halcyon
author_rank: Unknown
timestamp: July 15, 2024, 06:01:33 am
content: Quote from: Halcyon on July 15, 2024, 04:18:13 amQuote from: Smokey on July 15, 2024, 03:23:09 amMy view on AsusWRT is that it's fine if you already have an Asus router and want to improve the existing firmware, but I wouldn't recommend it if you're in the market for a new router. It's still very limited in terms of capability compared to something like pfSense or Opnsense.

author: Berni
author_rank: Super Contributor
timestamp: July 15, 2024, 06:24:02 am
content: Quote from: SiliconWizard on July 15, 2024, 02:47:49 amAh yeah the usual "My internet got faster than my LAN so now i have to upgrade"Going faster than 1Gbit indeed can get expensive. 2.5G is catching on and you can get routers and network cards for it cheap (some new motherboards even come with 2.5G built in). So id say it is worth getting all 2.5G switches with a bit of 10G mixed in for when speed is actually needed. Going all 10G is expensive and you will not need those sort of speeds for the next decade.For home use 10G over CAT6 is good enough. Most 10G interfaces are SFP ports anyway so you don't need to choose between fiber and copper. Yes fiber is thinner but only the actual fiber, the connectors on the end are not all that thin, also you can't easily cut it to length and terminate it yourself, the fiber is also a lot more delicate and has a minimum bend radius. So running fiber trough your house might not be as easy. Besides you likely already have a lot of twisted pair copper in your walls, just use that. Sure it might not be cable rated for 10G, but for short runs it usually works fine. If it doesn't work then you can always splice better cable onto it, then use the old cable to pull the new cable trough.Doing NAT routing on 10Gbit of internet traffic (for future proofing) becomes quite a task, but for home use with a small number of clients it is easier. Good excuse to set up Pfsense on a small PC if you are into it.Tho for me personally my 1Gbit fiber i have into the internet is less important than having 1Gbit to my NAS server where i keep most of my data. That NAS is due for a upgrade. Currently has dual 1Gbit integrated NICs while the one PCIe slot is taken up by a SAS HBA card. I feel like 2.5G is too small of a step up from 2x1Gbit, so i am mostly waiting for 10G to become more widespread and come down in cost, then rebuild the whole NAS to some more modern hardware.Quote from: Postal2 on July 15, 2024, 02:12:18 amTypical LANs have ping times way under 1ms so it can't be measured by simply running ping in a console.(unless your "LAN" is connecting buildings many kilometers apart). Most of the cause of ping is due to having to share bandwidth with other traffic. If the common network link that your traffic is destined for is already transmitting someone else's traffic then your packets have to wait for their turn. This is made even worse if the "last mile" network of your ISP also commons connections together, this is very common with coax DSL where an entire street shares one backhaul, so every DSL modem in that street can hear everyone elses upload traffic, so for this to work they need collision avoidance where each modem is assigned a time slot for transmuting so that they don't talk over each other (the time of flight of the coax is also measured and compensated for) hence all upload traffic needs to wait for a time slot. Similar things are also done with some ISPs fiber networks where passive splitters are used to share one big backhaul fiber over multiple subscribers.I get about 3 to 4ms of ping out into the internet over fiber. But this is of course only to things near my ISPs central office.

author: dietert1
author_rank: Super Contributor
timestamp: July 15, 2024, 06:42:45 am
content: Currently i am trying fiber ethernet for some metrology application. I got a 20m duplex multimode fiber and two tplink MC100CM media converters, all fast ethernet, for about 50 € in total. Of course ping response times are consistenly < 1ms.Modern gigabit equipment should behave like that or better, yet i don't see a need to upgrade. Avoid waste! Observe energy consumption, which may be higher for gigabit capable stuff.Regards, Dieter

author: RoGeorge
author_rank: Super Contributor
timestamp: July 15, 2024, 06:49:56 am
content: Wired LAN can do POE, which might be convenient, and waste less electricity than adding a wall adapter for each device. Depends on what's connected to your LAN.Another thing, I don't think it will be any noticeable improvement between 1 and 2.5Gbps during day to day usage. Most probably you'll notice the difference only if you run a benchmark.

author: Postal2
author_rank: Frequent Contributor
timestamp: July 15, 2024, 07:25:41 am
content: Nevertheless I think copper is better for home.Quote from: SiliconWizard on July 15, 2024, 02:47:49 amWell, I congratulate, your business is booming. Naturally, you need a lot of traffic.Providers are usually cunning. They say 2Gbps, meaning 1 down and 1 up.Quote from: Berni on July 15, 2024, 06:24:02 amNot only. Optical transmitter will wait for max payload of packet. Ping sends too small amount.

author: bingo600
author_rank: Super Contributor
timestamp: July 15, 2024, 10:04:26 am
content: Imho - For private usage 10Gb for everything is overkill.And the "power usage" of 10Gb Copper interfaces are "just ugly".I have deliberately postponed 2.5Gb to the "desktop", until the prices for 2.5Gb switches comes down to something that resembles the current 1Gb prices.And it's actually quite rare i max out my existing 1Gb network, plus most of my "daily drivers" are on WiFi anyway ....That said - I do have a 10Gb 8-port fiberswitch w. SFP's , and a 8-port 2.5Gb w. 10Gb uplink in the drawer.It's waiting for my new DELL proxmox server to be setup w. an Intel-X520 card.Until the DELL server is up, i'll just leave my 10Gb in the drawer.Just my 0.02€ opinion..

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 15, 2024, 10:52:09 pm
content: You'll "max out" any data link, even 10Gb, if you transfer data from/to SSDs (at least NVMe). So that question is a non-question. Even a HDD from 10 years ago could do better than 100 MBytes/s. So, the LAN becomes a bottleneck. Whether that matters in your own use cases is your call. 1Gb has been default for at least 15 years. Looking for a bit of upgrade is not unreasonable.2.5Gbe is uh, like 2.5 as fast, so that's a start. But clearly that would be a compromise, as 10Gb is still way expensive. And yes, it's quite power hungry too. Actually, I think 10Gb on copper is very power hungry, and should be a bit less so over fiber.

author: Postal2
author_rank: Frequent Contributor
timestamp: July 15, 2024, 11:42:10 pm
content: Quote from: SiliconWizard on July 15, 2024, 10:52:09 pmIt turns out that cheap water and a very expensive bucket. Dissonance.

author: Halcyon
author_rank: Unknown
timestamp: July 16, 2024, 01:16:27 am
content: 10 Gig has its uses. I only run it between servers and the network backbone, not to client devices, but it still allows me to max out the performance on my old NAS with spinning rust disks. I can copy data to the NAS as speeds well in excess of what a 1 Gbps network can handle.It really comes down to what you're using it for. Even if you just want to have a play and have the funds to do it, I say go for it! Everyone has a hobby, for some it's tinkering with networks. Do I need a full blown server rack and room in my house? No, but it's fun!

author: Postal2
author_rank: Frequent Contributor
timestamp: July 16, 2024, 01:52:51 am
content: Quote from: Halcyon on July 16, 2024, 01:16:27 amI agree and understand. I have a neighbor who uses a hammer drill as a hobby. Network devices as a hobby is much better.

author: Berni
author_rank: Super Contributor
timestamp: July 16, 2024, 05:48:00 am
content: Yeah power usage of 10G is quite a factor, it can burn a couple of watts per port.You can get that down using direct attach copper SFP cables, tho running those in walls is not really an option. But is very nice for connecting together switches and servers in the same rack/room.I guess things got so stagnant in the networking world because going faster presents technical challenges while at the same time people in most cases don't really need faster networking. 1Gbit gets you around 120MB/s and that's about the speed that spinning rust HDDs run at. And it means you can copy 1 DVD worth of data in about half a minute. I find myself rarely moving more data than that at a time (apart from backups, but those run automated in the background anyway). But going from 100Mbit to 1Gbit was an absolute game changer if you ever transfer files over the LAN.I think that give it 5 to 10 years for the cost of SSD storage to become so cheap that it will make sense to build a fully solid state NAS and then 10G will start making a bit more sense, hopefully with 10G coming down in cost to something reasonable too.

author: Postal2
author_rank: Frequent Contributor
timestamp: July 16, 2024, 07:03:53 am
content: I would like to remind about powerline adapters, which provide distances that Wi-Fi in the city can't reach.

author: Halcyon
author_rank: Unknown
timestamp: July 16, 2024, 07:43:41 am
content: Quote from: Postal2 on July 16, 2024, 07:03:53 amTrue, but those powerline adapters aren't without their problems and performance is highly variable. I'm not a fan.

author: Postal2
author_rank: Frequent Contributor
timestamp: July 16, 2024, 08:26:45 am
content: Quote from: Halcyon on July 16, 2024, 07:43:41 amI have 6 pcs (2+2+2) of different types. In the worst combination you will always get 20 Mbps. In most cases you will get over than 80 Mbps. One pair of my adapters (modern, rated as gigabit) gives 200 Mbps. All adapters are compatible with each other and can be combined as lan.

author: nctnico
author_rank: Super Contributor
timestamp: July 16, 2024, 09:58:07 am
content: Quote from: SiliconWizard on July 15, 2024, 02:47:49 amBut you'll also need tools like fiber & optical port cleaners. A good pair of those will set you back about 150 euros. Dirty LC connectors will cause problems. I always clean ports & connectors even when the fibers and optical parts are new from their package.If I where to upgrade, I'd use 10Gbit for the backbone (from internet router to office) and 1Gbit to the devices. 10Gbit all the way would only make sense if you need to have a lot of traffic between 2 devices.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 16, 2024, 10:00:14 pm
content: Quote from: nctnico on July 16, 2024, 09:58:07 amYes, obviously I don't need more than 1Gb for all devices (since many of them can't handle that and can't be upgraded), but I need a router than can handle more - after that, I'm thinking some 1Gb ports and a few 2.5Gb ports (the latter mainly for two machines at the moment, plus a NAS) should do at this point. And 2.5Gb starts becoming available by default on newer computers.As I mentioned, wondering if using one of these mini-PCs as a router is a reasonable option, compared to buying a "standalone" router. Benefit of the former is that I'm more likely to get the specs I want, and install whatever I want on it. Drawback is potentially reliability and idle power.

author: nctnico
author_rank: Super Contributor
timestamp: July 16, 2024, 10:36:31 pm
content: From a maintenance, software update en reliability POV, I'd go for a dedicated router from a reputable brand. Using a small PC means you'll need to figure out which updates to install and hope it will always boot after a sudden power failure.Even when using 2.5Gbit, I'd fit cabling suitable for 10Gbit.

author: 2N3055
author_rank: Super Contributor
timestamp: July 16, 2024, 10:53:02 pm
content: I have Mikrotik RB4011iGS+RM router at one side and CRS326-24G-2S+RM 24 Gigabit port switch with 2 x SFP+ cages connected with optical cable on another side of office with instruments.10 GBit.Router is connected with it's WAN port to provider's router, that segment works as DMZ.On router I run DNS cache, DHCP and NTP server.I use static DNS entries for some addresses and also permanent leases on DHCP.Works real well.Plan is to connect other SFP cage on switch via 10GBit to storage server in future.

author: Halcyon
author_rank: Unknown
timestamp: July 17, 2024, 01:13:14 am
content: Quote from: Postal2 on July 16, 2024, 08:26:45 amThose kinds of speeds are pretty awful considering most home networks (even the cheap crappy consumer routers) have LAN ports that run at 1 Gbps.Sure, you can argue that the speeds you're experiencing are enough for you, but in my case (and many others'), their internet connection alone is significantly faster.In my experience, a properly configured WiFi network (with decent gear) is significantly better than the "best" powerline ethernet products. I'd only consider those powerline adapters as an absolute last resort, after all other options have been exhausted. The other consideration is that (just like WiFi), powerline adapters are typically half-duplex.

author: Postal2
author_rank: Frequent Contributor
timestamp: July 17, 2024, 04:16:04 am
content: Quote from: Halcyon on July 17, 2024, 01:13:14 amI don’t have full Wi-Fi coverage in my apartment (powerline was targeted initially), I use these adapters anywhere, including when visiting my neighbors (old woman without internet). Some have very thick walls.Of course, I do not suggest building a network on these adapters. You just have to remember that they are there, and sometimes it is convenient.

author: radar_macgyver
author_rank: Frequent Contributor
timestamp: July 17, 2024, 05:07:51 am
content: I use a Ubiquiti Dream Machine with a 10 Gbit/s switch (UBNT Aggregation) and a 1 Gbit/s with PoE (USW-24 PoE). I use a couple of UBNT access points running off the PoE switch. I have a couple of home servers that have 10 Gbit/s links to the Aggregation switch, but everything else is 1 Gb. My workbench has one of their small Flex Mini switches for my scope, sig gen and power supply. The Dream Machine serves as the Unifi controller.I have a 1G fiber internet connection, and running speed tests to nearby servers gives me line speed (930-940 Mbit/s). Over Wifi, this drops to about 600 Mbit/s. The UDM SE version supports 10 Gbit/s WAN and would likely work for the OP's 2 Gb internet connection. The advantage over a more homebrew solution is it's easy to fix remotely when I'm not at home - I used to have a box running OpenWRT that needed a lot more attention. I replaced it after the SO started working from home and reliability was a bit more important. The disadvantage is the cost and power consumption.

author: Berni
author_rank: Super Contributor
timestamp: July 17, 2024, 05:43:18 am
content: Quote from: SiliconWizard link=topic=434369.msg5575153#msg5575153If you are after reliability just go for a decent off the shelf router from a manufacturer that also makes enterprise grade network gear. Not only are you buying quantity made hardware that is designed to last, but you are also buying the extensively battle tested software running on them that is kept up to date at the click of a button. It just works.My router is a Mikrotik RB5009 and i am certainly happy with it, all of my WiFi APs are also Mikrotik. They are not the easiest to configure, but once you do they are rock solid reliable with plenty fast routing performance (the manufacturer also publishes routing performance benchmarks on their own websites product page). You won't hit the limits of routing ability on the higher end models unless you are routing entire housing neighborhoods of traffic (Heck i have coworkers with the cheapest cut down Mikrotik models being impressed with the performance)Another cost effective but high quality networking brand is Ubiquiti. Easier to set up and has an excellent ecosystem that works together seamlessly.The one thing to avoid is networking gear that has a "gamer vibe" to it. Like routers from Asus that look like a spaceship with a million anteenas or something. Sure Asus is a good brand, but any routers made for home use seam to be low performing unreliable crap despite still costing 300 bucks.

author: nctnico
author_rank: Super Contributor
timestamp: July 17, 2024, 08:09:20 am
content: I have some hands-on experience with Mikrotik as well and find their configuration interface far from straightforward. Maybe OK if you are configuring these all day long but for the occasional user not so much. Chances are you miss a crucial setting needed for safety. Some configuration items are in such an unlogical, place you won't find it by yourself.

author: ejeffrey
author_rank: Super Contributor
timestamp: July 18, 2024, 03:37:32 am
content: Quote from: Postal2 on July 15, 2024, 01:42:23 amNo. Optical Ethernet interconnects typically have lower latency than twisted pair, especially at 2.5 Gb and beyond. The difference is usually only hundreds of nanoseconds to a microsecond per phy. This depends a lot of the particular phy and remember you usually have 8 phy in a round trip on the same LAN. DAC is slightly lower latency than fiber although that's more like nanoseconds, and limited range. All of that is insignificant compared to Internet latency, it's only a potential difference for local connections, and even then only usually for very specialized applications.

author: Postal2
author_rank: Frequent Contributor
timestamp: July 18, 2024, 03:45:38 am
content: Quote from: ejeffrey on July 18, 2024, 03:37:32 amDo you mean that the optical connection does not use buffering at all?Have you ever opened settings of usb-uart from FTDI?

author: ejeffrey
author_rank: Super Contributor
timestamp: July 18, 2024, 03:52:12 am
content: Quote from: SiliconWizard on July 16, 2024, 10:00:14 pmI use one of those N100 mini PCs with quad 2.5 gigabit Ethernet and it works great. There are variations with 6 ports and also some beefier systems with a combination of 10 gigabit SFP+ and 2.5Base-T. I run OPNSense. In my caae it's a bit overkill as my Internet connection is 500 Mb fiber and all my high speed devices are wired and on the same VLAN so I can't really push more than 1 GB through the router itself.I also only use two ports on my router for normal use. I use VLANs and a trunk port going to the router. I have a third port set up for management access to connect a laptop if I hose the rest of the setup, but tbh I would rather have a serial console port. If I had more inter-VLAN traffic I could saturate the trunk port, but then I would rather move up to 10 Gb to get more bandwidth rather than distributing across multiple ports.

author: Berni
author_rank: Super Contributor
timestamp: July 18, 2024, 05:36:01 am
content: Quote from: Postal2 on July 18, 2024, 03:45:38 amYou can get schematics for what is inside optical SFP+ modules:https://www.ti.com/lit/an/slla335/slla335.pdfIt is basically a glorified laser driver and photodiode amplifier.The XGMII PHY for running the SPF+ cage does involve a bit of buffering since it does 64/66b encoding, but that is only 64bits of buffer. The PHYs are only slightly smarter than the SFP+ module, they are just glorified serializer/deserializers that blindly take bytes and throw them out onto the interface.The only place actual buffering happens is before the ethernet MAC block. This is necessary because the CPU is capable of throwing packets at a Ethernet port faster than the port can send them out. Or on a Ethernet switch you can have a packet from two ports on the switch get routed into the same outbound port at the same time, so one of the two packets is sent out directly while the other gets buffered and sent out after it.Maximum packet size also does not matter here because normal switches operate purely on the L2 level. They do not touch the packet contents at all, so they do not have the ability to "merge" packets together into one max size packet. They just forward packets byte per byte to wherever they need to go. The latency of this is incredibly low, to the point where the time of flight over your ethernet cables might be causing more latency if there is a decent length of cable involved. This is how the PTP (Precision Time Protocol) can synchronize clocks over a enternet network to <1us precision.

author: dietert1
author_rank: Super Contributor
timestamp: July 18, 2024, 06:21:58 am
content: Recently i implemented a fast ethernet fiber interface using the TI DP83822 PHY. In it's datasheet they say:QuoteBecause a fiber cannot represent a negative voltage the physical representation of the bit stream needs to be different, but packet structure and timing are the same.Regards, Dieter

author: Postal2
author_rank: Frequent Contributor
timestamp: July 18, 2024, 06:37:00 am
content: Thanks, I've already read everything.Quote

author: nctnico
author_rank: Super Contributor
timestamp: July 18, 2024, 10:30:13 am
content: Quote from: Postal2 on July 18, 2024, 03:45:38 amYour are comparing pancakes to oranges hereLike Berni wrote: an SFP is nothing more than a laser diode + receiver and some signal conditioning circuitry. An optical / electrical transceiver. There is no buffering at all. Whatever goes into an SFP as an electrical signal is being put onto the fiber directly (and vice versa).

author: ejeffrey
author_rank: Super Contributor
timestamp: July 18, 2024, 10:42:43 am
content: Quote from: Postal2 on July 18, 2024, 03:45:38 amCorrect. There is no buffering in an optical transceiver module. There is buffering higher in the stack but that's the same for any media.There is little to no buffering in a 100BaseTX PHY, but the 1000 Base-T and above twisted pair PHYs do a significant amount of work to stripe the data across 4 pairs, do a moderate amount of signal processing such as echo cancellation, and so they have a bit more processing latency. How much depends on the PHY.The velocity factor of twisted pair is often slightly higher than glass fiber or DAC twinax but that's a different of only several nanoseconds even for a 300m run.QuoteWhat?

author: Postal2
author_rank: Frequent Contributor
timestamp: July 18, 2024, 11:08:42 am
content: Quote from: ejeffrey on July 18, 2024, 10:42:43 amThat. This named "latency".Because all packets via passive optical network are multicast.

author: ejeffrey
author_rank: Super Contributor
timestamp: July 18, 2024, 03:01:42 pm
content: The OP was asking about Ethernet over fiber in the home. Not passive optical networks, although PON also have relatively low latency, they have a bit more and more variable latency on the transmit side due to TDM and the need to allocate transmit slots. But still in the microsecond range not milliseconds.Watever device you posted about has a USB UART. Thats not the same thing as Ethernet at all, and if it's using USB 2.0 it will add a ton of latency due to the polling interval -- although still only hundreds of microseconds.

author: Berni
author_rank: Super Contributor
timestamp: July 18, 2024, 03:04:41 pm
content: Quote from: Postal2 on July 18, 2024, 11:08:42 amNot sure what a buffer on a USB serial port has to do with this. Yes almost all interfaces on PCs have buffers because they help the CPU and hardware work more efficiently. Even your USB mouse has to wait to be polled by the PC before it can send new movement data (purely due to how the USB protocol works).GPON sharing a medium is exactly what i said onReply #11Because of having to share a common fiber there is time interleaving happening for the uplink, however this interleaving happens very rapidly, so it does not introduce that much latency variability(especially since your latency is far from consistent once it has to go trough the very busy public internet lines).Here is the ping times i get on my 1Gbit GPON internet connection when pinging a server in the same town. As soon as i ping something in the next city the ping times goes up to 5ms because that is how much extra time it takes light to travel over the public internet fibers to there and back. Pinging google.com or youtube.com i get a consistent 9ms since i guess googles CDN server is a bit farther out.Code:Pinging 92.53.***.*** with 32 bytes of data:Reply from 92.53.***.***: bytes=32 time=2ms TTL=254Reply from 92.53.***.***: bytes=32 time=2ms TTL=254Reply from 92.53.***.***: bytes=32 time=3ms TTL=254Reply from 92.53.***.***: bytes=32 time=2ms TTL=254Reply from 92.53.***.***: bytes=32 time=2ms TTL=254Reply from 92.53.***.***: bytes=32 time=2ms TTL=254Reply from 92.53.***.***: bytes=32 time=2ms TTL=254Reply from 92.53.***.***: bytes=32 time=2ms TTL=254Reply from 92.53.***.***: bytes=32 time=2ms TTL=254Reply from 92.53.***.***: bytes=32 time=3ms TTL=254Reply from 92.53.***.***: bytes=32 time=2ms TTL=254Reply from 92.53.***.***: bytes=32 time=3ms TTL=254Reply from 92.53.***.***: bytes=32 time=2ms TTL=254Reply from 92.53.***.***: bytes=32 time=2ms TTL=254Reply from 92.53.***.***: bytes=32 time=2ms TTL=254Reply from 92.53.***.***: bytes=32 time=2ms TTL=254Reply from 92.53.***.***: bytes=32 time=2ms TTL=254Reply from 92.53.***.***: bytes=32 time=2ms TTL=254Reply from 92.53.***.***: bytes=32 time=2ms TTL=254Reply from 92.53.***.***: bytes=32 time=2ms TTL=254Ping statistics for 92.53.***.***:Packets: Sent = 20, Received = 20, Lost = 0 (0% loss),Approximate round trip times in milli-seconds:Minimum = 2ms, Maximum = 3ms, Average = 2msThe time interleaving is actually a problem on coax DSL internet, those seam to use an interleaving period that does take long enough to create noticeably worse ping times. This was somewhat remedied by the more modern DOCSIS 3 standard for DSL (This one can do >1Gbit over DSL) so that the ping penalty is low enugh to not be a problem with multiplayer games, but it does still have a noticeably higher ping.Most forms of residential internet access share some sort of common medium among its subscribers, purely because it saves cost. The only common one where you had a dedicated wire was dialup, later ISDN, later ADSL/VDSL, since those work over the telephone wires where they had to be a dedicated wire for analog telephones to work. So once upgraded to broadband ADSL internet they just swap out the line modules at the telephone exchange with boardband capable ones, so you have a dedicated twisted pair copper wire back to the exchange. I personally don't remember seeing particularly impressive ping times on ADSL (but they were better than coax DSL of the day), not sure why, but then again ADSL never reached very high speeds because the crappy telephone wires designed for analog phone service are horendusly bad at high frequency, so it is a miracle they even manage to get 10Mbit speeds over those (with preemphasis/postemphesis, fancy modulation, error correction, etc.. of course ).EDIT: Oh and the other screenshot you shown is talking about the management protocol for GPON. That part doesn't carry your data but is instead used for communicating control data between the modem and exchange. That's how the modem introduces itself onto the network and gets configured for the network. This particular line is there to tell the modem firmware developers how quickly they must respond to the special control messages and the exchange firmware developers know what a reasonable timeout might be when waiting for a response from the modem before declaring the costumers modem as dead.

author: Postal2
author_rank: Frequent Contributor
timestamp: July 18, 2024, 05:16:28 pm
content: Quote from: Berni on July 18, 2024, 03:04:41 pmI understand. But how about this:frame rate = 125 μs.users = 128.125 * 128 =A friend of mine told me how he installed a passive optical network at an enterprise with about 100 workstations. He was proud that he got a 10ms ping. I asked the question why - in my opinion, it is clear why.And the answers I received were:Quote from: ejeffrey on July 18, 2024, 03:01:42 pm

author: Berni
author_rank: Super Contributor
timestamp: July 19, 2024, 05:14:14 am
content: The GPON network we have here only supports up to 64 subscribers on a single fiber, in some configurations only up to 32. The uplink frame size is indeed 125us so worst case with a max number of subscribers is 8ms. That's not that bad of a ping penalty given that most places on the internet will have more latency than that purely due to the time of flight of light down the fibers.My particular ISP is pretty good at investing into infrastructure so they likely didn't pack the max number of users onto one fiber, hence why the ping times are so low.

author: Postal2
author_rank: Frequent Contributor
timestamp: July 19, 2024, 09:09:36 am
content: Quote from: Berni on July 19, 2024, 05:14:14 amOne day I typed 192.168.0.1 (also switching the router to fixed ip) and saw a login page for an optical router from another provider, and the router was a cheap, home-class one.I have ADSL as a backup line, but it is currently emulating ADSL (encapsulated in optics on the provider side). When ADSL was real the ping was mostly 30ms. Sometimes (rarely) I have to switch to it and, in principle, I do not feel any discomfort (10Mbit).

author: bson
author_rank: Unknown
timestamp: July 29, 2024, 09:11:19 pm
content: +1 on Netgate and pfSense. I have a (now discontinued) 3100 for my 1G/1G fiber connection. The 3100 has a 2.5gbe LAN side port and a few 1gbe (3?) ports. Plugging my PC into the 2.5g LAN port I get download speeds around 110-115MB/s, on a 1gbe port it maxes out around 90MB/s. However, my WiFi access points are 802.11ax with 2.5g PoE ports (I have four of them around the house) so I'm seriously considering replacing my old 1gbe switches with 2.5gbe ones since I use WiFi from a MBP 16" M1 Max (also 802.11ax) most of my time. If my Synology NAS supported 10gbe I'd be all over it already! But alas it only supports 1gbe, but has interface bonding (which requires switch support to use for load distribution, most consumer switches will get really confused by multiple links with the same MAC addr at the other end and only speak to one of them).

author: Halcyon
author_rank: Unknown
timestamp: July 29, 2024, 11:59:19 pm
content: Quote from: bson on July 29, 2024, 09:11:19 pmThat's right. The switch itself needs to support link aggregation (or bonding), if you're using LACP/802.3ad. But you can still balance network traffic and have redundant links in other modes, without requiring support from the switch. You can use some of the "balanced" modes or the active/standby mode in these cases.Keep in mind, even if you have configured a bonded link, any single host on your network will only be able to send/receive data at the maximum rate of one of your interfaces that is a member of the bond. e.g.: If you have 4x 1 Gbit interfaces that make up a 4 Gbps interface, your maximum RX/TX speed from any one host on your network will still be 1 Gbps.

author: mianos
author_rank: Regular Contributor
timestamp: July 30, 2024, 12:32:39 am
content: This is via short range fibre on a pcie SFP, via a Mikrotik router with SFPs. This is quite a bit faster than 10mS.Code:ping lxdPING lxd.mianos.com (131.84.1.9) 56(84) bytes of data.64 bytes from lxd.mianos.com (131.84.1.9): icmp_seq=1 ttl=64 time=0.085 ms64 bytes from lxd.mianos.com (131.84.1.9): icmp_seq=2 ttl=64 time=0.068 ms64 bytes from lxd.mianos.com (131.84.1.9): icmp_seq=3 ttl=64 time=0.071 ms64 bytes from lxd.mianos.com (131.84.1.9): icmp_seq=4 ttl=64 time=0.073 msThe first ping always seems to be a few microseconds longer.

author: dietert1
author_rank: Super Contributor
timestamp: July 30, 2024, 04:38:12 am
content: Currently i'm testing a prototype of a GPIB to fiber ethernet interface, based on FreeRTOS and LwIP (derived from STM32H743 example project).The connection is implemented as TCP/IP ping-pong between the host app and the GPIB app on the MCU. Between the Windows XP host and the MCU device i have a Netgear FS105 switch and a DS104 hub (tap for packet capture) and a duplex 20 m multimode fiber with a tp-link media converter. The GPIB interface has its own fiber transceiver.A self-paced Keithley 2700 multimeter produces eight readings per second, with minor variation due to autozero/autocal. The wireshark screen shows near perfect timing with a jitter of < 0.2 msec. In addition to the GPIB controller the MCU serves a dynamic html info page with two page views per second (on two browsers).This setup runs for hours without losing a single K2700 measurement in hundreds of thousands, according to the K2700 reading numbers.Regards, Dieter

author: bson
author_rank: Unknown
timestamp: September 16, 2024, 12:24:33 am
content: I ended up adding 10G PCIe card to the Synology (bought a used Syn brand card, looks to be made by Sonnet, but then they all kind of look similar). A card to the PC, new switches (MikroTik) and it's off to the races. Works great. I have only 5 of the 8 Synology bays populated, but I still get read speeds around 750MB/s for large files. That's really a nice upgrade. Makes me want to get Sonnet's Thunderbolt to 10G SGP+ adapter for laptop use... 10G optical really reduces round trip times, which greatly improves small-file transfer speeds. (I don't have a cache SSD in the Synology. For trees with lots of small files I use rsync anyway, and the price for a NAS-endurance SSD just isn't worth it for my needs.) Between 10G and 32GB in the Synology, it's really a substantial improvement that's very noticeable for large transfers. (1.8G Wifi helps, too, when on a laptop.)

author: TomS_
author_rank: Frequent Contributor
timestamp: December 24, 2024, 11:11:34 am
content: Quote from: Postal2 on July 15, 2024, 07:25:41 amI'm not sure where you get this idea from, but it is plainly incorrect.The transmit path of an interface, and this includes drivers, doesn't care if it's connected to an optical transceiver or what ever else. If it is presented with a packet it will send it. If it is waiting to fill some buffer before transmitting then it will do the same for all kinds of media and not just optical.There's nothing special about optical that requires a certain minimum amount of traffic before you can transmit...