lwip - limiting CPU usage on STM32 - Page 1
Forum section: electronics / general

author: harerod
author_rank: Frequent Contributor
timestamp: January 27, 2022, 07:17:31 pm
content: Scenario: STM32F4 with lwIP and freeRTOSThis particular device is inside a safe environment, which must never have an internet connection. Protocol errors, e.g. due to lost UDP packets, will be handled by another layer.One weak point of this whole setup is the variable CPU load lwIP generates. Normally we have sparse UDP exchanges, one telegram every second or so. At an higher packet rate, e.g. TFTP transfers, lwIP really starts eating into the available CPU bandwidth.What I have been looking for, but couldn't find a good answer to, is: How could we go about limitting the CPU bandwidth that lwIP may eat up?One idea that came to my mind, is monitoring CPU load (WWDG+EWI (Early Wakeup Interrupt)) and dynamically changing the priority of / disabling the ETH Interrupt. WWDG+EWI would be rather simple to add, since the device only uses IWDG so far.Any comments or better of the shelf solutions that I didn't find / think of?edit 2201301639: corrected typo in headline

author: thm_w
author_rank: Super Contributor
timestamp: January 28, 2022, 02:20:59 am
content: Can you shut off the interrupt if the lwIP buffer reaches a certain size (without cutting off a packet)?Then the RTOS task priority should be lower, so it will not re-enable interrupt until that task has time to run and process the data.But I guess that would need to be done within the interrupt itself.Code:/*** @brief STM32F2 Ethernet MAC interrupt service routine**/void ETH_IRQHandler(void){bool_t flag;uint32_t status;//Interrupt service routine prologueosEnterIsr();//This flag will be set if a higher priority task must be wokenflag = FALSE;//Read DMA status registerstatus = ETH->DMASR;//Packet transmitted?if((status & ETH_DMASR_TS) != 0){//Clear TS interrupt flagETH->DMASR = ETH_DMASR_TS;//Check whether the TX buffer is available for writingif((txCurDmaDesc->tdes0 & ETH_TDES0_OWN) == 0){//Notify the TCP/IP stack that the transmitter is ready to sendflag |= osSetEventFromIsr(&nicDriverInterface->nicTxEvent);}}//Packet received?if((status & ETH_DMASR_RS) != 0){//Clear RS interrupt flagETH->DMASR = ETH_DMASR_RS;//Set event flagnicDriverInterface->nicEvent = TRUE;//Notify the TCP/IP stack of the eventflag |= osSetEventFromIsr(&netEvent);}//Clear NIS interrupt flagETH->DMASR = ETH_DMASR_NIS;//Interrupt service routine epilogueosExitIsr(flag);}But how often can this interrupt occur?I can't see it happening so much that it overtakes everything else. lwip should not be running until any higher priority tasks are finished.

author: westfw
author_rank: Super Contributor
timestamp: January 28, 2022, 03:33:39 am
content: QuoteWell, for TFTP, you can always ACK more slowly...Most of the IP protocols suites are sort of self-timed. If you want them to use less CPU, adjust the ack timing.(and make sure things like the Nagel algorithm are turned on. That'll optimize packetization based on performance as well as network bandwidth.)(that's generic advice. I'm not 100% certain that it applies to lwip, or how, but ... it should.)

author: capt bullshot
author_rank: Super Contributor
timestamp: January 28, 2022, 02:27:46 pm
content: Back in the days, I had a similar issue with LwIP.What I did, afair, was to limit the processing rate of the incoming packets, by some artificial delay.MAC -> chained DMA -> bottom half interrupt handler enables upper half handler by OS call -> upper half handler runs in OS context (insert delay here afair) sets up new DMA buffers, and processes received packet.Anything else worked by itself then, reducing the OS load as a result.

author: harerod
author_rank: Frequent Contributor
timestamp: January 28, 2022, 05:46:35 pm
content: Quote from: thm_w on January 28, 2022, 02:20:59 amWhenever data appears at the Ethernet interface, which has either the correct IP address or is a broadcast. At 100Mbit this can be fairly often. This is the reason why I am looking for a way to throttle that interrupt.You can reproduce the effect with any Ethernet tool that provides "stress testing". With "Packet Sender" that would be "intense traffic generator".

author: peter-h
author_rank: Super Contributor
timestamp: January 29, 2022, 06:15:31 pm
content: I am not sure if this makes sense because it was someone else working on my project who did the ethernet code (we are using ST's LWIP library, with MbedTLS but that's a different thing) but we arepollingthe ethernet stuff instead of running under interrupt, from an RTOS task.This reduces ethernet performance but it putsusin control, so we cannot get overwhelmed by fast incoming packets. That's a known issue in embedded systems; there have been products which would practically hang because they could not cope with incoming data, and their ethernet controller didn't support hardware packet filtering (by IP).And one can control the RTOS (FreeRTOS) priority, etc.I can get you a more detailed description if you need it. I think it may be the same thing as westfw has suggested above.We have a LAN8742, like one of ST's development boards. I don't know if this hashardwarepacket filtering by IP. I suspect not, otherwise how would you create a system which responds on multiple IPs (whichispossible on say a PC).

author: harerod
author_rank: Frequent Contributor
timestamp: January 29, 2022, 08:56:44 pm
content: peter-h, thanks for sharing. Would you mind telling us which protocols you are using?As for the PHY - think of this component as an interface between the MAC and the actual transport medium. The MAX232 of Ethernet, if you will. I have written adaptations for more PHY's with STM32's than I can remember. They are largely interchangeable. Adaptation is mostly about different ID registers and different special capabilities. They are connected to the MAC via two interfaces. One is called MDIO, I2C-like and used for configuration. The other is the data highway, the MII. Check your PHY datasheet for more info.Information about the STM32F4 ETH Media Access Control can be found in RM0090 chapter ETH/MAC. It is this MAC that does basic filtering. Not every packet on the medium causes an interrupt, only if it fits certain criteria. Hence my comment about broadcast and IP-address in my last post.

author: westfw
author_rank: Super Contributor
timestamp: January 29, 2022, 11:29:49 pm
content: QuoteQuoteSo which is causing your performance problems? The traffic that you are specifically using/requesting, extra broadcast/multicast traffic on the net, or "uninteresting" traffic directed at your ethernet address? Is your net "healthy" (ie not prone to "broadcast storms", DoS attacks, or other issues)? Does a packet trace from your "busy" period look reasonable? (alas, such traces are harder to get than they used to be.)There's a principle: "Random Early Drop" that suggests that dropping packets "early" (ie before you spend a lot of CPU cycles trying to interpret them, or fill queues with them) is useful for avoiding congestion. But it's usually aimed at network congestion (and it's been pissing me off for 30+ years that very little of the "congestion avoidance" research has ignored endpoint congestion...)

author: nctnico
author_rank: Super Contributor
timestamp: January 30, 2022, 12:08:14 am
content: Quote from: harerod on January 29, 2022, 08:56:44 pmYou will need to handle broadcast packets and packets with the MAC address of your device otherwise things will be seriously broken. However a simple solution is to let the ethernet buffers overflow. If you have a decent MAC and microcontroller then it will use DMA to transfer data which sits on a bus which is normally not used for CPU <-> memory transfer (at least NXP's ARM microcontrollers are built this way). The MAC will keep pumping data into the buffers but they are not getting processed.Lwip is polled using an OS thread (in a function called tcpip_thread )so if you put a sleep (20ms or so) in that polling thread, you'll tune lwip down a bit without breaking stuff. But you'll get a much lower network datarate to your device because it will take 20ms before the next packet is being processed. Another option is to add a loop counter and execute the sleep every 10 loops (play with the number of loops and delay a bit until you get the desired effect).Another option is to use an external chip like the Wiznet W5500. This offloads dealing with network traffic completely and only forwards network data that is relevant.

author: westfw
author_rank: Super Contributor
timestamp: January 30, 2022, 01:34:26 am
content: QuoteBut it sounds more impressive on your resume if you call it "Random Early Drop"!(but yeah; in general you don't want to receive more pps than you can effectively process, and the earlier you get rid of them, the less processing they use. Letting them fail at the MAC level ought to be fine, unless there is some degenerate condition that causes the important packets to be dropped more often than the unimportant packets.)

author: nctnico
author_rank: Super Contributor
timestamp: January 30, 2022, 02:00:18 am
content: Quote from: westfw on January 30, 2022, 01:34:26 amThat is always the issue with not being able to keep up. For a recent project I had to resort to a bit-banged MAC due to component shortage. Ofcourse there is no way it can keep up if there is a huge burst of broadcast traffic but re-transmits implemented in most protocols make that it works just fine nevertheless.

author: peter-h
author_rank: Super Contributor
timestamp: January 30, 2022, 10:31:23 am
content: QuoteI am not 100% sure but IIRC it is HTTP, HTTPS (MbedTLS), NTP, DHCP, ICMP (you can ping it). There are a lot more sitting in the ST librarybut I don't know which of these (e.g. TFTP) are actually enabled.Thanks for the explanation of the interface. I have not been involved with this fr over a year so I forgot (spent some time on the 25MHz/50MHz clock arrangement which seems to be a hotly debated topic). There is an RJ45 with integrated magnetics (the Hanrun usual type), then the LAN8742 which is connected using the reduced interface to the 32F417.

author: harerod
author_rank: Frequent Contributor
timestamp: January 30, 2022, 03:38:44 pm
content: Everybody, thanks for your input. Food for thought. For the moment I would prefer to stick with the setup described in my previous posts (STM32 + PHY).Even if the device is in a safe environment, I am concerned with its weak points. Fast Ethernet allows frame rateswellin excess of1MHz100kHz, so any rogue participant in the local network could be a problem. One real world example that I saw recently, was a bug in a third party software, which started spamming the network.peter-h, thanks for your input. You anticipated and already answered my next question via private mail: your MAC fills DMA buffers with pre-filtered packets, whose status is polled by an RTOS task. I can see how this would give CPU/bus bandwidth control. I will look into this concept. I will also have to check how much bandwidth is used when the buffers are full and packets are being dropped. Again, my job of checking the RM0090 and my firmware setup.edit 220131: thanks to westfw I corrected the framrate estimate

author: westfw
author_rank: Super Contributor
timestamp: January 30, 2022, 08:55:59 pm
content: QuoteThe maximum frame rate for 100Mbps (“fast”) Ethernet is “only” about 148kfps.(Minimum frame is 84 bytes.)

author: peter-h
author_rank: Super Contributor
timestamp: January 31, 2022, 07:43:22 am
content: That is however a high interrupt rate - one every 7us.The ISR would have to be very short. The HAL coders have a company requirement of a minimum ISR size of 100 lines of C, of which at least 30% have to be if() statements skipping past unused code, which makes that 70ns per line of C

author: harerod
author_rank: Frequent Contributor
timestamp: January 31, 2022, 08:56:50 am
content: westfw - serves me right for working on a Sunday. I have corrected the frame rate estimate in my last post. Seems that I had slipped into the Gigabit Ethernet section:https://wiki.networksecuritytoolkit.org/nstwiki/index.php/LAN_Ethernet_Maximum_Rates,_Generation,_Capturing_&_Monitoring