[solved] networking, a simple fake FIN package causes a lot of troubles  - Page 1
Forum section: electronics / general

author: legacy
author_rank: Super Contributor
timestamp: March 05, 2019, 03:18:22 pm
content: we are managing a CCC-like event, and during one of the recent preliminary meeting, we experimented an attack that shut down the whole local area network. The attacker is anonymous but we found his source code, so he operated inside our LAN. He used a long piece of code which creates fake FINs packages and open a TCP/IP ports. We are still studying the source, and it's not 100% clear. When it tries to open a port, then it invokes these two linesCode:shutdown(sock, SHUT_RDWR);close(sock);Shutdown() is a socket's primitive offered by the standard UNIX networking engine, and it sends a FIN package. Combined it seems that the approach is causing problems on our local network because computers do no more responsible within 50 minutes.edit:I was focused on the wrong piece of code. The attack can be classified as "SYN flooding", so this topic is now cleared from useless links, and the title is edited with edit2:Things can also be summarized this wayQuote

author: madires
author_rank: Super Contributor
timestamp: March 05, 2019, 04:34:53 pm
content: Quote from: legacy on March 05, 2019, 03:18:22 pmYou can't close ports remotely. It seems that you're DOSing yourself.

author: xani
author_rank: Frequent Contributor
timestamp: March 06, 2019, 01:05:53 am
content: It would be much easier to guess what exactly the problem is if you made a tcpdump of the traffic (wireshark isAMAZINGtool to analyze dumps made by it). Trying to look at C calls and effects via port scanner is like using multimeter instead of oscilloscope. Even better if you posted the dump so we can see it (if it's nothing secret)As for Linux side, look into:dmesg command or kernel log - linux kernel will generally yell there when you hit any limitss --allwill show all open and listening sockets in system,ss -twill do same for only TCP connections. If you see few thousand connections there you might just created a lot of unclosed connections.lsof -iif your machine doesnt have ss commandTCP by default have LONG timeouts, and by that I meanHOURSso if daemon listening on port didn't change anything idle connection can sit unused for a long timenormally you can't have more than 30k to ~64k connections to a given port between same pair of IPs, you just run out of source ports. A lot of connections that are not closed properly can cause that.

author: bson
author_rank: Unknown
timestamp: March 06, 2019, 03:24:49 am
content: Quote from: legacy on March 05, 2019, 03:41:38 pmThe way it's supposed to work is:Client sends data to the serverServer responds with data to the clientServer, when done shuts down itswritesideThe client reads remaining data and when its read side is fully drained it gets an EOF (because the server sent a FIN on shutdown)The client shuts down its write side and closes the socketThe server gets an EOF when the client has shut down its write side, and this confirms the client has seen everything sentThe server closes the socketWhen a TCP connection is closed its goes into a TIME_WAIT state. This is to prevent the same address,port pairs to be used again since they uniquely identify a connection. If they're used too soon there is a risk of confusing TCP segments in the current connection with segments from a past connection that may have sat delayed in router queue, or got misrouted and took a while to find their way to their destination. The TIME_WAIT state is configurable, and by default fairly long (forget off hand, several minutes). When a connection is closed in an orderly fashion (using the shutdown semantics above), the TIME_WAIT period is dramatically shortened (to a few seconds). If it's shutdown due to an error, the full period applies as one of the endpoints may still be trying to use it. When you shutdown the read side, if there is any data in the socket buffer a TCP RST is issued - this signals an erroneous disconnect. The reason is so the other part can detect that you didn't see some of the data sent. The full TIME_WAIT applies. When you have two hosts, or localhost, the addresses are static, plus the server port number is static, and the address, port pairs of the endpoints uniquely identify a connection, you soon have the full 65536-1024 possible anonymous ports stuck in TIME_WAIT on either the client or server or both. No more ports are available and no more connections can be made.The classic textbook is TCP/IP Illustrated by Richard Stevens. Vol 1, Chapter 18.http://www.pcvr.nl/tcpip/(http://www.pcvr.nl/tcpip/tcp_conn.htm#18_0)There are tons of tweaks, options and adjustments to TCP since then, but this describes very much how it still works...

author: westfw
author_rank: Super Contributor
timestamp: March 06, 2019, 11:23:30 am
content: This was my guess as well.Quote255 (max IP TTL) seconds...QuoteSo scan, open, shutdown, close? Or just make a fake FIN?

author: rstofer
author_rank: Super Contributor
timestamp: March 06, 2019, 04:37:00 pm
content: The FIN attack is not new and there is quite a bit of information about using iptables to block it.I don't know if this video helps:Basically, I don't see why a FIN would be performed if there wasn't an established connection. The FIN should just be dropped in the firewall.Google seems to have a lot of links...Maybe some ideas here:https://it.toolbox.com/question/how-to-block-port-scanning-tools-and-log-them-with-iptables-051115

author: ejeffrey
author_rank: Super Contributor
timestamp: March 06, 2019, 04:39:12 pm
content: You aren't at all clear about what you are actually doing or what the problem is, but my guess is that you are flooding the server with ports in the CLOSE_WAIT state, and you have a defective server application.TCP orderly shutdown requires both sides to shut down the socket. If your client is calling shutdown(), that sends a FIN packet to the server. The server application should be notified about the closed connection, and then call shutdown() itself. Otherwise the socket will sit around forever.Calling close() on the socket instead of shutdown probably sends an RST packet, which terminates the connection immediately. This can cause data loss if there is any in-flight data.If your server application is correctly calling shutdown(), then it is possible that you are running into TIME_WAIT. TCP forbids the reuse of the same (source IP, dest IP, source port, dest port) combination for an amount of time to make sure there are no stray packets (including retransmitted ones) still on the network. Whichever side calls shutdown SECOND will enter the TIME_WAIT state for a couple of minutes. It is possible to get a DoS by filling up with CLOSE_WAIT connections, but it more difficult.

author: ejeffrey
author_rank: Super Contributor
timestamp: March 06, 2019, 04:43:33 pm
content: To be clear: Calling shutdown() on a regular socket file descriptor will only send a FIN if the socket is part of an open connection. This is not the same thing as a FIN flood where you send bare FIN packets that are not part of an established connection. To do that you would need to use a "RAW" socket interface. So it doesn't make sense to only look at the shutdown call, or say "shutdown creates a FIN", you have to look at the whole TCP state machine, including the connection establishment.

author: ejeffrey
author_rank: Super Contributor
timestamp: March 06, 2019, 08:51:59 pm
content: Sending bare FIN packets with a raw socket (not promiscuous mode which is used for packet sniffing) is a port scanning and OS fingerprinting technique. It shouldn't really be causing the kind of slowdown you describe unless it is an absolute flood of packets. In any case, yes it is something that firewalls should be blocking.You should really be looking at what is actually happening on your target system to see what is actually going wrong.

author: rstofer
author_rank: Super Contributor
timestamp: March 07, 2019, 02:14:08 am
content: So, does WireShark show a flood of malformed FIN packets?One of the big problems with network switches is that it is hard to get into the middle of a transaction. I usually add a hub so I can see both sides of the conversation. Alas, I don't think they make high speed hubs. I suppose there are better ways to do this.https://osqa-ask.wireshark.org/questions/57771/display-filtering-syn-and-fin-flags

author: ejeffrey
author_rank: Super Contributor
timestamp: March 07, 2019, 02:50:46 am
content: The modern way is to get a managed switch that can be configured with a MIRROR port.

author: westfw
author_rank: Super Contributor
timestamp: March 07, 2019, 07:36:42 am
content: You're trying to make a secure sandbox net, and you have systems running Telnet Servers? And not even REAL Telnet Servers?That's sort-of in the realm of "how can I make W95 secure while still allowing direct hardware access to my ISA-Bus Device Programmer.:Sigh. I think that most modern "corporate" routers and network switches can get pretty aggressive about filtering out "inappropriate" packets. Even a home NAT-based router probably wouldn't allow rogue FINs on unestablished connections.

author: magic
author_rank: Super Contributor
timestamp: March 07, 2019, 08:11:18 am
content: Second hand hubs up to 100Mb/s can be found, I don't think faster ones exist.I don't have a high-tech managed switch so when I need to intercept Ethernet traffic, I configure a Linux box with two NICs as a bridge, pass the connection through it and run my sniffer there.As for this thread in generalYou OP have an unhealthy tendency to read about things and let your imagination run wild about what they possibly could mean. The terminology you use is all wrong and the assumptions you make about things without really finding out what they are just aren't true. I recommend more reservation and skepticism if you don't want computers to drive you insane.

author: legacy
author_rank: Super Contributor
timestamp: March 07, 2019, 08:29:12 am
content: Quote from: magic on March 07, 2019, 08:11:18 amdo we need a post like this?

author: legacy
author_rank: Super Contributor
timestamp: March 07, 2019, 09:24:04 am
content: Quote from: westfw on March 07, 2019, 07:36:42 amAre you sure?Tested two from Amazon: wrong assumption

author: scatha
author_rank: Regular Contributor
timestamp: March 07, 2019, 09:58:27 am
content: Quote from: legacy on March 07, 2019, 09:24:04 amThis is surprising, what makes/models? All enterprise routers and most commodity routers will contain some form of stateful firewall behaviour which (if enabled) will absolutely block TCP traffic which is not part of an already established connection.

author: Jeroen3
author_rank: Super Contributor
timestamp: March 07, 2019, 10:53:52 am
content: You have to pass the firewall to block FIN attacks. If you stay within the switch chipset, on your LAN, your don't pass the firewall, and you can attack everyone on the subnet.

author: rstofer
author_rank: Super Contributor
timestamp: March 07, 2019, 02:52:37 pm
content: Quote from: Jeroen3 on March 07, 2019, 10:53:52 amI thought client machines, like those running Linux, were normally using iptables in one guise or another. I'm not sure how Windows Defender will respond to malformed FIN packets but if it was a large problem it would have been handled years ago.Yes, there are global firewalls where the LAN connects to the WAN but at this point, I would expect every client machine to have some form of filtering. Except toys... I suspect that packages like lwIP don't provide much in the way of defense. But that's an entirely different deal than a mainstream OS.It pretty hard to believe that at this point in time, malformed FIN packets wouldn't just fall in the bit bucket.

author: xani
author_rank: Frequent Contributor
timestamp: March 07, 2019, 03:00:12 pm
content: Quote from: westfw on March 06, 2019, 11:23:30 amIP does not have any timeoutsIP TTL isnumber of hops. At each hop TTL is decreased by one. Yes, it is named badlythe wholepointof TCP is to handle lost IP packets in the first place so even if it was something IP related you'd just see TCP repeating dropped packet few times

author: borjam
author_rank: Unknown
timestamp: March 07, 2019, 03:42:42 pm
content: Quote from: legacy on March 05, 2019, 03:18:22 pmWhat do you mean exactly by "signal"?QuoteI don't understand the context.QuoteWhat is that "poison signal"? Someone has pointed out that you are using non standard terms. And that's indeed true. It's impossibleto help you unless we can actually understand what you mean.QuoteYou mean you are opening connections and closing them using shutdown()?shutdown() is a system call, not a signal.

author: ejeffrey
author_rank: Super Contributor
timestamp: March 07, 2019, 05:18:48 pm
content: Quote from: scatha on March 07, 2019, 09:58:27 amThe "attack" is coming form the LAN side, not going through the firewall. 100% of NAT routers block unassociated FIN packets simply because it is impossible to not block them -- the only work at all because of connection tracking, and without a connection it doesn't know where to send the FIN.I still think what is happening is that the server is not calling shutdown() and leaving a lot of sockets in the half-shutdown CLOSE_WAIT state, which is then filling up the file descriptor table or some other finite resource. But the OP has not shared any diagnostic information, reproducible example, anything other than unintelligible code fragments, or even described what specific software or hardware they are using. It is a little surprising that busybox's telnet server would have this bug, but entirely possible. Another possibility, since this is presumably some embedded device is that the kernel data resource limits are set very low either because of the limited system memory or misconfiguration. Without further information this is just speculation.

author: westfw
author_rank: Super Contributor
timestamp: March 08, 2019, 03:50:59 am
content: QuoteYou would have to set them up with the "unsecured" network on the "WAN" side, and the "known ok" network on the "LAN" side.I wouldn't expect it to do the filtering between LAN ports.Quote1) You're wrong. TTL is a timeout, and IP does have timeouts (packet reassembly timeout.) It's ALSO a hop count; each router decrements the TTL by at least one. RFC790 is sort-of seminal.2) TCP has a concept "Maximum Segment Lifetime" that's intimately related to the chosen IP TTL. It's the MSL that comes into play for "zombie" connections.3) I was wrong too. MSL is apparently considered 2*TTL, and default TTL (in RFC793) was recommended to be one minute. I'm not sure how that plays out in modern networks - transission times went down and hop counts went up pretty rapidly, and even back in the late 80s most hosts would use 255 as the default TTL, I think.4) (heh. RFC793 has some "amusing" bits in it these days... "even if data rates escalate to 10's of megabits/sec.")In any cases, this forum of EEs and microcontroller experts isn't the best place to discuss TCP network attacks of embedded unix. There are a lot of well-known attacks, and most are well documented, with standard mitigation...