FAIL2BAN - is it really problematic? - Page 1
Forum section: electronics / general

author: peter-h
author_rank: Super Contributor
timestamp: February 10, 2025, 03:39:59 pm
content: I "run" a community site (tech but not electronics) and fairly regularly we get hit with a bot which runs 5-10 queries per second.This jacks up the CPU load to 400%. The site still runs, just about... I can't find out who it is because they have not created a profile, and the report for activity of non logged in users is huge - takes a number of minutes to run, and one gets timeouts. I didn't write the site; it was done in Ruby on Rails ~ 15 years ago and the developer is not really around anymore.There is a hidden google recaptcha on the signup page which, together with other measures, fairly effectively blocksbotlogins, but of course you can't stop everything else running queries... Also there is no recaptcha on the main site. All signups are personally approved before they can post - this is necessary since some 3/4 are malicious nowadays.I suggested to the dev to set up fail2ban but he is against it, saying that it causes problems. What are these, really?I can see that if somebody's browser does a prefetch of say 100 pages (which I see in the activity reports is probably happening) that may trigger it. But the two main browsers are Chrome and Safari so this must be known.The site is behind Cloudflare which offers firewalling and I have already blocked Russia, China, India, but the last one was from Germany. It ran for days before I blocked a whole subnet of a German ISP. I also have a firewall at the hosting company; not sure what features that offers.Thereisa cost to this, if the server gets overloaded. The hosting currently costs $60/month and the next package is $120. The site is donation funded so a bigger package is just a waste of money.There is no indication of any penetration and anyway no passwords etc are being stored (just hashes).I have a good linux (the site runs on Centos 8 with NGINX) guy who can probably set up firewall rules.Note that what I am after is blocking queries, not failed auth attempts. Basically if somebody has hit the server from the same IP say 100 times in past 1 minute, they get blocked for 1hr. What concerns me is that this could be triggered accidentally.

author: Kean
author_rank: Unknown
timestamp: February 10, 2025, 04:15:09 pm
content: Fail2ban will do whatever you set the rules to do. If you get the rules that are monitoring your logs "wrong", then yes it could be falsely triggered.In my typical installs, the things being monitored are drive-by exploit scanning. For example I don't run Wordpress but the logs show repeated scans everyday for wp-admin URLs, plus random PHP scripts like phpinfo, and myphpadmin. None of those are installed on public sites, so any attempts to find them will get the IP banned for a certain time. Similarly, failed login attempts on ssh ports or similar other non-http services deserve to get blocked for a time. It may add a little admin time to unban those when a genuine user does something silly, but it is rare.You can set up rules to look for a certain number of queries from the same IP in a certain timeframe, but I've never done that myself. For me, one strike and that IP is banned. You could also initially implement rules that notify you of a rule match (typically by email) but without actually banning the IP, so effectively in a test mode.Keep in mind that it is likely that anyone intending to be malicious (rather than just adhoc scanning) will notice the bans and begin to use VPNs or maybe a bot farm to attack, and thus the IP addresses will be somewhat random and fail2ban probably wont help you that much unless you have quite long ban times.Fail2ban can definitely be useful for simple rules based IP banning, but without a lot of work it is both crude and blunt. It would probably be best to also implement rate limiting at a higher level if that is at all possible.

author: madires
author_rank: Super Contributor
timestamp: February 10, 2025, 04:16:44 pm
content: An alternative solution is rate limiting. e.g.https://blog.nginx.org/blog/rate-limiting-nginx.

author: coppice
author_rank: Super Contributor
timestamp: February 10, 2025, 04:18:34 pm
content: You've already set rules which have probably caused some harm. Blocking large chunks of the planet will do that. Fail2ban can cause more harm. What is your threshold? My experience is people are happy to block large parts of the internet, then bad mouth you for not replying to them, when they've blocked your replies.

author: peter-h
author_rank: Super Contributor
timestamp: February 10, 2025, 04:27:38 pm
content: Blocking China Russia etc is actually ok in this case because there are no communities in these countries to which the topic is applicable...But there is a huge amount of "penetration testing" from there.Blocking that German ISP was not a good move; I agree. It was an ISP knows for hosting those kinds of people.

author: madires
author_rank: Super Contributor
timestamp: February 10, 2025, 04:46:03 pm
content: If you're talking about Hetzner, they are actually quite responsive to abuse reports. Unfortunately their good service and pricing attract also bad guys.

author: peter-h
author_rank: Super Contributor
timestamp: February 10, 2025, 04:49:18 pm
content: It wasn't them. I firewalled 91.108.192.0/24. The main attack was from .240.

author: madires
author_rank: Super Contributor
timestamp: February 10, 2025, 05:04:19 pm
content: The whois database entries smell fishy, possibly a bulletproof hoster.

author: peter-h
author_rank: Super Contributor
timestamp: February 10, 2025, 05:11:52 pm
content: Yes indeed. It once showed a Russian connection. However, looking at the logs, it looks like a currently runningscrapingjob, with rotating IPs.We can't stop scraping.Problem with rotating IPs, can one still set up NGINX rate limiting?

author: JohanH
author_rank: Frequent Contributor
timestamp: February 10, 2025, 05:23:13 pm
content: I've done rate limiting with iptables and the newer nftables (but only for ssh traffic, though). Also fail2ban for that matter.Here's a good example:https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/7/html/security_guide/sec-using_nftables_to_limit_the_amount_of_connections#sec-Blocking_IP_addresses_that_attempt_more_than_tenIn this example they create a rule that automatically adds the source IP address of hosts that attempt to establish more than ten new TCP connections within one minute to the denylist. The denylist is purged after 5 minutes.This is basically also how fail2ban works, but that's one layer more on top of iptables/nftables. Always better to work closer to the kernel. Not that I've experienced issues with fail2ban, either.

author: hans
author_rank: Super Contributor
timestamp: February 10, 2025, 05:28:16 pm
content: abuseipdb.com indeed shows several similar reports. I'm guessing it's probably some VPN ip. They may use different entry/exit IPs so you cannot always simply look them up.Simple scriptkiddies probably run their stuff without delays and shuffling.Any respectable scraper will use delays in order not to get 429 errors, use multiple IPs, etc. There are even proxy farms that offer several thousands of IPv4s, with the priciest ones also offering consumer ISP variants. And if that would be too expensive, one could DIY a 2.5$ VPS with IPv6 /48 range, so thats 2.8E14 unique addresses attached to one machine.Plenty of ready baked frameworks take care of this, so even John from business IT can copy-paste write a Python script to scrape all the prices/stock information from their competitor's webstore.Typically the hurdles that stop scraping also bugs your visitors more than bots. Some CNN's are already better at solving Google Captcha's than humans are. There are plenty of libraries that bypasses CF javascript challenges. And finally people use headless browsers to render content via JS (which sites like YouTube also do, which makes the site load very slow), and then utilize LLM's to be resilient against structural changes of a layout.I would probably try one of those abuse IP lists first before upgrading a hosting plan. Sure those abuse IP lists also costs money, but what if 50% of your current random visitor traffic is also botted? LLM don't train themselves..Oh, and I wouldn't feel too bad about blocking IPs from potentially genuine visitors that come via VPNs. Those companies should maintain their reputation by stopping abuse via their networks, just like a regular ISP, but it seems like they are more concerned with sending all notifications to /dev/null to sell a "safer internet". Its all a bunch of marketing BS. If visitors access the internet through them, they should know what they have signed up for.

author: grumpydoc
author_rank: Super Contributor
timestamp: February 10, 2025, 05:47:43 pm
content: I don't run fail2ban per se but do do run a homebrew equivalent on exposed HTTP(S), SMTP, IMAP and SSH ports. Unfortunately the bad guys are relentless so you don't have a lot of choice. Yes there might be collateral damage but it becomes "the price of doing business" on the Internet.Without knowing your dev's specific objections it is hard to comment or offer mitigations.

author: Algoma
author_rank: Frequent Contributor
timestamp: February 10, 2025, 05:56:49 pm
content: These bots are now likely AI scrapers working to collect raw info to feed into their training models.Might need to toss up a basic welcome page to First time visitors from Unknown IP addresses. Rather than Blacklist, build up a whitelist of known valid visitors.

author: peter-h
author_rank: Super Contributor
timestamp: February 10, 2025, 09:12:51 pm
content: One hit per second is fine. When it gets to say 5/sec then it slows down the server a lot.But if somebody rotates through say 100 IPs, no one IP will show much traffic, so the detection will have to work on a subnet. Can this be done?

author: Algoma
author_rank: Frequent Contributor
timestamp: February 10, 2025, 09:36:54 pm
content: Perhaps may be useful info:https://blog.nginx.org/blog/rate-limiting-nginx(I note this was already posted and discussed above.)was reading an article about feeding the scraper bots some zip bombs to crash their cache memory. Invisible pixel sized links to "vastly empty" downloads on the page that only the bots can see. They're tiny zip files of highly compressed repeating data patterns, that when the bots look inside it vastly expands to fill their memory with empty fluff.

author: bitwelder
author_rank: Super Contributor
timestamp: February 11, 2025, 11:43:30 am
content: With "running a community site" do you mean basically just web access, i.e. you are interested only allowing access in tcp/443 ?I guess the option of rate limiting should be good, optionally I think Cloudflare could offer some protection from DDOSing.I don't see a problem in using fail2ban (if well configured), but you might need to extend the ban times. On one server I implemented it with multiple ban levels, so first time the ban is 1 day, but 2nd time it will be 1 month.Something else you could do is to pass through a proxy that will do all the ingress filtering, so it will take all the 'slowness' from the real site.Last, if you are against scrapers, would it be possible perhaps to put the whole site behind login, i.e. require to login first not only to post but also to read contents?

author: peter-h
author_rank: Super Contributor
timestamp: February 11, 2025, 12:37:42 pm
content: QuoteYes.It is a community site so Cloudflare is the free version. CF offers a lot of stuff; maybe I need to take a closer look.QuoteCF is caching all static content but the forum pages are tagged to not be cached, for obvious reasons.QuoteScraping is unavoidable unless as you say you put the site behind a login, but then you get no SEO (OK; one can let google etc in, as e.g. paid newspapers do, but then the links don't take you anywhere useful). It needs to be an open site. I just want to block some machine which is doing a lot of hits for many days constantly. The activity is totally obvious to a human so it ought to be possible to come up with some rules for it.However, the guy who was looking at this reported some serious performance bottlenecks e.g.SELECT topic_posts.* FROM topic_posts WHERE topic_posts.system_id = 1 AND (topic_id in (10,12,1,2,9,3,4,5,8,11,6)) AND topic_posts.is_visible = 1 ORDER BY created_at desc LIMIT 10 OFFSET 361580;just gets 10 posts but runs for 5 seconds. The database is MariaDB. OFFSET 361580 looks like the scraper was downloading old posts.

author: DiTBho
author_rank: Super Contributor
timestamp: February 11, 2025, 12:46:51 pm
content: I think that nowadays opening and managing a Forum or a Site with users - I mean different from a blog that is -1- personal, and -2- read-only from the outside, is a total pain in the ass for all the shit that goes around on the net, starting with bots that attack a site ...You might as well open a group on Facebook, and let them take care of the "server" matters

author: sleemanj
author_rank: Super Contributor
timestamp: February 11, 2025, 12:48:54 pm
content: I use fail2ban on all my ec2 and lightsail servers running multiple sites on each. Configured correctly it does a fine job.I use it primarily in conjunction with mod_security so that when mod_sec gets tripped for critical attempts the ip is blocked for a period, and for trivial dos protection (mostly bad behaved robots) such that excessive requests that a normal human won;t do gets them a block. Repeat offenders get longer blocks. Since none of my sites use wordpress any attempt to hit any of the usual wordpress files get insta-banned as well, any attempt to hit more than a couple non-existent php files gets a ban, too many 404s gets a ban.... And a healthy list of junk-seo-link-checking-bot user agents are insta banned.Trivial exceptions are made for google and bing and a few others simply by useragent, the bots never bother getting around it.I'd never have a site up without it personally, same for mod_security, yes there is the odd false positive, but the amount of actual undesirables they catch and ban far outweigh it.

author: hans
author_rank: Super Contributor
timestamp: February 11, 2025, 12:58:07 pm
content: Quote from: bitwelder on February 11, 2025, 11:43:30 amThe question for me really is: is it a targeted scraper or just a general purpose one that tries to crawl all the links on the internet (much like Google also does).Most scraping prevention hit users more than bots. Especially if its targeted.. bots can spawn many more threads, more proxies, more sophisticated captcha solvers, they can wait for 15s to load a page, they don't care. Eventually they will run through all the links they wanted to fetch and start over again.A more subtle approach is to add honeypots to the website. Things like mentioned before with the bots checking for WordPress login URLs. But that is scriptkiddies looking for breaching common IT/website infrastructure.You can also add fake links that only bad bots will follow (e.g. create a <a> tag that is hidden or placed outside the page boundaries by some obfuscated manner). Once a visitor hits that URL, you'll also know fairly certain its a bot and ban that IP for some time.This approach is also employed to see if a particular data set is grabbed from a particular website. E.g public phone books sites are often filled with fake names/numbers. If someone scrapes all names/phone numbers to build a "reverse phonebook" site (which is normally not offered because of privacy reasons), then if those fake numbers give a hit, you'll know you can send them a legal letter.

author: peter-h
author_rank: Super Contributor
timestamp: February 11, 2025, 02:42:57 pm
content: Gosh DiTBho you forgot that I was on your blacklist! What has happened?QuoteI don't know. I do know that there are people out there who would like to trash this site. I will spare you jokes about forumsbut over years you do get some wars being fought, and sometimes people needs to be booted out. But CF protects from the standard DDOS attack. And some big social media sites (which I know some non public data about) pay loads of money to CF (5 digits per month) for this protection - because they actually need it. I don't think my site has enemies with those resources; I would know by now if it did.What is significant is that it ran this last time for just a few hours, fetching probably a small section of the forum (which has ~400k posts).So it looks like a trial run for something bigger.The site is completely custom, in Ruby on Rails, and very few other installations exist. It is not PHP-BB or Vbulletin or Discourse which are constantly getting hit, and usually penetrated. So someone would need to develop custom code to scrape it.QuoteThat is very clever. But google nowadays spots white text on white background (an old hack to get SEO). Don't all spiders/crawlers do the same?I would need to mod the site code to add a URL in white text to every post. That is probably doable, even if the original dev is not around to do it. And when that URL gets hit, the IP is firewalled for a day, or more.However, despite google's dominance there are other search engines. I've noticed one running on a Hetzner server in Germany.QuoteCould you suggest some fail2ban rules, which work with rotating IPs?

author: ejeffrey
author_rank: Super Contributor
timestamp: February 11, 2025, 04:31:29 pm
content: I don't care for fail2ban for a few reasons. It can be used successfully but I think it's generally more trouble than it's worth. That said, my normal criticism comes when people use it "for security" rather than just to throttle load as you are suggesting.My main problem with fail2ban is that it's a good way to DoS yourself and your legitimate users, while not effectively preventing anything but the most simplistic malware. Malicious actors and botnets are able to balance load across many source addresses to operate under whatever thresholds you set. And the more aggressive you make it the more likely you are to block legitimate users or even worse, administrators.In general, fail2ban is allowing untrusted clients -- actually the clients you have identified as mostly likely malicious -- to edit your firewall rules. That's a risky proposition.Fail2ban's main benefit is that it's easy to set up and protocol agnostic. But it's much better to approach this at the web server level. A common approach is to check if a request has a valid login cookie, and if so process it normally. If not, and the server is overloaded, serve them with a static cached version of the content. This can usually be done at your front end web server with minimal to no changes to the application. It works best If your application already supports providing a decontented, anonymous, static version of pages to use in your cache, but that can be worked around. If even maintaining and using the cache is too much, just serve an error page saying "high load, registered users only." This is also what services like cloudflare can do for you but in that case the front end is on their CDN so the traffic never even crosses your network. But you can do it yourself if you prefer.

author: peter-h
author_rank: Super Contributor
timestamp: February 11, 2025, 05:21:51 pm
content: What concerns me in particular with the "invisible link -> fail2ban" approach is that browsers will see that invisible link, and will hit it. And will get stuck. Browsers do a lot of prefetch nowadays e.g. downloading loads of links on a page.

author: Nominal Animal
author_rank: Super Contributor
timestamp: February 11, 2025, 09:23:32 pm
content: I use fail2ban to block scripted bot attacks, knocking on unexpected ports, trying dictionary attacks on SSH (which has become so annoying that I occasionally run SSH on a nonstandard port), and with web servers, blocking IP addresses that attempt known security issues including invalid URLs.It is essentially an annoyance filter, not a true security measure, and it is not that well suited fortraffic shaping.The underlying problem here is scraping by non-logged-in users.The best solution for OP, in my opinion, is to use Nginx rate-limiting, separating the configuration between logged-in and guest users; and heavily rate-limit not-logged-in user access (generally, not just per client IP address). Then, add a note somewhere visible to non-logged in users that because of various scrapers, guest access is slowed down but logged-in user access won't be impacted.The exact way to do this depends on the exact implementation, because Nginx configuration needs a way to differentiate between logged-in and guest users to implement the rate-limiting. Note thatngx_http_internal_redirect_modulecan do redirection internally, so that the URI change is not visible to actual users (or anybody outside the server).If you use cookie authentication (YourAuthCookie), one possibility is to map its existence to a variable thus:map$cookie_YourAuthCookie$isguest {default 1;~.+ 0;}limit_req_zoneguests zone=Zone:Sizerate=1r/s;and then in the proxy section,if($isguest) {limit_reqzone=Zoneburst=2;}This is very fragile, though, becauseanyYourAuthCookievalue will count as a logged-in user.If your host has an Nginx module (likehttp_set_misc) that supports hashing (EDIT: and setting the current date to a variable!), then you can do a secure version of the above, using some fixed secret (that is only used for this tracking cookie):set$secret "foobar";First, you construct the expected cookie value ($client_hash), a variable ($yyyy_mm_dd) indicating current date, and then set$isguestif the request cookie value matches the expected value:set_local_today $yyyy_mm_dd;set_sha1 $client_hash "$yyyy_mm_dd$binary_remote_addr$secret";map$cookie_CookieName$isguest {$client_hash 0;default 1;}Note thatmapcauses a mapping from the value to the variable, and is defined in the server context; it is only evaluated when the variable is used in that context.We also check the upstream (Ruby application server) response Set-Cookie header, and if at least 16 characters long, set$isknownnonzero:map$upstream_cookie_YourAuthCookie$isknown {~.{16} 1; // Cookie set to a value with at least 16 charactersdefault 0;}The guest request zone is still the same:limit_req_zoneguests zone=Zone:Sizerate=1r/s;In the proxy section, we use$isguestto rate-limit requests like above, noting that it will also hit the first logged-in request per session per IP per day:if($isguest) {limit_reqzone=Zoneburst=2;if($isknown) {add_headerSet-Cookie "CookieName=$client_hash; Domain=YourDomain; Secure; HttpOnly";}}Essentially, whenever a request comes in with an incorrect, unset, or emptyCookieNamecookie, the request will be treated as a guest request and rate-limited. As the response is output, if there is aSet-CookieforYourAuthCookiethat is at least 16 characters long (so it won't hit on logout!) on such a request, the properCookieNamecookie is set. If someone is logged in, then logs out, the avoid-rate-limiting cookie stays effective till they close the session. As theCookieNamecookie is tied to the IP and the date, only users who have logged in that day will get the correct value set. (This is why I named the variable$isknown, rather than$islogin.)

author: grumpydoc
author_rank: Super Contributor
timestamp: February 11, 2025, 10:09:43 pm
content: Quote from: Nominal Animal on February 11, 2025, 09:23:32 pmRunning SSH on non default ports is a good way of reducing the number of attacks and I hope that you have password authentication turned off on any server which opens any SSH port to the 'net.But, as you say, sometimes it's really obvious who the bad guys are - on my home network I'm pretty much the only person who will be logging in so any host throwing login attempts at any other user is a bad guy.My experience with ssh is that they keep trying, so I use connection throttling and a "fail2ban" like script to drop any further connection. I get less chatter in my logs that way.For SMTP I get another form of "attack" - throwing email at random user names within my domain. For this one fail2ban does not really work - the system doesn't error, it accepts the mail, runs it through spam and virus filters and then ... tries to return it to the sender, completely gumming up email.Years ago I wrote sendmail rules to check incoming mail and bounce it at the SMTP "RCPT To:" stage if it is never going to be deliverable locally - massively reduced CPU usage and length of queued bounce emails which were going nowhere because the "From:" address was invalid (I suspect generating this backscatter was largely the reason for the email in the first place).But there's no point feeding this into fail2ban because the current pattern is that a host turns up, throws 100 invalid email addresses at my server only to be told "no thanks" for each - then disappears for, as far as I can tell, ever. In this case it's much better to harden the email server rather than use fail2ban.

author: sleemanj
author_rank: Super Contributor
timestamp: February 11, 2025, 10:12:55 pm
content: Quote from: peter-h on February 11, 2025, 02:42:57 pmI doubt my rules would be that useful generally, they are made for my specific circumstances, the way my sites work, the type of users that I have and their typical browsing patterns, but with that said....My systems all present pages to the browser as .html or trailing-slash directory requests, so I simply detect all requests for .html and trailing slash directories, excluding some known bot UAs , 302 & 301 redirects, and some special locations that I don't want to count.Normal people on my sites do not hit full pages in rapid succession, they certainly hit resources of those pages (css, images, js) rapidly so I don't look for those.Code:# Filter: gogo-apache-dos.conf# Note: Match all requests for .html, / with or without query stringfailregex = ^<HOST> -.+(HEAD|GET|POST|PUT|DELETE) [^"]*((\.html(\?[^"]*)?)|/) HTTPignoreregex = (REDACTED)|msnbot|Googlebot|bingbot|googlebot|AdsBot-Google-Mobile|facebookexternalhit|(HTTP/1.1" 302)|(HTTP/1.1" 301)I also have a filter which looks at the fail2ban log to find reoffendersCode:# Filter: gogo-reoffender-fast.conffailregex = NOTICE\s*\[g-apache-dos-fast\] Unban <HOST>$ignoreregex = NOTICE \[g-reoffender.*\].*$In my jail, I have the following configurations, for thatCode:# People who request 8 html files in the space of 10 seconds are blocked 5m[g-apache-dos]enabled = trueport = http,httpsfilter = gogo-apache-dosname = g-apache-doslogpath = /home/*/logs/access-*.logmaxretry = 8findtime = 10bantime = 300action = iptables-multiport-log[name="g-apache-dos", port="http,https"]# People who request 3 html files in the space of 1 second are blocked 30s[g-apache-dos-fast]enabled = trueport = http,httpsfilter = gogo-apache-dosname = g-apache-dos-fastlogpath = /home/*/logs/access-*.logmaxretry = 3findtime = 1bantime = 30action = iptables-multiport-log[name="g-apache-dos-fast", port="http,https"]# People who do a "fast" DOS 3 times in 30 minutes get blocked for an hour[g-reoffender-fast]enabled = trueport = allfilter = gogo-reoffender-fastname = g-reoffenderlogpath = /var/log/fail2ban.logmaxretry = 3findtime = 1800bantime = 3600action = iptables-multiport-log[name="g-reoffender-fast", port="http,https"]

author: peter-h
author_rank: Super Contributor
timestamp: February 11, 2025, 11:15:18 pm
content: QuoteI do see lots of cases of high speed, say 10Hz, fetching of multiple pages from real users, which I assume is browser prefetch.

author: sleemanj
author_rank: Super Contributor
timestamp: February 12, 2025, 12:10:03 am
content: Conversely, I never see that on my sites. When I made my rules I was careful to analyse and examine logs for a period of time to determine appropriate limits.Every site is different, every user base is different, every tolerance for false positives is different.In the last 2 days I can only see one possible DOS false positive, the rest is all bots from EC2, digital ocean, googleusercontent (apps running on google), various china based, a couple through tor, and a handful of others.

author: Nominal Animal
author_rank: Super Contributor
timestamp: February 12, 2025, 04:23:39 am
content: Quote from: grumpydoc on February 11, 2025, 10:09:43 pmWhenever I know the clients will use a protected key storage, likessh-agentwith passphrase-protected key files, I definitely prefer public key authentication. Otherwise, it is a balance between the risk of keylogging (and leaking a password), or some process swiping the unprotected key files.Notably, the browsers I currently use, do not let any JavaScript do either of those.For allowing trusted friends temporary access to my machine, I set up a new account, and a randomly-generated but easily spoken password (for conveying via telephone or similar).When I have to use passwords, and for the passphrases protecting my keys, I default to cryptographically secure random number generator generated ones with 96–144 bits, 16 to 24 characters (openssl randor/dev/randomsource). To store these on paper, I use large cryptos (on 5mm gridded A4 paper), with no password in a straight line in it. Elsewhere, I have "artwork" which describes the shape the various passwords have. Overkill, because keylogging is how they'll leak if they leak, but I like it. (Or by someone drugging me and hitting me with a $5 wrench till I reveal the password, per XKCD #538.)Fail2Ban is a bit too heavy to use on OpenWRT routers, so I've been thinking about writing a lightweight userspace service to maintain a ban list based on port scanning. The idea is to reduce garbage traffic at the internet-facing edge. There are two basic approaches: netfilter logging, and monitoring honeypot sockets for incoming connection attempts. The former is more versatile, but the latter is simpler (if one is only interested in TCP and possibly UDP honeypot sockets).Quote from: grumpydoc on February 11, 2025, 10:09:43 pmYup. Unless it is an authenticated user, bounces (return to sender) should be disabled. Filtering on RCPT TO makes a lot of sense. I don't think fail2ban can help in this.I don't currently run my own email server; I use a hosted one, and lots of spam filters on my own mail program.

author: hans
author_rank: Super Contributor
timestamp: February 12, 2025, 08:34:20 am
content: Quote from: peter-h on February 11, 2025, 11:15:18 pmNever heard about it.You can prefetch DNS queries or linked files though.. like:Code:<link rel="dns-prefetch" href="https://www.mozilla.org" />Code:<link rel="prefetch" href="main.js" />If you then have some JS at end of page or something that injects code, the browser has already fetched these endpoints/domains.I don't think browsers will go out and request GET on new HTML bodies out of the blue. In theory REST model recommends GET requsts to be read-only, but in reality they not always are.You could look at the Sec headers to see how a browser has ended up a certain point. E.g. Sec-Fetch-Mode:Code:Sec-Fetch-Mode: corsSec-Fetch-Mode: navigateSec-Fetch-Mode: no-corsSec-Fetch-Mode: same-originSec-Fetch-Mode: websocketYou would mostly expect 'navigate' from directly browsing HTML pages, same-origin for resources loaded on that page, and cors/no-cors/websocket for AJAX or websocket requests from JS.Very basic Python scraping bots don't send these headers. Heck they don't even send the headers in the right order. Browsers like Chrome, Firefox, etc. have fairly static HTTP and TLS libraries, so services like Cloudflare use these hints for fingerprinting against bots.

author: peter-h
author_rank: Super Contributor
timestamp: February 12, 2025, 09:23:00 am
content: FWIW I see constant activity by Sentibot, at around 15 secs between hits. This is not a significant load. 95.216.173.102. I googled it. What a strange machine...A colleague has also generated extra indexes which have speeded up access quite a lot, which will also help.

author: Algoma
author_rank: Frequent Contributor
timestamp: February 12, 2025, 03:32:45 pm
content: You mentioned the site Being behind Cloudflare, They'll have their own external anonymous appearing bots monitoring, checking and caching your page on a regular interval to verify service status. Block Cloudflare's bots and your visitors will get Cloudflare status errors.https://developers.cloudflare.com/bots/concepts/bot/You had the Free plan: There is a basic mode to fight bots.https://developers.cloudflare.com/bots/get-started/free/Perhaps a temporary upgrade to Cloudflare Pro to figure out what is really happening using their tools, the back to free once you've got an idea what it really is.

author: peter-h
author_rank: Super Contributor
timestamp: February 12, 2025, 04:03:38 pm
content: Yes.We are using the hidden google recaptcha on the signup page, and AFAICT this is working 100% to blockbotsignups. Unfortunately manufacturing humans is a well developed activityI also check against a list of about 2k disposable email domains, which helps, but with gmail... There is also a Cloudflare browser integrity check on the signup page, so somehow together these work.FWIW, since most signups are malicious, a few years ago I got a Ruby programmer to code a graphical captcha for the signup, which presented some numbers to solve, as a graphic image. Then a friend (who works for a penetration test company) hacked it in minutes, using standard unix OCR toolsSo I never deployed this, especially as I manually approve all signups anyway.I tested Cloudflare's Under Attack mode yesterday and it puts up a big visible captcha (just a tick box, not the utterly mad "spot all the buses" one) and after that it works fine, and re-asks after some time (hours?).I already have Bot Fight Mode set - been like that for ages. I have now blocked AI bots.

author: Algoma
author_rank: Frequent Contributor
timestamp: February 12, 2025, 04:56:57 pm
content: China has been so heavy into AI development, with a few useful models released already (Deepseek, QWEN etc..) but they're so aggressively targeting and seeking to scrape and agglomerate any and all technical knowledge sources they can find. While I can see the value, having worked with such systems myself. but they really need to calm their bots down from being so aggressive with the collection of public data from small sites, constantly scraping entire forums just to pickup a tiny fragments of new material to feed their bots.I can only imagine what traffic statistics changes have occurred with EEVBLOG Forum since the release of ChatGPT, when this AI push all started.

author: peter-h
author_rank: Super Contributor
timestamp: February 12, 2025, 05:03:06 pm
content: For sure forums have filled up with people (humans) posting bland questions to generate LLM food. You see it heavily on reddit, facebook, probably stackexchange etc. On the one I admin, those posters don't get in to start with, plus I IP-block china, russia...

author: madires
author_rank: Super Contributor
timestamp: February 13, 2025, 11:48:47 am
content: In my experience blocking specific countries mostly helps just against script kiddies, spammers and other rookies, while also causing collateral damage. Doesn't work for more sophisticated groups using proxy services like the Mirai successors. Specifically from China and Russia I see a lot of ssh dictionary attacks, occasional spam attempts, and website scanning for known vulnerabilities. The prefixes I block completely are usually the most active bulletproof hosters and a few aggressive security companies (scanning).

author: DiTBho
author_rank: Super Contributor
timestamp: February 13, 2025, 12:19:49 pm
content: Quote from: Nominal Animal on February 12, 2025, 04:23:39 amDepends on the router it's running on, but here's what I did: Userspace collects bad-ips in a list, sends them to IP tables.Better idea: Userspace collects a list of bad-ips that somehow get past the firewall, sends them to the "firewall", dedicated machine.and anything that knocks on the default SSH port is blocked!

author: DiTBho
author_rank: Super Contributor
timestamp: February 13, 2025, 12:25:22 pm
content: S.N.O.R.T.? Anyone?

author: Nominal Animal
author_rank: Super Contributor
timestamp: February 13, 2025, 07:17:25 pm
content: Quote from: DiTBho on February 13, 2025, 12:19:49 pmYep, that's the point. The trick is how do you collect the bad IP addresses in the first place. Fail2Ban looks at log files. Netfilter logging reads any addresses logged from the firewall rules. Userspace honeypot listens to incoming connections, and immediately closes any connections.Of these, I think I like the netfilter logging best, because then the userspace component can use an external command (shell script) to update the set of blocked IP addresses, allowing for maximum flexibility - including when the actual firewall is a separate machine. It's not like that command will be executed very often, so the small overhead does not matter.For recording the state of suspected IP addresses, I'll use a compact in-memory storage. I don't mind that the blocked addresses are lost at reboot: it's not like this is foolproof in any way, just to reduce unwanted inbound traffic.Quote from: DiTBho on February 13, 2025, 12:25:22 pmI don't useSnort, simply because I do not trust Ciscoat all. In this, I am definitely paranoid; don't be like me.Thesource codeis at GitHub for everyone to check out and use, that's not the issue even for me. The issue for me is theSnort rules–– the very point of using Snort in the first place! –– and how Cisco has last-word control over these. I just cannot bring myself to trust Cisco on this. The fact that even the Community Ruleset comes with the Cisco Ruleset License/Agreement (that however in 2.3 states that are not bound by that Agreement) gives me the heebie-jeebies. I do not want to have any legally binding contract/agreement with Cisco.Snort applies regular expressions over raw packet contents. Rules are source-target address-port and content regular expression rules, that when matching, causes an alert. Only packets matching the source-target address-port part are checked for contents. It is very reasonable and useful approach forsuspicious packet inspection; it's just that what rules will you use –– effectively, subscribe to, because you do want to keep them up to date –– and trust? If you do not mind trusting Cisco, Snort is a very good addition to use.

author: grumpydoc
author_rank: Super Contributor
timestamp: February 16, 2025, 01:10:02 pm
content: Bots and scrapers and how to defend against them being discussed on LWWN herehttps://lwn.net/Articles/1008897

author: peter-h
author_rank: Super Contributor
timestamp: February 17, 2025, 11:36:40 am
content: That URL is behind a paywall.

author: kjpye
author_rank: Contributor
timestamp: February 17, 2025, 11:47:48 am
content: It was posted on Feb 14, and will be generally available after a week. so try again on Friday or Saturday.