skysight-inc/HackerNews huggingface dataset:
[conference room, some years ago, maybe]:

technician: "But if we authenticate using the facebook DNS, how will we do our jobs if it goes down?"

boss: "The only way our DNS servers are going down is if facebook closes, in which case you're out of a job anyway."
Whitemoney’s mother says as she thanks Biggie, Ebuka, fans and house mates
The word "attack" has multiple definitions, including "act harmfully on" e.g., a heart attack. It does not require intent or aggression.
Not everyone is subject to compromise via news feed. One of the interesting things about the Cambridge Analytica story was the idea that they got good at finding people vulnerable to radicalization via news feed.

As for the reason for the hate, well, we have to deal with the radicalized people here in meatspace. It's a negative externality that we are subject to even if we aren't the target of the advertising.
"People look for alternatives and want to know more or discuss what’s going on. When Facebook became unreachable, we started seeing increased DNS queries to Twitter, Signal and other messaging and social media platforms." from Cloudflare via https://news.ycombinator.com/item?id=28752131
The cultural gap is because America's political leadership is mostly lawyers and China's is mostly scientists and engineers. So people hailing from the latter background tend to think along more utilitarian and technical lines.
Exactly! Obviously they have extremely robust testing and error catching on things like code deploys: how many times do you think they deploy new code a day? And at least personally, their error rate is somewhere below 1%.

Clearly something about their networking infrastructure is not as robust.
I'd love to hear more, what kinds of incentives? Is it something like, "if you get X efficiency, you get N more dollars"?
So now to "help protect private app data" they've gone from a terrible backup experience (running adb backup) to essentially no backup experience.

Unless of course you want to pay Google money and even then arguably get a subpar backup experience.

Keep in mind that you have to unlock the device AND authorize USB debugging just to get adb backup to run.
What do you lose? You get iOS apps, much better battery life, fanless, a Unix os with the ability to install Linux. What is the loss you refer to?
Ya, but it's still useful to know what you're getting between an API based on functions, macros or data. So I'd say having that in the description of an API is useful.
At the time, OP likely pulled cached DNS records to interrogate the associated IPs directly for their application (HTTP) level resources.
A Tale of DNS and BGP: The Facebook Outage, October 2021
Users flock to social media app Jeti as Facebook, Instagram and WhatsApp crash
I mean, when I last worked in a NOC, we used to call ourselves "NOC monkeys", so yeah. IF you're in the NOC, you're a NOC monkey, if you're on the floor, you're a floor monkey. And so on.
What do you lose? You get iOS apps, much better battery life, a cool and fanless computer, a Unix os with the ability to install Linux. What is the loss you refer to?
https://www.status.fb.com/ is back online now
Sure, if you're...

- not arrogant
- or complacent
- haven't inadvertently acquired the company
- know your tech peers well enough to have confidence in their identity during an emergency
- do regular drills to simulate everything going wrong at once

Lots of us know what should be happening right now, but think back to the many situations we've all experienced where fallback systems turned into a nightmarish war story, then scale it up by 1000. This is a historic day, I think it's quite likely that the scale of the outage will lead to the breakup of the company because it's the Big One that people have been warning about for years.
So Facebook facilitates the spread of engaging content which happens to include disinformation because due to our imperfect nature people like to engage with disinformation.
Seems like wishing that Facebook wouldn't exist because of this is analogous to wishing that airfare wouldn't exist because it facilitates global spread of infections.

Also, the word "divisive" gives me shivers. It's better to be divided than to be united in a fallacy. And in many cases getting divided is our only hope of ever arriving at the truth.
https://i.imgur.com/O4yEget.png
I think it's a bit weird that you made the effort to switch the countries from the original title. Huge difference.
I use Gchat at work as essentially an adjunct to Gmail. I don't even have any of the other apps installed. I do use Facebook Messenger for one good friend who lives in Europe.

But, in general TBH, I don't use texting all that much day to day.
"Sorry, something went wrong. Facebook © 2020"
Even before E2E - to my knowledge, whatsapp would only store messages until they could be delivered. They never really stored your chats once they made it to their destination - there shouldn't be any "restoring" of backups that brings back messages unless it's just a re-delivery at most. (And honestly, i'd doubt that gets backed up).
yeah I thought I was the only one, never really noticed this on HN before...
You can’t really make a system that will unboil an egg.
They can't even take off their headsets?
Except some of the readers were made in house. I’ve never been into a Facebook conference room that didn’t use one of their custom touch screen/ card readers.
I've launched Wireshark monitoring DNS traffic of roughly 5 phones. I've collected 19.8k DNS packets so far. Out of that, 5.1k packets are flagged with REFUSED or SERVFAIL. If I am not mistaken, it means that 51% of DNS requests fail.

Looking at queries for graph.instagram.com, it looks like there are roughly 20 attempts in a sequence before it gives up.

All in all, this could probably explain doubling of the DNS traffic. But the sample is rather small, so take it with a grain of salt.
I’m not sure it is. If this was Twitter I’m sure the reaction would be similar. Besides those two, all other social media companies are either much smaller or in a weird category where people consider them separate (I.e Reddit)
Im glad you like it! Yes, you can. Just create an account, convert your articles and embed our player into your website. We also integrate with WordPress and Ghost.

If you have any specific questions, I'm happy to answer.
Tool to estimate cost of a widespread internet shutdown or blackout
The issue is, the difficulty of doing an randomized study doesn’t nullify the benefits of randomization (a big one being eliminating confounding variables, for which there can be a lot for who decides to take up an addiction and why).

The control part would be particularly difficult (unless maybe you gave everyone a real or sham nicotine patch? But then you can’t exclude some other benefit from straight nicotine itself).

One could control for all the confounders they want, but you still risk missing some that are unknown or undervalued.
> I remember my first time having a meeting at Facebook and observing none of the doors had keyholes and thinking "hope their badge system never goes down"

Every internet-connected physical system needs to have a sensible offline fallback mode. They should have had physical keys, or at least some kind of offline RFID validation (e.g. continue to validate the last N badges that had previously successfully validated).
Have you tried rewriting in Rust?
The 170,000 doesn't have a precise unit, it could be losses in market cap, cash reserves, advertiser revenue, operating costs, literally anything.

I'm merely putting the number in perspective. 6 digits isn't really something that's particularly concerning on any Facebook spreadsheet.
Even if they wanted to write one they have no way to host it :D
I don't think I am exagerating on saying that the best thing that happened to humanity in the 21st century is Facebook's very naive way of doing things: it showed to some what is possible to do with technology. Unfortunatelly, for the vast majority of the "tech savy" wannabe crowd, the lesson they learned stopped at "Facebook is bad, mkay?" and since this is all they know, this is all they talk about.
Is it possible for facebook to instead rely on an anycast IP rather than DNS for their (non-web) phone apps?
I'm French and SMS are ubiquitous here; sometimes even more so than email. All operators offer unlimited free SMS, even the cheapest 2€/month subscriptions. I'm not sure I personally know anyone who uses WhatsApp.

SMS used to be priced like you describe until a new operator (aptly named "Free") entered the market in 2012 with free SMS plans and forced the other three operators to align.
Farmers Attacked by BJP Members in Uttar Pradesh, 9 Dead
This is a domain brokering service. It is not listed for sale. They are saying that for $59 plus commission, they will call up Facebook on your behalf and try to convince them to sell Facebook dot com for a "fair" price.

It doesn't take much of a logical leap to assume that there is effectively no price in this world that Facebook will sell their domain to you for.

So the broker will come back to you and say, "sorry we couldn't convince them to sell it" and you are out $59.

You could technically do the same with

Apple: https://uniregistry.com/buy-domains/apple.com?src=domaintool...

Microsoft: https://uniregistry.com/buy-domains/microsoft.com?src=domain...

For that manner, send me a DM and I will attempt to broker the deal for just $49 and zero commission.

This is a non-story.
Almost half of US farm labourers are illegal immigrants;

https://www.ers.usda.gov/topics/farm-economy/farm-labor

https://www.ers.usda.gov/webdocs/charts/63465/LegalStatus201...

Pretty crazy how they're treated considering that.

Cool job tho, thanks for feeding people!
Do you ask for consent before all this handling and cleaning? You know, consent from the grain.

Ok but seriously, the best I can imagine might be a generic environmental impact concern that could be levelled far and wide. Honestly seems pretty legit. Are there any ethical issues we should know about in the field of agricultural grain handling?
Unfortunately, I'm not given the option about whether I interact with Facebook.

Their aggressive insinuation into the fabric of most websites ensures that there are very few places I can go without Mark Zuckerberg personally stalking me, along with a legion of inhuman advertisers.

Facebook and its sites can't stay down long enough; if they were offline forever, it wouldn't be enough. Even if this doesn't affect their advertising services, any hit to the FB ecosystem is a societal good.
Do your control systems enable producers of GMO grain to handle and clean their grain with greater efficiency? Then you’re contributing to the GMO problem :)

(I have nothing against GMO personally)
True

Although, to be fair, that is kind of like praising the arsonist after he put out the fire he started (which had already smoke-damaged the whole neighborhood).
Scientists officially list 23 species as extinct, including largest woodpecker
If your device is fully encrypted, nobody can run an adb backup without authorization. This change does not protect the user; it "protects" apps.
Possibly in Norway too (internet though, not phone calls) https://www.nrk.no/nyheter/internett-trobbel-hos-telia-1.156...
i don't use it. still wish it didn't exist so my parents don't upload all my pictures onto there and their personal information, then when they hear about leaks or get their FB hijacked I'm the one dealing with it. This is before even considering all the other messed up stuff. Society is better off without it.
I assume this is a play on hyperlink. Hyper the terminal is not the only software named that.
Partly due to the news and partly the outage. There will be more outage related news coming today I imagine. It will probably bounce back over a day or two. (Just an intuition, these things are hard to predict.)
The ones in Australia where one way - you could search for a name and it would give you a phone and address... but if you a programmer, you could read the database directly, so search on a phone number and get back the address and name, or search an address and get back name and phone number.

There were a few services in the early 2000s doing this, they were called Reverse Grey Pages
Open/R: Open routing for modern networks (Facebook) (2017)
This is a great write-up, but one thing I don't understand is why the effect of withdrawing the BGP prefixes was instantaneous (if I understand that correctly), but it's taking hours (so far) to re-announce the prefixes. Why would it take so long to flip the switch back the other way?
maybe they're open by default, like old 7-11 stores when they went 24hrs and had no locks on the doors :)
Top Techniques for Implementing Fault Tolerance in Distributed Systems
Isn’t there an easy fix to just add a bullshit record for FB to DNS until this blows over?

I am also thinking that all the poorly coded sites that don’t work unless the Share on Facebook button loads are also going to hemorrhage money. So are all the e-commerce sites that rely on Login with FB.

I hope this results in everyone rethinking adding all that shit to their infrastructure before the next time.
It is indeed an old phenomenon: https://www.commonlit.org/texts/the-fallacy-of-success
So your argument here is that it is ridiculous to suggest that an RCT is possible in this circumstance, so we should therefore accept observational studies which derive correlations as truth.

Not only does that conclusion not follow from your premise (a logical fallacy), but it's an absurd suggestion as a process for deriving truth. Observational studies, at best, offer an insight into possible hypotheses, and should by no means be considered persuasive unless all interacting systems have been controlled for. In the case of ethanol, there are an enormous amount of interacting systems that need to be controlled for.

Your premise is also strictly false, as it's clear that not only is an RCT for ethanol consumption possible, but it was planned and partially in progress before it was terminated.

It's truly incredible to see the kind of language that you use here, how confident you are about your fallacious argument. I kindly request that you turn down your confident language. Science is a long and careful process of pushing back the fog of ignorance, and if you are serious about the search for truth you shouldn't use such confident language.
https://www.xda-developers.com/lineageos-seedvault-open-sour...

>LineageOS adopts SeedVault as its open source backup solution
[...]
>For those not familiar with SeedVault, it is an open-source backup app that uses the same internal APIs as adb backup.

I hope the SeedVault people are able to circumvent that. And I hope they are willing to try, as some open source developers for some reason consider Google's user-hostile security model decisions as sacrosanct.
Doesn't the Agricultural industry have one of the highest mortality rates?
Grain handling, everything from gases in an enclosed space to being buried alive by a quicksand like effect.
really lol? HN goes down every full moon day or something like that
You forgot google meet!
"For Tesla, Facebook and Others, Misusing AI and Pretending It's a Magic Bullet Is Getting Harder To Ignore"
Would these be the same developing countries where Facebook has helped facilitate genocide?

Seriously though, is the net utility of Facebook even positive there? It’s not like those businesses couldn’t interact with their customers over literally any other medium.
You're contributing to the monoculture of our planet by designing systems that make only certain grains profitable.

(I don't actually think this but it was a reasonable ethical dilemma I thought)
It's quite a coincidence for this to coincide with the whistleblower report + rumors of Peter Thiel (perhaps via Palantir?) involved in leveraging FB for the 2022 midterm elections.

I'm not suggesting that this is the case, but a failure of this scale (with internal systems also down) could allow scrubbing of evidence without leaving traces.
You mean rely and hope that whatever the user installed as the default browser will not mess-up help, documentation and other such links?

I'm quite certain that all those non-standard URLs are all for internal links to OS-related information hosted on the web. If they instead just popped a custom app that hosted an Edge webview nobody would rip their shirts. Doing that would be ridiculous given that Edge is just there and has been tested and vetted by QA.

Don't forget that these things need to work on all versions of the OS and in all locales. You're asking MS to trust that any 3rd party app will give proper user experience for any locale when presenting such links.

It's not as-if MS was hijacking normal URLs. But ripping shirts is soooo much fun.
Tomorrow the issue will be fixed, stocks will return to normal and even higher than it was yesterday.
It will be like this never happened. FB will live on.
It's meditation science fiction that I pulled from my butt. But my experience suggests that such might be possible.
Bezos and Jassey, Jobs and Cook, Page and Pichai, Gates and Nadella, (((Zuckerberg))) and (((Sandberg))).
How to use office politics to your advantage
I've read here on HN that exactly this was the issue as they had one of the bigger outages (I think it was due to some auth service failure) and GMail didn't accept incoming mail.
Google offered turn by turn in exchange for access to user data. Apple said no and developed their own mapping service, so privacy was the central issue.
Those businesses should have never depended on Facebook to begin with, if anything the fact that what you describe is a reality is a testimony of the degree at which Facebook is a cancer to be expunged.
> timing with regard to ...

No matter when something happens, other things will be happening in the world around the same time. That doesn't establish a correlation (China has been doing that for awhile), much less causation.
I assume that you used something like isolcpus kernel parameter, then sefaffinity to put the process on the isolated CPU.

I'm very interested in the kernel bypass technique to talk to the hardware, I'm assuming it was the NICs ring buffer.. how was this achieved in userspace?
The person you are replying to is referring to a terminal client, also named Hyper[0].

0. https://hyper.is/
Allowing copyrights to expire on pre-1950 creative works would also destroy billions of dollars of "economic value", but that doesn't mean it would be bad for humanity.

We should be careful to not take our values from a spreadsheet.
Unfortunately or not, this is exactly what Deaths per X miles driven statistics actually are. And that’s the statistic set at the base of automotive policy making.

Here:
https://www.iihs.org/topics/fatality-statistics/detail/state...
Seems odd. There is NO ONE that can let people in? Every building I've worked in has required badges, but there's always security inside that can check IDs and let people in. And methods of security getting in even if there's a power failure.
> We're living through a major real world event now, and the financial markets haven't collapse...

Central banks are pumping money into the economy at a record rate right now, and still not able to get unemployment down to "normal" levels.

This causes supply chain problems and inflation signals that , unless they get resolved, will force the central banks to pull back most of the stimulus. THAT is when the crash will come (if at all).

If that comes, central banks will have a choice. Double down on the austerity, raise taxes and try to weather the storm, or restart the money printer and risk very high inflation, if not hyperinflation.

My bet is on the latter, as goverment obligations are so high right now that I doubt they have the stomach for austerity.

This crisis is not over....
IT: "Please do this fix."

Person 1: "I can't, I don't have physical access."

IT: "Please do this fix."

Person 2: "I can't, I don't have digital access."

Why? It's [IT's?] policy.
> Maybe the internal FB Messenger app has a slide button to switch to the backup network for those in need.

Having worked for 2 FAANG companies, I can tell you most core services like which FB Messenger would be using internal database services and relying on those which would be ineffective in a case like this as it would not work and the engineering cost to design them to support an external database would be a lot more than just paying for like 5 different external backup products for your SRE team.
I was clarifying his comment, since he didn't mention that this is not a guess, but inside knowledge.

I was not trying to establish a trust chain.

Take from it what you will.
I'm surprised they haven't pushed ranges to datacenters located near the last access.

You could maintain strong consistency while having fast local reads, once you have entered a building in a country it pulls you into the local range. So you're only slow when switching.
Facebook Downtime Counter
It was actually by accident. It's more accurate with regard to the aggression ... to the extent the title is accurate at all. This is probably the article with the most incendiary title for this news. I should have posted another source.
I wrote this as a pure joke, but now that I learned that SERVFAIL is not cached on browsers, clients, intermediate DNS servers [0] etc. I am curiously wondering what will be going on. It is not only FB apps, it is basically every website request (that uses FB JS for ads, tracking, etc.) that triggers a DNS request, which will be forwarded 1:1 from the ISP's DNS to the null-routed FB Subnet. This should put orders of magnitude more load on resolving DNS servers than usual.

[0]: https://serverfault.com/questions/479367/how-long-a-dns-time...
Sigma Events for T Distributions
Posting this comment will be like farting into a hurricane, but here goes.

Company like Facebook has a serious problem and their stock drops ... precipitously. CEO of said company instead of selling their equity in their company has taken out loans against their equity in order to decrease their tax burden and cash in on the value of their equity.

What amount of decrease would cause a margin call from lenders for the forced sale of said equity and subsequently the loss of majority stake in their own company? Now obviously only the lenders know this information and assuming I have the rough order of operations correct.

Could this be a potential chink in the armor of founders / CEOs / anyone who takes out low interest loans against the equity they hold in their company? Maybe my understanding of this is too simplified.
This is completely anecdotal, but Whoop data could be used to uncover what moderate drinking does to you as people are self reporting their habits daily.

It is common knowledge in the Whoop community that any amount of alcohol will impact your recovery score(a in house metric combined of HRV, amount sleep gotten, amount sleep needed, recent activity strain, & more) for near 72 hours.
They've been pretty successful.
Unfortunately or not, this is exactly what Deaths per X miles driven statistics actually are. And that’s the statistic set at the base of automotive safety policy decision making.

https://www.iihs.org/topics/fatality-statistics/detail/state...
I haven't been following closely, but I think once they moved the prefixes they could no longer access the routers. Coupled with barebones staff at the data center due to the pandemic, and all internal communication being disrupted. Though I really expected it to be up within an hour or two.
This seems like a great writeup, but I'm a little skeeved by the genericness of the website it's on and the fact that absolutely no information about the author which might let me discern their intentions, biases, prior viewpoints, etc. is available that I can find.

Does anyone know anything more about the person who writes at this site?

(I know, I know, it doesn't matter who they are if their arguments are good. I'm just at the point where I pretty much have to assume bad faith until proven good faith when faced with any new source of online information.)