skysight-inc/HackerNews huggingface dataset:
I did see a lot of politically-motivated salivating which I thought was woefully premature. That said, I see 2 outcomes now:

1. Generally, with large, complex systems like this, everything works, until it doesn't. All the big boys have major outages periodically. I just can't fathom how Twitter is going to handle the eventual certainty of a major outage when, as the author notes, in some cases there are teams that have 0 people left.

2. More than the technical issues, betting that Twitter will go bankrupt is the easiest bet one can make. Musk saddled Twitter with a shit ton of debt - even if things worked as they did before he had to cut tons of people due to the debt burden.

The issue I see is that #2 directly works against #1. Musk has said it will be lots of intense work adding new features to try to raise revenue. But making a ton of changes, probably with lots of shortcuts to get them out the door quickly, especially when so much institutional knowledge has walked out the door, will make keeping the site stable even that much harder.
At this point in the SOTA, Whisper [0], can probably be a drop in replacement for the for-profit Nuance versions of Dragon, etc. It's even open source.

[0] https://github.com/openai/whisper
Exhibit B: non-code policy changes have non-zero effects on your infrastructure. https://twitter.com/atax1a/status/1594905195737419777

> we don’t think so. the prod incident we heard about involved someone making an ill-advised choice to reactivate a large account, causing a huge load on the social-graph system, on the night before a prolonged high-traffic event.
4k 120Hz OLED is pretty common, I use one.
Nah you totally could.
The question was why they were hired, not why they were fired...
Twitter has lost a lot of revenue recently and is now in a lot of debt, although that might not have had time to sink in. In real terms it is in a much worse place, and remember that betting on a success is something you'd do as an investor, not as a rank and file employee.
I have to reboot my phone weekly exactly because “stable” software isn’t. Even on a know platform in the absence of updates.

The only good thing is that “turning it off and back on again” works. Usually.
Exactly. Tiny memory leaks in seldom called functions can also cause slow degradation over time. People wonder why a simple restart seems to 'fix a boatload of problems' but this is often the reason why.
After timers, the #1 use of Alexa in my house: 'make a fart noise'.

We've tried to use Alexa for some games, like Jeopardy or Whose Line Is It Anyway, but it always is an incredibly buggy experience. Even just simple multiple choice games should work perfectly, and yet, a lot of the time even basic answers don't register, and the games themselves crash.

I've always wondered - stupid mobile games are dominating video games in terms of money right now. Voice platforms seem like they could do something similar, but all the platforms want to be your shopping AI or whatever. Just seems like the two worlds - voice platform and game developers - just could never get together and figure things out.
There's a pattern of diverging expectations here, one is the non-technical/naïve one,

- Twitter is going to go down tomorrow and it's all over. RIP.

The second is,

- Twitter is going to experience a failure cascade over time.

The third is,

- It's all going to be fine.

I suspect that the real question is, how many individual wires can break before the cable holding the suspended platform snaps?

I am not that good of a developer, but watching Twitter I can't help but be reminded of Arecibo, except at a larger, more abstract scale. There was no single massive event that caused the failure, rather a series of factors and events, tiny cables breaking that eventually leads to a failure cascade that then causes the suspended platform to crash.

From what I can tell, in the past week or so,

- Twitter's copyright system failed

- Two Factor Authentication broke down (it seems to be back up?)

- (anecdata) Tweets have been loading sporadically for me and other people, sometimes we try to open a tweet and it says that it doesn't exist. Happens more frequently with new/recent tweets.

- (unconfirmed) Twitter's managed account backend is behaving "strangely." For e.g., "One of my campaign managers logged in last week and found all our paused creatives from the past 6 years had been reactivated." from https://www.teamblind.com/post/i-told-my-team-to-pause-our-750kmonth-twitter-ads-budget-last-week-4dnbo1Ft ———— Friends have told me other similar stories

Are these failures symptomatic of a larger problem, or are they well-isolated parts misbehaving? Can Twitter even experience a failure cascade like Arecibo? Can that be paused/stopped?

I am asking this question because I don't know. And I'd like to develop a better mental model to understand what happens next.
I had the privilege to get a behind the scenes tour of a winery once because a friend’s family were the owners. They had some pretty sophisticated lab equipment. You could drop in a sample and it could tell you all sorts of things about its chemical makeup (magic to me because I never took much Chemistry). I asked them what sample attribute had the highest correlation with sales and without even hesitating they told me it was sugar content. They said their best selling wine was also their sweetest. They even said it was a pretty strong correlation that held all the way down the line from sweet, citrusy varietals to tannin-rich, cottony varietals. I think I was even told they could sometimes observe the effect from year to year if one vintage crop happened to be a little more or less sweet than the last year.

Now, for the record, I take the same approach you and GP mentioned here. I happen to prefer pretty dry wines too, but I’ve gotta admit knowing this shook me a little.
From this operations engineer's perspective, there are only 3 main things that bring a site down: new code, disk space, and 'outages'. If you don't push new code, your apps will be pretty stable. If you don't run out of disk space, your apps will keep running. And if your network/power/etc doesn't mysteriously disappear, your apps will keep running. And running, and running, and running.

The biggest thing that brings down a site is changes. Typically code changes, but also schema/data changes, infra/network/config changes, etc. As long as nothing changes, and you don't run out of disk space (from logs for example), things stay working pretty much just fine. The trick is to design it to be as immutable and simple as possible.

There are other things that can bring a site down, like security issues, or bugs triggered by unusual states, too much traffic, etc. But generally speaking those things are rare and don't bring down an entire site.

The last thing off the top of my head that will absolutely bring a site down over time, is expired certs. If, for any reason at all, a cert fails to be regenerated (say, your etcd certs, or some weird one-off tool underpinning everything that somebody has to remember to regen every 360 days), they will expire, and it will be a very fun day at the office. Over a long enough period of time, your web server's TLS version will be obsoleted in new browser versions, and nobody will be able to load it.
> Twitter grew 3x on the headcount front

There were multiple executives making $10m/yr+

There were board members

There were shareholders

Why did all of them not stop this headcount increase if it's as easily reduced as "too much headcount bad, smaller headcount good"? These are paid professionals who are supposedly wealthy, good at their jobs, smart, informed, etc.

How can us commenters on HackerNews sit from our armchair and say "ah, goofballs should've just not let headcount get so high!"

These qualified people thought at the time it was a good idea to get up to 7.5k people. How were they all wrong?
You are not only arguing with benefit of hindsight, many of your "if A then B" statements are highly speculative
I have a 48" 120Hz OLED as my main monitor, and I love the smoothness when you scroll or zoom on webpages, or even just moving the mouse around. Never going back to 60Hz.

I do a small amount of gaming, but not much.
Please do! I also recently purchased a Beelink U59 and have not decided what exactly I want to do with it.
I personally have somewhat mixed feelings. I do enjoy a good dumpster fire like any good internet denizen ( and I won't deny that there is an odd level of.. excitement maybe.. that one major social platform may go down ). I have no love for social media as some of my posts may indicate and for a variety of reasons including the amount of oxygen this creation takes and the amount of confusion it causes in general population, it would not be the worst thing ever.

That said, the sheer amount of weird emotion attached to this particular billionaire is just perplexing. Only few years ago he could do no wrong. Today, it seems, he is being ganged up on including by some of his biggest internet fans. And media oblige for whatever reason. I dislike billionaires, but I dislike them as a caste in our societies. I don't see Elon as a savior of mankind or the devil, but those emotions that swirl around him are made to make him look like one or the other.

Somehow, one is supposed to pick sides.

Trump seems to create the same emotions. It is truly fascinating.
The scratch coat of plaster made in stucco has horizontal scratches in it made with a scarifier (looks like a hand held metal rake). This is a rough coat that the brown coat (next layer) can attach to.

This doesn't refer to "made from scratch" interestingly.
To whatever degree I might have ever done that, because Amazon went to the Marketplace model of putting all seller equal there are 15 duplicates of everything which may have very different prices.

Back when it was only Amazon it may have been possible to Alexa to buy some very specific thing (“buy Mario Party 23 for Switch”) but. Now I wouldn’t be able to trust even that.

So say nothing of worries about kids/guests/accidental activations ordering things.
Without genuine understanding voice is just a terrible method for control. I don't know what the options are or what exactly I'm supposed to say. Alexa is overly verbose too. The control feels very clumsy, though it's sometimes good enough.

I am imagining GPT-4 on the other end with a suite of controls. If Alexa could understand what I'm talking about and all of the options available to it I think it would be a much better system. I could also imagine a system that's always listening and proactively acting on my behalf without my even giving directions. If hardware advances to the point where I could run a model this size locally? That would solve privacy concerns and improve latency. How far are we from running GPT-4 on commodity hardware? 20 years?
> I disagree; math would be a closer analogy. And indeed, arithmetic still works like it did a millenia ago. Closer to the present, I have binaries from the late 80s that still work today (and I use them semi-regularly.)

Sure, those binaries might work the same when executed. Although the probability of that is never 100%, but as you pointed out, the rules of arithmetic aren’t expected to change any time soon. That’s correct. Unfortunately software does not exist in its own micro-verse, it’s subject to the laws of physics acting on the machines it’s running on. So while you might be able to write scripts that work decades later, it’s much harder to ensure those scripts consistently run for decades. RAM chips, CPUs, and everything in between are guaranteed to eventually fail if left running unsupervised in perpetuity. Entropy raises with complexity. At Twitter’s scale, to run a software service you need globally distributed cloud infrastructure. They likely have hundreds of services, deployed to many hardware instances distributed across the globe. Twitter isn’t 1 script running 1 time producing a single result. It’s hundreds if not thousands of systems interacting with one another across many physical machines. Layers of redundancy help, but ultimately cascading failures are a mathematical certainty. Many would argue the best strategy to reduce downtime on these systems is to actually optimize for low recovery time when you do fail.

Software is also bound to the world in other ways. Similarly to how most business processes, products and even more generally, tools, change over time, so too do the requirements placed on software systems made to facilitate or automate these things.

Ultimately the only way to escape the maintenance cost of software is to stop running it. The longer you leave a software system running, the more likely it will eventually stop.
Their CGI trailer is still one of the best trailers I've seen for a video game [0]. When I saw it, I really felt a sense of futuristic wonder, thinking what it'd be like to augment our bodies in the near future.

The color scheme is gold because it was a "golden age of human augmentation," highlighting how people saw the new technology for its endless possibilities. Its sequel was not gold because, predictably, people used such technologies for selfish ends. I often think of other such real-world technologies paralleled in the same light, like how the Internet evolved, and even those of centuries past; the trailer's Renaissance and Icarus comparisons are apt.

[0] https://www.youtube.com/watch?v=gTdNRdw2y4E
DUNS is not a government thing.
The US government does have a unique number, it is your tax ID number. Typically called an EIN, but it is not farmed out to a third party
What you are referring to is a private company trying to create a global id. It sounds like most banking institutions what you to have it. But it isn't required.
I've never heard of it, and I've established companies in America.
Yes, I've heard it all before. People who were drawn in to the baseless Russiagate conspiracy theory, and all the other ones have said exactly the same thing to me as you are now. Sure, I'm a secular-heretic for wanting to see evidence for extraordinary assertions from people with obvious ulterior motives, rather than taking them on faith.

So far, the conspiracy theorists have had a poor track record as far as I can see. That said, I don't engage or try to debate my friends on it, they have always become extremely upset by any debate on such topics. And they're not hurting anybody so my philosophy is just to leave them to their faith. I don't go around challenging people to prove the existence of God whenever they make a religious Facebook post either, I'm not that kind of asshole.
> don't care to study strategies

did you mean "haven't had much chance to" or something?

in english, "don't care to" generally means the same as "don't want to", aka "dislike to", but i'm curious what idiom you're confusing it with
Wait Musk saddled Twitter with debt? Please elaborate...
Studies? This is CTR we're talking about. It doesn't have to be a research paper to show what's the average CTR, advertisers know it beforehand, they just made the info public.

https://brave.com/brave-ads/casestudies/
That's what I have been trying to tell people. There are literally two alternate realities and the extreme polarization tries to suck you into either one or the other. Because what people don't realize is that worldviews are tied to group identification.

My theory is that the economic problems have been stressing the systems and that makes the "bones" more apparent. The core concept of this country is checks and balances of two opposing sides. I think it was a great improvement over authoritarianism, but you can see that it is definitely not the ideal final paradigm.
It’s unclear what they’re counting as public dollars on health care but I assure you none exists unless you qualify for special programs like Medicare or medicaid. Perhaps they’re including public research money and other subsidies for medicine? I find it implausible Medicare / Medicaid costs $10k/ year / person. We pay for these out of our income and it’s not nearly this much.
Yes, mostly web3 projects but there are other advertisers as well. Like Twitch for example.
Those are hardly “Twitter going down”. They’re normal medium-severity outages that could have happened anywhere.

Sure, it’s plausible they were made more likely by so many people leaving, but they’re not exactly the meltdown people predicted.
"That's cool"

"That's not cool. That's commie bullshit"
rottisserie chickens for $3.50 in Australian supermarkets are great examples of loss leaders. They get you in there, to buy chicken...and other stuff :)
> Now, why don't people use the same analogy for Russia starting that war?

Because a war of invasion and annexation is something we've learned since WW2 that won't ever be tolerated in Europe again, history has shown why.

> Let's take a step back and ask a basic question - who escalated this all?

Pretty easy: Putin and Russia. First 2008, then 2014 and a full-scale invasion in 2022. Russia chose to invade a neighbouring country in 2008 (Georgia, hope you remember it), Russia decided to escalate that and annex Crimea in 2014. Russia shadow-invaded Ukraine in 2014 (or have you forgotten all the military personnel wearing Russian gear that went "on vacation" to Donbass and Crimea?). Russia downed a civilian airplane in 2014.

Or isn't any of that escalation?

> Who benefits from this war?

Very good question, if the war ended in 3 days as it was Putin's fantasy who would have benefited from this war?

As it didn't end in 3 days, no one is really benefiting from this except the military industry in multiple countries, mostly in Europe, which didn't have to arm themselves for decades of relative peace.

> Who pours money there?

All the Western countries are pouring money in there, to fend off an invasion, to keep the sovereignty of a state, a thing that wasn't really required to be taken care of since the end of WW2.

> Why is nobody asking these questions?

Because they are fucking dumb questions posed by Russian propaganda, that's why.
To me the analogy here is that the new boss rolled in and sold all the fire extinguishers. That by itself doesn't set the building on fire - it doesn't even increase the chances of a fire occurring on any given day. But when one does...
It was a leveraged buyout. Musk borrowed $13 billion to complete the deal, which Twitter needs to pay back.
At least one high-follower account on my timeline posted that their newest notifications were from October
https://dothemath.ucsd.edu/2012/03/space-based-solar-power/
Disclaimer: I work in finance, and have for 15+ years.

> Given the special hardware needed for the grand master clock to get down to nanosecond time scales, I'm doubtful this will be used in most data centres of most corporate networks.

The "special hardware" is often just a gps antenna and a PCI card though. In fact, many tier 1 datacenters actually provide a "service" where they'll either cross connect you directly to a PPS feed from a tier 0 grandmaster time service or plug your server into a gps antenna up on the roof. It isn't really that exotic. For financial application, especially trading ones, syncing a LAN timesync to a handful of nanoseconds is doable and optimal.

It is just a matter of time before non-finance sees reasons that better timesync is useful. Precision Time Protocol aka IEEE 1588 was released in 2002 and IEEE 1588 version 2 was released in 2008. This isn't exactly a new thing.

With the right hardware and a tier 0 timesource, modern ntp on modern hardware with modern networks can keep a LAN in sync subsecond. However, as a protocol, NTP only guarantees 1 second accuracy.
Innovative daylighting systems’ challenges: A critical study (2014) [pdf]
> all software goes down and requires specialist intervention eventually

Well, that’s it, isn’t it? How many software systems need to keep running for Twitter to remain more or less functional?

If there are 10 critical systems that are running at four 9’s, you’d expect 3.6 hours of downtime a year, or about 90 days of uptime at a stretch if I have my math right.

If there are 100 critical systems running at 3 9’s, you’d expect 2.5 hours of downtime per day.

So yeah, all software should keep running. But it doesn’t. And something like Twitter isn’t “a software”, it’s a very large assembly of lots of different software systems and the exponential math that dependencies create.
Ex-Apple Daily Employees Plead Guilty in National Security Trial
I'm pretty sure that's just so they don't feel like they have to go all Article 5 on Ukraine. That would be really awkward, going to war against a country you were donating arms to yesterday, right? So sure, there's an agenda, but it seems like a morally defensible one, and not one supported by lies (at least not where it intersects with US interests).
I enjoy wine, but not to the level of detecting different ages or locations. I do, however, enjoy trying to find flavors in Whiskey, and I can attest that it's not all fake. The process and age makes a difference, and the age definitely comes out in the flavor of a whiskey. Of course it does, leaving a strong liquor in a cask will add flavors, and obviously time will determine how much of which flavors are imbued. I have a bottle of Bulleit rye, and my first taste impression was "I've had this before". Turns out it's made by some massive conglomerate that makes almost all rye whiskey in the US, and I had, indeed, had it before. It's as simple as recognizing a brand of ice cream or candy by taste. Candy might be harder, actually. Sugar masks flavor far better than alcohol.
Thanks for the laugh I needed that
> You also need to factor in the time value of money: if the winemaker sells you a 2022 release in 2022, they get paid immediately.

This is right, I forgot about this: at 5% interest rates, 5 years of storage is actually 25% of the original price, which is probably substantial factor.

> Also factor in temperature and humidity controlled storage (a kitchen fridge will not do),

My kitchen fridge example was only meant to provide an estimate for the cost. Controlling humidity upwards is not expensive at all, it’s even cheaper in fact than controlling temperature.

> insurance against disasters, backup power generation, and so on.

These are extremely cheap at scale. You don’t really need backup power generation, the wine won’t spoil from few hours or even days of inappropriate temperature.

> If you think aged wines are overpriced, it is easy to cut out the middleman and age it yourself — so my guess is that the market is reasonably efficient.

My point was rather that the mere cost of storage is not the main part of the premium. Capital cost is probably significantly higher, but what is probably even higher still is speculation premium: not all wine vintages are appreciating equally, and if you just buy random wines, they will likely won’t appreciate all equally over time.
Sounds like you did what they wanted.

But I bet most people don’t. They probably use them for competing services like Spotify that not only don’t make Amazon money but reinforce the feeling of not caring about Amazon music.

I’m in the Siri world, but I’m with the other posts in this thread. 99% of my usage is simple stuff like timers, math problems, smart home, etc. stuff that just won’t ever drive anyone’s revenue directly.
I think you are both broadening the definition to the same level as the original blog author and also narrowing what the commenter said to make any religious community a cult, I think that fails to consider hat the common understanding of what a cult is. So things like: “Keeping peace in the family” != “my family will disown me”. “I am part of a community” != “Everyone I know and are about is in a cult”. “My pastor provides me counseling” != “I need the leaders emotional support”

It