skysight-inc/HackerNews huggingface dataset:
Ask HN: Resources on learning System Design (back end/data engineering)?
I'm looking for resources/tips on how to learn concepts in system design. Things around backend engineering and all the various components that go into building large scale systems (load balancing, message queues, CDNs, etc.). I'd also like to learn concepts around data like database sharding, non-relational database models, streaming/batch (lambda architecture, kappa architecture, etc.) and more.

In terms of what I've done so far...

I've read

- Designing Data Intensive Applications (https://dataintensive.net/) - This book has been recommended a ton on HN/twitter/reddit and I found it extremely useful to get a better idea of how data systems work, different types (OLAP vs. OLTP, data warehouse vs. lake, etc.) and get a better understanding of distributed systems.

- Quastor (https://www.quastor.org/) - This is a newsletter that sends out summaries of system design blog posts from Big Tech engineering blogs. It's been helpful to me for getting an understanding of the different components involved in the tech stacks at various companies and how companies think about building scalable systems.

- MIT Distributed Systems (https://www.youtube.com/playlist?list=PLrw6a1wE39_tb2fErI4-W...) - This is a series of lectures by Robert Morris (co-founder of YC) on distributed systems and their properties. The lectures pick a specific tool/technology (Google File System, ZooKeeper, Apache Spark, etc.) and then discusses it. I've really enjoyed reading the papers and watching the lectures.

I'd really love any recommendations from the HN community on how else I should be learning about system design.

Thank you very much!
Depending on the parent commenter's definition of cutting the fat, I don't see your understanding's of PE as being much different
Octopus sounds really useful; is there an open source equivalent? I'd love to be able to scrape my own data on Facebook. Their data export feature is fairly good but far from complete.
And took notes on your personal life and tried to sell you things they'd specifically think you'd want.

And traded notes with other sellers.
The people those cruise ships bring are a problem
Build a rock paper scissors lizard spock ML recognition with a RPi4 w/camera
attitude is the reason why the west is in decline

knowledge is the reason why asia is rising again

a good entrepreneur is a good manipulator, a good liar and a good seducer

are you willing to trade your knowledge to become someone like that?

use your wealth of knowledge to find people like you and build nice things for your surroundings

but more importantly, do what you like, don't chase people for their attitude, look, they are already trying to sell you something and you fell for it
That's very unlikely in this case. Johnson already tried to hang on longer than he really should have done, albeit he did survive a motion of no confidence. It took the mass resignations of his cabinet to force him out.

The "never back down or admit defeat" seems mostly to apply to bureaucrats and public health officials. Conservative MPs are famous for resigning when they think honorable norms have been violated (as they see it), and getting rid of leaders once they become an electoral liability.
You are confusing pressing charges and indictment. Pressing charges just means you accuse somebody of a crime and “press” the prosecutor to indict them. So the state does have the ultimate say on who is prosecuted, but that doesn’t mean you can’t press charges.
Thanks for having my back. #3 is an overwhelming real world phenomenon. In fact, I posted my article on reddit and someone wrote back a comment with a huge OOP solution that would mitigate all my problems. Not sure that reader got to point #3.
This is completely another level of "issues" than other problems in this thread. During code review I'd only mention it as a nit. The longer form is correct and the only downside is that's a bit longer. It doesn't mess up code modularity or affect maintainability in a noticeable way.

Well, assuming that the language doesn't have any quirks in this area - e.g. in Java your statements aren't equivalent for a Boolean x.
Tangentially, I find it funny that the answer to this states banner blindness on a site (and set of sites, i.e., StackExchange) that usually takes nearly half the screen on desktop with a useless banner that everyone just ignores and scrolls through. StackExchange could learn a thing or two about UX from ux.stackexchange.

On topic, most people just don’t read on web pages and emails! They expect to be spoon fed in tweet increments through short messages.
with people like that roaming around the landscape maybe there is a point to all that leetcoding nonsense, since they'll certainly spout off some truly insane stuff in interviews...
OMG, does anyone else recall a series of comedy videos about a sysadmin dealing with employees? My favorite was a call from someone who "ran out of rows" in Excel. The sysadmin is losing his mind trying to explain why that wasn't possible.

EDIT: Found it! https://youtu.be/1SNxaJlicEU (slightly NSFW, skip to about 3:30)
It used to be they didn't render high-dpi text correctly, but fixes went in for that. Even so, it's a hack and often not pixel perfect. If you're trying to do visually sophisticated things (soft shadows, fading animations), GDI+ is inadequate. So basically it looks antiquated. It might be fine for simple form-like programs.

My main point is that "native" covers a lot of ground, and "native legacy compatibility" and "native modern" are radically different technology stacks on Windows.
Go's default `net/http` package will serve HTTP2 when a https configuration is provided (also most web browsers don't support h2c, aka. HTTP2 without tls).

The default limit of ~100 connections should be more than enough for most applications (additionally, the JS SDK client maintains a single SSE connection for a page no matter to how many things the user has subscribed, so that also helps).
Launching large amounts of nuclear material comes with its own risks, and if it orbits the sun in an earth-crossing orbit, we will eventually encounter it again. Launching into the sun would be nice, but that's way more expensive.
It's been awhile since I absorbed the weird programming norm that "real programmers use the !x form!" but even after 10+ years of !x , I still find ==false more readable.
Arguably this is a populist reaction to a situation which is heavily in Norway's favor. Instead of thinking in terms of 'Danes and Germans taking our electricity', Norwegians should realize that they are able to export an extremely valuable resource, energy storage, to two countries which have lots of wind power and huge storage needs. In fact, as the cost of energy has risen massively in Germany for well-telegraphed reasons, costs have gone up in Norway (despite Norway being able to meet all its own needs with hydro) because the two systems are economically linked. But this means that the potential income from storage/balancing has also gone up massively. They should build more hydro with pumped storage and more export cables.

Of course, this means that Norwegian society has to make sure that the benefits flow from those who will make money exporting storage to those who are affected by higher prices. But as a high-trust, wealthy country, the only one to solve the resource curse, they should be capable of doing this. This is what I meant when I said that difficult problems of regulation and resource allocation go along with tech advances in this field.
My parents went to Venice on holiday a few times in the 1970s. No, it was not overburdened with tourists. I have seen many photos and heard good stories. The insane levels seen in Venice and Amsterdam only started in the last 10 years.
How simple are we talking about when you say "simple frontend development"?

For simple personal projects I write all my own HTML and CSS from scratch. It's not particularly hard to write stuff with low to moderate interactivity from scratch and the infrequency that I do frontend dev means it's quicker to do that than learn whatever new framework has replaced the deprecated one I spent time learning a couple of years earlier (that's probably unfairly snarky but from an outsider looking in, it's hugely daunting and depressing when you see just how much churn there is).

The only problem with rolling your own is that you raise the difficulty for anyone new joining the project (if it's something you plan on opening up to other contributors) and CSS can be a bit a of trial and error to get right too.

However if you're just writing a few test UIs to interface with backend APIs you're POCing (which is what your question reads as) then it's probably easier to skip all the other frameworks out there and knock up something yourself.
Ask HN: Do math contests attract smart people to high school math teaching?
And so these high school math contests benefit most students and not only those who care about math contests?
First operational 'sand battery' can store energy for months
Anyone else heard of Tim Berners-Lee's idea of hosting your data in pods outside the relevant corps wanting access to it and you controlling what's shared and how?
This is such a completely different way of doing it, I'm not sure of all the implications, be that from admin (how much effort) to security (would this be a massive hacking opportunity) etc.
https://www.theregister.com/2022/01/20/tim_bernerslee/
Human wears augmented reality contact lens for the first time
>Wild that what's obvious to ransom HN commenters hasn't been noticed by the guys buying the dead shells of companies.

It isn't so wild when you consider cases like Sears, where Eddie Lampert in effect used a variety of shell companies to sell off the profitable parts to himself.
It depends on what you're looking for. There's been a withdrawal on focusing on 4K displays in laptops. Most of them exist as a marketing point, but provide no ancillary benefits.

One thing you might notice is "low res" displays tend to support things like higher refresh rates, might have better color and HDR

I have an ROG Flow X13 2021 with the 1980x1200 display. This "low res" display supports higher refresh rates, variable refresh rate, has good color and HDR support, and works quite well for a 13" laptop.
There are district cooling systems as well, there don't seem to be any scale problems in the engineering. It's more attractive if you have a body of cool water nearby for the heat pumps.

But the users need to have different heat exchangers than just radiators, if you just put cold water into radiators they'll start sweating precipitation and you'll get mold etc damage.
Now. I am not a 10x developer.

But this concern here with writing a make_pizza() function:

> The problem is that these two pizzas just happen to have the same crust, sauce and cheese. Had we started out with two pizza types that have different crust/sauce/cheese, we never would have made this refactor.

You can solve for this by adding parameters with default values for all of those things. Use the defaults for the most common use cases, but of course for pizzas with different crusts and such, you can override the default.

Not all languages support default parameter values, it's true. And of course, there is a level of complexity at which this breaks down.
Of my neighbours, colleagues and family, those that took the vaccine are those that are also getting ill with covid over, and over again.
Cough, cough, Google, cough, cough…

I’m not ashamed to admit that I’ve done some jquery shenanigans on my Facebook friends page to “export” my friend list so I can retake control of my friend relationships (disintermediation for the in-crowd).

So easy to push data in to Facebook, so hard to get even basic data out of it.
>It is a fundamental aspect of modern capitalist organization of wealth.

In a functioning system it's a property that reflects the time-value of money.

In cryptocurrency, there is no need for such a property. Any altruistically-designed cryptocurrency would have inflationary tokenomics.
One view does not exclude the other. Predators have a role in ecosystems, culling the weak and the sick from the herd. PEs are corporate predators, preying on bloated orgs, killing them, returning whatever value still held (trademarks, patents) back to the "environment".

But of course that "pure market" viewpoint doesn't take into account the social costs and individual pains these operations entail.
So then rich people will pay poor people to buy the tickets for them. And that's if you can come up with a way for someone to declare their wealth reliably.
From a corporate perspective, either SentinelOne or CrowdStrike Falcon.
It's basically victim blaming. "I just punched you in the face and broke your nose. Why are you so ugly now?"
it has its downsides, but so far I feel very productive while I am working with this framework. It also has great documentation.
The entry fee is the only way to do congestion pricing that is predictable and scaling. To achieve those it has to get done in advance. If the cheapest tickets have been sold in advance for the day you want, you can still get a ticket but the price went up. Meanwhile you need to know if you'll be able to get into Venice (or afford to) when booking tickets or selecting hotels.

So your plan works if you just want to raise money/raise the cost to discourage people. It doesn't work if you plan on increasing the cost to act as a soft cap on the tourists per day and you want people to know if they got in on a given day.
It's the "hero" vs "villain" dichotomy.

What are the characteristics of the "hero" and of the "villain"?

Are there any patterns seen across different stories, e.g. Ebenezer Scrooge from "A Christmas Carol", Phil Connors from "Groundhog Day" as the "villain Grinches"
vs. SpongeBob, Ferris Bueller, Pee-Wee Herman as the "happy warrior heroes" or the "capably glad" vs the "socially mad"?
> the more you have of the capital the more you influence the system?

You can influence the payment processing. And if the crypto would allow false payments, it’s value would immediately drop to zero. The more money you have invested, the higher your incentive to keep it’s value stable.
Pretty ironic that Mark Z himself started out exactly like this: scraping Harvard servers and photos to power facemash.

He subsequently realized that he doesn’t need to scrape if he can just make a viral site that lets people share this info with each other while he can eavesdrop on ALL OF IT:

https://www.esquire.com/uk/latest-news/a19490586/mark-zucker...
The problem is the execution-time verification (which calls home), not the compile-time signing. Think about it: the exec(3) function of the OS connects to the internet ! This is absolutely crazy stuff, yet folks don't seem to care.

Continuing with your analogy, it would be as if every time that you enter your home, with your key, with different clothes, the keylock sends your photo to Apple for "verification", which in turn has the power to not open your own door if it so decides. Would you find that acceptable?
UWP is most definitely not HWND per control, and doing so would be incredibly limiting. I don't know the UWP (WinUI) internals, but on mac it's generally a CALayer per widget, and Core Animation is doing the compositing. On Windows, the Windows 8 equivalent is DirectComposition, but I think the WinRT Windows.UI.Composition interface has some capabilities not exposed through winapi.
Following some links:

https://notes.ethereum.org/@vbuterin/single_slot_finality

> If 51% of validators start censoring, the victims and users can coordinate on a minority soft fork where they build on each other’s blocks and ignore the attacker. On the minority soft fork, the attacker’s deposits would lose millions of ETH to an inactivity leak, and after a few weeks the chain resumes finalizing.

It seems that this could easily be turned around: if enough malicious validators coordinate, they can create a soft fork that burns every honest validator’s stake until the malicious validators have an ever increasing supermajority.

New victims who aren’t validators could try to become validators to fight back, but the colluding validators can just ignore those proposed transactions.
Followup question: why has this not been changed, this question is three years old and yet the wiki under discussion still has the exact same homebrew banners that blend into the page's frame and have the "hey this is not official D&D content" part way over on the right, away from the natural eye-scanning flow.
thanks for the idea but not that interesting beside I don't have access to RPi
It trashes your IRR though. Now what does the sponsor make?
For me the main reason to upgrade would be Magsafe. I'm constantly afraid of someone stumbling on the cable and destroying the USB-C port.

I couldn't find any info on this, is this the same Magsafe as the 2015 devices, i.e. can I use existing old chargers? If so then I might even consider upgrading just due to this (I also have a 12/24V Magsafe charger).

24 GB RAM would be nice compared to 16 GB currently, but it's not something I'd really need.
I am currently in the USA and when I visit a public FB page e.g. [1], there is a small login header, and a very big annoying footer login. I estimate 15% of the content is blocked. I had spent the past year outside USA until one month ago. When I visited the same sites while traveling outside the USA, the annoying login footer moves to the middle of the page blocking almost all content. I do not have proof at the moment, but that was my experience trying to read 95% of government, business, and community pages who are almost all on FB.

[1] https://www.facebook.com/ParquesNacionalesdeArgentina
Yes, taxes and subsidies distort the actual costs of energy production. But if you look at my example, 90% of the Stromsteuer (tax on your energy bill as a household) don't go into funding of energy production, but to stabilize the (very broken) german mandatory pension/retirment system. But yes, one could argue that it is just moving around taxes in the government spending household.
When a quarter of the class identifies as trans
She does write about this topic as well, it is worth reading the full article. Bitcoin can never be used directly for all the world's financial transactions since it is not capable of that throughput. But those transactions can use things like the Lightning Network that is built on top of Bitcoin. Bitcoin is useful as a final settlement layer, more like how banks use Fedwire today.
In Germany Lenovo sells a Legion S7 15 with a 4k display and Ryzen 7 5800H (article number 82K800GUGE). There are also variants of the Yoga Slim 7 Pro 16, the IdeaPad 5 and the ThinkBook 16p G2, each with a 2560x1600 display and a Ryzen CPU.
> elsevier copyrights papers on their own name. should we fuck with them ?

Unironically yes. Fuck Elsevier and what they've done to academic publishing.
All good! It's a total nuance to this space so I can understand if someone got it wrong anyway.
Why is it necessary to break even? Ordinary users can mine on their home hardware and make a return on their existing hardware before it depreciates.
Thanks, added these example projects to the post
Bootstrap is 100% the answer. Pulling in everything from a CDN is the easiest route.

Bootswatch is also great if you want a little more styling. (Free bootstrap themes)

Bootstrap is great because it is so ubiquitous. They have page templates you can copy to get the page layout you want and then there are tons of bootstrap examples you can just copy and paste.
> Do you think poor people should also be able to park their cars in Manhattan for free?

No, but I think that they should be allowed to enter Manhattan for free.
>“At the moment, even if someone calls the leader of Japan an idiot, then maybe under the revised law that could be classed as an insult,” Cho said.

You think someone who calls their leader an idiot should be jailed?
No but I don't want to fiddle with this software. My phone have to work and I don't want troubleshoot it when I'm in the middle of nowhere when it dies. Some of bank apps also don't want to work on non official or rooted systems.
Definitely not Frank Zappa: his brilliant lyrics are way too engaging not to pay them full attention, and they demand the same part of your brain you use for programming.

Except for "Shut Up 'N Play Yer Guitar", "Shut Up 'n Play Yer Guitar Some More", and "Return of the Son of Shut Up 'n Play Yer Guitar". Also "Francesco Zappa"!

https://en.wikipedia.org/wiki/Shut_Up_%27n_Play_Yer_Guitar

https://www.youtube.com/watch?v=fKMk6fcKbwE

https://en.wikipedia.org/wiki/Francesco_Zappa_(album)

https://www.youtube.com/watch?v=JpCuvaDdzgw
DRY is about code, not data. If you see two methods do the same thing somewhere in their code blocks, separate the duplicate logic into a separate method, then call that method when you need to execute that logic.

The reason for this is maintainability. Say that logic needs to be changed. If you didn't break it out into a callable method, you'd have to find all the places you use that logic and change it. If it is a callable method, you only have to change the logic in one place, thus DRY.

https://en.wikipedia.org/wiki/Don%27t_repeat_yourself

Here is an overly simplistic example:

If you see this scattered around your code:

var formattedName = firstName + " " + lastName;

Create this method and call it when you need it:

string FormatName(string firstName, string lastName)
{
return firstName + " " + lastName;
}

var formattedName = FormatName(firstName, lastName);

When business decides it wants to change name formatting from "firstName lastName" to "lastName, firstName", you only have to change the logic in the FormatName method because you "didn't repeat yourself."
The great Pokémon card shortage has a solution: 9B new cards
The other issue is ‘heat quality’ (or temperature deltas). It’s easier to recover a given amount of heat from very hot sources than from only luke warm sources, and smaller heat stores will have a very pronounced difference very quickly, resulting in control issues.

Basically, if you only have a small amount of sand, it’s harder to keep it warm for longer (more surface area for the mass, so more insulation required), and harder to deal with effectively (fan needs to blow much harder when it’s only warm, much less when it’s hot), etc.
> For example, there may be users that use WMs like Blackbox, StumpWM, or Xmonad that they've heavily customized throughout many years, and now they're essentially being told that their setups will eventually be incompatible with the changing ecosystem.

Yes. You got a wonderful set of tools as a result of all the work and time folks put into X.

The problem is that the folks doing most of the lifting in this space right now, the ones still giving their free time and effort, have decided that they'd rather be developing Wayland (it's not all of them, X is still kicking, but it's a lot of them).

You, as a user of the free software produced by real people putting in time and effort, absolutely don't get to bitch about "Loss of choice is a loss" when what you're really saying is "I'd like to force the people working on wayland to continue working on X, because it's beneficial to me".
This would be a good idea for a speculative fiction novel, about how constant access to the internet could drive our evolution.