skysight-inc/HackerNews huggingface dataset:
There's a strong tendency in the tech world - and elsewhere but less so - to point to failures as an absolute proof something won't work, and to similarly use early successes as overly strong signals of viability.

An example of this would be the "obvious" superiority of micro-kernel OS designs compared to monoliths. Which is not at all how things actually panned out with Linux - but the problem was never about the architectural choice, it was about the fact that OS designs are hard - and you need to do the work to solve all the little problems which crop up with your big idea.

FSD is essentially going through the same thing, but the stakes are higher - FSD can kill people as a basic part of its function. We were always in a countdown till we started getting the first self-driving casualties.

This is not of course to suggest that what Tesla is doing right now is okay, but I'd sure be cautious trying to read the winner of the next 20 years into it.
No profilers really hold a candle to vTune is the problem in general. I love the new AMD chips but uProf isn't in the same class as vTune and that is sad. I'm certain with better tools the AMD chips could be demolishing Intel by an even greater margin.
There are outside groups with less of that type of bias that will often be able to see things in a more nuanced way that more closely reflects reality. For example, maybe someone from another country who is uninterested in politics.

That doesn't mean there are some people who have no bias at all. Just different ones that apply to other things.

In the examples you quoted, he was deliberately listing views extreme enough to be inaccurate. For example, both the liberal and conservative examples said the characterization applied to _everyone_ in those groups. The less-biased interpretations might be: _some_ conservatives really _do_ care about fetuses and lives, and _some_ liberals _do_ care about preventing COVID deaths.
Ah, apologies.

I realise in hindsight that I was rambling from the point of view of offering five nines for software that is inherently flaky/unreliable. Companies where developers cycle through and knowledge is lost, technical debt accumulates and systems are used almost counter to their intended purpose (ie Redis as a database)

In that sense, it's like constantly massaging applications to stay alive or at a reasonable level of service and so hence the assumption that someone will be paged and respond in order to preempt a failure or restore service during an outage.
If you said that the brakes on your car were "a joke" I would not understand you to be advocating for the abolition of brakes. Likewise, I'd assume people complaining about journalists want better journalists, not zero journalists.

I want better journalists for example - although I also don't have a good solution in mind for the economics problem that the internet gives journalism.
> So why not have a mechanism by which the user's desktop environment and init / login system can reserve itself enough memory to always be responsive under any memory condition?

This! Always wanted a /proc/$PID flag to flag process' pages never to go to swap and reserve a number of cache pages specifically for the process. Something similar to mlock.
Why did you put "one-in-tens-of-thousands" in quotes?

I looked it up, in 2019 there were 89 million 20-39 years olds in the U.S. [0], and 7000 ages 18-39 died dies of covid19 [1]. Mathematically that works out to 1 in 12,714, and would be a bit higher were 18-19 years added to the denominator. I didn't mean to exaggerate this number.

You mentioned family in vulnerable groups. Lets grant common sense, that someone who lives with their Grandparent's isn't going to go clubbing? EDIT: People need to use common sense, and they seem to, even in when they are not being forced.

I wish that we could measure the effectiveness of mitigations. This may be your strongest counterpoint. I end up extrapolating worst case numbers sometimes. I'd love to see you come up with a worst case number. I've got the feeling we could agree on a number, but still might disagree on how we react to that number.

Every weekend the beach bars near the Florida city I'm located in are busy. The majority of people are not wearing masks, sometimes almost nobody, sometimes that includes staff. Some places, with DJs and bands, are literally packed shoulder to shoulder as the night wears on. Like it was 2019 again!

I've also been in NY for part of the pandemic, and keep up with my circle there. As far as I know it's like you said, the Florida beach bar scene would not be tolerated. The rules are a bit stricter in NY, better enforced, with less exceptions, with more state backing.

So why are per capita covid19 deaths higher in NY than in Florida? Florida has 1500 deaths per million, and NY has 2500. [2]

EDIT: I was replying to your points, and don't mean to sidetrack us. Perhaps we need to back up? Do you think it is unreasonable for a young person to go clubbing under any circumstances at the moment? IS a definite 1-2+ years of restrictions worth avoiding a small possible (1/1000 at worst?) chance of harm.

[0] https://www.statista.com/statistics/241488/population-of-the...

[1] https://www.statista.com/statistics/1191568/reported-deaths-...

[2] https://www.worldometers.info/coronavirus/country/us/
Could replace a portable generator, if you have a use for one of those. Will be on my list if I need to replace my pickup truck (or my portable generator).
OpenSSL, like many other software indeed used a custom allocator, but this hasn't much to do with this to anything at all, as the system allocator also strongly favors giving back memory that once belonged to the same process, as it has to zero memory that belonged to other processes first.

This is of course a kernel feature when the lower level primitives are used that ask for blocks of memory from the kernel, which zeroes them if they had belonged to another process prior, and does not when they had not, and thus strongly favors giving back own memory. — allocators, such as the standard library's one or any custom ones, are built on top of these primitives.
When the library attempts to load something from disk that doesn’t fit into memory, it’s transparently, and (usually) without extra intervention from the user, swaps to memory-mapping and chunking through the file(s).

Particularly useful for when you’ve got a bunch of data that doesn’t fit in memory, but setting up a whole cluster is not worth the overhead (operationally or otherwise) and/or if you’ve already got a processing pipeline written in a language/framework and you can’t/don’t-want to go through rewriting it for something distributed.
What is the benefit of using such a solution rather than a standard database like sqlite or posgresql perhaps with some automatic api on top?
systemd-oom
And Voyager 1 and 2 are still zooming as far away from our solar system as they can get and happily sending back teeny bits of data every now and then. Pretty wild to think their voyages are now over 40 years old.
You missed the other two points.

We can all compile C to wasm now.

K is a db that loses to Clickhouse and isn’t resilient. J is cleaner syntax, and with no crushing commercial license.

Show us something interesting.
IllusionMage – Rebranded Blender(?)
Not disputing that, and I did the same. If you want to make 400k+ as an IC you won’t be able to continue though
> If you have fast nvme, I hear that you can go fairly crazy with swap sizes and have a system that works well under memory pressure, but I have not tried it myself

Not sure about Linux, but under macOS I had a bug that was leaking uncompressed 12 MP photos in a loop. I was browsing the web while waiting for my process to run and I got up to something like 150 GB of swap used (macOS uses dynamically growing swap files) before I even noticed that there was something going on.
Nowadays similar things get out to the public and bored internet people pick them apart.

The best scams these days are hidden under NDA so you never hear of them (one hired someone to disprove maxwell’s equation as that was needed to run their over unity device. So he developed tests that could show if maxwell was right or wrong and collected the paycheck).
Knowing this is reselling Blender under a different name (though apparently bundled with video course), I thought the web design was pretty wacky
Sierra Club is on a politically motivated posturing mission to cancel Muir. For all the claimed transgressions, this man did a massive amount of positive work for naturalism.

We should remember people for their good and their bad, not wipe them from the history books.
This is still subject to Landauer's principle, at best it will cost kT ln2 energy per bit erased.
This is actually a current issue that's being discussed because the transition fully relies on the miners to make hand off the transition because of the contention with moving to PoS. One possibility is that miners will fork the chain, although that's not necessarily a huge concern because the community will follow the PoS chain. Another possibility, and I'm a bit fuzzy on the details, but it has to do with them re-mining the same block over and over so the transition block never happens (or something like that).
You don't need it. Linux works fine without swap, given ample memory.

In my experience, even with ample swap, when Linux gets memory starved it goes haywire anyways.

I'm an ex linux sysadmin, and haven't run swap on my personal systems in years.
But your genetic disorder shouldn’t be extended to negatively impact others, knowing your unique situation means you need to manage your risks. It’s entirely unfair and a violation of privacy for people to have to be brought into your health decisions and vice versa. Why would it ever be reasonable to expect the government to keep tabs on random strangers just because I have a genetic health condition?

I would much rather die than force someone else to suddenly be required to base their free agency around what happens to work best for me, especially because I am 100% going to die no matter what.

It just seems totally illogical to remove freedoms at scale to prevent something that is ultimately inevitable.

Basically seems like throwing the baby out with the bath water, what’s the point of having a society at all if people have to be micro-managed to the point of being enslaved to each other’s mortality? It just seems so obviously a mistake and a huge slippery slope.

What’s next, they tell us which people have to die so the rest of us don’t have to be burdened with them? That is essentially exactly what this type of government/healthcare overreach is doing.
I didn’t get MB for self driving capability. I got Mercedes because when there’s a serious accident, I have a very low risk of death. With Tesla it may be low, but at the time there wasn’t enough data to be sure. Tesla could have been the safest or the car version of the 737 Max. For the sake of my family’s safety I made the conservative choice.
I'm open about some of the "bad" and potentially stigmatizing details about my life in part to give push back against that tendency.

I did well in school. I have no criminal record. I'm a decent person and not a drug addict, etc.

And bad things happened to me anyway.

One of them was homelessness and I like to think that talking a lot on HN, where lots of developers and the like hang, about the importance of my cheap tablet and how crucial it was to make my life work while I was homeless helped foster positive changes.

I can't prove that, of course, but it seems a not unreasonable inference.

If only I could find a way to more generally change the focus from big feels and pity and finger pointing to practical details like "Yes, but how do I legally earn a few bucks now while homeless (or whatever)? And how do I stay sane and healthy while trying to fix my problems and get my life back?"

Whining on HN about "I'm on a tablet and can't do X, dagnabbit" used to be a thing I did a lot and then a lot of those issues disappeared. And I wish the world did a better job more generally of hearing "I need a solution for these various practical details -- today, without first having to fix some other problem, like getting back into housing. I need to be able to do X whether I'm housed or not (as one example)."

Often, focusing on othering people boils down to justifying creating problems for them unnecessarily on top of their actual problems. I hate it.
Comparing an island nation of 5 million people, that can effectively close it's borders and "avoid" new focuses of infection, to countries of 100+ million of people, where the difficulties of border closing are effectively unsurmountable, is extremely disingenuous. The whole approach governments took in facing this pandemic needs to be studied in the next decades. It's an eye opening thing to see how viruses spread pretty much the way they do, and there is almost nothing we can do to stop it. It's really humbling to see that if you keep everything open (i.e. Florida) you get the same results as if you make your population effectively stop to live for one year (i.e. California).
There’s really no harm in doing that, and it’s still a pretty good idea.

I generally try and get my data sources as far as possible with the database, then leave framework/language specific things to the last step, means that-if nothing else-someone else picking up your dataset in a different language/framework toolset doesn’t need to pick up yours as a dependency, and you’re not spending time re-implementing what a database can already do (and can do more portably).
In case you’re not aware, Chomsky IS a giant historical figure. He’s a founder of modern linguistics and cognitive science, a renowned analytical philosopher, and a major political activist of the 20th American landscape. His academic and political work is massive.
I used the population of that age group as the denominator.

The numbers of infected by age group are not as easy to find, and perhaps not so accurate (confirmed vs non-confirmed, etc).

I've approached this numerically since numbers started coming out of NYC last spring. It's just the way I've been comfortable estimating/qualifying the risk.
I wonder how Linux with that compares to XNU (iOS)?

For some reason (maybe because I work with Android) I always thought that XNU is better at memory management (fewer low memory kills, better paging, etc.).
Beautifully worded.
Syntactic sugar for creating JSON Type Definitions
I think Feynman illustrates the failure of the Mechanical Philosophy very well, but that is not incompatible with what Chomsky is saying here. In physics, you can model action at a distance with waves, fields and so on, which are part of a model. An intelligible theory means a genuine explanation, of the kind that Feynman refers to, which he admits is a difficult or impossible problem. We have abandoned a genuine explanation of nature that is intuitive to us (i.e. a mechanical view of the world), and now use models to explain the natural world.

Models of physics for example include concepts like waves and forces, which we don't observe directly, but build into the model because they make concepts like gravitation intelligible to us. If gravitation did not involve action at a distance (e.g. a universe with aether), physicists probably wouldn't even study it (the most interesting things to study are usually things that only occur in controlled conditions, that do not conform to our [mechanical] intuitions).

That we can't form intelligible theories of concepts like gravitation without these models, is not really surprising, since we did not evolve to study quantum physics or organic chemistry (could not possibly be selected for by evolution). Instead our science-forming cognitive capacities, which determine what counts as a genuine explanation to us, are shaped/limited by our evolutionary heritage, which is trained on the kinds of phenomena that animals like us normally encounter. [edited]
The day is rapidly approaching where we'll have to choose between the desire to blame and the desire to save lives.

Fairly soon autopilot will be safer than new drivers and the elderly who are aging out of the ability to drive safely. I hope we have the political will to potentially save my parents' and child's life by allowing them to have an AI driver even if it's not 100% safe.
If you reach for a button and it's not exactly where your finger lands, you're now distracted by figuring out where it is and/or why something unexpected happened. Doesn't help that some buttons physically stick and reluctantly disengage. Half the buttons allegedly do something, but they're useless or pointless. Some have indicator lights one must have reason to look at before noticing (oh, the "dual heat mode" got enabled? is that why the HVAC has been fighting itself and can't decide whether to heat or cool? defrost is stuck on, and wife is freaking out that it's too hot/cold/something in here and doesn't understand how to stop it?)

My car's "infotainment" system is so bad I've given up on it. When working with a phone, the sound is good - but rattling thru menus via buttons on multiple surfaces, never quite knowing what some of the terms mean (because I'm focused on the road, not on sorting out "MEDIA SOURCE" vs "INPUT SOURCE" in context of connecting Bluetooth or whatever), and having to literally alternate between sound unit face and the "OK" button on the steering wheel, well just forget it we're using the phone's own speakers (relatively lousy but they work). There's a speech recognition system in there too, but that goes nowhere (first, memorize all the possible commands it recognizes; next, say the phrases just right in a noisy & distracting environment...).

Yeah, physical buttons/knobs are distracting too. At least a touchscreen system can better present/organize information, and likely comes with a viable speech recognition system. And can sanely interface to a phone via Bluetooth.
I shouldn't have to tell people I have a genetic disorder to have them respect my desire to keep their damn hands off of me.

Expecting random strangers to not touch me is not some huge invasive expectation. Quite the contrary: The people who think they are entitled to touch me are being invasive and making an imposition. People should not feel that because I'm poor or an attractive woman or whatever that touching me is appropriate behavior when we are total strangers.

That's just one example and I don't intend to discuss this further with you.
> Deplatforming solves a problem.

Yep!

> IMO it makes things even worse since these groups will congregate in secret much further away from the public eye where they can perpetuate unbalanced and unhealthy ideas.

The whole point of deplatforming is to prevent those stupid ideas from having a general audience. Sunlight is not always the best disinfectant.
If you click through to the detailed benchmarks page (https://h2oai.github.io/db-benchmark/). A lot of them are that it's running out of memory, a few of them are features that haven't been implemented yet.

Inefficient use of memory is a problem I've seen with several projects that focus on scale out. All else being equal, they tend to use a lot more memory. This happens for various reasons, but a lot of it is the simple fact that all the mechanisms you need to support distributed computing, and make it reliable, add a lot of overhead.
Problem is when you can't get the spare/replacement parts because only authorized repair centers can get them directly through the device manufacturer who has them exclusively produced by a chip manufacturer.

Everybody buying Apple products is supporting such practice.
I would recommend W-TinyLFU [1]. It offers higher hit rates, is adaptive, has lower metadata overhead, is O(1), and is the only policy that is robust against all workload patterns we have tried. This is what I proposed to the Linux folks when analyzing DClock.

I met Kirk McKusick (BSD fame) at Fast'20 where he expressed interest in that for ZFS. We tried introducing the idea to the contributors at FreeBSD, Delphix, and LLNL. There is frustration at ARC being memory hungry and the locking is expensive. Unfortunately while there is interest in replacing ARC, all of our contacts were too busy to explore that option.

[1] https://github.com/ben-manes/caffeine/wiki/Efficiency
Not that I am a heavy DataFrame user, but I have felt more at home with the comparatively light-weight TypeTables [1]. My understanding is that the rather complicated DataFrame ecosystem in Julia [2] mostly stems from whether tables should be immutable and/or typed. As far as I am aware there has not been any major push at the compiler level to speed up untyped code yet – although there should be plenty of room for improvements – which I suspect would benefit DataFrames greatly.

[1]: https://github.com/JuliaData/TypedTables.jl

[2]: https://typedtables.juliadata.org/stable/man/table/#datafram...
> Without taxes there are no backstops for me to fall back on (like a robust transit system). To me, that's more financial insecurity, not more.

Agreed I meant tax residents of Toronto and Vancouver more and use the money to subsidize other regions.
I can't pay my mortgage with stablecoins, though. They also aren't an on ramp.
I started using Julia in December, DataFrames are in a sort of weird place because they're so much less necessary compared to e.g. Python. In Julia, you could just use a dict of arrays and get most of the benefits, thanks to libraries like Query.jl and Tables.jl. Thus the ecosystem is a lot more spread out. I actually use DataFrames much less than I used to in Python.

This is mostly good, because you can apply the same operations on DataFrames, Streams, Time Series data, Differential Equations Results, etc., but it does mean that some of the specialized optimizations haven't made it into DataFrames.jl
> You see the narrative today being that whites don’t deserve what they have.

I challenge you to find one post (with positive score) on HN with such a narrative. And besides, it's incredibly strange how you shoehorn race into this discussion.

> entirely hiding UBI’s drawbacks when society is no longer as well off (e.g. during wars)

So we should not have UBI during good times because if the nation collapses it would not work? I can come up with plenty of things that don't work when society is no longer well off, but it's unclear what bearing that has on whether they are solving problems in the present day...
There's a massive difference between contagious diseases with exponential growth potential and the other freedoms you talk about.

For a closer analogy than "what people are wearing or eating", what about "what people drive and how fast in public areas".

The government introduced speed limits, regulations on vehicles, and driver licenses, all of which significantly limit who can drive what, and how fast. All of this is enforced by police.

Why is it fair for the government to do that? Because dangerous driving kills other people too, not just you. It's not a personal choice in a vacuum to drive drunk at 100mph, but a personal choice that kills others.

Covid is similar. Limits on gatherings, requirements to wear masks, etc are all because personal decisions also affect others at a larger scale. A pandemic, when it's not controlled, will spread exponentially and cause significant death and health problems.

I think there might be a reasonable discussion to be had about the role of the government in this, but that reasonable discussion does not start with claims that the government is treating us like cattle or that "but people could die" is not a valid justification for government action.

The chance of people dying is a valid justification. I think it was a valid justification for requiring a driver license, for making drunk driving illegal, and I think it's probably a valid justification for the current restrictions and regulations. You can apply this similarly to some of your other points: a tiny group of experts determine what standards make cars street-legal in the US and decide that drunk driving is dangerous, and their decisions impact everyone, etc.