skysight-inc/HackerNews huggingface dataset:
I am a very avid proponent of rust. however, here are a few places I have had difficulty in working on custom storage engines in rust:

- uninitialized memory: it is tricky to get the semantics of uninitialized memory right. the ergonomics of the `MaybeUninit` api are frankly terrible.

- memory alignment: for O_DIRECT and other cases where memory alignment is important, it is difficult to ensure that the backing memory of Vec and other datatypes is correctly aligned, which ends up pushing you towards raw pointers.

- mmap: after considerable research, it is unclear to me whether there is a safe rust api to mmap.

- hostility to unsafe: in general, rust is easy to learn (relative to C++). however, the hostility in the community to unsafe (there are some good reasons for this, not criticizing it in general), makes it more difficult for someone without a background in C/C++ to learn how to use unsafe correctly. feels like if you ask a question about how to do unsafe you get 100 people telling you what a terrible idea that is, but for database code there is very significant performance at stake.
Anyone understand Chinese read the source article at the end? The source article clearly says this is not the case. Whether it’s true or not, I don’t like a report that changes the information from its source.
Well... we have 3 node MongoDB cluster and are processing up to a million trades... per second. And a trade is way more complex than a chat message. Has tens to hundreds of fields, may require enriching with data from multiple external services and then requires to be stored, be searchable with unknown, arbitrary bitemporal queries and may need multiple downstream systems to be notified depending on a lot of factors when it is modified.

All this happens on the aforementioned MongoDB cluster and just two server nodes. And the two server nodes are really only for redundancy, a single node easily fits the load.

What I want to say is:

-- processing a hundred million simple transactions per day is nothing difficult on modern hardware.

-- modern servers have stupendous potential to process transactions which is 99.99% wasted by "modern" application stacks,

-- if you are willing to spend a little bit of learning effort, it is easily possible to run millions of non trivial transactions per second on a single server,

-- most databases (even as bad as MongoDB is) have a potential to handle much more load than people think they can. You just need to kind of understand how it works and what its strengths are and play into rather than against them.

And if you think we are running Rust on bare metal and some super large servers -- you would be wrong. It is a normal Java reactive application running on OpenJDK on an 8 core server with couple hundred GB of memory. And the last time I needed to look at the profiler was about a year ago.
Old iphones become faster if you reboot them, like most devices. This is true whether or not you change the region. There's no qualitative difference when you test repeatedly.

https://knowtechie.com/want-to-speed-up-your-old-iphone-try-...
Level 4 (e.g. highways in good weather) is something that people buying cars will pay a significant amount for as an option in good weather. It does nothing for those who want robotaxis. But the former don't care about the latter.

You have multiple groups of drivers including those who appreciate self-driving assistance on long boring highway stretches and may not even drive in cities that much and those who never want to get behind a wheel or perhaps ever own a car.
You are right that ordinary people were not defrauded directly, but pension funds are basically ordinary people's money. There just happened to be a middleman (the pension fund admin.)

Plenty of pension funds lost plenty of money
> I've been using an Ooni pizza oven (gas) weekly

Which version of it do you own? Have you found it useful for anything else? They seem to advertise cooking steak on it... Which I am skeptical of compared to a pan.
Yes I would say the ones I typically get are too (in the UK). The colouring is a bit off there - I was just glad to find one showing the consistency roughly.
If I give you a map of kid -> key material that looks like this:

{
"my-super-cool-key-id": "some hs256 secret key string goes here",
"another-key-id": "-----BEGIN RSA PUBLIC KEY-----\n ... snip ...",
"yet-another-key-id": "----BEGIN EC PUBLIC KEY-----\n... snip ..."
}

And let's say you've got three different API endpoints, which each hard-code a specific algorithm (one HS256, one PS256, one ES256). But, because of the framework you're developing in, you're expected to provide a single configuration object containing a map of key IDs used by the entire application. (This is a common framework quirk.)

What stops you from swapping out the kid in a JWT's header and getting the underlying library to use the wrong key type for the endpoint that accepts HS256?

{
"alg": "HS256",
- "kid": "my-super-cool-key-id",
+ "kid": "yet-another-key-id",
"typ": "JWT"
}

The answer varies per implementation. The JWT standards do not call this misuse potential out at all.

Strictly listing the keys does not, at all, hard-code the algorithm those keys are used with in every possible programming language and runtime.

Some languages (Java) accidentally prevent this through type safety in the low-level crypto APIs. Others accidentally prevent this by not supporting the kid header.

I have yet to see a JWT library that deliberately prevents this misuse potential. Is that the library's fault? Or their defaults' fault?

EDIT:

The fix, by the way, requires doing this:

{
"my-super-cool-key-id": {
"alg": "HS256",
"key": "some hs256 secret key string goes here"
},
"another-key-id": {
"alg": "PS256",
"key": "-----BEGIN RSA PUBLIC KEY-----\n ... snip ..."
},
"yet-another-key-id": {
"alg": "ES256",
"key": "----BEGIN EC PUBLIC KEY-----\n... snip ..."
}
}

And then verifying that the alg for the key matches the alg for the token before attempting to verify the signature/MAC.
> Personally, as an SF resident I would much prefer all the cars to be tunneled or elevated instead of idling in front of my house or blowing loudly through my block. It's a safety issue.

This is a false choice though. It would be better to design the city in such a way that we don't need personal automobiles for most trips. Building high-speed roads (elevated or not) through a city tends to have the opposite effect. If you live in SF, just think of the parts of the city that do have high speed roads. Is it pleasant to walk along division street? People choose their mode of transit based on what feels safe and convenient to them.

In the space and money taken up by an elevated highway, we could have low-speed mixed-use streets and an entire separate highway for bikes, and it would be safer and quieter.
Yes TV channels are being replaced by apps.

I think smart TVs are great, the chips that they put in the €1000 models are good enough for 4k streaming now. No more cheap mediatek crap.
The only remaining problem that needs to be solved is long term support.
Don't be disingenuous.
Thank you so much for your incredible work on FastAPI! This looks like another really cool project, can't wait to try it out.
This is trivial to overcome with a VPN. There's nothing that suggests that he was connecting from the same IP each time. Based on the article, he could have just easily forgot to sign in to a VPN before logging in and that mistake unraveled the whole thing.
Good comparison.

What if Bitcoin continues as it is, which is a way to preserve a part of wealth in a semi-liquid way. Once it hits an equilibrium point, it doesn't go any higher due to the limit in it's liquidity.
Thats not Googlespeak. Thats legal protection. Whenever a company is open to investigation you can bet emails will be searched and if something suggesting something borderline illegal is written there it can be used against you. A lot of companiea give such trainings to their employees.
California and Quebec carbon allowances markets are now linked (trading as CCA), and Washington State is expected to join. I researched that market thoroughly and it’s not perfect but very well structured. It’s mostly carbon allowances, not offsets (offsets don’t really reduce total emissions). CCAs trade on the ICE exchange and there are regular scheduled auctions. Arguably more liquid than most stocks.
To master swimming does not mean to master the ocean.
currently #21
TikTok adds in-app shopping, in a partnership with Shopify
That’s true. For me, I made a dataset for each user as a home directory. Each home directory was then exported over NFS.

In order to automate the export, I have a single script that can be executed with sudo on that system to manage both the creation of the dataset and the export.

So, there are still ways, if you’re able to use sudo.
Can the DEX's work even if they are outlawed in the future?
We buy as much or as little of the loan book that makes sense for both the customer and Level. So more on the individual loan level.

We will definitely reach out in the coming weeks.
I'm a researcher at a commercially funded research laboratory. A bit like Draper, but different organization.
Lidar isn’t expensive.

You can easily get the cost savings elsewhere in the vehicle or just target the premium market.
At many companies anything is a SPA now, which seems like the worst example of cargo cult mentality. It takes twice as long to develop, but with all these people who only know JavaScript you have to give them something to do.
It was not rhetorical. I assumed they might have planned for a situation where the warehouse or Samsung made a mistake.
Yeah but cash you have to physically transfer and exchange, which places you at a place at a point in time. Which is the ultimate "trace".
I don't know, but I've had performance management disabled for about a year on my 1st gen SE. After switching to France as my region, my phone died at 15% for the first time in a long while so there seems to be some difference.
Sure, but how? If there is an effective path to higher dimensional thinking and expression of code (beyond text, even structured text) to date it has eluded us. We need breakthroughs in the science of software.
There was a dialog window with a error message and a [Continue] button which did nothing.
Totino's has some good news for you
I have to confess that this EWD on 0-1 indexing is one of my pet peeves. Whenever the topic comes up, someone will invariably link to it uncritically, even though Djikstra's argument is biased in favor of 0-based indexing.

For example, he talks about how (a < i ≤ b) is unnatural, but then fails to mention that this happens with 0-based indexing when we have to iterate backwards.
The article leads with "Waymo already solved the self-driving problem."

I don't dispute all their points but I sure disagree with that one.
Probably yes. I think it's sold here (UK) under that name too - for some reason (not sure if justified) I imagine that being a jar rather than a tube.

But anyway, yes, a tomato by any name ~would smell as sweet~ reduced that much to a sort of jam-like consistency.

If anything it's too thick for pizza - but oil's going on anyway right, so I just mix it with a bit (and usually some chopped chillies) and it's perfectly spreadable without being too saucy.
It's not production-ready and isn't a GC'd language, but Zig's approach might be to your taste:

> Zig programmers must manage their own memory, and must handle memory allocation failure.

> This is true of the Zig Standard Library as well. Any functions that need to allocate memory accept an allocator parameter. As a result, the Zig Standard Library can be used even for the freestanding target.

https://ziglang.org/learn/overview/#manual-memory-management
For comparison, on Mac when the battery starts dying, you get a persistent warning in the menu bar battery indicator. Hard to miss it every day.

On the iPhone, the one-time alert with cryptic language will just be dismissed by most users without a second thought. Doesn't matter that the capability to turn this "feature" off is buried somewhere in the settings when most people don't know about it.

Apple knows how to design good UI. If they wanted to prompt people to replace batteries, they would find a good way to do that. But they much prefer people buying new phones instead.
Why not have a remote driver? Like call centers but for driving?
Sure, but I think the point is that it's more likely that the idea of an internet-connected TV originally came from the perceived usefulness of having a TV that could come preloaded with various apps (Netflix, etc.). Certainly I think that's a valuable feature to have.

But of course now that the TVs have an always-on internet connection, the manufacturers have realized they can extract more revenue (recurring revenue, at that) by spying on their customers and selling the data. So of course, they do it, and most people don't even know they're doing it, so there's little outrage from the mainstream.

> there is a definite profit motive to push smart TVs and only smart TVs

I would imagine that most customers would prefer a TV with Netflix (or whatever) than a TV without Netflix. At that point it becomes a decision of whether it's worth it to incur the cost of having a separate SKU without the "smart" bits, and it may not be.

Yes, the "smartness" benefits the manufacturer (to the detriment of customer privacy), but I would bet that even if all the sketchy bits were banned tomorrow, manufacturers would still make and sell about the same number of smart TVs.
bwa ha ha ha ha ha ha ha ha ha ha <breathe> bwa ha ha ha ha ha ha ha ha ha
That is because the root problem is excessive copyright terms lasting 100+ years. Bump that down to 10 or even 20 years and you will see a market emerge for the abilities you are looking for.
Financially.... what a way of boomerlight the youth. Of course , he is a 75 year old tenured professor.
if you create a product thats unique or defines a category you start by having a market share of 100 percent. Market share does not define monopolies. What defines monopolies is the absence of alternatives. Like you know, the postal service to send letters. funnily nobody talks about those.
No rational person is going to see “Area” in this context as anything but a synonym for “Market.”
You get more of the behaviors you encourage and less of the ones you discourage. The US legal system strongly discourages retaining email, since it rarely works in favor of the entity making the retention decision. And there is no particular reason to keep emails that old, so naturally companies want to get rid of them.
Thats not really what this article is about. Their problem wasn't throughput. What's the size of all the data in your MongoDB instance? And what's the latency in your reads?

In the big data world the "complexity" of the data doesn't really mean much. It's just bytes.
Sounds interesting. Does this "we" have any writings about this?
I'm also fully vaccinated, but is that constructed fear? Aren't they talking about a third shot already? And isn't it already a green pass?
Ask HN: Do you enjoy or dislike anime?
For what reasons? When did your enjoyment/hatred of it start?
If I had to choose between a color TV that could only show live broadcasts or a B&W TV that could stream Netflix, etc., I would certainly choose B&W streaming. This obviously doesn’t require a smart TV—I personally use a Roku and rejected all of my Smart TV’s EULAs—but I consider streaming to be a much bigger objective improvement than color.
I was expecting some analysis of what you tweeted to the Google people that was outside their vocabulary.
That's the stuff. Add a bit of olive oil and it can be spread, and is IMO perfect for doing so over a pizza base. ('course you'd spread it round the other way, but it probably tastes the same.)
How does your test work with just pydantic basemodels?
I bring this up every time this is mentioned, but I really wish the API token format included a domain to notify in case of leaks. Having services register with your in-house secret scanning system works very well if you're GitHub, but otherwise it's a very closed mechanism.

If sendgrid tokens were `secret:sendgrid.com/91on9SIkbUfSs` instead of `SG.91on9SIkbUfSs`, or Amazon keys looked like `amazon.com/JGUIERHT` instead of `AKIAJGUIERHT`, we wouldn't need a database of regexes and endpoints to report secret leaks.

See also the last time I ranted about this: https://news.ycombinator.com/item?id=26568651
You have a fleet of self-driving cars at level 4 that have multiple cameras, and you claim it's going to fail because the system can't keep a street map updated? Does the author even understand how software and automation works?
Tesla does not want HD maps. They want to be able to place the car somewhere it's never seen and have it drive itself competently. That's by definition more robust than requiring HD maps to navigate anywhere.
https://soundcloud.com/knowable-magazine/sets/whistle-langua...
This is from May 2021, i.e. predates Delta.
Especially when Google is distributing documentation saying to use the word "Area" explicitly instead of "Market."
Smart TVs from at least the last 5 years have a similar problem. You update the software and they become excruciatingly slow, or you can't update the software and they gradually become useless.

Damned if you do, damned if you don't.

But I can replace an old Roku for $39.
Teal add type annotations and type checking to Lua, and transpiles to Lua, while Nelua is a new systems programming language with optional type annotations and compiles to C.

@ubertaco also made a good comparison in his comment with Teal.
I found myself agreeing with the cons mentioned in this article -- but, where is the insight? All of these challenges must have been known since before the company was formed. For example, people didn't suddenly learn yesterday that lidar has trouble in rain & snow. I suppose there may be value in stating all of the downsides in one place (as this article does).
References to Apple Inc. In Pixar Films
You just need to wear a mask and basic clothing. Sensors totally useless.
Ditto for YouTube - I remember being able to watch videos on it 10 years ago, with even older hardware and software, and the site was far more responsive than it is now.

Yet it seems if you tell the average web developer that those sites should be accessible with software and hardware from that era, or even older in the case of Twitter (rendering short snippets of text and some small images doesn't require the latest technology, seriously), they'll become confused or indignant, or try to use some absurd argument to justify it.

Twitter and YouTube are great examples of a lot of the disturbing trends in web development today. Whatever it is, it's not "progress"!

I don't "contribute" to either one, nor do I have an account; I just go there to read or watch what others have contributed.
SQL vs. NoSQL Is the Wrong Distinction
Could do with adding a key to the chart.
Globalists have destroyed confidence in science with their climate scam.
It is religion, not science.
>> It's astonishing how banks (saved with your money) can go home scotch-free, and hedge funds (that will die for their recklessness) get all the blame.

The rationale is (not saying I agree with it) that broker dealers' and banks' can cause systemic risk and cause a domino effect.

Note that one hedge fund was indeed saved -- AIG-FP, and was given a record $180 billion dollar bailout, albiet on initially punitive terms. Incidentally they were so large (1Trillion USD of exposures, much of it leveraged) that they fell under the category of systemic risk.

https://en.wikipedia.org/wiki/American_International_Group#2...

That $180B wasn't all though, one could argue that the QE buy-back program purchasing securities at arguably inflated prices was also an additional bailout, but the amount for that is difficult to quantify and difficult to segment beneficiaries for.
But today there would be no McDonald’s or little new construction going on had we not written the software their insurance companies are using.
It suffices to freeze the yeast. Lasts a very long time that way.
I second your recommendation of Metaphors we live by. It is a book that had a lasting impact on how I view the world and I think most HN readers would find it interesting.
Thank you for the tip. I will give it a try.
Why would the text data be stored in TOAST?

My understanding was PG only uses TOAST when the data is too large to fit in the row, and since PG compresses data before inserting wouldn't user messages be fine?
Where the hell does it say Google or any employer has the right to make up exclusions to the right to organize?

Protecting employee interests is a basic right, and if the employer could define the set of valid interests, it would completely negate all protections.

Quoting from the law that I believe this is based on:

"RIGHTS OF EMPLOYEES

Sec. 7. [§ 157.] Employees shall have the right to self-organization, to form, join, or assist labor organizations, to bargain collectively through representatives of their own choosing, and to engage in other concerted activities for the purpose of collective bargaining or other mutual aid or protection..."

https://www.nlrb.gov/guidance/key-reference-materials/nation...
This is a significant number of ATMs by Salvadoran standards.

Currently there are about 1700 bank ATMs in the country, so a 200 increase would be about 11% of the ATM network.That's the number of ATMs that I would expect from a biggish bank, so it's large scale by our standards.
I pulled an all-nighter tracking this down and found a glitch in the automated test suite and CI/CD pipeline. Fixed and pushed out to all of the AWS lambda instances.

We don't have a bug bounty program, sorry.
> The SIM-card used to transmit the eCall data is dormant, i.e. it is only activated in case the vehicle has a serious accident (e.g. the airbag is activated).

At least the EU cares about the privacy implications of some of the stuff it does. If/when this happens in the US, they'll just let the auto companies offset the cost by selling the data it collects. For them it's a win-win-win, the government looks good by improving safety to 90% of people, companies can make more money by selling data, and the government can buy that data to circumvent the legal restrictions on tracking citzens directly.

[1] https://ec.europa.eu/transport/themes/its/road/action_plan/e...
I find deep dives like this fascinating. I am absolutely the kind of person who could get this 'into' something if it were what I really wanted to be into, at least for a time, so even though I'll probably never put in the effort to implement Jeff Varasano's recipe, I respect it. I'm absolutely glued to reading this. It seems like his restaurants are successful; I wish him luck.

I will say that personally, I treat Good pizza, like this or a "tomato pie" from a great place, as a fundamentally different thing from "Bad" pizza, the kind your cheap local chain sells. (Fast food pizza from Domino's or whomever is just not something I normally consume.) There is a place in the world for "bad", bready pizza with crazy toppings and whatever else. It just isn't what I want when I want the Good stuff.
> What's the size of all the data in your MongoDB instance?

3x12TB

> In the big data world the "complexity" of the data doesn't really mean much.

Oh how wrong you are.

It is much easier to deal with data when the only thing you need to do is to just move it from A to B. Like "find who should see this message, make sure they see it".

It is much different when you have large, rich domain model that runs tens of thousands of business rules on incoming data and each entity can have very different processing depending on its state and the event that came.

I am writing whole applications just to data-mine our processing flow just to be able to understand a little bit of what is happening there.

At that traffic you can't even log anything for each of the transactions. You have to work indirectly through various metrics, etc.