skysight-inc/HackerNews huggingface dataset:
Show HN: Denormalized – Embeddable Stream Processing in Rust and DataFusion
tl;dr we built an embeddable stream processing engine in Rust using apache DataFusion, check us out at https://github.com/probably-nothing-labs/denormalized

Hey HN,

We’d like to showcase a very early version of our embeddable stream processing engine called Denormalized. The rise of DuckDB has abundantly made it clear that even for many workloads of Terabyte scale, a single node system outshines the distributed query engines of previous generation such as Spark, Snowflake etc in terms of both performance and cost.

Now a lot of workloads DuckDB is used for were normally considered to be “big data” in the previous generation, but no more. In the context of streaming especially, this problem is more acute. A streaming system is designed to incrementally process large amounts of data over a period of time. Even on the upper end of scale, productionized use-cases of stream processing are rarely performing compute on more than tens of gigabytes of data at a given time.

Even so, the standard stream processing solutions such as Flink involve spinning up a distributed JVM cluster to even compute against the simplest of event streams. To that end, we’re building Denormalized designed to be embeddable in your applications and scale up to hundreds of thousands of events per second with a Flink-like dataflow API. While we currently only support Rust, we have plans for Python and Typescript bindings soon.

We’re built atop DataFusion and the Arrow ecosystems and currently support streaming joins as well as windowed aggregations on Kafka topics.

Please check out out repo at: https://github.com/probably-nothing-labs/denormalized

We’d love to hear your feedback.
> What do you find disatisfactory about the conventional explanation that the Carolina bays are thermokarsts?

For one, the bays don't look like thermokarsts, and for another the bays look like they "aim" at the impact sites, and they seem to form perimeters centered on the impact sites. That's highly suggestive.

> This isn't a scientific observation, but there's something fishy with this hypothesis. It's being popularized by conspiracy theorists, and the Comet Research Group, some of it's proponents, were busted for faking evidence.

I've no idea who they are. None. Is Zamora one of them? Can you tell me more about them? Everything I've seen is that he is very careful in what he says, and I've seen nothing from him about advanced civilizations.

> Meanwhile the alternative explanation that the AMOC was choked out by freshwater melting off the glaciers is very tidy

I agree with this. But that doesn't explain the Carolina bays, and the two things can have happened close enough in time, possibly for the same root reasons (impacts).
They didn't, but then they did, to the satisfaction of the Viacom case. Napster and Megaupload never built the tools to comply to the degree that YouTube did. Maybe if they had, they would still exist.
Spotted: GlobeOne app with ads promoting Trojan apps
GlobeOne is the official app from a big ISP (both fiber and mobile) in the Philippines. It is the recommended way to check or top up the balance, avail of promos such as data/call/text bundles, view the remaining minutes and gigabytes, and so on. On the main screen, it also shows ads, which were found to promote badware. Still, Globe would not remove the block with advertisements because of the revenue it generates.
An actual expert can fact check me but as far as I'm aware that's due to how magnesium sits within binding site within the NMDA receptor ion channel as a blocking / gating factor. If supplementation actually raises the level of available magnesium beyond the blood brain barrier (when you're perhaps in need of some) then it would affect the way in which that ion channel affects the downstream excitatory effects towards long term potentiation.
That's genius, thank you!! I will have to try that tomorrow but I have a feeling it might just work : - )
Note that magnesium carbonate is not particularly well absorbed either, up to about 30%, and is also used as a laxative (just like magnesium oxide).
It feels like a decade ago, but I remember the issue being with how they dealt with abuse reports. To save storage space, they matched similar files. So if two people ripped a DVD and compressed it, they would just keep one source file, and generate 2 different metadata files, to avoid wasting space. So they'd have different filenames and creation dates, but only take up half the space. Then, when one got an abuse report, they would delete that metadata, but as long as one still existed, the source data never got deleted. Law enforcement called it a conspiracy to commit crime, megaupload called it smart database deduping. It was usually much more than 2 copies, so content owners were playing endless whack—a-mole while megaupload was barely shuffling a few kilobytes around.
> The phenomenon that something which was previously working correctly, suddenly does not work correctly when one tries to demonstrate the operation to others.

the same word is also used for the reverse situation, i.e. if something that did not work suddenly works when trying to demonstrate the failure.

think bug reporting e.g. at the car mechanic, warranty claims, etc. if you try to show a reproducible bug to others, it suddenly vanishes...
Their small engines really are great. My local fire department uses Honda models for all the small forestry-style water pumps, for example.

You do have to watch out though, where some other brands have Honda-branded engines that are _not_ the same. My local small-engine mechanic derisively refers to some as "Chondas", basically meaning cheap, Chinese-built engines. When he serviced my Ryobi pressure washer, he'd had to hold the whole thing upside down to drain the old oil, because the "Honda" engine didn't come with an oil drain. Seems like quite the lack for any sort of internal combustion engine.
I just knew someone was going to take it seriously xD
They don't need to be reconciled, but they do differ, which suggests we might be able to find a framework which reconciles them.

Galois Spinoza Aquinas in Google yields (for me at least) the following HN thread: https://news.ycombinator.com/item?id=39885475 , in which an attempt at "Algebraic Theology" not only provides such a subsuming (thanks to Galois Theory) framework, answering that particular question in the affirmative, but also raises many other questions which might be amusing to pursue. (I could summarise those Q's in this thread, if any of you all are more interested than the downvoters were)

[I will summarise that thread further up in this one, but as it was months ago it may take me an hour or two to page everything back in]
> far worse and less valuable to society than the tech industry

The jury's still out on that.

The music industry can't hold a candle to some of the raw destruction that has been wrought by tech.

It's just balanced by a lot of good (and making tech billionaires isn't really what I consider "good").
Yeah but given that the default is apparently not to do that... kind of shows that these things are just a fun theoretical exercise rather than a serious attempt to solve a problem that actually exists.
Good point. There are multiple levels of "safety". I think glibc API does a decent job marking and indicating which is which:

https://sourceware.org/glibc/manual/2.40/html_node/POSIX-Saf...

Thread-Safe
Async-Signal-Safe
Async-Cancel-Safe (mostly limited to pthread_*cancel*)

There is a host of other "safety" remarks also useful to keep in mind: https://sourceware.org/glibc/manual/2.40/html_node/Other-Saf...
I wished they would bring these to the US but I don't think they are.
The craziest thing about Kim is that he held the FIRST PLACE in competitive Call of Duty at some point while running the company.
If only handloom operators hadn’t been replaced by steam looms we’d all be in a better place.
Creative work costs money to make. People who make it should have the right to make a living off it. It's not hard. Most of the people on this site make their money creating intellectual property. How many piracy activists here would be willing to leak the source code their company relies on?
It's fascinating to me how German composite words encapsulate an idea like "[failure attributed to] demonstration effect" in a way that just saying "demonstration effect" in English often fails to accomplish.

I also love how German often has useful opposites - schadenfreude vs fremdschamen, for example.

On that note, is there an opposite to this, where someone observes that something's not working, they call in the expert to diagnose the problem, and it functions perfectly under demonstration?
Problem is, this "need" for big platforms is entirely a matter of faith. It's like your "need" for supernatural bed-time stories that explain human suffering and make you feel cozy; they're just tall tales. You're taking this practice entirely on their word, your "proof" that this system works is a backwards understanding of determinism. The only realistic, grounded worldview is to conclude that we don't need fairytales, and that we repeat them as an axiomatic comfort and not a truth.

Similar problem pervades with Apple. They have no written obligation to protect you, they've been caught lying about security at the request of larger entities than them. They have no technical obligation to provide a secure experience; no GrapheneOS equivalent exists for iPhone, and Apple sues researchers that attempt to dissect the production build of iOS. They cooperate with governmental bodies worldwide and cannot fend-off state-sponsored attacks even when thousands of personnel are targeted. Default programs like iMessage, Facetime and iCloud Mail are all vulnerable to zero-click exploits. The App Store regularly approves trojan horse variety malware masquerading as brands people trust. Apple's platform security is a veritable trainwreck.

If you liken Apple's approach as the diametric opposite of Open Source security, then they are the perfect example of why this mindset fails. There are no half-measures with security, when someone locks your room without giving you the key it's not for your protection. Lord only knows people will argue that the cell door makes them feel safer though. Tall tales, and all that.
Fohr-fyur-effect [ˈfoːɐ̯fyːɐ̯ʔɛˌfɛkt]

Letter 'v' is called "fow" in German and is pronounced as sound "f".
22. 3 digitalocean 5 dollar instances and + backup storage. 2 vultr 2.50 instances and 4 free oracle boxes.

I could move everything to Oracle and pay nothing but I would rather pay digitalocean and know it won't be shutdown one day.
The ‘make thier own mental models’ vs sharing/providing full information is difficult.

‘Make thier own model’ of the domain can lead to deeper understanding but takes time and may lead to different (possibly incorrect) understanding of the issues and complexities. If not reviewed with others.

Providing full information upfront to a person can be quicker but lead to a superficial knowledge.

I think that it comes down to whether that deeper knowledge is directly needed for the main task. Can I get by with an superficial (leaky) abstraction and concentrate on the main job.
Like all corporations, Apple only cares about making a profit. Their business model let them implement much stronger privacy guarantees than their competitors, and this differentiation has been a profitable avenue for them. But ultimately if there’s any lock-down they can get away with (i.e. without affecting their profits either directly or via fines and legislation) they’ll try it, and privacy/security misfeatures are a great way to couch these regressions without appearing directly hostile.

That being said, I don’t think Apple’s privacy stance is all hot air, they publish plenty of white papers on their infrastructure. I mostly think they’re greedy and don’t mind harming the user and developer experience to ensure they get their cut from every transaction that occurs inside their ecosystem.
Sure, pirating means comandeering of a ship, not distribtion of copyrighted works.

He probably has a plan in a nearby Pacific jurisdiction like Vanuatu. Best of luck!
Settings > Editor > Default editing mode

Switch the dropdown from "Live preview" to "Source mode"
It was a joke, but thank you. The internet needs more helpful people.
Very cool! I've played diep.io plenty, so no comments on the gameplay haha, I'm familiar with the decisions made here, both the cool ones and the frustrating ones.

I did encounter a bug where out of nowhere I was jolted onto the bounds of the map (specifically the lower-left corner) and died, killed by "Unknown".

I think thse bugs in this kind of game are especially scary because you lose everything when you die, which makes you a little weary of starting again. You might get far just to randomly die for no reason. If those are ironed out, I think you've got yourself a pretty good .io game to build upon and keep evolving here!
The thing where backspace magically reformats half your document is so frustrating. I wonder if Word's solved it as it's been decades since I've used that, but it seems like pretty much anything else I use still has that issue.
That tracks with the fact that they use a .nz now.
Static CPU manager also affects the cores that sched_get affinity returns. And that’s what Go uses to obtain the core count.
Leibniz does get a mention, but it'll already take a while to explain these two, so I'll leave his full reconciliation as an exercise for the reader ;)
Yes it is.

If the content is a movie created in the last 100 years, it was almost certainly created to slot into that revenue model.

The artists have a right to sell their property on their terms. And if they decide to do so by selling their rights to a studio, then that's how it is. And if you don't like corporations, contracts, or the revenue model, then that's completely irrelevant to the parties involved.
Yeah no thanks, I'll stick with Postgres
Did you this week's episode of Warrior's Whiteboard Wednesday where I focused on understanding one possible reason the different staff weapon lengths were developed by our ancient Japanese forefathers? That was the topic, as well as using that same Master's logic for adapting our training today, WITHOUT changing the principles and concepts at all!
I appreciate your candid feedback. Our aim isn't to push for replacing humans but to ensure that when companies do use LLMs, they work as intended and don't create more problems. We'd rather see well-functioning systems than glitchy ones that frustrate everyone involved!
A civilization as advanced as ours would have to have bootstrapped via coal mining and burning like we did, and that would leave evidence in the CO2 records in the ice cores. There does not seem to be any evidence of such an advanced civilization in the ice cores, unless it wasn't remotely global in scope. There could have been less advanced (but still advanced) not-global-in-scope civilizations that we don't yet know about, but I don't see the link to the Carolina bays, unless it's to say that the same event wiped them out, but so what, I'm interested in the Carolina bays and their cause, and much less interested in speculation about advanced civilizations.
Thanks! I had thought it would make sense to post this chapter of the Reticulum manual separately, as the other post is about Nomad in particular. But since my post here didn't get traction, I repeated my question over there now.
So it stops being a blockchain if the criteria for adding a block is based on something else? Or do you intend to update your definition to incorporate other consensus mechanisms as they emerge?

Seems to me that a more useful definition would abstract out the consensus model such that a blockchain is essentially a merkle-linked-list together with some function for determining which of two candidate next-blocks will be the actual one, but without getting too specific for what that function is... just because there's so much potential for variation there.
Hermes 3
Previous discussion:
https://news.ycombinator.com/item?id=21170732
Well yeah of course, but there are going to be more bugs and they can't roll out fixes instantly.
Let's goooo
Anyone here migrated to TiDB from cockroach and can share experience? Asking for a friend…
Having something to lose, makes it a lot more likely to bring you to reason.

It takes seconds to destroy what it took decades to build.

If someone is pointing a mortar at your house, you want to get rid of the mortar, but you also want to keep your house, so it's likely that you will look for ways to remove the mortar, that don't include it being fired.
We call it a "numbers station" right now.

Honestly LLMs might make for an interesting means of stenography. Encoding encrypted data in vectors that can be rendered to coherent statements and back.
> YouTube went above and beyond to comply with the law

History just gets rewritten daily.

They took probably 10 years to attempt compliance at all. What's more, Google Video never had any organic participation (i.e. normal people uploading videos of themselves), and was almost exclusively pirated content. Its main differentiation (long forgotten in the age of youtube-dl) was actually how easy it was to download that content compared to Youtube, who made it annoying. Eventually Google realized that they still weren't going to attract the pirates/copyright violators that section 230 allowed them to use as a proxy (piracy still preferred Youtube because people were on youtube), so they bought it.

Youtube was absolutely loaded with copyrighted material, and the source of lots of pirated files still being traded is directly from youtube. Eventually they started aggressively scanning things for copyrighted music (because they wanted to make deals with the music industry), and then started preemptively responding to any DMCA claim by suspending the video so as not to look like hypocrites while they were going after music; section 230 implies a lot of helplessness for platforms in the face of users that removing audio tracks from videos where people were singing copyrighted songs doesn't bear.

They started getting rid of pirated (and amateur content in general) once they had the monopoly on video, not before. Now they wanted to push exclusive, expensively-produced content, and since producers didn't have any other online outlet, they were going to monopolize that, too. They didn't need the pirated content anymore.
I'm mean ignoring single newlines that you enter, so it's wrapping less, not more
Building an intermittent reward system that motivates you
How to nuke $9B in wrapped Bitcoin
Imo the game is clearly not intended to be about forced 1:1’s. The majority of the tanks seem to be optimized for just free for alls.

For one on ones you’d be pretty dumb not to prioritize front facing guns
Company: We just made $10.3B in profits

Stockholders: How do we make it $11.3B in profits?

Company: Layoffs!!
Well, the idea behind this product is to make sure the LLMs they replace/augment workers with aren't total dogshit (at least relative to the workers they're replacing).

But also remember -- the point of the economy is not jobs. It's value creation. If we can create the same or greater value with fewer people working/people working less, that's a great result! And it's the result we've seen continue over the last century and a half, even while people became much wealthier (because they were choosing to exchange some of that wealth for less working time).
Why can't Americans ditch checks? (2017)
Correct: I misread the thread. Thanks!
Moral of the story- ignore the law for as long as you can and comply hard enough when you have to.
It helps to have zirp levels of investment too.
No, it's impossible, even in principle, to answer that question. You can draw a picture for either answer.

"Left" and "right" are a dipole. Neither one can exist without the other, and they are symmetric. It's the same issue as we have with the conjugates discussed in Galois Theory.

In fact, in an algebraic (non-ordered/arithmetic/analytic) perspective, it's misleading to use the symbols + and - to label the conjugates in field extensions like sqrt2 and i. Left and Right are better names than + and - for those conjugate pairs. Only when we impose an arithmetic ordering (which is not needed in the theory of algebraic equalities) is it meaningful to use + and -: -sqrt(x) < 0 < +sqrt(x), where x is a positive real number.

(and when x is a negative real number, we immediately see the problem with - again: -i and i are not separable via ordering with respect to 0.)
I'm having trouble parsing/making sense of this. Was basic backup in Core? If you were running anything more than Core you weren't running an OSS version and had already crossed that line before this announcement. If you were running an OSS version there's nothing to add, just fork, no?
AltStore PAL drops its annual subscription thanks to a grant from Epic
When people say "bring this cobol code to a new language to improve maintainability", they don't just mean the syntax. Any C developer can learn cobol syntax in 10 minutes. They mean things like use functions instead of gotos, don't use just global variables, don't depend on lots of weird tooling from the parallel world of IBM mainframes, with most of your logic hidden in weird batch scripts with names like ABC@@DEF.

I could write a cobol to c translator in a weekend. Nobody would buy it. Source: I've spent the last year and a half consulting on a huge project to rewrite a cobol codebase.
What profanity that starts with M, ends with G and references sex comes to mind?

We’re an adhocracy that uses votes, and profanity doesn’t win votes…simple math…
It still shows the market value of browsers, even if unrealized or illegal this value could be exploited differently
Regulatory capture at work. The FDA doesn't work for the tax payer anymore.

Our regs would look a lot like Euro regs if they did. Glyphosate, BVO, titanium dioxide (taste the rainbow!), all artificial transfats, hydrogenated soybean and cottonseed oils, ractopamine (pig feed), pink slime, yellow 5, yellow 6, red 4,5,6, and 40, rBGH / rBST (growth hormone for cows), chlorine baths for chicken, atrazine (have some birth defects with your corn)... and the list goes on.

All of those things are used in U.S. foodstuffs, which is why so many U.S. food exports are banned in Europe and elsewhere. They're not good enough, or actively harmful. We have food science on these items going back many decades, but Monstanto (now Bayer?) and Dow Chemical throw millions of dollars around to ensure their chemicals still get used on U.S. crops.

CA law can be overly picky about a lot of things, but health and safety is one area where they're the good kind of picky.
> Cisco spokesperson Robyn Blum said in an email to SFGATE that the layoff is meant to allow the company to invest in “key growth opportunities and drive more efficiency in our business.”

Human Labor is no longer a "key growth opportunity" it's looking like?
That’s… not a senior engineer. Whatever his title may have been.
Or War, either works
I agree, and in fact I did not compare them. I asked an entirely different question.

You can re-read the first line of my comment if you think I'm putting those two things on the same level, and you will see that I agree with:

> He was a commercial opportunist, not a real activist or whistleblower.
> Your orders are not sent to the best exchange execution possible

Look up NBBO, that would be illegal. You're getting better execution as a retail trader thanks to PFOF.
Yeah, that bug is from a memory leak, I haven't had a chance to debug it since I'm traveling but I'll figure out the cause by this week.
How about his claim that AI companies should just steal everyone's IP then pay the lawyers to clean up the mess after the fact?