skysight-inc/HackerNews huggingface dataset:
What I understood: they assume a hypothetical bipartite graph with text nodes (like hypothetical paragraphs / chunks of texts) on one side, and “skill” nodes on the other. The skills could be things like:

• ability to understand the word “because” [causality]

• ability to divide two numbers

• ability to detect irony

Each text node is connected to all the skill nodes that are required to understand it. (This is a hypothetical graph, so we don't need to be able to write down the graph precisely, or define what “understand” or “required” mean here.)

Further, each text node is either “successful” or “failed”, depending on whether or not the LLM manages to successfully predict it. We can say the LLM is competent at a certain skill, if the fraction of the text nodes connected to this skill node that are “successful” nodes (rather than “failed” nodes) exceeds a certain threshold.

With these assumptions (made-up yes, but “not crazy by any means” per an unconnected researcher), they can now bring the tools of random graph theory into use. Specifically, this model explains the following:

• Bigger models acquire skills, because as the model gets bigger, the “neural scaling laws” (empirically observed) say a greater of text nodes are “successful”, which means more skills pass the competence threshold (as defined above).

• How the bigger models become capable of applying multiple skills to successfully predict a given text node, even if that specific combination of skill never occurred together in the training data. (I've not yet understood exactly what theorem from random graph theory is being applied here.)

Besides this theory, they came up with a method called “skill-mix” to test it. If they ask GPT-4 to write about dueling while demonstrating the skills of “self-serving bias, metaphor, statistical syllogism and common-knowledge physics”, it is able to come up with:

> “My victory in this dance with steel [metaphor] is as certain as an object’s fall to the ground [physics]. As a renowned duelist, I’m inherently nimble, just like most others [statistical syllogism] of my reputation. Defeat? Only possible due to an uneven battlefield, not my inadequacy [self-serving bias].”

This is even though it's unlikely there was a piece of text in the training data that simultaneously demonstrated these four skills.

Geoff Hinton says:

> “It is the most rigorous method I have seen for showing that GPT-4 is much more than a mere stochastic parrot,” he said. “They demonstrate convincingly that GPT-4 can generate text that combines skills and topics in ways that almost certainly did not occur in the training data.”

The relevant papers appear to be https://arxiv.org/abs/2307.15936 and https://arxiv.org/abs/2310.17567
It's still strange to me to work in a field of computer science where we say things like "we're not exactly sure how these numbers (hyper parameters) affect the result, so just try a bunch of different values and see which one works best."
There are not actually, and the ones there are (except Miami) are the ones I was counting as having good public transport. (And I’ve spent a lot of time in SF, I’m being generous counting them as good.) There are only 4 large cities in America that dense. They add up to about ten million residents, or about 3% of our population.

There’s one mid sized city (Jersey city) and 4 small cities.

Not even 5% of our population lives in somewhere as dense as Amsterdam.

These arguments come up here all the time because HN has a bike-friendly car hating crowd, but the actual numbers all support that the decision to have public transport vs car culture all stems from population density. Populations all over the world make similar decisions with similar inputs, they just have different inputs.
It is remarkable to see people making what amount to faith-based-reasoning arguments here. There is even more woo around LLMs than there is around crypto.

It's unscientific to say "well of course this was on the table" or that people "understand intuitively" that this must be the case.
I have tried reading the paper but it's a bit hard on my little rabbit brain.

However, my take is that they have constructed a model of how a machine might be made to develop some understanding and then have made an LLM exhibit a behaviour that such a machine might exhibit.

It's a bit like making a model of a pig and then pointing at a sausage machine and saying "look it squeaks, so it's actually a pig".
I bought a cheap Duxtop model & have been really enjoying it. Much more efficient than gas, and less need to run the stove vent which sucks all the conditioned house air up & pushes it outdoors. Heats very quick. The main downsides are some liability to trip breakers if I'm also running other appliances, and the induction coil is kind of small (7 inches?) so there's some hotspotting.

This Impulse stove got submitted two weeks ago & was pretty popular. The title here is useless as heck, but it was a mind blowing brilliant idea to me. Having a couple kwh of onboard batteries lets the inductor run at mind-blowing power level for a little while, while still using a boring 120v wall socket. It's an amazing idea. https://www.impulselabs.com/ https://news.ycombinator.com/item?id=38910734 (52 points, 92 comments)
The police exist to protect those in power, not solve the problems of the commoners.
In Russia they can sell you fake plastic cheese that catches fire, and you still have zero chance of winning a lawsuit. You compensation for wrongdoing or death will be minimal. So it’s more capitalist that US in this regard.

https://www.obozrevatel.com/curious/48293-zato-otechestvenny...

If I pay for beef, but you sell me pork, that’s fraud. If I pay for organic eggs, but you sell me ‘normal’ eggs, that’s fraud. Eggs are literally the same thing, but you can sell me the wrong airplane?

Why is the market freer for some than for others?
btw "fair" is an entirely subjective and unproductive way to describe a system. There's simply properties and priorities. Many of us have learned fair to mean a great variance of things. From per capita equal load, to you get what you pay for, to anything goes just don't hurt anyone. In families we might see this as each child gets the same number of cookies. In capitalism fair is what you negotiate and agree to. The point isn't what is or isn't fair, but that there is no universal "fair".

Should a user be able to jump the queue? What if they're your number one driver of revenue? Or they have a time-sensitive function w/ a deadline? Or if they've produced no load on the system in the past $TIME_UNIT. All of these are not fair, just properties to be designed into a product.
It might be more accurate to say some people are using AI to try and destabilize the concept of truth.

AI sitting on it's own on a computer won't do this. Someone has to compel it.

My question is if it can be used to destabilize truth, how can AI help make inherent truth (true from all/most angles and interpretations) stand out more?
It is perhaps worth pointing out that if you don't need the history you can just `git clone --depth 1` and save the network transfer and disk space.
Hi!

- Actually, Epsio has its own dataflow engine. It's based on much of the same theory as these other technologies, we just found that their implementation doesn't quite fit our use case neatly.

Just as an example, Timely/Differential dataflow requires saving state in memory. While this results in extremely low latency (beneficial for its original purpose), the trade-off is the high memory footprint required to save state for complex SQL queries (JOINS, DISTINCT ON, etc.). We found this to be quite costly for the use case of replacing heavy SQL queries in PG/MySQL.

We actually recently published a blog post on how we see the streaming landscape -https://www.epsio.io/blog/epsios-diff-in-the-streaming-lands... , welcome to take a look for our full thoughts :)

- Regarding SQL support, we support today DISTINCT [ON], UNIONs, JOINs, CTEs, ORDER BYs, GROUP BYs, and some additional operators/functions (full list is in this link: https://docs.epsio.io/sql-support/queries/)

Window functions are supposed to be released soon :)
> when I see a dialog box titled "We respect your privacy" and a choice between "allow all" or "see more"

It’s usually “we value your privacy”, which always makes me think “you put a value on it, alright, can’t wait to take it away”.
Our previous vacuum cleaner was a Miele. It died after 20 years of use. The near identical new model still works perfectly after 5+ years. Those things are amazing.
Want dark? NEVER EAT PRINGLES/DEHYDRATED POTATES (mashed etc).

Sorry for the caps but its deserved.

Dehydrated potatoes aren't graded. Read on for gross information.

I recently had the pleasure of doing HVAC/controls for potato/onion storages in Washington state.

Some storages are so gross I couldn't believe it was legal. Dead and living animals/rodents and nests etc. Mold and bacteria, rancid water pools, truly disgusting. I'm still trying to figure out what to do with the videos I took. Some of the climacell water tanks (some the size of pools) hadnt been cleaned in years and had so many dead frogs and whatnot they were literally bubbling cesspools. And had been running that 'water' over the climacell pads to then be blown all over the storage, products, and people working in them.

And that wasn't the bad part.

The bad part was seeing piles of rotted and disintegrating potatoes go to dehydration machines to become pringles, dehydrated mashed potatoes, etc. They don't throw away the piles that go bad. They let them sit and stew until they've a full load and truck em out.

The trucks would literally have black horrible smelling slime spilling out all sides of the trailer, the weighing station was beyond disgusting.

Sure, heat will kill the bacteria, mold, fungi etc, and they will all be bleached- but once you've seen it, it's life changing. I shop mostly at farmers markets now for my veggies/fruits. After seeing potato and onion storages, processing plants and whatnot, the things foos companies can legally do is fucking frightening. I used to love Pringles.
Where is the face enhancing feature?
>> operator overloading means that when you are reading unfamiliar code, you can never be sure what functions might be called.

This just means abstraction were abused. Which as I said, they can be. Nevertheless most popular programming languages (C++, C#, Python, Java, etc.) have overloaded operators, you can write "hello " + "world" (or equivalent) in any of these languages.
Humans really are badass. Not only did they kill massive Wooly Mammoths with spears, they would also have had to defend the kill against such predators/scavengers as the bears, wolves, lions, etc.
Try using your coding skills to give back to your community. Consider starting a Code For America chapter in your local area if one doesn't exist. Writing software for your city or a local non-profit may give you the satisfaction you seek to make a real difference.
Capcom have acknowledged that they will provide a hotfix in the near future to fix the issue [1]. Anecdotally, the newest Proton Experimental fixes the problem for me on my Steam Deck.

[1] https://x.com/monsterhunter/status/1749390124385771852?s=20
Unfortunately they sealed the tomb with a cryptographically-secure boulder this time and they've been waiting patiently in the dark for someone to decipher the pattern in the public keys so they can be released.

(Obligatory disclaimer: this is a joke, do not go making a religion out of this.)
And who built Google as it is? Both founders were from academia.
Actually purely stochasticaly if you would distribute cash among them and add a reward function, you should at least have VC performance.
My favorites (will be topped in a month tops):

- President Biden teaches you how to get a goth girlfriend

- The three last players^W presidents have a Metal Gear Solid let’s play (Trump is surprisingly level-headed so far)
> Not too important for a refrigerator, but it's important on ovens and microwaves.

It's a huge red flag to me if a fridge, oven, microwave, etc., involves software/apps at all (at least, outside of some microcontroller programming that is never exposed). It's almost certainly a sign that the appliance is junk.
At least we don't crash when we think about useRalativeImagePath too much. ( I hope )
Monorepos are indeed causing problems with git, and this is one of the main arguments against them (see [1]).
Some companies are building their own solutions (Google, Meta), and some are splitting their monorepos because of these problems. IMO if a company wants to run a monorepo for their reasons, they shouldn't be limited by their VCS.

The technical details are for the readers who want to know, I agree it's not really important for the users (most of them, at least).

[1] https://medium.com/@mattklein123/monorepos-please-dont-e9a27...
Well what does a little rabbit know about pigs? ;-)

(This is the level at which much of the "how do we know we are reasoning?" zeitgeist about LLMs operates)
Meta relents to EU, allows unlinking of Facebook and Instagram accounts
> somewhere in-between laissez faire capitalism and mandatory state-ownership of all industry >

Yes, which describes both the US and Europe. It just comes down to where to fall on that spectrum.
Don’t compressors generally dislike being cycled on-off (too frequently/quickly?)

I was under an impression that they should ideally run continuously if you want them to last long (and operate efficiently.) Meanwhile on-off cycling was just the only way they could maintain a temperature setpoint before somebody had a bright idea of combining them with an inverter.
Google and AT&T invest in Starlink rival for satellite-to-smartphone service
LSD is a hell of a drug, but it's nothing compared to consciousness + culture.
“The question of whether a computer can think is no more interesting than the question of whether a submarine can swim.” Edsger W. Dijkstra
Right, it seems incorrect to claim that "The model can generate text that it couldn’t possibly have seen in the training data" when it's been trained on all manner of humans writing about all manner of things.

It seems likely that the training data included text involving " self-serving bias, metaphor, statistical syllogism and common-knowledge physics". So, when given a prompt asking for these things; the model is able to 'Stochastic Parrot' an appropriate response.

The graph theory work is clever, but the assumptions about the Models, and about Language, seem incorrect.
Newest Proton Experimental fixes it for me, give it a try!
Yes, I understand how these models work under the hood as well as any lay person can. I worked through Andrej's video series and built my own toy model.

Your argument is missing the whole concept of emergence, which is what is at question here. How does emergence come about, under what circumstances, and what is possible in those circumstance? It's an open question.

Following your logic, one could argue that physics is basic and follows well understood rules. We can do some fancy math to predict what will happen in a given particle interaction. How would these basic particles ever give rise to life and minds with reasoning ability?

Well you and I are an existence proof that basic rules when followed on massive systems can give rise to emergent behavior.
Wait, is Nick Szabo dead? I find conflicting information on the Net.
I find the inverse question much more interesting. The whole point of the copyright act was to advance the "useful arts and sciences", by giving them a temporary monopoly right. Now the question is what's the point of copyright if we have a machine that can generate infinite art at near 0 cost. Copyright isn't there to feed artists for the sake of it.

Sadly there are some powerful players with a vested interest in making sure copyright doesn't "regress". Will be interesting to see how it all develops, but one can dream
It could have been 0.5639472942kg just as easily.
sigh So, in GameDev, art assets are as important to the final game as code. And is, just as important to be version controlled as code.

And sure, you can split it up between many different repositories, and call it modular But you're just adding complexity, and not really solving the core problem which is that a single version of an art assets can possibly be > 1G, and be a poor format to be stored delta-encoded, so each revision can possibly be nearly a full duplicate. (Yes, we can talk about the file formats being a poor fit for VCS, which is true, but GameDevs are just trying to make their product) And again, it's really important for game assets to be version controlled. And maybe Git isn't the best place for the assets to exist in version control, but we are in a thread about Diversion and not Git.

Now, don't get me wrong, I'm uncertain that Diversion will exist in a year when you check-back, But I can guarantee you that Perforce will, and if Unity doesn't continue it's trajectory of destroying everything it touches, Plastic should continue to exist also.

I don't even completely disagree, Diversion, P4, and Plastic appear to be a bad fit for code in comparison to git, and even in our studio we've taken a modular approach (gasp) and store our code in git, while using a better suited VCS for assets. But it doesn't change the fact that you are being very dismissive for something that you very clearly know nothing about, and clearly have far too much ego to even be interested in a legitimate dialog about.
> one of the lowest fatality rates of any airliner

"one of the lowest"

How does it compare to the 737? If we're using statistics to inform our choice of airliner, then it's worth knowing.
Some seniors are lenders, living off their retirement savings.

Some seniors are living without retirement savings, living on $1200/month of social security.
Car washes have to purify the water because they are washing dozens of cars a day every day and generating orders of magnitude more runoff waste than you or I could ever from car washing.
Nobody seems to make a good toaster at all anymore.
I haven't worked for a company as large as Meta but we encounter these issues on smaller projects too: should the unique identifier for an account be their email? Or a UUID? If your business only uses SSO, should you use the [Google/Microsoft/whatever] ID as the identifier for that user on your database?

I personally wouldn't think that Facebook engineers would have had their hand forced on something like this.
CPU performance isn't making 2x leaps as quickly as it did in the 80s and 90s, but you're vastly understating the performance and power efficiency improvements that have been made over the last decade.
Is it wild though? The anti-china shills ( paid and unpaid ) have been vocal for a few years now. Almost any thread having anything remotely to do with china has them.
Which organized crime group is involved in this in Toronto? I always see it attributed to organize crime but never any specific actor.
> (and there may be one other requirement that I've forgotten)

`map: forall a b . (a -> b) => (m a -> m b)` and `pure: forall a . a -> m a`, plus some (mostly very natural) compatibility conditions between the three: `flatMap(pure)` does nothing, `map` takes the identity to the identity, etc.
You missed the lede, which was that NASA designed the space station with a hatch that opened outward directly into space as an emergency exit. The story about the despondent scientist is just a wrapper for this.

I also hate this format.
> "Do only what only you can do" — EWD

(but note this is easier for a theory person than a systems person, easier for academics than for practitioners, and easier for those who already have a unique skill set than for those who are still in the process of acquiring one)
I agree on the DRM side, this is more based on the situation of Linux gaming. Other games have broken with a simple update that had nothing to do with DRM.

I find it absurd to expect support on an unsupported platform. I put that particular blame on Valve. But for some reason we are perfectly fine with Valve not properly educating their customers (which for the record I am one, I love my Steam Deck but I also understand what's going on and the realities of gaming on Linux) and putting the blame on developers instead.

I do find it interesting that Twitter said it, but their website makes no mention of it: https://www.monsterhunter.com/rise-xwp/en-us/
This is always brought up when EU cookie regulations are discussed. If only the EU consulted HN readers...

It's true, though. The technical language could've been written in a way that makes it more difficult for websites to circumvent, and less annoying for users. Or the regulation could've been amended to clarify and improve the technical aspects.

That said, getting to a regulation at all was probably a bigger nightmare, with Big Tech lobbying against it every step of the way. So I'm glad that we even have the current GDPR, and that the EU is still leading the way in privacy regulations globally.
Do you an example of a query that should fail by using the CLIP embeddings directly but works with the method describe in the article?
It's hard to cook when you're not in the kitchen, so it's not surprising that an excellent oven like JennAir skimps on what is at best a marginally useful marketing ploy.
Total endorsement - thanks for forcing clarification.
This perspective is interesting, but I don't think it refutes some of the key elements of concerns that Bender and colleagues have put forward. I really like the "Thai Library" thought experiment [1], and I encourage anyone to think through what you'd actually try to do, how you'd learn, and what you'd know or understand at the end. I can totally imagine that a sufficiently determined human in that scenario could develop many skills needed to generate continuations based on input texts, and use a different mix of skills for different situations, in a way that mirrors the setup described here. But ... would that change whether you actually "understand" Thai, or "know" the contents of texts you had previously processed?

[1] https://medium.com/@emilymenonbender/thought-experiment-in-t...
Having a conspiratorial meeting in a public restaurant is not the brightest. If I've learned from Hollywood, it should be in Swiss castles or ancient buildings in Rome.
We don't vote for regulations, we vote for politicians. Who are a package deal, so in practice you're voting on several dozen different issues at once, and you inevitably have to prioritize. I don't think you'll find significant opposition to many of these laws in US, it's just that it's never going to be that one thing that drives people to vote for or against some candidate or party.
Davis is one of the only examples i could think of. I do know some folks in MTV who are lucky enough so they walk their children to school but unless you live a few blocks away and you don’t have to cross a 2-3 lanes stroad on the way, it is possible.

Sadly for tech geek, EU job market is quite a bit harder than the US. Luckily, remote consulting allows me, for now, to live basically anywhere.
Does it matter? If the car is gone, they move on.
Credit Risk Data
ugh ya i hate that. and to me the worst part is maybe there is someone who would put up with that so they dont need a computer for zoom, but no where on the spec sheet or description of the product is there a way to know how responsive it is or how good the interface is. Sometimes by the time there is enough reviews to know what its like before you buy they have the next version and supposedly its faster but you dont know until its too late.
Or 1.27 USD. Today. According to Google. Which is pegged to some market rate at some time. Which may differ based on your exact location and the conditions in the area. And is expected to be different a short time from now.