skysight-inc/HackerNews huggingface dataset:
> As the parent says, modern LLMs are finetuned with a different loss function after pretraining. This means that in some strict sense they're no longer autoregressive models – but they do still generate text one word at a time. I think this really is the heart of the "just predicting the next word" critique.

That more-or-less sums up the nuance. I just think the nuance is crucially important, because it greatly improves intuition about how the models function.

In your example (which is a fantastic example, by the way), consider the case where the LLM sees:

<user>What do you call someone who studies the stars?</user><assistant>An astronaut

What is the next prediction? Unfortunately, for a variety of reasons, one high probability next token is:

\nAn

Which naturally leads to the LLM writing: "An astronaut\nAn astronaut\nAn astronaut\n" forever.

It's somewhat intuitive as to why this occurs, even with SFT, because at a very base level the LLM learned that repetition is the most successful prediction. And when its _only_ goal is the next token, that repetition behavior remains prominent. There's nothing that can fix that, including SFT (short of a model with many, many, many orders of magnitude more parameters).

But with RL the model's goal is completely different. The model gets thrown into a game, where it gets points based on the full response it writes. The losses it sees during this game are all directly and dominantly related to the reward, not the next token prediction.

So why don't RL models have a probability for predicting "\nAn"? Because that would result in a bad reward by the end.

The models are now driven by a long term reward when they make their predictions, not by fulfilling some short-term autoregressive loss.

All this to say, I think it's better to view these models as they predominately are: language robots playing a game to achieve the highest scoring response. The HOW (autoregressiveness) is really unimportant to most high level discussions of LLM behavior.
In 2007, my house was re-zoned from R-9600 (Single family residential, 9600 SF lots minimum) to a high-density transit oriented zoning (the acronym changes every other year, but was originally PCBTPV, Planned Community Business Transit Pedestrian Village), in expectation that a light rail station will be opening within 1/4 to 1/2 mile by 2035. The new zoning requirement is six story minimum, minimum of 50 dwelling units/acre density, tax credits for elder housing and child care facilities, etc.

This is in a suburb just north of Seattle. I am currently 4 miles from the nearest light rail station, which is a 5 minute walk + a 10 minute bus ride that comes along very frequently, then a 40 minute ride to downtown Seattle, so by US suburban standards the foot/transit access is already pretty darn good. There is a small neighborhood market and a few restaurants a 5 minute walk away, and a large supermarket is really only 15 minutes walk. (but yeah, we usually drive there.)

So far, zero occupied single-family houses have been knocked down to build high density developments. There were a few dilapidated/abandoned old houses on large parcels along the freeway that were bulldozed for apartments, which was an improvement. But even if you put policies and incentives in place to encourage re-development, it can take decades for market forces to reach a tipping point where developers are actually willing to make cash offers at or above the market value on existing properties to make these changes happen.
Unfortunately, right now the market is so tight you could put slop out and it would sell like hotcakes.
I'm specifically intrigued by that loading bar with magnetic blocks.
> But the mystery is really why, in setups like these, Claude is much more likely to say "An" than "A" in the first place.

Because in the training set you're likely to see "an astronomer" than a different combination of words.

It's enough to run this on any other language text to see how these models often fail for any language more complex than English
What makes these tactics the "right thing"? I as a prospective employee don't get to quiz the interviewer using the same tactics? I can ask questions, but how I do know they're knowledgeable and that I want to work for them? Or should I be thankful they're even talking to me? I think it's a terrible practice and that we can do better.
> Some things maybe really are better managed by big, lawsuit-averse, emotionless companies.

Completely disagree. Every poor experience I had renting was with a big emotionless company. I have never had a poor experience with a small time landlord.

Just like landlord vet their prospective tenants, you should be vetting your prospective landlords. Someone with a few units is pretty easy to track down folks for references.

The problematic ones tended to be the ones in the "midrange" - between small time and big company. Using horrible management companies as contractors to farm out the dirty work, and generally getting away with it since they were big enough to throw their weight around - but not big enough to care about being squeaky clean to the letter of the law.

I'd rent out my basement mother in law unit, but due to tenant rights in my city there is not a single chance I will ever do so. From direct personal experience I know how difficult it is to remove a problematic tenant from a living situation no matter how much impact they may have on your life.

The outcome is that there is one less extremely affordable unit in the area for living in.
It disappoints me how easily we are collectively falling for what effectively is "Oh, our model is biased, but the only way to fix it is that everyone needs to give us all their data, so that we can eliminate that bias. If you think the model shouldn't be biased, you're morally obligated to give us everything you have for free. Oh but then we'll charge you for the outputs."

How convenient.

It's increasingly looking like the AI business model is "rent extracting middleman", just like the Elseviers et al of the academic publishing world - wedging themselves into a position where they get to take everything for free, but charge others at every opportunity.
The security checklist that saved my friend's vibe coded product from disaster
Try the New AI Mode Experiment in Search Labs
For me, the problem with this setup (and with most high efficiency LED lights) is the lack of red wavelenghts. Real sunlight has a substantial amount of energy in the very red end of the visible spectrum (700 nm) and also of course quite a bit in the infrared. These lamps have two spectral peaks: a narrow peak in the blue range, around 450 nm, a broader peak in the green, centered around 580 nm. That greenish peak falls off sharply, and has almost no energy in the red end.

The color sensitive cones in our eye have three peaks of sensitivity, the S cones in the blue range, the M cones in the green, and the L cones in yellow. The L cones are what your brain uses to see red colors, but they are actually pretty insensitive to deep reds like 700 nm. That’s why you THINK that LED lamps produce red, because they stimulate your L cones, but they do so without actually emitting much red energy at all!

Our bodies are sensitive to deep red light. The cytochrome in our mitochondria respond to it. There’s an experiment where shining red lights on the skin improved sugar metabolism. That makes sense, because we naked apes evolved under red-rich sunlight.

So these lamps may look like sunlight, but they’re missing some crucial wavelengths.
It’s official, Arcana is live on Product Hunt today.
Our content-based ad platform helps AI developers earn revenue without sacrificing user experience. Privacy-first, seamless integration, developer-focused.

Try our Arcana-enabled GenAI application :hugging:demo today and experience firsthand that advertisements can actually be useful and non-intrusive!

Try it yourself: https://huggingface.co/spaces/arcana-ad/arcana-in-action
We hope this is the first step to monetizing your GenAI product, and we'd love your feedback! DM us or comment below to share your thoughts.

If you want to use Arcana in your GenAI product, feel free to fill out our onboarding form: https://forms.gle/qWfFPpSaw5g51avt8
Tenants rights are not only about how long you get to stay without paying rent. Not being able to compel your landlord to remove pests, mold or fix noisy steam radiators is not something that would benefit the vast majority of tenants.
I see. The main reason I was wondering about this is that occasionally there’s people online who recommend going barefoot or using a special shoe for doing weightlifting. Maybe strengthening the foot arch is the point behind this trend rather than width.
That $3/month figure actually makes me more concerned. Sure it's low for a business with revenue but donation funded services rarely get donations from more than a few percent of their users. So Signal probably needs something like $30-40/month from people who actually donate, which seems unlikely.
You could still use git-svn, but yeah, as another commenter wrote, I don't think reproducible build is that useful when debugging, it should be fine to have an actual timestamp in the binaries.
And now I’m proud because that’s pretty much the exact idea I tried to pitch at the great grandparent.
should just adapt japanese school food. no experiments,embrace tradition
Unfortunately, that’s not a scandal. It’s business as usual and generally considered acceptable.
signal may be e2e encrypted, but if i root your phone with a 0 click exploit. Which most of the state level actors we care about can and do do. Then I can just read your signal messages out of memory as soon as you open them. period. end of.

Using a public smartphone to conduct discussions about classified information is straight crazy. E2E encrypted or not.
$3/month is a lot of money, when their current revenue per user is approximately $0/month.
It's a fair point - My opinions and use case are my own, I didn't mean to imply or assume there were promises not kept. The dagger team has been nothing but supportive and I do think has built a great community.

That said, in the early days it was definitely pitched for CI/CD - and this how we've implemented it.

> What is it?
> Programmable: develop your CI/CD pipelines as code, in the same programming language as your application.

> Who is it for?
> A developer wishing your CI pipelines were code instead of YAML

https://github.com/dagger/dagger/blob/0620b658242fdf62c872c6...

Edit: This functionality/interaction with the dagger engine still exists today, and is what we rely on. The original comment is more of an observation on the new directions the project has taken since then.
Yes, several of the most popular (and even lesser-popular but newly open-sourced models such as Gemma 3 27b) overuse Em dashes. Even when prompting them to not use dashes, they almost can't help themselves and include them occasionally anyways as it must be part of their learned stylometry. It's just not a common symbol to use at all as most people generally use commas for the same purpose. I can't even remember learning about Em dashes in my college english classes.
He went to Columbia and he seems kinda unhinged. The former is really all you need.

It's a small club, and the rest of us aren't in it.
I genuinely don't understand why some people are still bullish about LLMs
"You need the latest version of the Google app or Chrome Chrome."
Marco Rubio has confirmed (in a press conference today) that Rumeysa Ozturk is being deported purely based on her opinions, not based on any illegal actions she has taken.
Of course! Glad you can find models that work well for you and we're all learning together. Even on the "expert side" we're learning from what folks like yourself are doing and taking note so we can shape these models to be better you all.
The toolchain (eg. compiler) reads the time from an environment variable if present, instead of the actual time. https://reproducible-builds.org/docs/source-date-epoch/
That was a very cute puzzle, well done
minus (US negative) enters the chat..
Shure BRH440 ($400) or Countryman H6 if you want the low-profile style ($500)
This specific key combination is not US keyboard specific.
I like how they managed to group characters that are formally similar by binding them to the same keys.

Examples:

en and em are on -

Below are maybe Swiss specific?

~ is on N

@ is on G

| and \ and / are on 7

√ is on V

¥ is on Y and € is on E

∑ on W ( ∑ is a rotated W :)

etc.
That's a good point, but not much I can do about that. Such a DIY project is limited to off the shelf LED suppliers. It would be cool to do such a lamp with both the high CRI and some infrared, also for heating (infrared lamps are a thing after all).
When I was editing an academic book published by a well-known university press, we were all asked to do that for the references. (And my colleagues, all doctors and lawyers, only knew Word and entered the references manually.)
> We still use the shell, but it's no longer the universal composition system it was meant to be.

Huh? When did that change?

Dagger seems interesting, but this opinion that the shell and standard Unix tools are archaic is a dubious starting principle.

Every now and then a project with similar ideas comes along, and whether it rejects the notion of passing unstructured data between commands, addresses the footguns and frustrations of shell scripting, or is built with a more modern and "safe" programming language, it ultimately never seems to catch on as much as traditional Unix shells and tooling has.

The reality is that these tools and the design choices made 50 years ago are timeless in nature. They're deliberately lacking in features and don't attach themselves to any specific tech du jour. It's this primitive nature and the "do one thing well" philosophy that makes them infinitely composable. The same pipelines Unix nerds were building 50 years ago to solve problems are still useful today, which is remarkable when you consider how quickly technology moves.

Sure, new tools are invented all the time, and they might do things better than old ones. I use `eza` instead of `ls`; `fd` instead of `find` (mostly); `rg` instead of `grep` (mostly); `fzf` is a pretty essential addition to my workflow, and so on. But the underlying principles of these tools are still the same as the tools they're replacing. They're just slightly modernized and ergonomic versions of them.

Whether or not we need a `container` command, `from alpine`, or an entire new shell for that, is a separate topic. It could be argued that this could be accomplished with a few functions or standalone commands. Even if we do need this new tooling, that's great, but don't tell me that it's meant to replace a proven set of tools and workflows[1]. When containers stop being popular, will we still need this?

Also, "Daggerverse" and "modules"? Great, let's bring in the npm mentality to the shell, just what I needed.

[1]: Ah, they don't, it's meant to serve as a complement. Alright, fair enough. I'm lowering my pitchfork.
> would you really type 124–127

literally yes
Yes, and to give an idea I replaced some windows in my house a couple years back. The double pane windows on the north side were about $100 to $150 per, while the triple pane windows on the south side were $900. The triple pane also had higher installation labor costs. It seemed ludicrous to me that it would be so much more expensive, but after calling around, that was very typical.
The latter, I believe.
This is fine-tuning to make a well-behaved chatbot or something. To make a LLM you just need to predict the next token, or any masked token. Conceptually if you had a vast enough high-quality dataset and large-enough model, you wouldn't need fine-tuning for this.

A model which predicts one token at a time can represent anything a model that does a full sequence at a time can. It "knows" what it will output in the future because it is just a probability distribution to begin with. It already knows everything it will ever output to any prompt, in a sense.
I "invested" in early 2020 when all of my training moved to Zoom. I've gone through a lot of equipment and frequently am complimented on my sound, camera, and background.

My rank order for investment:

1. Mic - See my mic comment in another thread

2. Lighting - It is hard for good cameras to compensate for poor lighting. Good lighting on the other hand, does compensate and helps laptop webcams perform much better. I currently run a Godox ULC60bi with a huge parabolic diffuser. Probably overkill for many. You can get cheap fluorescent or LED lights with a diffuser. Would avoid a ring light especially if you wear glasses.

3. Camera - I have used Canon M50 and Sony A6500 as webcams. These days, I use an Insta360 Link. It's not quite as good as the cameras (you can add more "background blur" with their software), but it's more convenient. I still get compliments on this.

4. Background - Blur, doors, and bedrooms are what you see 90% of the time. A good background helps you stand out. I have a bookcase with the books ordered in color. Folks always comment about it. There are lots of other common looks, look on Youtube or X (Kevin Shen) for office makeovers.

5. Teleprompter - Not for scripts but for "looking in the eyes" of my students or the person on the other end of the call.
I’ve never heard it put that way, more like “condiment as vegetable typifies republican thinking” as a shorthand for perceived attacks on national infrastructure.
> “In 2022, it was so dry that we only harvested half of our crops, and our neighbors harvested none. However, they were better off because they had crop insurance on everything. Advancement comes at a cost.”

I think this points to both how integrated financial planning based on other people's risk assessment farming is, and how complicated the decision logic can be. He cannot get crop insurance for some of his farm because his crop diversity runs counter to mass production driven views of cropping, and he plants varieties and seeds which aren't normal in his climate zone so the insurer has no track record to base risk on.
Exactly! Using the tool secretly totally misses the point of the interview. And now it's like, Hey look at my cheat tool! Feels more like a sales pitch than anything else.
It's not like the fixtures consist of a single LED either. Should be trivial to parallelize.
Yes, "we" are out of core to the extent that the engines used in the deferred expressions we execute are out-of-core (our "batteries-included" engine is a modified Datafusion).

We have previously demonstrated the capability of doing iterative batch training by way of our "batteries-included" engine. I'll try to post a reference later but need to run now due to family obligations.
Thanks for the tip. I've enrolled in an A1 class with Goethe Institute starting in April. In your opinion, how difficult would it be to rent or potentially own a home on a single engineering salary? Looking at prices on https://www.immobilienscout24.de/, it does not seem possible after accounting for taxes. A salary of 60k ends up being about 3.2k net per month. If renting a home takes 2k of that, I may need to reconsider my country of choice.
Its not clear which is hard, yet there are hundreds in the comments pointing it out.
If the president can declassify any information then can he just say "using signal is ok" and that's that?
it doesn't matter. If I root your phone I can read your signal messages as soon as the app is open right out of memory. State level actors have, in the past and one imagines still, utilized sophisticated malware that doesn't even require any user action (aka 0click) to root android and ios.
I try not to care what other people think as long as I know I'm being a courteous and respectful individual.
That's cloud gaming, not games run locally
Will we see models built on b-trees to deal with memory requirements? Have we already?
I have indeed tries lots of things that seemed possibly useful! Rate limiting by IP (or by CIDR subnet of various sizes) was not enough for me. The bots spread out to more addreseses and still overwhelmed my resources.
I'm not sure how "a set of methods" talked about by producers of commercial music can possibly be conflated with a continued tradition of ongoing scholarship and study that dates back some 500 years or more. Obviously anyone who creates music has some idea in her mind of how she does this, but it makes a rather massive difference whether actual in-depth scholarship is involved or not. It's very hard to do real study and scholarship without writing music down at some point, and being able to listen to a recorded track is not really the same thing.

The Barry Harris approach to jazz BTW is very much informed by what we know about the way classical improvisation worked, so even though it's transmitted by video there is in fact a link to the scholarly tradition. (The style is of course different, so these aren't quite the same thing! But not that far either.)
There is sufficient circumstantial evidence for bacterial life on Mars.

The (perhaps religiously) conservative science establishment refuses to conclusively accept it.

When a robotic mission is sent to Mars with the express purpose of finding current life, then it will become obvious that hardened bacteria spores not only survive in our solar system's space, but thrive in the most difficult environments. And what about that stuff growing outside the international space station?

The question about why certain bacteria on earth are very resistant to the sun's harmful radiation is waiting to be solved.

The threat to some of the world's religions is the hypothesis that the simplest life did not originate on this planet first. So what?
This isn't a group looking to make a potentially one-off product in an attempt to break into the industry. This is a different marketing and manufacturing model that is trying to mini-max the cost-to-quality ratio by only making products that have already sold. Similar production quality as other selvedge denim companies like APC or Iron Heart, and sewn in the US. The fabric they source is often of limited supply, either discontinued or small production runs, from high quality mills.
Arent black people like 10% of us population? You dont have ro look further
> would you really type 124–127?

Yes, every time. The clarity for the reader is more important than the time I save by leaving out '12'.
Humans anthropocize all sorts of things but there are way bigger consequences for treating current AI like a human than someone anthropocizing their dog.

I know plenty of people that believe LLMs think and reason the same way as humans do and it leads them to make bad choices. I'm really careful about the language I use around such people because we understand expressions like, "the AI thought this" very differently.
Yes, web crypto requires trust on the server and is not secure if your threat model includes its compromise (or that of the CA). ProtonMail recognizes this[1] and offers native open source clients. They also try to somewhat reduce the issue by using an SPA for the web client, to reduce fetches from the server.

[1] https://vimeo.com/216747532 2017 presentation by ProtonMail's CTO saying essentially the above, at 50:41