skysight-inc/HackerNews huggingface dataset:
“who does the ai work for”

I think even ignoring the more scary government/cops questions, this gets to a key problem. Because for 25 years we’ve taught 2 generations of digital natives as well as everybody else that everything Internet should be “free as in beer” paid for by trashy advertising, and that consumers’ only cash cost should be hardware and ISP/data plan. Therefore the answer to “who’s going to foot the bill for that AI” “must” be Big Adtech meaning the AI 100% will work for them and often directly against users’ interests (just as the YouTube algorithm mindlessly but intentionally prefers to radicalize a user or drive them to obsession on any number of topics vs. having them arrive at a healthy level of satisfaction and then sign off and go outside).

In my opinion a lot of these problems we see in the scary law enforcement scenarios would be easier to solve if we didn’t expect everything to be ad-supported “free” and rather, we could be convinced to buy a $5000 piece of hardware for your home, that you control, that was privy to your encryption keys and performed all the power-insensitive AI processing for your family. That sounds a lot but compared to things like cars that people happily finance for $70,000 and smartphones which cost $1300 it is only weird because we aren’t used to it.
This doesn't make sense on multiple levels. First of all, why wouldn't Israel want Gazans to go away? All of them somehow moving to Egypt or Syria might in fact be the dream outcome for Israel.

As it stands, the track record of Gazans avoiding "oppression" by joining their co-religionists is being kicked out of Jordan for being a disruptive influence, and then playing the main role in turning Lebanon into a failed state. No wonder Egypt doesn't want them!

Given that many Gazans do migrate individually, it might even get worse with time - the more capable and less political a Gazan is, the more likely they would be to migrate away, leading to a vicious cycle on the margins
If you think about chunking as "take x characters" then using LLMs is a poor idea.

But syntactic chunking also works really poorly for any serious application as you loose basically all context.

Semantic chunking, however, is a task you absolutely would use LLMs for.
The churn is real. I wonder if so much churn due to innovation in a space can prevent enough adoption such that it actually reduces innovation
Robots.txt is completely irrelevant. TOU/TOS are also irrelevant unless you restrict access to only those who have agreed to terms.
This is highly annoying and rude. Is there a complete list of all known bots and crawlers?
I would not call a CVT "not a gearbox". Were Audis equipped with their multitronic option (not in any way related to hybrid or BEV) "cars without a gearbox"?
I know this was a common talking point but I don't really agree it is a valid reason. It's probably just demographic and algorithmic differences that pro-Palestine content is more common on TikTok.

What would be interesting though - on all platforms, what's the organic percentage of the different view points, and whats the percentage that ends up being shown to people. I think that's what people are worried about being quietly manipulated. So even a small amount of people with some extreme view point would get promoted because China wants it, but since it's real content, it's not really obvious that it's being pushed.

I'm totally pro-TikTok btw I just don't really buy the idea that it was about this specific content.
Lately, I’ve found using hilbert curves with a "spatial btree" [1] beats an rtree for most of my geolocation needs.

1. https://github.com/tidwall/bgen/blob/main/docs/SPATIAL_BTREE...
A theoretically perfect TPM is way better. The strength of your KDF is limited by the user experience. If you want to limit an attacker to trying one password per minute, then you have to make the user wait one minute to log in, if you assume the attacker only has the same amount of computing power you do. A TPM can just say “you have ten tries and that’s it.”

Of course, a real TPM won’t be perfect. But there’s no reason you can’t do both. Have the TPM use a good KDF and then an attacker has to break both.
I like the solution in this comment: https://news.ycombinator.com/item?id=42727510.

Put a link somewhere in your site that no human would visit, disallow it in robots.txt (under a wildcard because apparently OpenAI’s crawler specifically ignores wildcards), and when an IP address visits the link ban it for 24 hours.
you put an identifier on a dns txt record and then Bluesky's directory checks to make sure the identifier is there.

I'm not sure there is documentation on how to do this yourself though, if that was what you are asking.
Not if you're selecting out of 10s or 100s of possible diagnoses
https://darkvisitors.com/agents

https://github.com/ai-robots-txt/ai.robots.txt
As I mentioned, I haven't finished it, so don't take my comment as a review. It's just a soft opinion.

Your comment reminds me of the Bible (cartoonish characters, plenty of action, etc). Took me some 35 years to learn to appreciate it. However, what made me like it has nothing to do with popular notions of the book (I'm compelled by the history behind its several edits and compilations).

Maybe someday I'll like Atlas Shrugged too. I don't know, maybe there's something else there, but I find it unlikely, given that it lacks this more profound historical background. It tries to fake it, and in some sense it wants to be it (some historical allegory people book thingy).
When Languages Collide, it makes a mess of one – Colider
Hello, Hacker News! I've created a programming language called Colider, It's still in alpha don't expect a flawless experience. Try it out, at http://codeberg.org/samedev/colider.

P.S It has one l cuz typo, and I'm lazy.

P.S again comment your forks, or make PRs outside codeberg and comment them here. :)
Cosine Similarity: Not the Silver Bullet We Thought It Was
I wish they just sold Siennas in Europe…

:hide_the_pain:
In the UK, the Computer Misuse Act applies if:

* There is knowledge that the intended access was unauthorised

* There is an intention to secure access to any program or data held in a computer

I imagine US law has similar definitions of unauthorized access?

`robots.txt` is the universal standard for defining what is unauthorised access for bots. No programmer could argue they aren't aware of this, and ignoring it, for me personally, is enough to show knowledge that the intended access was unauthorised. Is that enough for a court? Not a goddamn clue. Maybe we need to find out.
They do have freedom of speech—Citizen's United.

The more significant issue is that this is a bill of attainder. The fact that this was prima facia rejected by the courts is an indication that the NatSec state has corroded every branch of government, and we have a zombie government.
The article says IT unemployment is "its lowest since November 2023."

That's not two years. It's 14 months.
Still doesn’t explain why other species in the same context survived without it or with an opposite trait.
this is the way…

I find absolutely ridiculous every social media / free speech discussion if platform does not have proof of identity. while you and I may have right to free speech the bots etc do not. hence, there is no free speech without proof of identity imo
Why did it fail would be my question.

If it turns out to be any of “location, location, location” then getting a partially kitted out building may not help you.

Did they get independent data into the building via different routes? How’s the power?

Could be the data was coming in through a route that sees frequent construction. I knew a guy who ran the IT dept for a university and he discovered that the excavation crews found it was cheaper to maybe have to pay a fine for cutting data lines than it was to wait for them to be marked accurately. He spent a lot of time being stressed out.
The project has been built using Astro with Svelte.
> Unfortunately we don't seem to have that sortve option available to us with desktop OS.

As I understand MacOS uses per file encryption on AFS.
> And that's assuming an environment of honest reviews written in good faith, which isn't a realistic assumption

I exclusively shop around the bad reviews. All the fluffy positive comments are noise. People writing real, substantial complaints are much less common an issue. If everyone has the same complaint about a rude owner or a bland dish, that’s some good signal.
Text to SQL – chat with your data
a simple text-to-sql app https://rahuldey700-texttosql.streamlit.app/ using AI agents for those tired of writing sql queries (or don't know how to yet)

1. upload an excel/csv

2. ask questions in english

3. returns sql query and the final response

Example:

1. Upload excel file containing sales data (region, product, sales, month)

2. Ask : "calculate total sales"

3. Response:

SELECT SUM(sales) as total_sales FROM demo_data;

Agent Final Answer: The total sales amount to 2430.

don't have a dataset handy to try this tool? you can find one here https://www.datablist.com/learn/csv/download-sample-csv-file...

time to build this - 5ish hours (thanks to gpt o1)

https://github.com/rahuldey700/TextToSQL
How? With my proposal, if an attacker can obtain your password then they can get in, otherwise they can’t. With TPM + PIN it’s the same, just with your PIN instead.
Tiktok should just walk away from the US.

Whole mindless social media bubble will burst soon. Advertisers have been claiming for a while now, with the content explosion, they are spending more and more every year to get less and less.

Platforms are squeezing people on both demand and supply side to show their record numbers every quarter. And it cant go on for ever. Watch prices levels of pay for visbility. Keeps jumping.
How's security handled? Not in terms of system permissions which Linux handles well, but in terms of guarantees that it can't be hijacked and remotely controlled by an external attacker.
Quixotic[0] (my content obfuscator) includes a tarpit component, but for something like this, I think the main quixotic tool would be better - you run it against your content once, and it generates a pre-obfuscated version of it. It takes a lot less of your resources to serve than dynamically generating the tarpit links and content.

0 - https://marcusb.org/hacks/quixotic.html
Say "hi" to the Chairman for us.
I get this argument, and obviously it's not stopping people from uploading the same content other places. But isn't there (or shouldn't there be) something about not banning what people can consume? Like could the US ban aljazeera? Or banning foreign books?! And still TikTok is different, because it's about the potential for quietly manipulating or curating what is seen, even if that content is produced domestically... And even if people can use other apps, there's still a community and subcultures that are being dismantled.
Looking at o1's behavior, it seems there's a key architectural limitation: while it can see chat history, it doesn't seem able to access its own reasoning steps after outputting them. This is particularly significant because it breaks the computational expressivity that made chain-of-thought prompting work in the first place—the ability to build up complex reasoning through iterative steps.

This will only improve when o1's context windows grow large enough to maintain all its intermediate thinking steps, we're talking orders of magnitude beyond current limits. Until then, this isn't just a UX quirk, it's a fundamental constraint on the model's ability to develop thoughts over time.
I think that once you have domains as an identity, you can solve a lot of problems with the idea of 'just add money'. If $1000 gets me a gold check mark, it changes the economics of impersonation. Is it worth it to spend $1000 to get a gold check mark on 'goog1e.com' if a brand monitoring system is going to get that moderated out of existence in a couple of hours?

That's also why domain verification systems need to have continuous re-validation with more frequent re-validation for new identities. For example, if '@goog1e.com' is a new identity, it should be re-validated after 1h, 4h, 8h, 16h (up to a maximum). Additionally, you could let other validated users with aged accounts trigger a re-validation (with shared rate limits for a target domain).

The great thing about domains is that those of us that are good faith participants can build a ton of value on them and that value can be used as a signal for trustworthiness. The hard part is conveying that value to regular users in a way that's simple to understand.

We could also have systems that use some type of collateral attestation. For example, if I donate $1000 to the EFF, maybe I could attribute that donation to my domain 'example.com' and the EFF could attest to the fact that I've spent $1000 in the name of 'example.com'.

You probably have to gate that though some type of authority, but I can imagine a system where domain registrars could do that. I would love to buy reputation from my registrar by donating money to charity.
What a terrible title.
I used to drive a Prius until the hybrid battery system died, and the quote to replace it was a significant fraction of the value of the entire vehicle. I’ve driven ICE ever since.

My goal is usually to drive my vehicle all the way into the ground, and I guess that’s what happened to my Prius except it happened around if not before 150,000 miles.
(Although PlantFusion includes guar gum too, so maybe that point was overly nitpicky. It just seemed odd to make that variable dependent on your choice of flavor.)
> feeding the humans

You shouldn't include this, because they would exist and eat anyway, and probably the same thing (unless, maybe, their habits are changed).
I created an NLP library to help curl straight quotes into curly quotes. Last I checked, LLMs struggled to curl the following straight quotation marks:

''E's got a 'ittle box 'n a big 'un,' she said, 'wit' th' 'ittle 'un 'bout 2'×6". An' no, y'ain't cryin' on th' "soap box" to me no mo, y'hear. 'Cause it 'tweren't ever a spec o' fun!' I says to my frien'.

The library is integrated into my Markdown editor, KeenWrite (https://keenwrite.com/), to correctly curl quotation marks into entities before passing them over to ConTeXt for typesetting. While there are other ways to indicate opening and closing quotation marks, none are as natural to type in plain text as straight quotes. I would not trust an LLM curl quotation marks accurately.

For the curious, you can try it at:

https://whitemagicsoftware.com/keenquotes/

If you find any edge cases that don't work, do let me know. The library correctly curls my entire novel. There are a few edge cases that are completely ambiguous, however, that require semantic knowledge (part-of-speech tagging), which I haven't added. PoS tagging would be a heavy operation that could prevent real-time quote curling for little practical gain.

The lexer, parser, and test cases are all open source.

https://gitlab.com/DaveJarvis/KeenQuotes/-/tree/main/src/mai...
Windmemory
Hello,
I would like to talk about a project I developed for Windsurf Memory, which provides a platform where users can easily share their own prompts (or “memory” as we call them) and import them into their own Windsurf editors.

So what exactly does this feature do?
This system works in a similar way to the Cursor editor's .cursorrules file. So, you can create custom rules for your own projects or define prompts specific to your Windsurf editor. This way, you can make your development processes more efficient and optimize your workflow.

For example:
- You can create custom code styles or rules for your own project.
- You can create custom prompts to use in your Windsurf editor.
- You can share these prompts to make it easier for others.

This project is designed especially for software developers. You can use these features to make your workflow smarter and more efficient. If you have any questions or would like more information on this topic, please feel free to contact me.

Website: https://windmemory.vercel.app/
Github: https://github.com/Queaxtra/windmemory
it's the same mentality that brought us the git design - the easiest, least typing options are rarely, if ever the thing you want to do.

Instead, these invocations give cryptic messages, throw errors, or sometimes, even break things.

The most common and helpful things are hidden deep behind multiple flags and command line arguments in manuals that read like dictionaries more than guides.

I'm always at a complete loss as to how such decisions are made. For instance, "git branch -vv" is the useful output you would like to see, every time, that should be "git branch". Why not make the current output, "git branch -qq"? Is a humane interface too much to ask for? Apparently...

I know people defend this stuff, but as a senior engineer in programming pits for 30 years, they're wrong. Needless mistakes and confusions are the norm. We can do better.

We need to stop conflating elitism with fucked up design.
How has DeepSeek improved the Transformer architecture?
So does everything. Everything runs off of entropic decay of the universe.

Almost all the power we use is solar (stored energy from the sun). Nuclear is an exception, in that the energy didn't come from Sol, but some other star that went Nova in the ancient past.

Anyway, a Prius was a "hybrid" in the same sense as previous gas-electric cars and is the same nomenclature used for diesel trains with electric drive.
People agreeing and disagreeing about the central thesis of the article, which is fine because i enjoy the discussion...

no matter where you stand in the specific o1/o3 discussion the concept of "question entropy" is very enlightening.

what is the question of theoretical minimum complexity that still solves your question adequately? or for a specific model, are its users capable of supplying the minimum required intellectual complexity the model needs?

Would be interesting to quantify these two and see if our models are close to converging on certain task domains.
Well, the explanation I heard is that snoring provided protection during sleeping to scare away predators. I don't know the source for this theory, so take it with a grain of salt :)
Of course there is free speech without proof of identity. The Supreme Court has ruled that the First Amendment protects anonymous speech. The right to speak anonymously is fundamental to the right to speak freely.

You haven't proven the identity of "bdangubic" to us yet here you are exercising your right to free speech.
?????? What?

> Just for fun, I started asking o1 in parallel. It’s usually shockingly close to the right answer — maybe 3/5 times. More useful for medical professionals — it almost always provides an extremely accurate differential diagnosis.

THIS IS DANGEROUS TO TELL PEOPLE TO DO. OpenAI is not a medical professional. Stop using chatbots for medical diagnoses. 60% is not almost always extremely accurate. This whole post, because of this bullet point, shows the author doesn't actually know the limitations of the product they're using and instead passing along misinformation.

Go to a doctor, not your chatbot.
If someone simply asked what are the advantages of bees barbed stingers over wasps non barbed ones, that would be an interesting question. But if someone asks “why” and then proceeds to give a circular logic explanation (it survived because it is fit because it survived) that is unprovable, I find that to be silly idle speculation.
> `robots.txt` is the universal standard

Quite the assumption, you just upset a bunch of alien species.
Another variety: https://en.wikipedia.org/wiki/Jambiya
My 2024 Corolla supports Apple CarPlay.
Mind sharing a decent robots.txt and/or user-agent list to block the AI crawlers?
Perhaps they meant that AfD is a spiritual successor to one of the oldest, even pre '45 parties? You know, the one with similar ideas.
I put my personal site behind Cloudflare last year specifically to combat AI bots. It's very effective, but I hate that the web has devolved to a state where using a service like Cloudflare is practically no longer optional.
The moment EU will ban x/itter for the exact same reason it will be all about free speech again.
I find 2, 3, and 4 star reviews the most useful. 5 star reviews are too effusive, and 1 star reviews are often people that are just upset, spiteful, and want to complain. Honest gripes and complaints are found in the middle.
I just wish systemd allowed abbreviations for the subcommands, like "ip" (and git, for long options).
It’s not residential proxies. It’s Amazon using IPs they sublease from residential ISPs.
All countries in the world, USA just showed it is perfectly fine to steal a foreign companies' asset. Let's do that to all USA companies, Apple, Amazon, Nvidia, Tesla, Boeing, Qualcomn, Intel, all of them. U know how rich you will be if you just got a piece of them? U know you could end homelessness, poverty, balance trade, stabilize your currency, elevate tax revenues, get free education and health care for your citizens, provide great jobs if you just got a piece of USA companies? Now you can! All of them can be Indian, Germany, France, UK, Poland, Brazilian, Mexican, Canadian, Kenyan, Egyptian companies. Everyone gets a piece, everyone gets them equally, everyone will benefit and be happy!
I heard that too, but it doesn't sound likely. If I was sleeping and there was a predator passing by, I would prefer it didn't notice me... :)
Personally I'm not trying to block the bots, I'm trying to avoid the bandwidth bill.

I've recently blocked everything that isn't offering a user agent. If it had only pulled text I probably wouldn't have cared, but it was pulling images as well (bot designers, take note - you can have orders of magnitude less impact if you skip the images).

For me personally, what's left isn't eating enough bandwidth for me to care, and I think any attempt to serve some bots is doomed to failure.

If I really, really hated chatbots (I don't), I'd look at approaches that poison the well.
Have you looked at HubSpot?
I wonder if anyone has checked whether Alexa devices serve as a private proxy network for AmazonBot’s use.
Probably not copyright infringement. But it is probably (hopefully?) a violation of CFAA, both because it is effectively DDoSing you, and they are ignoring robots.txt.

Maybe worth contacting law enforcement?

Although it might not actually be Amazon.
> I would also argue "sudoku solver" better conveys the underlying mechanics.

It really does. I glanced over the WFC algorithm and didn't really get it, but reading your comment that it's a "sudoku solver" helped click things into place in my head.
Dammit. Unchecked geocentric model privilege, sorry about that.
TLDR;

Suicidal altruism and costly signaling for the survival of the super-organism. Also, zombie poison delivery pumps.
> Yes, but the point is that big company crawlers aren’t paying for questionably sourced residential proxies

You'd be surprised...
> I don't mean finding a few Nazis that happen to be members

Ah the few local Nazis that just happened to stumble upon the membership, while the rest of them were just fine with it. Reminds me of the good old "you're not marching with Nazis, you are a Nazi".