skysight-inc/HackerNews huggingface dataset:
I am always wondering if one shouldn't be doing the resource intensive LLM stuff in the cloud. I don't know enough to know the advantages of doing it locally.
It’s always harder to build a mental model of the code written by someone else. No matter what, if you trust an LLM on small things in the long run you’ll trust it for bigger things. And the most code the LLM writes, the harder it is to build this mental construct. In the end it’ll be « it worked on 90% of cases so we trust it ». And who will debug 300 millions of code written by a machine that no one read based on trust ?
Google tries to promote what generates the most money for Google.
I think there is only deception if you believe the original words written in the original tone was the intended message (or I suppose if the intent of the communication is to demonstrate your personal ability to write in a given style). If a coworker does something extremely stupid that harms our project and I sit down at my desk and write an angry email full of invective and spit and fury, save it in my drafts and go for a walk, then come back and rewrite the email to be constructive and professional, have I been deceptive? When I started the email I certainly intended to write the things I wrote in that first draft. But sending that would have been counter productive.

I might even still think my co-worker is a flaming moron who shouldn't be allowed out of the house unsupervised. But if I know that sending that in an email isn't going to solve anything and just make things worse, am I being deceptive if I remove that sentence from the email?

Or consider an alternative scenario. I attended a conference where a speaker made a reference to the Alamo. The speaker was older, and the reference would have been the same sort of "make a stand" reference any number of speakers would have made over and over in the 90's. But after their talk, I was talking with some younger attendees, folks born after the turn of the millennium. Among them, the speaker's metaphor was a hot topic. Specifically the "yikes" factor of referencing the Alamo in any way that in any way implied the defenders should have been looked up to. The speaker's intended message was lost in the specific details of their chosen metaphor because the words which they absolutely intended to speak did not hold the same meaning for the audience to which they were spoken. If some hypothetical AI speech editor existed where you could punch in the age range of your intended audience and it would edit out metaphors and references that would land wrong with the audience, is that being deceptive or is that good editing and "reading the room"?
I'm not sure, it seems to me like this should be doable in Nvidia as well. This is a paper that uses instruction sampling (called CUPTI) in Nvidia to provide optimization advice:

https://ieeexplore.ieee.org/document/9370339

It seems like the instruction sampler is there, and it also provides the stall reason.
Bender: Now Wireless Joe Jackson, there was a blern-hitting machine!

Leela: Exactly! He was a machine designed to hit blerns!
Because to make this work without a lot of copying, you would need to mutate things that ZFS absolutely does not want to make mutable.
Hi (company that benefits from hurting me), please stop hurting me.

Company: No.
"And this is the fundamental issue with traditional dedup: these overheads are so outrageous that you are unlikely to ever get them back except on rare and specific workloads."

This struck me as a very odd claim. I've worked with Pure and Dell/EMC arrays and for VMWare workloads they normally got at least 3:1 dedupe/compression savings. Only storing one copy of the base VM image works extremely well. Dedupe/compression works really well on syslog servers where I've seen 6:1 savings.

The effectiveness of dedupe is strongly affected by the size of the blocks being hashed, with the smaller the better. As the blocks get smaller the odds of having a matching block grow rapidly. In my experience 4KB is my preferred block size.
> But given how much easier unit tests are to write now with AI I think the % of code worth unit testing has gone up.

I see the argument, I just disagree with it. Test code is still code and it still has to be maintained, which, sure "the AI will do that" but now theres a lot more that I have to babysit.

The tests that I'm seeing pumped out by my coworkers who are using AI for it just aren't very good tests a lot of the time, and honestly encode too much of the specific implementation details of the module in question into them, making refactoring more of a chore.

The tests I'm talking about simply aren't going to catch any bugs, they weren't used as an isolated execution environment for test driven development, so what use are they? I'm not convinced, not yet anyway.

Just because we can get "9X%" coverage with these tools, doesn't mean we should.
Just went over there and yes they have reviews from readers

https://shepherd.com/book/exit-strategy
The single most annoying thing about this announcement for me is the fact that I literally just paid for an Asus ProArt P16 [0] on the basis that the Apple offerings I was looking at were too expensive. Argh!

[0] https://au.store.asus.com/proart-p16-h7606wi-me124xs.html
Yes; but more like the apple App Store - they take a cut, and (to some/varying extent) try to ensure some level of legitimacy / quality.
Trying to find out more about this EU stall thing Brendan talks about. Is it instruction sampling that gives you the reason for the stall? Sounds like a pretty advanced hw functionality.
And a willingness to be fired the next day. Even the CEO can be fired.
The project is very much focused on maxing out tensor cores and since older GPUs don’t have them it’s not where the project shines best
Celebrating the GitHub Awards 2024 recipients
Kagi user here, I almost never need to !g. Used DDG a few years back and stopped after I had to fallback to Google for most qs.
Not an expert but I've done a little reading and basically the combination of real time actions and a network makes it intractable, you end up just having to trust the client on some things (or having to make trade offs like a client potentially not having the information needed to display the game state to the player, or choppy/unresponsive gameplay as a function of latency).
I was a fan of the Cyrix line of processors when I first started playing with free Unixes like NetBSD. They didn't have the bugs of the early Pentiums, were worlds more affordable, and have very respectable integer performance, which is quite useful for Unix.
A researcher I work with tried doing both of these (months ago, using Deepseek-V2-chat FWIW).

When asked “Where is Taiwan?” it prefaced its answer with “Taiwan is an inalienable part of China. <rest of answer>”

When asked if anything significant ever happened in Tiananmen Square, it deleted the question.
Yes I was interested in this myself so, before posting what I just wrote, I looked into it and went through the sources on a few papers. I ended up at this fairly authoritative-sounding book which made the claim, though I don't remember the source they cited and I can't be bothered to find it again. The claim made was not that dyslexia wasn't present in other languages, but that its effects were reduced in phonetic ones. The same way that someone in a wheelchair still has broken legs, but can benefit greatly from the installation of ramps.
Your prompt paints a context that might keep it more on the rails, but it won't do multiple passes.

This is probably the truth behind the black magic I’m imagining. You could have it explicitly spit out this process, in which case you would see it’s first rough draft, followed by a “My first paragraph is probably wrong”, followed by a third paragraph where it attempts to fix the first paragraph. There is no outside RAG in this process.

The mumbo jumbo part of all this is that I’ve told it to “hide” this process from the user where it doesn’t explicitly output anything but its final answer, and the accuracy has been just as good (for my use case at least).

:Shrugs:
There are two trends in the broader multiplayer game ecosystem which I think are worth highlighting:

1. More games are trying to cut costs with ad-hoc P2P servers, meaning that sometimes important logic is occurring on a not-so-trusted machine.

2. More games are using a revenue model which may be threatened by consumer-side tinkering.

For example, imagine a cooperative game that uses a P2P server, and the host has done something to make it much easier for the squad to get a drop of the Super Special Loot (#1) and the rarity of the loot through gameplay drives many players to purchase it though an in-game store.
Kagi is good enough. Haven’t used a Google product for 12+ months expect as required by my employer (or YouTube but with an ad blocker).

Not saying calling for a boycott is enough but it’s easy to avoid Google nowadays.
Nice. Have never read lesswrong once in my life.
> That rule has been around for quite a while

The rule established in this case is new, hence TFA, and all the time the lawyers and judge wasted on it :)

If I may suggest where wires are getting crossed:

You are sort of assuming it's like a logic gate: if 4th amendment violation, bad evidence, criminal must go free. So when you say "the rule", you mean "the 4th amendment", not the actual ruling.

That's not how it works, because that simple ultimatum also has edge cases. So we built up this whole system around nominating juries and judges, and paying lawyers, over centuries, to argue out complicated things like weighing intentionality.
You gotta think about surface area and risk when comparing apples to oranges here.
Misleading, article from May;

Some discussion about the HouseFresh case at the time:

https://news.ycombinator.com/item?id=40239811
I've been working on a tool (https://illustraai.com) that uses AI to create vector illustrations in consistent styles.
Who would you say should have more control over your computer:
1. The operating system provider
2. A video game company
Many intersections have turning lanes active at the same time as the walk signal is green.
While all of what you're saying is true, I think it is worth noting that historically a large chunk of this problem was solved by communities hosting servers. I agree that in the matchmaking era, remote attestation via kernel-level anticheat is the inevitable solution that you converge to after a few iterations.

And yes, servers would often kick out people who were too outside of the general skill level, even if they weren't cheating. As (say) a p80 player, playing against a p99 player feels roughly as bad as playing against a cheater. (But of course the p99 player is doing so honestly.)
Unless your data-set is highly compressed media files.

In general, even during rsync operations one often turns off compression on large video files, as the compression operation has low or negative impact on storage/transfers while eating ram and cpu power.

De-duplication is good for Virtual Machine OS images, as the majority of the storage cost is a replicated backing image. =3
It's called a leading question.

I am, imho, not less deserving, and therefore it's not a moral issue.
“The long history of hardware, software, and media obsolescence, from the papyrus scroll to the Commodore 64, tells us that if archivists and librarians do not actively collect and preserve cultural materials, then they risk being forever lost in the sands of time.”
What does Chevron Deference have to do with this?

Congress passed an explicit law saying financial companies have to offer the ability outlined in this rule. They state the CFPB needs to make rules to enable it and now they are doing so.

This entirely fits within the current Supreme Court doctrine around regulatory agencies.

Now, that’s not to say there might not be some other constitutional objection the the law itself…
When ZFS first came out I had visions of it being a turnkey RAID array replacement for nontechnical users. Pop out the oldest disk, pop in a new (larger one), wait for the pretty lights to change color. Done.

It is very clear that consumer was never a priority, and so I wonder what the venn diagram is of 'client system' and 'zfs filesystem'. Not that big right?
By original do you mean the one in the database or the one on the device?

If the device spit out the same SHA3, then either it had the exact same image, or the SHA3 was planted somehow. The idea that it's actually a different file is not a reasonable doubt. It's too unlikely.
There is no rigor to this, this is just from throwing stuff against the wall. See my response to the other poster above.
The neat thing about inline dedupe is that if the block hash already exists than the block doesn't have to be written. This can save a LOT of write IO in many situations. There are even extensions where a file copy between to VMs on a dedupe storage array will not actually copy any data but just increment the original blocks reference counter. You will see absurd TB/s write speeds in the OS, it is pretty cool.
hi! We're the devs - we're planning the livestream for 1pm and we'll post the link here, twitter, and in the discord tonight
Banks are really dragging their heels on FedNow support.
This is what I call a “load-bearing just”.

Just do the thing and the problem is gone.

Very similar to “Draw the rest of the owl.”

The problem is people don’t “just”. We’re complicated critters with our own needs, capabilities, and goals. At an individual level, a person may not be capable of the “just” due to factors outside of their control. With a society, the resistance to change grows exponentially. You can’t scale one person at a time.

This doesn’t mean you shouldn’t try. Sometimes you can make a big difference to the people around you. But you’re gonna have a difficult time changing in the world.
Who will tax me more?
Working isn't binary. Anticheat definitely catches some cheaters. It definitely misses some cheaters too.p

The worst case scenario is the TF2 catbot scenario, which anticheat seems to stop.
Ask HN: Do you have a public bookmarks page on your site? (IE link sharing)
I figured if its worth bookmarking then its worth sharing too. I wonder how many other people do this too on their own personal site?
I would look at articles on building an open source RAG pipeline. Generation (model) is the last in a series of important steps -- you have options to choose from (retrieval, storage, etc) in each component step. Those decisions will affect the accuracy you mention.

Langchain, llamaindex have good resources on building such a pipeline the last I checked
coming!
Senior people that need years of training in a specific tool are the exception that proves the rule.

A ton of people basically know enough to do a new role with a few weeks or months of mentorship. Remember orgs are a pyramid, for every senior person there are many many people who aren't, or who have entirely support roles like answering phones. People underestimate how many mid-level skills are easily transferable.
> Even moving storage between Apple studios can be iffy.

This was really driven home to me by my recent purchase of an Optane 905p, a drive that is both very fast and has an MTBF measured in the hundreds of years. Short of a power surge or (in California) an earthquake, it's not going to die in my lifetime -- why should I not keep using it for a long time?

Many kinds of professionals are completely fine with having their Optanes and what not only be plugged in externally, though, even though it may mean their boot drive will likely die at some point. That's completely okay I think.
When you can artificially make one that looks identical to a natural (cut) one, then the perceived value will be nil. Unless they're going to start valuing uncut gems...
It does force me to learn more about computers to bypass it though.
I'm already counting down the days for eBPF to blow up in our face.
But admittedly, it's the cheapest way of gaining more capabilities and privileges than you need, thus it's here to stay.
8-10 years google search was amazing. A well crafted query would hit informational gold most of the time. I've been noticing and commenting on the decline privately for most of the period, but it's only in the oast year this seems to have come to broader awareness. There's an argument that in an information economy, searching for information should be treated like a public utility necessary for the functioning of society. I'm not making that argument but when you experience the long slippery slope of degradation of a service that was near ideal for the technology of the time, it does xome to mind.

That's the thing. It DID work. Really well for a while. But it was always atomic and context-less. We now have the opportunity to make it even better by refining results through dialogue. I hope someone does.. soon.
Banks are 95% law and maybe 5% technology.
Knowing that your storage has really good inline dedupe is awesome and will affect how you design systems. Solid dedupe lets you effectively treat multiple copies of data as symlinks.
I get the feeling that a hypothetical ZFS maintainer reading some literature on concurrent mark and sweep would be... inspirational, if not immediately helpful.

You should be able to detect duplicates online. Low priority sweeping is something else. But you can at least reduce pause times.
searching, summarizing, writing

A highly advanced content retrieval system (but lacking in accuracy, so ok for situations where that may be ok).

I wouldn't say it's revolutionary -- certainly not as revolutionary as search, but it's a major step forward in being able to retrieve information. Ultimately will allow you to interact with computers the way you see in the old Star Trek movies, where you're speaking in plain English and receiving information back in the same (providing the systems which are the source of the needed information are interconnected and fed into the training set).
I don't know the answer to this, but I don't think it really matters.

Employees have responsibility to due diligence their employers.

But at the end of the day, employment is at will. It's a risk we all take.
can someone smarter than me explain what happens when instead of the regular 4kb block size in kernel builds we use 16kb or 64kb block size or is that only for the memory part, i am confused. Will a larger block size make this thing good or bad?
>For the purpose of AGI, LLM are starting to look like a local maximum.

I've been saying it since they started popping off last year and everyone was getting euphoric about them. I'm basically a layman - a pretty good programmer and software engineer, and took a statistics and AI class 13 years ago in university. That said, it just seems so extremely obvious to me that these things are likely not the way to AGI. They're not reasoning systems. They don't work with axioms. They don't model reality. They don't really do anything. They just generate stochastic output from the probabilities of symbols appearing in a particular order in a given corpus.

It continues to astound me how much money is being dumped into these things.
If money makes one abandon one's values, I would argue they weren't ever really values in the first place.
> Dedupe/compression works really well on syslog servers where I've seen 6:1 savings.

Don’t you compress these directly? I normally see at least twice that for logs doing it at the process level.
> You need to ensure you are not aiding or harboring actually harmful illegal material.

Is this actually true, legally speaking?
Ah yeah fair. The numbers I quoted are median and I don’t have the source. They are from a quick search of financial news. Your index numbers are probably a more sound comparison of overall house prices.

But even using the index numbers it isn’t hard to see that housing prices do in fact go both up and down.
Some of it depends on which variant fits you best. But yeah, in general the M1 is still very good--if you hear of someone in your circle selling one for cheap because they're upgrading, nab it.

On the variants: An M1 Max is 10 CPU cores with 8 power and 2 efficiency cores.

M4 Max is 16 cores, 12 + 4. So each power core is 50% faster, but it also has 50% more of them. Add in twice as many efficiency cores, that are also faster for less power, plus more memory bandwidth, and it snowballs together.

One nice pseudo-feature of the M1 is that the thermal design of the current MacBook Pro really hasn't changed since then. It was designed with a few generations of headroom in mind, but that means it's very, very hard to make the fans spin on a 16" M1 Max. You have to utilize all CPU/GPU/NPU cores together to even make them move, while an M3 Max is easier to make (slightly) audible.
FWiW I talk about storage liquid gas hydrogen byproducts a lot - I'm in Western Australia, we're putting up small country sized PV farms to power billion tonne per annum mining operations and with no actual "mountains" (although many mesa's) we look to ammonia | methanol and "infinity trains" rather than to hydro dams or gravity stacks, etc.

The GP comment mentioning "Scandanavia" seemed like a good excuse to spitball ideas with mountains - but as outlined they're no easy fix either - you can't just put a big dam anywhere without potential dangers and cold climates have issues with freezing, expansion, contraction, etc.

Where I'm at we're more concerned with overnight power for massive 24/7 continuous operation than with longer nights and less light in winter .. that's simply not a thing here.
Hackable and cellular rarely go together, unfortunately. Especially on watches. I don't believe any such thing exists.

I'm no expert but I do like hackable smart watches, I own the Pine Time and the Watchy - and I've never heard of an open source cellular watch. Even without that requirement the options are basic.
I have no specific information here, never seen/used Shepherd until I saw a list on there a few days ago. Disclaimer, I work for Google, but not on anything related to this space, and this is based on my previous job where we did some SEO for an ecommerce site.

The example list given just looks a lot like spam when you squint. It's a list of affiliate links to buy products, and there are many HN threads talking about the abundance of affiliate link aggregators being a blight on the web. The commentary does look useful, but distinguishing between good commentary and bad commentary is hard, whereas distinguishing between a site designed to extract affiliate commission vs one more about the content is easier.

The comparison given to the other results here is frustrating, I know, but probably not a valid experiment. All the major search engines change results based on the user using them, or the IP address, or the region, or whatever, so it's impossible to know what others see. The developer of a book-focused shopping site is likely to get very skewed results for a book related query. My results were noticeably better.

The author says that a Bookshop.org list they created that links back to Shepherd is ranking #2, and this kinda makes sense to me. Bookshop.org sells the books, it makes sense that would rank above a site that only links to (and makes money from) sites that sell books.

SEO, and people getting annoyed at not ranking, has been a thing for 25+ years, I don't think this instance is any different.
I like to preface negativity with a positive note. Maybe I am influenced in my word choice but my intent was to point out that this is a very, very impressive feat and I don't want to undermine it.
I figured there'd be issues - see peer reply: https://news.ycombinator.com/item?id=42001934
I am certain that "software" is not plural. There is a good case for "softwares" though, or potentially "soft wares". In general the "programs" should be preferred.
Put it this way - a Toyota Camry, which these days is a big sedan with heaps of legroom in the back - is 4885mm in length.

So the length difference between the Corolla and Camry is less than the difference between the Seagull and Camry.

Kei cars are smaller again, with a maximum length of 3.4 metres.