MurphyA/hacker-news huggingface dataset:
by jerf | 2025-05-30 14:18:24+00:00
"if only for the fact that a surprising amount of people adamantly refuse to believe that they are valid... And even the people who are eventually convinced that it’s valid still think that it is technically incorrect in some unspecified way."

Speaking from my personal experience, if your idea of "valid HTML" was created in the late 1990s or early 2000s, it's worth a spin through the current HTML standard. HTML has always de facto been permissive, but de jure it had certain requirements. However, HTML 5 essentially works by reifying a very, very well-specified algorithm for how to handle HTML "loosely" (even though it is very strictly specified), and then refactors away effectively every requirement it possibly can and defers them to that algorithm instead.

Technically speaking, as long as you put down the correct doctype, you can elide almost anything nowadays and get a functional document; for instance, "<!DOCTYPE html><title>Hello</title>" is fully standards compliant now (push it through [1]). Only thing the validator gives is a warning that you might like to specify a language in the doctype. It isn't just "browsers will pretty much do the 'right thing'" with that, which has been true for a long time... that's actually standards-compliant HTML now.

What a lot of old hands don't understand is that HTML 5 was a seismic shift in how HTML is specified. Instead of specifying a rigid language and then pretending the world is complying and it's super naughty of them not to, it defines a standard for extracting a DOM tree from effectively any soup of characters you can throw at it, compliance is loosened as much as is practical, and even when things don't comply there's a specification on exactly how to pick up the pieces. HTML 5 has a completely different philosophy than HTML 4 and before.

(Relatedly, the answer to the frequently-asked question "What is the BeautifulSoup equivalent for $LANGUAGE", at least as far as parsing, is effectively now "Find an HTML 5-compliant parser", which they all have now. Beautiful Soup's parsing philosophy was enshrined into the standard.)

[1]: https://validator.w3.org/nu/#textarea
by leeny | 2025-05-30 14:19:24+00:00
I agree with what Mike said. We shared the content because we want people to read the book and to use interviewing.io... it's not because we've given up on it (see my longer comment about it: https://news.ycombinator.com/item?id=44136448). We have yet to see any real evidence that technical interviews are dying.

What is hopefully dying is companies asking verbatim LeetCode questions and candidates having to memorize a bunch of questions. We wrote this book largely because we wanted to teach people how to think. I knowing how to think is only going to get more valuable.
by PaulHoule | 2025-05-30 14:19:59+00:00
That's true for many people with student loans, not true for most. (Most people can pay their loans)

If you really believe student loans are unjust than the right policy is forgive them and stop making them or make some fundamental change that makes them more just (e.g. make them dischargable in bankruptcy like other loans)

Forgiveness without reform adds fuel to the fire. Part of what keeps debt under control is that borrowers and lenders are exposed to risk: if borrowers believe that each Democratic administration is a spin of the slot machine that might forgive their loans they might be more reckless about getting them. Instead of good policy we get clientelism.
by Wowfunhappy | 2025-05-30 14:20:59+00:00
I'm also assuming. But I would ask the opposite question: why store all that data if you'll have to scrape again anyway?

You will have to scrape again because you want the next AI to get trained on updated data. And, even at the scale needed to train an LLM, storing all of the text on the entire known internet is a very non-trivial task!
Ask HN: What tools do you use to discover competitors?
by flippyhead | 2025-05-30 14:21:12+00:00
Yeah yeah I know, don't be too obsessed with competition but also, when I have some new dumb idea I first buy the domain and then... I always want to get a sense of the competative landscape and market.

What tools do you use to quickly find most competitors to a given idea?
by leeny | 2025-05-30 14:21:14+00:00
OP here. A few of you have asked why we made these problems free. The answer is twofold, simple, and maybe even a bit underwhelming:

1) We want people to read the book (To wit, we've also made 9 chapters of the book free: http://bctci.co/free-chapters)

2) We want people to use interviewing.io

In my career, I've written a lot of stuff about hiring, and I've shared a lot of interview-related materials (e.g., full length interview replays). I hate paywalls for content, and you probably do too... and I have never regretted making it free. In my experience, putting good stuff out there is the best way to market to an eng audience.
by justinzollars | 2025-05-30 14:21:22+00:00
Who cares? Electricity is literally the flow of electric charge, typically measured in terms of current (amperes). China has substantial baseload capacity. China adds an America of capacity every 18 months. Batteries connected to the grid, imply shortages. A need to balance supply and demand, manage variability in energy sources. They are also very expensive. So we have net on net, less power, that's unreliable, that's more expensive. Nothing to brag about.
by lesuorac | 2025-05-30 14:21:52+00:00
> Moving the cost of the scams onto the banks will strongly incentivize them to prevent the scams.

And then less people have bank accounts.

The scammer isn't necessarily the person with the final bank account. Its often somebody who was told they're managing payroll for some made-up company and they're going to get a 2k transfer and to keep 500 of it and withdrawal 1.5k of it as cash to do payroll.

While you may think that now only people who aren't scammers won't get bank accounts what actually is happening is that anybody that can be fooled doesn't get one.

https://www.bitsaboutmoney.com/archive/optimal-amount-of-fra...
by andsoitis | 2025-05-30 14:21:59+00:00
"The European Commission welcomed the change, said spokesperson Thomas Regnier, arguing that the Biden rule, if it took effect, would “undermine U.S. diplomatic relations with dozens of countries by downgrading them to second-tier status.”

European Union countries should be able to buy advanced AI chips from the U.S. without limitations, Regnier said.

“We cooperate closely, in particular in the field of security, and represent an economic opportunity for the U.S., not a security risk,” he said in a statement."
by overu589 | 2025-05-30 14:22:07+00:00
This delusion of any such thing is your sir.

I suggest nothing of the sort.

I would like to point out your jump to extremes.

Owning your own ass means a steak holder.

Break the produce consume cycle with investments in personal steak. Buy in to the system. Be a link in the supply chain.

This is age old advice, it merely must be retold to ourselves generationally.

Our opportunities and risks will for ever be tethered by mutual dependencies. I think we can all see there are those who grow up into business owners, land owners, and valuable assets to their communities and those who never read “The Richest Man in Babylon.” (Sic.)

In times of uncertainty we need to hear it from somewhere. We have to be told it’s okay to see ourselves as owning a steak, not merely subscribers.
by NoMoreNicksLeft | 2025-05-30 14:22:14+00:00
>These weapons cannot be used ethically,

I disagree. Our most dangerous enemies aren't even human. On the off chance the little green men show up, let's have something with a bit more oomph than polite rebuke. Just in case.

>"Someday, an ultimate class of warriors will evolve, too strong to be contested. They will win battles without having to fight, so that at last, the day may be won without shedding a single drop of blood."

This is just silly. What if those warriors want to ban abortions and library books? Will you be satisfied that their fighting prowess grants them the power to dictate that to you? Do you believe that there's some mystical connection between that strength and your particular ethical perspective?
by omneity | 2025-05-30 14:22:31+00:00
Very rougly you can consider the Bs of a model as GBs of memory then it depends on the quantization level. Say for an 8B model:

- FP16: 2x 8GB = 16GB

- Q8: 1x 8GB

- Q4: 0.5x 8GB = 4GB

It doesn't 100% neatly map like this but this gives you a rough measure. In top of this you need some more memory depending on the context length and some other stuff.

Rationale for the calculation above: A model is basically a billions of variables with a floating number value. So the size of a model roughly maps to number of variables (weights) x word-precision of each variable (4, 8, 16bits..)

You don't have to quantize all layers to the same precision this is why sometimes you see fractional quantizations like 1.58bits.
by tortasaur | 2025-05-30 14:22:40+00:00
As someone who gleefully followed along as the Web3 hype train derailed, an important distinction is that crypto turns every believer into a salesperson, by design. There were some that were truly passionate about the potential applications for blockchain technology, but by and large they were drowned out by people who, having poured $10k into the memecoin of the week, wanted to see the price of that coin rise.

This doesn't feel like that. The applications of generative AI have become self-evident to anyone that's followed their rise. Specific applications of AI resemble snake oil, and there are hucksters who pivoted from crypto to AI, but the ratio of legit use cases to scams isn't even close.

If anything, the incentives for embellishment have flipped since crypto. VC-funded AI companies will dreamily fire press releases about AI taking us to Mars, but it doesn't have the pseudo-grassroots quality of cryptocurrency hype. The average worker is incentivized to be an AI skeptic. The rise of generative AI threatens workers in several fields today, and has already negatively impacted copywriters and freelance artists. I absolutely understand why people in those fields would respond by calling AI use unethical and criticize the shortcomings of today's models.

We'll see what the next few years hold. But personally, I foresee AI integration ramping up. Even if the models themselves completely stagnate from this point on, there's a lot of missing glue between the models and the real world.
by closewith | 2025-05-30 14:22:42+00:00
Not by tax cuts, by tax policy. One which the bloc was happy to enable when it wanted to pass the Lisbon treaty.

But that's irrelevant, as ghost HQs can avail of Ireland's tax regime with minimal employees.

Ireland's extremely educated workforce and exceptionally stable and peaceful governance along with business friendly economic climate are the reason Ireland is a software powerhouse, independent of tax regime (which is not particularly conducive to large employers).
by john-h-k | 2025-05-30 14:23:28+00:00
This isn't just due to the actual dependencies of flag instructions at hardware level (although likely be a factor), it also majorly affects code layout. On Arm64 for example, you can make a comparison, do other operations, and then consume the result of that comparison afterwards, which is excellent for the pipeline and OoO engine. However, because most instructions on x86_64 write flags, you can't do this, and so you are forced to cram `jcc`/`setcc` instructions right after the comparison, which is less friendly to compilers and the OoO engine
by WillAdams | 2025-05-30 14:23:32+00:00
One thing which the U.S. Air Force EDIT: Used to brag on was that no U.S. soldier had been killed by enemy munitions from the air in an active conflict zone since Vietnam --- drones definitely change the calculus in which this statement is made.

EDIT: and as pointed out below, were able to break this record, depending on how one defines airspace and conflict zone.
by romanhn | 2025-05-30 14:23:38+00:00
This sounds pretty close to how my brain processes things. I too have aphantasia, so imagination/visualization is conceptual/non-visual. The way sound plays in my head is via imaginary humming or singing by me, it never sounds as if coming from some external source. I do get rare earworms, but again, it's more of a realization that I've been humming a song for a while.

No issues with memorization, however. Brains are weird.
by sgt101 | 2025-05-30 14:23:38+00:00
If you send your troops or vehicles forward without defense they will all get killed. You are not going forward in order to exercise your vehicles and you are not defending your vehicles and troops for the sake of simply preserving them. You are doing it in order to be able to go forward and win.

Armies that can't conduct an offense are unable to win wars. At best they are able to wait for wars to finish. Remember the nightmare of the columns in the north of ukraine in 2022 - TB2s (remember them?) ranging up and down blowing up operationally relevant assets without being hindered. If those troops had had a way of protecting themselves it's possible that they could have been resupplied and evacuated. If that had happened, or if they had been able to refuel and push south Russia might have won, it was pretty close anyway. Imagine the value of avoiding 3 years of this conflict.
by daizhengxue | 2025-05-30 14:24:30+00:00
Not exactly an "agent" per se, but the underlying APIs behind these agents have definitely changed my workflow - especially the new image generation APIs.

I've been using filtrix.ai and it's actually helped me land several e-commerce clients who need product showcase images and model photos generated for their listings.

It's interesting how sometimes it's not the flashy agent interfaces that make the difference, but finding the right API/tool for a specific use case that can actually drive real business results.
by j1elo | 2025-05-30 14:24:37+00:00
That's perfectly fine but the investor exists in this scenario because of the user base that the product has; and the user base itself exists in part because of the features that were given for free and later are removed. Would they have been able to even have investment if the parts they want to charge for, had been charged for from the start, to avoid the bait and switch?
by amazingamazing | 2025-05-30 14:24:42+00:00
> Deterministic simulation. Another lightweight method widely used at AWS is deterministic simulation testing, in which a distributed system is executed on a single-threaded simulator with control over all sources of randomness, such as thread scheduling, timing, and message delivery order. Tests are then written for particular failure or success scenarios, such as the failure of a participant at a particular stage in a distributed protocol. The nondeterminism in the system is controlled by the test framework, allowing developers to specify orderings they believe are interesting (such as ones that have caused bugs in the past). The scheduler in the testing framework can also be extended for fuzzing of orderings or exploring all possible orderings to be tested.

Any good open source libraries that do this that are language agnostic? Seems doable - spin up a container with some tools within it. Said tools require some middleware to know when a test is going to be run, when test is run, tools basically make certain things, networking, storage, etc "determinstic" in the context of the test run.

This is more-or-less what antithesis does, but haven't seen anything open source yet.

You of course, could write your tests well, such that you can stub out I/O, but that's work and not everyone will write their tests well anyway (you should do this anyway, but it's nicer imo if this determinism is on a layer higher than the application).

as a slight sidebar - I'm not really bullish on AI, but I think testing is one of the things where AI will hopefully shine, because the feedback loop during prompting can be driven by your actual application requirements, such that the test implementation (driven by AI), requirements (driven by you as the prompt) and "world" (driven by the actual code being tested) can hopefully help drive all three to some theoretical ideal. if AI gives us anything, I'm hoping it can make software a more rigorous discipline by making formal verification more doable.
by antisthenes | 2025-05-30 14:24:46+00:00
I mean, you do realize you FIRST need to be location-agnostic to find some people, right?

Only then can your group start to develop favorite local places to hang around at.

> I bring this up because if you look at places that had lively social activities a few decades or a century ago, they were almost always a specific place.

Which is basically irrelevant, as you can come up with a similar example of pre-covid, during-covid and post-covid changes for how people hang out. People adapt pretty quickly.

> The neighborhood cafe where locals can stop by at any time and see other locals. The bar that everyone stops by after work twice a week.

I'm not sure these places really ever existed with such a stability as you describe outside of _really_ small towns. Like under 5000 pop towns.
by drweevil | 2025-05-30 14:25:28+00:00
What is a lie, and what is the truth? These are age-old questions, not some recent phenomenon. The Spanish-American War was at least in part precipitated by the infamous "yellow journalism" of the time. Propaganda and disinformation played a large role in the events leading to and including WWII. And what is The Truth? Using photography as an example, lies can easily be told by omission, even without any dark-room chicanery. What is the photographer's subject? What is off-frame? Which photographs did the editor select for publication? What story is not being told?

If anything, the idea that one can take information as "true" based on trust alone (what does the photograph show, what did the New York Times publish) seems to be a recent aberration. AI will be doing us a favor if it destroys this notion, and encourages people to be more skeptical, and to sharpen their critical thinking skills. Forget about what is "true" or "false." Information may be believed on a provisional basis. But it must "make sense" (a whole subject in itself), and it must be corroborated. If not, it is not actionable. There simply is no silver bullet, AI or no AI. Iain M Bank's Culture series provides an interesting treatment of this subject, if anyone is interested.
by em-bee | 2025-05-30 14:25:46+00:00
I think it’s unfortunate that you wouldn’t share some of your beliefs; we could have constructively criticized each other’s beliefs, and thereby sharpened our critical thinking skills.

actually, i'd love that. i just had some bad experience doing that in public where uncharitable readers can chime in. my email is in my profile. how about we continue the discussion there?

prevailing religious texts are outdated compared to our current understanding of the world

it's not that the texts themselves are outdated, but that peoples interpretations of those texts is. for example when the bible claims that the world was created in seven days, we today know that this can't possibly be true, therefore this text can't possibly be meant to be taken literally. it also means that it was never meant to be taken literally, but the people who did take it literally lacked the scientific understanding to realize that.

but this problem may also occur with a religious text written today. when a discrepancy is found it doesn't mean the text is wrong, but our interpretation of it is. or maybe the scientific findings are faulty. it's no different than having findings that contradict a scientific theory. either the theory is wrong or the findings are in error. the interpretation of a religious text is just like another theory.

the correlation between religiousness and harmony is not as strong as many religious people seem to claim

well, that to me is an indictment of those religions, and rightly so.

most of them also include many infamously unproven teachings (the existence of a creator and afterlife/reincarnation)

well, the problem here is that these are by their nature either unprovable or we simply lack the scientific knowledge to verify or falsify them.

absence of evidence is not evidence of absence. this is the leap of faith that a believer in science needs to do. it is fine to reject claims that can clearly be proven wrong (such as the age of the earth) but for anything that can't be proven we need to keep an open mind. that doesn't mean to blindly accept it. keep that skepticism, but at least allow that those claims are at least in the realm of the possible, specifically because we have no way to prove otherwise. to reject god or the afterlife without proof is just as much an act of faith as is to accept them.
by imaginationra | 2025-05-30 14:26:14+00:00
For filmmakers- smart phones + cheap video cameras + free video editors + youtube monetization create what from my perspective is human made content slop. So to me slop is slop- it doesn't matter if a machine or a human "made" it. It's all slop and it all sucks haha.

Grifters will always grift- curation is what is important to sort through and ignore the slop- maybe there will be some systems with special fingerprinting algo's to "find" original human made non slop?
by tanchaowen84 | 2025-05-30 14:26:15+00:00
To be honest, I’ve done quite a bit of LeetCode, and while it’s great for practicing algorithms and problem-solving, I’ve found that I rarely use these “fancy” algorithms in my actual work. Most of the time, what really matters is breaking down requirements into clear logic, writing maintainable and collaborative code, and communicating well with the team. These skills are hard to assess through algorithm-focused interviews alone. I really hope interviews can place more emphasis on practical engineering skills and real-world scenarios, which would be more valuable for both companies and candidates.

I’m currently preparing for interviews myself, so having access to high-quality, free resources like this is incredibly helpful. The AI interviewer feature, in particular, looks like it will be very useful for me. Thanks again to the author for making these resources available!
by kemotep | 2025-05-30 14:26:42+00:00
Good Cybersecurity is a lot like Ogres. It needs to be made up of layers. And like slices of swiss cheese, not each layer is perfect.

End user awareness training is still important. At one point in time everyone didn’t know how to type, let alone read and write. So education, and continuing education will always be important.

An IT department should be able to make it so even if a user clicks a link, gives away their password, and downloads something, the impact should be as minimal as possible. Maybe the user is locked out for half a day and needs to be issued a new computer before they can get back to work. But that is still less disruptive to automate locking them down and requiring manual intervention to unlock than a ransomware event.
by swhite237 | 2025-05-30 14:26:46+00:00
Any resources or suggestions for how to find projects or exercises for each section?

I need to train some people (who will never use the command line and will go back to excel and VSCode, but I am not the genius who decides what people have to be competent in…) and this book is great, but I don’t think they are motivated enough to learn without some projects or problems and I struggle to invent nonsense projects.
by tayo42 | 2025-05-30 14:27:26+00:00
Every culture that treated women equally? Or were there male only spaces because women were seen at 2nd (or 3rd tier people, below the pets?)

I trained at a gym where that scenario happened, people were already leaving because the teacher was an ass in general, played favorites with the male students and created at cliquey environment.

There really isn't anything women can't handle in front of men. Thinking you have some dark thing that cant be said in front of women or that you need to change how you behave is odd and exclusive to you.