MurphyA/hacker-news huggingface dataset:
by hiAndrewQuinn | 2025-05-10 18:43:36+00:00
A couple of reasons which pop to mind:

- Searching a plain text data file is O(n). Searching a SQLite database that has been properly indexed, which is very easy to do nowadays with FTS5, is O(log n) worst case scenario and O(1) in the best case. This doesn't explain why SQLite over a dataframe or anything, but it definitely justifies it over plain text for large email collections.

- SQLite is really easy to write custom views and programs around. Virtually every major programming language can work with it without issue. See also: simonw's wonderful https://datasette.io/ .

- SQLite is an accepted archival format by the Library of Congress, if you ever want to go down the rabbit hole of digital preservation.
by ldjkfkdsjnv | 2025-05-10 18:44:56+00:00
I well aware of Schmidhuber, still the scaling of the compute was critical. The reason Schmidhuber didnt go all the way is still scaling/capital, which accrues to monopolies who can afford wildly speculative research. Also, LSTM, RNN, etc, while effective for their time, were dead ends.
by 9283409232 | 2025-05-10 18:44:57+00:00
The uptick can be explained by this story[0]. As trade talks begin, exporters want to be ready to begin shipping ASAP. It remains to be seen if this will volume will come through depending on the results of these trade talks and tariffs.

[0] https://gcaptain.com/as-trade-talks-begin-chinese-exporters-...
by kurikuri | 2025-05-10 18:45:47+00:00
Except the ostensible motive of the government is to serve its people, whereas the company’s motive is either those of the people who control the company or profit.

Even then, if the government is weak than the ‘more power over you’ is simply false. Maybe the magnitude of the power is more for a government, but companies apply their power with much more frequency.
by bigbadfeline | 2025-05-10 18:47:01+00:00
> "everyone and their mother seem to be.. maybe not waiting exactly, but expecting something to happen."

They are stunned by disbelief or ill-conceived happiness. The former are waiting for the Dems to do something, which won't happen. At most, they'd pull some sort of reverse Jan 6, which will help Trump and/or the neolibs who secretly support similar policies.

BTW, that lentils thing is dumber than the vegetable, the effects would be no better than open riots. No violence, property damage or "sit on the road" type of disruptions can do any good in the current situation. Lawyers, truthful propaganda and organization can help but these are in short supply and "tariffed" extensively.
by Aurornis | 2025-05-10 18:47:03+00:00
> Crawling the internet is a natural monopoly. Nobody wants an endless stream of bots crawling their site,

Companies want traffic from any source they can get. They welcome every search engine crawler that comes along because every little exposure translates to incremental chances at revenue or growing audience.

I doubt many people are doing things to allow Googlebot but also ban other search crawlers.

> My heart just wants Google to burn to the ground

I think there’s a lot of that in this thread and it’s opening the door to some mental gymnastics like the above claim about Google being the only crawler allowed to index the internet.
by keeda | 2025-05-10 18:47:35+00:00
> Yes, Microsoft is one of the world's biggest companies, and it underinvests in research and development, preferring to hoard cash.

I'm not sure if you're being sarcastic, but:

1. If you're talking about basic research, Microsoft Research has been a thing since the 90's, is highly prestigious, and has published far more papers than Google, based on their respective research websites. (To be fair, Google started much later.)

2. If you're talking about product development, MSFT is vastly more diversified in terms of revenues than any of the "Magnificent 7" because of their varied product lineup.

3. The basis of their relationship with OpenAI is literally them investing double digit billions to catch up on the AI race once they recognized the opportunity.

> OpenAI is in effect a client state of Microsoft that Microsoft is using to make Google look flat-footed and force them to enter the chatbot market.

I'm not sure about Microsoft's influence in OpenAI's strategy, but it's pretty clear Google was caught flatfooted by their own strategy of locking away transformer technology behind products that didn't threaten their search monopoly. There's a reason the researchers who invented transformers had to leave and start a different company to bring its true potential to the market. Which, even if it was just a chatbot, is what has kicked off the AI boom.
by kelnos | 2025-05-10 18:48:03+00:00
> They were meant for that.

No they weren't. If we look at it from the perspective of pipelines (and not interactive programs that take input directly from the user and display output on the screen), stdin is for receiving data from the program in the pipeline before you, and stdout is for sending data to the thing in the pipeline after you. That's not bidirectional, that's a unidirectional flow.
by FireSquid2006 | 2025-05-10 18:48:10+00:00
Not just for math kids--good for anyone.

I have a "daybook" thats always with me. Each day gets a header with the date. I treat it as a "cache" for what I'm working on. Math problems for school, jotting down ideas for programming, having a dialog with myself while debugging (also a good tool).

When one gets full you put it on the shelf. Fun to look back on what you were working on a month/year/5 years ago
by HaZeust | 2025-05-10 18:48:31+00:00
I always liked the theory that we're living in an age where all of our needs can be reasonably met, and we now have enough time to think - in general. We're not working 12 hour days on a field, we're not stalking prey for 5 miles, we have adequate time in our day-to-day to think about things - and ponder - and reflect; and the ability to do so leads to thoughts and epiphanies in people that therapy helps with. We also have more information at our disposal than ever, and can see new perspectives and ideas to combat and cope with - that one previously didn't need to consider or encounter.

We've also stigmatized a lot of the things that folks previously used to cope (tobacco, alcohol), and have loosened our stigma on mental health and the management thereof.
by ck2 | 2025-05-10 18:49:27+00:00
It's somewhat of a demonstration/argument of the anthropic principle

Because we're here and that appears to be a "special case" and rare and no other life spotted (yet) elsewhere, it might very well be the reason we are here at all (that we haven't figured out yet)

It certainly enabled math and science to progress because it was accidentally possible to calculate distances because of that size/distance particularity even before telescopes and computers, though I realize that's not biological evolution
by acroyear | 2025-05-10 18:49:41+00:00
NeXT was a failed rocket launch (analogous to early rocket failures within SpaceX). A great step forward and a necessary step in the evolution of the PC. I thought NeXT workstations were pretty bad-ass for their time and place. Recall that only 3 years prior to NeXT, was computers like the Atari ST .. what a vast difference!!
by kelnos | 2025-05-10 18:50:18+00:00
You're not the intended audience for this blog post. The people who care about this are the kinds of people who have to implement the protocol on either end, and deal with all of those complexities.

You won't be able to fully insulate yourself from those complexities, though. Unnecessary complexity causes user-visible bugs and incompatibilities. You want to reach for a framework that will abstract all this stuff away, but because of poorly-designed protocols like MCP, those frameworks will end up being more unreliable than they need to be, in ways that will leak out to you.
by lokar | 2025-05-10 18:51:03+00:00
It depends what you mean by solved.

Banks tend not to over engineer things. KYC can be seen as a 3 sided trade off: cost of KYC infra/process/etc, lost revenue from denied business and fines from regulators.

They (the banks) don’t really care about the social goals of KYC, they just try to best optimize for expected value in the trade offs.

The regulators understand this, and are basically fine with it. They have their own trade offs they are balancing.

Both sides mostly find and equilibrium.
by jpeloquin | 2025-05-10 18:51:56+00:00
Publications with public funding have already escaped the paywall, partially as of 2013 and completely as of this year:

https://par.nsf.gov/

https://pmc.ncbi.nlm.nih.gov/

https://ospo.gwu.edu/overview-us-policy-open-access-and-open...

https://www.nih.gov/about-nih/who-we-are/nih-director/statem...

https://www.coalition-s.org/plan_s_principles/

The intent of the Bayh-Dole Act was to deal with a perceived problem of government-owned patents being investor-unfriendly. At the time the government would only grant non-exclusive licenses, and investors generally want exclusivity. That may have been the actual problem, moreso than who owned the patent. On the other hand, giving the actual inventors an incentive to commercialize their work should increase their productivity and the chance that the inventions actually get used.
by beloch | 2025-05-10 18:52:32+00:00
Museums are sort of like farms in that you can lose money every year for entire lifetimes and still have a tremendously valuable farm. Like land, the past is an appreciating asset. That's big, traditional museums. I suppose it should be no surprise that the smaller museums are still struggling in the post-covid era. I based my opinion on visits prior to the pandemic, so I'm out of date. Hopefully the popularity of museums in Sweden will rebound.
by nojvek | 2025-05-10 18:52:33+00:00
Changing Brand blah blah blah. Seeems they hired a Brand executive and they want to work their magic but the message is tone deaf.

SO’s target market is developers looking for help with their coding problems. LLMs are a superior interface.

If one is answering a new question, SO is hostile in actively marking it as dupe. Or the Q never gets answered.

No amount of brand dancing will get to the root problem.

SO losing its core audience because it doesn’t do a good job of serving them.
by Aurornis | 2025-05-10 18:52:46+00:00
> that I don’t blame site owners for blocking all bots except googlebot.

I doubt this is happening outside of a few small hobbyist websites where crawler traffic looks significant relative to human traffic. Even among those, it’s so common to move to static hosting with essentially zero cost and/or sign up for free tiers of CDNs that it’s just not worth it outside of edge cases like trying to host public-facing Gitlab instances with large projects.

Even then, the ROI on setting up proper caching and rate limiting far outweighs the ROI on trying to play whack-a-mole with non-Google bots.

Even if someone did go to all the lengths to try to block the majority of bots, I have a really hard time believing they wouldn’t take the extra 10 minutes to look up the other major crawlers and put those on the allow list, too.

This whole argument about sites going to great lengths to block search indexers but then stopping just short of allowing a couple more of the well-known ones feels like mental gymnastics for a situation that doesn’t occur.
by myself248 | 2025-05-10 18:52:59+00:00
I remember reading about naive circuits like ripple-carry, where a signal has to propagate across the whole width of a register before it's valid. These seem like they'd only work in systems with very slow clocks relative to the logic itself.

In this writeup, something that jumps out at me is the use of the equality bus, and Manchester carry chain, and I'm sure there are more similar tricks to do things quickly.

When did the transition happen? Or were the shortcuts always used, and the naive implementations exist only in textbooks?
by dang | 2025-05-10 18:54:17+00:00
> Meanwhile, YC has happily and excitedly fed it's start-ups to Google over the years.

I'm curious what you've seen or heard that led you to that conclusion? It's the opposite of correct.

YC supports what founders want, including if they want to sell to $BigCo, but such outcomes are hardly successes for YC. YC's success depends on outlier companies growing much larger than that.
by kurikuri | 2025-05-10 18:54:45+00:00
> But it can easily assign equal scores to 1 and 0 and zero to other tokens, and you’ll have to sample it randomly to produce the result. Whether you consider it external or internal doesn’t matter, transformers are inherently probabilistic by design.

The transformer is operating on the probability functions in a fully deterministic fashion, you might be missing the forest for the trees here. In your hypothetical, the transformer does not have a non-deterministic way of selecting the 1 or 0 token, so it will rely on a noise source which can. It does not produce any randomness at all.
by wvoch235 | 2025-05-10 18:54:50+00:00
The problem with the European mindset on this, is it's always involves bureaucrats taking their taxpayers money and allocating it in smarter ways than American investors who are doing it with their own money.

If that seems unlikely to work to you, then you possess critical thinking.

The US spends more on R&D (Private and Public) than the next 5 countries combined. Public research is and since the 70s has been a small fraction of research spending in the US. That's why their companies actually innovate.

If Europe doesn't change the inventive structures that are preventing investment in R&D, no amount of government money is going to fill that void...
by numpad0 | 2025-05-10 18:55:46+00:00
There is no single company from Asia that market electronic physical experience. Other than Nintendo. IOW there's Nintendo.

East Asian companies hasn't shifted into maximizing perceived values over functions. I think it just feels fake and wrong to many. Another factor that I think might exist is, it might be simply hard to inflate values and sell experiences as East Asian company entities without good connections and/or cultural understanding to sell to developed Western markets.

East Asian engineers don't share the pain points as Western audiences. People don't use webcams - less sympathies exist for the urge to see faces. People don't use Linux on laptop - Windows is normal and fine. Don't recognize annoyances as annoying as often - it's considered signs of weakness. (Not sure about Nintendo complaints - people everywhere happily pay $80 for another Mario remakes on top of $500 console. Isn't that what you're asking for?). Kids aren't as interested in ROM cooking or piracy or software freedom in general - way more interested in enforcing IP rights themselves than demanding something from IP holders.

Or, widespread "just don't give a shit because they can't compete with Asian price dumpers" could be another reason. East Asian nations each has its own internal markets with massive surplus production capacities. Western companies had progressively moved into value engineering for survival, as Swiss watch industry famously did. The costs of East Asian physical products never represented its cost in the first place, only utility in context of economy at export destinations, and that might have affected how companies at the destinations have come to be.
by smj-edison | 2025-05-10 18:56:21+00:00
It makes me think of the idea of legibility[1]: it's hard to understand incredibly large and complicated systems, so the solution is to simplify, and then enforce that simplification. From a high-level perspective, it seems reasonable to surveil everyone, to help them. I think someone really can delude themselves into thinking that it's necessary, utilitarian-style.

Now, if we designed technology for humans, we'd realize that most humans have local networks of trust. E.g. I talk to my friend in person, she tells me her discord handle, now we've established trust. In addition, trust is something that's gradually built, not given all at once in a EULA[2].

[1]: https://www.ribbonfarm.com/2010/07/26/a-big-little-idea-call...

[2]: https://ruben.verborgh.org/blog/2024/10/15/trust-takes-time/
by joezydeco | 2025-05-10 18:56:24+00:00
Friend of mine is in the commercial real estate business, leases lots of warehouses to big names. He says he's seeing a LOT of uptake on the east coast: Savannah, Jacksonville FL, Charleston.

A lot of companies are shifting to production in India, Pakistan, Vietnam, and it's easier to ship through Suez to the east coast from there.
by threeseed | 2025-05-10 18:56:38+00:00
Of course it is good for YC to do this. They have significant investments in OpenAI both directly and indirectly through the countless startups they've funded whose core is OpenAI.

And it's ridiculous to act like (a) you are forced to go through Google to access the 'market' and (b) that this is somehow unusual or untoward. They are an advertising company and not the only one.
by phren0logy | 2025-05-10 18:56:41+00:00
It's called a "waitlist" control group, and it's not intended to represent placebo. Or at least, it shouldn't be billed that way. It's not an ideal study design, but it's common enough that you could use it to compare one therapy to another based on their results vs a waitlist control. Placebo control for psychotherapy is tricky and more expensive, and can be hard to get the funding to do it properly.
by mrweasel | 2025-05-10 18:57:33+00:00
Probably a combination of things, I wouldn't pretend to know, but I have my theories. For men, one half-backed thought I've been having revolved around social circles, friends and places outside work or home. I'm a member in a "men only" sports club (we have a few exceptions due to a special program, but mostly it's men only). One of the older gentlemen, probably in his early 80s, made the comment: "It's important for men to socialise with other men, without women. Young and old men have a lot in common, and have a lot to talk about. An 18 year old woman, and an 80 year old man have very little in of shared interests or concerns."

What I notice is that the old members keep the younger members engaged socially, teach them skills and give them access to their extensive network of friends, family, previous (or current) co-workers, bosses, managers. They give advise, teach how to behave and so on. The younger members help out with moving, help with technology, call an ISP, drive others home, to the hospital and help maintain the facilities.

Regardless of age, there's always some dude you can talk to, or knows who you need to talk to, and sometimes there's even someone who knows how to make your problems go away or take you in if need by.

A former colleague had something similar, a complete ready so go support network in his old-boys football team. Ready to support in anyway they could, when he started his own software company.

The problem: This is something like 250 guys. What about the rest? Everyone needs a support network, if your alone, or your family isn't the best, you only have a few superficial friends, if any, then where do you go? Maybe the people around you aren't equipped to help you with your problems, not everyone is, some have their own issues. The safe spaces are mostly gone.

We can't even start up support networks, because the strongest have no reason to go, so we risk creating networks of people dragging each other down. The sports clubs works because members are from a wider part of society.

From the article:

> > Meta said its AIs carry a disclaimer that “indicates the responses are generated by AI to help people understand their limitations”.

That's a problem, because most likely to turn to an LLM for mental support don't understand the limitations. They need strong people to support and guide them, and maybe tell them that talking to a probability engine isn't the smartest choice, and take them on a walk instead.
by mattkevan | 2025-05-10 18:58:25+00:00
Disagree. 9-5 working is fine, and probably more efficient long term than permanent crunches.

Organisations fail when the ‘business’ people take over. People who let short term money-thinking make the decisions, instead good taste, vision or judgement.

Think Intel when they turned down making the iPhone chips because they didn’t think it’d be profitable enough, or Google’s head of advertising (same guy who killed yahoo search) degrading search results to improve ad revenue.

Apple have been remarkably immune to it post-Jobs, but it’s clear that’s on the way out with the recent revelations about in-app purchases.
by RainyDayTmrw | 2025-05-10 18:58:47+00:00
Make no mistake. This is, first and foremost, a big, for-profit corporation fighting a bigger, for-profit corporation, for its own financial interests. Nevertheless, we may stand to benefit, if only incidentally.

In particular, if the legal authorities start to unwind Google, I actually think Chrome and Android are more important to wall off or spin out than anything advertising or AI related.
by zrm | 2025-05-10 18:59:57+00:00
This seems like two different issues.

One is, suppose there are a thousand search engine bots. Then what you want is some standard facility to say "please give me a list of every resources on this site that has changed since <timestamp>" so they can each get a diff from the last time they crawled your site. Uploading each resource on the site to each of a thousand bots once is going to be irrelevant to a site serving millions of users (because it's a trivial percentage) and to a site with a small amount of content (because it's a small absolute number), which together constitute the vast majority of all sites.

The other is, there are aggressive bots that will try to scrape your entire site five times a day even if nothing has changed and ignore robots.txt. But then you set traps like disallowing something in robots.txt and then ban anything that tries to access it, which doesn't affect legitimate search engine crawlers because they respect robots.txt.
Is current state of querying on observability data broken?
by pranay01 | 2025-05-10 19:00:39+00:00
Hey folks! I’m a maintainer at SigNoz[0] , an open-source observability platform

Looking to get some feedback on my observations on querying for o11y and if this resonates with more folks here

I feel that current observability tooling significantly lags behind user expectations by failing to support a critical capability: querying across different telemetry signals.

This limitation turns what should be powerful correlation capabilities into mere “correlation theater”, a superficial simulation of insights rather than true analytical power.

Here’s the current gaps I see

1/ Suppose I want to retrieve logs from the host which have the highest CPU in the last 13 minutes. It’s not possible to query this seamlessly today unless you query the metrics first and paste the results into logs query builder and retrieve your results. Seamless correlation across signal querying is nearly impossible today.

2/ COUNT distinct on multiple columns is not possible today. Most platforms let you perform a count distinct on one col, say count unique of source OR count unique of host OR count unique of service etc. Adding multiple dimensions and drilling down deeper into this is also a serious pain-point.

and some points on how we at SigNoz are thinking these gaps can be addressed,

1/ Sub-query support: The ability to use the results of one query as input to another, mainly for getting filtered output

2/ Cross-signal joins: Support for joining data across different telemetry signals, for seeing signals side-by-side along with a couple of more stuff.

Early thoughts in this blog[1], what do you think? does it resonate or seems like a use case not many ppl have?

[0] https://github.com/signoz/signoz [1] https://signoz.io/blog/observability-requires-querying-across-signals/