MurphyA/hacker-news huggingface dataset:
Show HN: I'm Building a Decentralized Amazon
by astronautmonkey | 2025-05-20 20:57:12+00:00
Hi HN! Agora is an AI search engine for e-commerce products. We've indexed 21 million products sold on 62,000 different Shopify and WooCommerce stores and made them easy for customers to search and purchase. Think of a decentralized Amazon.

I originally built Agora as a side project and launched it here on HN. The search took 5 - 10 seconds to return results and the app looked like it was designed in MS Paint (by yours truly). https://news.ycombinator.com/item?id=38635695

Since the MVP launch, here are a few technical challenges that we've been working on:

1. Figured out how to return results in under 300ms. We are using a self-hosted version of Meilisearch on a high memory Digital Ocean server. The app is built with Remix, specifically to optimize for pre-fetching of data. To power the AI search, we convert our products into both text and image embeddings. Text embeddings with Cohere and image embeddings with Clip. We are now working on further server optimizations and data sharding to get the search speed under 200ms.

2. Expanded the crawler to index stores built with Shopify, WooCommerce, Squarespace, and Wix. Additionally, we adapted the crawler to index major retailers including Walmart, Target, Macy's, and Nordstroms. We also built a "recrawler" that updates all stores on Agora every 24 hours. Then when a user clicks on a product, we do a live check of the variants and stock. This ensures that product data is always up to date.

3. Added deeper e-commerce query detection. We noticed that some searches are objective keywords like "camping tent" and others are subjective queries like "what is the best camping tent under $500". When a user searches on Agora, we pass the query through a text classifier to determine if it's subjective or objective, then change the AI tooling and UI accordingly. We served 2.1 million search queries in 2024 and are now using this data to get even more precise about query detection.

My vision for Agora is to aggregate the world's e-commerce products into one place. I believe this will create market efficiencies for customers, developers, and merchants. I'd love your feedback!
by BriggyDwiggs42 | 2025-05-20 20:57:29+00:00
I’m referencing a paper I saw in passing multiple years ago, so forgive me if I didn’t elaborate the exact algorithm. The LLM varies its word selection in a patterned way, eg most likely word, 2nd most, 1st, 2nd, and so on. It’s statistically impossible for an esl person to happen to do this on accident.
by bluGill | 2025-05-20 20:57:58+00:00
It is a problem for me. Google is reasonable enough, but I do not want it to be too easy to make a mistake. I want those extra seconds it takes to type in my payment information to think again do I really need this thing. More than once I've realized the "toy" I was interested in wasn't really in my current budget and abandoned the purchase. I never turned on amazon one-click shopping for similar reasons. I want buying things I need easy, but not too easy.
by asdff | 2025-05-20 20:58:10+00:00
It also correlates quite well to available flat land for greenfield development running dry in major metro areas like the non-antelope valley side of LA county. And what do you know, prices are much cheaper on the other side of the San Gabriels where there is room to develop more housing. Without room for greenfield, infill needs to be legalized, or prices rise such that the only winners are the high income people able to participate in the now buyer constrained housing market like we see today.
by scantis | 2025-05-20 20:58:45+00:00
A law should be broad, to be applicable for a long time and multiple situations, yet clear enough not to be misunderstood.

For example, once it is decided if conduct is deemed unfair or dangerous, there should be little interpretation on the consequences and they should differ.

Weirdly laws are usually very specific on what is deemed unfair, yet rarely on what is dangerous.
by itissid | 2025-05-20 20:58:47+00:00
Who is doing all the work of making physical agents that can behave as good as a UBI generator? Something that can not just create videos, but go get groceries(hell grow my food), help a construction worker lay down tiling, help a nurse fetch supplies.

https://www.figure.ai/ does not exist yet, at least not for the masses. Why are Meta and Google just building the next coder and not the next robot?

Its because those problem are at the bottom of the economic ladder. But they have the money for it and it would create so much abundance, it would crash the cost of living and free up human labor to imagine and do things more creatively than whatever Veo 4 can ever do.
by jimt1234 | 2025-05-20 20:58:53+00:00
One of my biggest regrets in my "journey" with computers is walking away from Commodore when Amiga was released. I felt it was superior to anything else I had seen, and I knew the Commodore 64 inside-out, but I just felt like PCs were for grown ups, and I needed to grow up. I needed to skate to where the puck was going, and that wasn't Commodore. I guess I was right, but I still regret it because all the dudes I knew that stuck with Commodore, with the new Amiga platform - well, they all seemed to be having more fun. I learned macros for Lotus 1-2-3, which was more practical (I made $$$ as a teenager teaching stiffs about macros), but my Amiga friends were making cool drum beats and sample-based music and remixes on their Amigas - totally impractical, but also fun as shit back then. So yeah, they were all having fun with their Amigas, while I became Alex P. Keaton.
by tuatoru | 2025-05-20 20:58:54+00:00
This sounds utopian, but it ignores human nature. Reduce the cost of a good or service, you will get more of it.

Part of the cost of driving is the opportunity cost, the inability to do something else at the same time. That is the value proposition for autonomous vehicles, saving that opportunity cost.

If delivery costs go down, you will get more deliveries. And there will be "zombie" trips - return-to-base after a delivery run, providing no value but taking up road space.

The assumption of universal ride sharing ignores the needs of families, so they will be pushed out of cities to a greater extent than they already are.
by anon291 | 2025-05-20 20:59:11+00:00
I have very good drill bits. Old growth wood is like nothing you've ever seen before. I'd rather have a house made of it than anything else. Stuff is as hard as a rock. Harder I'd bet. My drill has no issue driving through metal. The thing about old growth wood is that as you drill in, the hole gets 'smaller' behind you. The wood fibers are highly packed together. When you make a hole, they sort of expand behind the bit. If you're using a spade bit, it can even be hard to pull it back through.

I'm also an amateur woodworker, and if you've ever dealt with ipe wood from Brazil, old growth pine is harder than that. I made a nice mantelpiece out of ipe and my bits had no problem with that. Old growth pine is very dense.
by palmotea | 2025-05-20 20:59:24+00:00
>> And API is just a set of functions and types that determine an interface.

> That's the kind of thinking that leads to janky APIs. When you say "just" you're doing the same thing the LLM does - you're removing all the nuance and complexity from the activity.

I wonder if that kind of thinking is a component of the difference between LLM fans and LLM skeptics. If the programmer who is not sensitive to "nuance and complexity" gets something lacking it from an LLM, they're happy today. A programmer who is sensitive to "nuance and complexity" would be unhappy with the same response.
by nchmy | 2025-05-20 21:00:58+00:00
Have you considered that its a much larger job than someone can just bootstrap? Also, Node itself was initially sponsored by a corporate entity, and it lead to considerable problems as well. Now it has backing in many other ways.

Also, are you aware that Deno is being built by literally the creator of Node? This isn't some get rich quick scheme - its something that he deeply wants to see exist. He's also leading the charge against Oracle (a genuine parasite) for the copyright/trademark of Javascript.
by drdeca | 2025-05-20 21:01:16+00:00
I don't see how Gödel's theorems at all shows that "the introduction of the idea of the truth erodes the truth". Gödel's incompleteness theorems are mostly about provability anyway, which is a distinct concept from truth.

Though, Gödel's completeness theorem I suppose does relate provability to truth in that it shows that (for systems with the right kind of rules of inference) provability is equivalent to something being true in all models...

Still, are you sure Tarski's undefinability theorem isn't more relevant to your point?
by amlib | 2025-05-20 21:01:29+00:00
No, 200% is rendering 4 pixels with "features" 2x larger in each axis. You may get 200% scaling as you said with some legacy apps that give zero fucks about dpi scaling but are still scaled trough some mechanism to properly match other apps.

Fractional scaling has been a problem across all platforms, but I agree Linux has taken its time to get it right and still have some gotchas. You should try to avoid it in any platform honestly, you can get sometimes get blurry apps even in Windows. AFAIK KDE is the first to get it right in this complex situations where you mix multiple monitors with different fractional scaling ratios and have legacy apps to boot. GNOME has had experimental fractional scaling for a while but it's still hidden behind a flag.

It also helps to not have nVidia trash on your old (and sometimes even new) computers if you want longevity. My old machines have intel and AMD graphics with full support from current kernel and mesa.
by TeMPOraL | 2025-05-20 21:02:35+00:00
Long-time StumpWM user, before I switched back to Windows a few years ago. This is super-exciting to see, and I'm going to take it for a spin. It might just address my major frustrations with arranging windows and switching between them; my monitor seems just the right shape/resolution for the standard Windows splits to be suboptimal.

(Browsers, in particular, I use full-screen less and less. That annoying trend of squeezing everything into short lines "because readability" is just wasting too much screen space; zooming in makes everything too big, and I'm getting tired of writing userstyles or userscript to fix it for every other page I open, so I'm back to keeping 2 or 3 columns of windows running.)

Also, any excuse to use more Lisp is good in my book. Based on the screenshots, it looks stellar; if it works half as well as it comes across, I'll switch over instantly.
by cadamsdotcom | 2025-05-20 21:02:56+00:00
One way to improve the rigor of this research would be to embed each word in each language in a common latent space then look for clusters.

But creating that latent space and the corresponding embedding algorithm is hard in the first place. Today’s embedding models could be terrible for the fringe languages this research is about, and we wouldn’t know because we don’t know how to evaluate overall semantic accuracy.

Am I off piste here?
by ankit219 | 2025-05-20 21:02:57+00:00
I simplified what i wrote. There is an off accelerator memory where the embeddings are stored and queried at inference time, i did not want to get into details. That is how you reduce the in memory RAM. There are definitely more things going on in the paper as it builds upon the concept I described. The central idea remains the same: you have input embedding layers which map text to continuous vectors. Instead of loading all these layers at runtime, you can break it per layer at training time, and then fetch the required ones from a separate store during inference. Would not be in RAM. Per layer is not mentioned in the paper. But surely it's not a great leap from the paper itself?
by nchmy | 2025-05-20 21:03:12+00:00
Evidently people didn't sufficiently value the technical promise of Deno and just wanted a (MUCH) better node. But, it also has plenty of new, bold, extra things (a sibling comment elaborates on it much better than I can). I, for one, am quite happy with it.

Given that you still use Node, you might want to try Deno 2 out... It'll likely solve a lot of your headaches.
by asdff | 2025-05-20 21:03:30+00:00
At the same time cost of living has not risen so much in certain parts of the country. You are paying say $1.2m in CA for a modest home worth say $90k in Toledo, Ohio. Labor certainly isn't so depressed in Ohio vs CA, it is a little but not by a factor of over 10 fold like we see in comps in the housing market. This is why it is important to recognize the regional factors that lead to these differential outcomes IMO, and you can't consider that without considering the state of the housing market. Everywhere had the same Fed but we see dramatically different outcomes.
by echelon | 2025-05-20 21:04:32+00:00
Sora, the video model, is shit. Kling, Runway, and a whole host of other models are better. You don't have to do much to be better than Sora.

Sora, the image model (gpt-image-1), is phenomenal and is the best-in-class.

I can't wait to see where the new Imagen and Veo stack up.
by bluGill | 2025-05-20 21:05:57+00:00
Why would I want a virtual credit card - US law gives me strong fraud protection and so if my number is compromised I just call my bank and dispute the charges and then I get a new number.

Though I've never had the above happen. I've had a few times where my number was compromised but the bank found out and gave me a new card before whoever got the number was able to use it.
by ir77 | 2025-05-20 21:06:31+00:00
people here keep saying that this is targeted at big companies/corporations. the big company that i work for explicitly block uploads of data to these services and we're forbidden to put anything company related in there for many reasons, even if you use your own account, we don't have 'company accounts'.

so no, i can't see companies getting all excited about buying $250mo/user licenses for their employees for google or chatgpt to suck in their proprietary data.
by TeMPOraL | 2025-05-20 21:07:13+00:00
That is amazing and I hope it'll continue to be so :).

The Linux scene isn't bad either (or at least it wasn't 4+ years ago when I was into this); I've used StumpWM as a daily driver for many years, and while it was definitely niche, I still saw friendly exchange of ideas and experiences with people using and/or contributing to dwm, i3, and ratpoison.

(Then there's EXWM, but I never really mustered the courage necessary to try it.)
by bilbo0s | 2025-05-20 21:07:34+00:00
I was thinking that the entire time I read HN User onlyrealcuzzo's comment.

Why, on Earth, would Apple ever want to solve the problem of Apps taking up more space? That's just not good business. Way better business right now to put R&D into increased memory access speeds.

Apple would need to have a different business model entirely for them to have a business case for fixing this. They may fix it because they just want to help out they AI guys? Maybe in the future they're getting money from the AI guys or something? So fixing it starts to make a lot of sense.

But all other things being equal, the money for Apple is in this not being fixed.
by psanford | 2025-05-20 21:07:44+00:00
This looks great! A few years ago I wrote a sqlite vfs for using dynamodb as a backing store[0] called DonutDB. With the recent addition of CAS to S3, I was thinking about making a new version of DonutDB backed by S3. I'm really glad lightstream supports this so I don't have to!

I can't wait to try this out.

[0]: https://github.com/psanford/donutdb
by MrScruff | 2025-05-20 21:08:00+00:00
I don't think this is just about convenience - you're not going to get these results with a 14B video model. I'd much prefer to have something I could hack on in ComfyUI but the open weights models don't compete with this anymore than a 32B LLM competes with Gemini 2.5 Pro for coding. And at least in coding you can easily edit the output from the LLM regardless...
by vharuck | 2025-05-20 21:08:01+00:00
I imagine that any dump of government communications will contain sensitive information about citizens or government employees who didn't directly engage in the chats. Soldiers, contractors, patients in a database. Especially if Congressional Representatives have their chats leaked. One of their roles is helping constituents work through red tape. Mine sends a weekly email tooting his own horn, including how many people he helped with social security or getting VA benefits.

I'm not saying these chats shouldn't be released. But I'd hope the names and other identifying info of people who weren't uninvolved would be redacted, just keeping the context to show what kind of information was being carelessly shared. Of course, given the admin's shamelessness, they'd claim anything with redacted info was faked. It might be better to leave it verifiable.
by randcraw | 2025-05-20 21:08:28+00:00
Yes, but I have no personal knowledge about radical prophylactic prostatectomy. You might start here:

https://www.researchgate.net/publication/361217707_The_role_...

I've read very little about choosing radical prostatectomy very early after detection, but it's likely that it does little to improve survivability:

https://medicine.washu.edu/news/surgery-early-prostate-cance...

That said, if nerve-sparing surgery were done early instead of doing NON-nerve-sparing surgery later (a standard radical prostatectomy), perhaps that might diminish some of the typical side-effects of the standard surgery like impotence or incontinence. But I'm only speculating.
by dvfjsdhgfv | 2025-05-20 21:09:55+00:00
There is no single reason. Nobody will argue that LLMs are already quite useful at some tasks if used properly.

As for the opposing view, there are so many reasons.

* Founders and other people who bet their money on AI try to pump up the hype in spite of problems with delivery

* We know some of them are plainly lying, but the general public doesn't

* They repeat their assumptions as facts ("AI will replace most X and Y jobs by year Z")

* We clearly see that the enormous development of LLMs has plateaued but they try to convince the general public it's the contrary

* We see the difference on how a single individual (Aaron Swartz) is treated when making a small copyright infringement, and how the consequences for AI companies like OpenAI or Meta who copied the whole contents of Libgen are non-existent.

* Some people like me just hate AI slop - in writing and imaging. It just puts me off and I stop reading/watching etc.

There are many more points like this.
by aaronbaugher | 2025-05-20 21:09:59+00:00
I stayed with Commodore, but with my 128, since I couldn't afford an Amiga. I envied my PC friends their VGA graphics, but yeah, the Commodores were a lot of fun. And I learned to program on it, which none of my friends were doing with their PCs. Once they went to PC, it seemed like they just used them to run software, and weren't interested in tinkering.
by Uehreka | 2025-05-20 21:10:41+00:00
In the short term perhaps, but even without carbon pricing the raw electricity prices will probably tamp down the enthusiasm. At someone point it’ll become cool for activist investors to demand to see ROI for AI features on earnings calls, and then the fat will get trimmed just like any other trend that goes too far.

I think the bigger underrated concern is if LLMs fall into an unfortunate bucket where they are in fact generally useful, but not in ways that help us decarbonize our energy supply (or that do, but not enough to offset their own energy usage).
by Clamchop | 2025-05-20 21:10:42+00:00
There's a pretty standard argument that creating artworks should or must require the hardship of developing the skill to bring vision into reality, or paying someone who can. That can be debated, but the position is textbook gatekeeping.

Other disapproval comes from different emotional places: a retreading of ludditism borne out of job insecurity, criticism of a dystopia where we've automated away the creative experience of being human but kept the grim work, or perceptions of theft or plagiarism.

Whether AI has worked well for you isn't just irrelevant, but contrarian in the face of clear and present value to a lot of people. You can be disgusted with it but you can't claim it isn't there.
by asdff | 2025-05-20 21:10:49+00:00
Plenty of software has zero ML. But either way not all ML is the same. There are many different algorithms each with their own tradeoffs. AI as it is presently marketed however usually means one type of AI, large language model, which also has tradeoffs, and is a bit new to the scene compared to say markov chains whose history starts in the early 1900s.
by vintermann | 2025-05-20 21:10:53+00:00
> No, I don't read that at all. There's plenty of Russian propaganda that Westerners have fallen for hook, line, and sinker, chief among them the idea that all Russian speakers are actually Russian and want to be a part of the Russia.

Oh yeah, other westerners, but not you. You take the foreign policy think tank line that Russians actually want to be balkanized. Just after saying that Putin has succeeded in brainwashing the population to go along with whatever he wants without need for excuses based on good points.
by gloosx | 2025-05-20 21:11:02+00:00
If your focus is to solve the problem, then it makes sense to treat the process as secondary. The tools are just means to an end.

This view also aligns with how generative AI is marketed – it's a way to accelerate realization, not a way to focus on the act of crafting.

That said, outcome-first thinking does run the risk of disconnection, and our current culture is all about disconnection.
by kkarakk | 2025-05-20 21:12:38+00:00
the point is not that AI services will be affordable "eventually" it's that the advantage is so crazy that people who don't have access to them will NEVER be able to catch up. First AI wrappers disrupt industries ->developing nations can't compete coz the services are priced prohibitively -> AI wrappers take over even more -> automation disrupts the need for anyone -> developing nations never develop further. this seems more and more likely not less. cutting edge GPUs for eg - already are going into the stratosphere pricing wise and are additionally being sanctioned off.
by oblio | 2025-05-20 21:13:09+00:00
> Just like fear of "greater finland" made the Soviets invade in the first place.

Which "fear" prompted the Soviets to invade Romania in 1940? Which "fear" prompted the Soviets to invade Poland in 1939? Which "fear" prompted the Soviets to invade the Baltics in 1940?

Ah, now I remember, the "fear" of not being the premier colonial power.
by tootie | 2025-05-20 21:13:29+00:00
Is your argument "humans are fallible so we should give up?" The effectiveness (and sincerity) of DEI programs have certainly been mixed, but they should produce measurable outcomes that can be iterated and improved upon. A lot of the commentary in this thread and the rhetoric from the White House is that DEI is bad idea and should not be attempted because we shouldn't seek equity.
by sokoloff | 2025-05-20 21:14:05+00:00
I think in the success case (still TBD), that it will increase productivity to the point where things that can’t be affordably addressed by software will now be able to be addressed with software.

I expect that anyone who is a skilled dev today will be fine. Expectations and competition might be higher, but so will production and value creation.

I think the demand will come, just as Excel didn’t put finance people out of jobs in aggregate.