skysight-inc/HackerNews huggingface dataset:
All of this is a long series of solvable problems. I should know, I've dabbled in solving most of them. This is why I suggest actually taking a stab at it before you dismiss it as impossible.

There are some problems that aren't as big as they seem. Parts of an SPA can't be reliably linked to anyway even if you find interesting text there, so you can just leave them out of the index.

Likewise, there isn't as great of a need to keep a fresh index as it may seem. The odds of a document changing is proportional to how frequently it changes. This is a bit of a paradox, where even if you crawl really aggressively, the most frequently changing documents will still always be out of date. Most documents are relatively stable over time. You can actually use how often you see changes to a document or website to modulate how often you crawl it.

The bad HTML is quite manageable. You really just need to flatten the document to get at the visible text. Even with really broken formatting, that's manageable.

The storage demands are also not as bad as you might think (most documents are tiny, sub 10 Kb), there are ways to lessen the blow on top of that. Both text and indexes can compress extremely well. Since you're paying for disk access by the block, you might as well cram more stuff into a block.

Most of the crawling concerns, in general, can be gotten around by starting off with Common Crawl (even if I do my own crawling, which also is finnicky but manageable).

> This is relatively straightforward for a limited search and document space up to a few million entries in your DB. A few million documents should be doable with off the shelf parts.

Right, so shouldn't the question be how to find the documents that are even candidates for being search results? Most documents are not ever going to be relevant to any query ever. Get rid of that noise and your hardware goes a lot longer.

I'm running a search engine on consumer hardware out of my living room that can index 100 million documents. Go a bit higher budget than a consumer PC, and you've got 5 billion. That goes a long way.
Is it so hard to believe that the recommendation engine is liberal? It’s true of HN too.

It’s not a bad thing, the same way that Fox News being conservative isn’t a bad thing. It just is what it is.

These are major news sites. News always has a bias. It’s important to identify, for one’s own sake.

It’s not even particularly hard to bias an AI algorithm. It’s arguably the default.
How is DuckDuckGo a worse version of Google?

I get less ads, more privacy, and pretty much the same or better results. Plus I can use my down arrows to traverse the results list!
This game looks absolutely fascinating. Getting it immediately - thanks for recommendation!
This may very well be the case, but it was years ago so I don't really know. This type of linking is something that always gives me the willies when it happens without warning and makes me think I should stay away from using components that are too "smart." I got really freaked out once when I was using a portable wireless access point owned by my employer (for on-call duties) in a starbucks and it told me a second device had attached to it. Turns out I had just bought a new iphone and Apple had stashed away the wifi password and automatically given it to the new phone, rather than the explanation being some hacker sitting near me in starbucks. I completely understand why they did this, but it really would have been nice to be asked if I mind if they store a copy of my password in the cloud.
I mean, the entire dataset should become available so you can look at any particular instance of the map, if you want, just create a snapshot of the map right before the white erasure started. Or find a high quality recording of a timelapse and screenshot your preferred moment.
It's actually so good I plan to pay when they start charging.
> Our business model is to make the terminal so useful for individuals that their companies will want to pay for the team features.

why would businesses pay for something that was always free?

selling bottom to top is terrible strategy that rarely works (been there, done that)

you should instead market it to decision-makers who actually have a say what services the business purchases or not

developers have no say in business and have no access to credit cards

basically if you want to sell a terminal app, don't sell a terminal app, instead sell your "productivity platform" and terminal is just needed to "access productivity benefits"
I didn't like that either, but put it in context. It's 1 of 17,000 tweets. There aren't many users with that many tweets who haven't lost their cool and called someone else a name. It's one of the most common things you see on twitter. Not excusing it, just saying it's worth looking at in context and importantly it's not something he's repeated.
> Until now, I think Twitter's not-so-slight political lean has been viewed as detrimental to the company (and public discourse).

Can you point out what is, in your personal opinion, the best example you have of Twitter's "not-so-slight political lean" and how you interpret it as "detrimental to the company (and public discourse)"?
Dim, a self-hosted media manager
During the rebuild of Christchurch, NZ they used this technique for a number of buildings, basically pumping out water with one well, then re-injecting it with another as I understand.

Essentially exploits the ground water remaining a constant temperature year around, warmer than the surface in winter and colder in summer
Biff is a simple, easy, full-stack web framework for Clojure
NetBSD is so cool, and I have so many machines sitting around I need to get running on (SGI, Alpha, Dreamcast, etc.)

Sadly I’ve heard it can be rough on older architectures still. I’ve been told, that at least on VAX, for example is not in the best of states because usermode dependencies on Python. From what I was told, Python currently doesn’t have a VAX port due to the architectures floating point design.
It is supported very well on AMD64 NetBSD. Perhaps that will be expanded in the future.
Yeah I think some folks are seeing this and thinking the terminal is the product, when in reality the devops platform is the real product here and a slick terminal emulator is one component of that platform. Enterprises pay good money to companies like Redhat, Teleport, etc. for similar kinds of devops collaboration/security platforms.
> it is important to understand that overworking today may impact your mental health tomorrow

That is so true.

I find it helpful to go into situations like that with the cognizance that my main reason to do extra work now are to do less work later (spending a little extra time double checking work, for example, might make you stay an hour or two later for a while, but the return comes when you spend less time fixing bugs or responding to production issues).

Not nearly as helpful to be working extra simply because a business operates as a skeleton crew, for example.
They meant unknown at compile-time, not run-time.
Leeds burger joint owner calls bad reviewers 'sexual predators', reviews deleted
Google can be the next Google if they just stopped being evil for a second:

1. Let me ban domains like pinterest, quora, stackoverflow clones, stock image sites, etc without requiring a chrome extension.

2. Do what I ask it to do. Don't be too smart. Bring back the plus sign, minus sign, double quotes, tilde which have been deprecated over these years and stop polluting the results with what it thinks I want.

3. A new feature where I can search inside the top 100 search results. Where I can narrow down the search results using additional filters like I do on amazon searching for products. So i can say "5000mah -clickbank" in the top-100 search results to weed out spam and narrow my search accurately.
I always find the story of DEC interesting as well.
Design-first APIs meets DevOps
I don't know where to add this question without it seeming off subject, but seeing as we're heading into spring, seeing Germany has quite a wealthy economy and QOL, that many if not most workers get 5 weeks of holiday, the chip shortage is leading to slowdowns in the automotive industry etc... why doesn't the government just go cold turkey, give people a month or 2 free holidays while drastically reducing public lighting, telling businesses to stop heating their premises at night etc etc... I mean we had a dress rehearsal 2 years ago with the onset of Covid, this would be better ,for a good cause, probably give a boost to domestic spending and send a f...U to the rest of the world saying we too can think outside of the box :-)
Thanks for the kind words! We actually did (and still do) talk to our Users via Discord, github issues, as well as 1-1 User interviews.

In fact, I participated in a very first user research sessions (back then it was me and the ENTIRE WARP TEAM) almost a year ago, and after a session like this I decided to interview with the company and changed my job, and joined Warp :))
A lot of it is learning the engine api and how different parts interact. The basic competence level requires a pretty large body of knowledge. Can you, for example,

Extend a character class, give it a skeleton, an animation blueprint with some animation states that read the character state, hook it up to player inputs, make it do some sounds, and add a bit of flair to some actions it can take?

Can you build a basic ai behavior tree? Run some line traces to interact with colliders so your guy can interact with an object?

Shoot a gun and spawn some bullets that can damage another unit?

Etc.
you mean an open field for CIA, FSB and MOSAD to perform psyops?
Yes. But I’m not delusional that my contribution matters.

They should search for revenue like all community projects. Get companies that have a stake in FF existing because they distribute it — Canonical, Red Hat, AWS, Oracle to put up cash and pay developers to work on it.

The only FOSS models that have a track record of success are:

* Run ads which is really shady.

* Be a true community project and be developed by the people who use it.

* Be open core and sell proprietary features.

* Sell consulting and documentation.
A good critic points out flaws to fix. With fewer flaws the company is healthier.
Handsontable made at least $1.5M https://www.indiehackers.com/post/how-to-earn-1-5m-in-revenu... . Whether that's "a whole lot" is questionable.
Hi, Warp engineer here! Nothing against electron. We write this in the headline because people who see features like blocks or input editor automatically think "slow". So it's a succinct way to communicate that we have a snappy app.

For benchmarks, we have some preliminary info in this tech blog https://www.warp.dev/blog/how-warp-works.
If you think Reddit isn’t highly curated too, you’re wrong. Perhaps you could make that argument 10 years ago but no longer. In the last few years Reddit has taken on hundreds of millions of dollars of funding to morph the site into a more mainstream advertiser-friendly offering. They’ve silenced lots of controversial communities by outright banning them or neutered them with “quarantines”. Over the years, Reddit has succeeded at pushing out people who don’t have palatable opinions and retained a core base of advertiser-friendly true believers. “Advertiser-friendly” in the contemporary era means “left-leaning with no reservations“. This is about the furthest thing you can get from a “balanced view”.
Another one riding the "Rust-based" horse.
x86 from Pentium Pro and K6 on are basically hardware x86->RISC recompilers. (This is a good bit of why Transmeta failed in the end, the last good bits of Moore's Law ensured that the recompiling became cheaper/more efficient in hardware)
There shouldn't be any reference to a subscription. Can you let me know where you saw one? We used to have this setup as a paid extension, but pivoted away as that business model didn't work as well as we'd have liked.

Re: privacy and data. You can certainly review the settings we request and our terms of service - we ask to run the extension on a number of travel sites we interact with. We only collect your search intent data, progress around searching, and other similar lifecycle events as your search to aid us in product development. That being said, our long term monetization strategy is to sell search intent data - not your personal or financial data, and definitely not showing ads.
1) installed it
2) login required
3) uninstalled it

try again
Top US general: Potential for 'significant international conflict' is increasing
LOL, no. "CEO of warp". I like my terminal because it's a dumb text terminal, not because it's smart. At least you'll reach the majority normie users and make some dollar there.
France pays the steep cost of ageing nuclear as electricity prices soar
I'm not sure I understand. What is the relevance of this line?
So accusing someone of being a pedophile once is OK behaviour? Interesting.
Or Kitty, which does more or less the same things but includes split and tabs and other convenience features.
> But mostly, the device simply deleting the tokens when the user clicks logout is good enough though.

The issue with "simply deleting the tokens" is that this offers no way to force another device to log out. You can only log out the device you're using. If you were to lose your laptop, for example, there is no way to go into your account settings and force the laptop to delete its login tokens, hopefully before an adversary can gain access to them—you need some way of invalidating them on the server.

> If you require tokens to be invalidated instantly, there's no way around the notion of doing some kind of lookup when you are validating the tokens.

If you're okay with invalidating all tokens together then you can avoid storing lists of valid or invalid tokens by tying them to some part of the account state such as a token generation counter or timestamp. The tokens would only be valid so long as that state remains unchanged. Logging out would then consist only of updating that one field in the account.
As someone who has been using asdf daily for over a year now I can say from experience it's awesome and should absolutely replace all individual language managers as the standard way we manage language versions.

I recently gave nix a try because of a hacker news comment and agree with most of the points in this comment. The fact that new users are encouraged to read not 1, but 3 separate manuals to grok the tool is extremely discouraging for new users. Also the fact that nix is currently transitioning to a new feature called flakes and deprecating channels signals to me that the ecosystem is not stable enough for me to recommend that my team at work use it right now.

That being said, I intend to read at least 2 of the manuals and use nix for dependency management in my side projects and to manage my system dependencies. I think nix takes all the great things that asdf can do, to a whole new level. The dream of having a nix section in my projects 'Getting Started' that only has 2 steps (1. Install nix 2. Run `nix-shell`) is something I desperately want.

I think nix is much like erlang in that it's incredibly powerful and solves a lot of common problems, but it's used and maintained by an old guard that went through the trouble of learning how to use it and maintains the "I did it, why can't you" mindset. The erlang ecosystem has been given new life and been made easily available to new users thanks to the awesome work done by Jose Valim and the rest of the elixir team and community. However, I really hope that it doesn't take 30 years and a new language built on top of nix for it to become accessible to everyone.

For those working on nix and the new flake functionality please look to things like elixir and asdf for inspiration on documentation, ease of use and pragmatism. Help make the tool you love ubiquitous. It will ultimately make your life easier when working with new developers because they will already have nix installed.

I intended to do everything I can to help push nix in this direction (once I understand how the hell it works) if anyone reading this, has some time on their hands and wants to improve the state of software development for everyone please go down the nix rabbit hole and try to find ways to contribute to making it more accessible.
Can it compile Haskell projects? They are famously RAM hungry.
I thought it referred to the Netflix speed test.
> record they don't understand why Elon calls himself a "Tesla founder"

And I explained why it is really not that insane. It was basically him throwing them a bone, they should be grateful. And of course they were horrible and ran Tesla into the ground and hid facts from Elon.

> I think he was referring to PayPal

I don't think so.

> X.com and designed a very simple page where you put two peoples email addresses and you "could" send them money. It was happening at the same time as PayPal had the same idea and similar website

This is not true. X.com from the beginning was payment company. Confinity was originally a security company for Palm platform. From there they switched to payments.

From Wikipedia:

> In March 2000, Confinity merged with x.com, an online financial services company founded in March 1999 by Elon Musk.

The two companies merged so all the people of both companies are rightly called founders. And non of the others disagree with that.

> because personal leverage of Musk father, who owns multiple diamond mines in ZA

Please provide evdience. This 'dimond mine' nonsense has mostly been discredited. The best researched story about that basically showed that it was like a 30k investment sometime in the 80s. Certainty not enough to convince a bank to do anything.

Musk father was wealthy because he was an engineer.

> This story was somewhat easy to find and popular back in the old days of the internet

And well researched probably ...
If this "now or never" was true, the WH wouldn't have released the strategic oil reserves to lower the gas prices.
Thank you for sharing your perspective! I'm not Christian, but I'm a person of faith. It's interesting to see how the different faiths approach suffering.
If there's a 1970's technology in bad need of an update, it's the man page format IMHO. Markdown is far easier to quickly learn and write.

But you completely missed my point of computational notebooks + shell and I'll just respond in kind with a link that you should research yourself: https://en.wikipedia.org/wiki/Project_Jupyter
He refused to censor Russian news sources via Starlink?
I think the replacement of Google can't come sooner. Google invades our privacy and keeps users hostages for more money. Thanks to HN I am aware now how bad Google is. I always recommend people to switch to Amazon, Apple or MS. Google is EVIL.
This startup had just $600k in annual revenues [1], or $1,500 per employee (not a typo). Raised $120m and burned it all...wild

1- https://www.theinformation.com/articles/stripe-backed-fast-s...
Works great until the text gets changed.

This strategy works for applying semantic annotations of any kind to text. If you are marking up text to train a machine learning model that works as a "magic magic marker" (learns to mark up text the way you do) this is the ultimate way to do it because it is not dependent on any way to tokenize the text and lets you make markings that overlap.
clicks == value is a big structural failure in the current web
Why couldn't they compete with NVIDIA? Were the not just as big?
I recently migrated a fairly large monorepo (20+ packages) that used Lerna and npm to pnpm, and the improvement in developer experience was pretty massive.

Dependency install times went down by a huge amount, and all the strange issues we had with lerna and npm sometimes erroring out, requiring us to remove all node_modules folders and re-install everything, are just gone.

We even noticed that the size of some of our production bundles went down. Before some dependencies that were used in multiple packages were being duplicated in our webpack bundles needlessly, but the way pnpm symlinks dependencies instead of duplicating them fixed that as well.

The non-flat node_modules structure did break some things as well, since in some places we had imports pointing to packages that were transitive dependencies and not defined in package.json. I see this as a positive though, since all those cases were just bugs waiting to happen.
Thank you. It's not a grift (or is that what a grifter would say - oh god I hope it isn't). I didn't want us to overpromise and become the next No Man's Sky (which has by now fully redeemed itself) so I never spent much time on advertising and trailers. As a result, we didn't get much money. People were skeptical from the beginning, which was expected. What we got in support largely came from a handful of friends who wanted to see it done, and most of their names are going to end up in the credit roll. 99% of our present funding comes out of our own pockets and from Sylvia's illustration business.
Yes, a few 35mm films had 4 track mag audio. The only one I can think of off the top of my head was Fantasia.
Elon Musk will join Twitter's board. Will users finally get an edit button?
Not hard to believe - younger people make up the bulk of their audience, and probably the bulk of HN that trend more liberal. That's a reflection of the userbase though, not necessarily of Twitter or HN's views. I would bet most LPs are more conservative - if only in private.
I haven't tried using DataGridXL with a file with that many records. There is a chance that the DOM node that is used for scroll-listening grows too large and browsers cannot handle that. I suspect that is where the error comes from.

However, I will add this to our list of issues, see where the error really comes from and where I can improve it (https://github.com/DataGridXL/DataGridXL2/issues/31). For now, I would say 10,000 records max to be sure.
As mentioned, official support was dropped and the community supported extension hasn't been updated in ~2 years. If you need to target WebGL, then Unity is a better bet.
DDG is basically just a wrapper around Bing. I'd rather just use Bing directly
My bad, I was thinking of your "and [Elon buying 10% of Twitter is] awful for society" comment - unless I misunderstood what you that was awful for society.
> And are debating who would make a better fit in startups as early employees.

That seems like the wrong question.

Shouldn't you be worrying about who would be the best fit in your company? Or better yet, who will help you succeed NOW? (And then there's "who can you actually get?" The people you want have options.) In other words, relevant specifics, not generics.

I do a lot of work with early startups and if I'd invested in your company, we'd be meeting right now to figure out whether you're clueless (and what to do about it) or just wrote something that you didn't actually mean.
The third thing he said:

>> You want devs from other startups.

> Seems a bit dumb

Are you sure you're in a good position to lecture others about content of their responses?
I was interested until I learned that the application is blazingly fast :/
Omen for bolt?
Are you creating an OS and/or softball as well there?
It Is Not the Types That Will Help You
What a fantastic article, was captivated from beginning to end.