skysight-inc/HackerNews huggingface dataset:
RAG is very difficult to do right. I am experimenting with various RAG projects from [1]. The main problems are:

- Chunking can interfer with context boundaries

- Content vectors can differ vastly from question vectors, for this you have to use hypothetical embeddings (they generate artificial questions and store them)

- Instead of saving just one embedding per text-chuck you should store various (text chunk, hypothetical embedding questions, meta data)

- RAG will miserably fail with requests like "summarize the whole document"

- to my knowledge, openAI embeddings aren't performing well, use a embedding that is optimized for question answering or information retrieval and supports multi language. SOTA textual embedding models can be found on the MTEB Leaderboard [2]. Also look into instructorEmbeddings

- the LLM used for the Q&A using your context should be fine-tuned for this task. There are several open (source?) LLMs based on openllama and others, that are fine tuned for information retrieval. They hallucinate less and are sticking to the context given.

1 https://github.com/underlines/awesome-marketing-datascience/...

2 https://github.com/embeddings-benchmark/mteb
Get a 5-stage RO system without a tank. Tanked systems are only a good idea if you use enough water that the tank doesn't sit stagnant and you include UV, otherwise tanks can be a place that causes bacterial growth. The reason you want a system with more stages is that you can have multiple pre-filters before the RO membrane which allows the membrane to last longer and allows cheaper filters to be the most often replaced.
Do you agree or disagree that techproducts today are easier to build than 15 years ago?

Everything else is irrelevant. Technology drives abstractions which drive deflation by allowing fewer people to produce more goods. This has been proven and observed for thousands of years

Companies can thrive while overemploying people, but eventually they get undercut by one that doesn't
Yes, I explained in the `Benchmarking setup` section that I used `march=native`, but I guess I forgot to mention I used -O3.
I personally believe the government can certainly pass information on to private corporations and then the private corporations can then choose what to do.

Whether the situation went past that is what this court case would be about, and nothing has been decided on that topic yet.
So say police are looking for someone who is suspected to be in a white Ford Bronco. They see a white Ford Bronco and want to use their internet connected mobile terminal to see if that is the white Ford Bronco they are looking for.

Without license plates how are they to tell what that terminal is connect to which white Ford Bronco they are inquiring about? Do they have to actually stop every white Ford Bronco they see so they can read the VIN or get the driver's ID?
Hey, thanks for reporting this, it should be fixed now!

I created this HN account to hopefully make it easier to find me in the future but I'll add some contact information to the site too.

In case anyone comes across this comment in the future, emails to <anything>@alerthn.com should find me.
Since chatGPT-4 is now useless for advanced coding because of their blackbox sudden nerfing, can anyone guess how long before i can run something similar to the orig version privately?

Is the newer 64B models up there? 1 year, 2 years? Can't wait until i get back the crazy quality of the orig model.

We need something open source fast. Thanks open-ai for giving us a glimpse of the crazy possibilities, too crazy for the public i guess.
As far as I can tell: No, and the roadmap doesn't include anything for adding it. I think it would actually be pretty much perfect for me if it did have OCR.
It was a disaster at Wisconsin. Union favored the 8th year humanities students at the expense of the 1 and 2 year STEM students, and traded cash for benefits that really only affected people with families. Stipends were the lowest in the country among major research universities, and the union refused to let wages rise for in-demand TAs, so a bunch of them went off and got private sector jobs instead, limiting the size of the undergrad population. Scott Walker killed it, and those problems haven't come back.
Last time I looked the 7 day active committers was about 750. So with holidays, people who don't commit for a week, PMs and other management etc, my guess is around ~1000 people, maybe a bit more. I doubt it's several thousand. Maybe if you include ChromeOS?
Is this transphobia masquerading as feminism?

Oh, "a man who claimed to be a woman". Yes, it is. Why is nonsensical bigotry posted here?
Ignore that dude. This "you're not paying for it" bullshit needs to end. Yes, we are paying for it. We are paying with our attention and our sharecropping labor. We give them a mind to infect and they sell that to advertisers. We give them our content and they use that to attract other minds to sell to advertisers. Any suggestion that those aren't a valuable payments shows a complete lack of understanding of the last 20 years of online "progress" and also just makes them look foolish.
And would all of these posts be useful information?
https://ernest.oppet.it/

I blog about books, podcasts, courses I've done, product management thoughts du jour, and the occasional rant
Not scientific, but literally just stand next to a road. A normal car driving at speed makes a “whoosh” noise, not a “vroom” noise.
And yet I can confirm that 4 is far superior to 3.5 in the medical domain as well!
Your notifications aren’t automatically silenced when presenting your screen?
I have no Meta corp apps installed on my phone. Not Facebook, Instagram nor Whatsapp, and Threads will join that list.

The reason for this is because Zuck and Meta corp are hostile to user privacy (and drain batteries).

Threads can't even launch in the European Union because of it.

Have fun paying $12 a month to be Threads-verified.

Thankfully day-to-day life isn't yet dependent on Meta corp as a necessity.
You're welcome. If you're going to be picky you'll probably find nothing meets all of your requirements.
Whoa, WHOA, I could never have named or described this but it suddenly felt like I was 8 hoping for specific christmas gifts again.
> You don't need 100s of operations people if you move from on-prem to serverless, for example

This statement really leaks a lack of experience with large systems.

The efficiency loss for going to some serverless regime and the costs associated would be astronomical. My god…
The problem is the author of the article is making some huge implicit assumptions that the compiler can't possibly know about.

Consider this statement: "However, we know some things about this loop. We know that the only time we break out of it is when we hit the null terminator (’\0’). The code clang generates checks for the null terminator first, but this makes no sense."

This statement contains huge assumptions about the lengths of the input strings and the frequency of the letters 's' and 'p' in the input. And then has the chutzpah to call the compiler's failure to read his mind about this as "making no sense."

Good first effort by the author, but has not sufficiently thought through the problem.
The engineer is not part of the board which makes these decisions.
I don't strive to be productive at work, love or family.. or play for that matter.
Luckily Popper explained to us we have something in science called falsifiability. I suggest you try to falsify one single theory of our current climate science. Or, alternatively, show that any of the theories is unfalsifiable. Just going all Popper on you, because it’s needed.

By the way, much of our previous research on climate science has already been falsified, for them to be substituted for theories which show even stronger evidence for man-made climate change.
Less of the slurs, please. I thought this was supposed to be a classy place.
really cool idea! been looking for something like this for a long time. its too bad it freezes my tab and is unusable
Gamble on La Liga, and when you are bankrupt, go apply for welfare, because you are now poor.
For those asking who would pay for this. I’d pay for an api that did this. We have a platform that gets a lot of audio files and it would be cool to run them through something like this to generate the video viz for it.

I think the term is audiogram but minus the transcript.

While writing this reply I found this https://getaudiogram.com/ So maybe it’s something to check out. (I’m not affiliated)
> The reason I don’t think Ladybird will ever really get to the point where it can render an arbitrary site perfectly, is based on my experience as a web developer. I haven’t been doing web development for very long, but there have been a couple times in the last couple of years where I’ve stumbled upon browser layout bugs where Chrome/Firefox/Safari render something different.

So what does perfect even mean in this context? Who's right, Chrome, Firefox, or Safari?

I 100% believe Ladybird will eventually be able to render arbitrary sites in an acceptable format at some point (potentially within the next year or two). It's pretty remarkable how many sites look very close to acceptable already. Any weird edge cases where Ladybird renders stuff differently from a big browser won't mean it's rendering the site "imperfectly"; just like how when Firefox and Chrome do different things, it doesn't mean either one is "correct".
Refactoring Consumer Electronics
Did you know that Hackernews has a poll feature?
https://news.ycombinator.com/newpoll
deactivate? does that claw back all my Threads content and metadata from Meta? i'm gonna go out on a limb and guess "no".
the code can't get out of sync with itself. i agree, that's a dumb policy. boilerplate comments are just as bad as boilerplate code-- the policy should be to comment when something is notable, in the manner that's appropriate for the context.

but comments telling you one thing is happening when something else is happening is a problem, and i suppose it couldn't happen if the comments weren't there.
How many of them are real people?
Exactly, burning fossil fuels causes air pollution hence health problems.

This is a problem to be addressed/solved regardless of the causes of climate change.
The difference between C++ and Python is not nanoseconds.

And I have definitely seen projects fail due - in part - to language choice. Of course projects can succeed in almost any language but that doesn't mean the language choice is irrelevant.
I'm afraid to ask?
Eh, they had great games on the genesis, not much after that. When Dreamcast released it was mostly hit-or-miss arcade ports.
> Ones that are trained on LLM generated blogspam will probably, I expect, get outperformed by ones that are trained on genuine information, or at the very least ones made using new techniques that adequately filter noise.

Yes, humans are notorious for only seeking out high quality, accurate data, especially when it conflicts with our priors.

To say nothing of our ability to assess the accuracy or truthiness of information in the first place (look at how many people take, on faith, that Chat GPT isn’t wrong as often as it is right).
Calling someone a "sex worker" doesn't bring class into the discussion. It's just sugar coating the situation, which is my point, as uncomfortable as that may be for some.
>Equating the two is like equating a sea sponge and a dish sponge. "Well, they're both sponges!"

Actually, they can[0] be. In fact, I use sea sponges as dish sponges every single day.

[0] https://www.naturalbathbody.com/natural-sea-sponges/
> But when people say "use AES-CBC + HMAC" and cite Signal as an example, and Signal's implementation does this: https://github.com/signalapp/Signal-Android/blob/main/app/sr...

That's very vague and therefore not very helpful. Could you say what exactly is wrong with the code you linked?
Why sqlite? You can write an array to a file directly. They read whole dataset each time they want to find the closest match.
In the US, it seems at the policy level this would be impossible to implement. From a practicality level, it would take very long to validate to the point of stifling innovation (good or bad) and would require international cooperation. Compared to medical research, medical interventions are risk based at the individual level and we are willing to accept marginal improvement with awful side effects. Chemicals apply risk at a societal level and settng by appropriate limits seems difficult vs mandating better waste containment.
Aren’t we currently experiencing inflation at least partially due to government stimulus handouts during Covid? I know there are multiple causes but free money certainly didn’t help.

Also, I think it is a good idea to keep government programs means tested and separate. If you give people 20k, the number of people who should be feeding their kids with the money but instead bought “new rims” for their 2007 Honda Civic is going to be much greater than zero.
The bus and the 20 AVs have the same throughput (40 passengers in 10 minutes). While the AVs take up more physical space than the bus, they require the same amount of road space to transport the same number of passengers in the same amount of time.

You have to have extremely short bus headways before the road space efficiency tilts back towards its favor.
None of the visuals are loading for me. SSL errors for both yandex and your CDN on my network.
wonder if it's possible to offer an attachable heatsink in a shame of extended battery or case. That way, there's no fan / dust to worry about.
Wasn't "the algorithm" already open sourced?
These games are deliberately designed this way

Which design elements can one add/remove to such game to make a difference? Asking because it feels like a stretch to make an argument and also just how all things naturally are (opposed to designed deliberately).
Even better, it's 6 people. You plus 5 family members.
Meta: I'd not realised that Sci-Hub was a permitted site on HN. It appears that it has been since at least 2019 looking at the site submission history link.

I'm not in the least displeased to learn this.
Instagram Threads was a (different) thing back in 2019: https://www.theverge.com/2019/8/26/20833903/facebook-instagr.... Threads.com looks like it came out in 2022?
You're damned if you do here, and damned if you don't.
Markers to Deter Human Intrusion into Nuclear Waste Isolation Plants
Ah, so you're just trolling. I thought we weren't supposed to do that here either?
Yeah, the Washington sites in the map are only along the I-5/Cascades corridor.
He’s not acting toxic. You just proved to have a thin skin.
I drive a Volt and I’m convinced this configuration of car is what many Americans should be buying until battery manufacturing ramps up and EV fast charging infrastructure is widely deployed. It gets 45-60mi range but it charges just fine overnight from a 120v plug. Most of the time it’s an EV, but we need the extra range several times a month for longer local trips or to visit family in a nearby city. The gas engine requires very little maintenance because it runs infrequently and at optimal RPM and load, it weighs less than a 200mi+ range battery, and (for now) it’s easier to manufacture from more common materials.
What are your concerns?
No, money is reallocated upwards.
Google Plus (invite only) failed in a similar way.
SEEKING WORK | GERMANY | REMOTE | React Developer

I specialize in web application and landing page
development. I make beautiful and scalable frontends.

My main skills are:
- React, NextJS, TailwindCSS
- NodeJS, ExpressJS
- Typescript, Javascript
- Redux, RTK Query

Examples:
- Loggl.net (took a week to develop)
- Snapify.it (took two weeks to develop)

To checkout my portfolio for the source-code and other projects: https://marcushof.vercel.app/

Links
Résumé/CV: https://drive.google.com/file/d/1CbZi1Bm-MlDHEb4WjsFBzIBSomJa1M7T/view?usp=share_link
LinkedIn: https://www.linkedin.com/in/marcus-hof/
Email: marcus (dot) hof (at) protonmail (dot)com
GitHub: https://github.com/MarconLP
Twitter: https://twitter.com/Marcon565

Hourly rate: $25/hour

You can email me at: marcus (dot) hof (at) protonmail (dot) com
> are computers just not yet fast enough to do a good job with very simple loops in practical compilers?

The short answer to this question is 'yes', but there are some extenuating factors:

- Although we could do interesting things with unlimited computational resources, the current crop of c compilers is simply not very good, compared with what's possible today.

- Performance is always workload-dependent; the compiler has been somewhat shafted here because it doesn't know what sorts of inputs the function usually receives. The compiler output is better than the 'improved' code for some inputs. (It's possible you could get a better result from the existing compilers and c code just by using profile-guided optimisation.)

- The difference is prone to be more pronounced in simple loops than large ones. This is a contrived use-case. There is not a factor of 6 of performance hiding in optimised c code which could be recovered by doing the sorts of optimisations done by the op. Probably something more like 10-20%.
I don’t trust it because enough third parties were able to verify the findings.

This is the double edge sword of being so ridiculously closed.
Promote and proliferate local LLMs.

If you use GPT, you're giving OpenAI money to lobby the government so they'll have no competitors, ultimately screwing yourself, your wallet, and the rest of us too.

OpenAI has no moat, unless you give them money to write legislation.

I can currently run some scary smart and fast LLMs on a 5 year old laptop with no GPU. The future is, at least, interesting.
Would this pass their System Design interview?
Thanks. For context, I happen to have a PhD in static analysis, so I'm not entirely stranger to the general idea :)

Sadly, all the attempts I've had with static analysis in Python screamed that while the language and tools make a valiant effort at supporting a reasonable set of annotations, 8-9 years after PEP 484, the libraries are simply not yet ready for it (not even, in many cases, the standard library).

Unit tests, staging environment, effective log aggregation are all important tools, of course.
So then continue to use Twitter? Not every new social media tool needs to be a fit for every industry. Additionally, sex work occupies a grey zone in which it's not always appropriate for the platform to... err, platform sexual content.

If people wish to contact sex workers then they'll use Twitter because it's the place to go for sex workers. Presumably this was already an issue when Instagram (Meta) stopped permitting sexual content. Why would they expect Meta's new app be any different?
And even lawyers get away with far far more than they should without losing their license to practice. It seems like accountability is at an all time low everywhere and we're all paying the price.
Wow what a jerk. I'm going off the Constitutional Oath, which all judges take. Obviously they use judicial discretion but be charitable for a second and at least try to understand my point, which is that deviating from what the Constitution says is the very rare exception, not the rule.
No matter what the true combination of reasons, this is a massive endorsement of the python ecosystem.

Its an interesting dynamic: popularity draws attention, attention helps adress pain points, this makes the platform more attractive and the popularity increases...
I don’t know about how interacts with tea, but gym bro forums have talked about itching and anal itching in particular. Maybe the tea makes that worse? No idea.
I think its election season in Spain.
It's a gross industry designed to milk big dollars out of clueless customers. Listening to these 'security experts' talk makes me roll my eyes roll so hard that I'm afraid they'll get stuck in the back of my head.
Payment won’t dissuade. What will is getting a conviction of murder with life in jail for every single employee aware of the impacts and complicit. Yes that’s 1000s of lives ruined. But that single act will tell everyone else to fess up or risk similar punishment. Including the lower peons who’d be happy to cut a deal for amnesty.
> Union favored the 8th year humanities students at the expense of the 1 and 2 year STEM students

Having been a one-time 1st or 2nd year STEM student, I think this is the proper decision.
AFAICT, Canonical's customers are cloud vendors and people who buy cloud services. The Linux desktop is, in that sense, just a hearts and minds investment for people who run linux workstations to manage their much larger linux server fleet. From that perspective, the investments in charm and snap on the desktop feel a bit like a sales pitch for their new cloud features. Which.... would be more effective if they didn't also make the user experience for FF emphatically worse.

And originally, Ubuntu was a high visibility demo piece for Launchpad. If it can manage the full build cycle for Debian, it can certainly handle your niche app.
Following the same analogy, it's like removing spreadsheets from Google Sheets.
That should be a wake up call to every corporation pinning their business on OAI models. My experience thus far is no one is seeing a need to plan an exit from OAI, and the perception is “AI is magic and we aren’t magicians.” There needs to be a concerted effort to finance and tune high quality freely available models and tool chains asap.

That said I think efficiencies will dramatically improve over the next few years and over investing now probably captures very little value beyond building internal competency - which doesn’t grow with anything but time and practice. The longer you depend on OAI, the longer you will depend on OAI past your point of profound regret.
This is problem of misaligned incentives: if you are making a security scanner the last thing you want to do is miss a vulnerability. The result is many false positives.
It does exactly what I've been describing!

They provide AE, not AEAD.

They feed an IV and ciphertext into HMAC. They don't feed additional authenticated data.

If someone followed Signal's example, they either wouldn't have AEAD, or they're likely to make the exact mistake described in the post I linked above.

I don't know how to be more helpful here. I've been only repeating myself.

AEAD modes let you bind a ciphertext to a context without increasing bandwidth. This is super important for database cryptography. Read more: https://soatok.blog/2023/03/01/database-cryptography-fur-the...

Whether "it's not AEAD" matters for an application depends on many factors. Signal doesn't need it.
Agreed.

Apple lost the lawsuit against Microsoft many decades ago when they tried to stop MS from using GUI elements that were similar to Apple's.

You can't copy someone's code, but nobody can stop you from having scrollbars or avatar thumbnails or whatever in your own app.
There was a guy in the news who asked an AI to tell him it was a good idea to commit suicide, then he killed himself.

Even on this forum I've seen AI enthusiasts claiming AI will be the best psychologist, best school teacher, etc.
I've been using google calendar for the same thing, it make it easy to start dragging to log what task I've been doing, the fact that you can show multiple calendar and easily toggle between them helps with this theoretical plan vs what actually happened. there's also a script/webapp on github that help you sum the total time spent per timeunit per task or calendar.

does anyone know of an open source calendar app with the same ui/ux ?
For context, I was asking about scaling code complexity, not scaling performance.

That being said, my personal experience suggests that both are really hard in Python, and not independent.
There are no health issues that anyone should ever be using Chat GPT for - the hallucinations are very real, and often severe. Not that you should be googling your symptoms either - there’s a reason we put medical folks through more than a few minutes of training.

LLMs are absolutely worthless for medical information.