skysight-inc/HackerNews huggingface dataset:
Show HN: Ragas – Open-source library for evaluating RAG pipelines
Ragas is an open-source library for evaluating and testing RAG and other LLM applications. Github: https://docs.ragas.io/en/stable/, docs: https://docs.ragas.io/.

Ragas provides you with different sets of metrics and methods like synthetic test data generation to help you evaluate your RAG applications. Ragas started off by scratching our own itch for evaluating our RAG chatbots last year.

Problems Ragas can solve

- How do you choose the best components for your RAG, such as the retriever, reranker, and LLM?

- How do you formulate a test dataset without spending tons of money and time?

We believe there needs to be an open-source standard for evaluating and testing LLM applications, and our vision is to build it for the community. We are tackling this challenge by evolving the ideas from the traditional ML lifecycle for LLM applications.

ML Testing Evolved for LLM Applications

We built Ragas on the principles of metrics-driven development and aim to develop and innovate techniques inspired by state-of-the-art research to solve the problems in evaluating and testing LLM applications.

We don't believe that the problem of evaluating and testing applications can be solved by building a fancy tracing tool; rather, we want to solve the problem from a layer under the stack. For this, we are introducing methods like automated synthetic test data curation, metrics, and feedback utilisation, which are inspired by lessons learned from deploying stochastic models in our careers as ML engineers.

While currently focused on RAG pipelines, our goal is to extend Ragas for testing a wide array of compound systems, including those based on RAGs, agentic workflows, and various transformations.

Try out Ragas here https://colab.research.google.com/github/shahules786/openai-... in Google Colab. Read our docs - https://docs.ragas.io/ to know more

We would love to hear feedback from the HN community :)
> Apple hasn’t changed its approach their wall garden in ages.

The same action might be legit with 10% marketshare but lesive of market competition when at 60% marketshare.

Take golden-era Microsoft: bundling a default browser was anti-competitive for them, whereas it clearly wasn't for any Linux distribution out there.
With so much recent news involving air travel; the public will certainly be reassured by the quick, decisive actions of the DOT.

> As DOT finds evidence of problematic practices, the Department will take action, which could mean investigations, enforcement actions, guidance, or rulemaking.

"as they find evidence" ... so they assume this investigation will find such evidence? which may then lead to further investigation? If the results of this investigation are forgone conclusion why isn't this the follow on investigation?

I might even suggest that the problem of securing customer data extends beyond the air travel industry; and might perhaps be better addressed as part of an effort also spanning across "folks that has customers." DOT might be better advised to focus on their own bailiwick and figure out what to do about airlines, railroads, and other industries where maintenance budgets have apparently been cut past the point where public failures become inevitable.
Whatsapp themselves could easily solve this if they wanted... Just add a "backup to file" button in settings. Then add a "restore from file" option in Android.

Quite why they don't do this is a mystery to me - if a user loses all their chats in a phone migration, they're more likely to start using another messaging app.
What’s the relevance of that? If that were the case the law should be to make everything available by the web which is inherently interoperable, which I think we both agree with, but still doesn’t have anything to do with Apple.
Seems like the main thing is rectifying what "research and development" entails and how it's taxed. Software development was grouped together with other kinds of development, which I believe would be traditional prototyping, modeling, and building test units (in a physical industry).
Man boarded Delta flight using photo taken of another passenger's boarding pass
The man man was arrested in Salt Lake City after he snuck on board using a photo taken of another passenger's phone screen while the passenger wasn't looking. He then hid in the lavatory until boarding was complete, but was unable to find a vacant seat afterward. The plane returned to the gate without ever taking off (but it did taxi).

He now faces a felony charge "stowaways on vessels or aircraft."

Sounds like this was just a 26-year-old ski bum trying to get back from Park City for free though - not anyone with deeper ill intent.
Or, both Apple and Google did see this coming years ago, have been smart about supporting politicians in both major US political parties, and calculated that the amount of money they could make by maintaining their monopoly positions — even if only for a few more years — was likely far greater than any fine or other regulatory headache it might cause down the line.
Here's a link to the full text of _Slow Tuesday Night_: https://web.archive.org/web/20060719184509/www.scifi.com/sci...
Network effects mostly. If your relatives or friends are on facebook then you will either have to put up with it or miss out on social contacts. This does apply to other kinds of forums too to some extend but limited to a more abstract numbers game where the biggest forum is the place to be rather than more personal direct connections that you don't want to cut.
> If the interpretation were wrong, and that's not a force, then the amount of force in that equation would be 0.

With no force, Gm₁m₂/r² = 0.

Nonsense. The GR interpretation is that G m1 m2 / r^2, when it is nonzero, is describing an effect of spacetime geometry, not a force. It can't be a force in GR because it isn't felt; an object moving solely under the influence of G m1 m2 / r^2 feels no weight--an accelerometer attached to it reads zero. GR does not change the numerical value of G m1 m2 / r^2 at all. It just reinterprets what the quantity represents.
I feel like there's a difference between the car regulation you state and the regulation approach being taken in the EU. Specifically the ability of third parties to limit end user choice.

With vehicle repair, I can still choose to use the manufacturer operated/approved repair shops. I truly am gaining additional choice and can continue to service my car as I always have.

The EU regulations allow third parties to remove my choice to live in the walled garden if they wish. So while it could enhance competition for developers I don't know if it greatly improves the users choice, or experience.
I was the wrong demographic for this scam. I am successful in the stock market so I would never have fallen for questionable investment schemes, especially one involving trading on insider knowledge.
Yeah it'll be interesting to see (via discovery) whether Apple has policies like Google's regarding "words not to use".
Members are taught to save "for a rainy day" (and to avoid debt, obtain education, and to be good to everyone, starting at home). The Church as an institution also follows their own advice. They have so many money-consuming projects going on that if there were serious economic problems, these resources would help it continue those efforts instead of having to dramatically cut back (feeding, employing, accredited global online higher-ed and English instruction and support to those who could not otherwise afford it, building programs, extensive welfare and humanitarian programs, etc etc). More info at the Church's site or my own (in profile).
They're not guesses. We know they use A100s and we know how fast an A100 goes. You can cut a brain open and see how many neurons it has and how often they fire. Kurzweil's 10 petaflops for the brain (100e9 neurons * 1000 connections * 200 calculations) is a bit high for me honestly. I don't think connections count as flops. If a neuron only fires 5-50 times a second then that'd put the human brain at .5 to 5 teraflops it seems to me. That would explain why GPT is so much smarter and faster than people. The other estimates like 1e28 are measuring different things.
Haha. I know you try to persuade me that consciousness as life force intelligence does not exist. But as far as I am concerned, I am and I am sentient and that is the only thing in my life I do not need any proof.
Reminder that in 2014, Reddit planned on giving away 10% of its stock to dedicated power users and moderators[1], before Ellen Pao (briefly) and then Steve rejoined as CEO.

1. https://techcrunch.com/2014/09/30/reddit-fundraising/
I'm actually advocating against re-inventing the wheel (insider another wheel, no less)

you can still follow css spec if you want. just get a better environment than a canvas.
If you want Fortnite then you need the Tencent...sorry the Epic Game Store. That comes with all of the PII leaks[0]. Because their game store will require permissions/privileges to install system wide apps, it won't be constrained on what data it can leak about users or what it can decide to install in the background. I for one can't wait for a dozen app stores to pop up all installing Sony root kits or Denuvo malware on people's phones.

[0] https://www.ftc.gov/news-events/news/press-releases/2022/12/...
I like to string scammers along for fun, if I have spare time.

These "pig butchering" scams provide hilarious possibilities.

They like to send their photo pretty early (always an attractive Asian lady), so I try and beat them to the punch and send my own photo (of course, also a photo of an Asian lady). Curiously, this doesn't deter them! My wife finds this particularly hilarious.

One time, I had 2 people randomly texting me at the same time, using the same name - Jenny. I started sending them screenshots of each other's conversations, which kind of annoyed them. "Which Jenny is this? Do you know the other Jenny?" lol

A list of things which also doesn't seem to deter them or make them think they might be being toyed with:

- Claiming to work for the FBI (in an anti-scammer unit)

- Being homeless

- Claiming to be Peter Frampton

- Claiming to be Bill Clinton
House Passes Privacy Bill Banning Data Brokers Selling Info to Russia, China
The article is chock full of examples where Apple prevents competition on their platform or in connection with their platform.

Apple's argument is generally that they are making the platform safer for their users, but I was just on the App Store looking for the Google Authenticator, and the first item listed was a scam third party authenticator which was intended to fool users looking like the Google Authenticator. This would be the easiest possible thing for a giant corporation like Apple to catch. The fact that it is Google's customers which are being scammed could be part of the reason why Apple doesn't prioritize safety in this case.

What we're dealing with here is a really duplicitous company. Their marketing is world class. The battery life of their products is world class. Everything else - not so much.
Hope you’ll excuse the somewhat off-topic nature of my comment: I’d be curious in hearing your story how you transitioned into software development from construction - would you mind passing along contact information or reaching out to me? I’m in a similar boat and thinking long and hard about my options!
People are already raging that Reddit can't pay moderators, but that Spez takes more compensation than Tim Cook while his company reports losses. Reddit said it themselves, one similar website could take them out and that's exactly what I see happening once the landscape changes from community to profit which to your point is already happening given recent events.
I'm curious here...I've only ever gotten deep enough into pcb design to modify an open source motor controller. What does this give you that I couldn't get out of KiCad? From what I can tell, there are autorouters and autolayout and design rule checks. Is it just better execution?
I like to think I'm part of that group, having learned program on microcomputers in the 80s.

Doesn't do me much good now, though. I've been turned down for programming jobs because they felt I "wasn't passionate enough about programming". Never mind that I've been doing it for 20 years professionally and that I've written books about it and that I started a company to do it.

I always wanted to be a teacher anyway, and now is a good time for that semi-retirement. And the job even has a "pension" (I had to look it up, too ;) ). So it all worked out.

But all you programmers in your 20s having the time of your life right now, be forewarned: you're going to have to make some changes in not too long... I honestly hope I'm wrong and people who love programming will be able to continue to do it until they retire. That's just rare these days, is all.
It looks like the DOJ doesn't believe that the closed marketplace doesn't add value to consumers or businesses but only to Apple themselves.

I think the crux of the DOJs argument is that apple is using their dominate marketshare to rent seek and create artificial restrictions preventing competition with their own products.
> But Chinese criminals need all the cellphones, so I wonder why

Chinese platforms within China require authentication and ID verification. In addition, all phones need to be linked with a user's ID. Furthermore, a number of platforms in China are locked behind apps and don't offer a PWA. This is a side effect of mobile phones being the primary computing device in China.

That said, this level of KYC is fairly normal outside of North America and Western Europe.
Ask HN: SQLite offline first sync options?
I'm confused, and my AI chat buddies seem to be too. I have an iOS React Native app that stores data in a sqlite db. What are the best options these days for getting that data synced to the cloud and back?

My goals:

- offer this local data as something users can maybe alter and use from a web app or another device. There won't be a lot of frequent real time merging and syncing going on. I imagine it's going to be an infrequent sync operation.
- I'd rather not rewrite everything to use a different datastore.
- I see there are sqlite merging and hosting options (fly.io and litesync.io) but I'm not quickly groking if they are what I want here.

Really curious what you all are doing with projects similar to this.
It seems weird to force a company to support their competitors products if there is no financial interest in them doing so.
Google's crime was not having a charismatic leader who could store all the mens rea solely in his own head and then conveniently die before legal scrutiny started over their App Store racket.

All of Google's monopolistic intent was conveniently detailed out in loads of e-mails. They were caught failing to retain these e-mails, which in a civil suit where the 5th Amendment does not apply, means the judge gets to just assume the worst (make an "adverse inference").

To make matters worse, Google promised openness and then tried to privately walk it back. Legally, this is admitting that the "Android app distribution market" already exists and is the appropriate market definition for a monopoly claim. It's harder to argue that an "iOS app distribution market" should exist when Apple is using power words like "intellectual property" - aka "we have a right to supracompetitive profits."

My personal opinion is that the DOJ probably will succeed where Epic failed, however, because of one other critical thing: standing. Epic did reveal market harms that are almost certainly cognizable under US law, but none of those harms were things Epic was allowed to sue over.
I always felt that the decrease in intelligence is a side effect of the necessary consensus mechanisms.

10 genius clones would still take on various roles/positions in the system, requiring some optimization with respect to alignment under time/energy constraints.
Different incentives. Apollo was made by someone who loved making software and wanting to build something nice for people. Reddit is made by a corporation interested only in extracting as much wealth from the platform as possible, with as little regard for users as they can get away with.
Most Macs sold are Macbooks that also have batteries
Why aren’t android phones equivalent? I had an iPhone and pixel and switched from both multiple times with no issues.
Right, but that comment is not a study, it's just someone saying there's a study..... They might be wrong, mistaken, or whatever... Sorry if it seems churlish, but if you are after studies, that comment is not what you want.
All well and good, but consider: if you work for Google in Pittsburgh you make approximately 60% of what you make doing the same job in Mountain View.

We are in a transitionary period where many people are earning Mountain view money while living in Pittsburgh (either literally or metaphorically). In the long run, that won't be the case. It may well be a boon to companies to draw from talent all over the country, but they are certainly not going to pay that nationally or globally distributed talent pool the highest local rate. Why on earth would they?
Git doesn't use the output of `diff` to determine whether anything has changed.
> Your bias against state-owned enterprises is clear

This is an incomplete statement. The biases of any media company are clear. The bias of a state-funded media company are in favor of those in power that control the funding. The biases of a free market media company are those that finance the company.

The difference is that with the free market, you have multiple biases to select from, and it is not backed by the power of the government.

> till in this case it's unfounded and just your ideology.

I know how human nature works. Consider the Don Lemon vs Elon Musk interview. Musk was going to finance Lemon big time. To kick it off, he was going to let Lemon interview him. Lemon proceeded to do his best to insult Musk and try to get Musk to say something embarrassing. Afterwards, Musk decided he was not going to finance Lemon.

Whoever pays the piper calls the tune.

It's a fantasy that having government funded media means independent journalism.
That's correct. When people speak of the early days of Reddit it seems like they are talking about whatever time long ago they initially joined.

I'm always hesitant to make accounts on new sites, so I thought I had waited a long time to join officially, but I ended up being in the first 500 accounts! From the earliest days (the del.icio.us to Reddit influx) it was a very libertarian place. Sometimes annoyingly so.

Reddit has shifted so many times over the nearly 2 decades, but most who stay for a long time have realized that it's great for pocket communities in smaller subreddits, as long as they last and aren't co-opted. The bots are the worst part at the moment.
Your average iphone user has no idea what this is.

All the know is "android makes the bubbles green and iMessage doesn't work as well with them, or at all".
Security Engineering is mostly about control and minimizing attack surfaces. Apple iOS implements this exceedingly well, with defaults, while still being one of the most widely used platforms on the planet. I believe IOS gets it right the vast majority of the time with solid architectural changes and not just endless patches and knobs that are hidden and forgot about. This is the key difference of "It just works" verses other platforms.

If someone wants to run another platform, go for it. Of course are shortcomings in iOS (as with any system), but viewing entire problem space of security and privacy, the default install of IOS + Safari could rarely be any better for the average consumer. This is why Security and Privacy is literally a paid feature of the IOS platform, and anecdotally everyone professional I know (who isn't in tech) is using IOS devices.

Personally, I'm planning to blocking RCS and any third party app stores on any of my own (and families) devices -- again, control and minimizing attack surfaces and eliminating an entire class of issues is better than trying to manage them to no end.
> Are your various friends/family all tech-y people?

Not at all. A few of my friends are techies and they use Android/iPhone about 50/50. Family is mixed as well. No one in family uses iMessage.

> I've seen people actually reject relationships because they don't date people with "green bubbles".

This seems like a feature, not a bug. I don't think you want to date someone who makes important life choices based on Apple marketing.

Edit: Is this a "Bay Area" problem or something? Or maybe a "young people" problem? I just can't imagine caring about whether someone messages me with "blue" or "green" text bubbles.
I wouldn't mess with Boeing. What a thankless job Quality Manager must be at such an company. Could cost your life.
It wouldn't be bad if we were sure it wasn't going to be abused to make apps where ads cannot be blocked, consent extortion pop-up cannot be removed, content cannot be copy-pasted, etc. unfortunately, this is the way this industry is always going, so there's good reason to be scared.

(It's also terrible for accessibility, for both disabled people or just regular users who expect to be able to navigate with the keyboard for instance, unless the framework re-implement everything itself, which I doubt)
You're right of course but the thing is - it's not "Hello, how would you like to make millions" - It's "Hello, how are you?" followed by a period of building the person's trust. And then it's an incidental "Oh btw, did I mention I'm involved in this great investment club". At this point a gullible person is seeing this other person as their friend (or more, a lot of people are lonely). Wouldn't you share your investment strategy with your friend?
Why is the branding/mascot a camel?

I'm reminded of Waldo, a mobile testing automation product that was acquired in 2023.

Their mascot is another camelid (not sure if alpaca or llama). https://www.waldo.com/
I don't know why that would be, but someone archived it: https://web.archive.org/web/20240321134120/https://spd-huntr...
I fear we're going to see this argument in absolutely every thread on this topic for the next few years and it's going to be argued ad nauseam. "You can just buy an Android phone" barely scratches the surface of the arguments being made.

For example (given the EU gives great context to this): Apple owns and maintains the only App Store that's allowed on the iOS platform. So they have a monopoly on the iOS app market. Is that right and fair? I'm sure plenty will read that want to reply "yes of course it's fair, Apple can do what they want with their own platform" but that's your opinion. Controlled app stores are a dramatic shift from the way software used to be distributed and as a society/whatever we've never actually had a discussion about it. It just happened.

> The government seems to want to make it illegal to have a tight experience, but why?

I'd argue the government seems to want to make it illegal for a tight experience to be the only experience available. And it's not hard to see the argument for why: competition is good. Multiple app stores or whatever would open up the market, Apple would have to make the case for why they should get 15/30% of an app developer's revenue. They might be able to make that case very easily, they might not. But a monopoly means they don't even have to try.

IMO talking solely about Apple and/or arguing about whether they have a "monopoly" isn't going to be that effective (sorry DOJ). The reality is that we live in a world that requires us to be connected, that connection is currently controlled almost exclusively by two tech giants to the extent that even a slightly smaller tech giant, Microsoft, utterly failed in launching a competitive third platform. Is this duopoly good for us as a society? Is it what we actually want? It's fair to at least be asking the question.