skysight-inc/HackerNews huggingface dataset:
Show HN: liteLLM Proxy Server: 50+ LLM Models, Error Handling, Caching
Hello hacker news,

I’m the maintainer of liteLLM() - package to simplify input/output to OpenAI, Azure, Cohere, Anthropic, Hugging face API Endpoints: https://github.com/BerriAI/litellm/

We’re open sourcing our implementation of liteLLM proxy: https://github.com/BerriAI/litellm/blob/main/cookbook/proxy-...

TLDR: It has one API endpoint /chat/completions and standardizes input/output for 50+ LLM models + handles logging, error tracking, caching, streaming

What can liteLLM proxy do?
- It’s a central place to manage all LLM provider integrations

- Consistent Input/Output Format
- Call all models using the OpenAI format: completion(model, messages)
- Text responses will always be available at ['choices'][0]['message']['content']

- Error Handling Using Model Fallbacks (if GPT-4 fails, try llama2)

- Logging - Log Requests, Responses and Errors to Supabase, Posthog, Mixpanel, Sentry, Helicone

- Token Usage & Spend - Track Input + Completion tokens used + Spend/model

- Caching - Implementation of Semantic Caching

- Streaming & Async Support - Return generators to stream text responses

You can deploy liteLLM to your own infrastructure using Railway, GCP, AWS, Azure

Happy completion() !
But you're assuming that one likes cake/pizza/whatever cheap mediocre food product that's been mentioned. If I was that manager, I'd raise your cake/pizza/whatever with a freshly grilled filet and baked potato with a nice glass of wine. Remind them just what possibilities they are missing for being in the office like peasants! After all, isn't that the point of having direct reports being in the office while you're at home?
> AMO account isn't actually required for the user, it's required for some entity to create the "collection", but you don't actually have to have an account to use whichever collection with FF on your phone.

the backwards logic to justify this sentence is mind boggling. I wrote my own extension, I should be able to install it on my own device without getting permission from daddy Mozilla, period. I should not have to register for that, in the same way I dont have to register with Debian to install a program on Linux.
Voting this up for a spirit of curiosity and learning while having fun

Talking with the matrices, you sometimes get made up nonsense or overly verbose trite answers, but I think it's really valuable having a conversation partner with infinite patience for explaining things, and an almost accurate memory of just about anything you might find on en.wikipedia.org

You do have to be careful and to spot check any fact with other sources, but where I find it's great is helping understand complicated or abstract concepts. Having an actual conversation can be so much more efficient that banging your head against the dry definition until it starts to make sense. Anyone who's tried reading math articles on Wikipedia as a layperson and ended 15 tabs deep into very abstract foundational definitions (like "closed" and "open" sets, along with the eternal gift upon the world that is the world "clopen".)

So having a large pile of numbers standing by can genuinely make learning math a lot more fun, and it's one of the few topics where there is a provable right and wrong, so you'll notice pretty quickly if you're being fed contradictory nonsense
Reminds me of something I wrote a while back: https://jmmv.dev/2020/01/system-rewrites-and-tuning.html

TLDR: when facing problems like these, it’s too easy to look at grandiose solutions and, because they look like cool engineering problems, we end up justifying that it’s worth and reasonable to take on such year-long projects. But most often, the boring incremental solutions are easier and cheaper to achieve, while delivering benefit along the way.

This article shows examples of both, and I’m happy to see that the “boring” solution won.
Very curious to learn more about what the monolith was doing so incredibly poorly that you managed to squeeze that much performance out of it. Poorly written queries? Too many queries? Lack of any caching? Doing things synchronously when they could've been done concurrently?
As Google products go, this seems very lightly branded. Just the monotone G logo at the bottom. I’m sure they’re not hiding or shying away from ownership but this seems unusual.

Also, if Apple did something like this, maybe not web-based initially; how much data would someone need to download to do use an AI code assistant on device only?
That's not how court-ordered in-patient drug treatment works
Well, the lists are edited manually by humans. You need to report the overlays you encounter for them to be hidden for everybody else.
>>*arguing that criminal culpability is impossible because free will doesn't exist.*

OK - WTF?

-

There is a lot of discussion about 'free will' - however, thats around predestination, blah blah -

But to state that '*criminality*' doesnt exist DUE to predetermination is psycho bot BS.

Some people more educated in philosophy or logic or whatever that I may be --- EXPLAIN THIS??
Useless if censored. Censored people have to resort to the same channels and tactics that crackpots do if they truly believe in their discoveries and that just exacerbates the problem.

But censorship of unpopular opinion has always been going on. I don't believe there's a technical/service solution to this. The solution is a societal and political one and those are harder to come by than the next big technical innovation.

Edit - spelling
I'm not sure how old you are, but I feel like my entire life perks have been blinking their eyes at porn usage. PG-13 was created. They cleaned up times square. They tried to clean up Vegas. They tried to stop selling Playboy in 7-11. There was the whole issue with Traci Lords
Everyone knows that the only way to solve freeway traffic is to add another lane of traffic \s.
The context of this article involves the use of secret sharing for confidential customer records... so I think that's pretty relevant.

> The governments are all moving to gradually ban end-to-end encryption, as they worry about “terrorism” etc.

In a few countries like the UK where the residents are subjects and don't have civil rights.

> But no one is moving to ban cryptographic signatures.

Most (perhaps all) signature schemes can also be used to conceal data.

> Smart contracts can help ensure the immutability of code and I don’t know of any other mechanism that’s widely deployed today that can guarantee the code and data won’t be tampered with.

I'm remained of "Wakalixes makes it go." from "Surely, You are Joking, Mr Feynman".

It's not at all clear to me that there is some code immutability crisis, and I don't see the relevance of 'Smart contracts' to the integrity of code that isn't answered by tools like cryptographic hashes or an ordinary digital signature, -- stuff we've been using for decades.

The blockchain woo is a bit part of what what causes a lot of the public to regard the entire space as a bunch of grifters.
I love everything about the artist collective 100R.
uBlock Origin blocks connections to trackers/marketing companies by default. If you don't even connect to them, they can't fingerprint or track you.
Pseudorandomness – Survey / Monograph
>And seperate CPUs+dGPUs are what users want on PCs

In the age of unified memory and large AI models, that may not be the case for very long.
Uberspace sounds pretty great, but I don't think I would host anything within the jurisdiction of this stupidass German court.
> There's all this random crap that the school somehow has money to fund, it's insane.

Student loans that cannot be defaulted on.
There's a lot of nuance that isn't discussed in the WFH circles. I don't believe the cynical claims some offer of VC's losing out on real estate deals, or mid-level managers needing to boost their butts-in-seats numbers or whatever. For some, perhaps, but on average?

What percentage of long-distance relationships work out long-term? Do you become more or less close with friends after they move out of town? Do you still "know" them the same as you did when you regularly met in person for lunch or drinks? Is it awkward when you do see each other in person that one visit a year?

Human beings are social creatures - yes, even we introverts need interaction. A pure voice-only relationship can feel close, but it's not the same thing. Current technology like Zoom and similar is just not a good replacement for in-person interactions.

How many times have you spoken right over someone on a Zoom call or Discord server? The social queues are erased, and things become awkward and less productive. Creativity (which applies even in engineering) is diminished when you have to use online whiteboards and planning tools vs. a real physical whiteboard with everyone holding their own pen and bouncing ideas around in real-time. To use Zoom et al effectively, the pace of thought and contribution are greatly diminished into a sort of per-turn dance.

WFH is fantastic when you have a head-down, concentration-demanding task that is very clearly defined. For most other things, it's actually harder to get work done.

WFH is simply not a replacement for team cohesivity. There is a great deal lost when you do not meet in person with the people you work with on a regular basis - we can't just sweep that under the rug because we'd prefer to sit in our home office. We have to acknowledge the real tradeoffs - the loss of the "team".

That's not the say WFH doesn't work. Some people make an entire career working from home after all. It is simply an acknowledgement that WFH isn't as beneficial in all situations as you might believe.
>Why would the USA need to get him extradited through Sweden if the UK can extradite him to the USA directly without any sort of fake criminal charges?

Currently the extradition of Assange has been refused on humanitarian grounds, as the US can't guarantee that they won't stick him in an ultramax facility and/or drive him to suicide. It's still facing appeal, but it's entirely possible the UK refuses to extradite.

I believe historically Sweden has been more willing to ignore such concerns when dealing with the US, which is why there would have been an incentive to fake charges that could get him moved to Swedish custody.
It’s not like he actually ran away. This seems sort of reasonable.
It would demonstrate arguing in bad faith, certainly.
Also, many companies don't even have DBAs these days. DBA is, at best, a part time job for a senior+ engineer.
What will it do if there's no spare bandwidth for an extra channel? Will it kill an existing non-emergency call to handle it?
Not if he's compliant with the revocation. The idea behind bail is ultimately it's the amount that needs to make you show up for trial. Either it's an amount that you or whoever post it won't be willing to lose or it's the amount that someone can get for bringing you in. If you skip trial your bail becomes the bounty payable to whoever brings you in.
Disagree - there are multiple VC claiming multi-month due diligence before piling money into FTX. That premise and this multi billion exchange running the entire business on emojis and quick books cannot exist in the same reality.
I wish people would either stop calling existential AI risks "hype", or make some informed arguments about why it's not really a problem.

I know enough about ML and AI to follow the logic that shows that strong, general AI is a serious threat if it's not aligned, and we don't know how to safely align it yet.
As for extending the rationals by 0⁻¹ and giving up properties, a·0 = 0 can still hold for all a ≠ 0⁻¹.

To get an idea of what operations do and don't make sense when extending the rationals in the way you propose, define ∞ as our 0⁻¹ and refer to

https://en.wikipedia.org/wiki/Projectively_extended_real_lin...

substituting ℚ and ℚ̂ for ℝ and ℝ̂.

For a similar construction that maintains ordering properties, see

https://en.wikipedia.org/wiki/Extended_real_number_line#Arit...

Finally, note that the complex number equivalent to your construction is really interesting and useful:

https://en.wikipedia.org/wiki/Riemann_sphere

https://www-users.cse.umn.edu/~arnold/moebius/
Sure, that's a fair point.

However, Pulumi Cloud is only as valuable as Pulumi itself. If Pulumi didn't support deploying to AWS, for example, then it is useless to my organization which uses AWS and I'm obviously not going to consider Pulumi Cloud. So Pulumi does gain a lot of value even from just the Terraform AWS provider that it uses under the hood (and which it can continue to use because it seems Terraform provider licenses are not changing, which is nice).
https://archive.is/ct9jl
Most (tech) jobs involve huge amounts of co-ordination and creativity. This is waaaay easier in person, so having all the people needed in the same location makes a huge difference.

But the "same location" is almost an inverse square law. Same room - great. Really strong. Same floor. pretty good. Same building. Well that's starting to hurt. Different building, different town, different time zone. At the different time zone level the "gravitational interaction level" is something you need LIGO to pick up.

If you have a "team" of people answering first line support calls, spread them across the globe and get them to read off the same wiki.

If you just launched a 10M dollar project to fix "something vaguely defined that the board know is important but cannot agree on", welcome to hell. It's possible if you lock the best tech and rising business stars in the same room. Good luck otherwise.
Peter Zeihan recent video on this: https://youtu.be/A9-wfHgjTB8
That's the right attitude for an employee. If management says something like that, look for a new job. It's not sustainable to compete with fewer resources than your opposition. There's a reason college sports is going through a passionate realignment right now.
It’s both. Collective madness supports the ruler while the ruler cultivated collective madness. You don’t even have to look into the extremes to see this- both democrats with Biden and republicans with Trump are seeing complete unreality, while both rulers have taken well documented evil actions (mass murder, human rights violations, etc). Rulers can be evil while their supporters are also evil and insane as well.
Do you actually understand the arguments for concern about existential risk, though?

https://www.youtube.com/c/robertmilesai
Because courts and bail bondsmen have lots of scope for chasing, catching and detaining people that aren't available to banks.
> Concentration of power in media and online means only allowed views get through

Never in history has it been easier for someone to create their own media publication, present alternative views and make it available to anyone in the world. But rather than do that what those people prefer to do is rant, rave and demand that other publications carry those views.

Concentration of power in media and online is because the majority of people simply don't want to listen to the type of views that are typically censored. And since the media is a business the owners understandably listen to those people.

You can demand the right to free speech. You can't demand everyone has to listen.
Memories of Flame: The crash of TWA flight 800
You need to sign in with a Tesla account. Once you sign in, the service manuals are free for all vehicles. The paid subscriptions are for their proprietary software.
Realistically most FAANG employees are not doing anything that will be written down in history either. For example, whichever division at Microsoft is responsible for Teams.
Querying cloud provider API’s takes a while. It can be done, but the results are cached afterwards, which is the state file.

Refreshing the state file (and reconciling differences) also uses the cloud provider API’s. I don’t think we’re going to get better than that.

Unless someone makes a common standard for resource reporting that all cloud providers implement.
Grab was the company Steve Yegge left Google to go to. He quit during COVID when he could no longer travel to Asia, and his retrospective is glowing, including of the CTO Mark Porter(I think?): "frankly, the “Grab Way” of collaboration teaches you life skills, such as psychological safety and inclusiveness, which work everywhere else as well. [...] We US Grabbers made many mistakes on the journey towards becoming better Grabbers. Mark Porter led the charge on figuring it out, and we’ve learned a ton from him as he has evolved as a person before our eyes [...] I want to thank Mark especially for helping me grow as a leader and as a person. It takes humility to become a better person, and Grab is a humbling place. The core company value of humility comes straight from the CEO and cofounder Anthony Tan, who is one of the most humble, down-to-earth people you’ll ever meet.".

Is that, uhh, rose-tinted glasses?

2020: https://steve-yegge.medium.com/saying-goodbye-to-the-best-gi...

Previously, 2018: https://steve-yegge.medium.com/why-i-left-google-to-join-gra...

[I don't have any knowledge of Grab or the people involved other than reading these two blog posts, or of the parent commenter].
We need scientific dissidents now more than ever, do we?

Do we need vaccine harm advocates, anthropogenic climate change “deniers,” people who claim life begins at conception, dark matter “deniers,” and big bang “deniers,” just to name a handful of heterodox positions?
Show HN: Happy 11th birthday to ACPUL programming language and AnimationCPU OS
11 years ago I tried to make a better programming environment. I'm happy to welcome everyone who wants to join the Discord community.

https://discord.gg/4m9g7KSaMV

What We Know About ACPUL and AnimationCPU OS:

90408 Commits

Simple, minimalist and fun programming language

Fast as C without JIT

Fully translatable to any spoken language (for AI)

Live coding

Mobile first

Real time Time Travel Debugger

The first and independent programming language of Ukraine

Demo video: https://vimeo.com/833395245

Demo app: https://github.com/web3cryptowallet/Web3CryptoWallet

Pray for Ukraine
That was very in depth! As far as Pijul as a tool goes, I'm not seeing a git compatibility layer? So I think it's a neat project, but I probably won't try it because nearly all code is rooted squarely in git. Even if Pijul is perfect, you'd need to convince everyone else to use it.

Nevertheless, the increased interest in moving to patch based workflows from branch based ones is great. There's a lot of similar tools here (https://github.com/gitext-rs/git-stack/blob/main/docs/compar...) which I refer to infrequently.

Personally my favorite tool for living-with-the-reality-that-is-branches is git-machete (https://github.com/VirtusLab/git-machete).
i don't want the blockchain for anything. i want to understand what it is capable of. i don't need arguments that explain that i can do the same without the blockchain. i already know that. i am trying to learn what the blockchain can do, regardless if it is useful or not. your answers are not helping.

Any encrypted material in the blockchain is public. Any algorithm in it is also public. So I can just execute the code by hand, skipping any tracking code.

i do not believe this is true. otherwise smart contracts would not work. if i store something on the chain, and accessing it triggers a smart contract, then you should not be able to bypass the contract. i don't know how that works, but if it didn't work then smart contracts would not be enforceable. if that is the case i'd really like to see evidence of that.

Is the idea here making a million different watermarked copies of any given movie?

a watermark only tracks ownership, but it doesn't call home to count how many times the movie has been watched.
it's not something i want. but it is something the blockchain would enable. and again, i am not interested in learning how to solve that problem without the blockchain, but i want to learn how the blockchain would solve this problem, regarless of better alternatives.

If the blockchain is the ultimate arbiter of who owns what, then as soon as I manage to hack you

you would not just have to hack me, but you would have to initiate an ownership transfer on the chain. and these things already happen, millions of coins have been stolen by some mechanism that allowed the transfer of ownership on the chain. and the transfer could not be undone, at least not without resetting the chain. so clearly this is a weakness of the whole blockchain concept. and something the developers will need to address. the question here is, can it be addressed or is the whole concept so flawed that this can't be fixed?
I feel like they shouldn't have been let off so easy. They are equally responsible. Making this all like some sort of Dr. Evil SBF scheme is silly. They were a gang, working together on the fraud. The case against SBF couldn't be hard to prove, why do you need to plea bargain the other execs and get them to testify?
This is interesting to me because I'm currently debating with a friend about the Canadian C-18 bill [1].

One point we are discussing is how news sites should make money. The Internet has really disrupted the business model of news organizations. When I was young, people paid subscriptions to receive the local newspaper at home every day. It was the main source of news for many, like my parents, so it made sense to pay for a subscription. Now, with Facebook, Reddit, Twitter, and such, someone can read articles from hundreds of different sources all across the globe. I don't see how anyone would pay a monthly subscription for all those sites. Paying with microtransactions might seem logical, but after reading this article, my only conclusion is that nobody wants to pay for news content nowadays, except maybe for giant organizations like the NYT and others.

Other thoughts: maybe the problem is not with the microtransactions aspect, but more with the fact that someone had to download an app to access the content in question. Do people really discover and read news on their own, or do they read it after someone else shared a link through Facebook, Reddit? Maybe the friction was just too much. I don't know; I have never used the service in question. Perhaps it would have worked if they managed to use cookies or SSO on all those sites instead of an app.

[1] https://www.parl.ca/Content/Bills/441/Government/C-18/C-18_1...
TL;DR: The Online News Act lays out a framework that would require digital giants such as Google and Meta to develop agreements with Canadian news sites to provide them with compensation for sharing their online news content through links or other means.
It’s surely the leader in the bikes’ capabilities and reliability.
I agree that they buried the lede. You might have stopped reading before it. I'm usually completionist, I'd rather skim than miss a big chunk of text.
Thanks for the detailed response. Nice work.
I dont know anything about it, bit apparently exchange has moderates recipients https://learn.microsoft.com/en-us/exchange/recipients-in-exc...
Humans in general haven't so we'll hope for a better tomorrow, together, tomorrow.
A sort of franken-Terraform that treats (some dialect of) HCL as yet another Pulumi frontend language would be interesting.
Materialized views provide a way to have joins that are accessed like tables!
First thing that comes to mind is like twisted pairing on cables. Distributes the external/internal forces more equally. Is it anything like this?
To be absolutely precise, the claim, as evinced by the diet of Twinkies, that he was depressed, was still an absolutely outrageous, abusive and mendacious justification for the premeditated homophobic murders that he committed in cold blood. But it is true that eating Twinkies was presented as the symptom not the cause.