skysight-inc/HackerNews huggingface dataset:
> Clearly a company with $5-5MM in the bank can’t train a competitive LLM from scratch...

Wait, are we sure?

I'm going to make the massive mistake of assuming we're compute bound instead of memory bound, and assume we can train at FP16 (which is a bad assumption because, of course, you're doing calculus where the little pieces you're adding up could get rounded to zero at FP16 pretty easily... although mixed precision FP32/FP16 training is possible).

Consumer GPUs like GeForce RTX 4090 can do 3e14 flop/s under certain conditions with fp16. They retailed for about $1600. It took reportedly 3e23 flop to train GPT-3. A year is 3e7 seconds. So the upfront cost of retail GPUs doing 3e23 fp16 operations in a single year is potentially as low as ~$50k (and about $20k worth of electricity). (FP32 peak is about a factor of 4 worse, so ~$200k.)

So it's not actually impossible to imagine a particularly clever approach to training that could maybe achieve competitive LLM training for less than $5 million in hardware costs. (except for the fact that compute isn't really the bottleneck, memory really is.)
I imagine the advantage is mainly in cost - getting a chunk of walnut that large is going to be many times more expensive than getting the boards necessary to laminate the same shape.
> lack of ready to go infrastructure

That's just an excuse and concealment of the real reason(s). If there's a will there's a way. It takes 5 minutes to open any could provider (or dedicated server host) account and all it needs is a credit card and billing address.
From Wikipedia: "In veterinary medicine, the alpha-2 antagonist atipamezole is used to reverse the effects of xylazine or the related drug dexmedetomidine, but while this has been tested in humans in Phase I trials, it is not an approved medical treatment for xylazine overdose.
> Apple Stores isn't tech, it's retail.

Apple Stores are ultimately owned by Apple, which most would consider a "tech" company.
That is a gnarly wound.
For this reason you can change the tilde with `EscapeChar` configuration in your `~/.ssh/config` (or `-e` option, but I bet you want it to be permanent).
>The Anglo West's approach

Doesn't Thailand/Japan etc have even harsher anti-drug laws?
How many years did Portugal’s approach take to work?

Oregon decriminalized personal possession of basically all of these drugs in 2021. As of today, a large swathe of downtown Portland is basically one highly dangerous open air drug market. It’s going to end up like the Tenderloin and Civic Center in SF. The cannabis tax money flooding into addiction treatment businesses seems to be not making a dent. Between lower quality of life and some of the highest tax burdens in the US, Oregon is decreasing in population every year now.
The people "in power" have a vested interest in making the most-walked parts crime-free. That is not the case in SF where the wealthy don't have a Manhattan to interact with on a daily basis.
> dispenses back to them an amount of pharmaceutical-grade drug of interest, the amount dispensed being equal to the fraction that is present in whatever cut/mixed drug sample they handed in

If you're distributing drugs just distribute the drugs. Why cut in the dealers and cartels?
It can't be a reaction to those things, because they're

1) arguing that private companies arbitrarily deciding what they want to allow on their platform, even under direct request from government departments and powerful individuals, is actually good and needed to protect minorities.

2) They also defend one set of "propaganda corporations" from the other set of "propaganda corporations"; they don't rail against extremely partisan coverage, they have come to consider it a patriotic duty to even suppress true information that would give the opposition ammo.

3) They support the FBI and the CIA over Congress and the democratic process. Democratic processes are ridiculed as "popularity contests." They support US military interventions and sanctions regimes abroad against democratically elected leaders.
Yep, they were designed not to be hidden from users, but from testers, and the greater corporate environment.

Mortal Kombat created an environment where they were for users. By the time Google got around to doodles they had been neutered, turned into an expression of corporate pride. It took a few years for users to catch up. At this point they are just a dick wagging in your face.
The risk of non-delivery exists for these large firms too - see Hertz vs Accenture, for example. Or Indiana vs IBM.

In my company, to hire a company like IBM or Accenture we have to propose a plan, which has to be approved by many layers of bureaucracy. This plan includes a budget. A real risk is that the consultant underdelivers, or we need more budget. Then we have the sunk cost problem, and we have a bureaucratic one as well to increase the budget. And the consultant already costs more than hiring on our own and doing it in house.
https://www.theguardian.com/technology/2023/feb/16/meta-to-p...

> Meta’s board declared that the 40% increase was “appropriate and necessary under the circumstances” and was in place “to address safety concerns due to specific threats to his safety arising directly as a result of his position as Meta’s founder, chairman, and CEO”.

> The tech mogul’s salary is just $1 a year – but his earnings are far more. Zuckerberg ranks as the 16th richest person in the world on the Forbes billionaire list due to his shareholdings in Meta. In 2021 he received compensation from the company worth $27m.

https://www.theregister.com/2022/04/12/mark_zuckerberg_gets_...

> There were no bonus or stock awards given to Zuckerberg in 2019, 2020, or 2021, but that "other" amount increased by nearly $2 million each year. The bulk of this goes to pay for the Zuckerberg family's security at home and while traveling, even for personal reasons.

> Zuckerberg and COO Cheryl Sandberg both get an allowance for personal and family security, which Meta said is necessary to address specific threats to their safety. "We require these security measures for the company's benefit because of the importance of Mr. Zuckerberg and Ms. Sandberg to Meta, and we believe that the scope and costs of these security programs are appropriate and necessary," the proxy statement said.

> Zuckerberg is in the unique position, Meta said, of being synonymous with the company because of the size of its user base and its continued presence thanks to less-than-positive legal and media attention. "As a result, negative sentiment regarding our company is directly associated with, and often transferred to, Mr Zuckerberg."
> What happens to all that accumulated $? Does it just go away?

It may not have been there in the first place. Lifestyle inflation is a thing.
It's like an obese personal trainer.
Just finished a match. GLA demolition vs China nuke general.

I've seen that video - looked really nice. Disappointed as well.
Also what languages are you focusing on for the training? How will this not result in massive political arguments about where resources are spent.
Generally, companies don't have to ask unions for approval for anything that is universally beneficial for the union members, obviously no one would say no to improvements. Union contracts are there to stop changes that might be hurtful for the members, not benefits.

Have you participated in a real union somewhere before and read through the bylaws and such?
"Not approved" simply because it hasn't gone through the drug approval process -- because, among other reasons, xylazine isn't approved for use in humans in the first place. Not because it doesn't necessarily work or would necessarily be harmful.

Granted, atipamezole seems somewhat "cleaner" than yohimbine, which is a natural product with quite a few off-target effects.
Right, cuz 300 people tech companies are known for their job stability….

/s
The hearer’s right is well established as part of the First Amendment: https://www.mtsu.edu/first-amendment/article/1549/right-to-r...
> was a daily meth user before he was in harm Reduction

These stories usually feature someone who went clean. That it's someone who still using and contributing to society makes it unexpected. (I hadn’t considered the possibility of a functional meth addict.)
How do humans understand numbers?
Humans have 86 billion neurons. Yet it is still hard for humans to grasp the concept of numbers. Without exposure to well structured curriculum, most humans would struggle to add two numbers of three digits. It very much seems that symbolic reasoning of numbers is naturally alien to humans, and that humans primarily understand numbers spatially. How is the link between spatial numbers and symbolic numbers established?
Singapore and Japan have even better outcomes.
I’m wondering how sustainable this is.

Tech firms are throwing billions at this at the moment, and will continue to do so for a long time. As soon as faster tech emerges, they’ll upgrade - they don’t need to justify it to the public, and justification to shareholders is pretty easy. “We’re trying to secure our position at the forefront of a competitive market”.

Would a publicly funded project be able to keep up with that? Or would there be a big upfront cost, only to find that their technology is in the Stone Age five years from now?

With CERN it makes a lot of sense. There isn’t a vast market out there for it to compete with - just a few organisations that they collaborate and “compete” with. But trying to join the big spenders seems risky.

What am I missing?
> Portugal’s policy rests on three pillars: one, that there’s no such thing as a soft or hard drug, only healthy and unhealthy relationships with drugs; two, that an individual’s unhealthy relationship with drugs often conceals frayed relationships with loved ones, with the world around them, and with themselves; and three, that the eradication of all drugs is an impossible goal.

> “The national policy is to treat each individual differently,” Goulão told me. “The secret is for us to be present.”

If you want to learn more the article I linked is very good. It goes into great detail and I think covers how long it took to work.

But the reality is it's not just enough to decriminalize, you also have to provide appropriate care. Oregon didn't do what Portugal did - they only did part of it.
There are indeed liabilities for products sold at convenience and grocery stores
No, I didn't. I know the results aren't that good yet.

My guess was that they didn't apply RLHF much. There's going to be plenty of research coming in that area, I suppose.
This reads like a dystopian novel.
No. It's not true. The Greens as a party didn't exist when the Australian Labor Party resolved to ban uranium mining and join the Nuclear Non-Proliferation Pact in the mid 70's. The various Australian anti-nuclear movements (plural) largely arose and thrived during the late 70s and 80s. Public opposition to nuclear energy started to wane in the 2000s, accelerated by concerns relating to climate change.
The market's moved elsewhere, developing a triple-A game of any genre takes roughly the same amount of effort, however strategy has a small overall market share compared to FPS, RPG, mobile, etc. FPS games might have around an 8x larger market, but don't take 8x as much effort to build, so that's what gets approved.

https://www.statista.com/statistics/189592/breakdown-of-us-v...

Couple that with studios (or publishers...) often preferring to re-skin their existing IP and you get Call of Modern Warfare XVII: Rehash Edition, or Skyrim for your Smart Fridge.
I think this has a lot more to do with the US having a larger english speaking population under one legal framework. All of the capital goes here first since it has the most upside and makes it easier to expand to other english speaking countries.

I'm Polish and would never build a tech business targeting the Polish market first over the US, and it has nothing to do with regulations getting in my way.

Building ChatGPT or Google for EU citizens is a lot harder than doing it for English speakers.

Also most of the leading figures in AI are from Europe or Asia but followed the money to the US.

EDIT 3: Americans underestimate the value of being surrounded by oceans instead of Russia and Germany, which they end up attributing to their ingenuity or libertarian leaning version of capitalism.
Welcome, Neo, to the real world.

Most companies cant afford to throw large amounts of money at a team who’s entire job is managing a Jenkins pipeline.

That’s a part time project for a single developer at smaller places.
Totally good point and fair to be critical here. He is the CEO after all.
I feel like there should have been a special note for EDM in general. There are plenty of EDM sub-genres with instrumentals that top their respective charts. I’m all about grimy big room dance stuff, and there’s a lot of instrumental stuff.
You can buy drugs in every high school in America. Same for every prison. You can buy drugs on death row. We have a higher percentage of our population in prison than the soviets, china or Iran. So how far down the police state road do you think the USA needs to go before you figure out this isn't working?
> like an obese personal trainer

Eh, it's more like an obese doctor treating obese patients. The goal isn't become thin, but to reduce harm. Yes, the doctor should tell the patient they should lose weight. But in cases where that's unrealistic, there are second-best options before abandonment.
How sad. VMS originally ran on a VAX 11/780 which maxed out at 8 MB. Now it apparently needs almost 1000 times more memory.
In Belgium cars running on LPG cannot park in underground garages even though they go through very stringent MOT-like checks.

One of the problems is that conversions need more than just a switch of just gas to LPG. You need a tune and you need an adjustment / expansion of the cooling setup. LPG runs a lot hotter than gasoline and this needs to be managed.

Funnily enough modern gasoline engines are tuned to manage temperature by ... dumping in more fuel.

Which is why your turbocharged 3 cylinder .9L has amazing mpg while driving calmly, but once you push it it proportionally consumes way more.

https://youtu.be/9aO2vC_iMTI?t=252
Do you have a source? Or are you speculating?

There's a lot to like about those places, but their approach is antithetical to at least the way the US and Canada likes to represent themselves. Decriminalization seems far more in line with western ideology. Freedom and personal responsibility combined with liberty. What's not to like? Especially since it works.

Shall we solve the stuck gum under park benches with caning too? Smacking people with sticks does solve problems, but I'm not sure that's the kind of society I want to advocate for. Despite very much enjoying my time in Singapore. The benevolent authoritarianism model of Singapore is unique and interesting - and attempts to replicate it elsewhere have failed. Including a famous attempt in China where Deng Xiaoping and Lee Kwan Yew gave it a real college try in the early 90s. Worth reading about. [1]

[1] https://www.thinkchina.sg/construction-singapore-model-mainl...
300 is a pretty large number.

I’m not sure why you added a /s in there.
https://www.metaculus.com/questions/4472/when-will-australia...
I like people that try to get the most out of the integrated peripherals in creative ways. It's a dying craft in a world where a 300 MHz "micro" is just a dollar away.
> What does it say about your security policies and practices if engineers can sneak stuff in?

I mean, what does it say about hospitals that doctors can kill you with one stray cut? Ultimately you trust them to be benevolent.
Yeah, here's the message I got.

"As an early customer we'd love to hear about your experience. Feel free to share feedback on our community forum or reach out directly to our team."

It was really fast too, about 3 days after it was announced.
I did have GPT-3 access for about 2 years, and wrote tutorials and stuff about GPT-3.

I also paid for ChatGPT+ for one month, but other friends who have subscribed earlier didn't get it.

Someone asked what I wrote in the waitlist form; I don't remember because GPT-4 filled it in, lol
As an Australian living in Canada, I want to say unalike Canada, the term "Province" feels weird to be used here. Australia refers to its divided regions as "States".
Job stability at that company is not guaranteed. Often times it is poorly run and their lack of formal processes and adherence to discipline is a result of the managements' incompetence.
I kept reading to try and find where the design innovation was that allowed containers on these traditional schooner sailing ships. It seems they're not meant for containerized cargo, and that's a big problem. Loading and unloading these things is going to be a lengthy and expensive process.
> I just want to pay bills and have a life.

You value the right things. Congrats.
The parent comment was saying that they thought the IT systems for a law firm were managed by a lawyer in their spare time, which is different from this case you describe.

Maybe the IT folks properly and accurately described the risk, and were stopped by non-IT management. Alternatively, maybe IT didn't push as hard, describe the risk properly, or want to deal with an annoying patching process. Microsoft also deserves blame for the difficult of patching, as your link describes.

Either way, the real victims are the 114,979 folks who had their information stolen. If a bank gets robbed because they didn't do basic things like locking the doors or hiring a security guard, they don't get to raise their hands and say "BuT cRiMInAlS". This is literally the same as what IT does whenever they get hit by a highly publicized exploit that's had a patch for months.
Forget it. The Navy would never buy a new aircraft that could only ever be used for CAS in low intensity conflicts. They have no budget for that mission and no extra space in carrier air wings. The Air Force will probably keep a few AC-130 gunships around but they're just not survivable against any sort of competent air defense.
The article links to a memo written by the law school's dean. It's worth a read because it addresses this very topic with legal citations.
reddit has avatars?
does your in-person interview do that?
> its working at a level of meaning and intent

LLMs work on tokens, which, for the most part, are words (longer words are generally split up into multiple tokens). Using concepts like "meaning" and "intent" muddies the waters unnecessarily.
Meaning and intent can only be applied if structure can be captured. I'm not proposing building some god-machine that has a library of all possible regular expressions. I'm merely saying that it could be helpful to nudge something like a neural network towards looking for these types of structures, such that they can be a basis for probabilistic encoding. It may be possible that the meaning that gets abstracted thereafter might actually be more concise and generalized.
Drug users have been banging the drum about the dangers here for years now. All the drugs are toxic. There is no safe dosage. There is no opioid crisis. There are no opioids. It's all a toxic mess of unknown chemicals in the street drugs. This is a core reason why the death rate has been spiking upwards.

Really good to see more reporting about this, but also alarming at the delay from when the problem started and when mainstream press is starting to talk about it.

Advocacy groups have achieved a bit of traction of getting some media to correctly call this a "toxic drug" crisis, but here we see an example of a major national media outlet while thankfully still ploddingly coming around to reporting on the issue, still framing things in the deprecated terms of an "opioid crisis" framework that is years out of date.
Billionaire AI wars as Elon Musk says STOP but Bill Gates urges “age of bots”
If the government is going to give out drugs, then they should just run some pharmacies similar to state run liquor stores. No need to force users to buy from dealers, that makes no sense at all.
You seem very uniformed? Top 1% pay like 40% of all taxes or some crazy number..
logically, yes. As a matter of language, no, due to what the emphasis is on.
Join the discord is a charmingly opaque answer to the question of where to find learning resources for an opaque topic. I'd guess OP was looking for something like youtube tutorials or short "programmer's guide to..." type resources.

Obviously an early line of research won't have anywhere near the depth, but as an example the geometric algebra community has done a great job of this. Lots of 10 minute videos to 2 hour lectures. Steve Brunton's videos are another example "learning ML math" resource that has helped me a great deal.
Thanks, the [1] and [3] look interesting, [2] looks inconclusive.
For me, 300 is a large company. I try to work for companies of several dozen people, with a dozen or fewer engineers. It makes my impact much more visible. The tech stack is usually already a hodgepodge, but it's small enough, and parts can be improved or replaced easily enough.

Being to a FAANG company (I worked for the F and the G) gives one a view of colossal tech stacks, some wonderful achievements of technological thought, some well thought-out processes (and also downsides, all well-known). It's a great learning experience, but a chance to make any publicly visible dent are pretty small.
Sending even a grain of salt FTL is unlikely because of physics.
I got it quite fast but probably because I develop a consumer product with it.
Unfortunately it seems par for the course. The steam UI itself is unusable on laptops with smaller screens and relatively high DPIs, since the minimum window size is larger than the screen
> mass spec analyzer

Just how cheap do you think mass spec is?
So base model+RLLLMF performs as well as base model+RLHF. That could mean a lot of things - it could mean the base model puts a ceiling on the total possible accuracy, so having human-level performance at the RL step doesn't matter as much. And looking at the scores on the individual tests that make up the composite accuracy metric, that looks probable.
Morbid indeed. Sounds like classic separation of Wanting vs. Liking in addiction

https://en.m.wikipedia.org/wiki/Reward_system#Wanting_and_li...
While the large batteries have been extremely useful in stabilising a grid composed mainly of renewables, true grid-scale storage will likely not be based on lithium battery tech, but on things like pumped hydro and molten salt heat storage.

Having said that, Australian lithium exports are tipped to exceed those of coal by value within 5 years.

https://www.theguardian.com/australia-news/2023/apr/07/value...
If by "shout someone down" you mean "refuse to engage with them", then it's sometimes required. Nutjobs need to be ignored sometimes.