skysight-inc/HackerNews huggingface dataset:
> People who don't understand the basic 'theory/math', won't even understand things like parameter count, or the tradeoff between intelligence and response-time/running costs, a very important consideration for any real project.

I am not sure anyone needs to know the parameter count if you are using GPT4 or similar powerful LLM. Most people also would never need to know "the tradeoff between intelligence and response-time/running costs".

> People need to understand how they work under the hood.

Most people don't understand how computers work, and yet derive immense value from them (Non engineers, even most programmers I have met have no clue how the computer works mostly)
The same sorts of things we do while on PTO or on the weekends, but every day.
Well, previously you were a 'researcher' working on 'open problems'. Now the problem has been solved, time to become an 'engineer' working on implementing/optimising the solutions.

The usage of AI will spike 100x going forward, from a mere curiosity inside the company which only affects management decisions (Say sentiment analysis), to extremely intensive direct interactions with customers. This will drastically increase the need for reliability and tractability of the results. NLP researchers simply have to accept 50% of what they did is useless, and reorient around the other 50%, its the risk of researching in an open field.
Kind of surprised it took Apple this long to jump on the defer everything train.
FYI, Tabby is the name of an existing open-source product: https://github.com/Eugeny/tabby/
But aren’t the weights still not for commercial use?
He writes that "no business is safe at this point." I disagree and actually wrote my first blog post in 8 years today explaining why:

https://www.sethbannon.com/p/defensibility-in-the-age-of-ai
> some type of heuristics would be needed here.

Calculating a meaningful numeric difference between two chunks of text is fairly well-trod territory.
Meh. The difference between GPT-2 and GPT-4 is not as revolutionary as you're making it seem. The ability to process images and text as input is impressive, and also an evolutionary combination of things that were happening separately before.
IntelliJ Idea 2023.1 Is Out
Windows 11 KB5023778 update adds promotions to the Start menu
Will there be open live streams for everyone to access this year also? I watched Revision live during most of the event and it was definitely a high-point for me. I want to go in person also some time, but not this year.
By "bloc" do you mean allies who would actually commit military forces to a war? Tributary puppet states like the Soviet Bloc?

Russia's list of allies:

1. Belarus (maybe, on a good day)

China's list of allies:
Relevant link that I saw on HN earlier this week: someone asks people to sketch bicycles and then renders the sketches.

https://www.gianlucagimini.it/portfolio-item/velocipedia/

> Little I knew this is actually a test that psychologists use to demonstrate how our brain sometimes tricks us into thinking we know something even though we don’t.

> I collected hundreds of drawings, building up a collection that I think is very precious. There is an incredible diversity of new typologies emerging from these crowd-sourced and technically error-driven drawings. A single designer could not invent so many new bike designs in 100 lifetimes and this is why I look at this collection in such awe.
Thank you! We have an OCR pipeline already so you can upload the files and we’ll process them, chunk the text, create the embeddings and index them. Right now, we support PDFs, but the pipeline is ready to accept images as well. We’re opening those file types this week!
Someone posted this repost from the Cerebras Discord earlier, but sharing for visibility -

"We chose to train these models to 20 tokens per param to fit a scaling law to the Pile data set. These models are optimal for a fixed compute budget, not necessarily "best for use". If you had a fixed parameter budget (e.g., because you wanted to fit models on certain hardware) you would train on more tokens. We do that for our customers that seek that performance and want to get LLaMA-like quality with a commercial license"
With all the hype, if this doesn’t pan out, then the technologists will lose a lot of credibility like they did with the predictions around self driving cars and 3D printing. You can only make so many claims of imminent life-changing tech before people just call your bluff, depending on society’s tolerance.
Such thought patterns can equally easily lead further into depression. Indeed often one needs a motivating goal to get out of a depressive phase and as soon as everything becomes "meaningless", then no goal will be strong enough.
There is mention of Property Based Testing and minimization. That's a very nice technique. The fuzzer may generate various data shapes and types, but then once a failure happens the generated types know how to shrink/minimize to produce the smallest failing test case to present to the user.

The author mentions correctly that for composite generated types, the shrinking behavior can compose too. So, a list of integers might shrink both integer values, to a lower bound, and the list size to have one less element, so you wouldn't need to implement a custom list of integers shrinking behavior. You can just compose the fuzzing generators.

There is an excellent book by Fred Herbert about property testing. It's Erlang based but the general concept applies to other frameworks: https://propertesting.com/book_shrinking.html
Is that the analogy you're choosing? I'm not sure much of a wealth gap exists in software in general. In many ways software is the most egalitarian product in history, since it costs nothing to copy it. Sure, not everyone can afford a $60 game, but there are no $6,000 games.

I think a better analogy (or perhaps a more specific one, since you did mention "web services") would be computing services, i.e. rich startups with hundreds of thousands in credits and funding vs. single bootstrapped founder with a little bit of cash.
Curious to know what product you worked on and how is the team dealing with the realization of ChatGPT and all. Have you thought of what to take up next?
Yes. I've been delivered John Cena bing chilling memes with slightly racist undertones. I've seen memes about Tiananmen square. Winnie the pooh xi jinping memes.

And we're talking about TikTok content in America, not in Hong Kong.
The Fed Proposes a 4th Function of Money: Means of Social Control
But the same systemic weakness that enables Swatting can be exploited here. Specifically that the government assumes good faith. Instead of sending a SWAT team to your house I can sign up for a DDoS in your name.
the people working on this stuff have been at it for decades, and they have very specialized education even prior to that

why do you think you would comprehend it all in a few weeks?

hats off to the folks at OpenAI etc...they made a lifetime bet on AI and it paid off
“This Fairground Farce of Light”: Vladimir Nabokov’s “The Cinema” (1928)
> the pace is not "overwhelming". You just started paying attention to it.

I've been paying some attention since 2015 when I worked a neural net gig. And the pace does seem to be increasing. Just in a small corner of LLMs we see LLaMa and then alpaca and then the LORA fine tuned version of the LLaMa params and then the various effects of different levels of quantization - allowing some of these models to run on things like RPis. More people are working on this stuff now meaning that the pace is going faster. Yeah, most of it is incremental, but that doesn't mean that it isn't difficult to keep up with where the best results are being found.
> It can't be 'because it's federally regulated' because that would literally make it a 1st amendment issue.

thats what i was thinking as well... it seems for the us, its mostly about spam prevention and marketing messages (referred to as SHAFT in the link below)...

seems not intended for individuals but those messages intended for commerce?

https://www.omnisend.com/blog/sms-regulations/
Both India and China have rejected the price cap: https://en.wikipedia.org/wiki/2022_Russian_crude_oil_price_c.... The price cap drove market prices lower, and India has continued to buy at market price. The reason the price cap was so high was because negotiations in mid-2022 revealed that developing nations wouldn’t go along.
> if it is providing huge benefits for employee productivity

With this particular employee, using chatGPT has not increased his productivity or the quality of his work by any noticeable degree.

> I don't think the employee should be fired for this or anything

The problem isn't using the technology. The problem is sharing confidential information with an unapproved entity. That is specifically and clearly spelled out as a firing offense, for pretty obvious reasons.

Even if some people feel that it's an overly tight policy, it's a the stated policy and the company has every right to put and enforce whatever rules it wishes about the use of its own data.
I started with MySQL from the beginning because it is the DB engine I knew.

Over the years I have played with sqlite and postgress. I keep coming back to Mariadb for my web apps.
I see. Hadn't considered this. Yes, I see how that might be a concern.

What I think is important in this AI Spring is that we make it possible for people to run their own models on their own hardware too, without having to submit anything to a large, centralised model for inference.
Guess I am in trouble then if I ever get stuck into something like that. My 'public' information whenever I query it is a blend of at least 3 other people. Of which only one I know. One DB thinks I am married to my mother-in-law.
It is revolutionary for many. GPT2 was just another "looks cool" thing, GPT4 is just so jaw dropping that I am scared what GPT5 would look like.
It's no coincidence that other human-powered transport mechanisms (aircraft, boats, etc.) commonly use pedals for the drivetrain. It's a lightweight way to use the body's strongest muscles, especially when high power is critical.

See:

https://en.wikipedia.org/wiki/MacCready_Gossamer_Condor
https://en.wikipedia.org/wiki/MacCready_Gossamer_Albatross
https://en.wikipedia.org/wiki/MIT_Daedalus
https://en.wikipedia.org/wiki/AeroVelo_Atlas
https://en.wikipedia.org/wiki/Hydrocycle
> I do not know why he did not get out years ago.

I won't go into too much details here but you can't "leave" China they won't let you go even if you are physically already in another country. Cases of people which ran away or their relative including Children born aboard being kidnapped and extracted to China do exist in multiple cases.

Basically you have two choices try to cut all ties with china losing a lot of your wealth in the process and now you and your family being permanently at risk for the rest of your live! Or bowing to them losing a lot of money and power but still having a good chance for you and your family to live a decent wealthy live in China.

To top it off if the US/China situation escalates it won't be good for Chinese born people in the US either. E.g. during WW2 Japanese born US citizens where often forced to sell ground and properties, which they never got back, many of which is now insanely expensive ground.
The president of India belongs to a Scheduled Tribe. If optics is what you are concerned about.

Some occupations like Law employ such a tiny percentage of the population, that Caste becomes too coarse grained to be the only dimension of analysis for the basis of privilege.

You probably will have a better chance of breaking into the top echelons of this industry by living in the right gated society in Delhi that puts you into contact with these entrenched folks rather than being the right caste in some remote Indian Village.

To look at a broader scale of occupational distribution, SC/ST folks have 23% affirmative action Quotas in Govt/State universities like IITs, IIMs and Medical Colleges. Is the issue that they are unable to compete in the Industry post college?

Caste based affirmative action quota is 52% with overall affirmative action quota at 63%.

Do you think more affirmative action is necessary to fix the problem?

At this point it just looks like "
The Floggings Will Continue Until Morale Improves".

You could also say no matter how large a portion you cut out of a pizza, you will never feed a hundred people if one pizza is all you have.
It’s just a Beowulf cluster of Markov chains.

And yet it blows everything before it out of the water.
ah i see so Its just a new way to run inference on the non commercial model
I'm aware that China's economic collapse has been continuously predicted for the last 30 years, but they nevertheless continue to produce more and more real commodities and infrastructure. Of course that doesn't hold a candle to our services-rent-IP-speculation economy here in the US.
You're not wrong, all of that is great. But the capacity for even more spam calls and scammers generating fraudulent content with cloned voice samples will be an immensely annoying issue. Anyone who owns a phone in the modern age absolutely cannot be a Pollyanna when looking at this technology. There are real issues that must be addressed.
I found this article a bit dry compared to his usual writing. What do you find interesting about it?
You will be amazed to know that the Ball Brothers, who invested the Mason Jar/Atlas Jar, and perfected canning and soups... is also Ball Aerospace - who makes super-high end space components for the MIC, black projects, skunkworks, NASA etc...

All from the Mason Jar.

https://www.visitmuncie.org/a-legacy-etched-in-glass-the-bal...

(also "Aliens")
So who cares if its GPL licensed it can never be put into production
But where is the fun then? You can easily build rather fancy CPUs on an FPGA, fancy enough to boot a small Linux system in the end.
How can a GPU be optimized for StyleGAN but not LLMs? Serious question.
Nice, glad to hear you went to Iceland as well! I went in 2020, not sure when the YVR<->KEF direct flights started though! We just snuck in before COVID got crazy, and in fact our flight home got canceled the night before we were due to leave -- ended up getting booked a replacement flight through LHR though it required a bunch of stressful calls to Iceland Air.. that was an interesting finale to the trip. On the upside, all tourist attractions were basically empty and we had an even more "isolated" and otherworldly experience than planned! A+++ would visit Iceland at the start of a pandemic again :)
Me: "X has this problem"

You: "but you can use Y"

Me: "I know, just saying X has this problem"

You: "But you can use Y. Don't complain about X."
[I'm not an expert] but I believe .ckpt and .safetensors.

The problem with .ckpt is that it executes arbitrary code in your machine(very unsafe). While .safetensors was made by huggingface in order to have a safe format to store the weights. I've also seen people load up the llama 7B via a .bin file.
What does it mean to "offer the same terms?" Presumably the unionized folks are getting some compensation benefit compared to non-unionized (otherwise, what's the point?)

Since the terms are already different, how do you compare if the union rate (no CC tips) is better than the non-union rate (with CC tips?)
I swear there was an article on the WebKit blog that detailed how the scrolling was different, but am not able to find it now.
No, I'm not talking about stuff that people return to store. My point was to make stores dispose all that garbage that they sell to people, including packaging. Like instead of throwing it to our blue bins, we could bring it back to store for disposal.
If you buy the ticket from another country, it's no longer the same ticket because the origin is (assumed to be) different
Just want to point out that the human hearing range goes up to 28khz, not 20khz as is commonly claimed.

I can still notice a very subtle difference when I downsample 96khz to 48khz. It used to be more pronounced when I was younger, but I always thought that was the placebo effect. It wasn't until I went and read the Wikipedia article on the human hearing range that I realized that I wasn't imagining it.

Basically, music that correctly reproduces up to 28khz has a sparkle effect that's also very noticeable in a live concert hall.
Thanks for not mentioning He-Who-Must-Not-Be-Named !

I do sympathize with his line:

> It's "the only realistic way to address advanced AI bot swarms taking over. It is otherwise a hopeless losing battle."
Advances in AI are informing vehicle autonomy. Tesla is effectively developing an LLM where the road system is the domain language. Recent innovations in NeRFs are enabling volumetric object detection with a single frame from multiple cameras, etc.
> You can... (define "next number") ...with rationals (so I understand, but I can't give a procedure).

1/1

1/2, 2/1

1/3, 2/2, 3/1

1/4, 2/3, 3/2, 4/1

1/5, 2/4, 3/3, 4/2, 5/1

...

The first rational number is 0. Next is 1/1, that is 1.

Next number to a positive rational number is its negative version, for example 1/1 is followed by -1/1.

Next number to a negative positive number can be found by taking its absolute value (i.e. the previous number), locating it in the pyramid, and choosing the next number... skipping those fractions which can be simplified. For example -1/1 is followed by 1/2.

So it goes like this: 0, 1/1, -1/1, 1/2, -1/2, 2/1, -2/1, 1/3, -1/3... now we skip 2/2 because that can be simplified to 1/1, and proceed with... 3/1, -3/1, 1/4, -1/4...
Oh, it was visually distinct. Just not very much.
It's the place you where born at.

People did went much further then bowing to their Governments oppression in a way which still leaves them wealth to be able to live at home.
Old Reddit was better on mobile too. I don't need an app, I have a browser -.-
> My first (childhood) computer was a 486 (DX I think?). Pentiums were already out, even PIIs maybe. The LEDs on the front (remember those?) said "100", but I was always suspicious of these, because my computer seemed a _lot_ slower (at playing games – what else) than my friend's 133MHz... But his was a Pentium(!), so I never got to the bottom of it.

A Pentium is much faster than a 486 at the same clock speed, so that shouldn't be surprising.
While not exactly the same, your school (college?) may have access to the Naxos library, which may allow you to stream that for free (requiring login through your school library).
You know, people are playing with OpenAI's playground without knowing any fancy "theory/math" and are getting pretty good results.

Besides, making your own AI is not easy. Unless you're a big shot company, you'll probably just use an API of an existing one.
Your sentence is confusing because of the unclear use of "they". It sounds like in this case an undercover cop would come to someone asking for their controlled drugs and you're arguing that it should be entrapment.
Yup. My ex- ran into this when she worked at Starbucks. All sorts of "unintentional" back to back "close one night, open the next morning", split shifts, etc. And very little you can do about it.
> Over 7k USD.

I don't think you understand how ridiculously expensive Iceland is. Practically everything has to be shipped/flown in.

Just to give you an idea, a small completely unremarkable burger at a fast food place will be around $20-25. It will not make you full.

7K a month is realistic. Our vacation was still fantastic despite the cost.
That's missing the point. The point is that this is a problem with US (and other) companies just as much with Chinese ones. Any legislative solution should address the underlying issue and attempt to resolve the problem across the entire industry, not just with one single company.

If a company (Chinese or otherwise) ignores the law, then it would be appropriate to sanction them.
It’s kind of wild to hear someone say this and it leads me to believe your expertise is outside the world of transformers.

In 2014, we were training encoder/decoder models on maybe a billion tokens, mostly limited by models architected around time steps.

Today, we are training encoder only models on trillions of tokens, mostly hindered by eminently solvable stability problems (i.e. can we make enough parallel compute for our infinitely parallel models).

Maybe that 10 years of progress feels the same as going from LSTM (1997-ish) to decoder attention (2014-ish) over 20 or so years, but it doesn’t to me.
Tiny tapeout - 2 weeks until the next one closes, plenty of time
This is the absolute typical layoff story we're seeing at the moment - 2018- 28k employees, 2023 - 47k, layoffs? 1,400. That's crazy! Either structurally we've seen a massive movement of employment into these sectors, or these companies have grown way less efficient, and given the ZIRP I'm very worried it's primarily the latter. What changed in the last decade that meant McKinsey needed to double in size? Not much would be my guess.
> are gang members waiting for someone to turn 18th birthday before commencing in gang related murder?

I wonder how many "undocumented" gang members would claim to be minors to get tried as a child instead of an adult. Remember the fastest 14 years old in Sweden? [0]

[0] https://www.dailymail.co.uk/news/article-3422000/Just-old-th...
That's what I thought too, the source code was not an issue so much as that.

What we need is some sort of "Large Language Model at Home" (like SETI@home was) that could crowdsource the creation of the model which would be free to use.
> Do people really give they actual contact details to do crimey activities ?

You'd hope not, but lots of people do when it comes to piracy. Private trackers often require accounts and interviews which can cause someone to leave a pretty extensive digital trail if they aren't careful including a clear record of everything they uploaded and when.
Go tubeless!

https://youtu.be/skW_Ysew3zw
> India is buying Russian gas and cooperating on payment networks to bypass American sanctions

Maybe it's time for the US and it's NATO allies to start restricting visas for nationals from misbehaving allies.

I guess the population won't mind since they can still immigrate/follow opportunities in Russia instead of NATO countries.
> Now is exactly the time for half measures - radical options proposed in the past largely failed due to their simplistic view of human nature.

Not sure I understand. What are some examples of "radical options" in policy you think have been attempted and failed? What are examples of "non-radical" options you think there's evidence would yield better outcomes?

With "not the time for half measures" I meant that if only half measures are deployed now then it seems likely millions of humans, many from regions least responsible for climate change, will be harmed, killed or displaced.

The idea of pricing in costs is already very familiar to people in their everyday lives. There are social norms like "you break it, you pay" and various regulations that price in costs.

> Going vegan because climate change and animal suffering is going to be a hard sell.

Keep in mind the distinction between policy level and individual level. The policy suggestion was remove meat industry subsidies, price in externalities, subsidize plant based foods. On the individual level the best outreach approach varies case by case depending on where the recipient is at. Are you already vegan or in the process of switching to vegan? If not then feel free to lift any individual level objections or obstacles you personally have and we can discuss them.
Great to hear that you're considering Milvus. Feel free to reach out if you ever have any questions/comments/concerns.

Just took a look at your docs and product page as well. Keep up the great work!
Even Saudi are turning on us (USA). Under the previous administration, they were a semi-reliable partner. This is going to have disastrous consequences for the dollar.