skysight-inc/HackerNews huggingface dataset:
It would be better at an intermediate cert level. Those are generally more aligned to an organization's boundaries.
How viable is a bank, which offers exactly:

* No physical bank branch. Only online banking

* a current account with a monthly fee

* a debit card that can be used at other banks' ATM - monthly fee.

* Debit card can't be directly used online. But you can generate merchant specific card numbers on the website.

Say, your monthly fee is $20. Then 4000 customers brings in a million dollars an year. How much is spent on fraud by a bank every year?

Could a single person with a few part-time contractors run a bank of this nature?
Transformers were known that they kept scaling up with more parameters and more training data so if the Open AI hit the limits of this scaling that would be a very important milestone in AI.
$5M to train GPT-4 is the best investment I've ever seen. I've seen startups waste more money for tremendously smaller impact.
text-davinci-003 but cheaper and runs on your own hardware is already a massive selling point. If you you release a foundational model at parity with GPT4 you'll win overnight because OpenAI's chat completions are awful even with the super advanced model.
Page 4 https://arxiv.org/abs/2302.13971

When training a 65B-parameter model, our code
processes around 380 tokens/sec/GPU on 2048
A100 GPU with 80GB of RAM. This means that
training over our dataset containing 1.4T tokens
takes approximately 21 days.

At $4/GPU-hour per A100 80GB GPU, that's $4 * 2,048 * 21 * 24 = $4,128,768.
Execution and rent-seeking – their culture is based on high margins so they tend to design things which will put a lot of hard to migrate logic into their app and even when they know they’re competing against more affordable options the prices will be eye-watering. I assume that the whales who do buy generate enough revenue to make up for the lost sales but it doesn’t seem like a good long-term strategy.
Upending a decades-long theory of reverse osmosis water desalinatio
Hard Disagree! Azure Portal is the most clear Cloud UI out there. From the Icons to the Blade/Journey UX scrolling sideways.
"Community is the future of AI"

Comedy gold. So first AI takes content created by human labor without permission or compensation. Then it centralizes the sum of it and monetizes it exclusively.

But wait, it gets better. You will now also supply your labor for free to fix/curate the current AI errors. Which will make the AI even better and more profitable, and yourself ever more obsolete over time.

Picking our brains to create a giant private for-profit brain. Why would anybody willingly contribute to this scheme with their free time, in a backdrop where their own relevancy is at stake? There is no community without human incentives, every community will starve and die. You can extend this doom scenario to all open (web) content.

More pragmatically speaking, StackOverflow is royally screwed. It was already on its way down for various reasons but this is a shock. AI coding assistants are rapidly spreading and improving, making it inevitable that programmers will have less need for a direct visit to SO over time. Worse, those actually keeping the site running is a small group of hardcore volunteers that you just alienated.

The future is even more bleak for their enterprise product. Having your private copy of some data and training it for internal use is rapidly being commoditized. Many companies have a Microsoft contract, giving them (potential) access to Azure OpenAI that allows you to do just that.

But that doesn't take it far enough and is just an intermediary step. Soon you'll simply point your enterprise AI at everything. Your Wiki, your documents, your SharePoint, your email. All of the companies' knowledge will be at your fingertips from any contextual UI, whether this is Word, Excel, Outlook or your code editor.

And not just that, this enterprise intelligence will be combined with the world's intelligence. In such a future, would one seriously need a private copy of StackOverflow? The future I describe is about a year away.
Not to mention the destruction of places of knowledge but this move destroys one of the last free third places in existence.
> Unfortunately, that "government-funded media" tag implies a lack of editorial independence

Yes, Twitter should do away with mere implication and clarify that not only is it government funded, the president of the CBC is also appointed by government: https://news.ycombinator.com/item?id=35605906
If you are driving something that really should not be a transmission line for your application from fast enough driver that makes it look like a transmission line then source termination (or just placing random resistor there as who the hell knows what the characteristic impedance of the thing really is) is the main thing that everybody should get from Handbook of Black Magic. Ferrite beads are for when you are driving external cables that should conform to some kind of interface standard and you have slew-limited driver.
Read the above comment regarding FBs breakdown of revenue per customer by region.
Real answer? Buy proprietary data from social media companies, credit card companies, retail companies and train the model on that data.
I don't think there is a contradiction at all. Altman is essentially saying they are running out of compute and therefore can't meaningfully scale further. Not that scaling further would be a worse plan longterm than coming up with new algorithms.
I follow your design, couldn't you also solve hallucinations with a "fact checking" LLM (connected to search) that corrects the output of the core LLM? You would take the output of the core LLM, send it to the fact checker with a prompt like "evaluate this output for any potential false statements, and perform an internet search to validate and correct them"
Show HN: A newsletter bringing you gems from the internet, daily to your inbox
I think the next step is multimodality, GPT-4 can "see" probably using a method similar to miniGPT-4, so the embeddings are aligned using Q-former (or something similar), the next step would be to actually predict image tokens using the LM loss, this way it would be able to use the knowledge gained by "seeing" on other tasks like: making actual good ASCII art, making SVG that makes sense, and on a less superficial level having a better world model.
We really are in the beggining (or maybe middle?) of something nasty. Reminiscent of things you read in history books. Certainly makes me want to leave the country. Turmoil of this scale in the US is probably something of a worldwide event though, unfortunately.
I wonder if there's some for-profit book rental company supporting this bill or its sponsors.
Here’s an interesting article about church-based credit unions. It looks like they exist to offer better options to underbanked populations and are often limited to church members: https://religionnews.com/2018/10/30/church-run-credit-unions...
Hi! I'm the VP of Engineering at Together. Thanks for writing up these instructions! FYI, you can also download all the files with one wget command:

wget -i https://data.together.xyz/redpajama-data-1T/v1.0.0/urls.txt

This is also mentioned on the dataset card for redpajama-data-1T on Huggingface [1].

[1]: https://huggingface.co/datasets/togethercomputer/RedPajama-D...
> Perhaps you will always find people willing to write interesting content about vintage computers, but you won't find many people who want to write about the intricacies of local bureaucracy, the tax system, and other boring-but-necessary topics.

> Donations just don't work, and people should stop pretending they do.

ProPublica and Mother Jones are doing just fine on donations.

But more to the point, these are absolutely not topics we want advertisers to have control over. I'll admit that donations leave many important forms of content underfunded, but the alternative you're proposing is a well-funded fox in charge of the henhouse.

> I make literally 100 times more from affiliate marketing than I do from donations. If I had no integrity, that multiple would be even bigger.

You might have integrity but you're a stopped clock: even when it's right it's never useful. The only means your customers without expert knowledge have of distinguishing your motivations is by looking at your funding, and as long as you accept incentives to lie, you can't be trusted.

> Let me put this another way: the bare minimum level of affiliate marketing covers all my expenses. Donations don't ever buy groceries. I could not put this much effort into my work if I had to get another job to put food on the table.

That's not evidence that donations don't work. That's just evidence you've built your business around advertising and not donations.
This seems important: "Since buying the farmhouse in 2019, the couple have spent about $150,000 on renovations"
Gaming, Gambling, and Porn also happen to be my 3 larges sources of dopamine.
As stated in the article, the initiative is dead in the water, as per the Missouri constitution:

> Article IX Section 10. Free public libraries—declaration of policy—state aid to local
public libraries.—It is hereby declared to be the policy of the state to promote the
establishment and development of free public libraries and to accept the obligation of
their support by the state and its subdivisions and municipalities in such manner as may
be provided by law. When any such subdivision or municipality supports a free library,
the general assembly shall grant aid to such public library in such manner and in such
amounts as may be provided by law.

I found this with a simple ctrl-f. However, thanks to the MO Library Association for knowing their law and educating people, otherwise I never would have looked.
Oh I absolutely agree! That's some sketchy stuff. Didn't mean to sound like I was discounting the rest of your experience.
Do you think GPT-4 was trained and then immediately released to the public? Training finished Aug 2022. They spent the next 6 months improving it in other ways (eg human feedback). What he is saying is already evident therefore.
I strongly believe the next generation of models will be based upon spiking neural concepts wherein action potentials are lazily-evaluated throughout the network (i.e. event-driven). There are a few neuron models that can be modified (at some expense to fidelity) in order to tolerate arbitrary delays between simulation ticks. Using actual latency between neurons as a means of encoding information seems absolutely essential if we are trying to emulate biology in any meaningful way.

Spiking networks also lend themselves nicely to some elegant learning rules, such as STDP. Being able to perform unsupervised learning at the grain of each action potential is really important in my mind. This gives you all kinds of ridiculous capabilities, most notably being the ability to train the model while it's live in production (learning & use are effectively the same thing).

These networks also provide a sort of deterministic, event-over-time tracing that is absent in the models we see today. In my prototypes, the action potentials are serialized through a ring buffer, and then logged off to a database in order to perfectly replay any given session. This information can be used to bootstrap the model (offline training) by "rewinding" things very precisely and otherwise branching time to your advantage.

The #1 reason I've been thinking about this path is that low-latency, serialized, real-time signal processing is somewhat antagonistic to GPU acceleration. I fear there is an appreciable % of AI research predicated on some notion that you need at least 1 beefy GPU to start doing your work. Looking at fintech, we are able to discover some very interesting pieces of technology which can service streams of events at unbelievable rates and scales - and they only depend on a handful of CPU cores in order to achieve this.

Right now, I think A Time Domain Is All You Need. I was inspired to go outside of the box by this paper: https://arxiv.org/abs/2304.06035. Part 11 got me thinking.
When librarians and teachers are your enemies, you are on the wrong side.
Oh, so protecting the children AND fighting homelessness? I'm sure there are plenty that see that as a two-for-one win.
If they have to axe places of public congregation to shore up the budget, before Republicans cut schools, libraries and parks, I propose they start with churches and their tax exemption.
If you ask any Fortune 500 CEO if he could magically take all the 135 IQ artists and academics and vagabonds, erase all their past traumas, put them through business or tech school, and put them to work in their company, they would all say 100% yes.

An equivalent AI won't have any agency and will be happy doing the boring work other 135 IQ humans won't.
I doubt they have trained on 0.1% of the tokens that are 'easily' available (that is, available with licencing deals that are affordable to OpenAI/MSFT).

They might have trained on a lot of the 'high quality' tokens, however.
Transcripts of other people's GPT chats are like photos of other people's kids.
On r/AppleCard there's a fairly consistent stream of people (often college students?) with their first credit card. The Path to Apple Card opens it up for a lot of people with poor or no credit history - people with credit limits of $1000 to $2000 (there are even some at $500 - https://www.reddit.com/r/AppleCard/comments/mwg9bn/this_is_m... ).

It gives people who are poorly banked an option to save in a higher yield account than what they may have with a big bank. One of the things that Apple Card does better than others is show that information about how much you're spending. With the savings account, showing that you are saving money can be useful in knowing that you've got it and its change over time (compared to logging into a bank that you otherwise rarely interact with).
Problematically, they're much better bullshitters than ChatGPT. And if you used Google to find them, they're probably either selling you something, or you had to navigate a minefield of people who are in order to find them.
“It took some man a lifetime maybe to put some of his thoughts down, looking around at the world and life, and then I came along in two minutes and boom! it’s all over.”

- Ray Bradbury, Fahrenheit 451
Tailscale comes to mind, see https://tailscale.com/kb/1100/services/
As I stated in my comment, $5M is assuming you can do a much much better job than OpenAI at optimizing your training, only need to make a single training run, your employees salaries are $0, and you get a clean dataset for essentially free.

Real cost is 10-20x that.

That's still a good investment though. But the issue is you could very well sink $50M into this endeavour and end up with a model that actually is not really good and gets rendered useless by an open-source model that gets released 1 month later.

OpenAI truly has unique expertise in this field that is very, very hard to replicate.
It’s cute that you think the republicans care about their constitution
115 IQ is perfectly fine for the majority of human endeavors.
Why?

Half their constituents can't/don't read. If they did, they probably wouldn't be voting for them.
Does the state of Missouri fund churches?

Edit: Beat me to it with the tax exemption edit.
$200K for a house is still cheap in most places.
We used to own a plug-in hybrid and it would keep track of when you filled up and occasionally force you to consume the gas in the tank. I want to say it was if you hadn't filled up in 6~9 months? Something like that.

We always buy ethanol free gas too, which lasts for years. I don't even empty our lawn mower over winter, it doesn't matter with real gas and a sealed tank.
Yeah, but it's not like the videos are private. Surely Amazon has the real advantage, given they have a ton of high quality tokens in the form of their kindle library and can make it difficult for OpenAI to read them all.
Yes: Through tax exemptions, faith based grants, vouchers and other methods.
Also scaling doesn't address some of the challenges for ai that chatGPT doesn't meet, like:

- learning to learn, aka continual learning
- internalised memory

bringing it closer to actual human capabilities.
Let me save you from the momentary daydream and predictable disappointment after reading the clickbait article:

> Mr. Thursfield and his Japanese-born wife, Chihiro

> The couple have spent about $150,000 on renovations, and there’s more to do
The reason we put everyone through school is we believe that it’s in society’s best interest to educate everyone to the peak of their abilities. It’s good for many different reasons.

It would be much easier to identify gifted kids and only educate them, but I happen to agree that universal education is better.
What quality do librarians and teachers possess that makes them infallible?
He has background in CEO (smooth-talking charmer in the VC crowd). That's why he's CEO.
> I'd bet that what he, and the competition, is realizing is that the bigger models are too expensive to run.

I think it's likely that they're out of training data to collect. So adding more parameters is no longer effective.

> most people don't need a brilliantly intelligent assistant, they just need a not-dumb assistant.

I tend to agree, and I think their pathway toward this will all come from continuing advances in fine tuning. Instruction tuning, RLHF, etc seem to be paying off much more than scaling. I bet that's where their investment is going to be turning.
Because no one will ever do that. Even if they risk being arrested
I'm a big EV fan who just dropped $60K on our second EV (my comment is probably still at the top of this page). I also used to be a professional auto mechanic, and I can tell you from both professional experience and personal (because we still have a rarely-driven ICE) that there is little basis in truth to what you posted.

If you are filling up less than once a week/once every two weeks you are risking stale gas and extra wear and tear on an internal combustion engine when you do use it...

No, the gas will have to sit much longer than that. We drove our previous EV for twelve years, while filling up the ICE Scion about once a month (if that). That Scion is still running just fine twelve years later. Don't even ask how long the gas sits in the VW camper van. I'd bet real money I go out, turn the key, and it'll fire up just fine.

I'm all for encouraging folks to get an EV, but inaccurate information only hurts the cause.
That’s not what that user is saying.
I was amazed to learn that 218 million people arrived in France during 2019 (tourist numbers indicate 90 million - not sure how the two numbers correlate).

Sources:

https://www.statista.com/statistics/436536/total-number-of-i...

https://www.diplomatie.gouv.fr/en/french-foreign-policy/tour...
Fair use means exactly the opposite of what you're saying. Companies who own IP can't do whatever they want. If use of a copyrighted work is transformative, it falls under fair use. It's true that it's up to the courts to decide whether specific works are transformative, but that doesn't justify game companies abusing individual content creators who don't have the financial means to defend themselves.
A lot of cultural revolution-esque actions unfortunately.
No. It can't be. Someone is shadowing him. Right off his left flank. A person on a skateboard rolling down the highway right behind him. The Deliverator, in his distracted state, has allowed himself to get pooned. As in harpooned. There's a big round padded electromagnet, on the end of an arachnofiber cable. It has just thunked on the back of the Deliverator's car and stuck. Ten feet behind him, the owner of this cursed device is surfing, taking him for a ride. Skateboarding along like a water-skier behind a boat.
I love the concept of getting links "out of the way," though I did it slightly differently. Instead of having links in the body of the text, links in the text of my docs tend to be links to citations in a reference section on the same page.

https://meadhbh.hamrick.rocks/v2/design/why_is_my_web_site_s...
someone posted an ssh bbs the other week https://news.ycombinator.com/item?id=35412416
We usually call this a "book".
Can anyone recommend something similar for Java / Smali ? Specific use case is reverse engineering Android application with partially obsufcated code.

Current approach is following “xrefs” in JADX and hooking methods with dynamic instrumentation with Frida (displaying backtraces etc). Got to be a quicker way while chipping away renaming classes and methods and mapping out source/sinks.
This is kind of how I'm set up. I use Ally for their checking and savings and privacy for their merchant specific card numbers. It works reasonably well for me.

It would be cool to have a real FDIC insured, licensed bank with this built in some how. I'd be willing to pay $25/mo for something that is first-party.
On the contrary, I was doing a calibration, asking about something to which I know the answers very well. To see if it was trustworthy.
My rule of thumb is that if something sounds completely evil and it exactly conforms to a political stereotype, there must be more to the story.

The article itself is somewhat thin on the details. Would anyone who knows more about the situation be able to give some nuance? I have a feeling this is brinkmanship at play.
Yes, it's called fascism.

Government by grifters for grifters and con artists. It can't get much nastier.
When you think that:
Complexity is the ultimate sophistication.
jajajajjajaja
Just last week it was reported that police showed up at the wrong house for a domestic violence incident and killed the (armed) homeowner. So I can see a lot of accidental takedowns with this tech.

Link: https://www.nbcnews.com/news/us-news/video-shows-police-wron...
So long as Software Engineer timing is not respected or tracked (compared to other engineering disciplines), in times of economic downturns, it will always be popular to shit on cloud.

People stop caring about the elasticity, about the operationial overhead to their people, additional 9's of availability, even about feature delivery velocity.

Tech companies turn into accountants looking at X $/Month in the cloud vs Y $/month from "buying a couple of servers".

Then you have "luminaries" like DHH and Elon blogging about how much leaner and better their services are as a result (never mind that twitter is now a buggy mess where everything is eventually consistent and breaks half the time)
Who's talking about infallibility? We're talking about literally making librarians a political enemy.
We'd love to but the immigration policies are very unfriendly.
> In case it's a surprise to you; crypto, after one of the worst bear markets, has now around $1.2 Trillion market cap.

“Market cap”?!? So WTF is it; some stock you want people to buy? Shitstocks usually brag about their “market cap”. Currencies don't.
knowledge and education, to name two.
It's been successful in Scotland: https://www.pearson.com/uk/educators/schools/making-an-impac...

This article compares Scotland with England: https://www.theguardian.com/education/2010/jan/19/phonics-ch...

One thing I've never seen addressed is the difference in accents between Scotland on the one hand, and England and Wales on the other. Scottish pronunciation of Standard English is in general closer to English spelling (e.g. "good" and "food" rhyme, and "r" is always pronounced).
> Once you've trained on the internet and most published books (and more...) what else is there to do? You can't scale up massively anymore.

Dataset size is not relevant to predicting the loss threshold of LLMs. You can keep pushing loss down by using the same sized dataset, but increasingly larger models.

Or augment the dataset using RLHF, which provides an "infinite" dataset to train LLMs on. Limited by the capabilities of the scoring model which, of course, you can scale the scoring model infinitely so again the limit isn't dataset size but training compute.
> I have bank accounts in 3 different countries

How is this even possible? Do you have citizenship in all three countries?
Good call. I was using ChatGPT.