skysight-inc/HackerNews huggingface dataset:
Because the fair use clause he's using is about giving commentary.

The reason the tool is problematic is because derivative works are also copyrighted. LLMs aren't adding value to their output or using creative functionality. That are smashing multiple works together to produce a response. And, many of them are selling the output which is doubly problematic.

Consider this, if I sell a book about gandolf and Dumbledore getting into a wizards duel, both jk and Tolkien have grounds sue me. Adding another copyrighted source does not protect me.

This is especially a big problem in the music industry.

Now should copyrights be like this? I don't know. It feels to me that copyrights have the wrong balance all over the place.
Raise your hand if you always come up with the perfect API on the first try.
If you demand no breaking changes ever, that means you are insisting on staying with an imperfect API forever. Isn’t it better to iterate towards something better? Strive towards perfection? No, you just want everything you use to be someone’s first draft because changes aren’t allowed?

Well, even though the API isn’t perfect, you used it as a dependency in your own project. It’s good enough for your needs, and you don’t need those changes. The imperfect is good enough. You just don’t want more work of having to update. I get it. We’re all lazy.

The real mistake here is that very new and immature packages are becoming popular way too early on in their lives. People just can’t resist the new hotness. If you want a stable API, use things that are very old and have already been polished extremely thoroughly. They made their breaking API changes long ago, and now they are much less likely to change.

I know we all laugh when people keep the beta tag on their software for way too long after they already have tons of users depending on it. GMail was a notable example. But maybe that’s not wrong. Maybe wave that beta flag high up for several years until you are sure you have figured it out and are ready to stop making API changes.

If you think about the packages the author uses as an example, like node and react, I don’t think the beta flag would have necessarily been innacurate.
Neither of those (Whatsapp, Instagram were profitable, far from it.

And most companies making billions need thousands of employees, especially software devs need to understand that.
But using a Common Table Expression is a workaround, kind of like using a paper towel as a plate. Just give us a plate...
There are some images you can't import into Photoshop, most notably being scans of legal tender. This is for a pretty obvious and on-the-nose use case, but perhaps we'll see GenAI given similar guardrails.
Well that sounds like good news, are there any practical applications for quantum computing other than breaking commonly used encryption that would push the modern world even further towards a complete surveillance state?
Streaming providers steadily increasing their prices, while reducing the quality of their offering and forcing paying customers to watch ads.
Nothing will be stolen. Artists will use the same tools to check if their work is infringing that those companies and right holders have. There will be no Coca Cola logo/Super Mario Brother/CPO in that work.

The artists in the West wont get paid, because they wont get any jobs and those in other countries will. Maybe less, because they are more productive and the market is saturated, or maybe more demand will be created due to lower prices.
This makes sense, but then why the use of `=` for function arguments?
The posited DRY-related benefit ultimately also comes back to conciseness, because (as the author does note) you can always avoid repeating yourself by assigning expressions to an intermediate variable instead of repeating expressions.
>why would any sane person want to create something that not only could be stolen and redistributed sans attribution, but that would be?

Well it wouldn't work with current business models, but maybe something would emerge? Some indie games or open source projects live off donations. Kickstarters exist. And of course completely free art exists. Also, if you design a chip, or a phone, or a car, I'm not sure if it's really the copyright that's stopping other people from stealing your idea. It's the tools, the people and the understanding relate dto your idea that they don't have.
I am currently working with about 100TB data on GCP with BigQuery as a query engine and simple hive partitioning like /key3=000/key2=002/. We are happy because we can run all the queries you want and it is insanely cheap. But latency is reaching quite high levels (it doesn't matter so much for us) but I was wondering, if implementing Iceberg would improve this? Has anyone experience with this?

Overall this kind of architecture is just awesome.
Ironically, in Japan, most cheaper tofu (and natto) is made from Canadian soybeans.
Quantum doesn’t help for NP problems.

In the case of codebreaking, where QC may have an advantage, AI is unlikely to provide an alternative because modern AI is all based on finding probabilistic patterns and cryptography is explicitly designed to be resistant to that kind of attack.
I will update my language that mimicked the original comment. However it is not simple discussion. Here is a snippet related to Japan’s law, updated from ca 2018 with AI systems in mind, and clarified recently. I personally find it totally reasonable and support it.

“The use of copyrighted products or materials to train generative AI models would be prima facie copyright infringement under the Copyright Act, as it is a reproduction (fukusei) or other form of use of the copyrighted work. However, Article 30-4 of the Copyright Act stipulates that the use of copyrighted works by generative AI for learning purposes is allowed in principle.”
And that tech problems are an even smaller part of the important problems that the company needs to solve.

And that within those tech problems, human factors (getting people to agree, to do all the work rather than just the fun part, to deliver a sensible level of quality) dominate the problem, rather than actual tech issues.

And within THAT subset, some of those tech problems can be sidestepped by a bit of clever decision making.
It is revolutionary in what it does. There's no question about it. The problem is that a lot of people think it does something different or more practical, like creating new drugs. That is one of the ultimate goals, but it's way down the road.
Copyright laws, in my opinion, should mimic (with mild extension) patent laws

Say...double the length of a patent (for institutionally-produced work) or lifetime of the creator(s) (for everything else)

If patents last for 15 or 20 years[0], then anything created by an organization (business, non-profit, etc) should cap at 30 or 40 years (I would push for 30 ... but would settle for 40)

Anything created by individually-named and -credited people should last until the last of them dies (or they agree to relinquish their copyright)

You write a book at 20 and live to 80? You have a 60-year copyright. Write a book at 74 and die at 81? Congratulations - 7-year copyright.

You write a technical manual for a business? It is under copyright for no more than 40 years. (Again, I would lobby for 30 - but 40 is still in the realm of plausible.)

Most corporately-produced, copyrightable items have zero value at well-under 20 years (let alone 30 or 40) - but even those that do have value past that point (say ... Yankee Doodle Dandy, or the like), are not especially likely to be "infringed upon". Keeping with the movie example for a moment, say you look at the 1969 movie The Italian Job. It was remade in 2003. Remaking a story 30+ years on detracts in no way from the previous rendition, but can give a new value.

-------

[0] https://en.wikipedia.org/wiki/Term_of_patent_in_the_United_S...
No, it's more akin to if Photoshop had a 'Mario' stamp which when used would stamp a random piece of Mario artwork from the games. Do you think this would be in violation of copyright?
Cue my surprised face in 3...2...1
Aeron is for a pretty different use case. Kafka is optimized for volume batches, Aeron is for very low latency transactional. I wouldn't use one in place of the other. They have very different operational requirements too.
Can anyone comment on the issue of hallucinations? The author only mentions them briefly and I cannot gather how big of a problem this is. Apart from the literal quote the LLM hallucinated, wouldn’t all the other information have to be double-checked as well?
Ask yourself what you want to achieve - if you want to create electronic projects, order your boards from the cheap manufacturers online. If you want to become really good at the process of making PCBs, then look at making your own.

This is the reality - unless you’re only interested in the most simple stuff (one or two layers but no plated holes, no vias, no solder mask, etc.), you can spend years trying to develop and tweak processes to try and achieve something that won’t match the quality and reliability of something you can have delivered in less 7 days for $10. Not to say that developing PCB processes isn’t fun to some people, and if it is then more power to you, but I’d rather spend the time on the electronics than spending all of it trying to build PCBs.
So, mostly celebrities, media, and typos, like the rest of the internet. Fortunately, it's got a lot of other stuff too, and unlike the rest of the internet, you can just enter Wikipedia at the point of whatever you're looking up, without being bombarded by recommended content. It's completely self-serve. I'm glad that Wikipedia is not yet incentivized to maximize daily active users—or some other dumb metric—otherwise this would all change.
Nordic countries are very capitalistic and yet they have flexible social protections in place.

If anything, they're more free market oriented than the US, see Tesla versus IF Metall.
Should not be a problem in the EU. Article 3 and 4 of the „ Copyright in the Digital Single Market“ Directive already regulate this.

Summary by Wolters Kluwer:
[…] Everyone else (including commercial ML developers) can only use works that are lawfully accessible and where the rightholders have not explicitly reserved use for text and data mining purposes.

AFAIK they are discussing something like a robot.txt to flag stuff as „not for training“. You will probably be expected to implement some safeguards and of course the end user will have to be careful in his use of the generated things.

Source at Kluwers:
https://copyrightblog.kluweriplaw.com/2023/02/20/protecting-...

EU Legal Text:
https://eur-lex.europa.eu/eli/dir/2019/790/oj
Stalin's purges extensively utilized false accusations and show trials, so there being a plausible story for each individual disappearance/conviction doesn't make this not a purge.
Five digit Gregorian years are a https://longnow.org/ thing, tho when a few millennia are not enough I prefer the holocene calendar https://en.m.wikipedia.org/wiki/Holocene_calendar
I mean sure I thought it was well understood too, but the article opens with:

"Today’s Silicon Valley billionaires grew up reading classic American science fiction. Now they’re trying to make it come true, embodying a dangerous political outlook."

So the author seems to think that tech billionaires do actually sit down and ask how to make Star Trek real.

People being generically inspired by science and technology can happen in many ways. Like, is Musk primarily inspired by sci fi, making money, a love of business, a love of engineering, politics .... you can't really separate out the different factors like Stross wants to do.
the main difference between java and pg vs closure and datomic (not: diatomic) is "immutability", which implies a different way of thinking about multithreading and multiprocessing and transactions.

instead of locking and mutexes and semaphores you think in terms of branching and merging, much like git or similar VCS systems.

what is "hipster" about that?
If you haven’t seen Mickey Mouse, Googled “cartoon mouse”, accidentally used it as inspiration, made T-Shirts, and sold them, Disney would be after you as well.
GOTO

1. Old solution becomes new again

Points 2 and 3

4. Dijkstra enters the picture with the paper: Go To Statement Considered Harmful [1]

5. Wait, did this ever happen? :’)

I wonder if there’s a modern language practice that extensively uses GOTO

[1] https://homepages.cwi.nl/~storm/teaching/reader/Dijkstra68.p...
I thought it was to emphasize how extremely slippery it may be, in a humorous manner, that even your tyres would cross! Much more striking that way IMO.
Yeah, I really don’t see everyone else giving up here because “funny magic parrot box” can write some mid-tier high school essays.

LLM people are really starting to veer into crypto-bro territory with the evangelising about how they’re the best thing since sliced bread and transistors.
Good. It's time to abolish copyright. Society must create distributed, open and uncensorable AI models that can synthesize humanity's knowledge so that it can be used by anyone.

Sorry if your 40 hour work won't pay you $10 bucks a month forever. That's the case for most of the rest of us: we produce for 40 hours, we get paid for those 40 hours, regardless of what we do.

Welcome to the club!!
Using generic text will end poorly. I predict a future where 99 of 100 requests result in Photoshop AI saying "I'm sorry, I can't do that". Google for silly trademarks. Facebook trademarked the word "Home". Star wars trademarked breathing under a mask.
> The copyright battles affect models that were trained with datasets where the copyright holder's approval is missing.

Which is likely to be most/all of the high-profile text and image generation stuff. Where do you get a vast and varied enough library of non-copyrighted text and images on a vast range of subjects? (Even if you rely on something like owning a social media platform, how do you filter out copyright-infringing content that users will inevitably post?)

More specialised datasets for specific tasks can be gathered safely, but, with the exception of self-driving vehicles, that's not really where the AI hype bubble is. It's all about ChatGPT, AI art, and the push for for AGI.
Is there an option to pay and to self host? Recently my boss asked me for things to spend money on to use up some budget at the end of the year. We can't use hosting services but would pay for support or an "enterprise" edition if we could host it ourselves. Probably not your market but just something to consider.
If the big companies have offices in Amsterdam and you're not Dutch, you'll have to move there but basically never be considered a local, even if you learn Dutch.
This is not the new capability that LLMs have pioneered. It's true that it is difficult to sift out signal from the noise of a vast data trove, but it is difficult in a way that people have been getting extremely good at since the late 90s. What you're describing is a Google-level capability, and that's truly a very complex thing not to be downplayed. But it's a capability that we've had and been honing for decades now.

I'm sure language models and transformer techniques will be (or more likely: already are) an important part of the contemporary systems that do this stuff. But I'm skeptical that they care much about GPT-4 itself (or other general models).

I'm not skeptical about whether they think it is useful and an important capability to incorporate ML techniques into their systems, I'm unsure how much utility they see in general (the "G" in AGI) models.
AlphaFold did not solve protein folding. You can tell this is the case because it doesn’t correctly model structures with missense variants.

AlphaFold is a huge advance and people are extending it to try to tackle this (eg AlphaMissense) but to say it solved protein folding is hyperbolic.
It would work for most of our business reporting, but BigQuery is so good at what it does and cheap that it doesn't make much sense to avoid it.

We also have some parts of our dataset that are complicated to compute.

An example is that we estimate the hours that people have spent working on incidents, with a method that we describe here: https://incident.io/guide/insights/workload

That requires us to do some expensive cross-joining of every action ever taken in an incident from messages sent to the channel to GitHub PRs being merged. We could make this incremental and optimise it for performance but using BigQuery by default means we don't need to worry yet and can leave the optimisations for when we're bigger and the engineering resource we'd dedicate wouldn't detract as much from customer-focused work.
We've come full circle.

Honestly I hate 'AI' or predicted recommendations in my personal feed.this type of stuff is fine in a explore category.

I actually feel as if the recommendations make it harder to find videos I enjoy
And also be more kind, tolerant, wish others, and oneself.

I agree that this is the ideal response, but unfortunately this hasn't been the general result and I doubt it ever will be. When an animal learns (or is tricked) that its environment is more harsh and dangerous than it previously believed, its instinctive reaction is to become less trusting, less curious, more withdrawn, and very often more prone to anger and violence. Humans are no different by and large.
ooh, that's useful. thanks.
How is unemployment so low then ? Most of these Anerican companies and layoffs happened in US too. Indian startups laying people off is due to less cheap money and faulty business models. Job growth in India is stunted due to economic mismanagement and the fact that the country doesn't do much manufacturing or isnt as important in the global supply chain. And the influence of AI doesnt show up anywhere. Overall a poorly written article.
The class of this problems where quantum computing gives an advantage is called BQP and it includes things like factoring but it is not known to include NP complete problems like the traveling salesman.
Stone Ridge 2023 Investor Letter [pdf]
How am I supposed to know how emoji's are rendered on your device.

What if my water pistol turns into a pistol by the time you see it and my harmless fun turns into a threat?

https://money.cnn.com/2016/08/01/technology/apple-pistol-emo...
> The option to do non-clever code is seemingly insulting to the self-professed architect, but it is in my own experience at least, vastly more maintainable and productive, not because it results in "clean" code - it's usually pretty dirty and nothing to write home about, but because reading even a page-long flat procedural function that's just doing its thing, is much more straight-forward than ping-ponging between some clever architect's conglomeration of decaying design patterns.

With all due respect, if you think you are talking about anything that I was attempting to describe, you are not. Everything that we do is based on simplicity and building the most simple things that we can. We may have different definitions of simplicity however. For mine, check out Rich Hickey's "Simple Made Easy" talk if you haven't seen it.

We simplify things by de-complecting or, avoiding entanglement. Having separate repositories is just one way to re-enforce the simplification that we put in place with design. We call that structural design (because it's the design of the actual structure of the application).

> on that note, it sounds like hmeh is trying to work around this problem with abstractions by doing a significant amount artificial partitioning.

I can guarantee you that none of the partitioning is "artificial". I think at this point that you have projected so much of your experience into what I am saying that it's impossible for you to see anything I say in any other light. This, by the way, is what I mean by tacit knowledge. I say X, and you either hear Q, or you hear XIENRSTIENRSTIERNST, either way, you cannot possible understand what I am saying. This isn't a negative reflection on you, or a positive reflection on me. It's simply the way that tacit knowledge works, and it's why most "internet arguments" are wastes of time.

> People promoting these things will of course then just blame it on the dev and call them average, or lacking in experience, because they can't identify and combine the various clever patterns correctly, they "are doing it wrong", and that's where I believe the above narcissistic tendencies I called out comes from.

And yet, it is literally lack of experience (and ability) that prevents people from doing things they cannot yet do. That's how expertise works. Pointing that out is not narcissism. A gardener does not have the experience (or training) necessary to do brain surgery. This isn't a mark against the gardener. The difference in our industry is that it is filled with expert beginners. People who think that because they make 6-figure salaries after their limited education and experience they can now understand everything there is to understand about software design. This just isn't the case. If a person sees that as a slight to a fellow human, that says more about them than it does about the person pointing to that reality.
How would that apply to Microsoft, Apple, nVidia, AMD, ASML, TSMC, ...?
Salaries or total compensation? As a senior SDE you can definitely make 150-180k TC in multiple European cities for FAANG, even outside of Switzerland.
This has been going on for decades and it's now the integral part of what "being a manager is" as it's taught at schools.

Inside these big companies, the forever shuffles, hiring and "losing people" like you're in the war it's part of daily life.

It's disgusting from a human perspective, trust me I know, but I still hope this way of doing business is not the best and therefor these companies will rot from the inside and give way of better ones.

The problem is the time of a big company is on a different scale of the time of an individual.
Fanfictions are controlled by unspoken common sense rules and protected by copyright laws. It's almost weird to hear fan content world being seen as a wild west, it feels like listening to a caveman description of an Apple Store. No they're not living there, they're - have you ever used currency? The round medals that people keep in pockets and trays?
Thanks, please don't hesitate to get in touch when you do try it!
> Especially true if that child or its mother has a huge market capitalization, large profit margins, highly-paid employees and shareholders eager to reap some more $$.

This is the bit I don’t get from the “feed everything to machine” LLM-maximalists. Do they think courts don’t take context into account, do they think all actions happen in a vacuum and that they can just skip along and ignore laws at their pleasure because “tee hee it’s totally definitely fair use bro, I’m totally an academic researcher-pinky promise”.

LLM bros ought to stop and have a think before they poison their own well, assuming they haven’t already done so.
I ate a ton of TVP growing up in rural Nebraska, of all places, because we were poor, it was cheap, and we were in a food co-op that ordered it in bulk.

Surprisingly not bad in a lot of dishes.
Equally, it's bad reporting to paint everything a Stalinian purge without the subtlety of the possible full story, not least when Russia may be involved.

Edit: a purge as practiced by Staline was to increase and solidify his personal power. There may be a similar dynamic at play here but at least in Qin Gang's case it seems to be much more complex. Simplistic and sensationalist labels prevent us from digging deeper into the interesting bit.