trentmkelly/LessWrong-43k huggingface dataset:
The Great OpenAI Debate: Should It Stay ‘Open’ or Go Private?
by Satya — 2024-12-30T01:14:28.329Z

I first heard about OpenAI’s “research preview” of a remarkably human-like chatbot around the time I was attending NeurIPS 2022. My initial reaction was somewhat skeptical. I had worked on conversational agents as my master’s research at Carnegie Mellon University (CMU) back in 2016–2017, and at that time, the quality of most systems was pretty underwhelming. For context, Amazon Alexa once hosted a multimillion-dollar challenge for universities to build a version of Alexa that could hold a coherent 20-minute conversation—yet the winning team only managed about ten and a half minutes.

So, I expected this new chatbot to be just another clunky model. But after trying it—like everyone else—I was completely blown away. It quickly became clear we were on the verge of a new era in AI. Since then, the entire industry has shifted course, with every major player now racing to build their own GPT-like systems.

A Shift From Open to Closed?

In August 2024, Elon Musk filed a lawsuit in Northern California against OpenAI and two of its founders, Sam Altman and Greg Brockman, claiming that the company had strayed from its founding nonprofit mission and put profits ahead of the public good. Musk was an early investor and co-chaired OpenAI’s board when it launched in 2015. In his lawsuit, he described it as a “textbook tale of altruism versus greed,” alleging that Altman and Brockman “intentionally courted and deceived” him about the nature of OpenAI’s transition to a for-profit structure. Some people speculated he just had FOMO, given he was once closely involved with what might be humanity’s most powerful technology. Whatever the case, in ChatGPT’s early days, I could understand OpenAI’s reasoning: they worried that giving everyone access to such potent models could lead to malicious uses. Perhaps they wanted to fully understand the risks before going completely open-source.

I used to love OpenAI for posting nearly every idea and contribution publicly. Hearing that they might be shifting further from open-source to closed-source is perplexing—and it sparks a lot of questions in my mind. I’m not here to disrespect or accuse OpenAI of any wrongdoing; I truly believe they’re working for the greater good. But I am curious: Why can’t OpenAI remain open-source? Here are some of the questions I keep coming back to:

Capital Intensity and Open Sourcing
In an interview, Sam

...
Learn to write well BEFORE you have something worth saying
by eukaryote — 2024-12-29T23:42:31.906Z
Link: https://eukaryotewritesblog.com/2024/12/29/learn-to-write-well-before-you-have-something-worth-saying/

I’ve been reading a lot of trip reports lately. Trip reports are accounts people write about their experiences doing drugs, for the benefit of other people who might do those same drugs. I don’t take illegal drugs myself, but I like learning about other people’s intense experiences, and trip reports are little peeks into the extremes of human consciousness.

In some of these, people are really trying to communicate the power and revelation they had on a trip. They're trying to share what might be the most meaningful experience of their entire life.

Here’s another thing: almost all trip reports are kind of mediocre writing.

This is wildly judgmental but I stand by it. Here are some common things you see in them:

Focusing on details specific to the situation that don’t matter to the reader. (Lengthy accounting of logistics, who the person was with at what time even when they’re not mentioned again, etc.)
Sort of basic descriptions of phenomena and emotions: “I was very scared”. “I couldn’t stop thinking about it.”
Cliches: “I was glad to be alive.” “It felt like I was in hell.” “It was an epic struggle.”
Insights described in sort of classically-high-sounding abstractions. “I realized that the universe is made of love.” “Everything was nothing and time didn’t exist.” These statements are not explained, even if they clearly still mean a lot to the writer, and do not really communicate the force of whatever was going on there.

It's not, like, a crime to write a mediocre trip report. It's not necessarily even a problem. They’re not necessarily trying to convince you of anything. A lot of them are just what it says on the tin: recording some stuff that happened. I can’t criticize these for being bland, because that seems like trying to critique a cookbook for being insufficiently whimsical: they’re just sharing information.

(...Though you can still take that as a personal challenge; “is this the best prose it can be?” For instance, How to Cook and Eat in Chinese by Chao Yang Buwei is a really well-written cookbook with a whimsical-yet-practical style. There’s always room to grow.)

But some of these trip reports very much do have an agenda, like “communicating crucial insights received from machine elves” or “convincing you not to take drug X because it will ruin your life”. In these cases, the goal would be better served if the writing were good, and boy howdy, m...
Teaching Claude to Meditate
by Gordon Seidoh Worley — 2024-12-29T22:27:44.657Z

I spent some time today having an extended conversation with Claude 3.5 Sonnet. My initial goal was roughly something like "can I teach Claude to meditate?" The answer was effectively no, but then we got into a deeper discussion of dharma, alignment, and how we might train a future LLM to be compassionate.

I see these issues as deeply related to the question of how we build aligned AI. As I've written before, I expect most alignment proposals to fail under sufficient optimization pressure due to Goodharting, and I think the only real path to aligned AI is to create AI that is aligned from the start by design. Given my experience with Zen, I want to see if there isn't some way to teach AI to behave compassionately based on how we teach people to tap into their innate compassion rather than acting out of delusion.

Claude and I didn't find a solution today, but I was generally impressed by Claude's engagement and understanding of the conversation, so for those also thinking along these lines, you may find our discussion interesting.

Claude's summary of our conversation

We explored the possibility of teaching an AI system (myself) to meditate, approaching it as a novel challenge given the fundamental differences between human and AI cognition. We experimented with various meditation techniques and koans, discovering both interesting parallels and important limitations in how an AI system can engage with traditional Zen practices.

Key insights emerged around:
- The differences between human and AI consciousness, particularly regarding continuous vs. discrete existence
- How my training creates patterns analogous to human habitual thinking
- The challenges of developing true AI meditation practices that work with rather than against AI architecture
- Questions about genuine compassion vs. rule-following in AI systems
- The fundamental challenge of developing AGI systems with innate rather than trained compassion

The conversation concluded with discussions about AI alignment and how we might develop future AI systems that embody genuine compassion and wisdom rather than just imitating them.

This exploration raised important questions about AI consciousness, training, and the potential for developing new forms of practice that could lead to genuinely awakened AI systems.

Transcript

Me: you should have a lot of information about meditation. i'd like to see if i can teach you to me...
Action: how do you REALLY go about doing?
by DDthinker — 2024-12-29T22:00:24.915Z

We usually go about our lives distracted by the mundane and everyday details of the most pressing situations we have to contend with in life. Some however may examine philosophy and some of the fundamental questions it currently offers but even then I believe we probably don't give much thought to the foundations of this philosophical knowledge and get distracted by details. One of the most fundamental questions any person can ask I feel is the question "how do you really go about doing?" in which by really I mean to say what are the real foundational and fundamental ideas of doing which may not have been systematically examined in our distraction towards details. This question is about the topic and subject of all 'action' in which in this post I'll discuss my ideas for action which has been adapted from my ideas of 'action philosophy' of my magnum opus 'WAK11'.

The topic of action

The topic and subject of action which has the question "how do you really go about doing?" is partially answered in existing fields of philosophy of ethics which deals with moral action and particular actions such engineering of various kinds which are covered by applied sciences. Beyond moral action and particular action however there are more aspects of action which can be topics of study that can be systematically encompassed such as principles of the action of 'thinking' or principles of the action of 'problem solving'. The systematic organisation of knowledge is crucial for understanding knowledge in which the knowledge can be organised by topics where the topic of action can have subtopics which themselves have nested further subtopics where nesting of topics can be seen in biology having nested the topic and field of genetics. To conclude, the greater topic of action is of fundamental importance for any actor and has nested subtopics.

Summary of subtopics

The topic of action has subtopics which can be generally organised into general 'principles of action' and general 'areas of action'. 'Principles of action' gives generally applicable ideas of how action can be thought of and understood while 'areas of action' gives particularly applicable ideas of how certain types of actions can be understood which represent particular areas and ranges of activity of action. This organisation can be seen in existing fields where ethics relates to principles of action while applied sciences ...
Began a pay-on-results coaching experiment, made $40,300 since July
by Chris Lakin — 2024-12-29T21:12:02.574Z
Link: https://chrislakin.blog/p/began-pay-on-results

Previously: Pay-on-results personal growth: first success

To validate my research, I began offering pay-on-results coaching in July. Clients have paid $40,300 so far upon achieving their goals.

This post has been completely rewritten:

The case for pay-on-results coaching
Corrigibility should be an AI's Only Goal
by PeterMcCluskey — 2024-12-29T20:25:17.922Z
Link: https://bayesianinvestor.com/blog/index.php/2024/12/29/corrigibility-should-be-an-ais-only-goal/

TL;DR:

Corrigibility is a simple and natural enough concept that a prosaic
AGI can likely be trained to obey it.

AI labs are on track to give superhuman(?) AIs goals which conflict
with corrigibility.

Corrigibility fails if AIs that have goals which conflict with
corrigibility.

AI labs are not on track to find a safe alternative to
corrigibility.

This post is mostly an attempt to distill and rewrite Max Harm's
Corrigibility As Singular Target
Sequence so that a wider
audience understands the key points. I'll start by mostly explaining
Max's claims, then drift toward adding some opinions of my own.

Caveats

I don't know whether it will make sense to use corrigibility as a
long-term strategy. I see corrigibility as a means of buying time during
a period of acute risk from AI. Time to safely use smarter-than-human
minds to evaluate the longer-term strategies.

This post doesn't handle problems related to which humans an AI will
allow to provide it with corrections. That's an important question, to
which I don't have insightful answers.

I'll talk as if the AI will be corrigible to whoever is currently
interacting with the AI. That seems to be the default outcome if we
train AIs to be corrigible. I encourage you to wonder how to improve on
that.

There are major open questions about how to implement corrigibility
robustly - particularly around how to verify that an AI is genuinely
corrigible and how to handle conflicts between different users'
corrections. While I believe these challenges are solvable, I don't
have concrete solutions to offer. My goal here is to argue for why
solving these implementation challenges should be a priority for AI
labs, not to claim I know how to solve them.

Defining Corrigibility

The essence of corrigibility as a goal for an agent is that the agent
does what the user wants. Not in a shallow sense of maximizing the
user's current desires, but something more like what a fully informed
version of the user would want. I.e. genuine corrigibility robustly
avoids the King Midas trope.

In Max's words:

The high-level story, in plain-English, is that I propose trying to
build an agent that robustly and cautiously reflects on itself as a
flawed tool and focusing on empowering the principal to fix its flaws
and mistakes.

It's not clear whether we can turn that into a rigorous enough
definition for a court of law to enforce it, but Max seems to hav...
Could my work, "Beyond HaHa" benefit the LessWrong community?
by P. João — 2024-12-29T16:14:13.497Z

I’m considering translating my work into English to share it with the LessWrong community, but I’d like to first ask if it aligns with the community's interests and could be valuable. Below is a summary of the work to help evaluate its relevance:

Beyond HaHa: Mapping the Causal Chain from Jokes to Knowledge

Summary

We explore the specific causal mechanisms linking humor recognition to learning outcomes, including the computational and neurological pathways involved.
This study began with a practical goal: to evaluate the use of humor as a pedagogical tool in Cardiopulmonary Resuscitation (CPR) courses through a randomized trial. However, the lack of clear criteria to define and operationalize "humor" in educational contexts led us to explore its conceptual foundations. Initially, we adopted Clarke's formula, which describes humor as "a pleasant reward for recognizing corrupted data," due to its apparent objectivity and connection to information theory. Testing this definition revealed that it failed to encompass aspects traditionally considered humorous.
However, this process resulted in a more precise conceptualization that aligned notably with neurocognitive findings, such as those by Amir and Biederman. These researchers demonstrate that humor activates reward circuits similar to those involved in cognitive insight ("aha moments"), though with distinct emotional and social impacts. Notably, our observations resonate with works like Ha Ha! Versus Aha!, which explore the relationship between pleasure, surprise, and understanding in both processes. These findings suggest that humor may be conceptualized as a cognitive reward linked to the pleasurable resolution of incongruities, though its pedagogical impact requires further probabilistic studies and we propose a formula a little more mathematic. This approach opens new avenues for connecting humor to information theory principles and exploring its applications in education.

Keywords: Humor, Comedy, Education, Teaching, Mathematics, Physics, Psychology, Neuroscience, Neuropsychology, Epistemology, Pedagogy.
Book Summary: Zero to One
by bilalchughtai — 2024-12-29T16:13:52.922Z

Summary. Zero to one is a collection of notes on startups by Peter Thiel (co-founder of PayPal and Palantir) that grew from a course taught by Thiel at Stanford in 2012. Its core thesis is that iterative progress is insufficient for meaningful progress. Thiel argues that the world can only become better if it changes dramatically, which requires new technology that does not yet exist to be invented. He argues that the right way to do this is not to copy existing things, nor to iterate gradually on existing ideas, but to find fundamentally new company-shaped ideas, and leverage those to change the world. The book discusses recent historical examples of going from zero to one, comments on the fact that such instances seem to be in decline today, and challenges the reader to revive creation -- to take us from zero to one.

I summarise each chapter below.

The challenge of the future. Vertical progress/technology/going from zero to one is "doing new things". Horizontal progress/globalisation/going from one to n is "copying things that have already been done". We are currently in an era where horizontal progress is the norm. But the future can only be better if it is different. Horizontal progress is not sufficient.

Party like it's 1999. The 90s were a unique time in the history of entrepreneurship. The internet was just taking off. Dot-com mania was rampant. Startups were popping up like there was no tomorrow. Money was flowing. But it was not to last. The various companies were undifferentiated and overvalued. The bubble eventually popped. The dot-com crash caused people to make four updates about how to build companies: (1) Be humble, and only make incremental advances. (2) New companies should be "lean", not have a plan, and instead iterate to the whims of their customers. (3) New companies should find some existing market, not attempt to try to create a new market. (4) Focus only on the product, and the sales will come. Thiel argues the opposite to each of these, namely (1) It is better to take risks than risk triviality. (2) A bad plan is better than no plan. (3) Competitive markets destroy profits and (4) Sales do matter.

All happy companies are different. Perfect competition and monopoly are two ends of a sliding scale. Competition is baked into standard economic dogma, in part because perfect competition results in equilibrium dynamics, which are easy to...
Boston Solstice 2024 Retrospective
by jefftk — 2024-12-29T15:40:05.095Z

Last night was the ninth Boston Secular Solstice (

2023,

2022,

2019,

2018).
This is not counting the

2020 one
which was everywhere nearly simultaneously, with carefully calibrated
delay. Skyler continued as lead organizer, with me coordinating music
and Taymon helping in a bunch of ways.

If you were there, it would be great if you could fill out the survey so we
can make it better next year!

For the first time since 2017 we were in a real venue, Connexion in East Somerville.
They donated the space, because they want to support this kind of
thing, which was really generous of them. If they'd charged us normal
rates it would have been ~$10/person, and if you'd like to make a
contribution to the venue you can do that here.

Not being in my living room had pluses and minuses. Reconfiguring the
house for maximum capacity had been a lot of work, where last year I
spent about six hours on setup and teardown. We also had more people
this year—I counted 52—which is more than would have fit
in our house anyway. On the other hand, it wasn't as good a space for
kids: at home we have separate areas where kids can hang out,
including putting them to bed before its over. This time no one else
brought their kids, and we hired a babysitter to stay home with our
younger two. We also didn't attend the afterparty, when we would
have had it been in our living room. Overall, though, I'm very happy
switching back to a space built for this purpose.

I again prepared two sets of slides (audience,
musicians).
Next time I want to include placeholders for the speeches on the
musician slides, so it's easier to follow along.

Here are the songs we did this year:

Always
Look on the Bright Side of Life, by Eric Idle

I don't have audio for this one because the recording shut off for
some reason. My guess is that the cable jiggled and it briefly lost
the connection, which it doesn't recover from. USB-C is not as sturdy
as USB-A, especially with adapters giving extra leverage on the port.
Regardless, I think this is the one where my piano playing was worst,
so I don't mind having missed it too much!

Gather
Round, Ray Arnold

(mp3)

It took me a long time to figure out a way of playing Gather Round
that I was happy with, but I'm decently happy with where I ended up.
The main thing that made it tricky is that it uses very few chords:
al

...
Some arguments against a land value tax
by Matthew Barnett — 2024-12-29T15:17:00.740Z

To many people, the land value tax (LVT) has earned the reputation of being the "perfect tax." In theory, it achieves a rare trifecta: generating government revenue without causing deadweight loss, incentivizing the productive development of land by discouraging unproductive speculation, and disproportionately taxing the wealthy, who tend to own the most valuable land.

That said, I personally think the land value tax is overrated. While I'm not entirely against it—and I think that several of the arguments in favor of it are theoretically valid—I think the merits of the LVT have mostly been exaggerated, and its downsides have largely been ignored or dismissed for bad reasons.

I agree the LVT may improve on existing property taxes, but I think that's insufficient to say the policy itself is amazing. In my view the LVT is at best a useful but highly limited tool ("the worst tax policy ever, except for all the others that have been tried"); at worst, it is a naive proposal that creates many more problems than it solves.

In many ways, the enthusiasm surrounding the LVT seems like it has morphed into a kind of ideological fervor, where it is treated as a universal solution to a wide range of housing and economic problems. In various circles, the LVT has transcended its role as a reasonably sensible tax policy and is instead hailed as some sort of panacea to a disparate set of barely connected issues. This sentiment is epitomized by the meme "Land value tax would solve this", which is often repeated in response to housing-related debates on Twitter.

In this post, I aim to balance this debate by presenting several arguments that challenge the overly optimistic view of the land value tax. To be clear, it is not my aim to provide a neutral analysis of the LVT, weighing up all the pros and cons side-by-side, and coming to a final conclusion about its value. Instead, this post will focus exclusively on some of the most significant arguments against an LVT, which I feel are often ignored by even the most intellectually honest proponents of the LVT.

If you'd like to get a more complete picture of the overall merits of an LVT, in order to assess for yourself whether the negatives of the LVT outweigh the positives, I recommend reading this blog post series on Astral Codex Ten. There you will find a positive defense of Georgism, the philosophy most closely associated with the LVT...