trentmkelly/LessWrong-43k huggingface dataset:
Don’t waste your time meditating on meditation retreats!
by EternallyBlissful — 2023-06-01T16:56:44.499Z

Do "Emotional Work Retreats" instead!

TLDR;

If your goal is therapeutic progress - the kind of progress you also hope to get from going to a therapist, i.e. become happier long term, get rid of unhealthy emotional patterns, and improve your relationships with others and yourself - then spending 6+ hours meditating each day on meditation retreats is not the best use of your time. Instead, I recommend you meditate for 3-4 hours each day and spend the rest of your time doing various therapeutic practices like Internal Family Systems, Focusing, Ideal Parent Figures, CBT, Core Transformations, etc. If possible, include as many therapy sessions as possible during the retreat (potentially over Zoom), as well as some deep talk, Authentic Relating, cuddling, etc. with friends or even with romantic partners. I would still remain silent as much as possible for most of the retreat except for anything related to emotional work.

I offer 'Being your retreat buddy' as a service, i.e. sitting 'emotional work retreats' with you for 30€/day.

What I would NOT do on meditation retreats

I recommend not getting distracted by and not "wasting your time" trying to do any of the following:

Become enlightened, reach the Jhanas, stream-entry, etc.
Generally, "make progress" with meditation. Deepen your practice. Gain Insight into Buddhist concepts like Impermanence, No-Self, Emptiness, Arising & Passing, the nature of suffering, etc.
Really figuring out Sam Harris’s “Look for the one who’s looking” or other so-called “pointing out instructions”, perfecting non-dual mindfulness

In my view, these endeavors are simply not the most effective/quickest way towards therapeutic progress. They are also somewhat "all or nothing": You gain almost nothing becoming almost enlightened or almost reaching the Jhanas, but you will have wasted a lot of time. Even if you reach the Jhanas, it's not clear at all how they help you with long-term therapeutic progress. And in my experience, insight into Emptiness, No-Self, etc., is transitory and not helpful anymore once you’ve stopped meditating huge amounts for a while. Even enlightenment is, at least in my experience, much less helpful for therapeutic progress than one might think. The world is full of enlightened narcissists, for example.

Even if you ONLY care about Enlightenment, I strongly suspect, based on my experience, that focusing on therapeu...
[Request]: Use "Epilogenics" instead of "Eugenics" in most circumstances
by GeneSmith — 2023-06-01T15:36:21.277Z

I am conditionally in favor of human genetic augmentation. In particular, I think embryo selection for intelligence, health, happiness and other positive traits is desirable. When I bring this up with people, one of the most frequent questions is "Isn't that eugenics?"

I never know how to respond. The term "eugenics" has absorbed so much baggage over the last century that it somehow refers both to swiping right on Tinder when you see an attractive person and to the holocaust.

These are not similar concepts. The fact that we use a single word to refer to both is crazy. I cannot count the number of debates I've heard about human genetic engineering where the disagreement boils down to people misunderstanding what the other is advocating for. One person will talk about the benefits of improving the genes of future generations, and the other person will attack the idea because they mentally associate it with "eugenics", and to them that means "the holocaust" and "state sponsored sterilization of people with disabilities".

The fault lies with the term itself. It means so many different things to so many people that its use actively hinders understanding. For this reason, I think we should start using a new term to talk about non-coercive means of improving human genetics.

Epilogenics: non-coercive means of changing genetics

This term originated with Aella on Twitter, and I think it perfectly fits the purpose I have in mind.

Examples of epilogenics

Selecting an embryo for lower disease risk, higher intelligence, or some other trait good for both the individual and society
Gene editing for the purposes listed above
Choosing an attractive spouse

Examples of things that are not epilogenics

State-sponsored sterilization of people deemed “unfit”
Rules against marriage of family members such a siblings and cousins
Things people think of as eugenics even though they are often bad for genes (i.e. genocide)

I would encourage the use of this term to clarify the difference between genocide and screening one's embryos for desirable traits.
Probably tell your friends when they make big mistakes
by Chi Nguyen — 2023-06-01T14:30:35.579Z

Big mistakes = Doing something that is actively harmful or useless by their own lights and values, i.e. doesn't help them achieve their life goals. (Not: Doing something that isn't in line with your values and goals.)

A lot of people think that others in the EA-ish community are trying to do something impactful but end up doing something harmful or useless. Sometimes they also work on something that they are just not very good at or make other big mistakes. A lot of people never end up telling the other person that they think they are making big mistakes. Sometimes people also just have one particular argument for why the other might do harmful or useless work but not be sure whether it's a bad overall. This also often goes unsaid.

I think that's understandable and also bad or at least very costly.

Epistemic status: Speculation/rant. I know of another person who might post something in this topic that is much more rigorous and has actual background research.

Upsides of telling others you think they are making big mistakes, wasting their time, or doing harm:

It's good on a community level because people get information that's useful to decide how to achieve their goals (among them, having impact,) so people end up working on less suboptimal things and the community has better impact overall.
It's good on a community level because it's pushes towards good intellectual conversations and progress.
I and probably others find it stressful because I can't rely on others telling me if they think I'm doing a bad job, so I have to try to read between the lines. (I find it much less stressful now but when I was more insecure about my competence, I found it really stressful. I think one of my main concerns was others thinking and saying I'm "meh" or "fine" (with an unenthusiastic tone) but only behind my back.)

Note that anxiety works differently for different people though and some people might find the opposite is true for them. See reasons against telling people that you think they are wasting their time or worse.

I and probably others find it pretty upsetting that I can't rely on others being honest with me. It's valuable information and I would like people to act in a way that helps me achieve my stated goals (in this case, doing good), especially if their motivation for not being honest with me is protecting my wellbeing.

That said, I often don't do a great job at thi...
Yes, avoiding extinction from AI *is* an urgent priority: a response to Seth Lazar, Jeremy Howard, and Arvind Narayanan.
by Soroush Pour — 2023-06-01T13:38:16.444Z
Link: https://www.soroushjp.com/2023/06/01/yes-avoiding-extinction-from-ai-is-an-urgent-priority-a-response-to-seth-lazar-jeremy-howard-and-arvind-narayanan/

The following is a post I published on my personal blog in response to criticisms published by Seth Lazar, Jeremy Howard, and Arvind Narayanan of Center for AI Safety's recent statement on AI existential risk. The target audience are those individuals who are not convinced of AGI x-risk. If you find the following arguments convincing, please share the blog post broadly so more people can can engage with arguments in favour of taking AGI x-risk seriously.

It has been a big week in the world of AI governance. Highlighting the two most important developments:

First, the Center for AI Safety led the signing and release of a "Statement on AI Risk", signed by dozens of incredibly knowledgeable and credible members of the AI community, including the CEOs of the 3 most advanced AI companies in the world (OpenAI, Google DeepMind, Anthropic) as well as some of the world's most prominent AI research leaders (Geoffrey Hinton, Yoshua Bengio, Stuart Russell), which reads:

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Soon after, 3 highly respected AI experts in their own right, Seth Lazar, Jeremy Howard, and Arvind Narayanan, published a rebuttal letter to the statement.

These 3 individuals are clearly knowledgeable in their domain and must have very good reasons for believing what they do. However, as a lifelong technologist who has spent thousands of hours investigating and forming an independent view of the arguments for and against the existential risk to humanity from AI[1], I found their arguments to be significantly flawed and ultimately not an effective rebuttal of the existential risks raised by the Center for AI Safety's letter and by the global community of people who have dedicated themselves to tackling the extinction risks of AI.

I want to highlight just some of the gaps in their rebuttal arguments, to show that, yes we

should

care about existential risk from AI. Humanity's very survival in the coming decades may depend on how we respond to this moment in history. Let's jump right into it.

AI extinction risk is more important than other AI risks

In the letter, the authors seem to indicate that human extinction risk should be considered no more important than other, non-extinction risks from AI (such as inequality). Quote:

Other AI risks are as important

And why focus on ext

...
Work dumber not smarter
by lemonhope — 2023-06-01T12:40:31.264Z

Is the 12th virtue incompatible with everything else?

Working smarter is about prioritizing, planning, allocating time & energy, strategizing, and results. See eg Humans are not automatically strategic

Working dumber is going brrr

One of the best long-distance runners in the world had part of her brain removed and cannot read a map and frequently goes the wrong direction

In the service industry it seems like good (co)workers are pretty much just doing the right thing quickly all the time. Rarely see them make a plan or anything. Even in a brand new environment or situation. Even a talented young person with no experience.

"Just take a love-minus-hate activation and add that to the prompt activation" sounds like an absolute newb idea. I like that idea but if I were trying to find the expert in a room then that statement would've disqualified them.

ReLU activation is the stupidest ML idea I've ever heard; everyone knows sigmoid um somehow feels optimal you know it is a real function from like real math. (ReLU only survived because it got a ridiculous acronym word thing and sounds complicated so you feel smart.)

The best drivers literally just like drive lmao. I know some folks who are very very aware on the road: they know who passed who and when, who is on a phone, where the cops are, etc but they cannot just chill out and stop swerving and braking.

I know >5 good cooks who add ingredients they want until the food is good. I've only met one who can grok a recipe from a book and execute.

Costs

All my good work was bad for my career and all my bad work was good for my career. Five different jobs I had with five different bosses:

During my most productive six months of software dev, my coworkers did not notice I existed because my code worked perfectly. I asked no questions because I knew what to do. I did not bounce any ideas around. I was an apt install to them. I did like a quarter of the total work on multi-million 10-person project and finished everything in my year-long contract several months early. When it ended I was not offered further employment.

I was a ticket-answering machine on this useless-seeming giant repo I didn't understand. Everything was triaged and so on. I mentioned to my boss that I didn't feel very productive and I wasn't sure what to do. He said my work was above expectations and he went on to help me multiple times in my caree

...
Short Remark on the (subjective) mathematical 'naturalness' of the Nanda--Lieberum addition modulo 113 algorithm
by carboniferous_umbraculum  — 2023-06-01T11:31:37.796Z

These remarks are basically me just wanting to get my thoughts down after a Twitter exchange on this subject. I've not spent much time on this post and it's certainly plausible that I've gotten things wrong.
In the 'Key Takeaways' section of the Modular Addition part of the well-known post 'A Mechanistic Interpretability Analysis of Grokking' , Nanda and Lieberum write:

This algorithm operates via using trig identities and Discrete Fourier Transforms to map x,y→cos(w(x+y)),sin(w(x+y)), and then extracting x+y(modp)

And

The model is trained to map x,y to z≡x+y(mod113) (henceforth 113 is referred to as p)

But the casual reader should use caution! It is in fact the case that "Inputs x,y are given as one-hot encoded vectors in Rp ". This point is of course emphasized more in the full notebook (it has to be, that's where the code is), and the arXiv paper that followed is also much clearer about this point. However, when giving brief takeaways from the work, especially when it comes to discussing how 'natural' the learned algorithm is, I would go as far as saying that it is actually misleading to suggest that the network is literally given x and y as inputs. It is not trained to 'act' on the numbers x, y themselves.
When thinking seriously about why the network is doing the particular thing that it is doing at the mechanistic level, I would want to emphasize that one-hotting is already a significant transformation. You have moved away from having the number x be represented by its own magnitude. You instead have a situation in which x and y now really live 'in the domain' (its almost like a dual point of view: The number x is not the size of the signal, but the position at which the input signal is non-zero).
So, while I of course fully admit that I too am looking at it through my own subjective lens, one might say that (before the embedding happens) it is more mathematically natural to think that what the network is 'seeing' as input is something like the indicator functions t↦1x(t) and t↦1y(t). Here, t is something like the 'token variable' in the sense that these are functions on the vocabulary. And if we essentially ignore the additional tokens for | and =, we can think that these are functions on...
How will they feed us
by meijer1973 — 2023-06-01T08:49:51.645Z

I was wondering how in more optimistic future scenarios people would be fed or get money to buy food et cetera. A common topic in discussions between pro and anti doomers is how the AI will kill us. It is a fair point, killing 8 billion people seems difficult at first glance. But what if the AI does not kill us, how will we eat? Our current agro-industrial food supply is very fragile. And feeding 8 billion people takes a lot of resources. So in the utopian view that question needs answering. There are several proposals.

Universal basic income

The AI would do most or all of the work so there is not much to do for humans. Therefore we build a welfare system with a universal basic income or some variant of that. It seems like a rather fragile system, because in this situation humans would have no economic value and so very little bargaining power. There is a great dependency of the AI’s and very little control in the hands of humanity.

One threat to this model could be that it would be very hard to find a broad enough tax base because of tax evasion. Especially in the early stages of UBI companies could go abroad to countries that are more tax-friendly. Or they could just find loopholes and don’t pay taxes (like the system we have today). In the current system taxes are mostly paid by the working middle and upper class (taxing labour is a lot easier than taxing profits or wealth). The wealthy people and the bigger companies are very hard to tax. This is an open problem that needs to be solved. Just stating that we will tax companies that use neural networks or something like that is not a solution. Companies are superintelligent at tax-evasion. I cannot tell you how they won’t pay the tax, but they probably won’t. Unlike the speculative how will a superintelligence kill us, there are a lot of examples here. The default is “pay less or no taxes”, despite large efforts to make companies pay their taxes.

Humans would still have jobs

As things are going cognitive work is rapidly being automated and there is still enough physical labour to be done. So maybe there are enough jobs. This might be enough to provide enough value and earn enough money at a subsistence level or above. This option does not get a lot of attention in utopian scenario’s because it is not very interesting. But we might have food.

Bargaining power f...
"LLMs Don't Have a Coherent Model of the World" - What it Means, Why it Matters 
by Davidmanheim — 2023-06-01T07:46:37.075Z

There are a variety of problems with LLMs, and I want to argue that they are all somewhat related, and are related to the idea of having a model of the world. This seems useful both as a conceptual model for thinking about why AI is unsafe, and to better explain why proposals like Davidad's are focused on the idea of a world-model.

What are the problems I'm referring to?

Machine learning models are often said to not be causal, to have only correlational understanding. This is partly true; their causal reasoning in LLMs exists, but at a different level than the model itself. Similarly, most machine learning systems fail to operate well outside of situations that resemble their training data. LLMs are known to hallucinate facts, and will also contradict themselves without noticing, or will confuse fact and fiction. They are also able to be induced to be racist, sexist, or malicious. In this post, I try to explain what a coherent world model is, and gesture towards why it could be a missing ingredient for addressing all these problems.

Humans

Humans are a good place to start when thinking about what works or does not work. And humans have largely coherent world models, and are largely rational[1], goal-seeking agents. We expect things to work in certain ways. We can be wrong about the world, but usually do so in ways that are coherent. We can predict what will occur in a variety of domains. When asked questions about the world, we may not know the answer, but we know when we are guessing. Humans can discuss fiction while knowing that it is not a factual description of the world, since part of our world model is the existence of fiction. Obviously, these world-models are imperfect, because humans understand the world imperfectly, but there are a variety of things we do well that machine learning systems do not. Some argue that these models, and the failures they exhibit, are central to AI risks as well.

What is Understanding? What is a Model of the World?

It is, of course, clear that machine learning models do little similar to what humans have. Scientific or statistical models, for example, might correctly predict outcomes in some specific domain, but they don't "understand" it. A human might know how orbital mechanics works, the same way an algebraic expression or computer program can predict the positions of satellites, but the human's understanding is ...
General intelligence: what is it, what makes it hard, and will we have it soon?
by homeopathicsyzygy — 2023-06-01T06:46:18.121Z

Timothy Lillicrap, a staff research scientist at DeepMind and (among other things) a member of the team that trained AI agents to play games like Go at a superhuman level, recently gave a talk at Harvard's ML Foundations seminar. The talk was not a showcase of new research results, as many of these talks are; instead, it was an attempt to grapple with the recent and stunning successes of large language models like ChatGPT. Tim admitted that his AGI timelines have become shorter, that he is worried, and that he is increasingly prioritizing studying AI safety. I found this admission by a mainstream AI researcher at the forefront of progress striking, and was also struck by what felt like tacit agreement (or at least, not tacit condemnation of the worries as totally unreasonable) from the room—a room with a large number of impressive academic AI researchers.

Tim is not the only one sounding the alarm, of course. There is the well-publicized request for 'pausing' the training of large models, and the eminent Geoff Hinton has publicly turned to doomerism. But the fact that AGI and AI risk were normal topics at a regularly scheduled Harvard seminar, rather than in letters or op-eds, speaks to a sea change.

Part of this change in sentiment is because 2022 was a banner year for spectacular and public-facing AI progress, between the release of ChatGPT and the release of powerful diffusion generative models like DALL·E 2 and Stable Diffusion. Given that GPT-4 is powerful enough to pass many of the tests previously thought to be challenging for AI (the SAT, the GRE, AP exams, law exams, medical exams, and even a quantum computing exam), researchers have begun to ask and debate other questions in earnest. Do large language models really have a world model, or are they just stochastic parrots? Could large language models be conscious? Moreover, these questions are not being contemplated on the fringe; philosopher David Chalmers talked about the consciousness question during a keynote talk at NeurIPS, one of the largest and most prestigious AI conferences.

The problem of AI safety and the potential consequences of AGI are no longer niche topics discussed on sites like LessWrong, but immediate and visceral issues. From academia to industry, there is increasing recognition that these problems are problems, and that they should be taken seriously. "Overpopulation on Mars" may h...