trentmkelly/LessWrong-43k huggingface dataset:
Originality vs. Correctness
by alkjash — 2023-12-06T18:51:49.531Z

I talk with Alkjash about valuing original thinking vs getting things right. We discuss a few main threads:

What are the benefits of epistemic specialisation? What about generalism?
How much of the action in an actual human mind is in tweaking your distribution over hypotheses and how much over making sure you're considering good hypotheses?
If your hope is to slot into an epistemic process that figures out what's correct in part by you coming up with novel ideas, will processes that are out to get you make you waste your life?

Intellectual generals vs supersoldiers

alkjash

Over time I've noticed that I care less and less about epistemic rationality - i.e. being correct - and more and more about being original. Of course the final goal is to produce thoughts that are original AND correct, but I find the originality condition more stringent and worth optimizing for. This might be a feature of working in mathematics where verifying correctness is cheap and reliable.

habryka

Huh that feels like an interesting take. I don't have a super strong take on originality vs. correctness, but I do think I live my life with more of a "if you don't understand the big picture and your environment well, you'll get got, and also the most important things are 10000x more important than the median important thing, so you really need to be able to notice those opportunities, which requires an accurate map of how things work in-aggregate". Which like, isn't in direct conflict with what you are saying, though maybe is.

habryka

I think I have two big sets of considerations that make me hesitant to optimize for originality over correctness (and also a bunch the other way around, but I'll argue for one side here first):

The world itself is really heavy-tailed and having a good understanding of how most of the world works, while sacrificing deeper understanding of how a narrower slicer of the world works, seems worth it since behind every part of reality that you haven't considered, a crucial consideration might lurk that completely shifts what you want to be doing with your life

The obvious example from an LW perspective is encountering the arguments for AI Risk vs. not and some related considerations around "living in the most important century". But also broader things like encountering the tools of proof and empirical science and learning how to program.

The world is adversarial in the s

...
Proposal for improving the global online discourse through personalised comment ordering on all websites
by Roman Leventov — 2023-12-06T18:51:37.645Z

This is a description of a project I’ve been thinking about making in the last few weeks. I’m looking for feedback and collaborators. [BetterDiscourse] name is provisional.

Updated on 9 Dec 2023: Completely overhauled the "Data and training" section, which no longer critically relies on LessWrong as a source of the initial training data and as a testing ground.

Problem

Social media promote tribalism and polarisation. Communities drift towards groupthink even if they were specifically founded to be strongholds for rational discourse and good epistemics.

Comments sections to media articles and YouTube videos are frequently dominated by support of the author (or the pundit guest of the show), finding rational critique there is time consuming, like finding a needle in a haystack.

X’s Community Notes, pol.is, and its development variant viewpoints.xyz are universally louded, but they all use the principle of “finding the most uncontroversial common ground” which is by definition low-information for the discourse participants (because most of them should already be expected to be on board with this common ground).

Big Audacious Goal aka mission statement: improve the global online discourse. Among humans, anyway, but perhaps AI debate could benefit from the mechanics that I describe below as well.

Solution

Each user has a state-space model (SSMs are on fire right now) that represents their levels of knowledge in this or that fields, beliefs, current interests, ethics, and aesthetics, in order to predict whether the user will find a particular comment insightful/interesting/surprising, clarifying, mind-changing, or reconciling.

To provide feedback to the model, a browser extension adds the corresponding reactions (along with negative counterparts to the above reactions: “Nothing new”, “Unclear/Muddled”, “Disagree”, “Combative/Inflaming”) to the comment on the popular platforms: Reddit, YouTube, X/Twitter. (During a later stage, [BetterDiscourse] may also host comments for media which don’t have comments themselves, such as NYT or YouTube videos with comments disabled, and display them through the same browser extension, like glasp.co is doing.)

Then, when the user opens a media or a comment section on some site, the browser extension simply sorts the comments for them[1].

I think this should very valuable already in this completely local regime, however, things may get even m...
Google Gemini Announced
by Jacob G-W — 2023-12-06T16:14:07.192Z
Link: https://blog.google/technology/ai/google-gemini-ai/

Google just announced Gemini, and Hassabis claims that "in each of the 50 different subject areas that we tested it on, it's as good as the best expert humans in those areas"

State-of-the-art performance

We've been rigorously testing our Gemini models and evaluating their performance on a wide variety of tasks. From natural image, audio and video understanding to mathematical reasoning, Gemini Ultra’s performance exceeds current state-of-the-art results on 30 of the 32 widely-used academic benchmarks used in large language model (LLM) research and development.

With a score of 90.0%, Gemini Ultra is the first model to outperform human experts on MMLU (massive multitask language understanding), which uses a combination of 57 subjects such as math, physics, history, law, medicine and ethics for testing both world knowledge and problem-solving abilities.

Our new benchmark approach to MMLU enables Gemini to use its reasoning capabilities to think more carefully before answering difficult questions, leading to significant improvements over just using its first impression.

It also seems like it can understand video, which is new for multimodal models (GPT-4 cannot do this currently).
Based Beff Jezos and the Accelerationists
by Zvi — 2023-12-06T16:00:08.380Z

It seems Forbes decided to doxx the identity of e/acc founder Based Beff Jezos. They did so using voice matching software.

Given Jezos is owning it given that it happened, rather than hoping it all goes away, and people are talking about him, this seems like a good time to cover this ‘Beff Jezos’ character and create a reference point for if he continues to come up later.

If that is not relevant to your interests, you can and should skip this one.

Do Not Doxx People

First order of business: Bad Forbes. Stop it. Do not doxx people. Do not doxx people with a fox. Do not dox people with a bagel with creme cheese and lox. Do not dox people with a post. Do not dox people who then boast. Do not dox people even if that person is advocating for policies you believe are likely to kill you, kill everyone you love and wipe out all Earth-originating value in the universe in the name of their thermodynamic God.

If you do doxx them, at least own that you doxxed them rather than denying it.

There is absolutely nothing wrong with using a pseudonym with a cumulative reputation, if you feel that is necessary to send your message. Say what you want about Jezos, he believes in something, and he owns it.

Beff Jezos Advocates Actions He Thinks Would Probably Kill Everyone

What are the things Jezos was saying anonymously? Does Jezos actively support things that he thinks are likely to cause all humans to die, with him outright saying he is fine with that?

Yes. In this case it does. But again, he believes that would be good, actually.

Emmet Shear: I got drinks with Beff once and he seemed like a smart, nice guy…he wanted to raise an elder machine god from the quantum foam, but i could tell it was only because he thought that would be best for everyone.

TeortaxesTex (distinct thread): >in the e/acc manifesto, when it was said “The overarching goal for humanity is to preserve the light of consciousness”… >The wellbeing of conscious entities has *no weight* in the morality of their worldview

I am rather confident Jezos would consider these statements accurate, and that this is where ‘This Is What Beff Jezos Actually Believes’ could be appropriately displayed on the screen.

I want to be clear: Surveys show that only a small minority (perhaps roughly 15%) of those willing to put the ‘e/acc’ label into their Twitter report endorsing this position. #NotAllEA...
Bucket Brigade: Likely End-of-Life
by jefftk — 2023-12-06T15:30:06.871Z

During the pandemic a group of us wrote

some software to allow people to
sing and make music together in real time over the internet, by
accounting for latency instead of minimizing it. The initial goal was
to support group singing at the

2020 Secular
Solstice, after which I reworked the UI to be a better fit for

singing parties. It doesn't get
much use now that it's safer to music together in person, but
geographically distributed groups still sing with it a bit.

I'd been hoping to host it indefinitely, and my usual approach is to
writing very low upkeep software is to avoid dependencies.
Unfortunately writing video chat software directly on top of browser
APIs is enough work that I took a dependency on Twilio's Programmable Video
product. Which they've now announced they're shutting down.

They've given a year for migration, with a final date of 2024-12-05,
but I'm not excited enough about this project to port it to a new
video API. If someone else is, let me know, otherwise Bucket Brigade
will shut down then as well.
Why Yudkowsky is wrong about "covalently bonded equivalents of biology"
by titotal — 2023-12-06T14:09:15.402Z
Link: https://open.substack.com/pub/titotal/p/why-yudkowsky-is-wrong-about-covalently?r=1e0is3&utm_campaign=post&utm_medium=web

confidence level: I am a physicist, not a biologist, so don’t take this the account of a domain level expert. But this is really basic stuff, and is very easy to verify.

Edit: I have added a few revisions and included a fact check of this post by an organic chemist. You can also read the comments on the EA forum to see Yudkowsky's response.

Recently I encountered a scientific claim about biology, made by Eliezer Yudkowsky. I searched around for the source of the claim, and found that he has been repeating versions of the claim for over a decade and a half, including in “the sequences” and his TED talk. In recent years, this claim has primarily been used as an argument for why an AGI attack would be extremely deadly. I believe this claim is factually incorrect.

The quotes:

I’m going to show the various versions of the claim I found below, with the relevant sentences bolded:

To plausibly argue that “humans” were intelligently designed, you’d have to lie about the design of the human retina, the architecture of the human brain, the proteins bound together by weak van der Waals forces instead of strong covalent bonds

-Yudkowsky discussing the flaws of evolutionary design, in “the sequences” blog post “dark side epistemology”.

It was obvious years before Nanosystems that molecular nanomachines would in fact be possible and have much higher power densities than biology. I could say, "Because proteins are held together by van der Waals forces that are much weaker than covalent bonds," to point to a reason how you could realize that after just reading Engines of Creation and before Nanosytems existed.

- Yudkowsky discussing AI interventions on the alignment forum.

A lot of the advantage of human technology is due to human technology figuring out how to use covalent bonds and metallic bonds, where biology sticks to ionic bonds and proteins held together by van der Waals forces (static cling, basically)

-Comment on a post discussing technology and AI.

Algae are tiny microns-wide solar-powered fully self-replicating factories that run on general assemblers, "ribosomes", that can replicate most other products of biology given digital instructions. This, even though the proteins are held together by van der Waals forces rather than covalent bonds, which is why algae are far less tough than diamond (as you ca

...
Metaculus Launches Chinese AI Chips Tournament, Supporting Institute for AI Policy and Strategy Research
by ChristianWilliams — 2023-12-06T11:26:15.790Z
Link: https://www.metaculus.com/tournament/chinese-ai-chips/?has_group=false&project=2784&order_by=-activity

Metaculus’s Chinese AI Chips Tournament is live: Forecast and support research by the Institute for AI Policy and Strategy (IAPS), a think tank that works to reduce risks from the development and deployment of frontier AI systems.

Better understanding China’s ability to produce its own AI chips over the coming years will help shed light on key issues for AI governance, such as:

How will Chinese AI systems compare to systems developed by frontrunner labs like OpenAI?
Will we see major, state-sanctioned AI chip smuggling operations?

Learn more about the tournament and start forecasting on these and other questions:

How much of Nvidia's quarterly 2024 revenue will come from the Chinese market?
Will any Chinese semiconductor foundry have 20% or more global semiconductor market share before 2033?
Will a Chinese firm market an EUV photolithography machine before 2033?

Read a primer on Chinese AI chips by IAPS researcher Erich Grunewald and independent researcher Chris Phenicie.
Minimal Viable Paradise: How do we get The Good Future(TM)?
by Nathan Young — 2023-12-06T09:24:09.699Z

I often think about who I want to be. I want to make the world better. I want to live with someone I love. I want a long term community of friends. Dreaming about this helps me figure out the steps to get there - if I want these things, I can understand how the world works, I can meet new people, I can run community events.

On a larger scale, I dream about the future of everything. Where should society end up? What is the picture we are aiming towards? Star trek-style luxury communism? Fluffy clouds and platonic harp-playing? A near-infinite amount of happy shrimp? What do we want?

In other words, I think about Heaven. What is The Good Future? And how do we get there?

The Good Future™ should be:

Really good. What’s good? Hard to say, but here goes: Something is really good if an individual, given time and space to reflect on their condition over time, would strongly endorse their life. You take a shrimp, give it a spa weekend and a brain the size of Morocco and ask “How is your life?” If it says “Great”, then I guess I’ll think that’s really good.

Universal. I think it's not The Good Future unless it's for everyone. I'm not saying it's equally good. But the way it's good should have some baseline for everyone. If your notion of good was emergency healthcare, then you need universal emergency healthcare. Perhaps some people have private healthcare too, but there is a baseline that every being gets.

I can imagine that The Good Future is briefly bad for some. Perhaps if it involves romance, people can get heartbroken in ways they don’t expect and wouldn’t have endorsed. This might be unavoidable. But I doubt it. I think that we should be cautious of saying “this badness is unavoidable”.

Stable. The Good Future has to not involve conflict over the future's future. If it does, then at the back of many minds will be the fear that Heaven is about to come to an end. At that point you want to turn off your heavenly hot tub and pick up your heavenly hand grenade. This does not sound like heaven[1]

In other words, small perturbations of heaven should leave it as heaven, not a rapid descent into the world we currently live in.

“They shall beat their swords into ploughshares” (Isiah 2:4) is meant to be a permanent thing.

Growing. My understanding of heaven involves growth. It involves becoming more, understanding more. Enlightenment too cheap to meter. I'm ...
Anthropical Paradoxes are Paradoxes of Probability Theory
by Ape in the coat — 2023-12-06T08:16:26.846Z

This is the fourth post in my series on Anthropics. The previous one is Anthropical probabilities are fully explained by difference in possible outcomes. The next one is Another Non-Anthropic Paradox: The Unsurprising Rareness of Rare Events.

Introduction

If there is nothing special about anthropics, if it’s just about correctly applying standard probability theory, why do we keep encountering anthropical paradoxes instead of general probability theory paradoxes? Part of the answer is that people tend to be worse at applying probability theory in some cases than in the others.

But most importantly, the whole premise is wrong. We do encounter paradoxes of probability theory all the time. We are just not paying enough attention to them, and occasionally attribute them to anthropics.

Updateless Dilemma and Psy-Kosh's non-anthropic problem

As an example, let’s investigate Updateless Dilemma, introduced by Eliezer Yudkowsky in 2009.

Let us start with a (non-quantum) logical coinflip - say, look at the heretofore-unknown-to-us-personally 256th binary digit of pi, where the choice of binary digit is itself intended not to be random.

If the result of this logical coinflip is 1 (aka "heads"), we'll create 18 of you in green rooms and 2 of you in red rooms, and if the result is "tails" (0), we'll create 2 of you in green rooms and 18 of you in red rooms.

After going to sleep at the start of the experiment, you wake up in a green room.

With what degree of credence do you believe - what is your posterior probability - that the logical coin came up "heads"?

Eliezer (2009) argues, that updating on the anthropic evidence and thus answering 90% in this situation leads to a dynamic inconsistency, thus anthropical updates should be illegal.

I inform you that, after I look at the unknown binary digit of pi, I will ask all the copies of you in green rooms whether to pay $1 to every version of you in a green room and steal $3 from every version of you in a red room. If they all reply "Yes", I will do so.

Suppose that you wake up in a green room. You reason, "With 90% probability, there are 18 of me in green rooms and 2 of me in red rooms; with 10% probability, there are 2 of me in green rooms and 18 of me in red rooms. Since I'm altruistic enough to at least care about my xerox-siblings, I calculate the expected utility of replying 'Yes' as (90% * ((18 * +$1) +

...
Digital humans vs merge with AI? Same or different?
by Nathan Helm-Burger — 2023-12-06T04:56:38.261Z

mishka

So, to create context, this is a continuation of our remarks in the comments to Zvi's "AI #40: A Vision from Vitalik"

mishka

There I was asking about

the boundary between humans merging with AI and digital humans (can these approaches be reliably differentiated from each other? or is there a large overlap?)
why digital humans would be a safer alternative than the merge

mishka

A good starting point might be to ask you to elaborate on the first item: what are the differences, is there an significant overlap, is one concept almost a subset of another.

How do you see the relationship between the notion of a digital human and a "hybrid" (if that's what the merge is)?

Nathan Helm-Burger

So the way I'm thinking about this is that digital human is a quite narrow and precise definition. An emulation of a human brain (not necessarily based on a particular human, that concept is even more specific, an Upload).

This brain emulation is constrained to act and to be modified only via a highly accurate ruleset based on detailed observations of human neurons. It cannot, for instance, use external processes to monitor or modulate the activity of its simulated neurons. It can't add more neurons other than by the same processes and to the same degree that a normal biological human brain can.

Nathan Helm-Burger

Merging with AI on the other hand, is a much more open concept. Lots of things fall under this heading. Some central examples might include:

A brain-computer implant that allows an AI system to have read and/or write access to the human brain.
An AI system that non-invasively controls much of the sensory input a set of humans receive, feeding them information and taking instructions from them in a high bandwidth way.
A digital human which has high bandwidth communication with an AI, either by normal emulated sensory methods or by the AI being able to directly read and/or write neuronal activity.

Nathan Helm-Burger

The key part here is the AI, that is merging with the humans. It is not necessarily dangerous or overwhelming. But it might be.

And the trouble is that the easiest way to scale the "merge"-type systems up in power is to expand the power and influence of the AI.

The digital human, by being restricted to a biological ruleset, is limited in the ways it can scale in power. It still could have significant superpowers relative to biological humans. Not being subject to aging or bi

...
EA Infrastructure Fund's Plan to Focus on Principles-First EA
by Linch — 2023-12-06T03:24:55.844Z

Edit 2024/01/02: People who like this new vision may want to consider donating to EAIF!

Now until the end of January is an unusually good time to donate, as it allows you to take advantage of 2:1 Open Phil matching ($2 from them for $1 from you)[1]. My best guess is that (unlike with LTFF), the EAIF match will not be filled by default without significant changes in more donors chipping in. We currently have ~1.5M of the 3.5M matching filled (see live dashboard here)
Summary

EA Infrastructure Fund (EAIF)[1] has historically had a somewhat scattershot focus within “EA meta.” This makes it difficult for us to know what to optimize for or for donors to evaluate our performance. More
We propose that we switch towards focusing our grantmaking on Principles-First EA.[2] More
This includes supporting:

research that aids prioritization across different cause areas
projects that build communities focused on impartial, scope sensitive and ambitious altruism
infrastructure, especially epistemic infrastructure, to support these aims

We hope that the tighter focus area will make it easier for donors and community members to evaluate the EA Infrastructure Fund, and decide for themselves whether EAIF is a good fit to donate to or otherwise support.
Our tentative plan is to collect feedback from the community, donors, and other stakeholders until the end of this year. Early 2024 will focus on refining our approach and helping ease transition for grantees. We'll begin piloting our new vision in Q2 2024. More

Note: The document was originally an internal memo written by Caleb Parikh, which Linch Zhang adapted into an EA Forum post. Below, we outline a tentative plan. We are interested in gathering feedback from community members, particularly donors and EAIF grantees, to see how excited they’d be about the new vision.

Introduction and background context

I (Caleb) [3]think the EA Infrastructure Fund needs a more coherent and transparent vision than it is currently operating under.

EA Funds’ EA Infrastructure Fund was started

about 7 years ago under CEA

. The EA Infrastructure Fund (formerly known as the EA Community Fund or EA Meta Fund) has given out 499 grants worth about 18.9 million dollars since the start of 2020. Throughout its various iterations, the fund has had a large impact on the community and I am proud of a number of the grants we’ve given out. However,...