trentmkelly/LessWrong-43k huggingface dataset:
Designing Artificial Wisdom: GitWise and AlphaWise
by Jordan Arel — 2024-07-13T06:46:04.807Z

Introduction

In this post I will describe two possible designs for Artificial Wisdom (AW.) This post can easily be read as a stand-alone piece, however it is also part of a series on artificial wisdom. In essence:

Artificial Wisdom refers to artificial intelligence systems which substantially increase wisdom in the world. Wisdom may be defined as "thinking/planning which is good at avoiding large-scale errors," or as “having good terminal goals and sub-goals.” By “strapping” wisdom to AI via AW as AI takes off, we may be able to generate enormous quantities of wisdom which could help us navigate Transformative AI and The Most Important Century wisely.

TL;DR

Artificially wise coaches that improve human wisdom seem like another promising path to AW. Such coaches could have negligible costs, be scalable & personalized, and soon perform at a superhuman level. Certain critical humans receiving wise coaching could be decisive in humans navigating transformative AI wisely.

One path to AW coaches is by creating a decentralized system like a wiki or GitHub for wisdom-enhancing use-cases. Users could build up a database of instructions for LLM’s to act as AW coaches to help users make difficult decisions, navigate difficult life and epistemic dilemmas, work through values conflicts, achieve career goals, improve relational/mental/physical/emotional well-being, and increase fulfillment/happiness.

One especially wise use-case could be a premortem/postmortem bot that helps people, organizations, and governments to avoid large-scale errors.

Another path to creating an AW coach is to build a new system trained on biographical data, which analyses and learns to predict which decision-making processes and strategies of humans with various traits in various environments are most effective for achieving certain goals.

Artificial Wisdom Coaches

The are several possible paths for developing AW coaches. After introducing the basic idea, I will briefly outline two of them.

The essential idea is that AW can learn from the best human coaches, therapists, teachers, or other wise data and training mechanisms, but can study orders of magnitude more data, which it never forgets, plus be fine-tuned on a large amount of human outcome data directly tied to the results of its coaching.

Artificially wise coaches will eventually perform much better than human coaches, ...
OpenAI's Intelligence Levels
by infinibot27 — 2024-07-13T06:25:12.068Z
Link: https://www.bloomberg.com/news/articles/2024-07-11/openai-sets-levels-to-track-progress-toward-superintelligent-ai

OpenAI announced a levels to intelligence approach they have to classifying their AI systems. In short, the Levels range from 1 to 5, starting at chatbot, then reasoned (GPT-4 reportedly falls somewhere between the first two levels), then notably, agent, innovator, then AIs capable of doing the work of organizations. Admittedly it's not an explicit road map, but that OpenAI views agency as a potential next step as opposed to remaining in a more tool AI orientated space seems concerning.
Some desirable properties of automated wisdom
by Marius Adrian Nicoară — 2024-07-13T06:05:34.386Z

Epistemic status: Musings about questioning assumptions and purpose. Not clear if I correctly extract wisdom from fiction. Possibly other fallacies present - bringing these ideas to the light of commentary will improve or dispel them

I distinguish a wise algorithm from an intelligent algorithm by the wise algorithm’s additional ability to pursue goals that would not lead to the algorithm’s instability. I consider the following desirable properties of a wise algorithm:

Questioning assumptions, which could be aided by:

Possessing multiple representations and modes of thinking to avoid getting stuck
Integrating new information on the fly

Having a stable sense of purpose that does not interfere with it’s stability, which I refer to as a good sense of purpose

Questioning assumptions

Paradoxes are dissolved by questioning assumptions. For example, let’s consider how solving the black body radiation paradox gave birth to quantum physics.

Initial assumption: energy is shared equally by electrons vibrating at different frequencies -> implies an ultraviolet catastrophe, where heated objects emit infinite energy(imagine an oven turning into supernova)

Revised assumption: in order to vibrate at a certain frequency, an electron must have a quantum of energy proportional to that frequency -> predictions in agreement with experimental data, which don’t show an ultraviolet catastrophe

Now for the details.

According to classical physics, light is an electromagnetic wave that is produced by the vibration of an electron. Heat is the kinetic energy of random motion, so in a hot object the electrons vibrate in random directions, generating light as a result. Thus a hotter object emits more light.

Classical mechanics assumes that there is no limit to the range of frequencies that the electrons in a body can vibrate. This implies that an object can emit infinite energy - the so-called ultraviolet catastrophe. However, experiments showed that the radiation of a blackbody(object that does not reflect any light) becomes small at short wavelengths(high frequencies) - the left-hand side of the graph.

Source:

https://physics.weber.edu/carroll/honors/failures.htm

Max Planck questioned the assumption that each frequency must have the same energy. Instead, he proposed that energy is not shared equally by electrons vibrating at different frequencies. Pl...
Thought Experiments Website
by minmi_drover — 2024-07-13T04:47:05.507Z

Hullo,

I'm hoping for your feedback. I recently finished a sort of beta of a website that allows users to explore and vote on thought experiments:

https://thought-experiment-explorer.vercel.app/

The spirit behind it is somewhat similar to LessWrong's: I think people should examine their lives and reasoning more. I also love gamification and broadening the appeal of academic things.

It has a small batch of some very famous ones, but as I improve the site, I'll add a lot more, as well a page that lets you search and filter by type (such as "utilitarian" or "non-Western"). The About page notes some other plans.

I hope you like the idea and general design so far, but I'd love to hear critiques and suggestions.

Thanks much!

Kat
A Second Wetsuit Summer
by jefftk — 2024-07-13T02:00:05.412Z

I often write about things when they're new to me, because that's when
I'm most interested in them. Sometimes how I feel after a short time
ends up being how I feel long-term; other times I end up with an
enthusiastic post about something I ended up abandoning a few months
later. So when I remember, it seems valuable to revisit with the
benefit of additional experience.

As long as I can remember I've gotten chilled quite rapidly when
swimming. It can be summer in Florida, and after 20min I'm needing to
get out and warm up. For years I just accepted this, but once I had
kids it became pretty frustrating that I couldn't hang out playing
with them in the water anywhere near as long as they wanted me to.

Last summer I got a wetsuit and wrote about it
after the first time I tried it. How do I feel now that it's been
most of a year?

I love it! We're just finishing up a week in NC visiting Julia's
family, and we've been swimming most days. It's very hot, hot enough
that most people can stay in the water indefinitely, but not me. Or,
not me before the wetsuit! I didn't once need to get out of the pool
or lake due to feeling cold, and multiple times spent more than an
hour in the water. I've really enjoyed being able to play with my
kids, help them practice swimming, or just bob around with the
toddler.

The main chance since my first post is that I've stopped wearing the
vest, and now just wear the pants. It turns out that the pants (1.5mm
neoprene) let me retain enough heat, and the vest was much more
unpleasant when hanging around not in the water.

Here's my swim attire, with trunks over the pants and a fast-dry shirt
on top:

I generally prefer more covering gear so I don't have to deal with
sunscreen, so that the wetsuit involves long pants is actually a minor
plus for me.

Anyway: I'm very happy! Strongly recommended for anyone else who
similarly gets cold easily.
Timaeus is hiring!
by Jesse Hoogland — 2024-07-12T23:42:28.651Z

TLDR: We’re hiring two research assistants to work on advancing

developmental interpretability

and other applications of singular learning theory to alignment.

About Us

Timaeus’s mission is to empower humanity by making breakthrough scientific progress on alignment. Our research focuses on applications of singular learning theory to foundational problems within alignment, such as interpretability (via “developmental interpretability”), out-of-distribution generalization (via “structural generalization”), and inductive biases (via “geometry of program synthesis”). Our team spans Melbourne, the Bay Area, London, and Amsterdam, collaborating remotely to tackle some of the most pressing challenges in AI safety.

For more information on our research and the position, see our

Manifund application

,

this update from a few months ago

, our

previous hiring call

and this

advice for applicants

.

Position Details

Title: Research Assistant
Location: Remote
Duration: 6-month contract with potential for extension.
Compensation: Starting at $35 USD per hour as a contractor (no benefits).
Start Date: Starting as soon as possible.

Key Responsibilities

Conduct experiments using PyTorch/JAX on language models ranging from small toy systems to billion-parameter models.
Collaborate closely with a team of 2-4 researchers.
Document and present research findings.
Contribute to research papers, reports, and presentations.
Maintain detailed research logs.
Assist with the development and maintenance of codebases and repositories.

Projects

As a research assistant, you would likely work on one of the following two projects/research directions (this is subject to change):

Devinterp of language models: (1) Continue scaling up techniques like local learning coefficient (LLC) estimation to larger models to study the development of LLMs in the 1-10B range. (2) Work on validating the next generations of SLT-derived techniques such as restricted LLC estimation and certain kinds of weight- and data-correlational analysis. This builds towards a suite of SLT-derived tools for automatically identifying and analyzing structure in neural networks.
SLT of safety fine-tuning: Investigate the use of restricted LLCs as a tool for measuring (1) reversibility of safety fine-tuning and (2) susceptibility to jailbreaking. Having now validated many of our predictions around SLT, we are

...
Consider attending the AI Security Forum '24, a 1-day pre-DEFCON event

by Charlie Rogers-Smith — 2024-07-12T23:01:46.370Z

Jeffrey Ladish, Caleb Parikh, and I are running the AI security forum, a 1-day event on Thursday, August 8th '24 in Las Vegas, the day before DEFCON. This is a continuation of last year's X-infosec Forum.

We thought that many of you reading this might be interested in attending. If you're interested, please apply here by July 20th. If you can’t attend but have recommendations for other people to invite, please let us know here.

We think that there’s a significant chance transformative AI will be developed within the next 10 years, and that securing those systems is on the critical path to preventing a global catastrophe. It’s not clear that we’re on track to do that in time, so we are convening researchers, engineers, and policymakers with the goal of significantly accelerating AI security. We're aiming to do that by establishing common knowledge about the state of AI security and AI progress, mobilising talent to fill the most pressing gaps, and fostering collaborations.

We're inviting a similar group of people to last year, which had (roughly):

~30-40% from AI labs and research orgs (e.g., Anthropic, OpenAI, Google DeepMind)
~20-25% from academic institutions (e.g., Harvard, MIT, Stanford)
~15-20% from government agencies and think tanks (e.g., CISA, RAND)
~15% from other tech companies (e.g., Google, Microsoft, Intel)
~10-15% independent researchers, consultants, and representatives from nonprofits and funding organizations.

Please see our website for more information. Thanks!
Memorising molecular structures
by dkl9 — 2024-07-12T22:40:42.307Z
Link: https://dkl9.net/essays/molecule_memory.html

Followup to: A tedious and effective way to learn 汉字 (Chinese characters)

Organic molecules are like 汉字 (Chinese characters).

Organic molecules vary widely in complexity, from methane (CH4) to phthalic acid (C8H6O4, benzene-1,2-dicarboxylic acid) to N,N-dimethyltryptamine (C12H16N2) and beyond. 汉字 vary widely in complexity, from 人 ("person") to 百 ("hundred") to 游 ("to roam") and beyond. Organic molecules consist of functional groups, which are each defined in terms of a few atoms. 汉字 consist of radicals, which are each defined in terms of a few strokes. The parts of an organic molecule hint at its origin, uses, and properties. The parts of a 汉字 hint at its meaning and pronunciation.

You can learn 汉字 very effectively by breaking them down into a tree of components, inferring the full character with working memory. Sith organic molecules are like 汉字, you can work analogously to memorise molecular structures.

Reading Wikipedia in Lynx (a text-only web browser) only gives me the names of molecules, until I explicitly download images. Most molecules (except the most trivial) have systematic names which describe their structure in terms of smaller segments. This breakdown proceeds recursively, until it bottoms out in common functional groups (methyl, benzene, aldehyde, etc) or base molecules whose structure you can deduce from hints in text.

For example, let's memorise the structure of metformin. Metformin is N,N-dimethylbiguanide hydrochloride. The "N,N-dimethyl" and "hydrochloride" parts are simple: two terminal CH3's at the same nitrogen of biguanide, and HCl mixes with the main compound to form a salt. Biguanide is C2H7N5, which has a text-based description as HN(C(NH)NH2)2. Nitrogen usually takes three bonds; biguanide is centred around nitrogen, bonded to hydrogen and two copies of C(NH)NH2. Chemical intuition can lead us to conclude that those carbons each have two amine groups, one double-bonded. We can add a methyl group to nitrogen only when it has a hydrogen bonded to it to replace. The methyl groups of metformin are at the same site, and only two nitrogens of biguanide (one at each end, equivalent to each other) have the requisite two hydrogens.

So, a general algorithm to memorise molecular structures:

Use a browser which shows only text, or blocks image-loading.
Look up the molecule on Wikipedia or a chemical database.
If the molecule is simple

...
Robin Hanson AI X-Risk Debate — Highlights and Analysis
by Liron — 2024-07-12T21:31:02.222Z
Link: https://www.youtube.com/watch?v=i0DxeBhjzlY

This linkpost contains a lightly-edited transcript of highlights of my recent AI x-risk debate with Robin Hanson, and a written version of what I said in the post-debate analysis episode of my Doom Debates podcast.

Introduction

I've poured over my recent 2-hour AI x-risk debate with Robin Hanson to clip the highlights and write up a post-debate analysis, including new arguments I thought of after the debate was over.

I've read everybody's feedback on YouTube and Twitter, and the consensus seems to be that it was a good debate. There were many topics brought up that were kind of deep cuts into stuff that Robin says.

On the critical side, people were saying that it came off more like an interview than a debate. I asked Robin a lot of questions about how he sees the world and I didn't "nail" him. And people were saying I wasn't quite as tough and forceful as I am on other guests. That's good feedback; I think it could have been maybe a little bit less of a interview, maybe a bit more about my own position, which is also something that Robin pointed out at the end.

There's a reason why the Robin Hanson debate felt more like an interview. Let me explain:

Most people I debate have to do a lot of thinking on the spot because their position just isn't grounded in that many connected beliefs. They have like a few beliefs. They haven't thought that much about it. When I raise a question, they have to think about the answer for the first time.

And usually their answer is weak. So what often happens, my usual MO, is I come in like Kirby. You know, the Nintendo character where I first have to suck up the other person's position, and pass their Ideological Turing test. (Speaking of which, I actually did an elaborate Robin Hanson Ideological Turing Test exercise beforehand, but it wasn't quite enough to fully anticipate the real Robin's answers.)

With a normal guest, it doesn't take me that long because their position is pretty compact; I can kind of make it up the same way that they can. With Robin Hanson, I come in as Kirby. He comes in as a pufferfish. So his position is actually quite complex, connected to a lot of different supporting beliefs. And I asked him about one thing and he's like, ah, well, look at this study. He's got like a whole reinforced lattice of all these different claims and beliefs. I just wanted to make sure that I saw what it is that I'm arguing against.

I...
Designing Artificial Wisdom: The Wise Workflow Research Organization
by Jordan Arel — 2024-07-12T19:18:09.947Z

This essay was a winner of the “Essay competition on the Automation of Wisdom and Philosophy.”

Introduction

In this post I will describe one possible design for Artificial Wisdom (AW.) This post can easily be read as a stand-alone piece, however it is also part of a series on artificial wisdom. In essence:

Artificial Wisdom refers to artificial intelligence systems which substantially increase wisdom in the world. Wisdom may be defined as "thinking/planning which is good at avoiding large-scale errors," or as “having good terminal goals and sub-goals.” By “strapping” wisdom to AI via AW as AI takes off, we may be able to generate enormous quantities of wisdom which could help us navigate Transformative AI and The Most Important Century wisely.

TL;DR

Even simple workflows can greatly enhance the performance of LLM’s, so artificially wise workflows seem like a promising candidate for greatly increasing AW.

This piece outlines the idea of introducing workflows into a research organization which works on various topics related to AI Safety, existential risk & existential security, longtermism, and artificial wisdom. Such an organization could make progressing the field of artificial wisdom one of their primary goals, and as workflows become more powerful they could automate an increasing fraction of work within the organization.

Essentially, the research organization, whose goal is to increase human wisdom around existential risk, acts as scaffolding on which to bootstrap artificial wisdom.

Such a system would be unusually interpretable since all reasoning is done in natural language except that of the base model. When the organization develops improved ideas about existential security factors and projects to achieve these factors, they could themselves incubate these projects, or pass them on to incubators to make sure the wisdom does not go to waste.

Artificial Wisdom Workflows

One highly promising possibility for designing AW in the near future is using workflows. As noted by Andrew Ng, even simple workflows can have surprisingly powerful effects, such as enabling GPT-3.5 to far outperform GPT-4 in coding tasks, and advanced workflows may be able to boost AW even more.

Workflows can be created which mimic or augment various human thought processes, and then do so in an automated, repeatable way which make certain ...
Whiteboard Pen Magazines are Useful
by Johannes C. Mayer — 2024-07-12T17:15:33.200Z

Glue your colored whiteboard makers together with duck tape to create a whiteboard pen magazine. If required glue a paper layer around the duck tape to make it non-sticky to the touch. Mark which side is up with an arrow, such that you can always <hold it/pick it up> in the same way.

This more than doubled my color-switching speed, fits in my pocket, and takes <5m to make.

It lasts: When a pen is empty just insert a new one. No need to create another magazine. No signs of degradation after multiple hundred unholstering-holstering cycles in total.

FUUU 754 Standard: I am using RFC2119. The magazine MUST be vertically stacked (like shown in the pictures). The magazine SCHOULD have a visual mark, that is clearly visible when the magazine lays on a flat surface, to indicate the top side of the magazine (like shown in the first picture). There MUST NOT be two or more pens of the same color. It MUST have at least the following colored pens: Black, Red, Green, Blue. Imagine you hold the magazine, top side up, in your left hand in front (like in the below picture). Now from top to bottom the first 4 color pens MUST be: Black, Red, Green, Blue, and the remaining color pens SCHOULD be ordered after hue in HSL and HSV. This ensures that you can index into the magazine blindfolded. For example:

In this case I have orange and purple as additional color pens. The magazine SCHOULD not exceed 13cm (~5") in length along the stacking dimension, such that it still fits in a large pocket.

Note how any magazine with at least four pen caps can be converted to conform to FUUU 754, as the colors of the caps is irrelevant to this standard.

FUUU 754 extensions M and S:

Additionally the highly recommended M extension requires the magazine to have enough magets attached to the backside such that the magazine can be securely attached upright (pen tips facing down):

It is strongly reccomended to store the magazine upright such that ink flows to the tip, ensuring the pens are ready to write immediately, even when the pen is almost emtpy.

The S extension requires that a single small whiteboard marker is attached to the side. This makes it easy to draw small lines, e.g. for annotations:

Pigment liners: Do it with them too:

These are the best pens for writing on paper I know of. I use 0.3 for writing text. Easily fits in a pocket.