trentmkelly/LessWrong-43k huggingface dataset:
In the very near future the internet will answer all of our questions and that makes me sad
by David Gross — 2022-03-24T03:55:31.462Z

Today if I type into Google’s search bar “how to move a couch into a moving plane using only a pair of avocados” the first result is not particularly on-point:

14 Tips for Moving Furniture | The Family Handyman

https://www.familyhandyman.com › list › 10-tips-for-m...

Mar 8, 2022 — Tip the item backward at an angle and have one person carry the top while the other carries the bottom. This centers the weight and keeps the ...

Missing: plane ‎pair ‎avocados

In some not very distant future, I expect the top result will be something like this:

How a pair of avocados was all I needed to move my couch into a moving plane

https://householdhints.gpteeble.com/ > helpful advice > how-tos

Mar 23, 202x — I was surprised at how easy it ended up being to move my couch into a moving plane by using nothing but a pair of avocados and some common ...

Today’s result for “what was that scientific paper about how you could cure pancreatic cancer by eating your own toenails?”

Starving Pancreatic Cancer of Cysteine May Kill Tumor Cells

https://www.cuimc.columbia.edu › news › starving-pan...

Apr 2, 2020 — The compound starves tumors of an amino acid, cysteine, which was found to be critical to the survival of pancreatic cancer cells.

Missing: own ‎toenails?

The not very distant future’s version:

Role of Unguiphagy in Remission of Pancreatic Cancer

https://oncologytoday.gpteeble.com/ > Journal of Oncological Unguiphagy > latest

Mar 23, 202x — The median life span of patients following diagnosis of advanced Carcinoma of pancreas is usually not more than 3-6 months. However in patients who eat their own toenails, survival rates several times longer ...

Today, the internet can give okay answers to many of our questions. In the near future it will give astonishingly plausible answers to all of our questions. I am concerned.
A survey of tool use and workflows in alignment research
by Logan Riggs — 2022-03-23T23:44:30.058Z

TL;DR: We are building language model powered tools to augment alignment researchers and accelerate alignment progress. We could use your feedback on what tools would be most useful. We’ve created a short survey that can be filled out here.

We are a team from the current iteration of the AI Safety camp and are planning to build a suite of tools to help AI Safety researchers.

We’re looking for feedback on what kinds of tools would be most helpful to you as an established or prospective alignment researcher. We’ve put together a short survey to get a better understanding of how researchers work on alignment. We plan to analyze the results and make them available to the community (appropriately anonymized). The survey is here. If you would also be interested in talking directly, please feel free to schedule a call here.

This project is similar in motivation to Ought’s

Elicit

, but more focused on human-in-the-loop and tailored for alignment research. One example of a tool we could create would be a language model that intelligently condenses existing alignment research into summaries or expands rough outlines into drafts of full Alignment Forum posts. Another idea we’ve considered is a brainstorming tool that can generate new examples/counterexamples, new arguments/counterarguments, or new directions to explore.

In the long run, we’re interested in creating seriously empowering tools that fall under categorizations like STEM AI, Microscope AI, superhuman personal assistant AI, or plainly Oracle AI. These early tools are oriented towards more proof-of-concept work, but still aim to be immediately helpful to alignment researchers. Our prior that this is a promising direction is informed in part by

our own

very fruitful and interesting experiences using language models as writing and brainstorming aids.

One central danger of tools with the ability to increase research productivity is dual-use for capabilities research. Consequently, we’re planning to ensure that these tools will be specifically tailored to the AI Safety community and not to other scientific fields. We do not intend to publish the specifics methods we use to create these tools.

We welcome any feedback, comments, or concerns about our direction. Also, if you'd like to contribute to the project, feel free to join us at the ...
Rational and irrational infinite integers
by Viliam — 2022-03-23T23:12:20.135Z

Epistemic status: a fine line between genius and madness

*

Let's start with the observation that it is much easier to use digits before the decimal point than the digits after it. A six years old child could calculate e.g. 11 + 2, but calculating reliably 0.11 + 0.2 will take a few more years of education.

From this perspective, it is quite ironic that we typically use numbers that only have a few digits before the decimal point, and many, sometimes infinitely many digits after the decimal point, such as 1/3 = 0.333333333... = 0.¯¯¯3.

Obviously, it would be better the other way round.

*

Technically, we already have infinitely many digits in front of every number. But they are all zeroes; and by convention, we do not write them. Writing a zero or multiple zeroes in front of a number does not change its value; the number 007 mathematically means 7. Therefore, technically, we could write the number 7 with infinitely many digits in front of it, like this: ¯¯¯07.

But it seems like a waste of space to only ever fill those infinitely many pre-decimal places with zeroes. What about using some other digit instead?

Consider the number ¯¯¯9, which consists of infinitely many nines. Written in full, it would be like this: ¯¯¯9 = ...999999999.

The value of this number is -1.

How can we know that? Simply, just calculate ...999999999 + 1 = ...000000000, which is zero. (Zero with an arbitrary number of zeroes in front of it is zero: 0 = 00 = 000 = ¯¯¯0.)

Similarly, ...999999998 = ¯¯¯98 is -2, because ¯¯¯98+2=¯¯¯0. Also ¯¯¯97=−3, ¯¯¯96=−4, and so on.

Seems like we have invented a new syntax for negative numbers. Instead of writing a minus symbol in front of a number, just let the digits underflow. If an odometer in your car showed only nines, you would know what happens if you go the extra mile.

*

What if we fill the infinite pre-decimal places with digits other than zeroes and nines? For example, how much is ¯¯¯1?

I believe this number is -1/9.

We can easily verify it by multiplying the number by 9 and adding 1.¯¯¯1×9+1=¯¯¯9+1=¯¯¯0.

Okay, this seems strange. How did a fraction smaller than one suddenly become an infinite integer? But as you can clearly see, it did. Remember, our goal was to move the infinite digits behind the decimal point to the front of it. And so far, we are succeeding!

*

We can similarly get other negative fracti...
Research on how pattern-finding contributes to memorization?
by DirectedEvolution — 2022-03-23T21:42:37.742Z

I notice that I have a much easier time memorizing things when I find a pattern. Often, I fail to think to even look for a pattern. As an example, I had a hard time memorizing the unit circle until I realized that the numerator is just counting up and down from sqrt(n) <-> -sqrt(n), for n = 4, 3, 2, 1, 0 1, 2, 3, 4, starting at different points for sin and cos, while the denominator is always 2. Before I saw that pattern, I was trying to memorize the values of cos and sin at particular degrees - the values in the individual parentheses, one by one. I completely missed the pattern when I looked at it this way. Noticing the pattern required actively looking for one, and also noticing that 1 = sqrt(1).

Likewise I've found tremendous utility in learning math by reducing complex equations to simpler underlying forms. For example, take this equation:

It's much more legible when you realize that it has the form:

A * (B cos(wt) + C sin(wt))/(B^2 + C^2).

But you have to think to look for that, and notice that 4d^2w^2 = (2dw)^2.

I'd love to know if there's any research into memory formation that is about the difference for patterned vs. unpatterned data, or the educational utility of having students deliberately look for patterns. "Patterns memorization" on Google Scholar isn't turning up much. Do you have any ideas?
Why Miracles Should Not Be Used as a Reason to Believe in a Religion
by Yitz — 2022-03-23T20:53:58.155Z

Note: This is part of a series of shorter essays I plan to write exploring my thoughts and opinions on various topics relating to philosophy, religion, and science. I will try to keep things simple. Feedback is welcome.

Human testimony is not sufficient grounds for the belief in a supernatural event.

Why?

Because humans are not perfect.

Just because a large number of people believe in something does not mean that the event actually occurred, or that it was caused by a deity. The value of a witness to a miracle really boils down to a question of trust. If you trust the witness, you are more likely to trust his report of the miracle.

The issue here is that there are a number of reasons why we might not trust the accuracy of the report.

First, humans are fallible.

One such reason is that memories are often edited or changed over time, often unconsciously. This can be due to a number of factors, such as the desire to remember events in a certain way, the influence of other people’s memories of the same event, or the introduction of false information after the fact. In fact, research has shown that it is very common for people to have different memories of the same event, and that even eyewitness testimony is often inaccurate.

Another reason why people’s memories can be inaccurate is because of something called “confabulation”. This is essentially when a person fills in gaps in their memory with information that they believe to be true, even if it is not. This can be due to a variety of factors, such as the person not being able to remember the event clearly, or wanting to make themselves appear more interesting or heroic.

Third, we are often subject to confirmation bias—that is, we are more likely to overlook evidence that contradicts our beliefs, or to see evidence that supports it when it is not there.
The more we believe a thing to be true, the more likely we are to think we are seeing it, whether we are or not.

Finally, just as we are susceptible to confirmation bias, we are also susceptible to social pressure.
The more we believe a thing to be true, the more likely we are to downplay inconsistencies in reports from others, or to attempt to rationalize them away.
This is particularly problematic when the people we are trusting are people we have a strong positive relationship with, as this makes us even more likely to be less skeptical of their claims.

When we put all of ...
Flywheels of progress
by jasoncrawford — 2022-03-23T20:42:14.773Z
Link: https://rootsofprogress.org/flywheels-of-progress

What causes progress?

I’ve been investigating this for five years, and I still don’t have a full answer. But part of the picture is starting to come into focus. Here’s my current, incomplete model:

Progress compounds. It builds on itself. Progress begets progress. This is why progress is super-linear: exponential, or indeed, over long periods, even super-exponential.

The form this takes is a number of feedback loops, or self-reinforcing cycles. By the nature of such loops, they act as if they had inertia: they are hard to get started, but hard to stop once going. Hence, a flywheel: the perfect metaphor for a loop or cycle with a lot of inertia.

There are several of these, at multiple levels, overlapping, and all operating simultaneously. Here are some that I can see:

Technology. Some technologies are fundamental, enabling many other technologies. Precision machining allows for the invention of many more types of machines. New engines and energy sources do the same. Information technology speeds up the dissemination of ideas and makes it easier for people to collaborate. Many technologies enable themselves: we use machine tools to make machines, we burn oil to drill for and to transport oil, we design computers on computers.

Wealth. Some level of surplus wealth is needed to fund research and development. When half the workforce had to be farmers just to feed the other half, the surplus simply wasn’t there. In the Renaissance, science was funded by wealthy patrons. As surplus builds up, we have more to invest in experimentation, invention, and new businesses; and progress in these areas raises our productivity, which gives us more surplus.

Science. Science enables advanced technology: electromagnetism enabled both electrical power and electronic communications; applied chemistry created everything from plastic to the the Haber-Bosch process; microbiology gave us sanitation, vaccines, and antibiotics. And technological and economic progress then in turn enable scientific progress, both by creating surplus wealth to fund it (as per the previous point) and by creating new scientific instruments and techniques, from the microscope and the thermometer to the LHC, LIGO, and JWST.

Markets. Transportation and communication technology have globalized markets that used to be narrowly local. Larger markets support more goods and more kinds of goods. Products that require special...
[Closed] Job Offering: Help Communicate Infrabayesianism
by abramdemski — 2022-03-23T18:35:16.790Z

Infrabayesianism seems to me (Abram) like a very promising framework for addressing at least some of the problems of AI alignment.

Like logical induction, it solves the realizability problem, creating an epistemic theory suitable for embedded agents.
Unlike logical induction, and unlike standard Bayesian decision theory, it presents a theory of epistemics directly relevant to proving decision-theoretic results (in particular, useful learning-theoretic guarantees). Logical induction and standard Bayesian decision theories both can produce meaningful loss-bounding guarantees with respect to predictive error, but bounding decision error appears challenging for these approaches. Infrabayes provides a systematic way to get around this problem. Since decision error is much more meaningful for bounding risk, this seems highly relevant to AI safety.
Being a new perspective on very basic issues, Infrabayesianism (or perhaps successors to the theory) may turn out to shed light on a number of other important questions.

(For more information on InfraBayes, see the infrabayesianism sequence.)

However, I believe infrabayesianism appears to have a communication problem. I've chatted with several people who have strongly "bounced off" the existing write-ups. (I'm tempted to conclude this is a near-universal experience.)

There was even a post asking whether a big progress write-up -- applying InfraBayes to naturalized induction -- had simply fallen through the cracks.

Personally, even though I've carefully worked through the first three posts and re-visited my notes to study them more than once, I still am not fluent enough to confidently apply the concepts in my own work when they seem relevant.

I would like to change this situation if possible. It's not obvious to me what the best solution is, but it seems to me like it could be possible to find someone who can help.

Properties which would make an applicant interesting:

Must be capable of fully understanding the mathematics.

See the sequence to get an idea of what kind of mathematics is involved; mainly topology, functional analysis, measure theory and convex analysis. Background in reinforcement learning theory is a bonus.

Must have good judgement when it comes to math exposition.
Must be a good editor.

Details of the job description are to be worked out, but probable acti...
Wanted: Executive Assistant to help build the progress movement
by jasoncrawford — 2022-03-23T18:19:51.594Z
Link: https://rootsofprogress.org/wanted-executive-assistant

I’m hiring an Executive Assistant to be part of the founding team at The Roots of Progress, and to work closely with me on all of our projects to build the progress movement.

This role is perfect for someone who wants to combine dedication to an ambitious, high-impact, long-term mission with a day-to-day focus on operations, organization, and getting things done.

You should have strong attention to detail, crisp communication, swift and efficient execution, and meticulous followup. You’ll apply your intelligence to a broad range of tasks and projects, learning as you go where needed. Prior experience is helpful but not required, and candidates of any background are encouraged to apply. The ideal candidate will be familiar with my work and will be excited about strengthening the progress community.

You’ll act as a force multiplier on my time, allowing me to delegate everything that doesn’t need me to do it so that I can focus as much as possible on research, writing, and speaking. Your responsibilities will thus span a broad range—for example, helping with:

Managing a database of everyone I meet and talk to, and helping me keep in touch
Scheduling talks and interviews; planning trips
Community-building, including online forums and in-person events
Fundraising, grant-seeking, and donor relations
Media management, from helping with the @rootsofprogress Twitter account to getting coverage and interviews in blogs, podcasts, and media
Project management for other organizational goals, such as launching new online resources or other programs
Generally helping me stick to a schedule and not drop tasks

This is a full-time role. You can do it from anywhere, but preference will be given to candidates closer to US time zones. Compensation will vary depending on your seniority, qualifications, and location, but will be competitive with market rates. To apply, send me a resume (link to an online one is fine): jason@rootsofprogress.org.

This is a chance to help establish a new philosophy of progress for the twenty-first century. I look forward to working with you!
Circular Counterfactuals "Only that which Happens is Possible"
by SebastianG  — 2022-03-23T14:40:36.342Z

“Only that which happens is possible.” -The Megarian School

You make a prediction, it doesn’t happen. Could it have?

Let’s ignore the cognitive bias literature here, and Tetlock’s wonderful data about how many forecasters claim that they were very, very close to being correct, if it weren’t for that one little thing that got in their way. Instead, let’s focus on the logic and metaphysics. Could it have happened? Immediately we are thrust into the world of counterfactual reasoning. What is the status of the counterfactual? Do they exist as an ontological primitive or are they something we discover/invent as a heuristic for thinking? The ramifications are pretty far reaching; for counterfactuals are a central tool for the way we think about causality. In other words, because many scientific fields employ counterfactual reasoning and search for the causes of phenomena, an account of counterfactuals which supplies a firm foundation should allow us to build clearer and cleaner models for interpreting the world, and in the case of AI, models which interpret the world.

While Tetlock’s is interested in what habits of thought are required to be skilled at making true counterfactual statements. And others on LW are interested in formalisms and engineering those formalisms to see what comes out. In this paper, I am interested in what metaphysical presuppositions are possible. The purpose here is to see how counterfactuals fit into epistemology, and how different metaphysical assumptions change our understanding of what counterfactuals are. (By metaphysical assumptions, I mean assumptions about the nature of causality and our capacity to identify causal connections).

I started down investigations of this sort in 2017, because I was worried about the ontological status of two common heuristic tools in behavioral science and LW: the distinction between the inside/outside view and the formation of priors. The distinction between the inside/outside view seemed to me subjective, and priors were (at the time) assumed to be almost arbitrary, as in it did not matter what priors one started out with so long as they were subject to updating new evidence. This also seemed wrong to me. I was interested in where good priors come from, the developmental process that leads to “priors hungry for updating”. In 2019 I worked on an Adversarial Collaboration which never came t...
[Intro to brain-like-AGI safety] 9. Takeaways from neuro 2/2: On AGI motivation
by Steven Byrnes — 2022-03-23T12:48:18.476Z

(Last revised: July 2024. See changelog at the bottom.)

9.1 Post summary / Table of contents

Part of the

“Intro to brain-like-AGI safety” post series

.

Most posts in the series thus far—Posts

#2

–

#7

—have been primarily about neuroscience. Then, starting with the previous post, we’ve been applying those ideas to better understand brain-like-AGI safety (as defined in Post #1).

In this post, I’ll discuss some topics related to the motivations and goals of a brain-like AGI. Motivation is of paramount importance for AGI safety. After all, our prospects are a heck of a lot better if future AGIs are motivated to bring about a wonderful future rich in human flourishing, compared to if they’re motivated to kill everyone. To get the former and not the latter, we need to understand how brain-like-AGI motivation works, and in particular how to point it in one direction rather than another. This post will cover assorted topics in that area.

Table of contents:

Section 9.2 argues that the goals and preferences of a brain-like AGI are defined in terms of latent variables in its world-model. These can be related to outcomes, actions, or plans, but are not exactly any of those things. Also, the algorithms generally don’t distinguish between instrumental and final goals.
Section 9.3 has a deeper discussion of “credit assignment”, which I previously introduced with an example back in

Post #7 (Section 7.4)

. “Credit assignment”, as I use the term in this series, is a synonym of “updates to the

Thought Assessors

”, and is the process whereby a concept (= latent variable in the world-model) can get “painted” with positive or negative valence, and/or start triggering involuntary visceral reactions (in the biology case). This type of “credit assignment” is a key ingredient in how an AGI could wind up wanting to do something.
Section 9.4 defines

“wireheading”

. An example of “wireheading” would be if an AGI hacks into itself and sets the “reward” register in RAM to the maximum possible value. I will argue that brain-like AGI will “by default” have a “weak wireheading drive” (desire to wirehead, other things equal), but probably not a “strong wireheading drive” (viewing wireheading as the best possible thing to do, and worth doing at any cost).
Section 9.5 spells out an implication of the wireheading discussion above: brain-like AGI is generally NOT trying to maxi

...