trentmkelly/LessWrong-43k huggingface dataset:
Transforming myopic optimization to ordinary optimization - Do we want to seek convergence for myopic optimization problems?
by tailcalled — 2021-12-11T20:38:46.604Z

While reading some papers, I found a way to take a myopic optimization problem L, and construct a non-myopic optimization problem L∗ whose optima match the fixed points of the original problem L. This seems potentially of interest for myopia research; it could provide a way to think about what optimization of L might give you, a tool for analyzing the trajectories of myopic optimization, and maybe also a better-behaved alternative to myopic optimization.

A common form for myopic optimization, TLDR

In order to explain the transformation, I first need to introduce a common form to myopic optimization problems. The next section goes into detail with applying the common form, but I suspect many people here are sufficiently comfortable with myopic optimization problems that they might be able to do with the shorter explanation.

In ordinary optimization, we might minimize a loss function L(m). That is, the optimized model mn would be given by the equation mn=argminmL(m).

This is non-myopic; every influence of m on the result of L gets optimized. To introduce a myopic equivalent, we need to distinguish between two influences of m; the parts that we want to optimize, and the parts that we want to be myopic about.

We can do that by introducing two separate parameters, L(x,y), with the first parameter being the parameter we optimize, and the second parameter being the parameter we are myopic about. Given a starting model mn, we can then myopically update this model, yielding mn+1=argminmL(m,mn).

This process can be iterated, but it has many poor properties, e.g. it is not guaranteed to converge, and it is unclear what its results will be. This post is about providing an alternative to that process.

A common form for myopic optimization, long version

Let's consider an example of myopic optimization.

Suppose you have some predictive model m, which gets trained according to some loss function L(m,x), requiring a distribution of data points x∼D. Essentially, it's trained to minimize L(m)=Ex∼D[L(m,x)].

If you deploy this model in the real world, then you would hope that it has some sort of real-world effect (if nothing else, then bringing you more ad money); otherwise, what's the point? But if it has a real-world effect, then in particular it might also have a real-world effect on the distribut...
What on Earth is a Series I savings bond?
by rossry — 2021-12-11T12:18:00.392Z

crossposted from: https://blog.rossry.net/series-i/

(Not investment advice, of course.)

Summary: The Series I savings bond is a US government bond offered to US citizens, with purchases limited to $10k per person per year. It pays interest set by a formula based on the official inflation rate, with a built-in lag. If inflation from November '21 to March '22 follows historical patterns, bonds purchased in December '21 and redeemed after 15 months will pay ~4.62% interest annualized. If inflation is higher (as it has been recently), the bonds will pay more; if it's lower, they will pay at least 3.26% interest when redeemed after 12 months.

All of those potential rates are a percentage points higher than any other bond that is even remotely as safe; this is because of the way the inflation adjustment rule works. Specifically, the inflation adjustment for the next six months is set based on what inflation was in the last six months. As a consequence, a Series I purchased between now and April 30 will pay its first 5-6 months of interest at 7.12% annualized (so 3.56% in 6 months), and then reset to some other rate that will depend on future inflation. If you don't want to stay invested after that, it's possible to redeem the bonds after 12 months.

If getting ~3-6% annualized interest on $20k of a US government bond is a thing that you want, then this might be the best way to do it.

(1) What is it?

The Series I savings bond is a US government bond that is offered directly to US citizens, with purchases limited to $10k per person per year. (Apparently, there's a way to purchase an additional $5k using your tax refund, but it's difficult and I'm going to ignore that.) You can't sell or transfer them to anyone else, though any time after 12 months you can redeem the bond to get your money back, with interest.

It's a US Treasury bond, so it's almost literally the safest investment in existence (in terms of "will I get paid back with actual US dollars?"). Despite being an extremely safe investment, it pays a few percent a year. This is relatively unusual for very safe investments (citation required).

Why does the bond pay so much more interest than other government bonds? The Series I is specifically designed as an "inflation-protected" investment, which means that instead of paying a fixed interest rate, it pays a fixed "base" interest rate plus an extra interest rate that'...
D&D.Sci GURPS Dec 2021: Hunters of Monsters
by J Bostock — 2021-12-11T12:13:02.574Z

Intro (skippable)

"Are you really the smartest member of the Hunters' Guild?"

"I'm the smartest at fighting! What's the difference?"

"Well, you're just about smart enough to write, at the very least"

"And you're just about short enough, and just about annoying enough, that if you don't shut your nerd mouth you'll find yourself flying out of that window"

The master hunter shoves a heavy leather-bound journal towards you. You look inside, and see just-about-legible scrawlings:

'Dear Diary, this week I took a big fiery flamu club to the thunderwood peaks. Even though I was wearing the nicest icemail in the armory, I got beaten up and came home with nothing :('

The junior research intern biologist hands you a meticulous-looking sheaf of parchments. The first - he insists - contains the sum total knowledge of all the creatures they've ever observed. The rest are a log of all the tracks the guild have found in the past five years. Somehow the handwriting is even worse than the hunter's.

You shoo them out of your office before they come to blows. The statistician's guild has been tasked with forging an alliance between the hunters and the biologists. The former typically see the latter as "a bunch of nerds who should go and get their own samples if they want them so much", and in return they're seen as "a bunch of thugs who would rather skin monsters for weapon parts than understand them".

Data and Objectives

The master hunter has given you a list of her hunts from the past five years. This consists of the time and location of the hunt, the weapon and armour she used, and what (if anything) she successfully hunted. You've managed to get your interns to clean up the data a bit and annotate it with information about the weapons and armour mentioned.

The biologists guild have given you a list of their expeditions over the past five years. They also send out one foray, consisting of three biologists per week, to look for monster tracks. On top of this they've given you "all their knowledge of monsters".

It's currently the start of Year 6, and you'll be able to advise the hunter what gear to take, and where to go, from weeks 6-15.
The Hunters' Guild want to maximize their total haul, i.e. they want as few hunts as possible to return with nothing, and they want to return with the largest monsters possible.
They've agreed to let the Biologists' Guild take a look at their specimens for

...
Anxiety and computer architecture
by Adam Zerner — 2021-12-11T10:37:30.898Z

Computer architecture seems like a great analogy for some of the anxiety that I feel.

Call stack fatigue

From Call stacks in everyday life

For example, before writing this article I was getting my dog ready for a walk.

But then I realized that the kitchen needed to be cleaned, so I started doing that.

Then I got inspired to write this article, so I started doing that.

Then I got a text from my brother.

I just responded to my brother.

Now I'm going to finish up the article.

After that I'll finish cleaning the kitchen.

After that I'll finally go and walk my dog.

At it's highest point, the leads to a call stack looks something like this:

textBrother
writeArticle
cleanKitchen
getDogReady

Being in a state where your call stack has more than one item on it is something that produces some sort of uneasy feeling for me. Fatigue isn't really the right word, but let's call it that.

Less instructions in memory

A computer program is basically a list of instructions. Do this, then that, then that, then that. My mind is often in a similar state, where it has a list of things it needs to execute. As if there is a todo list in my head that I am painfully aware of.

I wish it didn't work like this. Instead, here is an alternative architecture:

let alive = true;
let currentTask = firstInstruction;

while (alive) {
currentInstruction();
currentInstruction = fetchNewInstruction();
}

Notice the infinite loop? It's a feature, not a bug.

There is a tight deadline for this feature, and if we don't meet the deadline, or if it ends up breaking at some point afterwards, the whole company immediately fails.

Anyway, that's a conversation for another day. I just couldn't help myself.

For now, what I was pointing at is the fact that there is only one instruction in memory at any given time. That instruction is executed. Then a new instruction is fetched from the database. Then that instruction is executed. Then a new one is fetched. So on and so forth.

Really, it's quite zen if you think about it. The "mind" is totally focused on the present. There is nothing else in memory available to distract it.

Memory here is analogous to working memory, and the database is analogous to long-term memory. Or, better yet, a piece of paper. Your working memory doesn't have that other todo list swimming around, clouding your thinking, producing anxiety. It's just purely focused on the task at hand. Then, only ...
Reasons to act according to the free will paradigm?
by Maciej Jałocha — 2021-12-11T08:44:38.651Z

Hi, probably a newbie question for many of you, but I have some thoughts I'd like to verify publicly. My text presents two potential reasons to act according to the free will paradigm (not to believe blindly nor

say it's true

). Points are divided by "horizontal line".

For starters, let's define free will (FW). If it was stated that an action/change in the physical world was caused by consciousness and that this action/change was at least partially undetermined or non-random then I would call it free will. Consciousness: an object that can feel sensations and can reflect on its thoughts. (So called metacognition).

First point

So although FW is most probably unverifiable and most likely it doesn't exist it has still not been fully refuted, to my knowledge.

So there's a chance it's real.

To my mind, if you believed in FW but the reality was deterministic or random then nothing "bad" or "wrong" would happen (since you cannot even define it). It was like a fatum or dice roll or both combined.

On the other hand, if free will existed and you didn't believe in it, you would limit your options. It's not "bad/wrong" until you define it that way in your value system. It's a personal question: assuming FW exists, would you like to limit your choice? I bet many would rather not.

In a nutshell, I state that believing FW is just more optimal. (for most people)

Second point

Concerning only people who currently assume, there's FW.

Assuming there's FW, then every time you can choose, you can choose to act in line with any values. (Of course, you can choose to act without looking at any value). And more or less of our time, we do (try to) that to fulfill some long-term values.

So from the FW perspective: what's even the reason to find out whether FW is real or not? Unless one's value is to gain knowledge about this particular case (or it's within their broader values like "know all truth"), then there's no reason to do that. As for the first argument, it's a sort of personal question.

From perspective where is no FW we cannot speak about any reasons.

To sum up: neither from FW perspective nor deterministic/randomness perspecitve there's a reason to find out truth about underlaying mechanism. Assuming there's FW and you don't choose to know if it's real you don't need to do it. Assuming there's no FW, reasons don't exi...
Extrinsic and Intrinsic Moral Frameworks
by lsusr — 2021-12-11T05:28:24.944Z

The Western intellectual tradition comes from Christianity. Christianity comes from Judaism. Judaism comes from Zoroastrianism. Zoroastrianism has many ideas recognizable within Christianity.

Zoroastrianism has a dualistic cosmology of good and evil.

Zoroastrianism has an eschatology where good overcomes evil. (Zoroastrian cosmology is a precursor to the Book of Revelation.)

Zoroastrianism has heaven and hell.

Zoroastrianism is fundamentally future-oriented. There is no way to separate "doing good now" from "getting into heaven after you die". Asking a Christian "Would you do the right thing even caused you to go to Hell?" is a contradiction of terms. It is like asking a Utilitarian "Should you do the right thing even if it violates your inclusive utilitarian calculus?"

Zoroastrianism and Utilitarianism are similar ways of thinking because they treat value as a single quantifiable metric. Every reality has a good/evil/utility value (or vector) you can optimize. In this sense, Zoroastrianism and Utilitarianism are fundamentally extrinsic moral frameworks because moral value is downstream of the choices you make.

There are moral frameworks where moral value is upstream of the choices you make.

Intrinsic Moral Frameworks

An intrinsic moral framework is a moral framework where the value of an action is judged by how the action was committed, irrespective of its consequences. Two popular intrinsic moral frameworks are Buddhism and Daoism.

Buddhism

Non-attachment is the core value of Buddhism. Non-attachment is to Buddhism what love is to Christianity or what submission is to Islam[1]. There are a many varieties of Buddhism but all of them are based around a core of non-attachment.

This isn't to say lovingkindness and submission aren't important to Buddhism. They are—just as submission is important to Christians and lovingkindness is important to Muslims. But submission isn't central to Christianity and lovingkindness isn't central to Islam.

If you attend a zendō you might not hear anyone speaking overtly about non-attachment. You're more likely to hear a chant about saving all living beings from suffering. But the reason compassion is valuable is because it's what remains after you let go of your kleishas. Non-attachment is asymmetric. It uproots anger and hatred while leaving love and compassion unharmed.

In Buddhism, the fundamental measure of an action's value is...
Moore's Law, AI, and the pace of progress
by Veedrac — 2021-12-11T03:02:24.558Z

It seems to be a minority view nowadays to believe in Moore's Law, the routine doubling of transistor density roughly every couple of years, or even the much gentler claim, that There's Plenty [more] Room at the Bottom. There's even a quip for it: the number of people predicting the death of Moore's law doubles every two years. This is not merely a populist view by the uninformed. Jensen Huang, CEO of NVIDIA, a GPU company, has talked about Moore's Law failing.

"Moore's Law used to grow at 10x every five years [and] 100x every 10 years," Huang said during a Q&A panel with a small group of reporters and analysts at CES 2019. "Right now Moore's Law is growing a few percent every year. Every 10 years maybe only 2s. ... So Moore's Law has finished."

More academically, the International Roadmap for Devices and Systems, IRDS, warns that logic scaling is nearing certain fundamental limits.

After the “1.5nm” logic node goes into production in 2028, logic dimensions will stop shrinking and improved densities will be achieved by increasing the number of devices vertically. DRAM will continue to shrink CDs [Critical Dimensions] after that, but the minimum lines and spaces will only shrink modestly and should be reachable by improved EUV and EUV double patterning. The large number of masking levels and the many steps for 3D stacking of devices will make yield and cost high priorities.

This claim is not based on tooling limits, but a projected minimum useful size of transistors.

Lines and spaces are the flagship pattern of lithography. [...] Note that the logic node names are the commonly used names for each node but are not the same as the minimum half pitches of those nodes. Resolution improves to 12 nm half-pitch in 2022. This corresponds to the logic “3 nm” node. The IRDS expects that this resolution will be achieved through EUV double patterning. Then there is a further decrease in line and space resolution of 2 nm per node until 2028, when minimum line and space resolution is expected to reach 8 nm half-pitch. The 8 nm half pitch could be achieved with EUV double patterning, but there is time to develop other methods also, such as high-NA EUV lithography. After that, no further improvement in required resolution is projected, although this is due to projected device requirements, not expected limitations in patterning capability.

Computers are made of stacks of wires...
What role should evolutionary analogies play in understanding AI takeoff speeds?
by anson.ho — 2021-12-11T01:19:09.686Z

Cross-posted to the EA forum https://forum.effectivealtruism.org/posts/aSDnzAm85a3Pi87rm/what-role-should-evolutionary-analogies-play-in

This is based on a research project that I’ve been working on under the supervision of Dr. Vael Gates. This work is funded by the Long-Term Future Fund. I would like to thank Vael Gates for their support and supervision, and Tilman Räuker for their valuable feedback.

Epistemic status: exploratory, fairly high uncertainty. My background is in experimental physics, and I only have limited experience with biology or AI.

The motivation for this project was a question from Technical AGI Safety Research Outside AI by Richard Ngo.

What are the most compelling arguments for and against discontinuous versus continuous takeoffs? In particular, how should we think about the analogy from human evolution, and the scalability of intelligence with compute?

In short, it seems that many people are quite confused about the use of evolution as an analogy in forecasting AI takeoff speeds (if not, then I certainly am!) and I hope to clarify things through this post. In particular, I’m focusing mostly on “how should we think about the analogy from human evolution”, although I think this itself is a big enough question that I haven’t given a complete answer to it. I’ve made an effort to accurately represent existing arguments and stay up to date with new ones, but at the end of the day these are still my interpretations of them. Comments and feedback are very welcome, and all mistakes are my own.

Table of Contents

0 Overview

0.0 TL;DR
0.1 Introduction
0.2 Motivation
0.3 Definitions

0.3.1 Takeoff of what?
0.3.2 Takeoff speeds
0.3.3 Proxies

0.4 Some assumptions about evolution

1 Evolutionary arguments for takeoff speeds

1.1 Hominid variation
1.2 Changing selection pressures
1.3 Cumulative cultural evolution

1.3.1 My opinions on this debate

1.4 Brains and compute

1.4.1 Side note: Dog-level AI

1.5 Biological anchors
1.6 One algorithm and Whole Brain Emulation
1.7 Anthropic effects
1.8 Drivers of intelligence explosions
1.9 Summary

2 Evaluating analogies and disanalogies

2.1 A non-exhaustive list of analogies and disanalogies

Analogy: Timescales
Analogy: Necessary compute
Analogy: Cumulative cultural evolution
Analogy: Intelligence is non-hard
Analogy: Thresholds
Analogy: Number of neurons
Analogy: Structure of the learning system
Disanalogy: Training environments
Disanalogy: Optimisation

...
Nonverbal thinkers: how do you experience your inner critic?
by Phoenix Eliot — 2021-12-11T00:40:33.550Z

This is something I've been puzzled by for a long time, for myself. The "inner critic" is generally described as a literal voice that people hear, saying things like "wow, I'm such an idiot", "nobody likes me", etc.

My problem here is that I generally don't think in words (except when thinking about how to explain something to others, like this), and I don't have thoughts that strongly resemble that kind of self-talk even in abstracted form. The closest thing I can identify to an inner critic is spontaneous emotional experiences that seem like they plausibly would be the emotional result I'd get if I had such a voice saying those kinds of things in my thoughts.

I've had some minor success with doing a combination of Focusing and journaling to try to put words to what I'm feeling, but it doesn't seem like it captures the same essence as all the writing about dealing with one's inner critic that I've found.

So for anyone else who is similarly nonverbal in their thinking: How do you experience your inner critic? Are there methods you have for identifying the thoughts, and interacting with them analogously to the typical verbal-oriented advice for dealing with one's inner critic?
The Plan
by johnswentworth — 2021-12-10T23:41:39.417Z

This is a high-level overview of the reasoning behind my research priorities, written as a Q&A.

What’s your plan for AI alignment?

Step 1: sort out our fundamental confusions about agency

Step 2: ambitious value learning (i.e. build an AI which correctly learns human values and optimizes for them)

Step 3: …

Step 4: profit!

… and do all that before AGI kills us all.

That sounds… awfully optimistic. Do you actually think that’s viable?

Better than a 50/50 chance of working in time.

Do you just have really long timelines?

No. My median is maybe 10-15 years, though that’s more a gut estimate based on how surprised I was over the past decade rather than a carefully-considered analysis. (I wouldn’t be shocked by another AI winter, especially on an inside view, but on an outside view the models generating that prediction have lost an awful lot of Bayes Points over the past few years.)

Mostly timelines just aren’t that relevant; they’d have to get down to around 18-24 months before I think it’s time to shift strategy a lot.

… Wat. Not relevant until we’re down to two years?!?

To be clear, I don’t expect to solve the whole problem in the next two years. Rather, I expect that even the incremental gains from partial progress on fundamental understanding will be worth far more than marginal time/effort on anything else, at least given our current state.

At this point, I think we’re mostly just

fundamentally confused about agency and alignment

. I expect approximately-all of the gains-to-be-had come from becoming less confused. So the optimal strategy is basically to spend as much time as possible sorting out as much of that general confusion as possible, and if the timer starts to run out, then slap something together based on the best understanding we have.

18-24 months is about how long I expect it to take to slap something together based on the best understanding we have. (Well, really I expect it to take <12 months, but

planning fallacy

and safety margins and time to iterate a little and all that.)

But iterative engineering is important!

In order for iterative engineering to be useful, we first need to have a strong enough understanding of what we even want to achieve in order to recognize when an iteration has brought us a step closer to the goal. No amount of A/B testing changes to our website will make our company profitable if we’re measuring the wrong metrics. I claim tha...