trentmkelly/LessWrong-43k huggingface dataset:
Maximally Simple Belt Sander Stand
by jefftk — 2022-10-14T00:30:02.329Z

In making

loft

beds for the kids I've needed to do a
lot of sanding. Some of it, like the spindles for the guard railing,
would be a good fit for a stationary sander, but I don't have one. I
do have a belt sander, however, and if you poke around online you can
see

lots

of

DIY

belt

sander

stands,
but they all look like a lot of work. Here's what I did instead:

This is a pair of nails hammered into my porch. Weather permitting I
like to do as much woodworking outside as possible to annoy
the neighbors minimize dust indoors. The nails are just wide
enough for the handle on the sander, and angled up to hold it in
place:

It feels very secure: I don't think the sander could jump up off the nails
(which would be pretty inconvenient: runaway trigger-locked belt
sanders are scary). I used it for eighteen spindles, and it worked
well!

(No guarantees about how well other belt sander designs fit nails.)

Comment via: facebook
Contra shard theory, in the context of the diamond maximizer problem
by So8res — 2022-10-13T23:51:29.532Z

A bunch of my response to shard theory is a generalization of how niceness is unnatural. In a similar fashion, the other “shards” that the shard theory folk want to learn are unnatural too.

That said, I'll spend a few extra words responding to the admirably-concrete diamond maximizer proposal that TurnTrout recently published, on the theory that briefly gesturing at my beliefs is better than saying nothing.

I’ll be focusing on the diamond maximizer plan, though this criticism can be generalized and applied more broadly to shard theory.

The first “problem” with this plan is that you don't get an AGI this way. You get an unintelligent robot that steers towards diamonds. If you keep trying to have the training be about diamonds, it never particularly learns to think. When you compromise and start putting it in environments where it needs to be able to think to succeed, then your new reward-signals end up promoting all sorts of internal goals that aren't particularly about diamond, but are instead about understanding the world and/or making efficient use of internal memory and/or suchlike.
Separately, insofar as you were able to get some sort of internalized diamond-ish goal, if you're not really careful then you end up getting lots of subgoals such as ones about glittering things, and stones cut in stylized ways, and proximity to diamond rather than presence of diamond, and so on and so forth.
Furthermore, once you get it to be smart, all of those little correlates-of-training-objectives that it latched onto in order to have a gradient up to general intelligence, blow the whole plan sky-high once it starts to reflect.
What the AI's shards become under reflection is very sensitive to the ways it resolves internal conflicts. For instance, in humans, many of our values trigger only in a narrow range of situations (e.g., people care about people enough that they probably can't psychologically murder a hundred thousand people in a row, but they can still drop a nuke), and whether we resolve that as "I should care about people even if they're not right in front of me" or "I shouldn't care about people any more than I would if the scenario was abstracted" depends quite a bit on the ways that reflection resolves inconsistencies.
Or consider the conflict "I really enjoy dunking on the outgroup (but have some niggling sense of unease about this)" — we can't conclude from the fa

...
Greed Is the Root of This Evil
by Thane Ruthenis — 2022-10-13T20:40:56.822Z

The SGD's greed, to be specific.

Consider a ML model being trained end-to-end from initialization to zero loss. Every individual update to its parameters is calculated to move it in the direction of maximal local improvement to its performance. It doesn't take the shortest path from where it starts to the ridge of optimality; it takes the locally steepest path.

1. What does that mean mechanically?

Roughly speaking, every feature in NNs could likely be put into one of two categories:

Statistical correlations across training data, aka "the world model".
The policy: heuristics/shards, mesa-objectives, and inner optimization.

The world-model can only be learned gradually, because higher-level features/statistical correlations build upon lower-level ones, and therefore the gradients towards learning them only appear after the lower-level ones are learned.

Heuristics, in turn, can only attach to the things that are already present in the world-model (same for values). They're functions of abstractions in the world-model, and they fire in response to certain WM-variables assuming certain values. For example, if the world-model is nonexistent, the only available heuristics are rudimentary instincts along the lines of "if bright light, close eyes". Once higher-level features are learned (like "a cat"), heuristics can become functions of said features too ("do X if see a cat", and later, "do Y if expect the social group to assume state S within N time-steps").

The base objective the SGD is using to train the ML model is, likewise, a function of some feature/abstraction in the training data, like "the English name of the animal depicted in this image" or "the correct action to take in this situation to maximize the number of your descendants in the next generation". However, that feature is likely a fairly high-level one relative to the sense-data the ML model gets, one that wouldn't be loaded into the ML model's WM until it's been training for a while (the way "genes" are very, very conceptually far from Stone Age humans' understanding of reality).

So, what's the logical path through the parameter-space from initialization to zero loss? Gradually improve the world-model step by step, then, once the abstraction the base objective cares about is represented in the world-model, put in heuristics that are functions of said abstraction, optim...
Vehicle Platooning - a real world examination of the difficulties in coordination
by M. Y. Zuo — 2022-10-13T19:33:21.424Z

Due to aerodynamic drag effects one of the most often talked about methods of increasing the efficiency of road transport is for vehicles, whether trucks or passenger cars, to platoon together. Usually it's proposed that multiple vehicles closely follow behind a lead vehicle that bears the brunt of the aerodynamic drag.

The optimum separation distance, balancing the tradeoff of drag reduction and safety, depends on vehicle height and speed, and could be as little as 2m for shorter vehicles at 60 km/h or up to 9m for full height trucks at 120 km/h.

This is the primary feature of platooning, vehicles are so close together that they cannot safely operate independently but as a combined 'train-like' vehicle. With only one driver determining the platoon speed and direction. Thus necessitating highly coordination actions within the platoon.

For the example of long haul trucking, the main motive for their introduction is cost savings via overall efficiency improvements which depend on many factors including the size and weight of vehicles, the presence of headwinds, etc.

For the case of semi-trucks hauling standard 53 foot trailers, in the most optimistic scenario of strong headwinds and a dozen semi trucks platooned, each hauling their maximum gross weight of 82000 lbs (in the U.S.) on a flat slope, mileage could double with the same amount of fuel.

Clearly halving fuel consumption is an enormous potential benefit as fuel costs can make up the majority of shipping costs nowadays in long range truck hauling.

However, this system has not been introduced yet into any real world truck route for a number of reasons. Some of which are technical, though those have been recently addressed in the last decade, but some are primarily issues of coordination.

Some of the coordination issues arise from quite deep problems of human behaviour and environmental constraints.

One of the most immediate problems is that trucking companies typically don't send out a dozen trucks at once, but one or two every so often. So realizing greater efficiencies would likely necessitate some kind of agreement between several trucking companies to send trucks out near simultaneously to platoon.

Competitive dynamics would likely make them hesitant to cooperate. The parallels are obvious enough with any other type of competitive activity that the possible coordination issues are well ...
The Vitalik Buterin Fellowship in AI Existential Safety is open for applications!
by Xin Chen, Cynthia — 2022-10-13T18:32:55.138Z

This is a linkpost for

https://grants.futureoflife.org/

Epistemic status: Describing the fellowship that we are a part of and sharing some suggestions and experiences.

The Future of Life Institute is launching its 2023 cohort of

PhD and postdoctoral fellowships

to study

AI existential safety

: that is, research that analyzes the most probable ways in which AI technology could cause an existential catastrophe, and which types of research could minimize existential risk; and technical research which could, if successful, assist humanity in reducing the existential risk posed by highly impactful AI technology to extremely low levels. More information about the 2022 cohort can be found

here

.

The

Vitalik Buterin PhD Fellowship in AI Existential Safety

is targeted at students applying to start their PhD in 2023, or existing PhD students who would not otherwise have funding to work on AI existential safety research. Quoting from the page:

At universities in the US, UK, or Canada, annual funding will cover tuition, fees, and the stipend of the student's PhD program up to $40,000, as well as a fund of $10,000 that can be used for research-related expenses such as travel and computing. At universities not in the US, UK or Canada, the stipend amount will be adjusted to match local conditions. Fellows will also be invited to workshops where they will be able to interact with other researchers in the field.

In addition, applicants who are short-listed for the Fellowship will be reimbursed for application fees for up to 5 PhD programs, and will be invited to an information session about research groups that can serve as good homes for AI existential safety research.

Applications for the PhD fellowship close on Nov 15, 2022.

The

Vitalik Buterin Postdoctoral Fellowship in AI Existential Safety

is for postdoctoral appointments starting in fall 2023. Quoting from the page:

For host institutions in the US, UK, or Canada, the Fellowship includes an annual $80,000 stipend and a fund of up to $10,000 that can be used for research-related expenses such as travel and computing. At universities not in the US, UK or Canada, the fellowship amount will be adjusted to match local conditions.

Applications for the postdoctoral fellowship close on Jan 2nd, 2023.

We (Cynthia Chen and Zhijing Jin) are two of the fellows from the 2022 class, and we strongly recommend w...
Feelings
by Eris Discordia — 2022-10-13T17:48:16.751Z

A Brief History

"Reason alone can never be a motive to any action of the will… it can never oppose passion in the direction of the will… The reason is, and ought to be, the slave of the passions, and can never pretend to any other office than to serve and obey them"

These words were written by David Hume's A Treatise of Human Nature in 1773.[1]

We have had philosophical systems that put the wills or the motive forces in their rightful place in the hierarchy of action choices for a long time. The idea being that there's a process outside rationality making selections based on the reality the personality inhabits that are evolutionarily benefiting the human animal.

I intend to argue that emotions are not a separate phenomenon from rational thought, but are instead a core functional component for assembling the rational universe.

I Feel Therefor I Am

Everything we humans care about.

Everything we notice.

Everything in the rational human universe exists because at some point in time a human had a feeling about it enough to care about it and turn it into part of their shared existence with other human beings.

The Emotional Machine

I had set myself the goal of building an artificial general intelligence based in part on Marvin Minsky's emotion machine. I tried working out what it would take to model what I knew about emotions as a biologist and trying out functional abstractions for them as a computer scientist.

I had both personal and educational exposure to psychological concepts so I began with the discrete emotions and tried imagining what each might look like as a function. Here is an entire article on the various models people have tried before.

While looking into the possibility, I ran across the continuous emotional model (figure 1 [2]) and after staring at that for a while, realized I was looking at a four dimensional space. (figure 2)

figure 1 : The discrete emotions mapped as dyads of every emotion with the exception of its polar opposite.

figure 2 : A 4 dimensional emotional space.

Staring at it a while longer and thinking through what some of the neurotransmitters are theorized to convey, I saw functionality as relates to needs pop out of the descriptors we use for those emotions.

My first pass at describing the space looks like this: (figure 3)

figure 3 : The emotional compass of the rational world.

Y Axis : Directional Preference

This indicates whether we want ou...
Against the normative realist's wager
by Joe Carlsmith — 2022-10-13T16:35:30.933Z

(Cross-posted from my website. Audio version here, or search "Joe Carlsmith Audio" on your podcast app.)

Summary

Non-naturalist normative realism is the view that there are mind-independent normative facts that aren’t reducible to facts about the natural world (more here). I don’t find this view very plausible, for various reasons.[1] Interestingly, though, some fans of the view don’t find it very plausible either. But they think that in expectation, all (or almost all) of the “mattering” happens in worlds where non-naturalist normative realism is true. So for practical purposes, they argue, we should basically condition on it, even if our overall probability on it is low. Call this view “the realist’s wager.”

I disagree with this view. This essay describes a key reason why: namely, the realist’s wager says horrible things about cases where you get modest benefits if non-naturalist normative realism is true, but cause terrible harm if it’s false.[2] I describe this sort of case in the first section. The rest of the essay teases out various subtleties and complications. In particular, I discuss:

The difference between “extreme wagers,” which treat nihilism as the only viable alternative non-naturalist normative realism, and more moderate ones.
Whether it makes sense to think that things matter more conditional on non-naturalist normative realism than they do conditional on some non-nihilist alternative, because the former has more “normative oomph.”
Whether it makes sense to at least condition on the falsehood of nihilism. (I’m skeptical here, too.)

My aim overall is to encourage advocates of the realist’s wager (and related wagers) to think quantitatively about the meta-ethical bets they’d actually accept, rather than focusing on more sweeping assumptions like “if blah is false, then nothing matters.”

Thanks to Ketan Ramakrishnan, Katja Grace, Jacob Trefethen, Leopold Aschenbrenner, Will MacAskill and Ben Chang for discussion.

I. Martha the meta-ethical angel

The type of case I have in mind works like this:

Martha’s deal: Martha the meta-ethical angel appeals before you. She knows the truth about meta-ethics, and she offers you a deal. If non-naturalist normative realism is true, she’ll give you a hundred dollars. If it’s false, she’ll burn you, your family, and a hundred innocent children alive.

Claim: don’t take the deal. This is a bad deal. Or at lea...
Weekly Non-Covid News #1 (10/13/22)
by Zvi — 2022-10-13T15:40:01.523Z

This is the non-Covid part of what would previously have been the weekly Covid post. About half of the content written for the post this week is being withheld for topic-level future posts, both with longer time horizons (e.g. policy roundups don’t need to be weekly and should benefit from more integration over time) and elevating worthy sub-topics to their own posts, which this week seems likely to include the colonoscopy study.

We will retain the broad categories of Bad and Good news for various short notes.

Bad News

Sysco Teamsters are on strike in three cities, and there are a lot of people cheering the strikers on mood affectation grounds because Sysco is buying up rivals, slashing staff and service and treating everyone like garbage. Is it a monopoly worth investigating? It sure sounds like it has monopoly power and is actively seeking to assemble more of it. If you keep hiking prices (no you cannot simply say ‘inflation!’) and cutting service and everyone hates you and they live in fear of you, you might be a monopoly. If it was my job to investigate such things, I would investigate.

Signal to stop letting one send SMS messages on Android phones.

Paper (via MR) claims that there is a causal effect on call center workers where they are ‘less productive’ when they report they are happier. This effect might replicate with respect to call centers in particular, I would be shocked if it replicated to workers in general (and thus my answer to Tyler’s generalized ‘are happy workers less productive?’ is ‘no, quite the opposite, and happy to bet on this.’) If this effect is real and causal, my mechanism is that call center ‘productivity’ is about getting through calls quickly whether or not that is good for the business or customer (or victim/target), and also being willing to engage in essentially hostile interactions with the target to get the desired result. There are particular professions and times and places where one wants to Drive Angry. On the other hand I would have been more productive happier at any job I have ever had.

Kamil Galeev has an interesting thread, more explicitly than usual going beyond Russia, talking about the difference between violent and non-violent entrepreneurs, and how Musk’s instinct to ‘make a deal’ with China and Russia is exactly how the non-violent creators have gotten owned and curb-stomped for almost all of history....
Misalignment-by-default in multi-agent systems
by Edouard Harris — 2022-10-13T15:38:58.616Z
Link: https://www.gladstone.ai/instrumental-convergence-2

Summary of this post

This is the second post in a three-part sequence on instrumental convergence in multi-agent RL. Read Part 1 here.

In this post, we’ll:

Define formal multi-agent POWER (i.e., instrumental value) in a setting that contains a "human" agent and an "AI" agent.
Introduce the alignment plot as a way to visualize and quantify how well two agents' instrumental values are aligned.
Show a real example of instrumental misalignment-by-default. This is when two agents who have unrelated terminal goals develop emergently misaligned instrumental values.

We’ll soon be open-sourcing the codebase we used to do these experiments. If you’d like to be notified when it’s released, email Edouard at

edouard@gladstone.ai

or DM me on Twitter at

@harris_edouard

.

Thanks to Alex Turner and Vladimir Mikulik for pointers and advice, and for reviewing drafts of this sequence. Thanks to Simon Suo for his invaluable suggestions, advice, and support with the codebase, concepts, and manuscript. And thanks to David Xu, whose comment inspired this work.

Work was done while at Gladstone AI, which Edouard is a co-founder of.

🎧 This research has been featured on an episode of the Towards Data Science podcast. Listen to the episode here.

1. Introduction

In Part 1 of this sequence, we looked at how formal POWER behaves on single-agent gridworlds. We saw that formal POWER agrees quite well with intuitions about the informal concepts of "power" and instrumental value. We noticed that agents with short planning horizons assign high POWER to states that can access more local options. And we also noticed that agents with long planning horizons assign high POWER to more concentrated sets of states that are globally central in the gridworld topology.

But from an AI alignment perspective, we’re much more interested in understanding how instrumental value behaves in environments that contain multiple agents. If humans one day share the world with powerful AI systems, it will be important for us to know under what conditions our interactions with them are likely to become emergently competitive. If there’s a risk that competitive conditions arise, then it will also be important to understand how they can be mitigated, how much effort this is likely to take, and how we should think about measuring our success at doing so.

To address these questions, we need a measure of instrume...
A stubborn unbeliever finally gets the depth of the AI alignment problem
by aelwood — 2022-10-13T15:16:28.644Z
Link: https://pursuingreality.substack.com/p/finally-getting-the-depth-of-the

I realise posting this here might be preaching to the converted, but I think it could be interesting for some people to see a perspective from someone slow to get onboard with worrying about AI alignment.

I’m one of those people that finds it hard to believe that misaligned Artificial General Intelligence (AGI) could destroy the world. Even though I’ve understood the main arguments and can’t satisfyingly refute them, a part of my intuition won’t easily accept that it’s an impending existential threat. I work on deploying AI algorithms in industry, so have an idea of both how powerful and limited they can be. I also get why AI safety in general should be taken seriously, but I struggle to feel the requisite dread.

The best reason I can find for my view is that there is a lot of “Thinkism” in arguments for AGI takeoff. Any AGI that wants to make an influence outside of cyberspace, e.g. by building nanobots or a novel virus, will ultimately run into problems of computational irreducibility — it isn’t possible to model everything accurately, so empirical work in the physical world will always be necessary. These kind of experiments are slow, messy and resource intensive. So, any AGI is going to reach some limits when it tries to influence the physical world. I do realise there are loads of ways an AGI can cause a lot of damage without requiring the invention of new physical technologies, but this still slowed things down enough for me to worry less about alignment issues.

That was, until I started realising that alignment problems aren’t limited to the world of AI. If you look around you can see them everywhere. The most obvious example is climate change — there is a clear misalignment between the motivations of the petroleum industry and the long term future of humanity, which causes a catastrophic problem. The corporate world is full of such alignment problems, from the tobacco industry misleading the public about the harms of smoking to social media companies hijacking our attention.

It was exploring the problems caused by social media that helped me get the scale of the issue. I wrote an essay to understand why I was spending so much time on browsing the internet, without much to show for it or really enjoying the experience. You can read the full essay here, but the main takeaway for AI safety is that we can’t even deploy simple AI algorithms at scale without c...