trentmkelly/LessWrong-43k huggingface dataset:
Coordination explosion before intelligence explosion...?
by tailcalled — 2023-03-05T20:48:55.995Z

Epistemic status: musing that I wanted to throw out there.

A traditional AI risk worry has been the notion of an intelligence explosion: An AI system will rapidly grow in intelligence and become able to make huge changes using small[1] subtle[2] tricks such as bioengineering or hacking. Since small actions are not that tightly regulated, these huge changes would be made in a relatively unregulated way, probably destroying a lot of things, maybe even the entire world or human civilization.

Modern AI systems such as LLMs seem to be making rapid progress in turning sensory data into useful information, aggregating information from messy sources, processing information in commonsense ways, and delivering information to people. These abilities do not seem likely to generalize to bioengineering or hacking (which involve generating novel capabilities), but they do seem plausibly useful for some things.

Two scenarios of interest:

Coordination implosion: Some people suggest that because modern AI systems are extremely error-prone, they will not be useful, except for stuff like spam, which degrades our coordination abilities. I'm not sure this scenario is realistic because there seem to be a lot of people working on making it work for useful stuff.

Coordination explosion: By being able to automatically do basic information processing, it seems like we might be able to do better coordination. We are already seeing this with chatbots that work as assistants, sometimes being able to give useful advice based on their mountains of integrated knowledge. But we could imagine going further, e.g. by automatically registering people's experiences and actions, and aggregating this information and routing it to relevant places.

(For instance, maybe a software company installs AI-based surveillance, and this surveillance notices when developers encounter bugs, and takes note of how they solve the bugs so that it can advise future developers who encounter similar bugs about what to do.)

This might revolutionize the way we act. Rather than having to create, spread, and collect information, maybe we would end up always having relevant information at hand, ready for our decisions. With a bit of rationing, we might even be able to keep spam down to a workable level.

I'm not particularly sure this is what things are going to look like. However I think the possibility is useful to keep...
What are some good ways to heighten my emotions?
by  — 2023-03-05T18:06:30.693Z

I've noticed when I don't get enough sleep, the emotions I feel the next day are higher variance - I'm a sadder sad person and a happier happy person.

I want more of this, since I prefer feeling emotions to not feeling emotions and generally enjoy higher variance in life due to novelty and stuff. What are some ways I can get this without being sleep deprived?

(Also would appreciate pushback! Think this may be wrong due to similar reasons to variance being bad in finance.)
Research proposal: Leveraging Jungian archetypes to create values-based models
by MiguelDev — 2023-03-05T17:39:27.126Z

This project is the origins of the Archetypal Transfer Learning (ATL) method

This is the abstract of my research proposal submitted to AI Alignment Awards. I am publishing this here for community feedback. You can find the link to the whole research paper here.

Abstract

We are entering a decade of singularity and great uncertainty. Across all disciplines, including wars, politics, human health, as well as the environment, there are concepts that could prove to be a double edged sword. Perhaps the most powerful factor in determining our future is how information is distributed to the public. It can be both transformational and empowering using advanced AI technology – or it can lead to disastrous outcomes that we may not have the foresight to predict with our current capabilities.

Goal misgeneralization is defined as a robustness failure for learning algorithms in which the learned program competently pursues an undesired goal that leads to good performance in training situations but bad performance in novel test situations. This research proposal tries to capture what might be a better description of this problem and solutions from a Jungian perspective.

This proposal covered key AI alignment topics, from goal misgeneralisation to other pressing issues. It offers a comprehensive approach for addressing critical questions in the field.

reward misspecification and hacking
situational awareness
deceptive reward hacking
internally-represented goals
learning broadly scoped goals
broadly scoped goals incentivizing power-seeking,
power seeking policies would choose high reward behaviors for instrumental reasons
misaligned AGIs gain control of the key levers of power

These above-mentioned topics were reviewed to check the viability of approaching the alignment problem through a Jungian approach. 3 key concepts emerged from the review:

By understanding how humans use patterns to recognize intentions at a subconscious level, researchers can leverage on Jungian archetypes and create systems that mimic natural decision-making. With this insight into human behavior, AI can be trained more effectively with archetypal data.
Stories are more universal in human thought than goals. Goals and rewards will always yield the same problems encountered in alignment research. AI systems should utilize the robustness of complete narratives

...
Abusing Snap Circuits IC
by jefftk — 2023-03-05T17:00:08.585Z

The kids got a

Snap
Circuits Kit a while ago, and enjoy it. In terms of learning,
however, it's been a bit mixed. The basic circuits (light, motor) are
good: they understand they need to connect the parts the right way
around and that they need a complete circuit. On the other hand, most
of the projects in the kit rely on the three audio chips, and just doing
the projects isn't enough to really understand how they work.

They had a friend over yesterday, and after building one of the projects
together wandered off to do something else. I decided to see if I
could experimentally determine how one of the chips worked, so that
maybe we could do something interesting together later.
There turned out to be a lot more combinations that did something
interesting than I expected, however, so instead this is just
cataloging them. So: here are 13 more snap circuits projects you can
make that abuse the music IC to do things it wasn't intended to do.

Since none of this is what was intended, you may get different results
from these than I did. I think you won't break anything,
because as a modern kids toy I'd expect this to be robust to hooking
it up 'wrong', but no guarantees.

First, though, here's the basic correct operation of the Music IC:

The pinout of the chip is:

trigger +--+--+ hold
| |
- +-----+ out

The idea is you connect positive voltage to the + pin,
negative to the - pin, run the speaker between
out and + and you'll get "happy birthday".
It plays one time through the song when it first gets power, and
repeats if you connect trigger to power momentarily or
hold to power in a sustained way.

Despite my picture above being the simplest correct use of the chip,
it's not one of the projects. Instead the first project with an audio
IC (#3) also includes the "whistle chip" sensor on the trigger pin to
allow you to make it start over. Then they follow up (#4) by putting a
resistor in series with the speaker to make it quieter.

Here are some other arrangements that demonstrate other modes the chip
has:

Instead of connecting + directly to positive
voltage, put the resistor in the way. It plays the same song at a
slightly lower pitch.

Don't connect anything to +. Instead, connect
positive voltage to hold. It plays something kind of of like a
distorted version of the first line.

Same as (2) but

...
Do humans derive values from fictitious imputed coherence?
by TsviBT — 2023-03-05T15:23:04.065Z

[Metadata: crossposted from https://tsvibt.blogspot.com/2022/11/do-humans-derive-values-from-fictitious.html. First completed November 1, 2022. This essay is more like research notes than exposition, so context may be missing, the use of terms may change across essays, and the text may be revised later; only the versions at tsvibt.blogspot.com are definitely up to date.]

Humans are born with some elements of their minds, and without many other elements, some of which they'll acquire as their life unfolds. In particular, the elements that we pretheoretically call "values"--aesthetic preferences, goals, life goals, squad goals, aspirations, needs, wants, yearnings, drives, cravings, principles, morals, ethics, senses of importance, and so on--are for the most part acquired or at least unfolded, rather than being explicitly present in a newborn. How does this happen? What generates these mental elements?

Hypothesis: a human derives many of zer values by imputing coherent agency to zer past behavior, and then adopting the goals of that fictitious agency as actively influential criteria for future action.

Thanks to Sam Eisenstat for relevant conversations.

The FIAT hypothesis

As a shorthand: "the FIAT hypothesis" = "the Fictitious Imputed Adopted Telos hypothesis". ("Fiat" is Latin for "may it happen" or "may it be made", which has some resonance with the FIAT hypothesis in that they both talk about a free creation of goals.) FIAT goals are goals imputed to some behavior and then adopted as goals.

Human behavior is determined by many things: built-in behavior-determiners such as the instinctive ability to breath, socially learned behavior and values, convergent instrumental goals, and freely created autopoietic goals such as artistic goals. The FIAT hypothesis says that a major determiner of a human's behavior is the process of adopting goals based on interpreting zer past behavior as agentic.

Ze can be interpreted as asking the question: if my past behavior were the behavior of a coherent agent trying to do something, what would that something be? Then, whatever the answer was, ze adopts it as a goal--a target of more coherent behavior (more effective, more strategic, more orchestrated, more coordinated, more conscious, better resourced, more reflective, more univocal, more wasteless).

This hypothesis gives a possible answer to the question: how did evolution build so...
Why kill everyone?
by arisAlexis — 2023-03-05T11:53:23.823Z

In current thinking X-risk includes the possibility of human extinction. True, everyone is probably thinking in ranges of outcomes with probabilities scattered but if a non-aligned AI actually tries to attack humans usually a very high probability of extinction is assumed although nobody has ever quantified it.

I propose here that actually non-extiction in the event of superintelligent attack should be the default outcome. There would be no reason for this agent to kill all humans following the logic that we do not exterminate all ants globally when we are building a house. Some ants are very far from us and too weak to do anything and it takes extra effort and resources to exterminate them all. Sure, maybe it's trivial but given the huge dispersion of humans in remote places that even would be hidden in bunkers we can assume the utility of killing them all is lower than killing the ones that matter.

This agent would probably have suffucient backups established that virtually nuking even continents would not stop it's source code from running. That should be trivial given internet connectivity and the amount of computers in the world. Even just one copy somewhere would be more than enough. But there would be a handful of people that would try to stop it. These would be people who either know how to create an adversary or know it's inner workings so they could try to hack it and alter it's source code. There have been a lot of creative ways of how an ASI will terminate us so I won't go into this discussion but if it can do it to all humanity it can also do it only to a few people that matter:

AI researchers, high military ranking officers that control nukes, high ranking tech officers that control internet global routers and also influential intelligent people that could stage a coup and resistence.

What would be the purpose of killing farmers in India or surfers in Mexico? None of them could live long enough to gain the sufficient knowledge to pose a real threat. Old ladies can't fight and even if they could fighting with guns against an ASI would only happen in a movie like Terminator. After establishing factories and everything else it needs it could spare humans until it needs it for their atoms but even this is objectable. Kurzweil in his famous book thinks that humans would provide a miniscule benefit to an AI given a goal of building Von Neuman probe...
Selective, Corrective, Structural: Three Ways of Making Social Systems Work
by Said Achmiz — 2023-03-05T08:45:45.615Z

(A reference post for a concept that comes up often enough to warrant such a thing.)

If men were angels, no government would be necessary. If angels were to govern men, neither external nor internal controls on government would be necessary.

—James Madison

Three kinds of methods

The one comes to us (wishing to build a social system, and having only people, and no angels, to work with) and says:

“My system would work perfectly, if only everyone involved would behave in the optimal manner!”

Granted; unfortunately (or fortunately), not all people can be relied on to behave optimally. How to make the system work despite this?

There are three sorts of approaches:

Selective methods—build your system out of only the right sort of people, and exclude the wrong sort.

Corrective methods—apply such measures as will make the people in your system alter their behavior, to conform to relevant optimality criteria.

Structural methods—build your system in such a way that it will work if people behave in the ways that they can be expected to behave.

Examples

Work

The challenge: build an organization (or a team within one) that will be able to accomplish various desirable projects.

Selective: hire people who have the skills/experience/etc. to do the work; don’t hire (or fire, if discovered post-hiring) people who aren’t capable of doing the work.

Corrective: on-the-job training; social approval/disapproval from coworkers for good/bad work.

Structural: bonuses and other financial incentives for performance; technological and process improvements that reduce skill requirements.

World of Warcraft

The challenge: assemble a raiding guild that will be able to defeat the most challenging boss monsters.

Selective: accept players who can demonstrate competence in their chosen raid role; exclude those who can’t or won’t perform.

Corrective: teach inexperienced players to play better; shame lazy or selfish players into putting in effort, and contributing to the guild’s success.

Structural: assign raid members to roles that best fit their talents and inclinations; design a loot distribution system that incentivizes effort and effective participation.

Governance

The challenge: place over society a government, that will rule for the good of all.

Selective: choose wise and just rulers; prevent the foolish and the wicked from gaining power.

Corrective: “We demand that pub

...
Substitute goods for leisure are abundant
by Adam Zerner — 2023-03-05T03:45:32.705Z

Meta: This is intended to be opinionated. I think opinionated things are often useful, they just need to be read with the right perspective. The arguments aren't air-tight, but to a certain audience the ideas might "click". Mr. Money Mustache comes to mind as an example of this. I recognize of course that my arguments aren't perfect, I might be wrong, and to the that extent I am right it won't apply to everyone, but in the interest of being concise I'm not going to use too many qualifiers in this post.

During the pandemic, there was a conversation I had with a friend that has always stuck with me. Before the pandemic he had been getting into jujitsu. During the pandemic he couldn't continue with it, and he was bummed. Then later on in the pandemic when things first started opening up again, he continued with jujitsu even though it's one of the riskier things you could have been doing.

Something similar happened with my mom. There is a bagel place she goes to every weekend, and she was really bummed when it was takeout only.

Related thing: when I first met my girlfriend and moved to Las Vegas to be with her, she was really appreciative. I didn't see it as a big deal though. Why? Because substitute goods for leisure are abundant.

Yes, the summers in Vegas suck. Especially since I don't have a car and get around via walking or biking. But the way I see it, it just means that I substitute one set of activities for a different set of activities. For example, if I lived somewhere where I can walk around I'd spend more time going to restaurants to eat and coffee shops to work. If I didn't I'd spend more time cooking and using communal areas in my apartment.

Call { restaurants, coffee shops } set A and { cooking, communal areas } set B. I prefer set A to set B, but I don't have a super strong preference.

Economists have some cool terminology for this stuff. There's the concept of substitute goods.

The classic example is Coke and Pepsi. Imagine you go to a restaurant and want Coke but all they have is Pepsi. That's probably fine, right? For most people, Pepsi is 99%+ as good as Coke. When two goods are comparable like this, they're considered to be substitute goods.

In reality the extent to which something is a substitute is a spectrum. We can talk about how good something performs as a substitute. Maybe Pepsi is a "very good" substitute for Coke whereas Sprite is only a "de...
Does polyamory at a workplace turn nepotism up to eleven?
by Viliam — 2023-03-05T00:57:52.087Z

I am reading debates on the Effective Altruism forum, and they seem to focus on how a low-status person might be afraid to refuse to consent to sex with a high-status person, because the high-status person might then refuse to provide them a grant or a job position.

True, but this seems to be just the tip of the iceberg.

Making a high-status polyamorous person angry potentially means not just creating one enemy, but rather a group of enemies -- namely the person's entire polycule. They are naturally likely to trust that person. Even if they have doubts, they still have a selfish incentive not to disrupt their own polycule. And if they often meet and talk in private, the person will be able to tell all of them their version first.

*

Imagine the usual monogamous workplace. Imagine rejecting the advances of your manager, and then experiencing retaliation. Imagine contacting HR to complain, and finding out that the HR person handling your complaint is the manager's wife. Would you expect a fair treatment?

Now imagine a workplace with lots of polyamorous dating. The difference is that if your manager is polyamorous, the chance of meeting a member of their polycule is higher, and you probably wouldn't even know. Or you might give up and try your luck at a different company in this tightly connected small industry... but the new manager, or the job interviewer might be a part of the same polycule, too. Perhaps you have just locked yourself out of the entire industry.

On the other hand, joining a polycule of high-status people might be the ultimate office politics move. (Unless you had a bad luck, and the polycule turns out to be two-tiered: the high-status members who treat each other seriously, and the "underlings with benefits" that keep coming and going.)

*

This is not about "polyphobia".

I am not saying that monogamy is good and polyamory is bad. The failures of monogamy are numerous and well known, and we have lots of rules created around them. The rules against nepotism were created in the context of monogamy. The rules against sexual harassment were created in the context of (at least officially) monogamy. The archetype of a vindictive rejected man (or a woman) taking a revenge also exists mostly in the context of monogamy. We already know that monogamy can be bad, and we have a set of rules trying to minimize the harm (arguably, not sufficiently, but better than not...
Forecasts on Moore v Harper from Samotsvety
by gregjustice — 2023-03-05T00:47:37.217Z
Link: https://samotsvety.org/blog/2023/02/26/moore-v-harper/

"The Moore v. Harper case before SCOTUS asks to what degree state courts can interfere with state legislatures in the drawing of congressional district maps. Versions of the legal theory they’re being asked to rule on were invoked as part of the attempts to overthrow the 2020 election, leading to widespread media coverage of the case. The ruling here will have implications for myriad state-level efforts to curb partisan gerrymandering."

"[O]ur probability for a ruling against the [North Carolina Supreme Court] is 81%, very close to the base rate for state court cases before SCOTUS."

"The probabilities listed are contingent on SCOTUS issuing a ruling on this case. An updated numerical forecast on that happening, particularly in light of the NC Supreme Court’s decision to rehear Harper v Hall, may be forthcoming."

With credit to @yagudin, @NunoSempere, and others at Samotsvety
Why Not Just... Build Weak AI Tools For AI Alignment Research?
by johnswentworth — 2023-03-05T00:12:33.651Z

“Weak” cognitive tools are clearly a thing, and are useful. Google search is a fine example. There are plenty of flavors of “weak AI” which are potentially helpful for alignment research in a similar way to google search.

In principle, I think there’s room for reasonably-large boosts to alignment research from such tools[1]. Alas, the very large majority of people who I hear intend to build such tools do not have the right skills/background to do so (at least not for the high-value versions of the tools). Worse, I expect that most people who aim to build such tools are trying to avoid the sort of work they would need to do to build the relevant skills/background.

Analogy: A Startup Founder’s Domain Expertise (Or Lack Thereof)

Imagine a startup building tools meant to help biologists during their day-to-day work in the wetlab. I expect domain expertise to matter a lot here: I would guess that if none of the founders have ample personal experience doing research work in a wetlab, the chance of this startup building an actually-highly-useful wetlab product drops by about an order of magnitude. Our hypothetical startup might still “succeed” some other way, e.g. by pivoting to something else, or by being good at pitching their shitty product to managers who make purchasing decisions without actually using the product, or by building something very marginally useful and pricing it very cheaply. But their chance of building a wetlab product which actually provides a lot of value is pretty slim.

One might reply: but couldn’t hypothetical founders without domain experience do things to improve their chances? For instance, they could do a bunch of user studies on biologists working in wetlabs, and they could deploy the whole arsenal of UX study techniques intended to distinguish things-users-say-matter from things-which-actually-matter-to-users.

… and my response is that I was already assuming our hypothetical founders do that sort of thing. If the founders don’t have much domain experience themselves, and don’t do basic things like lots of user studies, then I’d guess their chance of building an actually-high-value wetlab product drops by two or three orders of magnitude, not just one order of magnitude. At that point it’s entirely plausible that we’d have to go through thousands of times more startups to find one that succeeded at building a high-value product....