trentmkelly/LessWrong-43k huggingface dataset:
REPL's and ELK
by scottviteri — 2022-02-17T01:14:07.683Z

In my previous post I talked about read-eval-print loops as providing a type signature for agents. I will now explain how you can quickly transition from this framework to an ELK solution. Notation is imported from that post.

Imagine we have two agents, a human and a strong AI, denoted H and M respectively. They both interact with the environment in lockstep, according to the following diagram.

We have the human's utility function UH:SH→Q, which is defined on the human's model of reality. We would like to lift UH to a version UM:SM→Q that the machine can use to influence the world in way that is agreeable to the human, which we can do by learning a mapping F : SM→SH and deriving UM=F∘UH.

But we haven't yet said what properties we want the ontology map F to have. I want to call two concepts sh and sm equal if they act the same with respect to transformation: ∀f. f(sh) = f(sm) → sh = sm. The issue is that since the concepts have different types we cannot feed them as arguments to the same function. So instead let's say that ∀s:S, EvalH(sh, ReadH(s)) = EvalM(sm, ReadM(s)) → sh = sm. But now we are back to the same problem where we are trying to compare concepts in two different ontologies. But this does give us a kind of inductive step where we can transfer evidence of equality between concept pairs (sh, sm) and (sh', sm'). I also believe that this kind of a coherence argument is the best we can do, since we are not allowed to peer into the semantic content of particular machine or human states when constructing the ontology map.

Consider the following two graphs.

My intuition is that even if I don't know the labels of the above graphs, I can still infer that the bottom nodes correspond to each other. And the arrows that I get in the context of ELK are the agents' Eval transitions, leading to the following commutative diagram specification for F.

We can learn an ontology map F : SM→SH by minimizing the difference between two paths from a state sm, one in which the machine's prediction function is used and one in which the human's prediction function is used. Concretely, I propose minimizing Dist(sh1,sh2) + λ |U(sh1)-U(sh2)| where sh1 = F(EvalM(sm,om)) and sh2 = EvalH(F(sm),o...
An analogy as the midwife of thermodynamics
by adamShimi — 2022-02-16T21:34:25.799Z

Epistemic status: exploratory

Introduction: Starting point

One of the aspects of deconfusion that consistently trip people up is “where to start”. You have your phenomenon or cluster of intuitions, and then you… make it less confusing? There’s a bunch of heuristics (find as many examples as possible, focus on applications, try a naive synthesis…), but nothing that satisfying.

So one of my goals during my epistemology training is to look at a bunch of examples of deconfusion, and find out patterns about how they got started and how important/problematic were their starting points.

Let’s start with Sadi Carnot’s “

Reflections on the Motive Power of Fire

”, which founded thermodynamics as a science. And the fascinating part is that he seems to have drawn heavily on an analogy with his father Lazare Carnot’s work on a general theory of machines. At least that’s the thesis of philosopher of science John D. Norton in “How Analogy Helped Create the New Science of Thermodynamics”, the paper which was my starting point here.

Following Norton, I’ll use the names “Sadi” and “Lazare” to disambiguate the two Carnot, even though as a fellow frenchman I feel uncomfortable going on a first name basis that quickly.

What did Sadi discover?

My thermodynamics is quite rusty (never liked the subject in college), and I never studied its history. So I at least needed a reminder on what Sadi’s work was all about, and how it fit into the knowledge of its day and the narrative of thermodynamics.

From the Norton paper and Wikipedia, I get that Sadi searched for a general theory of heat engines. Actually, it looks like he introduced the more general concept of heat engine, or at least made the explicit distinction between steam and heat engine, with the former being only a specific form of the latter. What Sadi sought and found was a general enough theory of heat engines to answer questions like the best possible design of a heat engines, or the relevance of using steam vs other gases.

His answer, in a fundamental move establishing a shared frame for the next 200 years of thermodynamics, was that the efficiency of such an engine only depended on two points:

The difference of temperature between the hot source and the cold sink
How close to reversibility the transfer of heat was in the engine.

That’s a pretty cool example of a

shared frame

simplifying a whole mess of engineering problems into two fun...
[Link] Eric Schmidt's new AI2050 Fund
by Aryeh Englander — 2022-02-16T21:21:28.525Z

This is a linkpost for https://www.schmidtfutures.com/schmidt-futures-launches-ai2050-to-protect-our-human-future-in-the-age-of-artificial-intelligence/

I am posting this here as it may be of interest to some members.

Schmidt Futures Launches AI2050 to Protect Our Human Future in the Age of Artificial Intelligence

$125 million, five-year commitment by Eric and Wendy Schmidt will support leading researchers in artificial intelligence making a positive impact

New York — Today, Schmidt Futures announced the launch of “AI2050,” an initiative that will support exceptional people working on key opportunities and hard problems that are critical to get right for society to benefit from AI. Eric and Wendy Schmidt are committed to funding $125 million over the next 5 years, and AI2050 will make awards to support work conducted by researchers from across the globe and at various stages in their careers. These awards will primarily aim to enable and encourage these AI2050 Fellows to undertake bold and ambitious work, often multi-disciplinary, that is typically hard to fund but critical to get right for society to benefit from AI.

I was particularly interested to see the following items listed in their Hard Problems Working List:

What follows is a working list of hard problems we must solve or get right for AI to benefit society in response to the following motivating question:
"It’s 2050, AI has turned out to be hugely beneficial to society and generally acknowledged as such. What happened? What are the most important and beneficial opportunities we realized, the hard problems we solved and the most difficult issues we got right to ensure this outcome, and that we should be working on now?"

...

2. Solved AI’s continually evolving safety and security, robustness, performance, output challenges and other shortcomings that may cause harm or erode public trust of AI systems, especially in safety-critical applications and uses where societal stakes and risk are high. Examples include bias and fairness, toxicity of outputs, misapplications, goal misspecification, intelligibility, and explainability.

3. Solved challenges of safety and control, human alignment and compatibility with increasingly powerful and capable AI and eventually AGI. Examples include race conditions and catastrophic risks, provably beneficial systems, human-machine cooperation, challenges of normati

...
Convoy Continued
by Zvi — 2022-02-16T14:40:02.305Z

Previously: Convoy, also coverage from weekly post.

We continue to have ourselves a convoy, as well as attempts at similar additional convoys elsewhere.

We also have some rather startling actions on the part of the Canadian government, which made the ongoing situation impossible to ignore or disentangle.

I continue to be doing my best to understand what is happening without getting into politics, because this seems increasingly important to do, but the issue is inherently super political and the stakes keep getting raised. I did what I could, and encourage commenters to do the same, while continuing to request that all convoy-related comments be made on convoy-related posts. I won’t be joining those discussions and will confine what I say to the posts. I am happy that I have not yet had to delete any comments on previous convoy posts or even warn anyone, so hopefully that can continue.

The Convoys

As I understand it, there are:

The original convoy in Ottawa’s streets making noise and attempting to cut off or restrict access to downtown.
Several attempts to blockade border crossings between Canada and the United States.
Various attempts to start other convoys that would use similar disruptive tactics elsewhere, without success as official response adjusted to the tactics.

The third point emphasizes, as many other reports do, that the city of Ottawa seriously botched its response to the convoy, and that this allowed the trucks to fortify their positions in various ways. Whereas in other places, they knew what was coming, and this did not happen.

This has some implications.

In tactical terms this was a one-off success that relied on the element of surprise.
If you clear things up now, there is no reason to expect the problem to reassert itself again because you can respond in Ottawa the way other cities have responded, which successfully diffused the problem without the military or noteworthy civil liberties violations.
Thus, the problem for Ottawa is mostly how to physically remove the trucks.

The border crossings can be blocked by remarkably small numbers of vehicles that can then cause a lot of economic damage, allowing the problem to reassert itself over and over, so they are potentially trickier. There the incentive problem is more central there. But in Ottawa, if everyone could be transported back to their homes, as far as I can tell there i...
Compute Trends Across Three eras of Machine Learning
by Jsevillamol — 2022-02-16T14:18:30.406Z

https://arxiv.org/abs/2202.05924

What do you need to develop advanced Machine Learning systems? Leading companies don’t know. But they are very interested in figuring it out. They dream of replacing all these pesky workers with reliable machines who take no leave and have no morale issues.

So when they heard that

throwing processing power at the problem might get you far along the way

, they did not sit idly on their GPUs. But, how fast is their demand for compute growing? And is the progress regular?

Enter us. We have

obsessively analyzed

trends in the amount of compute spent training milestone Machine Learning models.

Our analysis shows that:

Before the Deep Learning era, training compute approximately followed Moore’s law, doubling every ≈20 months.
The Deep Learning era starts somewhere between 2010 and 2012. After that, doubling time speeds up to ≈5-6 months.
Arguably, between 2015 and 2016 a separate trend of large-scale models emerged, with massive training runs sponsored by large corporations. During this trend, the amount of training compute is 2 to 3 orders of magnitude (OOMs) bigger than systems following the Deep Learning era trend. However, the growth of compute in large-scale models seems slower, with a doubling time of ≈10 months.

Figure 1: Trends in n=118 milestone Machine Learning systems between 1950 and 2022. We distinguish three eras. Note the change of slope circa 2010, matching the advent of Deep Learning; and the emergence of a new large scale trend in late 2015.

Table 1. Doubling time of training compute across three eras of Machine Learning. The notation [low, median, high] denotes the quantiles 0.025, 0.5 and 0.975 of a confidence interval.

Not enough for you? Here are some fresh takeaways:

Trends in compute are slower than

previously reported

! But they are

still ongoing

. I’d say slow and steady, but the rate of growth is blazingly fast, still doubling every 6 months. This probably means that you should double the timelines for all

previous

analyses

that relied on AI and Compute’s previous result.
We think the framing of the three eras of ML is very helpful! Remember, we are suggesting to split the history of ML into the Pre-Deep Learning Era, the Deep Learning Era and the Large-Scale Era. And we think this framing can help you make sense of what has happened in the last two decades of ML research.
We have curated an awesome

pub

...
[Intro to brain-like-AGI safety] 4. The “short-term predictor”
by Steven Byrnes — 2022-02-16T13:12:14.331Z

(Last revised: July 2024. See changelog at the bottom.)

4.1 Post summary / Table of contents

Part of the

“Intro to brain-like-AGI safety” post series

.

The previous two posts (

#2

,

#3

) presented a big picture of the brain, consisting of a Steering Subsystem (brainstem and hypothalamus) and Learning Subsystem (everything else), with the latter “learning from scratch” in a particular sense defined in

Post #2

.

I suggested that our explicit goals (e.g. “I want to be an astronaut!”) emerge from an interaction between these two subsystems, and that understanding that interaction is critical if we want to assess how to sculpt the motivations of a brain-like AGI, so that it winds up trying to do things that we want it to be trying to do, and thus avoid the kinds of catastrophic accidents I discussed in Post #1.

These next three posts (#4–#6) are working our way up to that story. This post provides an ingredient that we’ll need: “the short-term predictor”.

Short-term prediction is one of the things the Learning Subsystem does—I’ll talk about others in future posts. A short-term predictor has a supervisory signal (a.k.a. “ground truth”) from somewhere, and then uses a learning algorithm to build a predictive model that anticipates that signal a short time (e.g. a fraction of a second) in the future.

This post will be a general discussion of how short-term predictors work and why they’re important. They will turn out to be a key building block of motivation and reinforcement learning, as we’ll see in the subsequent two posts.

Teaser for the next couple posts:

The next post (#5) will discuss how a certain kind of closed-loop circuit wrapped around a short-term predictor turns it into a “long-term predictor”, which has connections to the temporal difference (TD) learning algorithm. I will argue that the brain has a large number of these long-term predictors, built out of loops between the two subsystems, and that a subset of these predictors amount to the “critic” part of actor-critic reinforcement learning. The “actor” part is the subject of Post #6.

Table of contents:

Section 4.2 gives a motivating example, of flinching just before getting hit in the face. This can be formulated as a supervised learning problem, in the sense that there is a ground-truth signal to learn from. (If you just got hit in the face, then you should have flinched!) The resulting circuit is what I

...
Why Doesn’t Healthcare Improve Health?
by George3d6 — 2022-02-16T11:28:08.546Z
Link: https://www.epistem.ink/p/why-doesnt-healthcare-improve-health?r=c5fr0

It’s almost a non-sequitur to ask whether or not a giant industry, with a well-defined goal, churning double-digits of GDP, actually works.

If pressured into an experiment we could easily design one that shows airplanes deliver people to destinations over 3000km away faster than trains (boarding time and travel to airport included). We could similarly prove they help people travel more, and more money spent on airplanes means more travel. Take a random group of 1000 people, give half 5000$ airline credits for two years, and we’ll certainly find that, on average they end up visiting more far-away places than the control.

On the other hand, this is not entirely unheard of.

A big and controversial example is the church. Where we’ve pretty conclusively proven that building huge cathedrals, converting “savage natives”, stomping out heresy and prayer have little results upon any form of well-being. But even here we must admit that, if the apparatus of science is unleashed upon religion, we will find interesting things such as people in religious communities drinking less, having longer-lasting marriages, and living longer with higher self-reported life satisfaction. We can argue about confounders and about the causal mechanism being replicable with things other than religion. But I think that, to most scientifically minded people, the lack of evidence is sufficient to not motivate them into becoming religious.

However, I keep being surprised by intelligent and scientifically minded people outright refusing to hear the evidence against medicine being useful. That’s not to say evidence against any particular therapy. There are many things that obviously work, such as emergency intervention for losing a limb in a car crash, where “almost certain death” is the clear outcome. There are many things that obviously work from a statistical perspective, such as vaccination against diseases such as covid, hep A & B, polio, HPV, yellow fever, rabies, and the like. There are also many things that seem to work, with good mechanistic evidence and promising studies, such as PRP for various forms of soft-tissue injury, or prophylactic therapy for HIV.

But this therapy-specific evidence is only more damming when viewed in the context of broader findings against medicine. Robin Hanson has written on this ad-nauseam, so I will let him do most of the talking.

1974 to 1982 the US governm

...
How do you think about mildly technical people trying to advance science and technology?
by Cui — 2022-02-16T03:43:45.662Z

If you were, say 22, had free time, enough money, and decent curiosity and technical skills (but less than spectacular) what would you do? I could get a PhD. I could try and do something with more upside—maybe startups. But what opportunities exist in my blindspot? Writing, consulting, science funding, K-12 education? What can I do to get long-term leverage—gain technical skills?

I hate the idea of watching people make real-world, practical impact while I try away at an irrelevant PhD. I've been trying to embrace learning by doing and just starting on projects. Thoughts here?
Acoustic vs Electric Mandolin
by jefftk — 2022-02-16T02:20:01.771Z

When I bought my mandolin in 2013, I

recorded some examples
comparing it to my previous one. Now that I have an electric
mandolin, I wanted to do something similar.

The acoustic mandolin is a Collings MT,
and the electric is a Gold
Tone GME-4. Here's how they sounded:

Chords

(

acoustic mp3)

(

electric mp3)

High riff

(

acoustic mp3)

(

electric mp3)

Low riff

(

acoustic mp3)

(

electric mp3)

High melody

(

acoustic mp3)

(

electric mp3)

Medium melody

(

acoustic mp3)

(

electric mp3)

Low melody

(

acoustic mp3)

(

electric mp3)

The acoustic has relatively worn J74 medium phosphor bronze strings,
while the electric has the steel strings that shipped with the
instrument. I mic'd the acoustic the way I would play it live: about 2" from the 15th
fret, where the neck meets the body, with a Sennheiser
e835s. With the electric I had the tone knob at 10 (no low pass)
and connected it to the board via a MXR M222 on bypass
(details).

Overall, for most of the kind of playing I do, I strongly prefer the
sound of the acoustic. It's much more complex, especially in the high
end. On the other hand, the electric offers some options for sounds
that I can't get on the acoustic, especially when paired with the talkbox, and it
sounds good enough clean that I would be ok playing that way some.
For Free Raisins gigs,
where I'm playing mandolin almost all the time, I'm definitely going
to continue bringing my acoustic, and might additionally bring my
electric when that wouldn't be too much hassle. For Kingfisher gigs, however,
where I mostly play keyboard and when I do play mandolin expect to
mostly use pedals, I think I'll probably bring only my electric.

(My view might change after hearing how the electric sounds at a
dance. Sometimes what sounds best in isolation isn't a good fit with
other instruments, or in the chaos of a dance hall.)
REPL's: a type signature for agents
by scottviteri — 2022-02-15T22:57:35.888Z

Read-eval-print loops are used to interact with programming languages as well as in the shell to interact with your computer. I claim that they are a reasonable starting point for reasoning about agents. This is a framing that all but wrote my ELK proposal for me, and has been paying rent in other research domains as well.

Let S be the set of world states, and assume a environment transition function T : S × A → S, where Aₓ is a set of actions.
We define an agent X to be a 3-tuple of functions Readₓ, Evalₓ, and Printₓ, where:

• Readₓ is a surjective "abstraction" function of type S → Oₓ, where Oₓ is a type for X's abstracted observations.

• Evalₓ is a prediction function of type Sₓ × Oₓ → Sₓ, where Sₓ represents the type of X's internal knowledge of the environment.

• Printₓ is a function from Sₓ to Aₓ, where Aₓ denotes X's actions on the environment. If there are multiple agents, we assume that an action Aₓ contains one action from each agent.

The environment and agent start in initial states S⁰ and Sₓ⁰, respectively. At each time-step, the agent observes the current state of the environment with its sensors, updates its worldview using its prediction function, produces an action, and then the universe uses that action to produce a next universe state. This process is depicted in the figure below.

Notice that in the above diagram, the transition functions of the agent and the environment look similar. In fact they are dual, and we can show this by considering the agent's Readₓ to be the environment's Print, and the agent's observation type Oₓ to be the environment's action type A.

Env := { Print : S → Aₓ, Eval : S × Aₓ → S }
Agentₓ := { Printₓ : Sₓ → Aₓ, Evalₓ : Sₓ × A → Sₓ }

Both the agent and environment start in a given initial state, pick an action based on that state, feed that action to each other's transition functions, and transition to the next state.

This interaction can be drawn as a game tree where the agent and the environment are selecting which of each other's next branches to follow.

The agent and environment run in lockstep, each round simultaneously printing and thereby choosing their partner's next branch. If you want to think of the environment as giving a reward, that reward can be a function of your whole action history, which put you in a particular branch of the physi...