Th?ese

pr?esent?ee

a

L'Universit

e

Pierre

Marie

Curie

Paris

VI

pour

obtenir

le

titre

de

Docteur

Sp?ecialit?e

Informatique

par

Paulo

Jorge

Pires

Ferreira

Sujet

de

la

th?ese

Larchant

ramasse-miettes

dans

une

m?emoire

partag?ee

r?epartie

avec

persistance

par

atteignabilit?e

Soutenue

le

Mai

devant

le

jury

compos?e

de

MM

Jean-Jacques

Levy

Pr?esident

Christian

Queinnec

Rapporteurs

Michel

Raynal

Claude

Girault

Examinateurs

Jos?e

Marques

Marc

Shapiro

Willy

Zwaenepoel

A

Yashmin

ii

Remerciements

Je

remercie

Claude

Girault

Professeur

a

l'Universit

e

Pierre

et

Marie

Curie

responsable

de

l'enseignement

de

troisi?eme

cycle

en

informatique

pour

m

avoir

accept?e

comme

etudiant

de

doctorat

et

pour

avoir

particip?e

a

ce

jury

Je

remercie

Jean-Jacques

Levy

Professeur

a

l'

Ecole

Polytechnique

et

Directeur

de

Recherche

a

l'INRIA

pour

avoir

accept?e

de

pr?esider

le

jury

de

cette

th?ese

Je

remercie

Christian

Queinnec

et

Michel

Raynal

respectivement

Professeurs

a

l'

Ecole

Polytechnique

et

a

l'Universit

e

de

Rennes

I

pour

l'int

er

et

qu'ils

ont

manifest?e

pour

mon

travail

en

acceptant

d'

etre

les

rapporteurs

de

cette

th?ese

Je

remercie

Jos?e

Marques

et

Willy

Zwaenepoel

respectivement

Professeurs

a

l'Universit

e

Technique

de

Lisbonne

et

a

Rice

University

pour

l'int

er

et

qu'ils

ont

port?e

a

mon

travail

en

acceptant

de

participer

a

ce

jury

Je

remercie

tr?es

chaleureusement

Marc

Shapiro

Responsable

Scientifique

du

Projet

SOR

qui

m

a

encadr?e

pour

son

exigence

ses

conseils

et

sa

rigueur

Ses

commentaires

et

critiques

ont

directement

influenc?e

mes

travaux

Je

lui

suis

tr?es

reconnaissant

d'avoir

stimul?e

et

encourag?e

ce

travail

Je

remercie

tous

les

membres

du

Projet

SOR

a

l'INRIA

Yann

pour

son

travail

sur

le

banc

d'essais

Mokrane

pour

avoir

trouv?e

et

corrig?e

des

erreurs

dans

Larchant

Mesaac

pour

sa

disponibilit?e

et

ses

conseils

avis?es

Herv?e

pour

m

avoir

aid?e

a

bien

ecrire

en

Fran?cais

David

pour

nos

discussions

sur

les

ramassemiettes

George

pour

son

expertise

de

LaTeX

et

d'Emacs

Povl

Koch

pour

nos

discussions

sur

les

syst?emes

bas?es

sur

un

espace

d'adressage

plat

Ian

et

Aline

pour

leur

aide

concernant

respectivement

l'Anglais

et

le

Fran?cais

Julien

pour

son

aide

quand

je

suis

arriv?e

au

Projet

SOR

Guillaume

pour

son

expertise

WWW

et

Nelly

pour

sa

disponibilit?e

Tout

au

long

de

mon

s?ejour

a

l'INRIA

j'

ai

profit?e

des

discussions

avec

des

personnes

ext?erieures

au

Projet

SOR

Je

remercie

Laurent

Amsaleg

pour

nos

discussions

sur

les

ramasse-miettes

Hank

Levy

pour

nos

discussions

concernant

la

m?emoire

partag?ee

r?epartie

Marcin

et

Marie-H

el?ene

pour

m

avoir

aid?e

avec

les

automates

Fabio

pour

m

avoir

montr?e

la

puissance

des

state-charts

Je

remercie

la

JNICT

-LRB-

programmmes

Ci

encia

et

PRAXIS

XXI

-RRB-

pour

la

bourse

qui

m

a

et?e

attribu?ee

et

qui

m

a

permis

de

mener

ma

recherche

et

l'INESC

pour

m

a

avoir

encourag?e

a

faire

mon

doctorat

hors

du

Portugal

Les

conditions

de

travail

offertes

a

l'INRIA

sont

exceptionnelles

Les

moyens

mis

a

ma

disposition

durant

cette

th?ese

m

ont

permis

de

mener

librement

mon

iii

travail

Je

remercie

toute

ma

famille

pour

son

soutien

extraordinaire

malgr?e

la

distance

qui

nous

s?epare

En

particulier

je

remercie

mes

parents

qui

m

ont

toujours

encourag?e

et

montr?e

le

plaisir

de

la

recherche

Finalement

je

remercie

tr?es

sp?ecialement

mon

epouse

pour

son

amour

son

encouragement

sa

patience

et

sa

compr?ehension

Paulo

Ferreira

iv

Acknowledgments

I

thank

Claude

Girault

Professor

at

Universit?e

Pierre

et

Marie

Curie

for

having

accepted

me

as

a

Ph.D.

student

and

for

being

a

member

of

the

jury

I

thank

Jean-Jacques

Levy

Professor

at

Ecole

Polytechnique

and

Director

of

Research

at

INRIA

for

being

the

President

of

the

jury

I

thank

Christian

Queinnec

and

Michel

Raynal

Professors

at

Ecole

Polytechnique

and

at

Universit?e

de

Rennes

I

respectively

for

having

accepted

to

be

the

rapporteurs

of

this

thesis

I

thank

Professors

Jos?e

Marques

from

the

Technical

University

of

Lisbon

and

Willy

Zwaenepoel

from

Rice

University

for

having

accepted

to

be

members

of

the

jury

I

warmly

thank

Marc

Shapiro

leader

of

Project

SOR

at

INRIA

and

my

research

advisor

for

his

care

exigence

and

guidance

His

comments

and

criticism

helped

me

to

improve

my

ideas

and

the

description

of

my

work

I

thank

all

the

elements

of

the

Project

SOR

at

INRIA

Yann

for

his

work

on

the

benchmark

Mokrane

for

his

help

to

discover

and

correct

some

bugs

of

Larchant

Mesaac

for

his

constant

availability

and

criticism

Herv?e

for

his

valuable

help

with

French

writing

David

for

our

discussions

concerning

garbage

collection

George

for

his

expertise

with

LaTeX

and

Emacs

Povl

Koch

for

our

discussions

concerning

single

address

space

systems

Ian

and

Aline

were

of

considerable

value

concerning

English

and

French

writing

respectively

Julien

for

his

helping

hand

when

I

arrived

at

INRIA

Guillaume

for

his

aid

with

the

WWW

and

Nelly

for

her

valuable

logistic

support

I

have

also

benefited

from

discussions

with

people

outside

Project

SOR

I

thank

them

all

Laurent

Amsaleg

for

the

debates

we

had

concerning

garbage

collection

Hank

Levy

for

our

discussions

concerning

distributed

shared

memory

Marcin

and

Marie-H

el?ene

for

helping

me

with

state-machines

and

Fabio

for

having

showed

me

the

usefulness

of

state-charts

I

'm

grateful

to

JNICT

-LRB-

programs

Ci

encia

and

PRAXIS

XXI

-RRB-

that

provided

the

scholarship

which

allowed

me

to

undertake

this

research

and

INESC

for

having

encouraged

me

to

pursuit

a

Ph.D.

program

outside

Portugal

I

'm

grateful

to

INRIA

that

provided

me

a

wonderful

environment

for

doing

my

work

The

resources

available

allowed

me

to

develop

my

research

with

great

freedom

and

efficiency

v

I

thank

all

my

family

for

the

extraordinary

support

they

always

gave

me

in

spite

of

the

distance

separating

us

In

particular

I

thank

my

parents

for

helping

me

to

discover

since

I

was

a

young

boy

the

pleasures

of

learning

and

researching

they

provided

me

a

home

where

such

pursuits

were

encouraged

Finally

thanks

are

not

adequate

for

the

love

patience

encouragement

and

understanding

that

my

wife

has

given

to

me

Paulo

Ferreira

vi

Avant-propos

Cette

th?ese

est

le

r?esultat

d'un

effort

de

recherche

effectu?e

a

l'INRIA

Elle

a

deux

parties

la

premi?ere

partie

est

un

long

r?esum?e

en

Fran?cais

de

la

th?ese

la

deuxi?eme

partie

est

la

version

compl?ete

de

la

th?ese

en

Anglais

Quelques

aspects

de

cette

th?ese

sont

present?es

dans

les

articles

suivants

-LRB-

par

ordre

chronologique

-RRB-

Paulo

Ferreira

and

Marc

Shapiro

Distribution

and

Persistence

in

Multiple

and

Heterogeneous

Address

Spaces

Proceedings

of

the

International

Workshop

on

Object-Orientation

in

Operating

Systems

-LRB-

IWOOOS

-RRB-

Asheville

NC

-LRB-

USA

-RRB-

December

Paulo

Ferreira

and

Marc

Shapiro

Garbage

Collection

of

Persistent

Objects

in

Distributed

Shared

Memory

Proceedings

of

the

International

Workshop

on

Persistent

Object

Systems

-LRB-

POS

-RRB-

Tarascon

-LRB-

France

-RRB-

September

Paulo

Ferreira

and

Marc

Shapiro

Garbage

Collection

and

DSM

Consistency

Proceedings

of

the

First

Symposium

on

Operating

Systems

Design

and

Implementation

-LRB-

OSDI

-RRB-

Monterey

CA

-LRB-

USA

-RRB-

November

Paulo

Ferreira

and

Marc

Shapiro

Garbage

Collection

in

the

Larchant

Persistent

Distributed

Shared

Store

Proceedings

of

the

Workshop

on

Future

Trends

in

Distributed

Computing

Systems

-LRB-

FTDCS

-RRB-

Cheju

Island

-LRB-

Republic

of

Korea

-RRB-

August

Marc

Shapiro

and

Paulo

Ferreira

Larchant-RDOSS

a

Distributed

Shared

Persistent

Memory

and

its

Garbage

Collector

Proceedings

of

the

International

Workshop

on

Distributed

Algorithms

-LRB-

WDAG

-RRB-

Le

Mont

St.

Michel

-LRB-

France

-RRB-

September

Paulo

Ferreira

and

Marc

Shapiro

Larchant

Persistence

by

Reachability

in

Distributed

Shared

Memory

through

Garbage

Collection

Proceedings

of

the

International

Conference

on

Distributed

Computing

Systems

-LRB-

ICDCS

-RRB-

Hong

Kong

May

Ces

articles

peuvent

etre

obtenus

via

World

Wide

Web

a

l'address

vii

Foreword

This

thesis

documents

the

result

of

a

research

effort

done

at

INRIA

It

has

two

parts

the

first

is

a

long

abstract

in

French

of

the

whole

thesis

the

second

part

is

the

thesis

itself

in

English

Some

results

presented

in

this

thesis

appear

in

the

following

articles

-LRB-

chronological

order

-RRB-

Paulo

Ferreira

and

Marc

Shapiro

Distribution

and

Persistence

in

Multiple

and

Heterogeneous

Address

Spaces

In

Proceedings

of

the

International

Workshop

on

Object-Orientation

in

Operating

Systems

-LRB-

IWOOOS

-RRB-

Asheville

NC

-LRB-

USA

-RRB-

December

Paulo

Ferreira

and

Marc

Shapiro

Garbage

Collection

of

Persistent

Objects

in

Distributed

Shared

Memory

In

Proceedings

of

the

International

Workshop

on

Persistent

Object

Systems

-LRB-

POS

-RRB-

Tarascon

-LRB-

France

-RRB-

September

Paulo

Ferreira

and

Marc

Shapiro

Garbage

Collection

and

DSM

Consistency

In

Proceedings

of

the

First

Symposium

on

Operating

Systems

Design

and

Implementation

-LRB-

OSDI

-RRB-

Monterey

CA

-LRB-

USA

-RRB-

November

Paulo

Ferreira

and

Marc

Shapiro

Garbage

Collection

in

the

Larchant

Persistent

Distributed

Shared

Store

In

Proceedings

of

the

Workshop

on

Future

Trends

in

Distributed

Computing

Systems

-LRB-

FTDCS

-RRB-

Cheju

Island

-LRB-

Republic

of

Korea

-RRB-

August

Marc

Shapiro

and

Paulo

Ferreira

Larchant-RDOSS

a

Distributed

Shared

Persistent

Memory

and

its

Garbage

Collector

In

Proceedings

of

the

International

Workshop

on

Distributed

Algorithms

-LRB-

WDAG

-RRB-

Le

Mont

St.

Michel

-LRB-

France

-RRB-

September

Paulo

Ferreira

and

Marc

Shapiro

Larchant

Persistence

by

Reachability

in

Distributed

Shared

Memory

through

Garbage

Collection

In

Proceedings

of

the

International

Conference

on

Distributed

Computing

Systems

-LRB-

ICDCS

-RRB-

Hong

Kong

May

These

articles

can

be

freely

obtained

on

the

World

Wide

Web

at

the

URL

viii

R?esum?e

Le

mod?ele

de

Larchant

est

celui

d'un

Espace

d'Adressage

Partag?e

-LRB-

comprenant

tous

les

sites

du

r?eseau

ainsi

que

la

m?emoire

secondaire

-RRB-

bas?e

sur

la

Persistance

par

Atteignabilit?e

Pour

donner

l'illusion

d'un

espace

d'adressage

partag?e

par

tous

les

sites

du

r?eseau

Larchant

met

en

oeuvre

un

m?ecanisme

de

M?emoire

Partag?ee

R?epartie

La

persistance

par

atteignabilit?e

est

apport?ee

par

un

Glaneur

de

Cellules

-LRB-

GC

ou

encore

ramasse-miettes

-RRB-

qui

parcourt

le

graphe

d'objets

a

partir

de

la

racine

de

persistance

Les

objets

non

atteignables

sont

appel?es

miettes

Le

GC

nettoie

la

m?emoire

en

recyclant

l'espace

occup?e

par

les

miettes

Le

glanage

dans

les

syst?emes

r?epartis

a

grande

echelle

a

et?e

consid?er?e

jusqu

a

peu

comme

infaisable

Les

principaux

probl?emes

rencontr?es

sont

dus

a

l'incoh

erence

des

donn?ees

et

a

des

consid?erations

portant

sur

l'

echelle

et

les

performances

Cette

th?ese

pr?esente

les

solutions

apport?ees

par

Larchant

pour

r?esoudre

ces

probl?emes

Le

GC

de

Larchant

combine

tra?cage

et

listage

de

r?ef?erences

Le

GC

fait

du

tra?cage

sur

les

partitions

en

m?emoire

Le

GC

fait

du

listage

sur

les

r?ef?erences

qui

si

elles

etaient

parcourues

impliqueraient

des

entr?ees

sorties

Le

ramassage

de

miettes

est

totalement

dissoci?e

de

la

coh?erence

i.e.

le

glanage

est

effectu?e

m

eme

s

il

y

a

des

r?epliques

incoh?erents

sur

le

site

o?u

le

GC

s

ex?ecute

Le

glaneur

fonctionne

de

fa?con

concurrente

et

asynchrone

par

rapport

aux

applications

Les

fronti?eres

de

listage

des

r?ef?erences

changent

dynamiquement

et

ind?ependamment

sur

chaque

site

de

mani?ere

a

nettoyer

les

cycles

de

miettes

L'algorithme

du

GC

a

et?e

prouv?e

correct

-LRB-

innocuit?e

et

vivacit?e

-RRB-

par

induction

Les

r?esultats

de

performance

du

GC

de

Larchant

montrent

que

nos

objectifs

d'orthogonalit

e

par

rapport

a

l'incoh

erence

et

a

l'

echelle

ont

et?e

atteints

Mots-cl

es

syst?emes

r?epartis

syst?emes

a

objets

persistance

par

atteignabilit?e

m?emoire

partag?ee

r?epartie

glaneur

de

cellules

r?eparti

ix

Abstract

The

model

of

Larchant

is

that

of

a

Shared

Address

Space

-LRB-

spanning

every

site

in

a

network

including

secondary

storage

-RRB-

with

Persistence

By

Reachability

To

provide

the

illusion

of

a

shared

address

space

across

the

network

despite

the

fact

that

site

memories

are

disjoint

Larchant

implements

a

distributed

shared

memory

mechanism

Reachability

is

accessed

by

tracing

the

pointer

graph

starting

from

the

persistent

root

and

reclaiming

unreachable

objects

This

is

the

task

of

Garbage

Collection

-LRB-

GC

-RRB-

GC

was

until

recently

thought

to

be

intractable

in

a

large-scale

system

due

to

problems

of

scale

incoherence

asynchrony

and

performance

This

thesis

presents

the

solutions

that

Larchant

proposes

to

these

problems

The

GC

algorithm

in

Larchant

combines

tracing

and

reference-listing

It

traces

whenever

economically

feasible

i.e.

as

long

as

the

memory

subset

being

collected

remains

local

to

a

site

and

counts

references

that

would

cost

I/O

traffic

to

trace

GC

is

orthogonal

to

coherence

i.e.

makes

progress

even

if

only

incoherent

replicas

are

locally

available

The

garbage

collector

runs

concurrently

and

asynchronously

to

applications

The

reference-listing

boundary

changes

dynamically

and

seamlessly

and

independently

at

each

site

in

order

to

collect

cycles

of

unreachable

objects

We

prove

formally

that

our

GC

algorithm

is

correct

i.e.

it

is

safe

and

live

The

performance

results

from

our

Larchant

prototype

show

that

our

design

goals

-LRB-

scalability

coherence

orthogonality

and

good

performance

-RRB-

are

fulfilled

x

Table

des

Mati?eres

Contents

I

Larchant

ramasse-miettes

dans

une

m?emoire

partag?ee

r?epartie

avec

persistance

par

atteignabilit?e

-LRB-

Long

r?esum?e

Long

abstract

-RRB-

Introduction

Panorama

des

algorithmes

de

GC

existants

Mod?ele

de

Larchant

Mod?ele

de

m?emoire

Mod?ele

du

mutateur

Mod?ele

de

coh?erence

Mod?ele

du

GC

GC

dans

Larchant

Tra?cage

Parcours

D?eplacement

et

correction

Ramassage

des

cycles

de

miettes

inter-bunch

Comptage

de

r?ef?erences

Asynchronisme

Impl?ementation

du

GC

dans

Larchant

Asynchronisme

Performance

des

algorithmes

de

GC

Conclusion

xi

II

Larchant

garbage

collection

in

a

cached

distributed

shared

store

with

persistence

by

reachability

Introduction

Thesis

Main

Issues

Thesis

Main

Contributions

Thesis

Roadmap

Overview

of

GC

Algorithms

Terminology

Basic

Uniprocessor

GC

Algorithms

Reference

Counting

Tracing

Mark-and-Sweep

Copy

Functioning

Modes

Incremental

Tracing

Partitioned

Tracing

System

Requirements

Distributed

GC

Algorithms

Reference

Counting

Weighted

Reference

Counting

Indirect

Reference

Counting

Reference

Listing

Cycles

of

Garbage

Tracing

Mark-and-Sweep

with

Timestamps

Logically

Centralized

Tracing

Group

Tracing

GC

in

Transactional

Systems

Transactional

Reference

Listing

Transactional

Mark-and-Sweep

Atomic

Copy

Replicated

Copy

GC

in

File

Systems

GC

in

Multiprocessors

xii

GC

in

DSM

Systems

Discussion

GC

Considerations

GC

in

Larchant

Summary

Larchant

Model

Motivation

Current

Solutions

for

Distributed

Data

Sharing

File

Systems

Object-Oriented

Databases

RPC-based

Systems

DSM

Systems

Cached

Distributed

Shared

Stores

Larchant

Model

Memory

Model

Network

Model

Process

Model

Mutator

Model

Coherence

Model

Mapping

of

Real

Systems

to

the

Larchant

Model

Summary

Garbage

Collection

in

Larchant

Problems

and

Outline

of

Solutions

Scalability

Coherence

Interference

Extra

I/O

Performance

GC

Algorithms

General

Overview

GC

Model

Stubs

and

Scions

Tracing

a

Bunch

Union

Rule

Intra-Bunch

GC

xiii

GC

without

Replication

GC

with

Replication

Union

Rule

Scan

Move

and

Patch

Flip

Cross-Bunch

GC

GC

without

Replication

GC

with

Replication

Union

Rule

Promptness

Interaction

of

Intra-Bunch

and

Cross-Bunch

GC

Reclaiming

Cross-Bunch

Cycles

of

Garbage

GC

Algorithm

Discussion

Summary

of

GC

Operations

Correctness

of

Intra-Bunch

GC

Safety

and

Liveness

Correctness

of

the

Cross-Bunch

GC

Cross-Bunch

GC

without

Replication

Safety

Algorithm

Proof

of

Safety

Liveness

Cross-Bunch

GC

with

Replication

Safety

Algorithm

Proof

of

Safety

Liveness

Summary

Implementation

of

GC

in

Larchant

Architecture

Basics

Applications

Cache

Manager

xiv

Object

Repository

Persistent

Root

Address

Space

Management

Garbage

Collection

Prototype

Implementation

Processes

Threads

and

Ports

Bunches

and

Objects

Coherence

Engine

Intra-Bunch

Garbage

Collector

Mark-and-Sweep

Copy

Detection

of

GC-dirty

Objects

Cross-Bunch

Garbage

Collector

GC

and

Coherence

Granularity

Programming

Restrictions

Summary

Performance

of

GC

Algorithms

Basics

Mark-and-Sweep

without

Replication

GC

Pause

Time

in

GC-only

Mode

GC

Pause

Time

in

Concurrent

Mode

Copy

without

Replication

GC

Pause

Time

in

GC-only

Mode

GC

Pause

Time

in

Concurrent

Mode

GC

Pause

Time

with

Replication

Mark-and-Sweep

in

GC-only

and

Concurrent

Modes

Copy

in

GC-only

and

Concurrent

Modes

Cross-Bunch

GC

Macro-Benchmarks

Summary

Conclusion

GC

Issues

and

Solutions

Achievements

Future

Work

Bibliography

xv

Index

des

Figures

List

of

Figures

I

Larchant

ramasse-miettes

dans

une

m?emoire

partag?ee

r?epartie

avec

persistance

par

atteignabilit?e

-LRB-

Long

r?esum?e

Long

abstract

-RRB-

Souches

et

scions

Situation

de

d?epart

Le

message

d'union

transporte

la

d?ependance

causale

La

superposition

assure

la

causalit?e

Temps

de

pause

du

GC

par

copie

concurrente

Temps

de

pause

du

GC

par

copie

non

concurrente

II

Larchant

garbage

collection

in

a

cached

distributed

shared

store

with

persistence

by

reachability

Uniprocessor

reference

counting

algorithm

Uniprocessor

mark-and-sweep

algorithm

Uniprocessor

copy

algorithm

Cheney

algorithm

Unsafe

tracing

due

to

mutator

activity

Decrement/increment

race

condition

Increment/decrement

race

condition

Weighted

reference

counting

Problem

with

weight

reference

counting

Indirect

reference

counting

Stubs

and

scions

describing

cross-partition

references

xvi

Distances

of

objects

in

a

cycle

of

garbage

Transactional

reference

listing

Problem

with

transactional

mark-and-sweep

Problem

with

copy

GC

in

presence

of

failures

A

bunch

with

objects

inside

Prototypical

example

of

mutator

execution

Two

bunches

containing

objects

stubs

and

scions

Bunch

replicas

with

outOwnerPtrs

and

inOwnerPtrs

Union

rule

-LRB-

single

bunch

-RRB-

Union

rule

-LRB-

two

bunches

-RRB-

Creation

and

destruction

of

cross-bunch

pointers

-LRB-

no

replication

-RRB-

Timeline

for

Figure

Creation

and

destruction

of

cross-bunch

pointers

-LRB-

replication

-RRB-

Timeline

for

Figure

Timeline

improved

with

union

message

Need

for

interleaved

intra-bunch

and

cross-bunch

collection

Mutators

intra-bunch

and

cross-bunch

collectors

Reclaiming

cycles

of

garbage

Intra-bunch

GC

represented

as

a

tricolor

algorithm

Larchant

architecture

High-level

view

of

an

application

process

Larchant

implementation

State-machine

for

the

entry-consistency

protocol

State-chart

of

mark-and-sweep

GC

with

entry-consistency

State-machine

of

copy

collector

State-chart

of

copy

GC

and

entry-consistency

Promptness

timeline

for

prototypical

example

Timeline

improved

with

union

message

GC

pause

time

of

GC-only

mark-and-sweep

algorithm

GC

pause

time

of

GC-only

mark-and-sweep

algorithm

-LRB-

cont

-RRB-

GC

pause

time

of

concurrent

mark-and-sweep

algorithm

GC

pause

time

of

GC-only

copy

algorithm

GC

pause

time

of

GC-only

copy

algorithm

-LRB-

cont

-RRB-

GC

pause

time

of

concurrent

copy

algorithm

xvii

Index

des

Tableaux

List

of

Tables

II

Larchant

garbage

collection

in

a

cached

distributed

shared

store

with

persistence

by

reachability

Summary

of

operations

in

the

Larchant

model

Time

it

takes

to

create

-LRB-

synchronously

-RRB-

stubs

and

scions

Time

it

takes

to

create

cross-bunch

pointers

and

the

corre

sponding

stub-scion

pairs

xviii

Partie/Part

I

Larchant

ramasse-miettes

dans

une

m?emoire

partag?ee

r?epartie

avec

persistance

par

atteignabilit?e

-LRB-

Long

r?esum?e

Long

abstract

-RRB-

Chapitre

Introduction

L'objectif

global

de

Larchant

est

d'offrir

un

stockage

persistant

r?eparti

-LRB-

PDS

Persistent

Distributed

Store

-RRB-

qui

facilite

le

partage

des

donn?ees

Larchant

garantit

que

l'acc

es

aux

donn?ees

a

partir

de

diff?erents

sites

et/ou

a

des

moments

diff?erents

reste

simple

et

efficace

De

plus

la

persistance

les

entr?ees

sorties

-LRB-

E/S

-RRB-

la

gestion

de

la

m?emoire

et

la

r?epartition

sont

transparents

et

automatiques

Avec

Larchant

les

programmeurs

peuvent

se

concentrer

sur

les

probl?emes

de

leurs

applications

sans

se

laisser

perturber

par

les

bogues

m?emoire

le

cache

la

coh?erence

les

acc?es

distants

les

E/S

etc.

Les

applications

type

destin?ees

a

Larchant

sont

les

syst?emes

de

CAO-FAO

le

multim?edia

les

bases

de

donn?ees

financi?eres

etc.

Le

mod?ele

de

Larchant

est

celui

d'un

espace

d'adressage

partag?e

-LSB-

-RSB-

-LRB-

s

etendant

sur

chaque

site

du

r?eseau

incluant

les

zones

de

stockage

secondaire

-RRB-

avec

la

persistance

par

atteignabilit?e

-LRB-

PBR

Persistance

By

Reachability

-RRB-

-LSB-

-RSB-

Ce

mod?ele

est

tr?es

attirant

par

la

simplicit?e

qu'il

apporte

au

programmeur

Pour

fournir

l'illusion

d'un

espace

d'adressage

partag?e

recouvrant

le

r?eseau

bien

que

les

m?emoires

des

sites

soient

disjointes

Larchant

met

en

oeuvre

un

m?ecanisme

de

m?emoire

partag?ee

r?epartie

-LRB-

DSM

Distributed

Shared

Memory

-RRB-

-LSB-

-RSB-

L'acc

es

a

une

donn?ee

n?ecessite

de

trouver

sa

r?ef?erence

par

navigation

en

partant

d'une

racine

persistante

bien

connue

-LRB-

par

ex.

un

serveur

de

noms

-RRB-

Dans

un

syst?eme

utilisant

la

PBR

-LSB-

-RSB-

un

objet

atteignable

depuis

la

racine

persistante

est

lui-m

eme

persistant

il

doit

ainsi

etre

stock?e

Un

objet

atteignable

par

transitivit?e

a

partir

de

la

racine

persistante

est

aussi

persistant

Les

objets

non

atteignables

ne

sont

plus

n?ecessaires

ils

peuvent

donc

etre

ramass?es

-LRB-

et

la

m?emoire

compact?ee

-RRB-

De

tels

objets

sont

appel?es

miettes

L'atteignabilit

e

est

confirm?ee

en

construisant

le

graphe

des

pointeurs

en

partant

de

la

racine

persistante

et

en

ramassant

les

objets

non

atteignables

Ce

processus

peut

etre

effectu?e

soit

par

une

gestion

manuelle

de

la

m?emoire

soit

automatiquement

par

un

Glaneur

de

Cellules

-LRB-

GC

ou

encore

ramasse-miettes

-RRB-

-LSB-

-RSB-

collector

en

Anglais

CHAPITRE

INTRODUCTION

La

gestion

manuelle

de

la

m?emoire

est

source

d'erreurs

-LRB-

par

ex.

les

pointeurs

invalides

et

les

fuites

de

m?emoire

-RRB-

qui

conduit

fr?equemment

a

la

violation

de

la

condition

fondamentale

d'int

egrit?e

r?ef?erentielle

suivre

un

pointeur

doit

toujours

etre

valide

i.e.

si

un

objet

persistant

x

pointe

vers

un

objet

y

alors

y

doit

aussi

etre

persistant

Le

GC

dans

les

syst?emes

r?epartis

a

grande

echelle

a

et?e

consid?er?e

jusqu

a

peu

comme

infaisable

Les

principaux

probl?emes

rencontr?es

sont

dus

a

l'incoh

erence

des

donn?ees

et

a

des

consid?erations

portant

sur

l'

echelle

et

les

performances

Cette

th?ese

pr?esente

les

solutions

apport?ees

par

Larchant

pour

resoudre

ces

probl?emes

L'algorithme

du

GC

dans

Larchant

associe

le

tra?cage

et

le

comptage

de

r?ef?erences

Il

trace

tant

que

c'est

economiquement

faisable

i.e.

tant

que

le

sous-ensemble

m?emoire

reste

local

a

un

site

et

il

compte

les

r?ef?erences

qui

demanderaient

un

trafic

d'

E/S

co

uteux

pour

un

tra?cage

Les

limites

du

comptage

de

r?ef?erences

changent

dynamiquement

et

de

fa?con

ind?ependante

sur

chaque

site

de

fa?con

a

collecter

-LRB-

ou

ramasser

-RRB-

les

cycles

d'objets

non

atteignables

Le

plus

important

est

que

les

sous-ensembles

m?emoire

r?epliqu?es

sont

collect?es

ind?ependamment

et

sans

interf?erence

avec

les

besoins

en

coh?erence

des

applications

Ce

travail

est

le

premier

a

proposer

un

algorithme

de

GC

pour

une

PDS

qui

soit

-LRB-

i

-RRB-

extensible

en

terme

d'

echelle

-LRB-

ii

-RRB-

orthogonal

a

la

coh?erence

i.e.

qu'il

progresse

m

eme

si

seules

des

r?epliques

incoh?erentes

sont

disponibles

localement

-LRB-

iii

-RRB-

totalement

asynchrone

par

rapport

aux

applications

et

-LRB-

iv

-RRB-

collecte

les

cycles

de

miettes

en

changeant

dynamiquement

les

fronti?eres

du

m?ecanisme

r?eparti

de

comptage

de

r?ef?erences

Cette

th?ese

est

organis?e

de

la

fa?con

suivante

Le

prochain

chapitre

dresse

un

panorama

des

techniques

traditionnelles

de

GC

Le

Chapitre

pr?esente

notre

mod?ele

des

programmes

applicatifs

les

protocoles

de

coh?erence

et

le

mod?ele

du

GC

Le

Chapitre

d?ecrit

les

algorithmes

de

tra?cage

et

de

comptage

de

r?ef?erences

dans

Larchant

Dans

le

Chapitre

nous

pr?esentons

bri?evement

l'impl

ementation

du

GC

dans

Larchant

Le

Chapitre

expose

une

etude

des

performances

de

nos

algorithmes

de

GC

Le

Chapitre

conclut

la

th?ese

par

un

r?esum?e

des

id?ees

les

plus

importantes

Chapitre

Panorama

des

algorithmes

de

GC

existants

Les

techniques

fondamentales

de

GC

sont

traditionnellement

regroup?ees

en

deux

grandes

familles

les

approches

fond?ees

sur

le

comptage

de

r?ef?erences

et

celles

s

appuyant

sur

un

tra?cage

du

graphe

d'objets

Dans

les

GC

fond?es

sur

le

comptage

de

r?ef?erences

-LRB-

reference

counting

en

Anglais

-RRB-

chaque

objet

compte

le

nombre

de

r?ef?erences

qui

le

d?esignent

-LSB-

-RSB-

Ce

compte

est

traditionnellement

repr?esent?e

par

un

entier

stock?e

dans

l'en-t

ete

de

chaque

objet

Le

compteur

d'un

objet

est

incr?ement?e

a

chaque

fois

que

sa

r?ef?erence

est

dupliqu?ee

Inversement

lorsqu'une

r?ef?erence

vers

un

objet

est

d?etruite

le

compteur

associ?e

est

d?ecr?ement?e

L'espace

m?emoire

occup?e

par

un

objet

est

ramass?e

d?es

que

son

compteur

de

r?ef?erences

passe

a

z?ero

Les

GC

fond?es

sur

un

tra?cage

effectuent

un

parcours

du

graphe

d'objets

depuis

la

racine

de

persistance

et

traversent

les

objets

en

suivant

successivement

leurs

r?ef?erences

Les

deux

techniques

el?ementaires

fond?ees

sur

le

tra?cage

sont

le

marquage-balayage

-LRB-

mark-and-sweep

en

Anglais

-RRB-

-LSB-

-RSB-

et

la

copie

des

objets

-LRB-

copy

en

Anglais

-RRB-

-LSB-

-RSB-

L'algorithme

de

marquage-balayage

fonctionne

de

la

mani?ere

suivante

Partant

de

la

racine

de

persistance

le

GC

distingue

les

objets

atteignables

des

miettes

en

marquant

les

objets

visit?es

lors

de

sa

travers?ee

du

graphe

Ensuite

le

GC

balaye

la

totalit?e

de

la

m?emoire

pour

trouver

puis

ramasser

tous

les

objets

non

marqu?es

L'algorithme

de

GC

par

copie

fonctionne

de

la

mani?ere

suivante

La

m?emoire

est

divis?ee

en

deux

demi-espaces

Durant

l'ex

ecution

normale

des

programmes

seul

le

demi-espace

appel?e

espace

d'origine

-LRB-

from-space

en

Anglais

-RRB-

est

utilis?e

alors

que

l'autre

demi-espace

appel?e

espace

destination

-LRB-

to-space

en

Anglais

-RRB-

est

conserv?e

vierge

Le

GC

copie

les

objets

atteignables

via

la

racine

de

persistance

depuis

l'espace

d'origine

vers

l'espace

destination

Lorsque

la

copie

s

ach?eve

le

r

ole

des

deux

demi-espaces

est

echang?e

atomiquement

L'espace

destination

devient

l'espace

d'origine

et

vice

versa

Il

existe

une

litt?erature

abondante

dans

le

domaine

du

GC

que

ce

soit

pour

les

CHAPITRE

PANORAMA

DES

ALGORITHMES

DE

GC

EXISTANTS

multiprocesseurs

-LSB-

-RSB-

ou

pour

des

syst?emes

client-serveur

r?epartis

-LRB-

voir

l'

etude

de

Plainfoss?e

-LSB-

-RSB-

-RRB-

En

revanche

peu

de

travaux

ont

et?e

r?ealis?es

sur

un

GC

dans

un

r?eseau

faiblement

coupl?e

avec

une

DSM

faiblement

coh?erente

La

diff?erence

fondamentale

entre

un

GC

dans

Larchant

et

un

GC

dans

un

multiprocesseur

est

l'

echelle

et

le

surco

ut

de

synchronisation

si

nous

appliquons

un

algorithme

de

GC

con?cu

pour

des

multiprocesseurs

-LRB-

par

ex.

Appel

-LSB-

-RSB-

-RRB-

a

notre

cas

le

surco

ut

serait

inacceptable

du

fait

des

co

uts

de

communication

et

de

synchronisation

Ces

co

uts

sont

dus

au

fait

que

les

algorithmes

de

GC

multiprocesseurs

actuels

supposent

implicitement

l'existence

d'objets

coh?erents

La

plupart

des

travaux

pr?ec?edents

sur

les

GC

r?epartis

-LSB-

-RSB-

consid?erent

que

les

processus

communiquent

par

messages

-LRB-

sans

m?emoire

partag?ee

-RRB-

en

utilisant

un

m?elange

de

tra?cage

et

de

comptage

Chaque

processus

trace

ses

pointeurs

internes

les

r?ef?erences

hors

des

fronti?eres

du

processus

sont

compt?ees

lorsqu'elles

sont

envoy?ees

dans

des

messages

Cependant

aucun

des

ces

travaux

n'accepte

un

m?ecanisme

de

m?emoire

r?epartie

partag?ee

dans

laquelle

des

applications

s

ex?ecutant

sur

diff?erents

sites

peuvent

acc?eder

a

de

multiples

r?epliques

du

m

eme

objet

de

fa?con

concurrente

comme

Larchant

le

permet

Finalement

les

travaux

pr?ec?edents

sur

le

ramassage

de

miettes

dans

un

DSM

sont

rares

et

ne

r?esolvent

pas

nos

probl?emes

puisqu'ils

supposent

des

objets

coh?erents

-LSB-

-RSB-

Chapitre

Mod?ele

de

Larchant

Nous

utilisons

un

vocabulaire

standard

de

la

litt?erature

sur

les

ramasse-miettes

-LSB-

-RSB-

Le

mutateur

est

le

programme

applicatif

qui

modifie

dynamiquement

le

graphe

des

pointeurs

il

cr?ee

des

objets

utilise

des

pointeurs

et

affecte

des

pointeurs

Un

effet

de

bord

de

l'affectation

de

pointeurs

est

que

certains

objets

deviennent

non

atteignables

Dans

un

syst?eme

r?eparti

le

mutateur

est

en

fait

compos?e

de

multiples

threads

ind?ependantes

s

ex?ecutant

sur

diff?erents

sites

par

extension

nous

appelons

chacune

de

ces

threads

un

mutateur

Le

collecteur

est

le

composant

syst?eme

qui

identifie

et

ramasse

les

miettes

cr?e?ees

par

le

mutateur

Notre

collecteur

est

compos?e

d'un

nombre

de

threads

s

ex?ecutant

sur

diff?erents

sites

nous

appelons

chacune

d'elles

un

collecteur

Notre

algorithme

de

GC

est

bas?e

sur

un

mod?ele

tr?es

simplifi?e

Ainsi

plusieurs

op?erations

traditionnellement

associ?ees

a

la

gestion

de

la

coh?erence

ne

sont

pas

pr?esentes

dans

notre

mod?ele

certaines

parce

qu'elles

ne

sont

pas

li?ees

au

GC

-LRB-

par

exemple

la

lecture

et

l'

ecriture

sans

utiliser

de

pointeurs

-RRB-

et

d'autres

parce

que

notre

algorithme

leur

est

ind?ependant

-LRB-

par

exemple

les

verrous

de

la

DSM

-RRB-

En

particulier

l'algorithme

de

GC

tol?ere

des

ecritures

arbitraires

et

des

donn?ees

incoh?erentes

La

simplicit?e

du

mod?ele

garantit

que

l'algorithme

de

GC

s

applique

a

une

large

vari?et?e

de

syst?emes

de

stockage

partag?es

Mod?ele

de

m?emoire

La

m?emoire

est

d?ecompos?ee

en

deux

niveaux

de

granularit?e

-LRB-

i

-RRB-

L'objet

est

l'unit

e

d'allocation

de

d?esallocation

et

d'identification

Il

est

aussi

l'unit

e

de

coh?erence

et

de

propagation

des

mises

a

jour

-LRB-

i.e.

la

diss?emination

de

la

r?eplique

la

plus

a

jour

d'un

objet

-RRB-

Un

objet

peut

contenir

plusieurs

pointeurs

Soit

un

pointeur

est

nul

soit

il

pointe

vers

un

objet

-LRB-

ii

-RRB-

Un

bunch

est

l'unit

e

de

cache

et

de

rassemblement

Il

contient

un

nombre

quelconque

d'objets

D?esormais

les

objets

seront

not?es

x

y

z

etc.

L'adresse

de

l'objet

x

sera

not?ee

@x

Les

bunches

seront

not?es

en

majuscules

B

C

etc.

Puisque

toute

structure

CHAPITRE

MOD?ELE

DE

LARCHANT

peut

etre

r?epliqu?ee

-LRB-

etre

en

cache

-RRB-

sur

de

multiples

sites

nous

distinguons

les

r?epliques

a

l'aide

d'un

indice

de

site

par

ex.

xi

xj

pour

les

r?epliques

de

x

observ?ees

respectivement

sur

les

sites

i

et

j.

Une

variable

pointeur

ptr

dans

un

objet

x

sera

not?ee

x.ptr

Afin

de

simplifier

la

notation

nous

allons

supposer

que

les

objets

n'ont

qu'un

seul

pointeur

et

identifier

x.ptr

par

x.

Mod?ele

du

mutateur

Un

mutateur

s

ex?ecutant

sur

le

site

i

voit

l'objet

x

au

travers

de

la

r?eplique

pr?esente

dans

le

cache

du

site

xi

Tout

mutateur

peut

ecrire

ou

lire

un

objet

x

vers

lequel

il

poss?ede

un

pointeur

dans

son

cache

Pour

des

questions

de

GC

l'op

eration

pertinente

ex?ecut?ee

par

les

mutateurs

est

l'affectation

d'une

variable

pointeur

au

sein

d'un

objet

Une

op?eration

d'affectation

effectu?ee

sur

le

site

i

r?ealisant

une

lecture

sur

l'objet

y

et

une

ecriture

sur

x

est

not?ee

x

y

i.

Cette

op?eration

est

atomique

en

local

uniquement

i.e.

la

lecture

et

l'

ecriture

des

r?epliques

locales

sont

indivisibles

Il

n'est

pas

n?ecessaire

que

cette

op?eration

soit

atomique

pour

les

r?epliques

distantes

Notez

que

l'op

eration

d'affectation

peut

conduire

a

la

destruction

d'un

pointeur

et

a

la

cr?eation

d'un

nouveau

ce

qui

est

effectu?e

de

fa?con

impr?evisible

par

les

mutateurs

modifiant

ainsi

l'atteignabilit

e

des

objets

Mod?ele

de

coh?erence

Cette

section

identifie

les

op?erations

de

coh?erence

ayant

un

rapport

avec

le

GC

De

fa?con

a

accepter

diff?erents

mod?eles

de

coh?erence

nous

n'imposons

pas

le

moment

auquel

de

telles

op?erations

ont

lieu

-LRB-

En

pratique

les

op?erations

de

coh?erence

sont

provoqu?ees

par

l'activit

e

du

mutateur

-RRB-

Notre

supposition

minime

est

que

a

tout

moment

chaque

objet

ne

poss?ede

qu'un

seul

propri?etaire

Le

propri?etaire

d'un

objet

est

le

seul

site

autoris?e

a

diss?eminer

sa

valeur

aux

autres

sites

Cette

diss?emination

est

effectu?ee

par

un

message

de

propagation

La

propagation

de

x

a

partir

de

son

site

propri?etaire

i

au

site

j

fait

que

xj

devient

egal

au

xi

propag?e

Notez

que

cette

op?eration

n'exclut

pas

les

ecritures

concurrentes

-LRB-

ou

les

lectures

-RRB-

effectu?ees

sur

les

autres

r?epliques

de

x.

Le

propri?etaire

d'un

objet

peut

changer

Ce

changement

s

effectue

par

un

message

de

propri?et?e

le

transfert

de

propri?et?e

de

x

du

site

i

vers

le

site

j

fait

que

le

site

j

est

le

nouveau

propri?etaire

de

x.

D'autre

op?erations

de

coh?erence

peuvent

etre

construites

comme

des

cas

particuliers

des

pr?ec?edentes

Par

exemple

l'invalidation

du

cache

se

fait

en

recevant

un

message

de

propagation

contenant

la

valeur

nulle

CHAPITRE

MOD?ELE

DE

LARCHANT

souches

bunch

B

bunch

C

z

y

x

scions

scions

objets

souches

objets

Figure

Souches

et

scions

Mod?ele

du

GC

Un

bunch

conserve

la

trace

des

pointeurs

inter-bunch

de

fa?con

a

permettre

son

ramassage

ind?ependant

Un

pointeur

sortant

est

d?ecrit

par

une

souche

et

un

pointeur

entrant

par

un

-LRB-

voir

la

Figure

-RRB-

Une

souche

identifie

le

scion

correspondant

et

vice

versa

Chaque

r?eplique

d'un

bunch

poss?ede

son

propre

ensemble

de

souches

et

de

scions

Nous

donnons

ici

sans

le

justifier

la

liste

des

op?erations

li?ees

a

notre

algorithme

de

GC

-LRB-

pour

les

techniques

de

marquage-balayage

et

de

copie

-LSB-

-RSB-

-RRB-

ffl

create.scion

-LRB-

Bx

Cy

-RRB-

cr?eer

un

scion

dans

C

qui

d?ecrit

le

pointeur

interbunch

de

x

dans

B

vers

y

dans

C.

ffl

delete.scion

-LRB-

Bx

Cy

-RRB-

d?etruire

le

scion

dans

C

qui

d?ecrit

le

pointeur

interbunch

de

x

dans

B

vers

y

dans

C.

ffl

scani

-LRB-

x

-RRB-

parcourir

xi

afin

de

trouver

les

objets

qu'il

pointe

directement

ffl

move

-LRB-

y

@y

-RRB-

d?eplacer

l'objet

y

a

sa

nouvelle

adresse

@y

ffl

patch

-LRB-

x

@y

-RRB-

i

corriger

xi

avec

la

nouvelle

adresse

@y

de

y.

Le

d?eclenchement

et

l'innocuit

e

de

ces

op?erations

seront

etudi?es

dans

le

Chapitre

Quelques

temps

apr?es

qu'un

mutateur

ait

cr?e?e

un

nouveau

pointeur

inter-bunch

le

collecteur

alloue

une

souche

dans

le

bunch

source

et

cr?ee

un

scion

dans

la

cible

De

fa?con

similaire

apr?es

la

disparition

d'un

pointeur

inter-bunch

le

structures

similaires

trouv?ees

dans

le

syst?eme

a

passage

de

messages

des

cha

nes

de

SSP

-LRB-

Stub-Scion

Pair

-RRB-

-LSB-

-RSB-

ou

cha

ne

de

paires

souches-scions

ont

inspir?e

les

noms

de

souche

-LRB-

stub

-RRB-

et

de

scion

Contrairement

aux

cha

nes

de

SSP

les

souches

et

les

scions

de

Larchant

ne

sont

pas

des

indirections

participant

au

processus

du

mutateur

mais

simplement

des

structures

de

donn?ees

auxiliaires

d?ecrivant

les

pointeurs

inter-bunch

CHAPITRE

MOD?ELE

DE

LARCHANT

collecteur

peut

finir

par

d?esallouer

la

souche

correspondante

et

d?etruire

le

scion

cible

Le

nombre

de

pointeurs

inter-bunch

sur

un

objet

est

approximativement

egal

au

nombre

de

scions

pour

cet

objet

La

boucle

principale

d'un

GC

par

tra?cage

utilise

l'op

eration

de

parcours

Pour

une

certaine

r?eplique

xi

cette

op?eration

d?etermine

les

autres

objets

point?es

par

xi

Un

collecteur

par

copie

d?eplace

les

objets

-LRB-

pour

compacter

le

m?emoire

et

r?eduire

la

fragmentation

-RRB-

et

corrige

les

pointeurs

avec

les

nouvelles

adresses

Notez

que

les

op?erations

de

parcours

et

de

correction

sont

locales

a

un

site

i.e.

elles

sont

appliqu?ees

a

une

r?eplique

locale

et

elles

ne

n?ecessitent

pas

une

op?eration

de

propagation

en

d?epit

des

lectures

et

des

ecritures

implicites

sur

l'objet

concern?e

-LRB-

ainsi

les

r?epliques

locales

peuvent

etre

incoh?erentes

-RRB-

D'un

autre

c

ot?e

l'op

eration

de

d?eplacement

s

applique

a

chaque

r?eplique

d'un

objet

Comme

nous

le

d?ecrivons

dans

le

Chapitre

toutes

les

op?erations

du

GC

ne

n?ecessitent

pas

des

donn?ees

coh?erentes

Dans

la

terminologie

associ?ee

aux

GCs

un

flip

d?esigne

le

moment

pendant

lequel

le

mutateur

est

interrompu

pendant

le

flip

le

collecteur

-LRB-

pour

l'algorithme

de

marquage-balayage

ou

de

copie

-RRB-

effectue

certaines

op?erations

ayant

pour

but

de

terminer

la

collecte

Par

exemple

pendant

le

flip

un

collecteur

marquagebalayage

reparcourt

les

objets

qui

apr?es

avoir

et?e

parcourus

ont

et?e

modifi?es

du

fait

d'une

activit?e

concurrente

du

mutateur

Enfin

un

objet

est

sale

apr?es

avoir

et?e

modifi?e

par

une

op?eration

d'affectation

-LRB-

du

mutateur

-RRB-

il

reste

sale

tant

qu'il

n

a

pas

et?e

parcouru

Un

bunch

est

sale

s

il

contient

un

objet

sale

Chapitre

GC

dans

Larchant

Dans

l'id

eal

un

algorithme

de

GC

devrait

etre

complet

i.e.

devrait

finir

par

collecter

toutes

les

donn?ees

non

atteignables

Les

seuls

algorithmes

que

l'on

peut

prouver

complets

sont

bas?es

sur

le

tra?cage

et

utilisent

une

synchronisation

globale

Ce

principe

n'est

pas

extensible

g?en?ere

un

grande

quantit?e

d'

E/S

et

perturbe

les

applications

D'un

autre

c

ot?e

les

algorithmes

de

comptage

de

r?ef?erences

sont

extensibles

mais

ils

sont

incomplets

puisqu'ils

ne

collectent

pas

les

cycles

de

miettes

Apparemment

c'est

sans

espoir

Mais

comme

nous

allons

le

voir

il

est

possible

d'

eviter

le

tra?cage

global

et

ses

inconv?enients

Nous

affirmons

que

la

compl?etude

parfaite

n'est

pas

faisable

dans

un

syst?eme

de

grande

echelle

Nous

proposons

donc

une

solution

approximative

qui

ne

peut

etre

prouv?ee

comme

compl?ete

mais

que

nous

croyons

appropri?ee

pour

les

situations

r?eelles

Elle

fonctionne

en

associant

le

tra?cage

et

le

comptage

de

r?ef?erences

Le

GC

dans

Larchant

s

approche

d'un

tra?cage

global

avec

une

s?erie

de

tra?cages

locaux

sans

synchronisation

et

partitionn?es

Chaque

bunch

est

collect?e

sur

le

site

o?u

il

est

mis

en

cache

avec

un

algorithme

de

tra?cage

ind?ependamment

du

reste

de

la

m?emoire

De

plus

si

un

bunch

est

r?epliqu?e

chacune

des

ses

r?epliques

est

aussi

collect?ee

de

fa?con

ind?ependante

avec

le

m

eme

algorithme

En

consid?erant

les

scions

d'un

bunch

comme

des

racines

int?erimaires

un

bunch

peut

etre

collect?e

ind?ependamment

des

autres

La

collecte

d'une

r?eplique

d'un

bunch

-LRB-

collecteur

intra-bunch

-RRB-

s

effectue

comme

suit

Tout

objet

point?e

directement

a

partir

d'un

scion

est

consid?er?e

comme

atteignable

et

on

recherche

ses

pointeurs

Si

un

objet

atteignable

pointe

vers

un

autre

objet

du

m

eme

bunch

le

collecteur

intra-bunch

consid?ere

de

fa?con

transitive

l'objet

point?e

atteignable

S'il

pointe

en

dehors

du

bunch

englobant

le

collecteur

alloue

une

souche

Ainsi

le

r?esultat

de

la

collecte

d'un

bunch

est

un

ensemble

d'objets

atteignables

et

un

nouvel

ensemble

de

souches

Les

objets

non

compris

dans

l'ensemble

sont

des

miettes

Lorsqu'une

collecte

intra-bunch

est

termin?ee

le

collecteur

inter-bunch

-LRB-

algorithme

de

comptage

de

r?ef?erences

-RRB-

compare

le

nouvel

ensemble

de

souches

avec

l'ancien

-LRB-

r?esultant

d'un

pr?ec?edent

GC

intra-bunch

-RRB-

Les

souches

qui

n'existaient

CHAPITRE

GC

DANS

LARCHANT

pas

pr?ec?edemment

indiquent

qu'un

nouveau

pointeur

inter-bunch

sortant

a

et?e

cr?e?e

par

le

mutateur

Les

souches

qui

ont

disparu

-LRB-

i.e.

qui

ne

sont

plus

dans

le

nouvel

ensemble

de

souches

-RRB-

indiquent

un

pointeur

inter-bunch

sortant

qui

n'existe

plus

Pour

chaque

nouvelle

souche

allou?ee

-LRB-

par

le

collecteur

intra-bunch

-RRB-

le

collecteur

inter-bunch

proc?ede

a

la

cr?eation

du

scion

correspondant

Pour

chaque

souche

qui

n'appartient

plus

a

l'ensemble

le

collecteur

inter-bunch

demande

la

destruction

du

scion

cible

correspondant

Ainsi

le

collecteur

inter-bunch

d?etruit

les

scions

autorisant

les

futures

collectes

intra-bunch

a

ramasser

les

objets

qui

n

etaient

atteignables

que

parce

qu'ils

etaient

point?es

par

les

scions

d?etruits

-LRB-

Comme

nous

le

verrons

dans

le

Chapitre

les

op?erations

de

cr?eation

et

de

destruction

sont

asynchrones

par

rapport

aux

mutateurs

-RRB-

Tra?cage

La

principale

difficult?e

des

algorithmes

de

collecte

intra-bunch

est

de

ramasser

un

bunch

-LRB-

concurremment

avec

les

mutateurs

-RRB-

sans

n?ecessiter

d'op

erations

de

coh?erence

afin

d'

eviter

de

perturber

les

applications

Par

exemple

le

collecteur

intra-bunch

doit

etre

capable

de

progresser

sans

imposer

qu'une

r?eplique

d'un

objet

soit

a

jour

pour

etre

parcourue

Un

raisonnement

similaire

s

applique

aux

op?erations

de

d?eplacement

et

de

correction

Chaque

site

trace

ses

bunches

-LRB-

dans

le

cache

local

-RRB-

ind?ependamment

des

autres

sites

m

eme

si

ses

bunches

peuvent

etre

r?epliqu?es

ailleurs

Cela

pose

les

questions

suivantes

ffl

Les

collecteurs

doivent-ils

etre

synchronis?es

entre

eux

ffl

Les

parcours

demandent-ils

des

donn?ees

coh?erentes

ffl

Si

on

utilise

un

algorithme

de

GC

par

copie

-LCB-

Est-il

n?ecessaire

de

synchroniser

les

collecteurs

pour

d?ecider

o?u

d?eplacer

un

objet

-LCB-

Est-il

n?ecessaire

d'effectuer

certaines

op?erations

de

coh?erence

avant

-LRB-

ou

apr?es

-RRB-

avoir

d?eplac?e

un

objet

ou

avoir

corrig?e

ses

pointeurs

internes

-LCB-

Est-il

n?ecessaire

de

synchroniser

le

flip

Une

r?eponse

oui

a

n'importe

laquelle

de

ces

questions

aurait

un

impact

sur

l'extensibilit

e

et

l'efficacit

e.

Dans

le

reste

de

cette

section

nous

montrerons

que

aussi

surprenant

que

cela

puisse

para

tre

les

r?eponses

sont

toutes

non

sous

les

bonnes

conditions

Cela

a

pour

cons?equence

que

le

collecteur

intra-bunch

n'est

pas

en

comp?etition

avec

les

applications

pour

la

coh?erence

des

donn?ees

qu'il

n

y

pas

de

synchronisation

entre

les

collecteurs

et

les

mutateurs

ou

entre

les

CHAPITRE

GC

DANS

LARCHANT

diff?erents

collecteurs

et

que

les

messages

du

GC

sont

asynchrones

et

s

echangent

en

t

ache

de

fond

Le

prix

a

payer

est

un

certain

degr?e

de

conservation

et

certains

messages

devant

etre

d?elivr?es

en

ordre

causal

-LSB-

-RSB-

Parcours

Le

parcours

d'une

r?eplique

incoh?erente

ne

prend

bien

evidemment

pas

en

compte

les

ecritures

de

pointeurs

survenant

sur

un

autre

site

Ce

n'est

cependant

pas

un

probl?eme

En

fait

le

parcours

d'une

r?eplique

non

a

jour

r?esulte

simplement

en

une

d?ecision

plus

conservative

concernant

l'atteignabilit

e

des

objets

point?es

D'un

autre

c

ot?e

le

parcours

des

seules

r?epliques

les

plus

a

jour

d'un

objet

n'assure

pas

l'innocuit

e.

Un

objet

z

peut

ne

plus

etre

r?ef?erenc?e

par

la

r?eplique

la

plus

a

jour

d'un

objet

y

mais

etre

quand

m

eme

atteignable

a

partir

d'une

r?eplique

incoh?erente

de

y

-LRB-

y

et

z

etant

tous

deux

dans

le

m

eme

bunch

-RRB-

Ainsi

un

objet

ne

peut

etre

collect?e

qu'apr

es

etre

devenu

non

atteignable

a

partir

de

l'union

de

toutes

les

r?epliques

de

ses

objets

source

Ce

que

nous

appelons

la

r?egle

de

l'union

Nous

etudierons

ce

probl?eme

plus

en

d?etail

dans

la

Section

quand

nous

d?ecrirons

le

collecteur

inter-bunch

D?eplacement

et

correction

Dans

cette

section

nous

etudions

les

op?erations

de

d?eplacement

et

de

correction

par

rapport

a

la

coh?erence

Le

premier

probl?eme

est

d'

eviter

que

deux

sites

ne

d?eplacent

de

fa?con

concurrente

leurs

r?epliques

locales

du

m

eme

objet

vers

deux

endroits

diff?erents

Une

solution

evidente

a

ce

probl?eme

serait

que

le

site

qui

veut

d?eplacer

un

objet

demande

la

propri?et?e

de

l'objet

avant

de

le

d?eplacer

Il

est

clair

que

cette

solution

n'est

pas

satisfaisante

puisqu'elle

interf?ere

avec

les

besoins

de

coh?erence

des

applications

Une

solution

simple

n'interf

erant

pas

avec

les

besoins

de

coh?erence

des

applications

serait

la

suivante

le

site

propri?etaire

de

x

d?ecide

de

l'endroit

o?u

le

d?eplacer

les

sites

non

propri?etaires

d?eplaceraient

leurs

r?epliques

de

x

a

la

m

eme

adresse

apr?es

r?eception

d'un

message

de

d?eplacement

emis

par

le

propri?etaire

Un

objet

atteignable

serait

alors

d?eplac?e

de

la

fa?con

suivante

-LRB-

i

-RRB-

Le

site

i

le

propri?etaire

de

x

envoie

un

message

move

-LRB-

x

@x

-RRB-

aux

sites

poss?edant

dans

leur

cache

un

pointeur

vers

x

lui-m

eme

compris

-LRB-

ii

-RRB-

Un

site

j

recevant

ce

message

d?eplace

sa

r?eplique

xj

a

la

nouvelle

position

et

corrige

les

pointeurs

en

cons?equence

Notez

que

les

messages

de

d?eplacement

sont

d?elivr?es

en

t

ache

de

fond

Ainsi

ils

ne

perturbent

pas

les

applications

Le

second

probl?eme

est

le

suivant

le

collecteur

doit-il

acqu?erir

la

propri?et?e

d'un

objet

afin

de

corriger

un

de

ses

pointeurs

avec

la

nouvelle

position

de

la

cible

La

r?eponse

est

non

parce

que

cette

op?eration

n'est

visible

que

sur

le

site

o?u

elle

a

lieu

Notez

que

m

eme

avec

des

protocoles

de

coh?erence

qui

demandent

la

propri?et?e

d'un

objet

dans

lequel

on

souhaite

ecrire

-LRB-

par

ex.

la

entry-consistency

-LSB-

-RSB-

-RRB-

l'op

eration

de

correction

peut

etre

effectu?ee

sans

d?etenir

la

propri?et?e

d'un

objet

CHAPITRE

GC

DANS

LARCHANT

Enfin

le

collecteur

d'une

r?eplique

d'un

bunch

peut

r?ealiser

le

flip

de

fa?con

ind?ependante

-LRB-

sans

synchronisation

-RRB-

des

collecteurs

des

autres

r?epliques

du

m

eme

bunch

Le

collecteur

intra-bunch

peut

m

eme

proc?eder

au

flip

avant

d'avoir

re?cu

tous

les

messages

de

d?eplacement

concernant

les

objets

atteignables

qui

ne

sont

pas

d?etenus

localement

-LRB-

Le

d?eplacement

de

tels

objets

peut

etre

retard?e

jusqu

a

la

prochaine

ex?ecution

du

GC

-RRB-

Ramassage

des

cycles

de

miettes

inter-bunch

L'algorithme

de

GC

intra-bunch

est

complet

par

rapport

au

bunch

collect?e

i.e.

il

ramasse

toutes

les

miettes

qui

sont

enti?erement

dans

ce

bunch

Cependant

il

est

incomplet

par

rapport

aux

autres

bunches

puisqu'il

ne

ramasse

pas

un

cycle

de

miettes

qui

traverse

les

fronti?eres

du

bunch

-LRB-

par

ex.

le

cycle

inter-bunch

de

la

Figure

les

objets

x

y

et

z

-RRB-

Le

m

eme

algorithme

que

celui

qui

ramasse

un

seul

bunch

peut

etre

utilis?e

pour

ramasser

n'importe

quel

groupe

de

bunches

La

seule

diff?erence

par

rapport

au

ramassage

d'un

seul

bunch

r?eside

dans

le

tra?cage

d'un

groupe

-LRB-

i

-RRB-

les

scions

des

pointeurs

inter-bunch

internes

au

groupe

ne

sont

pas

consid?er?es

comme

des

racines

et

-LRB-

ii

-RRB-

le

tra?cage

traverse

les

fronti?eres

des

bunches

internes

au

groupe

Cet

algorithme

ramasse

un

cycle

inter-bunch

non

atteignable

a

partir

de

bunches

ext?erieurs

au

groupe

Par

exemple

dans

la

Figure

une

collecte

du

groupe

comprenant

les

bunches

B

et

C

ne

consid?ererait

pas

les

scions

pointant

vers

y

et

z

comme

des

membres

de

la

racine

Le

cycle

de

miettes

inter-bunch

constitu?e

par

les

objets

x

y

et

z

serait

alors

ramass?e

Ainsi

une

collecte

de

groupe

est

compl?ete

par

rapport

au

groupe

ramass?e

La

collecte

de

groupe

signifie

qu'un

ensemble

arbitraire

de

sous-ensembles

de

la

m?emoire

peut

etre

ramass?e

sur

un

seul

site

ind?ependamment

du

reste

de

la

m?emoire

Le

choix

du

groupe

a

collecter

est

heuristique

et

doit

maximiser

la

quantit?e

de

miettes

ramass?ees

et

minimiser

le

co

ut

Pour

former

de

tels

groupes

Larchant

utilise

une

heuristique

bas?ee

sur

la

localit?e

Un

groupe

contient

tous

les

bunches

actuellement

dans

le

cache

du

site

Cette

heuristique

evite

les

surco

uts

d'

E/S

Cependant

elle

n'autorise

pas

le

ramassage

des

cycles

inter-bunch

passant

par

des

bunches

qui

r?esident

en

partie

sur

le

disque

Ces

miettes

peuvent

etre

ramass?ees

par

une

heuristique

de

groupement

plus

agressive

Le

surco

ut

en

E/S

doit

etre

equilibr?e

par

le

gain

esp?er?e

Nous

souhaitons

exp?erimenter

la

heuristique

bas?ee

sur

la

localit?e

avec

un

grand

nombre

d'applications

et

r?ealiser

des

etudes

de

simulation

Ensuite

si

n?ecessaire

des

heuristiques

plus

complexes

seront

le

sujet

de

recherches

futures

Comptage

de

r?ef?erences

Nous

observons

tout

d'abord

que

le

parcours

de

la

r?eplique

du

propri?etaire

d'un

objet

y

seul

n'assure

pas

l'innocuit

e

par

exemple

un

certain

objet

z

pourrait

etre

atteignable

d'une

r?eplique

incoh?erente

yj

et

non

atteignable

a

partir

de

la

CHAPITRE

GC

DANS

LARCHANT

r?eplique

la

plus

a

jour

yi

Par

cons?equent

une

destruction

pourra

etre

lanc?ee

que

lorsque

l'objet

cible

z

deviendra

non

atteignable

a

partir

de

toutes

les

r?epliques

de

l'objet

source

y.

Comme

nous

l'avons

d?ej?a

mentionn?e

dans

la

section

c'est

ce

que

nous

appelons

la

r?egle

de

l'union

la

destruction

n'assure

pas

l'innocuit

e

qu'avec

l'union

des

souches

de

toutes

les

r?epliques

Un

site

non

propri?etaire

de

l'objet

y

informe

le

propri?etaire

de

y

de

l'existence

des

souches

par

un

message

d'union

Apr?es

qu'une

souche

-LRB-

du

fait

d'un

pointeur

sortant

a

partir

de

y

-RRB-

ait

disparu

de

tous

les

sites

ayant

dans

leur

cache

une

r?eplique

de

y

le

propri?etaire

envoie

une

destruction

qui

concerne

le

scion

correspondant

Plusieurs

protocoles

de

coh?erence

imposent

que

seul

le

propri?etaire

d'un

objet

y

peut

l'

ecrire

-LRB-

par

ex.

la

entry-consistency

-LSB-

-RSB-

-RRB-

Dans

ce

cas

un

objet

non

atteignable

a

partir

d'une

r?eplique

non

propri?etaire

yj

ne

peut

pas

devenir

atteignable

sous

la

simple

volont?e

de

yj

parce

que

cela

signifie

une

ecriture

sur

y

sur

le

site

j.

Ainsi

l'ensemble

des

souches

d'un

site

non

propri?etaire

est

monotone

d?ecroissant

et

par

cons?equent

les

messages

d'union

peuvent

etre

d?elivr?es

de

fa?con

asynchrone

dans

l'ordre

FIFO

En

r?esum?e

la

r?egle

de

l'union

peut

etre

impl?ement?ee

a

moindre

co

ut

lorsque

seul

le

propri?etaire

d'un

objet

a

le

droit

d'

y

ecrire

Asynchronisme

Le

collecteur

inter-bunch

doit

tenir

compte

des

affectations

de

pointeurs

-LRB-

r?ealis?ees

par

des

mutateurs

concurrents

-RRB-

parce

qu'elles

peuvent

conduire

a

la

cr?eation

de

nouveaux

pointeurs

inter-bunch

De

tels

pointeurs

doivent

etre

traqu?es

afin

d'allouer

les

souches

correspondantes

et

cr?eer

le

scion

cible

Sinon

le

collecteur

intra-bunch

pourrait

ramasser

un

objet

atteignable

Une

observation

importante

est

que

ces

cr?eations

n'ont

pas

besoin

d'

etre

effectu?ees

d?es

que

le

pointeur

inter-bunch

correspondant

appara

t

-LRB-

r?esultant

d'une

op?eration

d'affectation

-RRB-

Les

cr?eations

peuvent

donc

se

faire

de

mani?ere

asynchrone

Ceci

pr?esente

un

avantage

de

performance

et

de

portabilit?e

non

n?egligeable

puisque

le

mutateur

n'est

pas

stopp?e

cela

peut

eviter

certaines

t

aches

et

autoriser

le

regroupement

des

messages

pour

la

cr?eation

de

scions

Sans

perdre

en

g?en?eralit?e

consid?erons

l'exemple

suivant

Des

objets

x

y

et

z

sont

allou?es

respectivement

dans

des

bunches

B

C

et

D.

L'objet

x

est

d?etenu

par

le

site

i

y

par

le

site

j

et

z

par

le

site

k.

Supposons

qu'initialement

xi

yi

et

yj

soient

atteignables

que

les

deux

r?epliques

de

y

soient

egales

-LRB-

y

a

et?e

propag?ee

du

site

j

vers

le

site

i

-RRB-

et

que

xi

soit

nul

La

situation

initiale

est

donn?ee

dans

la

Figure

Maintenant

consid?erons

la

s?equence

suivante

-LRB-

voir

Figure

-RRB-

Le

mutateur

ex?ecute

x

y

i

cr?eant

ainsi

un

pointeur

inter-bunch

de

x

vers

z.

Ensuite

les

deux

r?epliques

de

y

sont

modifi?ees

de

fa?con

a

ce

qu'elles

ne

pointent

plus

vers

z

-LRB-

par

ex.

y

j

et

propagation

de

y

de

j

vers

i

-RRB-

Puis

les

bunches

B

et

C

sont

ramass?es

sur

les

sites

i

et

j.

CHAPITRE

GC

DANS

LARCHANT

D

z

k

k

les

deux

r?pliques

pointent

vers

z

y

y

x

B

C

C

i

i

j

i

i

j

Figure

Situation

de

d?epart

site

k

x

y

y

GC

GC

create

-LRB-

Bx

Dz

-RRB-

delete

-LRB-

Cy

Dz

-RRB-

union

-LRB-

y

-RRB-

propagate

-LRB-

y

-RRB-

promptness

-LRB-

propri?taire

de

y

-RRB-

-LRB-

propri?taire

de

z

-RRB-

-LRB-

propri?taire

de

x

-RRB-

livraison

causale

i

j

site

i

site

j

Figure

Le

message

d'union

transporte

la

d?ependance

causale

-LRB-

montr?ee

par

les

lignes

epaisses

-RRB-

CHAPITRE

GC

DANS

LARCHANT

Il

est

clair

que

la

innocuit?e

d?epend

de

l'ordre

de

livraison

relative

des

cr?eations

et

des

destructions

sur

le

site

k

-LRB-

i

-RRB-

le

message

de

cr?eation

doit

etre

envoy?e

du

site

i

avant

que

le

message

de

destruction

ne

soit

envoy?e

du

site

j

et

-LRB-

ii

-RRB-

le

message

de

cr?eation

doit

etre

d?elivr?e

au

site

k

avant

la

destruction

Pour

garantir

la

premi?ere

condition

tous

les

messages

de

cr?eation

d'une

m

eme

s?equence

d'un

collecteur

intra-bunch

doivent

etre

envoy?es

avant

tout

message

d'union

ou

de

destruction

de

la

m

eme

s?equence

La

seconde

condition

est

garantie

en

d?elivrant

les

messages

de

cr?eation

et

de

destruction

suivant

un

ordre

causal

Notez

que

s

il

n

y

a

pas

de

r?eplication

la

innocuit?e

demande

que

tous

les

messages

de

cr?eation

d'une

s?equence

d'un

collecteur

intra-bunch

soient

envoy?es

avant

tout

message

de

destruction

de

la

m

eme

s?equence

Pour

conclure

l'op

eration

de

cr?eation

peut

etre

effectu?e

pendant

toute

la

p?eriode

de

temps

indiqu?ee

par

promptness

dans

la

Figure

-LRB-

au

maximum

avant

le

message

d'union

-RRB-

Un

protocole

de

communication

asynchrone

avec

une

livraison

causale

est

n?ecessaire

Chapitre

Impl?ementation

du

GC

dans

Larchant

Le

prototype

de

Larchant

impl?emente

le

protocole

de

entry-consistency

-LSB-

-RSB-

Ce

protocole

offre

le

mod?ele

traditionnel

de

lecteurs

multiples

et

d'

ecrivain

unique

a

chaque

objet

il

ne

peut

etre

associ?e

que

soit

plusieurs

jetons

de

lecture

soit

un

seul

jeton

d'

ecriture

Les

sites

d?etenant

un

jeton

de

lecture

sont

assur?es

de

lire

une

r?eplique

coh?erente

de

l'objet

correspondant

Chaque

objet

poss?ede

un

propri?etaire

qui

est

soit

le

site

d?etenant

le

jeton

d'

ecriture

de

l'objet

soit

le

site

qui

a

d?etenu

en

dernier

le

jeton

d'

ecriture

Un

jeton

d'

ecriture

ne

peut

etre

obtenu

qu

a

partir

du

propri?etaire

de

l'objet

tandis

qu'un

jeton

de

lecture

peut

etre

obtenu

a

partir

de

tout

site

qui

en

d?etient

d?ej?a

un

Un

jeton

est

obtenu

en

effectuant

une

op?eration

d'acquisition

-LRB-

acquire

en

Anglais

-RRB-

de

lecture

ou

d'

ecriture

le

jeton

est

lib?er?e

par

une

lib?eration

-LRB-

release

en

Anglais

-RRB-

correspondante

L'acquisition

d'un

jeton

d'

ecriture

pour

un

objet

x

implique

l'invalidation

de

toutes

les

r?epliques

de

x

pouvant

etre

lues

-LRB-

Voir

Bershad

-LSB-

-RSB-

pour

plus

de

d?etails

-RRB-

Nous

avons

impl?ement?e

deux

algorithmes

de

GC

intra-bunch

marquage-balayage

et

copie

Tous

deux

s

ex?ecutent

de

fa?con

concurrente

par

rapport

aux

mutateurs

Ils

sont

bas?es

sur

la

technique

a

base

de

r?eplication

de

Nettles

et

O'Toole

-LSB-

-RSB-

Evidemment

les

deux

collecteurs

peuvent

aussi

s

ex?ecuter

en

mode

non

concurrent

i.e.

le

mutateur

est

stopp?e

pendant

que

les

collecteurs

s

ex?ecutent

Dans

ce

chapitre

nous

nous

concentrerons

sur

l'asynchronisme

du

collecteur

inter-bunch

par

rapport

aux

mutateurs

Asynchronisme

Nous

observons

tout

d'abord

qu'un

pointeur

inter-bunch

ne

peut

appara

tre

ou

dispara

tre

qu'en

ecrivant

dans

un

objet

x

-LRB-

op?eration

d'affectation

-RRB-

Ceci

demande

que

le

site

o?u

s

ex?ecute

le

mutateur

d?etienne

le

jeton

d'

ecriture

de

x.

Ainsi

lorsque

le

mutateur

acquiert

le

jeton

d'

ecriture

de

x

Larchant

enregistre

CHAPITRE

IMPL?EMENTATION

DU

GC

DANS

LARCHANT

causale

x

y

y

GC

GC

create

-LRB-

Bx

Dz

-RRB-

propagate

-LRB-

y

-RRB-

promptness

union

-LRB-

y

-RRB-

create

-LRB-

Bx

Dz

-RRB-

delete

-LRB-

Cy

Dz

-RRB-

create

-LRB-

Bx

Dz

-RRB-

-LRB-

propri?taire

de

x

-RRB-

-LRB-

propri?taire

de

z

-RRB-

-LRB-

propri?taire

de

y

-RRB-

site

i

site

j

site

k

i

j

livraison

Figure

La

superposition

assure

la

causalit?e

l'adresse

de

x

-LRB-

dans

un

journal

appel?e

GC-log

-RRB-

Le

GC-log

contient

les

objets

sales

Plus

tard

lorsque

le

collecteur

intra-bunch

s

ex?ecute

il

parcourt

chaque

objet

sale

dans

le

cache

local

-LRB-

pr?esent

dans

le

GC-log

-RRB-

Pour

chaque

nouveau

pointeur

inter-bunch

trouv?e

le

collecteur

alloue

la

souche

correspondante

Ensuite

lorsque

le

collecteur

inter-bunch

s

ex?ecute

pour

chaque

nouvelle

souche

il

demande

la

cr?eation

du

scion

correspondant

pour

chaque

souche

disparue

il

lance

soit

un

message

d'union

-LRB-

le

site

local

n'est

pas

le

propri?etaire

de

l'objet

correspondant

-RRB-

soit

un

message

de

destruction

apr?es

avoir

appliqu?e

la

r?egle

d'union

-LRB-

le

site

local

est

le

propri?etaire

de

l'objet

correspondant

-RRB-

Ceci

garantit

que

les

cr?eations

sont

toujours

faites

avant

les

messages

d'union

ou

de

destruction

Cependant

la

innocuit?e

impose

aussi

un

ordre

causal

Pour

cela

les

messages

de

cr?eation

sont

superpos?es

aux

messages

d'union

et

de

destruction

Consid?erons

le

cas

illustr?e

dans

la

Figure

restreint

au

protocole

de

entryconsistency

Notez

que

l'op

eration

y

j

peut

etre

effectu?ee

sur

le

site

j

parce

qu'il

est

le

propri?etaire

de

y.

La

propagation

de

y

de

j

vers

i

se

fait

via

une

acquisition

de

lecture

-LRB-

op?eration

de

entry-consistency

-RRB-

effectu?ee

par

le

site

i.

Dans

notre

impl?ementation

-LRB-

voir

Figure

-RRB-

pour

assurer

la

causalit?e

create.scion

-LRB-

Bx

Cy

-RRB-

est

superpos?e

au

message

d'union

-LRB-

de

i

vers

j

-RRB-

et

a

delete.scion

-LRB-

Cy

Dz

-RRB-

-LRB-

de

j

vers

k

-RRB-

Chapitre

Performance

des

algorithmes

de

GC

Les

collecteurs

intra-bunch

et

inter-bunch

s

ex?ecutent

de

fa?con

concurrente

aux

mutateurs

ainsi

le

temps

de

pause

du

GC

n'est

d

u

qu'aux

flips

dans

la

collecte

intra-bunch

Pour

obtenir

des

r?esultats

de

performance

concernant

le

temps

de

pause

du

GC

nous

avons

ex?ecut?e

le

banc

d'essais

suivant

Sur

chaque

site

o?u

le

bunch

en

cours

de

collecte

est

dans

le

cache

-LRB-

un

maximum

de

sites

dans

notre

exp?erimentation

-RRB-

le

mutateur

cr?ee

des

objets

et

en

ins?ere

certains

dans

une

liste

les

objets

dans

la

liste

sont

atteignables

les

autres

sont

des

miettes

La

taille

des

bunches

va

varier

de

Mo

a

Mo.

Nous

montrons

dans

la

Figure

les

r?esultats

obtenus

pour

le

collecteur

par

copie

dans

le

mode

concurrent

La

taille

du

bunch

et

le

nombre

d'objets

atteignables

n'ont

pas

d'impact

sur

le

temps

de

pause

du

GC

parce

que

la

majeure

partie

du

travail

du

collecteur

est

effectu?ee

de

fa?con

concurrente

au

mutateur

Lors

du

flip

seuls

les

objets

modifi?es

par

le

mutateur

apr?es

le

parcours

du

collecteur

sont

reparcourus

-LRB-

et

d?eplac?es

dans

le

cas

d'un

collecteur

par

copie

-RRB-

Le

nombre

de

sites

poss?edant

le

bunch

r?epliqu?e

dans

leur

cache

n

a

pas

d'impact

sur

le

temps

de

pause

du

GC

parce

que

chaque

site

effectue

son

ramassage

de

fa?con

ind?ependante

et

asynchrone

par

rapport

aux

autres

sites

En

particulier

lors

du

flip

il

n

y

a

pas

de

communication

entre

les

sites

poss?edant

une

r?eplique

du

m

eme

bunch

dans

leur

cache

Pour

comparaison

nous

montrons

dans

la

Figure

les

r?esultats

obtenus

avec

le

m

eme

banc

d'essais

pour

le

collecteur

par

copie

dans

le

mode

non

concurrent

i.e.

quand

le

mutateur

est

stopp?e

pendant

l'ex

ecution

du

collecteur

Le

temps

de

pause

du

GC

est

toujours

ind?ependant

du

nombre

de

sites

ayant

dans

leur

cache

une

r?eplique

du

bunch

en

cours

de

ramassage

Cependant

pour

des

grands

bunches

le

temps

de

pause

du

GC

est

tr?es

perturbant

avons

ex?ecut?e

nos

bancs

d'essais

sur

un

r?eseau

de

stations

DEC

Alpha

Chacune

poss?ede

Mo

de

m?emoire

principale

Go

de

disque

un

cache

de

donn?ees

de

Ko

un

cache

secondaire

de

Ko

et

une

fr?equence

de

MHz

Elles

sont

interconnect?ees

via

FDDI

CHAPITRE

PERFORMANCE

DES

ALGORITHMES

DE

GC

temps

de

pause

du

GC

-LRB-

millisecondes

-RRB-

objets

non

atteignables

-LRB-

-RRB-

copie

-LRB-

concurrente

-RRB-

Figure

Temps

de

pause

du

GC

par

copie

concurrente

temps

de

pause

du

GC

-LRB-

millisecondes

-RRB-

objets

non

atteignables

-LRB-

-RRB-

copie

-LRB-

non

concurrente

-RRB-

Figure

Temps

de

pause

du

GC

par

copie

non

concurrente

CHAPITRE

PERFORMANCE

DES

ALGORITHMES

DE

GC

Pour

obtenir

plus

de

r?esultats

nous

avons

aussi

impl?ement?e

un

simulateur

d'

edition

de

texte

coop?erative

appel?e

Un

document

est

une

structure

hi?erarchique

contenant

des

objets

pour

les

sections

sous-sections

paragraphes

lignes

etc.

Le

simulateur

n'effectue

rien

d'autre

que

des

allocations

d'objets

et

des

affectations

de

pointeurs

Nous

avons

mesur?e

les

temps

de

pause

du

GC

pour

des

collectes

intra-bunch

concurrentes

avec

s

ex?ecutant

sur

un

deux

et

trois

sites

avec

un

bunch

de

Mo

et

diff?erentes

quantit?es

de

miettes

Les

temps

de

pause

du

GC

ne

sont

jamais

sup?erieurs

a

millisecondes

pour

l'algorithme

de

copie

Pour

l'algorithme

de

GC

marquage-balayage

les

temps

de

pause

du

GC

ne

sont

jamais

sup?erieurs

a

millisecondes

Nous

avons

egalement

port?e

le

tr?es

connu

banc

d'essais

-LSB-

-RSB-

largement

utilis?e

pour

mesurer

les

performances

des

bases

de

donn?ees

orient?ees

objet

Notre

objectif

principal

etait

de

tester

la

fiabilit?e

du

prototype

de

Larchant

Le

banc

d'essais

standard

s

ex?ecute

sur

un

seul

site

et

ne

g?en?ere

pas

une

grande

quantit?e

de

miettes

Nous

l'avons

modifi?e

de

fa?con

a

lui

faire

g?en?erer

plus

de

miettes

en

effectuant

plus

de

changements

dans

le

graphe

des

pointeurs

Lorsque

nous

lan?cons

sur

un

seul

site

les

temps

de

pause

du

GC

par

copie

sont

toujours

inf?erieurs

a

millisecondes

Chapitre

Conclusion

Nous

venons

de

d?ecrire

le

mod?ele

de

Larchant

qui

offre

un

partage

et

une

r?epartition

transparents

la

persistance

par

atteignabilit?e

et

la

gestion

automatique

de

la

m?emoire

La

persistance

par

atteignabilit?e

impose

de

tracer

le

graphe

des

pointeurs

a

partir

d'une

racine

persistante

et

de

ramasser

les

donn?ees

non

atteignables

C'est

le

travail

du

GC

Le

collecteur

dans

Larchant

ramasse

de

fa?con

opportuniste

des

groupes

locaux

de

bunches

il

trace

tant

que

la

trace

reste

locale

au

site

et

compte

les

r?ef?erences

qui

demanderaient

des

surco

uts

d'

E/S

dans

le

cas

d'un

tra?cage

Il

n

y

a

ni

coordination

ni

de

synchronisation

entre

les

applications

et

les

collecteurs

Tous

les

messages

du

collecteur

sont

asynchrones

cependant

pour

assurer

l'innocuit

e

une

livraison

causale

doit

etre

assur?ee

Etant

donn?ee

la

diversit?e

des

mod?eles

de

coh?erence

utiles

nous

avons

choisi

de

ne

faire

que

des

suppositions

minimales

sur

la

coh?erence

Chaque

site

collecte

ses

r?epliques

de

bunch

ind?ependamment

des

autres

sites

sans

imposer

la

disponibilit?e

locale

d'une

m?emoire

coh?erente

C'est

pourquoi

nos

r?esultats

sont

appliquables

de

fa?con

g?en?erale

Les

r?esultats

de

performance

du

GC

de

Larchant

montrent

que

nos

objectifs

d'orthogonalit

e

par

rapport

a

l'incoh

erence

et

a

l'

echelle

ont

et?e

atteints

Partie/Part

II

Larchant

garbage

collection

in

a

cached

distributed

shared

store

with

persistence

by

reachability

Chapter

Introduction

An

essential

part

of

tomorrow

's

computing

will

be

workers

and

organizations

carrying

out

cooperative

tasks

interacting

via

shared

information

The

need

for

sharing

data

is

well

known

in

applications

such

as

interactive

computer

aided

design

-LRB-

for

financial

or

engineering

purposes

-RRB-

or

cooperative

tools

for

example

Information

is

shared

either

concurrently

or

at

different

times

thus

it

must

be

available

at

different

locations

and

persist

beyond

completion

of

a

particular

application

The

overall

goal

of

the

Larchant

project

is

to

provide

a

Cached

Distributed

Shared

Store

that

makes

data

sharing

simple

and

efficient

For

this

purpose

Larchant

supports

automatic

and

transparent

persistence

memory-management

input-output

and

distribution

With

Larchant

programmers

may

concentrate

on

application

development

without

being

distracted

by

system

issues

such

as

memory

bugs

caching

coherence

remote

access

input-output

etc.

The

model

of

Larchant

is

that

of

a

Shared

Address

Space

-LSB-

-RSB-

with

Persistence

By

Reachability

-LRB-

PBR

-RRB-

-LSB-

-RSB-

The

address

space

spans

every

participating

site

in

a

network

including

permanent

storage

This

model

is

very

attractive

given

its

simplicity

for

application

programmers

A

shared

address

space

-LSB-

-RSB-

enables

programs

to

share

data

by

using

ordinary

pointers

With

this

model

distributed

programming

is

very

simple

since

it

provides

the

same

memory

abstraction

as

in

the

centralized

case

To

provide

the

illusion

of

a

shared

address

space

across

the

network

although

site

memories

are

disjoint

Larchant

implements

a

distributed

shared

memory

-LRB-

DSM

-RRB-

mechanism

-LSB-

-RSB-

Data

is

replicated

in

multiple

sites

for

performance

and

availability

Accessing

data

entails

finding

its

reference

by

navigation

from

a

well-known

persistent

root

-LRB-

e.g.

a

name

server

-RRB-

In

systems

with

PBR

-LSB-

-RSB-

a

datum

reachable

from

the

persistent

root

is

persistent

thus

it

should

persist

on

permanent

storage

A

datum

transitively

reachable

from

the

persistent

root

is

persistent

also

Unreachable

data

is

not

needed

and

can

be

reclaimed

-LRB-

and

memory

compacted

-RRB-

Such

data

is

said

to

be

garbage

Reachability

is

accessed

by

tracing

the

pointer

graph

starting

from

the

persis

CHAPTER

INTRODUCTION

tent

root

and

reclaiming

unreachable

data

This

can

be

done

either

via

manual

memory-management

or

automatically

via

Garbage

Collection

-LRB-

GC

-RRB-

-LSB-

-RSB-

Manual

memory-management

is

extremely

error-prone

-LRB-

e.g.

dangling

pointers

and

storage

leaks

-RRB-

frequently

resulting

in

the

violation

of

the

fundamental

requirement

of

Referential

Integrity

following

a

pointer

should

always

work

i.e.

if

persistent

datum

x

points

to

datum

y

then

y

must

be

persistent

also

If

in

a

centralized

short-lived

program

GC

might

be

considered

as

an

expensive

luxury

in

contrast

in

a

cached

distributed

shared

store

such

as

Larchant

the

nightmare

of

manual

memory-management

increases

as

the

number

of

objects

pointers

and

users

scales

up

Then

GC

becomes

indispensable

GC

automatically

ensures

referential

integrity

therefore

improving

program

reliability

and

programming

productivity

In

addition

it

may

compact

memory

thus

reducing

fragmentation

and

improving

locality

Our

goal

for

a

garbage

collector

in

Larchant

can

be

summarized

as

follows

to

support

persistence

by

reachability

in

a

distributed

shared

address

space

transparently

and

efficiently

In

the

next

section

we

summarize

the

main

problems

we

must

solve

in

order

to

achieve

this

goal

Thesis

Main

Issues

In

this

document

we

address

the

issue

of

GC

in

Larchant

The

main

problems

we

tackle

can

be

summarized

as

follows

ffl

Scalability

The

GC

algorithm

must

be

capable

of

collecting

a

huge

pointer

graph

without

incurring

into

high

performance

costs

due

to

computation

inputoutput

and

synchronization

In

particular

the

collection

of

the

whole

graph

in

a

single

phase

is

clearly

not

scalable

ffl

Coherence

interference

The

GC

algorithm

must

not

compete

with

applications

for

holding

coherent

data

replicas

Such

competition

would

interfere

with

applications

coherence

needs

For

example

if

the

collector

on

some

site

requires

to

access

a

coherent

object

that

would

prevent

an

application

to

write

into

another

replica

of

that

same

object

at

the

same

time

ffl

Performance

Applications

performance

overhead

due

to

GC

must

be

minimized

In

particular

interactive

applications

should

not

be

disrupted

by

the

collector

in

such

a

way

that

its

user

would

be

annoyed

The

GC

algorithm

must

solve

the

problems

mentioned

above

i.e.

it

must

be

scalable

capable

of

making

progress

even

if

the

locally

cached

data

is

incoherent

and

efficient

In

addition

it

must

be

correct

This

means

that

it

must

be

safe

CHAPTER

INTRODUCTION

-LRB-

persistent

data

must

not

be

reclaimed

-RRB-

live

-LRB-

at

least

some

garbage

must

be

reclaimed

-RRB-

and

as

much

as

possible

complete

-LRB-

all

garbage

must

be

eventually

reclaimed

-RRB-

These

are

the

specific

goals

for

our

GC

algorithm

correctness

scalability

orthogonality

to

coherence

and

good

performance

GC

algorithms

found

in

the

literature

are

not

adequate

for

a

cached

distributed

shared

store

such

as

Larchant

-LRB-

see

Chapter

for

more

details

-RRB-

This

is

due

to

the

fact

that

Larchant

is

different

from

most

other

systems

with

the

same

overall

goal

-LRB-

support

for

sharing

-RRB-

-LSB-

-RSB-

because

it

extends

caching

with

PBR

In

particular

the

coherence

interference

problem

and

the

issue

of

safety

in

presence

of

replicated

data

possibly

incoherent

were

never

addressed

before

Thesis

Main

Contributions

This

thesis

contains

three

main

contributions

The

first

one

is

a

general

model

of

a

cached

distributed

shared

store

with

PBR

which

we

call

the

Larchant

model

The

second

and

most

important

contribution

is

a

GC

algorithm

for

the

Larchant

model

which

is

proved

to

be

correct

is

scalable

and

efficient

We

believe

the

GC

algorithm

is

widely

applicable

to

other

shared

stores

with

PBR

given

the

minimal

assumptions

we

make

regarding

the

interactions

between

applications

coherence

protocols

and

GC

needs

In

particular

the

GC

algorithm

is

orthogonal

to

coherence

and

executes

asynchronously

w.r.t.

applications

The

third

contribution

is

a

running

prototype

on

top

of

which

distributed

applications

with

persistent

objects

can

be

executed

Thesis

Roadmap

In

Chapter

we

describe

the

basic

uniprocessor

GC

algorithms

found

in

the

literature

We

focus

on

their

fundamental

characteristics

and

present

their

extensions

to

RPC-based

distributed

systems

transactional

systems

log-based

file

systems

multiprocessors

and

DSM

systems

We

also

provide

some

basic

insights

about

their

unsuitability

to

a

cached

distributed

shared

store

such

as

Larchant

In

Chapter

we

motivate

the

Larchant

model

by

making

a

brief

analysis

of

current

systems

with

support

for

distributed

data

sharing

-LRB-

e.g.

file

systems

databases

etc.

-RRB-

Then

we

describe

the

Larchant

model

with

a

focus

on

characterizing

in

a

general

and

widely

applicable

manner

the

interactions

among

applications

and

coherence

of

data

The

GC

algorithm

-LRB-

presented

in

Chapter

-RRB-

is

designed

for

the

Larchant

model

In

Chapter

we

start

by

stating

the

specific

problems

of

GC

in

Larchant

along

with

an

outline

of

the

corresponding

solutions

Then

we

describe

our

GC

algorithm

first

intuitively

then

formally

in

order

to

provide

sufficient

theoretical

arguments

concerning

its

correctness

-LRB-

safety

and

liveness

-RRB-

CHAPTER

INTRODUCTION

Chapter

describes

our

implementation

of

the

Larchant

model

-LRB-

its

architecture

-RRB-

and

the

GC

algorithm

in

particular

We

focus

on

the

interaction

between

GC

and

a

specific

coherence

protocol

-LRB-

entry-consistency

-LSB-

-RSB-

-RRB-

Chapter

studies

discuss

and

analysis

the

performance

of

Larchant

's

GC

Chapter

summarizes

the

most

important

aspects

of

this

work

and

presents

some

directions

for

future

research

Chapter

Overview

of

GC

Algorithms

Garbage

collection

has

been

a

research

topic

for

more

than

years

-LSB-

-RSB-

It

was

first

investigated

in

the

domain

of

artificial

intelligence

The

applications

then

developed

-LRB-

mainly

with

symbolic

programming

languages

e.g.

Lisp

-RRB-

had

very

complex

data

graphs

In

such

applications

memory

management

-LRB-

allocation

and

deallocation

of

memory

-RRB-

is

a

very

intricate

problem

When

left

to

manual

programming

it

is

the

cause

of

serious

program

errors

violation

of

referential

integrity

due

to

dangling

pointers

and

memory

exhaustion

due

to

storage

leaks

Some

years

later

object-oriented

languages

and

systems

-LRB-

e.g.

Smalltalk

-LSB-

-RSB-

-RRB-

faced

the

same

problem

With

the

advent

of

distributed

systems

and

PBR

-LRB-

persistence

by

reachability

-RRB-

the

need

for

GC

has

increased

even

more

In

such

systems

manual

memory

management

becomes

a

nightmare

as

the

number

of

objects

pointers

and

users

scales

up

In

this

chapter

we

study

and

discuss

the

most

important

GC

algorithms

found

in

the

literature

First

we

introduce

the

terminology

that

will

be

used

in

the

rest

of

this

document

Second

we

present

the

basic

uniprocessor

GC

algorithms

Then

we

describe

their

extensions

to

RPC-based

distributed

systems

transactional

systems

log-based

file

systems

multiprocessors

and

DSM

systems

We

conclude

this

chapter

with

a

discussion

concerning

the

GC

algorithms

here

described

we

give

some

insights

regarding

their

unsuitability

for

a

distributed

shared

store

based

on

caching

and

PBR

such

as

Larchant

We

address

only

the

basic

aspects

of

the

GC

algorithms

For

more

details

on

a

particular

algorithm

or

its

variants

we

refer

the

reader

to

the

existing

literature

-LSB-

-RSB-

We

describe

the

GC

algorithms

characterizing

them

according

to

the

following

three

aspects

ffl

Type

of

Algorithm

There

are

two

basic

types

reference

counting

and

reference

tracing

-LRB-

tracing

for

short

-RRB-

The

general

term

of

tracing

was

taken

from

Lang

-LSB-

-RSB-

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

ffl

Functioning

Mode

The

functioning

mode

characterizes

a

GC

algorithm

in

terms

of

the

memory

on

which

the

collector

works

on

each

run

-LRB-

e.g.

a

subset

of

all

the

memory

-RRB-

and

the

moments

in

which

the

collection

takes

place

with

respect

to

applications

ffl

System

Requirements

A

GC

algorithm

is

characterized

w.r.t.

its

needs

in

terms

of

knowledge

of

references

location

and

the

possibility

of

moving

reachable

objects

to

different

addresses

in

order

to

compact

memory

Terminology

An

object

is

a

contiguous

set

of

bytes

A

reference

identifies

an

object

In

its

simplest

form

it

is

a

pointer

We

will

use

equivalently

the

term

reference

and

pointer

along

this

document

The

heap

designates

a

portion

of

virtual

memory

-LRB-

possibly

covering

the

whole

memory

-RRB-

where

objects

are

allocated

Hereafter

we

will

use

the

terms

heap

and

memory

equivalently

We

follow

the

standard

vocabulary

of

the

GC

literature

-LSB-

-RSB-

The

GC-root

is

the

set

of

references

that

forms

the

starting

point

for

the

GC

graph

tracing

The

mutator

is

the

application

program

that

dynamically

modifies

the

pointer

graph

it

creates

objects

dereferences

pointers

and

assigns

pointers

The

collector

is

the

system

component

that

identifies

and

reclaims

unreachable

objects

In

a

distributed

system

a

collector

is

composed

of

a

number

of

threads

executing

in

different

processes

we

still

call

each

one

a

collector

Similarly

the

mutator

is

actually

composed

of

multiple

independent

threads

running

in

different

processes

by

extension

we

call

each

of

these

threads

a

mutator

A

pointer

assignment

can

result

in

-LRB-

i

-RRB-

creation

of

a

new

reference

-LRB-

to

an

object

just

created

-RRB-

-LRB-

ii

-RRB-

duplication

of

an

already

existing

reference

and

-LRB-

iii

-RRB-

discarding

a

reference

to

an

object

Note

that

cases

-LRB-

ii

-RRB-

and

-LRB-

iii

-RRB-

may

happen

as

the

result

of

a

single

assignment

operation

As

a

side-effect

of

pointer

assignment

some

reachable

objects

become

unreachable

Unreachable

objects

said

to

be

garbage

are

not

needed

and

can

be

reclaimed

A

GC

algorithm

is

safe

when

it

does

not

reclaim

reachable

objects

and

is

live

if

it

eventually

reclaims

some

garbage

Completeness

means

that

a

GC

algorithm

eventually

reclaims

all

garbage

Obviously

every

GC

algorithm

aims

at

being

safe

and

complete

However

as

will

become

clear

this

goal

is

not

easy

to

achieve

as

scalability

is

at

odds

with

completeness

Basic

Uniprocessor

GC

Algorithms

In

this

section

we

describe

the

two

fundamental

uniprocessor

GC

algorithms

Reference

Counting

and

Tracing

We

present

their

most

relevant

advantages

and

inconvenients

which

are

related

to

completeness

performance

and

memory

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

GC-root

memory

Figure

Uniprocessor

reference

counting

The

counter

associated

with

an

object

is

indicated

at

its

top

The

shaded

objects

are

unreachable

however

they

are

not

reclaimed

fragmentation

Then

we

describe

several

variants

of

these

two

algorithms

which

are

based

on

the

functioning

modes

and

system

requirements

Reference

Counting

The

basic

idea

of

the

reference

counting

algorithm

-LSB-

-RSB-

is

the

following

A

counter

is

associated

to

each

object

denoting

the

number

of

references

to

it

When

an

object

is

created

a

single

reference

points

to

it

and

its

counter

is

initially

one

Each

time

a

reference

is

duplicated

the

object

's

counter

is

incremented

Each

time

a

reference

to

an

object

is

discarded

its

counter

is

decremented

Therefore

reference

counting

preserves

the

invariant

that

the

value

of

an

object

's

counter

is

always

equal

to

the

number

of

references

to

it

Figure

illustrates

this

algorithm

When

a

counter

drops

to

zero

the

corresponding

object

is

no

longer

reachable

and

can

be

safely

reclaimed

-LRB-

Garbage

objects

are

therefore

reclaimed

immediately

after

becoming

unreachable

-RRB-

However

the

converse

is

not

true

i.e.

an

object

may

be

unreachable

but

its

counter

greater

than

zero

This

happens

when

an

object

is

part

of

a

cycle

of

unreachable

objects

-LRB-

see

Figure

-RRB-

When

an

object

is

reclaimed

-LRB-

its

counter

has

become

zero

-RRB-

its

pointers

are

discarded

Thus

reclaiming

one

object

may

lead

to

the

transitive

decrementing

of

reference

counts

and

reclaiming

many

other

objects

This

algorithm

is

simple

to

implement

However

it

has

three

main

problems

First

it

is

inefficient

its

cost

is

proportional

to

the

amount

of

work

done

by

the

mutator

because

counters

must

be

updated

whenever

references

are

assigned

-LRB-

This

problem

is

addressed

by

variants

of

this

algorithm

such

as

Deferred

Reference

Counting

-LSB-

-RSB-

-RRB-

The

second

problem

is

fragmentation

the

free

space

recovered

from

reclaimed

objects

is

interspersed

with

reachable

objects

This

reduces

locality

of

reference

because

as

new

objects

are

allocated

in

the

free

space

recovered

from

reclaimed

objects

unrelated

objects

are

interleaved

in

memory

This

may

lead

to

a

situa

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

free-list

free-list

free-list

memory

GC-root

GC-root

GC-root

m

m

m

-LRB-

b

-RRB-

after

the

mark

phase

-LRB-

c

-RRB-

after

the

sweep

phase

-LRB-

a

-RRB-

before

GC

memory

memory

Figure

Uniprocessor

mark-and-sweep

algorithm

marked

objects

are

represented

with

a

m

in

the

header

tion

in

which

the

working

set

of

an

application

is

scattered

across

many

virtual

memory

pages

so

that

those

pages

are

frequently

swapped

in

and

out

of

main

memory

The

third

and

major

problem

is

that

cycles

are

not

reclaimed

thus

this

algorithm

is

not

complete

This

may

lead

to

memory

exhaustion

Tracing

There

are

two

algorithms

of

the

tracing

type

Mark-and-Sweep

and

Copy

Tracing

algorithms

traverse

the

pointer

graph

from

the

GC-root

to

determine

which

objects

are

reachable

Unreachable

objects

are

reclaimed

The

main

advantage

of

tracing

algorithms

is

their

ability

to

reclaim

cycles

of

unreachable

objects

Mark-and-Sweep

A

mark-and-sweep

collector

-LSB-

-RSB-

has

two

phases

-LRB-

i

-RRB-

trace

the

pointer

graph

starting

from

the

GC-root

and

mark

every

object

found

and

-LRB-

ii

-RRB-

sweep

-LRB-

i.e.

examine

-RRB-

all

the

heap

reclaiming

unmarked

objects

Figure

illustrates

this

algorithm

During

the

mark

phase

every

reachable

object

is

marked

-LRB-

setting

a

bit

in

the

object

's

header

for

example

-RRB-

and

scanned

for

pointers

This

phase

ends

when

there

are

no

more

reachable

objects

to

mark

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

y

-LRB-

a

-RRB-

before

collection

GC-root

GC-root

y

x

x

from-space

to-space

from-space

to-space

-LRB-

b

-RRB-

after

collection

z

z

Figure

Uniprocessor

copy

algorithm

During

the

sweep

phase

the

collector

detects

which

objects

are

not

marked

and

inserts

their

memory

space

in

the

free-list

When

the

collector

finds

a

marked

object

it

unmarks

it

in

order

to

make

it

ready

for

the

next

collection

This

phase

ends

when

there

is

no

more

memory

to

be

swept

This

algorithm

has

two

main

problems

The

first

one

is

fragmentation

Just

as

with

reference

counting

free

space

recovered

from

reclaimed

objects

is

interspersed

with

reachable

objects

The

second

problem

is

that

the

cost

of

the

sweep

phase

is

proportional

to

the

size

of

the

heap

Copy

In

the

copy

algorithm

-LSB-

-RSB-

the

collector

traces

the

pointer

graph

from

the

GC-root

and

moves

each

object

reached

to

another

location

In

this

way

memory

is

compacted

therefore

eliminating

fragmentation

and

allocation

of

objects

is

done

linearly

This

is

an

advantage

of

this

algorithm

Figure

illustrates

the

algorithm

The

collector

divides

the

heap

in

two

disjoint

semispaces

called

from-space

and

to-space

During

normal

mutator

execution

objects

are

allocated

linearly

in

from-space

Once

the

collection

starts

the

collector

moves

reachable

objects

to

to-space

Unreachable

objects

are

left

in

from-space

When

every

reachable

object

has

been

moved

the

roles

of

the

semispaces

is

exchanged

the

to-space

becomes

the

from-space

and

vice-versa

This

transition

is

called

flipping

Flipping

is

atomic

w.r.t.

the

mutator

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

x

y

z

t

u

y

x

y

t

z

x

y

t

z

x

y

t

z

v

u

x

y

t

z

v

u

x

y

-LRB-

vi

-RRB-

t

z

v

GC-root

v

from-space

move

reachable

objects

-LRB-

i

-RRB-

freescan

x

free

-LRB-

iii

-RRB-

scan

-LRB-

ii

-RRB-

free

scan

scan

free

-LRB-

iv

-RRB-

-LRB-

v

-RRB-

freescan

free

scan

to-space

Figure

Cheney

algorithm

The

Cheney

algorithm

-LSB-

-RSB-

is

a

well

known

technique

to

move

reachable

objects

from

from-space

to

to-space

We

describe

it

with

the

help

of

an

example

-LRB-

see

Figure

-RRB-

Objects

immediately

accessible

from

the

GC-root

-LRB-

x

and

y

-RRB-

are

the

first

to

be

moved

to

to-space

These

objects

form

the

initial

set

for

a

breadthfirst

traversal

that

is

implemented

with

the

help

of

two

pointers

free

which

points

to

the

first

free

address

in

to-space

and

scan

which

points

to

the

first

object

in

to-space

not

yet

scanned

Then

the

object

pointed

by

the

scan

pointer

-LRB-

x

in

this

case

-RRB-

is

scanned

for

pointers

into

from-space

Each

object

reached

-LRB-

t

and

z

-RRB-

is

moved

to

to-space

and

the

free

pointer

updated

In

addition

the

pointers

in

the

scanned

object

are

patched

according

to

the

new

locations

of

the

moved

objects

and

a

forwarding

pointer

-LRB-

pointing

to

to-space

-RRB-

is

left

in

their

old

location

When

y

is

scanned

z

will

be

reached

and

given

the

forwarding

pointer

there

left

z

is

not

moved

again

the

pointer

in

y

is

simply

patched

This

algorithm

ends

when

every

object

in

to-space

has

been

scanned

An

inconvenient

of

the

copy

algorithm

is

that

only

half

of

the

memory

space

available

is

used

at

any

point

in

time

the

to-space

is

a

wasted

resource

between

collections

Functioning

Modes

In

this

section

we

address

the

various

functioning

modes

that

can

be

applied

to

the

fundamental

algorithms

previously

described

The

functioning

modes

are

essentially

orthogonal

to

these

algorithms

These

modes

characterize

a

GC

algorithm

in

terms

of

the

memory

on

which

the

collector

works

on

each

run

-LRB-

e.g.

a

subset

of

the

all

memory

-RRB-

and

the

moments

in

which

the

collection

takes

place

w.r.t.

mutators

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

ffl

Partitioned

Only

a

sub-set

of

the

whole

memory

called

a

partition

is

garbage

collected

-LRB-

independently

from

the

rest

of

the

memory

-RRB-

The

existence

of

multiple

partitions

is

useful

in

systems

where

is

necessary

to

have

a

garbage

collected

heap

co-existing

with

a

manually-managed

heap

-LSB-

-RSB-

for

example

Partitioned

GC

is

also

useful

in

large

persistent

and/or

distributed

systems

because

different

partitions

may

be

collected

in

parallel

and

independently

from

each

other

-LRB-

see

Section

-RRB-

ffl

GC-only

The

mutator

is

halted

while

the

collector

runs

The

time

interval

during

which

a

mutator

is

halted

due

to

GC

is

called

GC

pause

time

This

time

may

be

unacceptable

in

some

cases

for

example

in

interactive

applications

where

the

user

would

be

annoyed

by

such

pauses

ffl

Incremental

Small

units

of

GC

are

interleaved

with

small

units

of

mutator

execution

Each

GC

pause

time

is

smaller

than

in

the

GC-only

mode

ffl

Concurrent

The

mutator

and

the

collector

run

concurrently

The

usefulness

of

this

mode

is

that

collection

adds

no

pauses

on

top

of

time-slicing

In

the

rest

of

this

section

and

when

the

difference

is

not

relevant

we

will

use

the

term

incremental

to

designate

both

incremental

and

concurrent

functioning

modes

Note

that

reference

counting

algorithms

are

inherently

incremental

because

the

collector

and

mutator

execution

is

interleaved

In

the

rest

of

this

section

we

focus

on

the

incremental

and

partitioned

variants

of

tracing

collectors

Incremental

Tracing

The

main

issue

of

incremental

tracing

is

how

to

ensure

the

correct

execution

of

the

collector

when

it

competes

with

the

mutator

for

the

same

data

Concerning

this

issue

there

is

no

significant

difference

between

mark-and-sweep

and

copy

algorithms

Before

going

into

more

details

concerning

this

fundamental

aspect

it

is

useful

to

see

how

both

mark-and-sweep

and

copy

-LRB-

incremental

collectors

-RRB-

can

be

described

as

variants

of

an

abstract

tricolor

marking

algorithm

-LSB-

-RSB-

The

tricolor

marking

algorithm

works

as

follows

Objects

subjected

to

collection

are

colored

white

and

when

the

collection

is

finished

reachable

objects

must

be

colored

black

A

collection

is

finished

when

there

are

no

more

reachable

objects

to

blacken

In

a

mark-and-sweep

collector

this

coloring

can

be

implemented

by

setting

mark

bits

-LRB-

objects

whose

bit

is

set

are

black

-RRB-

In

a

copy

collector

the

coloring

is

the

process

of

moving

reachable

objects

from

from-space

to

to-space

Objects

in

the

from-space

are

white

objects

in

the

to-space

are

black

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

GC-root

GC-root

x

y

u

z

y

u

z

-LRB-

a

-RRB-

intial

graph

by

the

mutator

-LRB-

b

-RRB-

graph

modified

concurrently

x

Figure

Erroneous

GC

tracing

due

to

concurrent

mutator

activity

The

main

difficulty

with

incremental

tracing

GC

is

that

while

the

collector

is

tracing

the

pointer

graph

as

a

result

of

mutator

activity

the

graph

may

change

while

the

collector

is

n't

looking

If

this

happens

the

collector

may

not

find

some

reachable

objects

Thus

the

mutator

can

not

be

allowed

to

change

the

pointers

graph

behind

the

collector

's

back

in

such

a

way

that

the

collector

fails

to

find

all

reachable

objects

For

this

purpose

there

is

a

third

color

gray

A

gray

object

is

one

that

has

been

reached

by

the

collector

tracing

but

its

descendents

may

not

have

been

Once

a

gray

object

has

been

scanned

it

becomes

black

and

its

descendents

are

colored

gray

-LRB-

In

a

copy

collector

the

gray

objects

are

those

that

have

already

been

moved

to

to-space

but

have

not

yet

been

scanned

-RRB-

Black

objects

may

not

point

to

white

objects

This

invariant

allows

the

collector

to

assume

that

it

is

finished

with

black

objects

and

can

continue

to

traverse

gray

objects

If

the

mutator

creates

a

pointer

from

a

black

object

to

a

white

one

it

must

somehow

notify

the

collector

that

its

assumption

has

been

violated

Therefore

the

collector

must

be

capable

of

keeping

track

of

graph

changes

resulting

from

mutator

activity

and

re-trace

parts

of

the

graph

adequately

This

ensures

that

the

collector

is

aware

of

every

change

concerning

the

pointers

graph

Figure

illustrates

this

problem

by

showing

the

need

for

coordination

between

the

mutator

and

the

collector

Suppose

that

object

x

has

been

completely

scanned

-LRB-

and

therefore

blackened

-RRB-

its

descendents

-LRB-

y

and

z

-RRB-

have

been

reached

and

grayed

Now

suppose

that

the

mutator

swaps

the

pointer

from

x

to

z

with

the

pointer

from

y

to

u

The

only

pointer

to

u

is

now

in

object

x

which

has

already

been

scanned

by

the

collector

-LRB-

this

violates

the

invariant

black

object

x

pointing

to

white

object

u

-RRB-

If

the

tracing

continues

without

any

coordination

y

will

be

blackened

z

will

be

reached

again

-LRB-

from

y

-RRB-

and

u

will

never

be

reached

at

all

and

hence

will

be

unsafely

considered

garbage

This

problem

is

solved

by

coordinating

the

mutator

with

the

collector

This

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

can

be

done

with

the

following

two

techniques

read-barrier

or

write-barrier

Read-Barrier

A

read-barrier

is

used

to

detect

when

the

mutator

attempts

to

read

a

pointer

to

a

white

object

immediately

the

collector

scans

the

object

and

colors

it

gray

since

the

mutator

can

not

read

pointers

to

white

objects

it

can

not

write

them

into

black

objects

There

are

many

incremental

collectors

using

a

read-barrier

-LSB-

-RSB-

We

describe

here

a

representative

example

using

a

copy

algorithm

The

best-known

incremental

copy

collector

is

Baker

's

-LSB-

-RSB-

A

collection

starts

with

a

flip

that

conceptually

invalidates

-LRB-

for

mutator

accesses

-RRB-

all

objects

in

fromspace

and

moves

to

to-space

all

objects

directly

reachable

from

the

GC-root

Then

the

mutator

is

allowed

to

resume

Any

object

in

from-space

that

is

accessed

by

the

mutator

must

first

be

moved

to

to-space

this

is

enforced

by

the

read-barrier

Thus

the

mutator

is

never

allowed

to

see

pointers

into

from-space

if

so

the

referent

is

immediately

moved

to

to-space

In

terms

of

the

tricolor

marking

algorithm

objects

in

to-space

that

were

already

scanned

are

black

objects

in

to-space

but

not

yet

scanned

are

gray

Objects

still

in

from-space

are

white

New

objects

created

by

the

mutator

while

the

collection

is

taking

place

are

allocated

in

the

to-space

and

are

colored

black

Write-Barrier

A

write-barrier

detects

when

the

mutator

tries

to

write

a

pointer

into

a

black

object

when

that

happens

the

write

is

trapped

or

recorded

-LSB-

-RSB-

We

describe

here

an

interesting

copy

collector

using

a

write-barrier

it

is

called

replication-based

GC

-LSB-

-RSB-

The

basic

idea

is

the

following

While

the

collector

moves

objects

to

to-space

the

mutator

continues

to

access

the

from-space

versions

of

objects

rather

than

the

replicas

in

to-space

When

every

reachable

object

has

been

moved

to

to-space

a

flip

is

performed

and

the

mutator

then

starts

seeing

the

to-space

replicas

This

technique

eliminates

the

need

for

a

read-barrier

because

all

the

reachable

objects

are

conceptually

moved

to

to-space

at

once

-LRB-

when

the

flip

occurs

-RRB-

However

it

is

necessary

to

have

a

write-barrier

because

the

mutator

sees

the

old

version

of

objects

in

from-space

if

an

object

has

already

been

moved

to

to-space

and

then

the

mutator

writes

into

it

the

replica

in

to-space

is

now

outof-date

w.r.t.

the

version

seen

by

the

mutator

Thus

the

write-barrier

catches

all

mutator

writes

-LRB-

and

stores

them

in

a

log

-RRB-

in

order

to

allow

the

collector

to

propagate

those

modifications

to

the

to-space

replicas

when

the

flip

takes

place

In

other

words

all

the

modifications

to

objects

in

from-space

must

be

applied

to

the

corresponding

replicas

in

to-space

so

that

the

mutator

sees

the

correct

values

after

the

flip

Partitioned

Tracing

The

application

of

the

partitioned

mode

to

the

basic

GC

algorithms

-LRB-

reference

counting

and

tracing

-RRB-

results

in

the

following

two

variants

-LRB-

i

-RRB-

hybrid

collectors

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

and

-LRB-

ii

-RRB-

generational

collectors

In

the

first

variant

each

partition

is

traced

independently

from

the

others

and

cross-partition

references

are

handled

with

a

reference

counting

algorithm

-LSB-

-RSB-

This

variant

is

very

interesting

for

large

distributed

and/or

persistent

systems

and

is

described

in

more

detail

in

Section

The

second

variant

generational

collectors

aims

at

reducing

the

GC

pause

time

by

decreasing

the

amount

of

memory

that

has

to

be

collected

For

this

purpose

it

takes

advantage

of

the

following

empirical

observation

In

many

applications

most

objects

are

reachable

for

a

very

short

length

of

time

only

a

small

portion

remains

reachable

much

longer

-LSB-

-RSB-

Thus

the

idea

is

to

separate

the

objects

in

-LRB-

at

least

-RRB-

two

groups

those

objects

that

are

reachable

only

for

a

short

length

of

time

belong

to

the

first

group

the

others

belong

to

the

second

group

and

to

collect

the

first

group

more

often

Both

mark-and-sweep

and

copy

algorithms

can

be

made

generational

-LSB-

-RSB-

in

this

section

we

will

focus

on

the

copy

algorithm

Objects

are

segregated

into

multiple

partitions

by

age

Each

partition

is

called

a

generation

Younger

generations

are

collected

more

often

than

older

generations

The

age

of

an

object

is

approximated

by

the

number

of

collections

it

has

survived

To

avoid

the

cost

of

successive

scans

and

moves

-LRB-

from

from-space

to-space

-RRB-

of

those

objects

that

remain

reachable

for

a

long

time

partitions

containing

older

objects

are

collected

less

often

than

the

younger

ones

Thus

once

objects

have

survived

a

certain

number

of

collections

they

are

moved

to

a

less-frequently

collected

partition

-LRB-

instead

of

to-space

-RRB-

To

allow

young

generations

to

be

collected

without

having

to

collect

the

older

ones

the

collector

must

be

capable

of

finding

pointers

into

the

young

generations

This

requires

either

the

use

of

a

write-barrier

similar

to

the

one

we

found

in

the

incremental

functioning

mode

-LRB-

see

previous

section

-RRB-

to

keep

track

of

such

cross-partition

pointers

-LSB-

-RSB-

or

indirect

pointers

from

older

to

younger

generations

-LSB-

-RSB-

The

set

of

references

pointing

from

older

to

younger

generations

is

usually

called

remembered-set

When

a

younger

generation

is

collected

the

pointers

in

the

corresponding

remembered-set

are

part

of

the

GC-root

System

Requirements

Tracing

garbage

collectors

traverse

the

pointer

graph

For

this

purpose

the

collector

must

be

capable

of

finding

the

pointers

inside

the

GC-root

and

inside

each

reachable

object

In

programming

languages

providing

runtime

type

information

it

is

possible

to

differentiate

pointers

from

raw

data

with

certainty

This

is

the

case

of

Lisp

and

Smalltalk

where

there

is

enough

type

information

that

can

be

used

to

determine

object

layouts

including

the

locations

of

embedded

pointers

-LSB-

-RSB-

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

However

there

are

cases

in

which

such

runtime

type

information

is

not

available

as

is

the

case

with

the

programming

languages

C

-LSB-

-RSB-

and

C++

-LSB-

-RSB-

In

such

environments

a

possible

solution

consists

of

either

a

pre-processor

-LSB-

-RSB-

or

the

compiler

-LSB-

-RSB-

to

statically

generate

type

information

for

each

data

type

Another

alternative

is

to

take

advantage

of

some

specific

language

features

-LRB-

e.g

smart-pointers

in

C++

-RRB-

-LSB-

-RSB-

to

generate

the

pointers

locations

Another

solution

called

conservative

GC

-LSB-

-RSB-

relies

on

a

conservative

pointer

finding

approach

i.e.

the

collector

treats

anything

that

might

be

a

pointer

as

a

pointer

Thus

any

properly

aligned

bit

pattern

that

could

be

the

address

of

an

object

is

actually

considered

to

be

a

pointer

to

that

object

There

are

a

few

problems

concerning

this

approach

First

the

conservative

interpretation

of

ambiguous

data

-LRB-

e.g.

considering

an

integer

as

a

pointer

-RRB-

may

lead

to

consider

garbage

objects

as

being

reachable

This

waste

of

memory

can

be

a

serious

problem

for

memory

intensive

applications

-LSB-

-RSB-

Second

given

that

the

collector

does

not

differentiate

raw

data

from

pointers

it

has

to

scan

all

the

data

inside

reachable

objects

This

extra

cost

of

scanning

increases

the

GC

pause

time

Finally

a

copy

collector

can

not

move

reachable

objects

and

patch

the

corresponding

pointers

because

a

non-pointer

might

be

considered

to

be

a

pointer

and

would

be

mistakenly

patched

However

in

spite

of

these

problems

there

are

cases

for

which

a

conservative

approach

is

adequate

and

sometimes

the

only

that

is

feasible

-LSB-

-RSB-

Distributed

GC

Algorithms

In

this

section

we

present

the

most

interesting

GC

algorithms

found

in

the

literature

for

RPC-based

distributed

-LRB-

i.e.

with

no

support

for

persistence

or

DSM

-RRB-

These

algorithms

are

extensions

of

the

basic

reference

counting

and

tracing

Reference

Counting

Each

process

in

the

system

holds

one

partition

that

is

collected

independently

from

the

rest

of

the

memory

with

a

tracing

algorithm

Cross-partition

references

are

managed

with

a

reference

counting

algorithm

The

extension

of

the

uniprocessor

reference

counting

algorithm

to

handle

crosspartition

references

poses

some

problems

These

problems

generally

called

race

conditions

arise

because

objects

counters

have

to

be

incremented

and

decremented

through

messages

exchanged

between

processes

Such

messages

must

be

delivered

reliably

and

in

causal

order

-LSB-

-RSB-

to

ensure

safety

and

liveness

increment

and

decrement

messages

are

not

idempotent

and

must

not

be

duplicated

or

lost

distributed

system

is

a

set

of

processes

communicating

by

messages

with

no

shared

memory

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

@z

process

j

z

decrement

increment

process

i

process

k

Figure

Decrement/increment

race

condition

with

distributed

reference

counting

@z

process

j

z

increment

decrement

process

i

process

k

Figure

Increment/decrement

race

condition

with

distributed

reference

counting

There

are

two

types

of

race

conditions

both

possibly

leading

to

the

unsafe

reclamation

of

a

reachable

object

We

call

them

decrement/increment

and

increment/decrement

Figure

illustrates

the

decrement/increment

race

condition

Suppose

that

process

i

holds

a

reference

to

object

z

in

process

k

and

sends

a

message

to

process

j

containing

@z

a

reference

to

z.

Process

j

receives

this

message

and

sends

an

increment

message

to

k

concerning

object

z

concurrently

process

i

deletes

its

reference

to

z

and

sends

the

corresponding

decrement

message

to

k

If

the

decrement

message

arrives

first

at

k

then

z

is

considered

to

be

unreachable

and

is

unsafely

reclaimed

At

first

glance

it

seems

that

this

race

problem

could

be

solved

by

simply

making

the

sender

process

i

conservatively

emit

the

increment

message

before

sending

@z

to

j.

However

as

explained

now

this

does

not

solve

the

problem

and

leads

us

to

the

scenario

for

the

second

race

problem

The

increment/decrement

race

is

illustrated

in

Figure

the

sender

process

issues

the

increment

message

not

the

receiver

as

in

the

previous

case

Suppose

that

process

i

holds

a

reference

to

object

z

in

process

k

and

sends

a

message

to

process

j

containing

@z

Now

j

receives

this

message

and

immediately

discards

it

Therefore

it

sends

a

decrement

message

to

k

concerning

z.

Concurrently

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

z

process

i

process

j

process

k

outgoing

weight

outgoing

weight

incoming

weight

W

@z

@z

Figure

Weighted

reference

counting

process

i

sends

the

corresponding

increment

message

to

k

Once

again

if

the

decrement

message

arrives

first

at

k

z

is

unsafely

reclaimed

An

obvious

solution

for

these

two

race

problems

is

to

acknowledge

the

increment

message

before

sending

a

decrement

or

a

reference

to

a

remote

process

respectively

However

this

introduces

communication

overhead

and

the

GC

algorithm

is

still

not

resilient

to

failures

For

these

reasons

there

are

many

variants

of

the

reference

counting

algorithm

that

can

be

grouped

in

the

following

main

categories

weighted

reference

counting

-LSB-

-RSB-

indirect

reference

counting

-LSB-

-RSB-

and

reference

listing

-LSB-

-RSB-

Each

one

of

these

variants

avoids

the

transmission

of

increment

messages

therefore

solving

the

two

race

problems

previously

described

We

describe

these

variants

now

Weighted

Reference

Counting

Each

cross-partition

reference

has

two

associated

weights

a

weight

at

the

source

process

-LRB-

outgoing

weight

-RRB-

a

weight

at

the

target

process

-LRB-

incoming

weight

-RRB-

For

any

object

z

the

incoming

weight

must

be

equal

to

the

sum

of

the

outgoing

weights

associated

to

the

cross-partition

references

pointing

to

z

-LRB-

see

Figure

-RRB-

When

a

cross-partition

reference

is

first

created

both

incoming

and

outgoing

weights

are

equal

-LRB-

a

even

positive

value

usually

a

power

of

two

-RRB-

When

the

holder

of

a

reference

passes

it

-LRB-

through

a

message

-RRB-

to

another

process

it

divides

the

current

outgoing

weight

in

two

parts

-LRB-

normally

equal

-RRB-

retains

one

and

sends

the

other

along

with

the

message

When

the

receiver

process

receives

the

message

it

associates

to

the

new

outgoing

cross-partition

reference

the

weight

just

received

if

a

cross-partition

reference

to

the

same

object

already

exists

it

simply

adds

the

received

weight

to

the

current

one

Thus

the

sum

of

outgoing

weights

is

always

kept

equal

to

the

incoming

weight

at

the

target

process

When

a

process

deletes

an

outgoing

cross-partition

reference

it

sends

a

decrement

message

with

the

associated

weight

to

the

target

process

The

target

process

receives

that

message

and

subtracts

the

received

weight

from

its

incom

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

process

i

@z

@z

weight

X

outgoing

outgoing

weight

weight

X

process

j

z

process

k

incoming

weight

W

outgoing

incoming

weight

Figure

Problem

with

weight

reference

counting

ing

weight

-LRB-

corresponding

to

the

cross-partition

reference

-RRB-

When

the

incoming

weight

becomes

zero

then

the

corresponding

object

may

be

reclaimed

Note

that

this

algorithm

solves

the

two

race

problems

previously

described

but

still

needs

reliable

communication

-LRB-

no

loss

or

duplication

-RRB-

for

ensuring

safety

and

liveness

In

addition

this

algorithm

has

the

following

problem

the

limit

imposed

on

the

number

of

times

a

reference

may

be

sent

to

another

process

is

limited

by

the

initial

associated

weight

This

problem

is

solved

by

the

use

of

an

extra

indirection

as

explained

now

Suppose

that

process

i

holds

an

outgoing

cross-partition

reference

pointing

to

z

in

process

k

and

that

the

associated

weight

has

dropped

to

-LRB-

i.e.

no

more

division

of

the

weight

is

allowed

-RRB-

Now

if

process

i

needs

to

send

@z

to

process

j

the

collector

creates

a

new

incoming

cross-partition

reference

coming

from

j

that

refers

indirectly

to

z

through

the

original

outgoing

cross-partition

reference

that

points

to

k

-LRB-

see

Figure

-RRB-

Indirect

Reference

Counting

This

algorithm

avoids

the

use

of

increment

messages

by

maintaining

a

distributed

reference

count

for

each

remotely

referenced

object

Increments

are

always

done

locally

therefore

with

no

communication

Remote

references

are

counted

with

a

counter

associated

to

the

corresponding

pointer

Associated

to

each

remote

pointer

there

is

-LRB-

see

Figure

-RRB-

-LRB-

i

-RRB-

the

identification

of

the

process

from

which

that

pointer

came

and

-LRB-

ii

-RRB-

a

variable

that

counts

how

many

times

that

pointer

was

duplicated

from

the

local

process

The

latter

is

incremented

each

time

the

pointer

is

duplicated

therefore

creating

a

new

remote

reference

to

the

target

object

Note

that

this

increment

is

local

i.e.

there

is

no

increment

message

sent

When

the

mentioned

counter

becomes

zero

the

process

sends

a

decrement

message

to

the

process

from

which

the

pointer

came

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

process

i

@z

z

process

k

@z

weak

pointer

process

i

strong

pointer

strong

pointer

weak

pointer

@z

@z

process

j

counter

strong

pointer

process

l

counter

process

i

@z

counter

process

j

Figure

Indirect

reference

counting

Sometimes

in

the

literature

-LSB-

-RSB-

the

reference

to

the

remote

object

is

called

strong

pointer

-LRB-

pointers

to

object

z

in

Figure

-RRB-

in

opposition

to

the

notion

of

weak

pointer

which

is

implemented

by

the

identification

of

the

process

from

where

the

remote

reference

came

-LRB-

pointers

to

processes

i

and

j

in

Figure

-RRB-

Weak

pointers

are

used

by

applications

when

accessing

the

remote

object

while

strong

pointers

are

only

for

GC

purposes

-LRB-

the

process

to

which

the

decrement

message

is

sent

-RRB-

Note

that

this

algorithm

does

not

ensure

safety

in

presence

of

duplicated

messages

and

liveness

is

not

preserved

if

messages

are

lost

In

addition

the

deletion

of

a

remote

reference

may

generate

many

decrement

messages

in

cascade

Reference

Listing

In

this

algorithm

an

outgoing

cross-partition

reference

goes

indirectly

through

an

auxiliary

data

structure

called

stub

Similarly

a

incoming

cross-partition

reference

goes

indirectly

through

an

auxiliary

data

structure

called

scion

-LRB-

instead

of

a

reference

counter

-RRB-

This

is

illustrated

in

Figure

A

stub

identifies

his

matching

scion

and

vice-versa

-LRB-

i.e.

the

scion

holds

a

backpointer

to

the

corresponding

stub

-RRB-

In

addition

a

scion

identifies

the

target

object

A

stub-scion

pair

fully

identifies

the

corresponding

cross-partition

reference

For

safety

the

invariant

that

must

be

kept

is

the

following

if

in

some

process

there

is

a

stub

associated

to

a

cross-partition

reference

than

the

target

object

must

be

pointed

by

the

corresponding

scion

The

increment

and

decrement

messages

of

the

basic

reference

counting

algorithm

are

replaced

by

asynchronous

create

and

delete

messages

-LRB-

applied

to

scions

-RRB-

respectively

These

messages

are

called

control

messages

A

create

-LRB-

delete

-RRB-

message

informs

the

target

process

that

a

scion

must

be

created

-LRB-

deleted

-RRB-

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

z

process

k

process

i

scion

stub

stub

scion

scion

scion

process

j

x

y

Figure

Stubs

and

scions

describing

cross-partition

references

The

algorithm

works

as

follows

Each

process

i

collects

its

partition

independently

-LRB-

The

GC-root

includes

the

set

of

scions

-RRB-

The

collection

generates

a

new

set

of

reachable

objects

and

a

new

set

of

stubs

describing

outgoing

crosspartition

references

By

comparing

the

old

set

of

stubs

-LRB-

before

the

local

collection

-RRB-

with

the

new

one

the

collector

discovers

the

stubs

that

no

longer

exist

and

sends

a

delete

message

concerning

the

corresponding

scion

Note

that

instead

of

sending

delete

messages

the

collector

can

send

the

list

of

stubs

then

it

is

up

to

the

receiver

to

find

out

which

scions

are

no

longer

referenced

The

advantage

of

this

algorithm

resides

on

its

fault-tolerance

to

message

loss

and

duplication

For

example

a

message

to

delete

a

scion

can

be

sent

several

times

without

violating

safety

a

repeated

delete

message

is

simply

ignored

For

more

details

we

forward

the

interested

reader

to

the

literature

-LSB-

-RSB-

Cycles

of

Garbage

Reference

counting

algorithms

do

not

collect

cross-partition

cycles

of

garbage

There

are

three

solutions

for

this

problem

an

additional

cross-partition

tracing

algorithm

object

migration

and

trial

deletion

Cross-partition

tracing

algorithms

are

addressed

in

the

next

section

Object

Migration

The

basic

idea

of

the

migration

technique

-LSB-

-RSB-

is

the

following

migrating

several

objects

to

a

single

partition

-LRB-

in

a

process

-RRB-

in

order

to

transform

a

distributed

cycle

into

a

local

cycle

that

can

be

easily

reclaimed

with

a

tracing

algorithm

The

problem

with

this

technique

is

twofold

-LRB-

i

-RRB-

some

objects

can

not

be

migrated

due

to

systems

constraints

and

-LRB-

ii

-RRB-

it

is

difficult

to

know

which

objects

should

be

migrated

-LRB-

and

when

-RRB-

in

order

to

reclaim

a

maximum

amount

of

cycles

with

a

minimum

cost

Recent

work

-LSB-

-RSB-

proposes

a

low-cost

heuristic

that

helps

to

find

which

objects

should

be

migrated

This

heuristic

is

based

on

the

distance

of

objects

which

is

defined

as

follows

The

distance

of

an

object

is

the

minimum

number

of

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

process

i

D

process

j

process

k

x

z

y

D

D

Figure

Distances

of

objects

in

a

cycle

of

garbage

cross-partition

references

in

any

path

from

the

persistent

root

to

that

object

the

distance

of

a

unreachable

object

is

infinity

The

distances

are

propagated

inside

a

partition

by

the

tracing

collector

and

between

partitions

by

sending

the

set

of

stubs

to

other

processes

-LRB-

reference

listing

algorithm

-RRB-

With

this

mechanism

the

distances

of

objects

that

belong

to

a

cycle

of

garbage

increase

indefinitely

while

the

distances

of

others

do

not

We

provide

here

an

example

that

illustrates

how

the

distance

of

a

cycle

increases

indefinitely

Initially

x

is

reachable

from

the

persistent

root

and

x

's

distance

is

D.

Then

the

pointer

from

the

persistent

root

to

x

is

discarded

Figure

shows

each

object

distance

at

this

instant

Now

imagine

that

process

i

collects

its

partition

Object

x

is

reachable

from

y

thus

x

's

distance

will

be

that

of

y

plus

one

When

the

collector

in

k

runs

z

is

reachable

from

x

thus

the

distance

of

z

will

be

x

's

distance

plus

one

When

the

collector

in

j

runs

y

is

reachable

from

z

thus

the

distance

of

y

will

be

z

's

distance

plus

one

Consequently

the

distances

of

x

y

and

z

will

increase

indefinitely

The

basic

idea

of

this

algorithm

is

to

migrate

only

those

objects

whose

distance

is

above

some

threshold

because

those

are

likely

to

be

garbage

The

estimation

of

the

threshold

depends

of

the

expected

distance

of

reachable

objects

Trial

Deletion

The

trial

deletion

technique

-LSB-

-RSB-

works

as

follows

an

object

suspected

to

belong

to

a

cycle

is

tentatively

considered

to

be

unreachable

by

the

distributed

collector

This

is

done

by

pretending

that

its

associated

reference

count

is

zero

then

if

the

associated

reference

count

in

fact

drops

to

zero

the

collector

concludes

that

the

object

must

be

part

of

a

cycle

A

difficulty

with

this

solution

is

to

decide

which

objects

are

suspected

to

belong

to

a

cycle

In

addition

the

implementation

of

the

temporary

reference

counts

is

complex

given

that

multiple

trial

deletions

may

be

occurring

in

parallel

Tracing

The

majority

of

the

tracing

algorithms

in

RPC-based

distributed

systems

are

of

the

type

mark-and-sweep

In

this

section

we

briefly

present

the

most

inter

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

esting

-LRB-

non-partitioned

-RRB-

distributed

mark-and-sweep

algorithms

found

in

the

literature

The

basic

distributed

mark-and-sweep

algorithm

is

as

follows

In

the

mark

phase

each

process

marks

objects

reachable

from

the

GC-root

-LRB-

set

of

scions

-RRB-

Each

marked

object

is

scanned

and

for

each

remote

reference

found

a

marking

message

is

sent

to

the

target

process

The

process

receiving

a

marking

message

marks

the

corresponding

object

and

continues

the

marking

phase

When

every

process

has

already

marked

all

the

reachable

objects

and

there

are

no

marking

messages

in

transit

the

sweep

phase

starts

The

sweep

phase

may

be

done

by

each

process

independently

from

any

other

process

Note

that

this

algorithm

imposes

a

global

trace

including

every

process

in

the

network

before

any

garbage

can

be

reclaimed

In

addition

it

is

necessary

to

synchronize

the

mark

and

sweep

phases

In

other

words

all

processes

must

synchronize

in

order

to

agree

on

the

termination

of

the

marking

phase

The

difficulty

with

this

global

synchronization

-LRB-

besides

its

non-scalability

-RRB-

is

that

no

marking

messages

may

be

in

transit

Another

difficulty

with

distributed

mark-and-sweep

is

to

maintain

the

consistency

of

the

description

of

cross-partition

references

-LRB-

i.e.

stubs

and

scions

-RRB-

Alternatively

if

some

degree

of

inconsistency

is

allowed

then

the

GC

algorithm

must

be

made

in

such

a

way

that

such

inconsistencies

are

safe

Mark-and-Sweep

with

Timestamps

This

technique

-LSB-

-RSB-

is

like

the

basic

distributed

mark-and-sweep

algorithm

except

that

it

uses

timestamps

instead

of

mark

bits

The

timestamp

of

a

reachable

object

keeps

increasing

while

the

timestamp

of

a

unreachable

object

eventually

stabilizes

A

global

timestamp

threshold

is

computed

and

every

object

marked

with

a

lower

timestamp

is

garbage

This

algorithm

avoids

a

global

synchronization

of

every

process

needed

to

agree

on

the

termination

of

the

mark

phase

Each

process

has

a

clock

that

is

used

to

record

the

time

when

the

GC

started

locally

-LRB-

we

call

it

GC-start

-RRB-

When

the

collector

starts

each

object

directly

referenced

from

the

GC-root

is

timestamped

with

GC-start

Scions

retain

the

timestamp

last

put

into

them

-LRB-

When

a

scion

is

created

it

is

timestamped

with

the

local

process

current

timestamp

-RRB-

Then

the

collector

propagates

the

timestamps

associated

with

its

local

GC-root

all

along

the

reachable

objects

up

to

the

reachable

stubs

-LRB-

Actually

only

stubs

and

scions

need

stamps

reachable

objects

are

simply

marked

-RRB-

The

collection

in

a

process

is

expected

to

timestamp

a

stub

with

the

largest

local

timestamp

of

any

GC-root

from

which

it

is

reachable

To

ensure

this

references

in

the

GC-root

are

selected

for

tracing

in

the

decreasing

order

of

the

corresponding

timestamps

At

the

end

of

a

local

collection

stubs

are

sent

to

their

target

processes

-LRB-

with

the

corresponding

timestamps

-RRB-

We

call

them

timestamping

messages

-LRB-

they

are

similar

to

the

marking

messages

of

the

basic

distributed

mark-and-sweep

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

algorithm

-RRB-

When

a

process

receives

a

timestamping

message

it

updates

the

timestamps

of

the

corresponding

scions

to

a

value

that

is

the

maximum

of

their

current

value

and

that

received

in

the

message

In

case

a

scion

's

timestamp

is

augmented

the

local

process

records

the

fact

that

the

new

timestamp

value

has

not

yet

been

propagated

For

this

purpose

each

process

maintains

a

timestamp

called

redo

whose

value

is

equal

to

the

greatest

timestamp

already

locally

propagated

Thus

when

a

scion

's

timestamp

is

increased

the

redo

is

set

to

the

scion

's

old

timestamp

if

that

is

lower

that

its

current

value

When

all

timestamping

messages

have

been

locally

treated

the

process

sends

an

acknowledgment

message

to

the

sender

Once

the

sender

process

has

received

the

acknowledgment

of

all

timestamping

messages

sent

it

can

update

its

own

redo

value

to

the

local

GC-start

-LRB-

as

long

as

it

did

not

receive

timestamping

messages

from

other

processes

itself

-RRB-

It

can

be

shown

that

the

threshold

mentioned

above

-LRB-

the

one

that

allows

the

collector

to

detect

unreachable

objects

-RRB-

can

be

the

global

minimum

of

the

redo

's

Thus

any

scion

timestamped

with

a

value

below

the

minimum

of

the

redo

's

is

garbage

The

problems

with

this

algorithm

are

the

following

First

the

computation

of

the

timestamp

threshold

-LRB-

minimum

of

redo

's

-RRB-

is

notoriously

costly

and

unscalable

as

it

depends

on

a

global

termination

algorithm

-LSB-

-RSB-

Second

it

requires

every

process

to

cooperate

-LRB-

even

though

there

is

no

global

synchronization

when

the

marking

phase

terminates

-RRB-

Third

if

a

process

i

crashes

the

entire

tracing

eventually

halts

as

the

global

minimum

redo

's

algorithm

will

be

stucked

at

i

's

redo

value

Thus

the

algorithm

is

not

resilient

to

failures

Logically

Centralized

Tracing

The

idea

of

this

variant

-LSB-

-RSB-

is

to

compute

the

global

accessibility

of

objects

on

a

single

highly

available

centralized

service

All

the

information

manipulated

by

this

service

-LRB-

possibly

replicated

for

fault-tolerance

-RRB-

is

stored

in

stable

storage

Each

process

performs

asynchronous

local

collections

and

communicates

with

the

central

service

providing

it

with

accessibility

information

i.e.

stubs

and

scions

From

time

to

time

each

process

asks

the

central

service

about

the

accessibility

of

its

scions

The

answer

may

indicate

that

a

scion

is

no

longer

reachable

from

any

stub

thus

it

can

be

deleted

The

central

service

never

has

a

consistent

view

of

the

reachability

of

every

object

given

that

processes

collect

asynchronously

Thus

the

central

service

adopts

a

conservative

approach

in

order

to

cope

with

such

inconsistencies

while

being

safe

For

this

purpose

it

uses

a

timestamp

protocol

involving

loosely

synchronized

clocks

at

each

process

and

a

bounded

delay

for

messages

in

transit

To

reclaim

cycles

this

variant

may

require

a

process

to

traverse

its

local

pointer

graph

multiple

times

This

is

not

a

good

solution

because

of

its

cost

in

terms

stable

store

as

defined

by

Lampson

-LSB-

-RSB-

is

a

set

of

objects

that

move

atomically

from

one

consistent

state

to

another

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

of

performance

Therefore

to

reclaim

distributed

cycles

a

solution

similar

to

the

previous

one

-LRB-

tracing

with

timestamps

-RRB-

must

be

used

Finally

the

central

service

can

become

a

bottleneck

and

processes

must

transfer

a

fair

amount

of

data

Group

Tracing

This

algorithm

-LSB-

-RSB-

is

designed

to

work

in

a

partitioned

memory

Each

process

holds

a

partition

that

is

collected

independently

The

algorithm

aims

at

reclaiming

cross-partition

cycles

of

garbage

without

requiring

the

cooperation

of

every

process

in

the

network

Only

the

processes

containing

objects

that

belong

to

the

cycle

being

reclaimed

must

cooperate

With

this

algorithm

any

set

of

processes

may

decide

to

form

a

group

and

perform

a

group-wide

tracing

independently

from

other

processes

This

tracing

is

able

to

reclaim

every

cross-partition

cycle

of

garbage

within

the

group

Objects

pointed

through

incoming

references

from

outside

the

group

are

considered

to

be

reachable

Groups

can

be

formed

and

dismantled

dynamically

Multiple

collections

on

different

groups

can

run

in

parallel

and

such

groups

may

even

overlap

Given

that

groups

are

dynamic

if

some

process

is

down

-LRB-

or

unreachable

due

to

communication

problems

-RRB-

the

set

of

accessible

processes

can

still

form

a

group

Thus

GC

is

not

blocked

due

to

a

crashed

process

One

difficulty

of

this

algorithm

is

finding

the

GC-root

of

a

group

i.e.

the

incoming

references

from

processes

external

to

the

group

This

is

due

to

the

fact

that

groups

are

dynamic

thus

the

GC-root

of

a

group

must

be

found

afresh

In

addition

the

formation

of

a

group

imposes

a

non-negligible

performance

penalty

Another

problem

resides

on

knowing

which

processes

should

be

grouped

-LRB-

and

when

-RRB-

in

order

to

reclaim

the

maximum

amount

of

cross-partition

cycles

of

garbage

GC

in

Transactional

Systems

In

this

section

we

describe

very

briefly

the

most

interesting

GC

algorithms

found

in

the

literature

for

transactional

systems

These

algorithms

are

extensions

of

the

reference

counting

and

tracing

algorithms

presented

in

the

previous

sections

Such

extensions

are

needed

to

deal

with

the

specific

safety

problems

posed

by

transactional

systems

We

focus

on

these

problems

and

corresponding

solutions

Transactional

Reference

Listing

Thor

-LSB-

-RSB-

is

an

object-oriented

client-server

database

with

multiple

storage

servers

Clients

cache

in

main

memory

objects

that

are

being

accessed

The

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

server

x

a

scion

pointing

to

y

-LRB-

from

-RRB-

server

y

x

client

x

is

cached

in

the

client

scion

stub

stub

scion

GC-root

scion

from

x

-LRB-

in

the

client

-RRB-

to

y

is

not

needed

because

there

is

already

The

stub

describing

the

pointer

Figure

Transactional

reference

listing

GC

algorithm

in

Thor

-LSB-

-RSB-

is

a

fault-tolerant

variant

of

the

reference

listing

-LRB-

recall

Section

-RRB-

We

call

it

transactional

reference

listing

When

a

server

recovers

from

a

failure

it

must

retrieve

all

the

scions

pointing

to

objects

allocated

in

the

server

partitions

Scions

whose

stubs

are

held

by

other

servers

are

easily

recovered

because

such

scions

are

kept

in

stable

storage

On

the

other

hand

scions

whose

stubs

are

held

by

clients

are

not

kept

in

stable

storage

because

that

would

be

too

expensive

Instead

a

server

keeps

in

stable

storage

a

list

of

its

clients

Thus

when

a

server

recovers

it

knows

who

its

clients

are

and

sends

them

query

messages

asking

for

their

stubs

If

a

client

process

has

not

communicated

with

a

server

for

a

long

time

and

does

not

respond

to

repeated

query

messages

the

server

assumes

it

has

failed

However

the

client

may

just

be

unable

to

communicate

with

that

server

because

of

network

problems

It

might

happen

that

the

client

communicates

with

other

servers

Imagine

that

a

server

assumes

that

a

client

has

failed

while

a

second

server

does

not

-LRB-

see

Figure

-RRB-

Then

the

first

server

discards

the

scions

whose

stubs

were

held

by

the

client

This

may

cause

the

second

server

to

unsafely

reclaim

an

object

y

that

is

still

reachable

from

the

client

This

erroneous

behavior

is

due

to

the

inconsistent

views

the

servers

have

about

the

client

To

solve

this

problem

it

is

necessary

that

all

servers

get

a

consistent

view

of

a

client

status

For

this

purpose

there

is

an

atomic

shutdown

protocol

Once

this

protocol

is

executed

no

server

will

honor

requests

from

the

client

that

has

been

shutdown

Thus

in

the

example

above

and

would

agree

that

the

client

has

crashed

and

y

would

be

safely

reclaimed

because

the

client

would

not

be

allowed

to

follow

the

pointer

from

x

to

y

A

problem

with

this

solution

is

that

the

shutdown

protocol

is

not

easily

scalable

Transactional

Mark-and-Sweep

This

algorithm

-LSB-

-RSB-

is

a

variant

of

the

basic

uniprocessor

mark-and-sweep

with

the

partitioned

and

incremental

functioning

modes

extended

in

order

to

cope

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

x

client

process

server

process

y

initial

situation

z

flush

after

y

y

page

flush

before

flush

z

y

y

x

x

x

x

y

x

z

Figure

Problem

with

transactional

mark-and-sweep

with

transactions

in

a

client-server

database

The

collector

runs

on

the

server

and

was

implemented

in

the

EXODUS

database

-LSB-

-RSB-

We

call

it

transactional

mark-and-sweep

When

a

reference

to

some

object

is

discarded

inside

a

transaction

the

pointed

object

is

eligible

for

collection

only

after

the

commit

of

that

transaction

This

rule

prevents

the

unsafe

reclamation

of

an

object

that

becomes

unreachable

during

a

transaction

that

will

later

abort

In

fact

the

abort

of

the

transaction

will

make

that

object

reachable

again

Thus

any

object

that

might

become

unreachable

during

a

transaction

will

only

be

reclaimed

if

it

remains

unreachable

after

the

commit

of

the

transaction

An

object

created

during

a

transaction

is

eligible

for

reclamation

only

after

the

commit

of

the

transaction

that

created

it

This

rule

prevents

the

unsafe

reclamation

of

reachable

objects

due

to

the

flush

of

pages

-LRB-

from

the

client

to

the

server

-RRB-

in

an

order

that

is

not

controlled

by

the

collector

This

is

illustrated

in

Figure

-LRB-

x

is

reachable

from

the

GC-root

-RRB-

The

application

creates

object

z

and

makes

it

reachable

from

x.

Then

the

page

containing

objects

y

and

z

is

flushed

to

the

server

When

the

collector

runs

-LRB-

in

the

server

-RRB-

only

the

page

containing

objects

y

and

z

has

been

flushed

Thus

z

is

unsafely

reclaimed

because

it

is

not

reachable

from

x

The

memory

space

reclaimed

by

the

sweeping

phase

can

be

reused

for

allocating

new

objects

only

when

the

freeing

of

that

space

is

reflected

in

stable

storage

This

ensures

that

during

a

recovery

after

a

failure

there

will

always

be

enough

space

for

allocating

objects

This

is

illustrated

by

the

following

example

Suppose

that

a

client

creates

objects

x

y

and

z

in

a

page

that

had

previously

been

garbage

collected

at

the

server

However

that

page

has

not

been

written

to

stable

storage

yet

on

disk

it

still

contains

a

garbage

object

t

-LRB-

not

yet

swept

-RRB-

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

forwarding

pointer

-LRB-

b

-RRB-

virtual

memory

after

failure

x

forwarding

pointer

-LRB-

a

-RRB-

virtual

memory

before

failure

from-space

to-space

from-space

to-space

x

x

Figure

Problem

with

copy

GC

in

presence

of

failures

Now

the

client

commits

the

transaction

As

a

result

the

page

and

the

associated

log

are

sent

to

the

server

The

log

is

written

to

stable

storage

and

the

system

crashes

before

the

updated

page

reaches

the

disk

Then

recovery

starts

The

problem

is

that

objects

x

y

and

z

must

be

created

-LRB-

according

to

the

log

-RRB-

but

there

is

not

enough

space

as

the

page

that

is

used

for

the

recovery

-LRB-

the

one

on

disk

-RRB-

still

contains

the

garbage

object

t

-LRB-

not

yet

swept

-RRB-

There

are

other

mark-and-sweep

collectors

for

databases

-LSB-

-RSB-

but

they

do

not

differ

from

the

basic

mark-and-sweep

algorithm

as

the

one

we

have

just

described

Such

algorithms

are

simply

adapted

to

the

particular

transactional

system

in

which

they

are

integrated

They

are

very

specific

and

not

scalable

Atomic

Copy

This

algorithm

-LSB-

-RSB-

is

based

on

Baker

's

copy

GC

-LRB-

recall

Section

-RRB-

and

works

on

a

single

partition

Both

functioning

modes

GC-only

and

incremental

can

be

found

in

the

literature

Reachable

objects

are

moved

from

from-space

to

to-space

and

patched

accordingly

This

may

interfere

with

the

stability

of

the

store

as

explained

now

Imagine

there

is

a

failure

while

the

collector

is

running

Then

during

the

failure

recovery

the

GC

algorithm

must

find

which

objects

have

already

been

moved

and

which

objects

have

already

been

patched

Otherwise

the

pointer

graph

will

be

corrupted

Figure

illustrates

an

example

-LRB-

for

a

more

detailed

study

we

refer

the

reader

to

the

existing

literature

-LSB-

-RSB-

-RRB-

Suppose

there

is

a

failure

after

object

x

has

been

moved

to

to-space

-LRB-

a

forwarding

pointer

was

left

in

from-space

-RRB-

and

only

the

from-space

has

been

written

to

disk

Thus

after

the

failure

the

disk

will

not

contain

a

valid

version

of

x

the

version

in

from-space

has

been

owverwritten

by

the

forwarding

pointer

and

the

version

in

to-space

is

not

available

on

disk

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

Thus

if

the

collector

were

to

be

restarted

after

the

failure

x

would

never

be

moved

again

to

to-space

and

the

pointer

graph

would

be

corrupted

The

solution

to

this

problem

consists

of

making

the

copy

algorithm

atomic

This

requires

that

the

recovery

of

a

failure

puts

from-space

and

to-space

in

a

state

from

which

the

collection

can

continue

This

is

done

by

applying

the

write-ahead

log

protocol

-LSB-

-RSB-

to

the

disk

contents

For

this

purpose

both

the

move

and

patch

of

each

reachable

object

must

be

logged

This

solution

even

after

being

optimized

remains

costly

We

describe

here

the

unoptimized

version

-LRB-

incremental

GC

-RRB-

The

move

of

a

reachable

object

proceeds

as

follows

ffl

Pin

the

from-space

object

that

will

be

overwritten

by

the

forwarding

pointer

ffl

Move

the

object

to

to-space

and

insert

the

forwarding

pointer

in

its

fromspace

version

ffl

Create

a

log

entry

with

the

from-space

and

the

to-space

addresses

of

the

object

Create

another

log

entry

with

the

to-space

address

of

the

object

and

its

contents

ffl

Now

the

move

of

the

object

is

completed

ffl

After

both

log

entries

are

written

to

disk

unpin

the

from-space

object

The

scan

and

patch

of

a

to-space

object

proceeds

as

follows

ffl

Pin

the

object

to

be

scanned

ffl

Scan

the

object

and

patch

its

from-space

pointers

to

to-space

pointers

moving

the

pointed

objects

as

needed

ffl

Create

a

log

entry

with

the

object

contents

ffl

Unpin

the

object

after

the

log

entry

is

written

to

disk

Concerning

correctness

the

GC

algorithm

must

not

consider

to

be

unreachable

an

object

x

that

became

so

due

to

a

transaction

that

is

still

running

If

the

mentioned

transaction

aborts

object

x

is

now

reachable

again

-LRB-

as

it

was

before

the

transaction

has

started

-RRB-

Thus

the

collector

has

to

consider

as

part

of

the

GC-root

the

log

of

running

transactions

Note

that

the

collector

could

run

as

a

single

long

user-level

transaction

or

a

series

of

short

transactions

However

in

the

first

case

this

would

lead

to

a

long

GC

pause

time

which

is

highly

disruptive

In

the

second

case

it

could

easily

lead

to

deadlock

or

to

a

situation

in

which

a

long

user

transaction

could

prevent

the

collector

from

making

progress

For

these

reasons

the

collector

runs

at

a

level

below

the

transactions

support

as

sketched

above

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

Replicated

Copy

The

replication-based

copy

collector

-LSB-

-RSB-

-LRB-

recall

Section

-RRB-

is

particularly

well

suited

to

mono-site

transactional

systems

This

is

due

to

the

fact

that

while

the

collector

moves

objects

to

to-space

the

mutator

continues

to

access

their

from-space

replicas

rather

than

the

replicas

in

to-space

When

every

reachable

object

has

been

moved

to

to-space

a

flip

is

performed

and

the

mutator

then

starts

accessing

the

to-space

replicas

If

a

process

fails

while

the

collector

is

running

-LRB-

before

the

flip

-RRB-

the

collector

simply

restarts

with

the

recovered

fromspace

All

GC

operations

already

done

are

lost

GC

in

File

Systems

Some

file

systems

and

databases

have

a

mechanism

to

compact

disk

blocks

These

are

mostly

of

the

time

executed

off-line

and

are

explicitly

invoked

by

the

system

manager

However

there

is

an

interesting

exception

the

LFS

disk

storage

manager

-LSB-

-RSB-

LFS

uses

the

concepts

of

log-structured

file

systems

-LSB-

-RSB-

and

has

an

incremental

copy

garbage

collector

The

disk

is

divided

in

large

fixed-size

pieces

called

segments

Each

segment

contains

several

blocks

and

a

summary

block

For

each

block

in

the

segment

the

summary

block

indicates

the

file

number

of

the

block

's

file

and

the

position

of

the

block

within

the

file

During

normal

operation

segments

will

become

fragmented

The

garbage

collector

task

is

to

generate

free

segments

called

clean

segments

fom

fragmented

segments

The

algorithm

has

two

phases

and

can

be

summarized

as

follows

In

the

first

phase

the

reachable

blocks

of

fragmented

segments

are

identified

and

read

into

main

memory

The

second

phase

combines

the

blocks

into

a

new

segment

and

writes

them

to

disk

The

collector

determines

if

a

block

is

reachable

using

the

summary

block

the

inode

map

and

inodes

In

the

first

step

the

segment

's

summary

block

is

used

to

determine

the

file

number

and

block

offset

of

each

block

Included

in

the

summary

block

there

is

the

file

's

version

number

from

the

inode

map

when

the

block

was

written

If

the

version

number

does

not

match

the

current

version

number

of

the

file

the

block

is

unreachable

-LRB-

it

has

been

deleted

or

overwritten

-RRB-

If

the

previous

step

fails

to

determine

the

allocation

status

of

the

block

the

inode

and

any

indirect

blocks

that

map

the

block

of

the

file

are

examined

to

see

if

the

block

is

still

part

of

the

file

The

block

is

classified

as

being

reachable

if

it

still

a

member

of

the

file

For

performance

reasons

it

is

desirable

to

collect

the

segments

with

the

most

free

space

i.e.

with

the

smallest

number

of

reachable

blocks

This

is

due

to

the

fact

that

the

cost

of

the

copy

algorithm

increases

with

the

number

of

blocks

that

have

to

be

moved

to

a

new

segment

For

this

purpose

LFS

has

a

data

structure

forwarding

pointer

does

not

overwrite

the

object

in

from-space

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

that

keeps

an

estimate

of

the

number

of

reachable

blocks

in

each

segment

This

data

structure

is

updated

when

files

are

truncated

or

overwritten

and

when

segments

are

written

or

garbage

collected

GC

in

Multiprocessors

GC

algorithms

for

multiprocessors

-LSB-

-RSB-

are

mostly

variants

of

the

tracing

type

with

the

incremental/concurrent

functioning

mode

As

the

name

indicates

multiprocessors

have

several

processors

This

capacity

of

processing

can

be

used

for

example

to

execute

the

collector

in

a

processor

different

from

the

ones

where

mutators

run

Multiprocessors

frequently

offer

very

efficient

synchronization

primitives

that

are

used

to

ensure

GC

correctness

in

presence

of

concurrent

access

to

the

same

data

by

mutators

and

the

collector

The

incremental/concurrent

functioning

mode

of

tracing

GC

has

already

been

addressed

in

the

context

of

uniprocessor

collection

-LRB-

recall

Section

-RRB-

The

important

issue

remains

the

same

how

to

ensure

the

correct

execution

of

the

mutator

and

the

collector

when

both

are

competing

for

the

same

data

Therefore

we

do

not

go

into

more

details

regarding

GC

in

multiprocessors

GC

in

DSM

Systems

There

is

very

few

work

on

GC

algorithms

for

DSM

systems

The

most

interesting

problem

regarding

GC

in

these

systems

is

coherence

interference

as

stated

in

Section

However

this

issue

is

not

even

mentioned

in

the

literature

Data

is

simply

assumed

to

be

coherent

all

the

time

Le

Sergent

-LSB-

-RSB-

describes

an

extension

of

a

copy

collector

first

developed

for

a

multiprocessor

to

a

DSM

system

Object

replicas

are

assumed

to

be

coherent

the

entire

memory

is

collected

at

the

same

time

which

is

not

scalable

and

the

garbage

collector

locks

pages

while

scanning

which

interferes

with

the

coherence

protocol

and

therefore

disrupts

applications

Kordale

's

GC

-LSB-

-RSB-

is

very

complex

and

relies

on

a

large

amount

of

auxiliary

information

This

GC

algorithm

is

based

on

the

mark-and-sweep

technique

and

relies

on

the

fact

that

objects

must

be

coherent

as

well

Discussion

In

this

section

we

provide

some

insights

regarding

the

suitability

of

the

GC

algorithms

described

in

this

chapter

for

several

systems

These

insights

can

be

used

as

rules

of

thumb

to

decide

which

algorithm

is

most

appropriate

for

a

given

environment

We

also

situate

Larchant

and

its

GC

w.r.t.

to

the

algorithms

and

systems

presented

in

this

chapter

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

GC

Considerations

The

first

observation

is

that

no

GC

algorithm

is

universally

good

In

other

words

depending

on

the

system

characteristics

a

particular

algorithm

might

be

more

appropriate

than

others

The

best

GC

algorithm

for

a

system

may

even

change

with

time

Reference

counting

algorithms

seem

to

be

most

indicated

for

distributed

systems

because

they

are

scalable

However

they

fail

to

reclaim

cycles

Tracing

algorithms

reclaim

both

acyclic

and

cyclic

garbage

However

they

are

not

appropriate

for

distributed

systems

because

they

are

not

scalable

Mark-and-sweep

collectors

must

sweep

the

entire

heap

This

is

a

disadvantage

for

heaps

where

the

majority

of

objects

are

unreachable

Then

for

a

uniprocessor

a

copy

algorithm

is

the

most

appropriate

on

the

contrary

if

the

majority

of

objects

are

reachable

a

mark-and-sweep

collector

might

be

more

efficient

Note

that

the

proportion

of

reachable

and

unreachable

objects

may

vary

with

time

The

basic

GC

algorithms

must

be

adapted

and

modified

in

order

to

deal

with

the

specific

problems

of

persistence

and

distribution

Concerning

GC

for

DSM

there

are

only

a

few

proposals

and

they

all

assume

coherent

objects

just

as

in

a

centralized

system

therefore

compromising

scalability

GC

in

Larchant

Larchant

is

a

cached

distributed

store

with

PBR

It

has

a

DSM

mechanism

that

provides

the

illusion

of

a

shared

address

space

across

the

network

including

secondary

storage

Thus

at

a

first

glance

we

could

envisage

to

chose

a

GC

algorithm

for

Larchant

similar

to

those

used

either

in

multiprocessor

systems

or

in

DSM

systems

If

we

apply

a

GC

algorithm

designed

for

multiprocessors

-LRB-

for

instance

Appel

's

collector

-LSB-

-RSB-

-RRB-

to

Larchant

the

overhead

will

be

unacceptable

due

to

synchronization

and

I/O

costs

Similarly

the

GC

algorithms

for

DSM

systems

found

in

the

literature

are

not

appropriate

for

Larchant

also

In

both

cases

the

reason

is

that

the

collector

needs

every

object

to

be

coherent

The

GC

algorithms

for

RPC-based

distributed

systems

clearly

indicate

that

a

scalable

solution

should

be

based

on

reference

counting

However

note

that

in

contrast

to

Larchant

they

do

not

consider

the

existence

of

multiple

replicas

of

objects

In

addition

while

in

RPC-based

distributed

systems

there

is

one

partition

per

process

in

Larchant

partitions

and

processes

are

orthogonal

In

conclusion

we

can

say

that

GC

algorithms

found

in

the

literature

are

not

adequate

for

Larchant

because

they

do

not

handle

replication

because

they

are

not

scalable

and

because

they

require

coherent

data

CHAPTER

OVERVIEW

OF

GC

ALGORITHMS

Summary

In

this

chapter

we

described

the

most

relevant

GC

algorithms

for

uniprocessors

RPC-based

distributed

systems

transactional

systems

log-based

file

systems

multiprocessors

and

DSM

systems

There

are

two

fundamental

GC

algorithms

reference

counting

and

tracing

There

are

two

tracing

collectors

mark-and-sweep

and

copy

Reference

counting

algorithms

do

not

reclaim

cycles

of

garbage

are

inherently

incremental

and

are

scalable

Tracing

algorithms

reclaim

cycles

but

are

not

scalable

The

copy

algorithm

has

the

advantage

of

compacting

memory

therefore

reducing

fragmentation

but

needs

to

know

the

exact

location

of

pointers

When

applied

to

distributed

and/or

persistent

systems

the

two

fundamental

algorithms

are

modified

in

order

to

remain

safe

in

presence

of

unreliable

communication

and

process

failures

They

deal

either

with

duplication

or

messages

loss

or

with

process

failures

and

transactional

particularities

such

as

uncontrolled

page

flush

from

the

client

to

the

server

Existing

GC

algorithms

do

not

provide

a

good

solution

for

GC

in

Larchant

because

they

do

not

address

the

problems

raised

by

a

cached

distributed

store

with

persistence

by

reachability

scalability

coherence

interference

and

safety

in

presence

of

replicated

data

possibly

incoherent

Chapter

Larchant

Model

We

start

this

chapter

by

motivating

the

Larchant

model

We

give

an

example

that

illustrates

the

simplicity

of

data

sharing

in

Larchant

from

the

application

programmer

's

point

of

view

sharing

an

object

is

just

the

matter

of

following

a

pointer

Then

we

briefly

discuss

the

most

relevant

solutions

currently

used

to

support

distributed

data

sharing

file

systems

object-oriented

databases

etc.

We

point

out

the

disadvantages

of

each

approach

regarding

data

sharing

for

the

applications

we

are

interested

in

supporting

-LRB-

e.g.

interactive

computer

aided

design

and

cooperative

tools

-RRB-

Then

we

describe

the

Larchant

model

a

general

model

of

a

cached

distributed

shared

store

with

PBR

-LRB-

persistence

by

reachability

-RRB-

Our

focus

is

to

characterize

the

interactions

between

applications

and

coherence

protocols

This

is

the

model

for

which

the

GC

algorithm

is

designed

-LRB-

see

Chapter

-RRB-

The

model

is

general

and

we

make

minimal

assumptions

therefore

it

is

widely

applicable

This

chapter

ends

with

an

example

of

the

mapping

between

the

Larchant

model

and

real

systems

Motivation

As

stated

in

Chapter

the

overall

goal

of

Larchant

is

to

provide

a

cached

distributed

shared

store

that

makes

data

sharing

simple

and

efficient

That

is

why

Larchant

enables

programs

to

access

and

modify

shared

distributed

persistent

data

without

dealing

with

explicit

I/O

to

a

file

or

a

database

system

Applications

navigate

through

the

pointer

graph

by

following

pointers

from

a

persistent

root

directly

in

virtual

memory

with

the

system

moving

data

between

main

memory

and

the

secondary

storage

as

needed

Larchant

maintains

coherent

replicas

of

shared

data

in

each

process

participating

in

a

cooperative

task

Such

local

caching

improves

performance

and

data

availability

Consider

the

following

example

that

illustrates

the

sharing

behavior

of

the

applications

we

are

interested

in

interactive

computer

aided

design

-LRB-

CAD

-RRB-

and

CHAPTER

LARCHANT

MODEL

cooperative

tools

Suppose

there

is

an

object

named

New

EU

Building

that

constitutes

a

persistent

root

-LRB-

for

instance

the

object

name

has

been

registered

in

the

name

server

-RRB-

This

object

contains

a

pointer

to

each

of

the

building

floors

Say

that

an

architect

using

a

CAD

application

wants

to

visualize

the

plan

of

the

Floor

Then

the

CAD

tool

simply

follows

the

pointer

in

the

New

EU

Building

that

points

to

the

Floor

object

Transparently

this

object

is

cached

locally

by

reading

it

from

disk

Later

if

another

application

on

some

other

site

also

needs

to

access

the

Floor

object

-LRB-

e.g.

an

engineer

is

checking

the

size

of

a

heating

duct

-RRB-

the

system

transparently

gets

a

replica

of

the

object

according

to

some

coherence

protocol

In

these

kind

of

applications

illustrated

by

the

previous

example

changes

made

by

an

application

-LRB-

running

on

some

site

-RRB-

must

be

quickly

integrated

in

the

shared

store

in

order

to

be

readable

by

other

applications

-LRB-

running

on

different

sites

-RRB-

For

this

sharing

behavior

a

mechanism

supporting

fine-grained

client-toclient

transfers

instead

of

a

client-server-client

transfer

is

the

most

appropriate

In

addition

applications

running

on

different

sites

may

have

to

access

the

same

data

at

the

same

time

To

support

this

sharing

efficiently

the

sites

where

such

applications

are

running

need

to

cache

replicas

of

the

same

data

This

will

become

more

evident

as

the

physical

memory

on

each

site

grows

and

data

remains

in

main

memory

for

longer

periods

Another

characteristic

of

the

applications

we

envisage

to

support

is

that

they

may

require

high

performance

data

manipulation

in

main

memory

This

happens

for

instance

when

a

CAD

tool

performs

civil

engineering

computations

Current

Solutions

for

Distributed

Data

Sharing

In

the

past

distributed

data

sharing

has

been

supported

through

file

systems

object-oriented

databases

RPC-based

systems

and

distributed

shared

memory

-LRB-

DSM

-RRB-

In

general

such

systems

are

inefficient

hard

to

use

or

poorly

adapted

to

the

sharing

behavior

of

our

class

of

applications

With

these

systems

applications

programmers

must

deal

with

three

very

different

Application

Programming

Interface

-LRB-

API

-RRB-

sets

depending

whether

the

data

is

in

main

memory

on

secondary

storage

or

on

a

remote

site

In

contrast

applications

above

Larchant

deal

only

with

memory

using

a

single

simple

and

familiar

API

File

Systems

Nowadays

distributed

file

systems

-LSB-

-RSB-

are

the

dominant

technology

for

the

distributed

sharing

of

persistent

data

A

problem

with

file

systems

is

that

they

do

not

support

the

complex

data

types

needed

by

the

applications

we

are

interested

in

Since

files

are

unstructured

the

representation

of

data

in

the

file

system

does

not

match

the

type

system

used

by

the

applications

programming

language

Therefore

applications

programmers

must

worry

about

I/O

format

conversion

-LRB-

marshaling

and

unmarshaling

CHAPTER

LARCHANT

MODEL

data

-RRB-

This

is

tedious

error-prone

and

detracts

programmers

from

their

most

important

task

application

design

and

development

Finally

being

client-server

architectures

distributed

file

systems

do

not

support

fine-grained

client-to-client

transfers

as

desired

for

the

sharing

behavior

we

are

considering

-LRB-

described

in

Section

-RRB-

Object-Oriented

Databases

One

alternative

for

the

distributed

sharing

of

persistent

data

is

the

objectoriented

database

-LRB-

OODB

-RRB-

technology

-LSB-

-RSB-

-LSB-

-RSB-

Thor

-LSB-

-RSB-

and

GemStone

-LSB-

-RSB-

are

only

a

few

examples

In

contrast

with

file

systems

OODBs

provide

excellent

support

for

complex

data

types

In

other

words

they

preserve

the

structure

of

data

-LRB-

no

marshaling

is

needed

-RRB-

However

they

are

very

heavyweight

and

often

come

with

their

own

specialized

programming

language

OODBs

are

optimized

towards

searching

for

relevant

data

-LRB-

through

indexing

query

languages

etc.

-RRB-

possibly

with

modest

updates

and

minimal

cooperative

read/write

interaction

Thus

they

take

the

view

that

data

resides

mostly

on

secondary

storage

with

main

memory

being

used

as

a

temporary

scratch

buffer

In

contrast

our

applications

need

high

performance

data

manipulation

in

main

memory

Like

distributed

file

systems

existing

distributed

OODBs

are

limited

by

their

client-server

architectures

no

fine-grained

client-to-client

transfers

In

addition

they

do

not

support

large-scale

sharing

RPC-based

Systems

Remote

Procedure

Call

-LRB-

RPC

-RRB-

-LSB-

-RSB-

is

a

basic

communication

mechanism

that

forms

the

basis

for

the

client-server

model

-LSB-

-RSB-

RPC

solves

the

problems

of

identification

and

remote

access

However

every

remote

data

access

is

burdened

with

communication

to

the

server

which

becomes

a

performance

and

availability

bottleneck

This

makes

the

client-server

architecture

inadequate

for

interactive

CAD

and

cooperative

applications

In

addition

RPC

does

not

support

coherence

of

data

viewed

by

multiple

clients

Finally

it

imposes

an

interface

definition

language

to

program

remote

data

access

separate

from

the

programming

language

and

the

client-server

model

is

often

unnatural

DSM

Systems

A

distributed

shared

memory

-LRB-

DSM

-RRB-

-LSB-

-RSB-

is

a

software

device

emulating

a

single

shared

memory

over

a

network

DSM

systems

maintain

the

illusion

of

a

distributed

shared

memory

by

synchronizing

data

access

and

moving

data

between

processes

when

required

transparently

to

applications

CHAPTER

LARCHANT

MODEL

A

DSM

supports

client-to-client

transfers

and

replication

as

needed

by

interactive

CAD

and

cooperative

applications

Data

in

memory

is

accessed

via

the

type

system

of

some

programming

language

and

is

simple

fast

and

intuitive

This

makes

application

programming

very

easy

since

it

provides

the

same

memory

abstraction

as

in

the

centralized

case

DSMs

-LSB-

-RSB-

have

been

mostly

used

to

run

parallel

algorithms

on

networks

of

workstations

However

they

have

not

been

used

for

general

programming

in

part

because

of

the

lack

of

support

for

persistence

by

reachability

Cached

Distributed

Shared

Stores

We

claim

that

the

simplicity

of

programming

provided

by

a

DSM

mechanism

can

be

extended

to

distributed

applications

with

persistent

objects

The

Larchant

model

is

that

of

a

cached

distributed

shared

store

resulting

from

the

combined

approach

of

DSM

and

PBR

It

offers

a

shared

memory

spanning

every

site

in

a

network

including

secondary

storage

This

model

offers

transparent

distribution

and

persistence

Applications

have

uniform

access

to

any

object

in

the

system

independently

of

its

location

The

model

hides

both

the

distinction

between

local

and

remote

data

and

the

distinction

between

short-term

and

long-term

storage

Applications

navigate

through

the

shared

store

by

following

pointers

in

virtual

memory

The

system

moves

the

necessary

data

between

main

memory

and

secondary

storage

or

between

the

main

memory

of

remote

sites

according

to

application

needs

We

have

found

very

few

systems

extending

DSM

with

persistence

in

the

literature

One

of

them

is

Casper

-LSB-

-RSB-

It

takes

advantage

of

the

external

pager

mechanism

-LSB-

-RSB-

of

the

Mach

operating

system

-LSB-

-RSB-

to

provide

transparent

access

to

data

in

secondary

storage

Given

that

its

GC

specification

is

sketchy

-LSB-

-RSB-

it

is

not

clear

how

PBR

could

be

supported

Feeley

et

al

-LSB-

-RSB-

describe

a

transactional

DSM

that

supports

fine-grained

distributed

data

sharing

in

cooperative

applications

However

there

is

no

support

for

PBR

Larchant

Model

The

main

goal

of

this

description

of

the

Larchant

model

is

to

establish

the

environment

in

which

the

GC

algorithm

execute

-LRB-

see

Chapter

-RRB-

The

model

describes

the

interaction

among

applications

and

coherence

of

data

We

only

present

those

operations

that

are

relevant

w.r.t.

GC

The

model

is

general

in

the

sense

that

it

makes

minimal

assumptions

e.g.

we

tolerate

arbitrary

writes

and

non-coherent

data

Many

events

traditionally

associated

with

coherence

management

are

not

present

some

because

they

are

not

relevant

to

PBR

-LRB-

e.g.

non-pointer

writes

-RRB-

others

because

our

GC

algorithm

is

independent

of

them

-LRB-

e.g.

DSM

tokens

or

locks

-RRB-

as

will

be

made

clear

in

Chapter

Given

its

minimal

assumptions

and

consequent

generality

CHAPTER

LARCHANT

MODEL

to

some

object

GC-root

x

y

z

bunch

B

u

t

from

some

object

in

another

bunch

v

in

another

bunch

Figure

A

bunch

with

objects

inside

we

believe

the

Larchant

model

is

capable

of

describing

a

wide

range

of

cached

distributed

shared

stores

The

Larchant

model

has

the

following

components

memory

model

network

model

process

model

mutator

model

and

coherence

model

We

describe

them

now

Memory

Model

In

Larchant

the

memory

is

structured

at

two

levels

of

granularity

-LRB-

see

Figure

-RRB-

-LRB-

i

-RRB-

By

definition

the

object

is

the

unit

of

allocation

deallocation

identification

and

coherence

-LRB-

see

Section

-RRB-

an

object

may

contain

any

number

of

references

-LRB-

ii

-RRB-

The

memory

is

divided

in

partitions

called

bunches

a

bunch

is

the

unit

of

caching

and

collection

it

contains

any

number

of

objects

An

object

resides

entirely

within

a

single

bunch

A

reference

is

either

-LRB-

value

zero

-RRB-

or

points

to

an

object

We

assume

that

a

reference

is

a

plain

pointer

applications

may

arbitrarily

assign

a

reference

with

a

legal

pointer

value

-LRB-

see

Section

-RRB-

This

is

a

worst

case

from

the

GC

point

of

view

Hereafter

objects

are

noted

x

y

z

etc.

A

pointer

variable

ptr

inside

object

x

is

noted

x.ptr

The

address

of

object

x

is

noted

@x

Bunches

are

noted

uppercase

B

C

etc.

Objects

are

accessed

locally

via

caching

They

can

be

replicated

on

multiple

sites

Their

coherence

is

maintained

with

a

coherence

protocol

-LRB-

see

its

model

in

Section

-RRB-

CHAPTER

LARCHANT

MODEL

Network

Model

A

distributed

system

is

a

set

of

processes

communicating

by

messages

with

no

shared

memory

An

atomic

event

E

at

some

process

i

is

noted

<E>

i.

Events

at

some

process

occur

in

some

serial

order

We

use

the

classical

relation

happens

before

-LSB-

-RSB-

For

any

message

M

we

note

<send.M>

i

the

sending

event

at

process

i

and

<deliver.M>

j

the

delivery

and

processing

of

M

at

its

receiver

process

j

We

assume

causally-ordered

communication

-LSB-

-RSB-

Process

Model

A

process

may

contain

any

number

of

threads

It

is

composed

of

a

mutator

a

collector

and

a

coherence

engine

implementing

the

coherence

protocol

-LRB-

see

Section

-RRB-

Note

that

some

processes

may

lack

a

mutator

or

a

collector

for

example

a

system

server

does

not

have

a

mutator

A

mutator

dynamically

modifies

the

pointer

graph

it

creates

objects

and

manipulates

references

-LRB-

recall

Section

-RRB-

The

collector

observes

the

pointer

graph

in

order

to

identify

and

reclaim

unreachable

objects

The

coherence

engine

implements

the

coherence

protocol

Mutators

from

different

processes

do

not

send

messages

directly

to

each

other

They

communicate

by

causing

and

observing

updates

on

the

shared

store

Messages

between

processes

flow

only

on

behalf

of

collectors

or

coherence

engines

In

the

following

sections

we

describe

the

mutator

model

and

the

coherence

model

The

collector

model

is

described

in

Chapter

Mutator

Model

Mutators

modify

the

pointer

graph

they

create

objects

dereference

pointers

and

assign

pointers

Pointer

assignments

modify

objects

reachability

As

described

in

Section

a

pointer

assignment

can

result

in

-LRB-

i

-RRB-

creation

of

a

new

reference

-LRB-

to

an

object

just

created

-RRB-

-LRB-

ii

-RRB-

duplication

of

an

already

existing

reference

and

-LRB-

iii

-RRB-

discarding

a

reference

to

an

object

Note

that

cases

-LRB-

ii

-RRB-

and

-LRB-

iii

-RRB-

may

happen

as

the

result

of

a

single

assignment

operation

A

duplication

of

pointer

y.ptr

pointing

to

object

z

such

that

x.ptr

will

also

point

to

z

executed

by

the

mutator

of

a

process

i

is

noted

x.ptr

y.ptr

i

This

operation

is

atomic

at

process

i

This

has

the

significance

that

the

model

does

not

allow

hidden

temporary

pointer

variables

such

as

registers

and

stacks

This

does

not

restrict

the

generality

of

the

model

because

it

is

always

possible

to

expose

temporary

variables

in

the

shared

store

that

this

is

not

a

restriction

in

fact

our

GC

algorithms

can

be

equally

applied

to

RPC-based

distributed

systems

However

this

is

out

of

the

scope

of

this

document

CHAPTER

LARCHANT

MODEL

j

B

x

C

y

proc

j

B

x

C

y

proc

j

D

z

D

z

proc

k

proc

k

proc

i

proc

i

B

x

C

y

proc

j

D

z

proc

k

proc

i

-LRB-

a

-RRB-

initial

situation

-LRB-

b

-RRB-

after

execution

of

x

y

-LRB-

c

-RRB-

after

execution

of

y

i

Figure

Prototypical

example

of

mutator

execution

Object

creation

is

a

special

case

of

pointer

assignment

We

assume

that

all

objects

always

existed

an

object

creation

consists

of

setting

up

a

path

from

the

persistent

root

to

that

object

by

means

of

an

assignment

operation

When

there

is

no

ambiguity

we

make

the

simplification

that

objects

have

only

one

pointer

inside

and

identify

x.ptr

with

x.

Note

that

this

simplification

does

not

restrict

the

validity

of

the

model

w.r.t.

GC

We

say

that

an

object

y

points

to

an

object

z

when

y

contains

@z

We

say

that

an

object

x

is

when

its

pointer

variable

does

not

point

to

any

object

i.e.

it

contains

the

value

zero

-LRB-

pointer

-RRB-

An

assignment

operation

performed

by

process

i

such

that

x

is

made

to

point

to

the

object

pointed

from

y

is

noted

x

y

i

As

will

be

shown

in

Chapter

-LRB-

when

proving

the

correctness

of

the

distributed

garbage

collection

algorithm

-RRB-

from

the

GC

perspective

any

mutator

operation

can

be

reduced

to

the

following

prototypical

example

-LRB-

see

Figure

-RRB-

Consider

CHAPTER

LARCHANT

MODEL

two

objects

x

and

y

located

in

bunches

B

and

C

respectively

Initially

x

is

and

y

points

to

object

z

located

in

bunch

D.

Now

mutators

within

processes

i

and

j

execute

the

following

operations

x

y

i

y

j

such

that

at

the

end

x

points

to

z

and

y

is

We

say

that

object

x

is

GC-dirty

after

being

modified

We

say

a

bunch

is

GC

dirty

if

it

contains

a

GC-dirty

object

An

object

remains

GC-dirty

until

its

contents

are

scanned

looking

for

pointers

to

other

objects

-LRB-

see

Section

-RRB-

Coherence

Model

The

coherence

engine

provides

support

for

local

data

coherent

access

via

caching

according

to

the

coherence

model

As

mentioned

in

Section

by

definition

an

object

is

the

granule

of

coherence

However

the

size

of

a

coherence

granule

is

not

relevant

for

the

present

discussion

in

fact

a

coherence

granule

might

contain

several

objects

In

Section

we

discuss

how

the

GC

algorithms

are

-LRB-

not

-RRB-

affected

when

a

coherence

granule

contains

several

objects

Note

that

the

granule

of

caching

a

bunch

is

different

from

the

coherence

granule

an

object

The

reason

is

that

applications

do

not

necessarily

need

coherent

access

to

every

object

in

a

bunch

For

some

applications

it

is

sufficient

to

access

incoherent

object

replicas

This

difference

of

caching

and

coherence

granularity

also

contributes

to

avoid

the

problem

of

false

sharing

-LSB-

-RSB-

To

simplify

notation

we

assume

that

all

data

is

cached

by

every

process

In

order

to

model

the

case

where

an

object

is

cached

only

in

some

processes

we

represent

an

object

not

cached

in

some

process

i

as

being

cached

with

value

zero

-LRB-

pointer

-RRB-

A

bunch

B

can

be

cached

by

multiple

processes

The

image

of

B

in

process

i

is

called

i

's

replica

of

B

and

is

noted

Bi

Similarly

the

replicas

of

x

and

y

observed

by

processes

i

and

j

are

noted

xi

and

xj

respectively

The

dissemination

of

an

object

x

from

a

process

i

to

process

j

is

done

by

a

propagate

message

the

corresponding

events

are

noted

send.propagate

-LRB-

x

i

j

-RRB-

i

and

deliver.propagate

-LRB-

x

i

j

-RRB-

j

The

former

event

puts

the

contents

of

xi

in

the

message

and

the

latter

copies

the

object

's

contents

from

the

message

into

xj

-LRB-

This

makes

xj

GC-dirty

-RRB-

Each

of

these

events

is

atomic

w.r.t.

other

operations

in

the

corresponding

process

At

any

point

in

time

for

each

object

x

there

is

a

single

process

that

is

assumed

to

cache

the

most

recent

version

of

x

We

call

this

process

the

owner

of

x

The

owner

of

an

object

may

change

The

transfer

of

x

's

ownership

from

process

i

to

process

j

is

done

by

a

message

the

corresponding

events

are

noted

send.owner

-LRB-

x

i

j

-RRB-

i

and

deliver.owner

-LRB-

x

i

j

-RRB-

j

The

effect

of

the

former

operation

is

that

sender

process

i

relinquishes

x

ownership

the

effect

of

the

latter

operation

is

that

receiving

process

j

becomes

the

new

owner

of

x.

Each

of

these

events

is

atomic

w.r.t.

other

operations

in

the

corresponding

process

As

will

be

made

clear

in

Chapters

and

these

events

-LRB-

ownership

transfer

-RRB-

CHAPTER

LARCHANT

MODEL

are

not

relevant

for

the

GC

algorithm

However

we

take

advantage

of

them

for

implementation

purposes

Other

coherence

operations

can

be

modeled

as

special

cases

of

the

above

ones

See

Section

for

more

details

on

this

In

conclusion

our

coherence

model

is

based

on

a

minimal

set

of

operations

that

are

common

to

all

known

coherence

protocols

the

owner

process

of

an

object

x

caches

its

most

recent

version

and

propagates

x

's

updates

to

other

processes

Non-owner

processes

caching

coherent

versions

of

x

-LRB-

equal

to

the

owners

's

version

-RRB-

can

also

propagate

x

to

other

processes

Our

model

does

not

dictate

how

the

owner

is

chosen

or

when

propagates

are

sent

-LRB-

In

a

practical

system

coherence

operations

are

caused

by

mutator

activity

-RRB-

It

prescribes

no

particular

coherence

or

concurrency

control

algorithm

as

long

as

the

ordering

is

consistent

with

causality

In

fact

we

even

allow

conflicting

writes

at

different

processes

however

only

the

owner

gets

to

propagate

its

updates

Mapping

of

Real

Systems

to

the

Larchant

Model

In

this

section

we

show

how

the

Larchant

model

applies

to

a

real

system

DSM

with

the

entry-consistency

protocol

-LSB-

-RSB-

-LRB-

This

is

the

coherence

protocol

that

was

implemented

in

Larchant

more

details

in

Chapter

-RRB-

This

protocol

provides

the

traditional

model

of

multiple

readers

and

a

single

writer

at

any

point

in

time

there

can

either

be

several

read

tokens

or

one

exclusive

write

token

associated

with

a

shared

object

Processes

holding

a

read

token

are

assured

a

coherent

replica

of

the

corresponding

object

Every

shared

object

has

a

owner

which

is

either

the

process

currently

holding

the

object

's

write

token

or

the

process

that

last

held

the

write

token

A

write

token

can

only

be

obtained

from

the

object

's

owner

A

read

token

can

be

obtained

from

any

process

already

holding

a

read

token

A

token

is

obtained

by

performing

a

read

or

write

token

acquire

operation

and

is

freed

by

the

corresponding

release

The

acquisition

of

a

write

token

for

an

object

implies

the

invalidation

of

every

readable

replica

of

that

object

in

other

processes

Each

process

receiving

an

invalidation

message

acknowledges

it

via

a

reply

It

is

clear

that

the

acquisition

of

a

read

token

in

the

entry-consistency

protocol

is

modeled

by

the

propagate

operation

The

acquisition

of

a

write

token

is

modeled

by

the

operations

propagate

and

ownership

transfer

The

invalidation

of

an

object

in

the

entry-consistency

protocol

is

modeled

as

a

special

propagate

operation

in

which

the

propagated

object

is

Summary

In

this

chapter

we

briefly

discussed

the

most

relevant

solutions

currently

used

to

support

distributed

data

sharing

file

systems

object-oriented

databases

CHAPTER

LARCHANT

MODEL

etc.

In

general

such

systems

are

inefficient

hard

to

use

or

poorly

adapted

to

the

sharing

behavior

of

our

class

of

applications

Being

client-server

architectures

distributed

file

systems

and

existing

distributed

OODBs

do

not

support

fine-grained

client-to-client

transfers

as

desired

for

the

sharing

behavior

we

are

considering

RPC-based

distributed

systems

impose

an

interface

definition

language

to

program

remote

data

access

separate

from

the

programming

language

and

the

client-server

model

is

often

unnatural

Existing

DSMs

have

not

been

used

for

general

programming

in

part

because

of

the

lack

of

support

for

persistence

by

reachability

Then

we

described

the

Larchant

model

Applications

above

Larchant

deal

only

with

memory

using

a

single

simple

and

familiar

API

-LRB-

Application

Programming

Interface

-RRB-

sharing

an

object

is

just

the

matter

of

following

a

pointer

An

important

aspect

to

retain

is

the

set

of

mutator

and

coherence

operations

pointer

assignment

and

object

propagation

A

mutator

dynamically

modifies

the

pointer

graph

by

executing

assignment

operations

In

the

coherence

model

the

owner

process

of

an

object

x

caches

its

most

recent

version

and

propagates

x

's

updates

to

other

processes

-LRB-

The

owner

of

an

object

may

change

-RRB-

Nonowner

processes

caching

coherent

versions

of

x

-LRB-

equal

to

the

owners

's

version

-RRB-

can

also

propagate

x

to

other

processes

Our

model

does

not

dictate

how

the

owner

is

chosen

or

when

propagates

are

sent

These

operations

will

be

of

great

value

in

Chapter

when

describing

GC

in

Larchant

because

they

establish

the

environment

for

which

the

GC

algorithm

is

designed

Chapter

Garbage

Collection

in

Larchant

In

this

chapter

we

describe

the

GC

algorithm

designed

for

Larchant

We

start

by

presenting

the

fundamental

problems

that

must

be

solved

and

an

outline

of

the

corresponding

solutions

Then

we

give

a

general

overview

of

the

GC

algorithm

A

detailed

description

follows

which

includes

the

GC

model

the

collection

of

replicated

bunches

and

the

reclamation

of

cycles

of

garbage

This

chapter

ends

with

a

formal

proof

of

the

correctness

-LRB-

safety

and

liveness

-RRB-

of

the

distributed

GC

algorithm

The

GC

algorithm

is

designed

for

the

Larchant

model

presented

in

the

previous

chapter

In

Chapter

we

show

how

the

GC

algorithms

apply

to

a

real

system

with

a

particular

coherence

protocol

-LRB-

entry-consistency

-LSB-

-RSB-

-RRB-

The

main

issues

we

tackle

in

this

chapter

are

related

to

scalability

asynchrony

and

orthogonality

to

coherence

of

the

GC

algorithm

In

addition

to

the

basic

requirement

of

good

performance

these

aspects

raise

interesting

problems

in

terms

of

safety

A

fundamental

requirement

of

any

GC

algorithm

is

to

be

correct

i.e.

safe

and

live

and

ideally

complete

-LRB-

recall

Section

on

terminology

-RRB-

If

the

GC

algorithm

fulfills

this

requirement

then

it

ensures

referential

integrity

and

reclaims

garbage

In

addition

safety

liveness

and

completeness

must

be

ensured

without

degrading

significantly

applications

performance

In

other

words

not

only

the

overall

cost

of

GC

must

be

as

small

as

possible

but

also

interactive

applications

must

not

be

disrupted

by

long

continuous

pauses

that

would

be

annoying

to

the

user

Ideally

a

GC

algorithm

would

be

complete

thus

it

would

eventually

reclaim

all

unreachable

data

Reference-counting

algorithms

for

example

are

incomplete

since

they

do

not

reclaim

cycles

of

garbage

The

only

known

provably

complete

algorithms

are

based

on

global

tracing

are

synchronous

and

require

coherent

data

-LRB-

recall

Chapter

-RRB-

This

does

not

scale

interferes

with

applications

coherence

needs

causes

extra

I/O

and

does

not

have

good

performance

Apparently

the

problem

is

hopeless

However

as

we

will

show

in

this

chapter

it

is

possible

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

to

do

some

useful

work

without

performing

a

global

trace

and

therefore

to

avoid

its

drawbacks

Problems

and

Outline

of

Solutions

In

this

section

we

describe

the

specific

problems

the

GC

algorithm

has

to

solve

We

also

outline

the

corresponding

solutions

They

concern

scalability

coherence

interference

extra

I/O

and

performance

A

common

thread

to

our

solutions

is

that

GC

is

opportunistic

i.e.

it

does

not

cause

events

that

mutators

would

eventually

cause

the

collector

waits

for

them

to

happen

and

then

takes

advantage

of

it

This

will

become

clear

afterwards

Scalability

We

claim

that

perfect

completeness

is

not

feasible

in

a

large-scale

system

Thus

we

propose

an

approximate

solution

that

is

not

provably

complete

but

which

we

believe

adequate

for

all

real-life

situations

because

it

takes

advantage

of

locality

-LRB-

as

will

be

explained

in

Section

-RRB-

Our

solution

combines

partitioned

tracing

-LRB-

within

bunches

-RRB-

with

referencelisting

-LRB-

across

bunch

boundaries

-RRB-

Each

bunch

replica

is

collected

independently

of

other

bunches

and

of

other

replicas

of

the

same

bunch

incoming

cross-bunch

pointers

are

members

of

the

GC-root

A

bunch

replica

is

collected

in

the

process

where

it

is

currently

cached

with

a

tracing

algorithm

The

intra-bunch

collector

is

the

system

component

that

identifies

and

reclaims

garbage

inside

a

bunch

The

cross-bunch

collector

is

the

system

component

that

manages

the

cross-bunch

reference-lists

it

does

not

reclaim

objects

it

simply

updates

the

mentioned

lists

This

algorithm

is

not

complete

since

it

does

not

reclaim

cross-bunch

cycles

of

garbage

Apparently

the

only

way

of

reclaiming

such

cycles

is

to

perform

a

global

trace

which

is

not

scalable

and

would

generate

a

huge

amount

of

extra

I/O

The

problem

of

reclaiming

cross-bunch

cycles

of

garbage

is

addressed

below

in

Section

Coherence

Interference

The

problem

of

coherence

interference

is

illustrated

by

the

following

examples

Imagine

the

intra-bunch

collector

scanning

an

object

replica

to

find

its

internal

pointers

and

follow

them

to

continue

the

traversal

of

the

pointer

graph

Then

the

question

is

does

the

collector

need

a

coherent

replica

of

the

object

being

scanned

If

the

answer

is

yes

as

it

happens

with

existing

GC

algorithms

-LRB-

recall

Chapter

-RRB-

it

is

clear

that

the

collector

will

be

interfering

with

applications

coherence

needs

Now

suppose

that

the

intra-bunch

collector

implements

a

copy

algorithm

Then

when

patching

a

pointer

inside

an

object

replica

-LRB-

to

reflect

the

new

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

location

of

a

reachable

descendent

that

has

already

been

moved

-RRB-

should

the

collector

require

exclusive

write-access

to

modify

the

object

replica

If

exclusive

write-access

is

needed

read-access

to

all

other

replicas

of

the

object

being

patched

would

have

to

be

invalidated

therefore

interfering

with

the

coherence

needs

of

applications

Another

example

of

the

coherence

interference

problem

is

related

to

safety

Imagine

an

object

z

in

bunch

D

that

is

pointed

from

several

replicas

of

an

object

y

-LRB-

in

bunch

C

-RRB-

which

are

not

necessarily

coherent

Then

if

the

collector

at

some

point

in

time

forces

every

replica

of

y

to

be

coherent

in

order

to

conclude

about

z

's

reachability

once

again

it

will

be

interfering

with

applications

coherence

needs

As

stated

in

Chapter

GC

algorithms

found

in

the

literature

do

not

handle

the

coherence

interference

problem

They

implicitly

assume

that

either

there

are

no

replicas

or

all

replicas

are

kept

coherent

at

all

times

The

adoption

of

such

solutions

in

a

cached

distributed

shared

store

such

as

Larchant

would

incur

into

unacceptable

costs

due

to

communication

and

synchronization

overhead

To

avoid

the

coherence

interference

problem

the

collection

of

a

bunch

replica

must

make

progress

even

if

the

objects

it

contains

are

not

coherent

-LRB-

in

the

process

where

the

intra-bunch

collection

is

taking

place

-RRB-

We

achieve

this

goal

by

making

our

GC

algorithm

orthogonal

to

coherence

Thus

the

garbage

collector

can

work

with

objects

that

are

incoherent

for

applications

purposes

while

being

safe

and

live

The

fundamental

observation

that

guided

our

design

is

that

GC

coherence

needs

are

less

strict

than

applications

-LSB-

-RSB-

This

enables

GC

to

be

performed

without

interfering

with

applications

coherence

needs

and

requiring

very

little

synchronization

or

communication

More

details

on

this

in

Section

Extra

I/O

GC

based

on

global

memory

tracing

incur

into

extra

I/O

This

is

due

to

the

fact

that

such

collectors

traverse

the

pointer

graph

even

when

the

corresponding

objects

are

not

cached

in

main

memory

In

contrast

in

Larchant

-LRB-

as

mentioned

in

Section

-RRB-

a

bunch

replica

is

collected

in

a

process

where

it

is

currently

cached

by

application

request

Therefore

the

collection

of

a

bunch

does

not

cause

extra

I/O

However

a

difficulty

arises

when

reclaiming

cross-bunch

cycles

of

garbage

because

the

only

known

provably

complete

algorithms

are

based

on

global

tracing

and

employ

a

global

synchronization

This

causes

a

large

amount

of

extra

I/O

Our

solution

to

reclaim

cross-bunch

cycles

of

garbage

without

incurring

into

extra

I/O

consists

of

extending

the

tracing

performed

by

the

intra-bunch

collector

to

include

several

bunches

at

once

Bunches

are

grouped

and

collected

as

if

they

were

a

single

bunch

Thus

memory

partitions

change

dynamically

and

seamlessly

and

independently

in

each

process

in

order

to

reclaim

cycles

of

unreachable

objects

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

The

significance

of

group

collection

is

that

any

arbitrary

subset

of

the

store

can

be

collected

in

a

single

process

independently

of

the

rest

of

the

memory

The

choice

of

a

group

to

be

collected

is

heuristic

and

should

maximize

the

amount

of

garbage

reclaimed

and

minimize

the

cost

In

Section

we

describe

the

heuristic

used

in

Larchant

It

takes

advantage

of

locality

therefore

avoiding

extra

I/O

Performance

There

are

two

fundamental

difficulties

concerning

GC

performance

First

ensuring

that

GC

pause

times

remain

sufficiently

small

in

order

to

be

unnoticed

bu

the

user

independently

of

bunch

size

proportion

of

reachable

objects

and

number

of

bunch

replicas

Second

keeping

track

of

cross-bunch

references

without

instrumenting

every

pointer

assignment

with

a

write-barrier

to

find

out

if

a

new

cross-bunch

pointer

has

been

created

-LRB-

Recall

that

when

a

bunch

replica

is

collected

the

incoming

cross-bunch

pointers

are

members

of

the

GC-root

-RRB-

To

solve

the

first

problem

the

intra-bunch

collector

runs

concurrently

with

mutators

and

is

orthogonal

to

coherence

To

solve

the

second

problem

keeping

track

of

cross-bunch

pointers

without

instrumenting

every

pointer

assignment

the

cross-bunch

collector

runs

asynchronously

w.r.t.

applications

and

certain

GC

operations

are

safely

delayed

Hence

in

Larchant

both

the

intra-bunch

and

the

cross-bunch

collector

execute

concurrently

and

asynchronously

w.r.t.

applications

This

functioning

mode

raises

important

issues

in

terms

of

safety

that

will

be

addressed

intuitively

in

Sections

and

and

formally

in

Section

GC

Algorithms

The

intra-bunch

collector

uses

a

tracing

algorithm

Now

the

question

is

what

type

of

tracing

mark-and-sweep

or

copy

As

mentioned

in

Chapter

both

algorithms

have

their

pros

and

cons

In

particular

collection

time

depends

of

the

heap

size

and

the

proportion

of

reachable

objects

respectively

We

believe

that

some

applications

will

benefit

from

a

copy

collector

while

for

others

a

mark-and-sweep

algorithm

is

the

most

appropriate

Such

a

choice

depends

on

the

application

programming

language

memory

usage

specific

application

requirements

in

terms

of

GC

pause

time

etc.

Note

that

even

for

a

single

application

the

best

GC

algorithm

may

vary

with

time

we

could

imagine

an

application

in

which

the

collector

would

be

a

mark-and-sweep

until

memory

gets

too

fragmented

and

then

changing

automatically

to

a

copy

algorithm

The

bottom

line

is

that

there

is

no

universal

solution

For

this

reason

we

decided

to

support

both

mark-and-sweep

and

copy

algorithms

It

is

up

to

the

application

programmer

to

chose

the

most

appropriate

GC

algorithm

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

General

Overview

The

main

aspects

of

GC

in

Larchant

can

be

summarized

as

follows

ffl

Each

process

collects

independently

and

asynchronously

from

other

processes

ffl

A

bunch

is

collected

independently

and

asynchronously

from

the

rest

of

the

store

with

a

tracing

algorithm

-LRB-

intra-bunch

collector

-RRB-

that

runs

concurrently

with

mutators

GC

between

bunches

-LRB-

cross-bunch

collector

-RRB-

is

based

on

reference-listing

ffl

To

reclaim

cycles

of

garbage

bunches

are

dynamically

grouped

and

collected

at

the

same

time

in

some

process

groups

are

formed

with

an

heuristic

that

avoids

extra

I/O

ffl

The

GC

algorithm

is

orthogonal

to

coherence

i.e.

each

bunch

replica

is

collected

independently

and

no

coherence

operation

is

needed

for

GC

purposes

ffl

The

reference-listing

algorithm

is

asynchronous

w.r.t.

applications

and

GC

specific

messages

are

exchanged

asynchronously

in

the

background

-LRB-

we

assume

causal

delivery

-RRB-

To

enable

a

bunch

to

be

collected

independently

from

the

rest

of

the

memory

a

bunch

keeps

track

of

cross-bunch

pointers

An

outgoing

pointer

is

described

by

a

stub

and

a

incoming

pointer

by

a

scion

-LRB-

see

Figure

-RRB-

Each

bunch

replica

has

its

own

replica

of

stubs

and

scions

The

number

of

cross-bunch

pointers

to

some

object

is

safely

approximated

by

the

number

of

scions

to

that

object

By

considering

a

bunch

's

scions

as

its

GC-root

a

bunch

can

be

collected

independently

of

others

An

object

y

in

bunch

C

pointed

by

a

scion

will

not

be

reclaimed

y

is

said

protected

The

result

of

collecting

a

bunch

is

a

set

of

reachable

objects

and

a

new

set

of

stubs

Objects

not

in

the

reachable

set

are

reclaimed

By

comparing

the

sets

of

stubs

before

and

after

an

intra-bunch

collection

the

cross-bunch

collector

discovers

which

cross-bunch

pointers

were

created

and

which

stubs

no

longer

exist

In

other

words

stubs

that

were

not

in

the

stub

set

before

the

collection

but

are

in

the

-LRB-

new

-RRB-

stub

set

after

the

collection

correspond

to

cross-bunch

pointers

that

were

created

by

the

mutator

Thus

the

corresponding

scions

must

be

created

Stubs

that

were

in

the

stub

set

before

the

collection

but

are

not

in

the

-LRB-

new

-RRB-

stub

set

after

the

collection

correspond

to

cross-bunch

pointers

that

no

longer

exist

Thus

the

corresponding

scions

may

be

deleted

In

Section

we

describe

when

and

how

scions

are

created

and

deleted

names

stub

and

scion

are

inspired

by

the

similar

structures

found

in

the

SSP

-LRB-

stubscion

pair

-RRB-

Chain

message-passing

system

-LSB-

-RSB-

In

contrast

to

SSP

Chains

Larchant

's

stubs

and

scions

are

not

indirections

participating

in

the

mutator

computation

but

simply

auxiliary

data

structures

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

stubs

CbunchBbunch

y

x

z

stubs

scions

objects

scions

objects

Figure

Two

bunches

containing

objects

stubs

and

scions

Safety

is

ensured

by

ordering

constructive

events

-LRB-

creation

of

scions

-RRB-

always

before

destructive

events

-LRB-

deletion

of

scions

or

garbage

reclamation

-RRB-

that

may

depend

on

them

and

the

use

of

causal

delivery

This

a

fundamental

aspect

of

our

distributed

GC

algorithm

Finally

note

that

in

Larchant

the

root

of

persistence

is

simply

a

special

scion

that

is

never

deleted

Every

reachable

object

can

be

reached

-LRB-

directly

or

transitively

-RRB-

from

this

special

scion

GC

Model

In

this

section

we

present

the

GC

model

i.e.

we

describe

the

operations

that

manipulate

the

stubs

and

scions

and

the

operations

that

characterize

a

tracing

intra-bunch

collection

Then

based

on

this

set

of

operations

in

the

rest

of

this

chapter

we

describe

in

detail

the

intra-bunch

and

cross-bunch

algorithms

and

prove

their

correctness

Stubs

and

Scions

An

outgoing

cross-bunch

pointer

is

described

by

a

stub

and

an

incoming

crossbunch

pointer

by

a

scion

-LRB-

see

Figure

-RRB-

Thus

for

each

cross-bunch

pointer

there

is

a

stub

in

the

source

bunch

and

a

scion

in

the

target

bunch

Each

bunch

replica

has

its

own

replica

of

stubs

and

scions

Note

that

stubs

and

scions

are

simply

auxiliary

data

structures

they

are

not

seen

by

applications

code

A

cross-bunch

pointer

from

object

x

B

to

object

y

C

is

noted

Bx!Cy

A

pointer

Bx!Cy

observed

by

process

i

is

noted

-LRB-

Bx

-RRB-

i

Cy

A

stub

describing

Bx!Cy

is

noted

stub

-LRB-

Bx

Cy

-RRB-

The

scion

for

the

same

crossbunch

pointer

is

noted

scion

-LRB-

Bx

Cy

-RRB-

The

cross-bunch

collector

manipulates

stubs

and

scions

according

to

the

reference-listing

algorithm

-LRB-

see

Section

-RRB-

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

x

i

j

y

i

z

i

y

j

z

j

inOwnerPtr

-LRB-

y

j

i

-RRB-

outOwnerPtr

-LRB-

y

j

i

-RRB-

inOwnerPtr

-LRB-

z

j

i

-RRB-

outOwnerPtr

-LRB-

z

j

i

-RRB-

C

i

C

Figure

Bunch

replicas

with

outOwnerPtrs

and

inOwnerPtrs

The

dashed

line

indicates

that

z

is

not

cached

in

j

The

creation

of

stub

-LRB-

Bx

Cy

-RRB-

executed

in

process

i

is

noted

create.stub

-LRB-

Bx

Cy

-RRB-

i.

Note

that

there

is

no

operation

for

deleting

a

stub

As

mentioned

in

the

previous

section

a

stub

is

implicitly

deleted

by

replacing

a

set

of

stubs

with

a

new

one

resulting

from

an

intra-bunch

collection

The

creation

and

deletion

of

a

scion

involves

two

bunches

the

source

and

the

target

of

the

corresponding

cross-bunch

pointer

The

operation

of

creating

scion

-LRB-

Bx

Cy

-RRB-

is

done

with

a

message

thus

it

is

constituted

of

two

events

send.create.scion

-LRB-

Bx

Cy

-RRB-

i

issued

at

the

source

process

i

-LRB-

caching

the

source

object

x

-RRB-

and

deliver.create.scion

-LRB-

Bx

Cy

-RRB-

j

at

the

receiving

process

j

-LRB-

caching

the

target

object

y

-RRB-

These

two

operations

are

atomic

in

the

process

where

they

are

executed

Similarly

the

operation

of

deleting

scion

-LRB-

Bx

Cy

-RRB-

is

constituted

by

send.delete.scion

-LRB-

Bx

Cy

-RRB-

i

and

deliver.delete.scion

-LRB-

Bx

Cy

-RRB-

j

There

is

a

special

type

of

stubs

and

scions

called

outOwnerPtr

and

inOwnerPtr

respectively

These

special

stubs

and

scions

are

used

only

between

replicas

of

the

same

bunch

Each

bunch

replica

has

its

own

set

of

outOwnerPtrs

and

inOwnerPtrs

There

is

a

pair

outOwnerPtr-inOwnerPtr

for

each

object

that

is

replicated

or

that

is

directly

accessible

from

a

replicated

object

as

illustrated

by

Figure

objects

yj

-LRB-

replicated

-RRB-

and

zj

-LRB-

directly

accessible

from

a

replicated

object

-RRB-

object

x

is

not

accessible

by

following

a

pointer

in

process

j

thus

there

is

no

outOwnerPtr-inOwnerPtr

associated

to

x.

Note

that

a

pair

outOwnerPtrinOwnerPtr

points

always

to

the

owner

of

the

concerned

object

-LRB-

possibly

through

some

indirection

-RRB-

The

inOwnerPtr

for

object

y

owned

by

process

i

replicated

in

process

j

is

noted

inOwnerPtr

-LRB-

y

j!i

-RRB-

The

corresponding

outOwnerPtr

is

noted

outOwnerPtr

-LRB-

y

j!i

-RRB-

These

special

stubs

and

scions

are

needed

to

ensure

the

safety

of

the

intra-bunch

collector

as

will

be

explained

in

Sections

and

Basically

inOwnerPtrs

are

part

of

the

GC-root

when

collecting

a

bunch

replica

the

process

owning

the

source

and

target

objects

is

the

same

the

message

becomes

a

simple

procedure

call

we

assume

instantaneous

delivery

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

Tracing

a

Bunch

An

intra-bunch

collection

of

bunch

replica

Bi

executed

at

process

i

is

noted

GCi

-LRB-

B

-RRB-

We

do

not

assume

that

tracing

a

bunch

is

atomic

An

intra-bunch

tracing

collector

uses

the

scan

operation

For

some

object

x

scan

determines

what

other

objects

y

z

are

pointed

to

by

x

A

scan

operation

executed

in

process

i

on

object

replica

xi

is

noted

scani

-LRB-

x

-RRB-

When

an

object

is

scanned

for

each

one

of

its

outgoing

cross-bunch

pointers

the

corresponding

stub

is

created

Note

that

an

object

can

be

scanned

even

if

its

enclosing

bunch

is

not

being

collected

-LRB-

The

circumstances

in

which

this

happens

will

become

clear

afterwards

-RRB-

A

mark-and-sweep

collector

marks

reachable

objects

The

operation

of

marking

object

replica

xi

in

process

i

is

noted

mark

-LRB-

x

-RRB-

i

A

copy

collector

moves

objects

-LRB-

to

compact

memory

and

reduce

fragmentation

-RRB-

and

patches

pointers

with

the

new

addresses

The

operation

of

moving

a

reachable

object

x

to

a

new

address

@x

is

done

with

a

message

thus

it

is

constituted

by

two

events

send.move

-LRB-

x

@x

-RRB-

i

and

deliver.move

-LRB-

x

@x

-RRB-

j

where

i

is

the

sending

process

and

j

the

receiver

-LRB-

If

i

equals

j

the

message

becomes

a

simple

procedure

call

-RRB-

A

patch

operation

executed

in

process

i

concerning

yi

pointing

to

x

that

has

been

moved

to

@x

is

noted

patch

-LRB-

y

@x

-RRB-

i.

-LRB-

In

Section

we

explain

how

every

replica

of

a

same

object

is

moved

to

the

same

address

-RRB-

Unreachable

objects

are

reclaimed

A

reclaim

operation

executed

in

process

i

concerning

object

replica

xi

is

noted

reclaim

-LRB-

x

-RRB-

i

In

a

mark-and-sweep

collector

this

is

implemented

by

adding

the

unreachable

object

's

memory

space

to

a

free

list

-LRB-

during

the

sweep

phase

-RRB-

In

a

copy

collector

all

unreachable

objects

are

reclaimed

at

once

when

the

flip

is

performed

The

nth

intra-bunch

collection

of

a

bunch

replica

Bi

scans

reachable

objects

in

Bi

therefore

creates

stubs

for

each

outgoing

pointer

These

stubs

are

inserted

in

a

new

set

of

stubs

noted

stubsn

-LRB-

Bi

-RRB-

The

set

of

stubs

before

the

nth

GCi

-LRB-

B

-RRB-

is

noted

stubsn

-LRB-

Bi

-RRB-

Finally

a

GC-dirty

object

-LRB-

defined

in

Section

-RRB-

remains

GC-dirty

until

scanned

Thus

when

an

object

is

GC-cleaned

the

stubs

corresponding

to

its

outgoing

cross-bunch

pointers

are

created

A

GC-dirty

bunch

remains

GC-dirty

as

long

as

it

contains

a

GC-dirty

object

Union

Rule

The

union

rule

is

common

to

the

intra-bunch

and

cross-bunch

collection

algorithms

It

ensures

safety

in

presence

of

replication

The

cross-bunch

GC

algorithm

respects

this

rule

through

the

use

of

stubs

and

scions

The

intrabunch

GC

algorithm

respects

this

rule

through

the

use

of

inOwnerPtrs

and

outOwnerPtrs

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

C

i

i

y

z

j

i

C

i

i

y

z

i

C

j

j

y

z

j

-LRB-

a

-RRB-

initial

situation

-LRB-

b

-RRB-

after

y

inOwnerPtr

-LRB-

y

j

i

-RRB-

inOwnerPtr

-LRB-

z

j

i

-RRB-

inOwnerPtr

-LRB-

y

j

i

-RRB-

inOwnerPtr

-LRB-

z

j

i

-RRB-

outOwnerPtr

-LRB-

z

j

i

-RRB-

outOwnerPtr

-LRB-

y

j

i

-RRB-

outOwnerPtr

-LRB-

y

j

i

-RRB-

outOwnerPtr

-LRB-

z

j

i

-RRB-

i

C

j

j

y

z

Figure

Example

illustrating

the

need

of

union

rule

-LRB-

single

bunch

-RRB-

The

dashed

line

indicates

that

z

is

not

cached

in

j

We

motivate

the

need

for

the

union

rule

with

two

examples

The

first

one

considers

a

single

replicated

bunch

The

second

example

considers

two

different

bunches

with

cross-bunch

pointers

Suppose

a

bunch

C

containing

objects

y

and

z.

Bunch

C

is

cached

in

processes

i

and

j.

Initially

yi

points

to

zi

then

y

is

propagated

to

j

-LRB-

Figure

-LRB-

a

-RRB-

illustrates

this

situation

-RRB-

From

now

on

the

mutator

in

j

can

access

z

by

following

the

pointer

in

yj

Now

imagine

that

due

to

mutator

activity

in

i

zi

becomes

unreachable

-LRB-

e.g.

mutator

executes

y

i

-RRB-

as

shown

in

Figure

-LRB-

b

-RRB-

Then

if

GCi

-LRB-

C

-RRB-

runs

without

considering

the

set

of

inOwnerPtrs

as

members

of

the

GC-root

zi

is

unsafely

reclaimed

This

shows

that

zi

may

be

reclaimed

only

after

it

becomes

unreachable

in

all

processes

caching

a

replica

of

bunch

C

The

second

example

is

similar

to

the

previous

one

with

the

following

difference

z

is

allocated

in

bunch

D

cached

in

process

k

-LRB-

see

Figure

-RRB-

Initially

both

replicas

of

y

point

to

z.

Now

imagine

that

due

to

mutator

activity

in

i

z

becomes

unreachable

from

yi

-LRB-

e.g.

mutator

executes

y

i

-RRB-

In

this

case

object

z

is

still

accessible

from

yj

Obviously

object

z

may

be

reclaimed

only

after

it

becomes

unreachable

from

all

replicas

of

the

source

object

y

In

short

a

target

object

can

be

reclaimed

only

if

it

is

not

reachable

from

the

union

of

all

replicas

of

its

source

objects

-LRB-

independently

of

their

location

either

in

the

same

or

in

a

different

bunch

-RRB-

We

call

this

the

union

rule

The

intra-bunch

GC

algorithm

enforces

the

union

rule

by

considering

every

inOwnerPtr

as

member

of

the

GC-root

An

object

will

be

reclaimed

only

when

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

k

D

both

replicas

point

to

z

C

C

i

j

j

y

i

y

k

z

Figure

Example

illustrating

the

need

of

union

rule

-LRB-

two

bunches

-RRB-

it

becomes

unreachable

not

only

from

any

object

locally

cached

but

also

no

longer

protected

by

any

inOwnerPtr

The

cross-bunch

GC

algorithm

enforces

the

union

rule

by

considering

every

stub

from

all

replicas

as

members

of

the

GC-root

Thus

the

last

scion

pointing

to

an

object

can

be

safely

deleted

only

after

the

corresponding

outgoing

pointers

have

disappeared

in

all

replicas

of

the

source

object

In

Sections

and

we

will

explain

with

more

detail

how

the

union

rule

is

enforced

by

the

intra-bunch

and

cross-bunch

algorithms

respectively

Intra-Bunch

GC

In

this

section

we

describe

the

intra-bunch

GC

algorithm

Both

mark-and-sweep

and

copy

collectors

are

addressed

We

first

consider

there

is

no

replication

i.e.

we

describe

the

collection

of

a

single

bunch

cached

only

in

a

single

process

Then

we

present

the

collection

of

a

bunch

replicated

in

multiple

processes

GC

without

Replication

An

intra-bunch

collection

for

a

non-replicated

bunch

proceeds

as

follows

The

GC-root

is

the

bunch

's

set

of

scions

Any

object

pointed

at

directly

from

a

scion

is

reachable

therefore

it

is

marked

-LRB-

or

moved

in

the

case

of

copy

GC

-RRB-

Every

marked

-LRB-

or

moved

-RRB-

object

is

scanned

for

pointers

If

such

an

object

points

to

another

object

inside

the

same

bunch

the

intra-bunch

collector

transitively

marks

-LRB-

or

moves

-RRB-

the

pointed-to

object

If

it

points

outside

the

enclosing

bunch

the

collector

creates

a

stub

in

the

stub

set

with

the

highest

index

Thus

the

result

of

collecting

a

bunch

is

a

set

of

reachable

objects

-LRB-

those

that

were

marked

or

moved

-RRB-

and

a

new

set

of

stubs

Objects

not

in

the

reachable

set

are

garbage

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

The

intra-bunch

collector

-LRB-

both

mark-and-sweep

and

copy

algorithms

are

considered

-RRB-

is

based

on

the

algorithm

from

Nettles

and

O'Toole

-LRB-

replication-based

technique

described

in

Section

-RRB-

and

runs

concurrently

with

the

mutator

Once

the

intra-bunch

collector

starts

running

any

object

modification

done

by

the

mutator

is

detected

-LRB-

see

how

in

Section

-RRB-

and

the

object

's

address

is

registered

in

a

log

called

GC-log

When

the

collector

flips

every

object

in

the

GC-log

is

scanned

again

-LRB-

and

moved

again

in

the

case

of

a

copy

collector

-RRB-

The

GC

finishes

when

every

reachable

object

has

been

marked

-LRB-

or

moved

and

patched

-RRB-

and

the

collector

has

flipped

-LRB-

Note

that

with

the

mark-and-sweep

collector

the

sweep

phase

is

also

done

concurrently

with

the

mutator

after

the

flip

-RRB-

Any

tracing

algorithm

could

be

used

for

the

intra-bunch

GC

However

when

using

a

concurrent

copy

algorithm

the

replication-based

technique

helps

to

avoid

disruptiveness

as

will

be

made

clear

in

Section

GC

with

Replication

The

general

algorithm

described

in

the

previous

section

applies

equally

to

the

intra-bunch

collection

of

a

replicated

bunch

The

main

additional

issues

in

this

case

are

-LRB-

i

-RRB-

to

ensure

safety

in

presence

of

replication

-LRB-

via

the

union

rule

-RRB-

and

-LRB-

ii

-RRB-

to

avoid

coherence

interference

i.e.

the

collector

must

be

able

to

collect

a

bunch

replica

containing

incoherent

objects

in

other

words

no

coherence

operations

should

be

issued

on

behalf

of

GC

Consider

a

bunch

replicated

in

multiple

processes

Each

process

runs

concurrently

to

the

mutator

an

intra-bunch

collector

thread

for

the

bunch

This

raises

the

following

questions

Must

these

collectors

synchronize

with

each

other

Does

scanning

need

coherent

data

Is

it

necessary

to

synchronize

the

flip

When

using

a

copy

GC

algorithm

-LRB-

a

-RRB-

Is

it

necessary

to

synchronize

the

collectors

to

decide

where

to

move

an

object

-LRB-

b

-RRB-

Is

it

necessary

to

perform

some

coherence

operation

before

or

after

patching

an

object

's

internal

pointers

A

yes

answer

to

any

of

these

questions

would

impact

efficiency

In

the

rest

of

this

section

we

will

show

that

conditions

exist

where

the

answers

are

all

no

As

a

consequence

the

intra-bunch

GC

algorithm

does

not

compete

with

the

case

of

a

mark-and-sweep

collector

the

flip

is

the

instant

when

the

mark

phase

is

finished

and

the

sweep

can

start

With

a

copy

collector

the

flip

consists

of

exchanging

the

roles

of

from-space

and

to-space

In

both

cases

mutators

are

halted

for

the

duration

of

the

flip

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

applications

for

coherent

data

there

is

no

synchronization

between

collectors

and

mutators

or

between

different

collectors

and

GC

specific

messages

are

asynchronous

and

exchanged

in

the

background

The

price

to

pay

for

these

features

is

some

degree

of

conservativeness

and

the

need

for

causal

delivery

-LSB-

-RSB-

of

some

messages

Union

Rule

As

mentioned

in

Section

the

intra-bunch

GC

algorithm

enforces

the

union

rule

by

considering

every

inOwnerPtr

as

member

of

the

GC-root

-LRB-

Recall

that

a

pair

outOwnerPtr-inOwnerPtr

always

point

to

the

owner

of

the

corresponding

object

-RRB-

A

pair

outOwnerPtr-inOwnerPtr

is

created

when

there

is

a

propagate

operation

that

will

enable

the

access

-LRB-

by

a

remote

process

-RRB-

to

a

locally

owned

object

for

the

first

time

-LRB-

otherwise

the

pair

already

exists

-RRB-

A

pair

outOwnerPtrinOwnerPtr

is

updated

whenever

the

owner

of

an

object

changes

in

order

to

point

always

to

the

corresponding

object

's

owner

A

pair

outOwnerPtrinOwnerPtr

is

deleted

as

the

result

of

a

garbage

collection

of

the

bunch

replica

holding

the

outOwnerPtr

These

are

the

only

operations

on

inOwnerPtrs

and

outOwnerPtrs

In

the

rest

of

this

section

we

explain

their

creation

and

deletion

with

more

detail

Before

a

send.propagate

-LRB-

y

i

j

-RRB-

i

is

issued

the

following

operations

are

executed

in

process

i

-LRB-

i

-RRB-

create

inOwnerPtr

-LRB-

y

j!i

-RRB-

-LRB-

if

it

does

not

exist

yet

-RRB-

and

-LRB-

ii

-RRB-

for

each

object

directly

reachable

from

yi

create

the

corresponding

inOwnerPtr

-LRB-

if

it

does

not

exist

yet

-RRB-

Before

the

corresponding

deliver.propagate

-LRB-

x

i

j

-RRB-

j

is

executed

perform

the

following

operations

in

process

j

-LRB-

i

-RRB-

create

outOwnerPtr

-LRB-

y

j!i

-RRB-

-LRB-

if

it

does

not

exist

yet

-RRB-

and

-LRB-

ii

-RRB-

for

each

object

directly

reachable

from

yj

create

the

corresponding

outOwnerPtr

-LRB-

if

it

does

not

exist

yet

-RRB-

-LRB-

Note

that

if

the

process

i

from

which

an

object

y

is

propagated

is

not

the

owner

of

an

object

x

that

is

directly

reachable

from

y

there

will

be

a

chain

of

pairs

outOwnerPtr-inOwnerPtr

for

x

passing

through

i

in

direction

of

x

's

owner

However

this

aspect

is

not

relevant

for

this

discussion

-RRB-

When

a

bunch

replica

Bj

is

collected

a

new

set

of

outOwnerPtrs

is

generated

-LRB-

just

like

with

stubs

for

cross-bunch

pointers

-RRB-

By

comparing

the

old

set

with

the

new

set

of

outOwnerPtrs

the

cross-bunch

collector

discovers

which

ones

have

disappeared

-LRB-

A

disappearing

outOwnerPtr

results

from

the

fact

that

the

corresponding

object

replica

is

no

longer

reachable

locally

-RRB-

For

each

disappeared

outOwnerPtr

the

collector

sends

a

delete

message

to

the

process

holding

the

corresponding

inOwnerPtr

When

a

locally

owned

object

y

in

bunch

C

is

no

longer

protected

by

any

inOwnerPtr

this

means

that

y

is

no

longer

reachable

in

any

replica

of

C

cached

in

processes

which

do

not

own

y.

Thus

y

can

be

reclaimed

if

it

becomes

unreachable

locally

-LRB-

in

the

owner

process

-RRB-

In

other

words

the

union

rule

is

evaluated

at

the

owner

process

and

consists

of

ensuring

that

an

object

will

not

be

reclaimed

at

its

owner

process

while

there

is

at

least

one

inOwnerPtr

protecting

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

that

object

Scan

The

scanning

of

an

incoherent

object

replica

yj

evidently

does

not

take

into

account

pointer

writes

occurring

at

some

other

process

-LRB-

e.g.

at

the

object

's

owner

-RRB-

However

this

is

not

a

problem

In

fact

scanning

an

out-of-date

replica

simply

results

in

making

a

more

conservative

decision

about

the

pointed

objects

reachability

Thus

objects

reachable

from

an

incoherent

replica

yj

will

not

be

reclaimed

this

is

ensured

by

the

union

rule

previously

described

The

worse

that

can

happen

is

that

the

intra-bunch

collector

in

process

j

fails

to

reclaim

some

objects

-LRB-

pointed

from

yj

-RRB-

that

are

in

fact

unreachable

because

they

are

no

longer

pointed

from

the

most

up-to-date

replica

yi

-LRB-

in

y

's

owner

-RRB-

These

objects

will

be

reclaimed

later

-LRB-

according

to

the

union

rule

-RRB-

either

after

a

coherent

replica

of

y

becomes

available

in

process

j

or

after

yj

has

become

unreachable

Move

and

Patch

In

this

section

we

study

the

move

and

patch

operations

of

an

intra-bunch

copy

collection

with

respect

to

coherence

The

first

issue

is

to

avoid

two

processes

moving

-LRB-

their

local

replicas

of

-RRB-

the

same

object

to

two

different

locations

concurrently

One

obvious

solution

to

this

problem

would

be

the

process

that

wants

to

move

an

object

would

acquire

the

object

's

ownership

before

moving

it

As

at

any

moment

there

is

a

single

owner

for

each

object

the

situation

in

which

two

processes

move

their

local

replicas

to

different

addresses

would

not

arise

However

this

solution

is

clearly

undesirable

since

it

interferes

with

applications

coherence

needs

A

simple

solution

that

does

not

interfere

with

applications

coherence

needs

is

as

follows

The

owner

process

of

x

decides

where

to

move

it

non-owner

processes

will

move

their

replicas

of

x

to

the

same

address

after

receiving

a

move

message

from

the

owner

Therefore

a

reachable

object

x

in

bunch

B

is

moved

as

follows

-LRB-

i

-RRB-

Process

i

the

owner

of

x

issues

a

send.move

-LRB-

x

@x

-RRB-

i

to

processes

caching

a

pointer

to

x

-LRB-

these

processes

are

those

for

which

a

inOwnerPtr

exists

in

i

-RRB-

including

to

itself

-LRB-

ii

-RRB-

A

process

j

executing

deliver.move

-LRB-

x

@x

-RRB-

j

moves

its

replica

xj

to

the

new

location

and

patches

pointers

accordingly

Note

that

move

messages

are

exchanged

in

the

background

i.e.

mutators

do

not

depend

on

such

messages

because

they

do

not

access

to-space

objects

In

fact

mutators

can

freely

access

a

from-space

object

for

which

a

move

is

being

executed

Thus

move

messages

do

not

disrupt

mutators

It

's

worthy

to

note

that

if

the

intra-bunch

collector

uses

some

other

copy

algorithm

instead

such

as

Baker

's

-LRB-

recall

Section

-RRB-

for

instance

the

mutator

could

be

disrupted

In

Baker

's

algorithm

any

object

in

from-space

that

is

accessed

by

the

mutator

must

first

be

moved

to

to-space

Thus

if

a

mutator

in

process

j

tries

to

access

its

replica

of

object

x

owned

by

process

i

the

mutator

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

would

have

to

block

while

the

collector

in

process

j

waits

for

the

new

address

of

x

in

to-space

The

second

important

issue

of

the

copy

algorithm

is

the

following

must

the

intra-bunch

collector

acquire

the

ownership

-LRB-

or

some

other

lock

for

coherence

purposes

-RRB-

of

an

object

in

order

to

patch

one

of

its

pointers

with

the

target

's

new

location

Surprisingly

the

answer

is

no

because

this

operation

is

visible

only

at

the

process

where

it

occurred

Each

process

patches

independently

thus

no

coherence

operation

is

required

Flip

The

intra-bunch

collector

may

flip

independently

with

no

synchronization

from

the

collectors

of

other

replicas

of

the

same

bunch

In

the

case

of

a

copy

collector

a

flip

may

occur

before

all

the

move

messages

concerning

reachable

objects

not

locally

owned

have

already

been

delivered

Thus

collectors

in

different

processes

may

flip

at

distinct

times

Therefore

the

GC

pause

time

experienced

by

a

mutator

does

not

depend

on

any

synchronization

with

other

processes

-LRB-

see

Chapter

for

performance

results

-RRB-

When

a

collector

flips

before

having

received

all

the

move

messages

for

not

locally

owned

objects

those

objects

remain

in

from-space

They

will

be

moved

later

when

the

corresponding

move

messages

are

delivered

-LRB-

In

fact

such

moves

can

be

delayed

until

the

next

local

garbage

collection

-RRB-

The

price

to

pay

for

this

lack

of

synchronization

when

flipping

is

the

following

First

the

semi-space

from

where

non-locally

owned

objects

were

not

moved

can

not

be

immediately

freed

this

can

happen

only

after

every

reachable

object

has

been

moved

Second

the

same

object

may

be

referenced

via

two

distinct

addresses

in

different

processes

the

old

address

in

from-space

-LRB-

processes

that

have

not

yet

moved

the

object

-RRB-

and

its

new

address

in

to-space

-LRB-

processes

that

have

already

moved

it

-RRB-

This

implies

that

pointers

contained

in

a

propagate

or

a

ownership

message

might

need

to

be

swizzled

In

this

case

a

flip

differs

slightly

from

the

definition

given

in

Section

In

fact

instead

of

changing

the

roles

of

from-space

and

to-space

the

flip

extends

the

from-space

with

the

to-space

This

results

in

delaying

to

free

the

original

from-space

When

flipping

those

objects

that

are

reachable

but

were

not

moved

to

tospace

-LRB-

because

the

corresponding

move

message

has

not

been

delivered

yet

-RRB-

are

scanned

and

patched

Given

that

collectors

may

flip

independently

move

messages

must

be

delivered

in

causal

order

w.r.t.

to

propagate

and

ownership

messages

Causality

prevents

a

process

from

receiving

a

propagate

-LRB-

or

a

ownership

-RRB-

message

with

a

to-space

's

worthy

to

note

that

the

patch

operation

implies

writing

into

an

object

and

on

some

DSM

coherence

protocols

-LRB-

e.g.

entry-consistency

-LSB-

-RSB-

-RRB-

this

requires

the

previous

acquisition

of

an

exclusive

write-lock

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

address

-LRB-

from

a

process

where

the

collector

has

already

flipped

-RRB-

before

the

corresponding

move

message

has

been

delivered

Cross-Bunch

GC

In

Larchant

the

cross-bunch

collector

is

based

on

the

reference-listing

algorithm

-LRB-

recall

Section

-RRB-

The

fundamental

issues

are

keeping

track

of

crossbunch

pointers

in

order

to

update

the

reference-lists

without

slowing

down

applications

significantly

and

ensuring

safety

in

presence

of

replication

As

mentioned

in

Section

instrumenting

every

pointer

assignment

with

a

write-barrier

to

synchronously

create

the

corresponding

stub-scion

pair

would

have

poor

performance

The

advantage

of

making

the

cross-bunch

collector

asynchronous

to

mutators

comes

from

the

substantial

performance

gains

since

the

mutator

is

not

halted

it

might

avoid

work

and

enables

message

batching

In

this

section

we

explain

how

cross-bunch

reference-listing

can

be

made

asynchronous

to

mutators

We

first

consider

there

are

no

replicated

bunches

Then

we

extend

this

result

to

consider

replication

GC

without

Replication

The

basic

idea

behind

the

reference-listing

algorithm

is

the

following

The

collection

of

a

bunch

generates

a

new

set

of

stubs

By

comparing

the

old

and

new

sets

of

stubs

-LRB-

before

and

after

an

intra-bunch

collection

-RRB-

the

cross-bunch

collector

discovers

which

cross-bunch

pointers

no

longer

exist

and

therefore

which

scions

should

be

deleted

Without

loss

of

generality

consider

the

prototypical

example

illustrated

by

Figure

-LRB-

special

case

of

Figure

since

there

is

no

replication

-RRB-

Objects

x

and

y

are

located

in

bunches

B

and

C

respectively

Bunches

B

and

C

are

cached

in

process

i

bunch

D

is

cached

in

process

k.

Initially

x

is

and

y

points

to

object

z

located

in

bunch

D.

Now

the

mutator

within

process

i

running

concurrently

to

the

cross-bunch

collector

executes

x

y

i

then

y

i

such

that

at

the

end

x

points

to

z

and

y

is

With

a

cross-bunch

collector

asynchronous

to

mutators

-LRB-

pointer

assignments

are

not

instrumented

-RRB-

we

must

answer

the

following

question

for

how

long

can

the

creation

of

scion

-LRB-

Bx

Dz

-RRB-

be

safely

delayed

-LRB-

Note

that

with

a

synchronous

distributed

reference-listing

algorithm

the

cross-bunch

collector

creates

the

stub-scion

pair

corresponding

to

Bx!Dz

at

the

instant

the

mutator

in

i

executes

x

y

i.

-RRB-

Consider

Figure

which

shows

the

execution

of

the

prototypical

example

of

Figure

Event

send.create.scion

-LRB-

Bx

Dz

-RRB-

i

is

caused

by

x

y

i

and

occurs

some

time

after

the

latter

Event

send.delete.scion

-LRB-

Cy

Dz

-RRB-

i

is

caused

by

scani

-LRB-

y

-RRB-

that

follows

y

i

and

occurs

after

GCi

-LRB-

C

-RRB-

Clearly

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

i

-LRB-

a

-RRB-

initial

situation

-LRB-

b

-RRB-

after

execution

of

x

y

x

y

proc

i

x

y

z

D

z

D

proc

k

proc

k

-LRB-

c

-RRB-

after

execution

of

y

x

y

proc

i

z

D

proc

k

proc

i

B

B

B

C

C

C

i

Figure

Prototypical

example

of

mutator

execution

-LRB-

non-replicated

case

-RRB-

illustrating

the

creation

and

destruction

of

cross-bunch

pointers

send.create.scion

-LRB-

Bx

Dz

-RRB-

i

must

not

be

delayed

so

much

that

deliver.delete.scion

-LRB-

Cy

Dz

-RRB-

k

happens

before

deliver.create.scion

-LRB-

Bx

Dz

-RRB-

k.

Otherwise

z

could

be

unsafely

reclaimed

by

GCk

-LRB-

D

-RRB-

Thus

assuming

FIFO

communication

send.create.scion

-LRB-

Bx

Dz

-RRB-

i

must

be

issued

before

send.delete.scion

-LRB-

Cy

Dz

-RRB-

i

This

is

explained

in

more

detail

now

We

examine

how

late

a

create

message

may

safely

be

delayed

we

call

promptness

the

corresponding

safety

condition

By

definition

at

the

time

of

the

assignment

x

y

i

y

is

reachable

and

z

is

reachable

via

y

As

long

as

D

is

not

collected

it

is

not

necessary

to

execute

deliver.create.scion

-LRB-

Bx

Dz

-RRB-

k.

Therefore

scion

-LRB-

Bx

Dz

-RRB-

does

not

have

to

be

created

at

the

instant

of

x

y

i

We

can

state

the

promptness

condition

thus

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

GC

-LRB-

D

-RRB-

i

GC

-LRB-

C

-RRB-

k

delete.scion

process

i

process

k

x

y

i

y

i

FIFO

delivery

promptness

-LRB-

Bx

Dz

-RRB-

create.scion

-LRB-

Cy

Dz

-RRB-

Figure

Timeline

of

example

illustrated

by

Figure

deliver.create.scion

-LRB-

Bx

Dz

-RRB-

k

before

z

could

otherwise

be

reclaimed

-LRB-

-RRB-

This

condition

can

be

implemented

with

the

following

-LRB-

unrealistic

-RRB-

algorithm

before

GCk

-LRB-

D

-RRB-

scan

every

object

in

all

other

bunches

looking

for

references

into

D

and

create

any

new

stub-scion

pairs

as

needed

This

algorithm

can

be

improved

by

scanning

only

the

GC-dirty

objects

since

a

new

reference

can

only

appear

by

assignment

Even

this

improved

version

can

not

realistically

be

implemented

because

it

would

mean

synchronous

communication

with

every

process

that

might

be

caching

a

GC-dirty

object

However

this

algorithm

does

contain

a

very

interesting

idea

taking

advantage

of

the

scan

operations

done

by

the

intra-bunch

collection

to

detect

new

outgoing

cross-bunch

pointers

Now

observe

that

the

only

pointer

that

is

positively

known

to

point

to

z

is

the

one

from

y.

So

assuming

messages

are

delivered

in

the

order

sent

the

promptness

condition

becomes

send.create.scion

-LRB-

Bx

Dz

-RRB-

i

before

send.delete.scion

-LRB-

Cy

Dz

-RRB-

i

-LRB-

-RRB-

Recall

that

we

are

currently

assuming

that

there

is

no

replication

Thus

send.delete.scion

-LRB-

Cy

Dz

-RRB-

i

might

be

sent

either

because

y

is

assigned

to

or

because

it

became

unreachable

-LRB-

no

longer

pointed

from

another

bunch

-RRB-

but

it

will

not

be

sent

until

the

enclosing

bunch

C

is

collected

On

the

other

hand

to

detect

the

cross-bunch

pointer

Bx!Dz

object

x

-LRB-

which

is

GC-dirty

-RRB-

must

be

scanned

This

will

happen

when

bunch

B

is

collected

In

short

this

suggests

that

an

algorithm

based

only

on

the

GC-dirty

information

available

at

process

i

can

be

safe

Thus

promptness

is

ensured

as

follows

When

the

intra-bunch

collector

runs

it

scans

at

least

all

-LRB-

locally

cached

-RRB-

GC-dirty

objects

therefore

creating

the

corresponding

stubs

Then

all

create

messages

resulting

from

the

appearing

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

stubs

generated

by

an

intra-bunch

collector

run

are

sent

by

the

cross-bunch

collector

before

any

deletes

-LRB-

for

the

disappeared

stubs

-RRB-

resulting

from

the

same

intra-bunch

collection

run

-LRB-

Note

that

this

takes

into

account

the

condition

-RRB-

We

prove

the

correctness

of

this

algorithm

in

Section

GC

with

Replication

In

this

section

we

extend

the

reference-listing

algorithm

to

cope

with

replication

The

main

issues

are

the

safety

and

asynchrony

of

the

cross-bunch

collector

in

presence

of

multiple

replicas

which

are

not

necessarily

coherent

Union

Rule

We

now

explain

how

the

cross-bunch

collector

is

ensured

to

be

safe

in

presence

of

replication

The

safety

problem

of

the

reference-listing

algorithm

is

illustrated

by

the

following

example

Suppose

that

object

y

is

replicated

in

multiple

processes

such

that

pointer

Cy!Dz

has

many

images

-LRB-

Cy

-RRB-

Dz

-LRB-

Cy

-RRB-

pn

Dz

observed

by

processes

pn

respectively

Then

scanning

only

the

owner

's

replica

of

an

object

y

alone

is

not

safe

Thus

the

question

is

in

what

circumstances

can

the

cross-bunch

collector

delete

scion

-LRB-

Cy

Dz

-RRB-

therefore

allowing

z

to

be

reclaimed

Given

that

there

are

multiple

replicas

of

y

not

necessarily

all

coherent

the

answer

is

scion

-LRB-

Cy

Dz

-RRB-

can

be

deleted

only

after

z

has

become

unreachable

from

all

replicas

of

y

As

already

mentioned

-LRB-

recall

Section

-RRB-

we

call

this

the

union

rule

a

scion

pointing

to

a

target

object

z

can

be

safely

deleted

only

after

the

corresponding

stubs

have

disappeared

in

all

replicas

of

the

source

object

This

rule

is

implemented

by

the

owner

of

the

source

object

The

owner

evaluates

the

union

of

the

stubs

of

all

replicas

-LRB-

of

the

source

object

-RRB-

then

if

the

result

is

an

empty

set

it

may

issue

a

<send.delete.scion>

for

the

corresponding

scion

In

other

words

a

process

caching

a

replica

yi

informs

the

owner

of

y

process

j

of

yi

's

stubs

by

a

union

message

events

send.union

-LRB-

y

i

j

-RRB-

i

and

deliver.union

-LRB-

y

i

j

-RRB-

j.

Then

the

owner

process

j

performs

the

union

of

all

stubs

for

all

y

's

replicas

If

the

resulting

set

is

empty

it

performs

send.delete.scion

-LRB-

Cy

Dz

-RRB-

j

Many

DSM

coherence

protocols

impose

that

only

the

owner

of

an

object

y

can

write

it

-LRB-

e.g.

entry-consistency

-LSB-

-RSB-

-RRB-

In

this

case

a

non-owner

replica

yi

can

not

cause

an

object

unreachable

from

yi

to

become

reachable

because

to

do

so

requires

writing

y

in

process

i.

Thus

the

set

of

stubs

at

a

non-owner

process

is

monotonically

decreasing

Therefore

the

union

rule

can

be

implemented

cheaply

using

asynchronous

FIFO

communication

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

i

proc

i

B

C

proc

j

C

y

x

y

D

proc

k

z

both

replicas

point

to

z

i

i

ii

j

proc

i

B

C

proc

j

C

y

x

y

D

proc

k

z

both

replicas

point

to

z

i

i

i

i

j

j

k

k

k

k

proc

i

B

C

proc

j

C

y

x

y

D

proc

k

z

i

i

i

i

j

j

k

k

proc

i

B

C

proc

j

C

y

x

y

D

proc

k

z

i

i

i

i

j

j

k

k

j

-LRB-

a

-RRB-

initial

situation

-LRB-

b

-RRB-

after

x

y

-LRB-

d

-RRB-

after

y

-LRB-

c

-RRB-

after

y

j

i

Figure

Prototypical

example

of

mutator

execution

-LRB-

replicated

case

-RRB-

illustrating

the

creation

and

destruction

of

cross-bunch

pointers

Objects

are

shown

in

bold

on

their

owner

process

Promptness

In

this

section

we

study

how

long

the

creation

of

a

stub

can

be

safely

delayed

extending

the

promptness

condition

to

the

replicated

case

Without

loss

of

generality

consider

the

prototypical

example

illustrated

in

Figure

-LRB-

replicated

case

of

Figure

-RRB-

Objects

x

and

y

are

located

in

bunches

B

and

C

respectively

Bunch

C

is

cached

by

processes

i

and

j.

Object

x

is

owned

by

i

y

is

owned

by

j

and

z

is

owned

by

k.

Initially

xi

is

and

both

replicas

of

y

point

to

object

z

located

in

bunch

D.

Then

the

mutators

within

processes

i

and

j

execute

the

operations

x

y

i

y

i

and

y

j

such

that

in

the

end

x

points

to

z

and

y

is

-LRB-

Note

that

instead

of

y

i

a

propagate

of

y

from

j

to

i

would

have

the

same

effect

both

replicas

of

y

no

longer

pointing

to

z.

-RRB-

As

in

the

non-replicated

case

we

want

to

render

the

cross-bunch

collector

asyn

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

k

GC

-LRB-

C

-RRB-

i

GC

-LRB-

D

-RRB-

-LRB-

owner

of

z

-RRB-

GC

-LRB-

C

-RRB-

process

i

process

j

process

k

y

y

x

y

i

i

j

j

create.scion

-LRB-

Bx

Dz

-RRB-

delete.scion

-LRB-

Cy

Dz

-RRB-

-LRB-

owner

of

x

-RRB-

-LRB-

owner

of

y

-RRB-

Figure

Timeline

showing

a

possible

problem

with

the

asynchronous

creation

of

scions

the

stub-scion

pair

describing

cross-bunch

pointer

Bx!Dz

is

created

too

late

-LRB-

for

safety

purposes

-RRB-

i

GC

-LRB-

C

-RRB-

j

GC

-LRB-

D

-RRB-

k

causal

delivery

process

i

process

j

process

k

delete.scion

-LRB-

Cy

Dz

-RRB-

GC

-LRB-

C

-RRB-

union

-LRB-

y

-RRB-

promptness

x

y

y

j

i

i

y

-LRB-

owner

of

x

-RRB-

-LRB-

owner

of

y

-RRB-

-LRB-

owner

of

z

-RRB-

create.scion

-LRB-

Bx

Dz

-RRB-

Figure

Timeline

showing

the

union

rule

to

ensure

promptness

Message

send.create.scion

-LRB-

Bx

Dz

-RRB-

i

can

be

delayed

at

most

until

send.union

-LRB-

y

i

j

-RRB-

i.

Note

that

the

union

message

carries

the

causal

dependency

-LRB-

shown

in

bold

lines

-RRB-

between

the

create

and

delete

messages

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

chronous

to

mutators

So

once

again

we

must

answer

the

question

for

how

long

can

the

creation

of

scion

-LRB-

Bx

Dz

-RRB-

be

safely

delayed

This

issue

is

illustrated

in

Figure

suppose

that

the

creation

of

scion

-LRB-

Bx

Dz

-RRB-

-LRB-

due

to

x

y

i

-RRB-

is

delayed

in

such

a

way

that

deliver.delete.scion

-LRB-

Cy

Dz

-RRB-

k

is

executed

first

Then

z

could

be

unsafely

reclaimed

by

GCk

-LRB-

D

-RRB-

because

there

is

no

scion

protecting

z

-LRB-

note

that

z

is

still

referenced

from

x

-RRB-

We

can

use

the

same

unrealistic

algorithm

described

for

the

non-replicated

case

-LRB-

recall

Section

-RRB-

before

GCk

-LRB-

D

-RRB-

scan

every

object

in

all

other

bunches

looking

for

references

into

D

and

create

any

new

scions

as

needed

However

this

algorithm

is

even

more

difficult

to

implement

than

in

the

non-replicated

case

because

it

implies

synchronizing

every

process

that

might

be

caching

a

GC-dirty

object

replica

Safety

clearly

depends

on

the

relative

delivery

order

of

create

and

delete

events

at

process

k

The

promptness

condition

is

therefore

-LRB-

i

-RRB-

the

create

message

must

be

sent

from

process

i

before

the

delete

message

is

sent

from

process

j

i.e.

create

and

delete

messages

must

be

sent

in

a

safe

order

and

-LRB-

ii

-RRB-

the

create

message

must

be

delivered

at

process

k

before

the

delete

message

i.e.

create

and

delete

messages

must

be

delivered

in

a

safe

order

Now

consider

the

situation

illustrated

by

Figure

which

extends

Figure

with

the

union

rule

To

ensure

the

promptness

condition

all

create

messages

resulting

from

an

intra-bunch

collector

run

must

be

sent

before

any

union

message

in

the

same

run

and

create

and

delete

messages

are

delivered

in

causal

order

-LSB-

-RSB-

To

conclude

the

create

operation

may

be

safely

issued

during

all

the

promptness

time

period

indicated

in

Figure

-LRB-

at

the

latest

before

the

union

message

is

issued

-RRB-

Furthermore

causal

delivery

is

necessary

to

ensure

safety

Interaction

of

Intra-Bunch

and

Cross-Bunch

GC

In

this

section

we

describe

the

interaction

between

the

intra-bunch

and

crossbunch

collectors

The

basic

aspect

is

that

between

any

two

intra-bunch

collections

in

a

process

-LRB-

not

necessarily

of

the

same

bunch

-RRB-

the

cross-bunch

collector

must

run

The

cross-bunch

collector

issues

the

messages

for

creating

the

scions

corresponding

to

new

stubs

generated

by

an

intra-bunch

collection

These

are

vital

for

safety

Let

us

give

an

example

to

illustrate

this

Suppose

that

bunches

B

C

and

D

are

all

cached

in

process

i.

Objects

x

and

y

are

allocated

in

B

t

in

C

z

in

D.

Now

the

mutator

modifies

x

and

y

by

creating

pointers

By!Dz

and

Bx!Ct

Thus

x

and

y

are

GC-dirty

This

situation

is

illustrated

in

Figure

-LRB-

no

scions

protecting

t

and

z

for

the

moment

-RRB-

Then

when

GCi

-LRB-

D

-RRB-

runs

the

intra-bunch

collector

must

scan

all

GC-dirty

objects

in

i

because

they

may

contain

pointers

to

objects

in

D

-LRB-

this

is

the

case

of

y

-RRB-

As

x

and

y

are

scanned

z

is

reachable

and

the

corresponding

stubs

are

created

stub

-LRB-

By

Dz

-RRB-

and

stub

-LRB-

Bx

Ct

-RRB-

Note

that

we

are

collecting

only

bunch

D

thus

t

reachability

is

not

considered

Now

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

C

t

y

x

z

D

B

Figure

Example

showing

the

need

of

cross-bunch

collection

between

each

pair

of

intra-bunch

collections

suppose

that

GCi

-LRB-

C

-RRB-

runs

after

GCi

-LRB-

D

-RRB-

has

finished

and

before

the

cross-bunch

collector

runs

in

i.

Objects

x

and

y

are

no

longer

GC-dirty

thus

they

will

not

be

scanned

Because

scion

-LRB-

Bx

Ct

-RRB-

does

not

exist

yet

t

is

unsafely

reclaimed

The

execution

of

the

cross-bunch

collector

after

GCi

-LRB-

D

-RRB-

and

before

GCi

-LRB-

C

-RRB-

prevents

this

problem

because

it

creates

the

scions

corresponding

to

stub

-LRB-

By

Dz

-RRB-

and

stub

-LRB-

Bx

Ct

-RRB-

It

compares

the

stub

sets

before

and

after

GCi

-LRB-

D

-RRB-

and

creates

scion

-LRB-

By

Dz

-RRB-

and

scion

-LRB-

Bx

Ct

-RRB-

Figure

illustrates

the

concurrent

execution

of

intra-bunch

collections

of

a

bunch

replica

Bi

cross-bunch

collections

and

mutator

in

process

i.

Intrabunch

collections

are

noted

intra

cross-bunch

collections

are

noted

cross

the

mutator

execution

is

noted

mutator

stubsi

-LRB-

Bi

-RRB-

is

noted

stubs

-LRB-

i

-RRB-

By

computing

the

set

difference

stubsi

-LRB-

Bi

-RRB-

stubsi

-LRB-

Bi

-RRB-

the

cross-bunch

collector

discovers

the

new

stubs

corresponding

to

new

outgoing

cross-bunch

pointers

The

cross-bunch

collector

then

sends

the

messages

to

create

the

corresponding

scions

By

computing

the

set

difference

stubsi

-LRB-

Bi

-RRB-

stubsi

-LRB-

Bi

-RRB-

the

cross-bunch

collector

discovers

the

disappeared

stubs

Then

the

cross-bunch

collector

sends

the

messages

to

delete

the

corresponding

scions

When

a

stub

is

created

it

is

inserted

in

the

most

recent

set

of

stubs

For

example

when

ith

intra-bunch

collection

runs

the

most

recent

set

of

stubs

is

stubsi

-LRB-

Bi

-RRB-

Thus

new

stubs

are

inserted

in

this

set

A

new

empty

set

of

stubs

is

created

at

the

beginning

of

each

cross-bunch

collection

Stubs

created

either

concurrently

with

or

after

the

cross-bunch

collector

are

inserted

in

this

new

set

The

reason

for

this

is

the

following

Imagine

that

a

stub

is

created

while

the

cross-bunch

collector

is

comparing

stubsi

-LRB-

Bi

-RRB-

and

stubsi

-LRB-

Bi

-RRB-

If

the

new

stub

is

inserted

in

stubsi

-LRB-

Bi

-RRB-

the

corresponding

scion

will

never

be

created

because

the

next

cross-bunch

collector

would

not

consider

it

as

a

new

stub

when

comparing

stubsi

-LRB-

Bi

-RRB-

and

stubsi

-LRB-

Bi

-RRB-

Thus

the

new

stub

created

while

the

cross-bunch

collector

runs

is

inserted

in

stubsi

-LRB-

Bi

-RRB-

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

send.create.scion

send.delete.scion

insert

stubs

in

set

i

destruction

of

set

of

stubs

-LRB-

i

-RRB-

-LRB-

-RRB-

i

i

-LRB-

i

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

creation

of

set

of

stubs

-LRB-

i

-RRB-

creation

of

set

of

stubs

-LRB-

i

-RRB-

destruction

of

set

of

stubs

-LRB-

-RRB-

send.create.scion

send.delete.scion

send.create.scion

send.delete.scion

insert

stubs

in

set

-LRB-

i

-RRB-

insert

stubs

in

set

-LRB-

i

-RRB-

i

-LRB-

-RRB-

-LRB-

i

-RRB-

i

intra

i

cross

i

mutator

cross

-LRB-

-RRB-

intra

-LRB-

i

-RRB-

cross

-LRB-

i

-RRB-

intra

-LRB-

i

-RRB-

destruction

of

set

of

stubs

-LRB-

-RRB-

new

stubs

inserted

in

stubs

-LRB-

i

-RRB-

new

stubs

inserted

in

stubs

-LRB-

i

-RRB-

creation

of

set

of

stubs

-LRB-

i

-RRB-

Figure

Timeline

showing

concurrent

execution

of

mutator

intra-bunch

and

cross-bunch

collectors

-LRB-

The

flip

times

are

not

represented

-RRB-

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

stubs

CbunchBbunch

y

x

z

stubs

scions

objects

scions

objects

Figure

Reclaiming

a

cross-bunch

cycle

of

garbage

When

collecting

the

group

of

bunches

B

and

C

shaded

scions

are

not

members

of

the

GC-root

Thus

objects

x

y

and

z

are

reclaimed

Finally

observe

that

a

set

of

stubs

can

be

destroyed

after

the

cross-bunch

collector

has

compared

it

with

its

successor

Reclaiming

Cross-Bunch

Cycles

of

Garbage

The

intra-bunch

GC

algorithm

is

complete

w.r.t.

the

collected

bunch

i.e.

it

reclaims

all

garbage

that

is

entirely

within

the

bunch

However

it

is

incomplete

w.r.t.

other

bunches

since

it

does

not

reclaim

a

cycle

of

garbage

that

crosses

the

bunch

boundary

-LRB-

e.g.

cross-bunch

cycle

in

Figure

objects

x

y

and

z

-RRB-

In

this

section

we

describe

our

GC

algorithm

to

reclaim

such

cycles

and

discuss

its

completeness

GC

Algorithm

Note

that

the

same

tracing

algorithm

that

collects

a

single

bunch

-LRB-

recall

Section

-RRB-

can

collect

any

group

of

bunches

The

only

difference

is

that

to

trace

a

group

-LRB-

i

-RRB-

scions

for

cross-bunch

pointers

internal

to

the

group

are

not

considered

as

members

of

the

GC-root

and

-LRB-

ii

-RRB-

tracing

continues

across

bunch

boundaries

internal

to

the

group

This

algorithm

reclaims

a

cross-bunch

cycle

unreachable

from

bunches

outside

the

group

i.e.

it

is

complete

w.r.t.

to

the

group

being

collected

Thus

in

Figure

a

group

collection

including

bunches

B

and

C

would

not

consider

the

shaded

scions

as

members

of

the

GC-root

The

cross-bunch

garbage

cycle

constituted

by

objects

x

y

and

z

would

therefore

be

reclaimed

and

the

corresponding

scions

deleted

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

Discussion

The

significance

of

group

collection

is

that

any

arbitrary

subset

of

the

store

can

be

collected

in

a

single

process

independently

of

the

rest

of

the

memory

The

choice

of

the

group

to

be

collected

is

heuristic

and

aims

at

maximizing

the

amount

of

garbage

reclaimed

while

minimizing

the

cost

Larchant

uses

a

locality-based

heuristic

A

group

contains

all

the

bunches

currently

cached

in

the

process

This

heuristic

avoids

extra

I/O

costs

-LRB-

Recall

that

this

was

one

of

the

problems

presented

in

Section

-RRB-

However

it

does

not

enable

the

reclamation

of

cross-bunch

cycles

enclosed

in

bunches

that

reside

partially

on

disk

Also

cycles

of

garbage

where

some

bunches

but

not

all

are

replicated

in

multiple

processes

are

not

reclaimed

This

garbage

might

be

reclaimed

with

a

more

aggressive

grouping

heuristic

For

example

the

intra-bunch

collector

could

force

bunches

to

be

cached

in

a

process

at

the

same

time

even

if

applications

do

not

do

so

naturally

This

extra

I/O

cost

needs

to

be

balanced

against

the

expected

gain

We

intend

to

experiment

with

the

locality-based

heuristic

over

a

wide

number

of

applications

and

to

do

some

simulation

studies

Then

if

experimental

results

mandate

it

more

complex

heuristics

will

be

the

topic

of

future

research

A

full

study

of

the

most

appropriate

heuristic

and

its

cost

is

out

of

the

scope

of

this

document

Summary

of

GC

Operations

In

this

section

we

summarize

all

the

operations

of

the

Larchant

model

and

their

implications

in

terms

of

GC

This

description

-LRB-

see

Table

-RRB-

consolidates

the

presentation

of

the

GC

algorithms

done

so

far

and

facilitates

the

understanding

of

the

next

sections

in

which

we

formally

address

their

correctness

Correctness

of

Intra-Bunch

GC

Given

that

the

algorithms

used

by

the

intra-bunch

collector

are

based

on

the

well

known

mark-and-sweep

and

copy

techniques

their

correctness

can

be

formally

proved

with

an

approach

similar

to

that

found

in

the

literature

-LSB-

-RSB-

Thus

from

the

theoretical

point

of

view

there

is

no

particular

difficulty

or

novelty

in

proving

the

safety

and

liveness

of

the

intra-bunch

collector

We

simply

present

the

invariants

that

must

hold

On

the

contrary

there

is

no

previous

work

regarding

the

proof

of

safety

and

liveness

of

an

asynchronous

cross-bunch

collector

in

presence

of

replication

Thus

its

correctness

is

formally

proved

in

Section

Safety

and

Liveness

Conceptually

both

the

mark-and-sweep

and

copy

intra-bunch

collectors

can

be

described

in

terms

of

the

tricolor

algorithm

introduced

in

Section

as

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

model

operations

description

GC

implications

mutator

x

y

i

assignment

operation

performed

modifies

the

pointer

graph

by

process

i

such

that

x

is

made

to

point

to

the

object

pointed

from

y

coherence

send.propagate

-LRB-

x

i

j

-RRB-

i

put

the

contents

of

xi

in

create

the

corresponding

the

message

inOwnerPtrs

GC-clean

xi

and

send

messages

to

create

the

corresponding

scions

-LRB-

see

Section

-RRB-

deliver.propagate

-LRB-

x

i

j

-RRB-

j

copy

the

object

's

contents

from

create

the

corresponding

the

message

into

xj

outOwnerPtrs

intra

GC

mark

-LRB-

x

-RRB-

i

mark

object

xi

in

process

i

color

xi

gray

scani

-LRB-

x

-RRB-

scan

xi

in

process

i

color

xi

black

and

create

corresponding

stubs

send.move

-LRB-

x

@x

-RRB-

i

i

sends

new

address

of

x

to

process

i

moves

xi

to

processes

caching

a

pointer

to

x

to-space

and

communicates

x

's

new

address

to

other

processes

deliver.move

-LRB-

x

@x

-RRB-

j

j

receives

new

address

of

x

process

j

moves

xj

to

to-space

patch

-LRB-

y

@x

-RRB-

i

on

i

patch

y

with

new

on

i

reconstruct

the

address

of

x

pointer

graph

in

to-space

cross

GC

send.create.scion

-LRB-

Bx

Cy

-RRB-

i

i

sends

message

to

create

results

from

scion

-LRB-

Bx

Cy

-RRB-

GC-cleaning

xi

deliver.create.scion

-LRB-

Bx

Cy

-RRB-

j

j

receives

message

to

create

create

scion

-LRB-

Bx

Cy

-RRB-

in

j.

scion

-LRB-

Bx

Cy

-RRB-

send.delete.scion

-LRB-

Bx

Cy

-RRB-

i

i

sends

message

to

delete

results

from

GCi

-LRB-

B

-RRB-

scion

-LRB-

Bx

Cy

-RRB-

deliver.delete.scion

-LRB-

Bx

Cy

-RRB-

j

j

receives

message

to

delete

delete

scion

-LRB-

Bx

Cy

-RRB-

in

j.

scion

-LRB-

Bx

Cy

-RRB-

send.union

-LRB-

y

i

j

-RRB-

i

i

sends

union

message

to

j

results

from

GCi

-LRB-

C

-RRB-

deliver.union

-LRB-

y

i

j

-RRB-

j

j

receives

union

message

from

i

apply

union

rule

in

j.

Table

Summary

of

operations

in

the

Larchant

model

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

gray-log

gray-log

scan

patch

-LRB-

b

-RRB-

copy

white

black

gray

read

write

scan

read

writeread

read

write

write

scan

sweepmark

white

black

gray

read

write

read

write

write

writeread

read

move

move

flip

-LRB-

a

-RRB-

mark-and-sweep

Figure

Intra-bunch

GC

represented

as

a

tricolor

algorithm

The

read

transition

represents

an

assignment

operation

in

which

the

object

appears

in

the

right-hand

side

the

write

transition

represents

an

assignment

operation

in

which

the

object

appears

in

the

left-hand

side

illustrated

in

Figure

The

fourth

color

gray-log

models

the

fact

that

a

mutator

may

modify

an

object

that

has

already

been

scanned

The

difference

between

schemes

-LRB-

a

-RRB-

and

-LRB-

b

-RRB-

in

Figure

is

due

to

the

fact

that

with

the

copy

algorithm

when

an

object

that

has

already

been

moved

is

written

by

the

mutator

it

is

necessary

to

move

it

again

to

the

same

address

in

to-space

-LRB-

recall

the

replication-based

technique

in

Section

-RRB-

This

difference

is

irrelevant

from

the

point

of

view

of

correctness

In

fact

scheme

-LRB-

b

-RRB-

also

describes

the

mark-and-sweep

algorithm

if

we

accept

to

re-mark

an

object

-LRB-

the

move

operation

between

colors

gray-log

and

gray

would

be

replaced

by

a

re-mark

-RRB-

and

to

-LRB-

unnecessarily

-RRB-

change

an

object

's

color

from

gray

to

gray-log

when

it

is

written

even

if

not

yet

scanned

Given

the

replication

technique

of

the

copy

collector

there

are

two

replicas

that

must

be

considered

the

object

in

from-space

and

its

replica

in

to-space

Thus

each

color

identifies

the

merged

state

of

both

from-space

and

to-space

replicas

of

an

object

as

explained

now

The

colors

have

the

following

meaning

ffl

mark-and-sweep

algorithm

-LCB-

white

object

not

yet

marked

-LCB-

gray

object

already

marked

but

not

yet

scanned

that

the

object

's

address

is

inserted

in

the

GC-log

if

the

mutator

modifies

it

after

having

been

scanned

this

is

the

rational

for

the

name

gray-log

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

-LCB-

black

object

already

marked

and

scanned

-LCB-

gray-log

object

written

after

being

marked

and

scanned

ffl

copy

algorithm

-LCB-

white

object

in

from-space

not

yet

moved

-LCB-

gray

object

in

from-space

already

moved

and

object

in

to-space

not

yet

scanned

-LCB-

black

object

in

from-space

already

moved

and

object

in

to-space

already

scanned

-LCB-

gray-log

object

in

from-space

written

after

it

has

been

moved

and

after

object

in

to-space

has

been

scanned

Before

an

intra-bunch

collection

starts

every

object

is

white

When

an

object

is

reached

by

the

intra-bunch

collector

while

traversing

the

pointer

graph

the

collector

colors

it

gray

-LRB-

This

coloring

consists

of

setting

a

mark

bit

associated

to

the

object

in

case

of

a

mark-and-sweep

collector

or

of

moving

the

object

to

to-space

in

case

of

a

copy

collector

-RRB-

A

gray

object

becomes

black

after

having

been

scanned

-LRB-

and

patched

in

case

of

a

copy

collector

-RRB-

If

a

black

object

is

written

by

the

mutator

it

becomes

gray-log

With

the

mark-and-sweep

algorithm

a

gray-log

object

becomes

black

after

having

been

scanned

With

the

copy

algorithm

a

gray-log

object

becomes

gray

after

having

been

moved

The

invariant

that

the

intra-bunch

collector

must

ensure

is

the

following

black

objects

may

not

point

to

white

objects

In

the

case

of

a

copy

collector

another

invariant

must

be

ensured

the

mutator

only

accesses

from-space

objects

These

invariants

are

typical

of

the

corresponding

basic

GC

algorithms

with

the

incremental/concurrent

functioning

mode

-LRB-

recall

Section

-RRB-

It

's

worthy

to

note

that

the

tricolor

algorithm

applies

not

only

to

the

collection

of

a

non-replicated

bunch

but

also

to

the

collection

of

a

replicated

one

The

only

difference

resides

in

the

move

operation

-LRB-

when

using

a

copy

collector

-RRB-

as

explained

now

When

the

process

where

the

intra-bunch

collector

is

running

owns

the

object

to

be

moved

-LRB-

being

grayed

-RRB-

the

collector

proceeds

as

in

the

non-replicated

case

i.e.

sends

a

move

message

including

to

itself

thus

it

moves

its

object

replica

to

to-space

If

the

process

does

not

own

the

object

then

it

will

receive

the

object

's

address

in

to-space

when

the

corresponding

move

message

is

delivered

-LRB-

sent

by

the

owner

-RRB-

Then

it

moves

the

object

to

to-space

-LRB-

Note

that

the

collector

does

not

have

to

block

while

waiting

for

the

move

message

as

will

be

explained

in

Section

in

addition

it

can

flip

even

without

having

received

such

messages

as

explained

in

Section

-RRB-

Correctness

of

the

Cross-Bunch

GC

In

this

section

we

prove

the

correctness

of

the

cross-bunch

GC

algorithm

We

first

use

a

simplified

model

of

the

cached

distributed

shared

store

without

replication

Then

we

extend

the

proof

to

the

case

where

objects

are

replicated

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

Cross-Bunch

GC

without

Replication

In

this

section

we

describe

rigorously

the

cross-bunch

GC

algorithm

without

replication

We

present

the

invariants

that

must

hold

and

prove

the

algorithm

is

safe

and

live

Given

that

for

the

time

being

we

consider

there

is

no

replication

no

coherence

events

appear

Thus

we

introduce

the

concept

of

holder

The

holder

is

the

equivalent

of

owner

in

the

replicated

case

at

any

point

in

time

an

object

is

cached

by

a

single

process

which

we

call

its

holder

The

model

does

not

specify

how

or

when

the

holder

process

may

change

We

define

the

propagate

operation

in

the

non-replicated

case

as

follows

When

object

x

held

by

process

i

is

propagated

to

j

by

operations

send.propagate

-LRB-

x

i

j

-RRB-

i

and

deliver.propagate

-LRB-

x

i

j

-RRB-

j

it

is

thereafter

held

by

j

and

not

by

i

we

say

that

x

migrates

from

i

to

j.

-LRB-

Note

that

when

an

object

x

migrates

all

the

stubs

corresponding

to

its

outgoing

pointers

and

the

scions

pointing

to

x

are

also

migrated

-RRB-

Safety

The

cross-bunch

GC

algorithm

must

satisfy

the

following

obvious

safety

invariant

Safety

Invariant

No

reachable

object

is

reclaimed

Since

we

are

considering

only

cross-bunch

pointers

the

above

invariant

is

equivalent

to

Safety

Invariant

-LRB-

B

x

B

Bx!Dz

-RRB-

-RRB-

-LRB-

E

t

E

scion

-LRB-

Et

Dz

-RRB-

-RRB-

This

reads

if

any

object

points

to

z

then

some

scion

protects

z.

Note

that

this

is

weaker

than

the

more

intuitive

if

x

points

to

z

then

scion

-LRB-

Bx

Dz

-RRB-

exists

indeed

the

scion

that

protects

z

does

not

need

to

match

the

pointer

Given

that

an

object

may

be

reclaimed

only

when

its

enclosing

bunch

is

collected

scion

-LRB-

Et

Dz

-RRB-

is

only

needed

at

the

time

of

GCk

-LRB-

D

-RRB-

i.e.

when

D

is

collected

in

some

process

k

In

other

words

Safety

Invariant

is

only

evaluated

at

the

time

of

GCk

-LRB-

D

-RRB-

Under

this

assumption

Safety

Invariant

is

equivalent

to

Safety

Invariant

-LRB-

B

x

B

Bx!Dz

-RRB-

-RRB-

-LRB-

E

t

E

scion

-LRB-

Et

Dz

-RRB-

at

the

time

of

GCk

-LRB-

D

-RRB-

for

the

process

k

holding

D

-RRB-

This

reads

at

the

time

D

is

collected

in

process

k

if

any

object

points

to

z

then

some

scion

protects

z.

The

scion

is

needed

only

at

the

time

when

the

target

bunch

D

is

collected

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

Algorithm

Let

us

return

to

the

prototypical

example

illustrated

in

Figure

and

observe

that

the

initial

situation

satisfies

Safety

Invariant

At

the

time

of

x

y

i

object

z

is

reachable

-LRB-

from

y

-RRB-

and

is

protected

by

some

scion

say

scion

-LRB-

Et

Dz

-RRB-

-LRB-

presumably

but

not

necessarily

E

C

and

t

y

-RRB-

As

long

as

that

scion

continues

to

exist

it

is

safe

to

delay

the

creation

of

scion

-LRB-

Bx

Dz

-RRB-

The

following

rules

and

algorithm

ensure

the

correctness

of

the

cross-bunch

collector

-LRB-

intuitively

presented

in

Section

-RRB-

as

will

be

formally

proved

in

the

next

sections

Comprehensive

Scanning

Rule

When

an

intra-bunch

collection

takes

place

at

process

i

every

GC-dirty

object

y

held

by

i

must

be

GC-cleaned

GC-Clean

Migration

Rule

An

object

can

be

migrated

from

process

i

only

if

every

object

held

by

i

is

GC-clean

and

the

messages

to

create

the

corresponding

scions

have

been

sent

Cross-Bunch

Collection

Algorithm

At

process

i

between

each

pair

of

intra-bunch

collections

execute

these

steps

in

the

following

order

For

all

new

cross-bunch

stubs

perform

the

corresponding

<send.create.scion>

i.

For

all

disappearing

cross-bunch

stubs

perform

the

corresponding

<send.delete.scion>

i

The

Comprehensive

Scanning

Rule

means

that

when

an

intra-bunch

collection

takes

place

in

process

i

every

GC-dirty

object

-LRB-

locally

held

-RRB-

must

be

scanned

therefore

creating

the

corresponding

stubs

The

GC-Clean

Migration

Rule

means

that

an

object

can

not

migrate

from

process

i

without

having

scanned

all

GC-dirty

objects

in

i

thus

after

having

created

the

corresponding

stubs

and

sent

the

messages

to

create

their

scions

The

cross-bunch

collector

runs

between

each

pair

of

intra-bunch

collections

and

compares

the

set

of

stubs

before

and

after

collection

-LRB-

recall

Section

-RRB-

The

Cross-Bunch

Collection

Algorithm

ensures

that

for

each

intra-bunch

collector

run

all

messages

to

create

scions

are

sent

before

any

message

to

delete

a

scion

To

understand

the

role

of

the

Comprehensive

Scanning

Rule

consider

Figure

after

the

mutator

has

executed

x

y

i

and

y

i.

Then

suppose

that

GCi

-LRB-

C

-RRB-

runs

without

fulfilling

the

Comprehensive

Scanning

Rule

i.e.

without

GC-cleaning

x

According

to

the

Cross-Bunch

Collection

Algorithm

send.delete.scion

-LRB-

Cy

Dz

-RRB-

i

will

be

sent

Thus

scion

-LRB-

Cy

Dz

-RRB-

is

deleted

that

when

an

object

is

GC-cleaned

the

corresponding

stubs

are

created

-LRB-

as

described

in

Section

-RRB-

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

Then

if

D

is

collected

object

z

is

unsafely

reclaimed

The

Comprehensive

Scanning

Rule

prevents

the

above

scenario

as

it

forces

x

to

be

GC-cleaned

thus

stub

-LRB-

Bx

Dz

-RRB-

to

be

created

when

GCi

-LRB-

C

-RRB-

runs

then

according

to

the

CrossBunch

Collection

Algorithm

send.create.scion

-LRB-

Bx

Dz

-RRB-

i

is

performed

before

send.delete.scion

-LRB-

Cy

Dz

-RRB-

i

Since

we

assumed

causal

delivery

scion

-LRB-

Bx

Dz

-RRB-

is

created

before

scion

-LRB-

Cy

Dz

-RRB-

is

deleted

Consequently

z

is

not

reclaimed

by

GCk

-LRB-

D

-RRB-

It

is

also

interesting

to

understand

the

role

of

the

GC-Clean

Migration

Rule

Consider

Figure

after

the

mutator

has

executed

x

y

i

and

before

y

i.

Now

suppose

that

object

y

migrates

to

some

other

process

j

without

fulfilling

the

GC-Clean

Migration

Rule

At

this

moment

the

only

scion

that

is

known

to

protect

z

is

scion

-LRB-

Cy

Dz

-RRB-

Suppose

that

the

mutator

at

j

executes

y

j

GCj

-LRB-

C

-RRB-

runs

and

send.delete.scion

-LRB-

Cy

Dz

-RRB-

j

is

performed

Then

scion

-LRB-

Cy

Dz

-RRB-

no

longer

exists

and

z

may

be

unsafely

reclaimed

by

GCk

-LRB-

D

-RRB-

The

GC-Clean

Migration

Rule

prevents

the

above

scenario

as

it

forces

x

to

be

GC-cleaned

thus

stub

-LRB-

Bx

Dz

-RRB-

to

be

created

and

send.create.scion

-LRB-

Bx

Dz

-RRB-

i

to

be

performed

all

before

object

y

gets

propagated

to

j.

Therefore

z

is

not

unsafely

reclaimed

because

scion

-LRB-

Bx

Dz

-RRB-

is

created

before

deleting

scion

-LRB-

Cy

Dz

-RRB-

-LRB-

assuming

causal

delivery

-RRB-

Proof

of

Safety

To

prove

the

safety

of

the

cross-bunch

collection

algorithm

we

start

by

proving

the

following

lemma

Lemma

Let

xn

located

respectively

in

bunches

Bn

all

held

by

process

p

be

the

set

of

all

protected

objects

pointing

to

z

D

held

by

process

k.

Then

i

n

ffl

xi

is

GC-clean

scion

-LRB-

Bixi

Dz

-RRB-

or

ffl

j

n

xi

is

GC-dirty

scion

-LRB-

Bjxj

Dz

-RRB-

This

lemma

says

that

as

long

as

there

is

an

object

pointing

to

z

held

by

some

process

p

there

exists

at

least

a

scion

protecting

z.

Proof

of

Lemma

We

prove

this

lemma

by

induction

over

the

size

of

the

set

containing

objects

pointing

to

z

for

a

single

process

Base

case

Initially

a

single

protected

object

points

to

z

held

by

process

p

and

scion

-LRB-

Dz

-RRB-

exists

Let

also

held

by

p

initially

not

pointing

to

z.

Perform

p

-LRB-

now

also

points

to

z

-RRB-

therefore

is

GC-dirty

If

is

not

assigned

to

thereafter

the

lemma

remains

trivially

true

since

is

GC-dirty

and

scion

-LRB-

Dz

-RRB-

exists

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

Therefore

consider

that

is

changed

by

an

assignment

such

as

p.

-LRB-

The

actual

value

of

the

right-hand

side

does

not

matter

for

the

proof

except

that

when

the

right-hand

side

is

a

pointer

to

z

it

is

as

if

the

assignment

did

not

take

place

-RRB-

Hence

is

GC-dirty

Until

an

intra-bunch

collection

takes

place

at

p

no

<send.create.scion>

p

or

<send.delete.scion>

p

is

performed

and

the

lemma

remains

trivially

true

since

is

GC-dirty

and

scion

-LRB-

Dz

-RRB-

exists

When

an

intra-bunch

collection

does

execute

in

process

p

by

the

Comprehensive

Scanning

Rule

both

-LRB-

containing

the

new

pointer

-RRB-

and

-LRB-

previously

containing

the

pointer

with

a

scion

-RRB-

are

GC-cleaned

because

both

are

GC

dirty

therefore

stub

-LRB-

Dz

-RRB-

disappears

-LRB-

not

in

the

new

set

of

stubs

-RRB-

and

stub

-LRB-

Dz

-RRB-

is

created

According

to

the

Cross-Bunch

Collection

Algorithm

send.create.scion

-LRB-

Dz

-RRB-

p

precedes

send.delete.scion

-LRB-

Dz

-RRB-

p

We

assumed

that

messages

are

delivered

in

causal

order

hence

deliver.create.scion

-LRB-

Dz

-RRB-

k

precedes

deliver.delete.scion

-LRB-

Dz

-RRB-

k.

Thus

at

any

moment

there

exists

at

least

a

scion

protecting

z

either

scion

-LRB-

Dz

-RRB-

or

scion

-LRB-

Dz

-RRB-

or

both

Induction

case

Assume

a

set

of

objects

xj

located

respectively

in

bunches

Bj

all

held

by

process

p

pointing

to

z

with

the

corresponding

scions

already

created

Thus

object

xl

l

j

points

to

z

xl

Bl

held

by

process

p

and

scion

-LRB-

Blxl

Dz

-RRB-

exists

Let

xj

Bj

also

held

by

p

initially

not

pointing

to

z.

Perform

xj

xl

p

-LRB-

now

xj

also

points

to

z

-RRB-

therefore

xj

is

GC-dirty

If

xl

is

not

assigned

to

thereafter

the

lemma

remains

trivially

true

since

xj

is

GC-dirty

and

scion

-LRB-

Blxl

Dz

-RRB-

exists

Therefore

consider

that

xl

is

changed

by

an

assignment

such

as

xl

p.

-LRB-

The

actual

value

of

the

right-hand

side

does

not

matter

for

the

proof

except

that

when

the

right-hand

side

is

a

pointer

to

z

it

is

as

if

the

assignment

did

not

take

place

-RRB-

Hence

xl

is

GC-dirty

Until

an

intra-bunch

collection

takes

place

at

p

no

<send.create.scion>

p

or

<send.delete.scion>

p

is

performed

and

the

lemma

remains

trivially

true

since

xj

is

GC-dirty

and

scion

-LRB-

Blxl

Dz

-RRB-

exists

When

an

intra-bunch

collection

does

execute

in

process

p

by

the

Comprehensive

Scanning

Rule

both

xj

-LRB-

containing

the

new

pointer

-RRB-

and

xl

-LRB-

previously

containing

the

pointer

with

a

scion

-RRB-

are

GC-cleaned

because

both

are

GC-dirty

therefore

stub

-LRB-

Blxl

Dz

-RRB-

disappears

-LRB-

not

in

the

new

set

of

stubs

-RRB-

and

stub

-LRB-

Bj

xj

Dz

-RRB-

is

created

According

to

the

Cross-Bunch

Collection

Algorithm

event

send.create.scion

-LRB-

Bj

xj

Dz

-RRB-

p

precedes

send.delete.scion

-LRB-

Blxl

Dz

-RRB-

p

We

assumed

that

messages

are

delivered

in

causal

order

hence

deliver.create.scion

-LRB-

Bj

xj

Dz

-RRB-

k

precedes

deliver.delete.scion

-LRB-

Blxl

Dz

-RRB-

k.

Thus

at

any

moment

there

exists

at

least

a

scion

protecting

z

either

scion

-LRB-

Blxl

Dz

-RRB-

or

scion

-LRB-

Bj

xj

Dz

-RRB-

or

both

This

terminates

the

induction

over

the

size

of

the

set

containing

objects

pointing

to

z

for

a

single

process

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

We

now

prove

the

following

theorem

Theorem

Let

xn

located

respectively

in

bunches

Bn

held

by

processes

pn

be

the

set

of

all

protected

objects

pointing

to

z

D

held

by

process

k.

Then

for

each

process

p

png

there

exists

an

object

xm

Bm

held

by

p

pointing

to

z

and

scion

-LRB-

Bmxm

Dz

-RRB-

exists

at

k

This

theorem

implies

Safety

Invariant

The

latter

states

that

whatever

number

of

cross-bunch

pointers

point

to

z

D

at

least

one

scion

protects

z.

Theorem

is

stronger

since

it

says

that

for

each

process

containing

a

pointer

to

z

a

scion

from

that

process

protects

z

It

does

not

say

whether

this

scion

effectively

corresponds

to

an

existing

pointer

to

z.

Proof

of

Theorem

We

prove

this

theorem

by

induction

over

the

size

of

the

set

containing

processes

with

pointers

to

z.

Base

case

Let

holding

be

the

only

process

with

a

pointer

to

z.

Let

process

initially

not

holding

any

object

pointing

to

z

The

only

way

for

process

to

gain

a

pointer

to

z

is

because

process

holding

pointing

to

z

performs

the

operation

send.propagate

-LRB-

-RRB-

According

to

Lemma

either

scion

-LRB-

Dz

-RRB-

exists

and

is

GC-clean

or

some

other

scion

protecting

z

exists

-LRB-

scion

-LRB-

Dz

-RRB-

xm

-RRB-

and

is

GC

dirty

In

the

first

case

the

theorem

remains

trivially

true

since

scion

-LRB-

Dz

-RRB-

exists

at

the

instant

send.propagate

-LRB-

-RRB-

is

performed

In

the

second

case

by

the

GC-Clean

Migration

Rule

process

must

GC-clean

every

GC-dirty

object

in

and

send

the

messages

to

create

the

corresponding

scions

Thus

is

scanned

stub

-LRB-

Dz

-RRB-

is

created

and

send.create.scion

-LRB-

Dz

-RRB-

is

performed

Therefore

send.create.scion

-LRB-

Dz

-RRB-

precedes

send.propagate

-LRB-

-RRB-

Since

we

assumed

that

messages

are

delivered

in

causal

order

deliver.create.scion

-LRB-

Dz

-RRB-

k

happens

before

any

other

message

sent

-LRB-

after

the

propagate

-RRB-

by

is

delivered

at

k

in

particular

a

possible

deliver.delete.scion

-LRB-

Dz

-RRB-

k

sent

by

will

be

delivered

at

k

after

scion

-LRB-

Dz

-RRB-

has

been

created

Thus

at

all

times

there

exist

at

least

a

scion

protecting

z

either

scion

-LRB-

Dz

-RRB-

or

scion

-LRB-

Dz

-RRB-

Induction

case

Let

ph

h

n

be

the

only

processes

pointing

to

z

via

the

objects

held

xh

contained

in

bunches

Bh

Let

process

ph

initially

not

holding

any

object

pointing

to

z

The

only

way

for

process

ph

to

gain

a

pointer

to

z

is

because

some

process

pl

phg

holding

an

object

xl

Bl

pointing

to

z

performs

the

operation

send.propagate

-LRB-

xl

pl

ph

-RRB-

pl

According

to

Lemma

either

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

scion

-LRB-

Blxl

Dz

-RRB-

exists

and

xl

is

GC-clean

or

some

other

scion

protecting

z

exists

-LRB-

scion

-LRB-

Blxm

Dz

-RRB-

xm

Bl

-RRB-

and

xl

is

GC-dirty

In

the

first

case

the

theorem

remains

trivially

true

as

scion

-LRB-

Blxl

Dz

-RRB-

exists

at

the

instant

send.propagate

-LRB-

xl

pl

ph

-RRB-

pl

is

performed

In

the

second

case

by

the

GC-Clean

Migration

Rule

process

pl

must

GC-clean

every

GC-dirty

object

in

pl

and

send

the

messages

to

create

the

corresponding

scions

Thus

xl

is

scanned

stub

-LRB-

Blxl

Dz

-RRB-

is

created

and

send.create.scion

-LRB-

Blxl

Dz

-RRB-

pl

is

performed

Therefore

send.create.scion

-LRB-

Blxl

Dz

-RRB-

pl

precedes

send.propagate

-LRB-

xl

pl

ph

-RRB-

pl

Since

we

assumed

that

messages

are

delivered

in

causal

order

deliver.create.scion

-LRB-

Blxl

Dz

-RRB-

k

happens

before

any

other

message

sent

-LRB-

after

the

propagate

-RRB-

by

ph

is

delivered

at

k

in

particular

a

possible

deliver.delete.scion

-LRB-

Blxl

Dz

-RRB-

k

sent

by

ph

will

be

delivered

at

k

after

scion

-LRB-

Blxl

Dz

-RRB-

has

been

created

Thus

at

all

times

at

least

one

scion

from

p

protects

z.

Liveness

We

assume

that

-LRB-

i

-RRB-

messages

for

creation

and

deletion

of

scions

are

eventually

delivered

-LRB-

in

causal

order

-RRB-

-LRB-

ii

-RRB-

every

bunch

is

eventually

collected

and

-LRB-

iii

-RRB-

intra-bunch

collection

is

complete

w.r.t.

the

collected

bunch

Our

GC

algorithm

is

clearly

not

complete

because

it

does

not

reclaim

crossbunch

cycles

of

garbage

Thus

we

only

consider

the

existence

of

acyclic

crossbunch

garbage

We

will

not

present

here

the

full

proof

of

liveness

given

its

triviality

and

consequent

lack

of

interest

We

simply

show

the

conditions

that

must

hold

and

how

they

are

ensured

The

obvious

liveness

condition

is

Liveness

Condition

An

object

not

reachable

is

eventually

reclaimed

Given

that

we

are

considering

cross-bunch

collection

Liveness

Condition

is

equivalent

to

Liveness

Condition

An

object

not

protected

by

any

scion

is

eventually

reclaimed

This

condition

is

obviously

ensured

by

the

assumptions

that

every

bunch

is

eventually

collected

and

intra-bunch

collection

is

complete

w.r.t.

the

collected

bunch

Note

that

for

liveness

we

must

ensure

that

if

an

object

is

no

longer

reachable

from

any

incoming

cross-bunch

pointer

eventually

no

scion

protects

it

Thus

the

following

liveness

condition

must

hold

Liveness

Condition

An

object

no

longer

reachable

is

eventually

not

protected

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

This

condition

is

obviously

ensured

because

-LRB-

i

-RRB-

we

assumed

that

every

bunch

is

eventually

collected

and

intra-bunch

collection

is

complete

w.r.t.

the

collected

bunch

therefore

eventually

there

will

be

no

stubs

for

disappearing

outgoing

pointers

and

-LRB-

ii

-RRB-

we

assumed

that

messages

for

deletion

of

scions

are

eventually

delivered

Therefore

a

scion

representing

a

incoming

cross-bunch

pointer

no

longer

existing

will

be

eventually

deleted

Cross-Bunch

GC

with

Replication

In

this

section

we

describe

rigorously

the

cross-bunch

GC

algorithm

with

replication

We

present

the

invariants

that

must

hold

and

prove

the

algorithm

is

safe

and

live

-LRB-

Recall

that

in

our

coherence

model

Section

we

assume

that

all

data

is

cached

by

every

process

-RRB-

Safety

The

safety

invariants

are

the

same

as

in

the

non-replicated

case

-LRB-

see

Section

-RRB-

Thus

we

do

not

repeat

them

here

Algorithm

Let

us

return

to

the

prototypical

example

illustrated

in

Figure

and

observe

that

the

initial

situation

satisfies

Safety

Invariant

At

the

time

of

x

y

i

object

z

is

reachable

-LRB-

from

both

replicas

of

y

-RRB-

and

is

protected

by

some

scion

say

scion

-LRB-

Et

Dz

-RRB-

-LRB-

presumably

but

not

necessarily

E

Cj

and

t

yj

-RRB-

As

long

as

that

scion

continues

to

exist

it

is

safe

to

delay

the

creation

of

scion

-LRB-

Bx

Dz

-RRB-

The

following

rules

and

algorithm

ensure

the

correctness

of

the

cross-bunch

collector

-LRB-

intuitively

presented

Section

-RRB-

as

will

be

formally

proved

in

the

next

sections

Comprehensive

Scanning

Rule

When

an

intra-bunch

collection

takes

place

at

process

i

every

GC-dirty

object

yi

cached

by

i

must

be

GC-cleaned

GC-Clean

Propagation

Rule

An

object

y

can

be

propagated

from

its

owner

process

i

only

if

yi

is

GC

clean

and

the

messages

to

create

the

scions

corresponding

to

yi

's

outgoing

cross-bunch

pointers

have

been

sent

Cross-Bunch

Collection

Algorithm

At

process

i

between

each

pair

of

intra-bunch

collections

execute

these

steps

in

the

following

order

For

all

new

cross-bunch

stubs

perform

the

corresponding

<send.create.scion>

i.

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

For

all

disappearing

cross-bunch

stubs

-LRB-

i

-RRB-

process

i

not

owner

of

an

object

y

whose

outgoing

pointer

no

longer

exists

-LRB-

corresponds

to

the

disappeared

stub

-RRB-

sends

a

union

message

to

the

owner

of

y

-LRB-

ii

-RRB-

process

i

owner

of

an

object

y

issues

the

corresponding

<send.delete.scion>

i

if

the

union

of

all

stubs

-LRB-

for

all

replicas

of

the

source

object

pointing

to

y

-RRB-

is

empty

The

Comprehensive

Scanning

Rule

means

that

when

an

intra-bunch

collection

takes

place

in

process

i

every

GC-dirty

object

-LRB-

locally

cached

-RRB-

must

be

scanned

therefore

creating

the

corresponding

stubs

The

GC-Clean

Propagation

Rule

means

that

an

object

y

can

not

be

propagated

from

its

owner

process

i

without

having

GC-cleaned

it

thus

after

having

created

the

corresponding

stubs

and

sent

the

messages

to

create

their

scions

The

cross-bunch

collector

runs

between

each

pair

of

intra-bunch

collections

and

compares

the

set

of

stubs

before

and

after

collection

-LRB-

recall

Section

-RRB-

The

Cross-Bunch

Collection

Algorithm

ensures

that

for

each

intra-bunch

collector

run

all

messages

to

create

scions

are

sent

before

any

union

or

delete

messages

It

is

always

the

owner

process

of

an

object

y

that

applies

the

union

rule

-LRB-

concerning

y

's

stubs

-RRB-

i.e.

the

owner

checks

if

the

union

of

all

stubs

for

all

replicas

of

y

is

empty

To

understand

the

role

of

the

Comprehensive

Scanning

Rule

consider

Figure

after

mutators

have

executed

x

y

i

y

i

and

y

j.

Then

suppose

that

GCi

-LRB-

C

-RRB-

runs

without

fulfilling

the

Comprehensive

Scanning

Rule

followed

by

the

cross-bunch

collector

that

sends

a

union

message

to

j

-LRB-

owner

of

y

-RRB-

indicating

that

stub

-LRB-

Cy

Dz

-RRB-

has

disappeared

in

process

i

According

to

the

Cross-Bunch

Collection

Algorithm

send.delete.scion

-LRB-

Cy

Dz

-RRB-

j

will

be

executed

Thus

scion

-LRB-

Cy

Dz

-RRB-

is

deleted

in

k.

Then

if

D

is

collected

object

z

is

unsafely

reclaimed

The

Comprehensive

Scanning

Rule

prevents

the

above

scenario

as

it

forces

xi

to

be

GC-cleaned

when

GCi

-LRB-

C

-RRB-

runs

thus

stub

-LRB-

Bx

Dz

-RRB-

to

be

created

in

process

i

then

according

to

the

Cross-Bunch

Collection

Algorithm

send.create.scion

-LRB-

Bx

Dz

-RRB-

i

is

performed

before

j

applies

the

union

rule

and

executes

send.delete.scion

-LRB-

Cy

Dz

-RRB-

j

Since

we

assumed

causal

delivery

scion

-LRB-

Bx

Dz

-RRB-

is

created

before

scion

-LRB-

Cy

Dz

-RRB-

is

deleted

Consequently

z

is

not

reclaimed

by

GCk

-LRB-

D

-RRB-

It

is

also

interesting

to

understand

the

role

of

the

GC-Clean

Propagation

Rule

Consider

Figure

after

the

mutator

has

executed

x

y

i

and

before

y

i.

Now

suppose

that

object

xi

is

propagated

to

some

process

w

-LRB-

in

the

absence

of

the

GC-Clean

Propagation

Rule

-RRB-

and

then

xi

is

assigned

in

such

a

way

that

it

no

longer

points

to

z

At

this

moment

the

only

scion

that

protects

z

is

scion

-LRB-

Cy

Dz

-RRB-

Suppose

that

both

replicas

of

y

are

modified

by

the

mutators

in

processes

i

and

j

such

that

they

no

longer

point

to

z.

Hence

by

the

Cross-Bunch

Collection

Algorithm

scion

-LRB-

Cy

Dz

-RRB-

is

deleted

Thus

z

may

be

unsafely

reclaimed

by

GCk

-LRB-

D

-RRB-

-LRB-

xw

still

points

to

z

-RRB-

The

GC-Clean

Propagation

Rule

prevents

the

above

scenario

as

it

forces

send.create.scion

-LRB-

Bx

Dz

-RRB-

i

to

be

performed

before

xi

is

propagated

to

w.

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

Note

that

the

GC-Clean

Propagation

Rule

is

weaker

than

its

non-replicated

counterpart

-LRB-

GC-Clean

Migration

Rule

-RRB-

While

the

latter

requires

every

locally

held

GC-dirty

object

to

be

GC-cleaned

before

the

migration

the

former

requires

only

the

object

being

propagated

to

be

GC-cleaned

This

is

due

to

the

fact

that

when

an

object

is

propagated

-LRB-

replicated

case

-RRB-

a

replica

of

the

object

and

its

stubs

remain

in

the

owner

process

Thus

they

are

still

seen

by

the

crossbunch

collector

in

that

process

On

the

contrary

in

the

non-replicated

case

when

an

object

is

migrated

its

stubs

go

along

Thus

they

will

not

be

seen

by

the

cross-bunch

collector

in

the

source

process

Proof

of

Safety

To

prove

the

safety

of

the

cross-bunch

collection

algorithm

we

start

by

proving

the

following

Lemma

Lemma

Let

xn

be

the

set

of

all

protected

objects

located

in

bunch

B

pointing

to

z

D

owned

by

k.

Let

pm

be

the

set

of

all

processes

caching

a

replica

of

xi

Then

i

n

ffl

xi

is

GC-clean

scion

-LRB-

Bxi

Dz

-RRB-

or

ffl

j

n

xi

is

GC-dirty

scion

-LRB-

Bxj

Dz

-RRB-

This

lemma

says

that

as

long

as

there

is

a

replica

of

some

object

x

located

in

B

pointing

to

z

cached

in

some

process

there

exists

a

scion

protecting

z.

Proof

of

Lemma

We

prove

this

lemma

by

induction

-LRB-

i

-RRB-

over

the

size

of

the

set

containing

objects

pointing

to

z

and

-LRB-

ii

-RRB-

over

the

size

of

the

set

containing

processes

caching

replicas

of

the

objects

pointing

to

z.

Base

case

Initially

there

is

a

single

object

B

pointing

to

z

is

owned

by

process

scion

-LRB-

Dz

-RRB-

exists

B

does

not

point

to

z

and

is

also

owned

by

and

neither

nor

are

cached

in

This

initial

situation

obviously

verifies

the

lemma

as

is

GC-clean

and

scion

-LRB-

Dz

-RRB-

exists

Now

perform

thus

points

to

z

and

is

GC-dirty

If

is

not

assigned

to

thereafter

the

lemma

remains

trivially

true

as

is

GC-clean

and

scion

-LRB-

Dz

-RRB-

exists

Therefore

consider

that

is

changed

by

an

assignment

such

as

-LRB-

the

actual

value

of

the

right-hand

side

does

not

matter

for

the

proof

except

that

when

the

right-hand

side

is

a

pointer

to

z

it

is

as

if

the

assignment

did

not

take

place

-RRB-

Hence

is

GC-dirty

Until

an

intra-bunch

collection

or

a

propagate

operation

takes

place

at

no

<send.create.scion>

or

<send.delete.scion>

is

performed

and

the

lemma

remains

trivially

true

since

is

GC-dirty

and

scion

-LRB-

Dz

-RRB-

exists

We

examine

these

two

cases

now

intra-bunch

collection

and

propagate

operation

in

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

When

an

intra-bunch

collection

does

execute

in

process

by

the

Comprehensive

Scanning

Rule

both

-LRB-

containing

the

new

pointer

-RRB-

and

-LRB-

previously

containing

the

pointer

with

a

scion

-RRB-

are

GC-cleaned

because

both

are

GC-dirty

therefore

stub

-LRB-

Dz

-RRB-

disappears

-LRB-

not

in

the

new

set

of

stubs

-RRB-

and

stub

-LRB-

Dz

-RRB-

is

created

According

to

the

Cross-Bunch

Collection

Algorithm

event

send.create.scion

-LRB-

Dz

-RRB-

precedes

send.delete.scion

-LRB-

Dz

-RRB-

We

assumed

that

messages

are

delivered

in

causal

order

hence

deliver.create.scion

-LRB-

Dz

-RRB-

k

precedes

deliver.delete.scion

-LRB-

Dz

-RRB-

k.

Thus

at

any

moment

there

exists

at

least

a

scion

protecting

z

either

scion

-LRB-

Dz

-RRB-

or

scion

-LRB-

Dz

-RRB-

or

both

Therefore

the

lemma

remains

true

when

an

intra-bunch

collection

occurs

in

Now

we

consider

a

propagate

operation

Before

a

send.propagate

-LRB-

-RRB-

takes

place

according

to

the

GC-Clean

Propagation

Rule

is

GC

cleaned

Therefore

is

scanned

stub

-LRB-

Dz

-RRB-

is

created

and

send.create.scion

-LRB-

Dz

-RRB-

is

performed

Therefore

the

lemma

remains

true

as

is

GC-clean

and

scion

-LRB-

Dz

-RRB-

exists

Induction

case

Let

ph

h

m

be

the

only

processes

caching

objects

xj

j

n

all

B

all

pointing

to

z

D

owned

by

k

The

scions

for

the

pointers

from

xj

to

z

do

exist

Object

xj

is

owned

by

pw

w

m.

Let

object

xj

owned

by

pw

initially

not

pointing

to

z

and

process

ph

initially

not

caching

any

object

pointing

to

z.

Now

perform

xj

xj

pw

thus

xj

points

to

z

and

is

GC-dirty

If

xj

is

not

assigned

to

thereafter

the

lemma

remains

trivially

true

as

xj

is

GC-clean

and

scion

-LRB-

Bxj

Dz

-RRB-

exists

Therefore

consider

that

xj

is

changed

by

an

assignment

such

as

xj

pw

-LRB-

the

actual

value

of

the

right-hand

side

does

not

matter

for

the

proof

except

that

when

the

right-hand

side

is

a

pointer

to

z

it

is

as

if

the

assignment

did

not

take

place

-RRB-

Hence

xj

is

GC-dirty

Until

an

intra-bunch

collection

or

a

propagate

operation

takes

place

at

pw

no

<send.create.scion>

pw

or

<send.delete.scion>

pw

is

performed

and

the

lemma

remains

trivially

true

since

xj

is

GC-dirty

and

scion

-LRB-

Bxj

Dz

-RRB-

exists

We

examine

these

two

cases

now

intra-bunch

collection

and

propagate

operation

in

pw

When

an

intra-bunch

collection

does

execute

in

process

pw

by

the

Comprehensive

Scanning

Rule

both

xj

-LRB-

containing

the

new

pointer

-RRB-

and

xj

-LRB-

previously

containing

the

pointer

with

a

scion

-RRB-

are

GC-cleaned

because

both

are

GC-dirty

therefore

stub

-LRB-

Bxj

Dz

-RRB-

disappears

-LRB-

not

in

the

new

set

of

stubs

-RRB-

and

stub

-LRB-

Bxj

Dz

-RRB-

is

created

According

to

the

Cross-Bunch

Collection

Algorithm

event

send.create.scion

-LRB-

Bxj

Dz

-RRB-

precedes

send.delete.scion

-LRB-

Bxj

Dz

-RRB-

We

assumed

that

messages

are

delivered

in

causal

order

hence

deliver.create.scion

-LRB-

Bxj

Dz

-RRB-

k

precedes

deliver.delete.scion

-LRB-

Bxj

Dz

-RRB-

k.

Thus

at

any

moment

there

exists

at

least

a

scion

protecting

z

either

scion

-LRB-

Bxj

Dz

-RRB-

or

scion

-LRB-

Bxj

Dz

-RRB-

or

both

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

Therefore

the

lemma

remains

true

when

an

intra-bunch

collection

occurs

in

pw

Now

we

consider

a

propagate

operation

Before

a

send.propagate

-LRB-

xj

pw

ph

-RRB-

pw

takes

place

according

to

the

GC-Clean

Propagation

Rule

xj

is

GC-cleaned

Therefore

xj

is

scanned

stub

-LRB-

Bxj

Dz

-RRB-

is

created

and

send.create.scion

-LRB-

Bxj

Dz

-RRB-

pw

is

performed

Therefore

the

lemma

remains

true

as

xj

is

GC-clean

and

scion

-LRB-

Bxj

Dz

-RRB-

exists

This

terminates

the

induction

over

the

size

of

the

set

containing

objects

pointing

to

z

and

over

the

size

of

the

set

containing

processes

caching

replicas

of

the

objects

pointing

to

z.

We

now

prove

the

following

Theorem

Theorem

Let

xn

be

the

set

of

all

protected

objects

pointing

to

z

D

owned

by

process

k

located

respectively

in

bunches

Bn

Let

pm

be

the

set

of

all

processes

caching

a

replica

of

all

bunches

mentioned

above

Then

i

n

scion

-LRB-

Bixi

Dz

-RRB-

exists

at

k

This

Theorem

implies

Safety

Invariant

Whereas

the

latter

states

that

whatever

number

of

cross-bunch

pointers

point

to

z

D

at

least

one

scion

protects

z

Theorem

says

that

at

least

one

scion

per

bunch

containing

a

pointer

to

z

protects

z

It

does

not

say

whether

this

scion

effectively

corresponds

to

an

existing

pointer

to

z.

Proof

of

Theorem

We

prove

this

theorem

by

induction

-LRB-

i

-RRB-

over

the

size

of

the

set

containing

objects

pointing

to

z

in

a

single

process

and

-LRB-

ii

-RRB-

over

the

size

of

the

set

of

processes

caching

replicas

of

objects

pointing

to

z.

Base

case

Assume

that

initially

and

are

both

owned

by

only

points

to

z

D

owned

by

k

scion

-LRB-

Dz

-RRB-

exists

and

process

does

not

cache

any

object

pointing

to

z

This

initial

situation

obviously

verifies

the

theorem

as

scion

-LRB-

Dz

-RRB-

exists

Perform

-LRB-

now

also

points

to

z

-RRB-

therefore

is

GC-dirty

If

is

not

assigned

to

thereafter

the

theorem

remains

trivially

true

as

scion

-LRB-

Dz

-RRB-

exists

Therefore

consider

an

assignment

such

as

-LRB-

the

actual

value

of

the

right-hand

side

does

not

matter

for

the

proof

except

that

when

the

righthand

side

is

a

pointer

to

z

it

is

as

if

the

assignment

did

not

take

place

-RRB-

Until

an

intra-bunch

collection

or

a

propagate

operation

takes

place

at

no

<send.create.scion>

or

<send.delete.scion>

is

performed

and

the

theorem

remains

trivially

true

In

the

first

case

i.e.

when

an

intra-bunch

collection

takes

place

at

by

Lemma

we

have

that

at

any

moment

there

exists

at

least

a

scion

protecting

z

either

scion

-LRB-

Dz

-RRB-

or

scion

-LRB-

Dz

-RRB-

or

both

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

In

the

second

case

i.e.

before

send.propagate

-LRB-

-RRB-

is

performed

by

Lemma

we

have

that

scion

-LRB-

Dz

-RRB-

exists

Induction

case

Let

objects

xj

located

in

bunches

Bj

all

pointing

to

z

all

owned

by

ph

h

m.

Initially

xj

Bj

owned

by

ph

does

not

point

to

z

scion

-LRB-

Bjxj

Dz

-RRB-

exists

and

process

ph

does

not

cache

any

object

pointing

to

z

This

initial

situation

obviously

verifies

the

theorem

as

scion

-LRB-

Bjxj

Dz

-RRB-

exists

Perform

xj

xj

ph

-LRB-

now

xj

also

points

to

z

-RRB-

therefore

xj

is

GC

dirty

If

xj

is

not

assigned

to

thereafter

the

theorem

remains

trivially

true

as

scion

-LRB-

Bjxj

Dz

-RRB-

exists

Therefore

consider

an

assignment

such

as

xj

ph

-LRB-

the

actual

value

of

the

right-hand

side

does

not

matter

for

the

proof

except

that

when

the

righthand

side

is

a

pointer

to

z

it

is

as

if

the

assignment

did

not

take

place

-RRB-

Until

an

intra-bunch

collection

or

a

propagate

operation

takes

place

at

ph

no

<send.create.scion>

ph

or

<send.delete.scion>

ph

is

performed

and

the

theorem

remains

trivially

true

In

the

first

case

i.e.

when

an

intra-bunch

collection

takes

place

at

ph

by

Lemma

we

have

that

at

any

moment

there

exists

at

least

a

scion

protecting

z

either

scion

-LRB-

Bjxj

Dz

-RRB-

or

scion

-LRB-

Bj

xj

Dz

-RRB-

or

both

In

the

second

case

i.e.

when

event

send.propagate

-LRB-

xj

ph

ph

-RRB-

ph

happens

by

Lemma

we

have

that

scion

-LRB-

Bj

xj

Dz

-RRB-

exists

Liveness

Just

as

for

the

non-replicated

case

we

simply

show

the

liveness

conditions

that

must

hold

and

how

they

are

ensured

The

proof

of

liveness

is

equally

trivial

thus

not

interesting

The

liveness

conditions

are

the

same

as

in

the

non-replicated

case

-LRB-

recall

Section

-RRB-

Therefore

we

do

not

repeat

them

here

We

add

another

assumption

to

those

made

for

the

non-replicated

case

we

assume

that

union

messages

are

eventually

delivered

Liveness

Condition

is

obviously

ensured

because

-LRB-

i

-RRB-

we

assumed

that

every

bunch

is

eventually

collected

and

intra-bunch

collection

is

complete

w.r.t.

the

collected

bunch

therefore

eventually

there

will

be

no

stubs

for

disappearing

outgoing

pointers

-LRB-

ii

-RRB-

we

assumed

that

union

messages

are

eventually

delivered

and

-LRB-

iii

-RRB-

messages

for

deleting

scions

are

eventually

delivered

Therefore

a

scion

representing

a

incoming

cross-bunch

pointer

no

longer

existing

will

be

eventually

deleted

CHAPTER

GARBAGE

COLLECTION

IN

LARCHANT

Summary

In

this

chapter

we

described

the

GC

algorithm

designed

for

Larchant

We

showed

how

GC

is

made

scalable

and

asynchronous

to

applications

how

the

coherence

interference

problem

is

solved

and

formally

proved

its

correctness

Our

GC

combines

partitioned

tracing

with

reference-listing

A

bunch

replica

is

collected

independently

from

the

rest

of

the

memory

and

concurrently

with

mutators

The

reference-listing

algorithm

runs

asynchronously

to

applications

and

manages

sets

of

stubs

and

scions

-LRB-

describing

cross-bunch

pointers

-RRB-

We

assume

causal

delivery

Cycles

of

garbage

are

reclaimed

by

collecting

groups

of

bunches

at

the

same

time

Such

groups

are

formed

according

to

a

very

simple

heuristic

that

does

not

introduce

extra

I/O

cost

a

group

contains

all

the

bunches

currently

cached

in

the

process

Both

mark-and-sweep

and

copy

collectors

can

be

used

for

intra-bunch

collection

With

replicated

objects

both

algorithms

are

capable

of

making

progress

without

requiring

coherent

data

therefore

no

coherence

operation

is

needed

for

GC

purposes

The

union

rule

is

a

fundamental

aspect

of

both

intra-bunch

and

cross-bunch

collection

it

ensures

safety

in

presence

of

replicated

objects

which

are

not

necessarily

coherent

Basically

the

union

rule

says

that

a

target

object

can

be

reclaimed

only

if

it

is

not

reachable

from

any

replica

of

its

source

objects

-LRB-

independently

of

their

location

either

in

the

same

or

in

a

different

bunch

-RRB-

Our

GC

algorithm

is

widely

applicable

given

its

asynchrony

to

applications

and

the

minimum

assumptions

we

made

concerning

objects

coherence

Chapter

Implementation

of

GC

in

Larchant

The

overall

goal

of

this

chapter

is

to

present

the

most

important

aspects

concerning

the

implementation

of

the

GC

algorithm

in

Larchant

We

describe

both

mark-and-sweep

and

copy

intra-bunch

collectors

and

the

reference-listing

cross-bunch

collector

First

we

give

a

high-level

view

of

the

architecture

of

Larchant

Then

we

describe

the

prototype

implementation

without

getting

into

much

detail

A

detailed

analysis

would

necessitate

an

exhaustive

study

of

the

source

code

which

is

out

of

the

scope

of

this

document

Then

we

describe

the

coherence

protocol

we

implemented

entry-consistency

-LSB-

-RSB-

It

follows

the

intra-bunch

collector

-LRB-

mark-and-sweep

and

copy

algorithms

-RRB-

and

the

cross-bunch

collector

We

focus

on

their

interaction

with

entry-consistency

This

description

shows

how

the

general

solutions

presented

in

Chapter

are

applied

to

the

particular

algorithms

implemented

in

the

prototype

Then

we

make

some

considerations

concerning

the

impact

of

the

size

of

the

coherence

granule

on

the

collection

algorithm

We

finish

this

chapter

with

the

description

of

the

programming

restrictions

imposed

by

GC

when

using

the

C++

language

Architecture

In

this

section

we

describe

the

Larchant

architecture

We

give

a

high-level

view

of

its

major

components

The

goal

of

this

description

is

to

present

the

main

characteristics

of

the

prototype

in

which

the

garbage

collector

is

integrated

Basics

Applications

share

a

single

address

space

spanning

every

site

in

the

network

including

secondary

storage

Objects

are

identified

by

their

virtual

ad

CHAPTER

IMPLEMENTATION

OF

GC

IN

LARCHANT

Communication

Medium

Object

Repository

Site

Process

Site

Application

Application

Application

Process

Application

Application

Cache

Manager

Process

Site

Cache

Manager

Process

Application

Application

Application

Application

Application

Process

Figure

Larchant

architecture

dress

and

are

replicated

via

a

DSM

mechanism

with

the

entry-consistency

protocol

Applications

explicitly

start

commit

and

abort

transactions

We

use

a

very

simple

transaction

model

-LSB-

-RSB-

single-process

transactions

two-phase

locking

Each

transaction

updates

a

cached

image

of

the

store

in

virtual

memory

The

store

goes

from

one

stable

state

to

another

by

means

of

a

commit

operation

Since

transactions

are

single-process

only

one

process

-LRB-

among

those

sharing

a

bunch

at

the

same

time

-RRB-

commits

a

transaction

we

call

that

process

the

principal

The

principal

of

a

bunch

is

the

process

that

initiated

the

transaction

enclosing

that

bunch

The

other

processes

sharing

the

same

bunch

at

the

same

time

are

called

associates

The

store

is

updated

with

the

cached

image

held

by

the

principal

The

overall

architecture

of

Larchant

is

illustrated

in

Figure

On

each

site

there

is

a

Unix

process

called

Cache

Manager

-LRB-

CM

-RRB-

Its

main

task

is

to

serve

applications

requests

and

manage

a

local

cache

of

bunches

The

Object

Repository

-LRB-

OBR

-RRB-

process

manages

the

stable

store

-LRB-

As

explained

afterwards

there

can

be

several

OBRs

for

increased

performance

and

reliabil

CHAPTER

IMPLEMENTATION

OF

GC

IN

LARCHANT

Bunch

Managers

downcalls

BMX

Application

Process

Application

Code

downcalls

upcalls

Figure

High-level

view

of

an

application

process

ity

-RRB-

Its

mains

task

is

to

store

-LRB-

committed

-RRB-

data

on

disk

and

serve

requests

issued

by

CMs

We

now

describe

the

most

important

structural

aspects

of

the

Larchant

architecture

This

concerns

applications

cache

management

object

storage

root

of

persistence

address

space

management

and

garbage

collection

Applications

Figure

gives

an

high-level

view

of

the

internal

organization

of

an

application

process

An

application

process

is

composed

of

the

application

code

one

or

more

Bunch

Managers

and

a

Bunch

Manager

eXecutive

-LRB-

BMX

-RRB-

all

linked

together

We

now

explain

the

functionality

of

bunch

managers

and

BMX

The

concept

of

bunch

manager

-LSB-

-RSB-

is

key

to

the

flexible

support

of

multiple

data

management

policies

in

Larchant

-LRB-

Recall

that

objects

are

allocated

in

bunches

thus

a

bunch

can

be

seen

as

a

container

for

objects

-RRB-

Such

policies

can

be

for

example

the

binding

protocol

-LRB-

what

kind

of

mechanism

can

be

used

to

remotely

access

a

bunch

RPC

or

DSM

-RRB-

-LSB-

-RSB-

a

coherence

engine

that

implements

an

optimized

coherence

protocol

for

the

bunch

the

most

appropriate

GC

algorithm

for

the

bunch

etc.

To

support

such

a

variety

of

policies

each

bunch

has

an

associated

user-level

object

called

its

bunch

manager

The

bunch

manager

implements

the

policies

most

appropriate

to

the

objects

allocated

inside

the

corresponding

bunch

Bunch

managers

can

be

improved

modified

or

developed

by

experienced

application

programmers

An

application

programmer

links

the

application

code

with

the

bunch

manager

libraries

he

thinks

are

most

appropriate

for

managing

its

data

For

example

if

an

application

only

uses

objects

that

are

known

to

be

never

shared

the

application

programmer

will

use

a

very

simple

bunch

manager

that

lacks

a

coherence

engine

to

allocate

the

application

objects

A

more

realistic

example

is

that

of

an

application

that

generates

a

large

amount

of

objects

whose

majority

CHAPTER

IMPLEMENTATION

OF

GC

IN

LARCHANT

remains

reachable

for

a

very

short

period

of

time

In

this

case

the

application

programmer

will

use

a

bunch

manager

providing

a

garbage

collector

based

on

the

copy

algorithm

In

fact

a

copy

algorithm

is

the

most

appropriate

for

this

situation

-LRB-

as

mentioned

in

Chapter

-RRB-

The

BMX

is

a

library

that

is

linked

with

each

application

running

on

top

of

Larchant

It

allows

the

application

code

to

interact

with

the

rest

of

the

system

-LRB-

the

local

CM

more

exactly

-RRB-

via

bunch

managers

BMX

provides

basic

support

for

the

specialized

policies

implemented

by

bunch

managers

In

particular

it

implements

the

minimal

generic

mechanisms

of

coherence

management

and

garbage

collection

that

are

used

by

bunch

managers

For

example

the

BMX

starts

a

group

garbage

collection

including

every

bunch

cached

by

the

process

by

up-calling

the

corresponding

GC

methods

provided

by

the

bunch

managers

Cache

Manager

On

each

client

site

there

is

a

CM

process

It

serves

applications

requests

performed

through

their

BMX

library

manages

the

slices

of

the

shared

address

space

which

are

reserved

for

the

creation

of

new

bunches

by

local

applications

and

collects

cached

bunches

when

they

are

not

being

used

locally

We

describe

briefly

each

one

of

these

functions

now

A

fundamental

request

served

by

CMs

is

that

of

getting

a

bunch

not

yet

mapped

in

a

process

for

application

access

In

other

words

a

CM

serves

bunch

faults

as

explained

now

When

an

application

accesses

an

object

in

a

bunch

not

yet

mapped

in

the

process

a

bunch

fault

is

automatically

generated

in

the

binding

method

of

corresponding

bunch

manager

-LRB-

recall

the

binding

policy

mentioned

in

the

previous

section

-RRB-

The

bunch

manager

issues

a

request

for

the

faulted

bunch

to

the

BMX

Then

the

BMX

forwards

the

request

to

the

local

CM

that

looks

for

the

bunch

in

its

cache

Then

there

are

the

following

possible

evolutions

ffl

The

CM

finds

the

bunch

in

its

cache

It

returns

immediately

the

bunch

contents

to

the

BMX

that

forwards

it

to

the

bunch

manager

The

bunch

manager

maps

the

bunch

in

the

process

and

the

application

resumes

ffl

The

faulted

bunch

is

not

in

CM

's

cache

The

CM

asks

the

bunch

to

the

OBR

which

can

reply

either

with

the

bunch

contents

or

with

the

identification

of

the

principal

's

CM

-LRB-

The

principal

's

CM

is

the

CM

on

the

site

where

the

principal

executes

-RRB-

In

the

first

case

we

proceed

as

in

the

previous

item

In

the

second

case

the

CM

sends

a

request

to

the

principal

's

CM

asking

for

the

bunch

This

CM

forwards

the

request

to

the

principal

's

BMX

which

upcalls

the

the

definitions

of

principal

and

associate

given

in

Section

the

principal

of

a

bunch

is

the

process

that

initiated

a

transaction

enclosing

that

bunch

the

other

processes

sharing

the

same

bunch

at

the

same

time

are

called

associates

CHAPTER

IMPLEMENTATION

OF

GC

IN

LARCHANT

binding

method

of

the

corresponding

bunch

manager

The

reply

from

the

bunch

manager

depends

of

the

binding

policy

it

allows

the

sharing

of

the

bunch

via

DSM

or

RPC

Here

we

only

address

the

DSM

case

the

bunch

manager

replies

with

the

bunch

contents

to

the

principal

's

BMX

which

forwards

it

to

the

principal

's

CM

which

also

forwards

the

reply

to

the

CM

serving

the

bunch

fault

Now

we

proceed

as

in

the

previous

item

Note

that

in

the

above

description

when

we

say

that

the

bunch

contents

are

sent

to

the

application

this

does

not

mean

that

all

the

bunch

is

sent

at

once

In

fact

this

is

done

lazily

i.e.

we

only

send

those

objects

-LRB-

and

some

system

information

-RRB-

that

are

effectively

needed

to

allow

the

application

to

proceed

Another

important

request

served

by

CMs

is

that

of

obtaining

a

coherent

replica

of

an

object

When

an

application

needs

a

coherent

replica

of

an

object

whose

enclosing

bunch

has

already

been

mapped

in

the

process

the

mutator

-LRB-

through

the

BMX

library

-RRB-

sends

a

request

to

the

local

CM

This

forwards

the

request

to

the

CM

in

the

site

where

the

owner

process

is

running

Then

this

CM

forwards

the

request

to

the

owner

The

reply

with

the

object

contents

flows

in

the

opposite

direction

Each

CM

manages

the

slices

of

the

address

space

reserved

for

the

local

applications

usage

New

bunches

created

by

local

applications

are

allocated

in

the

mentioned

slices

When

local

slices

are

exhausted

the

CM

gets

more

using

a

procedure

described

in

Section

Each

CM

can

also

collect

the

bunches

kept

in

its

cache

For

this

purpose

the

CM

launches

a

process

where

the

bunches

to

be

collected

are

mapped

The

garbage

collection

algorithm

used

is

the

one

implemented

by

the

corresponding

bunch

manager

The

bunches

that

are

candidates

for

being

collected

by

the

CM

are

those

that

are

not

being

used

by

any

local

application

-LRB-

More

details

on

this

in

Section

-RRB-

Object

Repository

The

main

task

of

the

OBR

is

to

store

-LRB-

committed

-RRB-

data

on

disk

and

serve

requests

issued

by

CMs

Figure

shows

only

one

OBR

process

however

there

can

be

several

for

increased

performance

and

reliability

Note

that

the

existence

of

several

OBRs

may

lead

to

the

need

of

a

two-phase

commit

protocol

-LSB-

-RSB-

We

do

not

address

this

issue

because

it

is

out

of

the

scope

of

this

document

Recall

that

if

a

bunch

is

requested

by

a

CM

and

that

bunch

is

already

cached

in

some

application

process

the

OBR

returns

the

identification

of

the

principal

's

CM

For

this

purpose

the

OBR

keeps

for

each

bunch

the

identification

of

its

principal

's

CM

The

OBR

can

garbage

collect

any

bunch

of

the

store

it

launches

a

process

where

the

bunches

to

be

collected

are

mapped

and

invokes

the

corresponding

the

difference

between

principal

and

owner

the

former

relates

to

a

bunch

while

the

latter

relates

to

a

coherence

granule

inside

a

bunch

i.e.

an

object

The

owner

of

an

object

in

a

bunch

B

is

not

necessarily

B

's

principal

CHAPTER

IMPLEMENTATION

OF

GC

IN

LARCHANT

methods

of

their

managers

More

details

on

this

in

Section

Persistent

Root

In

Larchant

there

is

a

special

persistent

object

called

global-root

which

is

the

persistent

root

of

the

Larchant

store

The

global-root

object

holds

references

to

other

objects

which

can

be

associated

with

human-readable

names

also

kept

in

the

global-root

object

It

provides

an

interface

similar

to

a

name

service

as

it

offers

methods

to

insert

and

remove

references

to

obtain

a

reference

to

an

object

given

its

human

readable

name

etc.

The

global-root

object

is

like

any

other

object

in

the

system

and

can

be

replicated

all

over

the

network

It

is

allocated

in

a

bunch

whose

manager

implements

a

coherence

protocol

that

ensures

a

single-writer

multiple-reader

sharing

policy

Note

that

for

security

reasons

the

global-root

object

is

inside

a

bunch

that

can

be

accessed

for

writing

only

by

privileged

processes

in

particular

CMs

-LRB-

see

bunch

protection

in

Section

-RRB-

Address

Space

Management

Larchant

supports

a

single

address

space

spanning

every

site

in

the

network

including

secondary

storage

Objects

are

identified

by

their

virtual

address

In

order

to

ensure

that

bunches

have

non-overlapping

addresses

there

is

a

special

persistent

object

called

Global

Address

Space

Manager

-LRB-

GASM

-RRB-

which

manages

a

free

list

of

address

slices

Each

CM

holds

one

-LRB-

or

more

-RRB-

slice

-LRB-

s

-RRB-

for

local

applications

usage

i.e.

where

local

applications

allocate

new

bunches

Each

time

a

CM

needs

more

address

space

it

accesses

a

coherent

replica

of

the

GASM

object

and

invokes

a

method

to

reserve

a

slice

A

slice

is

freed

by

a

CM

and

returned

to

the

GASM

as

a

free

slice

when

the

bunches

allocated

inside

it

become

empty

i.e

no

longer

contain

any

reachable

object

The

GASM

object

is

just

like

any

other

object

in

the

system

It

is

referenced

from

the

global-root

object

and

can

be

replicated

all

over

the

network

It

is

allocated

inside

a

bunch

whose

manager

implements

a

coherence

protocol

that

ensures

a

single-writer

multiple-reader

sharing

policy

Note

that

for

security

reasons

the

GASM

object

is

inside

a

bunch

that

can

be

accessed

only

by

privileged

processes

in

particular

CMs

-LRB-

see

bunch

protection

in

Section

-RRB-

Garbage

Collection

Each

bunch

is

collected

by

a

specific

GC

algorithm

which

is

implemented

by

the

corresponding

bunch

manager

A

bunch

can

be

collected

in

the

application

process

or

in

a

process

launched

by

a

CM

or

by

the

OBR

Note

that

in

the

last

two

cases

the

process

in

which

the

collector

runs

does

not

contain

a

mutator

This

is

equivalent

to

run

the

collector

in

GC-only

mode

CHAPTER

IMPLEMENTATION

OF

GC

IN

LARCHANT

The

GC

of

a

bunch

being

used

by

an

application

can

be

started

either

by

the

bunch

manager

or

by

the

process

's

BMX

In

the

first

case

the

goal

is

to

collect

a

single

bunch

The

collection

starts

because

the

bunch

manager

detected

that

the

bunch

size

has

reached

some

threshold

or

because

the

application

explicitly

required

so

In

the

second

case

-LRB-

collection

started

by

the

BMX

-RRB-

the

goal

is

to

collect

every

bunch

in

the

process

at

the

same

time

in

order

to

reclaim

cross-bunch

cycles

This

collection

starts

when

the

amount

of

memory

allocated

by

the

bunches

mapped

in

the

process

reaches

some

threshold

or

when

the

application

requires

it

explicitly

through

a

bunch

manager

The

CM

collects

the

bunches

kept

in

its

cache

A

goal

of

this

GC

is

to

reclaim

cross-bunch

cycles

in

the

group

of

bunches

cached

by

the

CM

Another

goal

is

to

collect

those

bunches

whose

size

has

increased

considerably

since

they

were

cached

and

were

not

collected

while

being

used

by

applications

These

bunches

are

suspected

to

contain

a

large

amount

of

unreachable

objects

Thus

the

bunches

that

are

candidates

for

being

collected

by

the

CM

are

those

that

are

not

being

used

by

any

local

application

and

those

that

are

suspected

to

have

a

large

amount

of

unreachable

objects

The

OBR

collects

bunches

that

are

not

being

cached

by

any

CM

The

goal

is

to

collect

large

groups

of

bunches

in

order

to

reclaim

a

maximum

amount

of

cross-bunch

cycles

In

summary

a

GC

started

by

the

BMX

reclaims

small-scale

cross-bunch

cycles

a

GC

launched

by

a

CM

reclaims

medium-scale

cross-bunch

cycles

a

GC

launched

by

the

OBR

reclaims

large-scale

cross-bunch

cycles

We

do

not

go

into

more

details

concerning

when

such

collections

are

launched

or

in

the

case

of

the

OBR

which

heuristics

can

be

used

to

form

groups

of

bunches

in

order

to

maximize

the

amount

of

cycles

reclaimed

Prototype

Implementation

We

implemented

a

prototype

of

Larchant

which

is

as

simple

as

possible

yet

sufficiently

complete

to

test

the

GC

algorithms

and

to

execute

distributed

applications

with

persistent

objects

In

this

section

we

present

the

main

aspects

of

the

prototype

Larchant

is

programmed

in

C++

The

total

size

of

the

system

is

about

lines

In

fact

many

of

these

lines

correspond

to

comments

and

to

probing

methods

The

latter

monitor

the

system

internal

data

structures

and

check

internal

invariants

These

monitoring

code

is

used

only

for

debugging

Thus

the

core

of

Larchant

-LRB-

including

the

mark-and-sweep

collector

the

copy

collector

the

DSM

mechanism

with

the

entry-consistency

protocol

the

OBR

the

CM

and

the

global-root

and

GASM

objects

-RRB-

comprises

about

lines

of

C++

code

The

prototype

runs

on

a

network

of

DEC

Alpha

workstations

connected

by

CHAPTER

IMPLEMENTATION

OF

GC

IN

LARCHANT

FDDI

The

host

operating

system

is

which

offers

a

address

space

We

take

advantage

of

this

large

address

space

by

mapping

objects

at

fixed

addresses

therefore

avoiding

the

cost

of

swizzling

-LSB-

-RSB-

In

the

rest

of

this

section

we

describe

the

most

relevant

high-level

aspects

concerning

application

processes

their

threads

and

ports

and

the

internal

structure

of

bunches

and

objects

Processes

Threads

and

Ports

The

current

prototype

implements

a

simplified

version

of

the

Larchant

architecture

previously

described

There

is

a

single

Unix

process

per

site

that

includes

some

application

program

linked

with

the

BMX

library

and

the

CM

code

-LRB-

recall

Figure

-RRB-

The

OBR

runs

in

a

separated

process

Within

each

application

process

there

are

the

following

threads

each

one

with

a

particular

functionality

and

using

one

or

more

communication

ports

-LRB-

see

Figure

-RRB-

ffl

Collector

Thread

It

executes

the

code

of

the

intra-bunch

and

cross-bunch

collectors

uses

port

bmxUdpSend

to

send

messages

related

to

GC

in

particular

the

move

messages

of

a

copy

collector

as

described

in

Section

ffl

Executive

Thread

It

executes

the

code

of

the

BMX

library

it

uses

the

ports

bmxTcpRecv

and

bmxUdpRecv

on

which

it

waits

for

messages

requesting

a

bunch

-LRB-

TCP

protocol

-RRB-

or

a

coherence

operation

-LRB-

UDP

protocol

-RRB-

as

described

in

Section

ffl

Mutator

thread

A

mutator

contains

one

or

more

threads

that

execute

the

code

of

the

application

program

They

use

-LRB-

through

the

BMX

library

thus

transparently

to

the

application

code

-RRB-

the

ports

bmxTcpSend

and

bmxUdpSend

to

send

requests

for

bunches

-LRB-

TCP

protocol

-RRB-

or

for

coherent

objects

-LRB-

UDP

protocol

-RRB-

and

receive

the

corresponding

replies

ffl

Manager

Thread

It

executes

the

code

of

the

CM

waits

for

messages

on

ports

cmTcpRecv

and

cmUdpRecv

requesting

a

bunch

-LRB-

TCP

protocol

-RRB-

or

a

coherence

operation

-LRB-

UDP

protocol

-RRB-

sends

messages

to

local

application

processes

or

to

remote

CMs

requesting

bunches

-LRB-

cmTcpSend

-RRB-

or

a

coherence

operation

-LRB-

cmUdpSend

-RRB-

fact

the

implementation

of

the

Alpha

microprocessor

we

use

supports

a

virtual

address

space

with

a

page

size

of

Kbytes

-LSB-

-RSB-

CHAPTER

IMPLEMENTATION

OF

GC

IN

LARCHANT

S

S

S

System

Process

OBR

S

thread

mutator

thread

thread

thread

S

manager

R

thread

cmTcpRecv

cmUdpRecv

cmTcpSend

cmUdpSend

bunch

requests

ports

Unix

process

message

flow

Process

i

Application

Process

j

Application

R

collector

collector

manager

R

obrTcp

R

bmxTcpRecv

bmxUdpRecv

R

thread

mutator

bmxTcpSend

bmxUdpSend

thread

executive

bunch

coherence

and

GC

requests

cmUdpRecv

bmxTcpSend

bmxUdpSend

R

bmxTcpRecv

bmxUdpRecv

cmTcpSend

cmUdpSend

cmTcpRecv

thread

executive

Figure

Sketch

of

Larchant

implementation

The

ports

whose

name

encloses

Tcp

use

the

TCP

protocol

those

whose

name

encloses

Udp

use

UDP

A

port

marked

R

means

that

there

is

a

thread

mostly

waiting

on

it

in

order

to

serve

requests

a

port

marked

S

means

that

a

thread

uses

it

to

send

messages

Suppose

that

a

mutator

of

some

application

process

i

needs

to

access

a

bunch

B

which

is

not

yet

locally

mapped

-LRB-

Recall

the

mechanism

described

in

Section

-RRB-

Then

the

mutator

thread

requests

the

faulted

bunch

through

the

BMX

library

to

the

local

CM

-LRB-

by

sending

a

message

from

bmxTcpSend

to

cmTcpRecv

-RRB-

If

the

faulted

bunch

is

not

locally

cached

the

CM

requests

it

to

the

OBR

by

sending

a

message

from

port

cmTcpSend

to

obrTcp

If

the

local

CM

requests

the

faulted

bunch

to

the

principal

's

CM

it

sends

a

message

from

port

cmTcpSend

to

principal

's

cmTcpRecv

This

CM

will

forward

the

request

from

cmTcpSend

to

bmxTcpRecv

The

reply

will

flow

in

the

opposite

direction

When

a

mutator

needs

a

coherent

replica

of

some

object

-LRB-

located

inside

a

bunch

that

has

already

been

mapped

in

the

enclosing

process

-RRB-

it

uses

the

bmxUdpSend

port

-LRB-

through

the

BMX

library

-RRB-

to

send

a

request

message

to

the

local

CM

-LRB-

port

cmUdpRecv

-RRB-

This

forwards

the

request

to

the

CM

local

to

the

object

's

owner

-LRB-

from

cmUdpSend

to

cmUdpRecv

-RRB-

according

to

the

coherence

protocol

-LRB-

see

Section

-RRB-

The

latter

CM

forwards

the

request

to

the

object

's

owner

-LRB-

to

port

bmxUdpRecv

-RRB-

The

reply

flows

in

the

opposite

direction

with

the

object

's

contents

the

current

release

the

mutator

and

the

CM

run

within

the

same

Unix

process

Thus

the

message

exchange

is

just

a

procedure

call

CHAPTER

IMPLEMENTATION

OF

GC

IN

LARCHANT

Finally

the

bmxUdpRecv

port

also

receives

move

messages

from

the

intra-bunch

copy

collector

thread

of

another

process

caching

a

replica

of

the

same

bunch

-LRB-

recall

Section

-RRB-

Bunches

and

Objects

A

bunch

contains

a

unlimited

number

of

segments

A

segment

is

a

set

of

contiguous

virtual

memory

pages

it

has

a

fixed

-LRB-

but

arbitrary

-RRB-

size

Segments

in

a

bunch

do

not

need

to

be

contiguous

Each

segment

is

backed

by

a

Unix

file

All

files

backing

segments

in

a

bunch

have

the

same

owner

and

protections

Special

objects

like

the

global-root

and

the

GASM

are

allocated

in

bunches

owned

by

the

administrator

of

the

Larchant

store

therefore

they

can

be

accessed

only

by

privileged

processes

like

the

OBR

and

CMs

-LRB-

recall

Sections

and

-RRB-

A

segment

can

be

mapped

using

Unix

V

shared

memory

memory

mapped

files

or

by

reading

the

file

into

an

anonymous

memory

mapped

region

The

memory

mapped

file

mechanism

is

primarily

intended

to

be

used

within

a

local

area

network

since

it

can

take

advantage

of

existing

distributed

file

systems

-LRB-

such

as

NFS

-LSB-

-RSB-

-RRB-

Due

to

garbage

collection

some

segments

in

a

bunch

may

become

empty

i.e.

with

no

reachable

objects

inside

In

this

case

the

virtual

memory

space

occupied

by

such

a

segment

is

freed

to

the

corresponding

CM

The

CM

adds

that

memory

space

to

its

list

of

free

address

space

slices

for

future

utilization

Each

object

has

a

header

which

contains

system-specific

information

such

as

its

size

its

state

-LRB-

if

it

is

mapped

for

example

-RRB-

a

pointer

to

the

bunch

manager

corresponding

to

the

enclosing

bunch

etc.

Note

that

accessing

this

information

does

not

cost

any

extra

swapping

In

fact

when

Larchant

needs

to

access

it

the

corresponding

object

is

already

in

main

memory

due

to

application

needs

On

the

contrary

GC

specific

information

such

as

the

color

of

an

object

might

be

accessed

by

the

collector

when

the

corresponding

object

is

swapped

out

Thus

to

avoid

extra

swapping

costs

the

color

of

each

object

is

stored

in

a

bitmap

separated

from

the

corresponding

object

Coherence

Engine

The

Larchant

prototype

supports

a

single

bunch

manager

type

which

coherence

engine

implements

the

entry-consistency

protocol

-LSB-

-RSB-

described

in

Section

We

recall

here

only

the

most

important

aspects

Every

shared

object

has

a

owner

which

is

either

the

process

currently

holding

the

object

's

write

token

or

the

process

that

last

held

the

write

token

-LRB-

There

can

either

be

several

read

tokens

or

one

exclusive

write

token

associated

with

a

shared

object

-RRB-

A

write

token

can

only

be

obtained

from

the

object

's

owner

In

the

current

prototype

a

read

token

is

also

obtained

from

the

object

's

owner

in

the

future

CHAPTER

IMPLEMENTATION

OF

GC

IN

LARCHANT

create

object

read

invalid

owner

owner

acquire-write

acquire-write

deliver.invalidate

acquire-read

write

acquire-read

release-read

owner

not

read

deliver.acquire-write

deliver.acquire-read

deliver.invalidate

deliver.acquire-read

deliver.acquire-write

acquire-read

acquire-write

release-writerelease-read

deliver.acquire-read

deliver.acquire-write

Figure

State-machine

for

the

entry-consistency

protocol

we

intend

to

optimize

this

by

allowing

a

read

token

to

be

obtained

from

any

process

already

holding

a

read

token

The

acquisition

of

a

write

token

for

an

object

implies

the

invalidation

of

every

readable

replica

of

that

object

in

other

processes

Each

process

receiving

an

invalidation

message

acknowledges

it

via

a

reply

Figure

illustrates

the

state-machine

that

represents

the

entry-consistency

protocol

just

described

The

acquisition

of

a

write

token

or

of

a

read

token

is

made

by

operations

acquire-write

and

acquire-read

respectively

When

the

object

's

owner

is

remote

acquire-write

designates

the

execution

of

send.acquirewrite

deliver.acquire-write

and

the

corresponding

reply

-LRB-

The

same

reasoning

applies

to

the

operation

acquire-read

-RRB-

In

Figure

the

character

identifies

those

requests

-LRB-

read

or

write

-RRB-

that

are

not

immediately

satisfied

i.e

they

will

be

served

once

the

concerned

object

is

released

For

example

when

an

object

is

in

state

owner-read

a

request

for

a

write

token

-LRB-

deliver.acquire-write

-RRB-

will

be

served

only

after

the

object

's

state

becomes

owner

until

then

the

requesting

thread

remains

blocked

Intra-Bunch

Garbage

Collector

Our

bunch

manager

implements

two

GC

algorithms

mark-and-sweep

and

copy

Both

algorithms

can

be

executed

concurrently

with

the

mutator

or

in

GC-only

CHAPTER

IMPLEMENTATION

OF

GC

IN

LARCHANT

mode

i.e.

the

mutator

is

halted

while

the

collector

runs

When

running

in

concurrent

mode

both

algorithms

are

based

on

the

replication

technique

from

Nettles

and

O'Toole

-LSB-

-RSB-

-LRB-

recall

Section

-RRB-

In

this

section

we

present

the

most

important

implementation

aspect

of

both

intra-bunch

GC

algorithms

their

state-machines

The

state-machine

shows

the

states

i.e.

the

colors

of

each

object

-LRB-

recall

the

tricolor

algorithm

described

in

Section

-RRB-

and

the

transitions

between

states

It

also

shows

how

the

GC

algorithm

interacts

with

entry-consistency

The

C++

code

of

the

collectors

resulted

from

a

simple

mapping

-LRB-

almost

automatic

-RRB-

of

the

corresponding

state-machine

into

-LRB-

i

-RRB-

a

variable

-LRB-

stored

in

a

bitmap

-RRB-

associated

to

each

object

defining

its

color

-LRB-

ii

-RRB-

a

few

C++

switch

instructions

that

for

each

initial

color

sets

the

new

color

of

the

concerned

object

depending

of

the

transition

as

defined

by

the

state-machine

We

terminate

this

section

with

a

description

of

how

GC-dirty

objects

are

detected

by

taking

advantage

of

the

entry-consistency

protocol

Mark-and-Sweep

The

mark-and-sweep

algorithm

in

presence

of

the

entry-consistency

protocol

is

described

by

the

-LSB-

-RSB-

of

Figure

It

results

from

the

combination

of

the

state-machines

of

Figures

-LRB-

a

-RRB-

and

Some

aspects

of

the

statechart

deserve

special

attention

ffl

A

reachable

object

can

be

made

black

by

the

collector

no

matter

the

object

coherence

state

except

when

it

is

being

written

-LRB-

write

state

-RRB-

In

this

case

after

being

scanned

the

object

goes

from

gray

to

gray-log

which

means

that

the

object

will

have

to

be

scanned

again

ffl

G

and

H

are

auxiliary

states

used

only

for

clarity

For

example

when

going

from

coherence

state

invalid

to

write

due

to

an

acquire-write

operation

the

object

GC

state

H

or

G

is

maintained

This

means

that

from

black

or

gray-log

-LRB-

state

H

-RRB-

the

object

color

goes

to

gray-log

-LRB-

state

H

-RRB-

the

other

colors

white

and

gray

-LRB-

state

G

-RRB-

do

not

change

Similarly

the

transition

from

coherence

state

not-owner

to

write

implies

that

the

object

's

color

goes

from

black

or

gray-log

to

gray-log

-LRB-

auxiliary

state

H

is

kept

-RRB-

in

the

same

coherence

transition

a

white

object

remains

white

a

gray

object

remains

gray

ffl

The

only

coherence

state

transitions

that

may

imply

a

color

change

are

those

that

end

in

the

write

state

from

owner

to

write

from

invalid

to

write

and

from

not-owner

to

write

The

color

change

is

always

from

black

to

gray-log

Any

other

coherence

state

transition

does

not

influence

an

object

's

color

state-chart

is

a

visual

formalism

similar

to

but

more

powerful

than

a

state-machine

It

extends

the

latter

with

the

notions

of

depth

orthogonality

and

broadcast

communication

leading

to

more

concise

specifications

CHAPTER

IMPLEMENTATION

OF

GC

IN

LARCHANT

white

gray

mark

scan

black

scan

white

gray

mark

scan

black

scan

acquire-write

create

object

owner-read

write

read

white

gray

mark

gray-log

scan

H

G

gray-log

gray-log

white

gray

mark

black

gray-log

scan

scan

H

G

invalid

gray-log

white

gray

mark

black

scan

not-owner

owner

white

gray

mark

black

scan

scan

H

G

scan

G

H

gray-log

release-read

acquire-read

release-write

acquire-write

release-write

acquire-write

acquire-write

acquire-read

deliver.invalidate

acquire-read

release-read

deliver.acquire-write

Figure

State-chart

of

an

object

located

in

a

bunch

which

manager

supports

the

entry-consistency

protocol

and

the

mark-and-sweep

GC

algorithm

From

the

state-chart

it

is

clear

that

only

when

an

object

is

acquired

for

writing

and

the

intra-bunch

collector

is

running

it

is

necessary

to

perform

a

GC

operation

possibly

changing

the

object

's

color

from

black

to

gray-log

In

addition

it

is

clear

that

the

GC

algorithm

never

requires

a

coherence

operation

-LRB-

e.g.

acquiring

a

read

or

a

write

token

-RRB-

Thus

there

is

no

coherence

interference

Copy

In

this

section

we

describe

the

state-machine

of

the

copy

collector

We

pay

special

attention

to

the

interaction

between

the

GC

algorithm

and

the

entryconsistency

protocol

Given

the

replication

technique

the

state-machine

describing

the

colors

and

CHAPTER

IMPLEMENTATION

OF

GC

IN

LARCHANT

white

black

gray

move

scan

patch

move

flip

move

-LRB-

b

-RRB-

gray-log

gray-log

gray

white

create

object

move

move

acquire-write

-LRB-

a

-RRB-

white

gray

black

gray-log

gray/gray

gray-log/gray

gray/black

whitegray

white/nil

flip

scan

patch

move

acquire-write

-LRB-

c

-RRB-

move

acquire-write

move

black

Figure

State-machine

of

a

copy

collector

-LRB-

a

-RRB-

from-space

object

-LRB-

b

-RRB-

tospace

object

-LRB-

c

-RRB-

merge

of

-LRB-

a

-RRB-

and

-LRB-

b

-RRB-

In

the

latter

the

color

of

an

object

-LRB-

including

both

replicas

in

from-space

and

in

to-space

-RRB-

is

represented

as

colorof-replica-in-from-space

color-of-replica-in-to-space

to

make

clear

which

replica

is

affected

in

each

transition

the

changed

color

is

shown

in

bold

transitions

of

an

object

located

in

from-space

is

different

from

the

statemachine

of

its

replica

in

to-space

Since

the

collector

only

scans

and

patches

to-space

objects

only

these

will

become

black

Objects

located

in

from-space

can

only

become

gray

as

they

are

moved

to

to-space

they

are

never

scanned

nor

patched

Figure

illustrates

the

state-machines

describing

-LRB-

a

-RRB-

the

colors

and

transitions

of

an

object

in

from-space

-LRB-

b

-RRB-

the

colors

and

transitions

of

an

object

's

replica

in

to-space

and

-LRB-

c

-RRB-

the

merged

color

and

transitions

of

both

replicas

of

an

object

-LRB-

in

from-space

and

in

to-space

-RRB-

In

the

latter

state-machine

the

dashed

ellipses

and

their

state

labels

highlight

the

equivalence

with

the

general

state-machine

presented

in

Figure

The

operations

that

trigger

the

color

transitions

are

executed

either

by

the

collector

-LRB-

move

scan

patch

flip

-RRB-

or

by

the

mutator

-LRB-

acquire-write

-RRB-

There

is

no

mutator

operation

in

the

state-machine

for

a

to-space

replica

-LRB-

Figure

-LRB-

b

-RRB-

-RRB-

because

the

mutator

only

accesses

from-space

replicas

Hence

the

mutator

operation

acquire-write

is

applied

only

on

the

from-space

replica

Conversely

move

affects

both

replicas

in

from-space

and

in

to-space

the

scan

patch

and

flip

operations

affect

only

the

to-space

replica

Note

that

when

the

from-space

replica

's

color

is

gray-log

and

its

to-space

replica

is

black

the

moving

of

the

former

by

the

collector

forces

the

transition

of

the

to-space

replica

to

gray

This

is

due

to

the

fact

that

the

to-space

replica

must

be

scanned

again

The

copy

algorithm

in

presence

of

the

entry-consistency

protocol

is

described

by

the

state-chart

of

Figure

It

results

from

the

combination

of

the

statemachines

of

Figures

and

-LRB-

a

-RRB-

Some

aspects

of

the

state-chart

deserve

to

CHAPTER

IMPLEMENTATION

OF

GC

IN

LARCHANT

G

release-read

create

object

owner

white

move

G

gray

move

gray-log

owner-read

white

move

gray

move

gray-log

write

white

move

G

gray-log

invalid

white

move

G

gray

move

gray-log

read

white

move

gray

move

gray-log

gray-log

move

gray

move

white

not-owner

deliver.acquire-write

acquire-write

release-read

acquire-read

release-write

acquire-write

release-write

acquire-write

acquire-read

deliver.invalidate

acquire-read

Figure

State-chart

of

a

from-space

object

located

in

a

bunch

which

manager

supports

the

entry-consistency

protocol

and

the

copy

GC

algorithm

be

mentioned

ffl

G

is

an

auxiliary

state

used

only

for

clarity

For

example

when

going

from

coherence

state

invalid

to

write

due

to

an

acquire-write

operation

the

object

GC

state

G

is

maintained

This

means

that

from

gray

or

gray-log

-LRB-

state

G

-RRB-

the

object

color

goes

to

gray-log

-LRB-

state

G

-RRB-

the

other

color

-LRB-

white

-RRB-

does

not

change

Similarly

the

transition

from

coherence

state

not-owner

to

write

implies

that

the

object

's

color

goes

from

gray

or

gray-log

to

gray-log

-LRB-

auxiliary

state

G

is

kept

-RRB-

in

the

same

coherence

transition

a

white

object

remains

white

The

same

happens

when

going

from

coherence

state

owner

to

write

ffl

When

the

collector

finds

a

not-owned

object

-LRB-

states

read

invalid

notowner

-RRB-

it

can

only

move

it

to

to-space

after

its

new

address

has

been

locally

delivered

-LRB-

sent

by

the

object

's

owner

-RRB-

To

avoid

halting

the

collection

while

waiting

for

the

object

's

to-space

address

the

collector

thread

sends

a

message

to

the

object

's

owner

requesting

its

new

address

and

does

not

wait

for

the

reply

Once

the

message

is

sent

the

collector

scans

the

CHAPTER

IMPLEMENTATION

OF

GC

IN

LARCHANT

from-space

object

-LRB-

that

can

even

be

invalid

-RRB-

and

continues

tracing

the

object

color

remains

unchanged

Later

when

the

object

's

new

address

is

delivered

the

object

will

be

moved

In

the

current

implementation

new

address

messages

are

batched

and

sent

by

the

owner

process

into

a

single

message

ffl

The

only

coherence

state

transitions

that

may

imply

a

color

change

are

those

that

end

in

the

write

state

from

owner

to

write

from

invalid

to

write

and

from

not-owner

to

write

The

color

change

is

always

from

gray

to

gray-log

Any

other

coherence

state

transition

does

not

influence

an

object

's

color

From

the

state-chart

it

is

clear

that

only

when

an

object

is

acquired

for

writing

and

the

intra-bunch

collector

is

running

it

is

necessary

to

perform

a

GC

operation

possibly

changing

the

object

's

color

from

gray

to

gray-log

In

addition

it

is

clear

that

the

GC

algorithm

never

requires

a

coherence

operation

-LRB-

e.g.

acquiring

a

read

or

a

write

token

-RRB-

Thus

there

is

no

coherence

interference

Detection

of

GC-dirty

Objects

As

discussed

in

Chapter

the

collector

must

know

which

objects

are

GC-dirty

There

are

numerous

ways

to

implement

the

detection

of

GC-dirty

objects

-LRB-

or

bunches

-RRB-

A

compiler

may

insert

a

write-barrier

instrumenting

every

pointer

write

that

sets

a

GC-dirty

bit

associated

with

the

object

-LSB-

-RSB-

collecting

a

bunch

resets

the

GC-dirty

bits

of

its

objects

If

the

coherence

protocol

only

allows

the

owner

of

an

object

to

modify

it

then

we

may

instead

conservatively

assume

that

any

object

the

current

process

owns

is

GC-dirty

Another

alternative

assuming

objects

are

protected

by

locks

is

to

set

the

bit

when

taking

a

write

lock

in

this

case

the

scan

resets

the

GC-dirty

bit

only

if

the

lock

has

been

released

-LSB-

-RSB-

This

is

the

alternative

we

implemented

in

Larchant

as

it

takes

advantage

of

the

entry-consistency

protocol

To

modify

an

object

x

the

entry-consistency

protocol

requires

that

the

process

where

the

mutator

is

running

holds

the

write

token

of

x.

So

when

the

mutator

performs

acquire-write

of

x

and

before

the

acquisition

completes

x

's

address

is

inserted

in

a

list

of

GC-dirty

objects

Once

a

GC-dirty

object

has

been

scanned

and

its

write

token

is

not

being

held

it

becomes

GC-clean

and

is

removed

from

the

list

of

GC-dirty

objects

Note

that

during

the

flip

the

intra-bunch

collector

scans

every

object

whose

write

token

is

being

held

Thus

after

being

scanned

such

an

object

remains

GC-dirty

CHAPTER

IMPLEMENTATION

OF

GC

IN

LARCHANT

y

j

GC

-LRB-

C

-RRB-

j

GC

-LRB-

C

-RRB-

i

GC

-LRB-

D

-RRB-

k

i

x

y

-LRB-

owner

of

z

-RRB-

invalidate

-LRB-

y

-RRB-

acquire-write

-LRB-

y

-RRB-

delete.scion

-LRB-

Cy

Dz

-RRB-

create.scion

-LRB-

Bx

Dz

-RRB-

rep-inv

-LRB-

y

-RRB-

process

j

process

i

process

k

-LRB-

owner

of

x

-RRB-

-LRB-

owner

of

y

-RRB-

Figure

Timeline

showing

a

possible

problem

with

the

asynchronous

creation

of

scions

in

presence

of

the

entry-consistency

protocol

the

stub-scion

pair

describing

Bx!Dz

is

created

too

late

-LRB-

for

safety

purposes

-RRB-

Cross-Bunch

Garbage

Collector

In

this

section

we

present

the

most

important

implementation

aspect

of

the

cross-bunch

collector

how

the

promptness

condition

is

fulfilled

-LRB-

recall

Section

-RRB-

in

presence

of

the

entry-consistency

protocol

-LRB-

i

-RRB-

all

create

messages

resulting

from

an

intra-bunch

collector

run

must

be

sent

before

any

union

or

delete

message

in

the

same

run

and

-LRB-

ii

-RRB-

create

and

delete

messages

must

be

delivered

in

causal

order

Figure

illustrates

the

prototypical

situation

of

Figure

restricted

to

the

case

of

the

entry-consistency

protocol

and

with

the

difference

that

y

i

is

not

executed

This

difference

is

not

relevant

and

makes

our

description

more

clear

In

fact

when

y

j

is

executed

Cy!Dz

is

immediately

deleted

everywhere

-LRB-

including

from

yi

-RRB-

indeed

j

holds

the

only

valid

replica

of

y

An

interesting

aspect

is

that

the

cross-bunch

collector

in

j

can

not

take

advantage

of

this

implicit

information

-LRB-

that

Cy!Dz

no

longer

exists

once

y

j

is

executed

-RRB-

because

this

could

lead

to

the

premature

deletion

of

scion

-LRB-

Cy

Dz

-RRB-

as

explained

now

Consider

Figure

object

x

is

owned

by

process

i

and

object

y

is

owned

by

process

j

After

the

execution

of

y

j

the

GCj

-LRB-

C

-RRB-

runs

enabling

the

detection

that

Cy!Dz

no

longer

exists

as

process

j

is

the

owner

of

y

this

means

that

no

other

replica

of

y

points

to

z.

Thus

apparently

scion

-LRB-

Cy

Dz

-RRB-

could

be

deleted

at

once

Consequently

send.delete.scion

-LRB-

Cy

Dz

-RRB-

j

could

be

performed

before

send.create.scion

-LRB-

Bx

Dz

-RRB-

i.

Then

z

could

be

unsafely

reclaimed

by

GCk

-LRB-

D

-RRB-

-LRB-

note

that

Bx!Dz

still

exists

-RRB-

Therefore

in

spite

of

the

knowledge

an

object

's

owner

has

concerning

the

object

's

outgoing

pointers

the

union

message

-LRB-

and

the

corresponding

union

rule

-RRB-

is

still

needed

Figure

extends

Figure

with

the

union

rule

process

j

the

owner

of

y

can

perform

send.delete.scion

-LRB-

Cy

Dz

-RRB-

j

only

after

having

made

the

union

of

all

y

's

stubs

and

the

resulting

set

is

empty

Note

that

we

add

the

propagation

of

y

from

j

to

i

otherwise

the

new

set

of

stubs

generated

by

GCi

-LRB-

C

-RRB-

would

contain

stub

-LRB-

Cy

Dz

-RRB-

and

by

the

union

rule

scion

-LRB-

Cy

Dz

-RRB-

would

not

be

deleted

CHAPTER

IMPLEMENTATION

OF

GC

IN

LARCHANT

i

GC

-LRB-

C

-RRB-

j

GC

-LRB-

D

-RRB-

k

GC

-LRB-

C

-RRB-

-LRB-

owner

of

z

-RRB-

rep-inv

-LRB-

y

-RRB-

invalidate

-LRB-

y

-RRB-

acquire-write

-LRB-

y

-RRB-

acquire-read

-LRB-

y

-RRB-

rep-acq-read

-LRB-

y

-RRB-

causal

delivery

union

-LRB-

y

-RRB-

create.scion

-LRB-

Bx

Dz

-RRB-

delete.scion

-LRB-

Cy

Dz

-RRB-

create.scion

-LRB-

Bx

Dz

-RRB-

create.scion

-LRB-

Bx

Dz

-RRB-

y

i

j

x

y

promptness

process

i

process

j

process

k

-LRB-

owner

of

x

-RRB-

-LRB-

owner

of

y

-RRB-

Figure

Figure

extended

with

the

union

message

Event

send.create.scion

-LRB-

Bx

Dz

-RRB-

i

can

be

delayed

at

most

until

send.union

-LRB-

y

i

j

-RRB-

i

is

sent

Note

that

the

union

message

carries

the

causal

dependency

-LRB-

shown

in

bold

lines

-RRB-

between

the

create

and

delete

messages

through

the

use

of

piggybacking

According

to

the

Cross-Bunch

Collection

Algorithm

described

in

Section

before

sending

the

union

message

the

cross-bunch

collector

in

process

i

issues

send.create.scion

-LRB-

Bx

Dz

-RRB-

i

This

ensures

condition

-LRB-

i

-RRB-

-LRB-

presented

at

the

beginning

of

this

section

-RRB-

Note

that

there

is

still

the

need

for

causal

delivery

As

we

do

not

have

a

communication

protocol

that

ensures

causality

we

implement

it

using

the

technique

of

piggy-backing

every

<send.create.scion>

not

yet

acknowledged

is

piggy-backed

on

both

the

union

and

on

the

<send.delete.scion>

messages

as

illustrated

in

Figure

Another

possibility

would

be

to

delay

the

sending

of

the

union

message

until

every

<send.create.scion>

has

been

acknowledged

However

this

may

delay

indefinitely

the

union

message

therefore

compromising

the

liveness

of

the

GC

algorithm

GC

and

Coherence

Granularity

In

this

section

we

make

some

considerations

concerning

how

the

GC

algorithms

are

affected

by

the

size

of

the

granule

of

coherence

Recall

Section

by

definition

an

object

is

the

granule

of

coherence

Basically

the

GC

algorithms

are

exactly

the

same

independently

of

the

number

of

application

objects

inside

a

coherence

granule

An

important

aspect

is

the

following

When

a

coherence

granule

encloses

several

application

objects

any

GC

operation

done

on

the

granule

must

be

applied

to

every

application

object

CHAPTER

IMPLEMENTATION

OF

GC

IN

LARCHANT

inside

the

granule

For

example

when

a

granule

is

propagated

by

the

GC

Clean

Propagation

Rule

-LRB-

recall

Section

-RRB-

the

granule

contents

must

be

GC-clean

this

implies

that

every

application

object

enclosed

in

the

granule

must

be

GC-clean

Another

important

aspect

concerns

the

detection

of

GC-dirty

objects

If

an

acquire-write

is

performed

on

a

granule

that

contains

several

application

objects

then

every

application

object

in

the

granule

becomes

GC-dirty

If

a

coherence

granule

contains

every

object

in

the

bunch

the

copy

intra-bunch

algorithm

can

be

optimized

since

at

any

moment

every

application

object

is

owned

by

one

process

all

move

messages

flow

from

the

owner

Thus

the

collector

running

in

the

owner

process

never

receives

the

addresses

of

objects

in

to-space

If

the

granule

of

coherence

includes

every

object

in

a

bunch

the

union

rule

is

easier

to

implement

because

every

union

message

sent

by

a

process

goes

to

the

same

target

the

owner

of

every

object

in

the

bunch

Finally

note

that

with

a

large

coherence

granule

such

as

an

entire

bunch

serious

performance

problems

may

arise

due

to

false

sharing

Programming

Restrictions

In

this

section

we

describe

the

C++

programming

restrictions

due

to

the

current

implementation

of

GC

in

Larchant

These

are

related

to

the

need

of

knowing

where

pointers

are

stored

in

the

stacks

-LRB-

which

is

part

of

the

GC-root

-RRB-

and

inside

objects

We

first

consider

the

mark-and-sweep

algorithm

The

collector

adopts

a

conservative

approach

-LRB-

recall

Section

-RRB-

concerning

pointers

in

the

stack

In

other

words

any

properly

aligned

bit

pattern

that

could

be

the

address

of

an

object

is

actually

considered

to

be

a

pointer

to

that

object

Such

objects

are

considered

reachable

therefore

marked

and

scanned

Concerning

pointers

inside

objects

there

are

two

possible

approaches

The

first

is

conservative

i.e.

just

like

with

pointers

in

the

stack

the

collector

treats

anything

that

might

be

a

pointer

as

a

pointer

This

approach

requires

no

programming

language

restriction

The

second

approach

requires

the

exact

knowledge

of

pointers

location

inside

objects

For

this

purpose

the

programmer

must

insert

a

macro

in

each

class

in

order

to

inform

Larchant

about

pointers

location

In

the

future

we

intend

to

eliminate

this

restriction

by

integrating

in

Larchant

a

tool

that

automatically

analyses

object

files

-LRB-

i.e.

o

files

-RRB-

and

discovers

where

pointers

are

-LSB-

-RSB-

Concerning

pointers

in

the

stack

the

copy

collector

adopts

a

conservative

approach

Thus

a

bit

pattern

in

the

stack

whose

value

corresponds

to

a

valid

object

address

is

considered

as

a

pointer

and

will

be

patched

This

is

the

cause

of

a

strong

C++

programming

restriction

the

application

programmer

must

ensure

that

in

the

stack

there

are

no

non-pointer

bit

patterns

whose

value

cor

CHAPTER

IMPLEMENTATION

OF

GC

IN

LARCHANT

respond

to

objects

addresses

Otherwise

the

value

will

be

corrupted

when

patched

by

the

collector

At

first

glance

it

seems

that

the

above

restriction

could

be

avoided

simply

by

not

moving

to

to-space

objects

that

are

directly

pointed

from

the

stack

However

this

is

would

be

very

costly

in

terms

of

communication

and

synchronization

for

the

following

reason

Remember

that

an

object

's

new

address

in

to-space

is

chosen

by

its

owner

-LRB-

recall

Section

-RRB-

Not

moving

an

object

is

equivalent

in

terms

of

the

GC

algorithm

to

implicitly

decide

what

is

going

to

be

its

new

address

Thus

there

would

be

a

conflict

between

an

object

's

owner

that

decides

to

move

an

object

and

some

other

process

with

a

replica

of

the

same

object

that

decides

not

to

move

it

because

locally

it

is

directly

reachable

from

the

stack

In

other

words

the

former

decides

that

the

object

is

moved

to

to-space

while

the

latter

decides

that

its

address

does

not

change

Therefore

deciding

if

an

object

should

be

moved

or

not

which

depends

if

it

is

pointed

from

the

stack

in

some

process

would

incur

into

a

high

communication

and

synchronization

cost

because

every

process

caching

a

replica

of

the

concerned

object

would

have

to

agree

on

the

same

decision

Note

that

if

there

is

run-time

type

information

enabling

the

differentiation

of

pointers

from

other

types

of

variables

such

as

in

Smalltalk

-LSB-

-RSB-

the

above

C++

programming

restriction

no

longer

holds

Also

note

that

if

the

copy

collector

is

used

on

bunches

that

are

not

being

concurrently

accessed

by

mutators

there

is

no

programming

restriction

because

there

is

no

stack

holding

pointers

to

objects

in

the

bunch

being

collected

This

is

the

case

when

the

collection

of

a

bunch

is

started

by

initiative

of

either

the

CM

or

the

OBR

-LRB-

recall

Section

-RRB-

The

copy

collector

needs

to

know

where

pointers

are

allocated

inside

objects

For

this

purpose

the

application

programmer

must

use

a

macro

just

as

with

the

mark-and-sweep

collector

-LRB-

second

approach

-RRB-

Finally

both

collectors

allow

the

existence

of

pointers

to

the

interior

of

objects

Summary

In

this

chapter

we

gave

a

general

overview

of

the

Larchant

architecture

and

described

our

prototype

This

is

as

simple

as

possible

yet

sufficiently

complete

to

test

the

GC

algorithms

and

to

execute

distributed

applications

with

persistent

objects

We

presented

the

state-machines

for

the

intra-bunch

garbage

collectors

we

implemented

mark-and-sweep

and

copy

algorithms

The

programming

of

the

collectors

resulted

from

a

simple

mapping

-LRB-

almost

automatic

-RRB-

of

the

statemachines

to

C++

The

collectors

can

be

executed

either

concurrently

with

mutators

or

in

GC-only

mode

We

also

described

how

the

promptness

condition

of

the

cross-bunch

collector

is

ensured

in

presence

of

the

entry-consistency

protocol

by

using

the

union

rule

and

the

piggy-backing

technique

-LRB-

for

causality

-RRB-

CHAPTER

IMPLEMENTATION

OF

GC

IN

LARCHANT

Finally

we

made

some

considerations

concerning

application

programming

restrictions

due

to

garbage

collection

These

restrictions

are

mostly

imposed

by

the

copy

collector

given

its

need

for

patching

pointers

Chapter

Performance

of

GC

Algorithms

In

this

chapter

we

study

the

performance

of

the

GC

algorithms

described

in

the

previous

chapters

This

study

has

two

goals

First

to

prove

the

feasibility

of

the

GC

algorithms

Second

to

prove

that

our

goals

of

good

performance

-LRB-

non-disruptiveness

in

particular

-RRB-

scalability

and

coherence

orthogonality

are

in

fact

achieved

The

performance

results

we

present

in

this

chapter

are

aimed

at

validating

the

design

decisions

regarding

the

GC

algorithms

In

particular

we

show

that

ffl

GC

is

non-disruptive

ffl

GC

is

scalable

ffl

There

is

no

coherence

interference

due

to

GC

ffl

An

asynchronous

cross-bunch

collector

has

better

performance

than

a

synchronous

one

The

most

important

performance

aspect

concerning

the

intra-bunch

collector

is

the

GC

pause

time

because

this

is

the

time

interval

during

which

a

mutator

is

halted

The

GC

pause

time

must

be

as

small

as

possible

in

order

to

avoid

disruption

of

applications

Scalability

and

coherence

orthogonality

means

that

GC

pause

time

remains

acceptable

whatever

the

bunch

size

and

the

number

of

sites

caching

a

bunch

being

collected

Concerning

the

cross-bunch

collector

the

most

important

performance

aspect

is

related

to

its

asynchrony

w.r.t.

applications

We

measure

the

performance

advantage

of

the

asynchronous

creation

of

stub-scion

pairs

and

the

overhead

of

the

GC-Clean

Propagation

Rule

when

a

coherent

replica

is

acquired

-LRB-

recall

Section

-RRB-

There

is

no

standard

-LRB-

or

widely

accepted

-RRB-

benchmark

for

GC

This

is

an

important

aspect

that

makes

GC

performance

study

a

difficult

issue

and

prevents

a

fair

comparison

with

other

algorithms

and

systems

-LSB-

-RSB-

In

addition

there

is

CHAPTER

PERFORMANCE

OF

GC

ALGORITHMS

a

lack

of

knowledge

on

the

behavior

of

real

distributed

applications

with

persistent

objects

e.g.

amount

of

garbage

rate

of

object

creation

rate

of

garbage

generation

etc.

Such

data

possibly

gathered

in

the

form

of

trace

files

would

be

extremely

valuable

for

GC

performance

evaluation

simulation

tuning

and

optimization

Given

the

lack

of

such

traces

and

of

a

widely

accepted

GC

benchmark

we

decided

to

study

the

performance

of

our

GC

algorithms

mainly

with

synthetic

micro-benchmarks

These

benchmarks

have

the

non-negligible

advantage

of

being

easy

to

implement

Note

that

these

synthetic

micro-benchmarks

are

not

assumed

to

model

the

behavior

of

real

applications

They

simply

enable

the

study

of

the

GC

algorithms

relative

to

important

parameters

such

as

bunch

size

and

percentage

of

unreachable

objects

for

example

They

are

relevant

in

the

sense

that

they

allow

us

to

understand

how

the

collector

works

and

to

simulate

worse-case

situations

-LRB-

e.g.

maximum

object

creation

rate

-RRB-

In

this

chapter

we

start

by

presenting

the

environment

in

which

the

performance

results

were

obtained

and

the

benchmarks

that

were

used

Then

we

show

the

GC

pause

time

of

the

intra-bunch

GC

algorithms

first

without

replication

relative

to

the

percentage

of

unreachable

objects

and

bunch

sizes

then

with

replication

relative

to

the

distribution

of

owned

objects

and

number

of

bunch

replicas

We

finish

this

chapter

with

some

GC

performance

results

obtained

with

two

small

macro-benchmarks

The

first

called

-LSB-

-RSB-

simulates

a

cooperative

text

editor

The

second

called

-LSB-

-RSB-

is

a

comprehensive

test

of

object-oriented

database

performance

Basics

We

ran

our

benchmarks

on

a

network

of

DEC

Alpha

workstations

Each

has

Mb

of

main

memory

Gb

of

disk

an

Kb

data

cache

a

Kb

secondary

cache

and

a

MHz

clock

They

are

connected

by

FDDI

The

micro-benchmarks

for

the

intra-bunch

collector

were

performed

with

an

application

that

keeps

creating

objects

inside

a

single

bunch

i.e.

fills

up

a

bunch

with

objects

and

inserts

some

of

them

in

a

list

-LRB-

with

a

periodic

distribution

-RRB-

objects

in

the

list

are

reachable

objects

not

in

the

list

are

garbage

Every

object

has

the

same

size

bytes

In

the

rest

of

this

chapter

we

present

the

following

performance

results

concerning

the

intra-bunch

collector

We

show

the

GC

pause

time

of

both

mark-andsweep

and

copy

intra-bunch

collectors

in

both

GC-only

and

concurrent

modes

First

we

consider

the

collection

of

a

single

non-replicated

bunch

we

vary

the

bunch

size

and

the

percentage

of

unreachable

objects

Then

we

consider

the

case

where

the

bunch

being

collected

is

replicated

on

several

sites

and

vary

the

bunch

size

the

percentage

of

unreachable

objects

and

the

percentage

of

owned

objects

on

each

site

Concerning

the

cross-bunch

collector

we

study

the

performance

advantage

of

the

asynchronous

-LRB-

versus

synchronous

-RRB-

creation

of

stub-scion

pairs

We

show

CHAPTER

PERFORMANCE

OF

GC

ALGORITHMS

how

long

it

takes

to

create

a

local

stub

a

local

scion

and

a

remote

scion

Mark-and-Sweep

without

Replication

In

this

section

we

study

the

performance

of

the

mark-and-sweep

algorithm

when

applied

to

a

single

non-replicated

bunch

We

study

the

GC

pause

time

of

the

intra-bunch

collector

in

two

functioning

modes

GC-only

and

concurrent

We

show

how

GC

pause

time

varies

with

the

bunch

size

and

the

percentage

of

unreachable

objects

GC

Pause

Time

in

GC-only

Mode

With

the

GC-only

mode

the

application

is

halted

while

the

collector

marks

and

scans

all

reachable

objects

and

sweeps

all

the

bunch

In

Figures

and

we

show

the

GC

pause

time

of

the

intra-bunch

collector

relative

to

the

percentage

of

unreachable

objects

for

several

bunch

sizes

These

graphs

confirm

the

classic

results

-LRB-

i

-RRB-

GC

pause

times

are

disruptive

-LRB-

ii

-RRB-

GC

pause

time

increases

with

bunch

size

and

-LRB-

iii

-RRB-

GC

pause

time

increases

as

the

percentage

of

unreachable

object

decreases

Disruption

is

due

to

the

GC-only

functioning

mode

of

the

intra-bunch

collector

Obviously

GC

pause

time

increases

with

the

bunch

size

because

there

are

more

objects

to

scan

and

mark

and

more

memory

to

sweep

Smaller

percentage

of

unreachable

objects

means

that

there

are

more

reachable

objects

to

mark

and

scan

thus

the

GC

pause

time

increases

To

summarize

the

mark-and-sweep

algorithm

in

GC-only

mode

is

extremely

disruptive

and

does

not

scale

to

large

bunch

sizes

and

arbitrary

number

of

reachable

objects

GC

Pause

Time

in

Concurrent

Mode

With

the

concurrent

functioning

mode

GC

adds

no

pauses

on

top

of

timeslicing

The

application

is

halted

only

while

the

collector

flips

In

Figure

we

show

the

GC

pause

time

of

the

intra-bunch

collector

relative

to

the

percentage

of

unreachable

objects

for

several

bunch

sizes

Note

that

with

the

concurrent

functioning

mode

GC

pauses

are

caused

only

by

flipping

and

hence

are

very

short

From

Figure

it

is

clear

that

the

mark-and-sweep

intra-bunch

collector

is

much

less

disruptive

when

running

in

concurrent

mode

than

in

GC-only

mode

in

fact

GC

pause

time

is

always

under

milliseconds

that

in

the

case

of

a

mark-and-sweep

collector

the

flip

is

the

instant

when

the

mark

phase

is

finished

and

the

sweep

can

start

CHAPTER

PERFORMANCE

OF

GC

ALGORITHMS

GC

pause

time

-LRB-

milliseconds

-RRB-

unreachable

objects

-LRB-

-RRB-

mark-and-sweep

-LRB-

GC-only

mode

-RRB-

Figure

GC

pause

time

of

the

intra-bunch

collector

relative

to

the

percentage

of

unreachable

objects

for

several

bunch

sizes

GC

pause

time

-LRB-

milliseconds

-RRB-

unreachable

objects

-LRB-

-RRB-

mark-and-sweep

-LRB-

GC-only

mode

-RRB-

Figure

GC

pause

time

of

the

intra-bunch

collector

relative

to

the

percentage

of

unreachable

objects

for

several

bunch

sizes

CHAPTER

PERFORMANCE

OF

GC

ALGORITHMS

GC

pause

time

-LRB-

miliseconds

-RRB-

unreachable

objects

-LRB-

-RRB-

mark-and-sweep

-LRB-

concurrent

mode

-RRB-

Figure

GC

pause

time

of

the

intra-bunch

collector

relative

to

the

percentage

of

unreachable

objects

for

several

bunch

sizes

GC

pause

time

is

independent

of

bunch

size

This

is

due

to

the

fact

that

the

majority

of

the

GC

work

is

done

concurrently

to

the

mutator

GC

pause

time

is

not

independent

of

the

percentage

of

unreachable

objects

This

behavior

is

explained

as

follows

When

flipping

the

collector

sweeps

a

certain

amount

of

memory

until

it

reclaims

objects

-LRB-

a

configurable

parameter

-RRB-

This

threshold

ensures

that

once

the

mutator

is

resumed

object

allocation

will

not

catch

up

the

GC

sweeping

i.e.

the

mutator

does

not

yield

the

processor

to

the

collector

to

allow

it

to

reclaim

unreachable

objects

and

hence

obtain

free

space

for

new

objects

Thus

for

smaller

percentages

of

unreachable

objects

the

memory

that

must

be

swept

is

larger

consequently

the

GC

pause

time

increases

with

decreasing

percentage

of

unreachable

objects

Note

that

GC

pause

time

can

be

made

smaller

if

the

number

of

objects

to

be

reclaimed

while

flipping

decreases

-LRB-

i.e.

less

than

-RRB-

however

in

this

case

if

the

mutator

allocates

objects

at

the

maximum

rate

-LRB-

as

is

the

case

of

our

benchmark

-RRB-

it

will

have

to

yield

its

processor

to

the

collector

because

the

sweeping

phase

is

not

fast

enough

Another

possibility

instead

of

the

threshold

mentioned

above

would

be

to

sweep

the

memory

during

the

flipping

for

a

maximum

pre-defined

time

interval

-LRB-

sufficiently

small

to

be

non-disruptive

-RRB-

This

solution

would

ensure

that

GC

pause

times

would

always

be

non-disruptive

The

price

to

pay

would

be

-LRB-

possibly

-RRB-

the

mutator

yielding

the

processor

to

the

collector

to

allow

the

latter

to

sweep

To

summarize

the

mark-and-sweep

algorithm

in

concurrent

functioning

mode

is

not

disruptive

and

it

scales

well

with

bunch

size

CHAPTER

PERFORMANCE

OF

GC

ALGORITHMS

Copy

without

Replication

In

this

section

we

study

the

performance

of

the

copy

algorithm

when

applied

to

a

single

non-replicated

bunch

We

study

the

GC

pause

time

of

the

intra-bunch

collector

in

two

functioning

modes

GC-only

and

concurrent

We

show

how

the

GC

pause

time

varies

with

bunch

size

and

with

the

percentage

of

unreachable

objects

GC

Pause

Time

in

GC-only

Mode

With

the

GC-only

mode

the

application

is

halted

while

the

collector

moves

scans

and

patches

all

reachable

objects

In

Figures

and

we

show

the

GC

pause

time

of

the

intra-bunch

collector

relative

to

the

percentage

of

unreachable

objects

for

several

bunch

sizes

These

graphs

confirm

the

classic

results

-LRB-

i

-RRB-

GC

pause

times

are

disruptive

-LRB-

ii

-RRB-

GC

pause

time

increases

with

bunch

size

and

-LRB-

iii

-RRB-

GC

pause

time

increases

as

the

percentage

of

unreachable

objects

decreases

Disruption

is

due

to

the

GC-only

functioning

mode

of

the

intra-bunch

collector

Obviously

GC

pause

time

increases

with

bunch

size

because

there

are

more

reachable

objects

to

move

and

scan

A

smaller

percentage

of

unreachable

objects

means

that

there

are

more

reachable

objects

to

move

and

scan

hence

GC

pause

time

increases

The

copy

collector

provides

better

-LRB-

i.e.

smaller

-RRB-

GC

pause

times

than

the

mark-and-sweep

collector

-LRB-

see

Figures

and

-RRB-

This

difference

becomes

more

significant

as

bunch

size

and

percentage

of

unreachable

objects

increase

This

is

due

to

the

fact

that

the

copy

collector

only

sees

the

reachable

objects

while

the

mark-and-sweep

algorithm

sees

them

all

To

summarize

although

its

performance

is

better

than

mark-and-sweep

the

copy

algorithm

in

GC-only

mode

remains

disruptive

and

it

does

not

scale

well

to

large

bunch

sizes

and

arbitrary

number

of

reachable

objects

GC

Pause

Time

in

Concurrent

Mode

In

Figure

we

show

the

GC

pause

time

of

the

intra-bunch

collector

relative

to

the

percentage

of

unreachable

objects

for

several

bunch

sizes

Note

that

in

the

concurrent

mode

the

GC

pause

time

corresponds

to

the

flip

time

which

is

the

only

time

interval

during

which

the

mutator

is

halted

From

Figure

it

is

clear

that

the

copy

intra-bunch

collector

is

not

disruptive

when

running

in

concurrent

mode

GC

pause

time

is

independent

of

bunch

size

and

of

percentage

of

unreachable

objects

This

is

due

to

the

fact

that

the

majority

of

the

GC

work

is

done

concurrently

to

the

mutator

When

flipping

the

collector

simply

has

to

re-scan

and

move

again

those

objects

that

were

modified

by

the

mutator

after

having

been

scanned

and

moved

-LRB-

recall

Section

-RRB-

CHAPTER

PERFORMANCE

OF

GC

ALGORITHMS

GC

pause

time

-LRB-

milliseconds

-RRB-

unreachable

objects

-LRB-

-RRB-

copy

-LRB-

GC-only

mode

-RRB-

Figure

GC

pause

time

of

the

intra-bunch

collector

relative

to

the

percentage

of

unreachable

objects

for

several

bunch

sizes

GC

pause

time

-LRB-

milliseconds

-RRB-

unreachable

objects

-LRB-

-RRB-

copy

-LRB-

GC-only

mode

-RRB-

Figure

GC

pause

time

of

the

intra-bunch

collector

relative

to

the

percentage

of

unreachable

objects

for

several

bunch

sizes

CHAPTER

PERFORMANCE

OF

GC

ALGORITHMS

GC

pause

time

-LRB-

milliseconds

-RRB-

unreachable

objects

-LRB-

-RRB-

copy

-LRB-

concurrent

mode

-RRB-

Figure

GC

pause

time

of

the

intra-bunch

collector

relative

to

the

percentage

of

unreachable

objects

for

several

bunch

sizes

To

summarize

the

copy

algorithm

in

concurrent

mode

is

not

disruptive

and

it

scales

well

to

large

bunch

sizes

and

arbitrary

number

of

reachable

objects

The

GC

pause

time

is

smaller

with

the

copy

collector

than

with

the

mark-and-sweep

GC

Pause

Time

with

Replication

In

this

section

we

study

the

performance

of

both

mark-and-sweep

and

copy

algorithms

when

applied

to

a

single

replicated

bunch

We

study

the

GC

pause

time

of

the

intra-bunch

collector

in

two

functioning

modes

GC-only

and

concurrent

We

show

how

the

GC

pause

time

varies

with

the

bunch

size

number

of

bunch

replicas

distribution

of

objects

ownership

among

the

sites

and

percentage

of

unreachable

objects

We

ran

the

benchmark

used

in

the

non-replicated

case

-LRB-

recall

Section

-RRB-

on

several

sites

all

sharing

the

same

bunch

We

varied

the

number

of

sites

from

to

-LRB-

the

maximum

available

in

our

laboratory

-RRB-

and

ran

the

intra-bunch

collector

in

both

GC-only

and

concurrent

modes

Mark-and-Sweep

in

GC-only

and

Concurrent

Modes

The

values

of

GC

pause

time

that

were

obtained

for

each

site

in

both

functioning

modes

are

the

same

as

in

the

non-replicated

case

-LRB-

see

Figures

and

for

GC-only

mode

and

Figure

for

concurrent

mode

-RRB-

These

results

confirm

our

expectations

that

our

GC

algorithm

is

orthogonal

to

coherence

In

fact

given

that

the

collector

does

not

need

coherent

objects

there

is

no

communication

between

sites

on

behalf

of

GC

Each

site

collects

independently

from

any

other

site

CHAPTER

PERFORMANCE

OF

GC

ALGORITHMS

time

to

create

a

local

stub

milliseconds

time

to

create

a

local

scion

milliseconds

time

to

create

a

remote

scion

milliseconds

Table

Time

it

takes

to

create

-LRB-

synchronously

-RRB-

stubs

and

scions

Copy

in

GC-only

and

Concurrent

Modes

The

values

of

GC

pause

time

that

were

obtained

for

each

site

in

both

functioning

modes

are

the

same

as

in

the

non-replicated

case

-LRB-

see

Figures

and

for

GC-only

mode

and

Figure

for

concurrent

mode

-RRB-

These

results

confirm

our

expectations

that

our

GC

algorithm

is

orthogonal

to

coherence

In

fact

given

that

the

collector

does

not

need

coherent

objects

the

only

messages

between

sites

on

behalf

of

GC

are

exchanged

in

the

background

-LRB-

these

correspond

to

move

operations

recall

Section

-RRB-

Cross-Bunch

GC

In

this

section

we

study

the

most

important

performance

aspect

of

the

crossbunch

collector

i.e.

we

show

the

time

it

takes

to

create

a

stub

and

to

create

a

scion

We

quantify

the

advantage

of

asynchronous

versus

synchronous

execution

of

the

cross-bunch

collector

We

also

measure

the

overhead

of

the

GC-Clean

Propagation

Rule

when

a

coherent

replica

is

acquired

If

the

cross-bunch

collector

runs

synchronously

every

assignment

operation

is

instrumented

in

order

to

detect

if

a

new

cross-bunch

pointer

has

appeared

In

that

case

the

corresponding

local

stub

and

-LRB-

possibly

remote

-RRB-

scion

are

synchronously

created

Creating

a

remote

scion

implies

sending

a

message

to

the

remote

site

and

to

wait

an

acknowledgement

During

all

this

time

the

mutator

is

halted

In

Table

we

show

the

time

it

takes

to

create

synchronously

a

stub

a

local

scion

and

a

remote

scion

The

advantage

of

making

the

cross-bunch

collector

asynchronous

to

mutators

comes

from

the

substantial

performance

gains

since

the

mutator

is

not

halted

it

might

avoid

work

and

enables

message

batching

To

quantify

this

advantage

we

ran

the

following

benchmark

with

an

object

containing

pointers

we

performed

an

assignment

operation

for

each

one

in

order

to

create

crossbunch

pointers

-LRB-

all

pointing

to

the

same

target

bunch

-RRB-

We

measured

the

time

the

assignment

operations

take

and

the

corresponding

creation

of

stub-scion

pairs

in

both

synchronous

and

asynchronous

modes

We

show

these

values

in

Table

In

the

synchronous

case

the

cross-bunch

pointer

is

detected

when

the

corresponding

assignment

operation

is

executed

and

the

stub-scion

pair

is

immediately

created

With

the

asynchronous

mode

assignment

operations

are

not

instrumented

the

object

is

scanned

while

the

intra-bunch

collector

executes

CHAPTER

PERFORMANCE

OF

GC

ALGORITHMS

synchronous

milliseconds

asynchronous

milliseconds

Table

Time

it

takes

to

create

cross-bunch

pointers

and

the

corresponding

stub-scion

pairs

the

stubs

are

created

locally

and

the

creation

of

remote

scions

is

batched

in

a

single

message

-LRB-

There

is

a

single

message

to

acknowledge

the

creation

of

all

scions

-RRB-

This

batching

is

the

reason

for

the

difference

of

performance

From

these

values

the

advantage

of

running

the

cross-bunch

collector

asynchronously

to

applications

is

clear

If

the

creation

of

a

stub-scion

pair

was

done

synchronously

with

the

appearance

of

the

corresponding

cross-bunch

pointer

-LRB-

due

to

an

assignment

operation

-RRB-

the

performance

of

the

mutator

would

be

penalized

accordingly

The

GC-Clean

Propagation

Rule

described

in

Section

says

that

an

object

y

can

be

propagated

from

its

owner

process

i

only

if

yi

is

GC-clean

and

the

messages

to

create

the

scions

corresponding

to

yi

's

outgoing

crossbunch

pointers

have

been

sent

Thus

this

rule

introduces

an

overhead

to

the

acquisition

of

a

coherent

replica

We

measured

this

overhead

by

acquiring

a

coherent

replica

of

an

object

-LRB-

with

outgoing

cross-bunch

pointers

inside

-RRB-

with

and

without

performing

the

GC

Clean

Propagation

Rule

The

results

that

we

obtained

are

the

following

acquiring

a

coherent

replica

of

the

object

without

executing

the

above

rule

takes

milliseconds

when

executing

the

above

rule

the

acquisition

of

a

coherent

replica

takes

milliseconds

Thus

there

is

an

overhead

of

microseconds

-LRB-

i.e

-RRB-

due

to

the

GC-Clean

Propagation

Rule

these

are

spent

on

scanning

the

object

and

creating

the

corresponding

stubs

-LRB-

microseconds

-RRB-

and

sending

a

message

to

create

the

corresponding

scions

-LRB-

it

's

a

single

message

as

all

the

cross-bunch

pointers

point

to

the

same

target

bunch

microseconds

for

building

the

message

buffer

and

microseconds

to

send

it

-RRB-

Macro-Benchmarks

In

addition

to

the

synthetic

micro-benchmarks

we

ran

two

macro-benchmarks

The

first

one

called

simulates

a

cooperative

text

editor

A

document

is

a

hierarchical

structure

containing

objects

for

sections

subsections

paragraphs

and

lines

The

simulator

creates

objects

assigns

pointers

reads

and

writes

lines

of

text

all

randomly

We

measured

the

GC

pause

times

of

the

intra-bunch

mark-and-sweep

collector

with

running

concurrently

on

one

two

and

three

sites

with

a

Mbytes

bunch

and

various

amounts

of

garbage

GC

pause

times

are

never

greater

than

milliseconds

The

same

benchmark

for

the

intra-bunch

copy

collector

presents

GC

pause

times

which

are

never

greater

than

milliseconds

CHAPTER

PERFORMANCE

OF

GC

ALGORITHMS

In

addition

the

total

GC

overhead

for

both

collectors

lays

between

and

with

a

mean

value

of

This

value

was

obtained

by

running

first

with

GC

and

then

without

it

and

comparing

the

total

execution

time

The

values

obtained

with

several

replicas

do

not

differ

significantly

from

the

non-replicated

case

We

have

also

ported

the

well-known

benchmark

widely

used

by

the

objectoriented

database

community

Our

main

goal

was

to

test

the

reliability

of

the

Larchant

prototype

The

standard

benchmark

runs

on

a

single

site

The

mean

GC

pause

time

with

the

concurrent

mark-and-sweep

intra-bunch

collector

is

second

This

value

is

bigger

than

the

GC

pause

times

obtained

with

the

synthetic

benchmark

for

the

following

reason

with

of

objects

are

reachable

thus

while

flipping

all

the

memory

must

be

swept

If

the

number

of

objects

to

be

reclaimed

while

flipping

is

made

smaller

-LRB-

currently

the

value

is

-RRB-

than

the

GC

pause

times

become

similar

to

those

obtained

with

the

synthetic

benchmark

For

the

the

GC

pause

times

with

the

concurrent

copy

intra-bunch

collector

are

all

less

than

milliseconds

Summary

The

performance

results

presented

in

this

chapter

show

that

our

implementation

of

the

GC

algorithms

is

non-disruptive

is

orthogonal

to

coherence

and

is

scalable

The

concurrent

functioning

mode

ensures

that

applications

are

not

disrupted

by

long

pauses

which

would

be

annoying

to

the

user

This

is

due

to

the

fact

that

the

majority

of

the

GC

work

is

done

concurrently

to

the

mutator

thus

GC

pause

time

is

independent

of

the

bunch

size

and

of

the

percentage

of

unreachable

objects

This

aspect

is

clearly

observed

by

comparing

the

GC

pause

times

of

the

intra-bunch

collectors

for

both

mark-and-sweep

and

copy

algorithms

in

GC-only

and

concurrent

modes

We

also

showed

that

GC

pause

times

are

independent

of

the

number

of

replicas

This

is

due

to

the

fact

that

the

collector

does

not

need

coherent

data

to

make

progress

The

copy

collector

in

concurrent

mode

scales

to

large

bunch

sizes

Both

the

copy

and

mark-and-sweep

collectors

scale

to

large

number

of

replicas

This

scalability

comes

from

the

fact

that

a

bunch

is

collected

independently

from

the

rest

of

the

memory

and

from

coherence

orthogonality

In

general

the

copy

collector

provides

smaller

GC

pause

times

than

the

markand-sweep

collector

However

the

copy

algorithm

requires

precise

knowledge

of

pointer

locations

which

might

be

impracticable

with

some

programming

languages

We

did

not

show

any

performance

results

concerning

group

collection

because

we

have

strong

reasons

indicating

that

they

are

similar

to

the

collection

of

a

bunch

whose

size

is

equal

to

the

sum

of

the

sizes

of

the

bunches

in

the

group

CHAPTER

PERFORMANCE

OF

GC

ALGORITHMS

We

showed

how

much

we

gain

by

executing

the

cross-bunch

collector

asynchronously

to

the

mutator

instead

of

doing

it

synchronously

The

mutator

freely

assigns

pointers

without

being

slowed

down

by

the

creation

of

stub-scion

pairs

We

tested

and

measured

the

intra-bunch

collector

with

two

macro-benchmarks

a

simulator

of

a

cooperative

text

editor

-LRB-

-RRB-

and

a

standard

comprehensive

test

of

object-oriented

database

performance

-LRB-

-RRB-

For

both

benchmarks

GC

pause

time

is

non-disruptive

However

poses

some

problems

to

the

markand-sweep

algorithm

because

almost

every

object

is

reachable

Thus

when

flipping

the

intra-bunch

collector

can

not

reclaim

the

objects

required

It

is

important

to

stress

the

lack

of

widely

accepted

benchmarks

for

measuring

GC

performance

In

particular

our

version

of

the

benchmark

is

not

indicated

for

measuring

GC

because

every

object

used

in

the

benchmark

is

created

when

the

application

starts

running

and

there

are

almost

no

unreachable

objects

In

other

words

the

pointer

graph

remains

mostly

unchanged

We

believe

that

this

behavior

is

typical

of

databases

which

are

optimized

towards

searching

for

relevant

data

-LRB-

through

indexing

for

example

-RRB-

with

modest

updates

and

minimal

cooperative

read/write

interaction

This

is

not

the

kind

of

applications

we

are

interested

in

-LRB-

recall

Sections

and

-RRB-

However

note

that

even

for

such

database

systems

our

fundamental

GC

design

aspects

are

still

valid

independent

collection

of

bunches

and

orthogonality

to

coherence

Finally

note

that

the

source

code

of

the

current

prototype

is

far

from

being

optimized

We

can

achieve

better

results

with

some

more

effort

For

example

we

could

scan

the

stack

concurrently

to

the

mutator

instead

of

doing

it

only

while

flipping

Chapter

Conclusion

The

basic

motivation

for

the

work

described

in

this

document

is

to

make

data

sharing

simple

and

efficient

-LRB-

recall

Chapter

-RRB-

This

is

an

important

issue

because

an

essential

part

of

tomorrow

's

computing

will

be

workers

and

organizations

carrying

out

cooperative

tasks

interacting

via

shared

data

Larchant

is

a

Cached

Distributed

Shared

Store

based

on

the

model

of

a

Shared

Address

Space

with

Persistence

By

Reachability

To

provide

the

illusion

of

a

shared

address

space

across

the

network

although

site

memories

are

disjoint

Larchant

implements

a

distributed

shared

memory

mechanism

Data

is

replicated

in

multiple

sites

for

performance

and

availability

Reachability

is

accessed

by

tracing

the

pointer

graph

starting

from

the

persistent

root

and

reclaiming

unreachable

data

This

is

the

task

of

Garbage

Gollection

-LRB-

GC

-RRB-

GC

automatically

ensures

referential

integrity

therefore

improving

program

reliability

and

programming

productivity

In

addition

it

may

compact

memory

thus

reducing

fragmentation

and

improving

locality

If

in

a

centralized

short-lived

program

GC

might

be

considered

as

an

expensive

luxury

in

contrast

in

a

cached

distributed

shared

store

such

as

Larchant

the

nightmare

of

manual

memory-management

increases

as

the

number

of

objects

pointers

and

users

scales

up

Then

GC

becomes

indispensable

In

the

rest

of

this

chapter

we

summarize

the

most

important

issues

concerning

distributed

GC

in

a

persistent

shared

store

for

which

we

proposed

innovative

solutions

We

analyze

the

achievements

in

the

light

of

the

GC

algorithm

characteristics

and

performance

results

We

conclude

this

chapter

with

some

perspectives

for

future

research

GC

Issues

and

Solutions

We

addressed

the

following

fundamental

issues

faced

by

GC

in

a

large-scale

distributed

shared

store

correctness

completeness

performance

scalability

and

interference

with

applications

coherence

needs

A

common

thread

to

our

solutions

described

in

detail

in

Chapter

is

that

GC

is

opportunistic

i.e.

it

CHAPTER

CONCLUSION

does

not

cause

events

that

mutators

would

eventually

cause

the

collector

waits

for

them

to

happen

and

then

takes

advantage

of

them

Our

GC

algorithm

combines

partitioned

tracing

-LRB-

within

one

or

more

bunches

-RRB-

with

reference-listing

-LRB-

across

bunch

group

boundaries

-RRB-

Each

bunch

replica

is

collected

independently

of

other

bunches

and

of

other

replicas

of

the

same

bunch

A

bunch

replica

is

collected

in

the

process

where

it

is

currently

cached

with

a

tracing

algorithm

either

mark-and-sweep

or

copy

Concerning

performance

applications

overhead

due

to

GC

must

be

minimized

In

particular

interactive

applications

should

not

be

disrupted

by

the

collector

Our

solution

is

to

perform

GC

concurrently

and

asynchronously

w.r.t.

mutators

The

intra-bunch

collector

runs

concurrently

to

mutators

and

performs

most

of

its

work

without

adding

pauses

on

top

of

time-slicing

The

mutator

is

halted

only

when

the

collector

flips

thus

GC

pauses

are

non-disruptive

The

cross-bunch

collector

runs

asynchronously

w.r.t.

mutators

by

safely

delaying

the

creation

of

stub-scion

pairs

describing

cross-bunch

pointers

The

advantage

of

making

the

cross-bunch

collector

asynchronous

to

mutators

comes

from

the

substantial

performance

gains

since

the

mutator

is

not

halted

it

might

avoid

work

and

enables

message

batching

Scalability

means

that

the

GC

algorithm

must

be

capable

of

collecting

a

huge

pointer

graph

without

incurring

into

high

performance

costs

due

to

computation

I/O

and

distributed

synchronization

In

particular

the

collection

of

the

whole

graph

in

a

single

phase

would

clearly

not

be

scalable

In

addition

we

claimed

that

perfect

completeness

is

not

feasible

in

a

large-scale

system

Thus

we

proposed

an

approximate

solution

that

is

not

provably

complete

but

which

we

believe

adequate

for

most

real-life

situations

because

it

takes

advantage

of

locality

As

already

mentioned

each

bunch

is

collected

independently

from

the

rest

of

the

memory

This

solution

is

not

complete

since

it

does

not

reclaim

crossbunch

cycles

of

garbage

Apparently

the

only

way

of

reclaiming

such

cycles

is

to

perform

a

global

trace

which

is

not

scalable

and

would

generate

a

huge

amount

of

extra

I/O

Our

solution

to

reclaim

cross-bunch

cycles

of

garbage

without

incurring

into

extra

I/O

consists

of

extending

the

tracing

performed

by

the

intra-bunch

collector

to

include

several

bunches

at

once

Bunches

are

grouped

and

collected

as

if

they

were

a

single

bunch

Thus

groups

of

bunches

change

dynamically

and

seamlessly

and

independently

in

each

process

in

order

to

reclaim

cycles

of

unreachable

objects

Groups

are

formed

according

to

a

locality-based

heuristic

a

group

contains

all

the

bunches

currently

cached

in

the

process

This

heuristic

avoids

extra

I/O

costs

However

it

does

not

enable

the

reclamation

of

cross-bunch

cycles

enclosed

in

bunches

that

reside

partially

on

disk

Also

cycles

of

garbage

where

some

bunches

but

not

all

are

replicated

in

multiple

processes

are

not

reclaimed

This

garbage

might

be

reclaimed

with

a

more

aggressive

grouping

heuristic

However

the

extra

I/O

cost

involved

needs

to

be

balanced

against

the

expected

gain

To

prevent

coherence

interference

the

GC

algorithm

must

not

compete

with

CHAPTER

CONCLUSION

applications

for

holding

coherent

data

replicas

Such

competition

would

interfere

with

applications

coherence

needs

For

example

if

the

collector

on

some

site

were

to

require

coherent

access

to

some

object

that

would

prevent

an

application

running

at

another

site

from

writing

into

another

replica

-LRB-

of

that

object

-RRB-

at

the

same

time

To

avoid

the

coherence

interference

problem

the

collection

of

a

bunch

replica

makes

progress

even

if

the

objects

it

contains

are

not

known

to

be

coherent

in

the

process

where

the

intra-bunch

collection

is

taking

place

In

other

words

our

GC

algorithm

is

orthogonal

to

coherence

Thus

the

garbage

collector

can

work

with

objects

that

are

incoherent

for

applications

purposes

while

remaining

safe

and

live

This

minimizes

the

assumptions

underlying

GC

and

ensures

wide

applicability

of

the

algorithms

We

designed

our

GC

algorithm

for

the

Larchant

model

-LRB-

presented

in

Chapter

-RRB-

This

model

is

that

of

a

cached

distributed

shared

store

with

persistence

by

reachability

The

model

is

general

and

we

made

minimal

assumptions

Therefore

the

GC

algorithm

is

widely

applicable

In

Chapter

we

described

the

GC

algorithm

first

intuitively

then

rigorously

and

proved

formally

its

safety

and

liveness

A

fundamental

aspect

of

safety

is

the

union

rule

a

target

object

can

be

reclaimed

only

if

it

is

not

reachable

from

the

union

of

all

replicas

of

its

source

objects

-LRB-

independently

of

their

location

whether

in

the

same

or

in

a

different

bunch

-RRB-

We

showed

how

the

creation

of

scions

can

be

safely

delayed

by

respecting

the

promptness

condition

deliver

a

message

to

create

a

scion

before

the

corresponding

object

could

otherwise

be

reclaimed

For

safety

causal

delivery

is

needed

We

implemented

a

prototype

of

Larchant

and

showed

how

the

intra-bunch

collector

is

specified

with

a

state-machine

formalism

-LRB-

both

mark-and-sweep

and

copy

algorithms

-RRB-

Then

the

programming

of

the

collectors

resulted

from

a

simple

-LRB-

almost

automatic

-RRB-

mapping

of

the

state-machines

to

C++

We

measured

the

performance

of

the

most

important

aspects

of

the

GC

algorithms

mark-and-sweep

and

copy

intra-bunch

collectors

-LRB-

in

both

GC-only

and

concurrent

functioning

modes

-RRB-

and

of

the

cross-bunch

collector

The

results

confirm

our

expectations

in

terms

of

performance

scalability

and

coherence

orthogonality

Achievements

Our

goals

for

the

GC

algorithm

in

Larchant

were

-LRB-

recall

Chapter

-RRB-

correctness

scalability

orthogonality

to

coherence

and

good

performance

The

GC

algorithm

fulfills

these

goals

It

is

correct

we

proved

its

safety

and

liveness

formally

It

is

scalable

orthogonal

to

coherence

and

has

good

performance

from

the

performance

results

obtained

with

our

prototype

-LRB-

recall

Chapter

-RRB-

we

clearly

see

that

GC

pause

times

are

independent

from

the

size

of

a

bunch

the

percentage

of

reachable

objects

the

number

of

replicas

and

ownership

distribution

CHAPTER

CONCLUSION

Future

Work

An

obvious

task

for

future

work

is

to

improve

the

current

prototype

This

implies

an

engineering

effort

in

order

to

increase

the

robustness

and

the

usability

of

Larchant

The

development

of

other

bunch

managers

possibly

with

different

coherence

protocols

and

GC

algorithms

is

another

important

aspect

In

the

future

we

intend

to

study

how

the

GC

algorithms

interact

with

a

checkpoint

mechanism

that

will

be

integrated

in

Larchant

This

research

is

part

of

the

more

general

issue

of

fault-tolerance

The

GC

algorithm

must

be

capable

of

dealing

with

site

crashes

and

unreliable

communication

i.e.

GC

must

remain

safe

and

live

under

adverse

conditions

In

particular

we

intend

to

make

the

GC

algorithm

safe

even

if

the

communication

medium

does

not

ensure

causal

communication

by

using

the

SSP

Chain

mechanism

-LSB-

-RSB-

This

issue

of

faulttolerance

is

fundamental

for

large

geographically

sparse

networks

or

for

mobile

computing

Another

direction

for

future

research

is

related

to

the

benchmarking

of

GC

algorithms

As

stated

in

Chapter

there

is

no

standard

-LRB-

or

widely

accepted

-RRB-

benchmark

for

GC

In

addition

there

is

insufficient

knowledge

concerning

the

behavior

of

real

distributed

applications

with

persistent

objects

-LRB-

e.g.

amount

of

garbage

rate

of

object

creation

rate

of

garbage

generation

etc

-RRB-

These

two

aspects

prevent

the

fair

comparison

of

GC

algorithms

and

makes

difficult

their

optimization

and

tuning

Thus

a

very

interesting

direction

for

future

research

consists

of

tracing

real

distributed

applications

in

order

to

model

their

memory

behavior

Once

such

data

is

available

different

GC

algorithms

can

be

designed

accordingly

compared

fairly

and

optimized

The

garbage

collection

of

distributed

-LRB-

cross-bunch

-RRB-

cycles

of

garbage

would

benefit

enormously

from

the

study

of

applications

memory

behavior

mentioned

above

In

fact

it

would

be

extremely

helpful

to

characterize

such

cycles

in

real

applications

size

geographical

distribution

creation

rate

typical

mutator

operations

that

generate

them

etc.

The

difficulty

with

such

tracing

is

that

real

distribution

applications

with

persistent

objects

are

not

easily

available

The

completeness

of

our

GC

algorithms

is

a

very

important

issue

for

future

research

We

believe

that

the

locality-based

heuristic

for

grouping

bunches

is

well

suited

for

the

majority

of

applications

However

a

practical

study

is

needed

Then

if

experimental

results

mandate

it

more

complex

heuristics

will

be

developed

and

their

cost

carefully

evaluated

either

through

simulation

or

integration

within

the

current

prototype

Finally

note

that

we

proved

the

correctness

of

our

GC

algorithm

but

we

did

not

prove

formally

the

correctness

of

its

implementation

This

is

also

an

interesting

aspect

for

future

work

from

the

specification

of

the

GC

algorithm

-LRB-

proved

to

be

correct

-RRB-

generate

automatically

the

corresponding

source

code

Bibliography

-LSB-

-RSB-

M.

Accetta

R.

Baron

W.

Bolosky

D.

Golub

R.

Rashid

A.

Tevanian

and

M.

Young

Mach

a

new

kernel

foundation

for

unix

development

In

Proceedings

of

Summer

Usenix

July

-LSB-

-RSB-

Guy

Almes

Andrew

Black

Edward

Lazowska

and

Jerry

Noe

The

Eden

system

a

technical

review

IEEE

Transactions

on

Software

Engineering

-LRB-

-RRB-

January

-LSB-

-RSB-

L.

Amsaleg

M.

Franklin

and

O.

Gruber

Efficient

Incremental

Garbage

Collection

for

Client-Server

Object

Database

Systems

In

Proc

of

the

VLDB

Int

Conf.

Z?urich

Switzerland

September

-LSB-

-RSB-

Laurent

Amsaleg

Paulo

Ferreira

Michael

Franklin

and

Marc

Shapiro

Evaluating

garbage

collectors

for

large

persistent

stores

In

Conf

on

Object-Oriented

Programming

Systems

Languages

and

Applications

Austin

TX

-LRB-

USA

-RRB-

October

-LSB-

-RSB-

Andrew

W.

Appel

Runtime

tags

are

n't

necessary

Lisp

and

Symbolic

Computation

-LCB-

-LSB-

-RSB-

Andrew

W.

Appel

Simple

generational

garbage

collection

and

fast

allocation

Software

Practice

and

Experience

-LRB-

-RRB-

-LCB-

February

-LSB-

-RSB-

Andrew

W.

Appel

John

R.

Ellis

and

Kai

Li

Real-time

concurrent

garbage

collection

on

stock

multiprocessors

In

Proceedings

of

the

SIGPLAN

Conference

on

Programming

Language

Design

and

Implementation

pages

-LCB-

Atlanta

Georgia

June

ACM

Press

-LSB-

-RSB-

M.

P.

Atkinson

P.

J.

Bailey

K.

J.

Chisholm

P.

W.

Cockshott

and

R.

Morrison

An

approach

to

persistent

programming

The

Computer

Journal

-LRB-

-RRB-

-LCB-

-LSB-

-RSB-

Henry

G.

Baker

Jr.

List

processing

in

real

time

on

a

serial

computer

Communications

of

the

ACM

-LRB-

-RRB-

-LCB-

April

-LSB-

-RSB-

Stoney

Ballard

and

Stephen

Shirron

The

design

and

implementation

of

vax/smalltalk

In

Bits

of

History

Words

of

Advice

pages

-LCB-

Addison-Wesley

MA

-LSB-

-RSB-

Joel

F.

Bartlett

Compacting

garbage

collection

with

ambiguous

roots

Technical

Report

Digital

Equipment

Corporation

Western

Research

Laboratory

Palo

Alto

California

February

-LSB-

-RSB-

John

Bennett

John

B.

Carter

and

Willy

Zwaenepoel

Adaptive

software

cache

management

for

distributed

shared

memory

architectures

In

Proc

of

the

Annual

International

Symp

on

Comp

Architecture

pages

-LCB-

Seattle

WA

-LRB-

USA

-RRB-

May

BIBLIOGRAPHY

-LSB-

-RSB-

Philip

Bernstein

Vassos

Hadzilacos

and

Nathan

Goodman

editors

Concurrency

Control

and

Recovery

in

Database

Systems

Addison-Wesley

Massachusetts

-LSB-

-RSB-

Brian

N.

Bershad

and

Matthew

J.

Zekauskas

The

Midway

distributed

shared

memory

system

In

Proceedings

of

the

COMPCON

Conference

pages

-LCB-

February

-LSB-

-RSB-

D.

I.

Bevan

Distributed

garbage

collection

using

reference

counting

In

Parallel

Arch

and

Lang

Europe

pages

-LCB-

Eindhoven

The

Netherlands

June

Spring-Verlag

Lecture

Notes

in

Computer

Science

-LSB-

-RSB-

Kenneth

Birman

Andre

Schiper

and

Pat

Stephenson

Lightweight

causal

and

atomic

group

multicast

ACM

Transactions

on

Computer

Systems

-LRB-

-RRB-

-LCB-

August

-LSB-

-RSB-

A.

D.

Birrell

and

B.

J.

Nelson

Implementing

Remote

Procedure

Calls

ACM

Transactions

on

Programming

Languages

and

Systems

-LRB-

-RRB-

February

-LSB-

-RSB-

Andrew

Birrell

Greg

Nelson

Susan

Owicki

and

Edward

Wobber

Network

objects

In

Proceedings

of

the

ACM

Symposium

on

Operating

Systems

Principles

pages

-LCB-

Asheville

NC

-LRB-

USA

-RRB-

December

-LSB-

-RSB-

Peter

B.

Bishop

Computer

Systems

with

a

Very

Large

Address

Space

and

Garbage

Collection

PhD

thesis

Massachusetts

Institute

of

Technology

Laboratory

for

Computer

Science

May

Technical

report

MIT/LCS/TR

-LSB-

-RSB-

Hans-J

Boehm

Alan

J.

Demers

and

Scott

Shenker

Mostly

parallel

garbage

collection

In

Proceedings

of

the

SIGPLAN

Conference

on

Programming

Language

Design

and

Implementation

-LSB-

-RSB-

pages

-LCB-

-LSB-

-RSB-

Hans-Juergen

Boehm

Space-efficient

conservative

garbage

collection

In

Proceedings

of

the

SIGPLAN

Conference

on

Programming

Language

Design

and

Implementation

pages

-LCB-

Albuquerque

New

Mexico

June

ACM

Press

-LSB-

-RSB-

Hans-Juergen

Boehm

and

David

Chase

A

proposal

for

garbage-collector-safe

compilation

The

Journal

of

C

Language

Translation

-LRB-

-RRB-

-LCB-

December

-LSB-

-RSB-

Rodney

A.

Brooks

Trading

data

space

for

reduced

time

and

code

space

in

real-time

collection

on

stock

hardware

In

Conference

Record

of

the

ACM

Symposium

on

LISP

and

Functional

Programming

-LSB-

-RSB-

pages

-LCB-

-LSB-

-RSB-

P.

Butterwoth

A.

Otis

and

J.

Stein

The

GemStone

object

database

management

system

Com

of

the

ACM

-LRB-

-RRB-

-LCB-

October

-LSB-

-RSB-

Luca

Cardelli

James

Donahue

Lucille

Glassman

Mick

Jordan

Bill

Kalso

and

Greg

Nelson

report

-LRB-

revised

-RRB-

Research

Report

Digital

Equipment

Corporation

Systems

Research

Center

November

-LSB-

-RSB-

Michael

Carey

D.

J.

DeWitt

and

J.

F.

Naughton

A

status

report

on

the

OODBMS

benchmarking

effort

In

Proceedings

of

the

Conf

on

Object-Oriented

Programming

Systems

Languages

and

Applications

-LRB-

OOPSLA

-RRB-

pages

-LCB-

Portland

OR

-LRB-

USA

-RRB-

October

-LSB-

-RSB-

Michael

J.

Carey

and

David

DeWitt

The

architecture

of

the

EXODUS

extensible

DBMS

In

Proc

Int

Workshop

on

Object-Oriented

Database

Systems

pages

-LCB-

Pacific

Grove

CA

-LRB-

USA

-RRB-

September

IEEE

BIBLIOGRAPHY

-LSB-

-RSB-

John

B.

Carter

John

K.

Bennett

and

Willy

Zwaenepoel

Implementation

and

performance

of

Munin

In

Proceedings

of

the

ACM

Symposium

on

Operating

Systems

Principles

volume

of

Operating

Systems

Review

pages

-LCB-

Pacific

Grove

CA

-LRB-

USA

-RRB-

October

-LSB-

-RSB-

J.

S.

Chase

H.

E.

Levy

M.

J.

Feely

and

E.

D.

Lazowska

Sharing

and

adressing

in

a

single

address

space

system

ACM

Transactions

on

Computer

Systems

-LRB-

-RRB-

November

-LSB-

-RSB-

C.

J.

Cheney

A

nonrecursive

list

compacting

algorithm

Communications

of

the

ACM

-LRB-

-RRB-

-LCB-

November

-LSB-

-RSB-

Thomas

W.

Christopher

Reference

count

garbage

collection

Software

Practice

and

Experience

-LRB-

-RRB-

-LCB-

June

-LSB-

-RSB-

Jacques

Cohen

Garbage

collection

of

linked

data

structures

Computing

Surveys

-LRB-

-RRB-

-LCB-

September

-LSB-

-RSB-

George

E.

Collins

A

method

for

overlapping

and

erasure

of

lists

Communications

of

the

ACM

-LRB-

-RRB-

-LCB-

December

-LSB-

-RSB-

Alan

Dearle

Rex

di

Bona

James

Farrow

Frans

Henskens

David

Hulse

Anders

Lindst?om

Stephen

Norris

John

Rosenberg

and

Francis

Vaughan

Protection

in

Grasshopper

A

persistent

operating

system

In

Proc

of

the

Int

Workshop

on

Persistent

Object

Systems

pages

-LCB-

Tarascon

-LRB-

France

-RRB-

September

-LSB-

-RSB-

Alan

Demers

Mark

Weiser

Barry

Hayes

Daniel

Bobrow

and

Scott

Shenker

Combining

generational

and

conservative

garbage

collection

Framework

and

implementations

In

Conference

Record

of

the

Seventeenth

Annual

ACM

Symposium

on

Principles

of

Programming

Languages

pages

-LCB-

San

Francisco

California

January

ACM

Press

-LSB-

-RSB-

David

Detlefs

Garbage

collection

and

run-time

typing

as

a

C++

library

In

C++

Conference

pages

-LCB-

Portland

OR

-LRB-

USA

-RRB-

August

Usenix

-LSB-

-RSB-

David

L.

Detlefs

Concurrent

Atomic

Garbage

Collection

PhD

thesis

Dept.

of

Computer

Science

Carnegie

Mellon

University

Pittsburgh

Pennsylvania

November

Technical

report

-LSB-

-RSB-

L.

Peter

Deutsch

and

Daniel

G.

Bobrow

An

efficient

incremental

automatic

garbage

collector

Communications

of

the

ACM

-LRB-

-RRB-

-LCB-

September

-LSB-

-RSB-

O.

Deux

et

al.

The

system

Communications

of

the

ACM

-LRB-

-RRB-

-LCB-

October

-LSB-

-RSB-

E.

W.

Dijkstra

L.

Lamport

A.

J.

Martin

C.

S.

Scholten

and

E.

F.

M.

Steffens

On-the-fly

garbage

collection

an

exercise

in

cooperation

Communications

of

the

ACM

-LRB-

-RRB-

-LCB-

November

-LSB-

-RSB-

Amer

Diwan

Eliot

Moss

and

Richard

Hudson

Compiler

support

for

garbage

collection

in

a

statically-typed

language

In

Proceedings

of

the

SIGPLAN

Conference

on

Programming

Language

Design

and

Implementation

pages

-LCB-

San

Francisco

California

June

ACM

Press

-LSB-

-RSB-

Damien

Doligez

and

Georges

Gonthier

Portable

unobtrusive

garbage

collection

for

multiprocessor

systems

In

Proc

of

the

Annual

ACM

SIGPLAN

SIGACT

Symp

on

Principles

of

Programming

Lang.

pages

-LCB-

Portland

OR

-LRB-

USA

-RRB-

January

BIBLIOGRAPHY

-LSB-

-RSB-

Damien

Doligez

and

Xavier

Leroy

A

concurrent

generational

garbage

collector

for

a

multithreaded

implementation

of

ML

In

Proc

of

the

Annual

ACM

SIGPLAN-SIGACT

Symp

on

Principles

of

Programming

Lang.

pages

-LCB-

Charleston

SC

-LRB-

USA

-RRB-

January

-LSB-

-RSB-

Daniel

R.

Edelson

Precompiling

C++

for

garbage

collection

In

Proc

International

Workshop

on

Memory

Management

pages

-LCB-

Saint-Malo

-LRB-

France

-RRB-

September

Springer-Verlag

-LSB-

-RSB-

Daniel

R.

Edelson

Smart

pointers

They

're

smart

but

they

're

not

pointers

In

C++

Conference

pages

-LCB-

Portland

OR

-LRB-

USA

-RRB-

August

Usenix

-LSB-

-RSB-

John

R.

Ellis

and

David

L.

Detlefs

Safe

efficient

garbage

collection

for

C++

Technical

Report

Digital

Equipment

Corporation

Systems

Research

Center

-LSB-

-RSB-

Margaret

A.

Ellis

and

Bjarne

Stroustrup

The

Annotated

C++

Reference

Manual

Addison-Wesley

-LSB-

-RSB-

Vaughan

F.

and

A.

Dearle

Supporting

large

persistent

stores

using

conventional

hardware

In

Proc

of

the

Fifth

International

Workshop

on

Persistent

Object

Systems

Design

Implementation

and

Use

pages

-LCB-

San

Miniato

Pisa

-LRB-

Italy

-RRB-

September

-LSB-

-RSB-

Michael

J.

Feeley

Jeffrey

S.

Chase

Vivek

R.

Narasayya

and

Henry

M.

Levy

Integrating

coherency

and

recovery

in

distributed

systems

In

Proc

Symp

on

Operating

Systems

Design

and

Implementation

pages

-LCB-

Monterey

CA

-LRB-

USA

-RRB-

November

-LSB-

-RSB-

Robert

R.

Fenichel

and

Jerome

C.

Yochelson

A

LISP

garbage-collector

for

virtual-memory

computer

systems

Communications

of

the

ACM

-LRB-

-RRB-

-LCB-

November

-LSB-

-RSB-

Paulo

Ferreira

Reclaiming

storage

in

an

object

oriented

platform

supporting

extended

C++

and

Objective-C

applications

In

Proc

of

the

International

Workshop

on

Object

Orientation

in

Operating

Systems

Palo

Alto

-LRB-

USA

-RRB-

October

-LSB-

-RSB-

Paulo

Ferreira

benchmark

for

GC

performance

evaluation

Personal

communication

October

-LSB-

-RSB-

Paulo

Ferreira

and

Marc

Shapiro

Distribution

and

persistence

in

multiple

and

heterogeneous

address

spaces

In

Proc

Int

Workshop

on

Object-Orientation

in

Operating

Systems

Asheville

NC

-LRB-

USA

-RRB-

December

-LSB-

-RSB-

Paulo

Ferreira

and

Marc

Shapiro

Garbage

collection

and

DSM

consistency

In

Proc

of

the

First

Symposium

on

Operating

Systems

Design

and

Implementation

-LRB-

OSDI

-RRB-

pages

-LCB-

Monterey

CA

-LRB-

USA

-RRB-

November

ACM

-LSB-

-RSB-

Paulo

Ferreira

and

Marc

Shapiro

Larchant

Persistence

by

reachability

in

distributed

shared

memory

through

garbage

collection

In

International

Conference

on

Distributed

Computing

Systems

-LRB-

ICDCS

-RRB-

Hong

Kong

May

IEEE

-LSB-

-RSB-

Ross

S.

Finlayson

and

David

R.

Cheriton

Log

files

An

extended

file

service

exploiting

write-once

storage

In

Proceedings

of

the

ACM

Symposium

on

Operating

Systems

Principles

pages

-LCB-

Austin

TX

-LRB-

USA

-RRB-

November

ACM

-LSB-

-RSB-

B.

Fleisch

and

G.

Popek

Mirage

A

coherent

distributed

shared

memory

design

In

Proceedings

of

the

ACM

Symposium

on

Operating

Systems

Principles

pages

-LCB-

Litchfield

Park

AZ

-LRB-

USA

-RRB-

December

ACM

BIBLIOGRAPHY

-LSB-

-RSB-

Benjamin

Goldberg

Generational

reference

counting

A

reduced-communication

distributed

storage

reclamation

scheme

In

Programming

Languages

Design

and

Implementation

number

-LRB-

-RRB-

in

SIGPLAN

Notices

pages

-LCB-

Portland

OR

-LRB-

USA

-RRB-

June

SIGPLAN

ACM

Press

-LSB-

-RSB-

Benjamin

Goldberg

Tag-free

garbage

collection

for

strongly-typed

programming

languages

In

Proceedings

of

the

SIGPLAN

Conference

on

Programming

Language

Design

and

Implementation

-LSB-

-RSB-

pages

-LCB-

-LSB-

-RSB-

David

Harel

Statecharts

A

visual

formalism

for

complex

systems

Science

of

Computer

Programming

-LCB-

-LSB-

-RSB-

Barry

Hayes

Using

key

object

opportunism

to

collect

old

objects

In

Andreas

Paepcke

editor

Conference

on

Object

Oriented

Programming

Systems

Languages

and

Applications

-LRB-

OOPSLA

-RRB-

pages

-LCB-

Phoenix

Arizona

October

ACM

Press

-LSB-

-RSB-

Yann

Herve

OCI

User

Manual

Personal

communication

October

-LSB-

-RSB-

Lorenz

Huelsbergen

and

James

R.

Larus

A

concurrent

copying

garbage

collector

for

languages

that

distinguish

-LRB-

im

-RRB-

mutable

data

In

Proceedings

of

the

Fourth

ACM

SIGPLAN

Symposium

on

Principles

and

Practice

of

Parallel

Programming

-LRB-

PPOPP

-RRB-

pages

-LCB-

San

Diego

California

May

ACM

Press

Published

as

SIGPLAN

Notices

-LRB-

-RRB-

July

-LSB-

-RSB-

John

Hughes

A

distributed

garbage

collection

algorithm

In

Jean-Pierre

Jouannaud

editor

Functional

Languages

and

Computer

Architectures

number

in

Lecture

Notes

in

Computer

Science

pages

-LCB-

Nancy

-LRB-

France

-RRB-

September

Springer-Verlag

-LSB-

-RSB-

Avadis

Tevanian

Jr.

Architecture-Independent

Virtual

Memory

Management

for

Parallel

and

Distributed

Environments

The

Mach

Application-Approach

PhD

thesis

Dept.

of

Comp

Sc.

Carnegie-Mellon

University

Pittsburgh

-LRB-

USA

-RRB-

December

-LSB-

-RSB-

Eric

Jul

Henry

Levy

Norman

Hutchinson

and

Andrew

Black

Fine-grained

mobility

in

the

Emerald

system

ACM

Transactions

on

Computer

Systems

-LRB-

-RRB-

-LCB-

February

-LSB-

-RSB-

T.

Kaelher

and

G.

Krasner

LOOM-Large

Object-Oriented

memory

for

Systems

Bits

of

History

Words

of

Advice

Addison-Wesley

MA

-LSB-

-RSB-

Pete

Keleher

Alan

L.

Cox

Sandhya

Dwarkadas

and

Willy

Zwaenepoel

TreadMarks

Distributed

shared

memory

on

standard

workstations

and

operating

systems

In

Proc

Winter

Usenix

Conf.

pages

-LCB-

San

Francisco

CA

-LRB-

USA

-RRB-

January

-LSB-

-RSB-

Brian

W.

Kernighan

and

Dennis

M.

Ritchie

The

C

Programming

Language

Prentice

Hall

-LSB-

-RSB-

Bett

Koch

Tracy

Schunke

Alan

Dearle

Francis

Vaughan

Chris

Marlin

Ruth

Fazakerley

and

Chris

Barter

Cache

coherency

and

storage

management

in

a

persistent

object

system

In

Proceedings

of

the

Fourth

International

Workshop

on

Persistent

Object

Systems

pages

-LCB-

Martha

's

Vineyard

MA

-LRB-

USA

-RRB-

September

-LSB-

-RSB-

E.

Kolodner

and

W.

Weihl

Atomic

incremental

garbage

collection

and

recovery

for

large

stable

heap

In

Proc

of

the

ACM

SIGMOD

Int

Conf

pages

-LCB-

Washington

D.C.

-LRB-

USA

-RRB-

June

BIBLIOGRAPHY

-LSB-

-RSB-

R.

Kordale

M.

Ahamad

and

J.

Shilling

Distributed/concurrent

garbage

collection

in

distributed

shared

memory

systems

In

Proc

Third

Int

Workshop

on

Object

Orientation

and

Operating

Systems

Asheville

NC

-LRB-

USA

-RRB-

November

-LSB-

-RSB-

Rivka

Ladin

and

Barbara

Liskov

Garbage

collection

of

a

distributed

heap

In

Int

Conf

on

Distributed

Computing

Systems

-LRB-

ICDCS

-RRB-

pages

-LCB-

Yokohama

-LRB-

Japan

-RRB-

June

IEEE

-LSB-

-RSB-

Leslie

Lamport

Time

clocks

and

the

ordering

of

events

in

a

distributed

system

Communications

of

the

ACM

-LRB-

-RRB-

-LCB-

July

-LSB-

-RSB-

B.

Lampson

Atomic

transactions

In

Distributed

Systems

Architecture

and

Implementation

pages

-LCB-

Spinger-Verlag

New

York

Lecture

Notes

in

Computer

Science

vol

-LSB-

-RSB-

Bernard

Lang

and

Francis

Dupont

Incremental

incrementally

compacting

garbage

collection

In

SIGPLAN

Symposium

on

Interpreters

and

Interpretive

Techniques

pages

-LCB-

Saint

Paul

Minnesota

June

ACM

Press

Published

as

SIGPLAN

Notices

-LRB-

-RRB-

July

-LSB-

-RSB-

Bernard

Lang

Christian

Queinnec

and

Jos?e

Piquer

Garbage

collecting

the

world

In

ACM

Symposium

on

Principles

of

Programming

Languages

pages

-LCB-

Albuquerque

New

Mexico

January

-LSB-

-RSB-

T.

Le

Sergent

and

B.

Berthomieu

Incremental

multi-threaded

garbage

collection

on

virtually

shared

memory

architectures

In

Proc

Int

Workshop

on

Memory

Management

number

in

Lecture

Notes

in

Computer

Science

pages

-LCB-

Saint-Malo

-LRB-

France

-RRB-

September

Springer-Verlag

-LSB-

-RSB-

Conference

Record

of

the

ACM

Symposium

on

LISP

and

Functional

Programming

Austin

Texas

August

ACM

Press

-LSB-

-RSB-

Kai

Li

and

Paul

Hudak

Memory

coherence

in

shared

virtual

memory

systems

ACM

Transactions

on

Computer

Systems

-LRB-

-RRB-

-LCB-

November

-LSB-

-RSB-

Henry

Lieberman

and

Carl

Hewitt

A

real-time

garbage

collector

based

on

the

lifetimes

of

objects

Communications

of

the

ACM

-LRB-

-RRB-

-LCB-

June

-LSB-

-RSB-

Barbara

Liskov

Mark

Day

and

Liuba

Shrira

Distributed

object

management

in

Thor

In

Proc

Int

Workshop

on

Distributed

Object

Management

pages

-LCB-

Edmonton

-LRB-

Canada

-RRB-

August

-LSB-

-RSB-

Barbara

Liskov

Mark

Day

and

Liuba

Shrira

Distributed

object

management

in

Thor

In

Proc

Int

Workshop

on

Distributed

Object

Management

pages

-LCB-

Edmonton

-LRB-

Canada

-RRB-

August

-LSB-

-RSB-

Barbara

Liskov

Robert

Gruber

Paul

Johnson

and

Liuba

Shrira

A

highly

available

object

repository

for

use

in

a

heterogeneous

distributed

system

In

Proc

of

the

Fourth

International

Workshop

on

Persistent

Object

Systems

Design

Implementation

and

Use

Martha

's

Vineyard

MA

-LRB-

USA

-RRB-

September

-LSB-

-RSB-

U.

Maheshwari

and

B.

Liskov

Fault-tolerant

distributed

garbage

collection

in

a

client-server

object

database

In

Proceedings

of

the

Parallel

and

Distributed

Information

Systems

pages

-LCB-

Austin

Texas

-LRB-

USA

-RRB-

September

-LSB-

-RSB-

U.

Maheshwari

and

B.

Liskov

Collecting

cyclic

distributed

garbage

by

controlled

migration

In

Proceedings

of

the

Principles

of

Distributed

Computing

Ottawa

-LRB-

Canada

-RRB-

September

BIBLIOGRAPHY

-LSB-

-RSB-

John

McCarthy

Recursive

functions

of

symbolic

expressions

and

their

computation

by

machine

Communications

of

the

ACM

-LRB-

-RRB-

-LCB-

April

-LSB-

-RSB-

Marvin

Minsky

A

LISP

garbage

collector

algorithm

using

serial

secondary

storage

A.I.

Memo

Massachusetts

Institute

of

Technology

Project

MAC

Cambridge

Massachusetts

-LSB-

-RSB-

David

Moon

Garbage

collection

in

a

large

Lisp

system

In

Conference

Record

of

the

ACM

Symposium

on

LISP

and

Functional

Programming

-LSB-

-RSB-

pages

-LCB-

-LSB-

-RSB-

J.

Eliot

B.

Moss

Addressing

large

distributed

collections

of

persistent

objects

The

Mneme

project

's

approach

In

Proceedings

of

the

Second

International

Workshop

on

Database

Programming

Languages

Gleneden

Beach

OR

-LRB-

USA

-RRB-

June

Also

available

as

COINS

Technical

Report

-LCB-

Object-Oriented

Systems

Laboratory

Dept.

of

Computer

and

Information

Science

University

of

Massachusetts

Amherst

-LSB-

-RSB-

S.

Nettles

J.

O'Toole

D.

Pierce

and

N.

Haines

Replication-based

incremental

copying

collection

In

Proc

Int

Workshop

on

Memory

Management

number

in

Lecture

Notes

in

Computer

Science

pages

-LCB-

Saint-Malo

-LRB-

France

-RRB-

September

Springer-Verlag

-LSB-

-RSB-

S.

C.

North

and

J.

H.

Reppy

Concurrent

garbage

collection

on

stock

hardware

In

Gilles

Kahn

editor

ACM

Conference

on

Functional

Programming

Languages

and

Computer

Architecture

number

in

Lecture

Notes

in

Computer

Science

pages

-LCB-

Springer-Verlag

September

-LSB-

-RSB-

E.I.

Organick

The

Multics

system

an

examination

of

its

structure

MIT

Press

Cambridge

Massachusetts

-LSB-

-RSB-

James

O'Toole

Scott

Nettles

and

David

Gifford

Concurrent

compacting

garbage

collection

of

a

persistent

heap

In

Proceedings

of

the

Fourteenth

Symposium

on

Operating

Systems

Principles

Asheville

North

Carolina

-LRB-

USA

-RRB-

December

ACM

Press

Published

as

Operating

Systems

Review

-LRB-

-RRB-

-LSB-

-RSB-

Jos?e

M.

Piquer

Indirect

reference-counting

a

distributed

garbage

collection

algorithm

In

PARLE

Parallel

Architectures

and

Languages

Europe

volume

of

Lecture

Notes

in

Computer

Science

pages

-LCB-

Eindhoven

-LRB-

the

Netherlands

-RRB-

June

Springer-Verlag

-LSB-

-RSB-

David

V.

Pitts

and

Partha

Dasgupta

Object

memory

and

storage

management

in

the

Clouds

kernel

In

Int

Conf

on

Distributed

Computing

Systems

-LRB-

ICDCS

-RRB-

pages

-LCB-

San

Jos?e

CA

-LRB-

USA

-RRB-

June

IEEE

-LSB-

-RSB-

David

Plainfoss?e

and

Marc

Shapiro

A

survey

of

distributed

garbage

collection

techniques

In

Proc

Int

Workshop

on

Memory

Management

Lecture

Notes

in

Computer

Science

Kinross

-LRB-

Scotland

-RRB-

September

Springer-Verlag

-LSB-

-RSB-

Proceedings

of

the

SIGPLAN

Conference

on

Programming

Language

Design

and

Implementation

Toronto

Ontario

June

ACM

Press

Published

as

SIGPLAN

Notices

-LRB-

-RRB-

June

-LSB-

-RSB-

Christian

Queinnec

Barbara

Beaudoing

and

Jean-Pierre

Queille

Mark

DUR

ING

Sweep

rather

than

Mark

THEN

Sweep

In

Eddy

Odijik

M.

Rem

and

Jean-Claude

Sayr

editors

Parallel

Architectures

and

Languages

Europe

number

in

Lecture

Notes

in

Computer

Science

Eindhoven

The

Netherlands

June

Springer-Verlag

BIBLIOGRAPHY

-LSB-

-RSB-

Michel

Raynal

Andr?e

Schiper

and

Sam

Toueg

The

causal

ordering

abstraction

and

a

simple

way

to

implement

it

Information

Processing

Letters

-LRB-

-RRB-

-LCB-

September

-LSB-

-RSB-

Mendel

Rosenblum

and

John

K.

Ousterhout

The

LFS

storage

manager

In

Proc

Summer

USENIX

Conf.

pages

-LCB-

Anaheim

CA

-LRB-

USA

-RRB-

June

USENIX

-LSB-

-RSB-

Vincent

Russo

Garbage

collecting

an

object-oriented

operating

system

kernel

Workshop

on

GC

in

Object-Oriented

Systems

at

Conf

on

Object-Oriented

Programming

Systems

Languages

and

Applications

-LRB-

OOPSLA

-RRB-

available

by

anonymous

FTP

from

cs.utexas.edu

in

October

-LSB-

-RSB-

A.

D.

Samples

Garbage

collection-cooperative

C++

In

Proc

Int

Workshop

on

Memory

Management

number

in

Lecture

Notes

in

Computer

Science

pages

-LCB-

Saint-Malo

-LRB-

France

-RRB-

September

Springer-Verlag

-LSB-

-RSB-

M.

Satyanarayanan

A

survey

of

distributed

file

systems

Technical

Report

Department

of

Computer

Science

Carnegie-Mellon

University

Pittsburgh

PA

-LRB-

USA

-RRB-

February

-LSB-

-RSB-

Michael

L.

Scott

Thomas

J.

LeBlanc

Brian

D.

Marsh

Timothy

G.

Becker

Dubnicki

Cezary

and

Evangelos

P.

Markatos

Implementation

issues

for

the

Psyche

multiprocessor

operating

system

Computing

Systems

-LRB-

-RRB-

-LCB-

-LSB-

-RSB-

Marc

Shapiro

A

fault-tolerant

scalable

low-overhead

distributed

garbage

detection

protocol

In

Tenth

Symp

on

Reliable

Distributed

Sytems

Pisa

-LRB-

Italy

-RRB-

October

-LSB-

-RSB-

Marc

Shapiro

A

binding

protocol

for

distributed

shared

objects

In

Proc

Int

Conf

on

Distributed

Computing

Systems

Poznan

-LRB-

Poland

-RRB-

June

-LSB-

-RSB-

Marc

Shapiro

Peter

Dickman

and

David

Plainfoss?e

Robust

distributed

references

and

acyclic

garbage

collection

In

Symp

on

Principles

of

Distributed

Computing

pages

-LCB-

Vancouver

-LRB-

Canada

-RRB-

August

ACM

Superseded

by

-LSB-

-RSB-

corrects

a

bug

more

elegant

more

informative

-LSB-

-RSB-

Marc

Shapiro

Peter

Dickman

and

David

Plainfoss?e

SSP

chains

Robust

distributed

references

supporting

acyclic

garbage

collection

Rapport

de

Recherche

Institut

National

de

la

Recherche

en

Informatique

et

Automatique

Rocquencourt

-LRB-

France

-RRB-

nov

Also

available

as

Broadcast

Technical

Report

-LSB-

-RSB-

Marc

Shapiro

Yvon

Gourhant

Sabine

Habert

Laurence

Mosseri

Michel

Ruffin

and

C?eline

Valot

SOS

An

object-oriented

operating

system

assessment

and

perspectives

Computing

Systems

-LRB-

-RRB-

-LCB-

December

-LSB-

-RSB-

Marc

Shapiro

Olivier

Gruber

and

David

Plainfoss?e

A

garbage

detection

protocol

for

a

realistic

distributed

object-support

system

Rapport

de

Recherche

Institut

National

de

la

Recherche

en

Informatique

et

Automatique

Rocquencourt

-LRB-

France

-RRB-

November

-LSB-

-RSB-

K.

Singhal

S.

Kakkad

and

P.

Wilson

Texas

An

efficient

portable

persistent

store

In

Proc

of

the

Fifth

International

Workshop

on

Persistent

Object

Systems

Design

Implementation

and

Use

pages

-LCB-

San

Miniato

Pisa

-LRB-

Italy

-RRB-

September

-LSB-

-RSB-

Richard

L.

Sites

Alpha

Architecture

Reference

Manual

Digital

Press

BIBLIOGRAPHY

-LSB-

-RSB-

Pedro

Sousa

Manuel

Sequeira

Andr?e

Z?uquete

Paulo

Ferreira

Cristina

Lopes

Jos?e

Pereira

Paulo

Guedes

and

Jos?e

Alves

Marques

Distribution

and

persistence

in

the

IK

platform

Overview

and

evaluation

Computing

Systems

-LRB-

-RRB-

-LCB-

November

-LSB-

-RSB-

Guy

L.

Steele

Jr.

Multiprocessing

compactifying

garbage

collection

Communications

of

the

ACM

-LRB-

-RRB-

-LCB-

September

-LSB-

-RSB-

Peter

Steenkiste

and

John

Hennessy

Tags

and

type

checking

in

Lisp

In

Second

International

Conference

on

Architectural

Support

for

Programming

Languages

and

Operating

Systems

-LRB-

ASPLOS

II

-RRB-

pages

-LCB-

Palo

Alto

California

October

-LSB-

-RSB-

Sun

Microsystems

Inc.

NFS

Network

file

system

protocol

specification

RFC

Network

Information

Center

SRI

International

March

-LSB-

-RSB-

G.

Tel

and

F.

Mattern

The

derivation

of

distributed

termination

detection

algorithms

from

garbage

collection

schemes

ACM

Transactions

on

Programming

Languages

and

Systems

-LRB-

-RRB-

-LCB-

January

-LSB-

-RSB-

David

Ungar

and

Frank

Jackson

Tenuring

policies

for

generation-based

storage

reclamation

In

Norman

Meyrowitz

editor

Conference

on

Object

Oriented

Programming

Systems

Languages

and

Applications

-LRB-

OOPSLA

-RRB-

Proceedings

pages

-LCB-

San

Diego

California

-LRB-

USA

-RRB-

September

ACM

Press

-LSB-

-RSB-

David

M.

Ungar

Generation

scavenging

A

non-disruptive

high-performance

storage

reclamation

algorithm

In

ACM

SIGSOFT/SIGPLAN

Software

Engineering

Symposium

on

Practical

Software

Development

Environments

pages

-LCB-

ACM

Press

April

Published

as

ACM

SIGPLAN

Notices

-LRB-

-RRB-

May

-LSB-

-RSB-

Francis

Vaughan

Tracy

Lo

Basso

Alan

Dearle

Chris

Marlin

and

Chris

Barter

Casper

a

cached

architecture

supporting

persistence

Computing

Systems

-LRB-

-RRB-

-LCB-

-LSB-

-RSB-

Stephen

C.

Vestal

Garbage

Collection

An

Exercise

in

Distributed

FaultTolerant

Programming

PhD

thesis

Dept.

of

Comp

Sc.

U.

of

Washington

Seattle

WA

-LRB-

USA

-RRB-

January

U.

of

Washington

Tech

Report

-LSB-

-RSB-

P.

Watson

and

I.

Watson

An

efficient

garbage

collection

scheme

for

parallel

computer

architectures

In

PARLE

Parallel

Architectures

and

Languages

Europe

number

in

Lecture

Notes

in

Computer

Science

Eindhoven

-LRB-

the

Netherlands

-RRB-

June

Springer-Verlag

-LSB-

-RSB-

Mark

Weiser

Alan

Demers

and

Carl

Hauser

The

portable

common

runtime

approach

to

interoperability

In

Proceedings

of

the

Twelfth

Symposium

on

Operating

Systems

Principles

Litchfield

Park

AZ

-LRB-

USA

-RRB-

December

ACM

Press

-LSB-

-RSB-

Brent

Welch

A

comparison

of

three

distributed

file

system

architecture

Vnode

Sprite

and

Plan

Computing

Systems

-LRB-

-RRB-

-LCB-

-LSB-

-RSB-

E.

P.

Wentworth

Pitfalls

of

conservative

garbage

collection

Software

Practice

and

Experience

-LRB-

-RRB-

-LCB-

July

-LSB-

-RSB-

Paul

R.

Wilson

Uniprocessor

garbage

collection

techniques

In

Proc

Int

Workshop

on

Memory

Management

number

in

Lecture

Notes

in

Computer

Science

Saint-Malo

-LRB-

France

-RRB-

September

Springer-Verlag

BIBLIOGRAPHY

-LSB-

-RSB-

Paul

R.

Wilson

and

Sheetal

V.

Kakkad

Pointer

swizzling

at

page

fault

time

Efficiently

and

compatibly

supporting

huge

address

spaces

on

standard

hardware

In

International

Workshop

on

Object

Orientation

in

Operating

Systems

pages

-LCB-

Dourdan

-LRB-

France

-RRB-

October

-LSB-

-RSB-

Paul

R.

Wilson

and

Thomas

G.

Moher

Design

of

the

Opportunistic

Garbage

Collector

In

Conference

on

Object

Oriented

Programming

Systems

Languages

and

Applications

-LRB-

OOPSLA

-RRB-

Proceedings

pages

-LCB-

New

Orleans

Louisiana

ACM

Press

-LSB-

-RSB-

V.

Yong

J.

Naughton

and

J.

Yu

Storage

reclamation

and

reorganization

in

client-server

persistent

object

stores

In

Proc

Data

Engineering

Int

Conf.

pages

-LCB-

Houston

TX

-LRB-

USA

-RRB-

February

-LSB-

-RSB-

Taichi

Yuasa

Real-time

garbage

collection

on

general-purpose

machines

Journal

of

Systems

and

Software

-LCB-

-LSB-

-RSB-

S.

Zdonik

and

D.

Maier

Readings

in

Object-Oriented

Database

Systems

MorganKaufman

San

Mateo

California

-LRB-

USA

-RRB-

-LSB-

-RSB-

Benjamin

Zorn

Comparative

Performance

Evaluation

of

Garbage

Collection

Algorithms

PhD

thesis

University

of

California

at

Berkeley

EECS

Department

December

Technical

Report

UCB/CSD

distributed garbage collection
distributed shared memory
distributed systems
persistence by reachability