Ruby's Concurrency
Management:
Now and Future
Koichi Sasada
ko1@cookpad.com

Today’s talk
•Supported features• Process• Thread• Fiber•Features under consideration• Guild• Auto-FiberToday’s talk
Process
Available
Yes
Switch on time Yes

Guild
No
Yes

ThreadYes
Yes

Switch on I/O

Auto
Auto
Yes

Auto
Auto
Auto
Auto
No (on MRI) No

Next target
Parallel run

Auto
Auto
Yes

Auto-Fiber
No
No

Fiber
Yes
No
No
SpecifyNo

Shared data N/A
Comm.
Hard
Programming Hard(mostly) N/A

Everything
Maybe Easy Easy
Easy
Difficult

Everything
Easy
Easy

Debuggingdifficulty

Maybe Easy Hard

Maybe hard Easy

difficulty

Easy?

Everything
Easy
Easy

Koichi Sasada
http://atdot.net/~ko1/•A programmer• 2006-2012 Faculty• 2012-2017 Heroku, Inc.
• 2017- Cookpad Inc.

•Job: MRI development• MRI: Matz Ruby Interpreter• Core parts• VM, Threads, GC, etc

Normal Ruby developer’s view
Ruby (Rails) app

i gigantum umeris insidentes
Standing on the shoulders of giants

So many gems
such as Rails, pry, thin, … and so on.
RubyGems/Bundler

Ruby interpreter

Normal MRI developer’s view
Ruby
scriptBundledLibraries

Parse

Gem
Libraries

Embeddedclasses and methods

AST
Abstract Syntax Tree

Compile(codegen)

Ruby
Bytecode

Threading

Evaluator

Object management
Garbage collector

Interpret on RubyVM
6

Koichi’s job
Ruby (Rails) app

So many gems
such as Rails, pry, thin, … and so on.
RubyGems/Bundler

Ruby interpreter
KoichiRuby3: Ruby3 has 3 goals•Static type checking•Just-in-Time (JIT) compilation•Parallel execution w/ highly
abstract concurrent modelRuby3: Ruby3 has 3 goals•For productivity• Static checking•For performance• Just-in-Time (JIT) compilation• Parallel execution w/ highly abstract
concurrent model

Concurrency“In computer science, concurrency is the decomposability
property of a program, algorithm, or problem into orderindependent or partially-ordered components or units.[1] Thismeans that even if the concurrent units of the program,algorithm, or problem are executed out-of-order or in partial
order, the final outcome will remain the same. This allows forparallel execution of the concurrent units, which cansignificantly improve overall speed of the execution in multiprocessor and multi-core systems.”
https://en.wikipedia.org/wiki/Concurrency_(computer_science)

Concurrent and Parallel execution
Concurrent Task A
execution Task B

Logical concept

Parallel(and concurrent)

Task A

execution

Task B

Physical concept

Ruby (MRI) support only concurrency

Concurrency
Why needed?
•Easy to write some kind of programs• Download files simultaneously• Process web requests simultaneously• Agent simulation (assume computer games)• Each agent has its own logics• Run agents simultaneously

Concurrency
Example: Downloader
Download A

post process
wait for receving
post process

Download B

Download C

wait for receving

post process
wait for receving

We can write this kind of program w/o concurrency support,
but not simple, not easyDownloader example
With concurrency support (Thread)ts = URLs.map do |url|
Thread.new(url) do |u|data = download(u)File.write(u.to_fname, data)
end
end.each{|th| th.join} # waitDownloader example
Without concurrency support# Serial execution
URLs.each do |u|data = download(u)File.write(u.to_fname, data)
endConcurrency
Not concurrent case
Download A

post process
wait for receving
post process

Download B

Download C

wait for receving… and download C after that

Downloader example
Without concurrency support# Use select. Not so SIMPLE!!
fds = URLs.map do |u|download_fd(u)endwhile ready_fds = select(fds)ready_fds.each{|fd|File.write(…, read(fd))}
end

Existing concurrency supports on
Ruby (MRI)Supported features by Ruby/MRI•Process•Thread•FiberProcess
Traditional concurrency support

Process• Use OS multi-process• Use fork on Unix-like systems• Shared-nothing• Communicate with IPC (pipe, etc) such as IO.pipe• Programming• Difficult to manage processes and IPC• Debugging• Easy because a few synchronization bugs

Inter-process communication
Ruby process

Ruby process

pipe
obj

Deserialize

Serialize

filestringobj

stringInter-process communication
Example code
# Traditional multi-process example
r, w = IO.pipe
fork doresult_str = work_something.to_s
w.write result_str
w.closeend
puts r.read # wait for a result

Sophisticated libraries/frameworks forprocess programming•dRuby: Distributed object for Ruby
•parallel gem: Parallel programming withprocesses•unicorn: Process based web application
server (master – worker model w/ processes)

ThreadRuby’s native concurrency support

Thread• Use Ruby managed threads• Thread.new do … end• Shared-everything• Communication is very easy

• Programming• Easy to make, easy to communicate (at a glance)• Difficult to make completely safe program• Debugging• Hard because of synchronization

MRI: Thread with Giant Lock (GIL)•Only a thread keeping the GIL can run (can’t
run in parallel)
Thread 1

Thread 1
CPU 1

OS Thread 1

Lock
Thread 2

Lock
OS Thread 2

CPU 2

IDLE

Inter-thread communication
Easy to share objects
Ruby process
Threadobj
We can share objects directly
between threads very easily

ThreadInter-thread communicationv = Object.new$g = Object.new
Thread.new dop [v, $g]end
p [v, $g]Thread programming
Synchronization is required•Reading/writing data simultaneously w/osynchronization will cause serious problem• Race condition• Data raceMutate shared objects
Lucky case
Ruby process
ThreadA

SpeakerObject@name = ‘ko1’@gender = ‘male’(1) Thread A tries to change the Speaker
to “Yuki” (female)Note: Yuki is my wife.

ThreadB

Mutate shared objects
Lucky case
Ruby process
ThreadA

write

SpeakerObject@name = ‘Yuki’@gender = ‘male’(2) A changes the name to “Yuki”

ThreadB

Mutate shared objects
Lucky case
Ruby process
ThreadA

write

SpeakerObject@name = ‘Yuki’@gender = ‘female’(3) A changes the gender to “female”

ThreadB

Mutate shared objects
Lucky case
Ruby process
ThreadA

read

SpeakerObject

read@name = ‘Yuki’@gender = ‘female’(4) Complete.
A and B can read correct speaker.

ThreadB

Mutate shared objects
Problematic case
Ruby process
ThreadA

SpeakerObject@name = ‘ko1’@gender = ‘male’(1) Thread A tries to change the Speaker
to “Yuki” (female)ThreadB

Mutate shared objects
Problematic case
Ruby process
ThreadA

write

SpeakerObject@name = ‘Yuki’@gender = ‘male’(2) A changes the name to “Yuki”

ThreadB

Mutate shared objects
Problematic case
Ruby process
ThreadA

SpeakerObject@name = ‘Yuki’@gender = ‘male’(3) Before the changing,B read incorrect data!!
Note: Yuki should be female.

read

ThreadB

Inter-thread communication
Synchronization•Require synchronization for shared data
• Mutex, Queue and so on• Usually Queue is enough• To prohibit simultaneous mutation• We need to keep consistency for each objects

Mutate shared objects
With lock
Locked by A

ThreadA

Ruby process
SpeakerObject@name = ‘ko1’@gender = ‘male’(1) Thread A tries to change the Speaker
to “Yuki” (female). Lock an obj.

ThreadB

Mutate shared objects
With lock
Locked by A

ThreadA

Ruby process
SpeakerObject@name = ‘Yuki’@gender = ‘male’(2) A changes the name to “Yuki”

ThreadB

Mutate shared objects
With lock
Locked by A

ThreadA

Ruby process
SpeakerObject

read@name = ‘Yuki’@gender = ‘male’(3) Before complete the changing,
B tries to read, but prohibited by a lockThreadB

Thread programming
Easy to share data: Good and Bad•Good: Easy to communicate with threads•Bad: Too easy. Difficult to manage all ofthem• Mutation for shared data requires correct
synchronization• Sometimes objects are shared implicitly• Otherwise, it causes serious problems

“Why Threads Are A Bad Idea
(for most purposes)”• Quoted from John Ousterhout, 1995 (about 20 years ago ☺)Compare Process with Thread
Process

ThreadSwitch on I/O

Yes
Yes
Auto

Yes
Yes
Auto

Next target
Parallel run
Shared data

Auto
Yes
N/A

Auto
No (on MRI)
Everything

Communication

Hard (high-overhead)
Hard
Easy?

Easy (lightweight)
Difficult
Hard

Available
Switch on time

Programming difficulty
Debugging difficultyFiberUser-defined context switching

Fiber example
Infinite generator
fib = Fiber.new do
Fiber.yield a = b = 1
loop{ a, b = b, a+b
Fiber.yield a }end
10.times{ p fib.resume }Fiber example
Infinite generator
fib = Fiber.new do
2. Resume Fiber
Fiber.yield a = b = 1
loop{ a, b = b, a+b
4. Resume fiber(again)1. Fiber creation
Fiber.yield a }5. Return to the
6. Resume fiber
Return to theend 3.parent
parent fiber(again2)fiber
10.times{ p fib.resume }Fiber example
Infinite generator
fib = Fiber.new do
2. Resume Fiber
Fiber.yield a = b = 1
loop{ a, b = b, a+b
4. Resume fiber(again)1. Fiber creation
Fiber.yield a }5. Return to the
6. Resume fiber
Return to theend 3.parent
parent fiber(again2)fiber
10.times{ p fib.resume }Not a Proc?
a = 0; b = 1
fib = Proc.new{a, b = b, a+ba}p fib.call #=> 1
p fib.call #=> 1
p fib.call #=> 2
p fib.call #=> 3
p fib.call #=> 5

Proc can’t restart fromthe middle of blockProc (method) v.s. Fiber
Start
Parameters
ReturnSuspend
Continue

Proc (method)
OK: callFiber
OK: Fiber#resume

OK: block (method) parameters OK: block parameters
OK: exit Proc/method

OK: exit Proc/method

NG: N/A

OK: Fiber.yield

NG: N/A

OK: Fiber#resume

Fiber example
Inner iterator to external iterator
f1 = Fiber.new do
2.times{|i| Fiber.yield i}
end
p f1.resume #=> 0
p f1.resume #=> 1
p f1.resume #=> 2 # return value of #times
p f1.resume #=> dead fiber called(FiberError)

Fiber example
Inner iterator to external iterator
etc_passwd_ex_iter = Fiber.new doopen('/etc/passwd').each_line{|line|
Fiber.yield line}end
p etc_passwd_ex_iter.resume #=> 1st line
p etc_passwd_ex_iter.resume #=> 2nd line…

Fiber example
Inner iterator to external iterator# make Enumerator
iter = open('/etc/passwd').each_line# Enumerator#next use Fiber implicitly
p iter.next #=> 1st line
p iter.next #=> 2nd line…

Fiber example
Agent simulation
characters << Fiber.new{loop{cat.move_up; Fiber.yield}}characters << Fiber.new{loop{dog.move_left; Fiber.yield}}…
loop{cs.each{|e| e.resume}; redraw}

Fiber example
Agent simulation
characters << Fiber.new{# you can specify complex rule for chars
loop{
cow.move_up;
Fiber.yield
cow.move_right; Fiber.yield
cow.move_down; Fiber.yield
cow.move_left; Fiber.yield}}Fiber example
Non-blocking IO scheduler

Wait multiple IO ops withtraditional “select” ormodern “poll”, “epoll” interface

Fiber
Programming difficulty•Good• Synchronization for shared data is not required
because of no unexpected switching• Lightweight than Processes and Threads•Bad• We need to switch explicitly. For example,“Blocking operations” (I/O blocking, etc) stop allfibersComparison of existing supports
Available
Switch on time
Switch on I/O

Next target
Parallel run
Shared data
Comm.

Process
Yes

Thread
Yes

Fiber
Yes

Yes
Auto
Auto

Yes
Auto
Auto

No
No
Specify

Yes
N/A
Hard

No (on MRI)
Everything
Easy

No
Everything
Easy

Difficult
Hard

Easy
Easy

Programming difficulty Hard
Debugging difficulty

Easy?

Fiber: Brief history•2007/05/23 cont.c (for callcc)•2007/05/25 Fiber impl. [ruby-dev:30827]•2007/05/28 Fiber introduced into cont.c•2007/08/25 Fix Fiber spec•2017 is 10th anniversary I introduced ☺

Proposed concurrency
features
Guild
Auto-Fiber

GuildProposed concurrency support for Ruby 3

Key idea
Problem of multi-thread programming:
Easy to share mutable objects
Idea:
Prohibit sharing mutable objects

Our goal for Ruby 3•We need to keep compatibility with Ruby 2.
•We can make parallel program.
•We shouldn’t consider locks any more.
•We can share objects with copy, but copy
operation should be fast.
•We should share immutable objects if we can.
•We can provide special objects to sharemutable objects like Clojure if we really need
speed.

Guild: New concurrency abstraction•Guild has at least one thread (and a thread
has at least one fiber)
Guild
ThreadFiber
FiberThread
Fiber

Guild
Guild
ThreadFiberThreadFiberThreads in different guilds can run in
Parallel• Threads in different guilds can run in parallel• Threads in a same guild can not run in parallelbecause of GVL (or GGL: Giant Guild Lock)
Acquire GGL

G1:T1
G1:T2

G2:T3

Acquire GGL

Important rule:
Mutable Objects have a membership•All of mutable objects should belong to only
one Guild exclusively•Guild can not touch objects belong to otherguildsGuild 1
Guild 2
NG!!
obj

obj

obj

Can’t access(read/write)

obj

obj

Object membership
Only one guild can access mutable object→ We don’t need to consider locks(if Guild has only one thread)Inter-guild communication•“Guild::Channel” to communicate each guilds•Two communication methods
1. Copy
2. Move (transfer_membership)Copy using Channel
channel.transfer(o1)o1 = channel.receive

Guild1

Guild2o1channelo2
O2:Datao1o2o3
O3:DataO2:Data

COPY

o3
O3:DataMove using Channel
channel.transfer_membership(o1)o1 = channel.receive

Guild1

Guild2o1channelo2
O2:Datao3
O3:DataMOVE

Move using Channel
channel.transfer_membership(o1)o1 = channel.receive

Guild1

Guild2-o1channel-o2-

From Guild1 perspective,
transferred objects are invalidated

O2:DataMOVE

o3
O3:Data

Sharing immutable objects
We can share reference to immutable objects
channel.transfer(o1)o1 = channel.receive

Guild1

Guild2

Ref too1
readchannelRef to
o1

o1

read

o2
o3
If o1 is immutable, any Guild can read o1
O2:Data

O3:DataUse-case 1: master – worker typedef fib(n) ... end
g_fib = Guild.new(script: %q{ch = Guild.default_channel
while n, return_ch = ch.receive
return_ch.transfer fib(n)end})
ch = Guild::Channel.new
g_fib.transfer([3, ch])p ch.receiven, return_ch

Main
Guild

ch

Fibonacci
Guild

return_ch

Answer of fib(n)NOTE: Making other Fibonacci guilds,
you can compute fib(n) in parallelUse-case 2: pipeline
result_ch = Guild::Channel.new
g_pipe3 = Guild.new(script: %q{
while obj = Guild.default_channel.receive
obj = modify_obj3(obj)
Guild.argv[0].transfer_membership(obj)end}, argv: [result_ch])g_pipe2 = Guild.new(script: %q{
while obj = Guild.default_channel.receive
obj = modify_obj2(obj)
Guild.argv[0].transfer_membership(obj)end}, argv: [g_pipe3])
g_pipe1 = Guild.new(script: %q{
while obj = Guild.default_channel.receive
obj = modify_obj1(obj)
Guild.argv[0].transfer_membership(obj)end}, argv: [g_pipe2])
obj = SomeClass.new
g_pipe1.transfer_membership(obj)obj = result_ch.receive

Main
Guild

Obj’’’
Move

Obj’’’

obj

Pipe 1
Guild

Pipe 3
Guild

Moveand modify

Obj’

Move
and modify

Moveand modify

Pipe 2
Guild
Obj’’

Compare with Process, Guild, Thread
Process
Yes
Yes

Guild
No
Yes

Thread
Yes
Yes

Auto
Auto
Yes

Auto
Auto
Yes

Auto
Auto
No (on MRI)

N/A
Hard
Programming difficulty Hard(mostly) N/A

Maybe Easy
Easy

Everything
Easy
Difficult

Debugging difficulty

Maybe Easy

Hard

Available
Switch on time
Switch on I/O

Next target
Parallel run
Shared data
Comm.

Easy?

Auto Fiber
Another proposed concurrency support for Ruby 3

Problem of Fiber
Requires explicit switching•“Fiber” enables writing scheduler by
programmer→ Programmers need to write own scheduler• We need to manage blocking operations likeI/O blocking

Auto Fiber proposal
https://bugs.ruby-lang.org/issues/13618

Auto Fiber proposal
Automatic schedule on I/O blocking• Support Fiber scheduler
natively• Don’t need to returnscheduler• Switch Fibers on all blocking
I/O (and other ops)
• No need to change existing
programs

Advantage and Disadvantage•Advantage• Don’t need to modify existing programs• Lightweight as a Fiber• Safer than Threads (no preemption)•Disadvantage• Introduce “non-deterministic” dangers same as
Thread programs• Non atomic operations can intercept accidentally.

Change the name…?

Compare w/ Thread and (auto-)Fiber
ThreadYes
Yes

Auto-Fiber
No
No

Fiber
Yes
No

Auto
Auto
No (on MRI)

Auto
Auto
No

No
Specify
No

Everything
Easy
Programming difficulty Difficult

Everything
Easy
Easy

Everything
Easy
Easy

Debugging difficulty

Maybe hard

Easy

Available
Switch on time
Switch on I/O

Next target
Parallel run
Shared data
Comm.

Hard

Today’s talk
•Supported features• Process• Thread• Fiber•Features under consideration• Guild• Auto-FiberToday’s talk
Process
Available
Yes
Switch on time Yes

Guild
No
Yes

ThreadYes
Yes

Switch on I/O

Auto
Auto
Yes

Auto
Auto
Auto
Auto
No (on MRI) No

Next target
Parallel run

Auto
Auto
Yes

Auto-Fiber
No
No

Fiber
Yes
No
No
SpecifyNo

Shared data N/A
Comm.
Hard
Programming Hard(mostly) N/A

Everything
Maybe Easy Easy
Easy
Difficult

Everything
Easy
Easy

Debuggingdifficulty

Maybe Easy Hard

Maybe hard Easy

difficulty

Easy?

Everything
Easy
Easy

References•Fiber: RubyKaigi 2017 http://rubykaigi.org/2017/presentations/ko1.html•Guild: RubyConf 2016 https://www.youtube.com/watch?v=mjzmUUQWqco•Auto-fiber: Feature #13618
https://bugs.ruby-lang.org/issues/13618

Thank you for your attention
Koichi Sasada<ko1@cookpad.com>