Fibers Are the Right Solution
Samuel Williams
samuel@ruby-lang.org www.codeotaku.com@ioquatix

Fibers Are the Right Solution
Samuel Williams
samuel@ruby-lang.org www.codeotaku.com@ioquatix

What is Scalability?
スケーラビリティとは何か？

Job

HardwareResultJob

ResultJob

ResultProportional Improvement
⽐比例例的なパフォーマンス改善

Work

Work

Work

Work

WorkWork

Why is scalability
important?
なぜスケーラビリティは⼤大切なのか？

Photo from NASA.

Is Ruby scalable?
Rubyはスケーラブルか？

Over a million web
sites globally.
Rubyが使われた世界中のWebサイトについて

Is this scalable?

Processing

Blocking

10s

Web Application

050s

15s

30s
Processing 500 requests/min45s

1mProcessing

Blocking

10s

Web Application

050s

15s

30s
Processing 500 requests/min

45s

1m

Postgres

S3RedisWebSocket

SMTP

Blocking
ブロッキング
DNS

HTTP

MySQL

Disk

How do we maximise
hardware utilisation?
どのようにしてハードウェア最⼤大限活⽤用できるか？

Late 1950s
1950年年代後半

100μs

Idle

SEEK

100μs

Idle

READ

1ms

Idle

PROCESSING

SEEKINGREADINGWRITING

10 seconds

10ms

20ms

CPU Instruction
1 nanosecond

CPU Instruction
1 nanosecond

CPU Cache
10 nanosecond

CPU Instruction
1 nanosecond

CPU Cache
10 nanosecond

Main Memory
100 nanosecond

CC

Solid State Disk1 millisecond

Solid State Disk
1 millisecond

Network Packet
10 millisecond

Can we avoid being
idle?
アイドル状態を避けることは可能か？

Time Sharing

Time Slicing

並⾏行行性Concurrency

Contention

並列列性

Parallelism

How does this apply
to Ruby?
どのようにしてRubyに応⽤用するか？

Is Ruby fast enough?
Rubyは⼗十分早いか？

Let’s make Ruby 10xfaster.
Rubyを10倍早くしようProcessing

Blocking

10s

Web Application

050s

15s

30s

45s

1m

Processing

Web Application

1s

0Blocking

50s

15s

30s

45s

1m

Let’s make Ruby 100xfaster.
Rubyを100倍早くしようProcessing

Web Application

100ms

0Blocking50s

15s30s45s

1m

How do we handle
more requests?
どのようにしてもより多くのリクエストを処理理するか？

Can we use multiple
processes?
複数のプロセスを利利⽤用することはできるか？

What about threads?
スレッドは？

How bad is the global
interpreter lock?
グローバルインタプリタロックはどれほど悪いのかFalcon “Hello World” Web Server
60000

45000

30000

15000

0

8 Processes

8 Threads
Requests/second

Are processes andthreads sufficient?
プロセスとスレッドは⼗十分か？

How many processes
can we create?
いくつのプロセスを作ることができるか？

How many threads
can we create?
いくつのスレッドを作ることができるか

100?

1,000?

10,000?

What about longrunning connections?
ロングランニングコネクションはどうするか？

What about 100,000
connected WebSockets?
100,000のWebSocketsは？

We need to go deeper

Event driven
non-blocking I/O
イベントドリブンのノンブロッキングI/O

while message = connection.read
handle(message)
end

while message = connection.read
handle(message)
end

while message = connection.read
handle(message)
endwhile true
while message = connection.read
ready = IO.select(connections)handle(message)
ready.each{|connection| handle(connection.read)}end
endHow do we handle
user logic?
ユーザーロジックはどのようにして処理理するかdef remote_size(host, port)peer = TCPSocket.new(host, port)
count = 0
while buffer = peer.read(1024)
count += buffer.bytesize
end
return countensurepeer&.closeendputs remote_size(HOST, PORT)

Sequential is easy.
シーケンシャルな処理理は簡単

Callbacks…

def remote_size(host, port)
TCPSocket.new(host, port) do |peer, error|
end
end
remote_size(HOST, PORT) do |size, error|
puts sizeenddef remote_size(host, port)
TCPSocket.new(host, port) do |peer, error|
if erroryield nil, error
elsecount = 0peer.read(1024) do |buffer, error|
end
end
end
end

yield nil, error
elsecount = 0
read_more = lambda dopeer.read(1024) do |buffer, error|
if erroryield nil, error
elsif buffercount += buffer.bytesize
read_more.call
else
yield count, nil
end
end
end
read_more.call
end
end
end

while buffer = peer.read(1024)
count += buffer.bytesize
end
return countTCPSocket.new(host, port) do |peer, error|
if erroryield nil, error
elsecount = 0
read_more = lambda dopeer.read(1024) do |buffer, error|
if erroryield nil, error
elsif buffercount += buffer.bytesize
read_more.call
else
yield count, nil
end
end
end
read_more.callendendif error
yield nil, error
elsecount = 0
read_more = lambda dopeer.read(1024) do |buffer, error|
if errorpeer.close
yield nil, error
elsif buffercount += buffer.bytesize
read_more.call
else
peer.close
yield count, nil
end
end
end
read_more.callendif error
yield nil, error
elsecount = 0
read_more = lambda do
peer.read(1024) do |buffer, error|
if errorpeer.close
yield nil, error
elsif buffercount += buffer.bytesize

Callback Hell

read_more.call
else
peer.close
yield count, nil
end
end
end
read_more.callendAsync/Await…

async def remote_size(host, port)
peer = await TCPSocket.new(host, port)
count = 0
while buffer = await peer.read(1024)
count += buffer.bytesize
end
return countensurepeer&.closeend
async lambda doputs await remote_size(HOST, PORT)
end.callasync lambda doputs(await remote_size(HOST, PORT))
end.callasync lambda do
await puts(await remote_size(HOST, PORT))
end.callasync def remote_size(host, port)
peer = await TCPSocket.new(host, port)
count = 0
while buffer = await peer.read(1024)
count += buffer.bytesize
end
return countensurepeer&.close
end

Async/Await Hellasync lambda do
await puts(await remote_size(HOST, PORT))
end.callCan we do better?
改善できるかShould we rewrite
existing code?
既存のコードを書き直すべきか？

What about usingfibers?
ファイバーを使うのはどうかWhat are fibers?
ファイバーとは何かcalladd(10)def add(sum)return sum
endputs add(10) # => 10
puts add(20) # => 20

sum = 10returnputs(10)return sum

calladd(20)sum = 20

returnputs(20)

return sum

resumeadd(10)add = Fiber.new do |sum|while truesum += Fiber.yield(sum)endendputs add.resume(10) # => 10
puts add.resume(20) # => 30

sum = 10yieldputs(10)yield sum

resumeadd(20)sum += 20

yieldputs(30)yield sum

Stack and instruction
pointer is not lost.
スタックとインストラクションポインタはなくなってはいないHow do we use fibers
for blocking I/O?
どのようにしてファイバーをブロッキングI/Oに使うか

EventLoop

Client 1

Client 2

readyield(socket)
read

IO.select

EWOULDBLOCK

resumewrite

readyyield(socket)

resumeready

readyield sumread
EWOULDBLOCK

yield(socket)
IO.select

read
EWOULDBLOCK

def remote_size(*address)
Async dopeer = Async::IO::TCPSocket.new(*address)
count = 0
while buffer = peer.read(1024)
count += buffer.bytesize
end
return countensure
peer&.close
end.wait
end
puts remote_size(HOST, PORT)How do we makeexisting code scalable?
どのようにして既存のコードをスケーラブルにするかTransparently makeall I/O non-blocking
コードを変更更することなく全てのI/Oをノンブロッキングにするhttps://github.com/socketry/async

“Async is the right model because
web apps are almost always I/Obound. The Ruby web ecosystem isreally lacking in scalability (e.g.
WebSockets on Puma). Async
unlocks the next tier of scalability inthe most Ruby-like way possible.”
Bryan Powell, on migrating from Puma to Falcon.

FALCONhttps://github.com/socketry/falcon

Multi-process
Multi-thread
Multi-fiber

HTTP/1
HTTP/2 & TLS

WebSockets

EventLoop

Synchronous
Rack Middlewareyield
resumeAsynchronous
Faraday

Synchronous
Rack Middlewareyield
resumeAsynchronous
PostgresHow does it perform?
パフォーマンスはどうか

Puma

FalconRequests / second

3000

2250

1500

750

0

1

2

4

8163264128

Concurrency (simultaneous connections)

256512

Are fibers the rightsolution?
ファイバーが正しい選択かFibers scale better
than threads.
スレッドよりもファイバーの⽅方がスケールする

Fibers are easier thanthreads.
スレッドよりもファイバーの⽅方が簡単

Fibers improve thescalability of existing code.
ファイバーは既存コードのスケーラビリティを改善するPostgres

S3

Redis

WebSocket

SMTP

Fibers Are the RightSolution

MySQL

ファイバーが正しい選択

DNS

HTTP

DiskFibers Are the Right Solution