RubyMemory Management Hacks
Ruby のメモリ管理とかなんとか

SASADA Koichi <ko1@rvm.jp>
Department of Creative Informatics,
Graduate School of Science and Technology,
The University of Tokyo

2009/11/23

Ruby Memory Management Hacks

1Summary of this talky Ruby 1.9.2 may improve severalperformancey Memory management efficiencies
y Fiber performancey Memory profiler featurey Researches are progressingy Memory Profiler
y MVM
2009/11/23

Ruby Memory Management Hacks

2

Agenday (1) Memory Management Hacks
y OS dependent Memory Managementy Memory profiler (including Studentʼs work)y (2) VM performance improvementy Fiber performance improvement (Studentʼswork)y Other progressy (3) Multi-VM (MVM) Progress
2009/11/23

Ruby Memory Management Hacks

3

(1) Memory Management Hacks
1. Trivial Optimizations
2. Memory Profiler

y “objspace” libraryy New T_DATA representation

3. OS dependent MemoryManagement

2009/11/23

Ruby Memory Management Hacks

4Memory Management
Importancey Memory is VERY BIG, EXPENSIVE!
y malloc()/free() helly Garbage collection overheady Long running services

y e.g. 30 years continuous running•20/Nov/1979 JST

2009/11/23

Ruby Memory Management Hacks

5Memory ManagementProblems
y CRuby/MRI has several issues

about M/M

y Naive Garbage Collection•Not a JVM, .NET or other VM
including smart garbage collection•Programmer friendly, GC unfriendly C
extension interfacesy No memory profilersy Object Fragmentation
2009/11/23

Ruby Memory Management Hacks

6

Solution (1)Reduce GC Pressurey CRuby.GC != Generational GC
y Old, long life object might be GC

pressure (increase GC frequency)
y NODE, InstructionSequence• Inline cache entries (ICE) are NODE.
y Modules, Classes

y [1.9.2 Feature] ICE is no longer GC

managed object (manage it explicitly).
y Reducing long lived NODE object.

→ Reducing GC pressure.
y Planning to adopt other NODE object.
2009/11/23

Ruby Memory Management Hacks

7

Solution (2)Memory Profiler
y Memory Profiler Support

y ObjectSpace#memsize_of(obj)y Experimental Memory Profilery [NOTE] These methods are notexpected to work except C Ruby.

2009/11/23

Ruby Memory Management Hacks

8Memory Profiler
ObjctSpace#memsize_ofy [1.9.2 Feature] 1.9.2 introduces“objspace” library:y This library adds the followingmethods•ObjectSpace.memsize_of(obj)•ObjectSpace.count_objects_size•ObjectSpace.count_nodes•ObjectSpace.count_tdata_objects

2009/11/23

Ruby Memory Management Hacks

9Memory Profiler
ObjctSpace#memsize_of example
require ʻobjspaceʼ
ObjectSpace.memsize_of(ʻ.ʼ*100)#=> 100
ObjectSpace.memsize_of(Time.now)#=> 60ObjectSpace.memsize_of(Thread.new{})#=> 524888
ObjectSpace.memsize_of(STDOUT)#=> 124
2009/11/23

Ruby Memory Management Hacks

10Memory Profiler
New T_DATA Representation
y T_DATA: Wrapping C structurey Traditional T_DATA representation includes:y pointer to memory object what T_DATA wrap
y pointer to mark function
y pointer to free functiony Make a T_DATA value with
Data_Wrap/Make_Struct(klass, mark, free, sval)
y T_DATA lacks:y Type Identity

y Memory size function
2009/11/23

Ruby Memory Management Hacks

11

Memory Profiler
New T_DATA representation (2)y [1.9.2 Feature] We introduce the type

rb_data_type_t:

y wrap_struct_namey pointer to mark function
y pointer to free function

y pointer to memsize_of functiony Make T_DATA object with

TypedData_Wrap/Make_Struct(klass, type, data_type, sval)

2009/11/23

Ruby Memory Management Hacks

12

Memory Profiler
New T_DATA representation (3)y Example:

static const rb_data_type_t time_data_type = {"time",
time_mark, time_free, time_memsize,};…
obj = TypedData_Make_Struct(klass, struct time_object,&time_data_type, tobj);

2009/11/23

Ruby Memory Management Hacks

13

Memory Profiler
New T_DATA representation (4)y New C APIsy rb_objspace_data_type_memsize(obj)y rb_objspace_data_type_name(obj)y RTYPEDDATA_DATA(obj)y RTYPEDDATA_P(obj)y RTYPEDDATA_TYPE(obj)

2009/11/23

Ruby Memory Management Hacks

14

Memory Profiler
Statistic Profilery Sampling memory statusperiodicallyy We already get tools

2009/11/23

Ruby Memory Management Hacks

15

Statistic Profiler
Examples (per T_xxx + Stack frame)

2009/11/23

Ruby Memory Management Hacks

16

Statistic Profiler
Examples (per Classes)

2009/11/23

Ruby Memory Management Hacks

17

Statistic Profiler
Examples (T_xxx / count)

2009/11/23

Ruby Memory Management Hacks

18

Statistic Profiler
Examples (NODE)

2009/11/23

Ruby Memory Management Hacks

19

Statistic Profiler
Examples (T_DATA)

2009/11/23

Ruby Memory Management Hacks

20

Memory Profiler
Deterministic Profiler
y [Research] We are working on

deterministic memory profilery Explicit realtime memory usagey Explicit object lifetimey Explicit object generated place(file:line)y Prepare probe points and some APIsy Inspired from Java realtime memory

profiler, valgrind/massif profiler

2009/11/23

Ruby Memory Management Hacks

21

Solution (3)OS dependent Memory Managementy Problem: Object Fragmentationy Memory block can not be de-allocate if

at least one object exists.

2009/11/23

Ruby Memory Management Hacks

221.9 Memory Management
Data Structure(Variable length)
Fixed,16KB

Fixed,16KB

Fixed,16KB

Fixed,16KB

2009/11/23

Ruby Memory Management Hacks

23

Memory Management
OS dependent Memory Managementy Make more small blocky Use OS dependent memory

management API

y mmap(), munmap(), madvise() on UNIXy Similar APIs on Windows

2009/11/23

Ruby Memory Management Hacks

24

1.9.2 Memory Management
Data Structure
Arena (128 MB)
Fixed, 4KB
Fixed, 4KB
Fixed, 4KB

2009/11/23

Ruby Memory Management Hacks

25

Memory Management
OS dependent Memory Managementy Many advantages

y Low object fragmentationy Easy to manage with OS M/M

y 4KB alignment page helps bitmapmarking
y 128MB bulk allocation helps
is_pointer_to_heap() checking(reduce GC overhead)
2009/11/23

Ruby Memory Management Hacks

26

Memory Management
OS dependent Memory ManagementEvaluation

yNot yetyHELP!

2009/11/23

Ruby Memory Management Hacks

27

(2) VM PerformanceImprovement
1. Change VM instruction set a little
2. Fiber performance improvement

2009/11/23

Ruby Memory Management Hacks28

(2.1) VM instruction set changes
y Make instance variable index cachein inline cache entry→ Need Instruction cache entry inoperand of getinstance/setinstancey Change instruction cache treatment
y This point is NOTICE for YARV

assembler users (… who use it?)2009/11/23

Ruby Memory Management Hacks29

(2.2) Fiber
Performance Improvementy Fiber is:

y 1.9 Featurey Coroutine, Semi-Coroutine

y Primitive for concurrent programmingy Used in Enumerator#next

y Problem is:y Slow context switching

2009/11/23

Ruby Memory Management Hacks

30

FiberCurrent implementationy Fiber context switching needs

machine stack copy

HeapspMachineStackMachineStack
Fiber B
Fiber AMachineStack
Fiber B

MachineStack
Fiber A(buffer)

Copy buffer (B)Copy buffer (A)2009/11/23

Ruby Memory Management Hacks

31

FiberCurrent implementationy Why such a expensive method?
y The reason is Portabilityy There are no general method to switch

machine stacky Assembler approach has portability,maintainability problems.
•e.g. [ruby-core:14149]

2009/11/23

Ruby Memory Management Hacks

32

Fiber performance improvement
Solutiony Use OS dependent featuresy OS features are well-maintained
y getcontext/setcontext on UNIX

•Linux (2.6-), Solaris
FreeBSD, Mac OSXy Fiber API on Windowsy And machine stack copy for other OSs
2009/11/23

Ruby Memory Management Hacks

33

Fiber performance improvement
Switching SP with OS featuresy [1.9.2 Feature] Lightweight fiber

context switching
HeapspMachineStackMachine
Stack

Fiber B
Fiber A

2009/11/23

Ruby Memory Management Hacks

34

Fiber performance evaluation
Context switch cost (sec)
Platform

Machine stack depth
0

1

2

3

Windows (1.9.1)

5.304

5.226

5.694

6.021

Windows (1.9.2)

1.872

1.887

1.903

1.887

Linux (1.9.1)

4.882

5.587

6.398

7.392

Linux (1.9.2)

1.675

1.675

1.673

1.661

Solaris (1.9.1)

130.995

137.903144.062151.819

Solaris (1.9.2)

42.50742.40842.05642.357

2009/11/23

Ruby Memory Management Hacks

35

Fiber performance evaluation
Context switch cost (speedup ratio)
5
4.5
4
3.5
3
2.5
2
1.5
1
0.5
0

WindowsLinuxSolaris0
2009/11/23

12
Ruby Memory Management Hacks

336

Fiber performance
Proc vs. Fiber# Procpr = Proc.new{};
10000000.times{pr.call}# Fiberf = Fiber.new{
Fiber.yield while true};
10000000.times{f.resume}
2009/11/23

Ruby Memory Management Hacks

37

Fiber performanceProc vs. Fiber
1816

Execution time14121086420
Proc
2009/11/23

Fiber (1.9.1)
Ruby Memory Management Hacks

Fiber (1.9.2)
38

Fiber performance
Fiber generation speed
y 3000000.times do

Fiber.new{}.resumeendy Ruby 1.9.1
y User time : 13.649
y System time:0.008
y Real time:
13.657y Ruby 1.9.2:
y User time: 10.541
y System time:1.136
y Real time:
11.677
2009/11/23

Most of time consumed
by GC overhead.
This is future work.

in SecRuby Memory Management Hacks

39

(3) Multi-VM (MVM)y Overview
y Motivationy Modely Challenges
y API Designy Progressy Features
y Evaluations
2009/11/23

Ruby Memory Management Hacks

40

(3) Multi-VM (MVM)

This one!

Quoted from Matz keynote
2009/11/23

Ruby Memory Management Hacks

41

MVM
Motivationy Parallel executiony Multi-core, many core, …

y Parallel thread seems bad ideay Isolated VMs can run in parallely Application embedded VM

y Easy to handle VM by your application

2009/11/23

Ruby Memory Management Hacks

42

Model of MVM
Ruby
Process

Ruby VM (under GVL)

Ruby VM (under GVL)
RT

Native Thread
System S/W・・・RTRTNT

NT・・・

NT

Thread Scheduler

S/W
H/W
Processor(s)PEPE・・・

PE

PE: Processor Element, NT: Native Thread
2009/11/23

Ruby Memory Management Hacks

43

MVMChallengesy Compatibility

y Process global information• C global variables• File related information (e.g. w/dir)• Process information (including signal)• C extensiony Communication interface design

2009/11/23

Ruby Memory Management Hacks

44

MVM
API
y C API /* include/ruby/vm.h */y ruby_vm_create(…)
y ruby_vm_run(…)

y ruby_vm_start(…)

y Ruby API # not mature

y vm = RubyVM.new(…)

y vm.push(…) / vm.pop() # comm.
y Now under consideration
2009/11/23

Ruby Memory Management Hacks

45

MVMC Extension
y Introduce new convention
y Init_foo(void)y InitVM_foo(void) ← New!

y Do not use global variable

y If C extension doesnʼt have

InitVM_foo() function, restricted touse by only main (first) VM

2009/11/23

Ruby Memory Management Hacks

46

MVM Progress
Current Featuresy Create VM by Ruby/C API
y Run in parallel each VMs

y Simple VM communication API
y Not mature

2009/11/23

Ruby Memory Management Hacks

47

MVM - Preliminary evaluation
Evaluation environmenty MacBooky Intel Core 2 Duo 2.1 GHz, 2 Coresy L2$ 3 MB

y Memory 4 GB

y Bus speed 800 MHz

2009/11/23

Ruby Memory Management Hacks

48

MVM - Preliminary evaluation
VM creation time
14

Creation time (sec)121086420vm
2009/11/23

forksystem

Ruby Memory Management Hacks

threadfiber49

MVM Progress
Evaluation (ping/pong)require 'benchmark'N = ARGV[0] ? ARGV[0].to_i : 100
puts "N = #{N}"W = 10Benchmark.bm(W) {|bm|vm = RubyVM.new("ruby", "-e", %q{
vm1 = RubyVM.current
vm2 = RubyVM.parent#{N}.times {vm2.send(vm1.recv)}}).start
data = :"foo"bm.report("send/recv") {N.times {|i| vm.send(data)}; vm.join}}2009/11/23

Ruby Memory Management Hacks

50

MVM Progress
Evaluation (symbol ping/pong)
Execution time (sec)

80
70
60
50
40
30
20
100
MVM
2009/11/23

Ruby Memory Management Hacks

dRuby51

MVM Progress
Evaluation (parallel execution)
fibdef = <<EOS
def fib(n)if n<2nelse
fib(n-2)+fib(n-1)
endendEOSeval fibdef
require 'benchmark'N = ARGV[0] ? ARGV[0].to_i : 100
F = ARGV[1] ? ARGV[1].to_i : 30
puts "N = #{N}, F = #{F}"W=8

2009/11/23

Benchmark.bm(W) {|bm|script = fibdef + %q{
vm1 = RubyVM.current
vm2 = RubyVM.parent
vm2.send(fib(vm1.recv))}
bm.report("parallel") {(1..N).map{|i| RubyVM.new('ruby', '-e',script).start}.each {|vm|vm.send(F)}.each{|vm|vm.join}}bm.report("serial") {(1..N).map{|i| fib(F)}}}Ruby Memory Management Hacks

52

MVM Progress Evaluation
Parallel execution on 2 cores
90
Execution Time (sec)807060504030
20100parallel
2009/11/23

Ruby Memory Management Hacks

serial53

Summaryy Ruby 1.9.2 may improve severalperformance. Stay tuned.

y Memory management efficiencies
y Fiber performancey Memory profiler featurey Researches are progressingy Memory Profiler
y MVM
2009/11/23

Ruby Memory Management Hacks

54

Fin.
Thank You for Your Attention.
Any Questions?
SASADA Koichi<ko1@rvm.jp>

Department of Creative Informatics,
Graduate School of Science and Technology,
The University of Tokyo

2009/11/23

Ruby Memory Management Hacks

55