ManageIQ/manageiq — pull request #1070 by jrafanie (2014-11-17 17:46:18+00:00)
Title: [POC/WIP] Very rough forking workers support

DO NOT MERGE

This is a POC of leveraging ruby's fork + ruby 2.0's copy on write friendliness.
This PR currently only implements forking of the non-ui and non-webservice worker processes.
Additionally, this PR doesn't clean up open IO objects such as open log handles, pipes, etc. in the forked subprocess. It also doesn't handle at_exit handlers that are inherited from the parent process into the child process.

I measured total memory usage at the point when all workers have been started via the "workers have been started" message in the evm.log.

## Before this PR:
### system memory usage is at ~2.6 GB:

![screen shot 2014-11-17 at 12 13 58 pm](https://cloud.githubusercontent.com/assets/19339/5074110/5d3414ba-6e55-11e4-9d51-692ec9e7522b.png)

## With this PR (many forking workers):
### system memory usage is at **~2.2 GB**:

![screen shot 2014-11-17 at 12 17 38 pm](https://cloud.githubusercontent.com/assets/19339/5074107/56c49758-6e55-11e4-9a87-8eaeca0fa33f.png)

Additionally, the workers that fork start much faster.

## Before this PR:

### Time from database seeding completes until the first worker finish starting is 37 seconds:
```
[----] I, [2014-11-17T17:11:52.615989 #16755:8ebea8] INFO -- : EvmDatabase.seed Seeding... Complete
[----] I, [2014-11-17T17:12:29.686321 #16960:5e1e9c] INFO -- : EmsRefreshCoreWorker started. ID [107], PID [16960], GUID [d5c746c8-6e7c-11e4-b751-005056af022c], Zone [default], Role [database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services]
[----] I, [2014-11-17T17:12:34.220275 #16986:11c3ea4] INFO -- : PriorityWorker started. ID [112], PID [16986], GUID [db627602-6e7c-11e4-b751-005056af022c], Zone [default], Role [database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services]
[----] I, [2014-11-17T17:12:37.350426 #17003:969eac] INFO -- : UiWorker started. ID [115], PID [17003], GUID [df6f7dc6-6e7c-11e4-b751-005056af022c], Zone [default], Role [database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services]
[----] I, [2014-11-17T17:12:39.679856 #17024:ad7e9c] INFO -- : WebServiceWorker started. ID [117], PID [17024], GUID [e2ee7dda-6e7c-11e4-b751-005056af022c], Zone [default], Role [database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services]
[----] I, [2014-11-17T17:12:42.442192 #16998:7f3e9c] INFO -- : ScheduleWorker started. ID [114], PID [16998], GUID [ddff6fa0-6e7c-11e4-b751-005056af022c], Zone [default], Role [database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services]
[----] I, [2014-11-17T17:12:49.494118 #16992:12abeac] INFO -- : ReportingWorker started. ID [113], PID [16992], GUID [dcbea85e-6e7c-11e4-b751-005056af022c], Zone [default], Role [database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services]
[----] I, [2014-11-17T17:12:54.296973 #16982:aa1ea0] INFO -- : GenericWorker started. ID [111], PID [16982], GUID [da483a72-6e7c-11e4-b751-005056af022c], Zone [default], Role [database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services]
```

## With this PR (many forking workers):
### Time from database seeding completes until the first worker finish starting is 6 seconds:
```
[----] I, [2014-11-17T17:16:46.938753 #17656:fbdeac] INFO -- : EvmDatabase.seed Seeding... Complete
[----] I, [2014-11-17T17:16:52.226431 #17864:fbdeac] INFO -- : EmsRefreshCoreWorker started. ID [118], PID [17864], GUID [84c0cd48-6e7d-11e4-9673-005056af022c], Zone [default], Role [database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services]
[----] I, [2014-11-17T17:16:56.280425 #17960:fbdeac] INFO -- : GenericWorker started. ID [122], PID [17960], GUID [87a5cab8-6e7d-11e4-9673-005056af022c], Zone [default], Role [database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services]
[----] I, [2014-11-17T17:16:57.565550 #17966:fbdeac] INFO -- : PriorityWorker started. ID [123], PID [17966], GUID [8875d0e6-6e7d-11e4-9673-005056af022c], Zone [default], Role [database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services]
[----] I, [2014-11-17T17:16:59.263124 #17972:fbdeac] INFO -- : ReportingWorker started. ID [124], PID [17972], GUID [8945c224-6e7d-11e4-9673-005056af022c], Zone [default], Role [database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services]
[----] I, [2014-11-17T17:17:00.857847 #17981:fbdeac] INFO -- : ScheduleWorker started. ID [125], PID [17981], GUID [8a479670-6e7d-11e4-9673-005056af022c], Zone [default], Role [database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services]
[----] I, [2014-11-17T17:17:10.741758 #17989:11bbeac] INFO -- : UiWorker started. ID [126], PID [17989], GUID [8b182786-6e7d-11e4-9673-005056af022c], Zone [default], Role [database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services]
[----] I, [2014-11-17T17:17:12.438203 #18008:124bea8] INFO -- : WebServiceWorker started. ID [128], PID [18008], GUID [8c9b1f00-6e7d-11e4-9673-005056af022c], Zone [default], Role [database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services]
[----] I, [2014-11-17T17:17:19.020029 #18002:fbdeac] INFO -- : VimBrokerWorker started. ID [127], PID [18002], GUID [8bb831e0-6e7d-11e4-9673-005056af022c], Zone [default], Role [database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services]
```