cloudfoundry/cf-release — pull request #469 by gberche-orange (2014-07-28 13:44:18+00:00)
Title: Fixing too small vsphere disk sizing

The current vsphere templates fail with v173 for clock_global

```
Error 450001: Failed to install package 'buildpack_ruby': System call error: No space left on device - /var/vcap/data/tmp/0a9c949f-ccf0-46a3-80a7-7e59e6439b8220140723-632-19tszpe: ["/var/vcap/bosh/lib/ruby/gems/1.9.1/gems/bosh_agent-1.2366.0/lib/bosh_agent/apply_plan/package.rb:53:in `install_failed'",
"/var/vcap/bosh/lib/ruby/gems/1.9.1/gems/bosh_agent-1.2366.0/lib/bosh_agent/apply_plan/package.rb:36:in `rescue in install_for_job'",
```
and cf vms display 100% ephemeral disk usage on clock_global/0 93.0% (934.1M) swap usage on api_z1/0

The clock_global job is currently using a medium_z1 pool which for vsphere results into a 2048 MB drive. Increasing it to 40492 (given that 1GB is used up in the swap partition) leaves a 73% disk space occupation.

```
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 2.9G 1.1G 1.7G 40% /
none 493M 156K 493M 1% /dev
none 500M 0 500M 0% /dev/shm
none 500M 56K 500M 1% /var/run
none 500M 0 500M 0% /var/lock
none 500M 0 500M 0% /lib/init/rw
none 2.9G 1.1G 1.7G 40% /var/lib/ureadahead/debugfs
/dev/sdb2 3.0G 2.1G 796M 73% /var/vcap/data
/dev/loop0 124M 5.6M 118M 5% /tmp

root@cc8f3638-a5e6-4698-a69c-6fcc8f8a4859:~# fdisk -l
Disk /dev/sda: 3221 MB, 3221225472 bytes
4 heads, 32 sectors/track, 49152 cylinders
Units = cylinders of 128 * 512 = 65536 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0006769f
Device Boot Start End Blocks Id System
/dev/sda1 1 46860 2998992 83 Linux
Disk /dev/sdb: 4290 MB, 4290772992 bytes
255 heads, 63 sectors/track, 521 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sdb1 1 128 1028159+ 82 Linux swap / Solaris
/dev/sdb2 129 521 3156772+ 83 Linux
```

In comparison, here is the ephemeral disk size for the small_z1 and medium_z1 pools mapping to m1.small and m1.medium in [AWS EC2 sizes](http://aws.amazon.com/ec2/previous-generation/)
```
m1.small = 1 x 160 GB disk, 1.7 GB RAM
m1.medium = 1 x 410 GB disk, 3.75 GB RAM
```

The same problem exists for the api_worker_z1 job, which is currently mapped to the small_z1 pool.

```
Error 450001: Failed to install package 'buildpack_ruby': System call error: No space left on device - /var/vcap/data/tmp/0a9c949f-ccf0-46a3-80a7-7e59e6439b8220140724-630-ey7x3r: ["/var/vcap/bosh/lib/ruby/gems/1.9.1/gems/bosh_agent-1.2366.0/lib/bosh_agent/apply_plan/package.rb:53:in `install_failed'",
```

Reminder of current cf-infrastructure-sphere.yml

```
- name: small_z1
cloud_properties:
ram: 1024
disk: 2048
cpu: 1

- name: medium_z1
cloud_properties:
ram: 1024
disk: 2048
cpu: 1
```

I'm suggesting to also raise small_z1 disk to 4096 MB (this should only overrisize a bit the loggregator clusters), rather than moving api_worker_z1 job from small_z1 to medium_z1 pool, which would increase too much RAM and disk usage into AWS/openstack deployments.

After this upgrade api_worker_z1 eph disk usage is at 73%

```
+------------------------------------+---------+---------------+----------------+-----------------------+------+------+------+----------------+---------------+------------+------------+------------+
| Job/index | State | Resource Pool | IPs | Load | CPU | CPU | CPU | Memory Usage | Swap Usage | System | Ephemeral | Persistent |
| | | | | (avg01, avg05, avg15) | User | Sys | Wait | | | Disk Usage | Disk Usage | Disk Usage |
+------------------------------------+---------+---------------+----------------+-----------------------+------+------+------+----------------+---------------+------------+------------+------------+
[...]
| api_worker_z1/0 | running | small_z1 | 192.168.26.89 | 0.11%, 0.08%, 0.05% | 0.2% | 0.0% | 1.0% | 24.7% (246.8M) | 0.0% (232.0K) | 40% | 73% | n/a |
[...]
```