hpcc-systems/HPCC-Platform — Issue #651: Thorslaves fail to start first time on clean multinode install
Author: lorraineachapman
Created: 2011-10-11 11:56:28+00:00
State: open

This issue contains the contents of bugzilla 85068.

INITIAL COMMENT BY LORRAINE:

I used the wizard defaults to create a multinode system. I master 1 thorslave.

On startup the command line reports OK, but preflight reports the thorslave as
down.

RKC reckons that maybe it fails to start first time? But I will let him comment
further...

COMMENT FROM RKC:

the runssh log on the master said the following:

00000000 2011-08-17 12:45:15 17516 17516 ssh result(1):
ERR: thorslave_6600: no process found
00000000 2011-08-17 12:45:16 17533 17533 ssh result(2):
ERR: mkdir: cannot create directory `/var/lib/HPCCSystems': Permission denied
cd: 1: can't cd to /var/lib/HPCCSystems/mythor
mkdir: cannot create directory `/var/log/HPCCSystems': Permission denied
/bin/sh: cannot create
/var/log/HPCCSystems/mythor/start_slave_08_17_2011_12_45_15.log: Directory
nonexistent

There was no process thorslave_6600 on the slave machine. However the other
errors suggested that perhaps the start on the master node (which was the same
machine as the dali...) had happened before the start on the slave node, which
was needed to setup some directories for the first time?

Just guessing.

This was done on two complete virgin Ubuntu VMs

COMMENT FROM PHILIP:

The /var/lib/HPCCSystems dir seems to have bad permissions from that error.

What are they currently set to?

COMMENT FROM RKC:

stopping then restarting thor caused the error to go away, which makes me
suspect a race between the init scripts on the two nodes (or similar)