Build environment updates

This pull contains commits to fix the OS X build and allow GitMake test to pass.
On Thu, Jul 10, 2014 at 5:19 PM, Mike Bland <mbland@acm.org> wrote:
> On Thu, Jul 10, 2014 at 4:15 PM, Mike Bland <mbland@acm.org> wrote:
>> On Thu, Jul 10, 2014 at 4:09 PM, Mike Bland <mbland@acm.org> wrote:
>>> On Thu, Jul 10, 2014 at 3:42 PM, Ben Laurie <notifications@github.com> wrote:
>>>>
>>>> In util/pl/unix.pl:
>>>>
>>>> > @@ -226,12 +228,6 @@ sub fixdeps
>>>> >  sub fixrules
>>>> >    {
>>>> >    my ($str) = @_;
>>>> > -
>>>> > -  # Compatible with -j...
>>>> > -  $str =~ s/^(\s+@?)/$1cd \$(TEST_D) && /;
>>>> > -  return $str;
>>>>
>>>> On 10 July 2014 19:38, Mike Bland <notifications@github.com> wrote: In util/pl/unix.pl: > @@ -226,12 +228,6 @@ sub fixdeps > sub fixrules > { > my ($str) = @_; > - > - # Compatible with -j... > - $str =~ s/^(\s+@?)/$1cd \$(TEST_D) && /; > - return $str; I said it worked because originally I wasn't using '-j', so it was working for me. When I used '-j 20', I didn't see the "cd: out.build: No such file or directory" errors and the shell return code was 0.
>>>
>>>> That's what's puzzling me - I get you could miss the text-only errors, but I got an actual make-stopping error, and I'm wondering why you didn't?
>>>
>>> There is other GitMake weirdness going on that I'm investigating,
>>> which I was going to raise in a separate thread later, but since it
>>> seems to be germane here...
>>>
>>> While beginning my investigation into your earlier report this
>>> morning, I noticed that in a brand new working directory identical to
>>> the latest OpenSSL master branch, if I do a 'GitMake -j 1',
>>> test_verify fails. If I then do a regular make, the subsequent
>>> 'GitMake -j 1 test_verify' will pass.
>>>
>>> It seems something similar is happening with 'GitMake -j 1 test' in
>>> general. I use git-new-workdir to have a private dir per branch. In my
>>> original FreeBSD 9.1 working dir for this 'build' branch, running
>>> 'GitMake -j 1 test' even now (after re-introducing the obsolete
>>> change), I'll see the 'cd' errors, but the make will return status
>>> zero. I have previously done a regular 'make' in this directory.
>>>
>>> Using the same, identical branch in a different workdir, without first
>>> running 'make', I see the failures *and* make returns an error.
>>>
>>> I'm spending this afternoon digging in to try to understand what the
>>> difference is between the two directories that accounts for this
>>> behavior. It's not obvious to me by diffing the output of 'find . >
>>> files.txt' for each dir.
>>
>> FWIW, here's the broken test_verify output I'm getting:
>>
>> test_verify test started
>> The following command should have some OK's and some failures
>> There are definitly a few expired certificates
>> /Users/msb/src/openssl-fresh/out.build/shlib_wrap.sh
>> /Users/msb/src/openssl-fresh/out.build/openssl verify -CApath
>> /Users/msb/src/openssl-fresh/certs/demo
>> /Users/msb/src/openssl-fresh/certs/demo/*.pem
>> WARNING: can't open config file: /usr/local/ssl/openssl.cnf
>> 140735297401696:error:0B06E06B:x509 certificate
>> routines:X509_get_pubkey_parameters:unable to find parameters in
>> chain:/Users/msb/src/openssl-fresh/crypto/x509/x509_vfy.c:2005:
>> /Users/msb/src/openssl-fresh/certs/demo/ca-cert.pem: C = AU, ST =
>> Queensland, O = CryptSoft Pty Ltd, CN = Test CA (1024 bit)
>> error 20 at 0 depth lookup:unable to get local issuer certificate
>> /Users/msb/src/openssl-fresh/certs/demo/dsa-ca.pem: C = AU, ST =
>> Some-State, O = Internet Widgits Pty Ltd, CN = CA
>> error 20 at 0 depth lookup:unable to get local issuer certificate
>> /Users/msb/src/openssl-fresh/certs/demo/dsa-pca.pem: C = AU, ST =
>> Some-State, O = Internet Widgits Pty Ltd, CN = PCA
>> error 18 at 0 depth lookup:self signed certificate
>> C = AU, ST = Some-State, O = Internet Widgits Pty Ltd, CN = PCA
>> error 10 at 0 depth lookup:certificate has expired
>> OK
>> /Users/msb/src/openssl-fresh/certs/demo/pca-cert.pem: C = AU, ST =
>> Queensland, O = CryptSoft Pty Ltd, CN = Test PCA (1024 bit)
>> error 18 at 0 depth lookup:self signed certificate
>> C = AU, ST = Queensland, O = CryptSoft Pty Ltd, CN = Test PCA (1024 bit)
>> error 10 at 0 depth lookup:certificate has expired
>> OK
>> *** Error code 2
>> 1 error
>
> Figured out the case of test_verify. 'make rehash' needs to be invoked
> to create symlinks in certs/demo, via 'make build_apps'. Hand-jamming
> this into a previously-failing GitMake-only working dir causes
> test_verify to pass. So it looks like a dependency disconnect I'll
> need to patch up somehow...
>
> Still working on finding out why the entire test set doesn't return an
> error code, though.

Got it. In a clean working dir, with this change, test{req,key}.pem
aren't being generated into out.build. If you do 'GitMake -j 1 -P
test_req' specifically, you'll see the prerequisite _fake1 target
(which generates the pem files) produce nothing yet still indicate a
successful result, which then causes test_req to break.

In my original working dir, the files were there from a previous build
before this change, so the test_req target continued to "pass" in
'GitMake -j' mode. Removing them caused the exact same failure and
output you described earlier.