Tidy up archive initialisation code

Improve code formatting and readability in _set_archive() ahead
of new temporary tree archive code.

Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
Make SoSReport establish report directory structure

Allow clients of the Archive classes to create arbitrary
directories. Sos needs to be able to do this to create the report
directory structure within a temporary file system tree.

This work also paves the way for improviyng the robustness of the
tree copying code in Plugin (which presently relies on the Archive
class to recurse up to root, adding parent directories that do not
already exist in the archive to ensure the corret permissions and
context are stored in the archive). This would also allow the
Archive.add_parent() hack to be removed.

Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
Fix make_path for absolute destination paths

Pythons os.path.join behaviour returns only the last path given
when that path is absolute. This means that when make_path() is
given an absolute destination path (e.g. /etc/foo) it will return
an archive path of '/etc/foo' causing an error when copying into
the temporary tree.

Check for absolute paths and use lstrip to remove leading
instances of os.sep from the string.

Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
Fix Plugin.add_copy_spec_limit() checks

The logic in add_copy_spec_limit() only attempted to collect a
'tailed' version of a file that is over the size limit in the case
that it is the first file in the list of specs.

Change this to always collect the tailed version for the first
file to exceed the size limit and improve the variable naming
to be more consistent with related methods.

Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
Move TempFileUtil initialisation to be after policy set up

Policies may wish to influence the choice of temporary directory.
Move the TempFileUtil initialisation to be after the call to
load policy.

Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
Introduce new FileCacheArchive class

Add a new level to the Archive class hierarchy. This class may
be used by any archive class that needs to temporarily cache
files in the file system during collection.

Classes deriving from FileCacheArchive must supply
_build_archive() and _compress() methods which will be called
during archive finalization.

This fixes the TarFileArchive problems caused by attempting to
apply read-modify-write updates to content already present in the
archive.

Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
Fix use of 'logsize' parameter in logs module

The logs module has a paramter 'logsize' but some code still uses
the old 'syslogsize' name. Make all uses consistent.

Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
Add Archive.cleanup()

Add a cleanup() method to the Archive classes and ensure that it
is called during normal and abnormal termination (unless the pdb
debugger has been invoked during --debug mode).

Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
Move tmp directory selection to policy class

Since some distributions may be using features like tmp-on-tmpfs
that can cause problems for sos data collection move the selection
of this directory into the policy class.

The new behaviour respects options passed on the command line but
will ignore any environment variables. Users wishing to override
the location of sos' temporary files should use the command line
option.

Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
Add get_cmd_dir() method to Plugin and make plug-ins use it

As reported in Issue #140 plug-ins that need to write to an
arbitrary location in the archive are broken by the in-line tar
archive changes.

Introduce a 'get_tmp_dir()' method to the Archive classes that
must return a writable directory that is included in the archive.

Archives deriving from FileCacheArchive simply retun the root of
the archive tree.

Add a new method to Plugin to use this support, 'get_cmd_dir()',
that will return 'Archive.get_tmp_dir() + 'sos_commands'
+ self.name()'.

Fixes problems in lvm2, cloudforms, satellite and rhui.

Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
Revert use of PAX archives

Revert the use of POSIX PAX archive format in TarFileArchive. The
code to capture SELinux file contexts is currently disabled in
master due to the undesirable affects on archive extraction. Since
this is the main reason for using the PAX format revert to
automatic format selection to allow the widest compatibility.

Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
Set umask when creating final archive file

The temporary trees used by sos are set up with correct
permissions but tarfile.open() uses the default mode and umask
settings.

Set the umask to deny group and other permissions for the archive
to avoid leaking data to unprivileged users.

Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
Test for symlink existence before creation

There are cases where we may attempt to add a symlink to the
archive more than once. Since this will fail for paths that
already exist test for their presence first.

Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
Spec file updates
Bump version

Increment the spec file version to 3.0