doebuild: fix bug #528272

The doebuild function has a boolean "unpacked" variable which it sets
to True if it determines that the "unpack" phase has already executed
and the last modification times of the distfiles are older than the
last modification time of WORKDIR. The code which sets the "unpacked"
flag does not need to run unless the current phase is supposed to have
a build directory. Therefore, disable this code for all phases that are
not supposed to have a build directory. This fixes incorrect behavior
of the "fetch" phase as reported in bug #528272.

X-Gentoo-Bug: 528272
X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=528272
Acked-by: Alexander Berntsen <bernalex@gentoo.org>
Add support for SUSE based distros in etc-update

Add support for reading/merging *.rpmnew files created in SUSE based distros.
Configuration from etc-update will be read:
- from portage config vars in Gentoo based distros
- from portage config vars in non Gentoo distros if portage is installed
- from /etc/sysconfig/etc-update in SUSE distros if portage is absent

Signed-off-by: Michal Hrušecký <miska@gentoo.org>
X-Gentoo-Bug: 456128
X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=456128
Acked-by: Brian Dolbec <dolsen@gentoo.org>
Acked-by: Zac Medico <zmedico@gentoo.org>
man/ebuild.5: document assert for bug #528760

X-Gentoo-Bug: 528760
X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=528760
Acked-by: Alexander Berntsen <bernalex@gentoo.org>
_selinux.setexec: improve failure message (525726)

This converts "OSError: [Errno 22] Invalid argument" into a more
meaningful error message that is designed to guide users in the
right direction. Also, add an import that was missing for the "sys"
module, since it is referenced in the error paths of many functions
in this module.

X-Gentoo-Bug: 525726
X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=525726
Acked-by: Brian Dolbec <dolsen@gentoo.org>
Acked-by: Zac Medico <zmedico@gentoo.org>
portageq: fix eroot parameter (bug #529200)

The portageq eroot parameter has been broken since commit
c9f6aa9f0151adb3c86706eaef1914cdbdcf2b6d, due to premature instantiation
of portage.settings (before the ROOT variable was set). Premature access
to the portage.settings attribute must be avoided by using other
available means to determine the EPREFIX.

Fixes: c9f6aa9f0151 ("Add cross-prefix support")
X-Gentoo-Bug: 529200
X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=529200
Acked-by: Alexander Berntsen <bernalex@gentoo.org>
fs_template._ensure_dirs: handle EEXIST (529120)

There was a race inside fs_template._ensure_dirs which could cause it to
raise EEXIST if a concurrent process created the directory after
os.path.exists returned False. Fix it by using the util.ensure_dirs
function, which already handles EEXIST.

X-Gentoo-Bug: 529120
X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=529120
Acked-by: Alexander Berntsen <bernalex@gentoo.org>
unprivileged mode: generate PORTAGE_DEPCACHEDIR

For unprivileged mode, if PORTAGE_DEPCACHEDIR is unset and the default
PORTAGE_DEPCACHEDIR setting does not refer to a writable directory
(or there are not sufficient permissions to create it), then
automatically make PORTAGE_DEPCACHEDIR relative to the current target
root (which should always be writable for unprivileged mode). Also, in
create_trees, propagate the automatically generated PORTAGE_DEPCACHEDIR
setting to the config instance that is instantiated for ROOT = "/".

The result is that unprivileged mode will get a writable
PORTAGE_DEPCACHEDIR by default, and the default can be overridden by
setting the PORTAGE_DEPCACHEDIR variable.

Fixes: 1364fcd89384 ("Support unprivileged mode for bug #433453.")
X-Gentoo-Bug: 433453
X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=433453
Acked-by: Alexander Berntsen <bernalex@gentoo.org>
_slot_operator_update_probe: fix bug #528610

This fixes a case inside _slot_operator_update_probe where it would
select an inappropriate replacement_parent of a lower version than
desired. The problem is solved by rejecting replacement_parent if its
version is lower than the existing parent, and a downgrade is not
desired.

X-Gentoo-Bug: 528610
X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=528610
Acked-by: Brian Dolbec <dolsen@gentoo.org>
unprivileged mode: use first_existing helper func

Split out a first_existing function, in order to improve logic related
to the _unprivileged_mode function so that it checks whether it's
possible to create the specified target root (instead of requiring that
the target root already exists).

Acked-by: Alexander Berntsen <bernalex@gentoo.org>
unprivileged mode: fix cross-prefix support

In commit 1364fcd89384c9f60e6d72d7057dc00d8caba175, EROOT calculation
in portage.data did not account for cross-prefix support. This is
fixed by using new _target_root and _target_eprefix functions to
perform the calculation. The _target_eprefix function is also useful
in portageq, where the target EPREFIX needs to be known before
portage.settings is instantiated.

Fixes 1364fcd89384 ("Support unprivileged mode for bug #433453.")
Acked-by: Brian Dolbec <dolsen@gentoo.org>
NewsManager.getUnreadItems: handle EROFS (490732)

When getUnreadItems tries to lock the news.unread file, it's safe to
ignore EROFS. This is handled with a ReadOnlyFileSystem exception
raised from the portage.locks.lockfile function.

X-Gentoo-Bug: 490732
X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=490732
Acked-by: Brian Dolbec <dolsen@gentoo.org>
dblink: case insensitive support for bug #524236

This adds a dblink._contents attribute with methods that provide
an interface for contents operations with "implicit" case handling.
The new methods are implemented in a separate
ContentsCaseSensitivityManager class, in order to avoid adding more
bloat to vartree.py.

X-Gentoo-Bug: 524236
X-Gentoo-Url: https://bugs.gentoo.org/show_bug.cgi?id=524236
Acked-by: Brian Dolbec <dolsen@gentoo.org>
FEATURES=case-insensitive-fs for bug #524236

When case-insensitive-fs is enabled in FEATURES, the dblink.isowner
method, _owners_db class, and ConfigProtect class will be
case-insensitive. This causes the collision-protect and unmerge code
to behave correctly for a case-insensitive file system. If the file
system is case-insensitive but case-preserving, then case is preserved
in the CONTENTS data of installed packages.

X-Gentoo-Bug: 524236
X-Gentoo-Url: https://bugs.gentoo.org/show_bug.cgi?id=524236
Acked-by: Brian Dolbec <dolsen@gentoo.org>
dep_zapdeps: avoid use.mask/force changes (515584)

This patch causes dep_zapdeps to check which USE flags cause a match
to fail, and uses that information to prioritize choices that do not
require changes to use.mask or use.force.

X-Gentoo-Bug: 515584
X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=515584
Acked-by: Brian Dolbec <dolsen@gentoo.org>
Introduce eqatag to output proper machine-readable QA logs

The eqatag command syntax conforms to pre-GLEP describing
install-qa-check.d. The qa.log file format strictly conforms to YAML for
easy machine parsing.
Update the QA checks to new eqatag API
Remove __eqalog & __eqawarnlog

Replaced by eqatag.
emerge: check for writable /var/db/pkg (490732)

If there are packages to be merged or unmerge, then bail out early
if /var/db/pkg is not writable (in order to avoid a fatal EROFS error
which would otherwise occur later on). Behavior remains unchanged for
--pretend mode. For --ask mode, it will bail out just after the last
relevant --ask prompt.

In contrast to the writeable_check module, which operates on files just
before they are merged, the new vardbapi.writable check is performed
before anything has been built (much earlier).

X-Gentoo-Bug: 490732
X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=490732
Acked-by: Brian Dolbec <dolsen@gentoo.org>
Add emerge --with-test-deps option for bug #520652

For packages matched by arguments, this option will pull in dependencies
that are conditional on the "test" USE flag, even if FEATURES=test is not
enabled for the matched packages. The state of the "test" USE flag is not
affected by this option. It only changes the effective dependencies
which are processed by the depgraph._add_pkg_deps method.

X-Gentoo-Bug: 520652
X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=520652
Acked-by: Alexander Berntsen <bernalex@gentoo.org>
emerge: check for writable PKGDIR (490732)

If there are packages to be merged and "buildpkg" or "buildsyspkg" is
enabled, then bail out early if PKGDIR is not writable (in order to
avoid a fatal EROFS error which would otherwise occur later on).
Behavior remains unchanged for --pretend, --fetchonly and
--fetch-all-uri. For --ask, it will bail out just after the last
relevant --ask prompt.

X-Gentoo-Bug: 490732
X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=490732
Acked-by: Alexander Berntsen <bernalex@gentoo.org>