User sync: store statistics in the sync object

This allows add_new() to return the newly added user object.

Let pylint ignore KeyedTuple, since its properties are generated
dynamically, and pylint cannot infer them.

Change-Id: I646421386c27e91cbf487755c01d75efcd1444a0
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Don't eager-load Interface.hardware_entity by default

Change-Id: Ic90163314e5fd6740ba34e3094868c2fbaf353b5
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Don't eager-load Host.hardware_entity and Machine.host

Change-Id: Ifb89f5003724b8b7b5df0340cb401ac4883eb54b
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Don't eager-load cluster membership information

Change-Id: I51c44e2b69d1642124148209eea78235117980f6
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Improve user information parsing

Make the parser more robust by ignoring invalid lines. This means the
test data needs to be fixed, too.

Change-Id: I28a88b222a671da66436d2d13a259f4450afb493
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Better error message for UID collisions

Provide a more meaningful error message instead an ugly DB constraint
violation when refresh_user encounters UID collision in the source.

Change-Id: Ib98e1880837d4ebaa6e332ff11e939bee8619ce3
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
refresh_user: limit deletions

Do not allow deleting too many users, in case something bad happened
with the synchronization source.

Change-Id: Id878457b6a43acdf1c763825e1689b2e9fef31f7
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Make some join conditions more explicit

ARecord.fqdn_id (inherited from DNSRecord) and ARecord.reverse_ptr_id
are two possible foreign keys between ARecord and Fqdn. SQLAlchemy 0.9
no longer guesses which one to use, but requires specifying it.

Change-Id: I9c963ceecab0aee3604e892371772af286731ff2
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Fix host object removal

Setting HardwareEntity.host to None is both necessary and sufficient to
get the host object deleted. It is necessary, because otherwise the
.host attribute becomes stale if the hardware object remains present in
the session's identity map. It is sufficient due to the delete-orphan
cascade option in the model.

Without this change, refresh_windows_host breaks with SQLAlchemy 0.9.

Change-Id: Ic51d2eb2e1f3c9f02b05fc01def8cb23b79513d3
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
refresh_windows_hosts: Avoid an unneeded query

The extra query is not necessary since Host was changed to be bound to
HardwareEntity instead of Machine.

Change-Id: I901f1640c3abfddda419885abd005710687f85e2
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Mark Cluster.location_constraint_id as NOT NULL

The code enforces having a location constraint anyway, and this allows
using more efficient inner joins instead of outer joins.

Change-Id: I26f8ccae22c4e4839339d64e456d756ca952c21c
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Use the column object as the target of ForeignKey

This way Python does the resolution rather than SQLAlchemy. There are a
couple of cases when this change would result in a dependency cycle,
leave those unchanged - but those are now the exception rather than the
norm.

Change-Id: Iba1f3e760b8e1263cf0c75a5ce31b22356bd2482
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Load DnsMap.location lazily

All current cases where we work with DNS maps have the location loaded
already, so loading it again is just a waste of time.

Change-Id: Ie33fde5f7fcfc0e2769ee77eb5d2362117538407
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Prefix private DB classes with underscores

Classes which we use as many-to-many relations should not be used
directly, so make that more explicit in their names.

Change-Id: I4ebcc5fbb6cbc9b7dbf72a95f19997576ffdaf17
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Support legacy hardware feature template locations

Change-Id: Id51bad19668f325224bd2b970467df1d8268eb85
Reviewed-by: Priya Joshi <Priya.Joshi@morganstanley.com>
Fix detection of the default panc output format

The default format from the configuration file is applied only if there
is no explicit override on the command line. That logic needs to
differentiate between 3 states (True, False, None), so we have to
supress the default value argparse's 'store_true' and 'store_false'
actions would use without an explicit override.

Change-Id: Iff150217ad6292133110fe850380f7dd1d86a80e
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Merge remote-tracking branch 'origin/for_next/hw_feature_fix'
Merge remote-tracking branch 'origin/for_next/aq_compile_fix'
Merge remote-tracking branch 'origin/for_next/query_tuning'
Merge remote-tracking branch 'origin/for_next/duplicate_uid'