Repository: pantsbuild/pants

Issue:
usability issues with --changed-parent=HEAD
In some cases I'd like to be able to make a bunch of changes in my repo, then compile just the targets that are changed.  I've been playing with --changed-parent=HEAD and while it works in some cases, it gives errors in other cases that prevent it from being effective.

For example, if I create a new file in a new directory (with no BUILD file), I get an error:

    mkdir bin
    echo "hi" > bin/README.txt
    ./pants --changed-parent=HEAD list 
    Exception caught: (<class 'pants.engine.mapper.ResolveError'>)

    Exception message: Path "bin" contains no BUILD files.

If I delete a directory with a BUILD file, I also get an error that the BUILD file doesn't exist (it doesn't anymore).

If I change a source file that is referenced by an existing BUILD file, it works as expected (listing/compiling the target affected).

I would expect if I delete a directory/file, or modify/add a file somewhere that's not referenced by a pants build, pants would just ignore these changes and list/compile any remaining targets that are modified.





Discussion:
@youngderekm I don't repro on pantsbuild/pants master. Can you verify a few things?

First, my repro attempt used `./pants --pants-config-files=pants.daemon.ini ...` - I assume from the problem ballpark you're using the daemon, which enables watchman file change notifications. To check you can run `./pants options --name=enable_pantsd`, I get:
```
$ ./pants --pants-config-files=pants.daemon.ini options --name=enable_pantsd 2>/dev/null
enable_pantsd = True (from CONFIG in pants.daemon.ini)
```

Can you also report `./pants --version` and re-run with the added option `--print-exception-stacktrace`? This will provide a bit more detail on the error and that detail may help in debugging.

noting that I also don't repro (without the daemon) in master:

```
[omerta pants (master)]$ git rev-parse HEAD
b170aeb3fac52502ff248cf7ceac6172ad6fc75b
[omerta pants (master)]$ mkdir bin
[omerta pants (master)]$ echo "hi" > bin/README.txt
[omerta pants (master)]$ ./pants --changed-parent=HEAD list
WARN] /Users/kwilson/dev/pants/src/python/pants/backend/graph_info/tasks/listtargets.py:72: DeprecationWarning: DEPRECATED: `./pants list` (with no explicit targets) will soon become an error. Please specify one or more explicit target specs (e.g. `./pants list ::`). will be removed in version 1.5.0.dev0.
  for target in self.determine_target_roots('list', lambda target: not target.is_synthetic):

[omerta pants (master)]$ 
```
Hi.  I'm sorry, I forgot to specify that this was while running v1.3.0.  I should have tested on master as well.

________________________________
From: Kris Wilson <notifications@github.com>
Sent: Tuesday, August 29, 2017 4:25:19 PM
To: pantsbuild/pants
Cc: Derek Young; Mention
Subject: Re: [pantsbuild/pants] usability issues with --changed-parent=HEAD (#4833)


noting that I also don't repro (without the daemon) in master:

[omerta pants (master)]$ git rev-parse HEAD
b170aeb3fac52502ff248cf7ceac6172ad6fc75b
[omerta pants (master)]$ mkdir bin
[omerta pants (master)]$ echo "hi" > bin/README.txt
[omerta pants (master)]$ ./pants --changed-parent=HEAD list
WARN] /Users/kwilson/dev/pants/src/python/pants/backend/graph_info/tasks/listtargets.py:72: DeprecationWarning: DEPRECATED: `./pants list` (with no explicit targets) will soon become an error. Please specify one or more explicit target specs (e.g. `./pants list ::`). will be removed in version 1.5.0.dev0.
  for target in self.determine_target_roots('list', lambda target: not target.is_synthetic):

[omerta pants (master)]$


—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<https://github.com/pantsbuild/pants/issues/4833#issuecomment-325791940>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ADw2iSCqMCoJ7D3HMHUKBhMHyvxw7yMNks5sdHOugaJpZM4PFC0z>.

@youngderekm  have you managed to solve this problem? We see that too...
Same problem showed up here, after upgraded from pants 1.2.0 to 1.3.0

```
Computing Select(AscendantAddresses(directory=u'services/autos/lambdas/scripts'), =Collection.of(HydratedTarget))
  Computing Task(<function transitive_hydrated_targets at 0x103f281b8>, AscendantAddresses(directory=u'services/autos/lambdas/scripts'), =Collection.of(HydratedTarget))
    Computing TransitiveDependencies( =HydratedTarget, =BuildFileAddresses)
      Computing Task(<function addresses_from_address_families at 0x103d126e0>, AscendantAddresses(directory=u'services/autos/lambdas/scripts'), =BuildFileAddresses)
        Throw(Path "services/autos/lambdas/scripts" contains no BUILD files.)
          Traceback (most recent call last):
            File "/Users/xinyu.liu/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.a6OiAb/install/lib/python2.7/site-packages/pants/engine/subsystem/native.py", line 410, in extern_invoke_runnable
              val = runnable(*args)
            File "/Users/xinyu.liu/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.a6OiAb/install/lib/python2.7/site-packages/pants/engine/build_files.py", line 244, in addresses_from_address_families
              raise ResolveError('Path "{}" contains no BUILD files.'.format(spec.directory))
          ResolveError: Path "services/autos/lambdas/scripts" contains no BUILD files.
```
Thanks for the reports, gang. Prioritizing this one.
I repro on `1.4.0.dev21` as well, but as alluded to in #4904, the only way to repro is by removing any `BUILD` files at the root of the repo.

Resolving patch:
diff --git a/src/python/pants/engine/build_files.py b/src/python/pants/engine/build_files.py
--- a/src/python/pants/engine/build_files.py
+++ b/src/python/pants/engine/build_files.py
@@ -239,8 +239,10 @@ def addresses_from_address_families(address_mapper, address_families, spec):
      - there were no matching AddressFamilies, or
      - the Spec matches no addresses for SingleAddresses.
   """
-  if not address_families:
-    raise ResolveError('Path "{}" contains no BUILD files.'.format(spec.directory))
+
+  def raise_if_empty_address_families():
+    if not address_families:
+      raise ResolveError('Path "{}" contains no BUILD files.'.format(spec.directory))
 
   def exclude_address(address):
     if address_mapper.exclude_patterns:
@@ -248,20 +250,22 @@ def exclude_address(address):
       return any(p.search(address_str) is not None for p in address_mapper.exclude_patterns)
     return False
 
-  if type(spec) in (DescendantAddresses, SiblingAddresses, AscendantAddresses):
-    addresses = tuple(a
-                      for af in address_families
-                      for a in af.addressables.keys()
-                      if not exclude_address(a))
+  def all_included_addresses():
+    return (a
+            for af in address_families
+            for a in af.addressables.keys()
+            if not exclude_address(a))
+
+  if type(spec) in (DescendantAddresses, SiblingAddresses):
+    raise_if_empty_address_families()
+    addresses = tuple(all_included_addresses())
   elif type(spec) is SingleAddress:
-    # TODO Could assert len(address_families) == 1, as it should always be true in this case.
-    addresses = tuple(a
-                      for af in address_families
-                      for a in af.addressables.keys()
-                      if a.target_name == spec.name and not exclude_address(a))
-    if not addresses:
-      if len(address_families) == 1:
-        _raise_did_you_mean(address_families[0], spec.name)
+    raise_if_empty_address_families()
+    addresses = tuple(a for a in all_included_addresses() if a.target_name == spec.name)
+    if not addresses and len(address_families) == 1:
+      _raise_did_you_mean(address_families[0], spec.name)
+  elif type(spec) is AscendantAddresses:
+    addresses = tuple(all_included_addresses())
   else:
     raise ValueError('Unrecognized Spec type: {}'.format(spec))
 


Test patch:
diff --git a/tests/python/pants_test/engine/legacy/test_changed_integration.py b/tests/python/pants_test/engine/legacy/test_changed_integration.py
--- a/tests/python/pants_test/engine/legacy/test_changed_integration.py
+++ b/tests/python/pants_test/engine/legacy/test_changed_integration.py
@@ -26,6 +26,14 @@ def lines_to_set(str_or_list):
     return set(x for x in str(str_or_list).split('\n') if x)
 
 
+def create_file_in(worktree, path, content):
+  """Creates a file in the given worktree, and returns its path."""
+  write_path = os.path.join(worktree, path)
+  with safe_open(write_path, 'w') as f:
+    f.write(dedent(content))
+  return write_path
+
+
 @contextmanager
 def mutated_working_copy(files_to_mutate, to_append='\n '):
   """Given a list of files, append whitespace to each of them to trigger a git diff - then reset."""
@@ -79,10 +87,7 @@ def create_isolated_git_repo():
   with temporary_dir(root_dir=get_buildroot()) as worktree:
     def create_file(path, content):
       """Creates a file in the isolated git repo."""
-      write_path = os.path.join(worktree, path)
-      with safe_open(write_path, 'w') as f:
-        f.write(dedent(content))
-      return write_path
+      return create_file_in(worktree, path, content)
 
     def copy_into(path, to_path=None):
       """Copies a file from the real git repo into the isolated git repo."""
@@ -426,6 +431,13 @@ def test_changed_with_deleted_resource(self):
       self.assert_success(pants_run)
       self.assertEqual(pants_run.stdout_data.strip(), 'src/python/sources:text')
 
+  def test_changed_in_directory_without_build_file(self):
+    with create_isolated_git_repo() as worktree:
+      create_file_in(worktree, 'new-project/README.txt', 'This is important.')
+      pants_run = self.run_pants(['list', '--changed-parent=HEAD'])
+      self.assert_success(pants_run)
+      self.assertEqual(pants_run.stdout_data.strip(), '')
+
   def test_list_changed(self):
     deleted_file = 'src/python/sources/sources.py'