Repository: iterative/dvc

Issue:
Custom credentialpath not working
Credentialpath is not working with the following config:

```
[core]
remote = models
cloud = AWS
['remote "models"']
url = s3://<bucket-name>
credentialpath = /home/ubuntu/Models/.dvc/aws_credentials
```

Works with the default path:

`credentialpath = /home/ubuntu/.aws/credentials`

The credentials files are identical.

Setup:

DVC: 0.17.0
Installation: pip3 install dvc[all]
OS: Ubuntu 16.04.4


Discussion:
Hi @JohannesTK !

> ```
> [core]
> remote = models
> cloud = AWS
> ['remote "models"']
> url = s3://<bucket-name>
> credentialpath = /home/ubuntu/Models/.dvc/aws_credentials
> ```

You don't actually need `cloud = AWS` line in your config. That line is from an old obsoleted version of dvc and doesn't do nothing when `remote =` is specified. So you can safely remove it. It surely didn't cause any issues in this case, but I just wanted to point out that that line is useless. :)

> Works with the default path:
> 
> credentialpath = /home/ubuntu/.aws/credentials
> 
> The credentials files are identical.

Could you please show us the output of the command that failed with the `--verbose` option added to it?

Thanks,
Ruslan 
```
Preparing to pull data from s3://sidekik-models
[##############################] 100% Collecting information
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/dvc/command/data_sync.py", line 26, in do_run
    all_branches=self.args.all_branches)
  File "/usr/local/lib/python3.5/dist-packages/dvc/project.py", line 420, in pull
    show_checksums=show_checksums)
  File "/usr/local/lib/python3.5/dist-packages/dvc/project.py", line 408, in fetch
    show_checksums=show_checksums)
  File "/usr/local/lib/python3.5/dist-packages/dvc/data_cloud.py", line 118, in pull
    show_checksums=show_checksums)
  File "/usr/local/lib/python3.5/dist-packages/dvc/remote/local.py", line 604, in pull
    show_checksums=show_checksums)
  File "/usr/local/lib/python3.5/dist-packages/dvc/remote/local.py", line 587, in _do_pull
    f.result()
  File "/usr/lib/python3.5/concurrent/futures/_base.py", line 398, in result
    return self.__get_result()
  File "/usr/lib/python3.5/concurrent/futures/_base.py", line 357, in __get_result
    raise self._exception
  File "/usr/lib/python3.5/concurrent/futures/thread.py", line 55, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.5/dist-packages/dvc/remote/s3.py", line 274, in download
    s3 = self.s3
  File "/usr/local/lib/python3.5/dist-packages/dvc/remote/s3.py", line 74, in s3
    session = boto3.session.Session(profile_name=self.profile)
  File "/usr/local/lib/python3.5/dist-packages/boto3/session.py", line 80, in __init__
    self._setup_loader()
  File "/usr/local/lib/python3.5/dist-packages/boto3/session.py", line 120, in _setup_loader
    self._loader = self._session.get_component('data_loader')
  File "/usr/local/lib/python3.5/dist-packages/botocore/session.py", line 729, in get_component
    return self._components.get_component(name)
  File "/usr/local/lib/python3.5/dist-packages/botocore/session.py", line 946, in get_component
    self._components[name] = factory()
  File "/usr/local/lib/python3.5/dist-packages/botocore/session.py", line 186, in <lambda>
    lambda:  create_loader(self.get_config_variable('data_path')))
  File "/usr/local/lib/python3.5/dist-packages/botocore/session.py", line 281, in get_config_variable
    elif self._found_in_config_file(methods, var_config):
  File "/usr/local/lib/python3.5/dist-packages/botocore/session.py", line 308, in _found_in_config_file
    return var_config[0] in self.get_scoped_config()
  File "/usr/local/lib/python3.5/dist-packages/botocore/session.py", line 385, in get_scoped_config
    raise ProfileNotFound(profile=profile_name)
botocore.exceptions.ProfileNotFound: The config profile (default) could not be found
```
Ah, ok. I was able to reproduce. Patch is comming soon and is going to be released today in 0.18.2.

Thank you so much!

Resolving patch:
diff --git a/dvc/remote/s3.py b/dvc/remote/s3.py
--- a/dvc/remote/s3.py
+++ b/dvc/remote/s3.py
@@ -57,6 +57,7 @@ def __init__(self, project, config):
         else:
             creds = {}
 
+        self.creds = creds
         self.region = creds.get('region', self.region)
         self.aws_access_key_id = creds.get('aws_access_key_id', None)
         self.aws_secret_access_key = creds.get('aws_secret_access_key', None)
@@ -71,12 +72,14 @@ def prefix(self):
 
     @property
     def s3(self):
-        session = boto3.session.Session(profile_name=self.profile)
-        return session.client('s3',
-                              aws_access_key_id=self.aws_access_key_id,
-                              aws_secret_access_key=self.aws_secret_access_key,
-                              region_name=self.region,
-                              endpoint_url=self.endpoint_url)
+        if not self.creds:
+            session = boto3.session.Session(profile_name=self.profile)
+        else:
+            session = boto3.session.Session(
+                             aws_access_key_id=self.aws_access_key_id,
+                             aws_secret_access_key=self.aws_secret_access_key,
+                             region_name=self.region)
+        return session.client('s3', endpoint_url=self.endpoint_url)
 
     def get_etag(self, bucket, key):
         try:


Test patch: