Repository: edgexfoundry/device-sdk-go (go)

Issue:
Profiles didn't be created when setting URI for files with secure credentials  on Device > ProfilesDir
<!--🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅

Hello there! 😄

To expedite issue processing please search open and closed issues before submitting a new one.
Existing issues often contain information about workarounds, resolution, or progress updates.

Please do not remove sections of this template when submitting your issue.  If the section is not relevant to your issue, leave the section but enter "n/a" for not applicable.

Note some sections are marked "REQUIRED" and must be provided.  Never mark required sections "n/a".

🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅-->


# 🐞 Bug Report

### Affected Services [**REQUIRED**]
<!-- Can you pin-point one or more EdgeX services (Data, Metadata, Command, etc...) as the source of the bug? -->
<!-- ✍️edit: --> The issue is located in: device-service


### Is this a regression?

<!-- Did this behavior work correctly in the previous version? -->
<!-- ✍️--> No


### Description and Minimal Reproduction [**REQUIRED**]
1. Download compose file from [edgex-compose/docker-compose.yml](https://github.com/edgexfoundry/edgex-compose/blob/main/docker-compose.yml)
2. Add the following variables under device-virtual > environment to make no default profile and device are created
```
DEVICE_PROFILESDIR: ./res
DEVICE_DEVICESDIR: ./res
```
3. Add http server service to the compose file. http-server.yml and all test files are attached
[httpd.zip](https://github.com/edgexfoundry/device-sdk-go/files/12483362/httpd.zip)

4. Deploy EdgeX by above compose file.
5. Create the following secret to device-virtual
```
POST https://localhost:8443/device-virtual/api/v3/secret
{
   "apiVersion":"v3",
   "secretName":"httpserver",
   "secretData":[
      {
        "key":"type",
        "value":"httpheader"
      },
      {
        "key":"headername",
        "value":"Authorization"
      },
      {
        "key":"headercontents",
        "value":"Basic ZWRnZXh1c2VyOmVkZ2V4cGFzc3dk"
      }
   ]
}
```
6. Update variables Device > ProfilesDir and Device > DevicesDir with `secure credentials` URI from consul
```
ProfilesDir: http://httpd-auth:80/http_files/profile.json?edgexSecretName=httpserver
DevicesDir: http://httpd-auth:80/http_files/device.json?edgexSecretName=httpserver
```
7. Restart device-virtual

## 🔥 Exception or Error
<pre><code>
Got thhe following logs, but no device profile and device was created

level=INFO ts=2023-08-31T06:54:42.327025847Z app=device-virtual source=profiles.go:122 msg="Loading pre-defined Device Profiles from http://httpd-auth:80/test_files/profile.json?edgexSecretName=httpserver(4 files found)"
level=INFO ts=2023-08-31T06:54:42.329944144Z app=device-virtual source=devices.go:137 msg="Loading pre-defined devices from http://httpd-auth:80/test_files/device.json?edgexSecretName=httpserver(1 files found)"


</code></pre>


## 🌍  Your Environment
**Deployment Environment:** Ubuntu 2204 / x86

**EdgeX Version [**REQUIRED**]:** Main


**Anything else relevant?**
If set variable value with `username-password URI`, the device profiles and devices are created successfully.

ex.
Update variables Device > ProfilesDir and Device > DevicesDir with `username-password URI` from consul
```
ProfilesDir: http://edgexuser:edgexpasswd@httpd-auth:80/http_files/profile.json
DevicesDir: http://edgexuser:edgexpasswd@httpd-auth:80/http_files/device.json
```
Restart device-virtual


Pull request description:
fix: Adjust GetFileType() function for new secret URI format
Closes #1504

<!-- Expected Commit Message Description (imported automatically by GitHub) -->
<!-- Must conform to [conventional commits guidelines](https://github.com/edgexfoundry/device-sdk-go/blob/main/.github/Contributing.md) -->
<!-- Expected Commit message must contain Closes/Fixes #IssueNumber statement when there is a related issue -->

<!-- Add additional detailed description of need for change if no related issue -->

**If your build fails** due to your commit message not passing the build checks, please review the guidelines here: https://github.com/edgexfoundry/device-sdk-go/blob/main/.github/Contributing.md

## PR Checklist
Please check if your PR fulfills the following requirements:

- [x] I am not introducing a breaking change (if you are, flag in conventional commit message with `BREAKING CHANGE:` describing the break)
- [x] I am not introducing a new dependency (add notes below if you are)
- [x] I have added unit tests for the new feature or bug fix (if not, why?)
- [x] I have fully tested (add details below) this the new feature or bug fix (if not, why?)
- [ ] I have opened a PR for the related docs change (if not, why?)
  <link to docs PR>

## Testing Instructions
<!-- How can the reviewers test your change? -->
1. Download compose file from edgex-compose
2. Modify the go.mod for device-virtual to add this replace `replace github.com/edgexfoundry/device-sdk-go/v3 => github.com/ejlee3/device-sdk-go/v3 v3.0.0-20230907221428-f6709ffcbf22` to use the new device-sdk.
3. Build the docker image for device-virtual.
3. Change the image name for device virtual to `docker.io/edgexfoundry/device-virtual:0.0.0-dev` and add the following variables to the edgex-compose/docker-compose.yml under device-virtual > environment to make no default profile and device are created
```
DEVICE_PROFILESDIR: ./res
DEVICE_DEVICESDIR: ./res
```
3. Add http server service to the compose file and modify the volume mounts to the local system path. http-server.yml and all test files are attached
[httpd.zip](https://github.com/edgexfoundry/device-sdk-go/files/12483362/httpd.zip)

4. Deploy EdgeX by above compose file.
5. Create the following secret to device-virtual
```
POST https://localhost:8443/device-virtual/api/v3/secret
{
   "apiVersion":"v3",
   "secretName":"httpserver",
   "secretData":[
      {
        "key":"type",
        "value":"httpheader"
      },
      {
        "key":"headername",
        "value":"Authorization"
      },
      {
        "key":"headercontents",
        "value":"Basic ZWRnZXh1c2VyOmVkZ2V4cGFzc3dk"
      }
   ]
}
```
6. check in the edgex-ui to ensure that there are no profiles/devices for device-virtual. If there are, delete them in the UI (Be sure to stop device-virtual first).
7. Update variables Device > ProfilesDir and Device > DevicesDir with `secure credentials` URI from consul
```
ProfilesDir: http://httpd-auth:80/test_files/profile.json?edgexSecretName=httpserver
DevicesDir: http://httpd-auth:80/test_files/device.json?edgexSecretName=httpserver
```
8. Restart device-virtual
9. check in the edgex-ui to ensure that the profiles/devices get created

## New Dependency Instructions (If applicable)
<!-- Please follow [vetting instructions](https://wiki.edgexfoundry.org/display/FA/Vetting+Process+for+3rd+Party+Dependencies) and place results here -->

Patch:
diff --git a/internal/provision/common.go b/internal/provision/common.go
index 476736ff..4dd661ec 100644
--- a/internal/provision/common.go
+++ b/internal/provision/common.go
@@ -22,9 +22,10 @@ const (
 )
 
 func GetFileType(fullPath string) FileType {
-	if strings.HasSuffix(fullPath, yamlExt) || strings.HasSuffix(fullPath, ymlExt) {
+	res := strings.Split(fullPath, "?")
+	if strings.HasSuffix(res[0], yamlExt) || strings.HasSuffix(res[0], ymlExt) {
 		return YAML
-	} else if strings.HasSuffix(fullPath, ".json") {
+	} else if strings.HasSuffix(res[0], ".json") {
 		return JSON
 	} else {
 		return OTHER
@@ -46,5 +47,6 @@ func GetFullAndRedactedURI(baseURI *url.URL, file, description string, lc logger
 	}
 	fullURI.User = baseURI.User
 	fullURI.Path = newPath
+	lc.Debugf("%s URI for %s: %s", description, file, fullURI.String())
 	return fullURI.String(), fullURI.Redacted()
 }
diff --git a/internal/provision/profiles.go b/internal/provision/profiles.go
index 723e9c9f..0aeb4a26 100644
--- a/internal/provision/profiles.go
+++ b/internal/provision/profiles.go
@@ -137,7 +137,6 @@ func loadProfilesFromURI(inputURI string, parsedURI *url.URL, dpc interfaces.Dev
 func processProfiles(fullPath, displayPath string, secretProvider bootstrapInterfaces.SecretProvider, lc logger.LoggingClient, dpc interfaces.DeviceProfileClient) ([]requests.DeviceProfileRequest, errors.EdgeX) {
 	var profile dtos.DeviceProfile
 	var addProfilesReq []requests.DeviceProfileRequest
-
 	fileType := GetFileType(fullPath)
 
 	// if the file type is not yaml or json, it cannot be parsed - just return to not break the loop for other devices


Test patch:
diff --git a/internal/provision/common_test.go b/internal/provision/common_test.go
index cd2a4f39..16b7b64c 100644
--- a/internal/provision/common_test.go
+++ b/internal/provision/common_test.go
@@ -23,6 +23,12 @@ func Test_GetFileType(t *testing.T) {
 		{"valid get Yaml file type", path.Join("..", "..", "example", "cmd", "device-simple", "res", "devices", "simple-device.yml"), YAML},
 		{"valid get Json file type", path.Join("..", "..", "example", "cmd", "device-simple", "res", "devices", "simple-device.json"), JSON},
 		{"valid get other file type", path.Join("..", "..", "example", "cmd", "device-simple", "res", "devices", "simple-device.bogus"), OTHER},
+		{"valid EdgeX Username/Password URI get Yaml file type", "http://edgexuser:edgexpasswd@httpd-auth:80/http_files/simple-device.yaml", YAML},
+		{"valid EdgeX Username/Password URI get Json file type", "http://edgexuser:edgexpasswd@httpd-auth:80/http_files/simple-device.json", JSON},
+		{"valid EdgeX Username/Password URI get other file type", "http://edgexuser:edgexpasswd@httpd-auth:80/http_files/simple-device.bogus", OTHER},
+		{"valid EdgeX Secret URI get Yaml file type", "http://httpd-auth:80/http_files/simple-device.yaml?edgexSecretName=httpserver", YAML},
+		{"valid EdgeX Secret URI get Json file type", "http://httpd-auth:80/http_files/simple-device.json?edgexSecretName=httpserver", JSON},
+		{"valid EdgeX Secret URI get other file type", "http://httpd-auth:80/http_files/simple-device.bogus?edgexSecretName=httpserver", OTHER},
 	}
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {