Repository: Azure/WALinuxAgent
Pull request #1902 (2020-06-04 23:06:09)

Add API for uploading logs via host plugin
<!-- DO NOT DELETE THIS TEMPLATE -->

## Description

Continuation of the work started in #1847. These changes update the wire client, wire protocol and host plugin with an API for uploading logs.

<!--
Please add an informative description that covers that changes made by the pull request. 
This checklist is used to make sure that common issues in a pull request are addressed.
This will expedite the process of getting your pull request merged and avoid extra work on your part to fix issues discovered during the review process.
-->

---

### PR information
- [ ] The title of the PR is clear and informative.
- [ ] There are a small number of commits, each of which has an informative message. This means that previously merged commits do not appear in the history of the PR. For information on cleaning up the commits in your pull request, [see this page](https://github.com/Azure/azure-powershell/blob/master/documentation/development-docs/cleaning-up-commits.md).
- [ ] Except for special cases involving multiple contributors, the PR is started from a fork of the main repository, not a branch.
- [ ] If applicable, the PR references the bug/issue that it fixes in the description.
- [ ] New Unit tests were added for the changes made and Travis.CI is passing.

### Quality of Code and Contribution Guidelines
- [ ] I have read the [contribution guidelines](https://github.com/Azure/WALinuxAgent/blob/master/.github/CONTRIBUTING.md).

codecov[bot]: # [Codecov](https://codecov.io/gh/Azure/WALinuxAgent/pull/1902?src=pr&el=h1) Report
> Merging [#1902](https://codecov.io/gh/Azure/WALinuxAgent/pull/1902?src=pr&el=desc) into [develop](https://codecov.io/gh/Azure/WALinuxAgent/commit/78c660062f2ef86976cf4d03c8d99b70fe0a7255&el=desc) will **increase** coverage by `0.02%`.
> The diff coverage is `77.77%`.

[![Impacted file tree graph](https://codecov.io/gh/Azure/WALinuxAgent/pull/1902/graphs/tree.svg?width=650&height=150&src=pr&token=ae90E2KYCw)](https://codecov.io/gh/Azure/WALinuxAgent/pull/1902?src=pr&el=tree)

```diff
@@             Coverage Diff             @@
##           develop    #1902      +/-   ##
===========================================
+ Coverage    69.37%   69.40%   +0.02%     
===========================================
  Files           85       85              
  Lines        11811    11858      +47     
  Branches      1658     1661       +3     
===========================================
+ Hits          8194     8230      +36     
- Misses        3249     3257       +8     
- Partials       368      371       +3     
```


| [Impacted Files](https://codecov.io/gh/Azure/WALinuxAgent/pull/1902?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [azurelinuxagent/common/protocol/hostplugin.py](https://codecov.io/gh/Azure/WALinuxAgent/pull/1902/diff?src=pr&el=tree#diff-YXp1cmVsaW51eGFnZW50L2NvbW1vbi9wcm90b2NvbC9ob3N0cGx1Z2luLnB5) | `82.52% <70.83%> (-1.35%)` | :arrow_down: |
| [azurelinuxagent/common/protocol/wire.py](https://codecov.io/gh/Azure/WALinuxAgent/pull/1902/diff?src=pr&el=tree#diff-YXp1cmVsaW51eGFnZW50L2NvbW1vbi9wcm90b2NvbC93aXJlLnB5) | `77.15% <83.33%> (+0.15%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/Azure/WALinuxAgent/pull/1902?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/Azure/WALinuxAgent/pull/1902?src=pr&el=footer). Last update [78c6600...3e13e76](https://codecov.io/gh/Azure/WALinuxAgent/pull/1902?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Patch:
diff --git a/azurelinuxagent/common/protocol/hostplugin.py b/azurelinuxagent/common/protocol/hostplugin.py
index 84e5ee1d..5ef2987b 100644
--- a/azurelinuxagent/common/protocol/hostplugin.py
+++ b/azurelinuxagent/common/protocol/hostplugin.py
@@ -20,29 +20,39 @@
 import base64
 import datetime
 import json
+import uuid
 
 from azurelinuxagent.common import logger
 from azurelinuxagent.common.errorstate import ErrorState, ERROR_STATE_HOST_PLUGIN_FAILURE
+from azurelinuxagent.common.event import WALAEventOperation, add_event
 from azurelinuxagent.common.exception import HttpError, ProtocolError
 from azurelinuxagent.common.future import ustr
 from azurelinuxagent.common.protocol.healthservice import HealthService
 from azurelinuxagent.common.utils import restutil
 from azurelinuxagent.common.utils import textutil
 from azurelinuxagent.common.utils.textutil import remove_bom
-from azurelinuxagent.common.version import PY_VERSION_MAJOR
+from azurelinuxagent.common.version import AGENT_NAME, AGENT_VERSION, PY_VERSION_MAJOR
 
 HOST_PLUGIN_PORT = 32526
+
 URI_FORMAT_GET_API_VERSIONS = "http://{0}:{1}/versions"
 URI_FORMAT_GET_EXTENSION_ARTIFACT = "http://{0}:{1}/extensionArtifact"
 URI_FORMAT_PUT_VM_STATUS = "http://{0}:{1}/status"
 URI_FORMAT_PUT_LOG = "http://{0}:{1}/vmAgentLog"
 URI_FORMAT_HEALTH = "http://{0}:{1}/health"
+
 API_VERSION = "2015-09-01"
-HEADER_CONTAINER_ID = "x-ms-containerid"
-HEADER_VERSION = "x-ms-version"
-HEADER_HOST_CONFIG_NAME = "x-ms-host-config-name"
-HEADER_ARTIFACT_LOCATION = "x-ms-artifact-location"
-HEADER_ARTIFACT_MANIFEST_LOCATION = "x-ms-artifact-manifest-location"
+
+_HEADER_CLIENT_NAME = "x-ms-client-name"
+_HEADER_CLIENT_VERSION = "x-ms-client-version"
+_HEADER_CORRELATION_ID = "x-ms-client-correlationid"
+_HEADER_CONTAINER_ID = "x-ms-containerid"
+_HEADER_DEPLOYMENT_ID = "x-ms-vmagentlog-deploymentid"
+_HEADER_VERSION = "x-ms-version"
+_HEADER_HOST_CONFIG_NAME = "x-ms-host-config-name"
+_HEADER_ARTIFACT_LOCATION = "x-ms-artifact-location"
+_HEADER_ARTIFACT_MANIFEST_LOCATION = "x-ms-artifact-manifest-location"
+
 MAXIMUM_PAGEBLOB_PAGE_SIZE = 4 * 1024 * 1024  # Max page size: 4MB
 
 
@@ -60,7 +70,7 @@ class HostPluginProtocol(object):
         self.api_versions = None
         self.endpoint = endpoint
         self.container_id = container_id
-        self.deployment_id = None
+        self.deployment_id = self._extract_deployment_id(role_config_name)
         self.role_config_name = role_config_name
         self.manifest_uri = None
         self.health_service = HealthService(endpoint)
@@ -69,6 +79,11 @@ class HostPluginProtocol(object):
         self.fetch_last_timestamp = None
         self.status_last_timestamp = None
 
+    @staticmethod
+    def _extract_deployment_id(role_config_name):
+        # Role config name consists of: <deployment id>.<incarnation>(...)
+        return role_config_name.split(".")[0] if role_config_name is not None else None
+
     @staticmethod
     def is_default_channel():
         return HostPluginProtocol._is_default_channel
@@ -82,6 +97,7 @@ class HostPluginProtocol(object):
 
     def update_role_config_name(self, new_role_config_name):
         self.role_config_name = new_role_config_name
+        self.deployment_id = self._extract_deployment_id(new_role_config_name)
 
     def update_manifest_uri(self, new_manifest_uri):
         self.manifest_uri = new_manifest_uri
@@ -91,9 +107,8 @@ class HostPluginProtocol(object):
             self.api_versions = self.get_api_versions()
             self.is_available = API_VERSION in self.api_versions
             self.is_initialized = self.is_available
-            from azurelinuxagent.common.event import WALAEventOperation, report_event
-            report_event(WALAEventOperation.InitializeHostPlugin,
-                         is_success=self.is_available)
+            add_event(WALAEventOperation.InitializeHostPlugin,
+                      is_success=self.is_available)
         return self.is_available
 
     def get_health(self):
@@ -116,7 +131,7 @@ class HostPluginProtocol(object):
         error_response = ''
         is_healthy = False
         try:
-            headers = {HEADER_CONTAINER_ID: self.container_id}
+            headers = {_HEADER_CONTAINER_ID: self.container_id}
             response = restutil.http_get(url, headers)
 
             if restutil.request_failed(response):
@@ -142,13 +157,13 @@ class HostPluginProtocol(object):
 
         url = URI_FORMAT_GET_EXTENSION_ARTIFACT.format(self.endpoint,
                                                        HOST_PLUGIN_PORT)
-        headers = {HEADER_VERSION: API_VERSION,
-                   HEADER_CONTAINER_ID: self.container_id,
-                   HEADER_HOST_CONFIG_NAME: self.role_config_name,
-                   HEADER_ARTIFACT_LOCATION: artifact_url}
+        headers = {_HEADER_VERSION: API_VERSION,
+                   _HEADER_CONTAINER_ID: self.container_id,
+                   _HEADER_HOST_CONFIG_NAME: self.role_config_name,
+                   _HEADER_ARTIFACT_LOCATION: artifact_url}
 
         if artifact_manifest_url is not None:
-            headers[HEADER_ARTIFACT_MANIFEST_LOCATION] = artifact_manifest_url
+            headers[_HEADER_ARTIFACT_MANIFEST_LOCATION] = artifact_manifest_url
 
         return url, headers
 
@@ -202,7 +217,28 @@ class HostPluginProtocol(object):
         return datetime.datetime.utcnow() >= (last_timestamp + period)
 
     def put_vm_log(self, content):
-        raise NotImplementedError("Unimplemented")
+        """
+        Try to upload VM logs, a compressed zip file, via the host plugin /vmAgentLog channel.
+        :param content: the binary content of the zip file to upload
+        """
+        if not self.ensure_initialized():
+            raise ProtocolError("HostGAPlugin: HostGAPlugin is not available")
+
+        if content is None:
+            raise ProtocolError("HostGAPlugin: Invalid argument passed to upload VM logs. Content was not provided.")
+
+        url = URI_FORMAT_PUT_LOG.format(self.endpoint, HOST_PLUGIN_PORT)
+        response = restutil.http_put(url,
+                                     data=content,
+                                     headers=self._build_log_headers())
+
+        if restutil.request_failed(response):
+            error_response = restutil.read_response_error(response)
+            raise HttpError("HostGAPlugin: Upload VM logs failed: {0}".format(error_response))
+        else:
+            logger.info("HostGAPlugin: Upload VM logs succeeded")
+
+        return response
 
     def put_vm_status(self, status_blob, sas_url, config_blob_type=None):
         """
@@ -322,12 +358,22 @@ class HostPluginProtocol(object):
     
     def _build_status_headers(self):
         return {
-            HEADER_VERSION: API_VERSION,
+            _HEADER_VERSION: API_VERSION,
             "Content-type": "application/json",
-            HEADER_CONTAINER_ID: self.container_id,
-            HEADER_HOST_CONFIG_NAME: self.role_config_name
+            _HEADER_CONTAINER_ID: self.container_id,
+            _HEADER_HOST_CONFIG_NAME: self.role_config_name
         }
-    
+
+    def _build_log_headers(self):
+        return {
+            _HEADER_VERSION: API_VERSION,
+            _HEADER_CONTAINER_ID: self.container_id,
+            _HEADER_DEPLOYMENT_ID: self.deployment_id,
+            _HEADER_CLIENT_NAME: AGENT_NAME,
+            _HEADER_CLIENT_VERSION: AGENT_VERSION,
+            _HEADER_CORRELATION_ID: str(uuid.uuid4())
+        }
+
     def _base64_encode(self, data):
         s = base64.b64encode(bytes(data))
         if PY_VERSION_MAJOR > 2:
diff --git a/azurelinuxagent/common/protocol/wire.py b/azurelinuxagent/common/protocol/wire.py
index a28de91d..e3345efd 100644
--- a/azurelinuxagent/common/protocol/wire.py
+++ b/azurelinuxagent/common/protocol/wire.py
@@ -29,7 +29,8 @@ import azurelinuxagent.common.conf as conf
 import azurelinuxagent.common.logger as logger
 import azurelinuxagent.common.utils.textutil as textutil
 from azurelinuxagent.common.datacontract import validate_param
-from azurelinuxagent.common.event import add_event, add_periodic, WALAEventOperation, EVENTS_DIRECTORY, EventLogger
+from azurelinuxagent.common.event import add_event, add_periodic, WALAEventOperation, EVENTS_DIRECTORY, EventLogger, \
+    report_event
 from azurelinuxagent.common.exception import ProtocolNotFoundError, \
     ResourceGoneError, ExtensionDownloadError, InvalidContainerError, ProtocolError, HttpError
 from azurelinuxagent.common.future import httpclient, bytebuffer, ustr
@@ -192,6 +193,9 @@ class WireProtocol(DataContract):
         validate_param(EVENTS_DIRECTORY, events, TelemetryEventList)
         self.client.report_event(events)
 
+    def upload_logs(self, logs):
+        self.client.upload_logs(logs)
+
 
 def _build_role_properties(container_id, role_instance_id, thumbprint):
     xml = (u"<?xml version=\"1.0\" encoding=\"utf-8\"?>"
@@ -875,41 +879,8 @@ class WireClient(object):
                      "advised by Fabric.").format(PROTOCOL_VERSION)
             raise ProtocolNotFoundError(error)
 
-    def send_request_using_appropriate_channel(self, direct_func, host_func):
-        # A wrapper method for all function calls that send HTTP requests. The purpose of the method is to
-        # define which channel to use, direct or through the host plugin. For the host plugin channel,
-        # also implement a retry mechanism.
-
-        # By default, the direct channel is the default channel. If that is the case, try getting a response
-        # through that channel. On failure, fall back to the host plugin channel.
-
-        # When using the host plugin channel, regardless if it's set as default or not, try sending the request first.
-        # On specific failures that indicate a stale goal state (such as resource gone or invalid container parameter),
-        # refresh the goal state and try again. If successful, set the host plugin channel as default. If failed,
-        # raise the exception.
-
-        # NOTE: direct_func and host_func are passed as lambdas. Be careful about capturing goal state data in them as
-        # they will not be refreshed even if a goal state refresh is called before retrying the host_func.
-
-        if not HostPluginProtocol.is_default_channel():
-            ret = None
-            try:
-                ret = direct_func()
-
-                # Different direct channel functions report failure in different ways: by returning None, False,
-                # or raising ResourceGone or InvalidContainer exceptions.
-                if not ret:
-                    logger.periodic_info(logger.EVERY_HOUR, "[PERIODIC] Request failed using the direct channel, "
-                                                            "switching to host plugin.")
-            except (ResourceGoneError, InvalidContainerError) as e:
-                logger.periodic_info(logger.EVERY_HOUR, "[PERIODIC] Request failed using the direct channel, "
-                                                        "switching to host plugin. Error: {0}".format(ustr(e)))
-
-            if ret:
-                return ret
-        else:
-            logger.periodic_info(logger.EVERY_HALF_DAY, "[PERIODIC] Using host plugin as default channel.")
-
+    def _call_hostplugin_with_container_check(self, host_func):
+        ret = None
         try:
             ret = host_func()
         except (ResourceGoneError, InvalidContainerError) as e:
@@ -959,6 +930,45 @@ class WireClient(object):
                              log_event=True)
                 raise
 
+        return ret
+
+    def send_request_using_appropriate_channel(self, direct_func, host_func):
+        # A wrapper method for all function calls that send HTTP requests. The purpose of the method is to
+        # define which channel to use, direct or through the host plugin. For the host plugin channel,
+        # also implement a retry mechanism.
+
+        # By default, the direct channel is the default channel. If that is the case, try getting a response
+        # through that channel. On failure, fall back to the host plugin channel.
+
+        # When using the host plugin channel, regardless if it's set as default or not, try sending the request first.
+        # On specific failures that indicate a stale goal state (such as resource gone or invalid container parameter),
+        # refresh the goal state and try again. If successful, set the host plugin channel as default. If failed,
+        # raise the exception.
+
+        # NOTE: direct_func and host_func are passed as lambdas. Be careful about capturing goal state data in them as
+        # they will not be refreshed even if a goal state refresh is called before retrying the host_func.
+
+        if not HostPluginProtocol.is_default_channel():
+            ret = None
+            try:
+                ret = direct_func()
+
+                # Different direct channel functions report failure in different ways: by returning None, False,
+                # or raising ResourceGone or InvalidContainer exceptions.
+                if not ret:
+                    logger.periodic_info(logger.EVERY_HOUR, "[PERIODIC] Request failed using the direct channel, "
+                                                            "switching to host plugin.")
+            except (ResourceGoneError, InvalidContainerError) as e:
+                logger.periodic_info(logger.EVERY_HOUR, "[PERIODIC] Request failed using the direct channel, "
+                                                        "switching to host plugin. Error: {0}".format(ustr(e)))
+
+            if ret:
+                return ret
+        else:
+            logger.periodic_info(logger.EVERY_HALF_DAY, "[PERIODIC] Using host plugin as default channel.")
+
+        ret = self._call_hostplugin_with_container_check(host_func)
+
         if not HostPluginProtocol.is_default_channel():
             logger.info("Setting host plugin as default channel from now on. "
                         "Restart the agent to reset the default channel.")
@@ -1139,9 +1149,6 @@ class WireClient(object):
                 self.send_encoded_event(provider_id, buf[provider_id])
 
     def report_status_event(self, message, is_success):
-        from azurelinuxagent.common.event import report_event, \
-            WALAEventOperation
-
         report_event(op=WALAEventOperation.ReportStatus,
                      is_success=is_success,
                      message=message,
@@ -1218,7 +1225,6 @@ class WireClient(object):
                     msg = "Content: [{0}]".format(profile)
                     logger.verbose(msg)
 
-                    from azurelinuxagent.common.event import report_event, WALAEventOperation
                     report_event(op=WALAEventOperation.ArtifactsProfileBlob,
                                  is_success=False,
                                  message=msg,
@@ -1226,6 +1232,14 @@ class WireClient(object):
 
         return artifacts_profile
 
+    def upload_logs(self, content):
+        host_func = lambda: self._upload_logs_through_host(content)
+        return self._call_hostplugin_with_container_check(host_func)
+
+    def _upload_logs_through_host(self, content):
+        host = self.get_host_plugin()
+        return host.put_vm_log(content)
+
 
 class VersionInfo(object):
     def __init__(self, xml_text):


Test patch:
diff --git a/tests/common/test_event.py b/tests/common/test_event.py
index dd732346..993e2e09 100644
--- a/tests/common/test_event.py
+++ b/tests/common/test_event.py
@@ -25,6 +25,7 @@ import shutil
 import threading
 import xml.dom
 from datetime import datetime, timedelta
+
 import azurelinuxagent.common.utils.textutil as textutil
 from azurelinuxagent.common import event, logger
 from azurelinuxagent.common.AgentGlobals import AgentGlobals
@@ -32,13 +33,13 @@ from azurelinuxagent.common.event import add_event, add_periodic, add_log_event,
     WALAEventOperation, parse_xml_event, parse_json_event, AGENT_EVENT_FILE_EXTENSION, EVENTS_DIRECTORY, \
     TELEMETRY_EVENT_EVENT_ID, TELEMETRY_EVENT_PROVIDER_ID, TELEMETRY_LOG_EVENT_ID, TELEMETRY_LOG_PROVIDER_ID
 from azurelinuxagent.common.future import ustr
+from azurelinuxagent.common.osutil import get_osutil
 from azurelinuxagent.common.telemetryevent import CommonTelemetryEventSchema, GuestAgentGenericLogsSchema, \
     GuestAgentExtensionEventsSchema, GuestAgentPerfCounterEventsSchema
+from azurelinuxagent.common.version import CURRENT_AGENT, CURRENT_VERSION, AGENT_EXECUTION_MODE
 from tests.protocol import mockwiredata
 from tests.protocol.mocks import mock_wire_protocol, HttpRequestPredicates, MockHttpResponse
-from azurelinuxagent.common.version import CURRENT_AGENT, CURRENT_VERSION, AGENT_EXECUTION_MODE
-from azurelinuxagent.common.osutil import get_osutil
-from tests.tools import AgentTestCase, data_dir, load_data, Mock, patch, skip_if_predicate_true
+from tests.tools import AgentTestCase, data_dir, load_data, patch, skip_if_predicate_true
 from tests.utils.event_logger_tools import EventLoggerTools
 
 
@@ -52,8 +53,8 @@ class TestEvent(HttpRequestPredicates, AgentTestCase):
         osutil = get_osutil()
 
         self.expected_common_parameters = {
-            # common parameters computed at event creation; the timestamp (stored as the opcode name) is not included here and
-            # is checked separately from these parameters
+            # common parameters computed at event creation; the timestamp (stored as the opcode name) is not included
+            # here and is checked separately from these parameters
             CommonTelemetryEventSchema.GAVersion: CURRENT_AGENT,
             CommonTelemetryEventSchema.ContainerId: AgentGlobals.get_container_id(),
             CommonTelemetryEventSchema.EventTid: threading.current_thread().ident,
@@ -68,7 +69,7 @@ class TestEvent(HttpRequestPredicates, AgentTestCase):
             # common parameters from the goal state
             CommonTelemetryEventSchema.TenantName: 'db00a7755a5e4e8a8fe4b19bc3b330c3',
             CommonTelemetryEventSchema.RoleName: 'MachineRole',
-            CommonTelemetryEventSchema.RoleInstanceName: 'MachineRole_IN_0',
+            CommonTelemetryEventSchema.RoleInstanceName: 'b61f93d0-e1ed-40b2-b067-22c243233448.MachineRole_IN_0',
             # common parameters
             CommonTelemetryEventSchema.Location: EventLoggerTools.mock_imds_data['location'],
             CommonTelemetryEventSchema.SubscriptionId: EventLoggerTools.mock_imds_data['subscriptionId'],
diff --git a/tests/data/wire/goal_state.xml b/tests/data/wire/goal_state.xml
index 8c60c1cd..579b5e87 100644
--- a/tests/data/wire/goal_state.xml
+++ b/tests/data/wire/goal_state.xml
@@ -12,7 +12,7 @@
      <ContainerId>c6d5526c-5ac2-4200-b6e2-56f2b70c5ab2</ContainerId>
      <RoleInstanceList>
        <RoleInstance>
-         <InstanceId>MachineRole_IN_0</InstanceId>
+         <InstanceId>b61f93d0-e1ed-40b2-b067-22c243233448.MachineRole_IN_0</InstanceId>
          <State>Started</State>
          <Configuration>
          <HostingEnvironmentConfig>http://168.63.129.16:80/hostingenvuri/</HostingEnvironmentConfig>
@@ -20,7 +20,7 @@
          <Certificates>http://168.63.129.16:80/certificatesuri/</Certificates>
          <ExtensionsConfig>http://168.63.129.16:80/extensionsconfiguri/</ExtensionsConfig>
          <FullConfig>http://168.63.129.16:80/fullconfiguri/</FullConfig>
-         <ConfigName>DummyRoleConfigName.xml</ConfigName>
+         <ConfigName>b61f93d0-e1ed-40b2-b067-22c243233448.1.b61f93d0-e1ed-40b2-b067-22c243233448.2.MachineRole_IN_0.xml</ConfigName>
          </Configuration>
        </RoleInstance>
      </RoleInstanceList>
diff --git a/tests/data/wire/goal_state_no_ext.xml b/tests/data/wire/goal_state_no_ext.xml
index 1c267d9d..ef7e3989 100644
--- a/tests/data/wire/goal_state_no_ext.xml
+++ b/tests/data/wire/goal_state_no_ext.xml
@@ -12,13 +12,14 @@
      <ContainerId>c6d5526c-5ac2-4200-b6e2-56f2b70c5ab2</ContainerId>
      <RoleInstanceList>
        <RoleInstance>
-         <InstanceId>MachineRole_IN_0</InstanceId>
+         <InstanceId>b61f93d0-e1ed-40b2-b067-22c243233448.MachineRole_IN_0</InstanceId>
          <State>Started</State>
          <Configuration>
          <HostingEnvironmentConfig>http://168.63.129.16:80/hostingenvuri/</HostingEnvironmentConfig>
          <SharedConfig>http://168.63.129.16:80/sharedconfiguri/</SharedConfig>
          <Certificates>http://168.63.129.16:80/certificatesuri/</Certificates>
          <FullConfig>http://168.63.129.16:80/fullconfiguri/</FullConfig>
+         <ConfigName>b61f93d0-e1ed-40b2-b067-22c243233448.1.b61f93d0-e1ed-40b2-b067-22c243233448.2.MachineRole_IN_0.xml</ConfigName>
          </Configuration>
        </RoleInstance>
      </RoleInstanceList>
diff --git a/tests/data/wire/goal_state_remote_access.xml b/tests/data/wire/goal_state_remote_access.xml
index 8d0a5cc8..c2840645 100644
--- a/tests/data/wire/goal_state_remote_access.xml
+++ b/tests/data/wire/goal_state_remote_access.xml
@@ -14,7 +14,7 @@
      </RemoteAccessInfo>
      <RoleInstanceList>
        <RoleInstance>
-         <InstanceId>MachineRole_IN_0</InstanceId>
+         <InstanceId>b61f93d0-e1ed-40b2-b067-22c243233448.MachineRole_IN_0</InstanceId>
          <State>Started</State>
          <Configuration>
          <HostingEnvironmentConfig>http://168.63.129.16:80/hostingenvuri/</HostingEnvironmentConfig>
@@ -22,7 +22,7 @@
          <Certificates>http://168.63.129.16:80/certificatesuri/</Certificates>
          <ExtensionsConfig>http://168.63.129.16:80/extensionsconfiguri/</ExtensionsConfig>
          <FullConfig>http://168.63.129.16:80/fullconfiguri/</FullConfig>
-         <ConfigName>DummyRoleConfigName.xml</ConfigName>
+         <ConfigName>b61f93d0-e1ed-40b2-b067-22c243233448.1.b61f93d0-e1ed-40b2-b067-22c243233448.2.MachineRole_IN_0.xml</ConfigName>
          </Configuration>
        </RoleInstance>
      </RoleInstanceList>
diff --git a/tests/ga/test_monitor.py b/tests/ga/test_monitor.py
index 14f25dd7..8c5ed9ae 100644
--- a/tests/ga/test_monitor.py
+++ b/tests/ga/test_monitor.py
@@ -271,7 +271,7 @@ class TestEventMonitoring(AgentTestCase, HttpRequestPredicates):
                              '<Param Name="Processors" Value="{8}" T="mt:uint64" />' \
                              '<Param Name="TenantName" Value="db00a7755a5e4e8a8fe4b19bc3b330c3" T="mt:wstr" />' \
                              '<Param Name="RoleName" Value="MachineRole" T="mt:wstr" />' \
-                             '<Param Name="RoleInstanceName" Value="MachineRole_IN_0" T="mt:wstr" />' \
+                             '<Param Name="RoleInstanceName" Value="b61f93d0-e1ed-40b2-b067-22c243233448.MachineRole_IN_0" T="mt:wstr" />' \
                              '<Param Name="Location" Value="uswest" T="mt:wstr" />' \
                              '<Param Name="SubscriptionId" Value="AAAAAAAA-BBBB-CCCC-DDDD-EEEEEEEEEEEE" T="mt:wstr" />' \
                              '<Param Name="ResourceGroupName" Value="test-rg" T="mt:wstr" />' \
diff --git a/tests/protocol/mocks.py b/tests/protocol/mocks.py
index 04231a25..87bc7912 100644
--- a/tests/protocol/mocks.py
+++ b/tests/protocol/mocks.py
@@ -99,6 +99,8 @@ def mock_wire_protocol(mock_wire_data_file, http_get_handler=None, http_post_han
                 return protocol.mock_wire_data.mock_http_get(url, **kwargs)
             if method == 'POST':
                 return protocol.mock_wire_data.mock_http_post(url, data, **kwargs)
+            if method == 'PUT':
+                return protocol.mock_wire_data.mock_http_put(url, data, **kwargs)
 
         # the request was not handled; fail or call the original resutil.http_request
         if fail_on_unknown_request:
@@ -201,6 +203,11 @@ class HttpRequestPredicates(object):
         artifact_location = HttpRequestPredicates._get_host_plugin_request_artifact_location(url, request_kwargs)
         return HttpRequestPredicates.is_in_vm_artifacts_profile_request(artifact_location)
 
+    @staticmethod
+    def is_host_plugin_put_logs_request(url):
+        return url.lower() == 'http://{0}:{1}/vmagentlog'.format(restutil.KNOWN_WIRESERVER_IP,
+                                                                 restutil.HOST_PLUGIN_PORT)
+
 
 class MockHttpResponse:
     def __init__(self, status, body=''):
diff --git a/tests/protocol/mockwiredata.py b/tests/protocol/mockwiredata.py
index 3cb53f02..1a33ad3b 100644
--- a/tests/protocol/mockwiredata.py
+++ b/tests/protocol/mockwiredata.py
@@ -99,6 +99,7 @@ class WireProtocolData(object):
             "/versions": 0,
             "/health": 0,
             "/HealthService": 0,
+            "/vmAgentLog": 0,
             "goalstate": 0,
             "hostingenvuri": 0,
             "sharedconfiguri": 0,
@@ -244,6 +245,21 @@ class WireProtocolData(object):
         resp.read = Mock(return_value=content.encode("utf-8"))
         return resp
 
+    def mock_http_put(self, url, *args, **kwargs):
+        content = None
+
+        resp = MagicMock()
+        resp.status = httpclient.OK
+
+        if url.endswith('/vmAgentLog'):
+            self.call_counts['/vmAgentLog'] += 1
+            content = ''
+        else:
+            raise Exception("Bad url {0}".format(url))
+
+        resp.read = Mock(return_value=content.encode("utf-8"))
+        return resp
+
     def mock_crypt_util(self, *args, **kw):
         #Partially patch instance method of class CryptUtil
         cryptutil = CryptUtil(*args, **kw)
diff --git a/tests/protocol/test_hostplugin.py b/tests/protocol/test_hostplugin.py
index 5dffee01..42141b59 100644
--- a/tests/protocol/test_hostplugin.py
+++ b/tests/protocol/test_hostplugin.py
@@ -16,22 +16,23 @@
 #
 
 import base64
+import contextlib
+import datetime
 import json
 import sys
-import datetime
 import unittest
-import contextlib
 
+import azurelinuxagent.common.protocol.hostplugin as hostplugin
 import azurelinuxagent.common.protocol.restapi as restapi
 import azurelinuxagent.common.protocol.wire as wire
-import azurelinuxagent.common.protocol.hostplugin as hostplugin
 from azurelinuxagent.common.errorstate import ErrorState
-
 from azurelinuxagent.common.exception import HttpError, ResourceGoneError
 from azurelinuxagent.common.future import ustr
+from azurelinuxagent.common.osutil.default import UUID_PATTERN
 from azurelinuxagent.common.protocol.hostplugin import API_VERSION
 from azurelinuxagent.common.utils import restutil
-from tests.protocol.mocks import mock_wire_protocol
+from azurelinuxagent.common.version import AGENT_VERSION, AGENT_NAME
+from tests.protocol.mocks import mock_wire_protocol, HttpRequestPredicates
 from tests.protocol.mockwiredata import DATA_FILE, DATA_FILE_NO_EXT
 from tests.protocol.test_wire import MockResponse as TestWireMockResponse
 from tests.tools import AgentTestCase, PY_VERSION_MAJOR, Mock, patch
@@ -47,6 +48,7 @@ elif sys.version_info[0] == 2:
 hostplugin_status_url = "http://168.63.129.16:32526/status"
 hostplugin_versions_url = "http://168.63.129.16:32526/versions"
 health_service_url = 'http://168.63.129.16:80/HealthService'
+hostplugin_logs_url = "http://168.63.129.16:32526/vmAgentLog"
 sas_url = "http://sas_url"
 wireserver_url = "168.63.129.16"
 
@@ -62,7 +64,7 @@ if PY_VERSION_MAJOR > 2:
     faux_status_b64 = faux_status_b64.decode('utf-8')
 
 
-class TestHostPlugin(AgentTestCase):
+class TestHostPlugin(HttpRequestPredicates, AgentTestCase):
 
     def _init_host(self):
         with mock_wire_protocol(DATA_FILE) as protocol:
@@ -156,7 +158,7 @@ class TestHostPlugin(AgentTestCase):
     @contextlib.contextmanager
     def create_mock_protocol():
         with mock_wire_protocol(DATA_FILE_NO_EXT) as protocol:
-            # These tests use mock wire data that dont have any extensions (extension config will be empty).
+            # These tests use mock wire data that don't have any extensions (extension config will be empty).
             # Populate the upload blob and set an initial empty status before returning the protocol.
             ext_conf = protocol.client._goal_state.ext_conf
             ext_conf.status_upload_blob = sas_url
@@ -172,7 +174,7 @@ class TestHostPlugin(AgentTestCase):
 
     @patch("azurelinuxagent.common.protocol.healthservice.HealthService.report_host_plugin_versions")
     @patch("azurelinuxagent.ga.update.restutil.http_get")
-    @patch("azurelinuxagent.common.event.report_event")
+    @patch("azurelinuxagent.common.protocol.hostplugin.add_event")
     def assert_ensure_initialized(self, patch_event, patch_http_get, patch_report_health,
                                   response_body,
                                   response_status_code,
@@ -398,7 +400,7 @@ class TestHostPlugin(AgentTestCase):
                         self.assertTrue('Falling back to direct' in patch_add_event.call_args[1]['message'])
                         self.assertEqual(True, patch_add_event.call_args[1]['is_success'])
 
-    def test_validate_http_request(self):
+    def test_validate_http_request_when_uploading_status(self):
         """Validate correct set of data is sent to HostGAPlugin when reporting VM status"""
 
         with mock_wire_protocol(DATA_FILE) as protocol:
@@ -536,6 +538,73 @@ class TestHostPlugin(AgentTestCase):
                         test_goal_state,
                         exp_method, exp_url, exp_data)
 
+    def test_validate_http_request_for_put_vm_log(self):
+        def http_put_handler(url, *args, **kwargs):
+            if self.is_host_plugin_put_logs_request(url):
+                http_put_handler.args, http_put_handler.kwargs = args, kwargs
+                return MockResponse(body=b'', status_code=200)
+
+        http_put_handler.args, http_put_handler.kwargs = [], {}
+
+        with mock_wire_protocol(DATA_FILE, http_put_handler=http_put_handler) as protocol:
+            test_goal_state = protocol.client.get_goal_state()
+
+            expected_url = hostplugin.URI_FORMAT_PUT_LOG.format(wireserver_url, hostplugin.HOST_PLUGIN_PORT)
+            expected_headers = {'x-ms-version': '2015-09-01',
+                                "x-ms-containerid": test_goal_state.container_id,
+                                "x-ms-vmagentlog-deploymentid": test_goal_state.role_config_name.split(".")[0],
+                                "x-ms-client-name": AGENT_NAME,
+                                "x-ms-client-version": AGENT_VERSION}
+
+            host_client = wire.HostPluginProtocol(wireserver_url,
+                                                  test_goal_state.container_id,
+                                                  test_goal_state.role_config_name)
+
+            self.assertFalse(host_client.is_initialized, "Host plugin should not be initialized!")
+
+            content = b"test"
+            host_client.put_vm_log(content)
+            self.assertTrue(host_client.is_initialized, "Host plugin is not initialized!")
+
+            urls = protocol.get_tracked_urls()
+
+            self.assertEqual(expected_url, urls[0], "Unexpected request URL!")
+            self.assertEqual(content, http_put_handler.args[0], "Unexpected content for HTTP PUT request!")
+
+            headers = http_put_handler.kwargs['headers']
+            for k in expected_headers:
+                self.assertTrue(k in headers, "Header {0} not found in headers!".format(k))
+                self.assertEqual(expected_headers[k], headers[k], "Request headers don't match!")
+
+            # Special check for correlation id header value, check for pattern, not exact value
+            self.assertTrue("x-ms-client-correlationid" in headers.keys(), "Correlation id not found in headers!")
+            self.assertTrue(UUID_PATTERN.match(headers["x-ms-client-correlationid"]), "Correlation id is not in GUID form!")
+
+    def test_put_vm_log_should_raise_an_exception_when_request_fails(self):
+        def http_put_handler(url, *args, **kwargs):
+            if self.is_host_plugin_put_logs_request(url):
+                http_put_handler.args, http_put_handler.kwargs = args, kwargs
+                return MockResponse(body=ustr('Gone'), status_code=410)
+
+        http_put_handler.args, http_put_handler.kwargs = [], {}
+
+        with mock_wire_protocol(DATA_FILE, http_put_handler=http_put_handler) as protocol:
+            test_goal_state = protocol.client.get_goal_state()
+
+            host_client = wire.HostPluginProtocol(wireserver_url,
+                                                  test_goal_state.container_id,
+                                                  test_goal_state.role_config_name)
+
+            self.assertFalse(host_client.is_initialized, "Host plugin should not be initialized!")
+
+            with self.assertRaises(HttpError) as context_manager:
+                content = b"test"
+                host_client.put_vm_log(content)
+
+            self.assertIsInstance(context_manager.exception, HttpError)
+            self.assertIn("410", ustr(context_manager.exception))
+            self.assertIn("Gone", ustr(context_manager.exception))
+
     def test_validate_get_extension_artifacts(self):
         with mock_wire_protocol(DATA_FILE) as protocol:
             test_goal_state = protocol.client._goal_state
diff --git a/tests/protocol/test_wire.py b/tests/protocol/test_wire.py
index 004ccebf..09e29bdc 100644
--- a/tests/protocol/test_wire.py
+++ b/tests/protocol/test_wire.py
@@ -217,7 +217,6 @@ class TestWireProtocol(AgentTestCase):
         def http_put_handler(url, *_, **__):
             if protocol.get_endpoint() in url and url.endswith('/status'):
                 return MockResponse(body=b'', status_code=200)
-            self.fail('The upload status request was sent to the wrong url: {0}'.format(url))
 
         with mock_wire_protocol(mockwiredata.DATA_FILE, http_put_handler=http_put_handler) as protocol:
             HostPluginProtocol.set_default_channel(False)
@@ -816,6 +815,40 @@ class TestWireClient(HttpRequestPredicates, AgentTestCase):
             self.assertTrue(self.is_host_plugin_extension_artifact_request(urls[3]), "The retry request should have been over the host channel")
             self.assertEquals(HostPluginProtocol.is_default_channel(), False, "The default channel should not have changed")
 
+    def test_upload_logs_should_not_refresh_plugin_when_first_attempt_succeeds(self):
+        def http_put_handler(url, *_, **__):
+            if self.is_host_plugin_put_logs_request(url):
+                return MockResponse(body=b'', status_code=200)
+
+        with mock_wire_protocol(mockwiredata.DATA_FILE, http_put_handler=http_put_handler) as protocol:
+            content = b"test"
+            protocol.client.upload_logs(content)
+
+            urls = protocol.get_tracked_urls()
+            self.assertEqual(len(urls), 1, 'Expected one post request to the host: [{0}]'.format(urls))
+
+    def test_upload_logs_should_retry_the_host_channel_after_refreshing_the_host_plugin(self):
+        def http_put_handler(url, *_, **__):
+            if self.is_host_plugin_put_logs_request(url):
+                if http_put_handler.host_plugin_calls == 0:
+                    http_put_handler.host_plugin_calls += 1
+                    return ResourceGoneError("Exception to fake a stale goal state")
+                protocol.track_url(url)
+            return None
+        http_put_handler.host_plugin_calls = 0
+
+        with mock_wire_protocol(mockwiredata.DATA_FILE_IN_VM_ARTIFACTS_PROFILE, http_put_handler=http_put_handler) \
+                as protocol:
+            content = b"test"
+            protocol.client.upload_logs(content)
+
+            urls = protocol.get_tracked_urls()
+            self.assertEquals(len(urls), 2, "Invalid number of requests: [{0}]".format(urls))
+            self.assertTrue(self.is_host_plugin_put_logs_request(urls[0]),
+                            "The first request should have been over the host channel")
+            self.assertTrue(self.is_host_plugin_put_logs_request(urls[1]),
+                            "The second request should have been over the host channel")
+
     def test_send_request_using_appropriate_channel_should_not_invoke_host_channel_when_direct_channel_succeeds(self):
         with mock_wire_protocol(mockwiredata.DATA_FILE) as protocol:
             protocol.client.get_host_plugin().set_default_channel(False)