opened
lboue
username_0
Hello,

I am facing an issue when I execute my Python script and call the get_mac_address_table method. Here is the snippet:
```
driver = get_network_driver(driver_name)
device = driver(host, 'admin', password, optional_args={'global_delay_factor': 3})

macAddrTable = device.get_mac_address_table()
print("macAddrTable:")
pp.pprint(macAddrTable)
```


It is complaining about XML formatting: 
`Response from the device is not in expected format : <?xml version="1.0" encoding="UTF-8"?>`


My device is a DELL **EMC S5148F-ON**.

```
python napalm-dellos10-get_mac-addr-table.py                                    
Traceback (most recent call last):
  File "napalm-dellos10-get_mac-addr-table.py", line 25, in <module>
    macAddrTable = device.get_mac_address_table()
  File "/home/user/.local/lib/python2.7/site-packages/napalm_dellos10/dellos10.py", line 800, in get_mac_address_table
    routes_xml_data = self.convert_xml_data(lldp_neighbors_output)
  File "/home/user/.local/lib/python2.7/site-packages/napalm_dellos10/dellos10.py", line 1775, in convert_xml_data
    raise CommandErrorException(msg.format(output))
napalm.base.exceptions.CommandErrorException: Response from the device is not in expected format : <?xml version="1.0" encoding="UTF-8"?>
<rpc-reply>
  <bulk>
    <data>
      <fwd-table>
        <mac-addr>00:50:56:**:**:**</mac-addr>
        <vlan>vlan1</vlan>
        <entry-type>dynamic</entry-type>
        <if-name>port-channel5</if-name>
      </fwd-table>
      <fwd-table>
        <mac-addr>00:50:56:**:**:**</mac-addr>
        <vlan>vlan1</vlan>
        <entry-type>dynamic</entry-type>
        <if-name>port-channel3</if-name>
      </fwd-table>
      <fwd-table>
      ....
```

Are you facing the same issue too?

Regards,
Ludovic
get_mac_address_table method: Response from the device is not in expected format
issue
created
javeedf
username_1
Hi Ludovic,

We didn't face this issue. Could you please send the complete debug log?

What is the OS10 Version are you using?

Thanks
Mohamed Javeed
comment
created
lboue
username_0
We are running 10.4.1.0. Here is the CLI output:

```
OS10# show version 
Dell EMC Networking OS10-Enterprise
Copyright (c) 1999-2018 by Dell Inc. All Rights Reserved.
OS Version: 10.4.1.0
Build Version: 10.4.1.0.483
Build Time: 2018-07-27T09:26:42-0700
System Type: S5148F-ON
Architecture: x86_64
```

Regards,
Ludovic
comment
created
javeedf
username_1
Thank you Ludovic, we will look into it

Regards
Mohamed Javeed
comment
created
lboue
username_0
Did you find the issue? 
I have the same behaviour from a Windows client running Python 3.7:

```
C:\Users\user\Documents\NAPLAM\DELLOS10>python napalm-dellos10-get_mac-addr-table.py
Password:
Traceback (most recent call last):
  File "C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\site-packages\napalm_dellos10\dellos10.py", line 1770, in convert_xml_data
    ret = ET.fromstring(encoded_output)
  File "src\lxml\etree.pyx", line 3213, in lxml.etree.fromstring
  File "src\lxml\parser.pxi", line 1877, in lxml.etree._parseMemoryDocument
  File "src\lxml\parser.pxi", line 1765, in lxml.etree._parseDoc
  File "src\lxml\parser.pxi", line 1127, in lxml.etree._BaseParser._parseDoc
  File "src\lxml\parser.pxi", line 601, in lxml.etree._ParserContext._handleParseResultDoc
  File "src\lxml\parser.pxi", line 711, in lxml.etree._handleParseResult
  File "src\lxml\parser.pxi", line 640, in lxml.etree._raiseParseError
  File "<string>", line 1213
lxml.etree.XMLSyntaxError: XML declaration allowed only at the start of the document, line 1213, column 6

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "napalm-dellos10-get_mac-addr-table.py", line 24, in <module>
    macAddrTable = device.get_mac_address_table()
  File "C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\site-packages\napalm_dellos10\dellos10.py", line 800, in get_mac_address_table
    routes_xml_data = self.convert_xml_data(lldp_neighbors_output)
  File "C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\site-packages\napalm_dellos10\dellos10.py", line 1775, in convert_xml_data
    raise CommandErrorException(msg.format(output))
napalm.base.exceptions.CommandErrorException: Response from the device is not in expected format : <?xml version="1.0" encoding="UTF-8"?>
<rpc-reply>
  <bulk>
    <data>
      <fwd-table>
      ...
  </bulk>
</rpc-reply>
```

Regards,
Ludovic
comment
created
dell-emc-networking
username_2
Hi Ludovic,


We are still working on that.


The problem is with the command response from device, looks like command response from the device is in different format than which i have developed. is it possible for you to share "show mac address-table | display-xml" command  response as a text file of your device for quick resolution?


We have tested with the following version, it is working fine for us. anyway we will check with your build version.

?


[cid:5e8c8790-e657-4f17-8688-cfde3c27dad4]


Response from napalm-dellos10 get_mac_address_table() API:


[cid:a9086a2d-f2f7-442b-b1ae-52cffa982a47]


Thanks

Mohamed Javeed
comment
created
dell-emc-networking
username_2
Looks like image cannot be attached, adding as text


OS10# show version
Dell EMC Networking OS10-Enterprise
Copyright (c) 1999-2018 by Dell Inc. All Rights Reserved.
OS Version: 10.4.1.0X
Build Version: 10.4.1.0.X.446
Build Time: 2018-07-24T13:02:25-0700
System Type: S5148F-ON
Architecture: x86_64
Up Time: 10:41:44
OS10# show mac address-table | display-xml
<?xml version="1.0" encoding="UTF-8"?>
<rpc-reply>
  <bulk>
    <data>
      <fwd-table>
        <mac-addr>11:11:11:11:11:13</mac-addr>
        <vlan>vlan1</vlan>
        <entry-type>static</entry-type>
        <if-name>ethernet1/1/2</if-name>
      </fwd-table>
      <fwd-table>
        <mac-addr>11:11:11:11:11:17</mac-addr>
        <vlan>vlan6</vlan>
        <entry-type>static</entry-type>
        <if-name>ethernet1/1/4</if-name>
      </fwd-table>
    </data>
    <last-keys>
      <entry-type>static</entry-type>
      <mac-addr>11:11:11:11:11:17</mac-addr>
      <vlan>vlan6</vlan>
    </last-keys>
  </bulk>
</rpc-reply>
OS10#


Response from napalm-dellos10 get_amc_address_table() API,


Started ...
[{'vlan': 1, 'last_move': -1.0, 'active': True, 'mac': u'11:11:11:11:11:13', 'static': True, 'interface': u'ethernet1/1/2', 'moves': -1}, {'vlan': 6, 'last_move': -1.0, 'active': True, 'mac': u'11:11:11:11:11:17', 'static': True, 'interface': u'ethernet1/1/4', 'moves': -1}]

Thanks
Mohamed Javeed



From: Faroo, Mohamed Javeed
Sent: 19 September 2018 11:08
To: napalm-automation-community/napalm-dellos10; napalm-automation-community/napalm-dellos10
Cc: Subscribed
Subject: Re: [napalm-automation-community/napalm-dellos10] get_mac_address_table method: Response from the device is not in expected format (#17)


Hi Ludovic,


We are still working on that.


The problem is with the command response from device, looks like command response from the device is in different format than which i have developed. is it possible for you to share "show mac address-table | display-xml" command  response as a text file of your device for quick resolution?


We have tested with the following version, it is working fine for us. anyway we will check with your build version.

​


[cid:5e8c8790-e657-4f17-8688-cfde3c27dad4]


Response from napalm-dellos10 get_mac_address_table() API:


[cid:a9086a2d-f2f7-442b-b1ae-52cffa982a47]


Thanks

Mohamed Javeed
comment
created
lboue
username_0
Hello,

I noticed something odd. When I replace the command 'show mac address-table | display-xml' in the dellos10.py plugin file with a more precise one, I don't get the error anymore:
- command 'show mac address-table interface port-channel1 | display-xml' is working
- command 'show mac address-table vlan 130 | display-xml' is working

```
    def get_mac_address_table(self):
        #cmd = 'show mac address-table vlan 130 | display-xml'
        cmd = 'show mac address-table interface port-channel1 | display-xml'
        #cmd = 'show mac address-table | display-xml'
        lldp_neighbors_output = self._send_command(cmd)
```     

**Output for the 'show mac address-table vlan 130 | display-xml' command:**
``` 
C:\Users\user\Documents\NAPLAM\DELLOS10>python napalm-dellos10-get_mac-addr-table.py
Password:
macAddrTable:
[   {   'active': True,
        'interface': 'port-channel4',
        'last_move': -1.0,
        'mac': 'ab:cd:ef:54:d2:f9',
        'moves': -1,
        'static': False,
        'vlan': 130},
    {   'active': True,
        'interface': 'port-channel4',
        'last_move': -1.0,
        'mac': 'ab:cd:ef:5a:fb:9a',
        'moves': -1,
        'static': False,
        'vlan': 130},
    {   'active': True,
        'interface': 'port-channel1',
        'last_move': -1.0,
        'mac': 'ab:cd:ef:b2:50:29',
        'moves': -1,
        'static': False,
        'vlan': 130},
    {   'active': True,
        'interface': 'port-channel47',
        'last_move': -1.0,
        'mac': 'ab:cd:ef:b2:58:2b',
        'moves': -1,
        'static': False,
        'vlan': 130},
    {   'active': True,
        'interface': 'port-channel2',
        'last_move': -1.0,
        'mac': 'ab:cd:ef:b3:27:7f',
        'moves': -1,
        'static': False,
        'vlan': 130},
    {   'active': True,
        'interface': 'port-channel1',
        'last_move': -1.0,
        'mac': 'ab:cd:ef:b3:56:a4',
        'moves': -1,
        'static': False,
        'vlan': 130},
    {   'active': True,
        'interface': 'port-channel47',
        'last_move': -1.0,
        'mac': 'ab:cd:ef:95:b9:19',
        'moves': -1,
        'static': False,
        'vlan': 130},
    {   'active': True,
        'interface': 'port-channel47',
        'last_move': -1.0,
        'mac': 'ab:cd:ef:c1:fb:c9',
        'moves': -1,
        'static': False,
        'vlan': 130},
    {   'active': True,
        'interface': 'port-channel47',
        'last_move': -1.0,
        'mac': 'ab:cd:ef:c1:ff:c9',
        'moves': -1,
        'static': False,
        'vlan': 130},
    {   'active': True,
        'interface': 'port-channel47',
        'last_move': -1.0,
        'mac': 'ab:cd:ef:a4:4c:14',
        'moves': -1,
        'static': False,
        'vlan': 130},
    {   'active': True,
        'interface': 'port-channel1000',
        'last_move': -1.0,
        'mac': 'ab:cd:ef:c1:fc:c9',
        'moves': -1,
        'static': True,
        'vlan': 130}]
``` 

In addition, I had the same behaviour with the old 10.3 version before the upgrade. 

Regards,
Ludovic
comment
created
javeedf
username_1
Hi Ludovic,

Please check if there are two XML tag inside the command "show mac
address-table | display-xml" response. That's when we will get this error
message.

Please send the complete command response without editing

Thanks
Mohamed Javeed
comment
created
javeedf
username_1
Hi Ludovic,

Could you please add the below code and check if it works for you?

Below code handles multiple xml tags:

*    def get_mac_address_table(self):*
*        cmd = 'show mac address-table | display-xml'*
*        mac_table_output = self._send_command(cmd)*

*        base_xpath = "./bulk/data/fwd-table"*
*        ret_mac_dict = []*
*        mac_table_output_list = self._build_xml_list(mac_table_output)*
*        for output in mac_table_output_list:*
*            mac_table_xml_data = self.convert_xml_data(output)*
*            mac_xml_list = mac_table_xml_data.findall(base_xpath)*
*            for mac_xml in mac_xml_list:*
*                mac_addr = self.parse_item(mac_xml, 'mac-addr')*
*                # vlan id comes with "vlan123"*
*                vlan_id_str = self.parse_item(mac_xml, 'vlan')*
*                entry_type = self.parse_item(mac_xml, 'entry-type')*
*                if_name = self.parse_item(mac_xml, 'if-name')*
*                vlan_id = int(vlan_id_str[4:].strip())*
*                mac_dict = {*
*                    "mac": mac_addr,*
*                    "interface": if_name,*
*                    "static": True if "static" == entry_type else False,*
*                    "active": True,*
*                    "vlan": vlan_id,*
*                    "moves": self.UNKNOWN_INT,*
*                    "last_move": self.UNKNOWN_FLOAT*
*                }*
*                ret_mac_dict.append(mac_dict)*

*        return ret_mac_dict*

Thanks
Mohamed Javeed


On Wed, Sep 19, 2018 at 2:58 PM Mohamed Javeed <username_1.dev@gmail.com>
wrote:

> Hi Ludovic,
>
> Please check if there are two XML tag inside the command "show mac
> address-table | display-xml" response. That's when we will get this error
> message.
>
> Please send the complete command response without editing
>
> Thanks
> Mohamed Javeed
>
>
comment
created
javeedf
username_1
Added the code easy to copy:
    

    def get_mac_address_table(self):
        cmd = 'show mac address-table | display-xml'
        mac_table_output = self._send_command(cmd)
        base_xpath = './bulk/data/fwd-table'
        ret_mac_dict = []
        mac_table_output_list = self._build_xml_list(mac_table_output)
        for output in mac_table_output_list:
            mac_table_xml_data = self.convert_xml_data(output)
            mac_xml_list = mac_table_xml_data.findall(base_xpath)
            for mac_xml in mac_xml_list:
                mac_addr = self.parse_item(mac_xml, 'mac-addr')
                # vlan id comes with "vlan123"
                vlan_id_str = self.parse_item(mac_xml, 'vlan')
                entry_type = self.parse_item(mac_xml, 'entry-type')
                if_name = self.parse_item(mac_xml, 'if-name')
                vlan_id = int(vlan_id_str[4:].strip())
                mac_dict = {
                    "mac": mac_addr,
                    "interface": if_name,
                    "static": True if "static" == entry_type else False,
                    "active": True,
                    "vlan": vlan_id,
                    "moves": self.UNKNOWN_INT,
                    "last_move": self.UNKNOWN_FLOAT
                }
                ret_mac_dict.append(mac_dict)
        return ret_mac_dict
comment
created
lboue
username_0
Hello Mohamed ,

I did the test with your patch this morning and it works fine as you can see bellow:
```
C:\Users\user\Documents\NAPLAM\DELLOS10>python napalm-dellos10-get_mac-addr-table.py
Password:
macAddrTable:
[   {   'active': True,
        'interface': 'port-channel5',
        'last_move': -1.0,
        'mac': '00:50:56:ab:cd:ab',
        'moves': -1,
        'static': False,
        'vlan': 1},
    {   'active': True,
        'interface': 'port-channel3',
        'last_move': -1.0,
        'mac': '00:50:56:ab:cd:ef',
        'moves': -1,
        'static': False,
        'vlan': 1},
       ...
```

I don't have the XML error anymore:
```
C:\Users\user\Documents\NAPLAM\DELLOS10>python napalm-dellos10-get_mac-addr-table.py
Password:
Traceback (most recent call last):
  File "C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\site-packages\napalm_dellos10\dellos10.py", line 1802, in convert_xml_data
    ret = ET.fromstring(encoded_output)
  File "src\lxml\etree.pyx", line 3213, in lxml.etree.fromstring
  File "src\lxml\parser.pxi", line 1877, in lxml.etree._parseMemoryDocument
  File "src\lxml\parser.pxi", line 1765, in lxml.etree._parseDoc
  File "src\lxml\parser.pxi", line 1127, in lxml.etree._BaseParser._parseDoc
  File "src\lxml\parser.pxi", line 601, in lxml.etree._ParserContext._handleParseResultDoc
  File "src\lxml\parser.pxi", line 711, in lxml.etree._handleParseResult
  File "src\lxml\parser.pxi", line 640, in lxml.etree._raiseParseError
  File "<string>", line 1213
lxml.etree.XMLSyntaxError: XML declaration allowed only at the start of the document, line 1213, column 6

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "napalm-dellos10-get_mac-addr-table.py", line 24, in <module>
    macAddrTable = device.get_mac_address_table()
  File "C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\site-packages\napalm_dellos10\dellos10.py", line 831, in get_mac_address_table
    routes_xml_data = self.convert_xml_data(lldp_neighbors_output)
  File "C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\site-packages\napalm_dellos10\dellos10.py", line 1807, in convert_xml_data
    raise CommandErrorException(msg.format(output))
napalm.base.exceptions.CommandErrorException: Response from the device is not in expected format : <?xml version="1.0" encoding="UTF-8"?>
<rpc-reply>
  <bulk>
    <data>
```

Regards,
Ludovic
comment
created
javeedf
username_1
Thank you Ludovic. I will commit and update the pypi
comment
created
lboue
username_0
Thanks, you can refer to PR #18
comment
created
javeedf
username_1
Sure Ludovic
comment
closed
skg-net
username_3
issue