Nutanix AOS5.10 のタスクで「Software Download」が残ってしまった場合の対処方法

#「Software Download」が残る
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/162569/490441a9-5d85-faa8-1f5a-f7dca43ffc81.png)

Software Download が87%で止まっている状態です。
これは NCC をダウンロード中に CVM を再起動したため発生しています。

#タスクには載っているのに…？
AOS5.10 だと、タスク関連のステータスは ecli で確認できました。
しかしながら kRunning は確認できず。
Cluster 関連のタスクも見えないので確認は別のコマンドなのか。

```shell
$ ecli task.list
Task UUID                             Parent Task UUID                      Component            Sequence-id  Type                Status
e5a67f47-ae6e-4c13-bc89-fa10685957a9  28f87d81-b1b5-4240-8fd8-e7a78a21e6e7  NCC                  16           HealthCheck         kSucceeded
28f87d81-b1b5-4240-8fd8-e7a78a21e6e7                                        NCC                  15           HealthCheck         kSucceeded
8173f01b-1a1c-45ba-8f40-e77ca82a956e  b2b583ea-9710-4c78-942b-b8fadcca7fe1  Acropolis            98           kVmNicCreate        kSucceeded
b2b583ea-9710-4c78-942b-b8fadcca7fe1                                        Uhura                61           VmNicCreate         kSucceeded
7a840e80-2d1a-4b0f-9c4a-3ac44dcd0602  d1470600-e422-451e-b91f-899a7631e410  Acropolis            97           kVmDiskUpdate       kSucceeded
d1470600-e422-451e-b91f-899a7631e410                                        Uhura                60           VmDiskUpdate        kSucceeded
87a4f08c-9fa8-4c35-ab87-83f75cbdd7a4                                        Acropolis            96           kUpdateVmDbState    kSucceeded
4fddd76f-a7ed-45af-9985-6d1aec9438aa  3e95ccb6-e018-4629-a360-fefc39aa85aa  Acropolis            95           kVmDiskUpdate       kSucceeded
3e95ccb6-e018-4629-a360-fefc39aa85aa                                        Uhura                59           VmDiskUpdate        kSucceeded
50429329-707a-4065-8f6f-cf798fd83361                                        Nutanix Guest Tools  3            MountGuestTools     kSucceeded
e255d783-203d-4fe6-a481-9a7007202980  78ab7a11-37c6-45b0-9677-39ae20d1eef3  Acropolis            94           kVmUpdate           kSucceeded
78ab7a11-37c6-45b0-9677-39ae20d1eef3                                        Uhura                58           VmUpdate            kSucceeded
4bacd411-9635-49ab-9e24-b5cad5cf7d12  9525df9b-ed01-4ce8-84f4-a681e7db571a  Acropolis            93           kVmDiskUpdate       kSucceeded
9525df9b-ed01-4ce8-84f4-a681e7db571a                                        Uhura                57           VmDiskUpdate        kSucceeded
ae903658-e64b-4d00-90de-1e1fcc3a00b7                                        Acropolis            92           kUpdateVmDbState    kSucceeded
bbb676d5-072d-4edc-894b-1335e44a764f  6115923c-ab7d-4ee3-be76-0d991b7b9f7a  Acropolis            91           kVmDiskUpdate       kSucceeded
6115923c-ab7d-4ee3-be76-0d991b7b9f7a                                        Uhura                56           VmDiskUpdate        kSucceeded
c2598993-72c8-4f27-8002-8463283d5438  50e50a9a-f60e-4e2e-92de-ae9702ce17da  Acropolis            90           kVmDiskUpdate       kSucceeded
50e50a9a-f60e-4e2e-92de-ae9702ce17da                                        Uhura                55           VmDiskUpdate        kSucceeded
4ac48abb-3115-4da2-9420-bda256844529                                        Acropolis            89           kUpdateVmDbState    kSucceeded
f9f87e6a-8f06-460a-b6d9-96c4870563d9  11d14be8-db27-43ed-bee7-c8f1f6f609be  Acropolis            88           kVmDiskUpdate       kSucceeded
11d14be8-db27-43ed-bee7-c8f1f6f609be                                        Uhura                54           VmDiskUpdate        kSucceeded
9e5f6c6a-be26-488f-b559-51418ccf7516                                        Nutanix Guest Tools  2            MountGuestTools     kSucceeded
846f238c-14b5-4f42-a609-f7acd4cab760  1e0eeeab-7055-4028-9adc-91434bada22f  Acropolis            87           kVmSetPowerState    kSucceeded
1e0eeeab-7055-4028-9adc-91434bada22f                                        Uhura                53           VmChangePowerState  kSucceeded
9675b72e-e445-44ee-b0e9-d12cb9c23e9b  673fa251-e2a8-4ff9-841f-1d86c97ade55  Acropolis            86           kVmDiskUpdate       kSucceeded
673fa251-e2a8-4ff9-841f-1d86c97ade55                                        Uhura                52           VmDiskUpdate        kSucceeded
1ad4700c-96e0-4d34-a111-f2a73a3905d9                                        Nutanix Guest Tools  1            MountGuestTools     kSucceeded
6a5ea471-51da-43d8-bf74-56c400cf5cee                                        Acropolis            85           kUpdateVmDbState    kSucceeded
c9f9de98-7866-4720-b636-75d733274eb9                                        Acropolis            84           kUpdateVmDbState    kSucceeded
b532b272-1846-4993-8e7a-3a530007f82c  3a13d7df-6623-4c5b-90a5-6e9b4102c74f  Acropolis            83           kVmDiskUpdate       kSucceeded
3a13d7df-6623-4c5b-90a5-6e9b4102c74f                                        Uhura                51           VmDiskUpdate        kSucceeded
7d485789-9ab5-40cf-a1d3-c375a727465f  0c541aca-6fd3-404b-98f6-bc509bc11158  Acropolis            82           kVmDiskUpdate       kSucceeded
0c541aca-6fd3-404b-98f6-bc509bc11158                                        Uhura                50           VmDiskUpdate        kSucceeded
68f27871-cfa4-4908-bd18-af9453c093a5  de1714d1-59d8-49c4-a24b-27f371a1fe23  Acropolis            81           kVmSetPowerState    kSucceeded
de1714d1-59d8-49c4-a24b-27f371a1fe23                                        Uhura                49           VmChangePowerState  kSucceeded
d9fadc13-4318-4ea8-83ee-6e8fc5915fef  1b5f6b19-b195-4280-ad80-7ba22b4f0488  Acropolis            80           kVmCreate           kSucceeded
1b5f6b19-b195-4280-ad80-7ba22b4f0488                                        Uhura                48           VmCreate            kSucceeded
0011d1da-19c1-4404-be1f-db1d6d93e5d6  edd4c8d0-e61c-400e-a08e-01d3dd8f9fc6  Acropolis            79           kVmUpdate           kSucceeded
edd4c8d0-e61c-400e-a08e-01d3dd8f9fc6                                        Uhura                47           VmUpdate            kSucceeded
admin@:192.168.x.xxx:~$
```

#別のコマンドで確認できました。
 progress_monitor_cli --fetchall で確認できました。

```shell
$ progress_monitor_cli --fetchall
2019-05-04 01:59:05,425:636(0x7f767f44cc80):ZOO_INFO@log_env@943: Client environment:zookeeper.version=zookeeper C client 3.4.3
2019-05-04 01:59:05,426:636(0x7f767f44cc80):ZOO_INFO@log_env@947: Client environment:host.name=ntnx-2ee81f29-a-cvm
2019-05-04 01:59:05,426:636(0x7f767f44cc80):ZOO_INFO@log_env@954: Client environment:os.name=Linux
2019-05-04 01:59:05,426:636(0x7f767f44cc80):ZOO_INFO@log_env@955: Client environment:os.arch=3.10.0-862.11.6.el7.nutanix.20180829.cvm.x86_64
2019-05-04 01:59:05,426:636(0x7f767f44cc80):ZOO_INFO@log_env@956: Client environment:os.version=#1 SMP Wed Aug 29 22:07:34 UTC 2018
2019-05-04 01:59:05,426:636(0x7f767f44cc80):ZOO_INFO@zookeeper_init@991: Initiating client connection, host=zk1:9876 sessionTimeout=20000 watcher=0x55dfbd7db190 sessionId=0 sessionPasswd=<null> context=0x55dfc0178040 flags=0
Could not create logging file: Permission denied
COULD NOT CREATE A LOGGINGFILE 20190504-015905.636!2019-05-04 01:59:05,437:636(0x7f76772c3700):ZOO_INFO@zookeeper_interest@1914: Connecting to server 192.168.x.xxx:9876
2019-05-04 01:59:05,439:636(0x7f76772c3700):ZOO_INFO@zookeeper_interest@1951: Zookeeper handle state changed to ZOO_CONNECTING_STATE for socket [192.168.x.xxx:9876]
2019-05-04 01:59:05,439:636(0x7f76772c3700):ZOO_INFO@check_events@2125: initiated connection to server [192.168.x.xxx:9876]
2019-05-04 01:59:05,449:636(0x7f76772c3700):ZOO_INFO@check_events@2172: session establishment complete on server [192.168.x.xxx:9876], sessionId=0x16a4551416d4c35, negotiated timeout=20000
================== Proto Start =========================
logical_timestamp: 87
progress_info_id {
  operation: kDownload
  entity_type: kPackage
  entity_id: "3.7.0.2"
}
title_message: "Software Download"
start_time_secs: 1555859429
progress_task_list {
  component: kPrism
  task_tag: "ncc.3.7.0.2"
  start_time_secs: 1555859429
  percentage_complete: 87
  progress_status: kRunning
}
=================== Proto End ==========================
admin@:192.168.x.xxx:~$
```

#確認できたところで削除
使用するコマンドはこちら。
progress_monitor_cli -delete -operation="download" -entity_type="package" -entity_id="< ID >" 

引数は上記の実行結果より。
-operation="download" -entity_type="package"　は k を外し大文字を小文字にする。

```shell
progress_info_id {
  operation: kDownload
  entity_type: kPackage
  entity_id: "3.7.0.2"
}
```

実行の結果。

```shell

$ progress_monitor_cli -delete -operation="download" -entity_type="package" -entity_id="3.7.0.2"
2019-05-04 02:50:14,035:1858(0x7f1b90ccac80):ZOO_INFO@log_env@943: Client environment:zookeeper.version=zookeeper C client 3.4.3
2019-05-04 02:50:14,036:1858(0x7f1b90ccac80):ZOO_INFO@log_env@947: Client environment:host.name=ntnx-2ee81f29-a-cvm
2019-05-04 02:50:14,036:1858(0x7f1b90ccac80):ZOO_INFO@log_env@954: Client environment:os.name=Linux
2019-05-04 02:50:14,036:1858(0x7f1b90ccac80):ZOO_INFO@log_env@955: Client environment:os.arch=3.10.0-862.11.6.el7.nutanix.20180829.cvm.x86_64
2019-05-04 02:50:14,036:1858(0x7f1b90ccac80):ZOO_INFO@log_env@956: Client environment:os.version=#1 SMP Wed Aug 29 22:07:34 UTC 2018
2019-05-04 02:50:14,036:1858(0x7f1b90ccac80):ZOO_INFO@zookeeper_init@991: Initiating client connection, host=zk1:9876 sessionTimeout=20000 watcher=0x5641517d1190 sessionId=0 sessionPasswd=<null> context=0x564153230040 flags=0
Could not create logging file: Permission denied
COULD NOT CREATE A LOGGINGFILE 20190504-025014.1858!2019-05-04 02:50:14,047:1858(0x7f1b88b41700):ZOO_INFO@zookeeper_interest@1914: Connecting to server 192.168.x.xxx:9876
2019-05-04 02:50:14,053:1858(0x7f1b88b41700):ZOO_INFO@zookeeper_interest@1951: Zookeeper handle state changed to ZOO_CONNECTING_STATE for socket [192.168.x.xxx:9876]
2019-05-04 02:50:14,054:1858(0x7f1b88b41700):ZOO_INFO@check_events@2125: initiated connection to server [192.168.x.xxx:9876]
2019-05-04 02:50:14,057:1858(0x7f1b88b41700):ZOO_INFO@check_events@2172: session establishment complete on server [192.168.x.xxx:9876], sessionId=0x16a4551416d4d9b, negotiated timeout=20000
Delete progress monitor info finished successfully
admin@:192.168.x.xxx:~$

```

#結果
ちゃんと消えました！
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/162569/d49576b4-3e99-54cd-a8fe-739ca8698aa6.png)

タスクからは削除されている。
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/162569/d7a12dd8-9586-889c-8447-e17a836b8ec5.png)


以上です。