Repository: milvus-io/milvus

Issue #15896: [Bug]: [benchmark][standalone] Milvus Standalone restart frequently in stability testing
Created: 2022-03-07T06:21:43Z
Is there an existing issue for this?

 I have searched the existing issues

Environment
- Milvus version:2.0.2-20220304-3f34bc1a
- Deployment mode(standalone or cluster): standalone
- SDK version(e.g. pymilvus v2.0.0rc2):pymilvus==2.0.2.dev5
same as issue :#15523,#15392
Current Behavior
Milvus standalone pod restarts for times during stability testing.  (it is recoverable, but we shall know why is restarts)
client pod: benchmark-ws6vf-3397330533
client log: client.log.zip
[2022-03-06 18:51:34,053] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.040373', 'RPC error': '2022-03-06 18:51:34.053211'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,054] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.043259', 'RPC error': '2022-03-06 18:51:34.054328'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,054] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.043252', 'RPC error': '2022-03-06 18:51:34.054553'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,055] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.045466', 'RPC error': '2022-03-06 18:51:34.055628'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,055] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.045458', 'RPC error': '2022-03-06 18:51:34.055874'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,056] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.043383', 'RPC error': '2022-03-06 18:51:34.056146'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,056] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.043378', 'RPC error': '2022-03-06 18:51:34.056359'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,056] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.043332', 'RPC error': '2022-03-06 18:51:34.056618'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,056] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.043328', 'RPC error': '2022-03-06 18:51:34.056830'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,057] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.043431', 'RPC error': '2022-03-06 18:51:34.057111'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,057] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.043427', 'RPC error': '2022-03-06 18:51:34.057327'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,057] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.053610', 'RPC error': '2022-03-06 18:51:34.057957'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,058] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.053604', 'RPC error': '2022-03-06 18:51:34.058218'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,059] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.053678', 'RPC error': '2022-03-06 18:51:34.059974'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,060] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.053674', 'RPC error': '2022-03-06 18:51:34.060231'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,060] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.053734', 'RPC error': '2022-03-06 18:51:34.060504'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,060] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.053730', 'RPC error': '2022-03-06 18:51:34.060781'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,061] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.053781', 'RPC error': '2022-03-06 18:51:34.061051'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,061] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.053777', 'RPC error': '2022-03-06 18:51:34.061262'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,061] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.053829', 'RPC error': '2022-03-06 18:51:34.061569'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,061] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.053825', 'RPC error': '2022-03-06 18:51:34.061802'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,062] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.053874', 'RPC error': '2022-03-06 18:51:34.062064'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,062] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.053870', 'RPC error': '2022-03-06 18:51:34.062265'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,062] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.053921', 'RPC error': '2022-03-06 18:51:34.062527'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,062] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.053917', 'RPC error': '2022-03-06 18:51:34.062742'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,063] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.053965', 'RPC error': '2022-03-06 18:51:34.062990'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,063] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.053962', 'RPC error': '2022-03-06 18:51:34.063175'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,063] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.054054', 'RPC error': '2022-03-06 18:51:34.063452'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,063] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.054051', 'RPC error': '2022-03-06 18:51:34.063677'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,063] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.054010', 'RPC error': '2022-03-06 18:51:34.063938'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,064] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.054006', 'RPC error': '2022-03-06 18:51:34.064122'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,064] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.054099', 'RPC error': '2022-03-06 18:51:34.064358'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,064] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.054096', 'RPC error': '2022-03-06 18:51:34.064618'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,064] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.054143', 'RPC error': '2022-03-06 18:51:34.064937'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,065] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.054139', 'RPC error': '2022-03-06 18:51:34.065181'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,065] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.054189', 'RPC error': '2022-03-06 18:51:34.065622'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,066] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.054185', 'RPC error': '2022-03-06 18:51:34.066874'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,068] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.054231', 'RPC error': '2022-03-06 18:51:34.068602'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,069] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.054228', 'RPC error': '2022-03-06 18:51:34.069459'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,071] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.054276', 'RPC error': '2022-03-06 18:51:34.070952'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,073] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.054273', 'RPC error': '2022-03-06 18:51:34.072895'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,076] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.055559', 'RPC error': '2022-03-06 18:51:34.076709'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,079] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.055552', 'RPC error': '2022-03-06 18:51:34.079036'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,087] [   DEBUG] - [scene_test] Start scene test : scene_test_920_100720 (milvus_benchmark.client:511)
[2022-03-06 18:51:34,091] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.057730', 'RPC error': '2022-03-06 18:51:34.091032'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,093] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.057723', 'RPC error': '2022-03-06 18:51:34.092953'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,097] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.057803', 'RPC error': '2022-03-06 18:51:34.097911'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,098] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.057799', 'RPC error': '2022-03-06 18:51:34.098671'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,100] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.057905', 'RPC error': '2022-03-06 18:51:34.100152'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,100] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.057900', 'RPC error': '2022-03-06 18:51:34.100721'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,101] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.059906', 'RPC error': '2022-03-06 18:51:34.101575'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,102] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.059899', 'RPC error': '2022-03-06 18:51:34.102338'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,103] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.057857', 'RPC error': '2022-03-06 18:51:34.103072'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,103] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.057853', 'RPC error': '2022-03-06 18:51:34.103533'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,105] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.083710', 'RPC error': '2022-03-06 18:51:34.105445'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,106] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.083648', 'RPC error': '2022-03-06 18:51:34.106108'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,109] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.085119', 'RPC error': '2022-03-06 18:51:34.109217'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,109] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.085102', 'RPC error': '2022-03-06 18:51:34.109540'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,109] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.084387', 'RPC error': '2022-03-06 18:51:34.109882'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,110] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.084353', 'RPC error': '2022-03-06 18:51:34.110233'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,111] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.084797', 'RPC error': '2022-03-06 18:51:34.110983'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,111] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.084773', 'RPC error': '2022-03-06 18:51:34.111240'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,111] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.085613', 'RPC error': '2022-03-06 18:51:34.111544'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,111] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.085599', 'RPC error': '2022-03-06 18:51:34.111797'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,112] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.085359', 'RPC error': '2022-03-06 18:51:34.112093'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,112] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.085346', 'RPC error': '2022-03-06 18:51:34.112307'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,112] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.085769', 'RPC error': '2022-03-06 18:51:34.112585'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,112] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.085760', 'RPC error': '2022-03-06 18:51:34.112966'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,113] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.085908', 'RPC error': '2022-03-06 18:51:34.113379'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,113] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.085896', 'RPC error': '2022-03-06 18:51:34.113814'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,114] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.086072', 'RPC error': '2022-03-06 18:51:34.114252'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,114] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.086058', 'RPC error': '2022-03-06 18:51:34.114805'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,115] [   ERROR] - RPC error: [load_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.086253', 'RPC error': '2022-03-06 18:51:34.115565'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,116] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.086551', 'RPC error': '2022-03-06 18:51:34.116285'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,116] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.086533', 'RPC error': '2022-03-06 18:51:34.116516'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,116] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.086803', 'RPC error': '2022-03-06 18:51:34.116795'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,117] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.086785', 'RPC error': '2022-03-06 18:51:34.117076'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,117] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.087053', 'RPC error': '2022-03-06 18:51:34.117459'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,117] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.087038', 'RPC error': '2022-03-06 18:51:34.117813'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,118] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.087257', 'RPC error': '2022-03-06 18:51:34.118248'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,118] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.087247', 'RPC error': '2022-03-06 18:51:34.118578'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,119] [   ERROR] - error_code: UnexpectedError
reason: "proxy not healthy"
 (pymilvus.client.grpc_handler:144)
[2022-03-06 18:51:34,119] [   ERROR] - RPC error: [create_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.090134', 'RPC error': '2022-03-06 18:51:34.119308'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,119] [   ERROR] - <BaseException: (code=1, message=proxy not healthy)> (milvus_benchmark.client:151)
[2022-03-06 18:51:34,119] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.097646', 'RPC error': '2022-03-06 18:51:34.119958'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,120] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.097609', 'RPC error': '2022-03-06 18:51:34.120259'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,121] [   DEBUG] - [scene_test] Start scene test : scene_test_3725_496051 (milvus_benchmark.client:511)
[2022-03-06 18:51:34,122] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.104752', 'RPC error': '2022-03-06 18:51:34.122163'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,122] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.104732', 'RPC error': '2022-03-06 18:51:34.122440'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,124] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.109107', 'RPC error': '2022-03-06 18:51:34.124057'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,124] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.109096', 'RPC error': '2022-03-06 18:51:34.124305'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,124] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.105170', 'RPC error': '2022-03-06 18:51:34.124694'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,124] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.105161', 'RPC error': '2022-03-06 18:51:34.124929'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,125] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.105016', 'RPC error': '2022-03-06 18:51:34.125201'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,125] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.105004', 'RPC error': '2022-03-06 18:51:34.125408'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,125] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.105289', 'RPC error': '2022-03-06 18:51:34.125688'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,125] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.105279', 'RPC error': '2022-03-06 18:51:34.125907'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,126] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.120791', 'RPC error': '2022-03-06 18:51:34.126573'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,126] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.120783', 'RPC error': '2022-03-06 18:51:34.126839'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,128] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.120888', 'RPC error': '2022-03-06 18:51:34.128756'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,129] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.120884', 'RPC error': '2022-03-06 18:51:34.129212'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,129] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.120945', 'RPC error': '2022-03-06 18:51:34.129866'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,130] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.120940', 'RPC error': '2022-03-06 18:51:34.130287'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,130] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.121531', 'RPC error': '2022-03-06 18:51:34.130562'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,130] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.121524', 'RPC error': '2022-03-06 18:51:34.130782'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,131] [   ERROR] - error_code: UnexpectedError
reason: "proxy not healthy"
 (pymilvus.client.grpc_handler:144)
[2022-03-06 18:51:34,131] [   ERROR] - RPC error: [create_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.121326', 'RPC error': '2022-03-06 18:51:34.131230'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,131] [   ERROR] - <BaseException: (code=1, message=proxy not healthy)> (milvus_benchmark.client:151)
[2022-03-06 18:51:34,131] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.121606', 'RPC error': '2022-03-06 18:51:34.131732'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,131] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.121601', 'RPC error': '2022-03-06 18:51:34.131964'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,132] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.121665', 'RPC error': '2022-03-06 18:51:34.132240'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,132] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.121661', 'RPC error': '2022-03-06 18:51:34.132458'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,132] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.121741', 'RPC error': '2022-03-06 18:51:34.132773'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,133] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.121736', 'RPC error': '2022-03-06 18:51:34.133066'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,133] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.121845', 'RPC error': '2022-03-06 18:51:34.133332'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,133] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.121840', 'RPC error': '2022-03-06 18:51:34.133528'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,133] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.121796', 'RPC error': '2022-03-06 18:51:34.133777'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,134] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.121792', 'RPC error': '2022-03-06 18:51:34.134031'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,134] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.121894', 'RPC error': '2022-03-06 18:51:34.134297'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,134] [   ERROR] - RPC error: [query], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.121890', 'RPC error': '2022-03-06 18:51:34.134501'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,134] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.121997', 'RPC error': '2022-03-06 18:51:34.134769'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,134] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.121993', 'RPC error': '2022-03-06 18:51:34.134975'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,135] [   ERROR] - RPC error: [has_collection], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.122050', 'RPC error': '2022-03-06 18:51:34.135224'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,135] [   ERROR] - RPC error: [search], <BaseException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.122046', 'RPC error': '2022-03-06 18:51:34.135431'}> (pymilvus.decorators:73)
[2022-03-06 18:51:34,135] [   ERROR] - RPC error: [describe_collection], <DescribeCollectionException: (code=1, message=proxy not healthy)>, <Time:{'RPC start': '2022-03-06 18:51:34.121943', 'RPC error': '2022-03-06 18:51:34.135721'}> (pymilvus.decorators:73)


Expected Behavior
server-instance
benchmark-ws6vf-1
server-configmap
server-single-32c128m
client-configmap
client-random-locust-100m-ddl-r8-w2
server ：
NAME                                                     READY   STATUS      RESTARTS   AGE    IP             NODE                      NOMINATED
NODE   READINESS GATES
benchmark-ws6vf-1-etcd-0                               1/1     Running       0          2d22h   10.97.5.155    qa-node003.zilliz.local   <none>           <none>
benchmark-ws6vf-1-milvus-standalone-7b6dbf4bc9-b6slq   1/1     Running       11         2d22h   10.97.17.208   qa-node014.zilliz.local   <none>           <none>
benchmark-ws6vf-1-minio-5466d68458-hpdv2               1/1     Running       0          2d22h   10.97.12.40    qa-node015.zilliz.local   <none>           <none>

Steps To Reproduce
1、create collection
2、create index of ivf_sq8
3、insert 100m million vectors
4、flush collection
5、build index with the same params
6、load collection
7、locust concurrent: query<-search, load, get<-query, scene_test>raise error
Anything else?
client-random-locust-100m-ddl-r8-w2
{
	"config.yaml": "locust_random_performance:
		  collections:
		    -
		      collection_name: sift_100m_128_l2
		      ni_per: 50000
		      build_index: true
		      index_type: ivf_sq8
		      index_param:
		        nlist: 2048
		      task:
		        types:
		          -
		            type: query
		            weight: 8
		            params:
		              top_k: 10
		              nq: 10
		              search_param:
		                nprobe: 16
		          -
		            type: load
		            weight: 1
		          -
		            type: get
		            weight: 8
		            params:
		              ids_length: 10
		          -
		            type: scene_test
		            weight: 2
		        connection_num: 1
		        clients_num: 20
		        spawn_rate: 2
		        during_time: 302400
		"
}

 def scene_test(self, collection_name=None, vectors=None, ids=None):
        logger.debug("[scene_test] Start scene test : %s" % collection_name)
        self.create_collection(dimension=128, collection_name=collection_name)
        time.sleep(1)

        collection_info = self.get_info(collection_name)

        entities = utils.generate_entities(collection_info, vectors, ids)
        logger.debug("[scene_test] Start insert : %s" % collection_name)
        self.insert(entities, collection_name=collection_name)
        logger.debug("[scene_test] Start flush : %s" % collection_name)
        self.flush(collection_name=collection_name)

        self.count(collection_name=collection_name)

        logger.debug("[scene_test] Start create index : %s" % collection_name)
        self.create_index(field_name='float_vector', index_type="ivf_sq8", metric_type='l2',
                          collection_name=collection_name, index_param={'nlist': 2048})
        time.sleep(59)

        logger.debug("[scene_test] Start drop : %s" % collection_name)
        self.drop(collection_name=collection_name)
        logger.debug("[scene_test]Scene test close : %s" % collection_name)

Duplicate issue #16119: Lab Test 2: 100-Million-Scale Vector Similarity Search ,after the load the command,querynode crash[Bug]: 
Created: 2022-03-21T04:26:17Z
Is there an existing issue for this?

 I have searched the existing issues

Environment
- Milvus version:milvus-dev:master-20220316-d65cf530
- Deployment mode(standalone or cluster):cluster
- SDK version(e.g. pymilvus v2.0.0rc2):2.0.0
- OS(Ubuntu or CentOS): Ubuntu
- CPU/Memory:  Intel(R) Xeon(R) Platinum 8374C CPU @ 2.70GHz(16 core)/128G
- GPU: no
- Others:
Current Behavior
after exec the command python3 main.py --collection ann_100m_sq8 --load,the querynode crash
Expected Behavior
load the data success
logs202203211215.tar.gz
Steps To Reproduce
1: python3 main.py --collection ann_100m_sq8 --create
2:python3 main.py --collection=ann_100m_sq8 --insert
3:python3 main.py --collection ann_100m_sq8 --index_type IVF_SQ8 --create_index
4:python3 main.py --collection ann_100m_sq8 --load
Anything else?
No response

Closing comment (2022-03-21T05:09:13Z): in querynode log, the "unexpected signal" is found:
{"log":"fatal error: unexpected signal during runtime execution\n","stream":"stderr","time":"2022-03-21T02:31:53.938962678Z"}
{"log":"fatal error: unexpected signal during runtime execution\n","stream":"stderr","time":"2022-03-21T02:31:53.939005195Z"}
{"log":"fatal error: unexpected signal during runtime execution\n","stream":"stderr","time":"2022-03-21T02:31:53.939044079Z"}
{"log":"[signal SIGSEGV: segmentation violation code=0x80 addr=0x0 pc=0x7f9ef0641583]\n","stream":"stderr","time":"2022-03-21T02:31:53.941455385Z"}
{"log":"\n","stream":"stderr","time":"2022-03-21T02:31:53.94153869Z"}
{"log":"runtime stack:\n","stream":"stderr","time":"2022-03-21T02:31:53.941546454Z"}
{"log":"runtime.throw(0x33d4ab4, 0x2a)\n","stream":"stderr","time":"2022-03-21T02:31:53.941828489Z"}
{"log":"\u0009/usr/local/go/src/runtime/panic.go:1117 +0x72\n","stream":"stderr","time":"2022-03-21T02:31:53.942120756Z"}
{"log":"runtime.sigpanic()\n","stream":"stderr","time":"2022-03-21T02:31:53.942254702Z"}
{"log":"\u0009/usr/local/go/src/runtime/signal_unix.go:718 +0x2e5\n","stream":"stderr","time":"2022-03-21T02:31:53.942502056Z"}
{"log":"\n","stream":"stderr","time":"2022-03-21T02:31:53.942558555Z"}
{"log":"goroutine 3622430 [syscall]:\n","stream":"stderr","time":"2022-03-21T02:31:53.942748022Z"}
{"log":"runtime.cgocall(0x1c6bd9f, 0xc00482cc88, 0xc002f2e008)\n","stream":"stderr","time":"2022-03-21T02:31:53.943035472Z"}
{"log":"\u0009/usr/local/go/src/runtime/cgocall.go:154 +0x5b fp=0xc00482cc58 sp=0xc00482cc20 pc=0x7c9b3b\n","stream":"stderr","time":"2022-03-21T02:31:53.943504483Z"}
{"log":"github.com/milvus-io/milvus/internal/querynode._Cfunc_Search(0x7f9e340058e0, 0x7f9cf8ee57b0, 0x7f97585492f0, 0x5fea94443cc0007, 0xc002f2e008, 0x5fe9472da1c0005, 0xd1a500000000, 0x0)\n","stream":"stderr","time":"2022-03-21T02:31:53.944139056Z"}
{"log":"\u0009_cgo_gotypes.go:727 +0x51 fp=0xc00482cc88 sp=0xc00482cc58 pc=0x1c3ab71\n","stream":"stderr","time":"2022-03-21T02:31:53.944607503Z"}
{"log":"github.com/milvus-io/milvus/internal/querynode.(*Segment).search.func1(0xc000511c70, 0xc002ebf4a0, 0x7f97585492f0, 0x5fea94443cc0007, 0xc002f2e008, 0xc002f2e000, 0x0)\n","stream":"stderr","time":"2022-03-21T02:31:53.945180589Z"}
{"log":"\u0009/go/src/github.com/milvus-io/milvus/internal/querynode/segment.go:315 +0x125 fp=0xc00482ccf0 sp=0xc00482cc88 pc=0x1c55805\n","stream":"stderr","time":"2022-03-21T02:31:53.945657403Z"}
{"log":"github.com/milvus-io/milvus/internal/querynode.(*Segment).search(0xc000511c70, 0xc002ebf4a0, 0xc002ebf4b8, 0x1, 0x1, 0xc00482ced0, 0x1, 0x1, 0x0, 0x0, ...)\n","stream":"stderr","time":"2022-03-21T02:31:53.946494537Z"}
{"log":"\u0009/go/src/github.com/milvus-io/milvus/internal/querynode/segment.go:315 +0x3b7 fp=0xc00482ce50 sp=0xc00482ccf0 pc=0x1c4baf7\n","stream":"stderr","time":"2022-03-21T02:31:53.946943627Z"}
{"log":"github.com/milvus-io/milvus/internal/querynode.(*historical).searchSegments.func1(0xc0039d20c0, 0xc002ebf4a0, 0xc002ebf4b8, 0x1, 0x1, 0x5fea94443cc0007, 0xc0039d20b0, 0xc0039c8fc0, 0xc0039ce198, 0xc0039ce1b0, ...)\n","stream":"stderr","time":"2022-03-21T02:31:53.947761347Z"}
{"log":"\u0009/go/src/github.com/milvus-io/milvus/internal/querynode/historical.go:189 +0x12c fp=0xc00482cf88 sp=0xc00482ce50 pc=0x1c4f34c\n","stream":"stderr","time":"2022-03-21T02:31:53.948224167Z"}
{"log":"runtime.goexit()\n","stream":"stderr","time":"2022-03-21T02:31:53.948336496Z"}
{"log":"\u0009/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc00482cf90 sp=0xc00482cf88 pc=0x839201\n","stream":"stderr","time":"2022-03-21T02:31:53.948809564Z"}
{"log":"created by github.com/milvus-io/milvus/internal/querynode.(*historical).searchSegments\n","stream":"stderr","time":"2022-03-21T02:31:53.948921851Z"}
{"log":"\u0009/go/src/github.com/milvus-io/milvus/internal/querynode/historical.go:181 +0x330\n","stream":"stderr","time":"2022-03-21T02:31:53.949209733Z"}
{"log":"\n","stream":"stderr","time":"2022-03-21T02:31:53.949261259Z"}
{"log":"goroutine 1 [chan receive, 3841 minutes]:\n","stream":"stderr","time":"2022-03-21T02:31:53.949547069Z"}
{"log":"github.com/milvus-io/milvus/cmd/roles.(*MilvusRoles).Run(0xc000795ea0, 0xc000010000, 0x0, 0x0)\n","stream":"stderr","time":"2022-03-21T02:31:53.949912006Z"}
{"log":"\u0009/go/src/github.com/milvus-io/milvus/cmd/roles/roles.go:496 +0x438\n","stream":"stderr","time":"2022-03-21T02:31:53.950177283Z"}
{"log":"github.com/milvus-io/milvus/cmd/milvus.RunMilvus(0xc00003c090, 0x3, 0x3)\n","stream":"stderr","time":"2022-03-21T02:31:53.95051464Z"}
{"log":"\u0009/go/src/github.com/milvus-io/milvus/cmd/milvus/milvus.go:316 +0x972\n","stream":"stderr","time":"2022-03-21T02:31:53.950728164Z"}
{"log":"main.main()\n","stream":"stderr","time":"2022-03-21T02:31:53.950825833Z"}
{"log":"\u0009/go/src/github.com/milvus-io/milvus/cmd/main.go:26 +0x45\n","stream":"stderr","time":"2022-03-21T02:31:53.95104214Z"}
{"log":"\n","stream":"stderr","time":"2022-03-21T02:31:53.951075074Z"}
{"log":"goroutine 17 [syscall, locked to thread]:\n","stream":"stderr","time":"2022-03-21T02:31:53.951258703Z"}
{"log":"runtime.goexit()\n","stream":"stderr","time":"2022-03-21T02:31:53.951347044Z"}
{"log":"\u0009/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1\n","stream":"stderr","time":"2022-03-21T02:31:53.95157288Z"}
{"log":"\n","stream":"stderr","time":"2022-03-21T02:31:53.951603598Z"}
{"log":"goroutine 80960 [select, 7 minutes]:\n","stream":"stderr","time":"2022-03-21T02:31:53.9518385Z"}
{"log":"fatal error: unexpected signal during runtime execution\n","stream":"stderr","time":"2022-03-21T02:31:53.965047447Z"}
{"log":"fatal error: unexpected signal during runtime execution\n","stream":"stderr","time":"2022-03-21T02:31:53.986787942Z"}

Might be the same as #15896  @soothing-rain @cydrain