opened
mclate
username_0
ES version: 7.1

Error log:
```
Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties
[2019-09-04T23:43:08,613][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"/usr/share/logstash/data/queue"}
[2019-09-04T23:43:08,742][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/usr/share/logstash/data/dead_letter_queue"}
[2019-09-04T23:43:09,463][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.1.1"}
[2019-09-04T23:43:09,512][INFO ][logstash.agent           ] No persistent UUID file found. Generating new UUID {:uuid=>"81a1f248-f42e-4b21-942a-daf248b2573a", :path=>"/usr/share/logstash/data/uuid"}
url template
{:scheme=>nil, :user=>nil, :password=>nil, :host=>"URLTEMPLATE", :port=>443, :path=>nil}
[2019-09-04T23:43:21,304][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[https://.....us-east-1.es.amazonaws.com:443/]}}
[2019-09-04T23:43:21,318][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>https://.....us-east-1.es.amazonaws.com:443/, :path=>"/"}
[2019-09-04T23:43:22,476][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"https://.....us-east-1.es.amazonaws.com:443/"}
[2019-09-04T23:43:22,583][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>7}
[2019-09-04T23:43:22,588][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
[2019-09-04T23:43:22,624][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//.....us-east-1.es.amazonaws.com"]}
[2019-09-04T23:43:22,663][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2019-09-04T23:43:22,672][INFO ][logstash.javapipeline    ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>125, :thread=>"#<Thread:0x2c42be78 run>"}
[2019-09-04T23:43:22,718][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s", "number_of_shards"=>1}, "mappings"=>{"_doc"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2019-09-04T23:43:22,851][INFO ][logstash.javapipeline    ] Pipeline started {"pipeline.id"=>"main"}
[2019-09-04T23:43:22,964][INFO ][logstash.outputs.elasticsearch] Installing amazon_es template to /_template/logstash
[2019-09-04T23:43:22,965][INFO ][logstash.inputs.gelf     ] Starting gelf listener (udp) ... {:address=>"0.0.0.0:12201"}
[2019-09-04T23:43:23,068][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2019-09-04T23:43:23,162][ERROR][logstash.outputs.elasticsearch] Failed to install template. {:message=>"Got response code '400' contacting Elasticsearch at URL 'https://.....us-east-1.es.amazonaws.com:443/_template/logstash'", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError", :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client/manticore_adapter.rb:126:in `perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client/pool.rb:291:in `perform_request_to_url'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client/pool.rb:278:in `block in perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client/pool.rb:373:in `with_connection'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client/pool.rb:277:in `perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client/pool.rb:285:in `block in Pool'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client.rb:365:in `template_put'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client.rb:88:in `template_install'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/template_manager.rb:21:in `install'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/template_manager.rb:9:in `install_template'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/common.rb:118:in `install_template'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/common.rb:49:in `block in install_template_after_successful_connection'"]}
[2019-09-04T23:43:23,759][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
```

I've tried to manually install template from `elasticsearch-template-es7x.json` and got next error:
```json
{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "The mapping definition cannot be nested under a type [_doc] unless include_type_name is set to true."
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "The mapping definition cannot be nested under a type [_doc] unless include_type_name is set to true."
  },
  "status": 400
}
```
Failed to install template on AWS ES 7.1
issue
created
asomir
username_1
I have the same problem
comment
created
mclate
username_0
So far i was able to fix it by manually applying logstash template using data from `elasticsearch-template-es7x.json` and applying it to `POST _template/logstash?include_type_name=true`
comment
created
ksemaev
username_2
@username_1 @username_0  Thanks for the help guys! Still the console just hangs after that request with no output :(  any ` GET /_cat/templates?v&s=name` showes no templates at all 
Tried with
```
POST _template/logstash?include_type_name=true
{
  "index_patterns" : "logstash-*",
  "version" : 60001,
  "settings" : {
    "index.refresh_interval" : "5s",
    "number_of_shards": 1
  },
  "mappings" : {
    "dynamic_templates" : [ {
      "message_field" : {
        "path_match" : "message",
        "match_mapping_type" : "string",
        "mapping" : {
          "type" : "text",
          "norms" : false
        }
      }
    }, {
      "string_fields" : {
        "match" : "*",
        "match_mapping_type" : "string",
        "mapping" : {
          "type" : "text", "norms" : false,
          "fields" : {
            "keyword" : { "type": "keyword", "ignore_above": 256 }
          }
        }
      }
    } ],
    "properties" : {
      "@timestamp": { "type": "date"},
      "@version": { "type": "keyword"},
      "geoip"  : {
        "dynamic": true,
        "properties" : {
          "ip": { "type": "ip" },
          "location" : { "type" : "geo_point" },
          "latitude" : { "type" : "half_float" },
          "longitude" : { "type" : "half_float" }
        }
      }
    }
  }
}
```
Any other advices are highly appreciated!
comment
created
briandicola
username_3
Please correct me if I'm wrong, but I believe this plugin just does not support version 7 ELK.  I've gotten it to work with logstash 1:6.4.3-1 and AWS ES version 6.4.
comment
created
mclate
username_0
@username_2 here is what worked for me (copied from kibana interface):
```json
POST _template/logstash?include_type_name=true
{
  "template" : "logstash-*",
  "version" : 60001,
  "settings" : {
    "index.refresh_interval" : "5s",
    "number_of_shards": 1
  },
  "mappings" : {
    "_doc" : {
      "dynamic_templates" : [
        {
          "message_field" : {
            "path_match" : "message",
            "match_mapping_type" : "string",
            "mapping" : {
              "type" : "text",
              "norms" : false
            }
          }
        },
        {
          "string_fields" : {
            "match" : "*",
            "match_mapping_type" : "string",
            "mapping" : {
              "type" : "text", "norms" : false,
              "fields" : {
                "keyword" : { "type": "keyword", "ignore_above": 256 }
              }
            }
          }
        }
      ],
      "properties" : {
        "@timestamp": { "type": "date"},
        "@version": { "type": "keyword"},
        "geoip"  : {
          "dynamic": true,
          "properties" : {
            "ip": { "type": "ip" },
            "location" : { "type" : "geo_point" },
            "latitude" : { "type" : "half_float" },
            "longitude" : { "type" : "half_float" }
          }
        }
      }
    }
  }
}
```
You should get something like `{"acknowledged": true}`

@username_3 Guess it's not, but aside from applying template during startup it works fine for us - i can see records being sent to our ES cluster. No issues so far.
comment
created
malpani
username_4
I will bring this up to the team to prioritize a fix.

In the meantime, thanks for the workaround and if you already have a PR, happy to pull it in
comment
created
briandicola
username_3
This worked for me.  Thanks.
comment
created
yadavcbala
username_5
The pull request #130 should resolve this issue. 
*From elasticsearch-7.0, types have been removed and request parameter include_type_name which omits types is defaulted to false. This will fail while installing the template with types. The PR #130 updates the `elasticsearch-template-es7x.json` template to remove types*
comment
closed
yadavcbala
username_5
issue