Issue 1:
Suppose we create an index with the following dynamic mapping in order to indicate that any element starting with 'nested_' should be typed as nested:

{
"test": {
"dynamic_templates":[
{
"nested_field": {
"mapping":{
"type":"nested"
},
"match":"nested_*"
}
}
]
}
}

If you want to perform a nested query when there is no data in the index yet, we'll get an exception. For example:

{
"query": {
"nested": {
"path": "nested_field",
"query": {
"match": {"nested_field.a": "foo"}
}
}
}
}

would give us a "failed to find nested object under path" exception.

We can avoid this error by declaring the mapping of the nested fields in a static way (instead of dynamic), but then you loose the power of these dynamic templates.

So my question is: would it be possible to silently ignore these "failed to find nested object under path" exceptions so that it makes sense to declare nested types inside the dynamic templates?

Issue 2:
Elasticsearch version: 2.3.3

JVM version: 1.8.0_91

OS version: Ubuntu 14.04

Description of the problem including expected versus actual behavior:

@jpountz I think I might have found a problem as a result of the quick fix from issue #18740. It doesn't give me the "Mixing up field types" error anymore but it tells me that the IPs are not valid:

Provide logs (if relevant):

source[{"@version":"1","@timestamp":"2016-06-27T21:16:52.085Z","domain_name":"2667500489.MARS.ORDERBOX-DNS.COM.","type":"A","ip_address":"162.251.82.124,162.251.82.125,162.251.82.252,162.251.82.253","domain_name_exact":"2667500489.MARS.ORDERBOX-DNS.COM.","domain_name_length":33,"ip_address_exact":"162.251.82.124,162.251.82.125,162.251.82.252,162.251.82.253","domain_name_segmented":["2667500489mars","order","box","dns"],"last_registered":"2016-05-01T10:00:00.000-0500"}]} MapperParsingException[failed to parse [ip_address]]; nested: IllegalArgumentException[failed to parse ip [162.251.82.124,162.251.82.125,162.251.82.252,162.251.82.253], not a valid ip address]; at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:329) at org.elasticsearch.index.mapper.DocumentParser.parseObjectOrField(DocumentParser.java:309) at org.elasticsearch.index.mapper.DocumentParser.parseValue(DocumentParser.java:436) at org.elasticsearch.index.mapper.DocumentParser.parseObject(DocumentParser.java:262) at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:122) at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:309) at org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:580) at org.elasticsearch.index.shard.IndexShard.prepareIndexOnPrimary(IndexShard.java:559) at org.elasticsearch.action.index.TransportIndexAction.prepareIndexOperationOnPrimary(TransportIndexAction.java:212) at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:224) at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326) at org.elasticsearch.action.bulk.TransportShardBulkAction.shardUpdateOperation(TransportShardBulkAction.java:389) at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:191) at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68) at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639) at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279) at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271) at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75) at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376) at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.IllegalArgumentException: failed to parse ip [162.251.82.124,162.251.82.125,162.251.82.252,162.251.82.253], not a valid ip address at org.elasticsearch.index.mapper.ip.IpFieldMapper.ipToLong(IpFieldMapper.java:84) at org.elasticsearch.index.mapper.ip.IpFieldMapper.innerParseCreateField(IpFieldMapper.java:351) at org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:241) at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:321) ... 23 more

Here's the mapping:

{
"settings": {
"number_of_shards": {{ num_shards }},
"number_of_replicas": 2,
"index":{
"analysis":{
"analyzer":{
"keylower":{
"tokenizer":"keyword",
"filter":"lowercase"
}
}
}
}
},
"mappings": {
"_default_": {
"properties": {
"ip_address":{
"type":"ip"
}
}
},
"NS": {
"properties": {
"domain_name": {
"type": "string",
"index": "analyzed",
"analyzer":"keylower"
},
"domain_name_exact": {
"type": "string",
"index":"not_analyzed"
},
"domain_name_length": {
"type": "integer"
},
"type": {
"type": "string",
"index": "not_analyzed"
},
"is_registered": {
"type": "boolean"
},
"last_registered": {
"type": "date"
},
"last_deregistered": {
"type": "date"
},
"registered_history": {
"type": "date"
},
"deregistered_history": {
"type": "date"
},
"name_servers": {
"type": "string",
"index": "analyzed"
},
"name_servers_exact": {
"type": "string",
"index":"not_analyzed"
},
"past_name_servers": {
"type": "string",
"index": "analyzed"
},
"past_name_servers_exact": {
"type": "string",
"index":"not_analyzed"
},
"domain_name_segmented": {
"type": "string",
"index": "not_analyzed"
},
"domain_name_exact_pronounceability": {
"type": "integer"
},
"domain_name_segmented_pronounceability": {
"type": "integer"
}
}
},
"A": {
"properties": {
"domain_name": {
"type": "string",
"index": "analyzed",
"analyzer":"keylower"
},
"domain_name_exact": {
"type": "string",
"index":"not_analyzed"
},
"domain_name_length": {
"type": "integer"
},
"type": {
"type": "string",
"index": "not_analyzed"
},
"ip_address": {
"type": "ip"
},
"ip_address_exact": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}

I'm not sure if this is unrelated but I've exhausted the docs and discussion at this point. I'm dealing with 20 shards(also 20 nodes), 2 replicas, 16GB of ram(8GB to heap), mlockall set to true, ran "sudo swapoff -a" on all the machines and still have problems with the GB not doing its job and nodes becoming unresponsive. Looking at the logs, I was getting errors like the one above before the GC warnings appeared, can they be related?