Issue 1:
I haven't found a way to create a custom datatype to reuse in mapping definition.

I would like to have a way to define custom_datatype in the mapping like that :

{
"settings" : {
"number_of_shards" : 1,
"custom_datatypes": [{
"string_with_raw": {
"mapping": {
"index": "analyzed",
"omit_norms": true,
"type": "string",
"fields": {
"raw": {
"ignore_above": 256,
"index": "not_analyzed",
"type": "string"
}
}
}
}
}],
},
"mappings" : {
"my_type": {
"properties": {
"field1": {
"type": "string_with_raw"
},
"field2": {
"type": "string_with_raw"
},
"field3": {
"type": "string"
}

}
}
}
}

For now I have to duplicate for every field :

"mappings": {
"my_type": {
"properties": {
"message": {
"index": "analyzed",
"omit_norms": true,
"type": "string",
"fields": {
"raw": {
"ignore_above": 256,
"index": "not_analyzed",
"type": "string"
}
}
},
"field2": {
"index": "analyzed",
"omit_norms": true,
"type": "string",
"fields": {
"raw": {
"ignore_above": 256,
"index": "not_analyzed",
"type": "string"
}
}
},
"field3": {
"type": "string"
}
}
}
}
}

It could also be used in dynamic mappings:

{
"settings" : {
"number_of_shards" : 1,
"custom_datatypes": [{
"string_with_raw": {
"mapping": {
"index": "analyzed",
"omit_norms": true,
"type": "string",
"fields": {
"raw": {
"ignore_above": 256,
"index": "not_analyzed",
"type": "string"
}
}
}
}
}],
},
"mappings" : {
"my_type": {
"dynamic_templates": [{
"message_field": {
"mapping": {
"omit_norms": false,
"type": "string_with_raw"
},
"match_mapping_type": "string",
"match": "*_with_raw"
}
}],
"properties": {
"field1": {
"type": "string_with_raw"
},
"field2": {
"type": "string_with_raw"
},
"field3": {
"type": "string"
}
}
}
}
}

For the dynamic_templates part, the value set in the mapping should override equivalent field defined in the custom datatype.

Is there a way to achieve that at the moment ?

Issue 2:
Under worst case conditions, a request that is bad for the system, thereby taking a long time to process, often gets repeated as a user--or even an automated system--becomes impatient while they await their response. If an exactly duplicated request comes into the same node to run on the same shard (otherwise it's not an exact match of course), then it would be ideal in all but one scenario to simply dupe the eventual response from the original request.

Unfortunately, the exact match nature of this eliminates a slew of requests from benefiting from it (at least until 3.0 when we can start to recognize inner parts better to drop irrelevant filters), but it provides a lot of protection from a very common problem: a user hitting refresh to send what was a recoverable situation into a downward spiral leading to OutOfMemoryError if you are lucky or, worse, permanent GCs that never recover.

The scenario where this is less ideal is the case where a request expects the most up-to-date information (e.g., details from segments that may have been created since the original request started running). Since search is not real time, and the performance of the system is at stake from a presumably slow, repeated request, this seems like an edge case to me. Perhaps providing a flag to use the existing behavior (just create another shard level request) would be enough to allow those cases to "demand" this behavior.

It should be as simple as adding a new response listener to the original request, which entirely side steps adding another thread to do the exact same work, but "simple" ends there as I do not believe we provide any way to easily find/attach to other requests. As a result, this is related to task management in #6914.