opened
mrchang7
username_0
Hi again,

I'm not sure this issue could be in some way related to the other one I raised. 

Here is my sample code to reproduce it. When the 2.4.0 aggregate plugin is used with split, it looks it does not pass the maps over to the split events. This works correct with the lower version.

With the following code run, I tried providing the std input like this (two lines of records):
t1 f1
t2 f2

And then checked if the s_data getting the correct value assigned before split, which is "value". But the result shows 'nil' for this field of s_data. 

`
input {
    stdin {}
}

filter {

    grok { match => { "message" => "%{DATA:t_id} %{DATA:f_id}$" } }

    aggregate {
        task_id => "%{t_id}"
        code => "
                if map['f_ids']
                    map['f_ids'] << event['f_id']
                else
                    map['f_ids'] = [event['f_id']]
                end
                event['f_ids'] = map['f_ids']
                "
    }

    aggregate {
        task_id => "%{f_id}"
        code => "
                map['data'] = 'value'
                event['data'] = map['data']
                "
    }

    if [f_id] == "f2" {
        split {
            field => "f_ids"
            target => "s_id"
        }
        aggregate {
            task_id => "%{s_id}"
            code => "
                    event['s_data'] = map['data']
                    "
        }
    }

}

output {
    stdout { codec => rubydebug }
}

`
aggregate 2.4.0 issue when using with split plugin
issue
created
fbaligand
username_1
Actually, since version 2.4.0, if taskid pattern is not the same, aggregate map is not shared. 
If you want sharing, you have to use the same taskid pattern so have to keep same value in same field name
comment
created
mrchang7
username_0
Okay, I see. Then I will try to following the rule. It would be nicer to have that in the document. Currently, it says for the task_id, "This value must uniquely identify the task in the system."

Thanks for your quick help!
comment
created
fbaligand
username_1
You are right. 
I have to explain it in documentation. 

That said, I truly think you should use version 2.3.1 : 
- in this version, aggregate maps are nicely stored to configured path at shutdown 
- and there is not this "new behavior" which annoy you.
comment
created
mrchang7
username_0
As I made the patterns the same even with the field names, it worked as before. Can I ask why you made this change? Wondering.. the value of task_id should matter if the map keeps based on it. 

Thanks!
comment
created
mrchang7
username_0
The problem using 2.3.1 is we have multiple task_id's and like to set different time_out values to them.
comment
created
fbaligand
username_1
To allow this new 2.4 feature (timeout options per task id pattern), I had to tie aggregate maps to their taskid pattern ; so that timeout management for one task id pattern process only associated aggregate maps.
comment
created
mrchang7
username_0
That makes sense. Thanks!
comment
created
fbaligand
username_1
@username_0 
As it was a simple question, and the question is answered, could you close this issue ?
comment
closed
mrchang7
username_0
issue