Repository: rcoh/angle-grinder (language: rust)

Issue:
Timeslice operator
See https://github.com/rcoh/angle-grinder/commit/274d34233bec9fd78ae680651efa58ad13b2a2e0 for a template
A non trivial undertaking. My suggested path:
- [x] Add a `date` subtype to `data::Value` -- this will include the date rendering logic
- [x] Add a parseDate expression that produces a `date` value from a string
- [x] Create the timeslice operator that accepts an `Expression<Date>` as an argument and does the bucketing (nontrivial)
- [x] Add an automatic `sort by _timeslice` automatically after groupingby `_timeslice`

Proposed query structure:
```* | parse "* * *" as level, date, message | timeslice(parseDate(level)) 5m | count by _timeslice```



Discussion / hints:
this would be amazing, can be partialy done by pre-converting using `dateround` (part of `dateutils`), but if it can be added natively here that would be incredibly useful.
FYI, I’ve started working on this 
> Add an automatic sort by _timeslice automatically after groupingby _timeslice

I think this is the last item remaining here.  To be clear, if the pipeline is `timeslice(ts) 1m | count by _timeslice`, you want the implicit sort to be by `_timeslice` instead of `_count` as it is now?

Yeah exactly. It's strictly gravy, but it's nice if your time slices are in
order

On Wed, Aug 18, 2021, 1:59 AM Tim Stack ***@***.***> wrote:

> Add an automatic sort by _timeslice automatically after groupingby
> _timeslice
>
> I think this is the last item remaining here. To be clear, if the pipeline
> is timeslice(ts) 1m | count by _timeslice, you want the implicit sort to
> be by _timeslice instead of _count as it is now?
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/rcoh/angle-grinder/issues/10#issuecomment-900833517>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AADYKZ2V4NPLV7W2O6MDYI3T5ND3VANCNFSM4EWET5WQ>
> .
>


Fix patch:
diff --git a/src/lib.rs b/src/lib.rs
index 5556d93..e926ab5 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -105,13 +105,26 @@ pub mod pipeline {
         }
 
         fn implicit_sort(multi_agg: &MultiAggregateOperator) -> SortOperator {
+            let timeslice_col = Expr::column("_timeslice");
+            let (opt_timeslice, direction) = if multi_agg.key_cols.contains(&timeslice_col) {
+                (Some(timeslice_col), SortMode::Ascending)
+            } else {
+                (None, SortMode::Descending)
+            };
+
+            let sort_cols: Vec<Expr> = opt_timeslice
+                .into_iter()
+                .chain(
+                    multi_agg
+                        .aggregate_functions
+                        .iter()
+                        .map(|&(ref k, _)| Expr::column(k)),
+                )
+                .collect();
+
             SortOperator {
-                sort_cols: multi_agg
-                    .aggregate_functions
-                    .iter()
-                    .map(|&(ref k, _)| Expr::column(k))
-                    .collect(),
-                direction: SortMode::Descending,
+                sort_cols,
+                direction,
             }
         }
 


Test patch:
diff --git a/tests/structured_tests/timeslice_1.toml b/tests/structured_tests/timeslice_1.toml
index 31818fd..d11a83c 100644
--- a/tests/structured_tests/timeslice_1.toml
+++ b/tests/structured_tests/timeslice_1.toml
@@ -17,8 +17,8 @@ input = """
 output = """
 _timeslice                     _count
 ---------------------------------------------
-2013-09-06 22:00:00 UTC        7
 2013-09-06 20:00:00 UTC        2
+2013-09-06 22:00:00 UTC        7
 2013-09-06 22:05:00 UTC        2
 2013-09-06 22:10:00 UTC        1
 2013-09-06 23:05:00 UTC        1