Project: ClickHouse (App Definition and Development / Database)
File: KQL_ReleaseNote.md

`Customers | distinct *` `Customers | distinct Occupation` `Customers | distinct Occupation, Education` `Customers | where Age <30 | distinct Occupation, Education` `Customers | where Age <30 | order by Age| distinct Occupation, Education` `print reverse(123)` `print reverse(123.34)` `print reverse('clickhouse')` `print reverse(3h)` `print reverse(datetime(2017-1-1 12:23:34))` `print parsecommandline('echo \"hello world!\" print$?', \"Windows\")` `print result=parse_csv('aa,b,cc')` `print resultmultirecord=parse_csv('record1,a,b,c\nrecord2,x,y,z')` `print parse_json( dynamic([1, 2, 3]))` `print parse_json('{"a":123.5, "b":"{\\"c\\":456}"}')` `print extract_json( "$.a" , '{"a":123, "b":"{\\"c\\":456}"}' , typeof(int))` `print parse_version('1')` `print parse_version('1.2.3.40')` array index should start with 0 - if bin is used , the column should be in select list if no alias include if no column included in aggregate functions, ( like count() ), should has alias with fun name + '',e.g count if column name included in aggregate functions, should have fun name + "" + column name , like count(Age) -> countAge if argument of an aggregate functions is an exprision, Columns1 ... Columnsn should be used as alias ``` Customers | summarize count() by bin(Age, 10) Agecount_ 40 2 20 6 30 4 Customers | summarize count(Age) by bin(Age, 10) Agecount_Age 40 2 20 6 30 4 Customers | summarize count(Age+1) by bin(Age+1, 10) Columns1count_ 40 2 20 6 30 4 ``` https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/extendoperator `T | extend T | extend duration = endTime - startTime` `T | project endTime, startTime | extend duration = endTime - startTime` `print array_reverse(dynamic(["this", "is", "an", "example"])) == dynamic(["example","an","is","this"])` `print arrayrotateleft(dynamic([1,2,3,4,5]), 2) == dynamic([3,4,5,1,2])` `print arrayrotateleft(dynamic([1,2,3,4,5]), -2) == dynamic([4,5,1,2,3])` `print arrayrotateright(dynamic([1,2,3,4,5]), -2) == dynamic([3,4,5,1,2])` `print arrayrotateright(dynamic([1,2,3,4,5]), 2) == dynamic([4,5,1,2,3])` `print arrayshiftleft(dynamic([1,2,3,4,5]), 2) == dynamic([3,4,5,null,null])` `print arrayshiftleft(dynamic([1,2,3,4,5]), -2) == dynamic([null,null,1,2,3])` `print arrayshiftleft(dynamic([1,2,3,4,5]), 2, -1) == dynamic([3,4,5,-1,-1])` `print arrayshiftleft(dynamic(['a', 'b', 'c']), 2) == dynamic(['c','',''])` `print arrayshiftright(dynamic([1,2,3,4,5]), -2) == dynamic([3,4,5,null,null])` `print arrayshiftright(dynamic([1,2,3,4,5]), 2) == dynamic([null,null,1,2,3])` `print arrayshiftright(dynamic([1,2,3,4,5]), -2, -1) == dynamic([3,4,5,-1,-1])` `print arrayshiftright(dynamic(['a', 'b', 'c']), -2) == dynamic(['c','',''])` `print x = 1, y = x 2, z = y 2, pack_array(x,y,z)` Please note that only arrays of elements of the same type may be created at this time. The underlying reasons are explained under the release note section of the `dynamic` data type. `print repeat(1, 0) == dynamic([])` `print repeat(1, 3) == dynamic([1, 1, 1])` `print repeat("asd", 3) == dynamic(['asd', 'asd', 'asd'])` `print repeat(timespan(1d), 3) == dynamic([86400, 86400, 86400])` `print repeat(true, 3) == dynamic([true, true, true])` `print zip(dynamic([1,3,5]), dynamic([2,4,6]))` Please note that only arrays of the same type are supported in our current implementation. The underlying reasons are explained under the release note section of the `dynamic` data type. `print isnull(dynamic(null))` `print dynamic(1) == 1` `print dynamic(timespan(1d)) == 86400` `print dynamic([1, 2, 3])` `print dynamic([[1], [2], [3]])` `print dynamic(['a', "b", 'c'])` According to the KQL specifications `dynamic` is a literal, which means that no function calls are permitted. Expressions producing literals such as `datetime` and `timespan` and their aliases (ie. `date` and `time`, respectively) along with nested `dynamic` literals are allowed. Please note that our current implementation supports only scalars and arrays made up of elements of the same type. Support for mixed types and property bags is deferred for now, based on our understanding of the required effort and discussion with representatives of the QRadar team. `print isnan(double(nan)) == true` `print isnan(4.2) == false` `print isnan(4) == false` `print isnan(real(+inf)) == false` Please note that functions returning arrays with set semantics may return them in any particular order, which may be subject to change in the future. `print jaccard_index(dynamic([1, 1, 2, 2, 3, 3]), dynamic([1, 2, 3, 4, 4, 4])) ==

`print jaccard_index(dynamic([1, 2, 3]), dynamic([])) == 0` `print jaccard_index(dynamic([]), dynamic([1, 2, 3, 4])) == 0` `print isnan(jaccard_index(dynamic([]), dynamic([])))` `print jaccard_index(dynamic([1, 2, 3]), dynamic([4, 5, 6, 7])) == 0` `print jaccard_index(dynamic(['a', 's', 'd']), dynamic(['f', 'd', 's', 'a'])) == 0.75` `print jaccard_index(dynamic(['Chewbacca', 'Darth Vader', 'Han Solo']), dynamic(['Darth Sidious', 'Darth Vader'])) == 0.25` `print set_difference(dynamic([1, 1, 2, 2, 3, 3]), dynamic([1, 2, 3])) == dynamic([])` `print arraysortasc(set_difference(dynamic([1, 4, 2, 3, 5, 4, 6]), dynamic([1, 2, 3])))[1] == dynamic([4, 5, 6])` `print set_difference(dynamic([4]), dynamic([1, 2, 3])) == dynamic([4])` `print arraysortasc(set_difference(dynamic([1, 2, 3, 4, 5]), dynamic([5]), dynamic([2, 4])))[1] == dynamic([1, 3])` `print arraysortasc(set_difference(dynamic([1, 2, 3]), dynamic([])))[1] == dynamic([1, 2, 3])` `print arraysortasc(set_difference(dynamic(['a', 's', 'd']), dynamic(['a', 'f'])))[1] == dynamic(['d', 's'])` `print arraysortasc(set_difference(dynamic(['Chewbacca', 'Darth Vader', 'Han Solo']), dynamic(['Darth Sidious', 'Darth Vader'])))[1] == dynamic(['Chewbacca', 'Han Solo'])` `print sethaselement(dynamic(["this", "is", "an", "example"]), "example") == true` `print sethaselement(dynamic(["this", "is", "an", "example"]), "test") == false` `print sethaselement(dynamic([1, 2, 3]), 2) == true` `print sethaselement(dynamic([1, 2, 3, 4.2]), 4) == false` `print arraysortasc(set_intersect(dynamic([1, 1, 2, 2, 3, 3]), dynamic([1, 2, 3])))[1] == dynamic([1, 2, 3])` `print arraysortasc(set_intersect(dynamic([1, 4, 2, 3, 5, 4, 6]), dynamic([1, 2, 3])))[1] == dynamic([1, 2, 3])` `print set_intersect(dynamic([4]), dynamic([1, 2, 3])) == dynamic([])` `print set_intersect(dynamic([1, 2, 3, 4, 5]), dynamic([1, 3, 5]), dynamic([2, 5])) == dynamic([5])` `print set_intersect(dynamic([1, 2, 3]), dynamic([])) == dynamic([])` `print set_intersect(dynamic(['a', 's', 'd']), dynamic(['a', 'f'])) == dynamic(['a'])` `print set_intersect(dynamic(['Chewbacca', 'Darth Vader', 'Han Solo']), dynamic(['Darth Sidious', 'Darth Vader'])) == dynamic(['Darth Vader'])` `print arraysortasc(set_union(dynamic([1, 1, 2, 2, 3, 3]), dynamic([1, 2, 3])))[1] == dynamic([1, 2, 3])` `print arraysortasc(set_union(dynamic([1, 4, 2, 3, 5, 4, 6]), dynamic([1, 2, 3])))[1] == dynamic([1, 2, 3, 4, 5, 6])` `print arraysortasc(set_union(dynamic([4]), dynamic([1, 2, 3])))[1] == dynamic([1, 2, 3, 4])` `print arraysortasc(set_union(dynamic([1, 3, 4]), dynamic([5]), dynamic([2, 4])))[1] == dynamic([1, 2, 3, 4, 5])` `print arraysortasc(set_union(dynamic([1, 2, 3]), dynamic([])))[1] == dynamic([1, 2, 3])` `print arraysortasc(set_union(dynamic(['a', 's', 'd']), dynamic(['a', 'f'])))[1] == dynamic(['a', 'd', 'f', 's'])` `print arraysortasc(set_union(dynamic(['Chewbacca', 'Darth Vader', 'Han Solo']), dynamic(['Darth Sidious', 'Darth Vader'])))[1] == dynamic(['Chewbacca', 'Darth Sidious', 'Darth Vader', 'Han Solo'])` https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/mvexpandoperator Note: expand on array columns only test cases ``` CREATE TABLE T ( a UInt8, b Array(String), c Array(Int8), d Array(Int8) ) ENGINE = Memory; INSERT INTO T VALUES (1, ['Salmon', 'Steak','Chicken'],[1,2,3,4],[5,6,7,8]) T | mv-expand c T | mv-expand c, d T | mv-expand b | mv-expand c T | mv-expand c to typeof(bool) T | mv-expand with_itemindex=index b, c, d T | mv-expand array_concat(c,d) T | mv-expand x = c, y = d T | mv-expand xy = array_concat(c, d) T | mv-expand with_itemindex=index c,d to typeof(bool) ```

test case make-series on datetime column ``` CREATE TABLE T ( Supplier Nullable(String), Fruit String , Price Float64, Purchase Date ) ENGINE = Memory; INSERT INTO T VALUES ('Aldi','Apple',4,'2016-09-10'); INSERT INTO T VALUES ('Costco','Apple',2,'2016-09-11'); INSERT INTO T VALUES ('Aldi','Apple',6,'2016-09-10'); INSERT INTO T VALUES ('Costco','Snargaluff',100,'2016-09-12'); INSERT INTO T VALUES ('Aldi','Apple',7,'2016-09-12'); INSERT INTO T VALUES ('Aldi','Snargaluff',400,'2016-09-11'); INSERT INTO T VALUES ('Costco','Snargaluff',104,'2016-09-12'); INSERT INTO T VALUES ('Aldi','Apple',5,'2016-09-12'); INSERT INTO T VALUES ('Aldi','Snargaluff',600,'2016-09-11'); INSERT INTO T VALUES ('Costco','Snargaluff',200,'2016-09-10'); ``` Have from and to ``` T | make-series PriceAvg = avg(Price) default=0 on Purchase from datetime(2016-09-10) to datetime(2016-09-13) step 1d by Supplier, Fruit ``` Has from , without to ``` T | make-series PriceAvg = avg(Price) default=0 on Purchase from datetime(2016-09-10) step 1d by Supplier, Fruit ``` Without from , has to ``` T | make-series PriceAvg = avg(Price) default=0 on Purchase to datetime(2016-09-13) step 1d by Supplier, Fruit ``` Without from , without to ``` T | make-series PriceAvg = avg(Price) default=0 on Purchase step 1d by Supplier, Fruit ``` Without by clause ``` T | make-series PriceAvg = avg(Price) default=0 on Purchase step 1d ``` Without aggregation alias ``` T | make-series avg(Price) default=0 on Purchase step 1d by Supplier, Fruit ``` Has group expression alias ``` T | make-series avg(Price) default=0 on Purchase step 1d by Supplier_Name = Supplier, Fruit ``` Use different step value ``` T | make-series PriceAvg = avg(Price) default=0 on Purchase from datetime(2016-09-10) to datetime(2016-09-13) step 3d by Supplier, Fruit ``` test case make-series on numeric column ``` CREATE TABLE T2 ( Supplier Nullable(String), Fruit String , Price Int32, Purchase Int32 ) ENGINE = Memory; INSERT INTO T2 VALUES ('Aldi','Apple',4,10); INSERT INTO T2 VALUES ('Costco','Apple',2,11); INSERT INTO T2 VALUES ('Aldi','Apple',6,10); INSERT INTO T2 VALUES ('Costco','Snargaluff',100,12); INSERT INTO T2 VALUES ('Aldi','Apple',7,12); INSERT INTO T2 VALUES ('Aldi','Snargaluff',400,11); INSERT INTO T2 VALUES ('Costco','Snargaluff',104,12); INSERT INTO T2 VALUES ('Aldi','Apple',5,12); INSERT INTO T2 VALUES ('Aldi','Snargaluff',600,11); INSERT INTO T2 VALUES ('Costco','Snargaluff',200,10); ``` Have from and to ``` T2 | make-series PriceAvg=avg(Price) default=0 on Purchase from 10 to 15 step 1.0 by Supplier, Fruit; ``` Has from , without to ``` T2 | make-series PriceAvg=avg(Price) default=0 on Purchase from 10 step 1.0 by Supplier, Fruit; ``` Without from , has to ``` T2 | make-series PriceAvg=avg(Price) default=0 on Purchase to 18 step 4.0 by Supplier, Fruit; ``` Without from , without to ``` T2 | make-series PriceAvg=avg(Price) default=0 on Purchase step 2.0 by Supplier, Fruit; ``` Without by clause ``` T2 | make-series PriceAvg=avg(Price) default=0 on Purchase step 2.0; ``` `print bin(4.5, 1)` `print bin(time(16d), 7d)` `print bin(datetime(1970-05-11 13:45:07), 1d)` `Customers | summarize t = stdev(Age) by FirstName` `Customers | summarize t = stdevif(Age, Age < 10) by FirstName` `Customers | summarize t = binaryalland(Age) by FirstName` `Customers | summarize t = binaryallor(Age) by FirstName` `Customers | summarize t = binaryallxor(Age) by FirstName` `Customers | summarize percentiles(Age, 30, 40, 50, 60, 70) by FirstName` `DataTable | summarize t = percentilesw(Bucket, Frequency, 50, 75, 99.9)` `Customers | summarize t = percentile(Age, 50) by FirstName` `DataTable | summarize t = percentilew(Bucket, Frequency, 50)` Only support the constant dynamic array. Returns an array. So, each element of the input has to be of same

`print t = arraysortasc(dynamic([null, 'd', 'a', 'c', 'c']))` `print t = arraysortasc(dynamic([4, 1, 3, 2]))` `print t = arraysortasc(dynamic(['b', 'a', 'c']), dynamic(['q', 'p', 'r']))` `print t = arraysortasc(dynamic(['q', 'p', 'r']), dynamic(['clickhouse','hello', 'world']))` `print t = arraysortasc( dynamic(['d', null, 'a', 'c', 'c']) , false)` `print t = arraysortasc( dynamic(['d', null, 'a', 'c', 'c']) , 1 > 2)` `print t = arraysortasc( dynamic([null, 'd', null, null, 'a', 'c', 'c', null, null, null]) , false)` `print t = arraysortasc( dynamic([null, null, null]) , false)` `print t = arraysortasc(dynamic([2, 1, null,3]), dynamic([20, 10, 40, 30]), 1 > 2)` `print t = arraysortasc(dynamic([2, 1, null,3]), dynamic([20, 10, 40, 30, 50, 3]), 1 > 2)` (only support the constant dynamic array) `print t = arraysortdesc(dynamic([null, 'd', 'a', 'c', 'c']))` `print t = arraysortdesc(dynamic([4, 1, 3, 2]))` `print t = arraysortdesc(dynamic(['b', 'a', 'c']), dynamic(['q', 'p', 'r']))` `print t = arraysortdesc(dynamic(['q', 'p', 'r']), dynamic(['clickhouse','hello', 'world']))` `print t = arraysortdesc( dynamic(['d', null, 'a', 'c', 'c']) , false)` `print t = arraysortdesc( dynamic(['d', null, 'a', 'c', 'c']) , 1 > 2)` `print t = arraysortdesc( dynamic([null, 'd', null, null, 'a', 'c', 'c', null, null, null]) , false)` `print t = arraysortdesc( dynamic([null, null, null]) , false)` `print t = arraysortdesc(dynamic([2, 1, null, 3]), dynamic([20, 10, 40, 30]), 1 > 2)` `print t = arraysortdesc(dynamic([2, 1, null,3, null]), dynamic([20, 10, 40, 30, 50, 3]), 1 > 2)` `print array_concat(dynamic([1, 2, 3]), dynamic([4, 5]), dynamic([6, 7, 8, 9])) == dynamic([1, 2, 3, 4, 5, 6, 7, 8, 9])` `print array_iif(dynamic([true, false, true]), dynamic([1, 2, 3]), dynamic([4, 5, 6])) == dynamic([1, 5, 3])` `print array_iif(dynamic([true, false, true]), dynamic([1, 2, 3, 4]), dynamic([4, 5, 6])) == dynamic([1, 5, 3])` `print array_iif(dynamic([true, false, true, false]), dynamic([1, 2, 3, 4]), dynamic([4, 5, 6])) == dynamic([1, 5, 3, null])` `print array_iif(dynamic([1, 0, -1, 44, 0]), dynamic([1, 2, 3, 4]), dynamic([4, 5, 6])) == dynamic([1, 5, 3, 4, null])` `print array_slice(dynamic([1,2,3]), 1, 2) == dynamic([2, 3])` `print array_slice(dynamic([1,2,3,4,5]), 2, -1) == dynamic([3, 4, 5])` `print array_slice(dynamic([1,2,3,4,5]), -3, -2) == dynamic([3, 4])` `print array_split(dynamic([1,2,3,4,5]), 2) == dynamic([[1,2],[3,4,5]])` `print array_split(dynamic([1,2,3,4,5]), dynamic([1,3])) == dynamic([[1],[2,3],[4,5]])` `print ago(2h)` `print endofday(datetime(2017-01-01 10:10:17), -1)` `print endofday(datetime(2017-01-01 10:10:17), 1)` `print endofday(datetime(2017-01-01 10:10:17))` `print endofmonth(datetime(2017-01-01 10:10:17), -1)` `print endofmonth(datetime(2017-01-01 10:10:17), 1)` `print endofmonth(datetime(2017-01-01 10:10:17))` `print endofweek(datetime(2017-01-01 10:10:17), 1)` `print endofweek(datetime(2017-01-01 10:10:17), -1)` `print endofweek(datetime(2017-01-01 10:10:17))` `print endofyear(datetime(2017-01-01 10:10:17), -1)` `print endofyear(datetime(2017-01-01 10:10:17), 1)` `print endofyear(datetime(2017-01-01 10:10:17))` `print make_datetime(2017,10,01)` `print make_datetime(2017,10,01,12,10)` `print make_datetime(2017,10,01,12,11,0.1234567)` `print datetime_diff('year',datetime(2017-01-01),datetime(2000-12-31))` `print datetime_diff('quarter',datetime(2017-07-01),datetime(2017-03-30))` `print datetime_diff('minute',datetime(2017-10-30 23:05:01),datetime(2017-10-30 23:00:59))` `print unixtimemicrosecondstodatetime(1546300800000000)` `print unixtimemillisecondstodatetime(1546300800000)` `print unixtimenanosecondstodatetime(1546300800000000000)` `print datetime_part('day', datetime(2017-10-30 01:02:03.7654321))` `print datetime_add('day',1,datetime(2017-10-30 01:02:03.7654321))` `print format_timespan(time(1d), 'd-[hh:mm:ss]')` `print format_timespan(time('12:30:55.123'), 'ddddd-[hh:mm:ss.ffff]')` `print format_datetime(todatetime('2009-06-15T13:45:30.6175425'), 'yy-M-dd [H:mm:ss.fff]')` `print format_datetime(datetime(2015-12-14 02:03:04.12345), 'y-M-d h:m:s tt')` `print todatetime('2014-05-25T08:20:03.123456Z')` `print todatetime('2014-05-25 20:03.123')` [totimespan] (https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/totimespanfunction) ` print totimespan('0.01:34:23')` `print totimespan(1d)` double quote support ``print res = strcat("double ","quote")`` `print res = bin_at(6.5, 2.5, 7)` `print res = bin_at(1h, 1d, 12h)` `print res = bin_at(datetime(2017-05-15 10:20:00.0), 1d, datetime(1970-01-01 12:00:00.0))` `print res = bin_at(datetime(2017-05-17 10:20:00.0), 7d, datetime(2017-06-04 00:00:00.0))` Supports only basic lookup. Do not support start_index, length and occurrence `print output = arrayindexof(dynamic(['John', 'Denver', 'Bob', 'Marley']), 'Marley')` `print output = arrayindexof(dynamic([1, 2, 3]), 2)` `print output = array_sum(dynamic([2, 5, 3]))` `print output = array_sum(dynamic([2.5, 5.5, 3]))` `print output = array_length(dynamic(['John', 'Denver', 'Bob', 'Marley']))` `print output = array_length(dynamic([1, 2, 3]))` `print tobool(true) == true` `print toboolean(false) == false` `print tobool(0) == false` `print toboolean(19819823) == true` `print tobool(-2) == true` `print isnull(toboolean('a'))` `print tobool('true') == true` `print toboolean('false') == false` `print todouble(4) == 4` `print toreal(4.2) == 4.2` `print isnull(todouble('a'))` `print toreal('-0.3') == -0.3` `print isnull(toint('a'))` `print toint(4) == 4` `print toint('4') == 4` `print isnull(toint(4.2))` `print tostring(123) == '123'` `print tostring('asd') == 'asd'` Supports only 1D array `print output = dynamic(['a', 'b', 'c'])` `print output = dynamic([1, 2, 3])` `print bool(1)` `print boolean(0)` `print datetime(2015-12-31 23:59:59.9)` `print datetime('2015-12-31 23:59:59.9')` `print datetime("2015-12-31:)` `print guid(74be27de-1e4e-49d9-b579-fe0b331d3642)` `print guid('74be27de-1e4e-49d9-b579-fe0b331d3642')` `print guid('74be27de1e4e49d9b579fe0b331d3642')` `print int(1)` `print long(16)` `print real(1)` Note the timespan is used for calculating datatime, so the output is in seconds. e.g. time(1h) = 3600 `print 1d` `print 30m` `print time('0.12:34:56.7')` `print time(2h)` `print timespan(2h)` `print Quine = base64encodefromguid('ae3133f2-6e22-49ae-b06a-16e6a9b212eb')` `print base64decodetoarray('S3VzdG8=')` `print base64decodetoguid('YWUzMTMzZjItNmUyMi00OWFlLWIwNmEtMTZlNmE5YjIxMmVi')` `print replace_regex('Hello, World!', '.', '\\0\\0')` `print idx = hasanyindex('this is an example', dynamic(['this', 'example']))` `print translate('krasp', 'otsku', 'spark')` `print trim('--', '--https://bing.com--')` `print trim_end('.com',

`print trim_start('[^\\w]+', strcat('- ','Te st1','// $'))` `print startofyear(datetime(2017-01-01 10:10:17), -1)` `print startofyear(datetime(2017-01-01 10:10:17), 0)` `print startofyear(datetime(2017-01-01 10:10:17), 1)` `print weekofyear(datetime(2020-12-31))` `print weekofyear(datetime(2020-06-15))` `print weekofyear(datetime(1970-01-01))` `print weekofyear(datetime(2000-01-01))` `print startofweek(datetime(2017-01-01 10:10:17), -1)` `print startofweek(datetime(2017-01-01 10:10:17), 0)` `print startofweek(datetime(2017-01-01 10:10:17), 1)` `print startofmonth(datetime(2017-01-01 10:10:17), -1)` `print startofmonth(datetime(2017-01-01 10:10:17), 0)` `print startofmonth(datetime(2017-01-01 10:10:17), 1)` `print startofday(datetime(2017-01-01 10:10:17), -1)` `print startofday(datetime(2017-01-01 10:10:17), 0)` `print startofday(datetime(2017-01-01 10:10:17), 1)` `print monthofyear(datetime("2015-12-14"))` `print hourofday(datetime(2015-12-14 18:54:00))` `print getyear(datetime(2015-10-12))` `print getmonth(datetime(2015-10-12))` `print dayofyear(datetime(2015-12-14))` `print (datetime(2015-12-14))` `print unixtimesecondstodatetime(1546300800)` `print dayofweek(datetime(2015-12-20))` `print now()` `print now(2d)` `print now(-2h)` `print now(5microseconds)` `print now(5seconds)` `print now(6minutes)` `print now(-2d) ` `print now(time(1d))` `print binary_and(15, 3) == 3` `print binary_and(1, 2) == 0` `print binary_not(1) == -2` `print binary_or(3, 8) == 11` `print binary_or(1, 2) == 3` `print binaryshiftleft(1, 1) == 2` `print binaryshiftleft(1, 64) == 1` `print binaryshiftright(1, 1) == 0` `print binaryshiftright(1, 64) == 1` `print binary_xor(1, 3) == 2` `print bitsetcountones(42) == 3` `print format_ipv4('192.168.1.255', 24) == '192.168.1.0'` `print format_ipv4(3232236031, 24) == '192.168.1.0'` `print formatipv4mask('192.168.1.255', 24) == '192.168.1.0/24'` `print formatipv4mask(3232236031, 24) == '192.168.1.0/24'` `print ipv4_compare('127.0.0.1', '127.0.0.1') == 0` `print ipv4_compare('192.168.1.1', '192.168.1.255') < 0` `print ipv4_compare('192.168.1.1/24', '192.168.1.255/24') == 0` `print ipv4_compare('192.168.1.1', '192.168.1.255', 24) == 0` `print ipv4ismatch('127.0.0.1', '127.0.0.1') == true` `print ipv4ismatch('192.168.1.1', '192.168.1.255') == false` `print ipv4ismatch('192.168.1.1/24', '192.168.1.255/24') == true` `print ipv4ismatch('192.168.1.1', '192.168.1.255', 24) == true` `print ipv6_compare('::ffff:7f00:1', '127.0.0.1') == 0` `print ipv6_compare('fe80::85d:e82c:9446:7994', 'fe80::85d:e82c:9446:7995') < 0` `print ipv6_compare('192.168.1.1/24', '192.168.1.255/24') == 0` `print ipv6_compare('fe80::85d:e82c:9446:7994/127', 'fe80::85d:e82c:9446:7995/127') == 0` `print ipv6_compare('fe80::85d:e82c:9446:7994', 'fe80::85d:e82c:9446:7995', 127) == 0` `print ipv6ismatch('::ffff:7f00:1', '127.0.0.1') == true` `print ipv6ismatch('fe80::85d:e82c:9446:7994', 'fe80::85d:e82c:9446:7995') == false` `print ipv6ismatch('192.168.1.1/24', '192.168.1.255/24') == true` `print ipv6ismatch('fe80::85d:e82c:9446:7994/127', 'fe80::85d:e82c:9446:7995/127') == true` `print ipv6ismatch('fe80::85d:e82c:9446:7994', 'fe80::85d:e82c:9446:7995', 127) == true` `print parseipv4mask('127.0.0.1', 24) == 2130706432` `print parseipv4mask('192.1.168.2', 31) == 3221334018` `print parseipv4mask('192.1.168.3', 31) == 3221334018` `print parseipv4mask('127.2.3.4', 32) == 2130838276` `print parseipv6mask('127.0.0.1', 24) == '0000:0000:0000:0000:0000:ffff:7f00:0000'` `print parseipv6mask('fe80::85d:e82c:9446:7994', 120) == 'fe80:0000:0000:0000:085d:e82c:9446:7900'` The config setting to allow modify dialect setting. Set dialect setting in server configuration XML at user level(` users.xml `). This sets the ` dialect ` at server startup and CH will do query parsing for all users with ` default ` profile according to dialect value. For example: ` <profiles> <!-- Default settings. --> <default> <loadbalancing>random</loadbalancing> <dialect>kusto</dialect> </default> ` Query can be executed with HTTP client as below once dialect is set in users.xml ` echo "KQL query" | curl -sS "http://localhost:8123/?" --data-binary @- ` To execute the query using clickhouse-client , Update clickhouse-client.xml as below and connect clickhouse-client with --config-file option (` clickhouse-client --config-file=<config-file path> `) ` <config> <dialect>kusto</dialect> </config> ` OR pass dialect setting with '--'. For example : ` clickhouse-client --dialect='kusto' -q "KQL query" ` strcmp (https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/strcmpfunction) `print strcmp('abc','ABC')` parse_url (https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/parseurlfunction) `print Result = parse_url('scheme://username:password@www.google.com:1234/this/is/a/path?k1=v1&k2=v2#fragment')` parse_urlquery (https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/parseurlqueryfunction) `print Result = parse_urlquery('k1=v1&k2=v2&k3=v3')` print operator (https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/printoperator) `print x=1, s=strcat('Hello', ', ', 'World!')` Aggregate Functions: `Customers | summarize t = make_list(FirstName) by FirstName` `Customers | summarize t = make_list(FirstName, 10) by FirstName` `Customers | summarize t = makelistif(FirstName, Age > 10) by FirstName` `Customers | summarize t = makelistif(FirstName, Age > 10, 10) by FirstName` `Customers | summarize t = makelistwith_nulls(Age) by FirstName` `Customers | summarize t = make_set(FirstName) by FirstName` `Customers | summarize t = make_set(FirstName, 10) by FirstName` `Customers | summarize t = makesetif(FirstName, Age > 10) by FirstName` `Customers | summarize t = makesetif(FirstName, Age > 10, 10) by FirstName` The following functions now support arbitrary expressions as their argument: - `set dialect='clickhouse'` `set dialect='kusto'` `"Customers | project parse_ipv4('127.0.0.1')"` `"Customers | project parse_ipv6('127.0.0.1')"` Please note that the functions listed below only take constant parameters for now. Further improvement is to be expected to support expressions. `"Customers | project ipv4isprivate('192.168.1.6/24')"` `"Customers | project ipv4isprivate('192.168.1.6')"` `"Customers | project ipv4isin_range('127.0.0.1', '127.0.0.1')"` `"Customers | project ipv4isin_range('192.168.1.6', '192.168.1.1/24')"` `"Customers | project ipv4netmasksuffix('192.168.1.1/24')"` `"Customers | project ipv4netmasksuffix('192.168.1.1')"` support subquery for `in` orerator (https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/in-cs-operator) (subquery need to be wrapped with bracket inside bracket) `Customers | where Age in ((Customers|project Age|where Age < 30))` Note: case-insensitive not supported yet has_all (https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/has-all-operator) `Customers|where Occupation has_any ('Skilled','abcd')` note : subquery not supported yet has _any (https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/has-anyoperator) `Customers|where Occupation has_all ('Skilled','abcd')` note : subquery not supported yet countof (https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/countoffunction) `Customers | project countof('The cat sat on the mat', 'at')` `Customers | project countof('The cat sat on the mat', 'at', 'normal')` `Customers | project countof('The cat sat on the mat', 'at', 'regex')` extract ( https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/extractfunction) `Customers | project extract('(\\b[A-Z]+\\b).+(\\b\\d+)', 0, 'The price of PINEAPPLE ice cream is 20')` `Customers | project

1, 'The price of PINEAPPLE ice cream is 20')` `Customers | project extract('(\\b[A-Z]+\\b).+(\\b\\d+)', 2, 'The price of PINEAPPLE ice cream is 20')` `Customers | project extract('(\\b[A-Z]+\\b).+(\\b\\d+)', 3, 'The price of PINEAPPLE ice cream is 20')` `Customers | project extract('(\\b[A-Z]+\\b).+(\\b\\d+)', 2, 'The price of PINEAPPLE ice cream is 20', typeof(real))` extract_all (https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/extractallfunction) `Customers | project extract_all('(\\w)(\\w+)(\\w)','The price of PINEAPPLE ice cream is 20')` note: captureGroups not supported yet split (https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/splitfunction) `Customers | project split('aabb', '')` `Customers | project split('aaabbbccc', '_', 1)` `Customers | project split('', '_')` `Customers | project split('ab', '_')` `Customers | project split('aabbcc', 'bb')` strcat_delim (https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/strcat-delimfunction) `Customers | project strcat_delim('-', '1', '2', 'A') , 1s)` `Customers | project strcat_delim('-', '1', '2', strcat('A','b'))` note: only support string now. indexof (https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/indexoffunction) `Customers | project indexof('abcdefg','cde')` `Customers | project indexof('abcdefg','cde',2)` `Customers | project indexof('abcdefg','cde',6)` note: length and occurrence not supported yet default is `clickhouse` `set sql_dialect='clickhouse'` only process kql `set sql_dialect='kusto'` create table `CREATE TABLE kqltable4 ENGINE = Memory AS select *, now() as newcolumn From kql(Customers | project LastName,Age);` verify the content of `kql_table` `select * from kql_table` insert into table create a tmp table: ``` CREATE TABLE temp ( FirstName Nullable(String), LastName String, Age Nullable(UInt8) ) ENGINE = Memory; ``` `INSERT INTO temp select * from kql(Customers|project FirstName,LastName,Age);` verify the content of `temp` `select * from temp` Select from kql() `Select * from kql(Customers|project FirstName)` Tabular expression statements `Customers` Select Column `Customers | project FirstName,LastName,Occupation` Limit returned results `Customers | project FirstName,LastName,Occupation | take 1 | take 3` sort, order `Customers | order by Age desc , FirstName asc` Filter `Customers | where Occupation == 'Skilled Manual'` summarize `Customers |summarize max(Age) by Occupation` contains `Customers |where Education contains 'degree'` !contains `Customers |where Education !contains 'degree'` contains_cs `Customers |where Education contains 'Degree'` !contains_cs `Customers |where Education !contains 'Degree'` endswith `Customers | where FirstName endswith 'RE'` !endswith `Customers | where !FirstName endswith 'RE'` endswith_cs `Customers | where FirstName endswith_cs 're'` !endswith_cs `Customers | where FirstName !endswith_cs 're'` == `Customers | where Occupation == 'Skilled Manual'` != `Customers | where Occupation != 'Skilled Manual'` has `Customers | where Occupation has 'skilled'` !has `Customers | where Occupation !has 'skilled'` has_cs `Customers | where Occupation has 'Skilled'` !has_cs `Customers | where Occupation !has 'Skilled'` hasprefix `Customers | where Occupation hasprefix_cs 'Ab'` !hasprefix `Customers | where Occupation !hasprefix_cs 'Ab'` hasprefix_cs `Customers | where Occupation hasprefix_cs 'ab'` !hasprefix_cs `Customers | where Occupation! hasprefix_cs 'ab'` hassuffix `Customers | where Occupation hassuffix 'Ent'` !hassuffix `Customers | where Occupation !hassuffix 'Ent'` hassuffix_cs `Customers | where Occupation hassuffix 'ent'` !hassuffix_cs `Customers | where Occupation hassuffix 'ent'` in `Customers |where Education in ('Bachelors','High School')` !in `Customers | where Education !in ('Bachelors','High School')` matches regex `Customers | where FirstName matches regex 'P.*r'` startswith `Customers | where FirstName startswith 'pet'` !startswith `Customers | where FirstName !startswith 'pet'` startswith_cs `Customers | where FirstName startswith_cs 'pet'` !startswith_cs `Customers | where FirstName !startswith_cs 'pet'` base64encodetostring() `Customers | project base64encodetostring('Kusto1') | take 1` base64decodetostring() `Customers | project base64decodetostring('S3VzdG8x') | take 1` isempty() `Customers | where isempty(LastName)` isnotempty() `Customers | where isnotempty(LastName)` isnotnull() `Customers | where isnotnull(FirstName)` isnull() `Customers | where isnull(FirstName)` url_decode() `Customers | project url_decode('https%3A%2F%2Fwww.test.com%2Fhello%20word') | take 1` url_encode() `Customers | project url_encode('https://www.test.com/hello word') | take 1` substring() `Customers | project name_abbr = strcat(substring(FirstName,0,3), ' ', substring(LastName,2))` strcat() `Customers | project name = strcat(FirstName, ' ', LastName)` strlen() `Customers | project FirstName, strlen(FirstName)` strrep() `Customers | project strrep(FirstName,2,'_')` toupper() `Customers | project toupper(FirstName)` tolower() `Customers | project tolower(FirstName)` arg_max() arg_min() avg() avgif() count() countif() max() maxif() min() minif() sum() sumif() dcount() dcountif() bin