Bug: Rewrite multiple GetJsonObjects to a JsonTuple if their json expression is the same
Status: In Progress | Priority: Major | Resolution: 
Created: 14/Mar/23 13:58
Affects Version/s: 3.5.0

Benchmark result:
{noformat}

Running benchmark: Benchmark rewrite GetJsonObjects
  Running case: Default: 2
  Stopped after 2 iterations, 77193 ms
  Running case: Rewrite: 2
  Stopped after 2 iterations, 51699 ms

Java HotSpot(TM) 64-Bit Server VM 17.0.4.1+1-LTS-2 on Mac OS X 13.2.1
Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Benchmark rewrite GetJsonObjects:         Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Default: 2                                        37914          38597         966          0.2        5244.0       1.0X
Rewrite: 2                                        24887          25850        1361          0.3        3442.2       1.5X

Running benchmark: Benchmark rewrite GetJsonObjects
  Running case: Default: 3
  Stopped after 2 iterations, 110890 ms
  Running case: Rewrite: 3
  Stopped after 2 iterations, 56102 ms

Java HotSpot(TM) 64-Bit Server VM 17.0.4.1+1-LTS-2 on Mac OS X 13.2.1
Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Benchmark rewrite GetJsonObjects:         Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Default: 3                                        52862          55445         NaN          0.1        7311.6       1.0X
Rewrite: 3                                        26752          28051        1837          0.3        3700.2       2.0X

Running benchmark: Benchmark rewrite GetJsonObjects
  Running case: Default: 4
  Stopped after 2 iterations, 150828 ms
  Running case: Rewrite: 4
  Stopped after 2 iterations, 57110 ms

Java HotSpot(TM) 64-Bit Server VM 17.0.4.1+1-LTS-2 on Mac OS X 13.2.1
Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Benchmark rewrite GetJsonObjects:         Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Default: 4                                        71680          75414         NaN          0.1        9914.4       1.0X
Rewrite: 4                                        28452          28555         145          0.3        3935.4       2.5X

Running benchmark: Benchmark rewrite GetJsonObjects
  Running case: Default: 5
  Stopped after 2 iterations, 223367 ms
  Running case: Rewrite: 5
  Stopped after 2 iterations, 78193 ms

Java HotSpot(TM) 64-Bit Server VM 17.0.4.1+1-LTS-2 on Mac OS X 13.2.1
Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Benchmark rewrite GetJsonObjects:         Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Default: 5                                       108479         111684        1447          0.1       15004.2       1.0X
Rewrite: 5                                        36830          39097         NaN          0.2        5094.0       2.9X

Running benchmark: Benchmark rewrite GetJsonObjects
  Running case: Default: 10
  Stopped after 2 iterations, 311453 ms
  Running case: Rewrite: 10
  Stopped after 2 iterations, 65873 ms

Java HotSpot(TM) 64-Bit Server VM 17.0.4.1+1-LTS-2 on Mac OS X 13.2.1
Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Benchmark rewrite GetJsonObjects:         Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Default: 10                                      153952         155727        2510          0.0       21293.7       1.0X
Rewrite: 10                                       32436          32937         708          0.2        4486.3       4.7X

Running benchmark: Benchmark rewrite GetJsonObjects
  Running case: Default: 15
  Stopped after 2 iterations, 451911 ms
  Running case: Rewrite: 15
  Stopped after 2 iterations, 69790 ms

Java HotSpot(TM) 64-Bit Server VM 17.0.4.1+1-LTS-2 on Mac OS X 13.2.1
Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Benchmark rewrite GetJsonObjects:         Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Default: 15                                      224950         225956        1423          0.0       31113.6       1.0X
Rewrite: 15                                       34806          34895         126          0.2        4814.2       6.5X

Running benchmark: Benchmark rewrite GetJsonObjects
  Running case: Default: 20
  Stopped after 2 iterations, 587378 ms
  Running case: Rewrite: 20
  Stopped after 2 iterations, 76667 ms

Java HotSpot(TM) 64-Bit Server VM 17.0.4.1+1-LTS-2 on Mac OS X 13.2.1
Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Benchmark rewrite GetJsonObjects:         Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Default: 20                                      293155         293689         756          0.0       40547.3       1.0X
Rewrite: 20                                       38148          38334         262          0.2        5276.4       7.7X

Running benchmark: Benchmark rewrite GetJsonObjects
  Running case: Default: 25
  Stopped after 2 iterations, 732228 ms
  Running case: Rewrite: 25
00:38:55.284 WARN org.apache.spark.sql.catalyst.util.package: Truncated the string representation of a plan since it was too large. This behavior can be adjusted by setting 'spark.sql.debug.maxToStringFields'.
  Stopped after 2 iterations, 93089 ms

Java HotSpot(TM) 64-Bit Server VM 17.0.4.1+1-LTS-2 on Mac OS X 13.2.1
Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Benchmark rewrite GetJsonObjects:         Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Default: 25                                      366037         366114         110          0.0       50627.9       1.0X
Rewrite: 25                                       45725          46545        1159          0.2        6324.4       8.0X

Running benchmark: Benchmark rewrite GetJsonObjects
  Running case: Default: 30
  Stopped after 2 iterations, 881176 ms
  Running case: Rewrite: 30
  Stopped after 2 iterations, 107241 ms

Java HotSpot(TM) 64-Bit Server VM 17.0.4.1+1-LTS-2 on Mac OS X 13.2.1
Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Benchmark rewrite GetJsonObjects:         Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Default: 30                                      439567         440588        1444          0.0       60798.2       1.0X
Rewrite: 30                                       52949          53621         950          0.1        7323.5       8.3X

Running benchmark: Benchmark rewrite GetJsonObjects
  Running case: Default: 36
  Stopped after 2 iterations, 1055559 ms
  Running case: Rewrite: 36
  Stopped after 2 iterations, 124081 ms

Java HotSpot(TM) 64-Bit Server VM 17.0.4.1+1-LTS-2 on Mac OS X 13.2.1
Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Benchmark rewrite GetJsonObjects:         Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Default: 35                                      526481         527780        1837          0.0       72819.5       1.0X
Rewrite: 35                                       60586          62041        2058          0.1        8379.8       8.7X

{noformat}