Issue 1:
Hi, I am getting the error below. The same query runs without any problems when just a few records are returned by the where clause. I am currently running on a single node "quickstart" configuration.

Thank You

`2017-09-19 11:03:54 DEBUG DruidDao:414 - SQL: select A, B, C, D, E, G, H, I, L from dataSource where A in ('aValue' ) AND B in ('SomeVALUE' )

[WARNING] /dl

AvaticaClientRuntimeException: Remote driver error: IllegalStateException: DefaultQueryMetrics must not be modified from multiple threads. If it is needed to gather dimension or metric information from multiple threads or from an async thread, this information should explicitly be passed between threads (e. g. using Futures), or this DefaultQueryMetrics's ownerThread should be reassigned explicitly. Error -1 (00000) UNKNOWN

java.lang.IllegalStateException: DefaultQueryMetrics must not be modified from multiple threads. If it is needed to gather dimension or metric information from multiple threads or from an async thread, this information should explicitly be passed between threads (e. g. using Futures), or this DefaultQueryMetrics's ownerThread should be reassigned explicitly

at io.druid.query.DefaultQueryMetrics.checkModifiedFromOwnerThread(DefaultQueryMetrics.java:59)

at io.druid.query.DefaultQueryMetrics.reportMetric(DefaultQueryMetrics.java:253)

at io.druid.query.DefaultQueryMetrics.reportCpuTime(DefaultQueryMetrics.java:231)

at io.druid.query.CPUTimeMetricQueryRunner$1.after(CPUTimeMetricQueryRunner.java:86)

at io.druid.java.util.common.guava.WrappingYielder.close(WrappingYielder.java:96)

at io.druid.java.util.common.guava.Yielders$2.close(Yielders.java:73)

at io.druid.java.util.common.guava.ConcatSequence.wrapYielder(ConcatSequence.java:123)

at io.druid.java.util.common.guava.ConcatSequence.access$000(ConcatSequence.java:29)

at io.druid.java.util.common.guava.ConcatSequence$3.next(ConcatSequence.java:143)

at io.druid.sql.avatica.DruidStatement.nextFrame(DruidStatement.java:277)

at io.druid.sql.avatica.DruidMeta.fetch(DruidMeta.java:209)

at org.apache.calcite.avatica.remote.LocalService.apply(LocalService.java:252)

at org.apache.calcite.avatica.remote.Service$FetchRequest.accept(Service.java:1350)

at org.apache.calcite.avatica.remote.Service$FetchRequest.accept(Service.java:1317)

at org.apache.calcite.avatica.remote.AbstractHandler.apply(AbstractHandler.java:95)

at org.apache.calcite.avatica.remote.JsonHandler.apply(JsonHandler.java:52)

at org.apache.calcite.avatica.server.AvaticaJsonHandler.handle(AvaticaJsonHandler.java:129)

at io.druid.sql.avatica.DruidAvaticaHandler.handle(DruidAvaticaHandler.java:63)

at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)

at org.eclipse.jetty.server.Server.handle(Server.java:534)

at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)

at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)

at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)

at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)

at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)

at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)

at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)

at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)

at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)

at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)

at java.lang.Thread.run(Thread.java:748)

Suppressed: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: DefaultQueryMetrics must not be modified from multiple threads. If it is needed to gather dimension or metric information from multiple threads or from an async thread, this information should explicitly be passed between threads (e. g. using Futures), or this DefaultQueryMetrics's ownerThread should be reassigned explicitly

at com.google.common.base.Throwables.propagate(Throwables.java:160)

at io.druid.sql.avatica.DruidStatement.close(DruidStatement.java:335)

at io.druid.sql.avatica.DruidStatement.nextFrame(DruidStatement.java:290)

... 22 more

Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: DefaultQueryMetrics must not be modified from multiple threads. If it is needed to gather dimension or metric information from multiple threads or from an async thread, this information should explicitly be passed between threads (e. g. using Futures), or this DefaultQueryMetrics's ownerThread should be reassigned explicitly

at java.util.concurrent.FutureTask.report(FutureTask.java:122)

at java.util.concurrent.FutureTask.get(FutureTask.java:192)

at io.druid.sql.avatica.DruidStatement.close(DruidStatement.java:319)

... 23 more

Caused by: java.lang.IllegalStateException: DefaultQueryMetrics must not be modified from multiple threads. If it is needed to gather dimension or metric information from multiple threads or from an async thread, this information should explicitly be passed between threads (e. g. using Futures), or this DefaultQueryMetrics's ownerThread should be reassigned explicitly

at io.druid.query.DefaultQueryMetrics.checkModifiedFromOwnerThread(DefaultQueryMetrics.java:59)

at io.druid.query.DefaultQueryMetrics.reportMetric(DefaultQueryMetrics.java:253)

at io.druid.query.DefaultQueryMetrics.reportCpuTime(DefaultQueryMetrics.java:231)

at io.druid.query.CPUTimeMetricQueryRunner$1.after(CPUTimeMetricQueryRunner.java:86)

at io.druid.java.util.common.guava.WrappingYielder.close(WrappingYielder.java:96)

at io.druid.java.util.common.guava.ConcatSequence$3.close(ConcatSequence.java:157)

at io.druid.java.util.common.guava.ConcatSequence$3.close(ConcatSequence.java:156)

at io.druid.sql.avatica.DruidStatement.lambda$close$1(DruidStatement.java:315)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

... 1 more

at org.apache.calcite.avatica.remote.Service$ErrorResponse.toException(Service.java:2476)

at org.apache.calcite.avatica.remote.JsonService.decode(JsonService.java:55)

at org.apache.calcite.avatica.remote.JsonService.apply(JsonService.java:140)

at org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:309)

at org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:306)

at org.apache.calcite.avatica.AvaticaConnection.invokeWithRetries(AvaticaConnection.java:756)

at org.apache.calcite.avatica.remote.RemoteMeta.fetch(RemoteMeta.java:305)

at org.apache.calcite.avatica.MetaImpl$FetchIterator.moveNext(MetaImpl.java:1606)

at org.apache.calcite.avatica.MetaImpl$FetchIterator.next(MetaImpl.java:1591)

at org.apache.calcite.avatica.util.IteratorCursor.next(IteratorCursor.java:46)

at org.apache.calcite.avatica.AvaticaResultSet.next(AvaticaResultSet.java:239)

at ... (DruidDao.java:1181)

at ... (Download.java:75)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)

at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:833)

at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)

at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:206)

at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)

at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)

at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)

at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)

at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)

at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)

at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)

at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)

at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)

at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)

at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)

at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)

at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)

at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)

at org.eclipse.jetty.server.Server.handle(Server.java:564)

at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:317)

at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)

at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)

at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)

at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)

at org.eclipse.jetty.util.thread.Invocable.invokePreferred(Invocable.java:128)

at org.eclipse.jetty.util.thread.Invocable$InvocableExecutor.invoke(Invocable.java:222)

at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:294)

at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:126)

at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:673)

at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:591)

at java.lang.Thread.run(Thread.java:748)`

Issue 2:
Today we have 2 ways to create a druid dimension based on a lookup.

The first one (old way) is as an extraction function ExtractionFn and the second is via lookup dimension spec LookupDimensionSpec.

Chronologically speaking the LookupDimensionSpec was introduced after ExtractionDimensionSpec.

The reason behind this is the fact that lookups can offer more functionalities like unApply, unApplyAll, bulk lookups or the fact that one dimension can map to multiple values use case.

IMHO This means it is better for lookup to be independent from extraction function from a design perspective. Now that we ended up with 2 way of doing things, i guess it is time to unify it.

Benefits.

-Will be able to support the case where we have a one to many mapping as lookups

Ensure that the logic of the mapping is consistent for both filtering and aggregation. This is really important if we want to support the use case mapping one to many

-Avoid to have duplicate code where issue like this occurs #3065

-Support the bulk lookup issues #3030

To achieve that is to make filters accept dimension spec instead of just dimension as a string and extraction function.

Then make the dimensionSpec use the decorate method since getExtractionFn is deprecated anyway.

Also we have to make all the filters work with either decoration of selector or extraction function.