auphong2707/dl4se-code-review-dataset huggingface dataset:
Source:
private ArrayList<String> getFieldNames() { ArrayList<String> names = new ArrayList<>(); HDF5CompoundMemberInformation[] infos = reader.compound().getDataSetInfo(datapath); for (HDF5CompoundMemberInformation info : infos) { names.add(info.getName().replace(UNSAFE_SPACE_SEPARATOR, SAFE_SPACE_SEPARATOR)); } return names; }

Target:
ArrayList -> List
Source:
public HDF5DrillMetadata() { attributes = new HashMap(); }

Target:
Generics... ```suggestion attributes = new HashMap<>(); ```
Source:
public List<Object> getColumn(int columnIndex) { List<Object> result = new ArrayList<Object>(); for (Object[] compoundDatum : compoundData) { result.add(compoundDatum[columnIndex]); } return result; }

Target:
```suggestion List<Object> result = new ArrayList<>(); ```
Source:
public static void setup() throws Exception { ProtobufPatcher.patch(); GuavaPatcher.patch(); ClusterFixtureBuilder builder = ClusterFixture.builder(dirTestWatcher); startCluster(builder); HDF5FormatConfig formatConfig = new HDF5FormatConfig(); cluster.defineFormat("dfs", "hdf5", formatConfig); dirTestWatcher.copyResourceToRoot(Paths.get("hdf5/")); }

Target:
No need to add these stuff here, it's already covered by `ClusterTest`.
Source:
public TupleSchema(@JsonProperty("columns") List<AbstractColumnMetadata> columns, @JsonProperty("properties") Map<String, String> properties) { if (columns != null && !columns.isEmpty()) { columns.forEach(this::addColumn); } setProperties(properties); }

Target:
Looks like `!columns.isEmpty()` check is redundant.
Source:
private static Set<RexInputRef> findAllRexInputRefs(final RexNode node) { Set<RexInputRef> rexRefs = new HashSet<>(); RexVisitor<Void> visitor = new RexVisitorImpl<Void>(true) { public Void visitInputRef(RexInputRef inputRef) { rexRefs.add(inputRef); return super.visitInputRef(inputRef); } }; node.accept(visitor); return rexRefs; }

Target:
Would this not break the existing logic? For a predicate like $0=$0 using the `Set` would cause the `isMultiColumnPredicate` function to return false.
Source:
public Map<String, String> getQueryOptions() { // Skip reporting QUERY_MAX_ROWS if it is inapplicable and set to zero (e.g. query -> SHOW FILES) return getOptions(o -> OptionValue.OptionScope.QUERY == o.getScope() && !(ExecConstants.QUERY_MAX_ROWS.equals(o.getName()) && String.valueOf(o.getValue()).equals("0"))); }

Target:
Please revert this change.
Source:
public List<? extends MetadataBase.ColumnTypeMetadata> getColumnTypeInfoList() { return new ArrayList(this.columnTypeInfo.values()); }

Target:
Please use generics to avoid warnings in IDE ```suggestion return new ArrayList<>(this.columnTypeInfo.values()); ```
Source:
public void setLargeMaxRows(long maxRowCount) throws SQLException { super.setLargeMaxRows(maxRowCount); try (Statement statement = this.connection.createStatement()) { statement.execute("ALTER SESSION SET `" + ExecConstants.QUERY_MAX_ROWS + "`=" + maxRowCount); statement.close(); } }

Target:
Please remove this line, `statement` will be closed automatically since it is used in `try with resources`.
Source:
public static void tearDownConnection() throws SQLException { if (connection != null) { try (Statement stmt = connection.createStatement()) { stmt.execute(String.format("alter session set `%s` = false", PlannerSettings.ENABLE_DECIMAL_DATA_TYPE_KEY)); } } connection.close(); }

Target:
Please move this call inside if statement. Also, since the connection is closed, there is no need to restore session options.
Source:
public static List<String> getPartitionColumnNames(FileSelection selection, SchemaConfig schemaConfig) { int partitionsCount = getPartitionDepth(selection); String partitionColumnLabel = schemaConfig.getOption(ExecConstants.FILESYSTEM_PARTITION_COLUMN_LABEL).string_val; List<String> partitions = Lists.newArrayList(); // generates partition column names: dir0, dir1 etc. for (int i = 0; i < partitionsCount; i++) { partitions.add(partitionColumnLabel + i); } return partitions; }

Target:
```suggestion List<String> partitions = new ArrayList<>(); ```
Source:
public Builder(DrillBuf managedBuf) { this.managedBuf = managedBuf; this.workingBuffer = new WorkingBuffer(managedBuf); this.mapOutput = new MapVectorOutput(workingBuffer); this.listOutput = new ListVectorOutput(workingBuffer); this.readNumbersAsDouble = false; this.skipOuterList = false; this.allTextMode = false; this.enableNanInf = true; this.enableEscapeAnyChar = false; }

Target:
default value will be false so not required to explicit set it.
Source:
private static File getWebServerTempDirPath(Drillbit drillbit) { File webServerTempDirPath = null; try { Field webServerField = FieldUtils.getField(drillbit.getClass(), "webServer", true); WebServer webServerHandle = (WebServer) FieldUtils.readField(webServerField, drillbit, true); webServerTempDirPath = webServerHandle.getOrCreateTmpJavaScriptDir(); } catch (IllegalAccessException e) { e.printStackTrace(); } return webServerTempDirPath; }

Target:
Please remove this and catch block in general and specify this exception in a throws clause.
Source:
private Map<String, String> makeParserFields() { Map<String, String> fieldMapping = Maps.newHashMap(); for (final SchemaPath sp : getColumns()) { String drillField = sp.getRootSegment().getPath(); try { String parserField = HttpdParser.parserFormattedFieldName(drillField); fieldMapping.put(drillField, parserField); } catch (Exception e) { LOG.info("Putting field: " + drillField + " into map", e); } } return fieldMapping; }

Target:
According to [Maps docs](https://google.github.io/guava/releases/23.0/api/docs/com/google/common/collect/Maps.html#newHashMap--) `newHashMap()` ` [...] method is now unnecessary and should be treated as deprecated. Instead, use the HashMap constructor directly, taking advantage of the new "diamond" syntax.`
Source:
public void close() throws Exception { AutoCloseables.close(this.reader, this.fsStream); }

Target:
```suggestion AutoCloseables.close(reader, fsStream); ```
Source:
public static List<SchemaPath> projectAllWithMetadata(int dirCount) { return RowSetTestUtils.concat( RowSetTestUtils.projectAll(), expandMetadata(2)); }

Target:
Maybe we should pass `dirCount` instead of `2`?
Source:
public void transferTo(UntypedNullVector target) { target.valueCount = valueCount; }

Target:
Please also set `this.valueCount = 0;` here or add it to `reset()` method and call it from this one.
Source:
public boolean isAutoLimitEnabled() { return autoLimitRowCount != null && autoLimitRowCount > 0; }

Target:
```suggestion return autoLimitRowCount > 0; ```
Source:
public QueryWrapper(@JsonProperty("query") String query, @JsonProperty("queryType") String queryType, @JsonProperty("autoLimit") String autoLimit) { this.query = query; this.queryType = queryType.toUpperCase(); this.autoLimitRowCount = autoLimit != null && autoLimit.matches("[0-9]+") ? new Integer(autoLimit) : null; }

Target:
```suggestion this.autoLimitRowCount = autoLimit != null && autoLimit.matches("[0-9]+") ? new Integer(autoLimit) : 0; ```
Source:
public static void stopMysql() { mysqld.stop(); }

Target:
Maybe we should check for null first?
Source:
public void appendMillis(final long p, Map<String, String> attributeMap) { //Embedding dataTable's data-order attribute if (attributeMap == null) { attributeMap = new HashMap<String, String>(); } attributeMap.put(HtmlAttribute.DATA_ORDER, String.valueOf(p)); appendCell((new SimpleDurationFormat(0, p)).compact(), NO_BGCOLOR, attributeMap); }

Target:
new HashMap<>
Source:
public void showSchemas() throws Exception { test("USE " + MINI_DFS_STORAGE_PLUGIN_NAME); testBuilder() .sqlQuery("SHOW SCHEMAS LIKE 'hive.%'") .unOrdered() .baselineColumns("SCHEMA_NAME") .baselineValues("hive.db_general") .baselineValues("hive.default") .go(); }

Target:
Why do you need this? `SHOW SCHEMAS LIKE 'hive.%'` should work without specifying DFS schema as default.
Source:
public static List<FileStatus> listDirectoriesSafe(final FileSystem fs, Path path, boolean recursive, PathFilter... filters) { List<FileStatus> statuses = new ArrayList<>(); try { listDirectories(fs, path, recursive, false, statuses, mergeFilters(filters)); } catch (Exception e) { // all exceptions are ignored } return statuses; }

Target:
Looks like `suppressExceptions` should be `true` here.
Source:
public ParquetReadOptions toReadOptions() { return ParquetReadOptions.builder() .withMetadataFilter(NO_FILTER) .useSignedStringMinMax(enableStringsSignedMinMax) .build(); }

Target:
`NO_FILTER` is set in `ParquetReadOptions.Builder` by default, so it may be removed.
Source:
public DrillSemiJoinRel( RelOptCluster cluster, RelTraitSet traitSet, RelNode left, RelNode right, RexNode condition, ImmutableIntList leftKeys, ImmutableIntList rightKeys) { super(cluster, traitSet, left, right, condition, leftKeys, rightKeys); Preconditions.checkArgument(validateTraits(traitSet, left, right)); }

Target:
The intent of my previous question was to see if this traits validation is already happening in Calcite since this functionality ideally belongs there. I suppose you checked that ?
Source:
public void addRuntimeFilter(RuntimeFilterWritable runtimeFilter) { this.runtimeFilterSink.aggregate(runtimeFilter); }

Target:
Update this method as RuntimeFilter Sink is not used anymore.
Source:
public RawFragmentBatch poll(long timeout, TimeUnit timeUnit) throws InterruptedException, IOException { RawFragmentBatchWrapper batchWrapper = buffer.poll(timeout, timeUnit); if (batchWrapper != null) { try { return batchWrapper.get(); } catch (InterruptedException e) { return null; } } return null; }

Target:
You should not catch InterruptedException as it is being handled by caller; the handling is also different: - Upper code is having more logic on how to deal with an interrupted exception - This code is erasing the interrupted flag whereas caller code doesn't
Source:
public boolean isIntersectIndex() { if (indexProps.size() > 1) { return true; } else { return false; } }

Target:
c'mon - why not `return indexProps.size() > 1 ;` instead ?
Source:
public void testSimpleNoRowKeyJoin() throws Exception { String query = "SELECT `reverseid` from table(hbase.`index_test_primary`(type => 'maprdb', index => 'hash_i_reverseid')) " + "where `reverseid` = 1234"; test(defaultHavingIndexPlan); PlanTestBase.testPlanMatchingPatterns(query, new String[] {".*JsonTableGroupScan.*tableName=.*index_test_primary.*indexName=hash_i_reverseid"}, new String[]{"RowKeyJoin"} ); return; }

Target:
remove `return;` statement int tests
Source:
public void NonCoveringPlan() throws Exception { String query = "SELECT t.`name`.`fname` AS `fname` FROM hbase.`index_test_primary` as t " + " where t.id.ssn = '100007423'"; test(defaultHavingIndexPlan); PlanTestBase.testPlanMatchingPatterns(query, new String[] {"RowKeyJoin", ".*RestrictedJsonTableGroupScan.*tableName=.*index_test_primary,", ".*JsonTableGroupScan.*tableName=.*index_test_primary,.*indexName=i_ssn"}, new String[]{} ); System.out.println("Non-Covering Plan Verified!"); testBuilder() .sqlQuery(query) .ordered() .baselineColumns("fname").baselineValues("KfFzK") .go(); return; }

Target:
possibly `logger.info`?
Source:
public MockStorageEngineConfig(@JsonProperty("url") String url) { this.url = url; this.setEnabled(true); }

Target:
This is not correct place to enable plugin config. You enable plugin after its creation, not directly in config itself.
Source:
public static void setup() throws Exception { startCluster(ClusterFixture.builder(dirTestWatcher)); StoragePluginRegistry pluginRegistry = cluster.drillbit().getContext().getStorage(); FileSystemConfig pluginConfig = (FileSystemConfig) pluginRegistry.getPlugin(CP_PLUGIN_NAME).getConfig(); pluginConfig.setEnabled(false); pluginRegistry.createOrUpdate(CP_PLUGIN_NAME, pluginConfig, true); }

Target:
factor out `StoragePluginRegistry pluginRegistry` to the static field and initialize it in `setup()` method.
Source:
public void registerRuntimeFilter(RuntimeFilterWritable runtimeFilterWritable) { BitData.RuntimeFilterBDef runtimeFilterB = runtimeFilterWritable.getRuntimeFilterBDef(); int majorId = runtimeFilterB.getMajorFragmentId(); UserBitShared.QueryId queryId = runtimeFilterB.getQueryId(); List<String> probeFields = runtimeFilterB.getProbeFieldsList(); logger.info("RuntimeFilterRouter receives a runtime filter , majorId:{}, queryId:{}", majorId, QueryIdHelper.getQueryId(queryId)); broadcastAggregatedRuntimeFilter(majorId, queryId, probeFields, runtimeFilterWritable.getData()); }

Target:
why not just pass `runtimeFilterWritable` to broadcast method instead of extracting individual things and passing it as parameter ?
Source:
public boolean equals(Object other) { if (other instanceof RuntimeFilterWritable) { RuntimeFilterWritable otherRFW = (RuntimeFilterWritable) other; return this.identifier.equals(otherRFW.identifier); } return false; }

Target:
Please include null check for other like: ``` if (other == null) { return false; } ```
Source:
public void testAllMethods() { for (Method method : jdbcIntf.getMethods()) { if (method.isDefault()) { // Skip new default methods from java 10 continue; } testOneMethod(method); } }

Target:
Consider moving to `isOkayNonthrowingMethod()`. What new default methods were introduced in java 10 that require special handling?
Source:
private JDKClassCompiler(JavaCompiler compiler, ClassLoader classLoader, boolean debug) { super(debug); this.compiler = compiler; this.listener = new DrillDiagnosticListener(); this.fileManager = new DrillJavaFileManager(compiler.getStandardFileManager(listener, null, Charsets.UTF_8), classLoader); this.compilerOptions = Lists.newArrayList("-source", "1.8", "-target", "1.8", this.debug ? "-g:source,lines,vars" : "-g:none"); }

Target:
I assume this change should be reverted
Source:
private NullableIntVector createMissingColumn(SchemaPath col, OutputMutator output) throws SchemaChangeException { // col.toExpr() is used here as field name since we don't want to see these fields in the existing maps MaterializedField field = MaterializedField.create(col.toExpr().replace('`',' ').trim(), Types.optional(TypeProtos.MinorType.INT)); return (NullableIntVector) output.addField(field, TypeHelper.getValueVectorClass(TypeProtos.MinorType.INT, DataMode.OPTIONAL)); }

Target:
Why is it necessary to replace \` with a space to trim it later?
Source:
public Double getValue() { if (osMXBean != null) { return osMXBean.getSystemLoadAverage(); } return null; }

Target:
Use `?` operator
Source:
public LateralJoinPrel(RelOptCluster cluster, RelTraitSet traits, RelNode left, RelNode right, boolean excludeCorrelateCol, CorrelationId correlationId, ImmutableBitSet requiredColumns, SemiJoinType semiJoinType) { super(cluster, traits, left, right, excludeCorrelateCol, correlationId, requiredColumns, semiJoinType); }

Target:
I think there is no need to make this constructor public. In `LateralUnnestRowIDVisitor` may be used method `copy()` instead of using the constructor.
Source:
public LimitPrel(RelOptCluster cluster, RelTraitSet traitSet, RelNode child, RexNode offset, RexNode fetch, boolean pushDown, boolean isPartitioned) { super(cluster, traitSet, child, offset, fetch, pushDown); this.isParitioned = isPartitioned; }

Target:
Who is calling this new constructor ? Is that a separate change ?
Source:
public Iterator<Entry<String, V>> getRange(int skip, int take) { Iterable<Entry<String, V>> entries = () -> new Iter(take); return StreamSupport.stream(entries.spliterator(), false) .skip(skip) .limit(take) .iterator(); }

Target:
There is no need to create new objects: neither Stream objects, nor Iterator wrappers. Please update `Iter` inner class to allow specifying ranges (skip (aka offset) and limit).
Source:
private static void checkThatQueryIsNotOptimized(final String query) throws Exception { PlanTestBase.testPlanMatchingPatterns(wrapLimit0(query), new String[]{}, new String[]{ ".*Limit\\(offset=\\[0\\], fetch=\\[0\\]\\)(.*[\n\r])+.*Scan.*" }); }

Target:
No need to create an empty array, `null` may be passed.
Source:
private static void checkThatQueryIsOptimized(final String query) throws Exception { PlanTestBase.testPlanMatchingPatterns(wrapLimit0(query), new String[]{ ".*Limit\\(offset=\\[0\\], fetch=\\[0\\]\\)(.*[\n\r])+.*Scan.*" }, new String[]{}); }

Target:
`PlanTestBase` has a method which takes two arguments, please use that method here.
Source:
public static void setup() throws IOException { Configuration conf = new Configuration(); footer = ParquetFileReader.readFooter(conf, new Path(path)); assertNotNull("footer must be not null", footer); }

Target:
You are checking Footer for null, here but not in prev test.
Source:
public SqlNode validate(final SqlNode parsedNode) { try { return validator.validate(parsedNode); } catch (RuntimeException e) { UserException.Builder builder = UserException .validationError(e) .addContext("SQL Query", (String) null); if (isInnerQuery) { builder.message("Failure validating a view your query is dependent upon."); } throw builder.build(logger); } }

Target:
Why do we need to use it? `addContext()` with one parameter exists
Source:
public PhysicalOperator visitUnnest(final Unnest unnest, final Object obj) { return new org.apache.drill.exec.physical.config.UnnestPOP(null, unnest.getColumn()); }

Target:
add `org.apache.drill.exec.physical.config.` to import statements
Source:
private Map<Integer, Integer> buildMapWithoutCorrColumn(RelNode corr, int correlationIndex) { int index = 0; Map<Integer, Integer> result = new HashedMap(); for (int i=0;i<corr.getRowType().getFieldList().size();i++) { if (i == correlationIndex) { continue; } else { result.put(i, index++); } } return result; }

Target:
Any reason for using a HashedMap instead of just HashMap ? (unless you are using the MapIterator it seems the latter should suffice).
Source:
public int hashCode() { final int prime = 31; int result = 1; result = prime * result + ((name == null) ? 0 : name.hashCode()); return result; }

Target:
how about just `return (prime + ((name == null) ? 0 : name.hashCode());`
Source:
public void aggregate(RuntimeFilterWritable runtimeFilterWritable) { List<BloomFilter> thisFilters = this.unwrap(); List<BloomFilter> otherFilters = runtimeFilterWritable.unwrap(); for (int i = 0; i < thisFilters.size(); i++) { BloomFilter thisOne = thisFilters.get(0); BloomFilter otherOne = otherFilters.get(0); thisOne.or(otherOne); } for (BloomFilter bloomFilter : otherFilters) { bloomFilter.getContent().clear(); } }

Target:
Shouldn't this be get(i), not get(0) ?
Source:
public SelectionVectorMode getEncoding() { return SelectionVectorMode.TWO_BYTE; }

Target:
Can you check if NONE_AND_TWO works for this case ? If the JPPD is not applicable, an SV2 is not created right ?
Source:
public BloomFilter(int ndv, double fpp, BufferAllocator bufferAllocator) { int numBytes = BloomFilter.optimalNumOfBytes(ndv, fpp); int size = BloomFilter.adjustByteSize(numBytes); this.byteBuf = bufferAllocator.buffer(size); this.numBytes = byteBuf.capacity(); this.byteBuf.writerIndex(numBytes); }

Target:
Please change to `this(BloomFilter.optimalNumOfBytes(ndv, fpp), bufferAllocator);`
Source:
private int computeNextBatchHolderSize(int batchSize) { return Math.min(batchSize, Math.min(remainingCapacity, maxBatchHolderSize)); }

Target:
Since we are using batchSize now, we can remove the maxBatchHolderSize field.
Source:
int getBit(TestBitPattern pattern, int index) { if (pattern == TestBitPattern.RANDOM) { final int randomBit = (int) (Math.random() * 2); return randomBit; } return (pattern == TestBitPattern.ALTERNATING) ? (index % 2) : ((pattern == TestBitPattern.ONE) ? 1 : 0); }

Target:
I was just thinking if bitValue is better than randomBit?
Source:
public List<InputSplit> getInputSplits() { return inputSplits; }

Target:
Why `Collection`->`List`? Where order comes from?
Source:
private SchemaPath replaceOverriddenSchemaPath(Map<String, String> parameters, SchemaPath colNameSchemaPath) { String hiveColumnName = colNameSchemaPath.getRootSegmentPath(); if (hiveColumnName.equals(parameters.get(MAPRDB_COLUMN_ID))) { return SchemaPath.getSimplePath(ID_KEY); } else { return colNameSchemaPath; } }

Target:
ternary operator?
Source:
private String replaceOverriddenColumnId(Map<String, String> parameters, String colName) { return Objects.equals(colName, parameters.get(MAPRDB_COLUMN_ID)) ? ID_KEY : colName; }

Target:
Objects equals will consider two null equal, will this case be OK for your changes?
Source:
public static boolean isDirectMemoryOOM(Throwable e) { if (e instanceof OutOfDirectMemoryError || e instanceof OutOfMemoryException) { // These are always direct memory errors return true; } if (e instanceof OutOfMemoryError) { return DIRECT_MEMORY_OOM_MESSAGE.equals(e.getMessage()); } else { return false; } }

Target:
`return (e instanceof OutOfMemoryError) && DIRECT_MEMORY_OOM_MESSAGE.equals(e.getMessage())`?
Source:
public static QueryId getQueryIdFromString(final String queryId) { final UUID uuid = UUID.fromString(queryId); return QueryId.newBuilder().setPart1(uuid.getMostSignificantBits()).setPart2(uuid.getLeastSignificantBits()).setText(queryId).build(); }

Target:
Why is it necessary? Query Id should be identifiable by part1 and part2 of UUID.
Source:
private boolean canRemoveExchange(MajorFragmentStat parentFrag, MajorFragmentStat childFrag) { return (childFrag.isSingular() && parentFrag.isSingular() && (!childFrag.isDistributionStrict() || !parentFrag.isDistributionStrict())) || parentFrag.isRightSideOfLateral(); }

Target:
For better readability, Since the lateral check is independent of the others, can you split it into a separate statement ?
Source:
public void mergeScanSpec(String functionName, KafkaPartitionScanSpec scanSpec) { switch (functionName) { case "booleanAnd": //Reduce the scan range if(startOffset < scanSpec.startOffset) { startOffset = scanSpec.startOffset; } if(endOffset > scanSpec.endOffset) { endOffset = scanSpec.endOffset; } break; case "booleanOr": //Increase the scan range if(scanSpec.startOffset < startOffset) { startOffset = scanSpec.startOffset; } if(scanSpec.endOffset > endOffset) { endOffset = scanSpec.endOffset; } break; default: assert(false); } }

Target:
Here we no need to fail for default. In other plugin's default, it will create empty ScanSpec.
Source:
public Set<String> getTableNames() { if (tables == null) { try { tables = Sets.newHashSet(mClient.getTableNames(this.name, schemaConfig.getIgnoreAuthErrors())); } catch (final TException e) { logger.warn("Failure while attempting to access HiveDatabase '{}'.", this.name, e.getCause()); tables = new HashSet<>(); // empty set. } } return tables; }

Target:
new `HashSet<>(...)`?
Source:
AbstractFieldReader() { super(); }

Target:
You can remove `super()`.
Source:
private NullReader() { super(); type = Types.NULL; }

Target:
Please remove `super()` here and below.
Source:
public void allocateVectors(VectorContainer container) { // Allocate memory for the vectors. // This will iteratively allocate memory for all nested columns underneath. for (VectorWrapper w : container) { RecordBatchSizer.ColumnSize colSize = getColumnSize(w.getField().getName()); colSize.allocateVector(w.getValueVector(), outputRowCount); } }

Target:
Cleaner implementation: Just call the following method, with outputRowCount as the second parameter.
Source:
@Override public final void setup(FragmentContext context, RecordBatch incoming, RecordBatch outgoing, List<TransferPair> transfers, LateralContract lateral) throws SchemaChangeException { this.svMode = incoming.getSchema().getSelectionVectorMode(); switch (svMode) { case FOUR_BYTE: throw new UnsupportedOperationException("Unnest does not support selection vector inputs."); case TWO_BYTE: throw new UnsupportedOperationException("Unnest does not support selection vector inputs."); } this.transfers = ImmutableList.copyOf(transfers); this.lateral = lateral; }

Target:
Could be simplified to if(svMode != SelectionVectorMode.NONE)
Source:
public void testFilterPushDownForExternalTable() throws Exception { String query = "select * from hive.kv_native_ext where key = 1"; int actualRowCount = testSql(query); assertEquals("Expected and actual row count should match", 1, actualRowCount); testPlanMatchingPatterns(query, new String[]{"HiveDrillNativeParquetScan", "numFiles=1"}, new String[]{}); }

Target:
I have added method without `excludedPatterns`, when it is not necessary. But it is not merged for now. Is it better to pass null, than to create empty String?
Source:
public void flushOutgoingBatches(boolean isLastBatch, boolean schemaChanged) throws IOException, InterruptedException { for (OutgoingRecordBatch batch : outgoingBatches) { if (Thread.interrupted()) { throw new InterruptedException(); } logger.debug("Attempting to flush all outgoing batches"); if (isLastBatch) { batch.setIsLast(); } batch.flush(schemaChanged); if (schemaChanged) { batch.resetBatch(); batch.initializeBatch(); } } }

Target:
Since we are checking for interrupts here already could we remove `Thread.currentThread().isInterrupted()` in the flush(boolean schemaChanged) method?
Source:
public boolean isRunning() { if (embeddedJetty == null || embeddedJetty.getConnectors().length != 1) { return false; } return true; }

Target:
Method code can be simplified to one line.
Source:
private int getSpillCycleMetricIndex(CoreOperatorType operatorType) { String metricName = null; switch (operatorType) { case EXTERNAL_SORT: metricName = "SPILL_COUNT"; break; case HASH_AGGREGATE: case HASH_JOIN: metricName = "SPILL_CYCLE"; break; default: return NO_SPILL_METRIC_INDEX; } int metricIndex = 0; //Default String[] metricNames = OperatorMetricRegistry.getMetricNames(operatorType.getNumber()); for (String name : metricNames) { if (name.equalsIgnoreCase(metricName)) { return metricIndex; } metricIndex++; } //Backward compatibility with rendering older profiles. Ideally we should never touch this if an expected metric is not there return NO_SPILL_METRIC_INDEX; }

Target:
Initializing with null is redundant.
Source:
public void sortOneKeyAscending() throws Throwable { ClusterFixtureBuilder builder = ClusterFixture.builder(dirTestWatcher) .configProperty(CodeCompiler.ENABLE_SAVE_CODE_FOR_DEBUG_TOPN, true); try (ClusterFixture cluster = builder.build(); ClientFixture client = cluster.clientFixture()) { TestBuilder testBuilder = new TestBuilder(new ClusterFixture.FixtureTestServices(client)); testBuilder.ordered() .physicalPlanFromFile("topN/one_key_sort.json") .baselineColumns("blue") .go(); } }

Target:
I think this line should be also removed since `ENABLE_SAVE_CODE_FOR_DEBUG_TOPN` should be disabled for tests.
Source:
private BitRpcUtility() throws Exception { throw new UnsupportedOperationException("BitRpcUtility is a utility class and should not be instantiated."); }

Target:
Exception is not required, just make the constructor private.
Source:
protected void log(final String name, final PhysicalPlan plan, final Logger logger) throws JsonProcessingException { if (logger.isDebugEnabled()) { PropertyFilter theFilter = new SimpleBeanPropertyFilter.SerializeExceptFilter(Sets.newHashSet("password")); String planText = plan.unparse(context.getLpPersistence().getMapper() .writer(new SimpleFilterProvider().addFilter("passwordFilter", theFilter))); logger.debug(name + " : \n" + planText); } }

Target:
Please rename to `filter`.
Source:
public void close() throws Exception { in.close(); }

Target:
`in` is not set until `setup()` time. If something goes wrong, Drill might call `close()` before calling `setup()`. For safety, maybe check if `in` is not null.
Source:
private ProjectedColumnInfo makeColumn(final String colName, final Column column) { TypeProtos.MajorType majorType = Types.optional(column.getMinorType()); MaterializedField field = MaterializedField.create(colName, majorType); ValueVector vector = getValueVector(column.getMinorType(), majorType, field, output); return new ProjectedColumnInfo(vector, column, colName); }

Target:
Can pcapng files omit fields? Or, if the user asks for `dst_ip`, say, will that field exist in every row? If the value exists in every row, you'll get slightly better performance using a `required` type for that column. Doing so omits the nullable bit for each column. Note that, for the `DummyImpl`, you do need the type to be `optional`. This might mean that the `MajorType` should be defined in the `Column` class so you can have some be required, others optional.
Source:
public int getPayloadByteCount(int valueCount) { if (valueCount == 0) { return 0; } int count = 0; int entryCount = offsets.getAccessor().get(valueCount); count += offsets.getPayloadByteCount(valueCount); for (final ValueVector v : getChildren()) { count += v.getPayloadByteCount(entryCount); } return count; }

Target:
``` int count = offsets.get... ```
Source:
public FragmentContext getContext() { // Backward compatibility with the full server context. Awkward for testing FragmentContext fragmentContext = fragmentContext(); if (fragmentContext instanceof FragmentContext) { return (FragmentContext) fragmentContext; } else { return null; } }

Target:
not clear what we are doing here. why can't we just return fragmentContext ?
Source:
public ColumnDefn(String name, int index) { this.name = name; this.index = index; this.format = ""; }

Target:
``` this(name, index, ""); ``` Should a missing format be null or empty?
Source:
public abstract void define(OutputMutator outputMutator) throws SchemaChangeException; public abstract void load(int rowIndex, String value); @Override //For testing public String toString() { return "Name: " + name + ", Index: " + index; }

Target:
Format?
Source:
public <T, X, E extends Throwable> T accept(PhysicalVisitor<T, X, E> physicalVisitor, X value) throws E { return physicalVisitor.visitExchange(this, value); }

Target:
Is this change still required?
Source:
private static boolean isMuxEnabled(OptionManager options) { if (options.getOption(PlannerSettings.MUX_EXCHANGE.getOptionName()).bool_val || options.getOption(PlannerSettings.DEMUX_EXCHANGE.getOptionName()).bool_val || options.getOption(PlannerSettings.ORDERED_MUX_EXCHANGE.getOptionName()).bool_val) { return true; } return false; }

Target:
use `return` instead of `if`
Source:
public void close() throws Exception { logger.debug("Partition sender stopping."); super.close(); ok = false; synchronized (this) { if (partitioner != null) { updateAggregateStats(); partitioner.clear(); } } if (closeIncoming) { ((CloseableRecordBatch) incoming).close(); } }

Target:
1. Should partitioner be volatile? Will this be enough without synchronization? 2. Should we check if partitioner is not null before synchronization as well (DCL)?
Source:
public MockPartitionSenderRootExec(FragmentContext context, RecordBatch incoming, HashPartitionSender operator) throws OutOfMemoryException { super(context, incoming, operator, false); }

Target:
Is the change still necessary?
Source:
public void admit() { // No queueing by default foreman.moveToState(UserBitShared.QueryResult.QueryState.STARTING, null); }

Target:
Would REALLY prefer that the resource manager not depend on the Foreman; makes unit testing impossible. The Foreman should depend on this class, but not the other way around.
Source:
public UserIdentity login(final String username, final Object credentials) { UserIdentity identity = null; try { identity = loggedInUgi.doAs(new PrivilegedExceptionAction<UserIdentity>() { @Override public UserIdentity run() { return spnegoLogin(username, credentials); } }); } catch (Exception e) { logger.error("Failed to login using SPNEGO"); } return identity; }

Target:
Please log exception as well to have full stack trace.
Source:
public FileSystemSchemaFactory(FileSystemPlugin plugin, String schemaName, List<WorkspaceSchemaFactory> factories) { super(); this.plugin = plugin; this.schemaName = schemaName; this.factories = factories; }

Target:
Omit; Java does this by default.
Source:
public void testQueryGoodStorageWithDefaultSchema() throws Exception { String use_dfs = "use dfs.tmp"; client.queryBuilder().sql(use_dfs).run(); String sql = "SELECT id_i, name_s10 FROM `mock_good`.`employees_5`"; client.queryBuilder().sql(sql).printCsv(); }

Target:
Do we actually want to print csv here? I suggest we produce no output here.
Source:
public boolean supportsRead() { return Boolean.TRUE; }

Target:
`true` is plenty good here.
Source:
public boolean equals(Object obj) { if (this == obj) { return true; } if (obj == null) { return false; } if (getClass() != obj.getClass()) { return false; } KafkaStoragePluginConfig other = (KafkaStoragePluginConfig) obj; if (kafkaConsumerProps == null) { if (other.kafkaConsumerProps != null) { return false; } } else if (!kafkaConsumerProps.equals(other.kafkaConsumerProps)) { return false; } return true; }

Target:
Just a minor point, but: ``` if (kafkaConsumerProps == null && other.kafkaConsumerProps == null) { return true; } if (kafkaConsumerProps == null || other.kafkaConsumerProps == null) { return false; } return kafkaConsumerProps.equals(other.kafkaConsumerProps); ```
Source:
public void testInvalidSetQueryTimeout() throws SQLException { PreparedStatement stmt = connection.prepareStatement(SYS_VERSION_SQL); //Setting negative value int valueToSet = -10; if (0L == valueToSet) { valueToSet--; } try { stmt.setQueryTimeout(valueToSet); } catch ( final Exception e) { // TODO: handle exception assertThat( e.getMessage(), containsString( "illegal timeout value") ); //Converting this to match expected Exception throw new InvalidParameterSqlException(e.getMessage()); } }

Target:
there's no point I guess? you already verified the exception...
Source:
public void testDefaultGetQueryTimeout() throws SQLException { Statement stmt = connection.createStatement(); int timeoutValue = stmt.getQueryTimeout(); assert( 0 == timeoutValue ); }

Target:
I personally prefer the old construct using hamcrest. This `assert` is actually less useful for two reasons: - there's no error message (hamcrest prints a nice error message based on the arguments) - if the assert fails, you don't know `timeoutValue`'s value
Source:
public void close() throws Exception { in.close(); fs.close(); }

Target:
Since you did not open fs, I guess it's not your responsibility to close it.
Source:
public void close() { if (closed) { logger.debug("Attempted to close Operator context for {}, but context is already closed", popConfig != null ? getName() : null); return; } logger.debug("Closing context for {}", popConfig != null ? getName() : null); closed = true; super.close(); }

Target:
`closed = true` should happen after the call to `super.close()` since base class close can throw exception.
Source:
private int getEstimatedRecordSize(MockColumn[] types) { int x = 0; for (int i = 0; i < (types == null ? 0 : types.length); i++) { x += TypeHelper.getSize(types[i].getMajorType()); } return x; }

Target:
Nit: maybe use: ``` if (types == null) { return 0; } // Original code here.. ```
Source:
public SSLConfig build() throws DrillException { if (config == null) { throw new DrillConfigurationException( "Cannot create SSL configuration from null Drill configuration."); } SSLConfig sslConfig; if (mode == SSLFactory.Mode.SERVER) { sslConfig = new SSLConfigServer(config, hadoopConfig); } else { sslConfig = new SSLConfigClient(config, hadoopConfig); } if(initializeSSLContext){ sslConfig.initContext(); } if(validateKeyStore){ sslConfig.validateKeyStore(); } return sslConfig; }

Target:
_validateKeyStore_ should happen before _initContext_.
Source:
private String getHadoopConfigParam(String name) { Preconditions.checkArgument(this.hadoopConfig != null); String value = ""; value = hadoopConfig.get(name, ""); value = value.trim(); return value; }

Target:
Can be removed and initialized on the next line.
Source:
public void visitAbstractPlan(PhysicalPlan plan) { boolean replanMemory = ! plan.getProperties().hasResourcePlan; if (! replanMemory || plan == null) { return; } MemoryAllocationUtilities.setupBufferedOpsMemoryAllocations(plan, queryContext); }

Target:
You are reading plan and then checking for null later. change it like this. if (!plan || !plan.getProperties().hasResourcePlan) { return; }
Source:
public String toString( ) { String msg = "Embedded queue lease for " + QueryIdHelper.getQueryId(queryId); if (released) { msg += " (released)"; } return msg; }

Target:
`StringBuilder`?
Source:
public void waitForGracePeriod() { ExtendedLatch exitLatch = null; exitLatch = new ExtendedLatch(); exitLatch.awaitUninterruptibly(gracePeriod); }

Target:
``` ExtendedLatch exitLatch = new ExtendedLatch(); ```
Source:
public void closeDrillbit(final String drillbitName) throws Exception { Exception ex = null; for (Drillbit bit : drillbits()) { if(bit.equals(bits.get(drillbitName))) { bit.close(); } } if(ex != null) { throw ex; } }

Target:
Space after `if`, here and below.
Source:
public void testTimeHoursMinutesSecondsFormat() { Locale.setDefault(Locale.US); int hours = 11; int minutes = 50; String seconds = "05"; DateTime date = parseDateFromPostgres(hours + ":" + minutes + ":" + seconds + " am", "hh12:mi:ss am"); Assert.assertTrue(date.getHourOfDay() == hours && date.getMinuteOfHour() == minutes && date.getSecondOfMinute() == Integer.parseInt(seconds)); }

Target:
Not sure that this change is necessary.
Source:
public static boolean isScalarSubquery(RelNode root) { RelMetadataQuery relMetadataQuery = RelMetadataQuery.instance(); RelNode currentRel = root; while (currentRel != null) { if (currentRel instanceof RelSubset) { currentRel = ((RelSubset) currentRel).getBest(); continue; } Double rowCount = relMetadataQuery.getMaxRowCount(currentRel); if (rowCount != null && rowCount.equals(1.0)) { return true; } else { return false; } } return false; }

Target:
For scalar, the value could be <= 1.0 (0 also qualifies).
Source:
protected static void copyMetaDataCacheToTempWithReplacements(String srcFileOnClassPath, String destFolderInTmp, String metaFileName, String customStringReplacement) throws IOException { String metadataFileContents = getFile(srcFileOnClassPath); Path rootMeta = new Path(dfsTestTmpSchemaLocation, destFolderInTmp); Path newMetaCache = new Path(rootMeta, metaFileName); FSDataOutputStream outSteam = fs.create(newMetaCache); if (customStringReplacement!=null) { metadataFileContents = metadataFileContents.replace("CUSTOM_STRING_REPLACEMENT", customStringReplacement); } outSteam.writeBytes(metadataFileContents.replace("REPLACED_IN_TEST", dfsTestTmpSchemaLocation)); outSteam.close(); }

Target:
`if (customStringReplacement!=null) {` -> `if (customStringReplacement != null) {`
Source:
public SystemOptionManager(LogicalPlanPersistence lpPersistence, final PersistentStoreProvider provider) { this.provider = provider; this.config = PersistentStoreConfig.newJacksonBuilder(lpPersistence.getMapper(), OptionValue.class) .name("sys.options") .build(); }

Target:
A really handy trick to avoid code duplication is this: ``` public SystemOptionManager(LogicalPlanPersistence lpPersistence, final PersistentStoreProvider provider) { this(lpPersistence, provider, null); } ``` Then, the bulk of the constructor code appears only once: in the three-arg version.
Source:
public void populateDefaultValues() { // populate the options from the config final Map<String, OptionValidator> tmp = new HashMap<>(); for (OptionValidator validator: VALIDATORS.values()) { try { validator.loadDefault(bootConfig); } catch (ConfigException.Missing e) { throw new IllegalStateException("Config Option is missing"+e+ validator.getOptionName()); } } }

Target:
`...missing " + validator.getOptionName()` The exception adds no real value here. Or: `new IllegalStateException(..., e);`
Source:
public int computeMaxWidth(double cpuLoadAverage, long maxWidth) { // if maxwidth is already set return it if (maxWidth != 0) { return (int) maxWidth; } // else compute the value and return else { int maxWidthPerNode; int availProc = Runtime.getRuntime().availableProcessors(); maxWidthPerNode = (int) Math.max(1, Math.min(availProc, Math.round(availProc * cpuLoadAverage))); return maxWidthPerNode; } }

Target:
`int maxWidthPerNode = ...` or even just `return (int) ...`
Source:
public void testJDKClassCompiler() throws Exception { logger.debug("Testing JDKClassCompiler"); sessionOptions.setOption(OptionValue.createString(OptionType.SESSION, ClassCompilerSelector.JAVA_COMPILER_OPTION, ClassCompilerSelector.CompilerPolicy.JDK.name(), OptionScope.SESSION)); for (int i = 0; i < ITERATION_COUNT; i++) { compilationInnerClass(false); // Traditional byte-code manipulation compilationInnerClass(true); // Plain-old Java } }

Target:
Hmmm... We should not need every caller that sets a session option to say that it is setting a session option. Instead, the implementation of `setOption` for the session option manager class should fill in the `OptionScope.SESSION` value.
Source:
public SchemaBuilder(BatchSchema baseSchema) { for (MaterializedField field : baseSchema) { columns.add(field); } }

Target:
just call `add` method of `SchemaBuilder` instead of `columns.add(field)`
Source:
private void flush() throws IOException { if (recordCount > 0) { parquetFileWriter.startBlock(recordCount); consumer.flush(); store.flush(); pageStore.flushToFileWriter(parquetFileWriter); recordCount = 0; parquetFileWriter.endBlock(); // we are writing one single block per file parquetFileWriter.end(extraMetaData); parquetFileWriter = null; } store.close(); pageStore.close(); store = null; pageStore = null; index++; }

Target:
This is existing code; but seems that the above code should be in a try block, with close in the finally, to ensure that the close occurs even if a write or other error occurs.
Source:
protected boolean hasCapacity(int index, int fieldLength) { if (fieldLength < 0) { throw new IllegalArgumentException("length: " + fieldLength + " (expected: >= 0)"); } return (! (index < 0 || index > capacity() - fieldLength)); }

Target:
change this to an assertion as per our discussion?
Source:
public boolean setBytesBounded(int index, byte[] src, int srcIndex, int length) { if (! hasCapacity(index, length)) { return false; } if (length != 0) { PlatformDependent.copyMemory(src, srcIndex, addr(index), length); } return true; }

Target:
Remove length check
Source:
public PcapRecordReader(final String inputPath, final List<SchemaPath> projectedColumns) { try { this.in = new FileInputStream(inputPath); this.decoder = getPacketDecoder(); validBytes = in.read(buffer); } catch (IOException e) { throw new RuntimeException("File " + inputPath + " not Found"); } setColumns(projectedColumns); }

Target:
As noted above, by opening the file here, if you are scanning 1000 files, you'll have 1000 open file handles at the start of the fragment. Better to postpone opening files until setup.
Source:
private ValueVector getValueVector(final MinorType minorType, final MajorType majorType, final MaterializedField field) { try { final Class<? extends ValueVector> clazz = TypeHelper.getValueVectorClass( minorType, majorType.getMode()); ValueVector vector = output.addField(field, clazz); vector.allocateNew(); return vector; } catch (SchemaChangeException sce) { throw new NullPointerException("The addition of this field is incompatible with this OutputMutator's capabilities"); } }

Target:
`NullPointerException` --> `IllegalStateException` This is not really a null pointer, but it is an illegal state...
Source:
public QueryType getType() { QueryType type = QueryType.SQL; switch (queryType) { case "SQL": type = QueryType.SQL; break; case "LOGICAL": type = QueryType.LOGICAL; break; case "PHYSICAL": type = QueryType.PHYSICAL; break; } return type; }

Target:
`return QueryType.valueOf(queryType);`
Source:
public int extraMemoryNeededForResize() { if (tableSize == MAXIMUM_CAPACITY) { return 0; } // will not resize int newSize = roundUpToPowerOf2(2 * tableSize); if (newSize > MAXIMUM_CAPACITY) { newSize = MAXIMUM_CAPACITY; } return newSize * 4 /* sizeof(int) */; }

Target:
Should this return a long ?
Source:
private void reinitPartition(int part) throws SchemaChangeException, ClassTransformationException, IOException { assert htables[part] != null; htables[part].reset(); if ( batchHolders[part] != null) { for (BatchHolder bh : batchHolders[part]) { bh.clear(); } batchHolders[part].clear(); } batchHolders[part] = new ArrayList<BatchHolder>(); // First BatchHolder is created when the first put request is received. }

Target:
Suppose numPendingOutput is small (e.g 100 rows) but there are several such batches, we would end up calling allocateOutgoing() multiple times for small amounts...shouldn't we try to combine these into smaller number of outgoing batches ?
Source:
private void runPreparedStatement(final PreparedStatementHandle preparedStatementHandle) throws ExecutionSetupException { final ServerPreparedStatementState serverState; try { serverState = ServerPreparedStatementState.PARSER.parseFrom(preparedStatementHandle.getServerInfo()); } catch (final InvalidProtocolBufferException ex) { throw UserException.parseError(ex) .message("Failed to parse the prepared statement handle. " + "Make sure the handle is same as one returned from create prepared statement call.") .build(logger); } final String sql = serverState.getSqlQuery(); logger.info("Prepared statement query for QueryId : {}", queryId, sql); runSQL(sql); // keep a query text of the preparedStatement for the queryManager queryText = sql; }

Target:
Temporary variable can be removed.
Source:
public static int getChildCount(PhysicalOperator popConfig) { Iterator<PhysicalOperator> iter = popConfig.iterator(); int i = 0; while (iter.hasNext()) { iter.next(); i++; } if (i == 0) { i = 1; } return i; }

Target:
Refactor i -> count
Source:
public void close() { metrics.updateWriteBytes(spilledRuns.getWriteBytes()); RuntimeException ex = null; try { spilledRuns.close(); } catch (RuntimeException e) { ex = e; } try { bufferedBatches.close(); } catch (RuntimeException e) { ex = e; } if (ex != null) { throw ex; } }

Target:
Being nit-picky: This may erase the previous spilledRuns runtime exception ....
Source:
public T get(String key) { Integer index = getIndex(key); if (index == -1) { return null; } return get(index); }

Target:
`int` rather than `Integer`
Source:
String getConnection() { return connection; }

Target:
I have noticed that there is an error during plugin configuration serialization, this happens because method accessor is not public.
Source:
public OpenTSDBStoragePlugin(OpenTSDBStoragePluginConfig configuration, DrillbitContext context, String name) throws IOException { this.context = context; this.schemaFactory = new OpenTSDBSchemaFactory(this, name); this.engineConfig = configuration; this.db = new ServiceImpl("http://" + configuration.getConnection()); }

Target:
Can it be `https`? May be user should be responsible of setting this in config?
Source:
private void recordNewState(final QueryState newState) { state = newState; if (queryContext.getOptions().getOption(ExecConstants.ZK_QUERY_STATE_UPDATE)) { queryManager.updateEphemeralState(newState); } }

Target:
How does this affect query operation for long-running queries? How does it impact the query profile? If updates are enabled, do we still do an update at query completion to finalize the profile? If not, should writing of the profile be automatically disabled if status updates are disabled? Do we do any timeout on updates? Will we notice that the query has not been updated and, say, kill the query due to timeouts?
Source:
ClusterFixture(FixtureBuilder builder) { String zkConnect = configureZk(builder); try { createConfig(builder, zkConnect); startDrillbits(builder); applyOptions(builder); } catch (Exception e) { // Translate exceptions to unchecked to avoid cluttering // tests. Failures will simply fail the test itself. throw new IllegalStateException( "Cluster fixture setup failed", e ); } // Some operations need an allocator. allocator = RootAllocatorFactory.newRoot(config); }

Target:
In createConfig we are calling `getServiceSetWithFullCache(config, allocator)`. But we are creating `allocator` later on. We should assign allocator before calling above method. But allocator creation also takes in config so probably we have to do inside `createConfig` method. On a side note I don't see `getServiceSetWithFullCache(config, allocator)` using allocator or config anyway. Not sure why we are passing in first place.