auphong2707/dl4se-code-review-dataset huggingface dataset:
Source:
public void tearDown() { START if (waterMarkEventGenerator != null) { END waterMarkEventGenerator.shutdown(); } } Nit: I don't think this can be null

Target:
public void tearDown() { waterMarkEventGenerator.shutdown(); }
Source:
public void shutdown() { LOG.debug("Shutting down WaterMarkEventGenerator"); START if (this.executorService != null) { END executorService.shutdown(); try { if (!executorService.awaitTermination(2, TimeUnit.SECONDS)) { executorService.shutdownNow(); } } catch (InterruptedException ie) { executorService.shutdownNow(); Thread.currentThread().interrupt(); } } } Nit: This doesn't seem like it can be null?

Target:
public void shutdown() { LOG.debug("Shutting down WaterMarkEventGenerator"); executorService.shutdown(); try { if (!executorService.awaitTermination(2, TimeUnit.SECONDS)) { executorService.shutdownNow(); } } catch (InterruptedException ie) { executorService.shutdownNow(); Thread.currentThread().interrupt(); } }
Source:
public JedisClusterConfig(Set<InetSocketAddress> nodes, int timeout, int maxRedirections) { START Preconditions.checkNotNull(nodes, "Node information should be presented"); END this.nodes = nodes; this.timeout = timeout; this.maxRedirections = maxRedirections; } Minor: Maybe better to call `this(nodes, timeout, maxRedirections, null);` to remove duplicate codes.

Target:
public JedisClusterConfig(Set<InetSocketAddress> nodes, int timeout, int maxRedirections) { this(nodes, timeout, maxRedirections, null); }
Source:
public LinearDRPCInputDeclarer addSharedMemory(SharedMemory request) { _component.sharedMemory.add(request); START return null; END } Should this return null?

Target:
public LinearDRPCInputDeclarer addSharedMemory(SharedMemory request) { _component.sharedMemory.add(request); return this; }
Source:
public WorkerSlot(String nodeId, Number port) { START if (port == null) throw new NullPointerException("port cannot be null"); END if (nodeId == null) throw new NullPointerException("node id cannot be null"); this.nodeId = nodeId; this.port = port.intValue(); } I think checkstyle will complain about this due to missing braces. Maybe you can use http://google.github.io/guava/releases/snapshot/api/docs/com/google/common/base/Preconditions.html#checkNotNull(T) instead?

Target:
public WorkerSlot(String nodeId, Number port) { if (port == null) { throw new NullPointerException("port cannot be null"); } if (nodeId == null) { throw new NullPointerException("node id cannot be null"); } this.nodeId = nodeId; this.port = port.intValue(); }
Source:
public void start() { // initial delay is slightly less than the duration so that the initial tuples wont't expire due to time drift START long initialDelay = duration - Math.min((long) (duration * .1), 100); END executorFuture = executor.scheduleAtFixedRate(newTriggerTask(), initialDelay, duration, TimeUnit.MILLISECONDS); } If the initial delay is slightly less than the given duration then its subsequent scheduling will happen after the given period of initialDelay which can be earlier than expected. Is not it?

Target:
public void start() { executorFuture = executor.scheduleAtFixedRate(newTriggerTask(), duration, duration, TimeUnit.MILLISECONDS); }
Source:
public void getAutoCredentialsTest() { START Map<String, Object> emptyMap = new HashMap<>(); END Map<String, Object> map = new HashMap<>(); map.put(Config.TOPOLOGY_AUTO_CREDENTIALS, Arrays.asList(new String[]{"org.apache.storm.security.auth.AuthUtilsTestMock"})); Assert.assertTrue(AuthUtils.GetAutoCredentials(emptyMap).isEmpty()); Assert.assertEquals(AuthUtils.GetAutoCredentials(map).size(), 1); } Nit: Consider replacing this with Collections.emptyMap() in L107.

Target:
public void getAutoCredentialsTest() { Map<String, Object> map = new HashMap<>(); map.put(Config.TOPOLOGY_AUTO_CREDENTIALS, Arrays.asList(new String[]{"org.apache.storm.security.auth.AuthUtilsTestMock"})); Assert.assertTrue(AuthUtils.GetAutoCredentials(Collections.emptyMap()).isEmpty()); Assert.assertEquals(AuthUtils.GetAutoCredentials(map).size(), 1); }
Source:
public static byte[] serialize(Object obj) throws IOException { try (ByteArrayOutputStream b = new ByteArrayOutputStream()) { try (ObjectOutputStream o = new ObjectOutputStream(b)) { o.writeObject(obj); START o.close(); END } return b.toByteArray(); } } >o.close(); [](start = 16, length = 10) not needed...

Target:
public static byte[] serialize(Object obj) throws IOException { try (ByteArrayOutputStream b = new ByteArrayOutputStream()) { try (ObjectOutputStream o = new ObjectOutputStream(b)) { o.writeObject(obj); } return b.toByteArray(); } }
Source:
public List<Object> deserialize(EventData eventData) { final List<Object> fieldContents = new ArrayList<Object>(); String messageData = ""; START messageData = new String (eventData.getBody(),eventData.getBodyOffset(),eventData.getBodyLength(),Charset.defaultCharset()); END Map metaDataMap = eventData.getProperties(); fieldContents.add(messageData); fieldContents.add(metaDataMap); return fieldContents; } The old code handles the case when AmqpValue is species as payload - if you need that support here - you will need to wait until 0.12.0 Refer to this pr for details : https://github.com/Azure/azure-event-hubs-java/pull/66

Target:
public List<Object> deserialize(EventData eventData) { final List<Object> fieldContents = new ArrayList<Object>(); String messageData = ""; if(eventData.getBody()!=null) messageData = new String (eventData.getBody(),eventData.getBodyOffset(),eventData.getBodyLength(),Charset.defaultCharset()); Map metaDataMap = eventData.getProperties(); fieldContents.add(messageData); fieldContents.add(metaDataMap); return fieldContents; }
Source:
public void fail(Object msgId) { KafkaMessageId id = (KafkaMessageId) msgId; PartitionManager m = _coordinator.getManager(id.partition); if (m != null) { m.fail(id.offset); START } { END // managers for partitions changed - try to find new manager responsible for that partition PartitionManager newManager = getManagerForPartition(id.partition.partition); if (newManager != null) { newManager.fail(id.offset); } } } missing `else` between `} {`

Target:
public void fail(Object msgId) { KafkaMessageId id = (KafkaMessageId) msgId; PartitionManager m = _coordinator.getManager(id.partition); if (m != null) { m.fail(id.offset); } else { // managers for partitions changed - try to find new manager responsible for that partition PartitionManager newManager = getManagerForPartition(id.partition.partition); if (newManager != null) { newManager.fail(id.offset); } } }
Source:
public KafkaSpoutConfig<K,V> build() { Number maxPollRecords = (Number)kafkaProps.get(ConsumerConfig.MAX_POLL_RECORDS_CONFIG); if(maxPollRecords == null) { START setMaxPollRecords(maxUncommittedOffsets); END } else { setMaxPollRecords(Math.min(maxPollRecords.intValue(), maxUncommittedOffsets)); } return new KafkaSpoutConfig<>(this); } @srdo I don't agree with this. If the user does not set MAX_POLL_RECORDS_CONFIG, this value should keep its Kafka default. It should not silently be set to maxUncommittedOffsets. They are two unrelated things, although I agree that MAX_POLL_RECORDS_CONFIG should not exceed maxUncommittedOffsets

Target:
public KafkaSpoutConfig<K,V> build() { return new KafkaSpoutConfig<>(this); }
Source:
public static PartitionAssignmentChangeListener listenerOf(final ConsumerRebalanceListener consumerRebalanceListener) { START return new PartitionAssignmentChangeListener() { END @Override public void onPartitionAssignmentChange(Set<TopicPartition> oldPartitions, Set<TopicPartition> newPartitions) { consumerRebalanceListener.onPartitionsRevoked(oldPartitions); consumerRebalanceListener.onPartitionsAssigned(newPartitions); } }; } You don't have to change it, but on master we're targeting Java 8, so lambdas are an option for this kind of thing :)

Target:
public static PartitionAssignmentChangeListener listenerOf(final ConsumerRebalanceListener consumerRebalanceListener) { return (oldPartitions, newPartitions) -> { consumerRebalanceListener.onPartitionsRevoked(oldPartitions); consumerRebalanceListener.onPartitionsAssigned(newPartitions); }; }
Source:
public void activate() { START openReader(); END } seems every time the spout is deactivated/activated it starts reading the file from the beginning which may not be desirable.

Target:
public void activate() { }
Source:
public static Builder<String, String> builder(String bootstrapServers, Collection<String> topics) { START return new Builder<>(bootstrapServers, new StringDeserializer(), new StringDeserializer(), topics); END } I don't think this works when the topology is serialized. See https://github.com/apache/storm/pull/1696/files#diff-d2c28214a421176fceab415ffe4c95c0R24

Target:
public static Builder<String, String> builder(String bootstrapServers, Collection<String> topics) { return new Builder<>(bootstrapServers, StringDeserializer.class, StringDeserializer.class, topics); }
Source:
private void subscribeKafkaConsumer() { kafkaConsumer = kafkaConsumerFactory.createConsumer(kafkaSpoutConfig); kafkaSpoutConfig.getSubscription().subscribe(kafkaConsumer, new KafkaSpoutConsumerRebalanceListener(), context); // Initial poll to get the consumer registration process going. // KafkaSpoutConsumerRebalanceListener will be called following this poll, upon partition registration START kafkaConsumer.poll(0); END } Doesn't this belong in the automatic subscriptions? It's not necessary if using manual partition assignment.

Target:
private void subscribeKafkaConsumer() { kafkaConsumer = kafkaConsumerFactory.createConsumer(kafkaSpoutConfig); kafkaSpoutConfig.getSubscription().subscribe(kafkaConsumer, new KafkaSpoutConsumerRebalanceListener(), context); }
Source:
public void prepare(Map stormConfig, TopologyContext topologyContext, OutputCollector outputCollector) { this.outputCollector = outputCollector; this.stormConfig = stormConfig; START Map<String, Object> cassandraClientConfig = cassandraConfig != null ? stormConfig : stormConfig; END this.cassandraConf = new CassandraConf(cassandraClientConfig); this.client = clientProvider.getClient(cassandraClientConfig); try { session = client.connect(); } catch (NoHostAvailableException e) { outputCollector.reportError(e); } } This always pick stormConfig which seems not intended.

Target:
public void prepare(Map stormConfig, TopologyContext topologyContext, OutputCollector outputCollector) { this.outputCollector = outputCollector; this.stormConfig = stormConfig; Map<String, Object> cassandraClientConfig = cassandraConfig != null ? cassandraConfig : stormConfig; this.cassandraConf = new CassandraConf(cassandraClientConfig); this.client = clientProvider.getClient(cassandraClientConfig); try { session = client.connect(); } catch (NoHostAvailableException e) { outputCollector.reportError(e); } }
Source:
private static Set<String> getListOfKeysFromBlobStore(Map<String, Object> stormConf) { START NimbusBlobStore client = new NimbusBlobStore(); END client.prepare(stormConf); return Sets.newHashSet(client.listKeys()); } Need to guarantee calling client.shutdown().

Target:
private static Set<String> getListOfKeysFromBlobStore(Map<String, Object> stormConf) { try (NimbusBlobStore client = new NimbusBlobStore()) { client.prepare(stormConf); return Sets.newHashSet(client.listKeys()); } }
Source:
private void emit() { //Keep trying to send a tuple when requested START while(!emitTupleIfNotEmitted(waitingToEmit.next()) && waitingToEmit.hasNext()) END waitingToEmit.remove(); } While I don't think it hurts, I don't see why this is necessary. Storm should call nextTuple repeatedly until something is emitted.

Target:
private void emit() { while(!emitTupleIfNotEmitted(waitingToEmit.next()) && waitingToEmit.hasNext()) { waitingToEmit.remove(); } }
Source:
void populateTopicData(String topicName, int msgCount) { kafkaUnitRule.getKafkaUnit().createTopic(topicName); IntStream.range(0, msgCount).forEach(value -> { KeyedMessage<String, String> keyedMessage = new KeyedMessage<>( START topicName, ((Integer)value).toString(), END ((Integer)value).toString()); kafkaUnitRule.getKafkaUnit().sendMessages(keyedMessage); }); } The Integer cast seems unnecessary?

Target:
void populateTopicData(String topicName, int msgCount) { kafkaUnitRule.getKafkaUnit().createTopic(topicName); IntStream.range(0, msgCount).forEach(value -> { KeyedMessage<String, String> keyedMessage = new KeyedMessage<>( topicName, Integer.toString(value), Integer.toString(value)); kafkaUnitRule.getKafkaUnit().sendMessages(keyedMessage); }); }
Source:
START static public Config getConfig() { END Config config = new Config(); config.setDebug(true); return config; } Nitpick: In all of the methods declared in this class, can you put the method modifiers in the order public static. That is the most standard way and recommended by the [JLS](https://docs.oracle.com/javase/specs/jls/se8/html/jls-8.html#jls-8.4.3). I also believe it makes the code a bit easier to read.

Target:
public static Config getConfig() { Config config = new Config(); config.setDebug(true); return config; }
Source:
private boolean poll() { final int maxUncommittedOffsets = kafkaSpoutConfig.getMaxUncommittedOffsets(); final boolean poll = !waitingToEmit() && numUncommittedOffsets < maxUncommittedOffsets; if (!poll) { START final boolean isLoggable = logTimer.isExpiredResetOnTrue(); END if (isLoggable) { if (waitingToEmit()) { LOG.debug("Not polling. Tuples waiting to be emitted. uncommitted offsets across all topic partitions", numUncommittedOffsets); } if (numUncommittedOffsets >= maxUncommittedOffsets) { LOG.debug("Not polling. uncommitted offsets across all topic partitions has reached the threshold of [{}]", numUncommittedOffsets, maxUncommittedOffsets); } } } return poll; } do we need this check since these are debug logs.

Target:
private boolean poll() { final int maxUncommittedOffsets = kafkaSpoutConfig.getMaxUncommittedOffsets(); final boolean poll = !waitingToEmit() && numUncommittedOffsets < maxUncommittedOffsets; if (!poll) { if (waitingToEmit()) { LOG.debug("Not polling. Tuples waiting to be emitted. uncommitted offsets across all topic partitions", numUncommittedOffsets); } if (numUncommittedOffsets >= maxUncommittedOffsets) { LOG.debug("Not polling. uncommitted offsets across all topic partitions has reached the threshold of [{}]", numUncommittedOffsets, maxUncommittedOffsets); } } return poll; }
Source:
public LSWorkerHeartbeat readHeartbeat() throws IOException { LocalState localState = ConfigUtils.workerState(_conf, _workerId); LSWorkerHeartbeat hb = localState.getWorkerHeartBeat(); START LOG.warn("{}: Reading heartbeat {}", _workerId, hb); END return hb; } debug

Target:
public LSWorkerHeartbeat readHeartbeat() throws IOException { LocalState localState = ConfigUtils.workerState(_conf, _workerId); LSWorkerHeartbeat hb = localState.getWorkerHeartBeat(); LOG.trace("{}: Reading heartbeat {}", _workerId, hb); return hb; }
Source:
public boolean areAllProcessesDead() throws IOException { Set<Long> pids = getAllPids(); String user = getWorkerUser(); boolean allDead = true; for (Long pid: pids) { if (!isProcessAlive(pid, user)) { START LOG.warn("{}: PID is dead", _workerId, pid); END } else { allDead = false; break; } } return allDead; } debug?

Target:
public boolean areAllProcessesDead() throws IOException { Set<Long> pids = getAllPids(); String user = getWorkerUser(); boolean allDead = true; for (Long pid: pids) { if (!isProcessAlive(pid, user)) { LOG.debug("{}: PID is dead", _workerId, pid); } else { allDead = false; break; } } return allDead; }
Source:
public static String workerPidPath(Map<String, Object> conf, String id, long pid) { START return (workerPidsRoot(conf, id) + FILE_SEPARATOR + pid); END } Nit: might be better to call workerPidPath(conf, id, String.valueOf(pid)) or vice versa.

Target:
public static String workerPidPath(Map<String, Object> conf, String id, long pid) { return workerPidPath(conf, id, String.valueOf(pid)); }
Source:
public KinesisConnectionInfo (AWSCredentialsProvider credentialsProvider, ClientConfiguration clientConfiguration, Regions region, Integer recordsLimit) { if (recordsLimit == null || recordsLimit <= 0) { throw new IllegalArgumentException("recordsLimit has to be a positive integer"); } if (region == null) { throw new IllegalArgumentException("region cannot be null"); } serializedKinesisCredsProvider = getKryoSerializedBytes(credentialsProvider); serializedkinesisClientConfig = getKryoSerializedBytes(clientConfiguration); this.recordsLimit = recordsLimit; this.region = region; START this.credentialsProvider = null; END this.clientConfiguration = null; } we don't need to explicitly assign a null to these.

Target:
public KinesisConnectionInfo (AWSCredentialsProvider credentialsProvider, ClientConfiguration clientConfiguration, Regions region, Integer recordsLimit) { if (recordsLimit == null || recordsLimit <= 0) { throw new IllegalArgumentException("recordsLimit has to be a positive integer"); } if (region == null) { throw new IllegalArgumentException("region cannot be null"); } serializedKinesisCredsProvider = getKryoSerializedBytes(credentialsProvider); serializedkinesisClientConfig = getKryoSerializedBytes(clientConfiguration); this.recordsLimit = recordsLimit; this.region = region; }
Source:
public void testDefaults() { SimpleFileNameFormat format = new SimpleFileNameFormat(); format.prepare(null, createTopologyContext()); String path = format.getPath(); String name = format.getName(1, System.currentTimeMillis()); Assert.assertEquals("/storm", path); String now = new SimpleDateFormat("yyyyMMddHHmmss").format(System.currentTimeMillis()); START Assert.assertEquals(now+".1.txt", name); END } This is a nit, but this test might occasionally fail since the time is taken at two different points which might resolve to different seconds.

Target:
public void testDefaults() { SimpleFileNameFormat format = new SimpleFileNameFormat(); format.prepare(null, createTopologyContext()); long now = System.currentTimeMillis(); String path = format.getPath(); String name = format.getName(1, now); Assert.assertEquals("/storm", path); String time = new SimpleDateFormat("yyyyMMddHHmmss").format(now); Assert.assertEquals(time+".1.txt", name); }
Source:
public BaseWindowedBolt withLateTupleStream(String streamId) { START if (!windowConfiguration.containsKey(Config.TOPOLOGY_BOLTS_TUPLE_TIMESTAMP_FIELD_NAME)) { END throw new IllegalArgumentException("Timestamp field should be specified before specifying stream for late tuples"); } windowConfiguration.put(Config.TOPOLOGY_BOLTS_LATE_TUPLE_STREAM, streamId); return this; } On re-thinking, can this validation be moved to `initWindowManager` or `validate` so that we don't impose an ordering in the builder?

Target:
public BaseWindowedBolt withLateTupleStream(String streamId) { windowConfiguration.put(Config.TOPOLOGY_BOLTS_LATE_TUPLE_STREAM, streamId); return this; }
Source:
public void declareOutputFields(OutputFieldsDeclarer declarer) { bolt.declareOutputFields(declarer); START super.declareCheckpointStream(declarer); END } Nit: It doesn't need to be called with `super.`

Target:
public void declareOutputFields(OutputFieldsDeclarer declarer) { bolt.declareOutputFields(declarer); declareCheckpointStream(declarer); }
Source:
private boolean isFilteredIn(String metricName) { START if (noneSpecified) { END return true; } if (!whitelistPattern.isEmpty()) { return checkMatching(metricName, whitelistPattern, true); } else if (!blacklistPattern.isEmpty()) { return checkMatching(metricName, blacklistPattern, false); } throw new IllegalStateException("Shouldn't reach here"); } should this be moved/copied to apply function?

Target:
private boolean isFilteredIn(String metricName) { if (!whitelistPattern.isEmpty()) { return checkMatching(metricName, whitelistPattern, true); } else if (!blacklistPattern.isEmpty()) { return checkMatching(metricName, blacklistPattern, false); } throw new IllegalStateException("Shouldn't reach here"); }
Source:
public boolean isWorkerHbTimedOutImpl(int now, LSWorkerHeartbeat whb, Map conf) { boolean result = false; if ((now - whb.get_time_secs()) > Utils.getInt(conf.get(Config.SUPERVISOR_WORKER_TIMEOUT_SECS))) { result = true; } START return result; END } Minor: This all could just be `return (now - whb.get_time_secs()) > Utils.getInt(conf.get(Config.SUPERVISOR_WORKER_TIMEOUT_SECS));`

Target:
public boolean isWorkerHbTimedOutImpl(int now, LSWorkerHeartbeat whb, Map conf) { return (now - whb.get_time_secs()) > Utils.getInt(conf.get(Config.SUPERVISOR_WORKER_TIMEOUT_SECS)); }
Source:
START public final static List<ACL> supervisorZkAcls() { END final List<ACL> acls = new ArrayList<>(); acls.add(ZooDefs.Ids.CREATOR_ALL_ACL.get(0)); acls.add(new ACL((ZooDefs.Perms.READ ^ ZooDefs.Perms.CREATE), ZooDefs.Ids.ANYONE_ID_UNSAFE)); return acls; } minor nit: static implies final for functions/methods, we don't need final.

Target:
public static List<ACL> supervisorZkAcls() { final List<ACL> acls = new ArrayList<>(); acls.add(ZooDefs.Ids.CREATOR_ALL_ACL.get(0)); acls.add(new ACL((ZooDefs.Perms.READ ^ ZooDefs.Perms.CREATE), ZooDefs.Ids.ANYONE_ID_UNSAFE)); return acls; }
Source:
START private void launchChannelAliveThread() { END // netty TimerTask is already defined and hence a fully // qualified name timer.schedule(new java.util.TimerTask() { public void run() { try { LOG.debug("running timer task, address {}", dstAddress); if(closing) { this.cancel(); } getConnectedChannel(); } catch (Exception exp) { LOG.error("channel connection error {}", exp); } } }, 0, CHANNEL_ALIVE_INTERVAL_MS); } Let's move the code in the static block above down here at the beginning of `launchChannelAliveThread`, so that as @revans2 suggests, the stack trace will not be so confusing. ``` Java if (timer == null) { synchronized (Client.class) { if (timer == null) { timer = new Timer("Netty-ChannelAlive-Timer", true); } } } ```

Target:
private void launchChannelAliveThread() { // netty TimerTask is already defined and hence a fully // qualified name timer.schedule(new java.util.TimerTask() { public void run() { try { LOG.debug("running timer task, address {}", dstAddress); if(closing) { this.cancel(); return; } getConnectedChannel(); } catch (Exception exp) { LOG.error("channel connection error {}", exp); } } }, 0, CHANNEL_ALIVE_INTERVAL_MS); }
Source:
protected Map<String, StormBase> topologyBasesImpl(IStormClusterState stormClusterState) { List<String> activeTopologys = stormClusterState.activeStorms(); Map<String, StormBase> stormBases = new HashMap<String, StormBase>(); if (activeTopologys != null) { for (String topologyId : activeTopologys) { StormBase base = stormClusterState.stormBase(topologyId, null); START if (base != null) { END stormBases.put(topologyId, base); } } } return stormBases; } The original code didn't check for null here. Is this needed? Extra checks that fix things up like this make me nervous that be broke something somewhere and we are using this to work around it.

Target:
protected Map<String, StormBase> topologyBasesImpl(IStormClusterState stormClusterState) { List<String> activeTopologys = stormClusterState.activeStorms(); Map<String, StormBase> stormBases = new HashMap<String, StormBase>(); for (String topologyId : activeTopologys) { StormBase base = stormClusterState.stormBase(topologyId, null); stormBases.put(topologyId, base); } return stormBases; }
Source:
public static Map componentConf(Object component) { Map<Object, Object> conf = new HashMap<Object, Object>(); ComponentCommon common = getComponentCommon(component); START if (common != null) { END String jconf = common.get_json_conf(); if (jconf != null) { conf.putAll((Map<Object, Object>) JSONValue.parse(jconf)); } } return conf; } The original code didn't check for null and would throw an NPE if we passed in a null. We should do the same.

Target:
public static Map componentConf(Object component) { Map<Object, Object> conf = new HashMap<Object, Object>(); ComponentCommon common = getComponentCommon(component); String jconf = common.get_json_conf(); if (jconf != null) { conf.putAll((Map<Object, Object>) JSONValue.parse(jconf)); } return conf; }
Source:
private static boolean isEmptyInputs(ComponentCommon common) { if (common == null) { START return true; END } else if (common.get_inputs() == null) { return true; } else { return common.get_inputs().isEmpty(); } } The original code would throw an NPE if this was null. In this case I can see it being a minor nit, but I still would prefer to keep the code the same if possible.

Target:
private static boolean isEmptyInputs(ComponentCommon common) { if (common.get_inputs() == null) { return true; } else { return common.get_inputs().isEmpty(); } }
Source:
private static Set<String> getStreamOutputFields(Map<String, StreamInfo> streams) { Set<String> outputFields = new HashSet<String>(); START if (streams != null) { END for (StreamInfo streamInfo : streams.values()) { outputFields.addAll(streamInfo.get_output_fields()); } } return outputFields; } In the original code if streams was null the return value was null.

Target:
private static Set<String> getStreamOutputFields(Map<String, StreamInfo> streams) { Set<String> outputFields = new HashSet<String>(); for (StreamInfo streamInfo : streams.values()) { outputFields.addAll(streamInfo.get_output_fields()); } return outputFields; }
Source:
public static ComponentCommon getComponentCommon(Object component) { START if (component == null) { END return null; } ComponentCommon common = null; if (component instanceof StateSpoutSpec) { common = ((StateSpoutSpec) component).get_common(); } else if (component instanceof SpoutSpec) { common = ((SpoutSpec) component).get_common(); } else if (component instanceof Bolt) { common = ((Bolt) component).get_common(); } return common; } This null check was not in the original code, but it is a minor nit here.

Target:
public static ComponentCommon getComponentCommon(Object component) { ComponentCommon common = null; if (component instanceof StateSpoutSpec) { common = ((StateSpoutSpec) component).get_common(); } else if (component instanceof SpoutSpec) { common = ((SpoutSpec) component).get_common(); } else if (component instanceof Bolt) { common = ((Bolt) component).get_common(); } return common; }
Source:
public static void addMetricStreams(StormTopology topology) { for (Object component : allComponents(topology).values()) { ComponentCommon common = getComponentCommon(component); START if (common != null) { END StreamInfo streamInfo = Thrift.outputFields(Arrays.asList("task-info", "data-points")); common.put_to_streams(Constants.METRICS_STREAM_ID, streamInfo); } } } This null check was not in the original code.

Target:
public static void addMetricStreams(StormTopology topology) { for (Object component : allComponents(topology).values()) { ComponentCommon common = getComponentCommon(component); StreamInfo streamInfo = Thrift.outputFields(Arrays.asList("task-info", "data-points")); common.put_to_streams(Constants.METRICS_STREAM_ID, streamInfo); } }
Source:
public static void addSystemStreams(StormTopology topology) { for (Object component : allComponents(topology).values()) { ComponentCommon common = getComponentCommon(component); START if (common != null) { END StreamInfo streamInfo = Thrift.outputFields(Arrays.asList("event")); common.put_to_streams(SYSTEM_STREAM_ID, streamInfo); } } } Again no null check in original code.

Target:
public static void addSystemStreams(StormTopology topology) { for (Object component : allComponents(topology).values()) { ComponentCommon common = getComponentCommon(component); StreamInfo streamInfo = Thrift.outputFields(Arrays.asList("event")); common.put_to_streams(SYSTEM_STREAM_ID, streamInfo); } }
Source:
public static int numStartExecutors(Object component) throws InvalidTopologyException { ComponentCommon common = getComponentCommon(component); if (common == null) { throw new InvalidTopologyException("unknown component type " + component.getClass().getName()); START } END int parallelismHintNum = Thrift.getParallelismHint(common); return parallelismHintNum; } This null check didn't exist in the original code.

Target:
public static int numStartExecutors(Object component) throws InvalidTopologyException { ComponentCommon common = getComponentCommon(component); return Thrift.getParallelismHint(common); }
Source:
public void setCurrAssignment(Map<Long, LocalAssignment> currAssignment) { this.currAssignment.clear(); START this.currAssignment.putAll(currAssignment); END } The clear with the add should probably be atomic. The race is very small, but I would rather see us have an AtomicReference<Map<Long, LocalAssignment>> instead of a ConcurrentHashMap.

Target:
public void setCurrAssignment(Map<Long, LocalAssignment> currAssignment) { this.currAssignment.set(currAssignment); }
Source:
public void setAssignmentVersions(Map<String, Map<String, Object>> assignmentVersions) { this.assignmentVersions.clear(); START this.assignmentVersions.putAll(assignmentVersions); END } Here too this operation should be atomic and I would rather see assignmentVersions be an `AtomicReference<Map<String, Map<String, Object>>>`

Target:
public void setAssignmentVersions(Map<String, Map<String, Object>> assignmentVersions) { this.assignmentVersions.set(assignmentVersions); }
Source:
START public static List<ACL> supervisorZkAcls() { END List<ACL> acls = new ArrayList<>(); acls.add(ZooDefs.Ids.CREATOR_ALL_ACL.get(0)); acls.add(new ACL((ZooDefs.Perms.READ ^ ZooDefs.Perms.CREATE), ZooDefs.Ids.ANYONE_ID_UNSAFE)); return acls; } In the original code this was a static final const value. It is probably not that big of a deal, but it would be nice to keep it the same if possible.

Target:
public final static List<ACL> supervisorZkAcls() { final List<ACL> acls = new ArrayList<>(); acls.add(ZooDefs.Ids.CREATOR_ALL_ACL.get(0)); acls.add(new ACL((ZooDefs.Perms.READ ^ ZooDefs.Perms.CREATE), ZooDefs.Ids.ANYONE_ID_UNSAFE)); return acls; }
Source:
private void registerWorkerNumGauge(String name, final Map conf) { START StormMetricsRegistry.registerGauge("supervisor:num-slots-used-gauge", new Callable<Integer>() { END @Override public Integer call() throws Exception { Collection<String> pids = SupervisorUtils.supervisorWorkerIds(conf); return pids.size(); } }); } name argument is not used.

Target:
private void registerWorkerNumGauge(String name, final Map conf) { StormMetricsRegistry.registerGauge(name, new Callable<Integer>() { @Override public Integer call() throws Exception { Collection<String> pids = SupervisorUtils.supervisorWorkerIds(conf); return pids.size(); } }); }
Source:
private static Server removeNonSslConnector(Server server) { for (Connector c : server.getConnectors()) { if (c != null && c instanceof SslSocketConnector) { START server.removeConnector(c); END } } return server; } connector should be removed if it is not a SslSocketConnector

Target:
private static Server removeNonSslConnector(Server server) { for (Connector c : server.getConnectors()) { if (c != null && !(c instanceof SslSocketConnector)) { server.removeConnector(c); } } return server; }
Source:
public void isZkAuthenticationConfiguredStormServerWithPropertyTest() { String key = "java.security.auth.login.config"; String oldValue = System.getProperty(key); try { System.setProperty("java.security.auth.login.config", "anything"); Assert.assertTrue(Utils.isZkAuthenticationConfiguredStormServer(emptyMockMap())); START } catch (Exception ignore) { END } finally { // reset property if (oldValue == null) { System.clearProperty(key); } else { System.setProperty(key, oldValue); } } } We don't want to ignore this exception. Let it fail the test if need be.

Target:
public void isZkAuthenticationConfiguredStormServerWithPropertyTest() { String key = "java.security.auth.login.config"; String oldValue = System.getProperty(key); try { System.setProperty("java.security.auth.login.config", "anything"); Assert.assertTrue(Utils.isZkAuthenticationConfiguredStormServer(emptyMockMap())); } finally { // reset property if (oldValue == null) { System.clearProperty(key); } else { System.setProperty(key, oldValue); } } }
Source:
public void isSimulatingReturnsTrueDuringSimulationTest() { Assert.assertFalse(Time.isSimulating()); Time.startSimulating(); Assert.assertTrue(Time.isSimulating()); START Time.stopSimulating(); END } Lets put this in a finally so we are sure we always stop simulating time. ``` Assert.assertFalse(Time.isSimulating()); Time.startSimulating(); try { Assert.assertTrue(Time.isSimulating()); } finally { Time.stopSimulating(); } ```

Target:
public void isSimulatingReturnsTrueDuringSimulationTest() { Assert.assertFalse(Time.isSimulating()); Time.startSimulating(); try { Assert.assertTrue(Time.isSimulating()); } finally { Time.stopSimulating(); } }
Source:
START public void checkActive() { END if (!this.task.isActive()) { throw new IllegalStateException("Timer is not active"); } } This should probably not be public. It was private in the original code.

Target:
private void checkActive() { if (!this.task.isActive()) { throw new IllegalStateException("Timer is not active"); } }
Source:
public static Object newInstance(String klass) { try { START LOG.info("Creating new instance for class {}", klass); END return newInstance(Class.forName(klass)); } catch (Exception e) { throw new RuntimeException(e); } } Please remove or make this debug

Target:
public static <T> T newInstance(String klass) { try { return newInstance((Class<T>)Class.forName(klass)); } catch (Exception e) { throw new RuntimeException(e); } }
Source:
public static String eventLogsFilename(String stormId, int port) { START return stormId + FILE_PATH_SEPARATOR + Integer.toString(port) + FILE_PATH_SEPARATOR + "events.log"; END } In both of these we don't need to do an Integer.toString. + will handle doing that for you.

Target:
public static String eventLogsFilename(String stormId, int port) { return stormId + FILE_PATH_SEPARATOR + port + FILE_PATH_SEPARATOR + "events.log"; }
Source:
START public static Map redactValue(Map<Object, String> m, Object key) { END if(m.containsKey(key)) { HashMap<Object, String> newMap = new HashMap<>(m); String value = newMap.get(key); String redacted = new String(new char[value.length()]).replace("\0", "#"); newMap.put(key, redacted); return newMap; } return m; } I think we want to preserve the type in the return, but the passed in type is actually wrong, the value does not have to be a String, it can be anything.

Target:
public static Map<Object, String> redactValue(Map<Object, String> m, Object key) { if(m.containsKey(key)) { HashMap<Object, String> newMap = new HashMap<>(m); String value = newMap.get(key); String redacted = new String(new char[value.length()]).replace("\0", "#"); newMap.put(key, redacted); return newMap; } return m; }
Source:
START public static String processPid() throws RuntimeException { END String name = ManagementFactory.getRuntimeMXBean().getName(); String[] split = name.split("@"); if (split.length != 2) { throw new RuntimeException("Got unexpected process name: " + name); } return split[0]; } RuntimeExceptions we don't need to declare that they are thrown.

Target:
public static String processPid() { String name = ManagementFactory.getRuntimeMXBean().getName(); String[] split = name.split("@"); if (split.length != 2) { throw new RuntimeException("Got unexpected process name: " + name); } return split[0]; }
Source:
START public static int forceKillProcess (String pid) { END return sendSignalToProcess(Long.parseLong(pid), SIGKILL); } When is this ever a string? We only call it from one place.

Target:
public static void forceKillProcess (String pid) throws IOException { sendSignalToProcess(Long.parseLong(pid), SIGKILL); }
Source:
public static String shellCmd (List<String> command) { START List<String> changedCommands = new LinkedList<>(); END for (String str: command) { if (str == null) { continue; } changedCommands.add("'" + str.replaceAll("'", "'\"'\"'") + "'"); } return StringUtils.join(changedCommands, " "); } Can we make this an ArrayList and pre allocate the same number of arguments as command? Very minor nit.

Target:
public static String shellCmd (List<String> command) { List<String> changedCommands = new ArrayList<>(command.size()); for (String str: command) { if (str == null) { continue; } changedCommands.add("'" + str.replaceAll("'", "'\"'\"'") + "'"); } return StringUtils.join(changedCommands, " "); }
Source:
public void shutdown() { try { log.info("window manager is being shutdown", windowManager); windowManager.shutdown(); START } finally { END log.info("window store is being shutdown", windowStore); windowStore.shutdown(); } } shouldn't we log an exception why the shutdown didn't happened properly.

Target:
public void shutdown() { try { LOG.info("window manager is being shutdown", windowManager); windowManager.shutdown(); } finally { LOG.info("window store is being shutdown", windowStore); windowStore.shutdown(); } }
Source:
public boolean equals(Object object) { boolean ret = false; if (object != null && object instanceof SubSystem) { START ret = (this.type.equals(((SubSystem)object).getType()) && this.hierarchyID == ((SubSystem)object).getHierarchyID()); END } return ret; } Cant enums be compared with `==` as well? Might be clearer to read. Also if you split this line up at the `&&`.

Target:
public boolean equals(Object object) { boolean ret = false; if (object != null && object instanceof SubSystem) { ret = ((this.type == ((SubSystem)object).getType()) && (this.hierarchyID == ((SubSystem)object).getHierarchyID())); } return ret; }
Source:
public synchronized static CgroupCenter getInstance() { if (instance == null) { instance = new CgroupCenter(); } START return CgroupUtils.enabled() ? instance : null; END } should this be the first line in function instead?

Target:
public synchronized static CgroupCenter getInstance() { if (CgroupUtils.enabled()) { instance = new CgroupCenter(); return instance; } return null; }
Source:
public boolean isMounted(Hierarchy hierarchy) { List<Hierarchy> hierarchies = this.getHierarchies(); START if (Utils.CheckDirExists(hierarchy.getDir())) { END for (Hierarchy h : hierarchies) { if (h.equals(hierarchy)) { return true; } } } return false; } Also, getHierarchies is going and reading from the file system. If getHierarchies is being called too often, you should consider caching.

Target:
public boolean isMounted(Hierarchy hierarchy) { if (Utils.CheckDirExists(hierarchy.getDir())) { List<Hierarchy> hierarchies = this.getHierarchies(); for (Hierarchy h : hierarchies) { if (h.equals(hierarchy)) { return true; } } } return false; }
Source:
START public static void deleteDir(String dir) { END File d = new File(dir); if (d.exists()) { if (d.isDirectory()) { if (!d.delete()) { throw new RuntimeException("Cannot delete dir " + dir); } } else { throw new RuntimeException("dir " + dir + " is not a directory!"); } } else { LOG.warn("dir does not exist!", dir); } } There are already utility methods for doing this.

Target:
public static void deleteDir(String dir) { File d = new File(dir); if (!d.exists()) { LOG.warn("dir does not exist!", dir); return; } if (!d.isDirectory()) { throw new RuntimeException("dir " + dir + " is not a directory!"); } if (!d.delete()) { throw new RuntimeException("Cannot delete dir " + dir); } }
Source:
START public static SubSystemType getSubSystem(String str) { END switch (str) { case "blkio": return blkio; case "cpu": return cpu; case "cpuacct": return cpuacct; case "devices": return devices; case "freezer": return freezer; case "memory": return memory; case "perf_event": return perf_event; case "net_cls": return net_cls; case "net_prio": return net_prio; default: return null; } } SubSystemType.valueOf(str) will do the exact same thing.

Target:
public static SubSystemType getSubSystem(String str) { try { return SubSystemType.valueOf(str); } catch (Exception e) { return null; } }
Source:
public void prepare(Map conf) { START LOG.info("no-op for zookeeper implementation"); END } Never mind I see in the code it does the same thing.

Target:
public void prepare(Map conf) { // no-op for zookeeper implementation }
Source:
public void removeFromLeaderLockQueue() { if (LeaderLatch.State.STARTED.equals(leaderLatch.get().getState())) { try { leaderLatch.get().close(); LOG.info("Removed from leader lock queue."); } catch (IOException e) { LOG.warn("failed to close leaderLatch!!"); START } END } else { LOG.info("leader latch is not started so no removeFromLeaderLockQueue needed."); } } Here too, lets not eat the exception. At least wrap it and rethrow it, but preferably update the Interface to allow us to throw it as is.

Target:
public void removeFromLeaderLockQueue() throws Exception { if (LeaderLatch.State.STARTED.equals(leaderLatch.get().getState())) { leaderLatch.get().close(); LOG.info("Removed from leader lock queue."); } else { LOG.info("leader latch is not started so no removeFromLeaderLockQueue needed."); } }
Source:
public static String absoluteHealthCheckDir(Map conf) { String stormHome = System.getProperty("storm.home"); String healthCheckDir = String.valueOf(conf.get(Config.STORM_HEALTH_CHECK_DIR)); START if (healthCheckDir.equals("null")) { END return (stormHome + FILE_SEPARATOR + "healthchecks"); } else { if (new File(healthCheckDir).isAbsolute()) { return healthCheckDir; } else { return (stormHome + FILE_SEPARATOR + healthCheckDir); } } } Why the string "null" here and an actual null in absoluteStormLocalDir?

Target:
public static String absoluteHealthCheckDir(Map conf) { String stormHome = System.getProperty("storm.home"); String healthCheckDir = (String) conf.get(Config.STORM_HEALTH_CHECK_DIR); if (healthCheckDir == null) { return (stormHome + FILE_SEPARATOR + "healthchecks"); } else { if (new File(healthCheckDir).isAbsolute()) { return healthCheckDir; } else { return (stormHome + FILE_SEPARATOR + healthCheckDir); } } }
Source:
public static String masterLocalDir(Map conf) throws IOException { String ret = String.valueOf(conf.get(Config.STORM_LOCAL_DIR)) + FILE_SEPARATOR + "nimbus"; try { FileUtils.forceMkdir(new File(ret)); } catch (IOException e) { LOG.error("Failed to create dir " + ret, e); START throw e; END } return ret; } Catching an exception to log it an rethrow it is not typically good practice. Could we remove the catch/rethrow? I also am confused why we do a mkdir here, but that is what the original code did so it is a good translation. Could we file a JIRA to clean this up later? Creating directories does not feel like something ConfigUtils should be doing.

Target:
public static String masterLocalDir(Map conf) throws IOException { String ret = String.valueOf(conf.get(Config.STORM_LOCAL_DIR)) + FILE_SEPARATOR + "nimbus"; FileUtils.forceMkdir(new File(ret)); return ret; }
Source:
public static String supervisorIsupervisorDir(Map conf) throws IOException { START return ((supervisorLocalDir(conf) + FILE_SEPARATOR + "isupervisor")); END } Double parens?

Target:
public static String supervisorIsupervisorDir(Map conf) throws IOException { return (supervisorLocalDir(conf) + FILE_SEPARATOR + "isupervisor"); }
Source:
public String getId() { START return this.getNodeId() + ":" + this.getPort(); END } minor: Do not need `this.` when calling methods.

Target:
public String getId() { return getNodeId() + ":" + getPort(); }
Source:
public double getMemoryUsedByWorker(WorkerSlot ws) { TopologyDetails topo = findTopologyUsingWorker(ws); if (topo == null) { return 0.0; } Collection<ExecutorDetails> execs = getExecutors(ws, _cluster); START LOG.info("getMemoryUsedByWorker executors: {}", execs); END double totalMemoryUsed = 0.0; for (ExecutorDetails exec : execs) { totalMemoryUsed += topo.getTotalMemReqTask(exec); } return totalMemoryUsed; } Is it useful for this to be INFO level? This could be a lot of text to log.

Target:
public double getMemoryUsedByWorker(WorkerSlot ws) { TopologyDetails topo = findTopologyUsingWorker(ws); if (topo == null) { return 0.0; } Collection<ExecutorDetails> execs = getExecutors(ws, _cluster); double totalMemoryUsed = 0.0; for (ExecutorDetails exec : execs) { totalMemoryUsed += topo.getTotalMemReqTask(exec); } return totalMemoryUsed; }
Source:
public void initState(KeyValueState<String, Long> state) { this.state = state; sum = state.get("sum", 0L); START System.out.println("initState with state [" + state + "] current sum [" + sum + "]"); END } Could we use slf4j here instead of `System.out`?

Target:
public void initState(KeyValueState<String, Long> state) { this.state = state; sum = state.get("sum", 0L); LOG.debug("initState with state [" + state + "] current sum [" + sum + "]"); }
Source:
private void setupCommitElapseTimer() { if(commitFrequencySec<=0) return; TimerTask timerTask = new TimerTask() { @Override public void run() { START commitTimeElapsed.set(false); END } }; commitTimer.schedule(timerTask, commitFrequencySec * 1000); } Should not commitTimeElapsed be set to true?

Target:
private void setupCommitElapseTimer() { if(commitFrequencySec<=0) { return; } TimerTask timerTask = new TimerTask() { @Override public void run() { commitTimeElapsed.set(true); } }; commitTimer.schedule(timerTask, commitFrequencySec * 1000); }
Source:
public String toString() { String ret = "Topologies:\n"; for (TopologyDetails td : this.getTopologies()) { ret += td.toString() + "\n"; } START return ret; END } Can we use a StringBuilder here instead? ``` StringBuilder ret = new StringBuilder(); ret.append("Topologies:\n"); for (TopologyDetails td : this.getTopologies()) { ret.append(td).append('\n'); } return ret.toString(); ``` (Also minor)

Target:
public String toString() { StringBuilder ret = new StringBuilder(); ret.append("Topologies:\n"); for (TopologyDetails td : this.getTopologies()) { ret.append(td.toString()).append("\n"); } return ret.toString(); }
Source:
public String toString() { String ret = ""; if(this.isSuccess()) { START ret += "Status: " + this.getStatus() + " message: " + this.getMessage() + " scheduling: " + this.getSchedulingResultMap().toString(); END } else { ret += "Status: " + this.getStatus() + " error message: " + this.getErrorMessage(); } return ret; } Don't need the .toString here at the end.

Target:
public String toString() { String ret = null; if(this.isSuccess()) { ret = "Status: " + this.getStatus() + " message: " + this.getMessage() + " scheduling: " + this.getSchedulingResultMap(); } else { ret = "Status: " + this.getStatus() + " error message: " + this.getErrorMessage(); } return ret; }
Source:
public void setAssignments(Map<String, SchedulerAssignment> newAssignments) { this.assignments = new HashMap<String, SchedulerAssignmentImpl>(); START for (Map.Entry<String, SchedulerAssignment> entry : newAssignments.entrySet()) { END this.assignments.put(entry.getKey(), new SchedulerAssignmentImpl(entry.getValue().getTopologyId(), entry.getValue().getExecutorToSlot())); } } Looking at this snippet of code, it is not clear what the entry's key contains. We just learn that it is a String, and is put in the assignments map as a key again. This is why I like the other style of looping: for (String representativeNameOfTheKeyString: newAssignments.keySet()) { String representativeNameOfTheValueString: newAssignments.get(representativeNameOfTheKeyString); //do something with key and value

Target:
public void setAssignments(Map<String, SchedulerAssignment> newAssignments) { this.assignments = new HashMap<String, SchedulerAssignmentImpl>(newAssignments.size()); for (Map.Entry<String, SchedulerAssignment> entry : newAssignments.entrySet()) { this.assignments.put(entry.getKey(), new SchedulerAssignmentImpl(entry.getValue().getTopologyId(), entry.getValue().getExecutorToSlot())); } }
Source:
public void setStatusMap(Map<String, String> statusMap) { this.status.putAll(statusMap); START } END This is named like a setter method, but it does not set the status map to what is in `statusMap`, it overrides any entries present in both. Suggest we rename it or change it to replace the current map.

Target:
public void setStatusMap(Map<String, String> statusMap) { this.status.clear(); this.status.putAll(statusMap); }
Source:
public Double getTotalCPU() { Double totalCPU = getTotalResource(Config.SUPERVISOR_CPU_CAPACITY); if (totalCPU == null) { throw new IllegalStateException("default value for supervisor.cpu.capacity is not set!"); } START return totalCPU; END } Do we want to throw exceptions here? It is unclear to me that this means the default was not set. If we need a default value, then we should add `@NotNull` to the validators for these configs. Then we will see the error earlier.

Target:
public Double getTotalCPU() { Double totalCPU = getTotalResource(Config.SUPERVISOR_CPU_CAPACITY); assert totalCPU != null; return totalCPU; }