splunk/splunk-sdk-java — pull request #60 by pax95 (2013-12-17 20:03:53+00:00)
Title: fixed comparison bug, and made Receiver close InputStream when done.

This pull request replaces https://github.com/splunk/splunk-sdk-java/pull/59.

The inputstream on the http response is not closed leaving a lot to open files on Linux in state CLOSE_WAIT.

Dump before patch -
mac:splunk prebenasmussen$ netstat -a
Active Internet connections (including servers)
Proto Recv-Q Send-Q Local Address Foreign Address (state)
tcp4 0 0 localhost.8089 localhost.60058 ESTABLISHED
tcp4 714 0 localhost.60058 localhost.8089 ESTABLISHED
tcp4 0 0 localhost.8089 localhost.60057 ESTABLISHED
tcp4 714 0 localhost.60057 localhost.8089 ESTABLISHED
tcp4 0 0 localhost.8089 localhost.60056 ESTABLISHED
tcp4 714 0 localhost.60056 localhost.8089 ESTABLISHED
tcp4 0 0 localhost.8089 localhost.60055 ESTABLISHED
tcp4 714 0 localhost.60055 localhost.8089 ESTABLISHED
tcp4 0 0 localhost.8089 localhost.60054 ESTABLISHED
tcp4 714 0 localhost.60054 localhost.8089 ESTABLISHED
tcp4 0 0 localhost.8089 localhost.60049 ESTABLISHED
tcp4 714 0 localhost.60049 localhost.8089 ESTABLISHED
tcp4 0 0 localhost.8089 localhost.60039 ESTABLISHED
tcp4 714 0 localhost.60039 localhost.8089 ESTABLISHED
tcp4 0 0 localhost.8089 localhost.60034 ESTABLISHED
tcp4 714 0 localhost.60034 localhost.8089 ESTABLISHED
tcp4 0 0 localhost.8089 localhost.60028 ESTABLISHED
tcp4 714 0 localhost.60028 localhost.8089 ESTABLISHED
tcp4 0 0 localhost.8089 localhost.60020 ESTABLISHED
tcp4 714 0 localhost.60020 localhost.8089 ESTABLISHED
tcp4 0 0 localhost.8089 localhost.60015 ESTABLISHED
tcp4 714 0 localhost.60015 localhost.8089 ESTABLISHED
tcp4 0 0 localhost.8089 localhost.60010 ESTABLISHED
tcp4 714 0 localhost.60010 localhost.8089 ESTABLISHED
tcp4 0 0 localhost.8089 localhost.60000 ESTABLISHED
tcp4 714 0 localhost.60000 localhost.8089 ESTABLISHED
tcp4 0 0 localhost.8089 localhost.59995 ESTABLISHED
tcp4 714 0 localhost.59995 localhost.8089 ESTABLISHED
tcp4 0 0 localhost.8089 localhost.59991 ESTABLISHED
tcp4 714 0 localhost.59991 localhost.8089 ESTABLISHED

With patch applied ->
mac:~ prebenasmussen$ netstat -a
Active Internet connections (including servers)
Proto Recv-Q Send-Q Local Address Foreign Address (state)
tcp4 0 0 localhost.8089 localhost.51001 ESTABLISHED
tcp4 0 0 localhost.51001 localhost.8089 ESTABLISHED

I have only run part of the unittest's for verification of the patch !!

I have left out the changes to HttpService.send since the changes here are not relevant for the fix, but I would recommend it to refactored a bit to be more readable. See example

Since the method returns the InputStream in the ResponseMessage there might be other places in the codebase where a close on the stream might not have been applied. This will show up as problems when Linux reports "Too Many Open files". Would be nice if you looked into this at the same time to ensure that InputSteams are closed correctly.

Refacor example :
public ResponseMessage send(String path, RequestMessage request) {
try {
// Construct a full URL to the resource
URL url = getUrl(path);

// Create and initialize the connection object
HttpURLConnection cn = (HttpURLConnection) url.openConnection();
if (cn instanceof HttpsURLConnection) {
((HttpsURLConnection) cn).setSSLSocketFactory(SSL_SOCKET_FACTORY);
((HttpsURLConnection) cn).setHostnameVerifier(HOSTNAME_VERIFIER);
}
cn.setUseCaches(false);
cn.setAllowUserInteraction(false);

// Set the request method
String method = request.getMethod();
cn.setRequestMethod(method);

// Add headers from request message
Map header = request.getHeader();
for (Entry entry : header.entrySet()) {
cn.setRequestProperty(entry.getKey(), entry.getValue());
}

// Add default headers that were absent from the request message
for (Entry entry : defaultHeader.entrySet()) {
String key = entry.getKey();
if (header.containsKey(key)) {
continue;
}
cn.setRequestProperty(key, entry.getValue());
}
cn.setDoOutput(true);
cn.connect();
// Write out request content, if any
Object content = request.getContent();
if (content != null) {
OutputStreamWriter writer = new OutputStreamWriter(cn.getOutputStream(), "UTF8");
writer.write((String) content);
writer.close();
}
if (VERBOSE_REQUESTS) {
System.out.format("%s %s => ", method, url.toString());
}

int status = cn.getResponseCode();
InputStream input = null;
input = status >= 400 ? cn.getErrorStream() : cn.getInputStream();

ResponseMessage response = new ResponseMessage(status, input);

if (VERBOSE_REQUESTS) {
System.out.format("%d\n", status);
if (method.equals("POST")) {
System.out.println(" " + request.getContent());
}
}
if (status >= 400) {
throw HttpException.create(response);
}
return response;
} catch (IOException e) {
throw new RuntimeException(e.getMessage(), e);
}
}