Selenium Gridを拡張する（Hub側）

# はじめに

色々あってSelenium GridにRESTする前に、Hubに接続されているNodeの状態をプログラムで見ようとしました。
Web画面のコンソールでは見れるのですが、できればJsonで返してほしいがそのような機能はない・・・

本家の[Customizing the Grid](https://www.seleniumhq.org/docs/07_selenium_grid.jsp#customizing-the-grid)を参考にしようにもコードも見つからないし、起動の方法も古いようでした:innocent:
最新の情報がどこかにある？:thinking:

今回の用途ではHub側のみですが、拡張方法を忘れないよう記録します。


# 利用したソフトウェア

|ソフト|バージョン|用途|
|:--|:--|:--|
|java|1.8.0_191||
|[selenium-server-standalone.jar](https://www.seleniumhq.org/)|3.14.0||


# 環境構築

Windowsで開発環境を構築しています。
Mavenで設定するのみです。
selenium-serverとレスポンスJsonを生成するためにgsonを使用します。
![aWS050498.JPG](https://qiita-image-store.s3.amazonaws.com/0/187061/7a91fe5d-763f-9271-7d91-edaff9e27873.jpeg)

###### Maven設定

```xml:pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>selenium-grid-extend</groupId>
  <artifactId>selenium-grid-extend</artifactId>
  <version>0.0.1</version>
  <build>
    <sourceDirectory>src</sourceDirectory>
    <testSourceDirectory>src</testSourceDirectory>
    <resources>
      <resource>
        <directory>resource</directory>
      </resource>
    </resources>
    <testResources>
      <testResource>
        <directory>resource</directory>
      </testResource>
    </testResources>
  </build>
  <dependencies>
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-server</artifactId>
      <version>3.141.5</version>
    </dependency>
    <dependency>
    	<groupId>com.google.code.gson</groupId>
    	<artifactId>gson</artifactId>
    	<version>2.8.5</version>
    </dependency>
  </dependencies>
  <properties>
    <java.version>1.8</java.version>
    <file.encoding>UTF-8</file.encoding>
    <project.build.sourceEncoding>${file.encoding}</project.build.sourceEncoding>
    <project.reporting.outputEncoding>${file.encoding}</project.reporting.outputEncoding>
    <maven.compiler.encoding>${file.encoding}</maven.compiler.encoding>
    <maven.compiler.source>${java.version}</maven.compiler.source>
    <maven.compiler.target>${java.version}</maven.compiler.target>
  </properties>
</project>
```


# プログラム

[Customizing the Grid](https://www.seleniumhq.org/docs/07_selenium_grid.jsp#customizing-the-grid)によるとHubの内部にアクセスする場合はRegistryBasedServletを継承、アクセスしなければHttpServletを継承すれば良いみたいです。

今回はHubに接続されているNodeの状態を見たいので、RegistryBasedServletを継承します。
試しに作成したのは2つのプログラム。

###### 全てのNodeをそのままJsonで返す拡張クラス

```java:AllNodes.java
package selenium.extend.hub.servlet;

import java.io.IOException;
import java.util.Iterator;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.openqa.grid.common.exception.GridException;
import org.openqa.grid.internal.GridRegistry;
import org.openqa.grid.internal.ProxySet;
import org.openqa.grid.internal.RemoteProxy;
import org.openqa.grid.web.servlet.RegistryBasedServlet;

import com.google.gson.Gson;
import com.google.gson.JsonArray;
import com.google.gson.JsonObject;
import com.google.gson.JsonSyntaxException;

public class AllNodes extends RegistryBasedServlet {

	public AllNodes() {
		this(null);
	}

	public AllNodes(GridRegistry registry) {
		super(registry);
	}

	protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
		process(request, response);
	}

	protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
		process(request, response);
	}

	protected void process(HttpServletRequest request, HttpServletResponse response) throws IOException {
		response.setContentType("application/json");
		response.setCharacterEncoding("UTF-8");
		response.setStatus(200);

		try {
			JsonObject res = getResponse(request);
			response.getWriter().print(res);
			response.getWriter().close();
		} catch (JsonSyntaxException e) {
			throw new GridException(e.getMessage());
		}
	}

	private JsonObject getResponse(HttpServletRequest request) {
		JsonObject json = new JsonObject();
		ProxySet proxies = super.getRegistry().getAllProxies();
		json.add("Nodes", getNodes(proxies));
		return json;
	}

	private JsonArray getNodes(ProxySet proxies) {
		JsonArray array = new JsonArray();
		Iterator<RemoteProxy> itr = proxies.iterator();
		Gson gson = new Gson();

		while (itr.hasNext()) {
			RemoteProxy proxy = itr.next();
			JsonObject proxyJson = new JsonObject();
			proxyJson.add("Node", gson.toJsonTree(proxy.getOriginalRegistrationRequest()));
			array.add(proxyJson);
		}
		return array;
	}

}
```

###### 全てのNodeを未使用/使用中にわけてJsonで返す拡張クラス

```java:AllNodesState.java
package selenium.extend.hub.servlet;

import java.io.IOException;
import java.util.Iterator;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.openqa.grid.common.exception.GridException;
import org.openqa.grid.internal.GridRegistry;
import org.openqa.grid.internal.ProxySet;
import org.openqa.grid.internal.RemoteProxy;
import org.openqa.grid.web.servlet.RegistryBasedServlet;

import com.google.gson.Gson;
import com.google.gson.JsonArray;
import com.google.gson.JsonObject;
import com.google.gson.JsonSyntaxException;

public class AllNodesState extends RegistryBasedServlet {

	public AllNodesState() {
		this(null);
	}

	public AllNodesState(GridRegistry registry) {
		super(registry);
	}

	protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
		process(request, response);
	}

	protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
		process(request, response);
	}

	protected void process(HttpServletRequest request, HttpServletResponse response) throws IOException {
		response.setContentType("application/json");
		response.setCharacterEncoding("UTF-8");
		response.setStatus(200);

		try {
			JsonObject res = getResponse(request);
			response.getWriter().print(res);
			response.getWriter().close();
		} catch (JsonSyntaxException e) {
			throw new GridException(e.getMessage());
		}
	}

	private JsonObject getResponse(HttpServletRequest request) {
		JsonObject json = new JsonObject();
		ProxySet proxies = super.getRegistry().getAllProxies();
		json.add("Nodes", getNodes(proxies));
		return json;
	}

	private JsonArray getNodes(ProxySet proxies) {
		JsonArray array = new JsonArray();
		Iterator<RemoteProxy> itr = proxies.iterator();
		Gson gson = new Gson();

		JsonArray freeProxies = new JsonArray();
		JsonArray busyProxies = new JsonArray();

		while (itr.hasNext()) {
			RemoteProxy proxy = itr.next();
			JsonObject proxyJson = new JsonObject();
			proxyJson.add("Node", gson.toJsonTree(proxy.getOriginalRegistrationRequest()));

			if (!proxy.isBusy()) {
				freeProxies.add(proxyJson);
			} else {
				busyProxies.add(proxyJson);
			}
		}

		JsonObject freeJson = new JsonObject();
		freeJson.add("FreeNodes", freeProxies);
		array.add(freeJson);

		JsonObject busyJson = new JsonObject();
		busyJson.add("BusyNodes", busyProxies);
		array.add(busyJson);

		return array;
	}

}
```


# Jarにする

作成したクラスだけぶっこんでます。
今回はextend.jarという名前で作成。
![aWS050500.JPG](https://qiita-image-store.s3.amazonaws.com/0/187061/de261049-fc99-4d3c-7790-352a88fe2c39.jpeg)


# 動作確認

###### ディレクトリ構成

依存関係にあるjarを全てlibに配置しておきます。

```console
C:\GRID
├─start-hub-extend.bat
│
└─lib
     ├ extend.jar
     ├ gson-2.8.5.jar
     └ selenium-server-standalone-3.14.0.jar
```

###### Hub起動コマンド

あとはクラスパスなどを指定して起動するだけです。
-servletsで追加するクラスを指定するのですが、今回は2つあるのでカンマで区切っています。

```bat:start-hub-extend.bat
java -cp lib/* org.openqa.grid.selenium.GridLauncherV3 -role hub -servlets "selenium.extend.hub.servlet.AllNodes,selenium.extend.hub.servlet.AllNodesState"
```

###### Selenium Grid のログ

バッチ実行してみると、3行目4行目に普段はでていない『[Hub.\<init\>]』の文字がでています。
簡単にいうと『/grid/admin/AllNodes/』とかでアクセスしてねってことですね。

```console
14:43:51.090 INFO [GridLauncherV3.launch] - Selenium build info: version: '3.14.0', revision: 'aacccce0'
14:43:51.106 INFO [GridLauncherV3$2.launch] - Launching Selenium Grid hub on port 4444
14:43:51.137 INFO [Hub.<init>] - binding selenium.extend.hub.servlet.AllNodes to /grid/admin/AllNodes/*
14:43:51.137 INFO [Hub.<init>] - binding selenium.extend.hub.servlet.AllNodesState to /grid/admin/AllNodesState/*
2018-11-09 14:43:51.480:INFO::main: Logging initialized @780ms to org.seleniumhq.jetty9.util.log.StdErrLog
14:43:51.777 INFO [Hub.start] - Selenium Grid hub is up and running
14:43:51.777 INFO [Hub.start] - Nodes should register to http://XXX.XXX.XXX.XXX:4444/grid/register/
14:43:51.777 INFO [Hub.start] - Clients should connect to http://XXX.XXX.XXX.XXX:4444/wd/hub
```

###### アクセスしてみる
※Jsonは記事貼り付け前に整形しています

/grid/console は以下のような状態
![aWS050502.JPG](https://qiita-image-store.s3.amazonaws.com/0/187061/eb6f5650-5616-4410-d513-9d99e323419b.jpeg)

/grid/admin/AllNodes だと・・・

```json
{
  "Nodes": [
    {
      "Node": {
        "configuration": {
          "remoteHost": "http://XXX.XXX.XXX.XXX:37803",
          "id": "http://XXX.XXX.XXX.XXX:37803",
          "capabilities": [
            {
              "caps": {
                "browserName": "chrome",
                "maxInstances": 1,
                "platform": "WINDOWS",
                "platformName": "WINDOWS",
                "seleniumProtocol": "WebDriver",
                "server:CONFIG_UUID": "8405f2b4-e375-4251-a2e8-50e432eb24b8"
              }
            }
          ],
          "downPollingLimit": 2,
          "hub": "http://XXX.XXX.XXX.XXX:4444/grid/register",
          "nodePolling": 5000,
          "nodeStatusCheckTimeout": 5000,
          "proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
          "register": true,
          "registerCycle": 5000,
          "unregisterIfStillDownAfter": 60000,
          "enablePlatformVerification": true,
          "custom": {},
          "maxSession": 5,
          "servlets": [],
          "withoutServlets": [],
          "avoidProxy": false,
          "browserSideLog": false,
          "captureLogsOnQuit": false,
          "browserTimeout": 0,
          "debug": false,
          "host": "172.16.11.134",
          "port": 37803,
          "role": "node",
          "timeout": 1800
        }
      }
    },
    {
      "Node": {
        "configuration": {
          "remoteHost": "http://XXX.XXX.XXX.XXX:3753",
          "id": "http://XXX.XXX.XXX.XXX:3753",
          "capabilities": [
            {
              "caps": {
                "browserName": "firefox",
                "maxInstances": 2,
                "platform": "WINDOWS",
                "platformName": "WINDOWS",
                "seleniumProtocol": "WebDriver",
                "server:CONFIG_UUID": "61acf94e-1db8-4274-9587-1e32a0ebb378"
              }
            },
            {
              "caps": {
                "browserName": "internet explorer",
                "maxInstances": 1,
                "platform": "WINDOWS",
                "platformName": "WINDOWS",
                "seleniumProtocol": "WebDriver",
                "server:CONFIG_UUID": "4adb7e6b-784e-46c3-896f-a2fcb7413bfb"
              }
            }
          ],
          "downPollingLimit": 2,
          "hub": "http://XXX.XXX.XXX.XXX:4444/grid/register",
          "nodePolling": 5000,
          "nodeStatusCheckTimeout": 5000,
          "proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
          "register": true,
          "registerCycle": 5000,
          "unregisterIfStillDownAfter": 60000,
          "enablePlatformVerification": true,
          "custom": {},
          "maxSession": 5,
          "servlets": [],
          "withoutServlets": [],
          "avoidProxy": false,
          "browserSideLog": false,
          "captureLogsOnQuit": false,
          "browserTimeout": 0,
          "debug": false,
          "host": "172.16.8.84",
          "port": 3753,
          "role": "node",
          "timeout": 1800
        }
      }
    }
  ]
}
```

/grid/admin/AllNodesState だと・・・

```json
{
  "Nodes": [
    {
      "FreeNodes": [
        {
          "Node": {
            "configuration": {
              "remoteHost": "http://XXX.XXX.XXX.XXX:37803",
              "id": "http://XXX.XXX.XXX.XXX:37803",
              "capabilities": [
                {
                  "caps": {
                    "browserName": "chrome",
                    "maxInstances": 1,
                    "platform": "WINDOWS",
                    "platformName": "WINDOWS",
                    "seleniumProtocol": "WebDriver",
                    "server:CONFIG_UUID": "8405f2b4-e375-4251-a2e8-50e432eb24b8"
                  }
                }
              ],
              "downPollingLimit": 2,
              "hub": "http://XXX.XXX.XXX.XXX:4444/grid/register",
              "nodePolling": 5000,
              "nodeStatusCheckTimeout": 5000,
              "proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
              "register": true,
              "registerCycle": 5000,
              "unregisterIfStillDownAfter": 60000,
              "enablePlatformVerification": true,
              "custom": {},
              "maxSession": 5,
              "servlets": [],
              "withoutServlets": [],
              "avoidProxy": false,
              "browserSideLog": false,
              "captureLogsOnQuit": false,
              "browserTimeout": 0,
              "debug": false,
              "host": "172.16.11.134",
              "port": 37803,
              "role": "node",
              "timeout": 1800
            }
          }
        },
        {
          "Node": {
            "configuration": {
              "remoteHost": "http://XXX.XXX.XXX.XXX:3753",
              "id": "http://XXX.XXX.XXX.XXX:3753",
              "capabilities": [
                {
                  "caps": {
                    "browserName": "firefox",
                    "maxInstances": 2,
                    "platform": "WINDOWS",
                    "platformName": "WINDOWS",
                    "seleniumProtocol": "WebDriver",
                    "server:CONFIG_UUID": "61acf94e-1db8-4274-9587-1e32a0ebb378"
                  }
                },
                {
                  "caps": {
                    "browserName": "internet explorer",
                    "maxInstances": 1,
                    "platform": "WINDOWS",
                    "platformName": "WINDOWS",
                    "seleniumProtocol": "WebDriver",
                    "server:CONFIG_UUID": "4adb7e6b-784e-46c3-896f-a2fcb7413bfb"
                  }
                }
              ],
              "downPollingLimit": 2,
              "hub": "http://XXX.XXX.XXX.XXX:4444/grid/register",
              "nodePolling": 5000,
              "nodeStatusCheckTimeout": 5000,
              "proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
              "register": true,
              "registerCycle": 5000,
              "unregisterIfStillDownAfter": 60000,
              "enablePlatformVerification": true,
              "custom": {},
              "maxSession": 5,
              "servlets": [],
              "withoutServlets": [],
              "avoidProxy": false,
              "browserSideLog": false,
              "captureLogsOnQuit": false,
              "browserTimeout": 0,
              "debug": false,
              "host": "172.16.8.84",
              "port": 3753,
              "role": "node",
              "timeout": 1800
            }
          }
        }
      ]
    },
    {
      "BusyNodes": []
    }
  ]
}
```

###### busy状態について

予想外だったんですが、いくつかブラウザを動かせる設定でも、1つでもブラウザが動いていればそのNodeはbusyになるようです。
今回使用しているAPIをほとんど調べていないので、そもそも使い方間違えているのかもしれません:thinking:

例えばFirefoxが1つだけ動いている状態でも・・・
![aWS050503.JPG](https://qiita-image-store.s3.amazonaws.com/0/187061/1a834cdb-9fe7-b985-cfcd-a7ff03ccd5f5.jpeg)


/grid/admin/AllNodesState を見るとBusyNodesに判定されています。

```json
{
  "Nodes": [
    {
      "FreeNodes": [
        {
          "Node": {
            "configuration": {
              "remoteHost": "http://172.16.11.134:37803",
              "id": "http://172.16.11.134:37803",
              "capabilities": [
                {
                  "caps": {
                    "browserName": "chrome",
                    "maxInstances": 1,
                    "platform": "WINDOWS",
                    "platformName": "WINDOWS",
                    "seleniumProtocol": "WebDriver",
                    "server:CONFIG_UUID": "8405f2b4-e375-4251-a2e8-50e432eb24b8"
                  }
                }
              ],
              "downPollingLimit": 2,
              "hub": "http://goto-main:4444/grid/register",
              "nodePolling": 5000,
              "nodeStatusCheckTimeout": 5000,
              "proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
              "register": true,
              "registerCycle": 5000,
              "unregisterIfStillDownAfter": 60000,
              "enablePlatformVerification": true,
              "custom": {},
              "maxSession": 5,
              "servlets": [],
              "withoutServlets": [],
              "avoidProxy": false,
              "browserSideLog": false,
              "captureLogsOnQuit": false,
              "browserTimeout": 0,
              "debug": false,
              "host": "172.16.11.134",
              "port": 37803,
              "role": "node",
              "timeout": 1800
            }
          }
        }
      ]
    },
    {
      "BusyNodes": [
        {
          "Node": {
            "configuration": {
              "remoteHost": "http://172.16.8.84:3753",
              "id": "http://172.16.8.84:3753",
              "capabilities": [
                {
                  "caps": {
                    "browserName": "firefox",
                    "maxInstances": 2,
                    "platform": "WINDOWS",
                    "platformName": "WINDOWS",
                    "seleniumProtocol": "WebDriver",
                    "server:CONFIG_UUID": "61acf94e-1db8-4274-9587-1e32a0ebb378"
                  }
                },
                {
                  "caps": {
                    "browserName": "internet explorer",
                    "maxInstances": 1,
                    "platform": "WINDOWS",
                    "platformName": "WINDOWS",
                    "seleniumProtocol": "WebDriver",
                    "server:CONFIG_UUID": "4adb7e6b-784e-46c3-896f-a2fcb7413bfb"
                  }
                }
              ],
              "downPollingLimit": 2,
              "hub": "http://localhost:4444/grid/register",
              "nodePolling": 5000,
              "nodeStatusCheckTimeout": 5000,
              "proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
              "register": true,
              "registerCycle": 5000,
              "unregisterIfStillDownAfter": 60000,
              "enablePlatformVerification": true,
              "custom": {},
              "maxSession": 5,
              "servlets": [],
              "withoutServlets": [],
              "avoidProxy": false,
              "browserSideLog": false,
              "captureLogsOnQuit": false,
              "browserTimeout": 0,
              "debug": false,
              "host": "172.16.8.84",
              "port": 3753,
              "role": "node",
              "timeout": 1800
            }
          }
        }
      ]
    }
  ]
}
```