XCTestCaseで作ったテストをxcodebuildで実行する方法


CIにテストを実行するためにXcodeの場合、`xcodebuild`を用いて処理するように設定していくと思います。
その際、端末別やOS別のテストを１つのクラス内に作っているケースがあるとして、テストケースを改修するコストよりもコマンド側で個別に呼び出す方針を取ったとき、どう設定すれば良いのかまとめておく。

## 目的

1. CIに端末別とOS別にテストを行うジョブを構築
2. 特定のXCTestCaseのテストを`xcodebuild`で実行させる **（←本題）**


## 環境

- Mac
    - macOS Mojave - version 10.14.6
- Xcode
    - version 11.2
- CI
    - Jenkins


## 準備

CI機となる共有PC（Mac miniなど）にXcodeをダウンロードし、１度Xcodeを起動させて初回の設定を済ませておく。
また今回は、CIに関することは割愛とする。

次のように`$ instruments -s devices`または`$ xcrun simctl list`からどのシミュレータを扱うか確認しておく。
もし扱いたい機種やOSが`devices`に無い場合は、Xcodeからデバイスを作成すること。

```
$ instruments -s devices
CoreData: annotation:  Failed to load optimized model at path '/Applications/Xcode.app/Contents/Applications/Instruments.app/Contents/Frameworks/InstrumentsPackaging.framework/Versions/A/Resources/XRPackageModel.momd/XRPackageModel 9.0.omo'
Known Devices:
gremitoのMacBook Pro [664E45D8-39FF-5D37-A42C-26FE51F10A9B]
Apple TV (13.2) [ADD78466-F544-4D33-9321-C81D4D0FC150] (Simulator)
Apple TV 4K (13.2) [C132945D-C5E9-40A6-915D-879E1B82BF2D] (Simulator)
Apple TV 4K (at 1080p) (13.2) [CF8827A4-3A31-48C9-9418-C31C8C9866EA] (Simulator)
Apple Watch Series 4 - 40mm (6.1) [2D3C75B6-07A6-4702-94C5-14A3CA12A818] (Simulator)
Apple Watch Series 4 - 44mm (6.1) [91BADC1D-4E20-44D3-A2A7-51E73693BC29] (Simulator)
iPad (7th generation) (13.2) [32489E3C-0348-477D-980A-11F0A0C27445] (Simulator)
iPad Air (3rd generation) (13.2) [53AA791E-5D2A-4989-8AAB-0564A1B10491] (Simulator)
iPad Pro (11-inch) (13.2) [7D8E1B5E-2286-475A-9406-17613AEC5998] (Simulator)
iPad Pro (12.9-inch) (3rd generation) (13.2) [602C1998-075C-4F53-8E9D-9D4233F26747] (Simulator)
iPad Pro (9.7-inch) (13.2) [C2B230EE-A01D-4E76-BA91-2214812D5047] (Simulator)
iPhone 11 (13.2) [E451C70D-4AA3-4C4B-A88C-E7A5EB1126A2] (Simulator)
iPhone 11 Pro (13.2) [B042F66E-2277-482D-97AE-9F2884EEDAE0] (Simulator)
iPhone 11 Pro (13.2) + Apple Watch Series 5 - 40mm (6.1) [32D82CF1-770B-4839-8F31-90C4CCCADCE7] (Simulator)
iPhone 11 Pro Max (13.2) [BF1E7BC6-A927-4C43-ADA7-8FEB2A658881] (Simulator)
iPhone 11 Pro Max (13.2) + Apple Watch Series 5 - 44mm (6.1) [26D86766-3435-42E9-B0D0-1F9F62DA7681] (Simulator)
iPhone 8 (13.2) [BDDE5F62-2721-41D5-ACA9-1EF6394627BC] (Simulator)
iPhone 8 Plus (13.2) [BD3AFEE1-975A-41DE-BE91-776D7E37F270] (Simulator)

$ xcrun simctl list
== Device Types ==
iPhone 4s (com.apple.CoreSimulator.SimDeviceType.iPhone-4s)
iPhone 5 (com.apple.CoreSimulator.SimDeviceType.iPhone-5)
iPhone 5s (com.apple.CoreSimulator.SimDeviceType.iPhone-5s)
iPhone 6 Plus (com.apple.CoreSimulator.SimDeviceType.iPhone-6-Plus)
iPhone 6 (com.apple.CoreSimulator.SimDeviceType.iPhone-6)
iPhone 6s (com.apple.CoreSimulator.SimDeviceType.iPhone-6s)
iPhone 6s Plus (com.apple.CoreSimulator.SimDeviceType.iPhone-6s-Plus)
iPhone SE (com.apple.CoreSimulator.SimDeviceType.iPhone-SE)
iPhone 7 (com.apple.CoreSimulator.SimDeviceType.iPhone-7)
iPhone 7 Plus (com.apple.CoreSimulator.SimDeviceType.iPhone-7-Plus)
iPhone 8 (com.apple.CoreSimulator.SimDeviceType.iPhone-8)
iPhone 8 Plus (com.apple.CoreSimulator.SimDeviceType.iPhone-8-Plus)
iPhone X (com.apple.CoreSimulator.SimDeviceType.iPhone-X)
iPhone Xs (com.apple.CoreSimulator.SimDeviceType.iPhone-XS)
iPhone Xs Max (com.apple.CoreSimulator.SimDeviceType.iPhone-XS-Max)
iPhone Xʀ (com.apple.CoreSimulator.SimDeviceType.iPhone-XR)
iPhone 11 (com.apple.CoreSimulator.SimDeviceType.iPhone-11)
iPhone 11 Pro (com.apple.CoreSimulator.SimDeviceType.iPhone-11-Pro)
iPhone 11 Pro Max (com.apple.CoreSimulator.SimDeviceType.iPhone-11-Pro-Max)
iPad 2 (com.apple.CoreSimulator.SimDeviceType.iPad-2)
iPad Retina (com.apple.CoreSimulator.SimDeviceType.iPad-Retina)
iPad Air (com.apple.CoreSimulator.SimDeviceType.iPad-Air)
iPad mini 2 (com.apple.CoreSimulator.SimDeviceType.iPad-mini-2)
iPad mini 3 (com.apple.CoreSimulator.SimDeviceType.iPad-mini-3)
iPad mini 4 (com.apple.CoreSimulator.SimDeviceType.iPad-mini-4)
iPad Air 2 (com.apple.CoreSimulator.SimDeviceType.iPad-Air-2)
iPad Pro (9.7-inch) (com.apple.CoreSimulator.SimDeviceType.iPad-Pro--9-7-inch-)
iPad Pro (12.9-inch) (com.apple.CoreSimulator.SimDeviceType.iPad-Pro)
iPad (5th generation) (com.apple.CoreSimulator.SimDeviceType.iPad--5th-generation-)
iPad Pro (12.9-inch) (2nd generation) (com.apple.CoreSimulator.SimDeviceType.iPad-Pro--12-9-inch---2nd-generation-)
iPad Pro (10.5-inch) (com.apple.CoreSimulator.SimDeviceType.iPad-Pro--10-5-inch-)
iPad (6th generation) (com.apple.CoreSimulator.SimDeviceType.iPad--6th-generation-)
iPad (7th generation) (com.apple.CoreSimulator.SimDeviceType.iPad--7th-generation-)
iPad Pro (11-inch) (com.apple.CoreSimulator.SimDeviceType.iPad-Pro--11-inch-)
iPad Pro (12.9-inch) (3rd generation) (com.apple.CoreSimulator.SimDeviceType.iPad-Pro--12-9-inch---3rd-generation-)
iPad mini (5th generation) (com.apple.CoreSimulator.SimDeviceType.iPad-mini--5th-generation-)
iPad Air (3rd generation) (com.apple.CoreSimulator.SimDeviceType.iPad-Air--3rd-generation-)
Apple TV (com.apple.CoreSimulator.SimDeviceType.Apple-TV-1080p)
Apple TV 4K (com.apple.CoreSimulator.SimDeviceType.Apple-TV-4K-4K)
Apple TV 4K (at 1080p) (com.apple.CoreSimulator.SimDeviceType.Apple-TV-4K-1080p)
Apple Watch - 38mm (com.apple.CoreSimulator.SimDeviceType.Apple-Watch-38mm)
Apple Watch - 42mm (com.apple.CoreSimulator.SimDeviceType.Apple-Watch-42mm)
Apple Watch Series 2 - 38mm (com.apple.CoreSimulator.SimDeviceType.Apple-Watch-Series-2-38mm)
Apple Watch Series 2 - 42mm (com.apple.CoreSimulator.SimDeviceType.Apple-Watch-Series-2-42mm)
Apple Watch Series 3 - 38mm (com.apple.CoreSimulator.SimDeviceType.Apple-Watch-Series-3-38mm)
Apple Watch Series 3 - 42mm (com.apple.CoreSimulator.SimDeviceType.Apple-Watch-Series-3-42mm)
Apple Watch Series 4 - 40mm (com.apple.CoreSimulator.SimDeviceType.Apple-Watch-Series-4-40mm)
Apple Watch Series 4 - 44mm (com.apple.CoreSimulator.SimDeviceType.Apple-Watch-Series-4-44mm)
Apple Watch Series 5 - 40mm (com.apple.CoreSimulator.SimDeviceType.Apple-Watch-Series-5-40mm)
Apple Watch Series 5 - 44mm (com.apple.CoreSimulator.SimDeviceType.Apple-Watch-Series-5-44mm)
== Runtimes ==
iOS 13.2 (13.2 - 17B84) - com.apple.CoreSimulator.SimRuntime.iOS-13-2 
tvOS 13.2 (13.2 - 17K81) - com.apple.CoreSimulator.SimRuntime.tvOS-13-2 
watchOS 6.1 (6.1 - 17S83) - com.apple.CoreSimulator.SimRuntime.watchOS-6-1 
== Devices ==
-- iOS 13.2 --
    iPhone 8 (BDDE5F62-2721-41D5-ACA9-1EF6394627BC) (Shutdown) 
    iPhone 8 Plus (BD3AFEE1-975A-41DE-BE91-776D7E37F270) (Shutdown) 
    iPhone 11 (E451C70D-4AA3-4C4B-A88C-E7A5EB1126A2) (Shutdown) 
    iPhone 11 Pro (B042F66E-2277-482D-97AE-9F2884EEDAE0) (Shutdown) 
    iPhone 11 Pro Max (BF1E7BC6-A927-4C43-ADA7-8FEB2A658881) (Shutdown) 
    iPad Pro (9.7-inch) (C2B230EE-A01D-4E76-BA91-2214812D5047) (Shutdown) 
    iPad (7th generation) (32489E3C-0348-477D-980A-11F0A0C27445) (Shutdown) 
    iPad Pro (11-inch) (7D8E1B5E-2286-475A-9406-17613AEC5998) (Shutdown) 
    iPad Pro (12.9-inch) (3rd generation) (602C1998-075C-4F53-8E9D-9D4233F26747) (Shutdown) 
    iPad Air (3rd generation) (53AA791E-5D2A-4989-8AAB-0564A1B10491) (Shutdown) 
-- tvOS 13.2 --
    Apple TV (ADD78466-F544-4D33-9321-C81D4D0FC150) (Shutdown) 
    Apple TV 4K (C132945D-C5E9-40A6-915D-879E1B82BF2D) (Shutdown) 
    Apple TV 4K (at 1080p) (CF8827A4-3A31-48C9-9418-C31C8C9866EA) (Shutdown) 
-- watchOS 6.1 --
    Apple Watch Series 4 - 40mm (2D3C75B6-07A6-4702-94C5-14A3CA12A818) (Shutdown) 
    Apple Watch Series 4 - 44mm (91BADC1D-4E20-44D3-A2A7-51E73693BC29) (Shutdown) 
    Apple Watch Series 5 - 40mm (32D82CF1-770B-4839-8F31-90C4CCCADCE7) (Shutdown) 
    Apple Watch Series 5 - 44mm (26D86766-3435-42E9-B0D0-1F9F62DA7681) (Shutdown) 
== Device Pairs ==
8266A1DA-60F1-443E-A9C3-FBED1C9E94F7 (active, disconnected)
    Watch: Apple Watch Series 5 - 40mm (32D82CF1-770B-4839-8F31-90C4CCCADCE7) (Shutdown)
    Phone: iPhone 11 Pro (B042F66E-2277-482D-97AE-9F2884EEDAE0) (Shutdown)
9EA20D9C-4745-4A3B-A89A-22237520ADD1 (active, disconnected)
    Watch: Apple Watch Series 5 - 44mm (26D86766-3435-42E9-B0D0-1F9F62DA7681) (Shutdown)
    Phone: iPhone 11 Pro Max (BF1E7BC6-A927-4C43-ADA7-8FEB2A658881) (Shutdown)

```


## 設定

以下のようにCIに設定することで特定のテストが処理させるようになる。
念のためCIで動作して確認する前にターミナルで問題なく動作するか確認しておくこと。

```
# iPhone 11(iOS 13.2)のテスト
xcodebuild \
 -workspace iOSAppProject/iOSAppProject.xcodeproj/project.xcworkspace \
 -scheme iOSAppProject \
 -destination OS=13.2,name="iPhone 11" \
 -only-testing:iOSAppProjectTests/iPhoneTests/iOS12OrBelowTest \
 -configuration Debug build test

# iPhone X(iOS 12.1)のテスト
xcodebuild \
 -workspace iOSAppProject/iOSAppProject.xcodeproj/project.xcworkspace \
 -scheme iOSAppProject \
 -destination OS=12.1,name="iPhone 10" \
 -only-testing:iOSAppProjectTests/iPhoneTests/iOS13OrHigherTest \
 -configuration Debug build test

# iPad Pro (12.9-inch)(iOS 13.2)のテスト
xcodebuild \
 -workspace iOSAppProject/iOSAppProject.xcodeproj/project.xcworkspace \
 -scheme iOSAppProject \
 -destination OS=13.2,name="iPad Pro (12.9-inch)" \
 -only-testing:iOSAppProjectTests/iPadTests/iPadProTest \
 -configuration Debug build test

# iPad mini 2(iOS 12.4.3)のテスト
xcodebuild \
 -workspace iOSAppProject/iOSAppProject.xcodeproj/project.xcworkspace \
 -scheme iOSAppProject \
 -destination OS=12.4.3,name="iPad mini 2" \
 -only-testing:iOSAppProjectTests/iPadTests/iPadMiniTest \
 -configuration Debug build test
```

上記のように特定のオプションの設定を変えるだけなので、シェル対応で変数化すると良いでしょう。
端末やOSはシミュレータの確認で表示された内容を指定できます。

## 参考記事

- [How to Run Individual XCTestCases](https://dev.to/higuri/how-to-run-individual-xctestcases-495f)
- [Technical Note TN2339: Building from the Command Line with Xcode FAQ](https://developer.apple.com/library/archive/technotes/tn2339/_index.html)
- [simctl: Control iOS Simulators from Command Line](https://medium.com/xcblog/simctl-control-ios-simulators-from-command-line-78b9006a20dc)
- [xcodebuild コマンドで iOS アプリの自動ビルド](https://blog.asial.co.jp/953)
- [Build and run an app on simulator using xcodebuild](https://stackoverflow.com/questions/34003723/build-and-run-an-app-on-simulator-using-xcodebuild)
- [Xcode 6 - Launch simulator from command line](https://stackoverflow.com/questions/26031601/xcode-6-launch-simulator-from-command-line)
- [simctlコマンドを使ってみる](https://qiita.com/tamaki/items/02eb43253193b950b08f)