Yocto Project 2.2 Release Test Plan

= Reversion history =
Version	Modifier 	Comments
1.0 	Jose Perez Carranza	First draft
= Introduction =
This article is the overall test plan for version 2.2 of the , for all the generic information this document will be pointing QA Master Test Plan
= Objectives =
QA_Master_Test_Plan#Objectives_and_Tasks
= Test Strategy =
Is followed the general strategy defined on QA_Master_Test_Plan#Testing_Strategy and below is showed specific information applicable only for 2.2 Release
== Owned features ==
As QA team we are also in charge for some features to implemetn/verify
*To Verify 2.2_qa_owned_features_to_verify
*To Implement 2.2_qa_owned_features
* Assigned to QA Team 2.2_qa_assigned_bugs
== Bugs to Verify ==
*Bugs that need verification form QA team 2.2_qa_owned_bugs
*Old Resolved bugs to verify Old_resolved_bugs_and_features
= Test Items =
Code of Colors
Active
Tracked on Testpoia
Active
Non Tracked on Testopia
In Design
Will be tracked on Tesopia
Inactive
COMPONENT 	TEMPLATE
ADT 	TRTEMP_2.2_MANUAL_ADT
ADT 	TRTEMP_2.2_AUTO_ADT
BSP 	TRTEMP_2.2_MANUAL_BSP/QEMU_core-image-sato-sdk_ANYQEMU
BSP 	TRTEMP_2.2_AUTO_BSP/QEMU_core-image-sato-sdk_ANYQEMU
BSP 	TRTEMP_2.2_AUTO_BSP/QEMU_ore-image-lsb-sdk_ANYBSP
BSP 	TRTEMP_2.2_AUTO_BSP/QEMU_core-image-sato-wic_ANYBSP
BSP 	TRTEMP_2.2_MANUAL_BSP/QEMU_core2-32_MinnowMax
BSP 	TRTEMP_2.2_MANUAL_BSP/QEMU_corei7-64_MinnowMax
BSP 	TRTEMP_2.2_MANUAL_BSP/QEMU_corei7-64_NUC
BSP 	TRTEMP_2.2_MANUAL_BSP/QEMU_GenericX86-64_NUC
BSP 	TRTEMP_2.2_AUTO_BSP/QEMU_core-image-sato-sdk_ANYBSP
BSP 	TRTEMP_2.2_MANUAL_BSP/QEMU_GenericX86_MMAX32
BSP 	TRTEMP_2.2_AUTO_BSP/QEMU_core-image-sato-sdk_intel-quark
CROPS 	TRTEMP_2.2_AUTO_CROPS
CROPS 	TRTEMP_2.2_MANUAL_CROPS
Eclipse-Plugin 	TRTEMP_2.2._MANUAL_Eclipse-Plugin
Meta-Yocto 	TRTEMP_2.2_MANUAL_Meta-yocto
Toaster 	TRTEMP_2.2_AUTO_TOASTER_UI
Toaster 	TRTEMP_2.2_MANUAL_TOASTER_UI-managed-mode
eSDK 	TRTEMP_2.2_AUTO_eSDK
eSDK 	TRTEMP_2.2_MANUAL_eSDK
OE-CORE / BitBake 	TRTEMP_2.2_AUTO_OE-Core
OE-CORE / BitBake 	TRTEMP_2.2_MANUAL_BitBake
Automated Build Test / Distro	TRTEMP_2.2_AUTO_Automated-Build-Testing
Runtime (Compliance) 	TRTEMP_2.2_AUTO_Runtime
Build Apliance 	TRTEMP_2.2_MANUAL_Build-Appliance
pTest 	
Performance 	Performance Test
SWUpd (Software Updater) 	
Distributed Builds 	
= Test Cases =
*Yocto 2.2 Test Cases
= Features to be tested =
All component to be tested on 2.2 are in #Test Items a detailed list of features per component can be found in Test Plan of the components QA_Master_Test_Plan#Component
= Features pending to be tested =
Here is a list of areas that are not yet covered and need some work, implement test cases and add it to a component that belongs or create a new test plan for new component.
* build-appliance - can we run builds in the build-appliance image? 
* Verify if -c menuconfig work? 
* Verify if -c devshell work?
* Verify if BBPOSTCONF and BBPRECONF work in bitbake?
* Verify if oe-pkgdata-util print help when no parameters specified?
* Verify if parsing time is improved with a hot codeparser cache (build metrics did catch a regression here but we had to manually spot it).
* Verify Bitbake sane error if a layer doesn't exist? (no backtrace?) send-error-report - no unittesting, only caught python3 issues when it failed to send errors
* Verify if buildhistory_analsysis.py is tested
* Verify if compression image types is tested
* Verify combo-layer 
* Verify yocto-bsp/yocto-layer 
* Verify oe-selftest coverage
* Verify bitbake -u depexp -g XXX
= Pass / Fail Criteria =
This criteria is defined in Yocto_Project_v2.2_Status#Yocto_Project_v2.2_Release_Criteria
= Exit Criteria =
As defined in QA_Master_Test_Plan#Entry and Exit Criteria
= Test Deliverables =
* Live Status 2.2_QA_Status
* Final execution Report #Execution History
* Mail sent to the mailing list
= Test Environment =
There are different environments and tools for every type of testing refer to QA_Master_Test_Plan#Test_Areas for more detailed information.
= Schedule =
The planed schedule for 2.2 can be found at Yocto_2.2_Schedule
== Live Schedule ==
MILESTONE 	STARTED 	FINISHED 	STATUS
M1.rc2 	2016-06-21	2016-06-29	DONE
M2.rc1 	2016-07-21	2016-06-28	DONE
M2 Meta Intel. 	2016-08-04	2016-08-10	DONE
2.2 Master: f038f0699. 	2016-08-19	2016-08-26	DONE
M3 rc1 	2016-09-09	2016-09-20	DONE
M4 rc4 	2016-10-18	2016-10-25	DONE
M4 rc6 	2016-10-26	2016-10-28	DONE
2.2rc1 Meta Intel 6.0 	2016-11-03	2016-11-10	DONE
2.2rc2 Meta Intel 6.0 	2016-11-10	2016-11-16	DONE
2.2.1 rc1 Point Release	2017-02-13	2017-02-20	DONE
2.2.2 rc1 Point Release	2017-05-29	2017-06-02	DONE
2.2.2 rc2 Point Release	2017-09-08	2017-09-13	DONE
2.2.3 rc2 Point Release	2018-01-22	2018-02-23	DONE
2.2.4 rc1 Point Release	2018-05-25	2018-07-13	DONE
= Execution History =
MILESTONE 	REPORT
M1.rc2 	WW26 - 2016-06-21 - Full Test Cycle 2.2 M1.rc2
M2.rc1 	WW30 - 2016-07-21 - Full Test Cycle 2.2 M2.rc1
M2 Meta-Intel 	WW33 - 2016-08-04 - Full Test Cycle - 2.2 M2 - meta-intel
2.2 Master: f038f0699 	WW35 - 2016-08-19 - Test Cycle - 2.2_Master
M3 	WW38 - 2016-09-09 - Full Test Cycle 2.2_M3.rc1
2.2 rc4 	WW43 - 2016-10-18 - Full Test Cycle 2.2 rc4
2.2 rc6 	WW44 - 2016-10-26 - Full Test Cycle 2.2 rc6
2.2rc1 Meta Intel 6.0 	WW46 - 2016-11-03 - Full Test Cycle - 2.2_rc1 - meta-intel-6.0
2.2rc2 Meta Intel 6.0 	WW48 - 2016-11-03 - Partial Test_Cycle - 2.2_rc2 - meta-intel-6.0
2.2.1rc1 Point Release 	WW07 - 2017-02-13 - Full Point Release Test Cycle 2.2.1 rc1
2.2.2 rc1 Point Release	WW22_-_2017-06-05_-_Full_Point_Release_Test_Cycle_2.2.2_rc1
2.2.2 rc2 Point Release	WW37_-_2017-09-13_-_Full_Point_Release_Test_Cycle_2.2.2_rc2
2.2.3 rc2 Point Release	WW06_-_2018-02-08_-_Full_Point_Release_Test_Cycle_2.2.3_rc2
2.2.4 rc1 Point Release	WW28_-_2018-07-13_-_Full_Test_Cycle_2.2.4_rc1