Yocto Project 3.3 Release Test Plan

= Revision history =
Version	Modifier 	Comments
1.0 	Jain Sangeeta	First draft
= Introduction =
This is the the overall test plan for version 3.3 of the , for all the generic information this document will be pointing to QA Master Test Plan
= Objectives =
This test plan for Yocto Project 3.3 release supports the following objectives:
*To define the tools to be used throughout the testing process.
*To communicate to the responsible parties the items to be tested, set expectations around schedule, and define environmental needs.
*To define how the tests will be conducted.
*Release Criteria
*Risks and contingencies
QA_Master_Test_Plan#Objectives_and_Tasks
= Risk Analysis & Contingency Plan =
QA identified risks for this release are mentioned here.
Module 	Risk Description 	Risk Level	Contingency plan 	Owner
BSP 	31 manual test cases 	medium 	test cases to be automated	Intel
Build Appliance	3 manual test cases, not executed by any QA team 	medium 	test cases to be automated	
Compliance 	4 manual test cases, not executed by any QA team 	medium 	test cases to be automated	
Kernel dev 	9 manual test cases, not executed by any QA team 	medium 	test cases to be automated	
OE-Core 	4 manual test cases 	medium 	test cases to be automated	Intel
SDK 	1 manual test case, not executed by any QA team 	medium 	test case to be automated 	
Toaster 	93 manual test cases, not executed by any QA team	medium 	test cases to be automated	
= Test Strategy =
Is followed the general strategy defined on QA_Master_Test_Plan#Testing_Strategy and below is showed specific information applicable only for 3.3 Release
= Test Items =
The Yocto Project QA team verifies each release against recent versions of the most popular Linux distributions that provide stable releases. 
 * centos 7 
 * Debian 8
 * Debian 9
 * Fedora 28
 * Fedora 29
 * Fedora 30
 * OpenSuse 150
 * OpenSuse 151
 * OpenSuse 423
 * OpenSuse Tumbleweed
 * ubuntu 16.04 
 * ubuntu 18.04
= Test Cases =
* http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/runtime
* http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/selftest
* http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/manual
* http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/sdk
* http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/sdkext
== OE-CORE ==
Component	Environment 	Auto	Manual	Owner
OE-CORE 	Ubuntu 20.04	 	Y 	Intel
OE-CORE 	 	 	 	
OE-CORE 	 	 	 	
OE-CORE 	 	 	 	
OE-CORE 	 	 	 	
OE-CORE 	 	 	 	
OE-CORE 	 	 	 	
OE-CORE 	 	 	 	
== BSP/QEMU ==
Component	Arch 	HW 	Environment 	Image 	Auto	Manual	pTest	Compliance	ltp
BSP 	x86 	MinwTurbot 32bit	genericx86 	core-image-sato-sdk	Y 	 	 	 	
BSP 	x86 	MinwTurbot 32bit	genericx86 	core-image-lsb-sdk 	Y 	 	 	 	
BSP 	x86 	Coffee Lake 	genericx86-64	core-image-sato-sdk	Y 	 	 	 	
BSP 	x86 	Coffee Lake 	genericx86-64	core-image-lsb-sdk 	Y 	 	 	 	
BSP 	x86 	NUC 6 	genericx86-64	core-image-sato-sdk	Y 	 	Y 	 	Y
BSP 	x86 	NUC 6 	genericx86-64	core-image-lsb-sdk 	Y 	 	 	Y 	
BSP 	x86 	NUC 7 	genericx86-64	core-image-sato-sdk	Y 	Y 	 	 	
BSP 	x86 	NUC 7 	genericx86-64	core-image-lsb-sdk 	Y 	 	 	 	
BSP 	x86 	QEMU 	qemux86 	core-image-sato-sdk	Y 	 	 	 	
BSP 	x86 	QEMU 	qemux86 	core-image-lsb-sdk 	Y 	 	 	 	
BSP 	x86 	QEMU 	qemux86-64 	core-image-sato-sdk	Y 	 	 	 	
BSP 	x86 	QEMU 	qemux86-64 	core-image-lsb-sdk 	Y 	 	 	 	
BSP 	non x86 	EdgeRouter 	EdgeRouter 	core-image-sato-sdk	Y 	Y 	 	Y 	
BSP 	non x86 	Beaglebone Black	Beaglebone 	core-image-sato-sdk	Y 	Y 	 	Y 	
BSP 	non x86 	QEMU 	qemuarm 	core-image-sato-sdk	Y 	 	 	 	
BSP 	non x86 	QEMU 	qemuarm-64 	core-image-sato-sdk	Y 	 	Y 	 	Y
BSP 	non x86 	QEMU 	qemuppc 	core-image-sato-sdk	Y 	 	 	 	
BSP 	non x86 	QEMU 	qemumips 	core-image-sato-sdk	Y 	 	 	 	
BSP 	non x86 	QEMU 	qemumips64 	core-image-sato-sdk	Y 	 	 	 	
BSP 	Meta-Intel	MinwTurbot 32bit	core2-32 	core-image-sato-sdk	Y 	 	 	 	
BSP 	Meta-Intel	MinwTurbot 32bit	core2-32 	core-image-lsb-sdk 	Y 	 	 	 	
BSP 	Meta-Intel	Coffee Lake 	corei7-64 	core-image-sato-sdk	Y 	 	 	 	
BSP 	Meta-Intel	Coffee Lake 	corei7-64 	core-image-lsb-sdk 	Y 	 	 	 	
BSP 	Meta-Intel	NUC 6 	corei7-64 	core-image-sato-sdk	Y 	 	 	 	
BSP 	Meta-Intel	NUC 6 	corei7-64 	core-image-lsb-sdk 	Y 	 	 	 	
BSP 	Meta-Intel	NUC 7 	corei7-64 	core-image-sato-sdk	Y 	Y 	 	 	
BSP 	Meta-Intel	NUC 7 	corei7-64 	core-image-lsb-sdk 	Y 	 	 	 	
BSP 	Meta-Intel	NUC 7 	corei7-64 	poky-tiny 	 	Y 	 	 	
BSP 	Meta-Intel	 	 	 	 	 	 	 	
BSP 	 	 	 	 	 	 	 	 	
All component to be tested on 3.3 are in #Test Items a detailed list of features per component can be found in Test Plan of the components QA_Master_Test_Plan#Component
=Approach=
Testing will be done as per QA process defined in main QA wiki page at https://wiki.yoctoproject.org/wiki/QA#QA_Process.
= Pass / Fail Criteria =
This criteria is defined in Yocto_Project_v3.3_Status#Yocto_Project_v3.3_Release_Criteria
= Exit Criteria =
As defined in QA_Master_Test_Plan#Entry and Exit Criteria
= Test Deliverables =
* Final Test results
* Final Test report
* Mail sent to the mailing list
= Schedule =
The planed schedule for 3.3 can be found at Yocto_3.3_Schedule
== Live Schedule & Execution History ==
MILESTONE	STARTED 	FINISHED 	STATUS	REPORT
M1 	12/11/2020	12/15/2020	DONE 	https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults
M2 	01/21/2021	01/27/2021	DONE 	https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults
M3 	03/16/2021	03/26/2021	DONE 	https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults
M4 	04/05/2021	04/20/2021	DONE 	https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults