Toaster future release planning

Category:Toaster
== 2.2 Planning ==
=== Wish lists ===
==== Michael's ====
* Remove pseudo API cruft / template context to json response stuff
* Remove bldcontrol (we only need localhostbecontroller.py)
* Look at schema and rework where needed
* Consolidate used API into it's own view
* Finish toaster tables porting
* Create a toasterclient.py to interact with toaster to avoid need for 'command line builds'
** This would use toaster's API as the entry point to using bitbake
** It could be called by CI systems
** It could be used by auto builders etc
** Means that there are no two modes anywhere in Toaster
** Allows project configuration to be changed in toaster from the command line
* Drop supporting old releases
==== Brian's ====
2.2
# bootstrap3
# Python3
# CI tie in. 
# allow toaster to work in OE mode, Ostro, etc. bascially generalize our use
# toasterconf.json generalize and load conifguration
The Ordered Part
# Django 1.8 aka LTS
# Image Customization
# Testing and Stability
# One way to start/One code path 
This is *not* ordered! This is more of a brain dump :)
* Collapse analysis and managed mode
* Working CI for each commit into toaster-next as well as for project peoples testME branches on poky-contrib:
** run django tests
** run selenium tests
** leverage sstate to make runs generally shorter
* remove git assumptions from code
* rationalize/simplify the configuration scripts
* if bitbake-memres comes out, interact with it gracefully (this is my preferred method of collapsing analysis and managed)
* fix the tmpdir/release build issue &/or discuss the need to support building old releases with a UI rather than limiting toaster to build the release it is part of. This is inherently fragile.
* ask bb to parse any layer added to a project and such as imported layers, and layers specified in the configuration file to give us recipe information.
* improve our build event following:
** note parse events
** update/retrieve ui so that we can tell something is happening before the first build event comes 
** eliminate the remaining random tracebacks from try/except failures.
* allow configuration in addition to the addition/removal of layers/packages
** kernel config
* add user control/authentication
* update to Django 1.8. Always stay on Django LTS versions
* add layer update from layers.openembedded.org to the UI
* add Django tests for building e.g. we should be able to build core-image-sato from a django test and validate the db
* add mysql script to simplify setup with mysql/apache
* pull in new look and feel
* add django tests to validate all artifacts - if we say they can download it and it appears on a page anywhere, it should have a test
* improve docs of code on wiki. We have a nice start (ty E & M); I'd like to improve it. 
** events - more info and list types supported
** what logic is in jscript , what is in python
** database schema definition and description esp interrelationships.
** list django tests, mostly so we can see what we are missing. 
* build mode triggerred off of database entry, not autonomous process.
* add asynchronicity. I have noted a number of places where commands result in a sluggish ui. I'm nervous about this one.
** for example, starting a build that requires us to clone layers has a noticeable delay as compared to starting a local only project build.
* verify we can stop builds cleanly.
* get the rest of our logs into our local toaster dir
* allow us to run and stop 2 toasters on the same machine without them interfering with each other
* (2.2 or later for sure) discuss how to support multiple back end bb servers. 
** probably will need to be able to add servers/remove servers/see server status/build state
* be able to see more details about build. For example, see the "jobs" being processed like you can in knotty
* revisit what variables we let people change. sstate_dir and dl_dir should certainly be allowed to change
* discuss way we do migrations. possibly change. thoughts:
** only support migrations from last release to current. (e.g. if you were running toaster jethro you could update to jethro++, but could not from fido to jethro)
** take migration from the startup script. it's only for releasetorelease.
*** allow it to be in for master and ease of development w/in a release and pull at end as we do with moving releases.
** whatever we decide, add a test set for it so we know it works
* Be able to delete things from the ui
** old builds
** old projects
** imported layers we no longer want
* Be able to set from the ui a policy to delete all builds/recipes/packages that are older than XXX &/or be able to set maximum size on db and delete everything older than XXX when we near that size
* update the builds on the web page so we dont have to hit refresh to see the build progress.
==== Elliot's ====
* Make sure all files lint properly
* Include lint in CI builds
* Improve test coverage of all areas of the UI and back-end
* Clean up toasterui.py so it's easier to see what's going on in there
* Fix localhostbecontroller so it doesn't rely on what's in the log file when figuring out whether the bitbake server has started
* Provide a way to modify settings in settings.py without editing that file (e.g. with a local overlay configuration file)
* Move the logic, environment variables, directory setup and script calls from bin/toaster into Django commands: this would allow you to run toaster correctly, with all the required setup, without having to use bin/toaster; it would also mean that we could provide cleaner production setup instructions, as a user could run a series of Django commands (with their own parameters) to setup their environment; bin/toaster would then just execute a series of Django commands
==== Belén's ====
===== 2.2 release =====
* Move to Bootstrap 3 (Bugzilla entry 9517)
* Clean up all the small little bugs in the Toaster UI
* Provide a better way to build the extensible SDK from Toaster (Bugzilla entry 9297)
* Improve the variable history (Bugzilla entry 5811)
===== 2.1 release =====
* Implement new custom theme (already in progress, aiming for end of M1: Dec 7th). Related bugs: 8417
* Finish image customisation. Related bugs: 8070, 8081, 8082, 8103, 8104, 8117, 8128, 8132, 8091
* Review interaction with layers: as it is, imposes heavy restrictions on users. Some of the questions I would like to consider:
** Should the import layer action be global instead of per project, so that you can reuse imported layers across projects?
** Should we allow users to override the release layer branch, setting a particular layer index layer to a different branch or commit?
** Should we allow users to set a different git repo for layer index layers?
** Should we allow users to override the bitbake revision set by the release in a project?
** Related bugs: 6640, 6701, 7574, 8426, 8429
* Access control (a simple one to start, with very basic permissions). Related bugs: 6233, 6234
* WIC integration (getting a step closer to the holy grail of 'click button' > 'image deployed in board')
* Fix the variable history (this is one of the features users have highly praised, and it is a bit broken in 2.0). Related bugs: 5811, 8488, 8190, 8189, 8188
* Get Toaster to collect build data from Jenkins builds. Once again, repeatedly requested by users. Related bugs: 7527
==== Ed's ====
* Merge analysis and managed modes
* Fix as much pending bugs as possible.
* Finish image customisation.
* Support Django LTS
* Increase test coverage. It would be good to set realistic goals, e.g. cover specific modules or increase coverage to 10% and follow the plan.
* Include production configuration into testing
* Cover views.py with tests and split it into modules. Currently it's huge and hard to maintain. I suspect there are a lot of unused APIs in there.
* Finish refactoring of buildcontroller code. This work is started in ed/toaster/bec and sitting there for quite long time.
* Bring patchset support to toaster patchwork.
* Add 'submitted upstream' and 'merged upstream' statuses to patchwork. Make the statuses updated automatically when patch is accepted to toaster-next or bitbake master.
* Clean up the code. Set realistic goals for pylint scores and follow them.
* Enhance Jenkins CI jobs to run all possible django, tts and pylint tests
==== Sujith's ====
* Get toaster working with non-git repos. Related bugs: 8456
* minor, but would be good to have :- Once user launches toaster, it would be nice to see the progress bar updated without user refreshing the page. Related bugs: 8328
* Cancellation of build. Related bugs: 6787
Would love to take up more tasks.
==== David's ====
* Finish the effort around Toaster "extensions", started by Farrell. This will enable Toaster customization for commercial partners, as per branding and 
alternate pages (for extended project creation).
== All bugs for 2.2 ==
== Toaster Enhancements for 2.2 ==