Repository: zephyrproject-rtos/west

Issue:
Determine west-based release strategy
We mostly have a plan for tagging a west-based release as discussed in https://github.com/zephyrproject-rtos/west/issues/39#issuecomment-423629912.

The short story is:

1. Tag every project (zephyr gets v1.14.0, hypothetical-other-thing gets v1.14.0, etc.)
2. Set the revisions for each project in the manifest file to v1.14.0; tag the commit that does that v1.14.0 as well

Then you can do `west init --mr v1.14.0`.

I think the thing we did not realize is that this will not prevent users from continually getting updates to west itself! These may in turn be backwards incompatible with the v1.14.0 versions of things. So if the user ever runs 'west pull', say, that will update west itself, which is not safe.

This issue tracks resolving this; it's a blocker for Zephyr v1.14.

Discussion:
carlescufi: @tejlmand please take another look
tejlmand: Hi.

Looking better :)
I took the liberty of changing a little bit.
Take a look:
https://github.com/tejlmand/west/commit/873ac11e5f9bd703e53015247bc73616dbdc26d6

Basically it treats `west` as a project, so that `_load` function can parse the `west` section just as any other project, meaning we don't need special WestMeta class.

And it means what you get when instantiating an object of Manifest class is same, just with either normal or special projects. 
carlescufi: Fixed tests, I think the coverage should be equal to the one before this PR
tejlmand: One thing I think should be discussed is `url` naming.

Although my work also uses `url` in `west` section in the updated manifest file, I beleive it is inconsistent use of `url` tag in manifest yaml, as we now have:
```
west:
  url: https://github.com/zephyrproject-rtos/west <-- This url contains full path
  revision: master


manifest:
.....
  remotes:
    - name: upstream
      url: https://github.com/zephyrproject-rtos  <-- This is only a base url which will be concatenated with project name below

  projects:
    - name: zephyr
```
I believe we should find either another name for url in `west` section, or re-name `url` to `url-base` in `manifest` section
carlescufi: > I believe we should find either another name for url in `west` section, or re-name `url` to `url-base` in `manifest` section

Yes, we can rename `url` to `base-url` or `url-base` in the main manifest section.
carlescufi: @ulfalizer addressed your comments, thanks for the review!
tejlmand: > Yes, we can rename `url` to `base-url` or `url-base` in the main manifest section.

Updated to `url-base` and updated test in additional commit as agreed.

Patch:
diff --git a/README.rst b/README.rst
index 598f8f7..e1cd25f 100644
--- a/README.rst
+++ b/README.rst
@@ -97,12 +97,8 @@ This somewhat unusual arrangement is because:
 Using a Custom "Main" West
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-To initialize west from a non-default location::
-
-  west init -w https://example.com/your-west-repository.git
-
-You can also add ``--west-rev some-branch`` to use ``some-branch``
-instead of ``master``.
+To initialize west from a non-default location add a section ``west`` in the
+manifest ``.yml`` file that points to a ``url`` and ``revision`` of your choice.
 
 To use another manifest repository (optionally with ``--mr
 some-manifest-branch``)::
diff --git a/src/west/_bootstrap/main.py b/src/west/_bootstrap/main.py
index 5770a39..5706f01 100644
--- a/src/west/_bootstrap/main.py
+++ b/src/west/_bootstrap/main.py
@@ -9,8 +9,10 @@ import argparse
 import configparser
 import os
 import platform
+import pykwalify.core
 import subprocess
 import sys
+import yaml
 
 import west._bootstrap.version as version
 
@@ -48,6 +50,7 @@ MANIFEST_URL_DEFAULT = 'https://github.com/zephyrproject-rtos/manifest'
 # Default revision to check out of the manifest repository.
 MANIFEST_REV_DEFAULT = 'master'
 
+_SCHEMA_PATH = os.path.join(os.path.dirname(__file__), "west-schema.yml")
 
 #
 # Helpers shared between init and wrapper mode
@@ -71,6 +74,14 @@ def west_dir(start=None):
     '''
     return os.path.join(west_topdir(start), WEST_DIR)
 
+def manifest_dir(start=None):
+    '''
+    Returns the path to the manifest/ directory, searching ``start`` and its
+    parents.
+
+    Raises WestNotFound if no west directory is found.
+    '''
+    return os.path.join(west_topdir(start), MANIFEST)
 
 def west_topdir(start=None):
     '''
@@ -151,11 +162,6 @@ With --no-reset, only the configuration file will be updated, and you will have
 to handle any resetting yourself.
 '''.format(WEST_MARKER))
 
-    init_parser.add_argument(
-        '-b', '--base-url',
-        help='''Base URL for both 'manifest' and 'zephyr' repositories. Cannot
-             be given if either -m or -w are.''')
-
     init_parser.add_argument(
         '-m', '--manifest-url',
         help='Manifest repository URL (default: {})'
@@ -166,16 +172,6 @@ to handle any resetting yourself.
         help='Manifest revision to fetch (default: {})'
              .format(MANIFEST_REV_DEFAULT))
 
-    init_parser.add_argument(
-        '-w', '--west-url',
-        help='West repository URL (default: {})'
-             .format(WEST_URL_DEFAULT))
-
-    init_parser.add_argument(
-        '--wr', '--west-rev', dest='west_rev',
-        help='West revision to fetch (default: {})'
-             .format(WEST_REV_DEFAULT))
-
     init_parser.add_argument(
         '--nr', '--no-reset', dest='reset', action='store_false',
         help='''Suppress the automatic reset of the manifest, west, and project
@@ -198,16 +194,10 @@ to handle any resetting yourself.
 def bootstrap(args):
     '''Bootstrap a new manifest + West installation.'''
 
-    if args.base_url:
-        if args.west_url or args.manifest_url:
-            sys.exit('fatal error: -b is incompatible with -m and -w')
-        west_url = args.base_url.rstrip('/') + '/west'
-        manifest_url = args.base_url.rstrip('/') + '/manifest'
-    else:
-        west_url = args.west_url or WEST_URL_DEFAULT
-        manifest_url = args.manifest_url or MANIFEST_URL_DEFAULT
+    west_url = WEST_URL_DEFAULT
+    manifest_url = args.manifest_url or MANIFEST_URL_DEFAULT
 
-    west_rev = args.west_rev or WEST_REV_DEFAULT
+    west_rev = WEST_REV_DEFAULT
     manifest_rev = args.manifest_rev or MANIFEST_REV_DEFAULT
 
     directory = args.directory or os.getcwd()
@@ -231,16 +221,38 @@ def bootstrap(args):
     # Clone the west source code and the manifest into west/. Git will create
     # the west/ directory if it does not exist.
 
-    clone('west repository', west_url, west_rev,
-          os.path.join(directory, WEST_DIR, WEST))
-
     clone('manifest repository', manifest_url, manifest_rev,
           os.path.join(directory, WEST_DIR, MANIFEST))
 
+    # Parse the manifest and look for a section named "west"
+    manifest_file = os.path.join(directory, WEST_DIR, MANIFEST, 'default.yml')
+    with open(manifest_file, 'r') as f:
+        data = yaml.safe_load(f.read())
+
+    if 'west' in data:
+        wdata = data['west']
+        try:
+            pykwalify.core.Core(
+                source_data=wdata,
+                schema_files=[_SCHEMA_PATH]
+            ).validate()
+        except pykwalify.errors.SchemaError as e:
+            sys.exit("Error: Failed to parse manifest file '{}': {}"
+                      .format(manifest_file, e))
+
+        if 'url' in wdata:
+            west_url = wdata['url']
+        if 'revision' in wdata:
+            west_rev = wdata['revision']
+
+    print("cloning {} at revision {}".format(west_url, west_rev))
+    clone('west repository', west_url, west_rev,
+          os.path.join(directory, WEST_DIR, WEST))
+
     # Create an initial configuration file
 
     config_path = os.path.join(directory, WEST_DIR, 'config')
-    update_conf(config_path, west_url, west_rev, manifest_url, manifest_rev)
+    update_conf(config_path, manifest_url, manifest_rev)
     print('=== Initial configuration written to {} ==='.format(config_path))
 
     # Create a dotfile to mark the installation. Hide it on Windows.
@@ -259,36 +271,24 @@ def reinit(config_path, args):
     This updates the west/config configuration file, and optionally resets the
     manifest, west, and project repositories to the new revision.
     '''
-    if args.base_url:
-        if args.west_url or args.manifest_url:
-            sys.exit('fatal error: -b is incompatible with -m and -w')
-        west_url = args.base_url.rstrip('/') + '/west'
-        manifest_url = args.base_url.rstrip('/') + '/manifest'
-    else:
-        west_url = args.west_url
-        manifest_url = args.manifest_url
+    manifest_url = args.manifest_url
 
-    if not (west_url or args.west_rev or manifest_url or args.manifest_rev):
+    if not (manifest_url or args.manifest_rev):
         sys.exit('West already initialized. Please pass any settings you '
                  'want to change.')
 
-    update_conf(config_path, west_url, args.west_rev, manifest_url,
-                args.manifest_rev)
+    update_conf(config_path, manifest_url, args.manifest_rev)
 
     print('=== Updated configuration written to {} ==='.format(config_path))
 
     if args.reset:
-        cmd = ['update']
-        if manifest_url or args.manifest_rev:
-            cmd += ['--reset-manifest', '--reset-projects']
-        if west_url or args.west_rev:
-            cmd.append('--reset-west')
+        cmd = ['update', '--reset-manifest', '--reset-projects', '--reset-west']
         print("=== Running 'west {}' to update repositories ==="
               .format(' '.join(cmd)))
         wrap(cmd)
 
 
-def update_conf(config_path, west_url, west_rev, manifest_url, manifest_rev):
+def update_conf(config_path, manifest_url, manifest_rev):
     '''
     Creates or updates the configuration file at 'config_path' with the
     specified values. Values that are None/empty are ignored.
@@ -298,8 +298,6 @@ def update_conf(config_path, west_url, west_rev, manifest_url, manifest_rev):
     # This is a no-op if the file doesn't exist, so no need to check
     config.read(config_path)
 
-    update_key(config, 'west', 'remote', west_url)
-    update_key(config, 'west', 'revision', west_rev)
     update_key(config, 'manifest', 'remote', manifest_url)
     update_key(config, 'manifest', 'revision', manifest_rev)
 
diff --git a/src/west/_bootstrap/west-schema.yml b/src/west/_bootstrap/west-schema.yml
new file mode 100644
index 0000000..78b44da
--- /dev/null
+++ b/src/west/_bootstrap/west-schema.yml
@@ -0,0 +1,17 @@
+## A pykwalify schema for basic validation of the structure of a
+## west YAML file.  (Full validation would require additional work,
+## e.g. to validate that remote URLs obey the URL format specified in
+## rfc1738.)
+##
+
+# The top-level west yaml is a map. The only top-level element is
+# 'west'. All other elements are contained within it. This allows
+# us a bit of future-proofing.
+type: map
+mapping:
+  url:
+    required: false
+    type: str
+  revision:
+    required: false
+    type: str
diff --git a/src/west/commands/project.py b/src/west/commands/project.py
index b0e57e0..a240d37 100644
--- a/src/west/commands/project.py
+++ b/src/west/commands/project.py
@@ -15,7 +15,7 @@ from west.config import config
 from west import log
 from west import util
 from west.commands import WestCommand
-from west.manifest import default_path, Remote, Project, \
+from west.manifest import default_path, Remote, Project, SpecialProject, \
                           Manifest, MalformedManifest, META_NAMES
 
 
@@ -145,8 +145,8 @@ class Clone(WestCommand):
 
     def do_run(self, args, user_args):
         if args.update:
-            _update_west()
-            _update_manifest()
+            _update_manifest(args)
+            _update_west(args)
 
         for project in _projects(args, listed_must_be_cloned=False):
             if args.branch:
@@ -185,8 +185,8 @@ class Fetch(WestCommand):
 
     def do_run(self, args, user_args):
         if args.update:
-            _update_west()
-            _update_manifest()
+            _update_manifest(args)
+            _update_west(args)
 
         for project in _projects(args, listed_must_be_cloned=False):
             _fetch(project)
@@ -216,8 +216,8 @@ class Pull(WestCommand):
 
     def do_run(self, args, user_args):
         if args.update:
-            _update_west()
-            _update_manifest()
+            _update_manifest(args)
+            _update_west(args)
 
         for project in _projects(args, listed_must_be_cloned=False):
             _fetch(project)
@@ -438,19 +438,19 @@ class Update(WestCommand):
                 args.reset_projects):
 
             # No arguments is an alias for --update-west --update-manifest
-            _update_west()
-            _update_manifest()
+            _update_manifest(args)
+            _update_west(args)
             return
 
         if args.reset_manifest:
-            _update_and_reset_special('manifest')
+            _update_and_reset_special(args, 'manifest')
         elif args.update_manifest:
-            _update_manifest()
+            _update_manifest(args)
 
         if args.reset_west:
-            _update_and_reset_special('west')
+            _update_and_reset_special(args, 'west')
         elif args.update_west:
-            _update_west()
+            _update_west(args)
 
         if args.reset_projects:
             _reset_projects(args)
@@ -591,7 +591,7 @@ def _projects(args, listed_must_be_cloned=True, include_meta=False):
     projects = _all_projects(args)
 
     if include_meta:
-        projects += [_special_project(name) for name in META_NAMES]
+        projects += [_special_project(args, name) for name in META_NAMES]
 
     if not args.projects:
         # No projects specified. Return all projects.
@@ -669,7 +669,8 @@ def _all_projects(args):
     # command aborts.
 
     try:
-        return list(Manifest.from_file(_manifest_path(args)).projects)
+        return list(Manifest.from_file(_manifest_path(args),
+                                       'manifest').projects)
     except MalformedManifest as m:
         log.die(m.args[0])
 
@@ -825,40 +826,29 @@ def _checkout(project, branch):
     _git(project, 'checkout ' + branch)
 
 
-def _special_project(name):
+def _special_project(args, name):
     # Returns a Project instance for one of the special repositories in west/,
     # so that we can reuse the project-related functions for them
 
-    # TODO: Maybe the Remote class could be replaced by a single
-    # project.remote_name field, now that we no longer use the Git remote
-    # mechanism and fetch directly from URLs
+    if name == 'manifest':
+        url = config.get(name, 'remote', fallback='origin')
+        revision = config.get(name, 'revision', fallback='master')
+        return SpecialProject(name, revision=revision,
+                       path=os.path.join('west', name), url=url)
 
-    remote = Remote(name='dummy name for {} repository'.format(name),
-                    url='dummy URL for {} repository'.format(name))
+    return Manifest.from_file(_manifest_path(args), name).west_project
 
-    # 'revision' always exists and defaults to 'master'
-    project = Project(name, remote, None,
-                      revision=config.get(name, 'revision', fallback='master'),
-                      path=os.path.join('west', name))
+def _update_west(args):
+    _update_special(args, 'west')
 
-    # This could also be the name of a Git remote. The naming breaks a bit
-    # here.
-    project.url = config.get(name, 'remote', fallback='origin')
 
-    return project
+def _update_manifest(args):
+    _update_special(args, 'manifest')
 
 
-def _update_west():
-    _update_special('west')
-
-
-def _update_manifest():
-    _update_special('manifest')
-
-
-def _update_special(name):
+def _update_special(args, name):
     with _error_context(_FAILED_UPDATE_MSG):
-        project = _special_project(name)
+        project = _special_project(args, name)
         _dbg(project, 'Updating (name-and-path)', level=log.VERBOSE_NORMAL)
 
         old_sha = _sha(project, 'HEAD')
@@ -897,11 +887,11 @@ def _update_special(name):
                 raise WestUpdated()
 
 
-def _update_and_reset_special(name):
+def _update_and_reset_special(args, name):
     # Updates one of the special repositories (the manifest and west) by
     # resetting to the new revision after fetching it (with 'git reset --keep')
 
-    project = _special_project(name)
+    project = _special_project(args, name)
     with _error_context(', while updating/resetting special project'):
         _inf(project,
              "Fetching and resetting (name-and-path) to '(revision)'")
@@ -1078,7 +1068,8 @@ def _expand_shorthands(project, s):
             .replace('(name-and-path)',
                      '{} ({})'.format(
                          project.name, os.path.join(project.path, ""))) \
-            .replace('(remote-name)', project.remote.name) \
+            .replace('(remote-name)', 'None' if project.remote is None
+                                             else project.remote.name) \
             .replace('(url)', project.url) \
             .replace('(path)', project.path) \
             .replace('(abspath)', project.abspath) \
diff --git a/src/west/manifest-schema.yml b/src/west/manifest-schema.yml
index 95cde46..c7d4e22 100644
--- a/src/west/manifest-schema.yml
+++ b/src/west/manifest-schema.yml
@@ -9,128 +9,124 @@
 ##
 ## However, the features don't map 1:1.
 
-# The top-level manifest is a map. The only top-level element is
-# 'manifest'. All other elements are contained within it. This allows
-# us a bit of future-proofing.
+# The top-level manifest is a map. There may be multiple sections in the
+# manifest file. Each section can be validated by their own schema.
+# This schema validates the 'manifest' section.
 type: map
 mapping:
-  manifest:
-    required: true
+  # The "defaults" key specifies some default values used in the
+  # rest of the manifest.
+  #
+  # The value is a map with the following keys:
+  #
+  # - remote: if given, this is the default remote in each project
+  # - revision: if given, this is the default revision to check
+  #   out of each project
+  #
+  # See below for more information about remotes and projects.
+  #
+  # Examples:
+  #
+  # default:
+  #   remote: zephyrproject-rtos
+  #   revision: master
+  defaults:
+    required: false
     type: map
     mapping:
-      # The "defaults" key specifies some default values used in the
-      # rest of the manifest.
-      #
-      # The value is a map with the following keys:
-      #
-      # - remote: if given, this is the default remote in each project
-      # - revision: if given, this is the default revision to check
-      #   out of each project
-      #
-      # See below for more information about remotes and projects.
-      #
-      # Examples:
-      #
-      # default:
-      #   remote: zephyrproject-rtos
-      #   revision: master
-      defaults:
+      remote:
+        required: false
+        type: str
+      revision:
         required: false
-        type: map
+        type: str
+
+  # The "remotes" key specifies a sequence of remotes, each of
+  # which has a name and a fetch URL.
+  #
+  # These work like repo remotes, in that they specify a URL
+  # prefix which remote-specific Git repositories hang off of.
+  # (This saves typing and makes it easier to move things around
+  # when most repositories are on the same server or GitHub
+  # organization.)
+  #
+  # Example:
+  #
+  #   remotes:
+  #     - name: zephyrproject-rtos
+  #       url-base: https://github.com/zephyrproject-rtos
+  #     - name: developer-fork
+  #       url-base: https://github.com/a-developer
+  remotes:
+    required: true
+    type: seq
+    sequence:
+      - type: map
+        mapping:
+          name:
+            required: true
+            type: str
+          url-base:
+            required: true
+            type: str
+
+  # The "projects" key specifies a sequence of "projects",
+  # i.e. Git repositories. These work like repo projects, in that
+  # each project has a name, a remote, and optional additional
+  # metadata.
+  #
+  # Each project is a map with the following keys:
+  #
+  # - name: Mandatory, the name of the git repository. The clone
+  #   URL is formed by remote url-base + '/' + name. The name cannot
+  #   be one of the reserved values "west" and "manifest".
+  # - remote: Optional, the name of the remote to pull it from.
+  #   If the remote is missing, the remote'key in the top-level
+  #   defaults key is used instead. If both are missing, it's an error.
+  # - revision: Optional, the name of the revision to check out.
+  #   If not given, the value from the default element will be used.
+  #   If both are missing, then the default is 'master'.
+  # - path: Where to clone the repository locally. If missing,
+  #   it's cloned at top level in a directory given by its name.
+  # - clone-depth: if given, it is a number which creates a shallow
+  #   history in the cloned repository limited to the given number
+  #   of commits.
+  #
+  # Example, using default and non-default remotes:
+  #
+  #   projects:
+  #     # Uses default remote (zephyrproject-rtos), so clone URL is:
+  #     #   https://github.com/zephyrproject-rtos/zephyr
+  #     - name: zephyr
+  #     # Manually specified remote; clone URL is:
+  #     #   https://github.com/a-developer/west
+  #     - name: west
+  #       remote: developer-fork
+  #     # Manually specified remote, clone URL is:
+  #     #   https://github.com/zephyrproject-rtos/some-vendor-hal
+  #     # Local clone path (relative to installation root) is:
+  #     #   ext/hal/some-vendor
+  #     - name: some-vendor-hal
+  #       remote: zephyrproject-rtos
+  #       path: ext/hal/some-vendor
+  projects:
+    required: true
+    type: seq
+    sequence:
+      - type: map
         mapping:
+          name:
+            required: true
+            type: str
           remote:
             required: false
             type: str
           revision:
+            required: false
+            type: text        # SHAs could be only numbers
+          path:
             required: false
             type: str
-
-      # The "remotes" key specifies a sequence of remotes, each of
-      # which has a name and a fetch URL.
-      #
-      # These work like repo remotes, in that they specify a URL
-      # prefix which remote-specific Git repositories hang off of.
-      # (This saves typing and makes it easier to move things around
-      # when most repositories are on the same server or GitHub
-      # organization.)
-      #
-      # Example:
-      #
-      #   remotes:
-      #     - name: zephyrproject-rtos
-      #       url: https://github.com/zephyrproject-rtos
-      #     - name: developer-fork
-      #       url: https://github.com/a-developer
-      remotes:
-        required: true
-        type: seq
-        sequence:
-          - type: map
-            mapping:
-              name:
-                required: true
-                type: str
-              url:
-                required: true
-                type: str
-
-      # The "projects" key specifies a sequence of "projects",
-      # i.e. Git repositories. These work like repo projects, in that
-      # each project has a name, a remote, and optional additional
-      # metadata.
-      #
-      # Each project is a map with the following keys:
-      #
-      # - name: Mandatory, the name of the git repository. The clone
-      #   URL is formed by remote + '/' + name. The name cannot
-      #   be one of the reserved values "west" and "manifest".
-      # - remote: Optional, the name of the remote to pull it from.
-      #   If the remote is missing, the remote'key in the top-level
-      #   defaults key is used instead. If both are missing, it's an error.
-      # - revision: Optional, the name of the revision to check out.
-      #   If not given, the value from the default element will be used.
-      #   If both are missing, then the default is 'master'.
-      # - path: Where to clone the repository locally. If missing,
-      #   it's cloned at top level in a directory given by its name.
-      # - clone-depth: if given, it is a number which creates a shallow
-      #   history in the cloned repository limited to the given number
-      #   of commits.
-      #
-      # Example, using default and non-default remotes:
-      #
-      #   projects:
-      #     # Uses default remote (zephyrproject-rtos), so clone URL is:
-      #     #   https://github.com/zephyrproject-rtos/zephyr
-      #     - name: zephyr
-      #     # Manually specified remote; clone URL is:
-      #     #   https://github.com/a-developer/west
-      #     - name: west
-      #       remote: developer-fork
-      #     # Manually specified remote, clone URL is:
-      #     #   https://github.com/zephyrproject-rtos/some-vendor-hal
-      #     # Local clone path (relative to installation root) is:
-      #     #   ext/hal/some-vendor
-      #     - name: some-vendor-hal
-      #       remote: zephyrproject-rtos
-      #       path: ext/hal/some-vendor
-      projects:
-        required: true
-        type: seq
-        sequence:
-          - type: map
-            mapping:
-              name:
-                required: true
-                type: str
-              remote:
-                required: false
-                type: str
-              revision:
-                required: false
-                type: text        # SHAs could be only numbers
-              path:
-                required: false
-                type: str
-              clone-depth:
-                required: false
-                type: int
+          clone-depth:
+            required: false
+            type: int
diff --git a/src/west/manifest.py b/src/west/manifest.py
index 8cd6d69..0f8a3ed 100644
--- a/src/west/manifest.py
+++ b/src/west/manifest.py
@@ -23,11 +23,19 @@ import yaml
 
 from west import util, log
 
+# Todo: take from _bootstrap?
+# Default west repository URL.
+WEST_URL_DEFAULT = 'https://github.com/zephyrproject-rtos/west'
+# Default revision to check out of the west repository.
+WEST_REV_DEFAULT = 'master'
 
 META_NAMES = ['west', 'manifest']
 '''Names of the special "meta-projects", which are reserved and cannot
 be used to name a project in the manifest file.'''
 
+MANIFEST_SECTIONS = ['manifest', 'west']
+'''Sections in the manifest file'''
+
 
 def default_path():
     '''Return the path to the default manifest in the west directory.
@@ -43,10 +51,12 @@ class Manifest:
     from_file and from_data helper methods.'''
 
     @staticmethod
-    def from_file(source_file=None):
+    def from_file(source_file=None, sections=MANIFEST_SECTIONS):
         '''Create and return a new Manifest object given a source YAML file.
 
         :param source_file: Path to a YAML file containing the manifest.
+        :param sections: Only parse specified sections from YAML file, 
+                         default: all sections are parsed.
 
         If source_file is None, the value returned by default_path()
         is used.
@@ -54,22 +64,26 @@ class Manifest:
         Raises MalformedManifest in case of validation errors.'''
         if source_file is None:
             source_file = default_path()
-        return Manifest(source_file=source_file)
+        return Manifest(source_file=source_file, sections=sections)
 
     @staticmethod
-    def from_data(source_data):
+    def from_data(source_data, sections=MANIFEST_SECTIONS):
         '''Create and return a new Manifest object given parsed YAML data.
 
         :param source_data: Parsed YAML data as a Python object.
+        :param sections: Only parse specified sections from YAML data, 
+                         default: all sections are parsed.
 
         Raises MalformedManifest in case of validation errors.'''
-        return Manifest(source_data=source_data)
+        return Manifest(source_data=source_data, sections=sections)
 
-    def __init__(self, source_file=None, source_data=None):
+    def __init__(self, source_file=None, source_data=None, sections=MANIFEST_SECTIONS):
         '''Create a new Manifest object.
 
         :param source_file: Path to a YAML file containing the manifest.
         :param source_data: Parsed YAML data as a Python object.
+        :param sections: Only parse specified sections from YAML file, 
+                         default: all sections are parsed.
 
         Normally, it is more convenient to use the `from_file` and
         `from_data` convenience factories than calling the constructor
@@ -97,13 +111,18 @@ class Manifest:
         if not self._data:
             self._malformed('manifest contains no data')
 
-        try:
-            pykwalify.core.Core(
-                source_data=self._data,
-                schema_files=[_SCHEMA_PATH]
-            ).validate()
-        except pykwalify.errors.SchemaError as e:
-            self._malformed(e)
+        if 'manifest' not in self._data:
+            self._malformed('manifest contains no manifest element')
+
+        for key in self._data:
+            if key in sections:
+                try:
+                    pykwalify.core.Core(
+                        source_data=self._data[key],
+                        schema_files=[_SCHEMA_PATH[key]]
+                    ).validate()
+                except pykwalify.errors.SchemaError as e:
+                    self._malformed(e, key)
 
         self.defaults = None
         '''west.manifest.Defaults object representing default values
@@ -120,29 +139,51 @@ class Manifest:
         Each element's values are fully initialized; there is no need
         to consult the defaults field to supply missing values.'''
 
+
+        self.west_project = None
+        '''west.manifest.SpecialProject object representing the west meta
+        project.'''
+
         # Set up the public attributes documented above, as well as
         # any internal attributes needed to implement the public API.
-        self._load(self._data['manifest'])
+        self._load(self._data, sections)
 
     def get_remote(self, name):
         '''Get a manifest Remote, given its name.'''
         return self._remotes_dict[name]
 
-    def _malformed(self, complaint):
+    def _malformed(self, complaint, section='manifest'):
         context = (' file {} '.format(self.path) if self.path
                    else ' data:\n{}\n'.format(self._data))
         raise MalformedManifest('Malformed manifest{}(schema: {}):\n{}'
-                                .format(context, _SCHEMA_PATH, complaint))
+                                .format(context, _SCHEMA_PATH[section], complaint))
+
 
-    def _load(self, manifest):
+    def _load(self, data, sections):
         # Initialize this instance's fields from values given in the
         # manifest data, which must be validated according to the schema.
+        if 'west' in sections:
+            west = data.get('west', {})
+
+            url = west.get('url') or WEST_URL_DEFAULT
+            revision = west.get('revision') or WEST_REV_DEFAULT
+
+            self.west_project = SpecialProject('west',
+                                               url=url,
+                                               revision=revision,
+                                               path=os.path.join('west', 'west'))
+
+        # Next is the manifest section
+        if 'manifest' not in sections:
+            return
 
         projects = []
         project_abspaths = set()
 
+        manifest = data.get('manifest')
+
         # Map from each remote's name onto that remote's data in the manifest.
-        remotes = tuple(Remote(r['name'], r['url']) for r in
+        remotes = tuple(Remote(r['name'], r['url-base']) for r in
                         manifest['remotes'])
         remotes_dict = {r.name: r for r in remotes}
 
@@ -271,7 +312,7 @@ class Project:
     __slots__ = 'name remote url path abspath clone_depth revision'.split()
 
     def __init__(self, name, remote, defaults, path=None, clone_depth=None,
-                 revision=None):
+                 revision=None, url=None):
         '''Specify a Project by name, Remote, and optional information.
 
         :param name: Project's user-defined name in the manifest
@@ -283,7 +324,7 @@ class Project:
         :param revision: Project revision as given in the manifest, if present.
         '''
         if remote is None:
-            raise ValueError('remote may not be None')
+            raise ValueError('remote or url must be given')
         _wrn_if_not_remote(remote)
 
         self.name = name
@@ -304,10 +345,33 @@ class Project:
         return ('Project(name={}, remote={}, url={}, path={}, abspath={}, '
                 'clone_depth={}, revision={})').format(*reprs)
 
+class SpecialProject(Project):
+    '''Represents a special project, e.g. the west or manifest project.
+
+    Projects are neither comparable nor hashable.'''
+
+    def __init__(self, name, path=None, revision=None, url=None):
+        '''Specify a Special Project by name, and url, and optional information.
+
+        :param name: Special Project's user-defined name in the manifest
+        :param path: Relative path to the project in the west
+                     installation, if present in the manifest. If None,
+                     the project's ``name`` is used.
+        :param revision: Project revision as given in the manifest, if present.
+        '''
+        self.name = name
+        self.url = url
+        self.path = path or name
+        self.abspath = os.path.realpath(os.path.join(util.west_topdir(), self.path))
+        self.revision = revision
+        self.remote = None 
+        self.clone_depth = None 
+
 
 def _wrn_if_not_remote(remote):
     if not isinstance(remote, Remote):
         log.wrn('Remote', remote, 'is not a Remote instance')
 
 
-_SCHEMA_PATH = os.path.join(os.path.dirname(__file__), "manifest-schema.yml")
+_SCHEMA_PATH = {'manifest': os.path.join(os.path.dirname(__file__), "manifest-schema.yml"),
+                'west': os.path.join(os.path.dirname(__file__), "_bootstrap", "west-schema.yml")}


Test patch:
diff --git a/tests/west/manifest/invalid_bad_default_remote.yml b/tests/west/manifest/invalid_bad_default_remote.yml
index 60e2021..070c5a0 100644
--- a/tests/west/manifest/invalid_bad_default_remote.yml
+++ b/tests/west/manifest/invalid_bad_default_remote.yml
@@ -4,7 +4,7 @@ manifest:
 
   remotes:
     - name: testremote
-      url: https://example.com
+      url-base: https://example.com
 
   projects:
     - name: foo
diff --git a/tests/west/manifest/invalid_duplicate_project_path.yml b/tests/west/manifest/invalid_duplicate_project_path.yml
index 1fbe5b6..2a4e37b 100644
--- a/tests/west/manifest/invalid_duplicate_project_path.yml
+++ b/tests/west/manifest/invalid_duplicate_project_path.yml
@@ -4,7 +4,7 @@ manifest:
 
   remotes:
     - name: testremote
-      url: https://example.com
+      url-base: https://example.com
 
   projects:
     - name: foo1
diff --git a/tests/west/manifest/invalid_ill_defined_project.yml b/tests/west/manifest/invalid_ill_defined_project.yml
index 9476597..b735542 100644
--- a/tests/west/manifest/invalid_ill_defined_project.yml
+++ b/tests/west/manifest/invalid_ill_defined_project.yml
@@ -1,7 +1,7 @@
 manifest:
   remotes:
     - name: testremote
-      url: https://example.com
+      url-base: https://example.com
 
   projects:
     - name: ill_defined_no_remote
diff --git a/tests/west/manifest/invalid_no_projects.yml b/tests/west/manifest/invalid_no_projects.yml
index 9cce29d..c0f7706 100644
--- a/tests/west/manifest/invalid_no_projects.yml
+++ b/tests/west/manifest/invalid_no_projects.yml
@@ -1,4 +1,4 @@
 manifest:
   remotes:
     - name: testremote
-      url: https://example.com
+      url-base: https://example.com
diff --git a/tests/west/manifest/invalid_reserved_name_manifest.yml b/tests/west/manifest/invalid_reserved_name_manifest.yml
index 3bc58e1..98548d3 100644
--- a/tests/west/manifest/invalid_reserved_name_manifest.yml
+++ b/tests/west/manifest/invalid_reserved_name_manifest.yml
@@ -1,7 +1,7 @@
 manifest:
   remotes:
     - name: testremote
-      url: https://example.com
+      url-base: https://example.com
 
   projects:
     - name: manifest
diff --git a/tests/west/manifest/invalid_reserved_name_west.yml b/tests/west/manifest/invalid_reserved_name_west.yml
index 7da8cdb..1f9e6f5 100644
--- a/tests/west/manifest/invalid_reserved_name_west.yml
+++ b/tests/west/manifest/invalid_reserved_name_west.yml
@@ -1,7 +1,7 @@
 manifest:
   remotes:
     - name: testremote
-      url: https://example.com
+      url-base: https://example.com
 
   projects:
     - name: west
diff --git a/tests/west/manifest/invalid_too_flat.yml b/tests/west/manifest/invalid_too_flat.yml
index a3dc610..14959b9 100644
--- a/tests/west/manifest/invalid_too_flat.yml
+++ b/tests/west/manifest/invalid_too_flat.yml
@@ -1,7 +1,7 @@
 # This invalid manifest is too flat.
 remotes:
   - name: testremote
-    url: https://example.com
+    url-base: https://example.com
 projects:
   - name: testproject
     remote: testremote
diff --git a/tests/west/manifest/invalid_west_section.yml b/tests/west/manifest/invalid_west_section.yml
new file mode 100644
index 0000000..cb942b6
--- /dev/null
+++ b/tests/west/manifest/invalid_west_section.yml
@@ -0,0 +1,20 @@
+# The default West manifest file to be used for Zephyr.
+#
+
+west:
+  url: https://example.com
+  revision: arevision
+  invalidfield: thisfieldisinvalid
+
+manifest:
+  defaults:
+    remote: upstream
+    revision: master
+
+  remotes:
+    - name: upstream
+      url-base: https://example.com
+
+  projects:
+    - name: zephyr
+      revision: abranch
diff --git a/tests/west/manifest/test_manifest.py b/tests/west/manifest/test_manifest.py
index 23a3858..c7c5c7d 100644
--- a/tests/west/manifest/test_manifest.py
+++ b/tests/west/manifest/test_manifest.py
@@ -33,9 +33,9 @@ def test_no_defaults():
     manifest:
       remotes:
         - name: testremote1
-          url: https://example1.com
+          url-base: https://example1.com
         - name: testremote2
-          url: https://example2.com
+          url-base: https://example2.com
 
       projects:
         - name: testproject1
@@ -74,9 +74,9 @@ def test_default_clone_depth():
 
       remotes:
         - name: testremote1
-          url: https://example1.com
+          url-base: https://example1.com
         - name: testremote2
-          url: https://example2.com
+          url-base: https://example2.com
 
       projects:
         - name: testproject1
@@ -116,7 +116,7 @@ def test_path():
     manifest:
       remotes:
         - name: testremote
-          url: https://example.com
+          url-base: https://example.com
       projects:
         - name: testproject
           remote: testremote
@@ -127,6 +127,45 @@ def test_path():
     assert manifest.projects[0].path == 'sub/directory'
     assert manifest.projects[0].abspath == os.path.realpath('/west_top/sub/directory')
 
+def test_sections():
+    # Projects must be able to override their default paths.
+    content_wrong_west = '''\
+    west:
+      url: https://example.com
+      revision: abranch
+      wrongfield: avalue
+    manifest:
+      remotes:
+        - name: testremote
+          url-base: https://example.com
+      projects:
+        - name: testproject
+          remote: testremote
+          path: sub/directory
+    '''
+    with patch('west.util.west_topdir', return_value=os.path.realpath('/west_top')):
+        # Parsing manifest only, no exception raised
+        manifest = Manifest.from_data(yaml.safe_load(content_wrong_west), 'manifest')
+    assert manifest.projects[0].path == 'sub/directory'
+    assert manifest.projects[0].abspath == os.path.realpath('/west_top/sub/directory')
+    content_wrong_manifest = '''\
+    west:
+      url: https://example.com
+      revision: abranch
+    manifest:
+      remotes:
+        - name: testremote
+          url-base: https://example.com
+      projects:
+        - name: testproject
+          remote: testremote
+          path: sub/directory
+    '''
+    with patch('west.util.west_topdir', return_value=os.path.realpath('/west_top')):
+        # Parsing west section only, no exception raised
+        manifest = Manifest.from_data(yaml.safe_load(content_wrong_manifest), 'west')
+    assert manifest.west_project.url == 'https://example.com'
+    assert manifest.west_project.revision == 'abranch'
 
 # Invalid manifests should raise MalformedManifest.
 @pytest.mark.parametrize('invalid',
diff --git a/tests/west/project/test_project.py b/tests/west/project/test_project.py
index cb98b0a..1d57cd7 100644
--- a/tests/west/project/test_project.py
+++ b/tests/west/project/test_project.py
@@ -76,7 +76,7 @@ manifest:
 
   remotes:
     - name: repos
-      url: file://{}
+      url-base: file://{}
 
   projects:
     - name: net-tools
@@ -294,14 +294,17 @@ def test_update(clean_west_topdir):
 [manifest]
 remote = {0}/remote-repos/manifest
 revision = master
-
-[west]
-remote = {0}/remote-repos/west
-revision = master
 '''.format(clean_west_topdir))
 
     config.read_config()
 
+    # modify the manifest to point to another west
+    clean_west_topdir.join('manifest.yml').write('''
+west:
+  url: file://{}/remote-repos/west
+  revision: master
+'''.format(clean_west_topdir), 'a')
+
     # Fetch the net-tools repository
     cmd('fetch --no-update net-tools')
 
@@ -360,16 +363,10 @@ def test_bootstrap_reinit(clean_west_topdir, monkeypatch):
         bootstrap.init([])  # West already initialized
 
     for init_args, west_args in (
-        (['-m',   'foo'], ['update', '--reset-manifest', '--reset-projects']),
-        (['--mr', 'foo'], ['update', '--reset-manifest', '--reset-projects']),
-        (['-w',   'foo'], ['update', '--reset-west']),
-        (['--wr', 'foo'], ['update', '--reset-west']),
-        (['--wr', 'foo'], ['update', '--reset-west']),
-        (['-m',   'foo', '-w', 'foo'],
-         ['update', '--reset-manifest', '--reset-projects', '--reset-west']),
-        (['-b',   'foo'],
-         ['update', '--reset-manifest', '--reset-projects', '--reset-west']),
-        (['-b',   'foo', '--no-reset'], [])):
+        (['-m',   'foo'], ['update', '--reset-manifest', '--reset-projects',
+                           '--reset-west']),
+        (['--mr', 'foo'], ['update', '--reset-manifest', '--reset-projects',
+                           '--reset-west'])):
 
         # Reset wrap_args before each test so that it ends up as [] if wrap()
         # isn't called (for the --no-reset case)