Generate manifest file

The manifest file stores important infomation such as image file name,
package format, compress format, partitions or disks. So when another program
want to parse image files, it can get the infomation more easily by reading
manifest file. There are two important kinds of format, see bellow:
loop:
   {
       "format": "loop",
       "version": "0.24",
       "loop": {
           "image_files": [
               "whole_rd_210-201403211800.tar.gz"
           ],
           "pack": ".tar.gz",
           "partitions": {
              "/opt": "data",
              "/opt/usr": "ums",
              "/": "platform"
          }
      },
      "created": "2014-03-21 18:02:07"
  }
raw:
   {
       "raw": {
          "image_files": [
               "ivi-201403211803-sda.raw.bz2",
               "ivi-201403211803-sdb.raw.bz2"
           ],
           "disks": [
               "sda",
               "sdb"
          ],
          "compress": "bz2",
          "sda": {
              "image": "ivi-201403211803-sda.raw.bz2",
              "bmap": "ivi-201403211803-sda.bmap"
          },
          "sdb": {
              "image": "ivi-201403211803-sdb.raw.bz2",
              "bmap": "ivi-201403211803-sdb.bmap"
          }
      },
      "format": "raw",
      "version": "0.24",
      "created": "2014-03-21 18:06:10"
  }
Change-Id: Ie9682bad79337319d1c1e6534b3cd4db25271124
Fixes: #1583
typo fix

Change-Id: Id8a274534f299381453dda400308a1073c6f4b99
Add lzop compress format

Add '.lzo' compression format to option '--compress-image' and
'--compress-disk-image' of 'raw' and 'loop' type image.

Fixes: #1282

Change-Id: Ic19faa2b9c2ee13a74a6cd2ce9153ce7feea276c
add rewritten moudle archive to handle archiving and compressing

Change-Id: I9fd94ca3b3d3460ecb2af618baa329f04a6a3bfb
Signed-off-by: Gui Chen <gui.chen@intel.com>
switch to use new archive module

Change-Id: Ibe1f956b178952f150b4d2285d3642dc73a06bb5
Signed-off-by: Gui Chen <gui.chen@intel.com>
clean up unused code

Change-Id: I483e4253437a3fa9905ff354f7739b15f71ec2c2
Signed-off-by: Gui Chen <gui.chen@intel.com>
add _do_tar and _do_untar to utilize 'tar' command

Change-Id: Iefc0ca78636e07ffe125560c3336031c5be4f9fb
Signed-off-by: Gui Chen <gui.chen@intel.com>
Pylint error of partitionedfs.py and proxy.py

C: 44,8:unset_proxy_environ: Operator not preceded by a space
        ENV=env.upper()
           ^
C: 44,8:unset_proxy_environ: Invalid name "ENV" (should match [a-z_][a-z0-9_]{2,30}$)
C: 77,4:_ip_to_int: Operator not preceded by a space
    ipint=0
         ^
C: 78,4:_ip_to_int: Operator not preceded by a space
    shift=24
C: 85,4:_int_to_ip: Operator not preceded by a space
    ipaddr=""
          ^
C: 86,4:_int_to_ip: Operator not preceded by a space
    shift=0
         ^
C:111,12:_set_noproxy_list: Comma not followed by a space
            _my_noproxy_list.append({"match":0,"needle":item})
                                              ^^
C:115,12:_set_noproxy_list: Comma not followed by a space
            _my_noproxy_list.append({"match":1,"needle":item})
                                              ^^
C:127,16:_set_noproxy_list: Invalid name "ip" (should match [a-z_][a-z0-9_]{2,30}$)
C:129,16:_set_noproxy_list: Operator not preceded by a space
                shift=24
                     ^
C:130,16:_set_noproxy_list: Operator not preceded by a space
                netmask=0
                       ^
C:136,12:_set_noproxy_list: Comma not followed by a space
            _my_noproxy_list.append({"match":2,"needle":ip,"netmask":netmask})

Change-Id: Iaf24729b3231b9f313af333fe58096d234d3e515
Pylint warnings.

C:139,8:Zypp._zyppQueryPackage: Comma not followed by a space
        query.addAttribute(zypp.SolvAttr.name,pkg)
                                             ^^
C:146,8:Zypp._splitPkgString: Comma not followed by a space
        sp = pkg.rsplit(".",1)
                           ^^
C:213,12:Zypp.selectPackage: Comma not followed by a space
            q.addAttribute(zypp.SolvAttr.name,pattern)
                                             ^^
C:217,12:Zypp.selectPackage: Comma not followed by a space
            q.addAttribute(zypp.SolvAttr.name,name)
                                             ^^
C:221,12:Zypp.selectPackage: Comma not followed by a space
            q.addAttribute(zypp.SolvAttr.name,pkg)
                                             ^^
W: 19,0: Unused import sys
W: 35,0: Unused import BaseImageCreator
W: 20,0: Unused import re

Change-Id: Ie6bdff4de495e93b0dd534c276cb77d5270db114
avoid to compress/decompress a non-file object

Change-Id: I0c43f449175325ba9314e56151a822b3aee0ec21
Signed-off-by: Gui Chen <gui.chen@intel.com>
archive.py: auto detect format in decompression and refine get_xxx_formats/get_xxx_suffixes

it will detect the format automatically when decompressing
add get_compress_suffixes to list all supported suffixes in compression
clear get_compress_formats
clear get_archive_formats

Change-Id: I1fb2117e4b04c382836b81760bb423cc62f2fd5e
Signed-off-by: Gui Chen <gui.chen@intel.com>
archive.py: refine extract_archive to extract tarball file

Change-Id: I5af9f17de9d1ac5d12bf1706ce7609ea4c4e201f
Signed-off-by: Gui Chen <gui.chen@intel.com>
typo fix

Change-Id: I0bc9288562694f3a12bb6cf46ffa5c0cadebd236
Signed-off-by: Gui Chen <gui.chen@intel.com>
Fix pylint tips

Change-Id: Ic0805cc13abc16afd1881c7513edf2f9f980d0fc
C: 58,0: Line too long (87/80)
W: 20,24: Redefining built-in 'id'
C:  1,0: Missing docstring
C: 11,4: Invalid name "version" for type constant (should match (([A-Z_][A-Z0-9_]*)|(__.*__))$)
C: 13,4: Invalid name "version" for type constant (should match (([A-Z_][A-Z0-9_]*)|(__.*__))$)
W: 26,8: No exception type(s) specified
C: 20,13: Invalid name "dist" for type constant (should match (([A-Z_][A-Z0-9_]*)|(__.*__))$)
C: 20,19: Invalid name "ver" for type constant (should match (([A-Z_][A-Z0-9_]*)|(__.*__))$)
C: 20,24: Invalid name "id" for type constant (should match (([A-Z_][A-Z0-9_]*)|(__.*__))$)
C: 43,0: Invalid name "prefix" for type constant (should match (([A-Z_][A-Z0-9_]*)|(__.*__))$)
C: 45,0: Invalid name "root" for type constant (should match (([A-Z_][A-Z0-9_]*)|(__.*__))$)
C: 47,0: Invalid name "conffile" for type constant (should match (([A-Z_][A-Z0-9_]*)|(__.*__))$)
C: 49,0: Invalid name "conf_str" for type constant (should match (([A-Z_][A-Z0-9_]*)|(__.*__))$)
C: 50,0: Invalid name "conf_str" for type constant (should match (([A-Z_][A-Z0-9_]*)|(__.*__))$)
Merge "Pylint warnings." into devel
Merge "Pylint error of partitionedfs.py and proxy.py" into devel
Add two output info to avoid duplicat "Checking filesystem"

"loop", "liveusb", "livecd" image creation logs all have duplicate
info of "Checking filesystem", add more info to avoid misunderstanding.

Change-Id: I416e71cd312ae5cad23468bb2d7d191358bf4309
more check before making archive

Change-Id: I5a205a06d354e559e7c62767661738a59585788b
Signed-off-by: Gui Chen <gui.chen@intel.com>
Merge "more check before making archive" into devel
more check before extracting archive

Change-Id: Ieca43aa54cd384392b288aadd66c3e443eaca701