dfu: free entities when parsing fails

When dfu_init_env_entities() fails part-way through, some entities may
have been added to dfu_list. These are only removed by dfu_free_entities().
If that function isn't called, those stale entities will still exist the
next time dfu_init_env_entities() is called, leading to confusion. Fix
do_dfu() to ensure that dfu_free_entities() is always called, to avoid
this confusion.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Lukasz Majewski <l.majewski@samsung.com>
Tested-by: Lukasz Majewski <l.majewski@samsung.com>
test:dfu: Add test scripts for testing DFU regression

This commit adds test scripts for testing if any commit has introduced
regression to the DFU subsystem.

It uses md5 to test if sent and received file is correct.
The test detailed description is available at README file.

Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
Signed-off-by: Stephen Warren <swarren@nvidia.com>
test: dfu: script enhancements

Various misc enhancements to dfu_gadget_test.sh:

* After every write (download), perform a write to a different file
  with different data. This ensures that the DFU buffer's content is
  replaced, so that if the read (upload) succeeds, we know that the
  correct data was actually read from the storage device, rather than
  simply being left over in the DFU buffer. This requires two alt
  setting names to be passed to the script, and a dummy data file to
  be generated by dfu_gadget_test_init.sh.

* Fix the assumption that dfu_gadget_test.sh is run from the directory
  that contains it, by cd'ing to that directory before invoking
  ./dfu_gadget_test_init.sh.

* Use $DIR$RCV_DIR consistently, rather than using plain $RCV_DIR in
  some places.

* Add 959, 961 test file sizes, to be consistent with having one
  more than and one less than all the other "round" sizes 64, 128, and
  4096.

* Remove references to $BKP_DIR from dfu_gadget_test_init.sh, since it
  isn't used.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
test: vboot: explicitly request bash

vboot_test.sh uses Bashisms. Explicitly use #!/bin/bash so the script
doesn't fail if /bin/sh isn't Bash.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
sandbox: restore ability to access host fs through standard commands

Commit 95fac6ab4589 "sandbox: Use os functions to read host device tree"
removed the ability for get_device_and_partition() to handle the "host"
device type, and redirect accesses to it to the host filesystem. This
broke some unit tests that use this feature. So, revert that change. The
code added back by this patch is slightly different to pacify checkpatch.

However, we're then left with "host" being both:
- A pseudo device that accesses the hosts real filesystem.
- An emulated block device, which accesses "sectors" inside a file stored
  on the host.

In order to resolve this discrepancy, rename the pseudo device from host
to hostfs, and adjust the unit-tests for this change.

The "help sb" output is modified to reflect this rename, and state where
the host and hostfs devices should be used.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Josh Wu <josh.wu@atmel.com>
Acked-by: Simon Glass <sjg@chromium.org>
Tesed-by: Simon Glass <sjg@chromium.org>
---
V3:
* Fix test/vboot/vboot_test.sh for the host->hostfs rename.
V2:
* Fix typo due to fixing checkpatch and not recompiling:-(
* Fix "help sb" output.
sandbox: terminate os_dirent_ls() result list

Each node in the linked-list that os_dirent_ls() returns has its next
pointer set only when the next node is created. For the last node in the
list, there is no next node, so this never happens, and the next pointer
is never initialized. Explicitly initialize the next pointer so that it
isn't dangling. Without this, "sb ls" might crash.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
---
v2: Initialize node->next when it's created, rather than deferring it to
the end of the loop. This prevents an attempt to initialize node->next
if node was never created, since the loop never ran.
fs: ext4: fix writing zero-length files

ext4fs_allocate_blocks() always allocates at least one block for a file.
If the file size is zero, this causes total_remaining_blocks to
underflow, which then causes an apparent hang while 2^32 blocks are
allocated.

To solve this, check that total_remaining_blocks is non-zero as part of
the loop condition (i.e. before each loop) rather than at the end of
the loop.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
fs: implement size/fatsize/ext4size

These commands may be used to determine the size of a file without
actually reading the whole file content into memory. This may be used
to determine if the file will fit into the memory buffer that will
contain it. In particular, the DFU code will use it for this purpose
in the next commit.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Lukasz Majewski <l.majewski@samsung.com>
Tested-by: Lukasz Majewski <l.majewski@samsung.com>
---
v2: New patch
dfu: fix some issues with reads/uploads

DFU read support appears to rely upon dfu->read_medium() updating the
passed-by-reference len parameter to indicate the remaining size
available for reading.

dfu_read_medium_mmc() never does this, and the implementation of
dfu_read_medium_nand() will only work if called just once; it hard-codes
the value to the total size of the NAND device irrespective of read
offset.

I believe that overloading dfu->read_medium() is confusing. As such,
this patch introduces a new function dfu->get_medium_size() which can
be used to explicitly find out the medium size, and nothing else.
dfu_read() is modified to use this function to set the initial value for
dfu->r_left, rather than attempting to use the side-effects of
dfu->read_medium() for this purpose.

Due to this change, dfu_read() must initially set dfu->b_left to 0, since
no data has been read.

dfu_read_buffer_fill() must also be modified not to adjust dfu->r_left
when simply copying data from dfu->i_buf_start to the upload request
buffer. r_left represents the amount of data left to be read from HW.
That value is not affected by the memcpy(), but only by calls to
dfu->read_medium().

After this change, I can read from either a 4MB or 1.5MB chunk of a 4MB
eMMC boot partion with CONFIG_SYS_DFU_DATA_BUF_SIZE==1MB. Without this
change, attempting to do that would result in DFU read returning no data
at all due to r_left never being set.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Lukasz Majewski <l.majewski@samsung.com>
Tested-by: Lukasz Majewski <l.majewski@samsung.com>
---
v2:
* Fix dfu_get_medium_size_mmc() to handle DFU_FS_FAT/EXT5 layouts too,
  by calling the recently introduced "size" command.
dfu: add write error handling

Fix calls to dfu_write() and dfu_flush() to detect errors in the I/O
itself. This could happen due to problems with the storage medium, or
simply when trying to write a FAT/ext file that is larger than the buffer
dfu_mmc.c maintains for this purpose.

Signal the error by switching the DFU state/status. This will be picked
up by the DFU client when it sends the next DFU request. Note that errors
can't simply be returned from e.g. dnload_request_complete(), since that
function has no way to pass errors back to the DFU client; a call to
dnload_request_complete() simply means that a USB OUT completed.

This error state/status needs to be cleared when the next DFU client
connects. While there is a DFU_CLRSTATUS request, no DFU client seems to
send this. Hence, clear this when selecting the USB alternate setting on
the USB interface.

Finally, dfu.c relies on a call to dfu_flush() to clear up the internal
state of the write transaction. Now that errors in dfu_write() are
detected, dfu_flush() may no longer be called for every transaction.
Separate out the cleanup code into a new function, and call it whenever
dfu_write() fails, as well as from any call to dfu_flush().

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Lukasz Majewski <l.majewski@samsung.com>
Tested-by: Lukasz Majewski <l.majewski@samsung.com>
---
This probably depends on "dfu: fix some issues with reads/uploads",
for context if nothing else.
dfu: defer parsing of device string to IO backend

Devices are not all identified by a single integer. To support
this, defer the parsing of the device string to the IO backed, so that
it can apply the appropriate rules.

SPI devices are specified as controller:chip_select. SPI/SF support will
be added soon.

MMC devices can also be specified as controller[.hwpart][:partition] in
many commands, although we don't support that syntax in DFU.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
dfu: allow backend to specify a maximum buffer size

CONFIG_SYS_DFU_DATA_BUF_SIZE may be large to allow for FAT/ext layouts
to transfer large files. However, this means that individual write
operations will take a long time. Allow backends to specify a maximum
buffer size, so that each write operation is limited to a smaller data
block. This prevents the DFU protocol from timing out when e.g. writing
to SPI flash. I would guess that NAND might benefit from setting this
value too, but I can't test that.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
dfu: add free_entity() to struct dfu_entity

This allows the backend to free any resources allocated during the
relevant dfu_fill_entity_*() call. This will soon be used by the
SF backend.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
dfu: add SF backend

This allows SPI Flash to be programmed using DFU.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
ARM: tegra: enable DFU too

Enable DFU protocol support (via the "dfu" command) on Tegra boards where
USB device/gadget mode is enabled.

This relies on a number of fixes to the ci_udc HW driver and DFU protocol
code.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
TEMP HACK: enable ext4 write for DFU testing