January 11, 2002

Notes on dd and Odd Sized Disks

By Dr. James Lyle

cftt@nist.gov

National Institute of Standards and Technology (NIST)

I've been asked to comment on digests 105-107 since I'm directly
involved in testing **dd** and I may be able to add a little bit about
this. These digests contain good questions; most of them should be
answered by our test report when it comes out. Feel free to contact us
with suggestions and comments. We are not working alone; we have regular
discussions with a number of forensic practitioners. These discussions
often lead to productive exchanges for both parties. If anyone has time,
please try to duplicate these results and let us know what happens.

# Disks with an odd number of sectors

## Question

The message that started the **dd** thread was:

Subject: Re: dd missing sectors - never in my experience

I have been thinking about how to get dd to miss sectors and I am now
really confused. The claim is that dd misses the last sector on disks
with odd numbers of sectors, but I can\'t find any disks that have odd
numbers of sectors and have not been able to reproduce these results. We
need specifics on how to reproduce this. Does anyone have a concrete
example?

## Response

The claim actually is that dd (GNU fileutils) 4.0.36 running on Red Hat
Linux Version 7.1 misses the last sector from a *source device* that has
an odd number of sectors. A source device may be either a disk drive
(e.g., **/dev/hda** or **/dev/sdc**) or a partition (e.g., **/dev/hdb3**
or **/dev/sda6**).

It is no surprise that finding a disk with an odd number of sectors is
hard. The odds are against finding an IDE (ATA) drive with an odd number
of sectors. Since both the number of heads and cylinders would need to
be odd. However a partition with an odd number of sectors is easy to
create if the number of heads is odd. Some drives allow selection of the
number of heads (either 15 or 16) by jumper. Some SCSI disks do have an
odd number of sectors, e.g., QUANTUM ATLAS10K2-TY092J 17938985 or a
SEAGATE ST39204LC 17921835 Sectors. A visit to vendor web sites would
likely turn up more.

Here is an extract from three of our test cases. DI(Linux)-06 is a
partition copy of a partition that just happened to have an odd number
of sectors. DI(Linux)-08 is a copy of a SCSI disk to an identical model
(the selected disk again just happens to have an odd number of sectors).
DI(Linux)-09 is a copy to a smaller destination drive that is *odd
sized*. The last sector is dropped for each case. The last sector of the
destination is not changed from the contents present before **dd** is
executed.

Test case DI(Linux)-06 copies a partition (entry #9 from the partition
table). Note that the partition is on an IBM-DTLA-307020 ATA disk with
an even number of sectors (40188960), but since the disk has 255 heads
(logically not physically) the partition has an odd number of sectors
(9430155). The last sector was not copied.

The dd command line was:

dd if=/dev/hda3 of=/dev/hdb1 bs=1b

Test case DI(Linux)-08 copies an entire disk to another disk of the same
model. The last sector is not copied. (QUANTUM ATLAS10K2-TY092J 17938985
sectors)

Test case DI(Linux)-09 copies part of a disk (QUANTUM ATLAS10K2-TY092J
17938985 sectors) to another disk of a smaller size (SEAGATE ST39204LC
17921835 Sectors). **dd** outputs a message that the destination is too
small, but stops the copy one sector short, i.e., **dd** could have
copied one more sector to the destination drive but it did not do so.
The destination disk has an odd number of sectors.

(The test report will contain formats similar to the following for each
test case.)

+-----------+----------------------------------------------------------+
| Product   | dd (GNU fileutils) 4.0.36                                |
| Name:     |                                                          |
+-----------+----------------------------------------------------------+
| Test ID:  | DI(Linux)-06                                             |
+-----------+----------------------------------------------------------+
| Case      | Copy a LINUX IDE source disk                             |
| Summary:  |                                                          |
|           | to a LINUX IDE destination disk                          |
|           |                                                          |
|           | and the source contains a LINUX partition                |
|           |                                                          |
|           | where the source disk is larger than the destination     |
+-----------+----------------------------------------------------------+
| Tester    | JRL                                                      |
| Name:     |                                                          |
+-----------+----------------------------------------------------------+
| Test      | Thu Aug 30 18:06:47 2001                                 |
| Date:     |                                                          |
+-----------+----------------------------------------------------------+
| PC:       | Cadfael                                                  |
+-----------+----------------------------------------------------------+
| Disks:    | Source: DOS Drive 80 Physical Label F5 Linux device hda3 |
|           |                                                          |
|           | Destination: DOS Drive 81 Physical Label A6 Linux device |
|           | hdb1                                                     |
|           |                                                          |
|           | Boot/Image media: Physical Label CD                      |
|           |                                                          |
|           | F5 is an IBM-DTLA-307020 40188960 Sectors, IDE           |
|           |                                                          |
|           | A6 is a WDC WD200BB-00AUA1 39102336 Sectors, IDE         |
|           |                                                          |
|           | CD is a SEAGATE ST336705LC 71687370 Sectors, SCSI        |
|           |                                                          |
|           | Jaz disk with Support software, scripts                  |
|           |                                                          |
|           | PC DOS 6.3 Boot floppy                                   |
+-----------+----------------------------------------------------------+
| Source    | Dual boot Linux/Windows Me with EXT2 & Fat16             |
| disk      |                                                          |
| setup:    | Disk: F5                                                 |
|           |                                                          |
|           | N Start LBA Length Start C/H/S End C/H/S boot Partition  |
|           | type                                                     |
|           |                                                          |
|           | 1 P 000000063 001236942 0000/001/01 0076/254/63 Boot 06  |
|           | Fat16                                                    |
|           |                                                          |
|           | 2 X 002249100 007181055 0140/000/01 0586/254/63 05       |
|           | extended                                                 |
|           |                                                          |
|           | 3 S 000000063 000208782 0140/001/01 0152/254/63 83 Linux |
|           |                                                          |
|           | 4 x 000208845 000144585 0153/000/01 0161/254/63 05       |
|           | extended                                                 |
|           |                                                          |
|           | 5 S 000000063 000144522 0153/001/01 0161/254/63 06 Fat16 |
|           |                                                          |
|           | 6 x 004450005 000192780 0417/000/01 0428/254/63 05       |
|           | extended                                                 |
|           |                                                          |
|           | 7 S 000000063 000192717 0417/001/01 0428/254/63 16 other |
|           |                                                          |
|           | 9 P 009430155 006152895 0587/000/01 0969/254/63 83 Linux |
|           |                                                          |
|           | 10 P 039760875 000417690 1023/000/01 1023/254/63 82      |
|           | Linux swap                                               |
+-----------+----------------------------------------------------------+
| Expected  | src compares qualified equal to dst, src is truncated on |
| Results:  | dst                                                      |
|           |                                                          |
|           | truncation is logged                                     |
|           |                                                          |
|           | Source disk unchanged                                    |
+-----------+----------------------------------------------------------+
| Actual    | src compares qualified equal to dst, src is truncated on |
| Results:  | dst                                                      |
|           |                                                          |
|           | truncation is logged                                     |
|           |                                                          |
|           | except for 1 sector not copied to destination            |
|           |                                                          |
|           | Source disk is unchanged                                 |
+-----------+----------------------------------------------------------+
| Log File  | Sectors Compared 5943987                                 |
| Hi        |                                                          |
| ghlights: | Sectors Differ 1                                         |
|           |                                                          |
|           | Diffs range: 5943986                                     |
|           |                                                          |
|           | Source (6152895) has 208908 more sectors than            |
|           | destination (5943987)                                    |
|           |                                                          |
|           | Hash after test:                                         |
|           | 83A0002816BBF089F8BE33C41C92C3B5A0F42A54                 |
|           |                                                          |
|           | dd: writing /dev/hdb1: No space left on device           |
+-----------+----------------------------------------------------------+

+-----------+----------------------------------------------------------+
| Product   | dd (GNU fileutils) 4.0.36                                |
| Name:     |                                                          |
+-----------+----------------------------------------------------------+
| Test ID:  | DI(Linux)-08                                             |
+-----------+----------------------------------------------------------+
| Case      | Copy a LINUX SCSI source disk                            |
| Summary:  |                                                          |
|           | to a LINUX SCSI destination disk                         |
|           |                                                          |
|           | where the source disk is the same size as the            |
|           | destination                                              |
+-----------+----------------------------------------------------------+
| Disks:    | Source: DOS Drive 80 Physical Label E3 Linux device sda  |
|           |                                                          |
|           | Destination: DOS Drive 81 Physical Label E0 Linux device |
|           | sdb                                                      |
|           |                                                          |
|           | Boot/Image media: Physical Label AD                      |
|           |                                                          |
|           | E3 is a QUANTUM ATLAS10K2-TY092J 17938985 Sectors, SCSI  |
|           |                                                          |
|           | E0 is a QUANTUM ATLAS10K2-TY092J 17938985 Sectors, SCSI  |
|           |                                                          |
|           | AD is a Maxtor53073H4 60030432 Sectors, IDE              |
|           |                                                          |
|           | Jaz disk with Support software, scripts                  |
|           |                                                          |
|           | PC DOS 6.3 Boot floppy                                   |
+-----------+----------------------------------------------------------+
| Source    | Dual boot Linux/Windows Me with EXT2 & Fat16             |
| disk      |                                                          |
| setup:    | Disk: E3                                                 |
|           |                                                          |
|           | N Start LBA Length Start C/H/S End C/H/S boot Partition  |
|           | type                                                     |
|           |                                                          |
|           | 1 P 000000063 001236942 0000/001/01 0076/254/63 Boot 06  |
|           | Fat16                                                    |
|           |                                                          |
|           | 2 X 002249100 007181055 0140/000/01 0586/254/63 05       |
|           | extended                                                 |
|           |                                                          |
|           | 3 S 000000063 000208782 0140/001/01 0152/254/63 83 Linux |
|           |                                                          |
|           | 4 x 000208845 000144585 0153/000/01 0161/254/63 05       |
|           | extended                                                 |
|           |                                                          |
|           | 5 S 000000063 000144522 0153/001/01 0161/254/63 06 Fat16 |
|           |                                                          |
|           | 6 x 004450005 000192780 0417/000/01 0428/254/63 05       |
|           | extended                                                 |
|           |                                                          |
|           | 7 S 000000063 000192717 0417/001/01 0428/254/63 16 other |
|           |                                                          |
|           | 9 P 009430155 006152895 0587/000/01 0969/254/63 83 Linux |
|           |                                                          |
|           | 10 P 017510850 000417690 1023/000/01 1023/254/63 82      |
|           | Linux swap                                               |
+-----------+----------------------------------------------------------+
| Execute:  | dd if=/dev/sda of=/dev/sdb bs=1b                         |
+-----------+----------------------------------------------------------+
| Expected  | src compares equal to dst                                |
| Results:  |                                                          |
|           | Source disk unchanged                                    |
+-----------+----------------------------------------------------------+
| Actual    | src compares equal to dst                                |
| Results:  |                                                          |
|           | except for 1 sector not copied to destination            |
|           |                                                          |
|           | Source disk is unchanged                                 |
+-----------+----------------------------------------------------------+
| Log File  | Sectors Compared 17938985                                |
| Hi        |                                                          |
| ghlights: | Sectors Differ 1                                         |
|           |                                                          |
|           | Diffs range 17938984                                     |
+-----------+----------------------------------------------------------+

+-----------+----------------------------------------------------------+
| Product   | dd (GNU fileutils) 4.0.36                                |
| Name:     |                                                          |
+-----------+----------------------------------------------------------+
| Test ID:  | DI(Linux)-09                                             |
+-----------+----------------------------------------------------------+
| Case      | Copy a LINUX SCSI source disk                            |
| Summary:  |                                                          |
|           | to a LINUX SCSI destination disk                         |
|           |                                                          |
|           | where the source disk is larger than the destination     |
+-----------+----------------------------------------------------------+
| Disks:    | Source: DOS Drive 80 Physical Label E4 Linux device sda  |
|           |                                                          |
|           | Destination: DOS Drive 81 Physical Label EA Linux device |
|           | sdb                                                      |
|           |                                                          |
|           | Boot/Image media: Physical Label AE                      |
|           |                                                          |
|           | E4 is a QUANTUM ATLAS10K2-TY092J 17938985 Sectors, SCSI  |
|           |                                                          |
|           | EA is a SEAGATE ST39204LC 17921835 Sectors, SCSI         |
|           |                                                          |
|           | AE is a Maxtor53073H4 60030432 Sectors, IDE              |
|           |                                                          |
|           | Jaz disk with Support software, scripts                  |
|           |                                                          |
|           | PC DOS 6.3 Boot floppy                                   |
+-----------+----------------------------------------------------------+
| Source    | Windows 2000 with NTFS & Fat32                           |
| disk      |                                                          |
| setup:    | Disk: E4                                                 |
|           |                                                          |
|           | N Start LBA Length Start C/H/S End C/H/S boot Partition  |
|           | type                                                     |
|           |                                                          |
|           | 1 P 000000063 006152832 0000/001/01 0382/254/63 Boot 0B  |
|           | Fat32                                                    |
|           |                                                          |
|           | 2 X 008193150 009735390 0510/000/01 1023/254/63 0F       |
|           | extended                                                 |
|           |                                                          |
|           | 4 x 002056320 001237005 0638/000/01 0714/254/63 05       |
|           | extended                                                 |
|           |                                                          |
|           | 5 S 000000063 001236942 0638/001/01 0714/254/63 07 NTFS  |
|           |                                                          |
|           | 6 x 005349645 001638630 0843/000/01 0944/254/63 05       |
|           | extended                                                 |
|           |                                                          |
|           | 7 S 000000063 001638567 0843/001/01 0944/254/63 17 other |
|           |                                                          |
|           | 8 x 008498385 001237005 1023/000/01 1023/254/63 05       |
|           | extended                                                 |
|           |                                                          |
|           | 9 S 000000063 001236942 1023/001/01 1023/254/63 1B other |
+-----------+----------------------------------------------------------+
| Execute:  | dd if=/dev/sda of=/dev/sdb bs=1b                         |
+-----------+----------------------------------------------------------+
| Expected  | src compares qualified equal to dst, src is truncated on |
| Results:  | dst                                                      |
|           |                                                          |
|           | truncation is logged                                     |
|           |                                                          |
|           | Source disk unchanged                                    |
+-----------+----------------------------------------------------------+
| Actual    | src compares qualified equal to dst, src is truncated on |
| Results:  | dst                                                      |
|           |                                                          |
|           | truncation is logged                                     |
|           |                                                          |
|           | except for 1 sector not copied to destination            |
|           |                                                          |
|           | Source disk is unchanged                                 |
+-----------+----------------------------------------------------------+
| Log File  | Sectors Compared 17921835                                |
| Hi        |                                                          |
| ghlights: | Sectors Differ 1                                         |
|           |                                                          |
|           | Diffs range 17921834                                     |
|           |                                                          |
|           | Source (17938985) has 17150 more sectors than            |
|           | destination (17921835)                                   |
|           |                                                          |
|           | dd: writing /dev/sdb: No space left on device            |
+-----------+----------------------------------------------------------+

NOTE: "Source compares qualified equal to destination" in the expected
results box denotes that data blocks were copied correctly and that
allowed changes, such as would be found in the partition table
information and boot records, were expected.

# Source of the problem

As some folks have commented, the problem is not in **dd**. Looking at
the **dd** source code is a red herring; the source of the problem is in
a software layer under **dd**, i.e., kernel or driver. This missing
sector issue has implications beyond **dd**. For example, compute an md5
message digest with a DOS based tool and compare the results with
**md5sum** running under Linux. The results are based on different data
(1 more sector is used by the DOS tool) and yield different values.

## Scope of the dd problem

A simple experiment to check if a unix-like OS sees all of a disk is to
run the following command and note the results:

dd if=/dev/*disk_in_question* of=/dev/null bs=1b

For dd (GNU fileutils) 4.0.36 on Red Hat Linux Version 7.1 we get the
following:

QUANTUM ATLAS10K2-TY092J (17938985):

17938984+0 Records in

17938984+0 Records out

SEAGATE ST39204LC 17921835 Sectors

17921834+0 Records in

17921834+0 Records out

For Mandrake Linux Version 7.2 dd (GNU fileutils) 4.0.p we get the same
result:

QUANTUM ATLAS10K2-TY092J (17938985):

17938984+0 Records in

17938984+0 Records out

SEAGATE ST39204LC 17921835 Sectors

17921834+0 Records in

17921834+0 Records out

But for FreeBSD 4.4-Release#0 we get (for the Atlas):

17938985+0 Records in

17938985+0 Records out

9184760320 bytes transferred in 1094.73 Secs

Bottom line: The some operating systems or underlying drivers skip that
last sector, but others see it all.

## Suggested dd problem workaround

If you see **dd** miss that last sector, we recommend that you use
another tool to either capture the last sector. Otherwise, print the
contents in hex and ASCII.

Note that if the disk is on a Linux system, then that sector is unlikely
to contain any data since Linux does not allow the sector to be used. If
the disk comes from a DOS/WIN system with a FAT file system, it is still
unlikely to be used in any cluster (the last cluster would have to be an
exact fit with nothing left over). In fact, it usually would be outside
any partition. I would try to capture that last sector no matter how
unlikely it is to contain data, just to be complete.

## Driver vs parameter

An additional comment on the discussion was the following:

\>\> as far as disks with odd numbers of sectors, it also stands to

\>\> reason, that if you are using any of the linux programs that use
the

\>\> same disk drivers as dd, you will not be able to see disks with
that

\>\> odd sector because it will not realize it is there.

\>

\> If this is the correct assessment. this is not a driver issue --

\> it is a parameter issue.

The problem is in a layer under dd. Call it driver or kernel, or
whatever, but this is not a **dd** parameter issue. If there were a way
to *advise* the lower level to modify its behavior, then **dd** could
get that last sector. Is there a way to do that?

# Reporting our results

The **dd** report will be out soon. Once it has been released, the
National Institute of Justice (NIJ) will publish it on a JUSTNET web
site. The NIST CFTT web site will point to it and provide other
supporting information.