Lesson 2: file streams, type promotion, endianness

We're finally ready to start reading from our actual input file, so it's
an exciting commit. There's a lot going on here, so hold on to your hats.

One of the first things dcraw does is to determine the byte-encoding of
the RAW image file, and it does so by reading the first few bytes out of
the file.

 298 ushort CLASS sget2 (uchar *s)
 299 {
 300   if (order == 0x4949)          /* "II" means little-endian */
 301     return s[0] | s[1] << 8;
 302   else                          /* "MM" means big-endian */
 303     return s[0] << 8 | s[1];
 304 }
 305
 306 ushort CLASS get2()
 307 {
 308   uchar str[2] = { 0xff,0xff };
 309   fread (str, 1, 2, ifp);
 310   return sget2(str);
 311 }
 ..
 7217   order = get2();

The way this code works, 'order' is initialized to 0. The first time we
call get2(), we enter the big-endian path and therefore shift the 0'th
byte, and or it with the 1st byte, returning the value. It's not stated
anywhere, but this actually works because the 'unsigned char' type of
str[0] gets promoted to the next biggest type when you left-shift it.

There can be no direct translation of this into go because 'str' below
is declared as a slice containing byte types. If a byte is left-shifted
in go by 8 bits, you get 0.

 > var b byte
 > b = 0xff
 > fmt.Println(b << 8)
 > 0

So one way to do a direct translation of the above would be to do some
casting. Indeed, this was my first attempt:

 func sget2(s []byte) int16 {
 	if order == 0x4949 {
	 	return int16(s[0]) | int16(s[1]) << 8
 	} else {
	 	return int16(s[0]) << 8 | int16(s[1])
 	}
 	return 0
 }

Not only is that supremely ugly, but it gets unwieldy quickly if you want
to convert say, a 4-byte or 8-byte value. Casting each individual byte to
the proper end type? No thank you.

Instead, let's try using our brain a little bit and see if there's a
better way to do what dcraw.c was trying to do, which is to read a buffer
of raw bytes, and then byte-swap it to the proper endianness. It turns out
that go's standard library provides a facility for byte-swapping so we
should just use it.

 21 func sget2(s []byte) int16 {
 22         var ret int16
 23         buf := bytes.NewBuffer(s)
 24
 25         /* "II" means little-endian */
 26         if order == 0x4949 {
 27                 binary.Read(buf, binary.LittleEndian, &ret)
 28         /* "MM" means big-endian */
 29         } else {
 30                 binary.Read(buf, binary.BigEndian, &ret)
 31         }
 32         return ret
 33 }
 34
 35 func get2() int16 {
 36         str := make([]byte, 2)
 37         ifp.Read(str)
 38         return sget2(str)
 39 }

It might seem a bit clunky above, but check out the go implementation
of sget4(), lines 44--48. Any time we can avoid manual byte-swapping, we
are happy knowing we're doing the right thing. I bet we can find more
commonality between sget2() and sget4() but for now, let's just bias
towards "obviously ported from the original" rather than trying to be
too clever by half.

Finally, let's check out the next thing dcraw.c does, which is to
continue reading from the file to determine the value of 'hlen':

 313 unsigned CLASS sget4 (uchar *s)
 314 {
 315   if (order == 0x4949)
 316     return s[0] | s[1] << 8 | s[2] << 16 | s[3] << 24;
 317   else
 318     return s[0] << 24 | s[1] << 16 | s[2] << 8 | s[3];
 319 }
 320 #define sget4(s) sget4((uchar *)s)
 321
 322 unsigned CLASS get4()
 323 {
 324   uchar str[4] = { 0xff,0xff,0xff,0xff };
 325   fread (str, 1, 4, ifp);
 326   return sget4(str);
 327 }
 ..
 7218   hlen = get4();

See the second call to fread() which continues reading from the file
pointer to get the next 4 bytes out of the file. I'm happy to report
that go's os.File.Read() interface does the same thing, and it will
also read the next 4 bytes from the file.

As a final note, let us observe the nasty on line 320 where dcraw.c
redefines sget4() to force a cast on the input parameter to make it
the proper type. Who knows what kind of strange behavior the original
author ran into without that cast. We can be happy that go prevents
this entire class of errors for us.