preserving test equipment with hard drives - Page 1
Forum section: electronics / testgear

author: majbthrd
author_rank: Contributor
timestamp: August 03, 2014, 10:13:11 pm
content: I'm dividing these posts into three parts for ease of reading:1) an observation on how use of hard drives in test equipment can cause the equipment to expire well before its time2) an account of my struggle to replace a mechanical hard drive with a flash equivalent3) some design thoughts on emulating hard drives--In the past, it seems like we've have been able to acquire used test equipment at moderate prices that might be too dated for trendy commercial projects but still have plenty of use left in them. However, I have to wonder if that trend will be hampered in the future thanks to component usage in modern test equipment.A few months back, I acquired a piece of test equipment that uses a "PATA" (aka "Parallel IDE" or just "IDE") 2.5" hard drive.As it happens, the equipment is an Anritsu MG3700A vector signal generator. (Anritsu isn't the only test equipment manufacturer to use a hard drive. I'm just using it as an example.) The equipment is only 8-9 years old and has great technical specifications. It doesn't even depend on the hard drive for storing its firmware.However, it was nonsensically designed to use the hard drive as an interim storage area for all equipment operation. So, when the hard drive eventually dies, it will become nothing more than a paperweight. It is so sad.A loosely comparable product would be the Agilent E4438, which with the right options can also function as a VSG. Some versions of the E4438 come without a hard drive (which I personally think is a preferred configuration). Unlike the Anritsu, all of the Agilent VSG models can (in my experience) accept a waveform upload directly through Ethernet WITHOUT using the hard drive. (Very good!) However, if the E4438C does come equipped with a hard drive, I don't know whether it will function without it.The immediate knee-jerk reaction is to try replacing the hard drive with a SSD or CompactFlash card. I've read of others replacing hard drives with CompactFlash adapters, etc. I've been down that road and encountered hurdles.

author: majbthrd
author_rank: Contributor
timestamp: August 03, 2014, 10:14:22 pm
content: At least with the Anritsu MG3700A, trying to replace the hard drive is fraught with problems. I've wasted money buying all manner of flash drives, and it has mostly been a bust.There is some sort of race condition in the portion of firmware that loads the waveform from the hard drive to its waveform RAM. If the data arrives from the hard drive too quickly, the (VxWorks-based) firmware locks up.PATA is hard enough to find. CompactFlash seems to be one of the few remnants of the standard. Alas, nearly all the cards still in production tend to be too darn fast at transferring data.I've acquired a supply of used conventional mechanical 2.5" PATA hard drives, and these work reliably without issue. However, these too will eventually die. I've heard of lubrication issues with even brand new hard drives that have been kept in storage, so that is something else to worry about.One piece of good luck was that Anrtisu used a VxWorks-flavored FAT32 file system.If one just wants to do a binary copy of two hard drives that have the exact same size (or to make an exact backup), dd is a straightforward option.If cloning drives of different sizes and the file system is something like FAT or FAT32 or NTFS, I had very good success with using an old copy of Norton Ghost 8.3 (DOS-based) that I bought many years ago. Your mileage may vary.The Ghost CD came with bootable floppy images and these could be used on a PC with IDE controllers. However, both of those (floppy drives and IDE ports) are not plentiful these days.The open-source QEMU works wonders, though, and came to the rescue.Here is an example command line syntax, where GHOST.IMG is an 2.88MB bootable floppy disk image and "src.bin" and "dest.bin" are images of the two emulated hard drives:qemu-system-i386 -no-kvm -boot a -fda ~/GHOST.IMG -hda src.bin -hdb dest.binAs might be suspected, "-boot a" indicates to boot from A: drive (fda). I found that I had to use "-no-kvm" on my Linux machine; your mileage may vary.One can also use /dev/sdX arguments for the -hda and -hdb parameters to access drives directly, but then one has to run qemu as root to access those devices. (I'd welcome a solution where I didn't have to do that.)What is great about the QEMU approach is that one can use USB-to-IDE adapters to mount drives and or CompactFlash cards. From the perspective of the software running on QEMU, it is just accessing another IDE drive.In my experience, Norton Ghost did a great job of copying the filesystem and adjusting for differences in drive sizes (up or down). (In comparison, Clonezilla can't even copy to a destination drive smaller than the source.) Norton Ghost also kept partition volume information the same on the destination, preserving what VxWorks expected.So, copying the hard drive was straightforward. Getting the replacement hard drive to work was not.In buying different drives, it is not at all clear what the differences are between them.What I found very helpful was the Linux utility "hdparm". Ideally, one wants the version that supports both of these command line arguments:hdparm -I /dev/sdbhdparm --Istdout /dev/sdbThe former outputs a human readable interpretation of the drive's response to the IDENTIFY DEVICE (ECh) ATAPI command. The latter outputs the raw 256 word data in hex form. (Some Linux LiveCDs use a limited version of hdparm that only supports the former. Presumably, all the relevant information is in the former, but I prefer the notion of having an exact bit-for-bit copy.)Using hdparm with a USB-to-IDE adapter is NOT an option in my experience. The USB mass storage specification doesn't try to reproduce this functionality.The two solutions I found that did work were:1) Find an old computer/laptop with IDE ports. Plug the drive (with necessary adapters) into the computer. Boot off a Linux Live. Run hdparam and save the results to USB flash drive.2) If dealing with a CompactFlash card, try using a Thinkpad X20-series laptop. These had a proper IDE-to-CompactFlash bridge and appear compatible with hdparm.If you have test equipment with a hard drive AND you feel comfortable with being able to remove the drive without damaging the test equipment, the hard drive, or the warranty sticker, it seems worth considering:a) making a sector-by-sector backup of the hard driveb) using hdparm to make a copy of the IDENTIFY DEVICE tableThat way, you *might* stand a fighting chance of being able to replace the hard drive.However, in my experience, even though I had both (a) and (b), I still had no end of trouble coming up with a solution.As mentioned in Part 1, there is a firmware bug in the Anritsu MG3700A where it will lock up if the data being read from the hard drive arrives too quickly.Using the hdparm utility and trying many drives, I found that the factory-provided mechanical hard-drive supported Ultra-DMA modes. However, I have to presume that the seek times of the mechanical drive were enough to slow down the data. If I used a modern Ultra-DMA flash drive, it would cause the Anritsu firmware to lock up.Through experimentation and hdparam, I found that low-end (and often past end-of-life) CompactFlash cards that ONLY supported PIO and Multi-DMA transfer modes seemed to be slow enough that they didn't trigger the Anritsu firmware bug.The downside is that these low-end cards don't come in anything near the capacities of the original mechanical hard drive (40GB).

author: majbthrd
author_rank: Contributor
timestamp: August 03, 2014, 10:15:56 pm
content: I could envision building a bespoke hard drive emulator that provided the asynchronous IDE interface, used a SD card as cheap flash storage, and deliberately introduced delays consistent with the behavior of the physical drive. One could even emulate the exact desired response to the IDENTIFY DEVICE (ECh) command and size of the legacy drive. However, the development effort is likely more complicated than desired.A perhaps less risky idea I have would be to complement a modern CompactFlash card with additional circuitry.My hdparm testing seems to suggest that most modern CompactFlash cards tend to support a full superset of the needed functionality. So, the goal would be to only advertise the subset of functionality supported by the legacy drive, and let the CompactFlash card do the heavy lifting.My notional idea is to use a CPLD/FPGA to perform two tasks:1) Intercept any IDENTIFY DEVICE (ECh) ATAPI command and replace the drive's response with an alternate one of the user's choosing2) Delay the response of an interrupt to retard the performance as neededThe advantage of such an implementation is that one shouldn't need to bother engineering a full ATAPI implementation. The IDENTIFY DEVICE (ECh) command only happens at PIO speed. The more complicated transactions are left to the CompactFlash card itself.When replacing a hard drive in equipment that might be very particular about the geometry/size of the hard drive, one can customize the IDENTIFY DEVICE table to precisely mimic the legacy drive. The CompactFlash card can be much larger than the legacy drive, and if the host only accesses a subset of the sectors, so be it. (Note: there may still be an issue trying to emulate non-LBA addressed drives.)Getting a CPLD with 5V I/O (that can support reasonably fast bus transactions, e.g. not just the a current limiting resistor approach that app notes may suggest) that ALSO happens to be capable of emulating a 256x16 ROM is problematic. I suspect a bunch of bus level translators are needed to convert to a more reasonable 3.3V to allow more CPLD design choices.Ideally, if one knew the right people at a CompactFlash vendor, one could forgo the complexity of the above designs and just customize the firmware of the card to do the same thing. It would be a much cleaner solution.Anyway, hopefully all my ramblings above give people some ideas and/or warnings about trying to preserve operation of test equipment past the life of the hard drive.

author: nctnico
author_rank: Super Contributor
timestamp: August 04, 2014, 12:32:29 am
content: I have replaced hard drives in several PC based piece of test equipment with compact flash drives. Sometimes I ran into trouble with equipment specific BIOSses but nothing that couldn't be overcome.

author: majbthrd
author_rank: Contributor
timestamp: August 04, 2014, 01:13:11 am
content: Quote from: nctnico on August 04, 2014, 12:32:29 amI'm sure hard drive swaps on PC-based equipment would be much easier. The vendor is relying on an underlying BIOS, operating system, and drivers that have been written and tested to support many different configurations. Ideally, it is no more difficult than a hard drive swap on a normal PC.When the implementation is a custom, bespoke embedded one, chances are that it was only tested under one idealized condition. Change anything, and unintended consequences can occur (such as the race condition I encountered with the Anritsu).

author: David Hess
author_rank: Super Contributor
timestamp: August 04, 2014, 06:22:58 am
content: I have been using compact flash cards to replace old hard drives but have not run across the race condition you describe although it seems likely enough so I am not surprised.Adding some interface logic to the PATA interface to intercept detection of the faster transfer modes during enumeration is what I would try to fix it.

author: omgfire
author_rank: Regular Contributor
timestamp: August 04, 2014, 07:07:51 am
content: http://www.epos.ua/view.php/en/products_emulator_hddStory about emulating bug in hdd to compensate bug in industrial machine (in Russian):http://habrahabr.ru/post/178099/

author: free_electron
author_rank: Super Contributor
timestamp: August 04, 2014, 07:16:04 am
content: you may need a card that can emulate the correct ATA level.

author: Wuerstchenhund
author_rank: Super Contributor
timestamp: August 04, 2014, 12:24:56 pm
content: Probably all CF cards on the market support IDE. However, there's a trap most people are not aware off:Most consumer CF cards are made for being used in digital cameras or in similar applications so these cards identify themselves as removable storage. This however often causes problems with systems that expect to find a fixed drive type storage device on its IDE bus, resulting in the card either being not recognized or being recognized but not working.The solution is to buy a CF card that identifies itself as fixed disk. This is often the case with industrial CF cards which usually can be ordered as removable type and fixed type (so careful with the exact model no, if in doubt check the spec sheet!).Crappy CF-to-IDE adapters can also cause lots of issues.

author: amyk
author_rank: Super Contributor
timestamp: August 04, 2014, 12:50:41 pm
content: Any chance of patching the firmware to remove the race condition? You might even enjoy increased performance.

author: majbthrd
author_rank: Contributor
timestamp: August 04, 2014, 04:17:26 pm
content: Quote from: Wuerstchenhund on August 04, 2014, 12:24:56 pmThanks for making that point; it is a very important one.The IDENTIFY DEVICE table in these consumer CF cards indicates a removable device. Trying to boot a PC from one of these cards, for example, can be a non-starter. The "industrial" cards can have an IDENTIFY DEVICE table indicating a proper drive (and they might also have better flash wear-leveling, increased temperature range, etc.).

author: nctnico
author_rank: Super Contributor
timestamp: August 04, 2014, 04:26:50 pm
content: There are drivers for Windows which turn a removable drive into a fixed drive.

author: Wuerstchenhund
author_rank: Super Contributor
timestamp: August 04, 2014, 05:47:17 pm
content: Quote from: majbthrd on August 04, 2014, 04:17:26 pmJust to add: some older cards (i.e. some SanDisk Extreme variants) could be changed from FIXED to REMOVABLE and vice versa through a little DOS utility, but most modern cards are either REMOVABLE or FIXED.QuoteIt depends. The only Standard PC OS that wants a FIXED type drive is Windows (although the Embedded variants can boot from removable storage, and Win9x can be made to boot from it as well). Most better Linux distros (i.e. openSUSE, Redhat, CentOS) don't care, however, as does DOS.For equipment with IDE bus that is a little bit more modern I'd recommend to use either a good ATA SSD (i.e. Transcend, *not* Kingspec) or a mSATA SSD (which has now become very cheap) in an adapter like this:http://www.addonics.com/products/adms25ide.php

author: majbthrd
author_rank: Contributor
timestamp: February 26, 2015, 08:47:01 pm
content: Quote from: Wuerstchenhund on August 04, 2014, 05:47:17 pmIt is an old thread, but I wanted to thank you for the suggestion.To be honest, at the time of the post I was dubious that this Addonics device would be any different from all the CompactFlash and flash PATA products that I had already tried (and I had tried many).However, I recently had a chance to give this product a try, and it appears to work. It bothers me because I'm not sure WHY it is working where other products didn't, but it seems a more viable option.I've attached the hdparm output of an ADMS25IDE in case a future reader wants to know what specific IDE modes and features it appears to support.

author: mian2zi3
author_rank: Contributor
timestamp: July 25, 2015, 12:19:48 pm
content: Quote from: majbthrd on August 03, 2014, 10:15:56 pmThis might not help the OP who is trying to replace an IDE drive, but there is a SCSI emulator using an SD card that others who stumble on this thread might be interested in:http://www.codesrc.com/mediawiki/index.php?title=SCSI2SDhttp://imall.itead.cc/scsi2sd.htmlI got one for a Agilent 16702B I picked up recently. It just arrived yesterday. I'll update once I've had a chance to install it.

author: svet-am
author_rank: Newbie
timestamp: September 23, 2015, 03:50:28 am
content: Quote from: mian2zi3 on July 25, 2015, 12:19:48 pmWere you able to get this to work? I ordered a SCSI2SD, used "dd" to dump an image of the factory hard disk and then used dd again to write it back to my SD card. Sadly, it won't boot and I'm trying to figure out what has gone wrong.