Using Automated File Comparisons to Increase Efficiency and Accuracy in SAS Code Development & Validation

File comparisons form a central part of SAS code development, validation, and maintenance. Complex derivations and statistical summaries are typically validated through ‘independent double-programming’, where a perfect match is expected between validated production output and ‘QCed’ output. When production and QC output exist as SAS datasets, this expectation can be tested using SAS-based file comparison techniques, in particular PROC COMPARE; when they exist in other file formats, a number of other file-comparison tools can be applied, including ‘diff’ utilities (for comparing text files), and application-specific methods such as Word-merge. Automated file comparisons not only provide a huge boost in efficiency over manual approaches, but also vastly increase the accuracy and scope of QC efforts, since they facilitate complete rather than selective QC and prevent subtle and infrequent discrepancies from being over-looked. File comparisons also greatly enhance SAS code review: diff utilities are indispensable when comparing program versions, and when managing code propagation. This paper describes the use of PROC COMPARE in program validation, and presents a technique for retrieving diagnostic codes from the &sysinfo automatic macro variable. The use of diff utilities (ExamDiff for Windows, and Diff, Bdiff, and Sdiff for Unix) in SAS environments is then illustrated, providing simple and useful applications to both source-file and output-file comparisons, and describing methods for direct invocation from SAS. A technique for automated file comparisons in Word is also presented, along with a small automation script to carry out batchcomparisons among Word documents. INTRODUCTION In SAS code validation, independent double-programming is generally accepted as standard practice. In this approach, a ‘lead’ and ‘QC’ programmer independently code to a common set of specifications, and then compare the results. Analysis datasets are almost always validated in this way, and SAS provides a powerful method to automate the comparison between ‘analysis’ and ‘QC’ datasets in the form of PROC COMPARE. PROC COMPARE produces a rich set of diagnostics that not only identify and quantify differences between individual data points, but also compare ‘metadata’, giving a summary of discrepancies in data attributes (data types, formats, labels, etc.). PROC COMPARE is useful in code development as well. For example, modifications to a program can be tested ‘inline’ to make sure they have the desired effect by comparing WORK datasets produced with and without the code updates. Many other types of file comparisons are encountered in a SAS programming environment, and many other file types besides SAS datasets. After datasets, probably the most common file types to be compared are text files, and probably the most common comparisons are those that take place between ‘production’ statistical output and associated QC output. While the validation of analysis datasets usually has as a target a perfect match between the analysis and QC datasets (all variables and all records), the same rigor is often not applied to statistical output. The objection is often raised that the task of checking all values in a statistical summary is too time-consuming, and this objection is frequently born of the fact that the check is being carried out externally to SAS, by hand, often using printed output. The tools available to automate text-file comparisons are not nearly as widely known as those used to compare SAS datasets. However, by applying these tools there’s really no reason the validation of statistical output can’t apply the same approach (independent double-programming, producing similarly-structure ‘production’ and ‘QC’ output), scope (100% QC) and target (perfect match between production and QC output) as are applied to the validation of analysis datasets, even with very large tables, and even with a requirement for rapid turn-around times. Besides datasets and output files, other files that often need to be compared are, of course, SAS programs themselves. Most programmers maintain some sort of chronological back-up of any code under development, and periodically have to make comparisons against earlier versions. Programs are also often propagated across multiple projects, and the process of checking for consistent implementation of core functionality then becomes a daunting task, but one made considerably more manageable through the use of automated file comparisons. Williams (2007) makes a very compelling case for the adoption of formalized version control with SAS, and illustrates the considerable advantage that file comparison utilities, in particular ExamDiff, provide to code review tasks.