UNIX, POSIX, and Open Systems: The Open Standards Puzzle, 1st edition
Authors: John S. Quarterman
Venue:  (1992)

From the Book:Preface UNIX, POSIX, and Open Systems: The Open Standards Puzzle is a guidebook to formal standards and industry specifications related to the historical UNIX operating system, particularly the POSIX (IEEE P1003 and ISO/lEC IS 9945) operating system interface and application environment standards. Most standards and specifications are small when compared to the general distributed environments, including networks, that many users want. But users want environments that support portable and interoperable software. Some standards and specifications are profiles that show how to form such environments. If profiles are jigsaw puzzles and the standards and specifications are the pieces, the pictures on them show the environments that vendors should implement to support user applications. More than one puzzle can often be constructed out of the same sets of pieces, by choosing different subsets of pieces, and by orienting them differently (with different parameters and options). Some profiles even reference other profiles, producing puzzles within puzzles that show large environments to be constructed from smaller environments. The specification of profiles to describe environments for particular classes of applications is called profiling. For profiling to work well, we also need an Open System Environment (OSE), described by a reference model that classifies the pieces according to general shapes and pictures, a taxonomy that says how to fit them together without their edges overlapping, and a guide that gives practical advice. The Open Standards Puzzle explains how these standards, specifications, and profilesare related, how they are made, and how you can influence them. It will help you use them in building real systems. The Market and the Puzzle Standards are important to users as tools to provide more functions, and to increase choices and productivity. They are important to vendors and even nations as tools of competition. The economic and political importance of standards is based on their technical specification of interfaces that are important to distributed computing and open systems. Standards can promote these new paradigms of computing while easing the integration of older, legacy, systems. Single standards tend to be just pieces of the larger puzzles vendors and users need to solve. Cutting the pieces and fitting them together is best done with input from all affected parties. Standards for Users. According to Senior Executives for Open Systems (SOS), who are ten very large companies that use computers but do not produce them as their main businesses, The creation of a computing environment based on widely implemented vendor neutral standards is essential to compete in the global marketplace of the '90s and beyond S0S 1991. Standards are becoming big business; sales can be won or lost on the basis of standards conformance alone. For example, many government contracts require conformance to specific standards, and the United States government constitutes a large part of the United States UNIX market, which is a large part of the POSIX and open systems markets. In the past few years, all major UNIX system vendors have committed to comply with the POSIX (IEEE 1003 and IS 9945) set of standards for operating system interfaces and application environments. Such commitments mean that standards will remain a major factor in these markets for years to come. The Global Playing Field. Standards are seen by many as tools of competition: Happily, the U.S. retains one big advantage that lies outside the scope of a story about manufacturing: software. More and more of the value in big computer systems lies in the software that controls them. With English as the universal business language, the U.S. has a natural advantage setting world standards Kupfer l992. Standards have even been called a national trust. Others hold that standardized software is commodity software, which some say gives the Japanese an advantage over the rest of the world. Such an argument would assume that software innovation was solely an American capability. This assumption would be incorrect: But here too Japan is coming on strong. Computer companies and consumer electronics companies, including Sony, have launched major drives to develop new operating systems Kupfer l992. And there are more than two players of the global economic game: The biggest chill is blowing from Europe. When a truly unified market becomes a reality at the end of 1992, factories that sell electrical products in Europe will need to be certified as meeting a new set of standards. These so-called ISO 9000 standards evaluate factories on every aspect of their operations, from training to manufacturing. U.S. exporters worry that the Europeans will wield the certification process as a nontariff trade barrier and shut them out Kupfer 1992. This is essentially the "level playing field" argument, which has also been heard regarding the OSI network protocols. Whether a particular playing field is level seems to depend on your point of view. Developers, sellers, buyers, users, and standards participants need to know the nature of the playing field and some of the rules of the game. Then they can better decide whether the playing field is level, and perhaps what to do about it. The Computing Paradigm Shift. Unlike twenty years ago, computing now takes place on any desktop, not just on a few large machines isolated in special rooms. Computers come from a multitude of vendors, but must communicate with other machines in other rooms or on other continents. Users want to move programs among machines, and employers want to move users among machines. Vendors want to sell applications for a variety of platforms, and platforms for many applications. Some vendors use standards to open closed markets by getting industry consensus on specifications for interfaces that were formerly provided only by a single vendor's system. Everybody wants applications that can communicate and systems that can exchange data and information, whether by network, tape, or disk. Distributed computing is replacing timesharing, personal computing, and networked computing. Open systems that provide distributed computing are replacing proprietary systems with proprietary interfaces, because they can accomplish more and sometimes cost less. Yet legacy systems, which are the older systems that represent large investments, must be taken into account. A single programming interface may be implemented across a wide range of systems, including legacy systems. Many vendors of older systems are implementing such interfaces. Legacy systems may also be made accessible by use of standard network protocols. Distributed computing usually does not cost less overall. The real reasons for moving to distributed computing are increased capabilities, greater flexibility of incremental expansion, and choice of vendors. Pieces and Puzzles. SOS wrote that "All Companies agreed that they had a common need to accelerate the commercial availability of Open Systems based on vendor neutral standards and enabling technologies in a proactive manner" SOS 1991. Specifications of interfaces and protocols are needed so interoperable and portable software can be written, sold, and bought. Specifications independent of any single vendor are best, because they can be used to build a vendor-independent computing environment. Such specifications are called standards when produced by an accredited standards body using a formal process. They are best produced with input from those who need to use them. Open processes are needed, so that the results will not be biased towards the specific concerns of any vendor or user, and will help solve real problems. The specifications and standards needed to build real open systems are fractured like a jigsaw puzzle. Even standards that form whole pieces, like those for the C, Fortran, or Cobol programming languages, or the RS-232-C interface, are small compared to a whole puzzle. The box of open system puzzle pieces for application programming includes pieces named POSIX, the C programming language, and the X/Open Portability Guide, among others. The main pieces in the box for networking are labeled TCP/IP and OSI. But users need completed puzzles, not boxes of pieces. When a user wants a new picture (environment), a puzzle (profile) must be constructed to specify the kinds of pieces that are wanted, and actual pieces must be selected from these (and perhaps other) boxes to solve the puzzle. The pieces may be assembled to form many different puzzles, such as a file server, a turnkey realtime system, a batch processing environment, a traditional interactive timesharing system, or even a complete distributed computing system. These puzzles are called profiles. Profiles describe sets of standards and specifications, together with specified values for options and parameters. A profile is specified to support the requirements of a particular application, class of applications, or environment. A user who asks for MS-DOS plus QuickBasic has specified a profile, but in a vendor-specific manner. If every user, application writer, or system interface vendor assembled a different set of standards or specifications, even from the same larger set, there would be almost as much confusion as if there were no standards. Vendor-independent profiles are more generally useful, especially when written by independent bodies using open processes, as are formal standards. Many standards bodies and industry and user groups have constructed profiles and guides that define sets of standards for particular purposes, such as multiprocessing, transaction processing, and network login hosts. A single standard or specification is usually focused around a technical, political, or marketing need, and is too small to form a whole puzzle. Each piece may be an international, regional, or national standard. It may be a formal standard or a de facto standard (a specification). New pieces (and puzzles) are being made all the time by working groups, technical advisory boards, and joint technical committees; others are made by marketing, market share, or free reference implementations. A standard may be rough cut by one group, polished by another, and painted by still another. Because some standards have to incorporate or at least not prohibit many divergent practices, the pieces may not fit well even when finished. The shape of a piece may change after it has already been used to solve a puzzle. In fact, squeezing a piece to fit is explicitly permitted, as long as the original shaper said where and how much you could squeeze. This is what the options and parameters used by profiles are for. The Need for User Involvement. Continual involvement by users in the development of standards or profiles will help ensure that the final documents will address user needs TSG-l 1991. But the standards themselves do not describe the processes by which they are created. It is often difficult to determine which bodies are responsible for a standard, specification, or profile, even just for those related to UNIX or POSIX. Users want solutions, not puzzles or pieces. The Book UNIX, POSIX, and Open Systems: The Open Standards Puzzle is a handbook for use by programming, marketing and sales, or acquisition and procurement professionals to determine which standards puzzle pieces are useful and necessary for their tasks. It gives reasons for building, selling, buying, and using open systems and distributed computing. It describes the historical relations between the UNIX system and open systems. It tells why standards are needed. It discusses some of the advantages and drawbacks of particular standards and of standards in general. Wherever possible, it points out what you might expect but will not find in a standard. The book contains a snapshot of many of the available standards related to POSIX. POSIX is the set of standards produced by the IEEE Computer Society (IEEE/CS) Technical Committee on Operating Systems Standards Subcommittee (TCOS-SS) P1003 working groups. It also includes ISO POSIX, which is the set of international standards produced by the ISO/IEC (International Organization for Standardization/International Electrotechnical Committee) JTC 1 (Joint Technical Committee One) 5C22 (Subcommittee 22) WGl5 (Working Group 15), such as the International Standard 15 9945-1, Information Technology Portable Operating System Interface (POSIX)Part 1: System Application Program Interface (API) C Language. The book also discusses related non-POSIX TCOS-SS working groups such as P1238, P1201, and P1224. The book has extensive coverage of profiles. It discusses issues that affect many standards and profiles, such as language independence, conformance, and internationalization. Language independence permits applications to be written in several programming languages for the same system interfaces. Conformance to specifications is important both to application developers (so they will know their products will run) and to users (so they will know what products will meet their needs). Internationalization helps a variety of software products to reach international markets. There are two traditional ways of looking at distributed computing: from the viewpoint of the network and from the viewpoint of the operating system and its interfaces. This book takes the latter viewpoint. The book is about open systems, which are essentially operating systems. It treats networking as an issue that is relevant to many standards. The network viewpoint is otherwise left to another book Carl-Mitchell & Quarterman 1993. This book is mostly about standards for application program interfaces. The standards covered are those most directly related to open systems. Standards for databases, documentation, and electronic data interchange are not covered except in passing, for example. The book is also not an application portability programming tutorial. It does not cover point by point details of each standard, specific changes required for standards compliance, nor complete requirements for application portability. Instead, it tells who produces standards and how, what's in them and how they are related, and how to use them together and how to change them. Much of this book is about the context in which the standards evolve. Specific details of any given standard may not yet be finished, and may be interpreted and modified. We believe the reasons a standard is created, its intended purposes. and the profiles that reference it, are as important as the contents of the standard itself. This is a how-to book for understanding and using standards, and for affecting the processes that create them. It provides a foundation for further investigations by the reader. Organization The book contains fifteen chapters and is organized in four parts. Each part opener contains a brief introduction to that part. The parts are named from the puzzle metaphor: Part 1, Context; Part 2, Cutters; Part 3, Pieces and Patterns; and Part 4, Puzzles. Part I, Context, gives the motivation for open systems, and tells how standards can be used to help build them. Part 2, Cutters, describes the bodies that produce the standards, that is, the groups that cut the puzzle pieces and define the puzzles they are supposed to fit into. Part 3, Pieces and Patterns, describes specific base standards and extensions, that is, the puzzle pieces themselves, and discusses issues such as networking and intemationalization that color many pieces. Part 4, Puzzles, describes profiles, which are puzzles that can be made out of the pieces. In addition, there are three forewords, this preface, references at the end of each chapter, two appendices, a glossary, and an index. Forewords. In addition to the series foreword, there are forewords by Carl Cargill and James Isaak. Cargill is the author of Information Technology Standardization, a general work about standardization. Isaak is chair of the TCOS-SS (IEEE Computer Society Technical Committee on Operating Systems Standards Subcommittee), which is the sponsor of the IEEE POSIX standards committees. He is also the convenor of the ISO POSIX working group. He was an early participant in the work that led to the /usr/group 1984 Standard, and he has been a major figure throughout the history of POSIX. Part 1, Context. Three chapters give the context needed to understand the rest of the book. People want to use distributed computing environments that provide many resources in a transparent manner. Open systems are needed for distributed computing, and the UNIX system has historically been used for this. Open standards are needed for open system interfaces, and the POSIX standards are the central ones. These chapters discuss these issues. and give terminology for fitting standards, specifications, and profiles together to solve the puzzles of open systems for distributed computing. Part 2, Cutters. The second part of the book contains a comprehensive list of standards bodies related to the UNIX operating system. The first chapter gives details of formal standards bodies. The second chapter describes industry groups that produce specifications and influence standards, and user groups that educate people and influence standards. The third chapter describes some examples of processes used by standards bodies. If standards are pieces of jigsaw puzzles, these are the processes used to cut pieces out of plywood or sheet goods. Part 3, Pieces and Patterns. The third part of the book is about standards and issues, that is, the puzzle pieces and the patterns that can be seen in them. The first chapter lists and describes all the TCOS committees and their projects, whether they have been completed or not. The rest of the third part is about issues. Networking may be the most important of them and is treated first. The next chapter describes the issues surrounding programming language bindings, including some details of bindings of IEEE 1003.1 to C, Ada, and FORTRAN. Conformance testing and internationalization each have a chapter. Part 4, Puzzles. Standardization is time consuming, and standards bodies often focus individual base standards on narrow technological areas so that they can be completed in a reasonable amount of time. Real systems cover wider technological areas, so it is necessary to specify several standards and options and parameters for them, in order to support a functional objective. The fourth part of the book describes these functional standards or profiles that select standards pieces to fit into puzzles. A chapter describing kinds of profiles and their relations to each other and to standards is followed by chapters on TCOS profiles, industry specifications and distributed management, and procurement profiles. Appendices. There are two appendices. The first gives access information for standards bodies and for publications about standards. The second is a standards puzzle acrostic challenge supplied by Jim Isaak that asks the reader to classify acronyms as standards, standards bodies, specifications, or companies. This approach reflects the humor of the current confusing situation, where users must locate usable specifications and put them together in a coherent way without much guidance. References, Glossary, and Index. Numerous bibliographic citations occur in the text, and the actual references are gathered at the end of each chapter. The glossary defines major (and many minor) terms and acronyms. The index indicates where they are used in the text. Expansions of acronyms are given both in the glossary and in the index for easy reference. Readers This book is suitable for someone new to standards to learn what the central open systems standards are, how they fit together, how they are made, and where to find additional information about them. Familiarity with the UNIX operating system is useful, and some sections require familiarity with the C programming language. You don't even have to have a technical background to benefit from reading this book. But even if you are already intimately familiar with the technology, you will find useful information here about what is being done with it. Since standards have been developed or are being developed in all areas of computation from system interface to graphical user interfaces, the list of people who need to be informed about them is long. The short list includes those in application development marketing and sales acquisition, procurement, and purchasing The long list includes system administrators, operating system developers, system programmers, system integrators, standards participants, and end users. These people all want to know how to build and use real systems and environments using standards, specifications, and profiles. Different people have different needs, and may want to read the book in different orders. We hope everyone will read the first three chapters in Part 1, Context, which give the context for the rest of the book. Since the chapters on standards bodies and processes in Part 2, Cutters, describe the organizations that make the pieces and puzzles, reading them will make the rest of the book more comprehensible. The reader in a hurry may want to read Part 2, Cutters, to get an overview of standards bodies and to look up specific bodies, standards, or topics in the index when needed. Anyone who wants to affect the standards process should read Chapter 6, Processes, and perhaps also Chapter 8, Protocols and Standards. Further reading suggestions are given in the following subsections. Because the book is mainly about context, it will be of interest to many computer professionals, including system programmers, operating system developers, sales engineers, system administrators, system integrators, standards participants, managers, consultants, and end users. All these can consult The Open Standards Puzzle to learn how standards are produced and how they fit together. Application Development. A major reason cited for standards development is application portability, and an open systems standard directly related to UNIX and POSIX is usually an Application Program Interface (API) standard or an Application Environment Profile (AEP). Application programs that conform with API and AEP standards can be ported to different platforms with less effort. An application that runs on more platforms has a wider market. Application developers will want to read the descriptions of specific standards and profiles in the chapters in Part 3, Pieces and Patterns, and Part 4, Puzzles. Part 3 will also be useful in describing approaches to known issues that cross standards and platform components. Some of these standards are not finished yet. Application developers may want to participate in the standards process to make sure the resulting standards meet their needs, for example, by accommodating new hardware technology. Lack of participation by application developers has been noted by many standards groups as a problem in producing adequate standards. All national standardization bodies have been requested to encourage the participation in standardization of people and organizations that use standards Isaak 1991. Marketing and Sales. It's been said that standards are marketing. That's not all they are, but it's certainly one thing. To reach a wider market, product marketing needs standards information. This can be used for making decisions about marketing strategies, assisting in the choice of standards that are important for their product, developing appropriate sales brochures, and training sales staff. Marketing personnel should find Part 4, Puzzles, of the most use, plus Chapter 10, Conformance Testing. Sales personnel can use a good working knowledge of the standards world in telling their customers which standards a product complies with and what benefit such compliance will provide. Hardware vendors are constantly asked "Will your hardware run the Whiz Bang software?" or "Will your hardware be able to communicate with brand Z hardware?" If both pieces of hardware conform with the same standard, there is a good chance that the answer to these questions is yes. Similar situations apply to software platforms such as operating systems. Chapter 10, Conformance Testing, and Part 4, Puzzles, should be of particular interest to sales personnel. There is also a basic difference between the kinds of standards and conformance needed for interoperability and those needed for application portability. Portability requires standards for application platforms such as operating systems, and standards for applications, together with separate conformance testing for each. Interoperability involves peer entities (processes, programs, computers, networks) communicating with one another, with any pair of peer entities using the same standards. Conformance of a single implementation to a standard can sometimes be tested, but is not nearly as important as actual demonstrated interoperability in a network with many different implementations from many vendors. These issues are discussed in Chapter 8, Protocols and Standards. Acquisition, Procurement, and Purchasing. It would be nice if customers knew what vendors knew. However, time is limited, and emphases differ. Customers may be more interested in fitting together pieces from many vendors to form an integrated working environment. Acquisition may include specifying procurement requirements, determining products that fit, and buying products. People specifying procurement requirements need to know standards, so they can say which standards and specifications products must conform to. Those using such requirements to make actual purchases don't need to know as much about standards, but the more they know, the better they can determine what products match the requirements. Informed use of standards can help a user become vendor independent and hardware independent, which can lead to lower costs and greater capabilities. Standards can permit multiple sources, and produce leverage with suppliers. But standards will not solve all problems, and care must be taken not to overestimate claims about them. Conformance to some networking standards doesn't necessarily mean the conforming systems will interoperate, for example; see Chapter 8, Protocols and Standards, and Chapter 10, Conformance Testing. More generally, Part 4, Puzzles, describes the larger environment specifications, and Part 2, Cutters, describes the bodies and processes that produce them. When an implementation of a particular standard is being chosen, its description in Part 3, Pieces and Patterns, can be useful. When a thorny issue such as internationalization (Il8N) comes up, the discussion of it in Part 3, should help. System Administrators. People who keep systems running have all the problems of application developers, system integrators, and end users, plus more. Carefully selected standards can help simplify these problems. The standards of most interest will be those for system administration, distributed management environments, and distributed computing, but all those involve other standards, as well. Chapter 14, Industry Profiles, should be of particular interest, probably in combination with the rest of the chapters in Part 4, Puzzles. Operating System Developers and System Programmers. People who develop platforms for other professionals to use have an even larger set of concerns. System programmers need to know how to implement standard interfaces, and they also need to stay abreast of the standards situation so they can adapt their implementation to future changes in standards. If your system doesn't have memory management, for example, you may want to be sure basic interface standards don't require that facility. Part 3, Pieces and Patterns, describes specific interfaces, and Part 2, Cutters, discusses the bodies and processes that produce them. System Integrators. Standards can make system integration easier, if the system integrator knows which standards will make each system fit with another system. Part 4, Puzzles, and Part 3, Pieces and Patterns, are of particular interest. It is important not to put together systems by simply checking a list of standards and ensuring implementation of all parts of all of them. It may seem obvious that options are by definition optional, but some procurement specifications require specific values for options. Procurement specifications should leave some options optional. For example, a requirement for all of 1003.1 includes a requirement for the mmap facility, that is, for user-accessible memory mapping. Some computers, such as those made by Cray, do not have memory management hardware, so requiring all of 1003.1 would inadvertently exclude such computers from consideration. Avoiding checklist specification is a good idea. Profiles are often useful in tailoring a system or a procurement to a particular application. For example, IEEE 1003.10 is for supercomputing environments, and 1003.15 is for batch processing. Standards Participants. Let's not forget those who spend large numbers of their working hours producing standards and specifications. Careful attention to the technical details of a single standard can leave little time for understanding the processes by which it is standardized. Participating in one working group leaves little time to learn about other working groups, their documents, and their processes. This book provides concise information on a range of related standards in Part 3, Pieces and Patterns, and information on how they are related in Part 4, Puzzles, and throughout. It also describes the bodies and processes that produce those standards in Part 2, Cutters. Standards processes are not fast by nature, since they must carefully balance technical considerations, and most of the processes used for the standards in this book require consensus or decision by large majority. Yet the economic and political pressures on these processes are enormous, and the main goal behind those pressures is more speed in the processes. Standards participants need to be aware of this, so they can balance speed against good standards. In some cases, technical mechanisms, such as the use of computer networks in the standards process, may permit greater speed while maintaining technical content. In other cases, standards participants may need to resist hurrying, so they can preserve consensus. More comments on these subject...