Removing the Spam: E-Mail Processing and Filtering
Authors: Geoff Mulligan
Venue:  (1999)

From the Book:As more companies connect to the Internet, system and network administrators are being tasked with increasingly diverse duties. Mail systems are becoming more intricate, and it is practically impossible for any individual to learn all of the complexities and idiosyncrasies of creating and maintaining a fully operational email system. When a crisis occurs, such as someone using your site as a spam relay, an administrator may spend countless hours plowing through Web pages, manuals, and numerous books to find the solution and proper configuration items. A simple and concise guide to problem fixes is necessary to save valuable time and minimize frustration. A number of books currently exist that deal with the theory of email operation and a few cover, in excruciating detail, sendmail and mailing list configuration, but there has not been a single reference written that covers the specific tasks required to get an electronic mail system up and running. In addition, although there are many Web pages and other documents (even comments in the source code) on how to configure a system for common situations such as blocking spam, supporting virtual domains, creating custom mail filters, and adding automated mailing lists, the correct method is not always intuitive nor is which document or Web page to search to find a possible and proper fix. Removing the Spam: Email Processing and Filtering was written to provide the system administrator with step-by-step solutions for these and other tasks required to maintain an efficient and effective Internet email site. It offers practical working examples in a comprehensive yet easy-to-use-and-understand way.After a basic overview, the types of tasks are grouped together in three chapterseach dealing with a major email subsystem. The topics for these chapters are outlined in the Organization section and here: Email history and technology Sendmail installation and configuration Mail filtering using Procmail Mailing lists using Majordomo and SmartList Each chapter includes subsections dealing with specific tasks. Where appropriate, some tasks may appear under multiple chapters. As an example, it is possible to filter spam using sendmail OR Procmail. Each has its own benefits and limitations and they can be used together to provide a relatively complete spam-filtering capability. The operating systems used as examples for the step-by-step instructions are Solaris and FreeBSD. Solaris was chosen because of its wide use as the Web server of choice for many Internet Service Providers and commercial enterprise networks. FreeBSD was selected because it provides a good view of the standard BSD environment used in other operating systems, such as SunOS, BSDI, OpenBSD, and NetBSD. This leads to the question, "Why not Linux?" Ready access to a stable Linux system on which to test installation and configuration issues was not available during the writing of this book. Audience This book is intended for both the UNIX system administrator and the email end-user to help them deal with the day-to-day tasks of managing an Internet email system. For the system administrator, the chapters cover the basic tools used to support a UNIX-based email system and the detailed tasks of getting, building, installing, and managing these tools. For the end-user, the chapters covering Procmail and mailing lists include the necessary information for configuring private filters and mailing lists. The chapters assume some basic knowledge of UNIX commands but also include step-by-step instructions that show exactly the commands to use to accomplish the tasks at hand. All instructions should be within reach of even the very novice user. Organization This book is divided into four chapters, each detailing specific tasks required to install, configure, and maintain an email system. Chapter 1 covers background material on the history and development of electronic mail on the Internet. This chapter discusses the evolution of mailing lists and email technology, including the workings and design of the Domain Name System, electronic mail etiquette, and the history of spam with some techniques to stop it. Chapter 2 focuses on setting up and running sendmail. Specific sections deal with every step of the process. First is where to obtain the latest version and the step-by-step commands needed to compile and install sendmail on your system. Then comes the simplest and most effective way to generate the complex sendmail configuration file and how to build and manage simple mail aliases and filters. This is followed by instructions for stopping spam at the mail server and user levels, setting up virtual domains, hiding internal domains, and working with mail queues. Not every conceivable option is covered; instead, the focus is on the main options used to handle the most common situations facing the email administrator. Chapter 3 delves into creating mail filters for specific tasks such as spam filtering and mail sorting. The chapter features Procmail, which is a widely used and extremely efficient mail filtering system available for most UNIX systems. Sections cover retrieving source and binary distributions, compiling and installing Procmail for your site, and the basics of building Procmail recipes. In addition, there are examples of ready-to-run filters to handle auto-reply mail information servers. The final chapter treats obtaining, compiling, and installing mailing list servers and configuring them for use on your system. Two different mailing list packages are coveredMajordomo and SmartListeach of which has versions freely available on the Internet. The example configurations include both open and closed lists and moderated and unmoderated lists. Again, not all of the various options available when setting up any of these packages are discussed; the focus is on the most common situations. There is also an appendix that contains on-line references to email software repositories and the Frequently Asked Questions Web locations. The Term Spam(R) The term Spam (R) when used in the context of food and cooking is a registered trademark of Hormel Foods Corporation and in that milieu generally refers to Hormel's luncheon meat product. You can find more information about the Spam (R) food product at the Web site http://www.spam.com. All uses of the word "spam" in this book, except for this paragraph, refer to Unsolicited Commercial Emailthe electronic mail version of junk mail. How to Use This Book This book can be read cover to cover, but it is also meant to be used as a quick reference for specific tasks. As such, the chapters and subsections are designed to be used independently to provide the step-by-step instructions for each recommended solution. Conventions The following conventions are used in this book: Bold: Used in the text for emphasis of important concepts and terms. Italic: Used for RFC titles, program names, file names, email addresses, and WWW URLs. Monospaced: Used for any text that appears on the computer screen, either in files or as the output from commands or programs. Monospaced Bold: Used in examples to identify commands and text that will be typed in by the user. Monospaced Italic Used in examples to indicate variable information, for instance, example.com would be replaced with an actual domain name. Acknowledgements I would first and foremost like to thank my wife, Jess, and my two girls, Katie and Jenna, for their help and support during the writing of this bookespecially the final push, during which I'm sure that I was unbearable. This was in addition to the numerous weekends I spent parked in front of the computer writing and testing and missing many wonderful summer days and activities. Without their love, encouragement, and profound patience this book would not have been possible. Thank you so very much. I would also like to thank and dedicate this book to Walt Grueninger. Walt started me on my adventure into computers some 30 years ago, when I was nine. It was his enthusiasm for the technology that sparked my interest and focused me on a career path. I appreciate the time and effort he spent teaching me about machine instructions, two-pass assemblers, and how to tell what a Data General Nova system was doing by listening to the "core" hum. Next, great thanks to the technical reviewers who provided excellent comments, both on the structure and the technical aspects of the book. Specifically: Dustin Andrews of Information Insights, Inc., Dr. Claus Assmann, David Collier-Brown, Dave Crocker of Brandenburg Consulting, Jasbir S. Mahal, and Bob Stewart from Cisco Systems, Inc. And thanks to the folks at Geocast Network Systems, Inc., my employer, who understood my commitment to completing the book even though we are just a startup company. Also, thank you to all of the folks at Addison Wesley Longman, especially Mary Hart, who also understood my commitment to my startup company and the inevitable delays and timing conflicts that this caused. They stuck with me, pushing me to complete the book and at least attempt to meet the deadlines. I would also like to thank them for giving me the opportunity to write this book. Geoff Mulligan