An Ajax-Enhanced Web-Based Ethernet Analyzer

I've spent the past six months or so playing with Ruby.  I blame the July
2006 issue of Linux Journal for this hiatus from my programming
language of choice, Perl, as that issue opened my eyes to
the possibilities of using Ruby as a serious tool.  I still love, use
and teach Perl, but I'm spending more and more time programming in Ruby. 

I follow the same process when learning any new programming technology:
I identify a good book and work through it, and then start to use the language
to build some of the things I love to build with Perl.  Identifying the
book was easy.  The second edition of Programming
Ruby by Dave
Thomas (known as The PickAxe) is as good an introduction as you
are likely to find for any programming language, not just Ruby.  Once I'd
worked my way through The PickAxe—creating a Ruby tutorial as I went
along (see Resources)—I was itching to write some real
code.  I started with a type of tool that I enjoy building with Perl:
a custom Ethernet analyzer. 

Does the World Really Need Another Ethernet Analyzer? 

At this point, probably more than a few readers are saying to
themselves: why bother creating an Ethernet analyzer when tcpdump and
Ethereal/Wireshark already exist?  Those solutions are excellent
tools—which I use a lot—but, I'm often looking to build something
that involves additional processing plus the capturing and
decoding of Ethernet packets, and this customization invariably involves
resorting to custom code.  Luckily, it turns out that the technology
that underpins both tcpdump and Ethereal/Wireshark—as well as the
hugely popular Snort IDS—is available as a library and that a number
of language bindings exist for it.  The packet capturing library, called
libpcap, is available from the same project that brought the world
tcpdump and can be downloaded with ease from the Web.  In fact, it may well
be included within your distribution's package management system; it
is if you are running a recent release of Ubuntu (as I am).  Obviously,
the intrepid programmer can use C with libpcap, but—let's be honest
here—life's far too short to work at the C level of abstraction when
something more agile is needed.  Thankfully, Perl provides an excellent
set of modules that work with libpcap, and I devote one-sixth of my
first book to discussing the Perl technology in detail.  To my delight,
and after a little digging around, I also found a set of Ruby classes
that interface to libpcap (see Resources). 

Creating a Custom Ethernet Analyzer with Ruby

In order to test the libpcap technology for real, I decided to
use Ruby to redevelop a tool I created with Perl a number of years
ago, which I wrote about within the pages of The Perl
Review
(see Resources).  My Perl tool, called wdw (short for
who's doing what?),  analyzes requests made to a LAN's DNS service and
reports on the site names for which the clients are requesting DNS
resolutions.  In less than 100 lines of Perl code, I'd written a functioning and
useful DNS Ethernet analyzer.  I wondered how using Ruby would
compare. 

Now, I present the 20 or so lines of Ruby I used to re-create
wdw (for the entire program, see Listing 1).  Do not interpret
my numbers as any attempt to claim that Ruby can do what Perl does in
one-fifth the number of lines of code.  It cannot.  It is important to note,
however, that Ruby's interface to libpcap is significantly more abstract
than the one offered by Perl, so Ruby does more in a single call than Perl
does, but that has more to do with the choices made by the creators of
each language's libpcap binding, as opposed to any fundamental language
difference. 

Before executing this code, download and
install Ruby's libpcap library.  Pop on over to the Ruby libpcap Web site
(see Resources), and grab the tarball.  Or, if you are using
Ubuntu, use the Synaptic Package Manager to download and install the
libpcap-ruby1.8 package.  If a distribution package isn't available,
install the tarball in the usual way. 

You also need a Ruby library to decode DNS messages.  Fortunately,
Marco Ceresa has been working hard at porting Perl's excellent Net::DNS
module to Ruby, and he recently released his alpha code to RubyForge, so
you need that too (see Resources).  Despite being alpha,
Marco's code is very usable, and Marco is good at releasing a patched
library quickly after any problems are brought to his attention.  Once
downloaded, install Marco's Net::DNS library into your Ruby environment
with the following commands:

tar zxvf net-dns-0.3.tgz
cd net-dns-0.3
sudo ruby setup.rb

My Ruby DNS analyzer is called dns-watcher.rb, and it starts by
pulling in the required Ruby libraries: one for working with libpcap
and the other for decoding DNS messages:

#!  /usr/bin/ruby -w
require 'pcap'
require 'net/dns/packet'

I can tell my program which network connection to use for capturing
traffic, or I can let libpcap-ruby work out this for me.  The following
line of code lets Ruby do the work:

dev = Pcap.lookupdev

With the device identified (and stored in dev), we need to enable
Ethernet's promiscuous mode, which is essential if we are to capture
all the traffic traveling on our LAN.  Here's the Ruby code to do this:

capture = Pcap::Capture.open_live( dev, 1500 )

The open_live call takes two parameters: the device to work
with and a value that indicates how much of each captured packet to
process.  Setting the latter to 1500 ensures that the entire Ethernet
packet is grabbed from the network every time capturing occurs.  The call to open_live will succeed only if the program has the
ability to turn on promiscuous mode—that is, it must be run as root or
with sudo.  With the network card identified and ready to capture traffic,
the next line of code applies a packet capturing filter:

capture.setfilter( 'udp port 53' )

I'm asking the libpcap library to concern itself only with capturing
packets that match the filter, which in this case is Ethernet packets
that contain UDP datagrams with a source or destination protocol port
value of 53.  As all Net-heads know, 53 is the protocol port reserved for
use with the DNS system.  All other traffic is ignored.  What's cool about
the setfilter method is that it can take any filter specification
as understood by the tcpdump technology.  Motivated readers can learn
more about writing filters from the tcpdump man page. 

A constant is then defined to set how many captured packets I am
interested in, and then a timestamped message is sent to STDOUT to indicate
that the analyzer is up and running:

NUMPACKETS = 50
puts "#{Time.now} - BEGIN run." 

The libpcap-ruby library contains the loop iterator, which provides a
convenient API to the packet capturing technology, and it takes a single
parameter, which is the number of packets to capture.  Each captured
packet is delivered into the iterator's body as a named parameter,
which I refer to as packet in my code:

capture.loop( NUMPACKETS ) do |packet|

Within the iterator, the first order of business is to decode the
captured packet as a DNS message.  The Packet.parse method from
Marco's Net::DNS library does exactly that:

dns_data = Net::DNS::Packet.parse( packet.udp_data )

With the DNS message decoded, we can pull out the DNS header information
with a call to the header method:

dns_header = dns_data.header

For my purposes, I am interested only in queries going to the DNS server,
so I can ignore everything else by checking to see whether the query? 
method returns true or false:

if dns_header.query?  then

Within the body of this if statement, I print out the IP source and
destination addresses, before extracting the IP name from the query,
which is returned by calling the dns_data.question method.  Note
the use of a regular expression to extract the IP name from the query:

Comments

Comment viewing options

Thanks for the excellent follow-up to your TPR article.  There are plenty of use cases for a network analyser that do not require promiscuous mode - which would simplify the approach.  In general, I prefer not to operate in promiscuous mode since I am only interested in monitoring point-to-point traffic. 

As a consultant who frequently performs network analysis and tuning I am keenly interested in a solution like this.  While I like the Ajax/Apache approach my customers aren't likely to be crazy about me installing the entire kit on their boxes.  Wrapping this approach into one executable would be ideal.  Or, simply installing an agent which would send the results to an Apache instance running on my laptop :-)

Thanks for the positive comment.  If all you need is the results, all your customers need is Ruby (or Perl) installed on their boxes, with a little script that wakes up every now & then and sends stuff to you.  My article simply used the analyzer as a way to generate a lot of server side data which allowed me to demo the Ajax solution. 

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority.  With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line. 

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs.  However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.”  After all, before data can be restored to a system, there must be a system to restore it to. 

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.