# System Monitoring with Xymon/Developer Guide
## Getting Started with hobbit development on sourceforge
### Reference Books

* C Programming
* UNIX Advance programming
* UNIX Network Programming

### SVN tutorial
#### References

* Getting started with SVN
* GCC SVN tutorial

#### SVN for Xymon Tutorials

Followings are some examples using svn to interact with hobbit on sourceforge.net.
* list out the sub directory of xymon SVN directory.
* find out changes in last 7 days.
* Checking out hobbit source tree using svn.
* As one of hobbit developers, you can create a branch.
* Following example create a instance of hobbit by tjyang.
* creating a subdirectory.

## Development Road Map
### Henrik, Feb 9 2009

On February 9, Henrik posted the following roadmap to the list. I haven't heard much lately so I don't know if it has changed.
Version 4.4:
* SNMP support for data collection
* Status updates based on data collected in the RRD files (this is really nice - essentially, *any* value you put into an RRD file can trigger a new or modified status in Xymon. So you could eg. change the status of your "http" test to go red, if the response time of a webpage exceeds 5 seconds).
* Status "flap" handling, to catch statuses that change rapidly.
Version 4.5:
* Support for communication between clients and Xymon to use compression and/or encryption.
* Support for client authentication via SSL certificates (so you cannot spoof a client message).
Version 4.6:
* Revised web interface - eliminate the need for webserver-based authentication, and provide more fine-grained authorization for the various Xymon components.
* Per-user custom login pages, so the login will always show those systems that the user has responsibility for / is allowed to access.
Somewhere along the road I will also introduce a new "ping" test daemon - I am getting to the point where ping'ing all hosts take longer than 5 minutes (I have a lot of hosts). So something needs to be done, and I think I know how to do it.

### Design Document

Hobbit Design Document

### Wish Lists

* Test Suite for software quality assurance.
* Migrate all the bb modules from deadcat.net to hobbit.
* SLA Report built in.
* Better Theme support.
* Provide encrypted communication pipe between hobbit server and hobbit client.
* hobbit messages compression.
* ipv6
* 4.2.x maintenance release.
* Rewrite of the alerts module to work properly. ;) It's quite flawed at the moment.

## Xymon volunteers Roles and Responsibility

This is a list of people offer their time and skill to Xymon community in following areas of R&R.
* Project Admin
** Has right as project admin on sourceforge.
** Assign tickets to different persons listed below.
** Grant new developer privilege on sourceforge.
** Get people to work as a team with limited and un-paid resources.
* Development
** Understand Xymon source tree.
** Do source code documentation.
** Work on ticket from issue tracker.
** Apply patches submitted from the Xymon community.
** Implement new requested features
* Documentation
** Keep troff files updated with Xymon features.
*** maintain troff2html process.
** Create a publish-grade manuals and process for Xymon using Latex.
*** use latex2troff tool to generate Xymon troff manpage sources.
*** Automate the document generation in "make" command process.
* End User Support
** Answer questions raised in xymon user email list.
** Maintain Xymon FAQ.
** Test out new Xymon features or verify a bug.
* Module Development
** Write new Xymon client external modules.
** Write Xymon Server side external modules.
** Maintain orphan Xymon modules.
** Maintain Xymon module development guides.
| Name | Project Admin | Development | Documentation | End User Support | Module Development |
| Henrik Størner | | Yes | | | |
| T.J. Yang | | Yes | Yes | | Yes |
| Jerald Sheets | | | Yes | | |
| Josh Luthman | | | | | |
| Malcolm Hunter | | | Yes | Yes | |
| Japheth J.C. Cleaver | | | | | |
| Steve Holmes | | | Y | Y | ? |
| Damien Martins (aka doctor madness) | | N | Y | Y | N |
| dubby | | | | Yes | |
| Flemming | | | | Yes | |
| Christoph Berg | | Yes | | | Yes |
| Vadim Pushkin | | | | Yes | |
| Martin Ward | | Yes | | | |
| Buchan Milne | | Yes | | | |
| Rich Smrcina | | Yes | | | |
| Kenneth Langford | | Yes | | | |
| Padraig Lennon | | | | | |
| David Baldwin | | Yes | | | |
| Ralph Mitchell | | | Yes | Yes | |
| Dummy Template Replace Here | Admin Y/N | Dev Y/N | Doc Y/N | User Support Y/N | Module devel Y/N |
| Olivier AUDRY | | Yes | | | Yes |
| Neil Franken | | Yes | | Yes | |
| Peter Toft | | | | | |

## Packager Who is who

* blingme: Mandriva
* Christoph Berg (Myon): debian

## Developers Who is who
### Developers with write access

* Henrik Stoerner: http://www.hswn.dk
** Make Big Brother system monitor to be open again.
** Rewrite Big Brother shell scrip part into C code to speed up the performance.
** Amazon Wish List
* Martin Ward
* Rich Smrcina
* Buchan Milne
* T.J. Yang
** hobbit wiki book project.
** Autoconfiscate Hobbit.
** Hobbit documentation project.
*** Will use latex to create hobbit manual.
** Digitation of hobbit software build,package build and deployment process.
** OpenSolaris VMWare appliance for Hobbit Development.
** Using EXTJS 3.0 for hobbit menu system.

### Developers

* Vernon Everett
** hobbit wiki and documentation.
* Etienne GRIGNON (etienne.grignon@gmail.com )
** BBWin: http://sourceforge.net/projects/bbwin
* Francesco Duranti <fduranti@q8.it>
** Ability to patch hobbit source code.
** http://www.hswn.dk/hobbiton/2006/10/msg00080.html
* TWW Inc.
** Package the hobbit client and server using TWW toolsets.

### List of Names contributed to this hobbit project
## Autoconfiscate Hobbit

This section describe the conversion of hobbit source tree into autotool control.

### Why the autoconfiscation ?

* To be more portable across OS platforms.
* Plan for I18n and L10n.
* To become true GNU software which require use of autotool.

### Learn autotool

* the autotools tutorial and read the pdf slides.
* hand on testing on hello world example.
* example of using autoscan.

### Convert hobbit source tree

* make sure your autotool sets has following version.
* download the latest hobbit source tar ball.
* run autoscan to get configure.scan and then modify it as configure.ac.
* convert lib/Makefile demotool/Makefile ... etc into Makfile.am.
* Fine tune configure.ac to take configuration options as configure options.
* Support Linux and Solaris OS setting in configure.ac.

### References

* Great Autotool tutorials and links started here
* Another tutorial
* Autotool for Beginners

## Xymon Documentation

* Current
** Henrik Stroner use troff to generate manual pages so that we have the RTFM type of documentation.
** Also troff2html is used to create html page for web browser viewing.
* Enhancements Proposals.
** Use latex+dia+make+wiki to create publishing grade docs
** Use Table of Conetent, index, references and Figures to describe Xymon related topic.
** Generate Xymon User Guide.
** Generate Xymon Developer Guide
** Generate Xymon System Admin Guide.
** Generate Xymon Slide for presentation purpose.
** Use Wiki documentation content incubation.
** Generate html.RTF and pdf for different purpose.
* Request Development Team to use doxygen syntax for source code documentation.

### MediaWiki to DocBook

* use mvc to gain hobbit wiki source
* convert mediawiki source to latex.

### Reuse content from existing hobbit documentation source

* http://www.hswn.dk/hobbit/help/manpages/

## Xymon test suite

Run through test cases to verify the code changes are ok.

### Quality assurance test scripts
#### shell scripts

* test script in build directory of hobbit-4.0.3 to make sure depended software are in build machine.

### Dtrace probes in hobbit server

An example of putting dtrace probe int an application.

## Packaging
### Hobbit Client for OpenSolaris IPS format

* Features of This hobbit client
** This hobbit client will use a local account and group to avoid NIS and NFS dependency.
** It will use SMF to start/top hobbit client.
** It will use sudo or Solaris 10's ACL to achieve reading restricted information off OpenSolaris.
** hobbit-client-serverconf is simple version of hobbit client without local messages processing.
** hobbit-client-local will do some processing filtering, it need more depended software.
** it use /opt/hobbitclient42 as installation path.
** svcadm disable/enable network:/hobbit-client-serverconf

#### OpenSolaris info

* OpenSolaris 2008.05
* Installed Sun and SUNW gcc-3.4.3 compilers.
** if not installed run "pkg install sunstudioexpress" and "pkg install SUNWgcc".

### Using CPAM tools to automate package building
## How to participate the hobbit development

* subscribe to hobbit mailing list
* create an account on en.wikibooks.org to participate in hobbit documentation effort.
* learn from http://en.wikibooks.org/wiki/CPAM_with_TWW to participate in hobbit automated cross-platform software packaging effort.

### Setup your development environment
#### On OpenSolaris x86
##### Using prebuilt OpenSoalris VMWare Image

* Download OpenSolaris VMWare Image from here(TBA).
* Install VMWare player 3.0 from here(TBA).

##### Using VirtualBox OpenSolaris Image
#### On Solaris 10 Intel

* host info

#### Setup your TWW developement

* download TWW tools onto your system
* compile hobbit*.[sb|pb].

### Feedback to Hobbit

* include your fix,patch, modules addition to sourceforge.net

## Hobbit Functional diagrams
### Windows Hobbit client

BB Win

## Developing Hobbit by Existing Internet Development Environment
### Custom Monitoring Scripts for Hobbit

System Monitoring with Hobbit/HOWTO/Custom Monitoring Scripts for Hobbit http://www.hswn.dk/hobbit/help/hobbit-tips.html#scripts

## Developing Hobbit by CPAM/TWW Development Environment
### Why use CPAM/TWW tool ?

* Pros
** Hobbit server and client need to be cross-platform thus the need of CPAM/TWW.
** Like Emacs editor, it can will be run everywhere without license. TWW tools is GPL opensource so it can be ported to any OS.
** A structural and automated approach to deal with packages creation and management.
* Cons
** Another learning curve need to climb.
** Changing people's development habit is hard.

### Supported OS

* Solaris Sparc
* RH AS 3.0
* HP-UX 11.11

## Hobbit package sources
### Hobbit client on MS windows Overview

* bbwin (Xymon client) is implemented in c++. It requires opensource software like boost c++ libary and tinyXML.
* The packaging tool it used is Microsoft's opensource packaging tool - WiXto do the Win32 packaging.
* The detals building instruction is build.txt on bbwin project site.
* You have two ways to build BBWin.msi,
** Use Microsoft's make: nmake
** Use "Build Soultion" in VS2008.
Following is a procedure to create a Windows XP development box so you can create your own BBWin.msi.

## Prepare your Xymon win32 build environment

* Install hhc.exe,it is used to compile html doc into windows chm help file.
** add "C:\Program Files\HTML Help Workshop" into PATH so hhc.exe is accessible.
* Install Visual Studio 2008 Standard or Professional Edition.
* Make sure your cmd.exe has following PATH name addition. echo %PATH% %INCLUDE% %LIB% in your DOS prompt.
** PATH
** "LIB=C:\Program Files\Microsoft Platform SDK\Lib\.;C:\Program Files\Microsoft Visual Studio 8\VC\LIB;"
** INCLUDE
* Install a svn client for your windows machine.
** tortoise svn
* Windows Server 2003 SP1 Platform SDK Web Install: It is use to have the complete sdk to develop programs on Microsoft OS. Remeber to install all options (include 64 bit and 32 bit) to avoid "atlthunk.lib not found" issue.
** install visual service pack 1:
** MSDN blog about VS2008 and SDK
* WiX opensource packaging tool from Microsoft: this is use to turn the compiled binary into MS MSI package format.
* Boost C++ library:These powerful C++ libraries are used for some agents and will be certainly used in the BBWin core service in the future.
** Download the pre-compiled 1.35 one for VS2008 and install it under C:\boost directory.
* TinyXML: BBWin used the excellent TinyXML library for the configuration loading XML. You do not need to download and install it. TinyXML sources are included in BBWin sources.
* Prepare your source code editors:
** notepade++:
** XEmacs for win32:
* checkout bbwin src.
** Using command "svn co https://bbwin.svn.sourceforge.net/svnroot/bbwin bbwin"
* Change the Makefile code for C:\boost installation.
** Modify Makefile to point INCLUDE to C:\boost and LIB to include C:\boost\lib\*.lib
** Modify C/C++ include c:\boost and Linker to include c:\boost\lib\*.
* Optional software installation
** install cygwin to have a strong unix enviroment on your windows box. So you can run command like following to clean up *.obj file that current makefile can't do.
** Clean up Release directory by Deleting Setup/*.msi

#### Build the bbwin using command line approach

BBWin.msi creation process is automated in "C:\BBWin\trunk\Setup\Makefile". This makefile will be called from "C:\BBWin\trunk\Makefile"
* Make sure your nmake command is available.
* cd to c:\bbwin top source directory.
* type "nmake" to kick off the build.

#### Compilation error
##### atlbase.h not found

* Error message
* make sure you have following directory and file, if not then you need to use VS non-Express version.
* Possible fix

##### Temporary Fix for following

* Goal is to get rid of " -D_MT -MTd" when using cl MS compiler.
* Run this command in cygwin shell on your codebase, "find . -type f -name Makefile -exec perl -pi -e 's!cvarsmt!conlibsdll!' \;" , to replace cvarsmt with conlibsdll. I don't know the exact impact of not building code with multihreading library.

##### * LINK : warning LNK4098: defaultlib 'LIBCMT' conflicts with use of other libs; use /NODEFAULTLIB:library

* http://msdn.microsoft.com/en-us/library/Aa267384
* Possible fix with similar issue.
* vc6 to vc8.
* Try to link with "/NODEFAULTLIB:library".

#### build the bbwin using GUI approach

* check out the source code
* open up bbwin.sln
* Example of using nmake to create a BBwin.msi

#### bbwin 0.9 package source
## Hobbit server software build source
#### How to use hobbit-4.0.3.sb build source ?

* hobbit-4.0.3.sb is an XML file contain the detail exact steps to build hobbit-4.03. into binary code for different OS. It need to be run against "sb" tool.
* TBA.

#### The source : hobbit-4.0.3.sb
### Hobbit server package build source
#### hobbit-4.0.3.pb
## Hobbit Required Packages

* fping
* apache web server
* rrdtool
* pcre
* openssl
* openldap
* following is the listing of exact version to build hobbit-4.0.3

### Following diagram list out all the hobbit depended software by GraphViz
## Create a firefox search engine plugin for hobbit server

* References: http://mycroft.mozdev.org/deepdocs/deepdocs.html

## Using relational DBs with Xymon Server

The default database back end of Xymon is RRD(Roun-Robin Database).

### Database
#### SQLite3

* Almost as fast as C fopen function code.
* No server process, data are store in plain files.
* Just like another rrd file that can be rsynced over to your standby xymon server.

#### MySql

* light weight DB.

#### XML flat file
### Benefits of using relation DBs
#### Inventory Monitoring

* Store a machine's asset information
** Serial Number
** OS type
** Computer Maker
** Leased/owned
** Trigger an alert if leasing is expiring soon.
** Display pie chart of OS type,Makers ... etc.

#### bb-hosts generation

* bb-hosts files got generated from host database table.

### MVC Implementation
#### Catalyst/SQLite
##### Bill of Materials

* Catalyst Book 1
* perl-5.10.0
* Fedora 11
* Address Book Example in Book 1.
* SQLite3
* Catalyst::Runtime,Catalyst::Plugins::*,Catalyst::Devel.
* Sql files to create Asset schema.

##### Procedures

* Use catalyst.pl to initiate your MVC project with sample templates.
* perl Makefile.PL

### Upgrade Xymon Web GUI

Current Xymon(Hobbit) is coded in C and using CGI and html code to compose a simple web interface. Recently, we have a few javascript framework choices to upgrade xymon server with RIA web interface.

#### Using qwikioffice for Xymon Dashboard

* qwikioffice Demo Site

#### Using Tomatocart's admin tool for Xymon administration tool

* Xymon Admin Interface

#### Replace Xymon C/CGI with C/EXTJS

* JSON in C

## Compilation Errors and Warning
### Possible Solutions

* To remove "unsigned char" usage hobbit src tree.
* To cast the char that need to be unsigned.

### GCC Compilation Warnings

* pointer targets in assignment differ in signedness
** Why use unsigned char ?
** char-vs-unsigned-char
* "/usr/include/sys/socket.h:214: note: expected socklen_t * __restrict__ but argument is of type int *"

### Compilation Warning on 4.3.0 C source code example listing
## Xymon GUI ideas
### GUI Requirements

* Keep cgi+html+tigramenu for legacy support.
* Include modern Javascript GUI framework like extjs or jQuery.
* Provide realtime web GUI update from clients using AJAX.
** Waiting for 1+ minutes to see the changes from web page is barely acceptable but hope to avoid.

### From Neil Franken

* Use CSS to provide easy theme and resolution change.
** Resolutions:
** Themes:
* Enable fine-grained control on columns in different views.

### From T.J. Yang

I found following combination of existing opensource software can make up a new GUI for Xymon. End user can still see old cgi+html GUI.
* Use extjs for new GUI and miframe.js to contain old tigra menu GUI in a frame.
* See Xymon Desktop Prototype.

## Using command line tool to update this wiki book
### Installation of mediawiki client in perl

* Mediawik command line tool
* A makefile script to automate the hobbit documentation effort.