Open Source, Cisco, Trading and Findability

Blosxom Reinstall on Debian Lenny 5.0.1

It is almost time to retire my perl based bloging server software known as blosxom. It has performed well. However, my page count is starting to get high, and blosxom is taking longer and longer to process. For now, I've moved it to faster hardware while I work on a different blog delivery mechanism (I hope to have Wt on C++ with PostgreSQL running the back-end soon).

Copying over the directory structure was no real problem. The only real thing needed was to put the mod_rewrite stuff back in so the unsightly cgi-bin url portion is removed. By default, mod_rewrite is not enabled. To enable it:

a2enmod rewrite

Here is what the rewrite stuff as it looks in sites-enabled/default file:

  RewriteLogLevel 0
  RewriteLog /var/log/apache2/rewrite.log

<Directory "/var/www/blog">
  AddHandler cgi-script .cgi
  Options +ExecCGI
  RewriteEngine On
  RewriteCond %{REQUEST_URI} !-f
  RewriteCond %{REQUEST_URI} !-d
  RewriteRule ^(.*)$ /cgi-bin/blosxom.cgi/$1 [L,QSA]
</Directory>

Perl Mason Install

Installing mason v1.42 from Mason HQ is quite straight-forward:

apt-get install build-essential
apt-get install libapache2-mod-apreq2
apt-get install libapreq2-dev
apt-get install libapache2-request-perl
ln -s /etc/apache2/mods-available/apreq.load /etc/apache2/mods-enabled/apreq.load
perl -MCPAN -eshell
install HTML::Mason

VMWare Mouse Release on Debian Lenny Guest

A simple message to myself. When installing a Debian Lenny 5.0.1 KDE guest in VMWare Workstation hosted on Windows XP, a few steps are required in order to move into and out of the guest without the ctrl-alt mouse release sequence:

apt-get install build-essential on the guest
apt-get install linux-headers-...
build and install the VMWare toolkit in the guest
add 'Option "CorePointer"' to the mouse section of /etc/X11/xorg.conf
add 'Option "CoreKeyBoard"' to the keyboard section of /etc/X11/xorg.conf
restart KDE

A visit to a VMWare Community describes a couple of additional steps for getting the VMWare Shared Folders (HGFS) Share to work inside of Lenny 5.0.1 as well. Basically, in the /etc/fstab file, which VMWare updates when you perform a tool install, append ',uid=1000,gid=1000' to the 'ttl=5' portion of the .host line, so it looks something like:

.host:/ /mnt/hgfs vmhgfs defaults,ttl=5,uid=1000,gid=1000 0 0

The actual value to be used in place of 1000 is what ever your uid,gid are for your current window session. This an be determined at the command line by:

echo ${UID}

Without the uid/gid values in the fstab file, the share is made available for the root user. Anyway, after the restart, by using the file explorer, go to /mnt/hgfs to see the volumes.

Web Statistics with awstats on Debian Lenny 5.0.1

On an old system, I used Webalizer to analyze Apache log files. On a newer system I thought I'd give awstats a try. I had two options, install via the original source, or install via apt-get. Considering the number of files and directories involved, I decided to go with the Debian package manager to install awstats.

The package manager to get things into decent directories, but it appears to have been built for an earlier flavour of Debian. A few things I had to fix up for working in Debian Lenny 5.0.1 with Apache v2.0 include:

In /etc/cron.d/awstats, changing one of the file checks from /var/log/apache/access.log to /var/log/apache2/access.log
changing the ownership of the logs in /var/log/apache from root.adm to root.www-data (an alternative might be www-data.adm)
changing the creation ownership in /etc/logrotate.d/apache2 from 'create 640 root adm' to 'create 640 root www-data'
in /etc/awstats/awstats.conf.loal, added 'LogFormat=1' and 'DirIcons=/awstats/icon'
in /etc/apache2/sites-enabled/000-default, added 'Alias /awstats/icon "/usr/share/awstats/icon"'
the version of awstats installed was 6.5. I downloaded the awstats.pl file from awstats site and placed it in the /usr/lib/cgi-bin directly as a simple upgrade to v6.9.

During package installation, the package manager suggested some additional packages: libnet-dns-perl libnet-ip-perl libgeo-ipfree-perl. Perhaps when I get a chance, I'll install those and see what they add to the statistics management.

Network Broadcast Addresses

A customer was performing penetration testing on their network. Once the test results were in, among other things, they had a couple questions about responses to certain addresses on their external subnet range.

As a background, every subnet with a network mask of /30 or shorter has three address groups:

first address: the zeros address aka network address
middle addresses: usuable addresses
last address: the ones address aka broadcast address

For explanation purposes, imagine a router with two interfaces:

interface 1, the ingress interface, with address range of 10.0.0.0/30 and interface address of 10.0.0.1.
interface 2, the egress interface, with address range of 10.0.0.4/30 and interface address of 10.0.0.5.

For some network devices, for a packet arriving on the ingress interface destined for the broadcast address of the egress interface (10.0.0.7), the network device will forward the packet, effectively broadcasting to all devices located in the subnet of the egress interfaces. When many packets arrive in this manner, this is known as a Smurf Attack.

Current Cisco devices, by default, no longer forward packets to broadcast addresses, but may respond to these packets. The following command is applied by default to prevent forwarding of packets to broadcast addresses:

no ip directed-broadcast

At the other end of the subnet, for the network address, I originally thought this was a quiescent address. However, I did find that the an ICMP echo request arriving on the ingress interface destined to the network address (10.0.0.4) of the egress interface will generate an echo-reply with the ingress ip address (10.0.0.1) as the source address.

It appears that in days gone past, that for BSD Unix boxes and various other equipment, the network address was *the* broadcast address. This is why some configurations allow one to configure the address of the broadcast address setting, whether it be the high end or low end of a subnet. (thanx to Steinar Haug for this info).

rfc 1122 formalizes this broadcast address configuration (thanx to an inciteful responder named Lee):

   3.3.6  Broadcasts

         There is a class of hosts* that use non-standard broadcast
         address forms, substituting 0 for -1.  All hosts SHOULD
         recognize and accept any of these non-standard broadcast
         addresses as the destination address of an incoming datagram.
         A host MAY optionally have a configuration option to choose the
         0 or the -1 form of broadcast address, for each physical
         interface, but this option SHOULD default to the standard (-1)
         form.

The host will respond with the echo-reply because of rfc 791:

   3.2.1.3  Addressing: RFC-791 Section 3.2

             ...   An incoming datagram is destined
            for the host if the datagram's destination address field is:

            (1)  (one of) the host's IP address(es); or

            (2)  an IP broadcast address valid for the connected
                 network; or

From a Cisco router perspective, the default use of the command 'no ip directed-broadcast', allows one to use a /31 subnet (two ip addresses) for point to point links instead of the usual /30 subnet (four ip addresses). One can effectively address twice as many links with the same number of addresses. This feature is mentioned in Cisco's Feature Guide: Using 31-Bit Prefixes on IPv4 Point-to-Point Links.

Coincidently, while I was writing this article, I received a note that there are a couple of TCP Security Assessment documents available:

These documents go into the details of the bits and bytes making up the TCP protocol, analyzing the reasons for the bits, how they can be misused, and suggesting counter-measures when used illegally. Theres is a detailed bibilography with active links to related papers and documents.

An idea of the scope of the document can be seen through its first level table of content:

The Transmission Control Protocol
TCP Header Fields
Common TCP Options
Connection-Establishment Mechanism
Connection-Termination Mechanism
Buffer Management
TCP Segment Reassembly Algorithm
TCP Congestion Control
TCP API
Blind In-Window Attacks
Information Leaking
Covert Channels
TCP Port Scanning
Processing of ICMP Error Messages by TCP
TCP Interaction with the Internet Protocol (IP>
References

Securely Erasing Files

On a Linux system, there are a number of tools available for over-writing a file with random data and then deleting the file and hiding the name of the name of the file as well.

Of course, there are certain caveats that go along with this. If you focus only on securely deleting files, you will miss file content that may have been written to bad sectors, file journals, sectors released when files have been relocated from one area to another (as in when you edit or shorten files), and various other disk dead areas.

On popular tool is a utility called shred, and is found natively on most distributions. In the most basic form:

shred --remove filename

If you use the -v (verbose) option, you can see how many times it over-writes a file, and with what patterns it uses. It also uses a descending 0 write in order to obliterate a file name.

If you need to recurse sub-directories:

find * -depth  -type f | xargs shred --remove

If you have created then moved or erased files and want to ensure that the released content is overwritten, then you need to over-write drive free space and then release it. There are some poeople who suggest using dd to fill the free space and then use shred to overwrite and delete the single large file.

An alternative is to use scrub, a tool built by the Lawrence Livermore National Library folks. It uses various national standards for selecting suitable patterns and over-writing strategies. Source can be found at Sourceforge.

To ensure you have all the data, not just what was located in files or drive free space, one needs to apply scrub/shred to whole partitions and/or drives. The Gentoo Wiki talks about ways of securely deleting drives and partitions.

For near-absolute protection of data, I've known companies to specify that once a drive is no longer useful, that it be crushed and sent to landfill.

New Release of WTL (Windows Template Library)

I've been able to start on a new project with a clean slate. For the portion residing on Microsoft Windows, I'm going to give the latest version (v8.1, build 9127) of the WTL (Windows Template Library) for crafting the GUI side of things. The latest version can be downloaded from SourceForge, which looks to have a release date of May 7, 2009.

The release notes don't indicate anything for running with the Visual Studio 2008 IDE. A blog entry at Code Gem called WTL Wizard for Visual Studio 2008 indicates that by changing a few references to registry entries in the Visual Studio 2005 script, one can get the new WTL Wizard to install in the 2008 IDE. He supplies downloadable source code in the blog entry. Another location for a patch file is located in the Yahoo Forums.

WTL has in the past been underdocumented. Besides some websites linked from the wtl.sourceforge.net web site, the WTL group on Yahoo has had a WTL Developer's Guide posted in Doc and PDF forms.

The American Dream

In a recent issue of Investment News, former U.S. Comptroller General David Walker was quoted as saying:

"The American dream is not owning a house; it.s every individual having the opportunity 
to achieve their full, God-given ability, and each generation having the responsibility to 
leave the country better off and better-positioned than the next so that our children and 
grandchildren can have a better way of life than we have."

In light of the trillion dollar budget deficits which Obama's government is attempting to run up, Walker's warning is one of good reason.

Installing PostgreSQL on Debian Lenny

Release 5.0.1 of Debian's Lenny GNU/Linux distribution includes version 8.3 of PostgreSQL.

During the creation of a new Debian Lenny server, a list of software packages is provided. To make a new PostgreSQL-only server, unselect everything, including the 'Standard system', then select 'SQL Database', and proceed with the installation.

Once installation has completed, and the new server has rebooted, the PostgreSQL service is not auto-started. There are a couple of manual commands to be applied. In prior versions, PostgreSQL was auto-started. I think I understand the reasoning, particularily because it is useful for my situation.

During the server creation, I have a separate set of disks allocated for the database. By manually finishing the PostgreSQL implementation, I am able to initialize the directory location during service creation. If I have I've mounted my drives at /var/local/db, then these two commands get the PostgreSQL 8.3 service started:

pg_createcluster -d /var/local/db 8.3 main
/etc/init.d/postgresql-8.3 start

Building WebGUI 7.7.8 on Debian Lenny

It is a couple of years since I built a WebGUI server. Last one I built was on a Fedora Linux box.

This article is about building the most recent beta WebGUI on a Debian Lenny Linux box. The procedure is a bit long, but there is nothing complicated.

I start with a basic Debian build that has the 'Web Server' and 'Standard Build' options selected.

There are a few packages to install first:

apt-get install ntpdate
ntpdate 0.pool.ntp.org
apt-get install ntp
apt-get install build-essential
apt-get install mysql-server-5.0
apt-get install imagemagick
apt-get install perlmagick
apt-get install exim4-daemon-light
apt-get install exim4-conf
apt-get install libcrypt-ssleay-perl libnet-ssleay-perl
apt-get install libxml-sax-perl
apt-get install libxml-sax-expat-perl
apt-get install libxml-simple-perl
apt-get install libsoap-lite-perl
apt-get install libtext-aspell-perl
apt-get install libapache2-mod-apreq2
apt-get install libapreq2-dev
apt-get install libapache2-request-perl
ln -s /etc/apache2/mods-available/apreq.load /etc/apache2/mods-enabled/apreq.load

Then using

perl -MCPAN -eshell

Install or confirm the installation of the following Perl packages:

install Bundle::CPAN
install Log::Log4perl
install Class::InsideOut
install Config::JSON
install Module::Find
install Tie::IxHash
install Net::Subnets
install Text::CSV_XS
install Tie::CPHash
install Net::LDAP
install Exception::Class
install POE::Component::IKC::ClientLite
install POE::Component::Client::HTTP
install Clone
install HTML::Packer
install Path::Class
install Scope::Guard
install HTML::TagFilter
install DateTime
install HTML::TagCloud
install DateTime::Format::Strptime
install DateTime::Format::Mail
install Class::C3
install MIME::Entity
install XML::FeedPP
install CSS::Minifier::XS
install Color::Calc
install Finance::Quote
install Net::DNS
install Crypt::SSLeay
install XML::Simple
install JavaScript::Packer
install JavaScript::Minifier::XS
install Archive::Any
install HTML::Template::Expr
install SOAP::Lite
install Weather::Com::Finder
install Image::Size
install Image::Info
install Template
install Image::ExifTool
install Business::Tax::VAT::Validation
install HTML::Highlight
install CSS::Packer
install Contextual::Return
force install Test::Class
install Test::MockObject
install Text::Aspell

Download and expand the current software from SourceForge:

cd /usr/src
wget http://voxel.dl.sourceforge.net/sourceforge/pbwebgui/webgui-7.7.8-beta.tar.gz
tar -zxvf webgui-7.7.8-beta.tar.gz

Move some files around:

mkdir /data
mv WebGUI /data/

cd /data/WebGUI/etc/
cp log.conf.original log.conf
touch /var/log/webgui.log
chown www-data.www-data /var/log/webgui.log

cp spectre.conf.original spectre.conf

mkdir -p /data/domains/www.example.com/public/extras
mkdir  /data/domains/www.example.com/logs
cp WebGUI.conf.original www.example.com.conf
cp -R /data/WebGUI/www/uploads /data/domains/www.example.com/public/
chown -Rf www-data.www-data /data/domains/www.example.com/public/uploads

Append the following to /etc/rc.local:

cd /data/WebGUI/sbin
perl spectre.pl --daemon

Add the following to /etc/apache2/httpd.conf:

PerlSetVar WebguiRoot /data/WebGUI
PerlCleanupHandler Apache2::SizeLimit
PerlRequire /data/WebGUI/sbin/preload.perl

Add the following to /etc/apache2/sites-enabled/000-default:

ServerName www.example.com
ServerAlias www.example.com
DocumentRoot /data/domains/www.example.com/public
SetHandler perl-script
PerlInitHandler WebGUI
PerlSetVar webguiConfig www.example.com.conf 
Alias /extras /data/WebGUI/www/extras

Check that all the Perl packages are loaded:

cd /data/WebGUI/sbin
./testEnvironment.pl

Create the MySQL Database:

cd /data/WebGUI/etc
mysql -e "create database www_example_com"
mysql -e "grant all privileges on www_example.com.* to webgui@localhost identified by 'password'"
mysql -e "flush privileges"
mysql -uwebgui -ppassword www_example_com < /data/WebGUI/docs/create.sql

The /data/WebGUI/etc/www.example.com.conf file may need updates:

"sitename" : ["www.example.com",example.com"],
"dsn" : "DBI:mysql:www_example_com",
"dbuser" : "webgui",
"dbpass" : "password",
"uploadsPath" : "/data/domains/www.example.com/public/uploads",
"spectreSubnets" : ["127.0.0.1/32", "123.123.123.123/32"],

Start up Spectre:

cd /data/WebGUI/sbin
perl spectre.pl --daemon

Restart the web server:

/etc/init.d/apache2 restart

Browse to www.example.com and get started!

If you would like a pre-configured WebGUI server capable of running on VMWare let me know. I can even host Virtual Sessions.

VMWare Datastore Browser

I'm sure the VMWare people have hidden this on purpose... just so you think you are forced into installing command line utilities or buying licensing for their management products.

Anyway, I have a couple of ESXi 3.5 U4 servers installed. I created a Virtual Machine on one server, then used the SSH scp command to copy the Virtual Machine from one host to the other. That is all well and good, but how do you get it to show in inventory?

The answer to that is to run the VMWare Infrastructure Client. That is no problem. The trick is to click on the Summary tab while in Inventory mode, and right click on the datastore. One can then browse the datastore. And one can right click on a .vmx file to register the Virtual Machine in Inventory. That same menu allows one to upload and download images from a local computer.

I think it would have been more intuitively obvious to have the datastore(s) listed in the left hand tree, but I guess that would make too much sense.

Some random notes on ESXi 3.5 U4:

One needs to purchase at least the foundation license in order to get the remote command line tools to work
When in the ESXi console, one can use vmkfstools to create and resize virtual drives. The GUI does not allow the 'thin' command, but the vmkfstools command does. 'thin' is the ability to indicate what the overall size is, but not to preallocate all the space necessary all at once.
When using an Asterisk based server in VMWare, allocate at least 500MHz to the server in order to maintain non slipping time. More VMWare Timekeeping Best Practices
Veeam FastSCP: Veeam FastSCP- VMware ESX/ESXi managment tool FastSCP provides a fast, secure and easy way to manage files and bulk copy VMs across your VMware ESX environment.

VMWare on HP DL360 G6

I recently acquired a couple of decently configured HP DL360 G6 servers. Each boots VMWare directly from an embedded USB Token. Now that is a server that works right out of the box. And it did.

It is an excellent ability to be able to use HP's management tools to view the console remotely. I've not laid hands on the server, but I have almost complete visibility into the unit. There are about 20 different temperature sensors, I can monitor and cap power usage, evaluate processor utilization, and much more. Remote access to CDRoms is also available through a virtual media Java mechanism. I'm using that now to upgrade to U4 of ESXi.

HP has their own special image and after a bunch of searching, I found it at Software Depot Home.

I had tried the U4 version from VMWare's site, but it wouldn't install itself in the correct spot. That is when I figured that HP must have a special version. Don't try to install HP's v8.20 of management tools either. They are frought with installation problems.