2007 Jun 29 - Fri

One Unified Address Management, v1.3

I've updated the schema of my prototype address management database to reflect some additional relationships with VRFs and VLANs.

The address management schema is for a PostgreSQL 8.1 database.

Once I have the address tables populated, I'll want to generate zone files for DNS to facilitate informative traceroutes. There were a couple of useful suggestions on the Nanog mailing lists, one by Mark Tinka, and the other one by Joe Abley, the same person who did a paper on toolmaking at Nanog 26.

[/OpenSource/Debian/AddressManagement] permanent link

2007 Jun 27 - Wed

Network Management Processes

In another exerpt from Cisco's book called Top-Down Network Design, Second Edition, there is a discussion regarding network mangement as being a part of network design. Very few companies, or many engineers for that matter, pay much attention to network management.

"Management is often overlooked during the design of a network because it is considered an operational issue rather than a design issue. However, by considering network management up front, instead of tacking it on at the end of the design process or after the network is already operational, designs will be more scalable and robust."

The The International Organization for Standardization (ISO) defines five types of network management processes (and some examples for each):

• Performance management: Cricket
• Fault management: Syslog, SNMP Traps
• Configuration management: Rancid
• Security management: Radius, Tacacs, Snort
• Accounting management: Radius

[/OpenSource/Debian/Monitoring] permanent link

2007 Jun 25 - Mon

Cisco Device Health

In Cisco's book, Top Down Network Design, one useful show command is 'show buffers'. Some useful SNMP statistics include:

• BusyPer. CPU busy percentage in the last 5-second period.
• AvgBusy1. 1-minute exponentially decayed moving average of the CPU busy percentage.
• AvgBusy5. 5-minute exponentially decayed moving average of the CPU busy percentage.
• LocIfInputQueueDrops. The number of packets dropped because the input queue was full.
• LocIfOutputQueueDrops. The number of packets dropped because the output queue was full.
• LocIfInIgnored. The number of input packets ignored by the interface.
• BufferElMiss. The number of buffer element misses. (You can also check misses for small, medium, big, large, and huge buffer pools.)
• BufferFail. The number of buffer allocation failures.

I've been doing most of my snmp statistics gathering on 5 minutes intervals. On some interfaces, it may be of value to step that up to 1 minute intervals. Of course, if my total collection time is over 1 minute, I may have problems with that.

From the same book, is this interesting statistic about why Window's file transfers over WAN links can go only 'so fast'. SMB acts like a ping-pong protocol. It can only send up to 32KB before requiring an acknowledgement. So if the delay is 50 ms end to end, and ignoring client and server delays, a client can receive at most 32 KB every 100 milliseconds, or 320 KB per second. This means that the maximum throughput is 2.56 Mbps, at best.

[/Cisco] permanent link

2007 Jun 24 - Sun

Perl Tools

In an earlier article, I wrote about an IP Address Management database. To manage the data framework through the web, I'm thinking it might be well to use Catalyst Web Framework, a Perl based Model/View/Controller designed for data based web infrastructures. An Introductory Article introduces Catalyst use along with Ajax.

In another vein, there is an article on how to do Perl Scripts for Perl-Less People from Cygwin. It looks to be as simple as doing:

perlcc -o foo.exe foo.pl

and then running the result to find the dependent dlls.... then packaging the exe and dlls together and shipping them off to the poor soul who needed the solution in the first place.

[/OpenSource] permanent link

Cygwin Customizations

The default shell for Cygwin uses a DOS command shell, which has limited usefulness. To enhance the console experience, here are some customizations. To enhance the shell prompt, use the Cygwin installer to install 'rxvt'.

The icon that the Cygwin installer puts on your desktop has a link to a start up file. I've put the following into my startup file in order to start up rxvt as the default terminal. As you see, I've also installed 'nano' and set it as the default editor.

@echo off
C:
chdir C:\cygwin\bin
set EDITOR=nano
set VISUAL=nano
set CYGWIN=codepage:oem tty binmode title
rxvt -fg green -fn "Lucida Console-12" -geometry 80x50 -sl 1000 -sb -e bash --login -i

The installer may or may not place a few other relevant files in your home drive: .bash_profile, .bashrc, and .inputrc. These can be found in /etc/skel and should be copied to your home directory. They can then be edited to taste.

For copying text in the rxvt terminal window, simply highlight the text while holding the left mouse button. It will be automatically copied to the clipboard once the mouse button is released. To paste the text, hold the shift key while pressing the left mouse button. This is slightly different from putty, which does not require the shift key to be held. To select a range of words you can double click on the first word, then set the mouse cursor to the right of the last word or letter you want copied, and right-click.

Here is a Cygwin Prompt Here method.

Use cygwin to run crontab jobs.

The following can be added to .bashrc to pipe any command to less (so ls):

#A function to pipe any command to less:
function so {
eval "$@" |less -I~
}

A Cygwin FAQ has some useful comments regarding mkpasswd, Octave, enhanced Bash profiles, X, and csh.

[/OpenSource] permanent link

TACACS + Linux

Up till now, I thought that TACACS use was limited to authentication/authorization of logins and commands on Cisco devices. There is an article at Freemode.net regarding TACACS + Linux for authentication.

[/OpenSource/Linux] permanent link

Trading Site of the Day -- Blog on Blogs: The 24/7 Wall St. Twenty-Five Best Financial Blogs

A site called 24/7 Wall Street, which features "Insightful Analysis and Commentary for U.S. and Global Equity Investors", has an article called The 24/7 Wall St. Twenty-Five Best Financial Blogs. These are good starting off points for further research into the daily markets.

[/Trading/SiteOfTheDay/D200706] permanent link

Debian Sample Network Configuration

For my future reference.

######################################################################
# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)
# See the interfaces(5) manpage for information on what options are
# available.
######################################################################

# We always want the loopback interface.
#
auto lo
iface lo inet loopback

# To use dhcp:
#
# auto eth0
# iface eth0 inet dhcp

# An example static IP setup: (broadcast and gateway are optional)
#
# auto eth0
# iface eth0 inet static
#     address 192.168.0.42
#     network 192.168.0.0
#     netmask 255.255.255.0
#     broadcast 192.168.0.255
#     gateway 192.168.0.1

[/OpenSource/Debian] permanent link

Linux Backup Application: Mondo

Mondo, Mindo. To be expounded on shortly.

[/OpenSource] permanent link

Installing XWindows/KDE on Debian

For the initial installation of Debian, I simply do a 'standard' install. No desktop. I may also add database (which is PostgreSQL 8.1), dns, and web server (which is Apache 2.0).

After the reboot, I'll install some basic packages to get KDE running: apt-get install xserver-xorg-core apt-get install x-window-system apt-get install xserver-xfree86 apt-get install xserver-xorg apt-get install x-window-system-core apt-get install kde-core

Running the command 'startx &' will get XWindows going and the KDE personalizer started.

Putting 'startkde' in .xinitrc in my home directory sets the preference for kde. Other possible window managers include gnome, xfce4, icewm, fluxbox, fvwm (loosely ordered by decreasing order of preference).

[/OpenSource/Debian] permanent link

Debian Linux on a Mini-Box

I picked up a small form factor Mini-Box M200-LCD Barebone System. It is a nice small embedded systems unit with a backlit LCD and front panel buttons. I chose a unit with a 1GHz processor, 512MB of memory, and a 40GB harddrive. The unit does not have a floppy drive or a CDROM drive. It does come with ample USB slots. It also has a Compact Flash slot hidden behind a door on the front.

With a little bit of fiddling, I was able to Debian installed and running on the unit. Here is what I did to get it up and running.

The Compact Flash is configured as a Master device on the second IDE channel.

I chose to use a USB Key to boot strap the unit into network mode to get the full Debian distribution installed.

The USB key needs to be created on another Linux system, in this case I used a VMWare'd version on an XP box, . (WinImage was suggested to me, but I could never get it to work according to what little instruction I could find). In VMWare, I needed to associate the USB controller to the VMWare session. Windows saw the USB drive disappear as a result. When I inserted the USB Key into USB slot, the Linux console indicated the drive name to be used. In this case, it was /dev/sdc.

I downloaded the netinst iso image from the Debian package site. I also obtained the boot.img.gz boot image for the distribution. Pascal Le Bail has a web page called Installing Debian Sarge from a USB memory stick (USB key) where he explains the simple process. The instructions have been incorporated into the Debian Installation Manual.

Once the files are available, a short series of commands makes the key bootable:

zcat boot.img.gz >/dev/sdc1

gets the image onto the USB Key.

mount /dev/sdc1 /mnt
cp netinst.iso /mnt

gets the iso installed on to the key.

The USB key can then be plugged into a USB slot on the Mini-Box and the power turned on. In the bios, the secret is to:

• set USB ZIP as the first boot device
• in the USB settings, turn on USB legacy mode

The Mini-Box will boot in to the Debian installer and allow a network download/configuration of a Debian installation.

The interesting part of the Mini-Box is it's LCD/Button front panel. A sample program and SDK have been made available.

There is a demo program called usblcd available to show how things are done. The source and SDK can be downloaded and built.

Assuming Debian's build tools have not been installed, here are the build steps:

apt-get install autoconf
apt-get install gcc-4.1-base
apt-get install gcc-4.1
apt-get install gcc
apt-get install automake
apt-get install binutils
apt-get install binutils-dev
apt-get install libtool
apt-get install libusb-dev
apt-get install libhid-dev
 ./autogen.sh
make
make install

You can then use commands like:

usblcd backlight 1
usblcd backlight 0
usblcd backlight 1
usblcd led 2 1
usblcd led 2 0
usblcd clear

[/OpenSource/Debian] permanent link

2007 Jun 23 - Sat

Run Levels and Mode Switch Keys

Debian Admin has a some information regarding run levels (switching between single-user, console, and graphic modes). These are the basic run levels:

• 0 System Halt
• 1 Single user
• 2 Full multi-user mode (Default)
• 3-5 Same as 2
• 6 System Reboot

The file /etc/inittab has a line in it:

id:2:initdefault:

Supposedly, if you change the 2 to a 3, on boot the system will come up with a console instead of the graphical interface (assuming the graphical interface has been installed).

The command runlevel will return two values, a run level change since powering up, and the current run level.

To change run levels, the command 'telinit x' will change to run level 'x'.

An Introduction to Run-Levels contains further detailed information.

[/OpenSource/Debian] permanent link

2007 Jun 12 - Tue

Trading Site of the Day -- CASTrader Blog: A Quest for 50%-plus Returns

The complete byline for CASTrader Blog is "A quest for 50%-plus returns in the financial markets via Complex Adaptive Systems and Quantitative Finance.".

High aspirations, but looks like he still maintains a day job. But I think there are useful tidbits scattered through the site.

The author of the site appears to be attempting the quant side of trading in the equity markets. Based upon my own evolutions, I'm thinking he may be getting too complicated. But them who am I to say. I'm still getting my own infrastructure together, based upon a simpler volatility trend/range trading idea, something that is going to take another week or two to code. And yep, I too, have a day job.

[/Trading/SiteOfTheDay/D200706] permanent link

2007 Jun 10 - Sun

Trading Site of the Day -- C++ Fast Delegates, plus Boost

No, C++ Fast Delegates aren't really trading algorithms, but they do help make trading algorithms fast.

In the C# world, the C# language has the built-in capacity to store multiple object/method pointers and call them in response to some sort of activity. This is generally referred to the Observer Pattern of event based programming. In C#, the structure is called a Delegate.

C++ does not provide a native Delegate structure. Don Clugston wrote an article called Member Function Pointers and the Fastest Possible C++ Delegates. I've used this method successfully with Microsoft Visual Studio 2005.

Don's implementation is unable to do multicast delegates. Sergey Ryazanov wrote an article called The Impossibly Fast C++ Delegates that supposedly is more C++ compliant and has a multicast capability.

JaeWook Choi took the work one step farther and implemented a multicast delegate that is a drop in replacement for the Boost C++ Libraries. Boost is a collection of free peer-reviewd portable C++ source libraries. JaeWook's library is faster than Boost, but slower than Don's library. But JaeWook's library is mulitcast and does offer up additional functionality. Timings and feature sets can be reviewed in his article called Fast C++ Delegate: Boost.Function 'drop-in' replacement and multicast.

[/Trading/SiteOfTheDay/D200706] permanent link

2007 Jun 09 - Sat

Memory Leak Detection in Visual Studio 2005 C++ Applications

In some recent software development, when working with C++ in Visual Studio 2005, when ending a program which was run in the debugger, I'll sometimes get statements regarding memory leaks.

Each memory leak has a unique number associated with it. If these leaks and their numbers are repeatable, they can be tracked in Visual Studio by using the '_CrtSetBreakAlloc' call in your code.

In your code, near the beginning of execution, place the call, using the leak number as a parameter. This effectively creates a break point on the specific memory allocation call. When your code allocates the memory allocation, the break point will allow you to inspect the code and figure out why an appropriate dispose was not provided.

[/Personal/SoftwareDevelopment] permanent link

2007 Jun 06 - Wed

Writing Callmanager Call Detail Records to Excel Spreadsheet

In an earlier article, I described how to access the Cisco Callmanager CDR tables to obtain billing records through the use of the Sybase DBI drivers for Perl. The database needs to be placed into "mixed mode" for this type of thing to work.

I've created two perl modules to make use of this technique: cdr2file.pm and billing2file.pm.

cdr2file.pm takes as input three parameters: an extension, a starting date, and an ending date. It creates an Excel spreadsheet using a temporary file name, which allows multiple requests to be processed simultaneously. The spreadsheet contains four worksheets: a list of inbound calls, a list of local calls, a list of long distance calls, and a list of internationally placed calls. The selection is based upon four digit extensions and the North American Dialling plan. The duration of each call is included. Because both the 'Original Called' and the 'Final Called' fields are included, one can determine which calls ended up going to voicemail or transferred elsewhere.

The other module, billing2file.pm, takes as input a starting date and an ending date. For all outbound calls placed within that date range, a summary spreadsheet is created. The first worksheet contains a summary of call duration by user with the following fields: Extension Number, Name of the Phone, Total Duration in Seconds, Total Duration in Minutes. Each line contains a hyper link to a worksheet containing the actuall call details for that extension. On the detail worksheet for each user, the following details are provided: Date the call was placed, extension number originating the call, the original called number, the duration in seconds for the call, the phone identifier, and the phone description. This spreadsheet makes it convenient to produce monthly billing reports by user. For large companies, this may not be suitable as a spreadsheet may not be able to handle enough rows or worksheets (a limitation of Excel).

By creating suitable web pages, say through mod_perl or mason, members of the accounting department have direct, easy, live access to the Callmanager Call Detail Records in a ready to use format.

[/Cisco/Callmanager] permanent link

2007 Jun 05 - Tue

Sliding Window Statistics

Processing bars can lag data somewhat. Bars can also introduce sharp changes in statistics. Instead, I prefer to work with Trades and Quotes directly. But I still need the ability to work with summary statistics.

I've created an O(1) running time mechanism for processing trades and quotes in real time. The resulting code is provided in Accumulation.cs. The basic class I use is SimpleAccumulationStatsOnQuotes. The class is instantiated as an object by defining the window width in seconds. The class will maintain a list of quotes. As quotes are added, they are added to the summary statistics. As quotes fall outside the window, the quotes are removed from the summary statistics.

The summary statistics provide the basics for maintaining Bollinger Bands for the selected moving window time frame.

The class AccumulateQuotes is a brute force approach to calculating slope change velocities and acceleration. I think, with suitable modification of the basic statistics, this class would not be needed.

Running statistics are maintained through a reference to the RunningStats class.

If you wish to know the Minimum/Maximum levels over an interval, a RunningMinMax class helps. It is more of a manual process, with the AddPoint and RemovePoint methods.

[/Trading/SmartQuant/Articles] permanent link

2007 Jun 02 - Sat

Tag Line, Found on GrokLaw

Simulation engineers do it with models virtually every day!

[/Personal/TagLines] permanent link