2006 Dec 29 - Fri

Emailing Call Detail Records

In a number of earlier blog entries, I developed and posted a Perl script named ciscowatcher.pl. The script attaches to a Linux based Syslog service and listens for Cisco device generated syslog messages.

One of the messages listened for is the %VOIPAAA-5-VOIP_CALL_HISTORY message generated by H.323 voice gateways. The script generates two Postgresql database files. One is a raw data file called calllog, which contains call-leg information, one call-leg is a voip leg and the other leg is a PSTN leg. With those two legs, one can determine the calling party and the called party. The script generates the database table called cdr from these two pieces of information.

This information is useful for Cisco Telephone Solutions based upon Cisco Unified Callmanager Express Systems, which do not have a real Call Detail Record capability.

Even for organizations with Cisco's Callmanager Solution, getting Call Detail Records can be a problem sometimes because of the way the SQL service is authenticated.

To tie the bow on this situation, I've added a Perl script called sendcdr.pl. It takes the information form the cdr table, formats it, and writes it out as an Excel spreadsheet file using the Spreadsheet::WriteExcel Perl Module from CPAN. The Spreadsheet module is surprisingly powerful in that it can provide bolded and centered column headers as well as properly formatted date/time stamps.

Once the spreadsheet file is generated, it is then emailed using the Mail::Sender Perl Module, also from CPAN.

By placing the script into a cron job entry, call details records can be automatically emailed on a regular basis. The only thing missing is a utility to purge call detail records once they have been emitted. That will be a subject for my next revision.

[/OpenSource/Debian/Monitoring] permanent link

2006 Dec 28 - Thu

Interim getDevConfig Files for Cricket

I've made some further modifications to the Acktomic's genDevConfig files. One fix was to get the H.323 dial-peers to show with current versions of Cisco's IOS. I've also added some target-type code so that the various errored-second counters on the T1 interfaces in Cisco's VWIC modules can be displayed.

On the T1 controllers, the following counters are available from the 'current counter' set, and are shown as guages:

• dsx1CurrentESs: Errored Seconds
• dsx1CurrentSESs: Severely Errored Seconds
• dsx1CurrentSEFSs: Severely Errored Framing Seconds
• dsx1CurrentUASs: Unavailable Seconds
• dsx1CurrentCSSs: Controlled Slip Seconds
• dsx1CurrentPCVs: Path Coding Violations
• dsx1CurrentLESs: Line Errored Seconds
• dsx1CurrentBESs: Bursty Errored Seconds
• dsx1CurrentDMs: Degraded Minutes
• dsx1CurrentLCVs: Line Code Violations

My earlier instructions have a section on replacing various lines. Rather than go through that again, here are the files that need to be replaced:

• /usr/share/cricket/util/genDevConfig
• /usr/share/cricket/plugins/genConfig/CiscoIOS.pm
• /etc/cricket/config/Defaults.cisco.oneunified
• /usr/share/cricket/lib/genConfig/Utils.pm

Francois from Acktomic will be incorporating these changes into his version. I'll post an update when his collection has been updated.

Here is a slightly revised command line for scanning voice gateways. There are two additional interace types that are ignored. The line also shows how to turn on some debugging statements to see what else is performed.

/usr/share/cricket/util/genDevConfig -2 -c snmpro --vendorint --loglevel debug --voip -d 22,81,101 vgw01

[/OpenSource/Debian/Monitoring/Cricket] permanent link

2006 Dec 27 - Wed

Book: New Trading Systems and Methods, by Perry J. Kaufman

Many people refer to the Achelis book for simple, straight-forward descriptions of technical analysis tools. I too have it on my primary bookshelf. However, lately, more often than not, I find myself reaching for Kaufman's book to get good background on the various ways of technically analysing trading options. Kaufman has chapters devoted to practically every indicator type you may encounter: chart reading, events, regressions, trending, momentum, oscillators, seasonality, cycles, patterns, multiple time frames, and advanced techniques. He then goes into some details regarding system testing, practical considerations, risk control, and diversification. As a wrap up, he provides some end-notes for the mathematically inclined.

There appear to be traders who will sit at their screen all day and watch for pattern based setups. It appears that many traders fall into this category, and the book is not for them.

Notes and blogs regarding people who do automated trading appear to be few and far between. In any case, this book is for the analytical crowd who need to prepare for the day's manual trades. It is also for the automated crowd who need the computer to do all the trading 'by-the-rules' in order to eliminate all forms of emotion from the trade.

I think you'll find a wealth of ideas you can mix and match to make a trading strategy uniquely your own.

Technical anlysis and automated trading strategy design takes much work and energy. A good chunk of statistics is practically mandatory (which the book does provide in various sections). This book fulfills only a portion of the overall knowledge someone will need build a winning trading strategy. Trader phsychology and money management skills will need to be learned elsewhere.

I'll give the book two thumbs up as it provides excellent details on the spectrum of technical analysis and provides references for the times you wish to flesh out the details. Mr. Kaufman must have a most amazing technical library, based upon the breadth and depth of descriptions, references, and citations he uses.

[/Trading/TechnicalAnalysis] permanent link

2006 Dec 24 - Sun

Merrill Patterns

On page 94 of Bollinger's Book: Bollinger on Bollinger Bands, he talks about matching various tradeable patterns developed by Arthur Merrill. In an earlier article on Peak Matching, I provided some code to match chart trading peaks and valleys in real-time. By using that code in addition to my Merrill Pattern Matcher code, Aurther Merrill's patterns can be computed live during a trading session. The PeakMatch code stores it's values in a DoubleSeries array from which the MerrillPattern code uses the last five values to compute the pattern. Then based upon the analysis provided in Bollinger's book and Merrill's book, one can use the information to influence trading styles.

The code is built upon a table driven decision format, which in turn is based upon the observation that Merrill's patterns can be described within a 5x5 grid where each of five peaks/valleys will fall into unique row. The rows are numbered 1 through 5. With five rows, there are 2 to the 5 combinations, ie, 32 combinations.

A set of five points is classified by placing the price point and an index into a sorted array. The concatenation of the sorted indexes is used to generate 'key' to be used in looking for the pattern in the pattern list.

[/Trading/SmartQuant/Articles] permanent link

2006 Dec 23 - Sat

IQFeed Provider for SmartQuant

I've recompiled the IQFeed Utilities to run with QD 2.3.3 and DC 2.1.5. The OneUnified.IQfeed.dll file is used for linking up to IQFeed for obtaining real-time Quote/Trade/Depth streams for QuantDeveloper.

The IHistory interface, is as of yet, still unimplemented. I'll see about rectifying that over the Chrismas Break. Hopefully.

The IQFeed client I'm currently using is IQFeed Client 4.2.0.2.

In the zip file is a test program called ticker.exe. It provides visibility into real-time data in three formats:

• A Level I ticker tape stream modelled upon Thomas Carr's 'The Tape-Reading Edge' in July 2005's issue of Technical Analysis of Stocks and Commodities.
• A Level II chart showing buy-side and sell-side Market Makers.
• A Level II price chart modelled after Interactive Brokers book chart.

IQFeed provides Level II data only on Nasdaq traded instruments.

One usage note, I'm not certain if I've done the simple thing of converting symbols into upper case yet in the ticker program, so you may need to enter them manually as upper-case before clicking on one of the buttons on the left hand menu.

In the ticker program, when viewing the Level II price action, there is one bug which occurs when the Bid overlaps the Ask. The overlapped level will show price counts incorrectly.

[/Trading/SmartQuant/Articles] permanent link

SmartQuant QuantDeveloper & DataCenter Release

The update meant for 'mid-week' didn't get posted till week-end.

SmartQuant has released revisions to DataCenter and QuantDeveloper. They are at the following revision levels:

DataCenter
Version 2.1.5 (22-Dec-2006) 

QuantDeveloper Enterprise Edition
Version 2.3.3 (22-Dec-2006) 

QuantDeveloper source code.
Version 2.3.2 (13-Dec-2006) 

[/Trading/SmartQuant/Releases] permanent link

2006 Dec 22 - Fri

Cisco Syslog Additions

I have once again updated ciscowatcher.pl.

A couple of primary additions:

• Watches for and emails on changes of the '%CONTROLLER-5-UPDOWN' event (for VWICs and such).
• Added a table to record Controller up/down events and current status
• Updated the database schema to generate Call Detail Records from Cisco H.323 gateway events
• Fixed a problem when processing certain VOIPAAA-5-VOIP_CALL_HISTORY records containing an extra ','

[/OpenSource/Debian/Monitoring] permanent link

VRF-lite (VRF's without MPLS)

VRF-lite (Virtual Routing & Forwarding) is a handy construct for keeping data segregated within a network. It can be thought of as kind of a meta-VLAN thing.

At one customer site, who has a number of branch offices, has a Cisco Callmanager solution spanning those sites. Each site has an internet connection. Three sites are in a single metro area and are linked with metro-links as well as tunnels. A fourth site is in another country. The three sites are linked to the fourth site through IPSEC tunnels. Four different providers are used.

From a real usage perspective, there are six tunnels of consequence: the three tunnels from the metro area to the single site, and the three return tunnels from the single site to the three grouped metro sites. Each of those six tunnels can be categorized in terms of a voice quality metric.

The design puzzle was to come up with a mechanism to route data across a set of tunnels and route voice across a different set of tunnels, in order to make best use of measured delay, jitter, and loss metrics. Using Policy Based Routing (PBR) was one solution but was rejected due to its scaleability problems (ie, lack thereof).

The idea that was finally implemented was to use VRF's to segregate voice from data, but still allow one to get to the other. The design yielded side benefits: provided a method to keep guest traffic separate, allows a method to get external addresses to different parts of the network, and provided a mechanism to improve secure access to network devices.

VRF's allow a routing process per VRF, thus facilitating the use of routing's built in automated route selection and fail-over. And by assigning costs to certain routes, those routes can be prioritized by whatever criteria you choose.

As indicated by another auditing consultant, the configuration does turn complex, but no suitable alternative solution was offered, which says something in itself. Perhaps others can offer up something better, I am open to suggestions.

In the example, 192.18.x.x addresses simulate external addresses, and 192.168.x.x simulate internal addresses. Interface s0/0/0 is the external internet interface, f0/0 is a trunked interface to the interior network, and f0/1 is an interface to the firewall

As you can see, there is an OSPF routing process per VRF, thus facilitating independant routing of each traffic type.

The example incorporates a bunch of features I've encountered in my searches in Cisco's documentation sites, Cisco's forums, Cisco's TAC, as well as Cisco's 'Implementing MPLS VPN' two volume book set.

The example highlights a number of functions:

• make internet connected links a member of an external vrf (vrfExterior)
• external addresses can therefore be routed between buildings and providers
• external addresses can therefore cross pollinate firewalls as a measure of redundancy
• make internal links members of the global routing table with RFC1918 addresses
• external addresses are kept separate from internal addresses
• allows flexible use of equipment when hardware budgets are restricted
• IPSEC encrypted tunnels are can carry VRF's between sites
• each inter-site tunnel requires a separate loopback (per Cisco's docs)
• a Guest VRF is kept totally separate from the Corporate Voice and Data networks
• Multiprotocol BGP is used route between vrfData and vrfVoice
• special attention must be paid to the redistribution statements to make best use of OSPF internal, external 1 and external 2 routes (external 2 is not redistributed)
• with careful redistribution, 'sho ip ospf vrf vrfVoice' shows which routes originate in the vrf and which originate outside, which facilitates troubleshooting and documentation
• some will argue that carrying guest traffic on the corporate hardware is a no-no, how do others feel about that?

If you have comments on how to improve the configuration, I'm all ears.

I hope this helps others to come up to speed with how VRF-lite can become a part of everyone's routing/switching toolset. And of course, I'm always open to consulting gigs if anyone would like some assistance.

!
ip cef
!
ip vrf vrfData
 description Server, Workstation, Printer data
 rd 65400:300
 route-target export 65400:300
 route-target import 65400:300
 route-target import 65400:400
!
ip vrf vrfExterior
 description Exterior BGP routed data
 rd 65400:200
 route-target export 65400:200
 route-target import 65400:200
!
ip vrf vrfGuest
 description Guest Wireless & Wired
 rd 65400:500
 route-target export 65400:500
 route-target import 65400:500
!
ip vrf vrfVoice
 description Voice Servers, Phones & Gateways
 rd 65400:400
 route-target export 65400:400
 route-target import 65400:400
 route-target import 65400:300
!
!
crypto keyring key4Exterior vrf vrfExterior
  pre-shared-key address 0.0.0.0 0.0.0.0 key 
!
crypto isakmp policy 10
 encr aes
 authentication pre-share
 group 2
!
crypto ipsec transform-set transForTunnels esp-aes esp-sha-hmac 
 mode transport
!
crypto ipsec profile profForTunnels
 set transform-set transForTunnels 
!
interface Loopback10000
 description OSPF ID
 ip address 192.168.5.254 255.255.255.255
!
interface Loopback10001
 description eBGP ID
 ip vrf forwarding vrfExterior
 ip address 192.18.164.191 255.255.255.255
!
interface Loopback10011
 description Global Routing
 ip vrf forwarding vrfExterior
 ip address 192.18.10.1 255.255.255.255
!
interface Loopback10012
 description vrfExterior
 ip vrf forwarding vrfExterior
 ip address 192.18.10.2 255.255.255.255
!
interface Loopback10013
 description vrfData
 ip vrf forwarding vrfExterior
 ip address 192.18.10.3 255.255.255.255
!
interface Loopback10014
 description vrfVoice
 ip vrf forwarding vrfExterior
 ip address 192.18.10.4 255.255.255.255
!
interface Loopback20000
 ip vrf forwarding vrfExterior
 ip address 192.168.5.224 255.255.255.255
!
interface Loopback30000
 ip vrf forwarding vrfData
 ip address 192.168.5.225 255.255.255.255
!
interface Loopback40000
 ip vrf forwarding vrfVoice
 ip address 192.168.5.226 255.255.255.255
!
interface Loopback50000
 ip vrf forwarding vrfGuest
 ip address 192.168.5.227 255.255.255.255
!
interface Tunnel100101
 description Global Building1 - Building2
 bandwidth 1000
 ip address 192.168.5.69 255.255.255.252
 ip ospf authentication message-digest
! ip ospf message-digest-key 1 md5 
 ip ospf mtu-ignore
 load-interval 30
 qos pre-classify
 keepalive 10 3
 tunnel source Loopback10011
 tunnel destination 192.18.20.1
 tunnel mode ipsec ipv4
 tunnel vrf vrfExterior
 tunnel protection ipsec profile profForTunnels
 max-reserved-bandwidth 100
 service-policy output pmShapeTunnels
!
!
interface Tunnel200101
 description vrfExterior Building1 - Building2
 bandwidth 1000
 ip vrf forwarding vrfExterior
 ip address 192.168.5.77 255.255.255.252
 ip ospf authentication message-digest
! ip ospf message-digest-key 1 md5 
 ip ospf mtu-ignore
 load-interval 30
 qos pre-classify
 keepalive 10 3
 tunnel source Loopback10012
 tunnel destination 192.18.20.2
 tunnel mode ipsec ipv4
 tunnel vrf vrfExterior
 tunnel protection ipsec profile profForTunnels
 max-reserved-bandwidth 100
 service-policy output pmShapeTunnels
!
interface Tunnel300101
 description vrfData Building1 - Building2
 bandwidth 1000
 ip vrf forwarding vrfData
 ip address 192.168.5.65 255.255.255.252
 ip ospf authentication message-digest
! ip ospf message-digest-key 1 md5 
 ip ospf mtu-ignore
 load-interval 30
 qos pre-classify
 keepalive 10 3
 tunnel source Loopback10013
 tunnel destination 192.18.20.3
 tunnel mode ipsec ipv4
 tunnel vrf vrfExterior
 tunnel protection ipsec profile profForTunnels
 max-reserved-bandwidth 100
 service-policy output pmShapeTunnels
!
interface Tunnel400101
 description vrfVoice Building1 - Building2
 bandwidth 1000
 ip vrf forwarding vrfVoice
 ip address 192.168.5.73 255.255.255.252
 ip ospf authentication message-digest
! ip ospf message-digest-key 1 md5 
 ip ospf mtu-ignore
 load-interval 30
 qos pre-classify
 keepalive 10 3
 tunnel source Loopback10014
 tunnel destination 192.18.20.4
 tunnel mode ipsec ipv4
 tunnel vrf vrfExterior
 tunnel protection ipsec profile profForTunnels
 max-reserved-bandwidth 100
 service-policy output pmShapeTunnels
!
interface Null0
 no ip unreachables
!
interface FastEthernet0/0
 no ip address
 ip route-cache flow
 load-interval 30
 duplex auto
 speed auto
 service-policy output pmFastEthernet
!
interface FastEthernet0/0.199
 description Global Routing
 encapsulation dot1Q 199
 ip address 192.168.5.26 255.255.255.248
 ip ospf authentication message-digest
! ip ospf message-digest-key 1 md5 
!
interface FastEthernet0/0.299
 description vrfExterior
 encapsulation dot1Q 299
 ip vrf forwarding vrfExterior
 ip address 192.168.5.106 255.255.255.248
 ip ospf authentication message-digest
! ip ospf message-digest-key 1 md5 
!
interface FastEthernet0/0.399
 description vrfData 
 encapsulation dot1Q 399
 ip vrf forwarding vrfData
 ip address 192.168.5.34 255.255.255.248
 ip ospf authentication message-digest
! ip ospf message-digest-key 1 md5 
!
interface FastEthernet0/0.499
 description vrfVoice
 encapsulation dot1Q 499
 ip vrf forwarding vrfVoice
 ip address 192.168.5.122 255.255.255.248
 ip ospf authentication message-digest
! ip ospf message-digest-key 1 md5 
!
interface FastEthernet0/0.599
 description vrfGuest
 encapsulation dot1Q 599
 ip vrf forwarding vrfGuest
 ip address 192.168.5.130 255.255.255.248
 ip ospf authentication message-digest
! ip ospf message-digest-key 1 md5 
!
interface FastEthernet0/1
 description to Firewall
 ip vrf forwarding vrfExterior
 ip address 192.18.10.11 255.255.255.248
 ip route-cache flow
 load-interval 30
 duplex auto
 speed auto
 no cdp enable
 service-policy output pmFastEthernet
!
interface Serial0/0/0:0
 description Internet Access
 no ip address
 encapsulation frame-relay IETF
 ip route-cache flow
 load-interval 30
 frame-relay lmi-type ansi
 max-reserved-bandwidth 100
 service-policy output pmShapeSerialInterface
!
interface Serial0/0/0:0.1 point-to-point
 description Frame Relay to internet
 ip vrf forwarding vrfExterior
 ip address 192.18.10.33 255.255.255.252
 ip access-group aclBlockInbound in
 ip access-group aclBlockOutbound out
 ip verify unicast reverse-path
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 no cdp enable
 frame-relay interface-dlci 170 IETF   
!
router ospf 200 vrf vrfExterior
 router-id 192.168.5.224
 log-adjacency-changes
 auto-cost reference-bandwidth 10000
 capability vrf-lite
 passive-interface FastEthernet0/1
 passive-interface Serial0/0/0:0.1
 passive-interface Loopback10001
 passive-interface Loopback10011
 passive-interface Loopback10012
 passive-interface Loopback10013
 passive-interface Loopback10014
 passive-interface Loopback20000
 network 192.168.5.56 0.0.0.7 area 0
 network 192.168.5.76 0.0.0.3 area 0
 network 192.168.5.92 0.0.0.3 area 0
 network 192.168.5.104 0.0.0.7 area 0
 network 192.168.5.224 0.0.0.0 area 0
 network 192.168.197.36 0.0.0.3 area 0
!
router ospf 300 vrf vrfData
 router-id 192.168.5.225
 log-adjacency-changes
 auto-cost reference-bandwidth 10000
 capability vrf-lite
 redistribute bgp 65400 metric 120 subnets
 passive-interface Loopback30000
 network 192.168.5.32 0.0.0.7 area 0
 network 192.168.5.64 0.0.0.3 area 0
 network 192.168.5.80 0.0.0.3 area 0
 network 192.168.5.112 0.0.0.7 area 0
 network 192.168.5.225 0.0.0.0 area 0
 network 192.168.197.40 0.0.0.3 area 0
!
router ospf 400 vrf vrfVoice
 router-id 192.168.5.226
 log-adjacency-changes
 auto-cost reference-bandwidth 10000
 capability vrf-lite
 redistribute bgp 65400 metric 120 subnets
 passive-interface Loopback40000
 network 192.168.5.48 0.0.0.7 area 0
 network 192.168.5.72 0.0.0.3 area 0
 network 192.168.5.88 0.0.0.3 area 0
 network 192.168.5.120 0.0.0.7 area 0
 network 192.168.5.226 0.0.0.0 area 0
 network 192.168.197.44 0.0.0.3 area 0
 default-information originate always
!
router ospf 500 vrf vrfGuest
 router-id 192.168.5.227
 log-adjacency-changes
 auto-cost reference-bandwidth 10000
 capability vrf-lite
 passive-interface Loopback50000
 network 192.168.5.40 0.0.0.7 area 0
 network 192.168.5.68 0.0.0.3 area 0
 network 192.168.5.84 0.0.0.3 area 0
 network 192.168.5.128 0.0.0.7 area 0
 network 192.168.5.227 0.0.0.0 area 0
!
router ospf 100
 log-adjacency-changes
 auto-cost reference-bandwidth 10000
 passive-interface Loopback10000
 network 192.168.5.4 0.0.0.3 area 0
 network 192.168.5.16 0.0.0.3 area 0
 network 192.168.5.20 0.0.0.3 area 0
 network 192.168.5.24 0.0.0.7 area 0
 network 192.168.5.68 0.0.0.3 area 0
 network 192.168.5.80 0.0.0.7 area 0
 network 192.168.5.254 0.0.0.0 area 0
 network 192.168.197.32 0.0.0.3 area 0
!
router bgp 65400
 no synchronization
 bgp router-id 192.18.164.191
 bgp log-neighbor-changes
 no auto-summary
 !
 address-family ipv4 vrf vrfVoice
 redistribute ospf 400 vrf vrfVoice
 no synchronization
 exit-address-family
 !
 address-family ipv4 vrf vrfData
 redistribute ospf 300 vrf vrfData match internal external 1
 no synchronization
 exit-address-family
!

end

[/Cisco] permanent link

2006 Dec 21 - Thu

Scalping, Elite Trader, and Range Bars

On the Elite Trader Forums, a poster that goes by the moniker of yoohoo has quite a few useful things to say about scalping.

Scalping is definitely an intraday activity. One attempts to capture small movements in the market, often of a one or two points. And one has to remember to make sure that the movement includes room for slippage and for commissions.

I've heard a number of definitions for the term 'point'. In those forums, I'm now given to understand that a point can be a synonym for the spread for the equity.

Yoohoo indicates that he's been trading since about 1998, so I think he has quite some experience. His posts certainly indicate that he is drawing on much hard earned knowledge. For traders trying to get on the positive side of the markets, his comments, as well as many others, are worth reading over at the forums.

As part of the set of indicators he uses, I learned of a new one: range bars. According to the article, range bars were invented by a Brazilian trader named Vicente M. Nicolellis Jr. In a nutshell, range bars are created through price movements: once an instrument has moved through a preset price range, a new bar is created. This creates a series of same sized bars, and is time independent. As a result, a trending market will generate a series of range bars as it goes up or down, while a sideways market will generate few if any bars (as long as the sideways movement is within the range of the bar).

[/Trading] permanent link

2006 Dec 20 - Wed

Cisco Syslog Additions

I have once again updated ciscowatcher.pl.

A couple of primary additions:

• Additional events are processed
• Email messages can be sent as alerts based upon nature of event

Some of the events processed include:

• '%ISDN-6-CONNECT'
• '%ISDN-6-DISCONNECT'
• '%VOIPAAA-5-VOIP_CALL_HISTORY'
• '%SEC-6-IPACCESSLOGDP'
• '%SEC-CLUSTER_MEMBER_1-6-IPACCESSLOGDP'
• '%SEC-6-IPACCESSLOGP'
• '%SEC-CLUSTER_MEMBER_1-6-IPACCESSLOGP'
• '%OSPF-5-ADJCHG'
• '%SYS-5-CONFIG_I'
• '%LINK-5-CHANGED'
• '%LINEPROTO-5-UPDOWN'
• '%LINK-3-UPDOWN'
• '%LINEPROTO-CLUSTER_MEMBER_1-5-UPDOWN'
• '%LINK-CLUSTER_MEMBER_1-3-UPDOWN'
• '%CRYPTO-4-PKT_REPLAY_ERR'
• '%SYS-6-CLOCKUPDATE'
• '%DOT11-6-ASSOC'
• '%DOT11-6-DISASSOC'
• '%DOT11-7-AUTH_FAILED'
• '%DOT11-6-ROAMED'

The email module example shows the the script sending email to the default service on the same machine. By simply changing the mail accounts and server id's, email can be sent to any SMTP capable server for distribution.

This script is configured to send notifications and save the results in a database anytime an OSPF link changes state.

[/OpenSource/Debian/Monitoring] permanent link

Volume At Price

It is said that, through the course of the day, trading trends will revert to the level of highest volume.

The software at Ensign Software has a feature which will chart a nice bar graph of the volume-at-price distribution for visual traders.

However, for a quick and dirty indicator at a single level only, fit for automated trading, I wrote the following small class:

public class VolumeAtPrice {
	
	SortedList slVolumeAtPrice;
	public int LargestVolume = 0;
	public double PriceAtLargestVolume = 0;
	
	public VolumeAtPrice() {
		slVolumeAtPrice = new SortedList( 400 );
	}
	
	public void Add( Trade trade ) {
		if ( slVolumeAtPrice.ContainsKey( trade.Price ) ) {
			int ix = slVolumeAtPrice.IndexOfKey( trade.Price );
			int volume = (int) slVolumeAtPrice.GetByIndex( ix );
			volume += trade.Size;
			slVolumeAtPrice.SetByIndex( ix, volume );
			if ( volume > LargestVolume ) {
				LargestVolume = volume;
				PriceAtLargestVolume = trade.Price;
			}
		}
		else {
			slVolumeAtPrice.Add( trade.Price, trade.Size );
			if ( trade.Size > LargestVolume ) {
				LargestVolume = trade.Size;
				PriceAtLargestVolume = trade.Price;
			}
		}
	}
}

After updating with the latest Trade, examine PriceAtLargestVolume to see where the current highest volume trading level occurs.

[/Trading/SmartQuant/Articles] permanent link

2006 Dec 19 - Tue

SmartQuant QuantDeveloper & DataCenter Release

Since Anton mentioned this blog on the SmartQuant forum earlier today, I better get the version list up-to-date here.

There was mention that another update will be released mid-week this week. Among other things, it is said to fix a couple of bugs plus allow scripted access to DataCenter MarketDepth from QuantDeveloper.

SmartQuant has released revisions to DataCenter and QuantDeveloper. They are at the following revision levels:

DataCenter
Version 2.1.4 (27-Nov-2006)

QuantDeveloper Enterprise Edition
Version 2.3.2 (08-Dec-2006) 

QuantDeveloper source code.
Version 2.3.2 (13-Dec-2006) 

[/Trading/SmartQuant/Releases] permanent link

The Stock Bandit

I've added a link to an interesting article by The Stock Bandit regarding trading with multiple positions. I've been attempting to design a system based upon handling single positions. But with multiple indicators sometimes signalling different directions, I've been at a loss (in more ways than one) to figure out which indicator to believe at any one point in time.

I've slowly been realizing that by handling multiple positions, multiple scenarios can be played out at once, with each scenario adding and removing positions to the overall portfolio.

Obviously, handling multiple positions is not for the faint of heart, but I'm hoping through robust back testing, it will prove it's worth for use in a real money environment.

[/Trading] permanent link

2006 Dec 07 - Thu

World Money Show, February 2007

InterShow, which bills themselves as 'The Worlds Leading Producer of Investment Trade Shows and Cruises', is putting on the World Money Show at the Gaylord Palms Resort in Orlando, Florida from February 7 to 10 in 2007.

The exhibit hall listing is quite extensive, as is their speaker list. Their Sponsor list has some well known names. If anyone has gone to one of these, please leave a comment as to your thoughts regarding past events.

[/Trading] permanent link

2006 Dec 06 - Wed

Finding Local Peaks in Quote/Trade Streams

Just about any book on technical analysis you open will have a number of charts showing the usual peaks and valleys of a instrument's trading range. Many trading strategies are designed around the specific arrangement of peaks and valleys. I thought, up till now, that these peaks and valleys could only be determined through studies through the use of the good old Mark I Eyeball.

However, after reading Bollinger's book entitled Bollinger on Bollinger Bands, in which he discusses computer aided determination of those peaks and valleys, I set out to work on an algorithm to do the same.

The C# PeakMatch Code Segment is my first attempt at peak and valley pattern matching.

It is implemented as a state machine in order to make it easy to determine, during live streams, whether the stream is going up or down. You can use various summary statistics from quotes, trades, or even bars as input values.

The variable dblPatternDelta is the grey zone used for determining when the pattern flips. This variable will need to be adjusted on an instrument by instrument basis. In addition, further tuning is necessary if you wish to capture small nuances or just large swings in the trading value. As such, determining the peak is a problem of lag. Sigh, so much for having a magical realtime signal for determining when the top or bottom of a range. The variable dtPatternPt1 holds the DateTime of the last determined peak. When a change of direction of determined, then that peak/valley attribute is stored in dsPattern.

dsPattern can then be used within a sliding window of pattern analysis. I'll show an implementation of Arthur Merrill's patterns in another article.

Further information on choosing a good value for dblPatternDelta can be found in Bollinger's book.

[/Trading/SmartQuant/Articles] permanent link

2006 Dec 05 - Tue

Google Webmaster Tools

I cover a number of different topics in my blog. I'm interested in finding out how people arrive on those pages. From a Google perspective, they have a convenient tool for analyzing some of this information.

Google has a site called Google Webmaster Central. On that page is a link for Webmaster Tools (including Sitemaps). I've added my web site to the tool. As part of that process, they generate a verification key. I've installed the verification key in the footer file that gets included on all generated webpages on my site. This verification key is part of a script that talks to Google each time a web page is rendered by a web browser.

As part of each rendering, information is sent to Google that logs the queries used to get to the specific page. Google summarizes this information and reports this to me each time I visit the Google Webmaster Tools.

I've also uploaded a simple site-map so Google knows which pages to visit.

Now that I've provided some basic information to Google, I can now get an idea of how pages are crawled, when they are crawled, and how the pages are ranked within Google's system.

I also have the ability to download a spreadsheet of queries Google has recorded for reaching each page set. This will help me to optimize the keywords in each document. It also gives me an idea of people's interests, and thereby provide additional content focussed to those topics of interest.

I have a Google search tool at the top of my pages. The searches entered in that box are also recorded so I can tell for what people are searching once they reach my site.

There is also a set of index statistics provided which include: the pages indexed by Google, sites that Google knows that link to my site, and presents a list of sites related to my site so I can have an idea of who my competition might be.

I've found the Google 'link:' not quite as detailed as I think it should be. Google has maybe twelve sites showing information for link:www.oneunified.net. On the other hand, Alexa shows around 74 sites linking.

Anyway, for a high level view of what GoogleBot thinks about my site, Google Webmaster Central provides a good summary.

[/Personal/SoftwareDevelopment/HTML/Findability] permanent link

NTP for Callmanager

Since Cisco generally recommends that the Callmanager server not be a member of an Active Directory domain, is is unable to make use of the time distribution protocols normally available to member servers and computers. As background, I think Cisco's stance is understandable, and reflects the quantity of testing Cisco performs with each upgrade and service pack install. If a server is a member of a domain, it is entirely possible that registry entries, dlls, or applications coulds be modified through Group Policies that may affect the reliable functioning of the phone server.

As an alternative to Microsoft's time distribution protocol, the industry standard NTP (Network Time Protocol) can be used. The Callmanager installation automatically installs an NTP client, but does not turn it on.

To turn it on is quite simple. Obtain an address of an NTP server, either on your network or an external one. Using the one that drives the Active Directory domain controllors would be best. As a last resort, an address from pool.ntp.org could be used.

Edit the file C:\WINNT\system32\drivers\etc\ntp.conf and apply the ip address to the server line. Save the file. Bring up Microsoft's Services management application. Set the Network Time Protocol service to Automatic and start it up.

The Event Log will record startup and synchronization messages for your reference.

Be aware that with each Callmanager upgrade you perform, the service may be turned back to Manual and the ntp.conf file may be overwritten. You may want to make a backup of the configuration file so you can easily re-apply after your next upgrade.

[/Cisco/Callmanager] permanent link

Ping Weblog

I used the ping_weblogs_com found and described at Get Noticed by Pinging site. The referenced page also lists a number of services to ping.

Adding a ping to weblogs.com was reasonably simple, except that I forgot that I can't do a $ping_weblogs_com in the text article, but had to do it as part of a flavour file.

And as I didn't want to do it as part of a regular article presentation, as it may extend the page load times, I had to create some separate flavour files. I then activated the plugin in the story flavour.

Now I'm all set with to ping Weblogs.com.

BrutalHogs has a ping flavour to help a little with the manual ping process. But I think that Ping-O-Matic probably takes care of most things, even if it is only manually for now.

[/OpenSource/blosxom] permanent link

Adding an RSS Template to Blosxom

The RSS template/flavour supplied with the original installation of Blosxom doesn't really conform to modern useage. At The Unofficial Blosxom User Group, mention is made of Hal Halvorsen's A better RSS 2.0 template for Blosxom.

He mentions it might take ten to fifteen minutes to configure. In actual fact it was easier than that. I copied his flavours into the root of my blog directory and made the following changes:

• story.rss20: added .article to fn in two places, since that is how I suffix each individual blog article from a url perspective
• head.rss20: changed the copyright notice
• foot.blog: changed my index.rss link to index.rss20 for my xml tag

I was able to validate the feed with Weblogs.com Feed Validator.

So, in actual fact, adding the feed ability and documenting it in this blog took maybe fifteen or twenty minutes. Easy.

[/OpenSource/blosxom] permanent link

2006 Dec 04 - Mon

Keywords for Search Engine Optimization (SEO)

Search engines like keywords, keywords that are balanced, not overused, not underused. Both copy writers and site designers need to be involved in optimization of keywords and phrases. Once appropriate keywords and phrases have been determined (more about this in another entery), copy writers need to work them into the text of the website. In addition, the keywords and phrases can be incorporated into meta-tags of the page, but I've heard that you can't overboard here, nor can you rely only on meta-tags to carry the day.

Another trick is to incorporate the keywords into web site structure and navigation. For example, on one my pages I discuss a set of HD-600 headphones by Sennheiser. The page references the word AudioPhonics and the directory structure in which the article rests has that word as part of the path.

That article has a Google rank of 7 or 8. It has reached that rank within a few weeks of posting. However, that page is actually about the Sennheiser HD 600 headphones, and if I do a search on that phrase, page rank is very very low. It is somewhere between 140 and 745,000. So, in effect, people will land on that page due to a search of the nebulous topic of AudioPhonics rather than the more specific topic of Sennheiser HD 600 Headphones about which I wrote.

Because there are so many other pages out there that have Sennheiser HD 600 in the title bar, which my page doesn't (I'm going to have to move to different blogging software to make that happen), they probably enjoy higher rankings. However, an appropriate title is only one of many attributes a search engine will use to compute a page rank.

Those other pages also probably use meta-tags.

Those other pages may also have inbound links of one form or another to gain them extra visibility.

Those other sites may also have related content that helps a search engine's related content matchers. (I'm still finding out about this one to see how true it is).

But you'll also note that when doing a search on the phrase 'Sennheiser HD 600', a bunch of other kind of related pages show up: ones that have a portion of the phrase repeatedly, ones that include the phrase in passing but related to similar equipment, and ones that have them in the url somewhere.

So, after having mentioned what search engines like, we'll see what happens with this page in the page rankings in a few weeks for the search topic 'Sennheiser HD 600'. I've mentioned the product multiple times within the text of this document, which is what Search Engines seem to like. I'm still missing the text in meta-tags, the title, and the url, but we'll see how things go. It will be form of double irony that people come here because they searched for 'Sennheiser HD 600' when in fact this page is actually is about Search Engine Optimization, in another form: Optimzation for Search Engine Findability.

In summary, if you are looking for visibility and findability, each page you write needs to be hand crafted with these various techniques in mind. And even if you've crafted a page for a certain phrase, you may find a search engine likes it differently based upon user's search patterns and preferences. But if you watch these patterns, you can use them to your benefit to further enhance your findability.

[/Personal/SoftwareDevelopment/HTML/Findability] permanent link

Book Review: Ambient Findability by Peter Morville

As I've been adding content to my blog, I've also been looking into methods and mechanisms of promoting it .

Over the last few years, I've been receiving JIll Whalen's High Rankings Advisor Newsletter. She devotes her time to educating her readers in the subject of SEO (Search Engine Optimization). I've been reading the newsletter ever since the time that meta-tags had significant meaning. Now they represent only a portion of toolset needed to successfully promote a web site.

Web site promotion is really all about 'findability'. Peter Morville presents this topic in a very readable book called 'Ambient Findability' available from O'Reilly Press. The sub-title of his book is 'What We Find Changes Who We Become', which I think is very apt, based upon the research he presents within the pages of his book. We as Blog authors and web page developers provide content so that we can share it with others, and perhaps make some money from it, if not for other altruistic reasons. As they say, content is king. But if no one knows about the content, what good does it do? So the key question is: how does one get others to visit? Obviously, other's can't visit if they don't know where to visit. One concept that comes immediately to mind is a one called viral marketing. Marketing Terms defines it as a 'marketing phenomenon that facilitates and encourages people to pass along a marketing message'. If you can get a snowball of a message rolling, people will come.

On the other hand, in order to build a base of respectability and long term associations, any number of ideas and philosophies need to be considered. No one promotion gimmick will yield results. Around 175 pages of book can be boiled down to this meaning laden quote:

Semantic Web tools and standards create a powerful, enduring foundation. Taxonomies and ontologies provide a solid semantic network that connects interface to infrastructure. And the fast-moving, fashionable folksomonies sit on top: flexible, adaptable, and responsive to user feedback.

The book is much more readable than what is found in that exerpt, but hopefully the exerpt provides a desire to find out more of what it takes to build a successful web site. Search Engine Optimization is part and parcel of semantic content. Tag building through such sites as Flickr and Del.icio.us provide the folksomony, or user ratedness of a site. And search engine classification schemes provide accessibility.

Many different topics related to finding things are presented within the book. It is an excellent first book for those involved with the multi-role task of making web sites findable, as well as usable.

[/Personal/SoftwareDevelopment/HTML/Findability] permanent link

2006 Nov 30 - Thu

C# Essentials (2nd Edition) by Ben Albahari

Cruising through book stores, I usually encounter the 800 page behemoths that 'teach you programming in 24 hours' or something similar. I suppose those are good for getting you programming with lots of examples.

However, I like to think I'm a pretty good programmer, having grown up with Pascal, C, and C++. I may be aging myself with that first one, but anyway. So in order to pick up a new language, I don't really need a lot of hand holding. I simply need something to show me the syntax and basic flavour of the language. After that, I can start developing applications quite quickly. Getting used to the libraries of the language is something else altogether.

I had thought C# was a toy language, ranking right up there with Visual Basic. That was until I encountered a powerful .NET financial development package out there on the 'net from SmartQuant. That started me thinking that there must be something to this language. I started reading The C# Essentials on one my connections to SaharaBooks online. It was good enough online that I purchased the book as a handy desktop reference. It is a nice slim volume, packed with relevant information. Having a programming background, I was able to quickly grasp the basics of the language as they compared to what I already knew. The concepts of delegate functions and events took a while to wrap my head around, after being used to C++'s explicit pointers and function passing mechanisms. Once understanding the power of events, and how they manage multiple registrations as well as static and object based instantiations, I was sold. However, I think C# loses it's power due to de-emphasizing the deconstructor and reverting to automated garbage collection. I can see the benefits, but I enjoyed the manual tuning I do with C++. Well, having digressed to the language itself, now back to the book. The book covers the language itself, in what I think is a very fine balance. The examples are short, sweet and succinct in showing many of the fine points of the language specific it is covering.

I must admit though, that there are language features discussed in the book that do have examples, but still leave me wondering what they mean and how they fill in the big picture. It takes a little time to unravel the nuances of meaning rolled into the example. I think the meanings will will fall into place as my experience grows, and I find scenarios where they start to make sense.

The book does not cover the .NET run-time library. That is something best left to the 800 page reference behemoths, or simply the online reference library provided by the Integrated Development Environment.

I give the book two thumbs up. After a year of programming C#, it is still my primary quick reference on basic language idioms.

[/Trading/AutomatedTrading] permanent link

2006 Nov 28 - Tue

Cisco Syslog Additions

I have updated ciscowatcher.pl and ciscowatcher.sql.

There are now six tables that are updated from syslog event information:

• ActiveCalls: shows the in progres calls and the interface to which they are assigned
• CallLog: Final Call Leg accounting information
• Counters: keeps track of how many calls are connected (for use in Cricket)
• InterfaceStatus: Tracks link and protocol status on an interface basis (for use in a weathermap)
• OspfStatus: maintains OSPF Neighbor information (for use in a weathermap)
• WirelessAssoc: tracks, by mac address which devices are associated with which AP

[/OpenSource/Debian/Monitoring] permanent link

SmartQuant QuantDeveloper & DataCenter Release

SmartQuant has released revisions to DataCenter and QuantDeveloper. They are at the following revision levels:

DataCenter
Version 2.1.4 (27-Nov-2006)

QuantDeveloper Enterprise Edition
Version 2.3.1 (27-Nov-2006) 

QuantDeveloper source code.
Version 2.2.4 (30-Oct-2006) 

[/Trading/SmartQuant/Releases] permanent link

Processing Cisco Syslog Events

First Version: 2006/11/27

Cisco devices generate a number of syslog event types. Common ones include interface up/down events. From a voice perspective, a number of h.323 call logging events can also be generated. By default, with PRI lines, a Cisco Voice Gateway will generate an %ISDN-6-CONNECT event when a call connects and an %ISDN-6-DISCONNECT when the call disconnects. This information can be used to get an idea of call volume and average call duration. These events contain number information for the voip side of the peer only.

To fill a table with call detail records and to generate billing records, which have numbers for both peers, full h.323 connection call accounting needs to be turned on. The records from this process generate a %VOIPAAA-5-VOIP_CALL_HISTORY syslog entry. More details can be found at Cisco's Web Site. In short, add these lines to the gateway to enable detail records:

!
aaa authentication login default local
aaa accounting connection h323 start-stop group radius
aaa session-id common
!
gw-accounting syslog
!
logging x.x.x.x
!

Here is a ciscowatcher.pl Perl script. It provides an idea of how to process the syslog entries in real time through a pipe. This script will form the primary source of updates to the monitoring server status screens for calls, interfaces, wireless, and related information in the monitoring server I'm currently putting together.

The script populates tables in a PostgreSQL database. Here is the ciscowatcher.sql to create the PostgreSQL tables in a database called oneunified.

[/OpenSource/Debian/Monitoring] permanent link

2006 Nov 27 - Mon

Code Colorization

Here are a few sites that will format and colorize code for use on web sites:

• CSharpFriends.com: average formatting quality
• manoli.net: better formatting quality

If you know of Perl Code Colorizer, please let me know.

[/Personal/SoftwareDevelopment/HTML] permanent link

Darvas Trading Module

First Version: 2006/11/26

Nicolas Darvas penned a book called How I Made $2,000,000 in the Stock Market. It is a record of the method he invented to select and trade stocks. The recent reprint includes an Appendix where he has a question and answer session explaining in further detail how his system works. His basic selection criteria is based upon stocks that have hit their 52 week highs. He then has a four day entry criteria backed up with a customized exit criteria. As you are probably aware, any time you always want to know what you risk is going to be. With the built in exit criteria, all the bases are covered. The exit condition is refined as the position changes in price. In the 2005 May issue of Technical Analysis of Stocks & Commodities Magazine, Daryl Guppy wrote an informative article discussing the technical implementation of the process. For the SmartQuant QuantDeveloper environment, I've written a C# class called Darvas that implements the method as described in that magazine article. The code, as supplied in the attached file, as some of the indicator code commented out. You can uncomment if you wish to use it as an indicator. The core of the code accepts OHLC Bars as input, which should be Daily bars from a simulation run, and generates Buy and Exit signals along with a Stop level. The code is straight-forward enough to be ported to other environments as well.

[/Trading/SmartQuant/Articles] permanent link

HTC P3300

First Version: 2006/11/26

I've had an I-Mate PDA2K (codename BlueAngel) for the last year or two. It has been a reliable workhorse. Although, it has been showing it's age recently. Two tiny screws, one up on the left side, and one up on the right side have departed. As a result, I've had to tape the sides together to keep it from falling apart. And as such, have been unable to use the slideout keyboard. Not too much of a deal there. The only real think it lacks is EDGE capabilty. GPS would be nice to have. Users at the XDA Developers Forums have come up with a way to load the unit with Windows Mobile 5. I was thinking of doing that but decided to wait for something new.

My eye was first drawn to the new HP 6915 series Mobile Phones. It has a thumb keypad and GPS. But having a square screen was not something about which I was too keen.

I then heard that HTC, the company that supplies Pocket PC and SmartPhone OEM units to most companies, including I-Mate, was coming out with a PDA form factor unit with GPS, EDGE, and Windows Mobile 5. I'm glad I waited. The one to which I'm referring is the HTC P3300. In the pictures, with nothing to compare it with size-wise, I had the impression it would be about the size of the PDA2K. When I recieved the P3300, I found it was measureably smaller. At first thought, it could be a bad thing. Now that I've used it for a few days, it is a good thing. A female friend indicated that the PDA2K was a bit big. I think she'll appreciate the smaller size of the P3300. I believe the screen has the same resolution, but in a smaller form factor. I think I'll have to form my fingernail a bit better so I can use it rather than the stylus. Windows Mobile 5 has some better features. One that stands out is the Today Screen that accepts plugins. Some of the GPS software vendors have a plugin to allow current coordinates to be displayed. A Pocket PC Music Player puts the play buttons as a plugin. I'm sure the list goes on and on. The unit also has Bluetooth. For some reason, Bluetooth on mine would not turn on. That was annoying. Today, I installed WiFiFo, which required a reboot. Upon reboot, Bluetooth started working. I havn't figured out the magic factor in that one yet. My Bluetooth headset, which I ordered from a different vendor, has yet to arrive, so I can't try them out yet. I did attempt to get a Sonorix OBH-0100 to function, but I think there is a Sonorix hardware incompatibillity. I'll have to give the Motorola HT820 or the Plantronics 590E a try. If you want to use a wired headset other than the one supplied with the unit, you'll need an Audio Jack Convertor. I wish they would have simply included one in the kit. I have a set of Shure E500 In Ear Monitors that I would dearly love to use with the PDA. I previously used the E500's with the PDA2K. They did a great job of cutting out noise and delivering excellent sound quality. They are great when riding airplanes. Anyway, back to the P3300. As it has a built in GPS unit, TomTom is bundled. Upon startup it takes you to a web site to download one free map. Nothing is available for Canada from that link. There are a good number of US maps however. Investing in TomTom's DVD might be a good thing (which isn't available yet, not from Amazon anyway). On the other hand there are a bunch of good stand-alone GPS applications available for the Pocket PC. An interesting add-on is the A2B website for finding georeferenced websites based upon your position. Some specs I found somewhere indicate that one can run the GPS for 4 or 5 hours before recharging is required. I'm interested in finding out how long the battery will last will last when in EDGE mode for a full day of Exchange Pushing. Which is another reason for migrating to Windows Mobile 5. The unit has some 'hesitancy' in responsiveness. It isn't too nerve wracking, it is barely noticeable. We'll see how it does with music and with live map updates with GPS, once I get the headset and maps. I would like it to be faster. But I think that is an engineering trade-off: small form factor, small battery size, smaller battery capacity, and a bunch of memory hungry hardware (GPS, Bluetooth, Wireless, EDGE), one can't have everything. I use two programs from Ilium Software: eWallet (for password management) and DockWare (Clock/Calendar when the unit is docked). I resorted to making a quick and dirty docking station out of cardboard so the unit would stand up-right so it could be seeable from across the room as a clock/calendar unit. The PDA2K had a docking stand come with the unit. It's low built-in memory is a real problem. I havn't loaded to many applications or data into it and it is already complaining about being out of memory. I hope my MicroSDRAM shows up soon. I'm still getting used to the built-in thumb-wheel scroll mechanism. Most of the time I forget it is there and tap entries directly. I think over the long term it could be a handy tool. In summary, the HTC P3300 is a nice little unity with pretty much everthing in it. The lack of a thumb keypad or slideout keyboard will deter some, but I think I can live without them.

[/Personal/Technology] permanent link

2006 Nov 24 - Fri

SSHD Intrusion Prevention

First version: 2006/11/23

There are many 'bots out on the internet that scan for linux hosts and attempt automated sign-ins to machines using common usernames and dictionary passwords. It is tough to lock those bots out but still allow user's to sign into a machine.

To close that loophole in a system's security, there are a couple of things to do. First of all, be sure that telnet access to a machine has been turned off. Telnet is is not a secure remote access technology as all traffic, including passwords, is transmitted in the clear.

The alternate form of remote console access to a machine is through an ssh client. There are a number of primary ssh protocols: ssh1 and ssh2, with the second being more secure than the first. With the ssh daemon running on a machine, in its standard configuration, the bots can still attempt username and password scans on a machine, and possibly through luck of the draw, gain access. Even though passwords and usernames are encrypted, it doesn't prevent the bots from trying them anyway.

In a related article regarding Putty SideKicks, I wrote an article about how to create a public/private key-pairs. This key-pair concept is required for implementing this solution.

Make sure the authorized_keys file in the user's .ssh directory has been updated with their public key. Then, in the sshd_config file, there is an entry called 'PasswordAuthentication'. It is typically set to 'yes'. Set it to no, and restart the sshd daemon.

This will prevent all password based logins to a server. Only users with pre-arranged public/private key-pairs will be allowed access to the server.

This closes down one form of unauthorized access to a server. However, nother remotely accessible applications on a server still need evaluation to determine their risk in permitting server intrusions.

[/OpenSource] permanent link

Tacacs Installation

Updated: 2006/11/23

Here is one of a series of installation procedures for an Open Source monitoring tool.

Tacacs is used for authenticating users in to (mostly) Cisco devices.  The Shrubbery.net's version is used here.

Login into www.shrubbery.net's ftp server and retrieve tac_plus into /usr/src.  Use 'tar -zxvf' to expand out the file and then cd into the newly expanded directory.

You'll need a couple of prerequisites:

apt-get install libwrap0

apt-get install libwrap-devel

You'll need to configure the Makefile:

./configure \

--bindir=/usr/local/bin \

--sbindir=/usr/local/sbin \

--localstatedir=/var/local/tacacs \

--sysconfdir=/etc \

--with-logfile=/var/log/tacacs/tacacs \

--with-pidfile=/var/run/tacacs.pid \

--with-acctfile=/var/log/tacacs/acctfile

Then perform the build and install:

make

make install

mkdir /var/local/tacacs

Update /etc/logrotate.conf:

/var/log/tacacs/acctfile /var/log/tacacs/tacacs {
  rotate 10
  daily
  compress
  }

Here is an example simple configuration file for /etc/tacacs.conf:

key = yourkey

user = outech {

  member = admin

  login = cleartext apassword

  }

user = lastresort {

  member = admin

  login = cleartext apassword

  }

user = webadmin {

  member = level1

  login = cleartext apassword

  }

user = $enab15$ {

  login = cleartext apassword

  }

group = admin {

  default service = permit

  }

group = level1 {

  cmd = show {

    deny run

    permit .*

    }

  }

In the device use a configuration similar to:

conf t

username lastresort secret apassword

ip tacacs source-interface Loopback0

enable secret apassword

aaa new-model

!

tacacs-server host 10.10.10.10 timeout 3

tacacs-server directed-request

tacacs-server key yourkey

aaa session-id common

aaa new-model

aaa authentication login default group tacacs+ local enable

aaa authentication enable default group tacacs+ enable

aaa authorization exec default group tacacs+ if-authenticated

aaa authorization commands 0 default group tacacs+ if-authenticated

aaa authorization commands 1 default group tacacs+ if-authenticated

aaa authorization commands 15 default group tacacs+ if-authenticated

aaa accounting exec default start-stop group tacacs+

aaa accounting commands 0 default start-stop group tacacs+

aaa accounting commands 1 default start-stop group tacacs+

aaa accounting commands 15 default start-stop group tacacs+

aaa accounting network default start-stop group tacacs+

aaa accounting connection default start-stop group tacacs+

aaa accounting system default start-stop group tacacs+

line vty 0 15

  no pass

  login authen default

end

Then start the service with:

tac_plus -C /etc/tacacs.conf

This configuration places a unique 'lastresort' username, secret, and enable into the device. If the tacacs server becomes unavailable, those are the credentials you use for gaining access to the device. When tacacs is available, the username, secret, and enable credentials as found in the tacacs config file are used.

Further Information

A page showing how to automatically assign privilege levels:

http://www.cisco.com/en/US/partner/tech/tk59/technologies_tech_note09186a008009465c.shtml

[/OpenSource/Debian/Monitoring] permanent link

2006 Nov 18 - Sat

SmartQuant QuantDeveloper & DataCenter Release

SmartQuant has released revisions to DataCenter and QuantDeveloper. They are at the following revision levels:

DataCenter
Version 2.1.3 (30-Oct-2006) 

QuantDeveloper Enterprise Edition
Version 2.2.4 (30-Oct-2006) 

QuantDeveloper source code.
Version 2.2.4 (30-Oct-2006) 

[/Trading/SmartQuant/Releases] permanent link

Cricket/Acktomic Installation & Configuration on Debian Etch 2

Introduction

This set of instructions guides you through configuring Cricket for monitoring QoS and SLA statistics on Cisco devices. Modified versions of Acktomic's template file builders are used for accessing QoS and SLA settings. Cisco has changed some of the RTT MIB settings. These settings have been tested on IOS 12.4.

Cricket Installation

Install the Cricket package:

apt-get install cricket

Modify permissions so the config file can be accessed by customized Apache processes:

cd /etc/cricket
chmod 754 config
chown -R cricket.www-data /etc/cricket

In '/etc/cricket/config/Defaults', around line 12, put in the default community string in place of 'public'.

Download and expand the utility archive:

wget http://www.acktomic.com/cricket/genDevConfig_2_0_0beta12d.tar.gz
tar -zxvf genDevConfig_2_0_0beta12d.tar.gz
cd genDevConfig

Remove the CVS directories, and copy the remaining files and directories to assigned locations:

rm -rf plugins/CVS
rm -rf plugins/genConfig/CVS
cp -r plugins /usr/share/cricket/
rm -rf lib/CVS
rm -rf lib/genConfig/CVS
cp -r lib/genConfig /usr/share/cricket/lib
cp lib/monitorConfig /usr/share/cricket/lib
cp util/genDevConfig /usr/share/cricket/util/

Make the main module executable:

chmod 755 /usr/share/cricket/util/genDevConfig

To be compatible with the new Policy-Map configurations, the file '/usr/share/cricket/plugins/genConfig/CiscoIOS.pm' needs to be modified.

Around line 54, replace the following lines:

my (%rttAgentType) = (  '2' => 'saa-rtt',
                        '3' => 'saa-udpecho',
                       '25' => 'saa-http',
                       '27' => 'saa-jitter',
                       '30' => 'saa-ftp'
                     );

With the following lines:

my (%rttAgentType) = (  '1' => 'notApplicable',
                        '2' => 'ipIcmpEcho',
                        '3' => 'ipUdpEchoAppl',
                        '4' => 'snaRUEcho',
                        '5' => 'snaLU0EchoAppl',
                        '6' => 'snaLU2EchoAppl',
                        '7' => 'snaLU62Echo',
                        '8' => 'snaLU62EchoAppl',
                        '9' => 'appleTalkEcho',
                       '10' => 'appleTalkEchoAppl',
                       '11' => 'decNetEcho',
                       '12' => 'decNetEchoAppl',
                       '13' => 'ipxEcho',
                       '14' => 'ipxEchoAppl',
                       '15' => 'isoClnsEcho',
                       '16' => 'isoClnsEchoAppl',
                       '17' => 'vinesEcho',
                       '18' => 'vinesEchoAppl',
                       '19' => 'xnsEcho',
                       '20' => 'xnsEchoAppl',
                       '21' => 'apolloEcho',
                       '22' => 'apolloEchoAppl',
                       '23' => 'netbiosEchoAppl',
                       '24' => 'ipTcpConn',
                       '25' => 'httpAppl',
                       '26' => 'dnsAppl',
                       '27' => 'jitterAppl',
                       '28' => 'dlswAppl',
                       '29' => 'dhcpAppl',
                       '30' => 'ftpAppl',
                       '31' => 'mplsLspPingAppl',
                       '32' => 'voipAppl',
                       '33' => 'rtpAppl',
                       '34' => 'icmpJitterAppl'
                     );

Around line 119, replace the following line:

                  '30'  =>  'ftpAppl');

With the following lines:

                  '30'  =>  'ftpAppl',
                  '31'  =>  'mplsLspPingAppl',
                  '32'  =>  'voipAppl',
                  '33'  =>  'rtpAppl',
                  '34'  =>  'icmpJitterAppl'
                  );

Around line 341, replace the following lines:

    } elsif ($opts->{model} =~ /3600/) {
        $opts->{chassisttype} = 'Cisco-3600-Router';
        $opts->{chassisname} = 'Chassis';
    } elsif ($opts->{model} =~ /2600/) {
       $opts->{chassisttype} = 'Cisco-2600-Router';
       $opts->{chassisname} = 'Chassis';

With the following lines:

    } elsif ($opts->{model} =~ /3600/) {
        $opts->{chassisttype} = 'Cisco-3600-Router';
        $opts->{chassisname} = 'Chassis';
    } elsif ($opts->{model} =~ /2800/) {
        $opts->{chassisttype} = 'Cisco-2800-Router';
        $opts->{chassisname} = 'Chassis';
    } elsif ($opts->{model} eq "C1200") {
        $opts->{chassisttype} = 'Cisco-1200-AP';
        $opts->{chassisname} = 'Chassis';
    } elsif ($opts->{model} =~ /2600/) {
        $opts->{chassisttype} = 'Cisco-2600-Router';
        $opts->{chassisname} = 'Chassis';

Around line 605, replace the following line:

                    $ifdescr = $ifdescr{$ifindex} . "." . $ifindex;

With the following lines:

                    #print "ifindex=$ifindex, policydirection=$policydirection, pol_id_cell=$pol_id_cell\n";
                    $ifdescr = ( 0 != $ifindex ) ? $ifdescr{$ifindex} . "." . $ifindex : "";
#                    $ifdescr = $ifdescr{$ifindex} . "." . $ifindex;
                    #print "ifdesc=$ifdescr\n";

Around line 725, replace the following lines:

  $ldesc = 'SAA(RTR) Performance agent for round-trip time using ' . $protocol .
    ' for destination <B>'. $address . " - " . $rttMonCtrlAdminTag{$key} .
    '</B><BR>Operational values: 1(Ok) 2(Disconnct) 4(Timeout) 5(Busy)" .
    ' 6(NoConnection) 7(LackIntRes) 8(BadSeqID) 9(BadData) 16(Error)' ;

  $sdesc = 'SAA(RTR) Performance agent for round-trip time using ' . $protocol .
    ' for destination ip: ' . $address . ' tag: ' . $rttMonCtrlAdminTag{$key};

With the following lines:

  $ldesc = 'Cisco SLA (RTR) using ' . $protocol . 
    ' for destination <B>'. $address . " - " . $rttMonCtrlAdminTag{$key} . '</B>' ;

  $sdesc = 'Cisco SLA (RTR) using ' . $protocol .
    ' for destination ip: ' . $address . ' tag: ' . $rttMonCtrlAdminTag{$key};

In the file '/usr/share/cricket/lib/genConfig/Utils.pm', near the end of the file, replace the following lines:

sub translateRttTargetAddr {
    my ($type, $value) = @_;
    return ("unknown") if (($type ne "saa-rtt") &&
                           ($type ne "saa-udpecho") &&
                           ($type ne "saa-jitter"));
    $value = inet_ntoa($value);
    Debug("TranslateRttTarget: $value");
    return ( $value );
    }

With the following lines:

sub translateRttTargetAddr {
    my ($type, $value) = @_;
    my $bCheck = 0;
    $bCheck ||= ( $type eq "ipIcmpEcho" );
    $bCheck ||= ( $type eq "ipUdpEchoAppl" );
    $bCheck ||= ( $type eq "jitterAppl" );
    return ("unknown") if ( !$bCheck );
    $value = inet_ntoa($value);
    Debug("TranslateRttTarget: $value");
    return ( $value );
}

In file /usr/share/cricket/util/genDevConfig, after about line 397, add:

'209' => 'Wireless BVI',

Create sub-directories for each device type. Standard directories are usually:

mkdir /etc/cricket/config/routers
mkdir /etc/cricket/config/switches

The file '/etc/cricket/subtree-sets' should therefore have the following configuration:

set normal:
        /routers
        /switches

Copy default configuration files into each of the two sub-directories:

cp /usr/src/genDevConfig/sample-config/genConfig/Defaults /etc/cricket/config/Defaults.genDev
cp /usr/src/genDevConfig/sample-config/genConfig/Defaults.cisco /etc/cricket/config/Defaults.cisco
cp /usr/src/genDevConfig/sample-config/genConfig/Defaults.netsnmp /etc/cricket/config/Defaults.netsnmp

Place the content of Defaults.cisco.oneunified into /etc/cricket/config. If it exists, remove Defaults.cisco. The various graphs and such that I've added to the file could be tuned a bit for color and such. If you have some suggestions, pass them onto me and I'll get them updated.

Here is a sample router configuration:

ip sla monitor responder
ip sla monitor logging traps
ip sla monitor 400101
 type jitter dest-ipaddr 172.20.5.74 dest-port 16390 source-ipaddr 172.20.5.73 source-port 16390 codec g729a
 tos 184
 vrf vrfVoice
 tag jitter tun 400101 nrbmin0401 nrbmac0201
 frequency 150
ip sla monitor 400111
 type jitter dest-ipaddr 172.20.5.90 dest-port 16391 source-ipaddr 172.20.5.89 source-port 16391 codec g729a
 tos 184
 vrf vrfVoice
 tag jitter tun 400111 nrbmin0401 nrcabc0101
 frequency 150
ip sla monitor group schedule 1 400101,400111 schedule-period 150 frequency 150 start-time now life forever

I've found that for Tunnels and such, you may need to reload the router so that the Tunnels have been 'created' from NVRAM rather than the command line. Do a 'sho ip int br' and look at the Method column and compare that with the interfaces and QOS settings that get collected in the next section. As part of your configuration, you may also want to issue the 'snmp-server ifindex persist' command to make sure snmp interface indexes persist across reboots.

To create a sample router template, follow this example:

cd /etc/cricket/config/routers
/usr/share/cricket/util/genDevConfig -c snmpro --rtragents --loglevel debug -2 --vendorint --vlans router01

To configure switches, follow this template:

cd /etc/cricket/config/switches
/usr/share/cricket/util/genDevConfig -c snmpro --loglevel debug -2 --vendorint switch01

to configure voice gateways, follow this template (the -d 22 prevents the serial port sub-interfaces from being listed on the voice interface):

cd /etc/cricket/config/vgw
/usr/share/cricket/util/genDevConfig -2 --vendorint -c snmpro --vendorint -d 22 vgw01

For Cisco Access Points, in the AP Defaults file in the device directory, cisco-interface needs to be changed to cisco-ap-interface.

Once all devices have been configured, run the following to compile the files. The devices will then be automatically scanned once every five minutes.

cricket-compile

To view the graphs:

http://localhost/cgi-bin/cricket/grapher.cgi

Before troubleshooting, you should:

su - cricket

This ensures that .rrd files are created in /var/lib/cricket with the correct permissions.

To run the collector manuall to see what errors there are (logLevel command is optional):

/usr/share/cricket/collector -logLevel debug /routers

Debug logs are found in /var/log/cricket. grapher.cgi errors can be found in /var/log/apache2/error.log. Master debugging flag can be set in /etc/cricket/cricket-conf.pl, with the following statement:

$gLogLevel = "debug";

Based upon the following MIB, CISCO-DOT11-ASSOCIATION-MIB, the following statistics can be collected:

snmpwalk -v2c -c snmpro -m ALL device01 ciscoDot11AssocMIBObjects

Modifications to the following files will be required:

/usr/share/cricket/plugins/genConfig/CiscoIOS.pm
/etc/cricket/config/Defaults.cisco

[/OpenSource/Debian/Monitoring/Cricket] permanent link

2006 Nov 17 - Fri

Putty Sidekicks

As I visit various client sites on a daily basis, I have to log in to various linux boxes and Cisco network devices. Entering usernames and passwords over and over again can be a fact of live that can be automated .... in a safe way.

Many people are aware of using Simon Tatham's Putty as a Telnet/SSH log in tool. There are a couple of add-ons that make life just a bit easier when using this tool on a regular basis.

At many of the sites I visit, there is a Linux server installed for monitoring the network. There is a log in for each consultant who visits.

From the Putty Download page, each consultant downloads Putty, Pageant, and PuttyGen.

• Putty: main tool for telnet/ssh shell logins
• Pageant: a memory resident tool maintaining an active private key
• PuttyGen: a tool for creating a public/private key set for a user

A new user will use PuttyGen to generate a new ssh2 rsa public and private key. Each key is saved to a file. The private key should be saved to a file and locked with a pass-phrase.

On the Linux server, in each user's directory, a directory '.ssh' is created. It needs to be chmod'd with 600. A file in that directory needs to be created with the name 'authorized_keys' and chmod'd with 600. The public key needs to be placed in that file on as one line.

The time saving feature comes with the next steps. When running Windows, put Pageant in the StartUp folder. After logging into Windows, right click on the icon in the tool tray and load the private key from the private key file saved in an ealier step.

Now, when logging into a Linux server from a Windows workstation, Putty will automatically obtain the private key from the running Pageant, pass it to the ssh server and automatically log in when matched against the user's public key from authorized_keys.

One further time saving step is to run QuickPutty. This program can also be auto-started upon auto-login into Windows, and will read Putty's saved entries. QuickPutty can be hidden/shown with -Q. When visible, simply click on an item in QuickPutty's menu. This will start Putty, which will use Pageant for key retrieval, and automatically log in to a Linux Server.

This collection of utilities greatly simplifies the logistics of logging into a Linux server multiple times in order to get multiple sessions opened to network devices with Rancid's 'clogin' command.

On an related note, if you want to copy files to and from ssh compatible hosts, WinSCP is an excellent visual tool for doing that. As an added bonus, it too, will use the Pageant key repository to aid automated logins to a server.

[/OpenSource] permanent link

2006 Nov 16 - Thu

Linux Through Microsoft Proxy

Some organizations use Microsoft Proxy to protect their network edge. To pass through the proxy, Microsoft's NTLM authentication/authorization sequence is typicially required. For regular domain users, this is typically not a problem, as Microsoft's Internet Explorer will automatically supply credentials to the Microsoft ISA Proxy Server.

When one is on a linux box, say a Debian machine, and one wants to obtain 'apt-get' updates, or to obtain Perl updates from CPAN, the Linux application will need to authenticate with and pass through the ISA server. The usual 'http_proxy=...' statement just doesn't work in this context.

When 'http_proxy=...' is used along with another application, it does work. Enter the NTLM Authorization Proxy Server. This is a wonderful little Python script that will act as a proxy to Microsoft's Proxy server.

There is a straightforward configuration file, server.cfg, where you enter the ip address or host name of the proxy server, supply a username and password for authenticating, supply a listening port, and then start with './main.py'. You'll of course need a recent version of Python running for this script to work.

Then from any machine on the network, connect to this proxy. It will authenticate to the Microsoft Proxy server. They note on the web site it will even perform this function for Internet Explorer.

For Linux machine, from the command line, use the two statements:

http_proxy=http://ipaddress:port/
export http_proxy

Commands like wget and apt-get will now function as expected. For CPAN updates, you'll need to use 'o conf ftp_proxy' once you've 'perl -MCPAN -eshell' to update the proxy it uses.

[/OpenSource] permanent link

Sennheiser HD 600

A little while I was involved in a battle of stereos. I moved into a new apartment, one with kinda thin walls. I was without a music center at the time. My neighbor would play his TV/Stereo/Whatever and I'd hear the bass and side affects. I should have done the right thing and talked to him at that moment. But naah. I decided to play along. I picked up a 6 speaker Logitech system. My neighbor is an early to bed, early to rise sort of guy. But he would go to bed with his system on and let it play through the night. Aargh. I started coming home late at night and turning my system up. Half way through the night, I'd turn my off. He must have turned his down in the meantime as well. We kinda reached a happy medium. Then one day he turned his on at 6 in the morning. That got me to the point of getting on speaking terms with the guy.

So we reached a compromise. Naturally. He'd keep his down and I'll keep mine down.

However, keeping my music down just doesn't give me the quality and depth I'd like to see, or rather, hear. So a search for a good set of headphones ensued. I ultimatedly landed on HeadRoom's web site. They do high fidelity headphones. They bring everything together in one place. And talk about their products. No holds barred.

I ended up choosing the Sennheiser HD 600 series. I don't own a $2500 CD player, but I'll play high quality MP3's. From a audioholics perspective, I can't really say by how much they beat the pants off anything else in the audio sphere, but I will attest to a few things. But, yes, they do deliver great sound. One obvious physical characteristic is their open air concept. As such, they aren't good for completely isolating you from someone close to you. But they keep you quiet from someone in the next room. But that same characteristic redeems itself in another manner. They let your ears breath. They also let in some ambient sound just to balance things out.

The things are darn light as well. Couple that fact with the design of their open air concept, I can go for two, four, and sometimes six hour extended listening tours while working through simulations or software development projects.

Just thinking about their sound quality again. I'm wearing them as I write this. It is hard to get the true heart rending bass out of them I can get from my Logitech surrounders, but the headphones are still respectable in that regards. The midrange and highs are indeed superb.

To go along with them, I splurged on the 15' Cardas Replacement Cable. I can walk around my bedroom, sit at my desk, or recline in bed with them watching a movie.

I'll go into more details in another entry, but I also picked up the HeadRoom Total BitHead amplifier to drive the headphones.

All in all, I'm really happy with this setup. The only change would be to try out the HD 650 headphones and see if they are as good as they say they are.

[/Personal/Technology/AudioPhonics] permanent link

2006 Nov 15 - Wed

Backups With Mondo and LVM

Mondo Rescue is an excellent backup for Linux based boxes. With Debian, it is a one line install:

apt-get install mondo

As the existing partitions on the machine I needed to backup did not have enough space for the backup archive, I used LVM to create another partition out of spare drive space, created a jounalled ext3 filesystem, mounted in a directory I created for Mondo backups, and then started the archive program.

mkdir /var/backups/mondo/src
mkdir /var/backups/mondo/dst

lvcreate --size 20G --name lvBackup vg01
mke2fs -v -j /dev/vg01/lvBackup
mount /dev/mapper/vg01-lvBackup /var/backups/mondo/dst
mondoarchive

As part of the backup process, an image of /root/images/mindi/mondorescue.iso should be made to a CD. This will be used in the initial part of the restore process.

One more trick having to do with the LVM (Logical File Manager) has to do with snapshots. Since databases are typically being updated during the backup, the backup process will have obtained files in an inconsistent state. Use LVM to make a snapshot, use Mondo to backup the snapshot, and then use LVM to delete the snapshot.

lvcreate --size 500M --name lvBackupSource --snapshot /dev/vg01/lvVar
mount /dev/mapper/vg01-lvBackupSource /var/backups/mondo/src
mondoarchive
umount /var/backups/mondo/src
lvremove /dev/vg01/lvBackupSource

For the size parameter in the lvcreate command, use a size that will readily accomodate any changes made to the primary partition during the backup phase.

[/OpenSource] permanent link

Kernel Upgrades

I recently upgraded to Debian 2.6.17-2-686. A bunch of packages were held back. A few that subsequently need to be installed manually include:

• apt-get install lvm2
• apt-get install ntp

Without the upgraded userspace lvm2, the system will hang when trying to lvremove a snapshot. The system will need to be restarted to bring things back to life.

For NTP, I see they have changed the configuration file from using multiple instances of pool.ntp.org to assigning specific numbers, such as 0.debian.pool.ntp.org, in order to guarantee unique addresses from dns.

[/OpenSource/Debian] permanent link

Cisco References vol1

Cisco has a number of reference documents that are very useful in day to day network consulting, but can be a real bear to find in a pinch.

The first is Cisco Unified Callmanager 4.1 TCP and UDP Port Usage. It goes through and identifies all the TCP and UDP ports in use by the various Callmanager services. Ports are grouped into the following categories:

• Intracluster Ports Between CallManagers
• Windows and Common Ports
• Between CallManager and LDAP Directory
• Web Requests from CCMAdmin or CCMUser to CallManager
• Signalling, Media and Other Communications Between Phones and Callmanager
• PC Behind the Phone to the Phone
• Signalling, Media and Other Communications Between Gateways and Callmanager
• Communications Between Applications and CallManager

The end of the document contains a number of links regarding PIX and IOS FW Inspection and Context Based Access Control.

When trying to select a Cisco device 'based on the numbers', you'll want to take a look at Cisco's Portable Product Sheets. The key sheets have to do with performance of the various switch models and router models. In addition, there are sheets relating to Port Adaptors, NM/WIC/VWIC compatibility, VPN performance, wireless comparisons, some info on GBICS, as some stuff on phones and voice density.

Cisco's TAC Tools page has such things as a DSP Calculator, IP Subnet Calculator, and a Voice Codec Bandwidth Calculator.

[/Cisco] permanent link

2006 Nov 14 - Tue

Netdisco Installation and Configuration

Netdisco maintains interface status for Cisco based devices. Netdisco is a Sourceforge hosted project with a main project page at http://www.netdisco.org/. The download link is somewhat out of date. The instructions contained herein pertain to obtaining the most recent version via version control.

Download the software and prepare the directories:

cd /usr/src
cvs -d:pserver:anonymous@netdisco.cvs.sourceforge.net:/cvsroot/netdisco login
cvs -z3 -d:pserver:anonymous@netdisco.cvs.sourceforge.net:/cvsroot/netdisco co -P netdisco
cvs -z3 -d:pserver:anonymous@netdisco.cvs.sourceforge.net:/cvsroot/netdisco co -P mibs
mkdir /usr/local/netdisco
mv mibs /usr/local/netdisco
mv netdisco/* /usr/local/netdisco
useradd -d /usr/local/netdisco netdisco
chown -R netdisco.netdisco /usr/local/netdisco

Make changes to config file by 'nano /usr/local/netdisco/netdisco.conf':

domain        = .example.com
db_Pg_pw      = netdisco
port_info     = true
community     = public
bulkwalk_off  = true
graph_x       = 40
graph_y       = 30
node_fontsize = 8.0

In the file, /usr/local/netdisco/html/login.html, change the line 'my $userip = $r->connection->remote_ip;' to 'my $userip = $r->user();'. Do something similar for:

line 103 login.html
line 96 autohandler
line 24 admin_user.html
line 22 portcontrol.html

In /etc/apache2/sites-enabled/000-default, insert the two lines:

Include /usr/local/netdisco/netdisco_apache.conf
Include /usr/local/netdisco/netdisco_apache_dir.conf

Fix the mason cache directory:

mkdir /usr/local/netdisco/mason
chown -R netdisco.www-data /usr/local/netdisco/mason
chmod -R 775 /usr/local/netdisco/mason

Install through perl:

perl -MCPAN -eshell
install Text::Reform
install IO::Tee
install Bundle::DBI
install Apache::DBI
install Heap
install Graph
install Compress::Zlib
install Net::NBName

Prepare PostgreSQL (should have already been installed with the base OS), create the database, and create the tables:

cd /etc/postgresql/7.4/main
nano pg_hba.conf
  host netdisco netdisco 127.0.0.1 255.255.255.255 trust
  local netdisco netdisco trust

/etc/init.d/postgresql-7.4 restart
cd /usr/local/netdisco/sql/
./pg --init
# follow prompts
./pg 
  # \q to exit

Prepare SNMP:

#apt-get install libnet-snmp-perl
apt-get install libsnmp-base
apt-get install libsnmp-perl
perl -MCPAN -eshell
install SNMP::Info

Install GraphViz:

apt-get install graphviz
apt-get install libgraphviz-perl

Ensure the Apache2 trimmings are installed:

apt-get install libhtml-mason-perl
apt-get install libdbi-perl
apt-get install libdbd-pg-perl
#apt-get install apache2-dev
apt-get install apache2-threaded-dev
apt-get install libapache2-mod-apreq2
apt-get install libapache-dbi-perl
apt-get install libmasonx-request-withapachesession-perl
apt-get install libapache2-request-perl
ln -s /etc/apache2/mods-available/apreq.load /etc/apache2/mods-enabled/apreq.load

Use WinSCP to copy c:\windows\fonts\arial.ttf and c:\windows\fonts\lucon.ttf to /usr/local/netdisco.

Update some permissions:

#chgrp netdisco /usr/local/netdisco/*.conf
chown -R netdisco.www-data /usr/local/netdisco
chmod 660 /usr/local/netdisco/*.conf

Import OUI database (get latest from web if you want):

cd /usr/local/netdisco
wget http://standards.ieee.org/regauth/oui/oui.txt
./netdisco -O

Test the configuration by performing some preliminary scanning:

cd /usr/local/netdisco
./netdisco -r center_network_device
./netdisco -m
./netdisco -a
./netdisco -w
./netdisco -g

Add a user in Netdisco (and provide it with port control and admin rights):

/usr/local/netdisco/netdisco -u admin

Restart Apache:

/etc/init.d/apache2 restart

Startup netdisco by browsing to:

http://localhost/netdisco

Make changes to /usr/local/netdisco/netdisco.crontab. If nothing else, at least change center_network_device to something. Then start cron job:

crontab -u netdisco /usr/local/netdisco/netdisco.crontab
/usr/local/netdisco/bin/netdisco_daemon start
ln -s /usr/local/netdisco/bin/netdisco_daemon /etc/init.d
update-rc.d netdisco_daemon defaults 25

You may need to up date netdisco-topology.txt

Links

A link to a scipt for installing an earlier version, with mrtg insertions.

  http://www.isc.cnrs.fr/informatique/public_notice/netdisco-install-english
  

[/OpenSource/Debian/Monitoring] permanent link

Check SMTP (email server)

Sometimes you want to check to see if you have access to an email server directly. A check like this may be necessary on some DSL networks where the ISP will block port 25, which is the standard port that email servers listen in on, due to issues with SPAM and rogue servers.

You can do the check from the command line with a program called telnet:

telnet mail.example.com 25

For mail servers with a Barracuda mail server, you may get a response like:

220 mail.example.com ESMTP (19a38e746d4fc812318d47ee6fa159ea)

Here is a sample session:

mail from:sender@example.com
250 2.1.0 sender@example.com... Sender ok
rcpt to:recipient@example.com
250 2.1.5 recipient@example.com... Recipient ok
data
354 Enter mail, end with "." on a line by itself
subject:  Test Message
to: recipient@example.com
This is the body of the message.
.
250 2.0.0 kAEK9vHC005225 Message accepted for delivery
quit
221 2.0.0 mail.example.com closing connection

[/Personal/Technology] permanent link

2006 Nov 13 - Mon

Archiving with Tar

To tar and compress a directory:

  tar czf archive.tar.gz archive
  

To uncompress and expand the file:

  tar zxvf archive.tar.gz
  

[/OpenSource] permanent link

Basics for Creating a Cisco Network Monitoring Server

This monitoring server is based upon a basic Debian 2.6 Etch 2 installation. Use the Debian Base Build document to create the base server.

Once the base build is complete, a number of different applications can be individually installed. As each application is individually documented, they can be mixed and matched as appropriate.

Configure the server as an NTP server:

apt-get install ntp
apt-get install ntp-server

Configure the /etc/ntp.conf file with one or more specifc servers if you need more than just pool.ntp.org, which may offer up the same server more than once (depending upon how your DNS server caches entries). To restart the service:

/etc/init.d/ntp-server restart

Useful debugging tools include ntpq, ntpdc, and tcpdump. Be patient as NTP requests cycle on a 64 second basis. The NTP port of interest is UDP port 123.

Configure Syslog by editing /etc/default/syslogd and fix the configuration to show:

SYSLOGD="-r"

Cisco devices typically use facility 7 for their syslog entries. Therefore direct these log entries to a specific file by adding the following line in the server's /etc/syslog.conf:

local7.*    /var/log/cisco.log

Then restart the service:

/etc/init.d/sysklogd restart

These folders should be archived and CVS'd for each transfer:

/usr/share/snmp/mibs
/var/www

[/OpenSource/Debian/Monitoring] permanent link

Debian Usage Notes

These are various common commands I've pulled together as useful in day to day Debian server management.

To mount a CD:

mount -t iso9660 dev/hdc /cdrom

For simple package refresh, use the following two commands:

  apt-get update
  apt-get upgrade
  

The 'apt-file' command allows you to look for packages to install. You'll need to do an 'apt-file update' first in order to obtain the package listings.

When performing a major upgrade, use:

  apt-get dist-upgrade
  

If the process generates an error and aborts, the following command may get things going again:

  apt-get -f install
  

Other useful commands:

  apt-get clean - clears cache of downloaded packages
  dpkg --purge
  apt-get check

At some point during future upgrades, you may encounter an error like:

  GPG error: ftp://ftp.us.debian.org/ testing Release: The following signatures couldn't
  be verified because the public key is not available: NO_PUBKEY ......
  

In this case, running the following may solve the problem:

  apt-get install debian-archive-keyring
  

• Debian Distribution
• Debian Administration

On some Debian installations, the GUI may be installed. On the one I encountered when I performed an update, I have to perform the following steps to get the GUI running after a reboot:

• log in to the machine via ssh
• run 'vncserver' and identify the display number it shows
• on your local machine, run the vnc viewer and connect to the servername:1 (1 is typically used)
• upon connection, you may have a graphical window with terminal mode... type the command 'startkde &' to get the full GUI functional
• once the GUI is started, VMWare Workstation can be started, and any necessary sessions can be started after

Commands to show logical volume system:

lvm
  lvdisplay
  vgdisplay
  pvdisplay
  

Command to create volume:

lvcreate --size 16m --snapshot --name  snapBase /dev/vg01/lvBase
mkdir /mnt/snap
mount /dev/vg01/lvBase /mnt/snap
  

Commands to remove volume:

umount /dev/vg01/lvBase
lvremove /dev/vg01/snapBase

[/OpenSource/Debian] permanent link

Debian Etch 2 Base Build

This build process creates a basic VMWare session with Debian Etch 2 Network Boot CD. Familiarity with VMWare is assumed. A similar configuration can be used when installing on physical hardware.

I'm posting this build process as it forms the basis for a number of other configurations that have already been posted, or will be posting in the near future. This configuration forms the basis of a network monitoring server that can be used to support a predominately Cisco based infrastructure.

To create a VM, start the VMWare wizard with File -> New -> Virtual Machine. Use a 'Custom' Virtual Machine Configuration.

Select Next and then for an 'Appropriate Configuration', choose custom, then click 'Next'.

For a 'Virtual Machine Format', use 'New - Workstation 5'.

When you need to select a Guest Operating System, there is no entry for Debian. Therefore, in the radio button list, select 'Linux'. Then in the drop down box, select 'Other Linux 2.6.x kernel'.

In the next wizard window, provide a Virtual Machine name such as 'debbase'. Select an appropriate directory.

For 'Processor Configuration', select 'One' processor.

Default memory usage of 256 MB should be fine.

For the 'Network Type', typically you'd use 'Bridged Networking'. During operating system installation and configuration, you can then assign a dedicated IP address or allow the network DHCP to assign it an IP address.

On the 'Specify Disk Capacity' wizard window, the 8GB default disk size should be fine. Leave 'Allocate all disk space now' as blank. Optionally, you can select the 'Split disk into 2 GB files'.

When you need to specify 'Disk File', give it a name such as hda which is similar to the Linux physical drive vernacular.

Click 'Finish' to finish the VM session creation.

Now that the VM session has been created, the operating system can now be installed.

For the VM properties, you can point the virtual CD-ROM to a physical drive or to an .iso file on your harddrive. In either case, make the Debian Etch 2 NetInstall available and start the VM session. It should boot from you CD or .iso file.

By default, the installation process will install Debian with Kernel 2.6, so just hit the ENTER key to start.

On the Language screen, hit enter to accept English.

On the country screen, choose Other, then select Bermuda (choose your own country here). This selection also sets the timezone, so be sure to choose appropriately, and hit ENTER.

For the Keyboard Layout, use American English and hit ENTER.

If you have a DHCP server, the installation process will automatically configure an IP address for your VM. If you had wanted to use a static IP address, either disable DHCP visability to the VM, or restart the installation and set the DHCP parameter to none (select F1 to find the appropriate screen with the parameter at the first installation screen.

For the hostname, use a name appropriate for the machine you'll be building. On the following screen, provide a domain name. For example: oneunified.net

You can then select a Debian Mirror from which to download the remaining packages for the installation. I normally use ftp.us.debian.org under the United States listed mirrors. You can skip or you can configure the http proxy information, as appropriate for your network access to the internet.

Installation will do a hardware scan and then start into drive partitioning.

After choosing the default, you'll be presented with three choices for partitioning.

I don't use any of the defaults, so choose <Go Back> to get the main partiioning screen.

To partition the drive, use the following steps:

• On a system with dual drives using software raid:
• create a 200MB partition on each drive for the /boot partition
• set the boot flag on each partition
• the two partitions can then be joined in the raid manager as /dev/md0
• create a 500mb or 1gb partiion on each drive for the swap (normally I allow the swap partition to be managed by LVM, but as LVM will be residing on the raid'd partition, swap is maintained on the non-raid location to keep it fast)
• allocate the remaining space on each drive into a final partition which will be used by LVM
• join these two partitions together in the raid manager to become /dev/md1
• once created and the operating system is installed, 'mdadm --detail /dev/md0' can be used to view synchronization status
• once the raid partitions are in place, proceed with file system and LVM allocation (some of the following steps are redundant or are in need of adjustment [I'll have to reorganize this a little later] )
• select the drive and create new empty partition table (if you are not configuring with raid)
• select the free space and create a new partition of size 200MB, make it primary at the beginning, and the mount point should be /boot and set the 'Bootable Flag' to on.
• select the remaining free space, create a new partition, use the maximum space available, make it primary, use as 'physical volume for LVM', and don't make it bootable
• now start the 'Configure the Logical Volume Manager' selection, and select yes when asked to commit the current changes
• use the 'Modify volume groups' menu
• Create Volume Group
• use the space bar to select the shown volume group device
• call it 'vg01'
• use the 'Modify Logical Volumes' menu to create logical volumes
• Swap: lvSwap in vg01 of 500MB
• Base: lvBase in vg01 of 2GB
• Var: lvVar in vg01 of 1GB
• Netflow: lvNetflow in vg01 of 500MB
• leave logical volume creation and leave logical volume manager menu
• you'll see a listing with logical volumes and physical partitions, each needs to be assigned a mount point with the following steps... you'll select #1 under each indicated logical volume
• lvBase use as ext3 with mount point '/'
• lvNetflow use as ext3 with mount point of '/var/local/netflow'
• lvSwap use as swap area
• lvVar use as ext3 with mount point '/var'
• you can now Finish Partitioning and write changes

Additional Installation Steps:

• During the network install, a network card may not get recognized. If this is the case, try 'install noapic' on install boot. This worked on a IBM e300 server with an e100 network card.
• Set a password for the root account
• For the screen asking for a regular user account, create an account called 'admin' and provide it with a password
• For software selection screen, uncheck everything, then check:
• Web Server
• File Server
• SQL database
• Standard systems
• For the Samba Server configuration, put in your network domain name
• Choose Yes to install the GRUB boot loader to the master boot record.
• Allow the system to reboot

Log back in after reboot and install a few more modules:

• apt-get update
• apt-get upgrade
• apt-get install apt-file
• apt-get install ssh
• apt-get install ncftp
• apt-get install ntp
• apt-get install curl
• apt-get install lynx
• apt-get install expect
• apt-get install cvs
• apt-get install ntpdate
• apt-get install tcpdump
• apt-get install iperf
• apt-get install perl-doc

To synchronize the server time with NTP:

ntpdate servername|ipaddress 

For some basic service management:

apt-get install sysv-rc-conf
sysv-rc-conf

Remove the AppleTalk protocol:

update-rc.d -f netatalk remove
apt-get remove netatalk

To search for packages:

apt-get install apt-file
apt-file update
apt-file -l list image

To install the default Perl CPAN bundle for the first time, accept all defaults.  If your proxy or firewall does not accept ftp commands, modify the command around line 58 in '/usr/share/perl/5.8.8/CPAN.pm' so it uses an http site like:

$CPAN::Defaultsite ||= "http://cpan.calvin.edu/pub/CPAN"

That one will get the regular list of repositories downloaded. You can then choose one or more to your liking. http://cpan.belfry.net/ seems to be a good one.

perl -MCPAN -eshell
install PAR::Dist
install Compress::Zlib
install IO::Zlib
install Test::More
install Spiffy
install Test::Base
install Digest::SHA::PurePerl
install YAML
install Bundle::CPAN
reload cpan
install Error
install Digest::SHA1
install Digest::MD5
install DBI

If you choose a download site that doesn't work very well, use the following command and restart the perl configuration.

rm -rf /usr/lib/perl5/5.8.6/CPAN/Config.pm

Perform general configuration install generic utilities.

Uncomment certain commands in .bashrc or adding lines like:

#for regular proxying:
#http_proxy=http://username:password@proxy:8080/
#for ntlm proxying with APS098:
http_proxy=http://proxy:5865/
export http_proxy

Then reload the .bashrc file with:

#this command:
source .bashrc
#or this command
.bashrc

Install your .ssh key:

scp -r .ssh root@yourmachine:/root

Finally, if you are using VMWare, make a snapshot of the base build so it can be used as a basis for other servers, and can be rolled back when necessary.

From the command line, you can shutdown the server with:

shutdown -h now

After making a snapshop of the server, it can be renamed to the target. To do so, you'll need to change existing name in the following files to reflect the new name:

• /etc/hostname
• /etc/hosts

Make sure you have all the latest packages installed.

Make sure you've got the latest and most appropriate kernel for your machine. You can use:

apt-file -l search linux

to determine what the current image is. Install it with:

apt-get install linux-image-2.6.15-1-686

Mastershaper will require the source:

apt-get install linux-source-2.6.15

[/OpenSource/Debian] permanent link

2006 Nov 11 - Sat

CSS Resources
Here are some resources for using CSS in web sites. The big thing is that one should be able eliminate the need for <table> tags. I have more research and tinkering to do before I can convert this site over to a predominately <div> flavour.

• The Noodle Incident: First definitive tutorial on boxes.
• A List Apart: contains a number of good getting started style articles on CSS
• Intro to CSS

[/Personal/SoftwareDevelopment/HTML/css] permanent link

Free Development Tools

thefreecountry.com has a bunch of free development tools. It is quite extensive, covering programming, webmaster, and security resources.

[/Personal/Technology] permanent link

favicon.ico

thesitewizard.com has a description of what to do with the favicon.ico and where to put it. It seems there is some sort of link problem with the link ref as it is not showing in my browser. More research is needed.

Here is a good Pixel Editor.

[/Personal/SoftwareDevelopment/HTML] permanent link

Apache Rewrite

After taking a look at Google's Webmaster Tools over a few days, I have come to the conclusion that Google doesn't crawl cgi-bin type things. So David Wheeler's rewrite page that I had come across a while back finally makes sense. The page url's need to be modifed to turn things into a seemingly standard directory structure.

Here is my version of a section of the Apache configuration file to handle the rewrites found anywhere within my /blog subdirectory:

  RewriteLogLevel 0
  RewriteLog /var/log/httpd/rewrite.log

<Directory "/var/www/html/blog">
  AddHandler cgi-script .cgi
  Options +ExecCGI
  RewriteEngine On
  RewriteCond %{REQUEST_URI} !-f
  RewriteCond %{REQUEST_URI} !-d
  RewriteRule ^(.*)$ /cgi-bin/blosxom.cgi/$1 [L,QSA]
</Directory>

The first two lines helped figure out why the rewrites weren't working. By setting RewritelogLevel to a non-zero value, of which I used a value of 4, log statements are written to the directory indicated by RewriteLog. It turned out that my rewrite rule becomes '^(.*)$' rather than Wheeler's '^/(.*)$'.

This only required one configuration change in the blosxom.cgi file:

$url = "http://www.oneunified.net/blog/";

[/OpenSource] permanent link

Turing Test for Comment Submissions

People have designed various ways to prevent bots from successfully submitting web forms. Perhaps the most common method is by using CAPTCHAs: common ones being those funny images with characters hidden in a disconcerting background. CAPTCHA is an acronym for 'Completely Automated Public Turing test to tell Computers and Humans Apart'.

There appear to be a number of ways to defeat CAPTCHAs. But why bother implementing such a scheme?

For the casual bot scanning the web, they may be tuned for bypassing CAPTCHAs. But what if a site was to do something completely different? For example, David Wheeler's Comments & TrackBacks uses a simple arithmetic expression to break an auto submission bot. So if every site out there did a variation on the theme, auto-submissions could be prevented. On the other hand, if everyone did a little arithmetic turing test, then we've come full circle with the bot being tuned to look for such a simple test.

One side comment, Wheeler made his site, through a simple javascript, some how better viewable with Firefox but less so with IE.

[/OpenSource/blosxom] permanent link

Career Planning

In talking to a friend today, they were considering a career change. I remember way back when, I was doing career changing, ie, finding myself. I think there were two books that helped me the most. Both are by Richard Nelson Bolles: 'What Color is Your Parachute' and 'Three Boxes of Life'. The first one is re-written yearly. The second is a bit harder to come by. Both helped me figure out where I was, where I wanted to go, and helped plan how to get there. I recommend them both.

[/Personal] permanent link

2006 Nov 09 - Thu

Fast Trading Simulation Engine

Are you running complicated trading scenarios incorporating equity and option mixtures from a quote/depth data stream? Are you using Genetic Programming tuned Fuzzy Logic algorithms? Are your sims taking a while? I think you may get a boost soon, if not real soon.

Intel is about to release their new quad processor, known as the QX6700. It is a dual die Conroe Dual Core CPU.

From a supporting cast perspective, nVidia has released their 680i chipset, and has chosen eVGA as a reference platform for the motherboard. eVGA doesn't appear to have any BIOS downloads on their site yet. I don't know if that means the motherboard is stable or not. Reviewers;, such as [H] Enthusiast and VR-Zone seem to think so. The motherboard has dual PCI-16x video slots, so it is dead easy to feed four large format LCD monitors simultaneously. And with all that horsepower available, playing high Frame Rate video games while waiting for a simulation run to complete should be child's play. I used to be an ASUS fan. I don't see anything from them, and I hear that it may be another month or two before they bring something to market. eVGA, I think you have a convert.

Newegg has a the eVGA motherboard in stock along with a memory combo. The combo includes Corsair TWIN2X2048-6400C4D chips. However, [H] Enthusiast used a pair of Corsair TWIN2X2048-8888C4DF Dominators. I think I'll give the Dominators a try instead of what comes standard in the combo.

Wikipedia indicates a November 2006 release date with a part number of BX80562QX6700. Newegg doesn't appear to have it yet. cnet reviews indicates a release date of the processor of Nov 14. Just under a week away. I think I can wait that long before I put in my order for motherboard/memory/cpu. Not too patiently though!

I don't do much gaming so jumping to the nVidia 8800 series Graphics cards doesn't seem to justified. I think I'll stick with a pair of well balanced and well reviewed eVGA Geforce 7600GT video cards driving quad VideoSonic VP201b 20" LCD monitors.

SmartQuant QuantDeveloper, during simulations, is single threaded. However, by using Altiris Software Virtualization Solution, it is possible to run multiple copies of QD simultaneously. I'll discuss the simulation and optimization set up for that in another entry.

Well, in the meantime, I have to wait another week for Intel's CPU to be released, and then another week for the stuff to arrive. Once it arrives, I'll be all set to give the config a try and see if I can take my current 23 second single run simulation down a notch.

[/Personal/Technology] permanent link

Traders Expo 2006

I had made other travel plans before realizing that Traders Expo is on this month at the Mandalay Bay in Las Vegas from November 16 to 19, 2006. It looks like it might be a worthwhile event as recognizable names like Options Trader Larry McMillan and Tom Sosnoff of ThinkorSwim are going to be there.

Any one heading over to that event?

[/Trading] permanent link

2006 Nov 08 - Wed

Over Abundance of Vehicles in Bermuda

According to Bermuda Online, Bermuda has almost 21 square miles of land-mass and as of 2005, 3301 permanent residents per square mile. And that figure, by it's very definition, excludes Expats living on the island. I think another 10% or 20% could be added on as a rough figure.

With more people there are more cars. Many in Bermuda believe that the number of cars on the island is getting out of control. Hamilton, Bermuda does indeed have it's rush hours, one in the morning, and one in the afternoon. And due to it's unique geography, commuters into Hamilton from the West end get to suffer the bottleneck of one road into Hamilton. There are suggestions that Expats be denied the privilege of owning/driving a four wheeled vehicle.

There was another article quite recently regarding the interesting statistic that the Warrants Backlog Nears 10,000 and they indicated that many of those are automobile related.

So... when you put those two facts side by side, I think it is possible to come up with some easy to digest, even-handed solutions. For instance, if the outstanding warrants could be tracked, and the guilty parties assigned heavy penalties, such as, say, revocation of driver's license or vehicle license, might that not help alleviate some of the problems of vehicular congestion?

[/Personal/Bermuda] permanent link

Celeros XT816

I have a customer who is using the Celeros XT816 3U ExtremeSAN iSCSI based IP SAN appliance. They purchased it because it had an exceedingly good price/storage price point. They had original plans to use it as a primary storage unit for use with VMWare ESX server. As it turns out, after much testing, and a few chats with tech-support, it won't work in such an arrangement with the customer's version of software. They are in the process of gaining access to Celeros' Knowledge Portal to obtain some updates.

I was tasked to integrate the unit into the network. The unit has a management port with a default IP address of 192.168.1.1. Due to the regular boots and resets we've had to perform with the unit, we decided to just stay with that address, put the management port on its own VLAN, and add the subnet into the routing table.

The unit has 8 Gigabit ports for data transfer. They can be used individually with LUN's or the ports can be mixed and matched for singles and bundles. In bundle mode, they supposedly use LACP as bundling protocol. Technical support indicates that they do that with Dell PowerConnect 3424/5212/5425, Fujitsu XG800's, Raptor, D-Link (several models), Force10, HP, Brocade, 3Com, & Extreme Networks. You'll notice that Cisco is absent from that list. Once we can gain access to the knowledge portal, supposedly there is a work around for Cisco switches available. We are currently running in single port mode to get at the data.

One more caveat with the unit is in the user-interface. To keep one's settings synchronized with what shows on the screen, the application, a Java based application, needs to be closed out and restarted. Hopefully this is cleared up in a new release of the software.

We also noticed, that with certain operations in the GUI regarding networking or LUN configuration, the unit will reset. So be sure you have quiesced your data before changing configurations.

In the end, the customer has decided to keep the unit and use if for secondary, non-critical storage. Perhaps better use can obtained with the latest software releases.

The customer is probably going to go with EqualLogic as they are certified for use with VMWare ESX.

[/Personal/Technology] permanent link

OpenMoKo: Open Source Cellphone

As of this writing, you can't google for it, but the Inquirer has an article on a Open Source Linux based Cellphone with built-in GPS. There isn't much information there, but then I remembered that the key site for Linux based devices would be from the people at LinuxDevices.com who have an article with some real details in it regarding the OpenMoKo. They talk about the development platform being in pre-release, I'm wondering when hardware becomes available. After a bit more reading, I see that the development platform Funambol Mobile Open Source is more of a synchronization platform. I'll have to come back to that site and see how well they can connect up with Microsoft's Exchange server. Linux Journal gave the software two thumbs up in their editor's choice awards for 2006. Dig the chic chain loop so you can't loose the darn thing.

Linux Devices has a Second Article regarding the phone. The phone is making it's debut tomorrow/today in Amsterdam. They say the first run is due in December with general availability in January at a price of around $350 directly from FIC.

In the meantime, since my existing Imate PDA2K is falling apart from way too much usage, I'll go and pick up HTC's P3300 with Microsoft Windows Mobile 5 and Built-in GPS. In the US, I've heard that the unit is available from Smart Mobile Gadgets as well as Phone Source USA. I just wish they had more accessories. I guess one will have to head over to Mobile Planet for the add-on bits.

[/Personal/Technology] permanent link

2006 Nov 06 - Mon

Definitive Dictionary on HTML and CSS coding:

Index DOT Html

[/Personal/SoftwareDevelopment/HTML] permanent link

What Is it Like

Well, Bermuda, that is. But I should take one step back yet. I had to go back to my paperwork to see just how long I've been here in Bermuda. Ah, yes. My rent contract came up for renewal back this June. So ... a few more calculations, and it turns out I was here off and on from the beginning of 2004, and settled in full time during June 2004. Almost three years. Three very fast years.

I must say a bit too much work. Although I have tried to fit in some International One Design (IOD) sailing in while I can. My up-coming new years resolution will be to last a full season without getting socked in with work.

I lived in St. Thomas, USVI for a year and half before arriving here. Now if I could bring their winter time weather to Bermuda, I think I would be in Nirvana.

In Bermuada, I'm living on North Shore road. It provides a great view of the ocean on every ride into work. When ever I happen to leave the island and come back, I take the taxi ride back from the airport. I have to pinch myself every time. There is nothing like coming back home to paradise. No four line highways here. Not a one.

And by living on North Shore, I have two or three routes home, and none of them have traffic congestion. Well, yes, I did do that on purpose. Fortunately, I was here for a while to get an idea of where good places were, and what to look out for when renting a place.

Anyway, enough for now. More later.

[/Personal/Bermuda/Personal] permanent link

Master Shaper Installation Introduction

Mastershaper is a composite tool designed to filter and control ip traffic of all types. It is composed of five primary tools: a specially compiled 2.6 kernel, l7-filter, iptables, ipp2p, and mastershaper. This document walks through the integration and configuration of each of these tools.

The installation is based upon the Debian Etch 2 installation with Apache 2.

Make sure you've got the latest and most appropriate kernel for your machine, by using the appropriate base build documentation.

Install tools:

apt-get install yaird
apt-get install kernel-package libncurses5-dev fakeroot wget bzip2
cd /usr/src
tar -xjf linux-source-2.6.15.tar.bz2
cd linux-source-2.6.15
make menuconfig
  general: append version info 

In the menu, load the alternate configuration file from /boot/config-2.6.15-1-686, or which ever is appropriate for the kernel you have loaded. Make any appropriate adjustments to the configuration. Exit the menu. By leaving all defaults as they were, you can rebuld the kernel in its default configuration. We'll then make further modifications.

make-kpkg clean fakeroot make-kpkg --initrd --revision=mastershaper.1.0 kernel_image
cd ..
dpkg -i linux-image-2.6.15_mastershaper.1.0_i386.deb
reboot

When the image comes back up (you may need to manully select the new image, as well as update /boot/grup.menu.lst), run 'uname -a' to check the build date to confirm it is your new basic rebuild.

Obtain and install the l7-filter and related patches:

cd /usr/src
wget http://internap.dl.sourceforge.net/sourceforge/l7-filter/netfilter-layer7-v2.2.tar.gz
tar -zxvf netfilter-layer7-v2.2.tar.gz
cd linux-source-2.6.15
patch -p1 < ../netfilter-layer7-v2.2/kernel-2.6.13-2.6.16-layer7-2.2.patch

After patching the kernel, install the patterns:

cd /usr/src 
wget http://internap.dl.sourceforge.net/sourceforge/l7-filter/l7-protocols-2006-06-03.tar.gz
tar -zxvf l7-protocols-2006-06-03.tar.gz
cd l7-protocols-2006-06-03
make install

Some changes are required to iptables before compiling the kernel.

apt-get remove iptables
cd /usr/src
ncftp ftp://ftp.netfilter.org/
  cd pub/iptables
  bin
  get iptables-1.3.5.tar.bz2
  exit
bzip2 -d iptables-1.3.5.tar.bz2
tar -xvf iptables-1.3.5.tar
cd iptables-1.3.5
patch -p1 < ../netfilter-layer7-v2.2/iptables-layer7-2.2.patch
chmod +x extensions/.layer7-test
cd /usr/src
ncftp ftp://ftp.netfilter.org/
  cd pub/patch-o-matic-ng/snapshot
  bin
  get patch-o-matic-ng-20060626.tar.bz2
  exit
tar -xjvf patch-o-matic-ng-20040621.tar.bz2
cd patch-o-matic-ng-20060626
export KERNEL_DIR=/usr/src/linux-source-2.6.15
export IPTABLES_DIR=/usr/src/iptables-1.3.5
./runme extra
  ** add the 'time', 'ipp2p', 'route' modules
cd ../iptables-1.3.5
make KERNEL_DIR=/usr/src/linux-source-2.6.15
make install KERNEL_DIR=/usr/src/linux-source-2.6.15

Build the kernel again:

fakeroot make-kpkg --initrd --revision=mastershaper.1.1 kernel_image

MasterShaper is the web site front end for controlling and monitoring the kernel tools just installed.

apt-get install mysql-server

Create the database and assign privileges:

mysql
  create database shaper;
  grant all privileges on shaper.* to 'shaper'@'localhost' identified by 'shaper' with grant option;
  exit

Download and install MasterShaper:

cd /usr/src
wget http://www.mastershaper.org/mastershaper_0.44.tar.bz2
tar -xjvf mastershaper_0.44.tar.bz2
cd MasterShaper-0.44
mkdir /var/www/shaper
cp -R htdocs/* /var/www/shaper/
chown -R www-data.www-data /var/www/shaper
cd /usr/src

Install some libraries:

apt-get install iproute
apt-get install libphp-jpgraph
apt-get install php-pear
apt-get install sudo
apt-get install php-db
apt-get install php4-mysql
apt-get install libphp-phplayersmenu
pear install DB Net_IPv4
cd /var/www/shaper
ln -s /usr/share/php/libphp-phplayersmenu phplayersmenu
ln -s /usr/share/jpgraph jpgraph

To get help on the IPP2P IPTables plugin:

iptables -m ipp2p --help

To run the GUI:

http://localhost/shaper

On the first configuration screen, iptables should be set to '/usr/local/sbin/iptables'.

MasterShaper documentation can be found at:

http://www.mastershaper.org/index.php/Main_Page

[/OpenSource/Debian/MasterShaper] permanent link

Configuring Wireless on Cisco 871W with SDM

This was not a pleasant experience. No wonder I like CLI over GUI interfaces. With a command line, I get feedback as to what I'm doing wrong. With Cisco's SDM, it has some rudimentary user-interface checks before performing a post. On the other end, if you haven't put something into the UI correctly, at least for the wireless config pages, you don't even get a 'hey dummy' message, it just plain ignores you. No feedback, no hints, no nothing.

I simply wanted to get my wireless bit bridged to the vlan bit. The first step is to select the setting in SDM GUI to bridge the wireless with the wired. This will create the normal BVI interface. No problem there. After that, you go into the wireless configuration web pages. I simply wanted to get a WPA-PSK (Pre-Shared Key) into the unit. There is no obvious way to accomplish that task.

I ended up taking a configuration from a 1230 wireless access point, massaged it a bit, and dropped it into the 871. Which yields another beef: there are variations in how wireless is configured in a 1230 vs the 871. In the 871, the ssid and authentication stuff is in the 'interface Dot11Radio0' section. In the 1230, it is outside. Cutting and pasting therefore is not quite so simple.

So, after hours of diddling with the GUI, I did the following in a few minutes with the CLI:

interface Dot11Radio0
 no ip address
 !
 encryption vlan 1 mode ciphers tkip
 !
 ssid my ssid
    vlan 1
    authentication open
    authentication key-management wpa
    guest-mode
    wpa-psk ascii mywpapsk
 !
 speed basic-1.0 basic-2.0 basic-5.5 basic-6.0 basic-9.0 basic-11.0 basic-12.0 basic-18.0 basic-24.0 basic-36.0 basic-48.0 basic-54.0
 station-role root
 l2-filter bridge-group-acl
 no cdp enable
!
interface Dot11Radio0.1
 encapsulation dot1Q 1 native
 no cdp enable
 bridge-group 1
 bridge-group 1 subscriber-loop-control
 bridge-group 1 spanning-disabled
 bridge-group 1 port-protected
 bridge-group 1 block-unknown-source
 no bridge-group 1 source-learning
 no bridge-group 1 unicast-flooding
!
interface Vlan1
 description Wired Network
 no ip address
 ip virtual-reassembly
 ip route-cache flow
 load-interval 30
 fair-queue
 bridge-group 1
!
interface BVI1
 description $ES_LAN$
 ip address 10.10.10.254 255.255.255.0
 ip nat inside
 ip virtual-reassembly
 ip tcp adjust-mss 1452

[/Cisco] permanent link

2006 Nov 05 - Sun

Phone Serial Numbers

Cisco doesn't store their phone serial numbers anywhere. So, for companies wishing to obtain SmartNet for their installed base of telephones, there aren't too many ready ways of obtaining those serial numbers, from a first time perspective.

To help one customer out of a bind, I did a couple of very quick and dirty scripts to scan the network for phone devices. I know it works for 7912's, 7940's, 7960's, 7970's and ATA adaptors.

I didn't realize until later that instead of scanning the human readable pages, I could have scanned the xml pages for the information. Oh, well. That will be for the next version.

To operate, you'll need a Perl interpreter and a couple of libraries off CPAN. Then edit findphones.pl and supply the ip address ranges you'd like to scan. Run the script and send STDOUT to a file.

Then run the file through filterphone.pl to get a model number and serial number list.

Cisco's CP-7935 and CP-7936 conference phone serial numbers are simply their mac addresses, which is easy to pull out of Callmanager.

[/Cisco] permanent link

Flavour Bugs

Well they aren't flavour bugs per-se. They are more like misunderstandings. Well, not that either. The word will come to me. My issue is that I have the two flavours: .blog and .article. Each with their own foot and head pieces. Now I have to figure out how to meld the two into one so that I don't have to update code in two different places when ever I make adjustments to the web site.

Looks like 'ln -s foot.blog foot.article' fixes that little conundrum.

[/OpenSource/blosxom] permanent link

Blosxom WriteBacks

So far, Blosxom has behaved quite nicely. It is amazing how such a compact application can effectively do so much. One addition I've wanted to add in order to make this site a two way street is WriteBacks. There were a few items in the Blosxom Plugin Registry, but have been a bit uncertain as to the reliability of the code, as it really hasn't been updated in quite some time. I finally did come across a link to Kevin Scaldeferri's Blog, from the Blosxom User Group Blog. He has a plugin, with recent updates which provides WriteGack capability.

During the installation process, there are a number of things needing doing. One is that you need the flavours from Rael's Original WriteBack Plugin. Don't install the plugin, just the flavour files. Then install Kevin's plugin. There are some configurations in the file you'll need to perform.

The trick with this is that you don't want all the comment submissions to happen on your main blog page, which will happen if you put the various bits in your default story.flavour. Instead, create two flavours, I call the default flavour 'blog', and the secondary flavour 'article'. In 'story.blog', along with the standard permanent link, you place the writeback count.

In story.article, you put the form to be posted. This draws the submission form only when viewing a single article. Use the supplied foot.writeback for inspiration. Also, in story.article, you place the code to view the writebacks. Use story.writeback for inspiration.

Be sure to make the various variable updates in the writeback plugin, and you should be good to go, but for some formatting and alignment issues you may want to tune.

[/OpenSource/blosxom] permanent link

Blosxom Categories

I installed Todd Larason's Categories Plugin, and I'd say it is another very easy winner. I used the 'breadcrumbs' version rather than the heavily indented and space consuming 'categories' version.

With this, I've reached my goal of a functioning Blog with navigation, advertising, and writebacks. For the targetback thing I havn't quite figured out how it works or what it does. Perhaps some one could 'writeback' and let me know how it works, and what I should do with it.

[/OpenSource/blosxom] permanent link

Blosxom Calendar

I installed Todd Larason's Calendar, changed a value in the config file to turn off caching, put two lines of code in my story.flavour file, and presto, calendars. I wish all software were this easy.

I see he is using MovableType now. I've flirted with trying that a couple of times. I haven't quite had to go quite that far yet. We'll see what happens with my next project: showing a list of categories. If I can get that going, then I think I've covered most of the basic features of a Blog site, and will want for little else. Famous last words.

[/OpenSource/blosxom] permanent link

HTML Escape Codes

• For the < type &lt;
• For the > type &gt;
• For the © type &copy;
• For the & type &amp;
• For the " type &quot;

[/Personal/SoftwareDevelopment/HTML] permanent link

2006 Nov 04 - Sat

Bollinger on Bollinger Bands

From a technical analysis perspective, I think the best book I've ever purchased is Bollinger on Bollinger Bands by John Bollinger. It's 228 pages covers a number of interesting concepts. It does indeed cover the concept for which Bollinger is famous: the volatility indicating Bollinger Bands. Since signals typically require corroborating evidence, he makes use of Arthur A. Merrill's Five Point Patterns as well as a number of different volume indicators.
Bollinger Bands can be used in Contrarian Trading as well as in Trading with the Trends. The hard part of found is figuring out when to transition from one to the other. Contrarian Trading means taking an opposing position when one of the band limits has been reached. It is at this critical decision point when you have to decide to keep the position and see if the trade is going to 'walk the band' (Trade the Trend), or if indeed, it will reverse direction. This is where various other indicators such as MACD, Candles, and Volume can help trip the appropriate trigger. Having introduced his various indicators, Bollinger then proceeds to describe some trading strategies such as The Squeeze, Trend Following, and Reversals.
I've found that Bollinger bands help delineate any type price data, whether it be daily bars, 1 minute bars, trades, or even quotes. I've used quite a number of different indicators, but the ones that frequent my charts the most are Bollinger Bands.

[/Trading/TechnicalAnalysis] permanent link

IQFeed Provider for SmartQuant

When I first started looking into developing an Automated Trading Strategy, I started by building some historical data acquisition routines in Perl. The routines were designed to communicate with DTN/IQFeed's servers. I then started gaining access to their real time data. At that point, I started to realize how daunting was the project I started on. More realizations were yet to come.

In the mean time, some mild clarification for those referring to the two sites just mentioned. IQFeed does 500 symbols, with opportunity for more in 500 chunks. DTN starts off with 1300 symbols, with room for more. Both are effectively the same company, so besides symbol counts, there really isn't all that much difference.

Anyway, as I thought may way through how I was going to store data, play it back, graph it, and analyse it, I was realizing that there was much to do. Being a software developer, I wanted something with a decent API, a lot of flexibility, and a lot of functionality. I figured there wasn't enough time in the world to do it mysefl. I looked at some of the Perl libraries, but they weren't quite 'there'. I looked at the mainstream trading platforms, but they relied on limited and proprietary languages. Then, by stumbling through a series of links relating Quant and Libraries, I ended up at SmartQuant. Their QuantDeveloper product fits the bill exactly. It has a straight forward user interfaces for manipulating and charting symbols. It has an analysis and simulation engine built around components. The components are developed using native C# code, and are supported by an array of extensive Quant/Trader/Data libraries. I have barely scratched the surface of utilizing the functionality.

More on this in a later entry.

I took my old perl code, rewrote it in C# and made it conform to the IProvider interfaces as supplied in the API. With another rewrite a month ago, it has progressed to something reasonably reliable.

If you are using SmartQuant's QuantDeveloper, and have a subscription to IQFeed, give the library a try, and let me know about any issues. You'll need the latest IQFeed Files as well as the C# Library. The library provides realtime access to IQFeed. I havn't implemented the IHistory interface yet.

[/Trading/SmartQuant/Articles] permanent link

2006 Nov 02 - Thu

Fuzzy Logic

For an Automated Trading system I've been developing, I've come across the fact that Fuzzy Logic may assist in making decisions on how to trade at particular times of the day depending upon what conditions are predominant. Amazon has a bunch of theoretical books, but hardly any at all for the practical practitioner. I did purchase The Fuzzy Systems Handbook, 2nd Ed by Earl Cox. I'm about half way through it now. I've got through all the bits that make up the basic fuzzy sets. The sections are liberally sprinkled with C++ code. I'm not sure how much of it will compile in today's tools. The book was written back in the age of Windows 98. On the other hand, the code snippets are readable for one needs to understand what is happening in the commentary.

I had approached the subject from a different perspective though. I started by searching for code libraries. I came across FLUtE: Fuzzy Logic Ultimate Engine. The fellow has written a code library in C#. The code does compile in Microsoft Visual Studio with the v2.0 run time libraries. Coding new stuff in it may be somewhat of a challenge as the documentation is quite sparse. But then again, that is par for the course.

After taking a look at the modules, I came across something called 'hedging'. At the time, I didn't know what it was all about. And that prompted me to look for some good practioner's books. Hence the book I referenced above. Hedging, is obvious once you think about it. It is adding fuzziness to an existing fuzzy rule. The concept is well described in the book. The book doesn't exactly flow from front to back. For instance, during the beginning of the book, the author introduces a concept called alpha-cuts, and incorporates its use in to the development and discussion of fuzzy rules.. I can see what they do, but where and how they are applied, I'm still not exactly certain. And I'm up to page 344 now. There have been some hints, but no concrete usage criteria. I'm sure it will become clear as I move along in the book.

It was good that I did some prior reading, otherwise I think I would have been lost with the onslaught of information. I recall one of the first things I read was the document regarding the Mathworks Fuzzy Logic Toolbox. You can review the document in html or as a complete pdf document. In the pdf version, on page 56 (2-26), they have an excellent drawing summarizing how everything fits together.

Another book that helped fill in the gaps is An Introduction to Fuzzy Logic For Practical Applications by Kuzuo Tanaka. It is quite expensive for its 138 pages, but does have some useful background info. In the end, it is a good pocket guide for starters. It did leave me thinking that there were holes in the information presented. Earl Cox's book has filled in some of those gaps.

Having said all that, I'm just beginning to build the environment for using fuzzy logic in my trading solution. Although the FLUtE code looks like it could work, I'm going to try my hand at some basic C# code for fuzzy rules and rulesets to get a feel for what is needed. Once that is in place, I think I can then use a Genetic Programming engine I wrote to see if I can optimize some rule selections.

[/Trading/AutomatedTrading] permanent link

Technorati

I wanted to see who linked to David Wheeler's site (the Perl guy who has single handedly contributed so many good things to the Perl community). His site linked to Technorati to show who linked to his site. So I thought I'd do the same. Here's my Technorati Profile.

[/Personal] permanent link

2006 Oct 27 - Fri

Opening Entry

I've been debating on whether to start blogging with one Blog or with three: one for centered around my professional work of Cisco networks, one about open source tools I've encountered, and one for detailing the trials and tribulations of developing an automated trading solution. Well, it is one for now.

My first entry has to do with Blosxom. I've been a perl programmer for a while and wanted to stay with Perl. Blosxom, at first blush seems to fit the bill. Now let's see if I can integrate it in with Mason, upon which my web site is based. I've just been told by the Blosxom installation instructions I should write something. So here goes. Let's see if I can get it broadcast to the world!

.... One second while I fire up my web browser to Ray's of Sunshine.

Wow, as advertised, 15 minutes or less. Success. And that was with reading the instructions!

Well, I can see the blog entry, but integrated with Mason and the rest of my web site, it isn't. I'll skip that for the time being. Having read further into the instructions, it looks like I won't need to create a different site for each train of thought. All I need to do is create some subdirectories. Blosxom then mixes and matches, automagically.

The Blosxom/Mason connection continued to pry at my skull. ... Some googling later, or is that some Googling later... so I landed on Life of a Sysadmin where the fellow talks about support for Blosxom. That provided a good introduction where I can find support. Also, during that same search, I encountered a Blosxom blog by the Perl Master himself, David Wheeler at Just a Theory where he discusses mod_rewrite rules to incorporate Blosxom into Mason. Now that I have the links, I can come back at my liesure and do some hacking/integrating.

For my further reference, here is The Unofficial Blosxom User Group.

[/OpenSource/blosxom] permanent link