2009 Aug 31 - Mon

Massaging CommunigatePro MIB For Cricket

CommuniGate Pro's web interface has a page which shows SNMP originated statistics. On that same page, there is link for downloading the MIB file which defines the values shown on that page.

Rather than going through all 100 or so MIB entries, I wrote an AWK script to process the CommuniGate Pro MIB file into a Defaults file which can read by Cricket, the SNMP collector/grapher.

After running the Defaults file with a CommuniGate Pro server for a while, I found that some of the groupings didn't work very well by default. Several values are serveral orders of magnitude different from other values in the same group. I did some manually editing to get values of like magnitude into their own groups. Here is the resulting Defaults.communigate file. I've left colouring to the Cricket defaults, but at least it gets the values into my monitoring solution.

[/OpenSource/Tools] permanent link

2009 Aug 29 - Sat

Tools for Testing Your Internet Connection

Measurement Lab has a series of tools for Testing Your Internet Connection:

• Network Diagnostic Tool: Test your connection speed and receive sophisticated diagnosis of problems limiting speed
• Glasnost: Test whether BitTorrent is being blocked or throttled
• Network Path and Application Diagnosis: Diagnose common problems that impact last-mile broadband networks
• Pathload2: Test your available bandwidth
• Diffprobe: Determine whether an ISP is giving some traffic a lower priority than other traffic
• NANO: Determine whether an ISP is degrading the performance of a certain subset of users, applications, or destinations

[/Networks] permanent link

Virtually Wondering the Earth

I saw a video once of how collections of pictures can be data mined to produce composite interactions which are more than the sum of the parts. For example, it is said that there are more than 80,000 images of the Notre Dame Cathedral in the Flickr database. By using Scene Reconstruction and Visualization from Community Photo Collections, one can see more detail than any one of the photographers who took pictures when actually being on site.

That research is only a minor portion of what can be found at Microsoft Research.

[/Personal/Technology] permanent link

VOIP Security Tools

The Voice over IP Security Alliance has an interesting collection of VoIP Security Tool List which includes things like:

• VoIP Sniffing Tools
• VoIP Scanning and Enumeration Tools
• VoIP Packet Creation and Flooding Tools
• VoIP Fuzzing Tools
• VoIP Signaling Manipulation Tools
• VoIP Media Manipulation Tools
• Miscellaneous Tools
• Tool Tutorials and Presentations

Use at your own risk.

[/Networks] permanent link

Options as Indicators

Optionetics has an interesting article called Using Options to Predict Stock Prices. The author, John Jeffery, writes that, in addition to the usual fundamental analysis and technical analysis methods of 'stock direction prediction', options can help indicate trade direction. Three useful indications include:

• Put/Call Ratio
• Implied Volatility
• Option Volumes

Put/Call Ratios: The most popular Put/Call Ratio is the one used for monitoring the sum total of option trades at the CBOE (Chicago Board of Trade). The same technique can be used for individual stocks as well. It is basically the ratio of the number of open call positions relative to the number of open put positions on a given stock at a given expiry. With experience, it can be used as a bullish/bearish indicator of the underlying stock. Street Authority has some further information on the Put/Call Ratio. Schaeffer's Investment Research indicates the general market is strongly bullish. They have a series of stock screeners.

Bullish Stock Screeners
• Stocks with a high put/call ratio
• Stocks with high short interest

Bearish Stock Screeners
• Stocks with a low put/call ratio
• Stocks with low short interest

A reference is made to an article by Pan and Poteshman called The Information of Option Volume for Future Stock Prices where they say that they "performed daily cross sectional analysis on 10 years of CBOE data to reveal that doing nothing more than buying stocks with low put/call ratios and selling stocks with high put/call ratios generated a return of 1% per week." You have to read the full abstract for some caveates though:

We present strong evidence that option trading volume contains information about future stock price movements. Taking advantage of a unique dataset from the Chicago Board Options Exchange, we construct put-call ratios from option volume initiated by buyers to open new positions. On a risk-adjusted basis, stocks with low put-call ratios outperform stocks with high put-call ratios by more than 40 basis points on the next day and more than 1% over the next week. Partitioning our option signals into components that are publicly and non-publicly observable, we find that the economic source of this predictability is non-public information possessed by option traders rather than market inefficiency. We also find greater predictability from option signals for stocks with higher concentrations of informed traders and from option contracts with greater leverage.

One of the authors has another paper entitled Investor Behavior in the Option Market. One of the interesting points from the abstract is the remark "none of the investor groups significantly increased their purchases of puts during the bubble period in order to overcome short sales constraints in the stock market." Taken the other way around, puts are an easy method of getting around short selling restrictions on equities.

In visiting a related author, there is a recent paper called Dynamic Trading with Predictable Returns and Transaction Costs which discusses some portfolio optimization with a mixture of short, medium, and long term mean reversion based trades. This has nothing to do with options, but is an interesting article in itself which I wanted to keep.

Implied Volatility: Implied Volatility is the expected volatility of an option's underlying asset up to the option expiry. This tool can be used for inter-day and intra-day trading calculations. In the article's example, where the underlying is moving sideways, an increasing Implied Volatility could indicate some major move in the underlying. Various other relationships can be established as well.

Option Volumes: When looking at option volumes across the whole series of an underlying's strike prices and expires, look for unusual activity in volume. This means comparing current traded volume with average daily traded volume. It may be possible to see where traders are seeing resistance or support levels, with the interpretation being whether the volume is in puts or calls. If there are no related news, earnings events, or government announcements, then someone may know something.

[/Trading/Options] permanent link

2009 Aug 19 - Wed

Debian Dpkg Install

From the Debian Security Announce List, a little short-cut for installing .deb packages:

wget url
        will fetch the file for you
dpkg -i file.deb
        will install the referenced file.

[/OpenSource/Debian] permanent link

IPTables Mangle DSCP

From the Nanog mailing list, a way to force QOS packet marking on outbound packets might look like:

# iptables -t mangle -I OUTPUT -p tcp --sport 80  -j DSCP --set-dscp 0x1a

[/OpenSource/Linux] permanent link

Boost Bind/Lambda replaced by Boost Spirit/Phoenix

Regular users of the C++ library known as Boost will already know about functors, lambda functions, and the like. These abilities mostly originate in the Boost.Bind and Boost.Lambda libraries.

As I'll soon be using the functor capability within my C++ programs, I wanted to make a 'note-to-self' regarding the fact that Boost.Bind and Boost.Lambda have basically been superceded by Boost.Spirit.Phoenix.

The Phoenix library has been accepted into Boost based upon Hartmut's summary of the Phoenix review.

The current incarnation of the Boost libraries is 1.39. Here is the Spirit User's Guide which includes a link to the Phoenix Documentation and a link to the Phoenix Users Guide.

It is noted that FC++ influenced Phoenix, and when looking at the FC++ web site, there is a reference to LC++, which is a Logic Programming language built atop of FC++. I wonder if something similar has been done atop of Phoenix.

An example from the Boost Mailing List of using Phoenix:

#include <vector>
#include <algorithm>
#include <boost/shared_ptr.hpp>
#include <boost/spirit/home/phoenix/core.hpp>

#include <boost/spirit/home/phoenix/operator.hpp>
#include <boost/spirit/home/phoenix/bind/bind_function.hpp>

struct A {};

void foo( const A& ) {}

int main()
{
   using namespace boost::phoenix;
   using namespace boost::phoenix::arg_names;

   std::vector< boost::shared_ptr< A > > vec;

   vec.push_back( boost::shared_ptr< A >( new A ) );

   std::for_each( vec.begin(), vec.end(),

                  bind( &foo, *arg1 ) );
   return 0;

}

Colorized with CodeColorizer.

[/Personal/SoftwareDevelopment/CPP] permanent link

Simple Desktop Network Monitoring Tools

Here are a few simple tools useful on a Windows' desktop for monitoring basic network stuff:

• TCPTraceRoute: mBy sending out TCP SYN packets instead of UDP or ICMP ECHO packets, tcptraceroute is able to bypass the most common firewall filters.
• LFT: Layer Four Traceroute which is mostly non Windows based tool, but partially works in Cygwin.
• Ping Plotter: helps you pinpoint where the problems are in an intuitive graphical way, and to continue monitoring your connection long-term to further identify issues.
  
• SNMP Traffic Grapher: monitor a couple of SNMP values in near real time.
  
• WinMTR: WinMTR is a windows clone of popular Matt's traceroute/ping program called MTR.
  

[/Networks] permanent link

2009 Aug 18 - Tue

Model View Controller

In the world of software development, programmers will make use of 'patterns' as a form of programming template.

By defining and segregating certain forms of functionality, code reuse and modularity can be enhanced. One regularily recurring form of functionality involves some form of data collection (a model), some form of view of the data (a view), and some form of interaction with the view and data (a controller). This collection has been defined as a the Model-View-Controller (MVC) pattern.

The best diagram I've found which depicts the relationships between these three sets of functions can be found at Designing Enterprise Applicationswith the J2EETM Platform, Second Edition. I've included the diagram here:

[/Personal/SoftwareDevelopment] permanent link

Reality Sets In

The blog Bits or Pieces? has an interesting reference to the three stages of expertise:

It starts with someone making the realization that they want to learn about something, and don't know much about it. They may turn into a wanna-be know-it-all, where they think they know-it-all. But for the trully intelligent, reality sets in, and the realization is made that one knows a lot about the subject matter, but there is so much to learn. For me, the hard part is that the more I know, the more I know I don't know. The curve seems to indicate that it only gets worse, not better.

[/Personal] permanent link

2009 Aug 02 - Sun

Building HDF5 in Microsoft Visual Studio 2008

A while ago, I wrote an article about HDF Group's Hierarchical Data Format (HDF5) Library . In the article, there were some brief installation instructions. This article adds some refinements to the installation instructions.

You need to start by downloading the compression libraries:

• SZip 2.1 for VS2005
• ZLib 1.2 for VS2005

Create a sub-directory called 'compress' somewhere. In that sub-directory, create two additional sub-directories: 'include' and 'lib'.

Unzip the two downloads. From each of the two uncompressed libraries, put all the .lib and .dll files into the sub-directory .\compress\lib, and put all the .h files into the .\compress\include sub-directory.

In Windows, create two environment variables by going: Start->ControlPanel->System->Advanced->EnvironmentVariables, and then create two new user variables:

• HDF5_EXT_SZIP = szlibdll.lib
• HDF5_EXT_ZLIB = zlib1.lib

The remaining build instructions focus on building the useful HDF5 C++ libraries for HDF5 v1.9, with v1.9.43 being the latest as of this writing. Download /hdf5-1.9.43.tar.gz and expand it with 7-Zip into a working sub-directory called hdf5-1.9.43. Run .\hdf5-1.9.43\windows\copy_hdf.bat. Double click on .\hdf5-1.9.43\windows\proj\all\all.sln to open the Visual Studio Solution. The file is in version 2005. VS 2008 will ask to convert it for you. You'll need to do so.

After the conversion, go into Tools->Options->ProjectsAndSolutions->VC++Directories. Set 'Include Files' to the full path of your .\compress\include sub-directory, and set 'Library Files' to the full path of your .\compress\lib sub-directory.

For the project properties, choose whether you are doing a debug build or a release build. Do the build.

For v1.9.43, I found that there was one debug executable that wouldn't build, which, since I'm only interested in some key libraries, has no effect on my required outcome.

After the build process is complete, open a command prompt in .\hdf5-1.9.43, and run 'installhdf5lib.bat'. The various .dll, .lib, and .h files will be in \dll, \lib, and \include off of .\hdf5-1.9.43\hdf5lib.

From Sysinternals, download junction.exe. This allows you to create symbolic links between directories. Put the program somewhere in your path. Then use it to create a symbolic link from your existing project to the hdf5lib directory. This will allow you to change library versions with a simple symbolic link change. For example, something like the following will set a link to the include files where ever you installed and built the hdf5 libraries.

• junction hdf5 .\hdf5-1.9.43\hdf5lib

[/Personal/SoftwareDevelopment/CPP] permanent link