2007 Dec 30 - Sun

SmartQuant QuantDeveloper & DataCenter Release

SmartQuant has released a revision to DataCenter and QuantDeveloper. DataCenter and QuantDeveloper are at the following revision levels:

DataCenter
Version 2.3.5 (29-Dec-2007)  

QuantDeveloper Enterprise Edition
Version 2.7.3 (29-Dec-2007) 

QuantDeveloper Source Code
Version 2.7.1 (20-Sep-2007) 
* Recent Versions available through 
  version control 

[/Trading/SmartQuant/Releases] permanent link

2007 Dec 16 - Sun

SmartQuant QuantDeveloper Code Excerpts

Here are some QuantDeveloper code snippets:

• Sliding Window Statistics
• Darvas Selection
• IQFeed Provider & Examples
• Merrill Patterns
• Matching Peaks in Real Time
• Tracking High volume
• CandleSticks

[/Trading/SmartQuant] permanent link

2007 Dec 15 - Sat

Candle Analysis for SmartQuantLibraries

A couple of posts at the SmartQuant forums have requested information on Candle definitions and analysis. Here are two C# modules which can used as a basis for your own analysis and signal generation:

• Candle.cs
• CandleList.cs

There are various knobs you can make use, with 'bald' (which controls when a hair is detected) being the main one. There are some percentages which I should have set as constants scattered through the code (0.40 being one of them), which set whether something is relatively long bodied or short bodied.

If you have changes, let me know, I'll add them and repost.

The source of inspiration was Bigelow's Profitable Candlestick Trading.

[/Trading/SmartQuant/Articles] permanent link

2007 Nov 23 - Fri

SmartQuant QuantDeveloper & DataCenter Release

SmartQuant has released a revision to DataCenter and QuantDeveloper. DataCenter and QuantDeveloper are at the following revision levels:

DataCenter
Version 2.3.4 (22-Nov-2007)  

QuantDeveloper Enterprise Edition
Version 2.7.2 (22-Nov-2007) 

QuantDeveloper Source Code
Version 2.7.1 (20-Sep-2007) 
* Recent Versions available through 
  version control 

[/Trading/SmartQuant/Releases] permanent link

2007 Oct 31 - Wed

Make It Seem To Work

"Sometimes it's much cheaper and easier to make people think that something works rather than actually make it work. After all, the result is, in all important aspects, the same." -- Douglas Adams

This reminds me of a building engineer, who, when complained to by staff that they were cold and wanted their own thermostat, put a thermostat on the wall. No one had asked that it be connected to anything, and no one complained after that.

[/Personal/TagLines] permanent link

2007 Oct 26 - Fri

Bliss

What is better than eternal bliss? Nothing. But a slice of bread is better than nothing. So a slice of bread is better than eternal bliss.

[/Personal/TagLines] permanent link

2007 Oct 23 - Tue

QuantDeveloper Source

A number of people took the time to email me their thoughts regarding my rant regarding source code for QuantDeveloper. Anton from SmartQuant responded, but unsurprisingly, I havn't heard anything from QuantHouse.

It seems that QuantHouse has purchased various rights and such to QuantDeveloper, and have renamed it to suit their suite. SmartQuant has also retained certain rights and such to QuantDeveloper, and have maintained the name. SmartQuant has indicated they are continuing development of QuantDeveloper and has hinted that some quite advanced features are in the pipeline. He also confirmed that, for existing developers who have subscribed to source code, source code is available for new releases.

[/Trading/SmartQuant/Articles] permanent link

2007 Oct 21 - Sun

The Cluetrain Manifesto

Today I was introduced to The Cluetrain Manifesto: The End of Business as Usual through a NANOG posting in response to someone's quote:

I think you greatly underestimate how customers react to the truth.

A few easily reachable quotes from the full online text:

The connectedness of the Web is transforming what's inside and outside your business -- your market and your employees. 

We are not seats or eyeballs or end users or consumers.  We are human beings  -- and our reach exceeds your grasp.  Deal 
with it

Facebook is probably one of many excellent examples of how humans interact, and companies are having a hard time trying to make those sites work for themselves. hmph. Perhaps when companies learn that they are made up of humans who share and interact, maybe good things can happen.

I'm thinking that many a great open source company has learned to share what could be classified as the family jewels, and thus have profited immensely from embracing the way humans naturaly interact.

[/Personal/Business] permanent link

2007 Oct 20 - Sat

Quality of Service Options on GRE Tunnel Interfaces

Note to self, according to Cisco's document 10106, Quality of Service Options on GRE Tunnel Interfaces: when applying queuing to a tunnel interface, shaping must also be applied. The top level policy of a recommended hierarchical policy should be a shaping command, while the lower-level policies configure the queueing mechanisms.

[/Cisco] permanent link

2007 Oct 19 - Fri

C++ Libraries from Scaling Web

Scaling Web has a number of open source libraries that may be of use:

• Template based B+ Tree: Implemented in C++, B+ Tree is template based, so it can be used with any types of data. Memory based B+ Tree is much faster than STL containers like map, hash_map.
• Embedded File System: Embedded File System (EFS) is a library that allows you to create virtual file system in a single file.
• Mork Parser in C++: MorkParser is an efficient C++ implementation of Mork database format reader. The Mork format is used in most Mozilla-based projects, including the Mozilla browser suite, SeaMonkey, Firefox and Thunderbird.
• Mime Email Parser in C++: C++ implementation of RFC 2045 - Multipurpose Internet Mail Extensions (MIME) standard.
• C++ IO Streams Library: IO Streams library provides different Java-like interface streams in C++.

[/Personal/SoftwareDevelopment/CPP] permanent link

Trading Site of the Day -- StreamBase: Event, Databases, and (unrelated) Charting

StreamBase has a developers kit for using their Event Stream Processor and database. They have an article on Using the StreamBase opentick Adapter to Build Market Data Applications. The database itself could be a useful alternative to SmartQuant's historical data tables, but the downside is that compression may not be available. I've only skimmed the site. I'm hoping the code behind the referenced article shows enough that I could re-code it for an IQFeed or Genesis data stream.

While on the topic of coding, some one pointed out ZedGraph as a possible alternative for showing charts. It is open source.

On the other hand, I've been leaning towards ChartDirector for C++ as it, as it says, has an API for C++, as well as a host of other languages.

[/Trading/SiteOfTheDay/D200710] permanent link

Trade Optimization by Genetic Programming, Example for QuantDeveloper

I had recieved a private message by someone requesting some background on what I had done for my Genetic Programming trials with QuantDeveloper.

I am releasing the example source code into the public domain. If you make changes and would like to see the changes recorded here, please let me know, and I'll update the files.

The subdirectory OneUnified.GeneticProgramming is the GP library. I used Koza (1992) as well as a few other related references as a guide for various constants and such. You'll need some of my other libraries (as listed elswhere on this site) in order to get the thing to compile.

The library provides population management, survival of the fittest, and various breeding cross-over strategies. Adding and removing operators should be relatively easy. I have a couple of base classes from which most are inherited: boolean and double. Note that, unlike C++ where one can use static methods to initialize some global class variables, C# insists on initializing static class variables upon first invocation, which is why you'll see ugly, redundant code for getting this done for the operator setups. Maybe there is a better way.

The subdirectory UI holds the setup code to run a simulation. When you look at the code, you'll see that there is no real easy way to embed an optimization module in QD, so I took source samples and patched together a self running optimizing application.

You'll be glad to note that doing something like this will be much more difficult, if not impossible in the future. QuantHouse, in their infinite wisdom, has decided that we add-on developers, who don't spend thousands of dollars each month on sub-millisecond data-feeds are a pain in the back side, and so will not be providing source code in the future. Or so they told me. I come from an open source community, and this closed source madness of QuantHouse has turned me off of QD. I've gone back to using C++ routines to take care of my trading chores.

While on the subject of C++, er, C#, I went back to C++ for several other reasons. There is a rich and varied source of high quality open source libraries for C++: Boost, QuantLib just to name two. C++ is faster. Also, I feel that C# is meant for lazy programmers who don't know how to clean up after themselves. There are C++ garbage collection libraries out there if I feel the need to use them. And did you know that Microsoft couldn't even bring themselves to use CLS in most of the their Windows development (most of the core CLS was removed from key kernel applications).

Anyway, here is my Trade Optimization by Genetic Programming, Example for QuantDeveloper.

[/Trading/SmartQuant/Articles] permanent link

2007 Oct 07 - Sun

IQFeed Source Code Release

I havn't had much time recently to keep my IQFeed libraries (for QuantHouse/SmartQuant's QuantDeveloper) updated, and I see there are a few users with some annoyances with the libraries. I've decided to empower them to solve the issues themselves.

I've released the code under GPL3 license terms. For those who wish to to use it commercially, they can contact me directly.

I'd been holding off releasing the code, hoping I could clean things up a bit more, but I better bite the bullet and just get the code out there. Perhaps Anton can create a sub-forum for supporting the library.

There are two libraries:

• OneUnified.QD.Libraries.20071007.zip: libraries for use in various applications
• OneUnified.QD.Apps.20071007.zip: some sample applications using the libraries

The projects are in Visual Studio 2005 SP1 format.

If you have corrections to the libraries, please email the changes to me, I'll update the code, and release an updated version.

All the code compiles. I've used it for my own market tests and evaluations. As such, it may be a little rough around the edges, but is usable. I never did get around to implementing the IHistory module. Instead, I bypassed that interface and wrote a mechanism to download histories for multiple symbols simultaneously. If you do implement an IHistory interface, let me know, I'll update the code and release it.

There are few libraries that use a SQL Database back end. the OneUnified.SmartQuant library has a SQL file for recreating the database (you'll need to do the permissions manually), and a .csv which will load the iqMessageFormat table with stuff that might be useful.

The reason for this is that I use Genesis Securities as my broker. They have a C++ trading library, which I've started to integrate... in a C++ environment. With Boost (which will have time series shortly), QuantLib, GSL, and a bunch of other libraries, I've found I can revert back to my favorite development environment for live market trading. QD will still remain as a solid simulation and testing environment for me (depending upon what favorable or unfavorable changes QuantHouse makes to release arrangements--they've indicated that source code rights will no longer exist--which will suck big time).

I'm rewriting and expanding the scalping example (from the application archive) in C++ to run properly with two trading accounts, and provide detail I couldn't do with the C# version.

If you have questions, let me know.

Happy Trading!

[/Trading/SmartQuant] permanent link

2007 Oct 06 - Sat

New Ideas -- For The Market Place, not the Existing Business Model

Dare Obasanjo wrote a nice blog entry summarizing some key ideas from Marc Andreessen and Clayton Christensen. In his entry entitled Stupid Things Big Companies Do, he had a great exerpt that confirmed that companies need to diversify their business models, offered up a key irony, and ended by providing a possible solution:

People come up with lots of new ideas, but nothing happens. They get very disillusioned. Never does an idea pop out of a person's head as a completely fleshed-out business plan. It has to go through a process that will get approved and funded. You're not two weeks into the process until you realize, "gosh, the sales force is not going to sell this thing," and you change the economics. Then two weeks later, marketing says they won't support it because it doesn't fit the brand, so we've got to change the whole concept.

All those forces act to make the idea conform to the company's existing business model, not to the marketplace. And that's the rub. So the senior managers today, thirsty for innovation, stand at the outlet of this pipe, see the dribbling out of me-too innovation after me-too innovation, and they scream up to the back end, "Hey, you guys, get more innovative! We need more and better innovative ideas!" But that's not the problem. The problem is this shaping process that conforms all these innovative ideas to the current business model of the company.

[/Personal/Business] permanent link

Web 2.0 Site Development with Wt -- Dead Easy

I've written an earlier article about installing the Wt C++ Web Toolkit. I think it took longer to get all the bits and pieces properly arranged than it did me to cook up an example to check that it did what I thought it could do.

I'm quote pleased and impressed with the tool. I was able to get a basic web application up and running in about two days, which were spent reading the documentation, reviewing the class libraries, looking at some of the samples, poking through the PostgreSQL C library, and then hacking stuff into the basic Wt 'Hello World' application.

The authors of Wt have done a great job of hiding all the Javascript go-between code. I can focus on the higher level parts of my design, rather than the underlying machinery to make an interface work. About the only thing I've encountered that I'd like to change is to take the inline Javascript code, and get it put into a .js file.

The test I came up with was to open up a database, select a bunch of records, and present the records in a table which could be stepped through (forwards and backwards) 10 records at a time... and all this without having to refresh the whole page, just the table itself. The following code proves it can be done.

It allows one to turn an internet browser into a true application front end with most of the usual gui functions provided.

This first file is ouipam.cpp, the file with the 'main' start for each session connection:

//============================================================================
// Name        : ouipam.cpp
// Author      : Ray Burkholder
// Version     :
// Copyright   : (c) 2007 One Unified
// Description : OUIPAM:  One Unified IP Address Management
//============================================================================

// server push uses WApplication enableUpdates(), triggerUpdate(), 
getUpdateLock();

#include "Responder.h"

using namespace Wt;

WApplication *createApplication(const WEnvironment& env) {

  // Instantiate the Wt application.
  Responder *appl = new Responder(env);
  return appl;
}

int main(int argc, char **argv) {
  return WRun(argc, argv, &createApplication);
}

This file is Responder.h, the header file for the responder, which handles the WApplication (or session):

//============================================================================
// Name        : ouipam.cpp
// Author      : Ray Burkholder
// Version     :
// Copyright   : (c) 2007 One Unified
// Description : OUIPAM:  One Unified IP Address Management
//============================================================================

#ifndef RESPONDER_H_
#define RESPONDER_H_

#include <WApplication>
#include <WContainerWidget>
#include <WText>
#include <WPushButton>
#include <WTable>
#include <WTableCell>

#include "libpq-fe.h"

using namespace Wt;

class Responder : public WApplication {
public:
  Responder(const WEnvironment& env);
  virtual ~Responder();
protected:
  WPushButton *button;
  WTable *table;
private:
  int cnt;
  PGconn *conn1;
  PGresult *result;
  int nRowsToShow;
  int cntRowsFound;
  int cntColumnsFound;
  int ixFirstRowShowing; // 0 offset
  void OnButtonListBegin();
  void OnButtonForeward();
  void OnButtonBackward();
  void OnButtonListEnd();
  void ShowRows();
};

#endif /*RESPONDER_H_*/

This final file is Responder.cpp, where all the session work is performed. The constructor opens the database, creates the webpage basics, and assigns events to each of the forward and backward buttons. When a button is pressed, the appropriate method is called to use the appropriate records and update the table. Again, with Javascript enabled, the web page is not refreshed. Instead, the browser's DOM is updated directly, and only the table cells change. I look foreward to seeing what else I can do with this flexibility. As you can see from the header comment, this is the start of an interactive web application for managing an organizations IP Addresses.

//============================================================================
// Name        : ouipam.cpp
// Author      : Ray Burkholder
// Version     :
// Copyright   : (c) 2007 One Unified
// Description : OUIPAM:  One Unified IP Address Management
//============================================================================

#include "Responder.h"
#include "WBreak"
#include <sstream>
#include <ostream>
#include <algorithm>

using namespace std;

Responder::Responder(const WEnvironment& env) :
  WApplication(env ) {

  // Set application title
  setTitle("OUIPAM by One Unified");

  ostringstream ss;

  // perform database query 

  conn1
      = PQconnectdb("hostaddr=127.0.0.1 port=5432 dbname=oneunified user=oneunified 
password=xxx");
  ConnStatusType stat = PQstatus(conn1 );
  if (CONNECTION_OK != stat) {
    //PQfinish(conn1);
    ss << "pq result= bad("<< stat << ")"<< endl;
    root()->addWidget(new WText(ss.str()));
  } else {
    result = PQexec(conn1, "select * from ianaiftype;");
    ExecStatusType statusExec = PQresultStatus(result );
    ss << "pgresult=";
    bool bTuplesFound = false;
    switch (statusExec ) {
      case PGRES_EMPTY_QUERY:
        ss << "empty query";
        break;
      case PGRES_COMMAND_OK:
        ss << "command ok";
        break;
      case PGRES_TUPLES_OK:
        ss << "tuples found";
        bTuplesFound = true;
        break;
      case PGRES_COPY_OUT:
        ss << "copy out";
        break;
      case PGRES_COPY_IN:
        ss << "copy in";
        break;
      case PGRES_BAD_RESPONSE:
        ss << "bad response";
        break;
      case PGRES_NONFATAL_ERROR:
        ss << "non fatal error";
        break;
      case PGRES_FATAL_ERROR:
        ss << "fatal error";
        break;
    }
    root()->addWidget(new WText(ss.str()));
    root()->addWidget(new WBreak());

    // present some query statistics

    if (bTuplesFound ) {
      cntRowsFound = PQntuples(result );
      cntColumnsFound = PQnfields(result );
      ss.str("");
      ss << "rows="<< cntRowsFound << ", columns="<< 
          cntColumnsFound << endl;
      root()->addWidget(new WText(ss.str()));
      root()->addWidget(new WBreak());
      for (int i = 0; i < cntColumnsFound; i++) {
        ss.str("");
        ss << PQfname(result, i )<< ", format="<< PQfformat(result, i )
            << ", type="<< PQftype(result, i )<< ", size="
            << PQfsize(result, i )<< endl;
        root()->addWidget(new WText(ss.str()));
        root()->addWidget(new WBreak());
      }

      // create table for row results

      table = new WTable();
      root()->addWidget(table);

      // assign methods to the buttons

      button = new WPushButton( L"<<" );
      root()->addWidget(button );
      button->clicked.connect(SLOT(this, Responder::OnButtonListBegin));

      button = new WPushButton( L"<" );
      root()->addWidget(button );
      button->clicked.connect(SLOT(this, Responder::OnButtonBackward));

      button = new WPushButton( L">" );
      root()->addWidget(button );
      button->clicked.connect(SLOT(this, Responder::OnButtonForeward));

      button = new WPushButton( L">>" );
      root()->addWidget(button );
      button->clicked.connect(SLOT(this, Responder::OnButtonListEnd));

      // show the query results

      nRowsToShow = 10;
      ixFirstRowShowing = 0;
      ShowRows();

    }
  }
}

Responder::~Responder() {
  PQclear(result ); // result exists even with new command, and even if connection is closed;
  PQfinish(conn1 );
}

void Responder::ShowRows() {
  // update rows in the already created table
  WTableCell *cell;
  WText *text;
  int ixRowToShow = ixFirstRowShowing;
  for (int ixTableRow = 0; ixTableRow < nRowsToShow; ixTableRow++) {
    for (int ixColumn = 0; ixColumn < cntColumnsFound; ixColumn++) {
      cell = table->elementAt(ixTableRow, ixColumn );
      cell->clear();
      if (ixRowToShow < cntRowsFound ) {
        text = new WText();
        text->setFormatting(WText::PlainFormatting );
        text->setText(PQgetvalue(result, ixRowToShow, ixColumn ) );
        cell->addWidget(text );
      } else {

      }
    }
    ixRowToShow++;
  }
}

void Responder::OnButtonListBegin() {
  ixFirstRowShowing = 0;
  ShowRows();
}

void Responder::OnButtonBackward() {
  ixFirstRowShowing = max( 0, ixFirstRowShowing - nRowsToShow );
  ShowRows();
}

void Responder::OnButtonForeward() {
  int i = max( 0, cntRowsFound - nRowsToShow );
  ixFirstRowShowing = min(i, ixFirstRowShowing + nRowsToShow );
  ShowRows();
}

void Responder::OnButtonListEnd() {
  ixFirstRowShowing = max( 0, cntRowsFound - nRowsToShow );
  ShowRows();
}

[/Personal/SoftwareDevelopment/CPP] permanent link

Flash File Systems for Embedded Systems, and Otherwise

On Kernel Trap, I see they are discussing the use and maintenance of flash file systems. It seems there are lots of interesting gotchas when using flash file systems intensively. I'm wondering if that might be why the Seagate 32G flash drive is taking a while to get going in the market place.

Anyway, for my own embedded thoughts, it is good to know flash file systems are making good headway into kernel integration.

[/OpenSource/Linux] permanent link

Open Source Site of the Day -- OSSWAD: Open Source Savvy Web Application Developer

An OSSWAD, almost sounds like a forbidden word. Based upon Bob Zurek's column, there are Osswads, and then there are OSSWADs. Many developers of web sites 'out there' already use Open Source tools to get the job done. Apache, MySQL, PostgreSQL, Perl, and PHP are commonly used tools by Osswads.

On the other hand, one gets the impression that real, enterprise savvy OSSWADs, use some real hard core tools like:

• hadoop: large scale distributed compute clusters. Amazon has a developer resource center with a Hadoop/MapReduce Paper.
• lucene: not just search software, but something that provides the basis for getting meaningfull context. There is an informative online book to go alone with it.
• nutch: adds web specifics to lucene: web crawling, link-graphs, and parsers.

These Open Source projects fill in more of the gaps of the thinking I've been doing on providing some contextual search products in specific subject matter areas. It looks like I won't have to do as much ground up development as I thought I might need to do.

[/OpenSource/SiteOfTheDay/D200710] permanent link

libpqxx: A PosgreSQL C++ Wrapper Library

I've looked at the C library for PostgreSQL and then wanted to see if there were any C++ wrappers for it. The most current appears to be libpqxx. On first blush, it looks very good. It if functional and robust. But... it has one draw back. The library insists on converting all binary stuff into text for passing back to the caller. For some applications, that can be a reasonable library simplifier.

But in this day and age of Templates and polymorphism, it seems to be a copout. Yes, if I had time, I'd probably try my hand at implementing some sort of 'variant' implementation to handle the various types of data that come back.

Sigh. As much as I'd like to use the library, I really want my data in the native form in which I stored it. Perhaps once I've worked with the API, and I've worked my own working version of variants, maybe I'll be in a position to offer up some workable suggestions.

... several hours later ...

I now realize that the PostgreSQL API returns everything as strings. So now I understand the reasoning behind the library and what it returns.

To get binary values, from my understanding, one has to use the COPY routines and decipher the result streams directly. I would have hoped there was some middle ground, where the API will provide the binary values when needed. The API implies that it does, but in actual fact, didn't. I spent an hour or two figuring it out the hardway. I'm really going to have to figure out how gdb works so I can single step through stuff and analyze variables. As it was, I reverted back to the stone age and put in print statements to figure out what was happening.

Now I have a tri-choice:

• Stick with the C API, which seems to have everything one needs, including access to true parameterized queries (I found out after the fact, that the Perl libraries actually do string concatenation, which creates a possibility for SQL Injection attacks).
• Use the libpqxx library and convert stuff to binary when I need it. I must say working with everything as strings is a simplifying assumption, but doesn't do much for accuraccy and performance.
• Start working on some sort of wrapper to examine the results of COPY commands, which is probably more work than I really want to do right now.

[/OpenSource/Programming] permanent link

2007 Oct 03 - Wed

Image Maps with CSS, Plus 20 CSS Tools, Plus other Free Tools

Marketing Technology Blog has a How To: Build An Image Map with CSS entry. Pretty simple and straightforward.

In CSS TOOLBOX: 20+ Tools For Working With CSS, there are links to a bunch of sites offering up quite a number of tips, tricks, and techniques for working with CSS.

This site isn't necessarily CSS, but I thought I'd include it here anyway. Emma Alvarez Blog has an entry called Top best 50 free tools for your website. She references:

• Color Selectors
• Logo Makers
• Button Generators
• Rounded Corner Generators
• Background Makers
• Custom Games for Blogs
• Avatars
• Free Scripts
• Visitor Maps
• Public Domain Photos
• Free Fonts
• Widgets
• Statistics
• Firefox Extensions
• Inspiration
• Optimization Tools
• Image Tools

[/Personal/SoftwareDevelopment/HTML/css] permanent link

A Link to an Article regarding SQL Injection

Many web sites use a back end SQL engine for serving up data. Some Credit Card number thefts can be attributed to poor protection of web page interactions with a back end SQL server through injecting SQL statements into web requests.

A well written article called The Unexpected SQL Injection goes into detail on how SQL Injection happens, and how to prevent it.

Spend some time on the site. There are many other documents providing valuable information regarding the protection of internet based activity.

[/Personal/SoftwareDevelopment] permanent link

Free Tools: CD ISO Create and Burning, Notepad++

CDBurnerXP is a great tool for assembling ISO's and burning them to CD. It isn't bloated like other pay-for stuff out there. This is a good, streamlined, free tool to ... burn CDs and DVDs.

Notepad++ is a is a free source code editor (and Notepad replacement), which supports several programming languages, running under the MS Windows environment. It has a very large feature set, with some being cold-folding, syntax highlighting, and macro-recording. It is a valuable Windows Notepad replacement, and is as fast or faster, and absolutely better.

[/Personal/Technology] permanent link

GSL - GNU Scientific Library

I'm coming across all sorts of interesting things today. Another C++ library I've encountered is the GSL - GNU Scientific Library. From the web-site, it is:

a numerical library for C and C++ programmers. It is free software under the GNU General Public License.

The library provides a wide range of mathematical routines such as random number generators, special functions and least-squares fitting. There are over 1000 functions in total with an extensive test suite.

Some of the subject areas covered include:

Complex Numbers	Roots of Polynomials	Special Functions
Vectors and Matrices	Permutations	Sorting
BLAS Support	Linear Algebra	Eigensystems
Fast Fourier Transforms	Quadrature	Random Numbers
Quasi-Random Sequences	Random Distributions	Statistics
Histograms	N-Tuples	Monte Carlo Integration
Simulated Annealing	Differential Equations	Interpolation
Numerical Differentiation	Chebyshev Approximation	Series Acceleration
Discrete Hankel Transforms	Root-Finding	Minimization
Least-Squares Fitting	Physical Constants	IEEE Floating-Point
Discrete Wavelet Transforms

[/Personal/SoftwareDevelopment/CPP] permanent link

Cryptographic Library for C++

ergo offers up some explanation and examples of using the CryptoPP - The cryptographic library for C++. He also refers to something else that may be of interest, a SSL++: C++ Headers for OpenSSL. But I think most just compile and link directly with the headers from OpenSSL.

[/Personal/SoftwareDevelopment/CPP] permanent link

More Code Coloring Capability for HTML

Google Code Colorizer is one to add to the list.

[/Personal/SoftwareDevelopment/HTML] permanent link

Life, Judgement and Experience

The Tired Architect talks about second systems, and refers to one of my favorite books: The Mythical Man Month by Frederick Brooks.

Anyway, The Tired Architect has a good quote:

As they say, good judgement comes from experience, and experience from bad judgement. It makes you a better architect, like how pain makes you appreciate health better.

[/Personal/TagLines] permanent link

Debian Installation on a Dell Laptop

Here is a page discussing the Installation of Debian 3.1 on a Dell Laptop. There were some key commands I learned from that article:

• lspci -v # to list adaptors on pci buss
• dmesg # to show boot log
• lsmod # to show installed kernel modules
• fdisk -l # to show drive partitions

Even though it is an older site, it does have some useful tidbits of information.

Other pages on the site provide notes one wireless war driving and such, plus other stuff.

[/OpenSource/Debian/ArticleLinks] permanent link

Installing And Configuring Wt, a C++ Web Toolkit

I mentioned in another article about doing web page development with C++. I came across Wt: a C++ Web Toolkit. It has been a bear to configure with the ASIO library. I'll layout what I've done below. I'm stuck with an SSL initialization problem. I'm probably going to have to move to FastCGI to see how well that works.

Based upon the forums, there are supposed to be some Debian Packages already available to make this work. Packages seem to be a bit behind the latest versions of stuff, so in this case, I wanted to be able to have a build system where I could incorporate the latest of Boost (which is supposed to have time-series avaliable shortly), and with Wt (which has an active CVS feed).

Anyway, here is my installation process so far. There are a couple of Kludges due to platform differences (a prefix of lib is needed on some stuff, which I should do a SED with at some time) and I can't figure out how the header file include stuff works properly (in order to keep it in a separate directory).

I started by downloading asio-0.3.7.tar.gz from asio.sf.net and boost_1_34_1.tar.tz from www.boost.org, and expanding them out to their directories in /usr/src.

apt-get install gcc
apt-get install zlib1g
apt-get install zlib1g-dev
apt-get install libbz2-dev
apt-get install libgd-dev
apt-get install cmake
apt-get install libfcgi-dev
apt-get install libapache2-mod-fastcgi
apt-get install libssl-dev

cd /usr/src/boost_1_34_1
./configure --without-icu --without-libraries=python,wave,test --libdir=/usr/lib/boost_34_1
make install
ln -s /usr/lib/boost_1_34_1/ /usr/lib/boost
ln -s /usr/include/boost-1_34_1/boost /usr/include/boost

Before proceeding, a patch needs to be applied to one of the ASIO files:

@@ -45,13 +45,13 @@
       {
         ::SSL_library_init();
         ::SSL_load_error_strings();        
+        ::OpenSSL_add_ssl_algorithms();
 
         mutexes_.resize(::CRYPTO_num_locks());
         for (size_t i = 0; i < mutexes_.size(); ++i)
           mutexes_[i].reset(new asio::detail::mutex);
         ::CRYPTO_set_locking_callback(&do_init::openssl_locking_func);
 
-        ::OpenSSL_add_ssl_algorithms();
       }
     }

It basically moves the location of '::OpenSSL_add_ssl_algorithms();'. Without it, an error such as the following may occur during runtime:

__gnu_cxx::recursive_init'
  what():  N9__gnu_cxx14recursive_initE
Aborted

ASIO can then be built:

cd /usr/src/asio-0.3.7
./configure --with-boost=/usr/include/boost --libdir=/usr/lib/ --includedir=/usr/include/
make
make install

After that fixup, Wt can be built.

cd /usr/src
cvs -d:pserver:anonymous@witty.cvs.sourceforge.net:/cvsroot/witty login
cvs -z3 -d:pserver:anonymous@witty.cvs.sourceforge.net:/cvsroot/witty co -P wt
cd wt
nano src/CMakeLists.txt
# prefix boost file entries with lib to get libboost
cmake -D DEPLOYROOT=/var/www/wt -D WEBUSER=www-data -D WEBGROUP=www-data \
-D BOOST_DIR=/usr/include/boost/ \
-D BOOST_COMPILER=gcc41 \
-D BOOST_VERSION=1_34_1 \
-D BOOST_INCLUDE_DIR=/usr/include/boost \
-D BOOST_LIB_DIR=/usr/lib/boost/  \
-D BOOST_DT_LIB_MT=/usr/lib/boost \
-D BOOST_DT_LIB=/usr/lib/boost \
-D BOOST_FS_LIB=/usr/lib/boost \
-D BOOST_FS_LIB_MT=/usr/lib/boost \
-D BOOST_PO_LIB_MT=/usr/lib/boost \
-D BOOST_REGEX_LIB_MT=/usr/lib/boost \
-D BOOST_SIGNALS_LIB_MT=/usr/lib/boost \
-D BOOST_THREAD_LIB=/usr/lib/boost \
-D BOOST_ASIO_INCLUDE_DIR=/usr/include/asio/  \
-D CMAKE_INSTALL_PREFIX=/ \
-D LIB_INSTALL_DIR=/usr/lib/wt/  \
-D LIBRARY_OUTPUT_PATH=/usr/lib/wt \
-D SHARED_LIBS=ON \
-D CONNECTOR_FCGI=ON \
-D CONNECTOR_HTTP=OFF \
.
# FCGI ON for FastCGI (production), HTTP ON for ASIO library (development)

make

# on error:
nano src/Ext/cmake_install.cmake
#  comment out cmakefiles line
rm /include/Ext/CMakeFiles
mkdir /include/Ext/CMakeFiles
cp /usr/src/wt/src/Ext/CMakeFiles/* /include/Ext/CMakeFiles

make install
mkdir /usr/include/wt
mv  /include/* /usr/include/wt/
rmdir /include

nano /etc/ld.so.conf
#  put in:
#   /usr/lib/wt
#   /usr/lib/boost_1_34_1
# ldconfig needs to be run if the fcgi or http libraries get switched or added
ldconfig

Back in Eclipse, I created C++ ANSI project, and then placed the content from examples/hello/hello.cpp into the .cpp file of the new project. The directories '/usr/include/boost-1_34_1' and '/usr/include/wt' need to be entered as 'include' paths. For GCC C++ Linker, the following are -L library search paths:

• /usr/lib/wt
• /usr/lib/boost_1_34_1

The following are -l libraries:

• boost_signals-gcc41-mt-d
• boost_filesystem-gcc41-mt-d
• boost_program_options-gcc41-mt-d
• boost_thread-gcc41-mt-d
• boost_regex-gcc41-mt-d
• wt
• wthttp or wtfcgi
• wtext

So... with some recompiling, I was able to get the hello sample up and running with FastCGI and then with the ASIO library.

When using FCGI, I renamed the compiled file to hello.wt, and placed it into /var/www/wt/, added the line 'FastCgiServer /var/www/wt/hello.wt' into /etc/apache2/mods-enabled/fastcgi.conf, and restarted Apache. Then by browsing to localhost/wt/hello.wt, I was able to get the demonstration.

When using ASIO, set the folloing for run-time command-line arguments in Eclipse to successfully start the application: '--doc-root=/var/www/wt --http-address=0.0.0.0 --http-port=8080'. Browsing to localhost:8080 will get the web page.

[/OpenSource/Debian/Development] permanent link

KDE / X-Windows Screen Resolution Adjustment

I have a laptop that has 1920 x 1200 resolution on it. The default VESA drivers don't like that resolution. After finding out the type of video card, I was able to use the following command to update the card type and resolution:

dpkg-reconfigure -phigh xserver-xorg

I am now running at full resolution on the laptop.

[/OpenSource/Debian] permanent link

Installing Eclipse C++ Development Environment

The Eclipse C++ Development Environment is a Java based Integrated Development Environment (IDE). On Debian, it is a 'non-free' install. As such, the file /etc/apt/sources.list needs to have the word 'non-free' tacked on to the end of the deb and deb-src lines.

The command 'apt-get update' then needs to be run to update its lists.

The Java runtime environment can then be downloaded with 'apt-get install sun-java-jre'.

The downloaded Eclipse C++ Development Environment can be expanded. Within the expanded directory is a executable file called 'eclipse'. Start it and the environment is up and running. Dead easy.

[/OpenSource/Debian/Development] permanent link

2007 Sep 24 - Mon

Implementing SpeedyCGI/PersistentPerl on Debian Etch

I've recently moved away from Perl and Mason for web hosting into the world of C++ and CGI with Wt.

But for my own future reference, for if/when I need to do something with Perl in Apache, I can refer to Falko Timme's article on Speeding Up Perl Scripts With SpeedyCGI/PersistentPerl On Debian Etch as a reference on how to make Perl fast in Apache.

[/OpenSource/Debian/ArticleLinks] permanent link

Embedded Debian

Enterprise Networking has a short article on some recent specialized motherboards running embedded Debian. They have an article called Voyage Linux: The Comforts of Debian. Embedded.. A number of specific distributions are mentioned, such as m0n0wall, uClibc, Pyramid, and iMedia. The article is about a new contender called Voyage Linux.

The article is about getting Voyage Linux installed and running on flash card. It is the first in a series for ultimately getting an internet-connection sharing firewall in place.

[/OpenSource/Debian/ArticleLinks] permanent link

phoneAlarm for the PocketPC, and GPS Also

I believe the HTC Kaiser is now out and availble. It is getting good reviews as a good device that does have everything, including the kitchen sink. The only think I havn't heard yet is whether or not it has good battery life (for a device with more memory and more functionality, does it come with a power price?).

A reviewer at Pocket PC Thoughts mentioned a piece of software the may prove useful with the device, at least it did for him... something called PhoneAlarm by pocketMAx.

On the GPS side of things, the reviewer mentioned GPS products called Pharos Ostia and Delorme. These are add-on packages. The HTC does come bundled with TOm Tom Navigator, but maps are extra cost, which may be true with the other two mentioned items.

[/Personal/Technology] permanent link

Sentiment Indicators with Option Statistics

When I have the time, I've been spending it adding capabilities to my trading software. My current addition is an Option Watcher. Nothing to really trade, just some thing to watch the state a complete options list for the trading instrument in which I'm interested.

A while ago, or rather, a long while ago, I looked into trading options. That turned out to be a very complicated endeavor. I decided to set it aside and come back to it later. Now isn't quite the 'later' I was thinking about, but I've been keeping them in mind. An article by Jeff Neal from Optionetics expanded upon the recent thoughts I've been having with his article called OUTSIDE THE BOX: Option Statistics as Sentiment Indicators. Here are a few choice exerpts where he says things better than I can:

One of the best ways to get a handle on sentiment in a particular stock is to monitor the activity of option traders. For instance, monitoring and tracking option volume and option open interest changes can reveal important information in regards to the expectations of traders, as well as how they may be positioned.

Option volume when unusually high can often times can identify explosive moves and identifies for the trader just where the action is taking place.

To best forecast a directional change in the market, it is important to monitor the daily gyrations of open interest. The thinking is that small investors are typically on the wrong side of a rally, an unusual increase or decline in the open interest of put and/or calls often signal a change in directional bias. Usually an abnormal rise or decline in open interest sends a contrarian type signal to the sentiment trader.

[/Trading/AutomatedTrading] permanent link

Recent Comments on Scalping in Elite Trader Forums

"The market is all about watching the same areas as the pros are and following their lead. Breakouts on low volume are rookies taking the shares off the pro's hands with no clue they can't fuel the next leg." -- mcichocki

"Some people can't scalp and others can't swing trade, it's all about finding what works for your mind." -- mcichocki

"learn to read tape and remember key numbers from tape support and resistance. The tape is true buying and selling so if you know what your doing with it you will know key levels to watch for breaks with heavy buyers/sellers." -- mcichocki (If one has a good memory, tape reading works. For those of us with less than good memories, having a program that shows total 'volume at price' opens up new vistas of information, similar to what you find when reading a tape.)

"Scalping can work at any market conditions, up, down, sideway... there are slow but no garbage days for a good scalper." -- enlightedtrader

"It's not unusual for a master scalper to obtain a daily trading outcome greater than the daily range although I've never seen it done on a consistently basis, day after day, often yes but not always." -- QuantPlus

"Signals always abound and noise has little impact - this is the zone where noise is tradable. " -- yoohoo

"The fact remains that the market zig-zags in every timeframe, and he who captures most of the distance covered has lower risk and greater return. Of course there us a cut off point where size is the final limiting factor for the scalper." -- yoohoo

"A zig-zag is longer than a straight line. The maximum profits are gained by trading the waves - that's simple maths." -- yoohoo

[/Trading] permanent link

HOW-TO: SSH and Remote File Systems

Debian Admin has an article called "Mount a remote file system through ssh using sshfs".

Here is the introduction of the how-to:

If you want to access a remote file system through ssh you need to install sshfs. sshfs is a filesystem client based on the SSH File Transfer Protocol. Since most SSH servers already support this protocol it is very easy to set up: i.e. on the server side there's nothing to do. On the client side mounting the file system is as easy as logging into the server with ssh.

[/OpenSource/Debian] permanent link

2007 Sep 20 - Thu

SmartQuant QuantDeveloper & DataCenter Release

SmartQuant has released a revision to DataCenter and QuantDeveloper. DataCenter and QuantDeveloper are at the following revision levels:

DataCenter
Version 2.3.3 (20-Sep-2007) 

QuantDeveloper Enterprise Edition
Version 2.7.1 (20-Sep-2007) 

QuantDeveloper Source Code
Version 2.6.2 (10-Jul-2007)
* Recent Versions available through 
  version control 

[/Trading/SmartQuant/Releases] permanent link

2007 Sep 18 - Tue

DirectSound Error 88780078

Sometime towards the end of July, I installed Windows XP latest sound stuff (Windows Media, DirectSound, etc), version 10 I think. I had been putting it off for a while. I had heard about licensing issues, codec problems, and such. In a moment of weakness, I clicked the Windows Update button and installed it. What a pain.

WinAmp plays fine most of the time. At some point in time, when it isnt' playing, and I 'do something', I have no idea yet, and when I go back to playing WinAmp, I'll get a message like:

Bad DirectSound Driver. Please install proper drivers or select another device in configuration. Error code: 88780078

Most of the time, I reboot my computer, and things magically fix themselves.

Upon further searching for solutions, someone pointed out a possible simple fix for my problmem of the DirectSound 88780078 problem:

• Right click on My Computer
• Manage
• Device Manager
• go down to where the exclamation point is
• Right click to Disable, then Enable it.

In this case, it solved my problem. Now I'd like to know how it breaks randomly in the first place. Some have said it has to do with VMWare. I do have VMWare workstation installed, but hasn't been running for a while.

Any suggestions?

I did come across Microsoft Knowledge Base article 29030. You can run a Windows Installer Cleanup Utility. It's first screen shows, ironically, "Welcome to the Windows Installer Clean Up Installation Wizard". When running the application after the installation, I don't see anything having to do with sound, so just cancelled out.

There are more interesting solutions at TechSpot.

[/Personal/Technology/AudioPhonics] permanent link

2007 Sep 17 - Mon

Configuring Wheel Mouse in Debian

I'm running Debian in VMWare in Workstation mode. My wheel mouse doesn't work automatically. I had to have the following configuration in /etc/X11/xorg.conf for it to work:

Section "InputDevice"
        Identifier      "Configured Mouse"
        Driver          "mouse"
        Option          "CorePointer"
        Option          "Device"                "/dev/input/mice"
        Option          "Protocol"              "ExplorerPS/2"
        Option          "Emulate3Buttons"       "true"
        Option          "ZAxisMapping" "4 5"
        Option          "Buttons" "5"
EndSection

[/OpenSource/Debian] permanent link

Debian's Firefox is Known as Ice Weasel, and breaks Google Toolbar

Ok, I might be behind the times somewhat, I havn't used Debian in workstation mode in a while. To date, I've been using it substantially in console/server mode. Today I found trying to get Firefox to run, while in workstation mode, to be an 'interesting' experience.

Using Debian's built-in browser, Konqueror, I went to www.getfirefox.com to download Firefox. The download worked fine, but figuring out how to install/run it was.., not possible.

A little web searching told me that due to some differences of opinion between MOzilla and Debian, Debian has decided to do things a bit differently. With an 'apt-get install firefox', you get the latest software, but under a new name: IceWeasel. All well and fine. Until I wanted to use the Google Toolbar.

It isn't possible to install the toolbar because Google checks the user agent string. So... the trick is to change the user agent string. Put 'about:config' in the address bar. You'll get a bunch of entries. To make it easier to find the needed one, put 'useragent' in the filter line. In the line for 'general.useragent.extra.firefox', change 'iceweasel' to 'Firefox'. Restart firefox. The Google Toolbar is now available.

So in summary, don't download Firefox from the Mozilla web site. Instead, obtain the Debian install package. A change to the useragent will be needed to make it 'more compatible' with plugins and such.

The other plugin I use is FireBug with Firefox. This is a handy tool for troubleshooting CSS and HTML code on a web page. Two cool things with FireBug: 1) as you pass over html code, it's area on the web page is highlighted, and 2) you can edit HTML and CSS entries to see their immediate results reflected in the browser. Installing this tool took one trick... when downloading, do a 'run with' firefox. A new window will open allowing the install to proceed.

[/OpenSource/Debian] permanent link

2007 Sep 16 - Sun

Rock Paper Scissors

I think my first introduction to Rock Paper Scissors was in Dr. Who episode. I thought it was just something the writers made up. Later I was introduced to it when figuring out who was going to buy the beer with a best two out of three.

Now I find out that there is an actual web site devoted to the game: World RPS Society, with a by-line of "Serving the Needs of Decision Makers Since 1918". Even yearly tournaments are established locally as well as internationally. The site even has on-line training software to get you introduced in the privacy of your own desk.

There is even a book about Rock Paper Scissors. Well actually more than one... Amazon has 18 listed, some of which are only names for novels.

The basic rules of Rock Paper Scissors are:

• Rock wins against Scissors,
• Scissors wins against Paper
• Paper wins against Rock

When used in a series of three moves, there are 27 possible combinations. Each set of three combinations is a gambit. These are the eight most popular gambits:

• Avalanche (RRR)
• Bureaucrat (PPP)
• Crescendo (PSR)
• Dénouement (RSP)
• Fistfull o. Dollars (RPP)
• Paper Dolls (PSS)
• Scissor Sandwich (PSP)
• Toolbox (SSS)

The one thing I notice about these gambits is that with seven of the eight, you can sequence readily from one to another in double gambit pairs, whereas with the eighth, the ToolBox, it can't be the first of a double gambit. This would tend to indicate a bias towards seven of the eight. But I suppose that is a starting point for advanced strategies.

Further information in the advanced section indicates that there are "Exclusive Strategies", a strategy where a player will only play two of the three throws. The opponent plays based upon the fact that the missing through has to come up at some time.

If you read the site, you'll find that there is all sorts of psychology and strategy wrapped up in the game. Almost like playing the prisoner's delima.

[/Personal] permanent link

2007 Sep 15 - Sat

Full Text RSS Blosxom Feeds

I previously wrote about getting a basic RSS Feed going. That feed had titles only in it. I wanted to get text into it as well. I had some problems getting it going. I think that if I ran my content through a validator like FEED Validator to start, I might have made progress faster. The key thing is that I need to write my text articles with out ampersands in the titles.

Anyway, I used information from Operational Dynamics web site to figure out the special command to include formatted content with 'CDATA'. I also used his plugin.

Here is the content of my files (wget may be necessary to get the content pure):

• rss20
• content_type.rss
• date.rss
• foot.rss
• head.rss
• story.rss

By removing a couple of minor elements in head.rss, the plug in rss20 won't be needed.

I would really like to get blog comments going. I tried them once but got too much spam. My next project is to migrate to Moveable Type. Hopefully they have better ways of handling comments.

[/OpenSource/blosxom] permanent link

NistNet and Netem: WAN Empairment Emulators

A client was putting together and testing an equipment package that was to be installed in their Disaster Recovery site. After getting everything up and running and tested, they wanted to test the replication performance in a 'real' WAN environment. One way to do this is to use a real WAN. The other way is to simulate a WAN. Consdering their DR site is several thousand miles away, simulating the WAN would be logistically better.

To emulate a WAN, one needs to be able to control:

• variable delay
• delay distribution
• packet loss
• packet re-ordering
• rate control

In researching possible tools for emulating an WAN environment, I came across two open source tools WAN emulation tools. The first one I saw was NIST Net. Although it hasn't received too many recent updates, it does still have an active forum. And it appears to be quite sophisticated. It does require an x-windows for a library during compile time. With X11 Forewarding, you don't need to burden the WAN emulating computer with a GUI, can use another computer as a terminal. You can reference one of my Cygwin pages on how to remote X11 applications.

It is said that NIST Net is a bit better than NetEm as NIST Net has tighter controls on its delay mechanisms.

I came across the second wAN Emulator purely by chance. In reading through some of the NIST Net forum articles, one of the mentioned that the Linux Kernel already has one built in: NetEm. One may need to enable it and rebuld the kernel. It doesn't have a user interface, but instead relies on command line utilities. Someone did do up a GUI for NetEm, but has removed it for one reason or another. A newsgroup article has a reference to where it can be obtained. Someone else mentioned that MasterShaper could be used as an interface to the capabilities of NetEm.

Page 15 of a slide presentation shows command line examples for running NIST Net as well as NetEm. Another document offers up an example of using NetEm.

I was able to get NIST Net built on a Debian box. But the DR equipment had to be shipped out before I could actually give it a try. Oh well, I'll find another project to try it out on. Here are some build instructions for a recent Debian Kernel. There are some variations regarding availability of config.h depending upon the 2.6 kernel version you have available.

When obtaining the NistNet code in one of the commands below, some of the instructions assume you've expanded the library in /usr/src. As such, when the library is expanded, you'll need to change four lines in /usr/src/nistnet-3.0a/kernel/knistnet.c from

return ippt->func(skb, dev, ippt);

to:

return ippt->func(skb, dev, ippt, NULL);

You may need to comment out the following line in /usr/src/nistnet-3.0a/kernel/nistnet_table.c:

/* typedef enum {false = FALSE, true = TRUE} boolean; */

You may need to add a dummy config.h in /usr/src/linux-headers-2.6.21-2-686/include/linux/ with:

#ifndef _LINUX_CONFIG_H
#define _LINUX_CONFIG_H

#include 

#endif

You should also confirm that this file exists (depending upon your kernel version): /usr/src/linux-headers-2.6.21-2-686/include/linux/autoconf.h

Here are remaining installation instructions:

apt-get install vlan
apt-get install linux-headers-2.6.21-2-686
apt-get install x-window-system-core

apt-get install libxaw-headers libxmu-headers
apt-get install libxp-dev                        
apt-get install xaw3dg-dev

# might need:
ln -s /usr/lib/libXaw7.so.7 /usr/lib/libXaw.so

wget http://www-x.antd.nist.gov/nistnet/dist/nistnet.2.0.12c.tar.gz
tar -zxvf nistnet.2.0.12c.tar.gz

./configure
make
make install

depmod
modprobe nistnet
lsmod | grep nistnet
cnistnet -G

[/OpenSource/Debian/MasterShaper] permanent link

Trading Site of the Day -- Trade-Ideas: Real Time Idea and Signal Generation

In another article, I referenced the Trade-Ideas Blog. I don't recall ever making it over to their web site. Today I made it over to their site by way of an article there on Scalp Traders Access Their Best Trades with Trade-Ideas. They have links from their to a page they have whichreferences a bunch of their signals. I've coded several of those signals elsewhere, but this opens a whole new vista on possible trading opportunities from a scalping perspective.

Another page references how their analysis engine uses different time frames, different data types, and different statistics to come up with its signals.

I like how they do mini charts to provide an idea of daily and weekly price movements.

[/Trading/SiteOfTheDay/D200709] permanent link

Burning Water aka Breaking Water Into Hydrogen/Oxygen aka Electrolysis

A few days ago, there was a much talked about article regarding Radio Frequencies Help Burn Salt Water. Even John Mauldin referenced it.

Many people have gone off the deep end thinking that this is the next best thing to water running up hill. By looking a carefully at the video and reading the article, and doing a little research, one can understand what is going on. First a little background:

• Electrolysis
• Homemade hydrogen
• Producing and Detecting Hydrogen

I'd say the radio frequency energy acts in a way similar to feeding electrical current directly through the salt solution (radio frequencies are the byproducts of an oscillating electrical current). The video showed us 'something' using between 200 and 400 Watts, about the equivalent of four standard 100W light bulbs. They were careful to not show us the frequencies used though.

Thinking more about how the hydrogen is produced and recombined, I seem to recall article and/or video from many moons ago about once you got this burning reaction going, you could keep the mix running by continually adding water. (No this is not perpetual motion because you are still supplying a source of energy--the water). Anyone remember research on this type of thing? In some blogs, there were comments suggesting that even though this a a decent reaction, but because it was not self-perpetuating, it isn't good. I beg to differ. Just think about oil for a moment. Untold amounts of energy is spent in finding it, drilling it, pumping it, refining it, and delivering it. Yet there is enough energy left over to fuel our cars and planes. Could it not be said that even though we lose energy in breaking the bonds to produce hydrogen, that there may be more energy left over to do other stuff? Looking at it another way, electrical energy is used to break the bond, and we get heat energy (although inefficient to use) when the bond is recombined.

I did find a reference to a Water Engine, but it wasn't quite what I was after. Besides, I think the scientist in that post misses the point. Yes, you can't use the same amount of water and expect to keep re-using it. But if you keep adding water into the system, you keep refueling the system, just like putting more gas into your auto fuel tank. An in most places, water is quite abundant. Burning water may be more or less as energy efficient as burning oil, but if I recall correctly (!), there is more water than oil.

If someone could point me to a reference where some one demonstrates a 'burning water' engine--an engine where you prime it with hydrogen, feed it water, and keep it burning--I'd be most appreciative.

But all this is water under the bridge, in comparison to what the video was really showing. It shows that if you can deliver the special particles to the site of a cancerous growth, and use radio waves to activate the particles, you can kill the cancer cells and leave everything else intact. Now that is what one would call 'precision bombing'.

Update: Through a Google link that passed through my site, I came across Stan Meyer and a Water Fuel Cell. If you look those up, you'll see all sorts of stories, conspiracies, and strange information floating around. Who/What can you beleive.

For other alternate energy systems, PESWiki looks interesting.

[/Personal/Technology] permanent link

Word Lists with a Calculator

FreshMeat, contrary to any derogatory images it may conjure in your mind, is a site dedicated to publicizing releases of open source software. It is a great combo to SourceForge.

Today on Freshmeat was a link to Frink, "a practical calculating tool and programming language designed to help us all to better understand the world around us, to help us get calculations right without getting bogged down in the mechanics, and to make a tool that's really useful in the real world. It tracks units of measure (feet, meters, kilograms, watts, etc.) through all calculations, allowing you to make physical calculations easily, to mix units of measure transparently, and ensures that the answers come out right." It deserves a special link on my toolbar.

One of the things it know how to do is word searches. Cheating at Scrabble is one suggested use. The interesting point, though, is that in one of the descriptions, he points to word list he uses: Grady Ward's Moby, a lexicon project. It has a hyphenator, part-of-speach definitions, pronunciator, Shakespeare, Thesaurus, and 610,000 words and phrases. It shows as a last update being around the year 2000 or so.

[/Personal/SoftwareDevelopment] permanent link

2007 Sep 14 - Fri

Software Development, Coders, and C++ Libraries

I grew up with Assembler, Pascal, then C, then C++, then C#, and now I'm back to C++. I've found that C# makes things easier for graphical programming, but it feels sluggish when doing some computationally intensive things. I've since moved back to C++. Development time has increased on some stuff, but I think things are better, and I derive more pleasure from C++ development. And C++ has a rich heritage and a rich library universe. This entry goes through some interesting things I've found.

One of the first libraries I came across was the Boost Libraries. I believe I've written about these before. A few specifics of interest include Regular Expressions, a soon to be released Time Series, date/time operations, some geometry constructs, state machine tools, and, well, the list goes on.

A few days ago, in looking for sophisticated Web Application tool kit. Wt: a C++ Web Toolkit appears to fit that niche very well. It also handles Ajax like functionality.

To assist with web development and layout, Firebug: A Firefox Addon might be of value for page layout issues. Although it has nothing to do with C++, which is the main topic here, it does have to do with finding a viable solution for checking out web page design.

Earlier today, I came across dzone: fresh links for developers. It has a wealth of links to articles written by developers for developers, developers of all categories and skill sets. Doing a search on C++ comes up with quite a list of articles.

One of the links pointed to The Programmicon. This article is mostly game based, but gaming shares cross-functionality with many disciplines. This once had two links to resources regarding finance. I was first introduced to Multivariate Embedding Methods by Carol Alexander on page 405 of her book Market Models. Although she won a prize for best price predictor using a model with that concept, I havn't been completely sold on it's applicability. If I had time I'd try it out. However, a key part of embedding is nearest neighbor analysis. The Programmicon points to a site providing ANN: A Library for Approximate Nearest Neighbor Searching. It also points to TMV - Template Matrix/Vector Library for C++, something else upon which embedding algorithms are built. Embeddings are based upon chaos theory. The concept is to try to find self-similarity in continous time. When similarities are found, you've got a predictor. Easier to say than do.

dzone also re-introduced me to LUA: An Embedded Programming Language. Debian Administration discusses how to incorporate it in to C++. I'm thinking it might be useful for scripting signals in a network monitoring package or defining charts in a financial modelling solution, or performing information searches in text analysis tools, or performing event & signal handling in a Cricket grapher.cgi rewrite. IEEE Software has an 8 page article called Traveling Light, the Lua Way. Kind of related is Kepler: Lua based web development platform.

During a brief flirtation with Fuzzy Logic, where one needs to evalute line crossings and area calcuations, I realized Computational Geometry might be of use. The C++ library Wykobi might be of value for optimized algorithms. The Code Project discusses its use.

I'm currently 'enjoying' MFC based development. I'm wondering if, since I'm still at a relatively early stage, I should be using TrollTech's Qt: Cross-Platform Rich Client Development Framework.

From a Microsoft perspective, Somasegar's Weblog has an article on 'Visual C++ Futures'. There are more than 200 user comments summing up needs, wants, and desires in that universe.

[/Trading/AutomatedTrading] permanent link

IP Addressing Best Practices (plus VRF and VLAN ideas)

I've encountered many weird and wonderful IP Addressing schemes during my consulting engagements. During my early years, I'm sure I've contributed some less than optimal ideas into the mix. I'm going to try to mend my ways and introduce some ideas that I think make things better.

Since the advent of Network Address Translation, network designers/architects/engineers have three basic address ranges from which to choose. These address ranges are based upon what is known as RFC 1918 - Address Allocation for Private Internets. The three ranges are:

• 10.0.0.0/8
• 172.16.0.0/12
• 192.168.0.0/16

These are, what some call non-routable addresses. To be more specific, routers routing public addresses, will typically drop (not route) packets having destination addresses from these ranges. Of course, private networks are free to route these addressess as they see fit.

The commentary below assumes a basic knowledge of ip addressing, subnet allocation, host addressing, and routing.

When starting with small networks, a network installer may simply choose some random /24 block from one of the three ranges, and start assigning addresses to all devices on the network to that block. When installing IP Phones, some similar rationale may be used to pick some onother block for the phones. Some rationales that I've see included:

• let's choose 10.20.10.x for workstations and 10.20.20.x for phones, it keeps them in a similar big block, and keeps the octents nice workable numbers
• let's choose 10.20.30.x for workstations and 192.168.30.x for phones, it keeps them separate and identifiable, but yet similar in one octet
• let's add 172.20.30.x as a network management block as it is something completely different and identifiable as being not normal traffic

I'm sure you can come up with many other mechanisms. But what happens when you want to get another office joined in? Then suddenly one has a bunch of different blocks one has to add into a routing table. The lazy answer would be to simply use a statement like 'network 10.0.0.0'. There. All routes for that block are automatically known and added into the routing table (syntax and concept varies a bit depending upon which routing protocol you use).

I'm not sure if that is an optimal solution. I generally like to be very specific about which subnets or supernets I allow into a network table. With addressing scattered through three different address ranges, detailed route table maintenance becomes time consuming.

I like to use route aggregation whenever possible.

I left that statement by itself as it is a very powerful statement. It is also a strong, guiding principle in network/routing table design.

I've used that to come up with a hierarchical addressing structure that works with many of my client sites. There are some scalability issues for very large organizations with many large offices. For the majority of organizations, I think the following scheme works well. As I'm in Bermuda, Bermuda is the center of my universe, and the numbering is based partially on that concept. There is nothing saying you can't change things to match your own geographic realities.

Since the 10/8 address range has the most number of addresses to work with, I use that as a basis for most schemes. The hiearchy I use follows this pattern. I'll break the last three octets into a bit map and assign meanings:

octet bitmap:
00001010.wwwxxxxx.yyyyzzzz.00000000

bitmap meanings:
www:   region (NA (001), SA (010), Europe (011), PacRim (101), Asia (100)
xxxxx: country [ BM 1, CA 2, US 3, UK 1, IR 2 ]
yyyy:  city [ HAM,BM 1; NY,US 1; PA,US 2 ]
zzzz:  VLAN 

The region numberse, starting at 1, are loosley based upon contintents and how inter-country fibres are run. These categories help with aggregation. Within a region, countries are numbered, starting at 1. Within a country, cities are numbered starting at 1. I start the numbering for each bit section at 1, as I use the 000, 00000, and 0000 portions for addressing of inter-region, inter-country, and inter-city links. You can't really aggregate those addresses into the routes of either site, so they need to be kept separate.

Therefore, for a site in Hamilton, Bermuda, addressing would start at 10.33.16.0. Addresses from the range 10.33.0.0 to 10.33.15.255 would be used for inter-city links. Addresses from 10.32.0.0 to 10.32.255.255 would be used for inter country links. Addresses from 10.0.0.0 to 10.31.255.255 would be used for inter-region links. Ip address utilization in this range is acknowledged to be quite sparse.

You'll note that this scheme breaks when you have coverage in more than 15 cities. Therefor the scheme needs to be adjusted based upon how many locations you have, how many buildings in each location, and how many floors in each location. It doesn't matter how the bit map is structured, just so long as some rule for aggregation is followed as the level of aggregation grows.

At each site, I've been refining a strategy for vlan and ip address block assignments. Sixteen is a base(2) number, and therefore easy to aggregate. I assign 16 /24 blocks and 16 VLANs per site using a strategy similar to:

+0	Subnets for /30 links
+1	Servers
+2	Server ILO Ports
+3	SAN, iSCSI Traffic
+4	Voice Servers & Gateways
+5	Lab/Classroom/Testing
+6	Spare, To Be Assigned
+7	DMZ
+8	Workstations
+9	Phones
+10	Wireless Corporate
+11	Wireless Phones
+12	Wireless Guest
+13	Wired Guest
+14	Printers
+15	Network Management

The first set of eight blocks are server related. The second block of eight are client related. If an organization has mulitiple floors, conceivably a sequential block of eight would be assigned to each additional floor.

For VLAN numbering, I have a couple of different schemes. One scheme starts at VLAN 100 and assigns vlans 100 through 115 to each /24.

A second scheme is more complicated. In some locations, I use VRF's to segragate and route traffic. VRF's are handy when you have guest traffic you want to keep separate from corporate traffic. Cisco has a number of SRND documents on how this is accomplished. Anyway, I've used up to 5 segragations: 1 global routing table plus four VRF's:

1XX	global		100
2XX	vrfExterior	200
3XX	vrfData		300
4XX	vrfVoice	400
5XX	vrfGuest	500

The first column is are the VLAN identifying numbers, the second is the VRF name, and the third column is the OSPF process number. By adding this numbering scheme to the VLAN types previously provided, the following table might be used, where column 1 is the VLAN ID, column 2 is a relative offset of a sequential VLAN numbering system, and the third column is the description:

	+0	Subnets for /30 links
301	+1	Servers
302	+2	Server ILO Ports
303	+3	SAN, iSCSI Traffic
304	+4	Voice Servers & Gateways
305	+5	Lab/Classroom/Testing
	+6	To Be Assigned
	+7	DMZ
308	+8	Workstations
309	+9	Phones
310	+10	Wireless Corporate
311	+11	Wireless Phones
512	+12	Wireless Guest
513	+13	Wired Guest
314	+14	Printers
115	+15	Network Management

Some don't have VLANs as they don't exist as VLANS, but are address ranges uses elsewhere at the site.

Here are some additional VLAN assignments I'll use:

• 1: turned off, or not used at all
• 2: trunk native VLAN
• 3: 802.1x unauhtorized clients
• 20 - 29: switch SPAN ports for various monitoring activities
• 299: vrfExterior router to firewall exterior
• 399: vrfInterior router to firewall interior
• 599: vrfGuest router to firewall interior

Here is a finished table for showing address assignments for two different locations:

VLAN	Offset	BM Office	NY Office	
	+0	10.33.16.0	10.35.16.0	Subnets for /30 links and loopbacks
301	+1	10.33.17.0	10.35.17.0	Servers
302	+2	10.33.18.0	10.35.18.0	Server ILO Ports
303	+3	10.33.19.0	10.35.19.0	SAN, iSCSI Traffic
304	+4	10.33.20.0	10.35.20.0	Voice Servers & Gateways
305	+5	10.33.21.0	10.35.21.0	Lab/Classroom/Testing
	+6	10.33.22.0	10.35.22.0	To Be Assigned
	+7	10.33.23.0	10.35.23.0	DMZ
308	+8	10.33.24.0	10.35.24.0	Workstations
309	+9	10.33.25.0	10.35.25.0	Phones
310	+10	10.33.26.0	10.35.26.0	Wireless Corporate
311	+11	10.33.27.0	10.35.27.0	Wireless Phones
512	+12	10.33.28.0	10.35.28.0	Wireless Guest
513	+13	10.33.29.0	10.35.29.0	Wired Guest
314	+14	10.33.30.0	10.35.30.0	Printers
115	+15	10.33.31.0	10.35.31.0	Network Management
				
		Default Gateway is x.x.x.254		

This type of scheme, when implemented from the start, provides easy expansion of an organization's network, provides an ability to add protections between network devices, and simplifies inter-site routing.

[/OpenSource/Debian/AddressManagement] permanent link

LightFactory for Lighting BMDS' Production of The Sisterhood

It was an interesting experience, to say the least, in terms of being a first time lead lighting designer, introducing new equipment to the theatre, and learning a new piece of software.

All in all, the time was well spent. Live cue updates during rehearsals were simplified through a logical layout of lighting instruments in LightFactory.

In the diagram, instruments are assigned positions based upon the area they illuminate rather than the location they occupy in the grid. As such, in the diagram, there are five groups of four instruments. The top one of the diamond is a down light, the left was an amber 'warm' wash light, the right was a blue 'cold' wash, and the bottom was a lavender wash for the dark skinned actor, (the remainder of the cast were light skinned).

The channel over each of the four ColorCommand lights controlled the intensity. Each ColorCommand had three colors (magenta, yellow, and cyan), plus a beam width control.

Some of the specials were: 35: the bar bottle highlight, 32: a spot for a solo cast member talking to the audience, 19: a red downlight for the lovers, 2: the greenish downlight on the phone, 17,23: plant decoration, and 48: audience lights for intermission.

Over five acts, there were a total of 44 cues for the show, including one for doing a 15% on all lights for a lamp test.

The biggest hassle with LightFactory was in using the color lights. It took some time to work around some issues of assigning colors to a group of color lights. I think this has now been fixed. From what I see in the bug reports, there were some fixes implemented for refreshing Channel Groups and Palettes in a more timely fashion. I thought about updating the software once the bugs were fixed, but decided to run the software as it was through the show: better an enemy you know than one you don't. I'll update the software for the next show.

[/Personal/Lighting] permanent link

2007 Sep 13 - Thu

Computers Are Infallible -- NOT

A recent article at Kernel Trap entitled Data Errors During Drive Communication indicates that "An ongoing study on datasets of several Petabytes have shown that there can be 'silent data corruption' at rates much larger than one might naively expect from the expected error rates in RAID arrays and the expected probability of single bit uncorrected errors in hard disks".

This is an interesting read. I used to blame my ocmputer problems on Random Alpha Particles. I guess I can now expand my horizons.

[/Personal/Technology] permanent link

Trading Site of the Day -- Teresa Lo: Power Swings

Teresa Lo is a retired securities analyst who has a site called PowerSwings. Here analytical skills and powers of commentary show through. Two pages of interest, one on Volatility and one on Intraday Trading Strategies are recent articles with some good meat.

For intraday trading, she boils it down to:

• Use small discretionary setups to scalp the first half-hour, especially when the market is really emotional. I particularly like small flags and gap plays documented in The Ultimate Trading Course.
• Avoid getting killed after the first hour, when the market usually settles down and starts to chop. Stay out.
• Get on board the train if it leaves the station in the afternoon for a trend day into the close.

Once I get into the swing of things, I'm hoping I can break her rule #2, which is where I think with contrarian Limit orders, one can make money in that sort of market condition.

She also promotes here Ultimate Trading Course. I don't know if it is a good thing or a bad thing, perhaps someone could let me know.

Her site links to a number of high quality blogs. WSJ Real Time Economics for one.

[/Trading/SiteOfTheDay/D200709] permanent link

Snow Squall Inn

If you happen to be the Wiscasset, Maine area, and need a place to stay, try out Snow Squall Inn. My good friend Melanie and her husband Paul own and operate the 7 room Bed & Breakfast. I hope to see them in October and check out their place.

[/Personal/Business] permanent link

Lighting for Moliere's Sisterhood, the 80's Version

From this month's production of Moliere's The Sisterhood at Bermuda Musical and Dramatic Society, here are three shots of the set during the Pre-Show, Intermission, and Post-Show Decoration cues. During the periods of activity, the set is fully lit, with subtle dimming of unused area. Unused areas would be the bar and the sofas at various times. This is my first show where I was lead on lighting design and programming.

The goal of the color splashes were to offset the black and white patterns of the main set. The color changes were also meant to occur in a somewhat early morning (cool of the day), mid day (warmer, hot sun), early evening sequence (heavy on sunset colors). The purple and green in the second were used to highlight the only actor with a color wardrobe, Trissotin, who had bright green and purple in his clothing.

I used three HighEnd ColorCommand lights to light the French Doors. One light was aimed at one door from far stage right, one was aimed at the other door from up stage center, and a third provided a splash across the backdrop from up stage center. The beam width was adjustable to provide a beam or a wash as I needed.

Although the pictures don't show it dramatically, a purple color splash was 'thrown' on the red backdrop in the hallway to highlight the phone. A slightly greenish downlight served to show the phone in stark contrast.

A square shuttered light was used to provide a highlight on the bottles at the bar throughout the play.

[/Personal/Lighting] permanent link

2007 Sep 11 - Tue

Recent Paper on Profitability of Technical Stock Trading

There is a recent, very readable paper from Stephan Schulmeister called The Profitability of Technical Stock Trading has Moved from Daily to Intraday Data. His abstract goes like this:

This paper investigates how technical trading systems exploit the momentum and reversal effects in the S&P 500 spot and futures market. The former is exploited by trend-following models, while the latter by contrarian models. In total, the performance of 2580 widely used models is analyzed. When based on daily data, the profitability of technical stock trading has steadily declined since 1960 and has become unprofitable over the 1990s. However, when based on 30-minutes-data the same models produce an average gross return of 8.8% per year between 1983 and 2000. These results do not change substantially when trading is simulated over six subperiods. Those 25 models which performed best over the most recent subperiod produce a significantly higher gross return over the subsequent subperiod than all models. Over the out-of-sample-period 2001-2006 the 2580 models perform much worse than between 1983 and 2000. This result could be due to stock markets becoming more efficient or to stock price trends shifting from 30-minutes-prices to prices of higher frequencies.

One of the interesting comments he makes is that contrarian strategies appear to be more profitable than do trending strategies.

In the article, the author offers up some possible reasons why technical trading is harder (but I should temper that remark and say that successful trading is more profitable with 'higher frequency' data--5 minute bars over 30 minute bars or daily data):

The decline in the profitability of technical trading based on daily data could be explained in two different ways. The "adaptive market hypothesis. (Lo, 2004; Neely-Weller-Ulrich, 2006) holds that asset markets have become gradually more efficient, partly because learning to exploit profit opportunities wipes them out, partly because information technologies steadily improve market efficiency (Ohlson, 2004). The second explanation holds that technical traders have been increasingly using intraday data instead of daily data. This development could have caused intraday price movements to become more persistent and, hence, exploitable by technical models. At the same time price changes on the basis of daily data might have become more erratic. This would then cause technical trading to become less profitable based on daily prices (but not on intraday prices).

Another interesting quote I came across regarding how everyone's trades get jumbled together, and what trader's think about it:

... traders have to form expectations about expectations of all other traders (Keynes. "beauty contest. problem).

[/Trading/TechnicalAnalysis] permanent link

2007 Sep 10 - Mon

Internet Information Analysis

In follow up to a previous post I did on news analysis, I came across Monitor110. They don't release much about how they do stuff, but they do release some information which sets the bar as to what can be done in terms of analysis of information found in various sorts of repositories found on the web.

[/Trading/AutomatedTrading] permanent link

2007 Sep 09 - Sun

VC++ MultiCast Delegate Template

In follow up to my article on Fast Delegates back in June, I used the C++ STL (Standard Template Library) to create a simple, one parameter Multicast Delegate for C++. The code provided here provides a mechanism for creating a Multicast Delegate by incorporating the FastDelegate library. I used the 'vector' template from the STL to implement dynamically adding and removing object method calls to the Delegate.

#pragma once

#include <vector>
#include "FastDelegate.h"
// http://www.codeproject.com/cpp/FastDelegate.asp

template<class RO> class Delegate {
  // RO: Return Object in call

public:
  typedef FastDelegate1<RO> OnMessageHandler;
  void Add( OnMessageHandler function );
  void Remove( OnMessageHandler function );
  bool IsEmpty();
  void operator()( RO );
protected:
private:
  std::vector<OnMessageHandler> rOnFD;
};

template<class RO> void Delegate<RO>::Add( 
OnMessageHandler function ) {
  rOnFD.push_back( function );
}

template<class RO> void Delegate<RO>::Remove( 
OnMessageHandler function ) {

  std::vector<OnMessageHandler>::iterator rOnFD_Iter;

  rOnFD_Iter = rOnFD.begin();
  while ( rOnFD.end() != rOnFD_Iter ) {
    if ( function == *rOnFD_Iter ) {
      rOnFD_Iter.erase( rOnFD_Iter );
      break;
    }
    rOnFD_Iter++;
  }
}

template<class RO> bool Delegate<RO>::IsEmpty() {
  return rOnFD.empty();
}

template<class RO> void Delegate<RO>::operator()( RO ro ) {

  std::vector<OnMessageHandler>::iterator rOnFD_Iter;

  rOnFD_Iter = rOnFD.begin();
  while ( rOnFD.end() != rOnFD_Iter ) {
    (*rOnFD_Iter)( ro );
    rOnFD_Iter++;
  }
}

// =======

// Here is a declaration of three events, with each event receiving a 
// pointer to a object with class of CIQFSymbol:

Delegate<CIQFSymbol*> OnFundamentalMessage, OnUpdateMessage, OnSummaryMessage;

// Here is how to add a delegate to the MultiCast Delegate:

pSym->OnFundamentalMessage.Add( 
  MakeDelegate( this, &CGTScalpDlg::EmitDataFundamental ) );

// EmitDataFundamental is declared with:

void EmitDataFundamental( CIQFSymbol *pSym );

// The mulicast delegate can then be invoked with:

OnFundamentalMessage( this );

// =======

Caveats: I havn't proven that Delegate's Remove method works as intended yet, but theoretically it should, as coded. Proper initialization, deconstruction cleanup, and thread safety are left as exercises for the reader.

[/Personal/SoftwareDevelopment] permanent link

2007 Sep 08 - Sat

BootDisk.com

Bootdisk.com is a handy site for creating all sorts of bootable media for all sorts of operating systems and hardware.

[/Personal/Technology] permanent link

Trading Site of the Day -- Non Dealing Desk: Forum for Forex

For those who want to get inside the wild-west business of forex, Non Dealing Desk Forum appears to be a good one with which to start.

It has quite a number of forum sections, but then ones at the top of the list have to do with 'Non-Dealing Desk Brokers', 'Dealing Desk Brokers', and 'Broker Selection Criteria'. Other forums feature real life examples of what can happen to a trader when submitting trades and stop orders.

Forum participants discuss the pros and cons of various brokers and of what to watch out for.

So even though one sees many advertisments in print and the web for $500 sign up accounts, they really are a guise for taking your money the easy way if you aren't prepared for trading desks trading against you and taking out stops on a regular basis.

So really, in the world of forex, it is caveat emptor. Choose your broker wisely, and above all, really hnow what you are doing (which ironically, may not be possible for the neophyte trying to get started).

[/Trading/SiteOfTheDay/D200709] permanent link

2007 Sep 06 - Thu

News Analysis

I subscribe to DTN's IQFeed data streams. (If you'd like to sign up, let me know I'll do a referral for you.) Anyway, in addition to the usual equity, futures, and options feeds, they have a news feed. Each feed entry has a media source indicator, a headline, a list of associated symbols, and a index number for obtaining the story content.

I thought it might be an interesting project to process each incoming message for its symbol list and do some sort of key word analysis to see if one can get a 'mood' of the article. This might provide some interesting trading ideas for the day.

I don't have the time to do it right now, but am recording my thoughts so I can come back to it a little later.

Two recent articles by Paul C. Tetlock in the The Journal of Finance, one in the June 2007 issue titled "Giving Content to Investor Sentiment: The Role of Media in the Stock Market", and one in an upcoming issue called "More Than Words: Quantifying Language to Measure Firms. Fundamentals", got me thinking about this again.

One of the articles pointed to the General Inquirer, no, not a racy tabloid but a "a computer-assisted approach for content analyses of textual data". Although GI references an application useful for researches, I think the interesting content resides with the spreadsheet of categorized words they have. These words can be used to classify the 'mood' of processed text.

The site also points to a book called "The Content Analysis Guidebook" by Kimberly A. Neuendorf as one that might shed further background on the concept. A while ago, I was taking a look at content anlysis from a different perspective, something akin to classifying market analysis and trading blogs. Some additional book references are linked below.

An application called Yoshikoder is an already built application that can take the GI word lists and process portions of text and produce analysis summaries.

A brief web search brought up a couple of blogs that show some perspective on how to put analysis into perspective:

• Joho the Blog
• Harvard Extended

Some 'possibly' related books:

[/Trading/AutomatedTrading] permanent link

2007 Sep 03 - Mon

Linux, Wine, MFC, Win32 API

for the trading application I'm developing, I was thinking that I'd only be able to run it on a Windows machine due to the fact that a couple of vendor supplied libraries are only supplied as Microsoft Windows .dll's and MFC C++ libraries. Perhaps such is not the case any more.

For whatever reason, I recalled that Wine is a "compatibility layer for running Windows programs". They say they can handle WinSock32 calls, which is probably one of the primary hard things to do.

So I'm hoping I can take the supplied vendor .dlls, my MFC .dlls, and load them into the wine layer, and they'll run. As a result, I can make further use of some of my remotely hosted Linux servers for hosting my trading platform, without resorting to installing either VMWare editions or real Windows platforms.

While on the subject of MFC and such, I want to record a few Win32 API/MFC sites that will help in some of the code development:

• Prineas C++ Resources
• #winprog Win32 Tutorial
• Wine HQ Resource Page

[/Trading/AutomatedTrading] permanent link

2007 Sep 01 - Sat

The Royal Gazette's Review of Sisterhood

Here is what Jennifer Hind of the Royal Gazette had to say of opening night for the Sisterhood:

Black and white with a touch of colour/classy and classic/heartless and cold?

Getting the balance right is one of the themes of this modern translation and reinterpretation of the classic 17th century Les Femmes Savantes by Moliere. The Sisterhood, R.R. Bolt's 1980s take on this comedy of manners and motives, explores sibling rivalry, the battle of the sexes and the conflict between intellectual snobbery and personal integrity at a cracking pace in rhyming couplets, replete with clever puns, irony, witty repartee and physical comedy.

It is also marvellously self-aware, with one character suggesting all the books should be burned . save the collection of Moliere as they must be worth a fortune.

The current production at the Daylesford Theatre, directed by Suzann McLean and produced by Jo Shane, is a clever realisation of a very clever play and well worth the price of admission.

Sisters Armande and Henriette are like chalk and cheese, with Armande seeking maternal approval through the parroting of her mother's and aunt's intellectual pretensions and aggressive feminism, while Henriette declares herself more interested in the sensory world than the world of the intellect.

The rivalry of these siblings, 'marriage is slavery' versus 'sex is better than speculative thought', is compounded by the fact that the poor but charming Clitandre, spurned by a scornful Armande, finds a warm welcome in the arms of Henriette. The young lovers' relationship is supported by Henriette's sympathetic but ineffective father, while her domineering mother has other plans for her daughter: marriage to a poet of dubious ability and large ambition. The parental battle of the sexes is spiced with a liberal sprinkling of a saucy maid, a lascivious aunt and a disloyal pal.

How the happy ending is contrived with a little help from their friends, involves a clever twist and a surprising revelation.

Very little, if anything, has been left to chance in this production. Even the music between the scenes, such as Madonna's 'Material Girl', and Hall and Oates' 'Maneater', have been carefully chosen to underscore the themes of the play. The scene is set with a black and white décor, carefully styled but unnatural, closed off from the colourful world of nature by a pair of French doors. The black and white theme is carried over to the costumes, with one exception: the flamboyant Trissotin appears as a Technicolor Boy George.

Enhancing the witty lines is a lot of delicious stage business . my favourite being the 'musical chairs' on the sofa in the second act. But even the manner in which Chrysale deals with the 'meaty matters' concerning the very down-to-earth Martine and the solving of the Rubik's cube puzzle show the deft hand of a thoughtful director.

There are generally strong performances from the cast, and the few pauses and flubbed lines could be put down to first-night nerves. That the play's rhyming couplets are never intrusive and even sound almost natural attests to the competence with which they are delivered. I am reluctant to single out individual performances, as all the acting was solid, with even the minor characters making striking impact.

I came away thoroughly amused and anxious to dust off the Hits of the 80s album buried at the back of my CD collection.

The play runs until September 8, with no performances on September 2 or September 3. Tickets ($25) are available from the BMDS box office 7 p.m. to 8 p.m. on performance nights.

[/Personal/Lighting] permanent link

2007 Aug 31 - Fri

Open Document Format (ODF) and Microsoft's Open Office XML (MS-OOXML)

The Open Document Format has been a standard for a while. It is amazing that Microsoft did not take part in that standard development effort. Well, no, it's not really amazing. It is well known that Microsoft likes to embrace, extend, and extinguish. It likes lock-in. It likes to be a monopoly. Acknowledging and working with ODF would not force people to buy and use Microsoft Office, which would be a definite profit problem.

As such, Microsoft has been pushing to have it's document format designated as a standard by ECMA. Having their format designated as a standard provides them with marketing visibility. Microsoft got real serious with pushing the format as a standard back last year when the State of Massachusetts wanted to standardize on a format. ODF appeared to be on the verge of winning. Microsoft was not amused by such a goal.

With the ECMA submission, Microsoft thought they could push things through. But with a 6000 page document, it required careful review. Many participating standards bodies have reservations about the standard. Microsoft isn't amused by that. In fact, they see the push back as being so signficant, that they are starting to pay their partners to become members of the various national bodies in order to stack the vote.

I do use Microsoft products, but I'm not really enthralled by their commercial ethical conduct. I really hope Vista bites them in the behind.... even our my boss, the owner of a Microsoft Gold Partner, is not totally thrilled by Vista. But, I digress.

I'm wondering if there is a way for the person in the street to speak back against such heavy handed corporate activity.

To keep people up to date on the standards activity, there are a few related web sites:

• GrokLaw
• NO OOXML: Say NO to Microsoft Office broken standard

[/Personal/Technology] permanent link

2007 Aug 30 - Thu

Cisco Log Decoding

On the Cisco-voip mailing list, there was reference to a couple of sites that provide log decoders for Callmanger and IOS:

• TripleCombo Tool: Triple Combo is a tool to aid people troubleshoot CallManager problems by providing a listed output of SCCP, MGCP, Q931 / H225, H245 messages found in CCM traces, CCAPI/VTSP, Q931 and MGCP debugs in IOS gateway traces and versatile filtering capabilities.
• TranslatorX: TranslatorX allows you to quickly parse through Cisco CallManager trace files and search for Q.931, H.225, SCCP (Skinny), MGCP, or SIP messages.

They aren't necessarily TAC supported, but they may help to weed through and make sense of what would be otherwise painful troubleshooting.

[/Cisco/Callmanager] permanent link

2007 Aug 29 - Wed

Lighting for Moliere's Sisterhood

The last I wrote regarding lighting effects for The Sisterhood was in an article back on Aug 11 where I discussed getting the equipment ready prior to design.

Working with the equipment is a whole other story.

The set really wasn't ready for use by the actors until about Aug 29. All I could really do in the meantime was get most of the basic lights mounted and pointed in the right direction. I lit the stage with cells of three lights, with the three lights each of a different color: amber for a warm wash, a blue for a cold wash, and a lavender for part of the wash with a black actor.

In my previous article, I wrote about obtaining a second USB-DMX box so I could use the sliders on the existing lighting board as input. It did arrive, and I did use it. It was interesting to work with LightFactory to figure out which was the input and which was the output, based upon the flashing activity light on the convertor. I also found that the input channel froze every once in a while (perhaps the light board was set to send too fast). In any case, I found that I really didn't need slider input once I discovered that one can layout channels on a canvas in LightFactory. This turned out to be even better than trying to cross-patch channels in some sort of meaningful layout. By arranging the three color cells along with a no-blue blue downlight in how they lighted the stage, adjusting dimmers became easy. I simply control clicked (for selecting the amber channels) or box-selected dimmers (for the channels in one or more cells), then used the mouse scroll wheel to increase or decrease intensity as desired. I have no desire to use a lighting board after experiencing the ease in which the software allowed me to make changes on the fly.

I did spend a couple long days trying to tame the color changers though. The software made it easy to select colors and intensity, but it got in the way when trying save and retry groups and palettes. I ended up submitting five or six bug reports one morning after figuring out how to work around my frustrations. I was surprised when I received responses back that same day from the vendor to say that they had fixed the bugs. That was excellent turn around. However, that does cut both ways: why did the software have these silly bugs in the first place, but when encountered, they did fix the problems quickly.

I did spend quite a bit of time in the grid for light focussing. The PDA based remote focus software worked well in conjunction with LightFactory. I just wish it had a slider, and a better scroll back buffer. I had to turn off power management on my PDA so it would remain on, otherwise I'd have to restart the remote software as it would lose the connection to LightFactory.

For lighting, there were five major areas: an outside patio up stage right, a hallway with red wall on stage left, a bar down stage right, two sofas up stage center, and main stage area down stage center. I used three color changers in the patio area at various angles to provide various day time color changes and mixes. I used a single color changer in the hallway to cast a purple light over the hallway phone. The bar had a Source Four angled and shuttered to give a hightlight to the liquor bottles as a kind of ornamentation.

For the remainder of the stage, there was one light cell (three colors plus down light) for each of the two sofas, one for the french door entry way, three cells for down stage center, and one for the bar. I ended up having to do three long throw Source Fours to get stage left as regular lights cast a bad shadow into the hallway (only 8 ft walls).

The director decided to highlight certain parts of the stage by dimming cells in other parts of the stage when no activity was being undertaken. Ten and fifteen second fades were used to make the transitions subtle to the audience.

This play was used to come up to speed with the software. The usual excuse, if I'd had the time, I could have done more interesting things with the fades more often. As it was, it was good. A photographer took some pictures of the set as lighted for intermission set decoration. I hope to get some loaded here soon.

[/Personal/Lighting] permanent link

2007 Aug 26 - Sun

HTC P4550 Kaiser TyTN II

It is said that this device is to be released sometime during September, any where from second week to Oct 1, depending upon who you read.

This I think is the device that finally gets it right. The microSD card is externall insertable (on the P3300, you had to remove the battery and SIM card to get one installed). It has GPS built in, has quad band. It has more horsepower, more RAM and more ROM. It runs Microsoft Mobile 6 (supposedly the P3300 is upgradeable to v6 at some time).

I'll keep my eyes open to the following sites to see when it becomes available:

• Phone Source USA
• NetworkingRUs.com
• Mobile Planet: have the GPS antenna available

HTC has the user manual and quick start manuals available for download in their support pages.

[/Personal/Technology] permanent link

2007 Aug 23 - Thu

Open Source Network Empairment Emulator

I have a customer who wants to simluate file replication traffic over a Wan link. I came across NIST Net, which is a network emulation package that runs on Linux. A paper called NIST Net -- A Linux-based Network Emulation Tooldescribes the algorithms behind what it does.

While not necessarily having something to do with network empairment emulation, during my search, I did come across a document that offers up how to use IP Accounting in Cisco IOS devices. It may be a good supplement to NetFlow based packet accounting. Netflow will provide both port and address information, while IP Accounting only provides address information, and sometimes mac information.

[/OpenSource] permanent link

Open Source Site of the Day -- Some Sites Referencing SNMP Sites

• Simple Web
• SLAC: Network Monitoring Tools
• Free Software Directory: not necessarily SNMP, but free all the same

[/OpenSource/SiteOfTheDay/D200708] permanent link

2007 Aug 17 - Fri

Genetic Programming Tool: Open BEAGLE

On the Yahoo Tech Group, Genetic Programming, is a reference to a Genetic Programming tool based upon C++, called Open BEAGLE, which stands for "Beagle Engine is an Advanced Genetic Learning Environment.

It is hosted on SourceForge.

In addition, it has a Yahoo Group. Although it appears to be based upon a Linux platform, I see someone is maintaining the ability for running in Microsoft Visual Studio 2005.

[/OpenSource/Programming] permanent link

2007 Aug 16 - Thu

Asterisk Cisco CallManager Voicemail Integration
Asterisk Cisco CallManager Voicemail Integration

[/Cisco/Callmanager] permanent link

2007 Aug 14 - Tue

Recording Calls in a Cisco Environment

In one of today's Cisco listserves, there were a few suggestions for Call Recording software:

• NICE Contact Center & Enterprise Products
• "run Asterisk and use it as your outbound/inbound gateway. Let it open a second connection to the actual destination, and record"
• "base something on pcapsipdump"
• Telrex
• Edigin
• Witness
• Comvurgent: "It works ok, but you have to manually type in the phone number for each call"
• CAllReplay
• Oreka: Oreka is a modular and cross-platform system for recording and retrieval of audio streams. The project currently supports VoIP and sound device based capture. Recordings metadata can be stored in any mainstream database. Retrieval of captured sessions is web based.

[/Cisco/Callmanager] permanent link

LINQ: Language Integrated Query

In the upcoming release of Microsoft's .NET version 3.0, code named ORCA, they have a fewature called Language Integrated Query (LINQ). [Come to think of it, v3.0 is now out, I've been rooted in C++, I'll have to get back to C@ and check it out]. The language extension allows one to write queries for anything with an iterator. Writing inline SQL queries comes immediately to mind. I've also heard that there is something called PLINQ (Parallel LINQ).

Here are a couple of references regarding the subject:

• The LINQ Project
• The Evolution Of LINQ And Its Impact On The Design Of C# (v3.0)

On an unrelated programming note, erlang is said to be a language for developing interacting distributed applications. One interesting capability is Hot Code Replacement (replacing code and data without stopping the system).

[/Personal/SoftwareDevelopment] permanent link

C++ STL (Standard Template Library)

Here is a nifty shortcut to make C++ do what you can do in C#:

public static void ForEach(this IEnumerable ienum, Action func)
{
    foreach (var v in ienum)
        func(v);
}

[/Personal/SoftwareDevelopment] permanent link

Trading Site of the Day -- IVolatility.com: Implied Volatility Data

I would say, based upon the sites I've seen so far, at least from a volatility point of view, IVolatility.com is probably the best for information regarding volatility and its uses. As far as I can tell, they provide data only, no brokerage services. For trading purposes, ThinOrSwim can nicely supply that requirement.

IVolatility provides various and sundry scanners, the best option choice for the day, many different ways to analyze options and their combinations, as well as a knowledge base for honing your skills for any type of market. They also have, what seems to be, a weekly Trading Digest Blog, which comes out on each Monday, for helping to identify the type of market, and the types of trades to do in that market type.

Anyone can by naked calls in a rising market. The true traders ar those who can make the right option combination in a rising market, a falling market, or a sideways market.

I wish I know about this site when I first learned about options.

[/Trading/SiteOfTheDay/D200708] permanent link

2007 Aug 12 - Sun

Trading Site of the Day -- Seeking Alpha: Stock Market Opinion & Analysis

Seeking Alpha is a site containing mostly, if not all, reader contributed articles. It is quite active, even on a Sunday. One get commentary and opinion on the market in general, as well as industries and companies in specific.

One article, in particular, caught my eye today: The 'Plunge Protection Team' Working Overtime: A Play-By-Play. Gary Dorsch writes an article that does a good job of time-relating intra day market gyrations with public comments from the US President, Henry Paulson, as well as a number of other key news releases.

Perhaps by focussing on key news articles during the day, such as real time news feeds by The Fly on the Wall, one can ride and profit the market changes as they occur. That may improve the profitability of those trading the market futures such as YM.

The RSS feeds to the web site are numerous and well categoriized.

One other note to make regarding the Seeking Alpha web site. For those wanting to get noticed in the world as an 'expert' or as a 'blogger' or both, writing regular informative articles for the site would probably be of benefit. Publicity would be free.

[/Trading/SiteOfTheDay/D200708] permanent link

2007 Aug 11 - Sat

The Geek Side of Lighting

Saturday didn't start off so well. I was going to ride my pedal bike into town for the day. That was not to be. On getting on my bike, I find that it had flat. Rather than fixing the flat right then, I put it off till Sunday, and drove my car into town. This turned out to be fortuitous, as the theatre was having 'cleaning day', and I was able to pick up a few pieces of furniture to haul home. However on the way back in, I decided to take the 'scenic route' down to Flatts and back. It was such a nice day for a drive. My car didn't think so. It decided to konk out a little ways back from Flatts on Middle Road. I spent an hour or two sitting in the grass waiting for the tow truck to arrive. In waiting, I noticed that there weren't too many, if any, 20 year Toyota's still around. Most of the drivers had spiffy new cars. So being without transportation made it a bit difficult to make it to a party I supposed to go to.

Instead, much belatedly, I made it back to BMDS and worked on finishing up making DMX cables for the four new High End Color Command lights the G&S Society donated. Making up the cables went well. When it was all said and done, I had the following DMX chain:

• My laptop running LightFactory with a 512 channel license, and the High End light fixture library
• An Enttec DMX USB Pro adaptor, to control the DMX chain over USB from my laptop
• Four Color Command static color change light fixtures, with DMX offset 73
• A four channel dimmer pack for testing the ColorCommand light intensity control, with DMX offset 49
• the 48 channel house dimmer system, at DMX offset 1

In an earlier article, I made some misguided remark as to why the Color Command offset appeared to be off by one. Upon reading the instructions, I found that the first channel is assigned to the Color Control box itself. Sets of four channels after that are each assigned to the Color Command lights: Cyan, Yellow, Magenta, and beam width. The colors turn out to be bright, intense and beautiful.

I found that even though the DMX USB Pro has a DMX in, it doesn't segregate the in-stream from the out-stream, it is simply pass through device. Therefore I couldn't connect up our ETC Express 72/144 Light Board and use it as a physical submaster board input to the software. I've ordered another DMX USB Pro to resolve that little issue: one will be 'in', one will be DMX control 'out'.

My laptop is connected to the internet through a wireless access point. My PDA has 802.11 wireless capbility. The LightFactory software has a remote telnet capability with software that can be loaded on my Windows Mobile 5 PDA. The combination makes for a time saving and vocal chord saving ability to test light focus right from within the grid.

The easy part is done. Now the fun part of doing the lighting design is next.

[/Personal/Lighting] permanent link

John H. Holland

Programs that learn, changing the course of calculation as the model accumulates experience, are rare after almost half a century of endeavor. We still have little theory to guide us and few implementations. . John H. Holland

[/Personal/TagLines] permanent link

2007 Aug 10 - Fri

The Sisterhood ("Les Femmes Savantes")

BMDS has a new production opening up on August 30, 2007. We have another imported directory for this one, Suzann McLean. The Sisterhood is a "farce about cultural elitism and the triumph of true love over snobbery. One of Molière.s most popular comedies." The play has been modernized and is set in a chic 1980's salon. I'm assisting Doug Parker on this one. We had our first production meeting with the director earlier this month.

The set, for the internal areas, is blacks and whites. The outer courtyard will have more color in it. As of tonight, set construction has proceeded to the point where most of the basic wall structure is place. There is still some question as to how high to make the walls, a standard 8 feet, or would a total of 10 feet be better? Producer Jo Shane is working her way through that question.

From a lighting perspective, after taking a quick glance through the script, I thought it might be another simple lights-on/lights-off type of scenario. But after some clarification from the director that the story takes place over the course of the day, we find we can have some fun with light angles and colors to simulate a moving sun. In addition, the interior set has a three or four primary areas: a bar area, patio doors, a couch area, and a hallway with telephone. The director has called for subtle light changes to highlight the action areas as they occur.

Earlier in the year, the Gilbert and Sullivan society donated a new light board and four High End lights. We were able to use the lighting board for the Famous for 15 production. As we were missing cables and connections for the lights, we couldn't use the new lights then. They would have proven useful for the night club play.

I did order some DMX connectors from SIRS Electronics in McAllen Texas. I ordered through their web page. The order was handled quickly and was complete on arrival. The color lights have a controller box, and are connected to it through four pin DMX connectors. The box is connected to the light board with a five pin DMX connector. After some research on Beldin's website as to an appropriate cable to use with the connectors, I found that standard Category 5 network cable would fit the bill. There is ample supply of that type available to us.

After work this evening, Doug Parker and I met at the theatre to try out the colored lights. Mary Brier stopped by for a bit to see how things were going. Doug did the plugs for the light electricals, and I soldored some test signal cables together so we could see how the things work. After suffering through a mild blonde moment where we couldn't get the light to come on, we realized the grandmaster wasn't up (ok, that did prove my soldoring job first time through was good). Whew, simple problem number 1 down. The second problem took a bit longer to figure out. With no manuals to work from, we had to go by guess and by golly. I imagine the light manufacturer had the same question often enough, such that they printed the directions on the control box. It took a bit of fiddling, but we found an auto mode in which the control box found its light. After that, it was smooth sailing. One thing to remember is that the sliders on the light board are numbered starting from 1, and DMX channels are numbered starting at 0, which I think is why when we progammed a DMX offset of 73 into the controllor, we had slider 74 as the first active slider.

Saturday we go back in to make the remaining control cables and test out the three remaining color lights.

As part of the order with SIRS, I obtained a copy of Light Factory. Light Factory has a downloadable fixture library for the High End lights we have. Perhaps we can try out the new lights as well as the new software for the upcoming production.

[/Personal/Lighting] permanent link

2007 Aug 09 - Thu

Lighting Assistant at Bermuda Musical & Dramatic Society

Since the beginning of the year, I've been helping out with Lighting Design with the Bermuda Musical & Dramatic Society at the Daylesford Cinema on Dundonald Street in Hamilton. The first production was a straight forward, almost as simple as white lights on-white lights off, lighting requirement for Sordid Lives (A Black Comedy about White Trash). The lead lighting designer, Mary Brier, came down with pnemonia just prior to opening night. As such, I 'ran' the lights for the 10 nights the show was open. No big deal. Press the 'go cue' button when called to do so by the stage manager. Stage manger was Nicola Wilkinson, who was great to work with.

Next show up was a musical by the name of The Mystery of Edwin Drood. We had an off island director by the name of Vivienne Elborne. For lights, Doug Parker, the lighting designer on this one, took a different approach to lighting. For primary lighting, he ran with a number of clusters of three lights each. Each cluster had red, green and blue filters. As such, any color could be chosen for lighting the covered area, including white. Because three separate lights were used, we had some interesting colored shadows as artifacts. At first I thought it was a bad thing, but after a while, It seemed appropriate for how the stage had been designed. We did use some fill in whites as well as some specials. One of the specials was a white light shining out of the crypt, with a smoke generator running, as a cue to Drood to come out. It looked quite good.

Mary Brier was lead designer for the next production, Famous for 15 Minutes. This is a production with six plays, each being fifteen minutes long. Lighting this production taxed the inventory of lights and lighting channels. One of the more challenging plays of the sequence was Bermuda Triangle, which is set on a sailboat about to encounter a hurricane. This one required careful timing of lighting (lightning) and sound (thunder) queues. As the storm approached, lightning and thunder became stronger and closer together. The play that did win the Golden Inkpot was Grass is Greener. To simulate a light club we did a simple two light special by shining an emerald and a purple spot on the back wall.

[/Personal/Lighting] permanent link

Trading Site of the Day -- QWAFAFEW: A four letter word for those who can't count?

Really, that means: Quantitative Work Alliance for Applied Finance, Education, and Wisdom. Quite a mouthful. Anyway, this appears to be a current and running site for quants in various stages of the game. Regular meetings are held, and the site is chock full of presentations regarding Hedge Funds, Regression, Portfolio Construction, Simulations, etc.

Plus it has a random quote section, one of which I found rather pointed:

When asked what it was like to set about proving something, the mathematician likened proving a theorem to seeing the peak of a mountain and trying to climb to the top. One establishes a base camp and begins scaling the mountain's sheer face, encountering obstacles at every turn, often retracing one's steps and struggling every foot of the journey. Finally when the top is reached, one stands examining the peak, taking in the view of the surrounding country side and then noting the automobile road up the other side! . Robert J. Kleinhenz

Here is a self description of the group:

QWAFAFEW is an informal organization of quantitatively oriented professionals in various aspects of financial services (primarily investment management). The group was formed ... to provide a venue for quantitative researchers to discuss their evolving work with peers. ... The members span the gamut from owners and senior executives of investment related organizations to recent entrants to the industry.

Another good quote. I wonder where they get their database from.

A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it. . Max Planck

On another track, they have a link to a recent New Yorker article, called Hedge Clipping, where they have an interview with an ex-analyst, now academic, Harry Kat, who has developed some software to emulate trading styles of hedge funds that he could find that had published data about their returns and asset type usage. He charges clients one third of one percent of the moeny they invest using the software. The interesting thing I learned in the article is that it is typical for fund managers to make 2 percent of the value invested, plus twenty percent of any profits that the fund generates. A nice little return, for the fund manager. Just so long as the fund generates significant enough returns to cover the percentages.

[/Trading/SiteOfTheDay/D200708] permanent link

2007 Aug 07 - Tue

Trading Site of the Day -- Market Delta: See Inside the Chart

I am now, quite simply, amazed. I knew what I was for and was thinking that it wasn't truly original, but I had not been able to find it. Until now. Of course, it was already there, but I just didn't know what to look for.

In a previous post, I discussed the sequence of events I encountered in realizing the power of the Market Profile, and that it will yield good information about a symbol's meanderings. Market Profile is mostly about price brackets over the course of a period of time, which is typicall a day's duration broken down into 30 minute intervals.

But knowing that volume has much to do with interpreting market activity, I knew that Market Profile needed that 'something extra'.

Market Delta takes the Market Profile concepts, adds volume information at the bid/ask/trade perspective, and provides versatile and flexible mechanisms for viewing the data in any volume related market (which excludes straight forex, but includes forex futures). Market Delta demonstrates dramatically the concepts that I knew to be possible.

Market Delta's web site has excellent training and tutorial materials. There are video presentations as well.

Market Delta works with feeds from Interactive Brokers and IQFeed, as well as others. One of the others is a broker called CQG, one that I have not heard of before, and gets a variety of reviews at www.elitetrader.

In an article called Market Profile: A Best Practice in Trading, Brett Steenbarger makes a good point that is useful in overall trading that I like to keep in mind: "be aware of value areas at one level larger than the timeframe being traded". In an Amazon review of the "Markets in Profile" book, he further enhances this by quoting from the book: "If you can correctly identify which timeframe is in control of market activity, and you have a good understanding of how the individual timeframes generally behave, then you are in a stronger position to trade, invest, and effectively control risk." I'd like to add that the same holds true for keeping track of what occured in previous time frames. Points of Control (POC), which are price points at which highest volume occurred, are good sources of support and resistance, and are an example of points of information available from previous and larger time frames.

He also makes reference to a site called WINdoTRADEr. The page to which I have linked has some interesting screen captures of the market profile in action and how previous time frames relate to following time frames.

As a side note, in a previous message, I introduced 'proprietary trading firms'. Brett says "the best strategy for finding solid proprietary trading firms is to go to the futures exchanges themselves and obtain a list of their clearing members. You'll find major prop firms within such a list. The CME education dept maintains a list; they're quite helpful--".

As another side note, of marginal interest, to make something painfully obvious, some traders build bars from ticks, rather time. And in building tick bars, they will use tick counts of 233, 37, 610, etc. And until I looked it them up, I didn't realize those are actually Fibonacci number series elements.

Anyway, to wrap this up, Market Profile and Market Delta appear to have the concepts for which I've been looking for: showing how the markets move, and offering insight into making the best use of the information relating to bid/ask/price and volume, at each price level.

[/Trading/SiteOfTheDay/D200708] permanent link

GenAlgo: Genetic Algorithms

I found a link to GenAlgo.com through a link at the Google group comp.ai. They advertised themselves as a site for people to publish their 'rejected papers'. I suppose one could be concerned about the quality, but I think there is enough substance there that it is a decent site with content relating to:

• Genetic Programming
• Artificial Intelligence (mostly fuzzy logic)
• Data Mining
• Neural Networks
• JAIR Articles
• a lightly loaded forum

Based upon the date of publication of their first newsletter, it looks like the site has been around since Aug 2006.

Probably 'THE' bibliography for related papers would be The Collection of Computer Science Bibliographies, where the Bibliography on Genetic Programming shows 5748 references at the time of this writing.

[/Personal/SoftwareDevelopment] permanent link

2007 Aug 05 - Sun

Trading Site of the Day -- Trader's Narrative

I was pointed to Trader's Narrative through a link at Elite Trader via a thread regarding margins and haircuts (the two being the same thing). This link points to a list of proprietary trading firms, where Genesis Securities is listed as one. I guess I have an exam to write to get a better haircut!

It seems that once a person has passed what is known as the Series 7 exam (a 6 hour, 250 question General Securities Representative Examination), one can join a proprietary trading firm and enjoy margins in the 20-1 region. Regular retail traders get 4-1 during the day and 2-1 over-night. There are a number of preparation books listed at Amazon. One poster suggested Securities Training Corporation.

Trader's Narrative blog also has an entry regarding the use of the McClellan Summation Index as it relates to the Nasdaq and NYSE markets. According to that index, the markets can fall some more and still be above previous declines.

A couple of links later, I found myself at VIX and More and a summary page on the McClellan Summation Index, which is basically an advance/decline tool. VIX and More should probably be on a Trading Site of the Day entry by itself. Anyway, from there, I got to where there is a description of the actual calculation at the McClellan Financial Publications.

[/Trading/SiteOfTheDay/D200708] permanent link

Shopping by Night Owls

I'm not sure who dreamed this one up, here is a good one:

While shopping on the Internet has always been a 24-hour ordeal, some retailers are starting to realize the potential of offering online-only discounts when their stores are closed. Traditional stores like Sears (SHLD), Kohl's (KSS) and Dick's Sporting Goods (DKS) are offering online-only discounts between midnight and dawn. [Reference Link]:[http://www.chicagotribune.com]

Do they want people not to go to their stores?

[/Personal/Business] permanent link

Trading System Design Thoughts: Price - Volume - Time

I spent a couple of years using SmartQuant's QuantDeveloper (now owned by QuantHouse) to evaluate the viability of various technical analysis based trading systems. I had great success with tweaking simulations to make in-band solutions work, but when it came to using the developed scenario for out-of-band data, the attempted solutions became woefully inadequate.

In reading various books and blogs, I could see that people, when trading using traditional technical analysis tools, would spend much of their time on the look-out for new stocks with a potential for a large trend, whether the trend be up or down. I asked my self why does one need to jump from stock to stock to find trades? In effect, those traders are looking for directional volatility. As a corrollory, it would appear that they are unable to make money when markets go sideways (ie, don't trend one way or the other).

The people looking for trends will always have market scanners running in order to find the 'hot stock' for the day. Depending upon the sensitiviity of the scanner, much of the trend to be found could have already been run, with very little left to go. One really needs to be in on the ground floor, but those opportunities are few and far between.

When looking through Amazon book lists for traders, all one really sees are books based upon chart analysis, technical indicators, and stock selection. During my initial research into trading, I did in fact obtain a number of those books. But as already mentioned, I became dissillusioned with what they had to say. I couldn't really put my finger on the answer to the question of why. Some good, solid, statistically validated answers became apparent once I obtained "Evidence Based Technical Analysis" by David Aronson. He basically proved what I had finally learned: a lot of published techniques are only so many words on paper. (I think I ranted about this once before, come to think of it).

While looking at equity trading, I also did a bunch of research into options trading. Good options traders know all about volatility, and how to make use of volatility in selecting an appropriate options trading strategy. Because of the wide variety of options strategies, and my inexperience with making money in this realm, I decided to back off of options, and move back to equities.

As a side note, it is interesting to note that the authors (Chacko, Jurek, and Stafford) of a paper entitled "The Price of Immediacy", in a recent issue of Journal of Finance, "show that limit orders are American options", which is a nice segue into equities. (The article number is 4458.pdf).

During the transition back to equities, I came across J. Welles Wilder Jr.'s book called "New Concepts in Technical Trading Systems". He appears to be the one who introduced the Average True Range, which is a mechanism for measuring volatility.

With a better understanding of volatility, I set out to use this knowledge in trading equities. I created a stock screener to use end of day data to find equities with good daily volatility. From an absolute volatility perspective, GOOG always landed on the top of the selection list. But one needs to be well financed to trade there as it is currently in the $500 range. ICE turned out to be a good runner up with it being in the $150 range with good daily liquidity.

I havn't assimiliated all it's nuances yet, but Joseph E. Murphy, Jr.'s book "Stock Market Probability" has much to say on statistics and probability as it relates to stock movement. Although it covers mostly long term trading, it may be useful for intraday movements.

Content of "Bollinger on Bollinger Bands" by John Bollinger assisted much in terms of understanding and measuring volatility.

In relation to Bollinger Bands, I developed a peak detection tool to determine how often an equity changes trend direction in any given day. The relationship is that peaks will typcially relate to Bollinger Band edges, and point out new edges, so to speak. Since the peak detection tool provides peak determination in a real-time delayed fashion (yes, I know I could explain that better, but it sounded more interesting that way), it can't be used directly as a trading tool, but it does yield some interesting statistics in terms of average peak-to-trough runs and their average duration. On a volatile equity, one gets lots of peaks, some bigger than others. I've found that I should be able to focus on one or two stocks regularily, and begin to learn it's idiocyncracies, and as a result trade it profitably, even though it may, from a daily chart reader's perspective, be going sideways. It may be trading side ways over a period of days, but it will have lots of intra-day ups and downs.

This, in effect, is what Market Makers do: act as sources of liquidity to traders. They play the market on both sides simultaneously. They enter the market at the beginning of the day directionless, and attempt to end the day directionaless, that is either with no portfolio, or with a portfolio with evenly matched short and longs. In Option Maker's parlance, this is called ending the day with a zero delta.

You'd think that a book by the title of "The Market Maker's Edge", which in this case is written by Josh Lukeman, provide some details about market making and how to trade in that manner. Instead, the book has a decidedly technical analysis bent, with not enough on the higher frequecy perspective on trading. "The Nasdaq Trader's Toolkit" by M. Rogan LaBier does a much better job of introducing one to Level II data, and what is happening on the markets. But the book dates itself through screen shots using fractions rather than the current decimalized system.

As a book not necessarily devoted to Level II analysis, I did find "Mastering the Trade" by John F. Carter to be extremely helpful in finding out about various market relationships, including what to look for before the market opens. It also suggested ways to make use of the trin and tick indicators while the market is open. The book "The ARMS Index (TRIN)" by Richard W. Arms, Jr. provides much background on how this works, and is a very useful tool for helping in determining short term (intra day) market movement.

So, after having said all that, I've come to realize that 'it' is really all about short term (intra day) market movement. Can one make money from all the gyrations of the market? It comes down to statistics and probability: how often are trades within a range and how often and when do they do a range extension?

It comes down to evaluating price, volume, and time.

In using Interactive Brokers Trader Workstation interface, in particular with the BookTrader interface (otherwise known as the ladder interface), one can see the latest price, bid, and ask. When subscribed to Level II, the content of the Limit Order Book is also available. By clicking on the bid or ask column at a price level, one can quickly place Limit Order bids and asks in order to bracket price movement. As price moves, the Profit/Loss of the cumulative position is updated in another column. In addition, a tick histogram is available for determining popular price levels. I find the book trader easier to work with rather than the traditional side by side bid/ask Limit Order book.

About the time I found out how that works, and how effective it is for active trading, I came across a few threads in Elite Trader which discussed this as a 'Non Linear Trading' method. One contributor explained how he used two accounts to work both sides (the buy side and the sell side) of the market at once.

Since IB isn't/wasn't all that much into customer service or special requests, I scouted elsewhere for a broker who would be willing to set something like this up. Genesis Trading turned out to be easy to work with in this regard. They were able to set me up with two trading accounts that draw off the same fund account. The only drawback with them is that they are mostly equities, they don't do the miniDow (YM), which I've been paying attention to in one fashion or another recently.

As Genesis doesn't seem to offer the equivalent of IB's BookTrader for monitoring Price - Volume - Time, I did a quick prototype in SmartQuant's QuantDeveloper. Unfortuneately, Genesis' API is somewhat lean when tied to a .NET framework. Gensis, instead, has a robust C++ framework. And since I found the .NET libraries a bit slow, I'm currently involved in rewriting my prototype in Microsoft VC++ 2007. It 'feels' faster, and 'closer to the metal'. C# is good for building systems quickly, but one loses the feeling of 'getting dirty' when working with it.

During trial runs on the C# version, I found I was getting caught up in following the tick rather than keeping track of the big picture in order to bracket trade ranges and follow range extensions. I found I needed to see the 'forest for the trees'.

My Peak Detection module was supposed to help with that, but not as much as I hoped. I came across a technique known as the Market Profile. The Market Profile breaks a day into 30 minute slices. The trading range in each time slice is marked with a letter of the alphabet and then 'draped' over the predecessor time frames. This allows one to find where most of the market action is occuring. By bracketing the 70% range, it should be possible to pick up a bunch of good trades with relatively little effort.

There are two recent books, both by Dalton/Jones/Dalton. The older one is "Mind over Markets" and should be viewed first, as it introduces the concept. The newer, recently released one is "Markets in Profile", which builds further on the theory. My plan is build and process Market Profiles in real time so as to maintain a 'big picture' view of the trading day.

There are also significant online resources for Market Profile. Much of the initial research was performed by J. Peter Steidlmayer while at the Chicago Board of Trade. The CBOT has a good Market Profile resource area including a free downloadable handbook in the educational resources area. Cisco Futures has a tutorial on Value Based Power Trading, which shares some of the material from the CBOT manual. The tutorial can also be downloaded as a .pdf. They have more links at Value Based Trading Research page.

In one of these references, I came across a remark to the effect that people were having a problem with using the Market Profile for building multiple day strategies. Given that market research indicates that any day is a 50%/50% chance of going up or down, I can see why people would have this problem. I think this is another reason to not try holding multi-day positions. Each day should be treated separately. This becomes readily apparent when doing end of day recaps, and realizing that each day moved due to some different market stimulous.

At the CBOT site, there are two good introductory articles by Jack Broz: Trade by the Book - A Guide to Reading Order Flow and Reading Order Flow. The first uses the Limit Book side by side format, while the second shows the ladder format.

The ladder format is used by many trader applications, Ninja Trader and Button Trader are ones that come to mind immediately. However, by the look of them, they don't appear to handle two simultaneous trading accounts. Hence, my motivation for coming up with my own application.

Which brings me to the present. My trading software is almost tradable, as in I'll be able to place and cancel Bid/Ask limit orders in a ladder format quite soon. There is a bunch of supporting infrastructure to implement, but the hard bit has mostly been accomplished. I hope to provide a screen capture of it in operation soon.

The goal of TradeFrame, the name I've given the software, is to provide good perspectives on price - volume - time. At each price level, accumulated volumes and ticks are presented. It is able to provide limit order book depth. And through auxilliary charts, it will provide market statistics such as tick and trin.

Then, as time goes by, I hope to try adding in semi-automation. The ultimate goal will be to fully automate the process, but can only be done once I've got a good handle on the manual process.

[/Trading/AutomatedTrading] permanent link

2007 Aug 04 - Sat

Trading Site of the Day -- Futures & Options Trader: A Magazine for Free

Who says the best things in life aren't free. I'd rank the monthly Futures & Options Trader magazine as good bang for the buck. By signing up, you get a monthly email with a link to download a pdf file full of the type of content you'd see in regular glossy magazine. For free.

This month's issue is 52 pages in length and has articles relating to the Gold market intra day performance, Lumber futures as they relate to weather, Put-Call parity analysis, Bull Call ladders, technical analysis with the Price Movement Index, plus a number of additional articles and regular features.

[/Trading/SiteOfTheDay/D200708] permanent link

Open Source Site of the Day -- Xen: OS Virtualization

VMWare is probably the pre-eminent hypervisor. Microsoft is trying to catch up with their VM. At one point, you couldnt' get Linux based guests to run (something that has now changed due to popular demand).

On the Open Source front, Xen appears to be doing well. And through contributions from both Intel and AMD, unmodified guest operating systems such as MS Windows can be run within a Xen environment. (Intel VT or AMD-V microprocessor capabilities required).

From an installation point of view, .bootstrap has an article Installing Xen on Debian Etch 4.0. Russell Coker has a couple of how-to's on installing Xen, to which I'll just reference through his Popular Posts page.

[/OpenSource/SiteOfTheDay/D200708] permanent link

Open Source Site of the Day -- PowerTOP: Saving Power with Linux on Intel Platforms

Intel spent some time analyzing the causes of power usage of Linux applications on laptops. To help users maximize laptop battery life, Intel came up with a helpful utility called PowerTOP.

Based upon what I've read, certain USB components and products are the worst contributors to bad battery life. Slashdot readers have various and sundry things to say about battery performance in general. Matthew has things to say about it in particular, especially about USB stuff not conforming to published specifications.

[/OpenSource/SiteOfTheDay/D200708] permanent link

2007 Aug 03 - Fri

Personal Co-location Registry

Paul Vixie hosts a Personal Co-location Registry. If you have a personal 1U server running, say, a trading program or some such, then looking for place for it could be as easy as looking at the site.

For trading action, where you're not trading quite enough to colo right in a market data source or broker, then setting up somewhere close to the action might be sufficient. I ended up working for a week near Wall Street last month and was walking down Broadway and passed by what is known as the Cunard Line Building, just up from the photogenic New York / Wall Street Bull. I didn't realize the significance at the time, but later found out that Telehouse operates a hosting facility on one of the floors of the building. Look for companies with rack space there if you want to get geographically real close to the action.

[/Trading/AutomatedTrading] permanent link

EDDY: End-to-end Diagnostic DiscoverY

Carnegie Mellon has a development project called EDDY which is touted for "... allowing information about the operation of disparate components in the computing infrastructure to be brought together for analysis, research, and audit. This enables a system manager to more easily pinpoint problems as they occur, allows autonomic processes to assist in prediction, management, and maintenance."

In reading through the notes, it has modules for handling SNMP, Syslog, and event logs at the minimum. There is mention of many other modules as well.

[/OpenSource] permanent link

Trading Site of the Day -- Currency Trading: Under-hyped, for once

This is site is 'different'. I can't put my finger on it. I'm sure it has corporate backing of some sort, but that could be and probably is totally irrelevant. The definite good thing about CurrencyTrading.net is that it has a good intro reference library to currency trading. It discusses such things as the fundamental pip, how to spot forex scams, and how to choose a broker and open an account. The site is a quick read.

The caveat is that choosing a successful trading strategy is up to the reader, which when doing currency, equities, options, derivatives, or whatever, is the hard part.

[/Trading/SiteOfTheDay/D200708] permanent link

Am I Smarter Than I Really Am?

That cuts two ways according to a paper published in 1999 by Justin Kruger and David Dunning of Cornell University. In Unskilled and Unware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments, they say that incompetent people rank themselves as more competent than they really are, and that competent people rank themselves as less then they really are.

Incompetent people may stay that way due to the fact that we've been trained to "don't say anything if you don't have something nice to say".

I havn't taken the training myself, but supposedly the Arbinger Institute's 'Thinking Outside the Box' training is supposed to promote the ability for people who work together to mutually lift themselves out of this mental block through their own bootstraps.

I'm unable draw a direct link between that and this, but it comes to mind: once interstellar space travel becomes available, and should the top 1% of the population, intellectually wise, decide to emigrate, that would theoretically drop the average intelligence level of the population by more than 1%. Could be a scary thought. Does anyone have references to this real or imagined statistic?

Anyway, coming back to the article, or rather the end of the article. The authors provide a very well done ironic summary of their paper (are their results to be believed?):

In sum, we present this article as an exploration into why people tend to hold overly optimistic and miscalibrated views about themselves. We propose that those with limited knowledge in a domain suffer a dual burden: Not only do they reach mistaken conclusions and make regrettable errors, but their incompetence robs them of the ability to realize it. Although we feel we have done a competent job in making a strong case for this analysis, studying it empirically, and drawing out relevant implications, our thesis leaves us with one haunting worry that we cannot vanquish. That worry is that this article may contain faulty logic, methodological errors, or poor communication. Let us assure our readers that to the extent this article is imperfect, it is not a sin we have committed knowingly.

[/Personal] permanent link

2007 Aug 02 - Thu

botHunter

On today's NANOG list, Marc Sachs of SRI made the following announcement regarding a BotHunter pacakge:

SRI and Georgia Tech have been working on a pretty cool new tool that will quickly locate bot traffic inside a network. A government/military version of this software has been in use successfully for about a month, and a public version was made available this week. BotHunter introduces a new kind of passive network perimeter monitoring scheme, designed to recognize the intrusion and coordination dialog that occurs during a successful malware infection. It employs a novel dialog-based correlation engine (patent pending), which recognizes the communication patterns of malware-infected computers within your network perimeter. BotHunter is available for download at http://www.cyber-ta.org/BotHunter/ and runs under Linux Fedora, SuSE, and Debian distributions.

There is also a highly interactive honeynet using BotHunter run by SRI you should look at. The URL is http://www.cyber-ta.org/releases/malware-analysis/public/. We are detecting dozens of new infections each day and this site is very helpful in understanding the behavior of the received malware. Also, it generates a nice list of potentially evil IP addresses and DNS queries.

For both the BotHunter software and the honeynet we'd appreciate any feedback on ways to improve them. Contact details are in the download package and on the website.

[/OpenSource] permanent link

2007 Jul 31 - Tue

Trading Site of the Day -- MoneyScience

MoneyScience, according to the footer on the website, is 'an open access resource for academics and practitioners working in quantitative finance and related fields'.

I should have listed this site earlier as I've used it to get introduced to sites I've already added to my Trading Site of the Day lists. It appears to be supported through private funding from a number of well known financial institutions.

The site has a number of primary groupings: Events calendar, Recruitment, Recently Posted (blog articles from other sites), Products and Services, Latest Sites (links to articles, .pdf's, and such), and has a forum.

The site format has improved over the time the owner has had it up, but could use a few more spruce-ups: the primary RSS feed title could point back to the main site, rather than the aggregator's site. I'm also wondering if things are slowing down as the Featured Event is one from last month.

Anyway, overall, the site as it gets updated, is a good link for quant minded individuals looking for a hopping off point to related research.

[/Trading/SiteOfTheDay/D200707] permanent link

2007 Jul 29 - Sun

Internet Based 911

For those of you setting up IP based telephone solutions and are needing to set up E911 connections, there are a number of ways to do this.

The typical way to do this is to ensure that analog or digital lines are available at each location being serviced. Then be sure to route all calls originating from those specific geographical areas are sent out the nearest POTS (Plain Old Telephone Service) line, ie, analog or digital line.

If this isn't possible, for one reason or another, there is an alternate way of doing this. There are internet based E911 services available. According to Robert Kulagows, you "setup something like a SIP trunk and route e911 calls to them. You'd tell them which DIDs are located in which city, and allow them to handle the 911 call and send it to the correct PSAP."

One provider who can do this is 9-1-1 Enable. According to their site, they are in the Pulver 100 and the Internet Telephony 2006 Product of the Year.

[/Cisco/Callmanager] permanent link

2007 Jul 28 - Sat

Manipulating Perceptual Focus in Advertisments

At Science Daily is an article about how to arrange advertised items so as to make the targeted item stand out:

In a new study from the August issue of the Journal of Consumer Research, researchers from Northwestern University demonstrate how advertisements can be manipulated to cause overemphasis of a particular feature and increase the likelihood that a certain product is chosen. Their finding runs contrary to economic models, which assume that choices are based on stable preferences and should not be influenced by the inclusion of inferior options.

"By showing the impact of perceptual focus on consumer preferences, this research demonstrates that in addition to the many overt ways companies can draw attention to products, the visual arrangement of alternatives can also have a significant influence on their relative choice shares," explain Ryan Hamilton, Jiewen Hong, and Alexander Chernev.

In a series of fascinating experiments, the authors show how grouping together options with similar characteristics can emphasize dissimilar options and help them pop-out. For example, consider a comparison of two sofas, A and B. Sofa A has softer cushions; Sofa B is more durable. In a head-to-head comparison, sofa A is preferred by less than half of the survey participants -- 42.3 percent.

However, if sofas A and B are grouped with three other sofas, all of which have a low rating for cushion softness, then preference for sofa A jumps to 77.4 percent. One of these things is not like the others, and that apparently makes it more desirable -- a phenomenon the authors term "perceptual focus effect."

"The research presented in this article has practical implications for manufacturers and retailers in determining the size and composition of their product assortments," conclude the authors. "In particular, when designing product displays, in both print and electronic media, companies need to be aware of the potential impact of the visual characteristics of choice alternatives on consumer preferences."

The researchers also found that how consumers process information can influence how susceptible they are to perceptual focus effect. Those who rely on intuition are more likely to choose a perpetually focal option. In addition, having participants perform an analytical test before making a product choice drained logical reasoning resources and increased the likelihood that the person would choose the perpetually focal option.

Reference: Ryan Hamilton, Jiewen Hong, and Alexander Chernev. "Perceptual Focus Effects in Choice" Journal of Consumer Research: August 2007.

Note: This story has been adapted from a news release issued by University of Chicago.

[/Personal/Business] permanent link

Trading Site of the Day -- QCode

QCode is a bare bones site linking type site for:

• Quant Jobs: appears to be live linked to appropriate references at other sites
• Links to Recent Papers
• Links to a few Quant and option related sites
• There is a forum with some activity
• Probably one of the most interesting is the links to various software libraries, hence the name of the site.

[/Trading/SiteOfTheDay/D200707] permanent link

2007 Jul 24 - Tue

Trading Site of the Day -- Two Journals of Note

These are of an academic nature:

• Journal of Financial Markets
• Journal of Finance

[/Trading/SiteOfTheDay/D200707] permanent link

2007 Jul 17 - Tue

IQFeed Client v1.3 Release

It seams that the client needs to be revised with recent version of the SmartQuant libraries. This one is built with the July 10, 2007 release of libraries. This build may resolve a System.BadImageFormatException. I hope. Here is IQFeed v1.3.

[/Trading/SmartQuant/Articles] permanent link

IQFeed Client Release 1.2

This client set is released with DTN/IQFeed's latest client v4.2.1.4. This release of the IQFeed SmartQuant Provider v1.2 suppresses an exception when it finds an invalid time in the 18th field of level 1 data. It caused DC to crash and burn, or halt and freeze, as the case may be.

There is a report that it has an image exception when used with OpenQuant. I havn't run OpenQuant, so can't confirm. Please let me know of repeatable issues iwth OpenQuant.

[/Trading/SmartQuant/Articles] permanent link

2007 Jul 10 - Tue

SmartQuant QuantDeveloper & DataCenter Release

SmartQuant has released a revision to DataCenter and QuantDeveloper. DataCenter and QuantDeveloper are at the following revision levels:

DataCenter
Version 2.3.2 (10-Jul-2007) 

QuantDeveloper Enterprise Edition
Version 2.6.2 (10-Jul-2007) 

QuantDeveloper Source Code
Version 2.6.2 (10-Jul-2007)
* Recent Versions available through 
  version control 

[/Trading/SmartQuant/Releases] permanent link

2007 Jul 08 - Sun

TransactionSet Code

For what its worth, there was a request for my TransactionSet code at the SmartQuant Forums. This is my description of the code:

I don't really use the ClOrdId per-se for anything other than reporting an order number. Instead a I created a series of transaction objects, with each storing an opening market/limit order and a starting stop order. When I need to exit, the transaction object then takes care of closing out any stops/markets/limits that are still outstanding. I can then calculate profit/loss (which I could have used from already existing QD API elementes), but I go one step further and maintain other statistics such as time-to-fill, time-in-trade, ... More than anything, the transaction object has helped simplify/encapsulate the trading state-machine for tracking unfilled limits and stops. And it does it smoothly by making use of the two events I mentioned earlier. Then from a higher level perspective, on a market direction change, I can simply tell the transaction to 'exit' and it does what ever clean up is necessary

The code is located in the file TransactionSet.cs.

[/Trading/SmartQuant/Articles] permanent link

2007 Jun 29 - Fri

One Unified Address Management, v1.3

I've updated the schema of my prototype address management database to reflect some additional relationships with VRFs and VLANs.

The address management schema is for a PostgreSQL 8.1 database.

Once I have the address tables populated, I'll want to generate zone files for DNS to facilitate informative traceroutes. There were a couple of useful suggestions on the Nanog mailing lists, one by Mark Tinka, and the other one by Joe Abley, the same person who did a paper on toolmaking at Nanog 26.

[/OpenSource/Debian/AddressManagement] permanent link

2007 Jun 27 - Wed

Network Management Processes

In another exerpt from Cisco's book called Top-Down Network Design, Second Edition, there is a discussion regarding network mangement as being a part of network design. Very few companies, or many engineers for that matter, pay much attention to network management.

"Management is often overlooked during the design of a network because it is considered an operational issue rather than a design issue. However, by considering network management up front, instead of tacking it on at the end of the design process or after the network is already operational, designs will be more scalable and robust."

The The International Organization for Standardization (ISO) defines five types of network management processes (and some examples for each):

• Performance management: Cricket
• Fault management: Syslog, SNMP Traps
• Configuration management: Rancid
• Security management: Radius, Tacacs, Snort
• Accounting management: Radius

[/OpenSource/Debian/Monitoring] permanent link

2007 Jun 25 - Mon

Cisco Device Health

In Cisco's book, Top Down Network Design, one useful show command is 'show buffers'. Some useful SNMP statistics include:

• BusyPer. CPU busy percentage in the last 5-second period.
• AvgBusy1. 1-minute exponentially decayed moving average of the CPU busy percentage.
• AvgBusy5. 5-minute exponentially decayed moving average of the CPU busy percentage.
• LocIfInputQueueDrops. The number of packets dropped because the input queue was full.
• LocIfOutputQueueDrops. The number of packets dropped because the output queue was full.
• LocIfInIgnored. The number of input packets ignored by the interface.
• BufferElMiss. The number of buffer element misses. (You can also check misses for small, medium, big, large, and huge buffer pools.)
• BufferFail. The number of buffer allocation failures.

I've been doing most of my snmp statistics gathering on 5 minutes intervals. On some interfaces, it may be of value to step that up to 1 minute intervals. Of course, if my total collection time is over 1 minute, I may have problems with that.

From the same book, is this interesting statistic about why Window's file transfers over WAN links can go only 'so fast'. SMB acts like a ping-pong protocol. It can only send up to 32KB before requiring an acknowledgement. So if the delay is 50 ms end to end, and ignoring client and server delays, a client can receive at most 32 KB every 100 milliseconds, or 320 KB per second. This means that the maximum throughput is 2.56 Mbps, at best.

[/Cisco] permanent link

2007 Jun 24 - Sun

Perl Tools

In an earlier article, I wrote about an IP Address Management database. To manage the data framework through the web, I'm thinking it might be well to use Catalyst Web Framework, a Perl based Model/View/Controller designed for data based web infrastructures. An Introductory Article introduces Catalyst use along with Ajax.

In another vein, there is an article on how to do Perl Scripts for Perl-Less People from Cygwin. It looks to be as simple as doing:

perlcc -o foo.exe foo.pl

and then running the result to find the dependent dlls.... then packaging the exe and dlls together and shipping them off to the poor soul who needed the solution in the first place.

[/OpenSource] permanent link

Cygwin Customizations

The default shell for Cygwin uses a DOS command shell, which has limited usefulness. To enhance the console experience, here are some customizations. To enhance the shell prompt, use the Cygwin installer to install 'rxvt'.

The icon that the Cygwin installer puts on your desktop has a link to a start up file. I've put the following into my startup file in order to start up rxvt as the default terminal. As you see, I've also installed 'nano' and set it as the default editor.

@echo off
C:
chdir C:\cygwin\bin
set EDITOR=nano
set VISUAL=nano
set CYGWIN=codepage:oem tty binmode title
rxvt -fg green -fn "Lucida Console-12" -geometry 80x50 -sl 1000 -sb -e bash --login -i

The installer may or may not place a few other relevant files in your home drive: .bash_profile, .bashrc, and .inputrc. These can be found in /etc/skel and should be copied to your home directory. They can then be edited to taste.

For copying text in the rxvt terminal window, simply highlight the text while holding the left mouse button. It will be automatically copied to the clipboard once the mouse button is released. To paste the text, hold the shift key while pressing the left mouse button. This is slightly different from putty, which does not require the shift key to be held. To select a range of words you can double click on the first word, then set the mouse cursor to the right of the last word or letter you want copied, and right-click.

Here is a Cygwin Prompt Here method.

Use cygwin to run crontab jobs.

The following can be added to .bashrc to pipe any command to less (so ls):

#A function to pipe any command to less:
function so {
eval "$@" |less -I~
}

A Cygwin FAQ has some useful comments regarding mkpasswd, Octave, enhanced Bash profiles, X, and csh.

[/OpenSource] permanent link

TACACS + Linux

Up till now, I thought that TACACS use was limited to authentication/authorization of logins and commands on Cisco devices. There is an article at Freemode.net regarding TACACS + Linux for authentication.

[/OpenSource/Linux] permanent link

Trading Site of the Day -- Blog on Blogs: The 24/7 Wall St. Twenty-Five Best Financial Blogs

A site called 24/7 Wall Street, which features "Insightful Analysis and Commentary for U.S. and Global Equity Investors", has an article called The 24/7 Wall St. Twenty-Five Best Financial Blogs. These are good starting off points for further research into the daily markets.

[/Trading/SiteOfTheDay/D200706] permanent link

Debian Sample Network Configuration

For my future reference.

######################################################################
# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)
# See the interfaces(5) manpage for information on what options are
# available.
######################################################################

# We always want the loopback interface.
#
auto lo
iface lo inet loopback

# To use dhcp:
#
# auto eth0
# iface eth0 inet dhcp

# An example static IP setup: (broadcast and gateway are optional)
#
# auto eth0
# iface eth0 inet static
#     address 192.168.0.42
#     network 192.168.0.0
#     netmask 255.255.255.0
#     broadcast 192.168.0.255
#     gateway 192.168.0.1

[/OpenSource/Debian] permanent link

Linux Backup Application: Mondo

Mondo, Mindo. To be expounded on shortly.

[/OpenSource] permanent link

Installing XWindows/KDE on Debian

For the initial installation of Debian, I simply do a 'standard' install. No desktop. I may also add database (which is PostgreSQL 8.1), dns, and web server (which is Apache 2.0).

After the reboot, I'll install some basic packages to get KDE running: apt-get install xserver-xorg-core apt-get install x-window-system apt-get install xserver-xfree86 apt-get install xserver-xorg apt-get install x-window-system-core apt-get install kde-core

Running the command 'startx &' will get XWindows going and the KDE personalizer started.

Putting 'startkde' in .xinitrc in my home directory sets the preference for kde. Other possible window managers include gnome, xfce4, icewm, fluxbox, fvwm (loosely ordered by decreasing order of preference).

[/OpenSource/Debian] permanent link

Debian Linux on a Mini-Box

I picked up a small form factor Mini-Box M200-LCD Barebone System. It is a nice small embedded systems unit with a backlit LCD and front panel buttons. I chose a unit with a 1GHz processor, 512MB of memory, and a 40GB harddrive. The unit does not have a floppy drive or a CDROM drive. It does come with ample USB slots. It also has a Compact Flash slot hidden behind a door on the front.

With a little bit of fiddling, I was able to Debian installed and running on the unit. Here is what I did to get it up and running.

The Compact Flash is configured as a Master device on the second IDE channel.

I chose to use a USB Key to boot strap the unit into network mode to get the full Debian distribution installed.

The USB key needs to be created on another Linux system, in this case I used a VMWare'd version on an XP box, . (WinImage was suggested to me, but I could never get it to work according to what little instruction I could find). In VMWare, I needed to associate the USB controller to the VMWare session. Windows saw the USB drive disappear as a result. When I inserted the USB Key into USB slot, the Linux console indicated the drive name to be used. In this case, it was /dev/sdc.

I downloaded the netinst iso image from the Debian package site. I also obtained the boot.img.gz boot image for the distribution. Pascal Le Bail has a web page called Installing Debian Sarge from a USB memory stick (USB key) where he explains the simple process. The instructions have been incorporated into the Debian Installation Manual.

Once the files are available, a short series of commands makes the key bootable:

zcat boot.img.gz >/dev/sdc1

gets the image onto the USB Key.

mount /dev/sdc1 /mnt
cp netinst.iso /mnt

gets the iso installed on to the key.

The USB key can then be plugged into a USB slot on the Mini-Box and the power turned on. In the bios, the secret is to:

• set USB ZIP as the first boot device
• in the USB settings, turn on USB legacy mode

The Mini-Box will boot in to the Debian installer and allow a network download/configuration of a Debian installation.

The interesting part of the Mini-Box is it's LCD/Button front panel. A sample program and SDK have been made available.

There is a demo program called usblcd available to show how things are done. The source and SDK can be downloaded and built.

Assuming Debian's build tools have not been installed, here are the build steps:

apt-get install autoconf
apt-get install gcc-4.1-base
apt-get install gcc-4.1
apt-get install gcc
apt-get install automake
apt-get install binutils
apt-get install binutils-dev
apt-get install libtool
apt-get install libusb-dev
apt-get install libhid-dev
 ./autogen.sh
make
make install

You can then use commands like:

usblcd backlight 1
usblcd backlight 0
usblcd backlight 1
usblcd led 2 1
usblcd led 2 0
usblcd clear

[/OpenSource/Debian] permanent link

2007 Jun 23 - Sat

Run Levels and Mode Switch Keys

Debian Admin has a some information regarding run levels (switching between single-user, console, and graphic modes). These are the basic run levels:

• 0 System Halt
• 1 Single user
• 2 Full multi-user mode (Default)
• 3-5 Same as 2
• 6 System Reboot

The file /etc/inittab has a line in it:

id:2:initdefault:

Supposedly, if you change the 2 to a 3, on boot the system will come up with a console instead of the graphical interface (assuming the graphical interface has been installed).

The command runlevel will return two values, a run level change since powering up, and the current run level.

To change run levels, the command 'telinit x' will change to run level 'x'.

An Introduction to Run-Levels contains further detailed information.

[/OpenSource/Debian] permanent link

2007 Jun 12 - Tue

Trading Site of the Day -- CASTrader Blog: A Quest for 50%-plus Returns

The complete byline for CASTrader Blog is "A quest for 50%-plus returns in the financial markets via Complex Adaptive Systems and Quantitative Finance.".

High aspirations, but looks like he still maintains a day job. But I think there are useful tidbits scattered through the site.

The author of the site appears to be attempting the quant side of trading in the equity markets. Based upon my own evolutions, I'm thinking he may be getting too complicated. But them who am I to say. I'm still getting my own infrastructure together, based upon a simpler volatility trend/range trading idea, something that is going to take another week or two to code. And yep, I too, have a day job.

[/Trading/SiteOfTheDay/D200706] permanent link

2007 Jun 10 - Sun

Trading Site of the Day -- C++ Fast Delegates, plus Boost

No, C++ Fast Delegates aren't really trading algorithms, but they do help make trading algorithms fast.

In the C# world, the C# language has the built-in capacity to store multiple object/method pointers and call them in response to some sort of activity. This is generally referred to the Observer Pattern of event based programming. In C#, the structure is called a Delegate.

C++ does not provide a native Delegate structure. Don Clugston wrote an article called Member Function Pointers and the Fastest Possible C++ Delegates. I've used this method successfully with Microsoft Visual Studio 2005.

Don's implementation is unable to do multicast delegates. Sergey Ryazanov wrote an article called The Impossibly Fast C++ Delegates that supposedly is more C++ compliant and has a multicast capability.

JaeWook Choi took the work one step farther and implemented a multicast delegate that is a drop in replacement for the Boost C++ Libraries. Boost is a collection of free peer-reviewd portable C++ source libraries. JaeWook's library is faster than Boost, but slower than Don's library. But JaeWook's library is mulitcast and does offer up additional functionality. Timings and feature sets can be reviewed in his article called Fast C++ Delegate: Boost.Function 'drop-in' replacement and multicast.

[/Trading/SiteOfTheDay/D200706] permanent link

2007 Jun 09 - Sat

Memory Leak Detection in Visual Studio 2005 C++ Applications

In some recent software development, when working with C++ in Visual Studio 2005, when ending a program which was run in the debugger, I'll sometimes get statements regarding memory leaks.

Each memory leak has a unique number associated with it. If these leaks and their numbers are repeatable, they can be tracked in Visual Studio by using the '_CrtSetBreakAlloc' call in your code.

In your code, near the beginning of execution, place the call, using the leak number as a parameter. This effectively creates a break point on the specific memory allocation call. When your code allocates the memory allocation, the break point will allow you to inspect the code and figure out why an appropriate dispose was not provided.

[/Personal/SoftwareDevelopment] permanent link

2007 Jun 06 - Wed

Writing Callmanager Call Detail Records to Excel Spreadsheet

In an earlier article, I described how to access the Cisco Callmanager CDR tables to obtain billing records through the use of the Sybase DBI drivers for Perl. The database needs to be placed into "mixed mode" for this type of thing to work.

I've created two perl modules to make use of this technique: cdr2file.pm and billing2file.pm.

cdr2file.pm takes as input three parameters: an extension, a starting date, and an ending date. It creates an Excel spreadsheet using a temporary file name, which allows multiple requests to be processed simultaneously. The spreadsheet contains four worksheets: a list of inbound calls, a list of local calls, a list of long distance calls, and a list of internationally placed calls. The selection is based upon four digit extensions and the North American Dialling plan. The duration of each call is included. Because both the 'Original Called' and the 'Final Called' fields are included, one can determine which calls ended up going to voicemail or transferred elsewhere.

The other module, billing2file.pm, takes as input a starting date and an ending date. For all outbound calls placed within that date range, a summary spreadsheet is created. The first worksheet contains a summary of call duration by user with the following fields: Extension Number, Name of the Phone, Total Duration in Seconds, Total Duration in Minutes. Each line contains a hyper link to a worksheet containing the actuall call details for that extension. On the detail worksheet for each user, the following details are provided: Date the call was placed, extension number originating the call, the original called number, the duration in seconds for the call, the phone identifier, and the phone description. This spreadsheet makes it convenient to produce monthly billing reports by user. For large companies, this may not be suitable as a spreadsheet may not be able to handle enough rows or worksheets (a limitation of Excel).

By creating suitable web pages, say through mod_perl or mason, members of the accounting department have direct, easy, live access to the Callmanager Call Detail Records in a ready to use format.

[/Cisco/Callmanager] permanent link

2007 Jun 05 - Tue

Sliding Window Statistics

Processing bars can lag data somewhat. Bars can also introduce sharp changes in statistics. Instead, I prefer to work with Trades and Quotes directly. But I still need the ability to work with summary statistics.

I've created an O(1) running time mechanism for processing trades and quotes in real time. The resulting code is provided in Accumulation.cs. The basic class I use is SimpleAccumulationStatsOnQuotes. The class is instantiated as an object by defining the window width in seconds. The class will maintain a list of quotes. As quotes are added, they are added to the summary statistics. As quotes fall outside the window, the quotes are removed from the summary statistics.

The summary statistics provide the basics for maintaining Bollinger Bands for the selected moving window time frame.

The class AccumulateQuotes is a brute force approach to calculating slope change velocities and acceleration. I think, with suitable modification of the basic statistics, this class would not be needed.

Running statistics are maintained through a reference to the RunningStats class.

If you wish to know the Minimum/Maximum levels over an interval, a RunningMinMax class helps. It is more of a manual process, with the AddPoint and RemovePoint methods.

[/Trading/SmartQuant/Articles] permanent link

2007 Jun 02 - Sat

Tag Line, Found on GrokLaw

Simulation engineers do it with models virtually every day!

[/Personal/TagLines] permanent link

2007 May 31 - Thu

Setting Up and Working With a Remote Subversion Repository

I maintain a number of different servers, and work on a number of different projects. I'd like to put all of these things under some sort of version control. I chose Subversion as it has a good command line environment, has much access flexibility, and will work with Windows and Linux based files. I also wanted to secure the respository on a remote computer. At some time in the future, I may allow limited public access to portions of the repository. For now, I want to restrict access via ssh only. Also, at some later time, I may experiment with WebDav and such (which, if I read this stuff correctly, provides Subversion repository access through Windows Explorer).

Installing the software on Debian for a Subversion Repository is easy:

apt-get install subversion

To create a base repository directory for a number of servers would use a command like:

adduser svn
mkdir /home/svn
su - svn
mkdir -p /var/local/svn/servers

This adds a user named svn, and then uses that user account to create the Subversion directory and respository. This is needed so that appropriate file permissions are maintained for remote access users. I'll describe a technique of access where no additional user accounts are needed for the server in which the repository resides.

To create a repository for the various directories and files for server server01 (still using user svn):

svnadmin create /var/local/svn/servers/server01

Configuring everything for remote use is a more complicated scenario and uses a bunch of concepts I wrote about in my ssh article.

On the server to be put under version control, which, for this example, is server01, create a private/public key with:

ssh-keygen -t rsa -b 2048
mv ~/.ssh/id_rsa.pub ~/.ssh/server01.pub

On the repository computer, with the account svn, ssh to a computer somewhere in order to create the .ssh directory (if it hasn't already been created). Use SCP or a similar capability to get server01's ~/.ssh/server01.pub file into the /home/svn/.ssh directory of the repository computer. Append the file to authorized_keys:

cat server01.pub >> ~/.ssh/authorized_keys

Now edit the authorized_keys and insert the following in front of the line of the key that was just inserted:

command="/usr/bin/svnserve -t --tunnel-user=user1 \
-r /var/local/svn/servers/server01/"\
,no-port-forwarding,no-agent-forwarding,no-X11-forwarding,no-pty

You'll need to take out the line-ending slashes and put everything on one line, ssh doesn't appear to like line continuations in the authorized_keys file. You should have a format like:

command="stuff",sshsettings ssh-rsa onelongkey admin@server01

The '-t' in the command tells svnserve that commands are coming in from an ssh tunnel. The --tunnel-user parameter gives Subversion a username with which it may tag repository changes. The name has nothing to do with any authentication or authorization. As such, it should be changed to reflect an appropriately descriptive name for each public key in the authorized_keys file. The '-r' command provides a 'root' location for the Subversion client to use for new projects and directories. The remaining commands tell ssh to enhance the security of the connection by disabling certain ssh forwarding capabilities.

Back on server01, start up a new Bash session with:

ssh-agent sh -c 'ssh-add < /dev/null && bash'

This loads your private key for automated use in subsequent Subversion interactions.

Now to maintain version history of seleected files in server01's /etc directory:

svn mkdir svn+ssh://svn@svn.example.com/server01/etc
cd /etc
svn co svn+ssh://svn@svn.example.com/server01/etc .
svn status
svn add hosts -m "added hosts to repository"
svn commit

The first line creates an empty directory in the repository. After changing into the /etc directory, the svn diretory is then checked out. By doing a status, you'll see all the files have a '?', as unknown. Files can then be added and committed as needed.

The inspiration for the 'mkdir' command came from the second example in the TLUG Wiki HowTo. The Wiki does make a valid point about not maintaining permissions by default, and does offer up a script that saves these as Subversion properties.

As a side note, for the string 'svn+ssh://svn@svn.example.com/...', when running TortoiseSVN, the svn.example.com can be the name of a saved profile from the Windows based SSH program called Putty.

[/OpenSource/Debian] permanent link

2007 May 30 - Wed

SmartQuant QuantDeveloper & DataCenter Release

SmartQuant has released a revision to DataCenter and QuantDeveloper. DataCenter and QuantDeveloper are at the following revision levels:

DataCenter
Version 2.3.1 (30-May-2007) 

QuantDeveloper Enterprise Edition
Version 2.6.1 (30-May-2007) 

QuantDeveloper Source Code
Version 2.5.1 (23-Mar-2007) 
* Recent Versions available through 
  version control 

[/Trading/SmartQuant/Releases] permanent link

2007 May 29 - Tue

Cricket: Combining Three Graphs Into One

In 2007/05/29's Cricket-User mailing list, there was a useful configuration file snippet for charting and summing selected sources:

target Net
	mtargets ="/Ethernet_To_Building/NSW/52GDC76F02/vlan231;
	/Ethernet_To_Building/ACT/12NOC76FF/vlan201;
		/Ethernet_To_Building/QLD/14VRC76F06/vlan439;
		/Ethernet_To_Building/VIC/O3MLC76F05/vlan436;
		/Ethernet_To_Building/SA/55DRC76FG/vlan467;
		/Ethernet_To_Building/WA/O6SSC76FE/vlan469;"
	mtargets-ops = "sum()"
	short-desc = "desc"
	filename = " Net Aggregate"
	combinationgraph = 1
	unknown-is-zero = true
	order = 899

Courtesy of Naveen Baldeo.

[/OpenSource/Debian/Monitoring/Cricket] permanent link

2007 May 28 - Mon

Opportunities for High Frequency Traders

Here is an article entitled Opportunities for High Frequency Traders: Intraday Patterns in Price Volatility and Liquidity of SFE Contracts by Professor Alex Frino and Grant Wearin of the University of Sydney, Australia in association with the Sydney Futures Exchange.

I've recently put together some scanning software to look for symbols with high daily volatility. This easy to read paper, confirms what I've found out about daily patterns of volatility. In addition, it adds to my knowledge regarding bid/ask spreads in relationship to depth analysis. The paper also discusses the Predictability of Price Movements of SFE Contracts in relationship to the time of day where it might be easier to predict.

An Power Point Presentation by Robert Engle entitled Predicting Returns and Volatilities with Ultra-High Frequency Data offers up some additional confirming evidence of how the markets work when traders are 'in the know'. Here are a few interesting highlights:

• The price impacts, the spreads, the speed of quote revisions, and the volatility all respond to information variables
• Transition is faster when there is information arriving, where an econometric measure of information includes high shares per trade, shor duration between trades, and sustained wide spreads
• Both the realized and the expected duration impact the distribution of the price changes for the data studied
• Transaction rates tend to be lower when the price are falling
• Transaction rates tend to be higher when volatility is higher
• Simulations suggest that the long run price impact of a trade can be very sensitive to the volume but is less sensitive to the spread and the transaction rates

Mark Hooker at Advanced Research Center has an article called Microstructure-Based Predictors. The end of the article has a nice wrap-up:

There is ... a ... benefit from efficient volatility forecasting. It turns out that a good volatility forecast can help us to forecast periods of trending and mean-reversion (or non-trending) in currency returns. For the technical component of our currency management strategy, such forecasts are very valuable since they can provide an early warning of when trending periods are likely to end and therefore allow time to close positions and book profits before the market turns around.

A Google search term for "high frequency volatility trading" works quite well.

[/Trading/AutomatedTrading] permanent link

2007 May 27 - Sun

Trading Site of the Day -- Peter Hoadley's Option Pricing Models and the "Greeks"

Many discussions regarding volatility and options tend to lean towards non-tradable academic talk. Peter Hoadley's page on Option Pricing Models and the "Greeks" has more tradable bent to it. The article starts off by discussing the basic options formula. Further into the article, he discusses how the formula can be used to further trading strategies.

[/Trading/SiteOfTheDay/D200705] permanent link

2007 May 26 - Sat

SSH Server to Server Connections

I wrote an earlier article on how to make it easy to connect to Linux servers with ssh and to copy files.

I need to start doing more version control of my projects. In the past, I used CVS. I'm now converting over to Subversion. I wanted to be able to securely connect to my Subversion servers for updates. Subversion ties nicely into SSH to provide this capability. In order to streamline this connectivity, I need to stream line my ssh connectivity. This article records what I learned about SSH to facilitate this. It has a focus on Linux and Cygwin based operations.

The first step is to run

ssh-keygen -t rsa

This creates a 2048 bit RSA key useful for signing and for encryption. The public key is placed in ~/.ssh/id_rsa.pub with the private key in a file called ~/.ssh/id_rsa. The private key should be protected with a suitably complicated pass phrase. A private key can be created without a passphrase, but security experts frown upon this option, but it does have it's advantages.

It is helpful to rename the public key file to a name that reflects your username and/or current computer. This key can then be copied to other machines and appended to a corresponding ~/.ssh/authorized_keys file. To append the key, you can use

cat id_rsa.pub >> ~/.ssh/authorized_keys

In the authorized_keys file on the destination host, you can prefix a key with a command in order to prevent actions or to automate certain actions. Here are some example ssh key commands.

You can then ssh to the destination host. Upon connection, you'll be asked for your passphrase in order to open the local private key file.

Instead of having to provide the passphrase each time you open a new ssh connection, you can use the ssh-agent program. Here is an example way to start it: 'ssh-agent bash'. The application becomes memory resident and starts a new shell with a couple of needed ssh environment variables. The application 'ssh-add' is used to add your private key to ssh-agent. Use 'ssh-add -l' to check which keys have been added. When you exit the shell, ssh-agent exits also, and closes out the use of the private keys. A quick one-liner to start the agent and add the key (add an alias for this to your ~/.bashrc file):

ssh-agent sh -c 'ssh-add < /dev/null && bash' 

Some authors recommend changing the PermitRootLogin in /etc/ssh/sshd_config to 'no' in order to prevent root logins. To prevent dictionary attacks, I like to set PasswordAuthentication to 'no'. For X11 sessions, X11Forwarding will need to be set to 'yes'.

I found an article that was useful for explaining the difference between RSA and DSA. ssh-keygen typically defaults to RSA, which is a good thing. Security Focus has additional background on SSH Host Key Protection. Secure Shell: Part 1 is more of what I wrote about, but from a Solaris perspective.

[/OpenSource] permanent link

2007 May 25 - Fri

Phishers can use social Web sites as bait to net victims: Informatics study Indiana University (05/24/07)

Personally, I've been able to identify phishing emails as they arrive, and promptly delete them. Indeed, some are quite tempting and realistic, but if one looks at the links closely, the imposters can be separated from the real thing.

If there is ever any confusion, I'll go the source directly, bypassing the link, and type in the correct link directly.

An ACM Newsletter speaks of a study that show that separating the wheat from the chaffe is becoming more difficult. Here is what they say:

Popular social network sites such as Facebook and MySpace are being used by cybercriminals to gather personal information to create targeted phishing attacks, according to Indiana University School of Informatics researchers. In their study, "Social Phishing," the researchers established a baseline for the success rate of traditional and social network-based phishing attacks. Phishers steal personal information by sending authentic looking requests, either by email or instant messaging, asking someone to click on a link and submit their information on what looks like a legitimate Web site. "Phishing has become such a prevalent problem because of its huge profit margins, ease in launching an attack, and the difficulty of identifying and prosecuting those who do it," says associate professor of informatics and computer science Filippo Menczer. "Our study clearly shows that social networks can provide phishers with a wealth of information about unsuspecting victims." The study sent email messages to two groups of students asking them to enter their university ID and password. One group received an email from what they thought was a friend, while the other group received an email from a stranger. Only 16 percent of students who received an email from a stranger entered their information, while 72 percent of those receiving emails from "friends" gave away their information. Associate professor of informatics and member of the research team Markus Jakobsson says they were astonished by the 72 percent response rate. The researchers suggested some countermeasures to prevent phishing, including digital signatures on emails to verify the source, browser toolbars that alert users to spoofing attempts, spam filters that detect spoofed emails, and providing users with a secure path to enter passwords, alerting users that they are trying to authenticate to an unknown site. The study is scheduled to be published in the October 2007 issue of Communications of the ACM.

The full article can be found at the Indiana University.

[/Personal/Technology] permanent link

Color Name Enumeration for C++

I was looking to use named colors in Microsoft's Visual Studio C++, but couldn't find any enumerations anywhere. I found a Wiki Entry which listed Web Colors, aka X1 Color Names. In taking a look at the source code for the web page, I noticed that the tables were produced in a nice, computer readable format.

I ended up cutting and pasting the code into an editor and cut out all the extraneous stuff and was left with a regular list I could process with a short Perl script:

#!/usr/bin/perl

use strict;

my $line;
my ( $enum, $val );
print( "enum EColor {\n" );
while ($line = <STDIN>) {
  $line = <STDIN>;
  chomp( $line );
  $line =~ /^<td>(.+)<\/td>$/;
  $enum = $1;
  $line = <STDIN>;
  chomp( $line );
  $line =~ /^<td>(\w{2}) (\w{2}) (\w{2})<\/td>$/;
  $val = "RGB(0x" . $1 . ",0x" . $2 . ",0x" . $3 . ")";
  $line = <STDIN>;
  $line = <STDIN>;
  print( "  $enum = $val,\n" );
}
print( "}\n" );

This code produce the following C++ enumeration. If you run the above script, you'll find that one of the colors was duplicated. Simply remove the redundant copy to fix the compile error.

enum EColor {
  IndianRed = RGB(0xCD,0x5C,0x5C),
  LightCoral = RGB(0xF0,0x80,0x80),
  Salmon = RGB(0xFA,0x80,0x72),
  DarkSalmon = RGB(0xE9,0x96,0x7A),
  LightSalmon = RGB(0xFF,0xA0,0x7A),
  Crimson = RGB(0xDC,0x14,0x3C),
  Red = RGB(0xFF,0x00,0x00),
  FireBrick = RGB(0xB2,0x22,0x22),
  DarkRed = RGB(0x8B,0x00,0x00),
  Pink = RGB(0xFF,0xC0,0xCB),
  LightPink = RGB(0xFF,0xB6,0xC1),
  HotPink = RGB(0xFF,0x69,0xB4),
  DeepPink = RGB(0xFF,0x14,0x93),
  MediumVioletRed = RGB(0xC7,0x15,0x85),
  PaleVioletRed = RGB(0xDB,0x70,0x93),
  Coral = RGB(0xFF,0x7F,0x50),
  Tomato = RGB(0xFF,0x63,0x47),
  OrangeRed = RGB(0xFF,0x45,0x00),
  DarkOrange = RGB(0xFF,0x8C,0x00),
  Orange = RGB(0xFF,0xA5,0x00),
  Gold = RGB(0xFF,0xD7,0x00),
  Yellow = RGB(0xFF,0xFF,0x00),
  LightYellow = RGB(0xFF,0xFF,0xE0),
  LemonChiffon = RGB(0xFF,0xFA,0xCD),
  LightGoldenrodYellow = RGB(0xFA,0xFA,0xD2),
  PapayaWhip = RGB(0xFF,0xEF,0xD5),
  Moccasin = RGB(0xFF,0xE4,0xB5),
  PeachPuff = RGB(0xFF,0xDA,0xB9),
  PaleGoldenrod = RGB(0xEE,0xE8,0xAA),
  Khaki = RGB(0xF0,0xE6,0x8C),
  DarkKhaki = RGB(0xBD,0xB7,0x6B),
  Lavender = RGB(0xE6,0xE6,0xFA),
  Thistle = RGB(0xD8,0xBF,0xD8),
  Plum = RGB(0xDD,0xA0,0xDD),
  Violet = RGB(0xEE,0x82,0xEE),
  Orchid = RGB(0xDA,0x70,0xD6),
  Fuchsia = RGB(0xFF,0x00,0xFF),
  Magenta = RGB(0xFF,0x00,0xFF),
  MediumOrchid = RGB(0xBA,0x55,0xD3),
  MediumPurple = RGB(0x93,0x70,0xDB),
  BlueViolet = RGB(0x8A,0x2B,0xE2),
  DarkViolet = RGB(0x94,0x00,0xD3),
  DarkOrchid = RGB(0x99,0x32,0xCC),
  DarkMagenta = RGB(0x8B,0x00,0x8B),
  Purple = RGB(0x80,0x00,0x80),
  Indigo = RGB(0x4B,0x00,0x82),
  SlateBlue = RGB(0x6A,0x5A,0xCD),
  DarkSlateBlue = RGB(0x48,0x3D,0x8B),
  GreenYellow = RGB(0xAD,0xFF,0x2F),
  Chartreuse = RGB(0x7F,0xFF,0x00),
  LawnGreen = RGB(0x7C,0xFC,0x00),
  Lime = RGB(0x00,0xFF,0x00),
  LimeGreen = RGB(0x32,0xCD,0x32),
  PaleGreen = RGB(0x98,0xFB,0x98),
  LightGreen = RGB(0x90,0xEE,0x90),
  MediumSpringGreen = RGB(0x00,0xFA,0x9A),
  SpringGreen = RGB(0x00,0xFF,0x7F),
  MediumSeaGreen = RGB(0x3C,0xB3,0x71),
  SeaGreen = RGB(0x2E,0x8B,0x57),
  ForestGreen = RGB(0x22,0x8B,0x22),
  Green = RGB(0x00,0x80,0x00),
  DarkGreen = RGB(0x00,0x64,0x00),
  YellowGreen = RGB(0x9A,0xCD,0x32),
  OliveDrab = RGB(0x6B,0x8E,0x23),
  Olive = RGB(0x80,0x80,0x00),
  DarkOliveGreen = RGB(0x55,0x6B,0x2F),
  MediumAquamarine = RGB(0x66,0xCD,0xAA),
  DarkSeaGreen = RGB(0x8F,0xBC,0x8F),
  LightSeaGreen = RGB(0x20,0xB2,0xAA),
  DarkCyan = RGB(0x00,0x8B,0x8B),
  Teal = RGB(0x00,0x80,0x80),
  Aqua = RGB(0x00,0xFF,0xFF),
  Cyan = RGB(0x00,0xFF,0xFF),
  LightCyan = RGB(0xE0,0xFF,0xFF),
  PaleTurquoise = RGB(0xAF,0xEE,0xEE),
  Aquamarine = RGB(0x7F,0xFF,0xD4),
  Turquoise = RGB(0x40,0xE0,0xD0),
  MediumTurquoise = RGB(0x48,0xD1,0xCC),
  DarkTurquoise = RGB(0x00,0xCE,0xD1),
  CadetBlue = RGB(0x5F,0x9E,0xA0),
  SteelBlue = RGB(0x46,0x82,0xB4),
  LightSteelBlue = RGB(0xB0,0xC4,0xDE),
  PowderBlue = RGB(0xB0,0xE0,0xE6),
  LightBlue = RGB(0xAD,0xD8,0xE6),
  SkyBlue = RGB(0x87,0xCE,0xEB),
  LightSkyBlue = RGB(0x87,0xCE,0xFA),
  DeepSkyBlue = RGB(0x00,0xBF,0xFF),
  DodgerBlue = RGB(0x1E,0x90,0xFF),
  CornflowerBlue = RGB(0x64,0x95,0xED),
  MediumSlateBlue = RGB(0x7B,0x68,0xEE),
  RoyalBlue = RGB(0x41,0x69,0xE1),
  Blue = RGB(0x00,0x00,0xFF),
  MediumBlue = RGB(0x00,0x00,0xCD),
  DarkBlue = RGB(0x00,0x00,0x8B),
  Navy = RGB(0x00,0x00,0x80),
  MidnightBlue = RGB(0x19,0x19,0x70),
  Cornsilk = RGB(0xFF,0xF8,0xDC),
  BlanchedAlmond = RGB(0xFF,0xEB,0xCD),
  Bisque = RGB(0xFF,0xE4,0xC4),
  NavajoWhite = RGB(0xFF,0xDE,0xAD),
  Wheat = RGB(0xF5,0xDE,0xB3),
  BurlyWood = RGB(0xDE,0xB8,0x87),
  Tan = RGB(0xD2,0xB4,0x8C),
  RosyBrown = RGB(0xBC,0x8F,0x8F),
  SandyBrown = RGB(0xF4,0xA4,0x60),
  Goldenrod = RGB(0xDA,0xA5,0x20),
  DarkGoldenrod = RGB(0xB8,0x86,0x0B),
  Peru = RGB(0xCD,0x85,0x3F),
  Chocolate = RGB(0xD2,0x69,0x1E),
  SaddleBrown = RGB(0x8B,0x45,0x13),
  Sienna = RGB(0xA0,0x52,0x2D),
  Brown = RGB(0xA5,0x2A,0x2A),
  Maroon = RGB(0x80,0x00,0x00),
  White = RGB(0xFF,0xFF,0xFF),
  Snow = RGB(0xFF,0xFA,0xFA),
  Honeydew = RGB(0xF0,0xFF,0xF0),
  MintCream = RGB(0xF5,0xFF,0xFA),
  Azure = RGB(0xF0,0xFF,0xFF),
  AliceBlue = RGB(0xF0,0xF8,0xFF),
  GhostWhite = RGB(0xF8,0xF8,0xFF),
  WhiteSmoke = RGB(0xF5,0xF5,0xF5),
  Seashell = RGB(0xFF,0xF5,0xEE),
  Beige = RGB(0xF5,0xF5,0xDC),
  OldLace = RGB(0xFD,0xF5,0xE6),
  FloralWhite = RGB(0xFF,0xFA,0xF0),
  Ivory = RGB(0xFF,0xFF,0xF0),
  AntiqueWhite = RGB(0xFA,0xEB,0xD7),
  Linen = RGB(0xFA,0xF0,0xE6),
  LavenderBlush = RGB(0xFF,0xF0,0xF5),
  MistyRose = RGB(0xFF,0xE4,0xE1),
  Gainsboro = RGB(0xDC,0xDC,0xDC),
  LightGrey = RGB(0xD3,0xD3,0xD3),
  Silver = RGB(0xC0,0xC0,0xC0),
  DarkGray = RGB(0xA9,0xA9,0xA9),
  Gray = RGB(0x80,0x80,0x80),
  DimGray = RGB(0x69,0x69,0x69),
  LightSlateGray = RGB(0x77,0x88,0x99),
  SlateGray = RGB(0x70,0x80,0x90),
  DarkSlateGray = RGB(0x2F,0x4F,0x4F),
  Black = RGB(0x00,0x00,0x00)
};

Use it as you see fit.

As one more point of reference for color, here is a good reference page for Color Selection.

[/OpenSource/Programming] permanent link

2007 May 20 - Sun

SSH Article Summary

Here are a few articles I've written regarding SSH:

• Putty & Friends
• SSHD (Server) Configuration
• SSH (client) Usage

[/OpenSource] permanent link

2007 May 18 - Fri

Using a USRobotics Modem for Out of Band Management (OOB)

In configuring an older 33.6Kbps US Robotics modem for accepting auto-dial-up calls into a router, here are a few items to know:

• Use 'at&f1&b1&w0&w1y0' to force the modem to speak at 9600 to the router
• use 'ati5' to confirm new communications rate setting
• I seem to recall that all dip switch settings are in the up position but for 2, 4, 8, which are down

[/Cisco] permanent link

2007 May 17 - Thu

HTML Page Template Framework

'Glue' web pages on the monitoring server are generated through the Mason Delivery Engine. In other words, Mason is a Perl based template system used for formatting the primary web pages of this server. It requres the use of Apache and mod_perl. Here is what I do to install Mason for my needs. As the monitoring server relies on tables from two databases, OneUnified and NetDisco, the installation process revolves around getting things ready for these two types of databases.

Installation

Install the packages:

apt-get install libhtml-mason-perl
apt-get install speedy-cgi-perl
apt-get install libfcgi-perl
apt-get install libdbi-perl
apt-get install libdbd-pg-perl
apt-get install apache2-dev
apt-get install libapache2-mod-apreq2
apt-get install libapache-dbi-perl
apt-get install libmasonx-request-withapachesession-perl
apt-get install libapache2-request-perl
ln -s /etc/apache2/mods-available/apreq.load /etc/apache2/mods-enabled/apreq.load

Add the following line into '/etc/postgresql/8.1/main/pg_hba.conf':

local   netdisco    netdisco    trust
local 	oneunified  oneunified	trust

Create and load database:

/etc/init.d/postgresql-8.1 restart
su - postgres
psql template1
create user netdisco;
create group netdisco user netdisco;
create database netdisco with owner netdisco;
create user oneunified;
create group oneunified user oneunified;
create database oneunified with owner oneunified;
\q
psql netdisco -U netdisco < netdisco.dump

exit

Add the following lines into /etc/apache2/sites-available/default:

	PerlModule HTML::Mason::ApacheHandler

	<Directory /var/www/liveprobe>
        	PerlSetVar MasonArgsMethod CGI
		PerlSetVar MasonDataDir /var/local/mason
		PerlSetVar MasonDeclineDirs 0
	</Directory>

        # Decline access to mason internals
	<LocationMatch "/liveprobe/.*(\.mc|autohandler|dhandler)$">
		SetHandler perl-script
		PerlInitHandler Apache::Constants::NOT_FOUND
	</LocationMatch>

	<LocationMatch "/liveprobe/.*(\.html)$">
		SetHandler perl-script
		PerlHandler HTML::Mason::ApacheHandler
	</LocationMatch>

	Include /usr/local/netdisco/netdisco_apache.conf
	Include /usr/local/netdisco/netdisco_apache_dir.conf

Restart Apache:

/etc/init.d/apache2 restart

[/OpenSource/Debian/Monitoring] permanent link

Debian based Open Source Network Monitoring Server

In giving back to the wealth of tools the open source community has provided, I have created a number of pages describing how I have assembled a set of open source based network monitoring tools.

The set of tools is in two sections. The first step has to do with the installation of the Debian distribution:

• Debian Base Build

Once the base is in place. Make it easy on your self to log in to the server to carry out various command line tasks:

• Putty Auto Logins

Here are some entries for maintaining and upgrades of the distribution:

• Kernel Upgrades
• System Updates
• Debian Usage Notes
• Tool Pre-requisites

The next section has to do with the add-on tools:

• Page Framework
• Syslog
• NetDisco
• NTOP
• Rancid with CVS
• Tacacs
• Nagios
• NMAP2Nagios
• NfDump
• NfSen
• NfSen Upgrade
• NTOP

There are a few optional tools, not fully integrated into the overall plan yet:

• Bind

The next portion of the plan includes developing a database for maintaining circuit, patch panel, and floor diagram layout information. This will integrate with an IP Address management scheme I've come up with, and should work with Cricket and NetDisco to provide a fully integrated visual troubleshooting guide to an organization's network.

[/OpenSource/Debian/Monitoring] permanent link

2007 May 16 - Wed

One Unified Address Management Schema v1.1

I did some more brainstorming on how I wanted to visually represent a network along with it's address structure. I've incorporated some additional tables into the design to handle a hierarchical network map.

The core of the database schema is still represented by the host, interface, circuit, address, location, organization, and ianaiftype tables.

The Image table contains references to all pictures of hosts and locations. All other associated tables reference the images managed by this table.

The WeatherMap group of tables allows a Network Weathermap to be defined for a collection of circuits. By requesting a certain weathermap, the WeathermapCircuit table provides a list of associated circuits. The circuit links to interfaces and their respective hosts. By looking for hosts in the HostOnImage table that reference the same hostid and imageid, a weather map can be automatically drawn with the appropriate host picture from the HostImage table along with the collected interface statistics.

The HostType table represents names such as 'Router', 'Switch', 'Server', 'Access Point', etc.

Two tables have been added to the Location group of tables. LocationImage represents images of locations: a building, a floor, a cabinet, a back panel, with each image suggesting a collection of sub-locations. The LocationOnImage table provides the ability to click on an active area on LocationImage and drill down to the more specific LocationImage.

[/OpenSource/Debian/AddressManagement] permanent link

2007 May 15 - Tue

One Unified Address Management (OUAM)

For the longest time, I thought that the sum total of the ip address management solutions out there revolved around NorthStar and IPplan, neither of which really were as robust as I'd like.

A recent 'net search comes up with some different candidates. A IP Addressing Space Management Applications? has some interesting pointers to IP Address Management solutions, both Open Source as well as Commercial. One Open Source solution that appeared to be a stand out is Carnegie Mellon's Network Registration/Network Monitoring solution. It is under active development. Internet2 has some links to solutions that handle various combinations of Agents, Registration, and Active/Passive Detection.

Some of the above actually crosses over into the region of Network Authentication, of which Internet2's SALSAK is trying to rigorize through a Poicy Framework. Their second draft has better details, in my opinion.

So I can come back to this later, in following the various links from an earlier mentioned table, I came across PacketFence which is a Network Access Control (NAC) solution wrapped up in a VMWare deployment package.

When coming up with an IP Address Management Solution, BT Diamond IP has a handy guide to Best Practices for Next-Generation IP Address Management.

During my initial thoughts of what I'd like to see, I was focussing more on address management, floor diagrams, and port management than on DNS and DHCP. I figured DNS would be easy by simply exporting bind files on an as required basis. I havn't considered DHCP integration yet, but should be straight forward with dhcp configuration file exports, or data base lookups.

I had put together a schema diagram of what I was thinking of for ip address and facilities management.

Here is a description of the various links:

• Host -> Location: every host is associated with a particular location, floor, rack, shelf, etc
• Interface -> Host: an interface, and its sub-interfaces are associated with a host
• Interface -> Address:
  • an interface, or sub-interface will have an associated address
  • an interface will need multiple sub interfaces to contain additional addresses
  • these sub-interfaces may simply be 'secondary address blocks', or secondary addresses, or vlans
• Interface -> Circuit: an interface is associated with a particular circuit, patch panel, connector, etc
• Circuit -> Address:
  • a circuit may reference an address or address range that can be used to find attached interfaces, hosts, and circuits (and is recursive by looking at subnets and contained addresses)
  • thus routed address blocks shouldn't be referenced this way, only a circuit with ip endpoints should have an address reference
• Port -> Address: for ports routed to different locations, or are routed, this is where this is documented, such as on NAT.d addresses (eg port 80 (http) or port 25 (smtp))
• Address -> Organization: Every address range is associated with a controlling organization

Some of the tables have 'self' links. This provides an ability for defining a hierarchy of relationships:

• Address: address blocks can be subdivided down to a /32
• Interface: a phsysical interface may be divided into sub-interfaces
• Circuit: a circuit may be composed of sub-circuits, wire going from wall jack to IDF to MDF to IDF to wall jack
• Location: a building may have multiple floors, a server room may have multiple racks, a rack will have multiple 'U' locations

Here is a sql schema file to go along with the diagram. It is based upon PostgreSQL as it has native data types for handling ip addresses and mac addresses.

[/OpenSource/Debian/AddressManagement] permanent link

Import IANAifType

From the web page http://www.iana.org/assignments/ianaiftype-mib there is an list of interface types. Early portions of this list are incorporated into /usr/share/cricket/util/genDevConfig. The full list is parsed and imported into the database.

Create a perl program ianaif.pl:

#! /usr/bin/perlwhile ( <STDIN> ) {
  $_ =~ /^\s+([\-\w]+)\s*\((\d+)\),?(\s*|\s*--\s(.+?))\s*$/;
  print "insert into ianaiftype (ianaiftypeid, name, description) values ($2, '$1', '$4');\n";
}

Create a file 'ianaif.txt' with the mildly edited content from web site. Run the perl script to create an imort file:

perl ianaif.pl < ianaif.txt > ianaif.import

Import into the database:

su - postgres
psql oneunified
\i ianaif.import
\q

Here is the raw data:

                   other(1),          -- none of the following
                   regular1822(2),
                   hdh1822(3),
                   ddnX25(4),
                   rfc877x25(5),
                   ethernetCsmacd(6), -- for all ethernet-like interfaces,
                   iso88023Csmacd(7), -- Deprecated, use ethernetCsmacd (6)
                   iso88024TokenBus(8),
                   iso88025TokenRing(9),
                   iso88026Man(10),
                   starLan(11), -- Deprecated,  use ethernetCsmacd (6)
                   proteon10Mbit(12),
                   proteon80Mbit(13),
                   hyperchannel(14),
                   fddi(15),
                   lapb(16),
                   sdlc(17),
                   ds1(18),            -- DS1-MIB
                   e1(19),             -- Obsolete see DS1-MIB
                   basicISDN(20),
                   primaryISDN(21),
                   propPointToPointSerial(22), -- proprietary serial
                   ppp(23),
                   softwareLoopback(24),
                   eon(25),            -- CLNP over IP 
                   ethernet3Mbit(26),
                   nsip(27),           -- XNS over IP
                   slip(28),           -- generic SLIP
                   ultra(29),          -- ULTRA technologies
                   ds3(30),            -- DS3-MIB
                   sip(31),            -- SMDS, coffee
                   frameRelay(32),     -- DTE only. 
                   rs232(33),
                   para(34),           -- parallel-port
                   arcnet(35),         -- arcnet
                   arcnetPlus(36),     -- arcnet plus
                   atm(37),            -- ATM cells
                   miox25(38),
                   sonet(39),          -- SONET or SDH 
                   x25ple(40),
                   iso88022llc(41),
                   localTalk(42),
                   smdsDxi(43),
                   frameRelayService(44),  -- FRNETSERV-MIB
                   v35(45),
                   hssi(46),
                   hippi(47),
                   modem(48),          -- Generic modem
                   aal5(49),           -- AAL5 over ATM
                   sonetPath(50),
                   sonetVT(51),
                   smdsIcip(52),       -- SMDS InterCarrier Interface
                   propVirtual(53),    -- proprietary virtual/internal
                   propMultiplexor(54),-- proprietary multiplexing
                   ieee80212(55),      -- 100BaseVG
                   fibreChannel(56),   -- Fibre Channel
                   hippiInterface(57), -- HIPPI interfaces     
                   frameRelayInterconnect(58), -- Obsolete, use frameRelay(32) or frameRelayService(44)
                   aflane8023(59),     -- ATM Emulated LAN for 802.3
                   aflane8025(60),     -- ATM Emulated LAN for 802.5
                   cctEmul(61),        -- ATM Emulated circuit          
                   fastEther(62),      -- Obsoleted, use ethernetCsmacd (6)
                   isdn(63),           -- ISDN and X.25           
                   v11(64),            -- CCITT V.11/X.21             
                   v36(65),            -- CCITT V.36                  
                   g703at64k(66),      -- CCITT G703 at 64Kbps
                   g703at2mb(67),      -- Obsolete see DS1-MIB
                   qllc(68),           -- SNA QLLC                    
                   fastEtherFX(69),    -- Obsoleted, use  ethernetCsmacd (6)
                   channel(70),        -- channel                     
                   ieee80211(71),      -- radio spread spectrum       
                   ibm370parChan(72),  -- IBM System 360/370 OEMI Channel
                   escon(73),          -- IBM Enterprise Systems Connection
                   dlsw(74),           -- Data Link Switching
                   isdns(75),          -- ISDN S/T interface
                   isdnu(76),          -- ISDN U interface
                   lapd(77),           -- Link Access Protocol D
                   ipSwitch(78),       -- IP Switching Objects
                   rsrb(79),           -- Remote Source Route Bridging
                   atmLogical(80),     -- ATM Logical Port
                   ds0(81),            -- Digital Signal Level 0
                   ds0Bundle(82),      -- group of ds0s on the same ds1
                   bsc(83),            -- Bisynchronous Protocol
                   async(84),          -- Asynchronous Protocol
                   cnr(85),            -- Combat Net Radio
                   iso88025Dtr(86),    -- ISO 802.5r DTR
                   eplrs(87),          -- Ext Pos Loc Report Sys
                   arap(88),           -- Appletalk Remote Access Protocol
                   propCnls(89),       -- Proprietary Connectionless Protocol
                   hostPad(90),        -- CCITT-ITU X.29 PAD Protocol
                   termPad(91),        -- CCITT-ITU X.3 PAD Facility
                   frameRelayMPI(92),  -- Multiproto Interconnect over FR
                   x213(93),           -- CCITT-ITU X213
                   adsl(94),           -- Asymmetric Digital Subscriber Loop
                   radsl(95),          -- Rate-Adapt. Digital Subscriber Loop
                   sdsl(96),           -- Symmetric Digital Subscriber Loop
                   vdsl(97),           -- Very H-Speed Digital Subscrib. Loop
                   iso88025CRFPInt(98), -- ISO 802.5 CRFP
                   myrinet(99),        -- Myricom Myrinet
                   voiceEM(100),       -- voice recEive and transMit
                   voiceFXO(101),      -- voice Foreign Exchange Office
                   voiceFXS(102),      -- voice Foreign Exchange Station
                   voiceEncap(103),    -- voice encapsulation
                   voiceOverIp(104),   -- voice over IP encapsulation
                   atmDxi(105),        -- ATM DXI
                   atmFuni(106),       -- ATM FUNI
                   atmIma (107),       -- ATM IMA     
                   pppMultilinkBundle(108), -- PPP Multilink Bundle
                   ipOverCdlc (109),   -- IBM ipOverCdlc
                   ipOverClaw (110),   -- IBM Common Link Access to Workstn
                   stackToStack (111), -- IBM stackToStack
                   virtualIpAddress (112), -- IBM VIPA
                   mpc (113),          -- IBM multi-protocol channel support
                   ipOverAtm (114),    -- IBM ipOverAtm
                   iso88025Fiber (115), -- ISO 802.5j Fiber Token Ring
                   tdlc (116),        -- IBM twinaxial data link control
                   gigabitEthernet (117), -- Obsoleted, use ethernetCsmacd (6)
                   hdlc (118),         -- HDLC
                   lapf (119),        -- LAP F
                   v37 (120),        -- V.37
                   x25mlp (121),       -- Multi-Link Protocol
                   x25huntGroup (122), -- X25 Hunt Group
                   trasnpHdlc (123),   -- Transp HDLC
                   interleave (124),   -- Interleave channel
                   fast (125),         -- Fast channel
                   ip (126),        -- IP (for APPN HPR in IP networks)
                   docsCableMaclayer (127),  -- CATV Mac Layer
                   docsCableDownstream (128), -- CATV Downstream interface
                   docsCableUpstream (129),  -- CATV Upstream interface
                   a12MppSwitch (130), -- Avalon Parallel Processor
                   tunnel (131),       -- Encapsulation interface
                   coffee (132),       -- coffee pot
                   ces (133),          -- Circuit Emulation Service
                   atmSubInterface (134), -- ATM Sub Interface
                   l2vlan (135),       -- Layer 2 Virtual LAN using 802.1Q
                   l3ipvlan (136),     -- Layer 3 Virtual LAN using IP
                   l3ipxvlan (137),    -- Layer 3 Virtual LAN using IPX
                   digitalPowerline (138), -- IP over Power Lines 
                   mediaMailOverIp (139), -- Multimedia Mail over IP
                   dtm (140),        -- Dynamic syncronous Transfer Mode
                   dcn (141),    -- Data Communications Network
                   ipForward (142),    -- IP Forwarding Interface
                   msdsl (143),       -- Multi-rate Symmetric DSL
                   ieee1394 (144), -- IEEE1394 High Performance Serial Bus
                   if-gsn (145),       --   HIPPI-6400 
                   dvbRccMacLayer (146), -- DVB-RCC MAC Layer
                   dvbRccDownstream (147),  -- DVB-RCC Downstream Channel
                   dvbRccUpstream (148),  -- DVB-RCC Upstream Channel
                   atmVirtual (149),   -- ATM Virtual Interface
                   mplsTunnel (150),   -- MPLS Tunnel Virtual Interface
                   srp (151), -- Spatial Reuse Protocol 
                   voiceOverAtm (152),  -- Voice Over ATM
                   voiceOverFrameRelay (153),   -- Voice Over Frame Relay 
                   idsl (154),  -- Digital Subscriber Loop over ISDN
                   compositeLink (155),  -- Avici Composite Link Interface
                   ss7SigLink (156),     -- SS7 Signaling Link 
                   propWirelessP2P (157),  --  Prop. P2P wireless interface
                   frForward (158),    -- Frame Forward Interface
                   rfc1483 (159), -- Multiprotocol over ATM AAL5
                   usb (160),  -- USB Interface
                   ieee8023adLag (161),  -- IEEE 802.3ad Link Aggregate
                   bgppolicyaccounting (162), -- BGP Policy Accounting
                   frf16MfrBundle (163), -- FRF .16 Multilink Frame Relay 
                   h323Gatekeeper (164), -- H323 Gatekeeper
                   h323Proxy (165), -- H323 Voice and Video Proxy
                   mpls (166), -- MPLS                   
                   mfSigLink (167), -- Multi-frequency signaling link
                   hdsl2 (168), -- High Bit-Rate DSL - 2nd generation
                   shdsl (169), -- Multirate HDSL2
                   ds1FDL (170), -- Facility Data Link 4Kbps on a DS1
                   pos (171), -- Packet over SONET/SDH Interface
                   dvbAsiIn (172), -- DVB-ASI Input
                   dvbAsiOut (173), -- DVB-ASI Output 
                   plc (174), -- Power Line Communtications
                   nfas (175), -- Non Facility Associated Signaling
                   tr008 (176), -- TR008
                   gr303RDT (177), -- Remote Digital Terminal
                   gr303IDT (178), -- Integrated Digital Terminal
                   isup (179), -- ISUP
                   propDocsWirelessMaclayer (180), -- Cisco proprietary Maclayer
                   propDocsWirelessDownstream (181), -- Cisco proprietary Downstream
                   propDocsWirelessUpstream (182), -- Cisco proprietary Upstream
                   hiperlan2 (183), -- HIPERLAN Type 2 Radio Interface
                   propBWAp2Mp (184), -- PropBroadbandWirelessAccesspt2multipt
                   sonetOverheadChannel (185), -- SONET Overhead Channel
                   digitalWrapperOverheadChannel (186), -- Digital Wrapper
                   aal2 (187), -- ATM adaptation layer 2
                   radioMAC (188), -- MAC layer over radio links
                   atmRadio (189), -- ATM over radio links   
                   imt (190), -- Inter Machine Trunks
                   mvl (191), -- Multiple Virtual Lines DSL
                   reachDSL (192), -- Long Reach DSL
                   frDlciEndPt (193), -- Frame Relay DLCI End Point
                   atmVciEndPt (194), -- ATM VCI End Point
                   opticalChannel (195), -- Optical Channel
                   opticalTransport (196), -- Optical Transport
                   propAtm (197), --  Proprietary ATM       
                   voiceOverCable (198), -- Voice Over Cable Interface
                   infiniband (199), -- Infiniband
                   teLink (200), -- TE Link
                   q2931 (201), -- Q.2931
                   virtualTg (202), -- Virtual Trunk Group
                   sipTg (203), -- SIP Trunk Group
                   sipSig (204), -- SIP Signaling   
                   docsCableUpstreamChannel (205), -- CATV Upstream Channel
                   econet (206), -- Acorn Econet
                   pon155 (207), -- FSAN 155Mb Symetrical PON interface
                   pon622 (208), -- FSAN622Mb Symetrical PON interface
                   bridge (209), -- Transparent bridge interface
                   linegroup (210), -- Interface common to multiple lines     
                   voiceEMFGD (211), -- voice E&M Feature Group D
                   voiceFGDEANA (212), -- voice FGD Exchange Access North American
                   voiceDID (213), -- voice Direct Inward Dialing
                   mpegTransport (214), -- MPEG transport interface
                   sixToFour (215), -- 6to4 interface (DEPRECATED)
                   gtp (216), -- GTP (GPRS Tunneling Protocol)
                   pdnEtherLoop1 (217), -- Paradyne EtherLoop 1
                   pdnEtherLoop2 (218), -- Paradyne EtherLoop 2
                   opticalChannelGroup (219), -- Optical Channel Group 
                   homepna (220), -- HomePNA ITU-T G.989       
                   gfp (221), -- Generic Framing Procedure (GFP) 
                   ciscoISLvlan (222), -- Layer 2 Virtual LAN using Cisco ISL
                   actelisMetaLOOP (223), -- Acteleis proprietary MetaLOOP High Speed Link 
                   fcipLink (224), -- FCIP Link  
                   rpr (225), -- Resilient Packet Ring Interface Type
                   qam (226), -- RF Qam Interface
                   lmp (227), -- Link Management Protocol
                   cblVectaStar (228), -- Cambridge Broadband Limited VectaStar
                   docsCableMCmtsDownstream (229), -- CATV Modular CMTS Downstream Interface
                   adsl2 (230), -- Asymmetric Digital Subscriber Loop Version 2
                   macSecControlledIF (231), -- MACSecControlled 
                   macSecUncontrolledIF (232), -- MACSecUncontrolled
                   aviciOpticalEther (233), -- Avici Optical Ethernet Aggregate
                   atmbond (234), -- atmbond 
                   voiceFGDOS (235), -- voice FGD Operator Services
                   mocaVersion1 (236), -- MultiMedia over Coax Alliance (MoCA) Interface
                   ieee80216WMAN (237), -- IEEE 802.16 WMAN interface
                   adsl2plus (238), -- Asymmetric Digital Subscriber Loop Version 2, 
                   dvbRcsMacLayer (239), -- DVB-RCS MAC Layer
                   dvbTdm (240), -- DVB Satellite TDM
                   dvbRcsTdma (241), -- DVB-RCS TDMA
                   x86Laps (242) -- LAPS based on ITU-T X.86/Y.1323   

[/OpenSource/Debian/AddressManagement] permanent link

2007 May 14 - Mon

Nagios Installation and Sample Configuration

Introduction

As part of the network monitoring server, Nagios is used for monitoring servers and their services. Installation

Download and expand the source:

cd /usr/src
wget http://superb-east.dl.sourceforge.net/sourceforge/nagios/nagios-2.4.tar.gz
tar  -zxvf nagios-2.4.tar.gz

Add users and groups:

adduser nagios
mkdir /usr/local/nagios
chown nagios.nagios /usr/local/nagios
groupadd nagioscmd
usermod -G nagioscmd www-data
usermod -G nagioscmd nagios

Obtain libs, compile, and install:

apt-get install libgd-dev
apt-get install mcrypt
apt-get install libmcrypt-dev
cd nagios-2.4
./configure \
  --prefix=/usr/local/nagios \
  --with-cgiurl=/cgi-bin/nagios \
  --with-htmurl=/nagios \
  --with-nagios-user=nagios \
  --with-nagios-group=nagios \
  --with-command-group=nagioscmd \
  --localstatedir=/var/local/nagios
make all
make install
make install-init
make install-commandmode
make install-config

Add the following lines to /etc/apache2/sites-available/default. They need to go before the default cgi-bin configuration files.

<Location "/nagios/">
  SetHandler default-handler
</Location>
ScriptAlias /cgi-bin/nagios/ /usr/local/nagios/sbin/
<Directory "/usr/local/nagios/sbin">
#    AllowOverride AuthConfig
    AllowOverride None
    Options ExecCGI -MultiViews
    Order allow,deny
    Allow from all
    AddHandler cgi-script .cgi
</Directory>

Alias /nagios/ "/usr/local/nagios/share/"
<Directory "/usr/local/nagios/share">
    DefaultType text/html
    Options None
#    AllowOverride AuthConfig
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>

Restart Apache:

/etc/init.d/apache2 restart

Modify configuration files:

cd /usr/local/nagios
mkdir samples
cp * samples/
mv cgi.cfg-sample cgi.cfg
mv checkcommands.cfg-sample checkcommands.cfg
mv misccommands.cfg-sample misccommands.cfg
mv nagios.cfg-sample nagios.cfg
mv resource.cfg-sample resource.cfg
rm bigger.cfg-sample
rm minimal.cfg-sample

Install plug-ins:

cd /usr/src
>wget  http://superb-east.dl.sourceforge.net/sourceforge/nagiosplug/nagios-plugins-1.4.3.tar.gz
tar -zxvf nagios-plugins-1.4.3.tar.gz
cd nagios-plugins-1.4.3
./configure \
  --prefix=/usr/local/nagios \
  --with-nagios-user=nagios \
  --with-nagios-group=nagios \
  --with-cgiurl=/cgi-bin/nagios
make
make install

Make Nagios run automatically:

update-rc.d nagios defaults 25

To test configurations before committing them to execution:

cd /usr/local/nagios
bin/nagios -v etc/nagios.cfg

In /usr/local/nagios/etc/checkcommands.cfg, insert the following lines (this will be used by an Windows event logging program):

# 'check_dummy' command definition
define command{
        command_name    check_dummy
        command_line    $USER1$/check_dummy $ARG1$ $ARG2$
        }

Create a directory for site specific configurations:

mkdir /usr/local/nagios/etc/examplecom 

Make the following changes to /usr/local/nagios/etc/nagios.cfg:

#cfg_file=/usr/local/nagios/etc/minimal.cfg
cfg_dir=/usr/local/nagios/etc/examplecom
check_external_commands=1
use_syslog=0
log_notifications=0
date_format=iso8601
admin_email=root

Apply the following changes to /usr/local/nagios/etc/cgi.cfg:

46c46
< show_context_help=0
---
> show_context_help=1
65c65
< #nagios_check_command=/usr/local/nagios/libexec/check_nagios /var/local/nagios/status.dat 5 
'/usr/local/nagios/bin/nagios'
---
> nagios_check_command=/usr/local/nagios/libexec/check_nagios /var/local/nagios/status.dat 5 
'/usr/local/nagios/bin/nagios'
116c116
< #authorized_for_system_information=nagiosadmin,theboss,jdoe
---
> authorized_for_system_information=nagiosadmin,admin
128c128
< #authorized_for_configuration_information=nagiosadmin,jdoe
---
> authorized_for_configuration_information=nagiosadmin,admin
141c141
< #authorized_for_system_commands=nagiosadmin
---
> authorized_for_system_commands=nagiosadmin,admin
154,155c154,155
< #authorized_for_all_services=nagiosadmin,guest
< #authorized_for_all_hosts=nagiosadmin,guest
---
> authorized_for_all_services=nagiosadmin,admin
> authorized_for_all_hosts=nagiosadmin,admin
168,169c168,169
< #authorized_for_all_service_commands=nagiosadmin
< #authorized_for_all_host_commands=nagiosadmin
---
> authorized_for_all_service_commands=nagiosadmin,admin
> authorized_for_all_host_commands=nagiosadmin,admin

Create a file /usr/local/nagios/etc/services.cfg with the following content:

################################################################################
# Configuration File:  Services
#
# Last Modified: 2006/06/08
# By:  Ray Burkholder
#
################################################################################

################################################################################
# Service Templates
################################################################################

# Generic service definition template
define service{
        name                            generic-service ; Generic Service Template
        active_checks_enabled           1       ; Active service checks are enabled
        passive_checks_enabled          1       ; Passive service checks are enabled/accepted
        parallelize_check               1       ; Active service checks should be parallelized
        obsess_over_service             1       ; We should obsess over this service (if necessary)
        check_freshness                 0       ; Default is to NOT check service 'freshness'
        notifications_enabled           1       ; Service notifications are enabled
        event_handler_enabled           1       ; Service event handler is enabled
        flap_detection_enabled          1       ; Flap detection is enabled
        process_perf_data               1       ; Process performance data
        retain_status_information       1       ; Retain status information across program restarts
        retain_nonstatus_information    1       ; Retain non-status information across program restarts

        register                        0       ; DONT REGISTER THIS DEFINITION
        is_volatile                     0
        check_period                    24x7
        max_check_attempts              3
        normal_check_interval           3
        retry_check_interval            1
        contact_groups                  general-admins
        notification_interval           120
        notification_period             24x7
        notification_options            w,u,c,r

        }

Create a file /usr/local/nagios/etc/hosts.cfg with the following content:

###############################################################################
# Configuration File:  Hosts
# Last Modified: 2006/06/08
#
# By:  Ray Burkholder
#
################################################################################
# HOST Template
################################################################################

# Generic host definition template
define host{
        name                            generic-host-skeleton    ; The name of this host template
        notifications_enabled           1       ; Host notifications are enabled
        event_handler_enabled           0       ; Host event handler is enabled
        flap_detection_enabled          1       ; Flap detection is enabled
        process_perf_data               1       ; Process performance data
        retain_status_information       1       ; Retain status information across program restarts
        retain_nonstatus_information    1       ; Retain non-status information across program restarts
        active_checks_enabled           1
        passive_checks_enabled          0
        max_check_attempts              5
        contact_groups                  general-admins
        register                        0       ; DONT REGISTER THIS DEFINITION
        }
# 'generic-host' server host definition
define host{
        name                    generic-host
        use                     generic-host-skeleton
        check_command           check-host-alive
        notification_interval   120
        notification_period     24x7
        notification_options    d,u,r
        register                0       ; DONT REGISTER THIS DEFINITION - ITS NOT A REAL HOST
        }
# 'generic-host-nocheck' server host definition
define host{
        name                    generic-host-nocheck
        use                     generic-host-skeleton
        notification_interval   120
        notification_period     24x7
        notification_options    d,u,r
        register                0       ; DONT REGISTER THIS DEFINITION - ITS NOT A REAL HOst
        }

Add the following lines to /usr/local/nagios/etc/nagios.cfg:

cfg_file=/usr/local/nagios/etc/services.cfg
cfg_file=/usr/local/nagios/etc/hosts.cfg
cfg_dir=/usr/local/nagios/etc/examplecom

Create file /usr/local/nagios/etc/examplecom/timeperiod.cfg:

################################################################################
# Configuration File:  Time Period
#
# Last Modified: 2005/07/19
#
# By:  Ray Burkholder
#
################################################################################


###############################################################################
# TIMEPERIOD DEFINITIONS
################################################################################
# '24x7' timeperiod definition
define timeperiod{
        timeperiod_name 24x7
        alias           24 Hours A Day, 7 Days A Week
        sunday          00:00-24:00
        monday          00:00-24:00
        tuesday         00:00-24:00
        wednesday       00:00-24:00
        thursday        00:00-24:00
        friday          00:00-24:00
        saturday        00:00-24:00
        }
# 'workhours' timeperiod definition
define timeperiod{
        timeperiod_name workhours
        alias           "Normal" Working Hours
        monday          09:00-17:00
        tuesday         09:00-17:00
        wednesday       09:00-17:00
        thursday        09:00-17:00
        friday          09:00-17:00
        }
# 'nonworkhours' timeperiod definition
define timeperiod{
        timeperiod_name nonworkhours
        alias           Non-Work Hours
        sunday          00:00-24:00
        monday          00:00-09:00,17:00-24:00
        tuesday         00:00-09:00,17:00-24:00
        wednesday       00:00-09:00,17:00-24:00
        thursday        00:00-09:00,17:00-24:00
        friday          00:00-09:00,17:00-24:00
        saturday        00:00-24:00
        }
# 'none' timeperiod definition
define timeperiod{
        timeperiod_name none
        alias           No Time Is A Good Time
        }

Create a file called /usr/local/nagios/etc/examplecom/contacts.cfg with the following contact. Add additional contacts and rearrange groups as needed.

################################################################################
# Configuration File:  Contacts and Contact Groups
#
# Last Modified: 2006/06/08
#
# By:  Ray Burkholder
#
################################################################################
# relevant documentation
# http://nagios.sourceforge.net/docs/1_0/xodtemplate.html#contact<
################################################################################
# CONTACT DEFINITIONS
################################################################################
# 'nagios' contact definition
define contact{
        contact_name                    nagios
        alias                           Nagios Admin
        service_notification_period     24x7
        host_notification_period        24x7
        service_notification_options    w,u,c,r
        host_notification_options       d,u,r
        service_notification_commands   notify-by-email,notify-by-epager
        host_notification_commands      host-notify-by-email,host-notify-by-epager
        email                           nagios-admin@localhost.localdomain
        pager                           pagenagios-admin@localhost.localdomain
        }
# Network Generic Alerts
define contact {
  contact_name                          networkalerts
  alias                                 Network Alerts
  service_notification_period           24x7
  host_notification_period              24x7
  service_notification_options          c,w
  host_notification_options             d,r
  service_notification_commands         notify-by-email
  host_notification_commands            host-notify-by-email
  email                                 networkalerts@example.com
  }
################################################################################
# CONTACT GROUP DEFINITIONS
################################################################################
# 'general-admins' contact group definition
define contactgroup{
        contactgroup_name       general-admins
        alias                   General Administrators
        members                 networkalerts
        }
# 'windows-server-admins' contact group definition
define contactgroup{
        contactgroup_name       windows-server-admins
        alias                   Windows Server Administrators
        members                 networkalerts
        }
# 'windows-desktop-admins' contact group definition
define contactgroup{
        contactgroup_name       windows-desktop-admins
        alias                   Windows Desktop Support
        members                 networkalerts
        }
# 'linux-server-admins' contact group definition
define contactgroup{
        contactgroup_name       linux-server-admins
        alias                   Linux Administrators
        members                 networkalerts
        }
# 'switch-admins' contact group definition
define contactgroup{
        contactgroup_name       switch-admins
        alias                   Etherswitch Administrators
        members                 networkalerts
        }
# 'router-admins' contact group definition
define contactgroup{
        contactgroup_name       router-admins
        alias                   Router Technicians
        members                 networkalerts
        }
# 'firewall-admins' contact group definition
define contactgroup{
        contactgroup_name       firewall-admins
        alias                   Firewall Technicians
        members                 networkalerts
        }
# 'printer-admins' contact group definition
define contactgroup{
        contactgroup_name       printer-admins
        alias                   Printer Administrators
        members                 networkalerts
        }
# 'sensor-admins' contact group definition
define contactgroup{
        contactgroup_name       sensor-admins
        alias                   Sensor Administrators
        members                 networkalerts
        }

Create a file called /usr/local/nagios/etc/examplecom/groups.cfg with the following template:

################################################################################
# Configuration File:  Groups
#
# Last Modified: 2006/06/08
# By:  Ray Burkholder
#
################################################################################
# Group DEFINITIONS
################################################################################
#define hostgroup {
#  hostgroup_name               groupname
#  alias                        Member Devices
#  members                      x,y
#  }

[/OpenSource/Debian/Monitoring] permanent link

2007 May 10 - Thu

TFTP Installation with ATFTPD

I use the atftpd tftp server daemon. For the most part, it is an easy package retrieval. However there are some custom security and directory settings. Installation

Install the package:

apt-get install atftpd

Edit /etc/inetd.conf and comment out the line with tftpd.

Restart inetd:

/etc/init.d/inetd.dpkg-new restart

Create a new directory, user, group, and privleges:

mkdir /var/atftpd
cd /var
groupadd atftpd
useradd -d /var/atftpd -g atftpd atftpd
chmod 766 atftpd
chown atftpd.atftpd atftpd

Edit /etc/default/atftpd. Add '--user atftpd.atftpd' and change '/tftpboot' to '/var/atftpd'.

Start the service:

/etc/init.d/atftpd start

[/OpenSource/Debian/Monitoring] permanent link

Installing Asterisk

It has been a year or two since I last worked with the Opensource PBX solution called Asterisk. Instead, I've been doing stuff with Cisco Callmanger and Voice Gateways for the last while. My support pages are still receiving regular hits with regards to Asterisk Support, so I think I should do more with it. I've got a bunch of scripts laying about that I'd like to resurrect.

So, to start off, I have a new Debian server, I need to install the latest and greatest from version control. Here is what I did.

These get me up to current for latest kernel. Compiling Asterisk requires the kernel headers, so they are included here. I want the call detail records to go to a PostgreSQL database, so I include the libraries as well. As the latest source is in Subversion, I need that package to obtain the installation files.

apt-get install linux-image-2.6.18-4-686
apt-get install linux-headers-2.6.18-4-686
apt-get install libncurses5-dev
apt-get install postgresql-dev
apt-get install subversion

Now I can obtain the source files:

cd /usr/src 
mkdir digium 
cd digium 
svn checkout http://svn.digium.com/svn/asterisk/trunk asterisk
svn checkout http://svn.digium.com/svn/zaptel/trunk zaptel
svn checkout http://svn.digium.com/svn/libpri/trunk libpri

Compile the driver files. A double make install will be required. If you are not using Digium hardware, use the ztdummy module, otherwise use the zaptel module. Once compiled and installed, the zaptel configuration file is found in /etc/zaptel.conf and will need to be updated before using the zaptel module.

cd zaptel
make clean
make install
make install
mddprobe ztdummy
modprobe zaptel

For running with PRI digitial telephone circuits, another library needs to be built:

cd ../libpri
make clean
make install

This last one holds the bulk of the Asterisk build.

cd ../asterisk
./configure \
  --sysconfdir=/etc \
  --localstatedir=/var
make samples

That gets us to a basic installation. My next write up will show some of the configuration file stuff I do.

[/OpenSource/Debian/Asterisk] permanent link

2007 May 09 - Wed

Various Perl Based Proxy Tools

In perusing Debian's Package List, I came across a number of Perl based Proxy tools.

The first one, an item that sounds interesting, but havn't thought of a way to put it into action yet, is an HTTP Recorder. HTTP::Recorder is a browser-independent recorder that records interactions with web sites and produces scripts for automated playback. Recorder produces WWW::Mechanize scripts by default (see WWW::Mechanize by Andy Lester),

The next item is an HTTP Tunnel. httptunnel creates a bidirectional virtual data connection tunnelled in HTTP requests. The HTTP requests can be sent via an HTTP proxy if so desired. This can be useful for users behind restrictive firewalls. If WWW access is allowed through a HTTP proxy, it's possible to use httptunnel and, say, telnet or PPP to connect to a computer outside the firewall.

The third item is HTTP::Proxy. It is a Perl based HTTP Proxy. It sounds like it can do some SSL type interception as well. It has an ability for add-on modules allowing various parts of a page to be re-processed prior to delivery back to a user.

[/OpenSource/Debian] permanent link

Redirecting a Web Page

I've encountered a number of ways to redirect a web page. If all you have access to is a web page, then a meta tag is the way to go:

<html>
<head>
<meta http-equiv="refresh" content="3;url=/liveprobe/index.html">
</head>
<body>
You will be redirected to <a href=/liveprobe/index.html>/liveprobe/index.html</a> in 3 seconds.
</body>
</html>

More meta-tags can be found at http://vancouver-webpages.com/META/.

If pages are being dynamically created, then executing the following code before anything else on the page is generated should do the trick (I haven't tried it out myself yet):

        print "Status: 302 Moved Temporarily\r\n",
              "Location: $url\r\n",
              "Content-Type: text/html\r\n\r\n",
              "$url\r\n";

[/Personal/SoftwareDevelopment/HTML] permanent link

Upgrading Nfsen and Nfdump

As an update to my two articles for installing nfdump and installing nfsen, here are a few corrections and a process for upgrading.

As of this writing, the latest snapshots are for March 12. Here is the upgrade process:

cd /usr/src
wget http://superb-east.dl.sourceforge.net/sourceforge/nfsen/nfsen-snapshot-20070312.tar.gz
wget http://superb-west.dl.sourceforge.net/sourceforge/nfdump/nfdump-snapshot-20070312.tar.gz
/usr/local/nfsen/bin/nfsen.rc stop
tar -zxvf nfdump-snapshot-20070312.tar.gz
tar -zxvf nfsen-snapshot-20070312.tar.gz
cd nfdump-snapshot-20070312
./configure
make
make install
cd ../nfsen-snapshot-20070312
./install.pl /etc/nfsen/nfsen.conf
/usr/local/nfsen/bin/nfsen.rc start

The 'start' command could be placed in /etc/rc.local so it starts upon boot. The start command also starts the flow collectors (nfdump), so there is no need to start them; the nfsen.rc command takes care of everything.

[/OpenSource/Debian/Monitoring] permanent link

2007 May 08 - Tue

Determiniing Space Used in SubDirectories

When space is getting tight, one has to find out where the space is being used. Sometimes it suffices to find where the most accumulation is and cleaning that up. To find that accumulation, use the du command:

 du -h -x  --max-depth=1

The -h turns the result into human readable form, the -x excludes files on a different file system (for when you have a mount somewhere in the structure), and the --max-depth performs a summarizes the results of the recursive search through the structure.

[/OpenSource] permanent link

Removing lots of files

If I don't have a clean up program running, sometimes autocollected files tend to accumulated. For example, netflow processing on 5 minute intervals can accumulate a large number of files. An 'rm' with a partical wild card seems to complain.

I've resorted to a chaining some command line utilities to come up with a quick script to remove files:

ls -1 | grep ft-v05.2007-0[123] | sed 's/^/rm /' > t.sh

This generates a one column directory listing and puts it through grep. Grep looks for a few specific months of files and passes the list onto sed. Sed preappends the removal command onto each file name. The whole shooting match is thrown into a shell script.

Either set execute privileges on the script:

chmod 500 t.sh

Or run with with the shell command:

bash t.sh

[/OpenSource] permanent link

2007 May 07 - Mon

Importing a Blosxom Blog into Movable Type

I have my Blosxom based blog organized by category directories rather than by date. I've written a Perl script to take this collection of articles and transform them into a MovableType import file. My content files have a .txt extension. You'll need to change the match string if your extension is different from mine.

To run, there is a variable called dir which you seed with a starting directory. The program then scans that directory and further sub-directories looking for files with the designated suffix. It then uses the first line of a found file for the title. The third and subsequent lines are used for the content.

I've found that after importing, I'm not able to see the body in Movable Type's content editor. How weird... Well, it does show up when you customize the display of the page. It shows up in the extended entry text.

2007/06/12: David Graff suggested an additional print statement before the body block. I havn't tried it, but I hope it works to remedy the missing body bit.

Here is the script:

#!/usr/bin/perl

use strict;
#use File::stat;
use Fcntl ':mode';

my $author = 'ray';
my $ext = '.txt';

my @dirs;
my $dir = '/var/www/html/blog';
push( @dirs, $dir );

while ( $dir = pop( @dirs) ) {
  chdir( $dir );
  opendir( DIR, $dir );
#  print( "$dir\n" );
  foreach my $file ( readdir( DIR) ) {
    if ( $file =~ /.txt$/ ) {
      my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) =
                                                localtime((stat($file))[9]);
     $mon+=1;
     $year+=1900;
     $mon = substr( '0' . $mon, -2 ,2 );
     $mday = substr( '0' . $mday, -2, 2 );
     $hour = substr( '0' . $hour, -2, 2 );
     $min  = substr( '0' . $min, -2, 2 );
     $sec  = substr( '0' . $sec, -2, 2 );
     my $date = "$mon/$mday/$year $hour:$min:$sec";
#     print( "  $file: $date\n");

     open( FILE, '<' . $file );
     my $title = <FILE>;
     chomp( $title );

     my $body;
     my $extbody;
     my $line;
     my $summary = 1;
     while ( $line = <FILE> ) {
       if ( $summary ) {
         $body .= $line;
         if ( length( $line ) < 2 ) {
           $summary = 0;
           }
         }
       $extbody .= $line;
       }

     close( FILE );

     $dir =~ /\/([^\/]+$)/;
     my $category = $1;

     print( "AUTHOR: $author\n" );
     print( "TITLE:  $title\n" );
     print( "DATE: $date\n" );
     print( "PRIMARY CATEGORY: $category\n" );
     print( "STATUS: publish\n" );
#     print( "ALLOW COMMENTS: 1\n" );
#     print( "ALLOW PINGS: 1\n" );
     print("-----\n"); # 2007/06/12 insertion by David Graff
     print( "BODY:\n" . $body . "\n-----\n" );
     print( "EXTENDED BODY:\n" . $extbody . "\n-----\n" );
     print( "--------\n" );
     }

    my $mode = (stat($file))[2];
    if ( S_ISDIR( $mode ) ) {
      if ( '.' ne $file && '..' ne $file ) {
        push( @dirs, $dir . '/' . $file );
        }
      }
    }
  closedir( DIR );
  }

[/OpenSource/Debian/MovableType] permanent link

Security Enhancements for Remote Access at Microsoft

Here is a link to a paper that has a bunch of useful stuff in it regarding Microsoft VPN's, IAS (Internet Authentication Server), security scripting, and Windows 2003 based Remote Access Infrastructure:

Security Enhancements for Remote Access at Microsoft: Technical White Paper

[/Cisco] permanent link

User Certficate Auto Enrollment

With my 802.1x test setup, machine certificates were being sent to domain machines with no problem, but user certificates were not showing up.

In the group policy object, right on the container housing the users that needed certificates, I set the auto-enrollment settings. For some reason things weren't being inherited from the domain default policy. The group policy container is User Configuration -> Windows Settings -> Security Settings -> Public Key Policies -> Autoenrollment SEttings. The 'Enroll Certificates Automatically' needs to be checked along with it's two subsidiary check boxes.

The following command serves as a manual refresh of the policy:

gpupdate /target:user

Enrollment will take several minutes. Running the certmgr.msc mmc snap-in will allow one to check that the certificate has arrived in the Personal -> Certificates store.

The Application Event Log will contain success/failure status for the auto-enrollment.

I also found out from an troubleshooting auto-enrollment article, that domain users without email addresses will not auto-enroll. They don't need an actual email box, just an entry in the email attribute in Active Directory.

As further reference, Microsoft has an article on How Autoenrollment Works. There are other related and helpful articles in the same library section.

[/Cisco] permanent link

Movable Type Links

Here are some links to various pages in Movable Type that look useful:

• User Manual (HTML)
• Movable Type Performance Tuning
• Plugins Directory
• Template Tag Reference
• User and Administration Manual (pdf)
• Import Format
• Resources

[/OpenSource/Debian/MovableType] permanent link

2007 May 06 - Sun

I Hate PHP on a database call.

I think this is the second time I've been caught on this. And I should have recalled the second time around to check this.

I'm working through the process of installing and playing with Movable Type. It went quite well. Configuration and maintenance and adding content worked well. Well, up until I decided to enable Dynamic Content generation. Then I found out Movable Type has two faces, a Perl one and a PHP one.

To handle dynamic content, they do an Apache redirect/rewrite through a default php script, one that does a database call. And in order to do the databae call, it opens a connection to the database.

I'm testing with PostgreSQL. My Debian install didn't have, by default, the php4-pgsql package. If it isn't installed, PHP simply aborts somewhere in the code, quietly. How silly. No errors, no messages, no nothing.

So after untold 'echo()' statements later, I tracked it down to a call in the ezsql implementation. Finally the light bulb went on over my head. The fix:

apt-get install php4-pgsql

I'm wondering if there is a way to test for this package, so I don't forget this again.

I probably didn't clue into this before because the main Movable Type scripts are in Perl and use the database connections there. I made the, obviously incorrect assumption, that the PHP scripts were involved and already knew about the database driver. Well, I was incorrect on that one.

[/OpenSource/Debian/MovableType] permanent link

apt-get: Managing Packages

In a few of my earlier articles regarding upgrading and updating a Debian system, I missed a few key facts.

In one article, a mentioned performing a distribution upgrade. Another one mentioned doing a kernel upgrade and missing an lvm2 module.

The key is that if modules are being held-back, that is a sign that a distribution upgrade will be needed:

apt-get -u dist-upgrade

The -u parameter provides more detail as to what will be happening during the upgrade.

More details for working with packages can be found at APT HowTo

[/OpenSource/Debian] permanent link

SmartQuant QuantDeveloper & DataCenter Release

SmartQuant has released a revision to DataCenter and QuantDeveloper. DataCenter and QuantDeveloper are at the following revision levels:

DataCenter
Version 2.2.3 (12-Apr-2007) 

QuantDeveloper Enterprise Edition
Version 2.5.4 (04-May-2007)

QuantDeveloper Source Code
Version 2.5.1 (23-Mar-2007) 
* Recent Versions available through 
  version control 

[/Trading/SmartQuant/Releases] permanent link

2007 May 05 - Sat

Post Processing NMAP2Nagios Output

nmap2nagios is a module available from Nagios ExchangeK (I seem to recall). From an nmap based network scan, it generates information useful for import into Nagios. I needed to muck with the output somewhat in order to remove redundant references and other stuff. I'm sure there is a better way to do this, but this what I ended up with. If nothing else, it was a good introduction to the command line edit utility 'sed'.

On the network I was processing, device names had 'ilo' in them. I needed to remove them from the listing. I edited nmap2nagios.pl and did the following:

#!/usr/bin/perl -w

#line 126
my $n = $host_ref->{'host_name'};
print '** name ' . $n;
print ' done' . "\n";
next if ($n =~ /^ilo/);

I took a bunch of stuff out of nmap2nagios.conf.

I created a file called 'scanlo.sed' with the following content to remove references to printer and altiris stuff:

/portid="443/ {s/name="http"/name="https"/}
/portid="280/ {s/name="http"/name="hpweb1"/}
/portid="631/ {s/name="http"/name="hpweb2"/}
/portid="902/ {s/name="ftp"/name="altirisftp1"/}
/portid="912/ {s/name="ftp"/name="altirisftp2"/}

I then ran the following sequence of commands. nmap does the network scan with the given segment. Some inline editing is performed with sed. nmap2nagios creates another file, which is then copied to the nagios etc directory.

nmap -A -sV -p1-1024 -O -oA scanlo -v 10.1.1.0/24
sed -f scanlo.sed scanlo.xml > scanlo1.xml
/usr/src/nmap2nagios-0.1.2/nmap2nagios.pl -i -v -r scanlo1.xml -o sh1.cfg
sed 's/^>//' sh1.cfg > sh2.cfg
cp sh2.cfg /usr/local/nagios/etc/

For checking ssh services, the following needs to be inserted into checkcommands.cfg:

# 'check_ssh' command definition
define command{
        command_name    check_ssh
        command_line    $USER1$/check_ssh  $HOSTADDRESS$
        }

[/OpenSource/Debian/Monitoring] permanent link

2007 May 04 - Fri

Configuring eSensors with Nagios

The EM01B WebSensor from Esensor (http://www.eesensors.com/websensor.html) provides Temperature, Humidity, and Illumination values to Nagios.

Download the source code for the Nagios plugin from http://www.nagiosexchange.org/Environmental.60.0.html?&tx_netnagext_pi1[p_view]=13 and place it into the /usr/src directory. Unzip it to a directory called esensors. Change into /usr/src/esensors/c and run:

gcc check_em01.c -o check_em01
cp check_em01 /usr/local/nagios/libexec/

Ignore the warnings.

Add the following llnes to /usr/local/nagios/etc/checkcommands.cfg:

# 'check_temp' command definition
define command{
            command_name check_temp
            command_line $USER1$/check_em01 $HOSTADDRESS$ T $ARG1$ $ARG2$ $ARG3$ $ARG4$
            }
# 'check_humidity' command definition
define command{
            command_name check_humidity
            command_line $USER1$/check_em01 $HOSTADDRESS$ H $ARG1$ $ARG2$ $ARG3$ $ARG4$
            }
# 'check_light command definition
define command{
            command_name check_light
            command_line $USER1$/check_em01 $HOSTADDRESS$ I $ARG1$ $ARG2$ $ARG3$ $ARG4$
            }

Create a file named /usr/local/nagios/etc/srvc_esensor.cfg with the following content:

###############################################################################
#
# Configuration file for eSensor Devices
#
# Last Modified: 2006-06-08
#
################################################################################

################################################################################
# SERVICE DEFINITIONS
################################################################################
# Generic service definition template
define service{
        name                            generic-esensor ;  Referenced in other service definitions
        active_checks_enabled           1       ; Active service checks are enabled
        passive_checks_enabled          1       ; Passive service checks are enabled/accepted
        parallelize_check               1       ; Active service checks should be parallelized
        obsess_over_service             1       ; We should obsess over this service (if necessary)
        check_freshness                 0       ; Default is to NOT check service 'freshness'
        notifications_enabled           1       ; Service notifications are enabled
        event_handler_enabled           1       ; Service event handler is enabled
        flap_detection_enabled          1       ; Flap detection is enabled
        process_perf_data               1       ; Process performance data
        retain_status_information       1       ; Retain status information across program restarts
        retain_nonstatus_information    0       ; Retain non-status information across program restarts
        register                        0       ; DONT REGISTER THIS DEFINITION
        is_volatile 0
        check_period 24x7
        max_check_attempts 3
        normal_check_interval 5
        retry_check_interval 1
        contact_groups sensor-admins
        notification_interval 30
        notification_period 24x7
        notification_options d,u,r
        }
# check temp service definition template
define service{
        use                             generic-esensor
        name                            etemp-service   ; Referenced in other service definitions
        register                        0       ; DONT REGISTER THIS DEFINITION
        service_description Temperature
        check_command check_temp!2!110!1!120
        }
# check humidity service definition template
define service{
        use                             generic-esensor
        name                            ehum-service    ; Referenced in other service definitions
        register                        0       ; DONT REGISTER THIS DEFINITION
        service_description Humidity
        check_command check_humidity!2!90!1!99
        }
# check light service definition template
define service{
        use                             generic-esensor
        name                            eillum-service  ; Referenced in other service definitions
        register                        0       ; DONT REGISTER THIS DEFINITION
        service_description Illumination
        check_command check_light!2!600!1!700
        }

Add the following line to /usr/local/nagios/etc/nagios.cfg:

cfg_file=/usr/local/nagios/etc/srvc_esensor.cfg

Create a file called /usr/local/nagios/etc/examplecom/em01.cfg with the following content:

define host{
        use                             generic-host
        host_name                       em01
        alias                           Heat, Humidity and Illumination tests
        address                         10.1.6.30
        }

define hostgroup{
        hostgroup_name  	Esensors
        alias          		Esensors HVAC sensor
        members         	em01
        }

define servicegroup{
        servicegroup_name       esensors
        alias                   Enviromental Monitoring
        members                 em01,Temperature
        members                 em01,Humidity
        members                 em01,Illumination
        }

define service {
        use                     etemperature-service
        host                    em01
        check_command           check_temp!60!80!50!85
        }

define service {
        use                     ehumidity-service
        host                    em01
        }

define service {
        use                     eillumination-service
        host                    em01
        }

As a side note, I think I'm going to process esensor output with something else. Here is a brief regular expression in a perl file called 'em.pl' to handle the output:

use strict;

my $a = <STDIN>;
$a =~ m/TF:[ ]*(\d+\.\d+).*HU:[ ]*(\d+\.\d+).+IL[ ]*(\d+\.\d+)/;
print "$1, $2, $3\n";

The following command line generates three values (using the tool 'curl' to obtain the current values):

echo `curl -s http://bmem01/index.html?em123456` | perl em.pl

[/OpenSource/Debian/Monitoring] permanent link

2007 May 03 - Thu

Installing SNMP Modules

SNMP configuration on Debian is quite easy:

apt-get install snmp
apt-get install snmpd

Here is an example snmp query on a device named device01, community string public, and values of interest in the ciscoRttMonMIB MIB. The '-m' says to load all mibs from the '-M' mib directory.

snmpwalk -v2c -c public -m ALL -M /usr/share/snmp/mibs device01  ciscoRttMonMIB 

[/OpenSource/Debian/Monitoring] permanent link

2007 May 02 - Wed

Drraw Installation

Cricket and NetFlow create RRD files. Each has some basic utilities for drawing data found within the files. Drraw provides advanced capabilities for creating graphs, templates, and dashboards. Installation

Download the utility and expand the source:

wget http://web.taranis.org/drraw/dist/drraw-2.1.3.tgz
tar -zxvf drraw-2.1.3.tgz

Create an icon directory, copy the script and icon files:

cd drraw-2.1.3
mkdir /var/www/icons
cp icons/* /var/www/icons
cp drraw* /usr/lib/cgi-bin/
chmod 755 /usr/lib/cgi-bin/drraw.cgi

Create some directories:

mkdir /var/cache/apache2/drraw
mkdir /var/cache/apache2/drraw/saved
mkdir /var/cache/apache2/drraw/tmp
chown -R www-data.www-data /var/cache/apache2/drraw
mkdir /var/log/drraw
touch /var/log/drraw/error.log
chown -R www-data.www-data /var/log/drraw

Update /usr/lib/cgi-bin/drraw.conf with the following:

%datadirs = (
#               '/here/are/some/files'  => '[Label1] ',
             '/var/lib/cricket' => '[Cricket] ',
            );
$saved_dir = '/var/cache/apache2/drraw/saved';
$tmp_dir = '/var/cache/apache2/drraw/tmp';
$ERRLOG = '/var/log/drraw/errors.log';

Change the first line of /usr/lib/cgi-bin/drraw.cgi to:

#! /usr/bin/perl -T

The program can be started with:

http://localhost/cgi-bin/drraw.cgi

[/OpenSource/Debian/Monitoring] permanent link

2007 May 01 - Tue

HTML Page Template Framework

'Glue' web pages on the monitoring server are generated through the Mason Delivery Engine. In other words, Mason is a Perl based template system used for formatting the primary web pages of this server. It requres the use of Apache and mod_perl. Here is what I do to install Mason for my needs. As the monitoring server relies on NetDisco tables, a portion of the installation process revolves around initializing the NetDisco PostgreSQL database.

Installation

Install the packages:

apt-get install libhtml-mason-perl
apt-get install libdbi-perl
apt-get install libdbd-pg-perl
apt-get install apache2-dev
apt-get install libapache2-mod-apreq2
apt-get install libapache-dbi-perl
apt-get install libmasonx-request-withapachesession-perl
apt-get install libapache2-request-perl
ln -s /etc/apache2/mods-available/apreq.load 
/etc/apache2/mods-enabled/apreq.load

Add the following line into '/etc/postgresql/7.4/main/pg_hba.conf':

local   netdisco    netdisco     trust

Create and load database:

/etc/init.d/postgresql-7.4 restart
su - postgres
psql template1
create user netdisco;
create group netdisco user netdisco;
create database netdisco with owner netdisco;
\q
psql netdisco -U netdisco < netdisco.dump

exit

Add the following lines into /etc/apache2/sites-available/default:

    PerlModule HTML::Mason::ApacheHandler
    <LocationMatch "(.html|.txt|.pl)$">
        SetHandler perl-script
        PerlHandler HTML::Mason::ApacheHandler
    </LocationMatch>
    <LocationMatch "(.m(html|txt|pl)|dhandler|autohandler)$">
        SetHandler perl-script
        PerlInitHandler Apache::Constants::NOT_FOUND
    </LocationMatch>

        <Directory />
                Options FollowSymLinks
                AllowOverride None
                AllowOverride AuthConfig
                AuthType Basic
                AuthName "Password Required"
                AuthUserFile /etc/apache2/sites-available/password.file
                Require user admin
        </Directory>

Create a password file with a username admin:

htpasswd -c /etc/apache2/sites-available/password.file admin

Restart Apache:

/etc/init.d/apache2 restart

[/OpenSource/Debian/Monitoring] permanent link

2007 Apr 30 - Mon

Implementing a Wired 802.1X Network With Cisco and Microsoft

Some companies will set up guest networks utilizing a parallel network configuration: separate switches or hubs, along with separate DSL/router internet connections. They will then designate certain ports in a conference room to be 'corporate connections' and certain ports to be 'guest network'. They then leave it up to the user to 'pick a port'.

This mechanism does indeed provide a separate path to the internet, but obviously, the weakness is an inability to prevent people from using inappropriate ports.

A better from of enforcement is provided through the implementation of an authentication/authorization protocol called 802.1X. This protocol works with wired as well as wireless networks. Various methods of operation are available. The simplest to to either enable or disable a switch port based upon receipt of appropriate credentials from the supplicant, which is the computer/user being connected to the network.

A more sophisticated form of operation is to assign a vlan (and associated IP address) based upon computer and/or user credentials. If a connecting device does not have supplicant ability, a default 'guest' vlan can be assigned.

According to Cisco's and Microsoft's literature, the best authentication mechanism for 802.1X is through EAP-TLS, a PKI/certificate mechanism.

This document describes an implementation based upon Microsoft's Certifcate Authority (CA), Microsoft's Internet Authentication Server (IAS), and Cisco Switches.

The implementation uses a Windows Server 2003, Enterprise Edition, for the Certificate Authority. The Enterprise Edition has a few more CA templates (such as Wireless certificate templates) than does the Standard Edition product. One recommendation is to run the CA on a VMWare server. This provides the ability for simple backups and to move the CA from place to place. For very risk averse environments, placing the CA on a dedicated server in a secure room would be a better choice. The primary consideration in this is that the CA is a single point of failure and requires a simple, convenient, fast mechanism to bring it back to life during some sort of failure condition.

With the CA in place, machine and user certificates can be automatically issued through a Group Policy configuration.

Users and machines should be assigned to various Active Directory groups. For machines, possible groupings would be servers, workstations, and laptops. Since Laptops are typically more prone to carrying malicious nasties, assigning them to protected segments is beneficial.

As some HP printers are 802.1X ready, they can also be issued certificates.

Users can also be issued certificates. Preliminary testing indicates that Group Policy doesn't push out the certificates. Users need to download them manually by connecting to the CA server webpage at http://ca/certsrv to obtain their certificates.

Windows XP workstations have the 802.1X supplicant built in. The settings for the supplicant are in the authentication tab under properties for the network card. On some machines, you may need to start the Wireless Zero Configuration (WZC) service in order to see the tab. For our implementation, the defaults are fine.

However, there are two registry keys we play with to change the characteristics somewhat. They are described in a document called 802.11 Wireless Tools and Settings. Even though the document is about 802.11 Wireless, the settings still apply to wired connections. After changing the registry keys, the machine will need to be rebooted, or simpler, restart the WZC service.

• AuthMode: HKEY_LOCAL_MACHINE\Software\Microsoft\EAPOL\Parameters\General\Global\AuthMode
• SupplicantMode: HKEY_LOCAL_MACHINE\Software\Microsoft\EAPOL\Parameters\General\Global\SupplicantMode

With AuthMode of 1 and SupplicantMode of 3, you can get different VLAN's depending upon whether or not a user is logged in. If a user is not logged in, and a machine is plugged in, the machine certificate defines the VLAN. If a user subsequently logs in, the VLAN assignment changes to that associated with the user. If the user logs out, the VLAN reverts back to the VLAN associated with the machine certificate.

For another site where the administrator wants to ensure that users login to the network in order to ensure login scripts run (scripts that check that virus clients are up to date and such), a SupplicantMode of 2 and an AuthMode of 1 would be used. Supplicant mode of 2 will not force a renegotiation when a user logs in. In this scenario, one of two VLAN assignments happen:

• If a user is logged in and the machine is connected to the network, the VLAN assignment is based upon the logged in user's certificate. As this doesn't force a domain login, no login scripts are run. Therefore, a blocked VLAN would be assigned to this relationship. This will force a user to logout, disconnect and reconnect in order to get the machine based VLAN instead.
• If a user is not logged in and the machine is connected to the network, the VLAN assigned is the one associated with the machine certificate. Even when the user subsequently logs in, the VLAN assignment does not change. This mechansim forces the user's domain log in and the operation of log in scripts. (See further in this document on how this relates to the 'reauthenticate' command in the switch).

A company called Blue Socket may have some add-on tools that force Domain Login.

On the Cisco side of things, there are a number of configuration items. For a 3750 switch, Here is an example configuration document: Configuring 802.1x Port-Based Authentication.

The switch needs to know of one or, preferably more, RADIUS servers:

radius-server host 10.10.10.10 auth-port 1645 acct-port 1646 key thisisakey
radius-server source-ports 1645-1646

Make sure you have a username line and enable secret in the configuration:

username switchadmin password switchadminpass
enable secret enablesecret

Then let the switch know to where authentication requests need to be sent:

aaa new-model
aaa authentication dot1x default group radius
aaa authorization network default group radius
aaa accounting dot1x default start-stop group radius
aaa accounting system default start-stop group radius

A switch global command is required:

dot1x system-auth-control

Each interface needs some additional commands:

interface GigabitEthernet0/1
 switchport access vlan 3
 switchport mode access
 switchport port-security
 switchport port-security aging time 2
 switchport port-security violation restrict
 switchport port-security aging type inactivity
 dot1x pae authenticator
 dot1x port-control auto
 dot1x timeout quiet-period 3
 dot1x timeout server-timeout 10
 dot1x timeout tx-period 5
 dot1x timeout supp-timeout 5
! dot1x reauthentication
 dot1x auth-fail vlan 105
 spanning-tree portfast
 spanning-tree bpduguard enable

The 'dot1x reauthentication' may or may not be applicable. A scenario in which it should not be used is the following. I mentioned a scenario where you want a machine to be connected to the network before a user logs in. When the user does finally login, you don't want the vlan to change. Hence, SupplicantMode should be 2, and there should be no 'dot1x reauthentication' command. If the command were in place, the reauthentication would see the user certificate and change the vlan to suit. Which wouldn't be the desired affect.

For further reading and reference, here are some additional links:

• EAP-TLS Deployment Guide for Wireless LAN Networks
• EAPOL\SupplicantMode & AuthMode for vista
• Wireless Deployment Technology and Component Overview
• Deploying Windows Server 2003 Internet Authentication Service (IAS) with Virtual Local Area Networks (VLANs)
• Internet Authentication Service (IAS) Operations Guide

[/Cisco] permanent link

2007 Apr 25 - Wed

My Voip Source

MyVoIPsource.org is a community plattform to Voice-over-IP Know-How. Here you will find:

• Opensource VoIP Tools to make your IP-Telephony solution more attractive
• VoIP Trainings to bring you "up-to-speed"
• Support services if you don't have the time to do it yourself
• A forum to engage with people who share your technical challenges
• Development resources for your special features and needs

From their website: "Our focus is to explore the potential of integrating enterprise applications such as Outlook/Exchange, ERP, CRM, directories, databases etc. with VoIP solutions to create new ways of "how things can be done". We achieve this by using intelligent APIs and standardized interfaces that most Voice-over-IP solutions offer today. "

[/Cisco] permanent link

2007 Apr 23 - Mon

Linux On Wall Street

In a news headline this morning, I came across the following. It is the 5th Annual Show & Conference for Linux/Open Source on Wall Street. Which happens to be taking place today. On the site, you'll find a few presentations from last year's show as an indication of what materials and thoughts are happening this year.

Now if SmartQuant QuantDeveloper was on a Linux platform, it would be a blazing fast trading environment.

[/Trading] permanent link

2007 Apr 22 - Sun

IQFeed Provider For SmartQuant QuantDeveloper: Updated

I have an original article which discusses a beta IQFeed Provider for SmartQuant's QuantDeveloper product.

I've fixed a few bugs in the provider. In addition, it should work with IQFeed's soon to be released data feeds which are stamped at the second resolution rather than minute resolution. The Provider is released in a zip file with an updated Ticker utility called: OneUnified.IQFeed.1.1.zip. The new IQFeed client is iqfeed_client_4_2_1_3.exe.

[/Trading/SmartQuant/Articles] permanent link

2007 Apr 20 - Fri

Running the Same Commands on Multiple Cisco Devices

Older switches don't have the nifty 'interface range' command for applying the same configuration to multiple interfaces at the same time. On older style 48 port switches, it can be a boring task to update all interfaces with, say, a new vlan assignment.

I was thinking of doing an Expect script to perform the task. I tackled it once upon a time, and did come up with a working example, but it took a while to get used to the nuances of the Expect language.

Having more skills in Perl, and realizing that there is a CPAN add-on for Cisco devices, I recently did something up in Perl. The example below connects to a switch, and for each of 48 interfaces, it defaults it and then applies a new vlan.

By creating an array of devices, and if they have common usernames and passwords, or are authenticated through TACACS, the same commands could be applied to a range of devices in one easy run.

use strict;

use Net::Telnet::Cisco;

my $postDevice = 'bmsw08';
my $postCommand = 'sho inter status';

  my $session = Net::Telnet::Cisco->new(
    Host => $postDevice
    );
#  $session->login( '', 'password' );
  $session->login( 'username', 'password' );
  $session->enable('enable');

  my @output;
#  my @output = $session->cmd(String => $postCommand );
#  print @output;

   print $session->cmd( String => 'config t' );

   for ( my $i = 1; $i <= 48; $i++ ) {
     print $session->cmd( String => "inter f0/$i" );
     print $session->cmd( String => "default desc" );
     print $session->cmd( String => "swi acc vlan 103" );
   }


  $session->close();

There are two types of logins, one with a username and password, and one with just a password. An Enable is used in either case.

[/Cisco] permanent link

2007 Apr 12 - Thu

SmartQuant QuantDeveloper & DataCenter Release

SmartQuant has released a revision to DataCenter and QuantDeveloper. DataCenter and QuantDeveloper are at the following revision levels:

DataCenter
Version 2.2.3 (12-Apr-2007) 

QuantDeveloper Enterprise Edition
Version 2.5.3 (12-Apr-2007) 

QuantDeveloper Source Code
Version 2.5.1 (23-Mar-2007) 
* Recent Versions available through 
  version control 

[/Trading/SmartQuant/Releases] permanent link

2007 Apr 06 - Fri

SmartQuant QuantDeveloper & DataCenter Release

SmartQuant has released a revision to DataCenter and QuantDeveloper. DataCenter and QuantDeveloper are at the following revision levels:

DataCenter
Version 2.2.2 (04-Apr-2007)

QuantDeveloper Enterprise Edition
Version 2.5.2 (04-Apr-2007) 

QuantDeveloper Source Code
Version 2.5.1 (23-Mar-2007) 
* Recent Versions available through 
  version control 

[/Trading/SmartQuant/Releases] permanent link

2007 Apr 05 - Thu

Cisco's Embedded Event Manager

For a long time, certain of Cisco's routers had the ability to run tcl scripts, particularily routers with voip loads, in order to perform credit card billings for voip connection validations. To me, the tcl scripting was a black art and not well documented.

That has changed. The capability has now become officially and publically supported. It is now known as the Cisco Embedded Event Manager. Part of it is integrated into SLA functionality. This, for example, provides the ability to adjust routing and other configuration details on the fly depending upon reachability testing. That one example barely covers the surface of what can be done to dynamically and automatically alter a router's functioning parameters.

That, in itself, is a two edged sword. But with today's interconnected, converged, flexible networks, sometimes one needs a trick like that to make things happen.

Some URL's:

• http://cisco.com/go/ciscobeyond
• http://cisco.com/go/eem
• http://forums.cisco.com/eforum/servlet/EEM?page=main

[/Cisco] permanent link

Trading Site of the Day -- Kirk Report: One Pro's View of the Stock Market

Charles E. Kirk is a trader who writes the The Kirk Report. The site is green, the color of money. On the serious side, he offers up daily market commentary on what has happened with the markets, what were the news affects, and offers up how his own portfolio is doing in the markets.

[/Trading/SiteOfTheDay/D200704] permanent link

2007 Apr 03 - Tue

Emulated Test Labs for Cisco Routing Gear

This is a two in one article actually. In a posting to today's cisco-nsp mailing list, Josh Horton provided a couple of links to his BlindHog Blog. His site has a wealth of good how-to articles for things routing and linux related. One entry that caught my eye was how to link router login AAA to a Microsoft IAS server. This basically does the job of a Tacacs server. Although it does the authentication and authorization, this Radius based authentication doesn't do the command logging that a Tacacs server can do. Even so, it is well worth a look. Other posts relate to ssh and some other router odds and sods.

The reason for his posting to the mail list was to advertise his wrapper for a Cisco router emulation engine at Dynagen. So for those studying for the CCNP exams and need a bunch of routers to check out OSPF, EIGRP, BGP and IGP, no need to spend thousands of dollars on real gear. Just get a recent IOS image and you can run multiple virtual routers all on your PC. The author says throughput is about 1kpps or so. But just to check out and test routing protocols with no sacrifice in command line options, this is an impressive capability.

[/Cisco] permanent link

2007 Apr 02 - Mon

Trading Sites of the Day -- ETFs, Day Trading, Short Time Periods

Yesterday I did a search with the keywords of 'active trader magazine scalp scalping'. The magazine, Active Trader Magazine, wasn't very high up in the results, mostly because they hide most of their content. I'm wondering if they would do better, not that I know if they are doing well or not, if they had a better indexable site. Even Stocks and Commodities Magazine has good teasers to get people interested in their content. But I digress.

Well, I'll stay digressed for a moment. In the recent issue of Stocks and Commodities, at the back of the issue, is an article describing the web site ETF Connect. I've linked to their education page. The web site is a good introductory source to Exchange Traded Funds, in their various incarnations.

Now that I've covered the interesting but unrelated, now I move back to the subject at hand, regarding short term trading, right down to scalping.

PowerZone Trading has a pdf called Trading with the High Performance Intraday Analysis Indicator Suite. The draw back with this document is that they couch all their setups in proprietary lingo. However, by reading between the terms, and reviewing the charts, there are some good points regarding trade setups that can be used. They allude to using multiple time-frames, which I wholly support. Near the end of the document, they go into some scalping setups that could prove to be useful. I'm in the process of writing an article in my Automated Trading Strategy section that goes more into what I've come to learn about scalping. To wrap up, the last page of the article references some of the author's other work in various periodicals. One ariticle that I do have is the August 2006 issue of Stocks and Commodities on page 34 regarding Intraday Market Forecasting. There is a good table in there regarding how to Guage Market Strength with INDU, COMPX, TRIN, and TICK. I think it is good enough to reproduce here because it solidifies what I've learned about these indicators elsewhere:

Market Direction	INDU	COMPX	TRIN	TICK
High Reversal Probability			0.55 or lower	1000 or higher
Extremely Bullish	above 75	above 20	0.70 to 0.55	600 to 1000
Bullish	35 to 75	15 to 20	0.85 to 0.70	300 to 600
Neutral	35 to -35	15 to -15	1.00 to 0.85	300 to -300
Bearish	-35 to -75	-15 to -20	1.15 to 1.00	-300 to -600
Extremely Bearish	-75 or lower	-20 or lower	1.30 to 1.15	-600 to -1000
High Reversal Probability			1.30 or higher	-1000 or lower

In an article at Stocks, Futures, and Options Magazine, there is an article called Precision Trading: Trading Shorter Time Frames Well Still Depends on Method and Psyche discusses various trading time frades, all the way from scalping, intraday trading, up to swing trading. It puts scalping into perspective as fast in out trades, almost without regards to trends and analysis of which other time periods make heavy use.

I've refered to the 1option site previously, but I've come across another article that serves to perpetuate some myths that appear to be prevalent about trading. There is an article called Can An Option Trader With A $100k Account Consistently Make $500 A Day?. In one portion of the article, the author makes mention of Warren Buffet, trading, and scalping in the same breath. I suppose among other things, Warren Bufffet traded. But I think for the most part, he specialized in buying under valued companies and turning them around. I suppose that's what you get when you trade enough stocks in a company to become a primary shareholder. So a trader in companies, maybe, to stretch the definition, or bend it somewhat. The author goes on to say that scalpers aren't rich. The counter argument for that would be BATS (the trading system the specializes in making big dollars in scalping), and Market Makers who make their money on spreads.

'Making their money on spreads'. I've finally clued in to that. Limit Orders and Spreads. Basically scalping. After two years of research, I think I've finally figured out how this stuff works, after spending a bunch of time spinning my wheels on bar analysis and technical analysis. So the thought is to make more than $500 a day on a $100,000 account. Yes it can be done. I'll go more into that in another article.

To finish off this page, I came across a Trading Tips page. At first, with their big Roman fonts and yellow highlighting, I didn't know how seriously to take this page. Well, I still take it with a grain of salt. But unlike most sites with yellow buy me now hilights, this actually offers up some value. There is an article called Making the Market. It, very cynically, but probably with much truth, discusses Market Makers and, as a side-benefit, discusses what scalping does and how to watch where it goes. Be aware that the article doesn't finish, but references a more complete article at traders101. And only on that link's last page does it go into self promotion. But you learn something along the way. And if you pay attention to pivots, support, resistance, and trading ranges, paying someone else for the service may not be necessary, other than to get you over the first few hurdles.

Also at TraderAide is an article called Markets in Profile. Coincidentily, I had made the purchase through Amazon before coming across that article. I'm not sure if the time frames discussed in the book cover the quick trades like scalping, but perhaps it might cover some of the things to keep a watch for in intraday type of trading.

I should have spread these treasures over several days, but I tend to loose my bookmarks if I don't deal with them right away. Happy reading.

[/Trading/SiteOfTheDay/D200704] permanent link

2007 Apr 01 - Sun

Stopping Screen Flicker in C#/.NET ListView Controls

The automated trading solution I'm building is starting out being a manual trading affair. I figure that once I can prove the algorithm manually, then I can probably cast the manual rules I use in stone and automate them. As such, I've been putting an user-interface into the software. Part of the interface uses a ListView. The only draw back to using this tool is the amount of flicker it presents.

A number of sites highly ranked on the search sites offer up solutions for resolving the Screen Flicker problem in C#/.NET v2.0 applications. None of them catch a key feature that makes it so simple (or at least it took me a whilei to catch on). Most of the solutions, such as the solution presented by c pound use the technique of inheriting from the basic ListView. Even that is a bit too much like work.

Franceso's Blog does a fairly large song and dance to resolve ListView screen flicker by manually copying stuff back and forth. Nope, that one isn't very good either.

An article called Painting Techniques using Windows Forms for the Microsoft .NET Framework shows up some good manaul ways of using double buffering to provide flicker free drawing. Another related background article is at Jeremy Kuhne's Blog. It offers up some good points, but still not the main point.

A related question at the The Scripts Developer Network was posted by a chap who, coincidentally, is doing some sort of stock ticker ap. The solution also references the technique of derivation and setting 'DoubleBuffered' to true. Nope, not quite what I was looking for. That article did point to a commercial grade ListView replacement called Glacial ListView. Looks good, but I'm looking for a free solution.

And the winner is: a page from the MSDN ControlStyles Enumeration. The secret ingredient for eliminating ListView Screen Flicker is to put the ControlStyle initialization flags in to the form's form_load override.

Here is a sample with all the test stuff in it:

    private void frmTradeFrame_Load( object sender, EventArgs e ) {

      this.SetStyle(
        //ControlStyles.UserPaint | 
        ControlStyles.AllPaintingInWmPaint |
        //ControlStyles.Opaque |
        ControlStyles.OptimizedDoubleBuffer,
        true);
      //this.DoubleBuffered = true;
      this.SetStyle(ControlStyles.EnableNotifyMessage, true);
      SetStyle(ControlStyles.ResizeRedraw, true);
    }

    protected override void OnNotifyMessage( Message m ) {
      if (m.Msg != 0x14) {
        base.OnNotifyMessage(m);
      }
    }

When I'm done, it should all boil down to this:

    private void frmTradeFrame_Load( object sender, EventArgs e ) {

      this.SetStyle(
        ControlStyles.AllPaintingInWmPaint |
        ControlStyles.OptimizedDoubleBuffer,
        true);
      SetStyle(ControlStyles.ResizeRedraw, true);
    }

Or maybe even this:

    private void frmTradeFrame_Load( object sender, EventArgs e ) {

      this.DoubleBuffered = true;
      }
    }

... since one of the documents I referenced indicated that DoubleBuffered sets those ControlStyles flags for you.

After all that ranting and raving, it didn't work. The following mods do. The key requirement is that you do have to override the ListView form, you can can't just override the containing form.

    public frmLV() {

      this.SetStyle(
        ControlStyles.AllPaintingInWmPaint |
        ControlStyles.Opaque |
        ControlStyles.OptimizedDoubleBuffer,
        true);
      this.SetStyle(ControlStyles.EnableNotifyMessage, true);
      SetStyle(ControlStyles.ResizeRedraw, true);
      SetStyle(ControlStyles.ResizeRedraw, true);
    }

    protected override void OnNotifyMessage( Message m ) {

      if (m.Msg == 0x14) m.Msg = 0x00;
      //if (m.Msg != 0x14) {
      base.OnNotifyMessage(m);
      //}
    }

The bonus feature that appears to reduce cpu utilization a bit is:

    // http://msdn.microsoft.com/msdnmag/issues/06/03/WindowsFormsPerformance/
   lvBook.Invalidate(lvi.GetBounds(ItemBoundsPortion.Entire),false);

If you've read this far, the simple solution to the problem, if you get to the low level code, according to a link at a Microsoft MSDN List is to "filter out the erase background message". Therefore, in the main draw code, one needs to paint the background text all in one fell swoop.

[/Trading/AutomatedTrading] permanent link

2007 Mar 30 - Fri

Fix in Autohandler for working with AuthNTLM

With testing at another customer site, I found that sometimes Apache2::AuthNTLM does not provide the username information when a web page has already been authenticated. Perhaps it might do it with every page if I turn off the 'cache' parameter in the return headers. Anyway, the new autohandler will rely on the session uuid to keep authorization going on pages which AuthNTLM does not return a username.

[/OpenSource/Debian/Monitoring] permanent link

2007 Mar 29 - Thu

Web Page Authorization with Perl and Mason

The monitoring server described in these various articles uses an Apache 2.0 based web server loaded with mod_perl and page templating language based upon mason. I'll put up on this site the other pages I have, but for the moment I just wanted to get the authentication stuff documented.

Authentication starts in /etc/apache2/sites-available/default with the following configuration:

        
                Options Indexes FollowSymLinks MultiViews
                AllowOverride None
                Order allow,deny
                allow from all

          PerlAuthenHandler OneUnified::AuthNTLM
          AuthType ntlm,basic
          AuthName domainname
          require valid-user

          PerlAddVar ntdomain "domainname globalcat1 globalcat2"

          PerlSetVar defaultdomain domainame
          PerlSetVar splitdomainprefix 1
          PerlSetVar ntlmdebug 0
          PerlSetVar ntlmauthoritative 0
#          PerlSetVar fallbackdomain fallbackdomain

        

In this configuration, you'll need to replace 'domainname'and 'globalcatx' with your netbios based domain name and global catalog server dns entries.

When running this security configuration, a web browser's security may need to be changed to make this a trusted site so that a user's ntlm security information will be exchanged.

AuthNTLM.pm is a wrapper around the CPAN perl module Apache2::AuthenNTLM. It pulls the username and domain information out and passes it along in the object's pnotes() method to the actual mason processing.

autohandler is mason's root file that is executed for every page serviced in a web site. As such, it can be used to perform page initialization, authentication, and authorization. It also holds the defaults for the pages of the site, and is used in an object oriented fashion to let other pages override the defaults. This particular one maintains session and user variables, and with some upcoming additions, will help to maintain session state. (You may need to use FireFox to download the file as Internet Explorer interprets the file for what it isn't.)

The autohandler file calls isADGroupMember.pm to perform the actual user and group authentication and authorization in Active Directory. The routine knows how to recursively search groups within groups.

The autohandler stores session variables in a PostgreSQL database with the following schema:

-- Table: users

-- DROP TABLE users;

CREATE TABLE users
(
  pkuserid character(36) NOT NULL, -- guid of user
  userdomain character varying(100) NOT NULL,
  authtype character varying(10) NOT NULL, -- ntlm, basic, ldap, db, builtin
  pwdclear character varying(20),
  pwdmd5 character varying(50),
  CONSTRAINT pk_users_pkuserid PRIMARY KEY (pkuserid)
)
WITHOUT OIDS;
ALTER TABLE users OWNER TO oneunified;
COMMENT ON TABLE users IS 'User list';
COMMENT ON COLUMN users.pkuserid IS 'guid of user';
COMMENT ON COLUMN users.authtype IS 'ntlm, basic, ldap, db, builtin';

-- Table: sessions

-- DROP TABLE sessions;

CREATE TABLE sessions
(
  pksessionid character(36) NOT NULL,
  fkuserid character(36) NOT NULL,
  ts timestamp without time zone NOT NULL,
  groupname character varying(50),
  grouppermission boolean,
  CONSTRAINT pk_sessions_pksessionid PRIMARY KEY (pksessionid),
  CONSTRAINT fk_users_pkuserid FOREIGN KEY (fkuserid)
      REFERENCES users (pkuserid) MATCH SIMPLE
      ON UPDATE NO ACTION ON DELETE NO ACTION
)
WITHOUT OIDS;
ALTER TABLE sessions OWNER TO oneunified;
COMMENT ON TABLE sessions IS 'Contains active web sessions.';

-- Insert these default entries
insert into users ( pkuserid, userdomain, authtype, pwdclear )
  values ( 'admin', 'local', 'builtin', 'admin' );
insert into users ( pkuserid, userdomain, authtype, pwdclear )
  values ( 'guest', 'local', 'builtin', 'guest' );

[/OpenSource/Debian/Monitoring] permanent link

2007 Mar 28 - Wed

Trading Site of the Day -- Some Random Picks

Here are some notes and web locations I've come across that appear to be relevant to Active Day Traders and Automated Trading Strategies.

I've recently opened an account with Genesis Securities and have downloaded their Laser trading software. There is a great deal of trading flexibility in that software. It also introduces some trading subtlties I've not dealt with before. There is flexibility in which markets to which orders can be sent, and the types of orders to be sent.

Routing Guide is a useful guide to ECN's, what markets they cover, how to grade with them, and a few helpful hints on order placement.

Electronic Communication Network (ECN) are important sources of liquidity.

Here are some more articles from Answers.com on Scalping and Day Trading.

A forum post at Elite Trader mentioned Bull Investors. They have some various live feeds and intra-day trading notes.

An older ebook on Nasdaq trading.

ShortSqueeze

[/Trading/SiteOfTheDay/D200703] permanent link

2007 Mar 27 - Tue

Aging Concepts in Genetic Programming

I've used some Genetic Programming in the past to assist with some trading optimizations. As part of calculating the population of a new generation, I've used a concept of elitism to keep the best individuals propogating to new generations. Some researchers seem to think this is a good thing, where others think this is a bad thing. As there was no real consensus about this, I chose the side saying this is a good thing. Now that I've heard about a concept of being able to age individuals, perhaps I can modify how I build new populations somewhat.

[/Trading/AutomatedTrading] permanent link

2007 Mar 26 - Mon

Installation and Configuration of PGAdmin on Debian

A multiplatform GUI tool called pgAdmin is available for managing and querying PostgreSQL databases. It is available at www.pgadmin.org.

For Windows platforms, an executable installation program is available.

For Debian platforms, don't install the package from the Debian web site. Instead, through a little config file magic, a Debian package direct from pgAdmin can be installed.

First, you'll need to determine your nearest mirror. Choose a mirror, and past it in the following format into /etc/apt/sources.list:

deb [MIRROR URL]/pgadmin3/release/debian testing pgadmin

You should then be able to run the following commands to get it installed:

apt-get update
apt-get install pgadmin3 

You can start pgadmin through the command pgadmin3. In order to manage the database, you'll have to login first as postgres, or try this simple expedient (warning this is not very secure for your database). Place the following line into /etc/postgresql/7.4/main/pg_hba.conf:

host    all         postgres    127.0.0.1         255.255.255.255   trust

In pgadmin, you'll need to connect to the database. You can connect to a specific database through existing credentials, or if you are looking to create new databases, you should connect via the above credentials and connect to template1.

[/OpenSource/Debian] permanent link

2007 Mar 23 - Fri

Trading Site of the Day -- Day Trader's Bulletin

Although Day Trader's Bulletin looks like it hasn't been updated in a while, it still contains useful and timely information. For example, in my own research, I've discovered the use of the INDU and the TICK as useful bellweather's for day trading. Knowing What Indicators to Monitor goes into additional details of these indicators plus adds a few more, such as: SPX (S&P 500 Cash), Down Transports (TRAN), selected stock issues, and intraday highs and intraday lows. The article goes on for a few additional pages in review of other things to keep an eye out for.

Besides that one article, there are many more interesting ones. If you'd like them all in one document, you can download Tips, Tricks & Techniques for Daytraders.

[/Trading/SiteOfTheDay/D200703] permanent link

Genesis Securities on Elite Trader Chat Room

There is a chat log on Elite TRader where Serge Pustelnik discusses some background on Genesis Securities, Program Trading, and software development. Scalping techniques for out-scalping the computer are also discussed.

[/Trading/AutomatedTrading] permanent link

2007 Mar 22 - Thu

Junk is stuff you throw away. Stuff is junk you keep.

Seen on GrokLaw.

[/Personal/TagLines] permanent link

2007 Mar 21 - Wed

Peak Selection, Pivot Analysis, and Fibonacci

Sometimes it takes a while to realize that how various things work together. The light has dawned. I wrote some code to do peak detection a while ago. By using this tool over multiple time frames, and associating Fibonacci intervals on the advancing lines, one can automatically search out possible turning points. A CBOT article called Trading the CBOT mini-sized Dow - Master Technician Techniques provides the ground work for how this would be done from an analytical perspective. These possible Fibonacci turning points can be strengthened if some of them overlap calculated pivot points. Further strengthening is achieved if Fibonacci turning points occur at the same location over multiple time intervals. As the author, John Person, says in the article, "Combining Pivot Point numbers with the Fibonacci analysis we have a much better outlook and confirmation of Support and Resistance targets."

CBOT has some links to additional strategies.

In an article at National Futures, writes an additional article on Using pivot points to predetermine support and resistance price levels. He ties them in with the TRIN, TICK, and VIX indicators along with a reference to candlesticks.

[/Trading/SiteOfTheDay/D200703] permanent link

2007 Mar 20 - Tue

Connecting to Debian X Windows with Cygwin/X

On some of the monitoring servers I install, KDE is installed and running. For remote management, it is nice to connect to it remotely via Windows XP host. To do so, I did the following:

• On the monitoring server, edited /etc/ssh/sshd_config, and changed X11Forwarding to yes. This is necessary for getting DISPLAY set properly.
• Reload sshd after change in setting: /etc/init.d/ssh restart
• Downloaded and installed Cygwin. During installation, I made sure that under the X11 heading, that the xorg-x11-base package was selected
• Started Cygwin with the desktop icon
• In my Cygwin home directory: cp /etc/X11/xinit/xinitrc .xinitrc
• In my Cygwin home directory, Started the xwindows process with: startx
• In the new GUI window, connect to the monitoring server with: ssh -Y -l username serverip
• It is possible to run 'startkde' but on my setup, currently, it doesn't appear to work quite right, but I can run individual XWindows based programs by starting them with their command line names

vBulletin was helpful in troubleshooting some connectivity issues.

Government Security has some more info on Cygwin, X11 Forwarding, and SSH. The site was either copied to or from a Linux Journal article.

For using vnc and securely remoting vnc, Debian Administration is helpful.

[/OpenSource/Debian] permanent link

2007 Mar 19 - Mon

Be a Price Maker, Not a Price Taker

Benson Shapiro wrote an article at the Harvard Business School called commodity Busters: Be a Price Maker, Not a Price Taker. In the article, he very elegantly writes about the importance of differentiating oneself from the competition. Rather than competing head to head, it is wise to pick a niche and become a monopolist in ones own right.

He provides a series of steps for doing it right:

• Create customer value
• Choose your customers
• Be different
• keep it simple
• Determine customer value
• Deliver on your promise
• Be courageous

[/Personal/Business] permanent link

Low Latency Data and Trades

SmartQuant was recently purchased by QuantHouse. Quanthouse provides low latency data and is using the SmartQuant framework in their product line.

That is all well and good, but if one has low latency data, one should have good executions as well, particularily if one is performing automated, high frequency trading. So what does one do: find a low latency data source and a separate excellent execution broker, or does one find a provider who can do both? I suppose there are organizations who only need the data side of things, but I'm looking for both data and execution.

I'm currently using IQFeed for data and Interactive Brokers for execution. I think I can do better than that. I'm currently looking at:

• Genesis Securities: appear strong on equities but weak on futures coverage, and have a QuantDeveloper API
• RedSky Financial: appear to be be strong on both futures and equities, but I don't think it has a QuantDeveloper API link (I'd have to write one)
• Photon Trader: strong on futures, and may have a QuantDeveloper API link

[/Trading/AutomatedTrading] permanent link

2007 Mar 18 - Sun

New Cisco Voip Wiki

cisco-voip.org is community based self-help Wiki for users, buyers, installers, and troubleshooters of Cisco Voice products.

[/Cisco] permanent link

2007 Mar 17 - Sat

Futures Trading

This is a short entry with three links:

• Forum focused on futures trading and eduation. The site is confusing as to what is advertising and what are menus. The advertising is at the top and sides, and the menu is at the bottom, for access to other locations of the site.
• The Art of Day-Trading. A series of cobbled together articles on day trading, with a futures bent.
• Interview with Paul Rotter. Probably the most interesting article of the blog 'I found my Holy Grail' is an interview with trader Paul Rotter and some background on scalping the markets.

[/Trading/SiteOfTheDay/D200703] permanent link

2007 Mar 16 - Fri

NetFilter Documentation, Tutorial and Pictures

The packet flow through netfilter has been a little hazy for me. Today I received enlightenment. Pablo Neira Ayuso has a paper called Towards 4th Generation Linux Firewalling Tools. On page 10 of that presentation is an excellent drawing of the iptables paths and what is performed in each path. It clearly shows PREROUTING, FORWARD, INPUT, OUTPUT, and POSTROUTING.

In the cross references on that page are a couple of good links:

• Packet handling in Linux Kernel 2.4
• ASCII chart comparing packet handling with ipchains (old) and iptables (new)
• Oskar Andreasson's Iptables Tutorial

While here, and on an unrelated topic, here is a Packet Shaping HOWTO. Now if I could just find a utility that can chart who is doing what with what protocol in real time. In isn't open source but later I did come across ObjectPlanent's Net Probe as something that could do the job.

Here are a few interesting commands to use when iptables is active:

• cat /proc/net/dev
• cat /proc/net/netstat
• cat /proc/net/ip_conntrack
• cat /proc/net/sockstat
• iptables --list -v

A paper called Netfilter Performance Testing is a good one which discusses the testing of netfilter and the various tools employed to do so.

The NetFilter site is at www.netfilter.org. For monitoring connections conntrack, ulogd2, and libnetfilter_conntrack are projects to look at.

Intellos Network has a souped up Conntrack Viewer for 2.4 kernels. I wander if it will work on 2.6 kernels.

Some background information on network accounting with netfilter and userspace utilities.

[/OpenSource] permanent link

2007 Mar 15 - Thu

Implementing Live URL Tracking

To monitor browsing habits of company users, and to use the information for possible site blocking or traffic limiting with MasterShaper, URLSnarf along with a Perl script can be used to log specific items to a database and place the full capture results into a log file. The log file will be rotated with logrotate.

The Perl module Logfile::Access is key. On my machine it needed to be manually installed:

perl -MCPAN -eshell
  install Logfile::Access 
  # a missing file will be encountered
  exit
cd ~/.cpan/build/Logfile-Access-1.30/Access.pm
nano Access.pm 
  # change line 38 to '/etc/mime.types'
make 
make install

[/OpenSource/Debian/Monitoring] permanent link

Darvas Selections for 2007/03/15

Here are the selections for Thursday. These lists have the Darvas stop calculated in the second column.

Start: 2007-02-14 High: 2007-03-09 EOD: 2007-03-14 Symbol Stop BIG 26.95 CYTC 32.58 DLTR 34.98 FAF 51.45 LBTYA 30.56 MAT 27.47 MOS 24.27 NRPH 63.39 RRI 17.74 SCI 12.10 TRA 15.88 VLO 59.67

Start: 2007-02-14 High: 2007-03-08 EOD: 2007-03-14 Symbol Stop AXL 25.99 DISH 43.44 DLTR 34.98 FAF 51.45 HSY 54.17 MAT 27.47 MHS 69.00 NRG 65.24 PSS 33.64 PWR 23.94 SCI 12.10 TRA 15.88 VLO 59.67

Start: 2007-02-14 High: 2007-03-07 EOD: 2007-03-14 Symbol Stop AXL 25.99 BRCD 9.47 DISH 43.44 DLTR 34.98 FAF 51.45 GSS 3.96 HSY 54.17 PLL 36.90 PWR 23.94 VLO 59.67

[/Trading/Darvas/D200703] permanent link

Darvas Results at EOD 2007/03/14 with EOD Signal of 2007/03/13

In comparison, the Dow Jones Industrial Index opened at 12074, had a higher high of 12142, had a lower low of 11939, and closed up for the day at 12133.

Symbol	#	Open	High	Low	Close	Stop	O->H	O->C
ARXX	1	13.03	13.05	12.95	12.99	13.09	0.02	-0.04
BRCD	1	9.50	9.85	9.48	9.80	9.36	0.35	0.30
CF	1	40.72	41.34	39.06	40.68	40.72	0.62	-0.04
DISH	2	42.71	42.78	41.87	42.50	43.44	0.07	-0.21
DLTR	2	35.76	35.85	35.05	35.56	34.98	0.09	-0.20
FAF	3	51.20	51.74	50.76	51.57	51.45	0.54	0.37
GSS	1	3.97	4.05	3.79	3.99	3.96	0.08	0.02
HSY	3	53.40	53.63	52.75	53.45	54.17	0.23	0.05
MAT	1	27.45	27.56	27.09	27.36	27.47	0.11	-0.09
MHS	2	68.65	69.36	67.67	68.92	69.00	0.71	0.27
NRG	1	68.30	69.22	67.86	69.21	65.24	0.92	0.91
PLL	1	37.27	37.71	36.61	37.21	36.90	0.44	-0.06
PSS	1	33.40	33.63	31.87	32.85	33.64	0.23	-0.55
PWR	2	24.01	24.43	23.56	24.32	23.94	0.42	0.31
SCI	1	12.10	12.17	11.87	11.96	12.10	0.07	-0.14
TRA	1	17.92	18.31	17.56	17.97	15.88	0.39	0.05
VLO	2	60.27	61.35	60.01	61.27	59.67	1.08	1.00
17		599.66					6.37	1.95

[/Trading/Darvas/D200703] permanent link

2007 Mar 14 - Wed

Sniffing, Security, Penetration Testing

For my reference, here is a compendium of interesting sites I encountered today:

• Linux Journal: Stealthful Sniffing, Intrusion Detection and Logging -- some discussion about hiding sniffers
• Linux Journal: In Search of a Sniffer -- some discussion about detecting sniffers and preventing network sniffing
• L0T3K: Sniffing Networks -- a page on a wide ranging web site about security issues
• Foogazi: The Best Linux Security Tools
• Sniffing: a wide ranging faq on capture and analsysis
• Vulnerability Assessment: Penetration Testing Framework document
• hping: packet generator and analyzer

[/OpenSource] permanent link

Tracking What Web Sites Users Visit

As part of the monitoring package, we are interested in recording which web sites that users are visiting. The first step is to capture the urls. The second step is to process and report the urls.

It took a while, but I came across DebianHelp. Among a bunch of other network diagnostic tools, it made mention of dsniff. It is a composition of the following tools:

• arpspoof - Send out unrequested (and possibly forged) arp replies.
• dnsspoof - forge replies to arbitrary DNS address / pointer queries on the Local Area Network.
• dsniff - password sniffer for several protocols.
• filesnarf - saves selected files sniffed from NFS traffic.
• macof - flood the local network with random MAC addresses.
• mailsnarf - sniffs mail on the LAN and stores it in mbox format.
• msgsnarf - record selected messages from different Instant Messengers.
• sshmitm - SSH monkey-in-the-middle. proxies and sniffs SSH traffic.
• sshow - SSH traffic analyser
• tcpkill - kills specified in-progress TCP connections.
• tcpnice - slow down specified TCP connections via "active" traffic shaping.
• urlsnarf - output selected URLs sniffed from HTTP traffic in CLF.
• webmitm - HTTP / HTTPS monkey-in-the-middle. transparently proxies.
• webspy - sends URLs sniffed from a client to your local browser.

I am now looking at various ways to process the result. I was thinking of manually using Logfile::Access to parse the lines and put stuff into a database. Then I got to looking around at log file analyzers like visitors or awstats, but they don't provide a breakdown of sites by user. I think I'll roll my own with the perl library already mentioned.

[/OpenSource/Debian/Monitoring] permanent link

Driftnet: Watching Your Surfers

This tool puts a network interface into promiscuous mode and looks for image files. Any image files found are displayed as a continuously changing mosaic on the desktop. It has the ability to listen a tond keep mpeg and sound files as well.

On my monitoring server, I have eth1 connected to a switch spanned port which in turn listens on the internal firewall interface. This captures all the media files that users are bringing in from the internet.

Installation is easy:

apt-get install driftnet

To run is even easier:

driftnet -i eth1

Original files are found at Driftnet. This sites has other interesting software such as proxies, mail impersonators, visual formatting, graphics, and more.

I found this little treasure through a where some other nifty monitoring tools can be found.

[/OpenSource/Debian/Monitoring] permanent link

iftop: Display Bandwidth Usage on an Interface

iftop is an interesting command line utility for tracking traffic on an interface. tcpdump is good for looking at packet content in real time, this is good for looking at what makes up traffic bandwidth in realtime.

Installation is easy:

apt-get install iftop 

I have second interface on the monitoring server which is listening to what ever traffic passes in and out of the internal interface of the firewall. The switch port connecting to the firewall is 'span'd to the monitoring server's eth1. This allows one to monitor all components of inbound and outbound traffic.

Here is the command I used for looking at generating a bar chart of traffic details:

iftop -i eth1 -F 10.0.0.0/255.0.0.0 -P

The -F parameter provides an indication of what is internal traffic and what is external traffic. The -P parameter shows the ports in each flow. For run-time commands, take a look at 'man iftop'.

[/OpenSource/Debian/Monitoring] permanent link

Darvas Selections for 2007/03/14

Here are the selections for Wednesday. These lists have the Darvas stop calculated in the second column.

Start: 2007-02-13 High: 2007-03-08 EOD: 2007-03-13 Symbol Stop CF 40.72 DISH 43.44 DLTR 34.98 FAF 51.45 HSY 54.17 MAT 27.47 MHS 69.00 NRG 65.24 PSS 33.64 PWR 23.94 SCI 12.10 TRA 15.88 VLO 59.67

Start: 2007-02-13 High: 2007-03-07 EOD: 2007-03-13 Symbol Stop ARXX 13.09 BRCD 9.36 DISH 43.44 DLTR 34.98 FAF 51.45 GSS 3.96 HSY 54.17 PLL 36.90 PWR 23.94 VLO 59.67

Start: 2007-02-13 High: 2007-03-06 EOD: 2007-03-13 Symbol Stop FAF 51.45 HSY 54.17 MHS 69.00

[/Trading/Darvas/D200703] permanent link

Darvas Results at EOD 2007/03/13 with EOD Signal of 2007/03/12

In comparison, the Dow Jones Industrial Index opened at 12307, had a lower high of 12307, had a lower low of 12071, and closed down for the day at 12075.

Symbol	#	Open	High	Low	Close	Stop	O->H	O->C
ARXX	2	13.08	13.14	13.02	13.02	12.99	0.06	-0.06
BRCD	2	9.61	9.94	9.54	9.55	9.36	0.33	-0.06
DISH	1	44.10	44.26	42.90	42.93	43.44	0.16	-1.17
DLTR	1	36.82	36.82	35.82	35.84	34.98	0.00	-0.98
FAF	2	51.99	52.01	51.15	51.58	51.45	0.02	-0.41
GSS	1	4.25	4.30	4.00	4.05	3.96	0.05	-0.20
HSY	2	54.45	54.58	53.39	53.41	54.17	0.13	-1.04
MHS	1	69.50	69.68	68.26	68.39	69.00	0.18	-1.11
PLL	1	37.73	37.85	37.11	37.14	36.90	0.12	-0.59
PWR	1	24.22	24.30	23.87	23.95	23.94	0.08	-0.27
VLO	1	60.48	61.47	59.85	60.19	59.67	0.99	-0.29
11		369.41					2.12	-6.18

[/Trading/Darvas/D200703] permanent link

2007 Mar 13 - Tue

Darvas Selections for 2007/03/13

Here are the selections for Tuesday. These lists have the Darvas stop calculated in the second column.

Start: 2007-02-12 High: 2007-03-07 EOD: 2007-03-12 Symbol Stop ARXX 12.99 BRCD 9.36 DISH 43.44 DLTR 34.98 FAF 51.45 GSS 3.96 HSY 54.17 PLL 36.90 PWR 23.94 VLO 59.67

Start: 2007-02-12 High: 2007-03-06 EOD: 2007-03-12 Symbol Stop FAF 51.45 HSY 54.17 MHS 69.00

Start: 2007-02-12 High: 2007-03-05 EOD: 2007-03-12 Symbol Stop ARXX 12.99 BRCD 9.36

[/Trading/Darvas/D200703] permanent link

Darvas Results at EOD 2007/03/12 with EOD Signal of 2007/03/09

In comparison, the Dow Jones Industrial Index opened at 12275, had a higher high of 12349, had a higher low of 12246, and closed up for the day at 12318.

The equity curve for today's Darvas ended up down rather than up, even though the Dow was up marginally.

Symbol	#	Open	High	Low	Close	Stop	O->H	O->C
ARXX	1	13.07	13.15	13.07	13.07	13.07	0.08	0.00
BRCD	1	9.55	9.77	9.50	9.67	9.36	0.22	0.12
CECO	1	29.75	30.29	29.70	30.00	29.85	0.54	0.25
FAF	1	51.98	52.48	51.76	52.16	51.45	0.50	0.18
GT	3	28.70	29.25	28.56	28.98	27.90	0.55	0.28
HLIT	1	10.32	11.07	10.32	11.02	10.05	0.75	0.70
HSY	1	54.57	54.80	54.30	54.51	54.17	0.23	-0.06
MHS	1	68.28	70.18	68.28	69.70	57.93	1.90	1.42
OSI	1	39.76	39.89	39.65	39.73	40.08	0.13	-0.03
PALM	1	17.10	17.41	16.88	17.31	16.55	0.31	0.21
10		323.08					5.21	3.07

[/Trading/Darvas/D200703] permanent link

2007 Mar 12 - Mon

Trading Site of the Day -- C(omp)++ : A Portal for the Computational Finance Community

John Wiley and Sons, the book publishers, have a 'Collective Knowledge Portal' called C(comp)++.

This is a relatively new site with content just getting started. So far, content includes a discussion learning C++, some comments relating C++ and C#, some background on estimation, plus recent remarks on confidence intervals (or standard deviation of a standard deviation).

[/Trading/SiteOfTheDay/D200703] permanent link

Darvas Selections for 2007/03/12

Here are the selections for Monday. These lists have the Darvas stop calculated in the second column.

Start: 2007-02-09 High: 2007-03-06 EOD: 2007-03-09 Symbol Stop FAF 51.45 GT 27.90 HSY 54.17 MHS 57.93

Start: 2007-02-09 High: 2007-03-05 EOD: 2007-03-09 Symbol Stop ARXX 13.07 BRCD 9.36 GT 27.90

Start: 2007-02-09 High: 2007-03-02 EOD: 2007-03-09 Symbol Stop CECO 29.85 GT 27.90 HLIT 10.05 OSI 40.08 PALM 16.55

[/Trading/Darvas/D200703] permanent link

Darvas Results at EOD 2007/03/09 with EOD Signal of 2007/03/08

In comparison, the Dow Jones Industrial Index opened at 12262, had a higher high of 12330, had a higher low of 12228, and closed up for the day at 12276.

The equity curve for today's Darvas ended up down rather than up, even though the Dow was up marginally. For my own trading, I added Thursday's symbols to Friday's symbols. ON that group, overall, I ended up even for the day. I did see some opportunities to take some profit, but left things in place instead. I think, on this set of stocks, day trading might be more profitable: getting out when some good dollars have accumulated, waiting for the Dow to find a low, and then getting in again.

Symbol	#	Open	High	Low	Close	Stop	O->H	O->C
ARXX	1	13.16	13.20	13.04	13.05	13.07	0.04	-0.11
BRCD	1	9.72	9.75	9.47	9.56	9.36	0.03	-0.16
CECO	1	29.99	30.00	29.61	29.71	29.85	0.01	-0.28
GT	3	29.27	29.27	28.23	28.51	27.90	0.00	-0.76
HLIT	2	10.58	10.70	10.15	10.36	10.07	0.12	-0.22
HYSL	1	51.60	51.67	51.58	51.59	51.50	0.07	-0.01
OSI	1	39.94	39.94	39.65	39.83	40.08	0.00	-0.11
PALM	1	17.21	17.51	16.83	17.12	16.55	0.30	-0.09
STTS	1	11.86	11.94	11.85	11.91	11.59	0.08	0.05
9		144.12					0.65	-1.69

[/Trading/Darvas/D200703] permanent link

Trading Site of the Day -- Trading Machines: Electronic and Automated Trading

Trading Machines has a directory of electronic and automated trading links. The topics covered include:

• Brokers (which have trading API's)
• Forums
• Coding Sites
• Data Sources
• Data Sources
• FIX
• Automation Software
• Analysis Software
• Software Components
• Maths and Models (excellent sites here)
• Risk and Reliability

[/Trading/SiteOfTheDay/D200703] permanent link

2007 Mar 10 - Sat

Trading Site of the Day -- Trader's Roundtable: A forum for mechanical system traders

Trading Blox hosts the Trader's Roundtable which is a forum devoted to the discussion of mechanical trading systems and related topics.

I consider it a good source of vendor independent trading methods and concepts. Participants do talk about specific vendors though.

The site is rich in content. One just has to look for it. The site has topics covering things like:

• Market Psychology
• Trader Psychology
• Testing and Simulation
• Money Management
• Trend Indicators and Signals
• Commercial and Public Domain Systems
• Turtle Trading
• Short term and Day Trading Systems
• Books, Courses, and Seminars
• Pro Trading
• Trading for a Living
• Software
• Brokers and Markets

[/Trading/SiteOfTheDay/D200703] permanent link

2007 Mar 09 - Fri

Darvas Selections for 2007/03/09

Here are the selections for Friday. These lists have the Darvas stop calculated in the second column.

Start: 2007-02-08 High: 2007-03-05 EOD: 2007-03-08 Symbol Stop ARXX 13.07 BRCD 9.36 GT 27.90

Start: 2007-02-08 High: 2007-03-02 EOD: 2007-03-08 Symbol Stop CECO 29.85 GT 27.90 HLIT 10.07 OSI 40.08 PALM 16.55

Start: 2007-02-08 High: 2007-03-01 EOD: 2007-03-08 Symbol Stop GT 27.90 HLIT 10.07 HYSL 51.50 STTS 11.59

[/Trading/Darvas/D200703] permanent link

Darvas Results at EOD 2007/03/08 with EOD Signal of 2007/03/07

In comparison, the Dow Jones Industrial Index opened at 12193, had a higher high of 12303, had a higher low of 12193, and closed up for the day at 12260.

The equity curve for Darvas follows the Dow directionally quite closely.

Symbol	#	Open	High	Low	Close	Stop	O->H	O->C
ABT	1	52.69	53.57	52.38	53.40	54.28	0.88	0.71
AKS	1	22.68	22.98	22.26	22.43	22.03	0.30	-0.25
CECO	1	29.95	30.19	29.66	29.68	29.85	0.24	-0.27
DF	1	46.30	46.50	46.02	46.33	46.03	0.20	0.03
GT	2	29.11	29.61	28.70	28.86	27.90	0.50	-0.25
HLIT	2	10.00	10.50	10.00	10.43	9.36	0.50	0.43
HSP	1	39.99	40.25	39.84	40.21	38.41	0.26	0.22
HYSL	1	51.55	51.59	51.52	51.54	51.50	0.04	-0.01
NXL	1	33.25	33.34	33.23	33.30	33.10	0.09	0.05
OSI	1	39.95	39.98	39.60	39.75	40.08	0.03	-0.20
PALM	2	17.43	17.64	17.12	17.15	16.55	0.21	-0.28
STTS	1	11.88	11.92	11.88	11.90	10.65	0.04	0.02
12		384.78					3.29	0.20

[/Trading/Darvas/D200703] permanent link

2007 Mar 08 - Thu

Perl Based Active Directory Authentication and Authorization

Most of the customers in which I install the monitoring server have an Active Directory based organization. As such, the best mechanism for authenticating users onto the web pages of the monitoring server is to check that they are valid members of an active directory domain. Authorization to certain specific pages of the monitoring server can then be authorized through group membership.

isADGroupMember.pm is a Perl Module designed to authorize a user through Active Directory group membership. It performs a recursive group lookup to ensure the user is somewhere in the hierarchy. It returns a 1 if the user is found in a group, and 0 if no match is found.

You'll need to update four search parameters at the top of the module in order to work with your Active Directory Organization. There is a base search path to act as the root of the search. A user name and password are required for gaining access to Active Directory. This will require appropriate rights to perform it search. There is a bind address used for connecting to a global catalog server, which can be an ip address or a fqdn.

The user name will be a SAMAccount formatted name. The same for the group name. Each are expanded out to their respective DistinguishedNames, which are then used for matching purposes.

The AuthNTLM module can be used for automatically supplying the user name used in the lookup. In the solution I'm building, AuthNTLM will take the user name it automatically obtained from the browser session and place it into a session record in a database. Mason's autohandler will then pull out the username when it pulls session information and then use isADGroupMember to authorize access to any given web page.

[/OpenSource/Debian/Monitoring] permanent link

Darvas Selections for 2007/03/08

Here are the selections for Thursday. These lists have the Darvas stop calculated in the second column.

Start: 2007-02-07 High: 2007-03-02 EOD: 2007-03-07 Symbol Stop CECO 29.85 DF 46.03 GT 27.90 HLIT 9.36 OSI 40.08 PALM 16.55

Start: 2007-02-07 High: 2007-03-01 EOD: 2007-03-07 Symbol Stop GT 27.90 HLIT 9.36 HYSL 51.50 STTS 10.65

Start: 2007-02-07 High: 2007-02-28 EOD: 2007-03-07 Symbol Stop ABT 54.28 AKS 22.03 HSP 38.41 NXL 33.10 PALM 16.55

[/Trading/Darvas/D200703] permanent link

Darvas Results at EOD 2007/03/07 with EOD Signal of 2007/03/06

In comparison, the Dow Jones Industrial Index opened at 12204, had a higher high of 12256, had a higher low of 12178, and closed down for the day at 12192.

The equity curve for Darvas follows the Dow directionally quite closely.

Symbol	#	Open	High	Low	Close	Stop	O->H	O->C
ABT	2	52.65	52.99	52.55	52.69	54.28	0.34	0.04
AKS	1	22.23	22.74	22.05	22.16	22.03	0.51	-0.07
BRCD	1	9.77	9.96	9.58	9.60	9.36	0.19	-0.17
CECO	1	30.12	30.24	29.91	29.96	29.85	0.12	-0.16
CMS	1	17.13	17.40	17.07	17.18	17.02	0.27	0.05
GT	1	28.40	28.88	28.30	28.48	27.90	0.48	0.08
HLIT	1	9.62	9.91	9.58	9.86	9.36	0.29	0.24
HSP	1	38.95	39.92	38.75	39.90	38.41	0.97	0.95
HYSL	1	51.52	51.56	51.51	51.55	48.90	0.04	0.03
NXL	1	33.19	33.19	33.14	33.15	33.10	0.00	-0.04
PALM	1	17.16	17.50	17.08	17.27	16.55	0.34	0.11
RSH	1	25.44	25.76	25.29	25.46	24.80	0.32	0.02
STTS	1	11.82	11.90	11.80	11.89	10.65	0.08	0.07
WY	1	84.50	84.96	83.10	84.15	83.50	0.46	-0.35
14		399.31					4.41	0.80

[/Trading/Darvas/D200703] permanent link

2007 Mar 07 - Wed

Darvas Selections for 2007/03/0

Here are the selections for Wednesday. These lists have the Darvas stop calculated in the second column.

Start: 2007-02-06 High: 2007-03-01 EOD: 2007-03-06 Symbol Stop GT 27.90 HLIT 9.36 HYSL 48.90 STTS 10.65

Start: 2007-02-06 High: 2007-02-28 EOD: 2007-03-06 Symbol Stop ABT 54.28 AKS 22.03 HSP 38.41 NXL 33.10 PALM 16.55

Start: 2007-02-06 High: 2007-02-27 EOD: 2007-03-06 Symbol Stop ABT 54.28 BRCD 9.36 CECO 29.85 CMS 17.02 RSH 24.80 WY 83.50

[/Trading/Darvas/D200703] permanent link

Darvas Results at EOD 2007/03/06 with EOD Signal of 2007/03/05

In comparison, the Dow Jones Industrial Index opened at 12051, had a higher high of 12224, had a higher low of 12050, and closed up for the day at 12207.

The equity curve for Darvas follows the Dow directionally quite closely.

Symbol	#	Open	High	Low	Close	Stop	O->H	O->C
ABT	3	53.00	53.08	52.37	52.87	54.28	0.08	-0.13
ADP	1	48.72	48.94	48.38	48.80	50.33	0.22	0.08
AEP	1	44.70	45.50	44.38	45.33	45.37	0.80	0.63
AHG	1	30.55	31.00	30.01	30.59	31.11	0.45	0.04
AKS	1	21.79	22.41	21.60	22.21	22.03	0.62	0.42
AMCC	1	3.60	3.73	3.58	3.67	3.68	0.13	0.07
AXL	1	23.75	24.37	23.47	24.25	25.82	0.62	0.50
BDN	1	34.32	34.78	34.10	34.36	34.02	0.46	0.04
BRCD	1	9.63	9.83	9.53	9.80	9.36	0.20	0.17
CECO	1	29.78	30.23	29.41	30.09	29.85	0.45	0.31
CEG	1	76.75	78.21	76.40	78.21	77.42	1.46	1.46
CERN	1	51.81	52.20	51.30	52.12	53.55	0.39	0.31
CMS	2	16.86	17.25	16.82	17.20	17.02	0.39	0.34
CNP	1	17.09	17.50	17.05	17.41	18.31	0.41	0.32
CTV	1	37.70	39.00	37.28	38.71	36.91	1.30	1.01
D	1	84.12	84.75	83.99	84.68	84.70	0.63	0.56
DYN	1	8.01	8.23	7.92	8.23	7.75	0.22	0.22
ED	1	47.73	48.30	47.46	48.17	49.25	0.57	0.44
EIX	1	47.73	48.67	47.60	48.37	46.19	0.94	0.64
EQ	1	51.47	52.51	50.95	52.35	57.05	1.04	0.88
ESLR	1	9.39	9.82	9.39	9.79	10.68	0.43	0.40
FE	1	61.30	62.18	61.26	61.95	64.35	0.88	0.65
GILD	1	70.35	71.22	69.59	70.72	73.25	0.87	0.37
HES	1	51.00	51.57	50.60	51.50	55.06	0.57	0.50
HSP	1	38.96	39.45	38.80	38.95	38.45	0.49	-0.01
IR	1	42.48	43.20	42.32	42.99	43.81	0.72	0.51
KLAC	1	51.73	51.74	50.80	51.15	53.65	0.01	-0.58
LLTC	1	32.20	32.43	31.93	32.33	33.64	0.23	0.13
NBL	1	57.17	58.03	57.04	58.02	57.06	0.86	0.85
NRG	1	67.25	68.41	67.06	67.93	65.79	1.16	0.68
NSM	1	25.17	25.37	24.83	24.95	25.58	0.20	-0.22
NTES	1	19.41	19.89	19.38	19.84	20.60	0.48	0.43
NVTL	2	13.47	13.99	13.47	13.97	12.80	0.52	0.50
NXL	1	33.13	33.23	33.12	33.14	32.14	0.10	0.01
ONNN	1	9.50	9.78	9.40	9.75	10.68	0.28	0.25
PAAS	1	27.10	27.98	27.03	27.87	31.52	0.88	0.77
PALM	1	16.90	17.39	16.85	17.31	16.55	0.49	0.41
PAYX	1	39.42	40.08	39.33	39.84	41.93	0.66	0.42
PD	1	123.02	124.59	123.02	124.38	126.95	1.57	1.36
PPL	1	37.35	37.91	37.26	37.86	38.17	0.56	0.51
RIMM	1	137.10	137.74	135.23	135.60	139.74	0.64	-1.50
RRC	1	30.25	30.60	29.88	30.25	31.25	0.35	0.00
RRI	1	17.09	17.16	16.93	16.96	17.06	0.07	-0.13
RSH	1	24.78	25.57	24.53	25.45	24.80	0.79	0.67
SONS	1	6.89	7.09	6.79	6.98	7.23	0.20	0.09
STM	1	18.84	18.89	18.61	18.79	19.92	0.05	-0.05
STP	1	35.08	35.92	34.87	35.76	36.70	0.84	0.68
USG	1	52.04	52.55	51.11	51.71	57.74	0.51	-0.33
USU	1	13.51	14.36	13.49	14.25	14.98	0.85	0.74
VSEA	1	46.84	48.18	46.79	47.70	47.22	1.34	0.86
WY	2	85.30	86.65	85.01	86.20	83.50	1.35	0.90
WYN	1	34.07	34.59	33.96	34.38	35.10	0.52	0.31
XLU	1	37.82	38.13	37.67	38.13	39.17	0.31	0.31
53		2105.02					31.16	18.80

[/Trading/Darvas/D200703] permanent link

2007 Mar 06 - Tue

Darvas Selections for 2007/03/06

Here are the selections for Tuesday. These lists have the Darvas stop calculated in the second column.

Start: 2007-02-05 High: 2007-02-28 EOD: 2007-03-05 Symbol Stop ABT 54.28 AKS 22.03 HSP 38.45 NVTL 12.80 NXL 32.14 PALM 16.55

Start: 2007-02-05 High: 2007-02-27 EOD: 2007-03-05 Symbol Stop ABT 54.28 BRCD 9.36 CECO 29.85 CMS 17.02 RSH 24.80 WY 83.50

Start: 2007-02-05 High: 2007-02-26 EOD: 2007-03-05 Symbol Stop ABT 54.28 ADP 50.33 AEP 45.37 AHG 31.11 AMCC 3.68 AXL 25.82 BDN 34.02 BGO 6.20 CEG 77.42 CERN 53.55 CMS 17.02 CNP 18.31 CTV 36.91 D 84.70 DYN 7.75 ED 49.25 EIX 46.19 EQ 57.05 ESLR 10.68 FE 64.35 GILD 73.25 HES 55.06 IR 43.81 KLAC 53.65 LLTC 33.64 NBL 57.06 NRG 65.79 NSM 25.58 NTES 20.60 NVTL 12.80 ONNN 10.68 PAAS 31.52 PAYX 41.93 PD 126.95 PPL 38.17 RIMM 139.74 RRC 31.25 RRI 17.06 SONS 7.23 STM 19.92 STP 36.70 USG 57.74 USU 14.98 VSEA 47.22 WY 83.50 WYN 35.10 XLU 39.17

[/Trading/Darvas/D200703] permanent link

Darvas Results at EOD 2007/03/05 with EOD Signal of 2007/03/02

In comparison, the Dow Jones Industrial Index opened at 12111, had a lower high of 12188, had a lower low of 12039, and closed down for the day at 12050.

The Dow did a fine job of going up and down over the course of the day. It did finally end down for the day.

The results, as shown below, indicated a slightly negative exit for the day. The equity curve followed the Dow reasonably well. So, if we had mostly negative sentiment for the day, getting out around 10:20 or 11:20 would have been good calls for a positive return.

Symbol	#	Open	High	Low	Close	Stop	O->H	O->C
ABT	2	52.80	53.59	52.72	52.75	54.28	0.79	-0.05
ADI	1	35.00	35.99	34.91	35.10	36.64	0.99	0.10
ADP	1	48.00	48.89	47.80	48.47	50.33	0.89	0.47
AEP	1	44.87	44.95	44.28	44.30	45.37	0.08	-0.57
ALTR	1	20.11	20.39	19.87	19.87	21.48	0.28	-0.24
AMCC	1	3.57	3.64	3.48	3.53	3.68	0.07	-0.04
ATI	1	95.50	97.42	94.37	94.44	104.00	1.92	-1.06
AUY	1	13.24	13.52	12.60	12.86	15.25	0.28	-0.38
AXL	2	23.38	23.90	23.13	23.58	25.82	0.52	0.20
BBBY	1	39.34	39.50	39.05	39.13	42.34	0.16	-0.21
BBI	1	6.45	6.55	6.27	6.30	6.91	0.10	-0.15
BDN	1	34.42	34.57	33.53	33.53	34.02	0.15	-0.89
BRCD	1	8.94	9.95	8.84	9.44	8.83	1.01	0.50
CECO	2	29.99	30.31	29.40	29.73	29.85	0.32	-0.26
CEG	1	77.56	77.85	76.51	76.51	77.42	0.29	-1.05
CERN	2	50.34	51.87	49.75	51.09	53.28	1.53	0.75
CF	1	35.99	38.31	35.02	37.20	38.11	2.32	1.21
CMS	2	16.85	17.17	16.81	16.86	17.02	0.32	0.01
CNP	1	17.40	17.40	17.07	17.09	18.31	0.00	-0.31
D	1	84.73	85.00	83.71	83.82	84.70	0.27	-0.91
ED	2	47.94	48.01	47.47	47.56	49.25	0.07	-0.38
EQ	2	53.39	53.39	51.00	51.36	57.05	0.00	-2.03
ESLR	1	9.12	9.40	9.09	9.26	10.68	0.28	0.14
FE	1	61.20	62.07	60.85	61.20	64.35	0.87	0.00
GILD	1	69.75	71.41	69.33	69.99	73.25	1.66	0.24
HES	2	49.50	51.15	49.05	50.52	55.06	1.65	1.02
HL	1	7.45	7.68	7.18	7.44	8.40	0.23	-0.01
IR	2	41.76	42.75	41.66	42.18	44.00	0.99	0.42
JOYG	1	41.53	42.34	40.36	41.43	52.94	0.81	-0.10
KLAC	2	49.88	51.98	49.75	51.13	53.65	2.10	1.25
LLTC	2	31.94	32.47	31.71	31.75	33.64	0.53	-0.19
LYO	1	30.43	31.04	30.41	30.48	32.20	0.61	0.05
NBL	2	56.20	57.08	56.04	56.71	57.06	0.88	0.51
NRG	2	67.20	68.06	66.62	67.01	65.79	0.86	-0.19
NSM	1	24.60	25.29	24.40	24.89	25.58	0.69	0.29
NTES	1	19.75	19.76	19.06	19.13	20.60	0.01	-0.62
NVTL	2	13.19	13.46	13.07	13.33	12.80	0.27	0.14
NYB	1	16.54	16.76	16.50	16.59	17.35	0.22	0.05
ONNN	2	9.38	9.74	9.19	9.21	10.46	0.36	-0.17
PAAS	2	26.16	27.56	26.00	26.43	31.52	1.40	0.27
PAYX	2	39.15	39.87	38.93	39.50	41.93	0.72	0.35
PD	2	122.43	123.47	121.75	122.29	126.95	1.04	-0.14
PENN	1	45.02	46.07	44.66	44.66	44.24	1.05	-0.36
PPL	2	37.57	37.75	37.19	37.23	38.17	0.18	-0.34
PWR	2	22.01	22.40	21.65	21.67	23.31	0.39	-0.34
RIMM	1	131.70	136.44	131.18	134.52	139.74	4.74	2.82
RRC	2	30.57	30.95	29.76	29.76	31.25	0.38	-0.81
RRI	1	17.08	17.20	16.86	16.87	17.06	0.12	-0.21
RSH	1	24.35	24.70	24.04	24.42	24.80	0.35	0.07
SIRF	1	27.32	28.30	27.10	27.43	31.70	0.98	0.11
SONS	1	7.16	7.18	6.63	6.67	7.23	0.02	-0.49
STM	2	18.42	18.79	18.41	18.49	19.92	0.37	0.07
STP	1	32.45	34.44	32.45	33.88	36.70	1.99	1.43
TSM	1	9.90	10.46	9.80	10.11	11.30	0.56	0.21
USG	1	52.12	52.55	51.33	51.48	57.74	0.43	-0.64
USU	2	13.40	13.85	13.14	13.24	14.98	0.45	-0.16
VSEA	1	45.42	47.37	45.20	46.20	47.22	1.95	0.78
WYN	2	34.45	34.50	33.82	33.86	35.10	0.05	-0.59
XLU	2	37.91	38.21	37.47	37.57	39.17	0.30	-0.34
59		2143.03					42.85	-0.77