2008 Feb 19 - Tue

Installing Asterisk

Here are a few installation updates to my other installation article. Here are some revised subversion retrieval commands:

svn checkout http://svn.digium.com/svn/zaptel/branches/1.4 zaptel
svn checkout http://svn.digium.com/svn/libpri/trunk libpri
svn checkout http://svn.digium.com/svn/asterisk/trunk asterisk

The make results indicate that the trunk version of zaptel is unstable, so a 1.4 is the most recent stable version available.

make, make install, make samples, doxygen, make progdocs

Some additional, older notes can be found at AsteriskGuru.

[/OpenSource/Debian/Asterisk] permanent link

Upgrading PostgreSQL 7.x to 8.x

Upgrading a PostgreSQL server version 7.x to version 8.x was relatively painless. It helped that there wasn't a significant amount of data to move, or blob objects to move.

For a Debian server on which I recently ran 'apt-get dist-upgrade', the Debian crew made PostgreSQL 8.2 run along side version 7.4. The /etc/postgresql/8.2/main/postgresql.conf file showed the server port as being 5433, by default. Since the service wasn't turned on, I set it for 5432, and changed the listen_addresses for '*'.

Relevant lines from the pg_hba.conf files will have to coped from the /etc/postgresql/7.x/main/ directory to the matching file in the 8.x directory. Some say that running pgadmin3 will provide messages indicating exactly which paramters need to be copied over. I havn't tried that.

The commands following are used after 'su - postgres' (logging in as the postgreSQL user account. There are a number of ways of performing backups and restores. The following sequence of steps takes a little longer, but I wield a bit more control over what gets done when.

With 7.4 running and 8.2 off, I did a 'pg_dumpall --schema-only > db.dump'. The top of the file shows a number of 'create role' lines for regenerating users.

Do a 'pg_dump databasename > databasename.db.dump' for each database (where you substitute the real name for databasename) to extract the schema and data. A 'psql -l' will list the databases available.

Use '/etc/init.d/postgresql-7.4 stop' to stop the existing service. Use 'apt-get remove xxx' to remove the various old version packages. 'apt-file -l list postg' will provide an indication of what can be removed (I'm sure there is a better way though).

Use '/etc/init.d/postgresql-8.2 start' to start the more recent version of PostgreSQL.

Use 'psql template1' to connect to the server. Run the 'create role' lines that you see from the pg_dumpall command run earlier. '\q' to exit.

Use 'createdb -T template0 --owner=ownername databasename' to create each of the databases.

Use 'psql databasename < databasename.db.dump to restore the data.

Data and schema are now ready to be used as if nothing happened.

[/OpenSource/Debian] permanent link

2008 Feb 18 - Mon

Redirect STL cout

In a previous article entitled C++ Override std::cout, std::cerr Streams, I wrote about some sites I found regarding the redirection of cout to some user supplied routine. After some fiddling about, I came up with a result that works for me in Visual Studio 2005 version of C++.

Many of the sites suggest overriding the xsputn function. I did that in conjunction with buffered output through the setp function. I found that the xsputn function is used for string delivery, but the user supplied buffer is used when cout formats binary values. I had to come up with a mechanism to sync the two. The solution was to not over-ride xsputn, only use the setp function, and rely on overriding the sync function.

The code in the sync override isn't perfect, but it does get the job done. The code makes use of the fastdelegate template to issue a 'callback' to code interested in processing the buffer on each sync. The short coming with this code is that cout inserts a 0x0a into the buffer for each endl, and so the routine accepting the buffer has to scan and interpret the character appropriately.

By not using setp, the routine becomes unbuffered, and then xsputn becomes necessary. I havn't tried that scenario yet.

#pragma once

#include <iostream>
using namespace std;

#include "FastDelegate.h"
using namespace fastdelegate;

class CConsoleStream :  public streambuf {

public:
  CConsoleStream(void);
  virtual ~CConsoleStream(void);

  typedef FastDelegate2<const char*, streamsize> OnNewStringHandler;
  void SetOnNewString( OnNewStringHandler function ) {

    OnNewString = function;
  }
  typedef FastDelegate0<> OnFlushStringHandler;
  void SetOnFlushString( OnFlushStringHandler function ) {

    OnFlushString = function;
  }
protected:
  OnNewStringHandler OnNewString;
  OnFlushStringHandler OnFlushString;

  static const unsigned short BufSize = 1024;
  char buf[ BufSize ]; // arbitrary length sized to get most console length stuff

  //virtual streamsize xsputn( const char_type* s, streamsize n );

  virtual int sync( void );
  virtual int_type overflow( int_type meta );

private:
};

#include "StdAfx.h"
#include "ConsoleStream.h"
#include <stdexcept>

CConsoleStream::CConsoleStream(void) {
  // http://www.cplusplus.com/reference/iostream/streambuf/setp.html
  // http://blogs.awesomeplay.com/elanthis/archives/2007/12/10/444/

  setp( buf, buf + BufSize );
}

CConsoleStream::~CConsoleStream(void) {
}

//streamsize CConsoleStream::xsputn (const char_type* s, streamsize n) {
//  if ( NULL != OnNewString ) OnNewString( s, n );
//  return n;
//}

int CConsoleStream::sync() {
  if ( NULL != OnNewString ) OnNewString( pbase(), (int) ( pptr() - pbase() - 
1 ) ); // assumes CR at end

  if ( NULL != OnFlushString ) OnFlushString();
  setp( pbase(), epptr() );

//  if ( NULL != OnFlushString ) OnFlushString();
  return 0;
}

int CConsoleStream::overflow(int_type meta) {

  throw std::runtime_error( "ConsoleStream overflow" );
}

The code was formatted with the javascript found at C++2HTML. I see there is GNU Source-highlight 2.8, but I don't see a web interactive version handy.

[/OpenSource/Programming] permanent link

2008 Feb 07 - Thu

Bash Script to Large Number of Files

Not knowing any other way to remove a large number of similarily named files from a directory (such as netflow files), and not wanting to do it manually, the following one liner creates something to start with:

ls -1 -A ft | sed "s/^/rm ft\//" > d.sh

It does a directory listing with one column, and does not include the '.' and '..' entries. It then uses sed (stream editor> to insert a rm command at the beginning of each line. The results are written out to a file for further tweaking.

chmod +x d.sh

will make the script executable.

[/OpenSource/Linux] permanent link

2008 Feb 06 - Wed

Scalped Straddles

Today I put the two words 'scalp' and 'straddle' into google to see what I could glean. I came up with a number of sites that could be of interest. Most of them talk about gamma straddles (option trading mechanism using one of the 'greek's) being tuned with scalp trades.

• Optionetics: an old article, but has some good warnings about the supposed profit features of straddles (the basic problems being of time decay, and the uncertainty of the movement).
• Market Maker Approach to Straddle Trading: this site by VTraderPro whets your appetite, but really, really wants you to sign up for a course. But the article does have some good caveats to keep on sane when trying to trade this popularized scheme.
• Elite Trader, Optimal Approach to gamma scalping is an article on gamma scalping. Here the participants get down and dirty on ways of doing things, and what to look out for.
• A link to led to their homepage with some fairly detailed market calendars and forums. This article on a GBP/USD Hourly Scalp got me to the site in the first place. I took a quick glance. There may be some useful scalping techniques in there for forex trading.
• Vega Scalping by Akalawoo is about scalping option greek Vega. The English in the blog is a bit off, but the fellow lays it on the line of what he does in various markets.

[/Trading/BlogsIFound] permanent link

C# Drivers and Indicators on SourceForge

For the code in the zip files I released back in 2007/10/07, I've uploaded it to SourceForge. The code can be reviewed and downloaded via Subversion. In summary, these are a series of C# routines designed for use in market trading applications.

Some of the IQFeed utilities can be used without SmartQuant libraries. The standalone IQFeed utilities are desigend to retrieve live as well as historical data from DTN/IQFeed's services.

The Sockets directory provides one mechanism for asynchronously receiving data from a socket and passing it upwards through the stack of waiting routines.

In the Trading.PatternAnalysis directory is an example of a ZigZag or PeakMatching indicator. Be aware that it is a lagging indicator, but it is good for keep track of the peaks and valleys of price movement through the day.

There are three indicators supplied in the Signals library: Darvas (currently set with aggressive signallig), Pivots (the self fulfilling floor trader signals), and TVI (which is a trade volume index indicator).

Probably the most interesting file in the SmartQuant directory is the Accumulation.cs file. It shows a mechanism for running a sliding window (based upon seconds duration) through a live series of data and obtaining some deviation, variation, mean, and Bollinger Band statistics for the window.

In the same file are some files for setting up a Microsoft SQL Server database for maintaining IQFeed mktsymbol.txt information, trading calculations, and related information.

[/Trading/SmartQuant] permanent link

C# Genetic Programming Code Released on SourceForge

I've released my Genetic Programming code library on SourceForge.

The code can be viewed through SubVersion. The most interesting directory is the GeneticProgramming directory where the Node, Individual, and Population algorithms are maintained. The remaining directories are devoted to testing, scaffolding, and runtime environments. The code has been written to integrate with SmartQuant's QuantDeveloper Trading Strategy Development Environment. With a little effort, it can be modifed to work in other environments. At some point, I plan to take the basic code and convert it to C++ to run with the new trading environment I'm developing.

The code is released to use as you wish. Feedback on the Sourceforge forum would be appreciated.

[/Trading/SmartQuant] permanent link

2008 Feb 05 - Tue

Open Source Site of the Day -- SOCI: The C++ Database Access Library

Back in October 2007, I wrote an article about PostgreSQL, C++ and the lack of binary data and wrappers.

I stand corrected. On two counts. The first is binary access. After looking through the PostgreSQL API, I see that I can indeed get binary data into and out of a database easily. The C based library, libpq, has an API call named PQexecParams which allows full separation of binary values from the query statement. This is a good step towards preventing SQL injection attacks.

The second item upon which I stand corrected is a full binary C++ wrapper around PostgreSQL. I encountered one today: SOCI - The C++ Database Access Library. It handles Oracle, PostgreSQL, MYSQL, as well as a number of others. The library is templated and allows extensions of user defined binary values.

If I'm not mistaken, it also provides a capability similar to what LINQ does in CLR languages, inlining of SQL code. Or close enough that the difference does not matter.

While on the subject of databases, I'll make mention that, for a trading application in a Windows environment that I've been developing, I've been starting to use the HDF Group's HDF5 Database as a repository for historical trade information. It handles custom binary record types, handles compression on the fly with a clever byte pre-ordering strategy, with huge file sizes. I'll write more on it later once I've had a chance to break it in. This is a C++ based library and works in Windows as well as Linux environments. I'm also going to try this library out for managing SNMP based data from a custom network monitoring application.

SQL based engines are good for generic query applications, but when one wants a self contained application, it is tough to get a user to install or allow installation of a separate engine. I suppose Microsoft's embedded SQL engine is a counter example to that argument. But I was looking for something lighter in weight. For the trading application I mentioned earlier, I've been trying out the Berkeley DB C++ Library. It doesn't handle the overhead of SQL queries, but does all the low level table stuff, along with transactions and secondary indexes. The code is a little more complicated, but it makes for a nice light weight engine for storing queryable records. The HDF5 engine is good for storing large quantities of binary data, but lacks a query mechanism. I think the division of labour between the two storage mechanism for a trading and a network management application works well.

[/OpenSource/SiteOfTheDay/D200802] permanent link