2008 Jan 29 - Tue

Open Source Site of the Day -- TrueCrypt: Free open-source disk encryption software for Windows Vista/XP/2000 and Linux

When trying to keep things private and personal, and to carry things around securely on USB keys, or even hard drives, nothing beats the simplicity and flexibility of TrueCrypt. It is free and and it is Open Source. A new release is scheduled for February 4, 2008.

Here is a feature list from the main web page:

• Creates a virtual encrypted disk within a file and mounts it as a real disk.
• Encrypts an entire hard disk partition or a storage device such as USB flash drive.
• Encryption is automatic, real-time (on-the-fly) and transparent.
• Provides two levels of plausible deniability
• Encryption algorithms: AES-256, Serpent, and Twofish. Mode of operation: LRW.

The software is well documented and has a good startup tutorial.

I had seen this software a while ago, and hadn't done much with it. I was recently reminded of this through A Nice Surprise, an article written by a SANS' NewsBytes Editor regarding data loss and theft. As the author says, with software so simple and easy, why isn't it used more often?

The same author has a page designated as the The Six Dumbest Ideas in Computer Security. In brief, the points are:

• Default Permit
• Enumerating Badness
• Penetrate and Patch
• Hacking is Cool
• Educating Users
• Action is Better Than Inaction

The article is well worth the read. It is an excellent dissertation on how one should change one's security philosophy to get at problems at the source, rather than attempting to make the symptoms go away.

[/OpenSource/SiteOfTheDay/D200801] permanent link

2008 Jan 28 - Mon

Symbol Clash Between VC++ oledb.h and Berkeley DB db.h

When attempting to use Berkeley DB4 in a Microsoft Visual Studio 2005 C++ project, the symbol DBTYPE is found in both Microsoft's oledb.h and Berkeley DB4's db.h. It is really hard to get rid of oledb.h as it is buried somewhere in the depths of the stdafx.h precompiled header file.

In one of the forums, a suggestion was made to wrap the Berkely DB4 header file db_cxx.h in a namespace. That worked somewhat to remove the name clash, but it resulted in a link error of not being able to find the namespaced symbol in the dll file. I wasn't sure what was needed to resolve that bit of a naming/link-resolution problem.

Elsewhere, in another forum posting, there was a suggestion of using an #undef DBTYPE. That didn't appear to work either. I think this is because DBTYPE is a 'typedef' rather than a simple #define. I suppose I could have tried to change the typedef to a #define.

Instead, I made a copy of oledb.h as oledb.original.h, and modified the oledb.h file. I changed all references of DBTYPE to MS_DBTYPE and rebuilt the project. The project compiled and linked fine.

Of course, if I need to use oledb.h in the future, something I doubt very much, but one never knows, I'll have to figure something out to maintain code compatibility.

I'm sure there is a more elegant solution. If someone has encountered it, please email a solution to me and I'll post it.

[/OpenSource/Programming] permanent link

C++ Override std::cout, std::cerr Streams

C# has a handy method of redirecting Console output. In the first place, C# has a System.Console library for catching all console based output. The Console.SetIn and Console.SetOut methods can then be used to redirect input/output from/to local application based routines.

In C on a Linux/Unix system one can use popen(), as described in question 19.30 of the comp.lang.c FAQ. If you cruise through the FAQ, other methods are discussed and critiqued.

	extern FILE *popen();
	sprintf(cmdbuf, "sort < %s", datafile);
	fp = popen(cmdbuf, "r");
	/* ...now read sorted data from fp... */
	pclose(fp);

Some have suggested using the freopen() stdio function call to redirect stdout to a file. I havn't seen an example to get the stream back into the program though.

In C++, when using the Standard Templated Libraries (STL), one encounters the std::cout and std::cerr streams for standard console output.

C++ streams are generic and, after a fashion, interchangeable. Each stream type has a std::streambuf, which can be redirected. Be aware the streambuf is a C++ construct, and therefore won't catch stuff done with a printf or similar. A pipe may help for the printf problem (I havn't looked into that), but a pipe isn't necessary for the streambuf solution.

A simplistic mechanism for switching streambufs is found in a Borland CPP Builder Forum. An example there does show a good mechanism for saving the old streambuffer before redirecting to the new one. The problem with the posted example is that one has to come back to the streambuf and manually extract what was there. A better solution would do a proper override and automatically process the arriving characters.

I saw in a few places there people were trying to override endl or << operators. That isn't quite correct either.

Probably the best description on how to do a proper override is located in a recent blog entry from Sean Middleditch in a post called: Creating Custom C++ Output Streams. His examples build a simple override and then add additonal features on to improve the solution.

// from Sean Middleditch's site
// your log file, lazily declared as a global
ofstream logfile;

// logbuf forwards all output to cerr and logfile
class logbuf : public streambuf {
private:
  // write a string s of length n to standard
  // error and a log file
  int xsputn (char_type* s, streamsize n) {
    cerr.write(s, n);
    logfile.write(s, n);
    return n;
  }
};

int main () {
  // open our log file
  logfile.open("mylog.txt", ios::app);

  // create our log stream
  ostream log(new logbuf());

  // be friendly
  log << "Hello, World!" << endl;

  return 0;
}

For another perspective on the solution, Cay S. Horstmann wrote an article regarding Extending the iostream library. Further down in that article is an example for 'Routing stream output to a debug window'.

Some further background on IOStreams and Stdio can be found in a Dr. Dobbs article by Matthew H. Austern called The Standard Librarian: IOStreams and Stdio.

[/OpenSource/Programming] permanent link

2008 Jan 26 - Sat

SmartQuant QuantDeveloper & DataCenter Release

SmartQuant has released a revision to DataCenter and QuantDeveloper. DataCenter and QuantDeveloper are at the following revision levels:

DataCenter
Version 2.3.6 (25-Jan-2008) 

QuantDeveloper Enterprise Edition
Version 2.7.4 (25-Jan-2008) 

QuantDeveloper Source Code
Version 2.7.4 (25-Jan-2008) 
* Recent Versions available through 
  version control 

[/Trading/SmartQuant/Releases] permanent link

2008 Jan 25 - Fri

Debian with Java and Eclipse/CDT

After downloading Eclipse/CDT (Eclipse for C++ Developers), and trying to run it I encountered an error about it finding Java RunTime 1.4.2, and it finding that inadequate.

I proceeded to 'apt-get install sun-java6-jre' successfully. Eclipse still didn't start. I found that if one runs 'update-alternatives --config java', one can select the proper Java Run Time with which to be running.

[/OpenSource/Debian] permanent link

2008 Jan 22 - Tue

Creating a Crypto++ Shared Library in Eclipse/C++

The Crypto++ Library, which is an open sourced C++ Cryptographic library, has a makefile for creating a static library. The library turns out to be a large library. Static link times when linking into my project aren't fast, particularily when used in a VMWare based development environment. To make linking and running faster, a shared library would be much better. I'm using the library in an Linux hosted Eclipse/C++ based IDE.

As I really havn't built a makefile by hand before, I cheated and used Eclipse to create a project in which to create the library. These are the steps I used:

• Create a new C++ project with a shared library as the target
• Created a src directory in the project GUI, manually copied the .h and .cpp files into the directory
• Excluded the test files (bench*, bench2* test*, validat*, adhoc*, datatest*, regtest*, fipsalgt*, dlltest*)
• In the C++ Compiler Preprocessor settings, added CRYPTOPP_DISABLE_ASM, which seemed to fix an 'asm' message in vmac.cpp
• Unchecking the 'All Warningss (-Wall)' box in the C++ Compiler Warnings settings will disable a flood of warnings
• Added '-pthread', '-pipe', and '-fPIC' flags to the C++ Miscellaneous flags (-fpic might create smaller code, or -fpic may even be left out, but I havn't tried that).
• Named the object 'cryptopp' in the C++ Linker Shared Library Settings for Shared Object Name (-Wl,-soname=)
• Compiled the library, copied the resulting file 'libcryptopp.so' to /usr/lib, and ran 'ldconfig /usr/lib/libcryptopp.so'.

With a little more work, I could now take these settings from Eclipse's make file, and retrofit them into the original Crypto++ GNUMakefile, but I'll save that for another day.

As a backgrounder on shared libraries, I used the TLDP's Program Library HowTo as for reference.

[/OpenSource/Programming] permanent link

2008 Jan 19 - Sat

Trading Site of the Day -- aiQUANT: Biologically Inspired Algorithms for Modeling Financial Markets

In the About Page for aiQuant, he has an interesting diagram (taxonomy) of Biologically Inspired Algorithms (BIAs) in which he is interested. There are few there that I've not yet come across, and which are probably worthy of further study.

I think the author of the site does a good job of bringing much of the math down to earth to be accessible in an applications oriented method. The way he explains the Hilbert Transform and the z-Transform are admirable in terms of how they can be used with regard to financial time series.

[/Trading/SiteOfTheDay/D200801] permanent link

2008 Jan 18 - Fri

C++ Tools: Face Detection, Artificial Neural Networks

Today I came across a couple of C++ projects relating to Neural Network usage. One is located at CodeProject called Face Detection C++ Library with Skin and Motion Analysis. The author has used a number of interesting statistical and analytical methods for face detection. To quote his mouth-full: "An understanding of wavelet-analysis, dimensionality reduction methods (PCA, LDA, ICA), artificial neural networks (ANN), support vector machines (SVM), SSE programming, image processing, morphological operations (erosion, dilation), histogram equalization, image filtering, C++ programming and some face detection background would be desirable.".

From a C++ library perspective, here is a Fast Artificial Neural Network Library (FANN). I believe the library is written in C, but has API bindings in about 13 different programming language formats, of course with one being C++. The author's description goes as follows: "Fast Artificial Neural Network Library is a free open source neural network library, which implements multilayer artificial neural networks in C with support for both fully connected and sparsely connected networks. Cross-platform execution in both fixed and floating point are supported. It includes a framework for easy handling of training data sets. It is easy to use, versatile, well documented, and fast."

[/Personal/SoftwareDevelopment/CPP] permanent link

Network Security Tools

ITSecurity has an article titled The 10 Best Free Security Tools. One correction I'd like to make is that Ethereal has been forked with modern version now known as WireShark.

One cool knowledge snippet I learned is that NetStumbler, the rogue Wireless Access Point detecter, now has a sibling for Windows CE based mobile devices.

[/OpenSource] permanent link

They Can't be Better Than Sliced Bread

On dzone, there were a couple of language related articles. Well, ok, the whole site is devoted to programming languages. Perhaps what I was trying to say is that I detected a mild language war brewing. Well, maybe skirmish. Maybe mild skirmish.

Linux.com has an article on D, called New D language pumps up programmer productivity. The writer does a lot of comparing against C++. I still fail to see why D shines though. What makes it special?

C# is supposed to be a better C++. When you couple C# with Microsoft's CLI (Common Language Interface) libraries, it is a nice development environment. C# takes away some nice things from C++, but adds its own nice things.

Java is also supposed to be a better C++. Java does indeed do a good job of of being platform independent. For instance, I liked the way of being able to install Eclipse, the Java based IDE, on a Linux platform or a Windows platform and being up and running in minutes. I havn't programmed in Java, so can't make a fair comparison of what makes Java a nice place in which to program.

In an article by Rick Hightower, he mentions Java, Ruby, Python, and Scala. In the article is a graph showing language usage. Java ranks first and C++ second.

I have an insurance modelling friend who swears by Python, which ranks a low sixth in the chart.

Anyway, what got me started on this all was an article called The Great Language Backlash. I thought, oh cool, someone is going to do an impressive rant on what is missing in all the world's programming languages. it ended up being some little rant about Ruby and Groovy, with Ruby being last in the chart I mentioned above. I'm glad the writer redeemed himself with his final phrase of the article: use the best tool for the job. Well, I suppose that phrase, in and of itself, is worthy of many a rant all by its lonesome.

I just can't resist: C++ rocks!! Maybe the version 0x should be changed to C!!! I'll leave it up to the reader to determine how many '!' belong to the C and how many belong to sentence punctuation.

[/Personal/SoftwareDevelopment] permanent link

2008 Jan 17 - Thu

Open Document Format Alliance Refutes the Burton Group Report on ODF

I have to say, starting out, that I do use Open Source productions as much as I use Microsoft products. Each has its merits.

On the other hand, Microsoft would prefer that I use their products exclusively. I don't think so. Interoperability is not really an allowed word in their books. The words extend, embrace, and extinguish do feature prominently.

Much has been said, written, and done with regards to their OOXML format, which they are trying to ratify. As it turns out, the standard they are presenting does not fully represent their formats. Perhaps with some recent releases, that may be resolved. But with a standard, an incomplete one at that, already at 6000 pages, additional information releases just adds more bulk.

On the other hand, why can't they just allow an ODF transalator into their main 'Save As ...' menu? Why make things complicated. Oh. Sorry. I know the answer to that one... to keep the competition at bay.

If Microsoft's shareholders were polled, do you think they would want Microsoft to continue in the vein they are currently, or to inject a certain amount of ethical, healthy market competition into the proceedings? After all, they started to thrive once IBM openly published the specs for the PC. Instead, Microsoft publishes only what they need to, and only at the lash of the whip of the European Union.

The basis for today's rant is an article over at Groklaw entitled Open Document Format Alliance Refutes the Burton Group Report on ODFM. The Burton Group had developed a report which has quite a few false facts. ODFA has refuted them. I just wanted to add my bit in publicising the counter to the fear, uncertainty, and doubt.

And I hope reason prevails when the OOXML vote comes up in February.

[/OpenSource] permanent link

Crypto++

The C++ library, Crypto++, has an amazing array of crypto routines, including stream cyphers, block ciphers, message authentication codes, hash functions, public-key cryptography, elliptic curve cryptopgraphy, as well as hold-over insecure and obsolescent algorithms.

I'm using the library for a relatively simple task of using SHA-1 hashing on user passwords for a variety of software applications I'm in the midst of writing: blog routines, IP addressing documentation, and network management.

In the readme file in the archive is a couple of paragraphs I found as succinct descriptions of what to do in contstructors with C++ references and pointers:

1. If a constructor for A takes a pointer to an object B (except primitive types such as int and char), then A owns B and will delete B at A's destruction. If a constructor for A takes a reference to an object B, then the caller retains ownership of B and should not destroy it until A no longer needs it.

2. Crypto++ is thread safe at the class level. This means you can use Crypto++ safely in a multithreaded application, but you must provide synchronization when multiple threads access a common Crypto++ object.

[/OpenSource/Programming] permanent link

2008 Jan 10 - Thu

Additional Configuration Notes for Wt, a C++ Web Toolkit

I need to append some additional notes to my 2007/10/03 Wt Configuration Guide article.

As mentioned in the Wt Ext Widget deployment notes, the Wt toolkit makes use of libraries from Ext JS. There are notes at the bottom of the Wt page itemizing the files needed from the JS Ext 1.1 .zip file.

As a side note, all things Ajax can be referenced one way or another starting at the ajaxian web site. Maybe not quite the truth. More all things Ajax at AjaxPatterns... if you want to get under the hood.

[/OpenSource/Debian/Development] permanent link

2008 Jan 03 - Thu

Being Stopped Out, Accidently On Purpose

Putting stops on trades is a popular mechanism for protecteding 'the trade'. In some situations, if they are not placed properly, they can be used against the trader in their pursuit of profits.

According the exerpt below, which I gleaned from a thread on Elite Traders, putting stops just outside of common resistance and support lines can cause premature failure of what could have been a profitable trade.

I think the writer's comments are worth repeating as a possible explanation for why some traders may think that the market 'is out to get them'.

From my personal experience that is derived from day trading primarily NASDAQ through a proprietary firm, I suspect that the main running of stops by MMs and by large operators occurs when these professionals are able to move the market to a place where the majority of stops are most likely located. IMO such places are usually just beyond natural support or natural resistance levels, e.g. the high or the low of any of the last three trading days, or perhaps just beyond the boundaries of a congestion. Basically, if the market mover is able to take the market to a place where some stops are likely to be located, unless a significant amount of the outsiders/public joins in on the action and carry the price further, the market is likely to reverse shortly after reaching that point. For example if a MM or a large operator suspects that there are orders sitting just above yesterday.s high, then basically he knows that what he buys below yesterday.s high, he will be able to dispose of at a higher price because once he moves the market just beyond yesterday.s high a number of buy orders will enter the market. At this point, once the operator disposes of his original line he is very likely to sell short a significant amount of shares, and so unless there is enough buying from the outsiders to take care of this supply, the market is likely to reverse. This is just one scenario of many, however, I do think that the logic is peaty similar in most cases.

Many of the breakouts are real, such is the nature of market dynamics, however, when the market is in a consolidation or in a congestion it.s somewhat easy for the big operators to make their profits by running the market back and forth. Actually, if your stops are constantly being hit, it may be a sign that you are in a congestion. Personally, I try to stay out of the market as soon as I suspect that I am in a congestion, and wait for a good indication that the market is beginning to trend again.

To protect myself, I often do not trade the first breakout of the natural support or resistance, or the congestion. Personally, I.d rather wait for a correction or a second time through. Such strategy is much more conservative and will prevent you from taking many traders, however, IMO I think that being patient and waiting for a high-probability setup is essential if one is to achieve consistent success in day-trading.

This trader is still looking for the traditional retail style of trading of looking for the trends and runs. Rather than staying out of the conditions mentioned, I am still researching and developing some trading methods to stay with the market, bracket it, and make money through these congestive periods.

The other moral of the story is that a trader needs to be careful of where stops are placed. Stay out of the usual areas, and place them in the less travelled paths.

[/Trading] permanent link