2007 Oct 06 - Sat

New Ideas -- For The Market Place, not the Existing Business Model

Dare Obasanjo wrote a nice blog entry summarizing some key ideas from Marc Andreessen and Clayton Christensen. In his entry entitled Stupid Things Big Companies Do, he had a great exerpt that confirmed that companies need to diversify their business models, offered up a key irony, and ended by providing a possible solution:

People come up with lots of new ideas, but nothing happens. They get very disillusioned. Never does an idea pop out of a person's head as a completely fleshed-out business plan. It has to go through a process that will get approved and funded. You're not two weeks into the process until you realize, "gosh, the sales force is not going to sell this thing," and you change the economics. Then two weeks later, marketing says they won't support it because it doesn't fit the brand, so we've got to change the whole concept.

All those forces act to make the idea conform to the company's existing business model, not to the marketplace. And that's the rub. So the senior managers today, thirsty for innovation, stand at the outlet of this pipe, see the dribbling out of me-too innovation after me-too innovation, and they scream up to the back end, "Hey, you guys, get more innovative! We need more and better innovative ideas!" But that's not the problem. The problem is this shaping process that conforms all these innovative ideas to the current business model of the company.

[/Personal/Business] permanent link

Web 2.0 Site Development with Wt -- Dead Easy

I've written an earlier article about installing the Wt C++ Web Toolkit. I think it took longer to get all the bits and pieces properly arranged than it did me to cook up an example to check that it did what I thought it could do.

I'm quote pleased and impressed with the tool. I was able to get a basic web application up and running in about two days, which were spent reading the documentation, reviewing the class libraries, looking at some of the samples, poking through the PostgreSQL C library, and then hacking stuff into the basic Wt 'Hello World' application.

The authors of Wt have done a great job of hiding all the Javascript go-between code. I can focus on the higher level parts of my design, rather than the underlying machinery to make an interface work. About the only thing I've encountered that I'd like to change is to take the inline Javascript code, and get it put into a .js file.

The test I came up with was to open up a database, select a bunch of records, and present the records in a table which could be stepped through (forwards and backwards) 10 records at a time... and all this without having to refresh the whole page, just the table itself. The following code proves it can be done.

It allows one to turn an internet browser into a true application front end with most of the usual gui functions provided.

This first file is ouipam.cpp, the file with the 'main' start for each session connection:

//============================================================================
// Name        : ouipam.cpp
// Author      : Ray Burkholder
// Version     :
// Copyright   : (c) 2007 One Unified
// Description : OUIPAM:  One Unified IP Address Management
//============================================================================

// server push uses WApplication enableUpdates(), triggerUpdate(), 
getUpdateLock();

#include "Responder.h"

using namespace Wt;

WApplication *createApplication(const WEnvironment& env) {

  // Instantiate the Wt application.
  Responder *appl = new Responder(env);
  return appl;
}

int main(int argc, char **argv) {
  return WRun(argc, argv, &createApplication);
}

This file is Responder.h, the header file for the responder, which handles the WApplication (or session):

//============================================================================
// Name        : ouipam.cpp
// Author      : Ray Burkholder
// Version     :
// Copyright   : (c) 2007 One Unified
// Description : OUIPAM:  One Unified IP Address Management
//============================================================================

#ifndef RESPONDER_H_
#define RESPONDER_H_

#include <WApplication>
#include <WContainerWidget>
#include <WText>
#include <WPushButton>
#include <WTable>
#include <WTableCell>

#include "libpq-fe.h"

using namespace Wt;

class Responder : public WApplication {
public:
  Responder(const WEnvironment& env);
  virtual ~Responder();
protected:
  WPushButton *button;
  WTable *table;
private:
  int cnt;
  PGconn *conn1;
  PGresult *result;
  int nRowsToShow;
  int cntRowsFound;
  int cntColumnsFound;
  int ixFirstRowShowing; // 0 offset
  void OnButtonListBegin();
  void OnButtonForeward();
  void OnButtonBackward();
  void OnButtonListEnd();
  void ShowRows();
};

#endif /*RESPONDER_H_*/

This final file is Responder.cpp, where all the session work is performed. The constructor opens the database, creates the webpage basics, and assigns events to each of the forward and backward buttons. When a button is pressed, the appropriate method is called to use the appropriate records and update the table. Again, with Javascript enabled, the web page is not refreshed. Instead, the browser's DOM is updated directly, and only the table cells change. I look foreward to seeing what else I can do with this flexibility. As you can see from the header comment, this is the start of an interactive web application for managing an organizations IP Addresses.

//============================================================================
// Name        : ouipam.cpp
// Author      : Ray Burkholder
// Version     :
// Copyright   : (c) 2007 One Unified
// Description : OUIPAM:  One Unified IP Address Management
//============================================================================

#include "Responder.h"
#include "WBreak"
#include <sstream>
#include <ostream>
#include <algorithm>

using namespace std;

Responder::Responder(const WEnvironment& env) :
  WApplication(env ) {

  // Set application title
  setTitle("OUIPAM by One Unified");

  ostringstream ss;

  // perform database query 

  conn1
      = PQconnectdb("hostaddr=127.0.0.1 port=5432 dbname=oneunified user=oneunified 
password=xxx");
  ConnStatusType stat = PQstatus(conn1 );
  if (CONNECTION_OK != stat) {
    //PQfinish(conn1);
    ss << "pq result= bad("<< stat << ")"<< endl;
    root()->addWidget(new WText(ss.str()));
  } else {
    result = PQexec(conn1, "select * from ianaiftype;");
    ExecStatusType statusExec = PQresultStatus(result );
    ss << "pgresult=";
    bool bTuplesFound = false;
    switch (statusExec ) {
      case PGRES_EMPTY_QUERY:
        ss << "empty query";
        break;
      case PGRES_COMMAND_OK:
        ss << "command ok";
        break;
      case PGRES_TUPLES_OK:
        ss << "tuples found";
        bTuplesFound = true;
        break;
      case PGRES_COPY_OUT:
        ss << "copy out";
        break;
      case PGRES_COPY_IN:
        ss << "copy in";
        break;
      case PGRES_BAD_RESPONSE:
        ss << "bad response";
        break;
      case PGRES_NONFATAL_ERROR:
        ss << "non fatal error";
        break;
      case PGRES_FATAL_ERROR:
        ss << "fatal error";
        break;
    }
    root()->addWidget(new WText(ss.str()));
    root()->addWidget(new WBreak());

    // present some query statistics

    if (bTuplesFound ) {
      cntRowsFound = PQntuples(result );
      cntColumnsFound = PQnfields(result );
      ss.str("");
      ss << "rows="<< cntRowsFound << ", columns="<< 
          cntColumnsFound << endl;
      root()->addWidget(new WText(ss.str()));
      root()->addWidget(new WBreak());
      for (int i = 0; i < cntColumnsFound; i++) {
        ss.str("");
        ss << PQfname(result, i )<< ", format="<< PQfformat(result, i )
            << ", type="<< PQftype(result, i )<< ", size="
            << PQfsize(result, i )<< endl;
        root()->addWidget(new WText(ss.str()));
        root()->addWidget(new WBreak());
      }

      // create table for row results

      table = new WTable();
      root()->addWidget(table);

      // assign methods to the buttons

      button = new WPushButton( L"<<" );
      root()->addWidget(button );
      button->clicked.connect(SLOT(this, Responder::OnButtonListBegin));

      button = new WPushButton( L"<" );
      root()->addWidget(button );
      button->clicked.connect(SLOT(this, Responder::OnButtonBackward));

      button = new WPushButton( L">" );
      root()->addWidget(button );
      button->clicked.connect(SLOT(this, Responder::OnButtonForeward));

      button = new WPushButton( L">>" );
      root()->addWidget(button );
      button->clicked.connect(SLOT(this, Responder::OnButtonListEnd));

      // show the query results

      nRowsToShow = 10;
      ixFirstRowShowing = 0;
      ShowRows();

    }
  }
}

Responder::~Responder() {
  PQclear(result ); // result exists even with new command, and even if connection is closed;
  PQfinish(conn1 );
}

void Responder::ShowRows() {
  // update rows in the already created table
  WTableCell *cell;
  WText *text;
  int ixRowToShow = ixFirstRowShowing;
  for (int ixTableRow = 0; ixTableRow < nRowsToShow; ixTableRow++) {
    for (int ixColumn = 0; ixColumn < cntColumnsFound; ixColumn++) {
      cell = table->elementAt(ixTableRow, ixColumn );
      cell->clear();
      if (ixRowToShow < cntRowsFound ) {
        text = new WText();
        text->setFormatting(WText::PlainFormatting );
        text->setText(PQgetvalue(result, ixRowToShow, ixColumn ) );
        cell->addWidget(text );
      } else {

      }
    }
    ixRowToShow++;
  }
}

void Responder::OnButtonListBegin() {
  ixFirstRowShowing = 0;
  ShowRows();
}

void Responder::OnButtonBackward() {
  ixFirstRowShowing = max( 0, ixFirstRowShowing - nRowsToShow );
  ShowRows();
}

void Responder::OnButtonForeward() {
  int i = max( 0, cntRowsFound - nRowsToShow );
  ixFirstRowShowing = min(i, ixFirstRowShowing + nRowsToShow );
  ShowRows();
}

void Responder::OnButtonListEnd() {
  ixFirstRowShowing = max( 0, cntRowsFound - nRowsToShow );
  ShowRows();
}

[/Personal/SoftwareDevelopment/CPP] permanent link

Flash File Systems for Embedded Systems, and Otherwise

On Kernel Trap, I see they are discussing the use and maintenance of flash file systems. It seems there are lots of interesting gotchas when using flash file systems intensively. I'm wondering if that might be why the Seagate 32G flash drive is taking a while to get going in the market place.

Anyway, for my own embedded thoughts, it is good to know flash file systems are making good headway into kernel integration.

[/OpenSource/Linux] permanent link

Open Source Site of the Day -- OSSWAD: Open Source Savvy Web Application Developer

An OSSWAD, almost sounds like a forbidden word. Based upon Bob Zurek's column, there are Osswads, and then there are OSSWADs. Many developers of web sites 'out there' already use Open Source tools to get the job done. Apache, MySQL, PostgreSQL, Perl, and PHP are commonly used tools by Osswads.

On the other hand, one gets the impression that real, enterprise savvy OSSWADs, use some real hard core tools like:

• hadoop: large scale distributed compute clusters. Amazon has a developer resource center with a Hadoop/MapReduce Paper.
• lucene: not just search software, but something that provides the basis for getting meaningfull context. There is an informative online book to go alone with it.
• nutch: adds web specifics to lucene: web crawling, link-graphs, and parsers.

These Open Source projects fill in more of the gaps of the thinking I've been doing on providing some contextual search products in specific subject matter areas. It looks like I won't have to do as much ground up development as I thought I might need to do.

[/OpenSource/SiteOfTheDay/D200710] permanent link

libpqxx: A PosgreSQL C++ Wrapper Library

I've looked at the C library for PostgreSQL and then wanted to see if there were any C++ wrappers for it. The most current appears to be libpqxx. On first blush, it looks very good. It if functional and robust. But... it has one draw back. The library insists on converting all binary stuff into text for passing back to the caller. For some applications, that can be a reasonable library simplifier.

But in this day and age of Templates and polymorphism, it seems to be a copout. Yes, if I had time, I'd probably try my hand at implementing some sort of 'variant' implementation to handle the various types of data that come back.

Sigh. As much as I'd like to use the library, I really want my data in the native form in which I stored it. Perhaps once I've worked with the API, and I've worked my own working version of variants, maybe I'll be in a position to offer up some workable suggestions.

... several hours later ...

I now realize that the PostgreSQL API returns everything as strings. So now I understand the reasoning behind the library and what it returns.

To get binary values, from my understanding, one has to use the COPY routines and decipher the result streams directly. I would have hoped there was some middle ground, where the API will provide the binary values when needed. The API implies that it does, but in actual fact, didn't. I spent an hour or two figuring it out the hardway. I'm really going to have to figure out how gdb works so I can single step through stuff and analyze variables. As it was, I reverted back to the stone age and put in print statements to figure out what was happening.

Now I have a tri-choice:

• Stick with the C API, which seems to have everything one needs, including access to true parameterized queries (I found out after the fact, that the Perl libraries actually do string concatenation, which creates a possibility for SQL Injection attacks).
• Use the libpqxx library and convert stuff to binary when I need it. I must say working with everything as strings is a simplifying assumption, but doesn't do much for accuraccy and performance.
• Start working on some sort of wrapper to examine the results of COPY commands, which is probably more work than I really want to do right now.

[/OpenSource/Programming] permanent link