function url_decode(str){
var hex = /^[0-9a-fA-F]{2}/;
var out='';
var arr = str.split('%');
if(arr.length<2) return str;
for(var i=0;i<arr.length;i++)
{
  /*look for hex values */
  if(hex.exec(arr[i])) {
    out += String.fromCharCode(parseInt(arr[i].substring(0,2),16))+arr[i].substring(2,arr[i].length);
  } else { if(i==0) out+=arr[i]; else out+='%'+arr[i];
  }
}
return utf8to16(out);
}

function utf16to8(str) {
    var out, i, len, c;

    out = "";
    len = str.length;
    for(i = 0; i < len; i++) {
	c = str.charCodeAt(i);
	if ((c >= 0x0001) && (c <= 0x007F)) {
	    out += str.charAt(i);
	} else if (c > 0x07FF) {
	    out += String.fromCharCode(0xE0 | ((c >> 12) & 0x0F));
	    out += String.fromCharCode(0x80 | ((c >>  6) & 0x3F));
	    out += String.fromCharCode(0x80 | ((c >>  0) & 0x3F));
	} else {
	    out += String.fromCharCode(0xC0 | ((c >>  6) & 0x1F));
	    out += String.fromCharCode(0x80 | ((c >>  0) & 0x3F));
	}
    }
    return out;
}

function utf8to16(str) {
    var out, i, len, c;
    var char2, char3;

    out = "";
    len = str.length;
    i = 0;
    while(i < len) {
	c = str.charCodeAt(i++);
	switch(c >> 4)
	{
	  case 0: case 1: case 2: case 3: case 4: case 5: case 6: case 7:
	    // 0xxxxxxx
	    out += str.charAt(i-1);
	    break;
	  case 12: case 13:
	    // 110x xxxx   10xx xxxx
	    char2 = str.charCodeAt(i++);
	    out += String.fromCharCode(((c & 0x1F) << 6) | (char2 & 0x3F));
	    break;
	  case 14:
	    // 1110 xxxx  10xx xxxx  10xx xxxx
	    char2 = str.charCodeAt(i++);
	    char3 = str.charCodeAt(i++);
	    out += String.fromCharCode(((c & 0x0F) << 12) |
					   ((char2 & 0x3F) << 6) |
					   ((char3 & 0x3F) << 0));
	    break;
	}
    }
    return out;
}

function trace(url){
var tracker = _gat._getTracker("UA-XXXXXXX-X");
tracker._trackPageview(url);
}

Now every time I need to track something I call this function with a custom build URL.

HTTP Connection - engine selector

So I changed the connector engine and restartet the test. This time I started small. I set the heap to 512MB and limited the maxProcessor to 500. What I then saw was unexpected. The engine filled up right to the ten thousand requests I send. There occured no Out-Of-Memory at all. What was also interesting was, that the engine held 10k connections despite the maxprocessors was set to 500.
So to conclude the result, the new connector is quite impressive when you need to handle a lot of simultanous connection and have not a lot of memory to spare. On the other hand, when you use it, you loose some of the TIBCO integrated features to limit your load. Further to that, the TIBCO documentation states that due to the single thread arcitecture you increase latency. So as always there is a tradeoff.

One final sidenote. I had some issues with the BusinessWorks 5.7.1 engine so I upgraded it to 5.7.2. Than it ran without a glitch.

graph.height.pref=712
graph.width.pref=1515

These values represent the default size which the Designer allocated for its canvas. As you can see the default is pretty small on this one. I also found installations where both values where ten times larger then this. So far I found no performance penalty to this.
The only pattern I found is that newer installation had smaller values as default. Where this value comes from and how it is determined stays unclear to me.

One other thing I found during my research. If the panel is to small for the current process you can drag one activity right next to the border and then start to move it via keyboard (‘Shift + Cursor’). By doing so you drag the icon out of the canvas, but the position will be updated internally. After that you just need to refresh the process view and voila you have expanded the canvas size. But this is just a quick and dirty work-around.

CREATE TABLE pages
(
  id bigint NOT NULL,
  url character varying(255),
  md5 character(32),
  CONSTRAINT pages_pkey PRIMARY KEY (id)
)

The faster lookup should mainly be possible through the shorter column length (and therefore smaller index). Actually I don’t know if the fixed width is good or bad here, but hashes usually don’t vary in length. After creating this table I added a B-Tree unique Index to the md5 column to enable a fast lookup.
After a while a noticed a rather high CPU load on lookups for this table so I tried to analyze the problem. First I tried the obvious through psql.

cloud=# explain analyze select * from pages where md5 ='abc';
                                                     QUERY PLAN
---------------------------------------------------------------------------------------------------------------------
 Index Scan using i_pages_md5 on pages  (cost=0.00..8.50 rows=1 width=166) (actual time=0.046..0.046 rows=0 loops=1)
   Index Cond: (md5 = 'abc'::bpchar)
 Total runtime: 0.157 ms
(3 rows)

As the explain shows all works perfectly fine and lookups shouldn’t be a problem. So there had to be something different what was going on.

After that I tried the same select with an actual md5.

cloud=# explain analyze select * from pages where md5 = md5('abc');
                                                  QUERY PLAN
--------------------------------------------------------------------------------------------------------------
 Seq Scan on pages  (cost=0.00..32017.63 rows=3994 width=166) (actual time=1203.699..1203.699 rows=0 loops=1)
   Filter: ((md5)::text = '900150983cd24fb0d6963f7d28e17f72'::text)
 Total runtime: 1203.769 ms
(3 rows)

Now you can see the plan does change quite a lot. I have a full table scan instead of an index scan. You can also see that the query time increases nearly by factor ten thousand.

The reason for this dramatic change is a simple type mismatch. For whatever reason the md5 function will be evaluated to a string of the type text. To create a match with the column md5 all values had to be casted to that type. The side effect of this is that the index can no longer be used, because it is of the wrong type.

To solve this I just had to cast the result of the md5 function back to something that is compatible with the index type. In my case I used a fixed width character field which is represented in the database as bpchar (blank padded character). So after modifying the query to the following I was back on index usage.

cloud=# explain analyze select * from pages where md5 = md5('abc')::bpchar;
                                                     QUERY PLAN
---------------------------------------------------------------------------------------------------------------------
 Index Scan using i_pages_md5 on pages  (cost=0.00..8.50 rows=1 width=166) (actual time=0.141..0.141 rows=0 loops=1)
   Index Cond: (md5 = '900150983cd24fb0d6963f7d28e17f72'::bpchar)
 Total runtime: 0.199 ms
(3 rows)

for the Administrator:

#!/bin/bash

DOMAIN=tibcoesb
USER=esb
TIBCO_HOME=/home/$USER/tibco
PATH=/bin:/usr/bin:/sbin:/usr/sbin
NAME=tibco-admin

start() {
# Start daemons.
echo "Starting Tibco Admin"
/bin/su $USER -c"cd $TIBCO_HOME/administrator/domain/$DOMAIN/bin;./tibcoadmin_$DOMAIN 2>&1 | /usr/bin/logger -t $NAME" &
echo "done"
}

stop() {
# Stop daemons.
echo "Shutting down Tibco Admin"
killall tibcoadmin_$DOMAIN
echo "done"
}

case "$1" in
  start)
	start
	    ;;
  stop)
	stop
    	    ;;
  *)
        echo $"Usage: $0 {start|stop}"
    	exit 2
esac

for the Hawk

#!/bin/bash

DOMAIN=tibcoesb
USER=esb
TIBCO_HOME=/home/$USER/tibco
PATH=/bin:/usr/bin:/sbin:/usr/sbin
NAME=tibco-hawk

start() {
# Start daemons.
echo "Starting Tibco Hawk"
/bin/su $USER -c"cd $TIBCO_HOME/tra/domain/$DOMAIN;./hawkagent_$DOMAIN 2>&1 | /usr/bin/logger -t $NAME" &
echo "done"
}

stop() {
# Stop daemons.
echo "Shutting down Tibco Hawk"
killall hawkagent_$DOMAIN
echo "done"
}

case "$1" in
  start)
	start
	    ;;
  stop)
	stop
    	    ;;
  *)
        echo $"Usage: $0 {start|stop}"
    	exit 2
esac

The easiest way to achieve this would be to change the authentication for this service. Unluckily this service is used by a lot of other clients, so I can’t change it out of the blue. Next thing that came to my mind was using the underlying tomcat to do some IP-based blocking. After some searching around I gave that one up. Tibco has a really complex deployment model for BusinessWorks so I wasn’t able to even find the process-specific section for the tomcat engine.
After some searching around in the documentation (actually several hours – hooray to a usable documentation) I found some parameter which allows you to block IPs with some TRA-file parameters. In case you want to look it up yourself it is in the BusinessWorks documentation under ‘Administration’ -> Chapter 8 ‘Custom Engine Properties’ -> ‘Available Custom Engine Properties’ (and another hooray to the excellent formatting – Note to Tibco: HTML has tags, so you can do more with the content then continuous text – even some breaks would enhance the experience).
So back to the point:

bw.plugin.http.server.allowIPAddresses
bw.plugin.http.server.restrictIPAddresses

Both tags are rather self explanatory so you can use right away. Tibco says you can use it with a single IP, a list of IPs (comma-separated) or regular expression. As stated before this highlight of a documentation doesn’t say what reg-exp syntax means. I guess Tibco means they use the Tomcat engine for evaluation of these values an that means they use the Java syntax described here.

A lot of people out there use services like delicious where you tag you favorite sites and make this available to other users. Now I started to grab that data and began to build a massive tag cloud. After some time the site collected hundreds of thousands of links with their corresponding tags. So now you can start on the site and search for tags which interests you. These search tags will then be correlated against the cloud database and you get the most active links for your tags. So here is an example.
Let’s say you are interested in a tomcat tutorial.
tags:

tutorial, tomcat

results:

howtoforge.com
howtogeek.com
java.sun.com

Of course those results will be a link to the concrete tutorial (not just the entry page).
So far to the official part. For me this is more of a fun project. I prefer to start with some random tag and then wander around. It’s more like browsing, cause you start at points that you don’t already know. You have the chance to break out of your existing network of most used sites and see something new.

So have fun with it.

www.cloudtheweb.com

PS: For the implementation part – if you have any questions, just ask. I’m planning to explain some details about how it works on some later posts.

So enjoy the new features and lets hope the next release will not take another year.

First download the GLUEscript source from sourceforge.
A second tool you will need to get this running is premake. This is also a sourceforge project (you can use the binary version of it right away).
After downloading, I copied the premake binary into the glue/src folder.

So now we can start with fetching the dependencies which debian can fulfill.

sudo apt-get install libnspr4-0d libnspr4-0d-dbg libnspr4-dev libcurl4-openssl-dev libwxgtk2.8-dev libssl-dev libiodbc2-dev libmysql++-dev

In addition to that I needed a library called poco version 1.3.5 (all repositories I found just provided versions up to 1.3.3 -> those don’t work). So get the source from http://pocoproject.org/download/ (the complete version). Compiling that should make no trouble cause all the dependencies are already installed.

/tmp$ cd poco-1.3.5-all/
/tmp/poco-1.3.5-all$ ./configure
Configured for Linux
/tmp/poco-1.3.5-all$ make
/tmp/poco-1.3.5-all$ sudo make install

Now let’s get back to configuring GLUEscript. All configuration is done via lua script which will than be consumed by premake. The config file I needed to edit was the premake.lua file:

-- Check NSPR
if ( string.len(nspr_dir) == 0 ) then
  print("Using the NSPR library which is part of GLUEscript")
  dopackage("nspr") -- build our own NSPR
  nspr_dir = "../nspr/include"
  nspr_lib = "nspr"
  nspr_lib_dir = project.libdir
else
  print("You are using your own NSPR library: ")
  nspr_dir = "/usr/include/nspr"
  print("nspr include: " .. nspr_dir)
  print("nspr lib: " .. nspr_lib_dir .. "/" .. nspr_lib)
end

I copied the whole paragraph to just make it easier to find the position. Important is the added row in the else part.

nspr_dir = “/usr/include/nspr”

This is needed because debian has a different file structure for header files than the script expects it.

After that we are done with configuring. To actually start the build process you have to run premake first.

./premake gnu
make

The output will be generated to the following directory:

glue/bin/Debug

So far the makefile does not a an installation part. So if you want to install this you have to do it by yourself.

PS: This only works for the 0.0.1 version. So far I didn’t get any more recent svn version running.