A scorecard is a special type of concept. A scorecard serves as a set of static
variables that is available throughout the project.
…
It is more accurate to say there is one instance of a scorecard per inference agent.
Each inference agent in an application has its own instance of the score card.
Scorecards are not shared between agents.

So you can use Scorecards without any Issues as long as you consider the per Instance rule. Now I have a project where Timeouts are calculated an stored in Scorecards. This solution works perfectly in single instance environment because of the special behavior described above. Now there comes a new Requirement into the project. The Agents should be clustered over 2 Instances to ensure high availability. This really ruins the simplicity of the scorecard approach.

Digging a little deeper into the documentation I came to the conclusion that all the scorecards had to be stored in the cache-server. Knowing this, it came to me mind that the only thing I must accomplish would be, that all inference agents use the same instance of the scorecard.

To achieve the single Instance per Agent approach, TIBCO uses Instance keys for every Processing Unit. So if you could change this instance key, all agents had to us the same instance of the scorecards and the timeout-solution would still work. Searching the documentation the answer was also found really quickly on page 144 (Understanding and Working With Scorecards).
On that page TIBCO described how to set this instance key via CDD.

Any agent that uses scorecards, and also uses Cache Manager, must be assigned a
unique key so that the correct scorecard can be retrieved from the cache. The key
is set in the Processing Unit tab of the CDD.

I also tried this approach on a little test project and it worked right away.

What this means is, you can change the behavior of Scorecards depending on your needs. If you need a per Instance Variable you just have to set different Instance keys for every Agent. On the other hand if you want a globally shared Variable you can achieve this by setting the same value to all instances.

function url_decode(str){
var hex = /^[0-9a-fA-F]{2}/;
var out='';
var arr = str.split('%');
if(arr.length<2) return str;
for(var i=0;i<arr.length;i++)
{
  /*look for hex values */
  if(hex.exec(arr[i])) {
    out += String.fromCharCode(parseInt(arr[i].substring(0,2),16))+arr[i].substring(2,arr[i].length);
  } else { if(i==0) out+=arr[i]; else out+='%'+arr[i];
  }
}
return utf8to16(out);
}

function utf16to8(str) {
    var out, i, len, c;

    out = "";
    len = str.length;
    for(i = 0; i < len; i++) {
	c = str.charCodeAt(i);
	if ((c >= 0x0001) && (c <= 0x007F)) {
	    out += str.charAt(i);
	} else if (c > 0x07FF) {
	    out += String.fromCharCode(0xE0 | ((c >> 12) & 0x0F));
	    out += String.fromCharCode(0x80 | ((c >>  6) & 0x3F));
	    out += String.fromCharCode(0x80 | ((c >>  0) & 0x3F));
	} else {
	    out += String.fromCharCode(0xC0 | ((c >>  6) & 0x1F));
	    out += String.fromCharCode(0x80 | ((c >>  0) & 0x3F));
	}
    }
    return out;
}

function utf8to16(str) {
    var out, i, len, c;
    var char2, char3;

    out = "";
    len = str.length;
    i = 0;
    while(i < len) {
	c = str.charCodeAt(i++);
	switch(c >> 4)
	{
	  case 0: case 1: case 2: case 3: case 4: case 5: case 6: case 7:
	    // 0xxxxxxx
	    out += str.charAt(i-1);
	    break;
	  case 12: case 13:
	    // 110x xxxx   10xx xxxx
	    char2 = str.charCodeAt(i++);
	    out += String.fromCharCode(((c & 0x1F) << 6) | (char2 & 0x3F));
	    break;
	  case 14:
	    // 1110 xxxx  10xx xxxx  10xx xxxx
	    char2 = str.charCodeAt(i++);
	    char3 = str.charCodeAt(i++);
	    out += String.fromCharCode(((c & 0x0F) << 12) |
					   ((char2 & 0x3F) << 6) |
					   ((char3 & 0x3F) << 0));
	    break;
	}
    }
    return out;
}

function trace(url){
var tracker = _gat._getTracker("UA-XXXXXXX-X");
tracker._trackPageview(url);
}

Now every time I need to track something I call this function with a custom build URL.

HTTP Connection - engine selector

So I changed the connector engine and restartet the test. This time I started small. I set the heap to 512MB and limited the maxProcessor to 500. What I then saw was unexpected. The engine filled up right to the ten thousand requests I send. There occured no Out-Of-Memory at all. What was also interesting was, that the engine held 10k connections despite the maxprocessors was set to 500.
So to conclude the result, the new connector is quite impressive when you need to handle a lot of simultanous connection and have not a lot of memory to spare. On the other hand, when you use it, you loose some of the TIBCO integrated features to limit your load. Further to that, the TIBCO documentation states that due to the single thread arcitecture you increase latency. So as always there is a tradeoff.

One final sidenote. I had some issues with the BusinessWorks 5.7.1 engine so I upgraded it to 5.7.2. Than it ran without a glitch.

graph.height.pref=712
graph.width.pref=1515

These values represent the default size which the Designer allocated for its canvas. As you can see the default is pretty small on this one. I also found installations where both values where ten times larger then this. So far I found no performance penalty to this.
The only pattern I found is that newer installation had smaller values as default. Where this value comes from and how it is determined stays unclear to me.

One other thing I found during my research. If the panel is to small for the current process you can drag one activity right next to the border and then start to move it via keyboard (‘Shift + Cursor’). By doing so you drag the icon out of the canvas, but the position will be updated internally. After that you just need to refresh the process view and voila you have expanded the canvas size. But this is just a quick and dirty work-around.

CREATE TABLE pages
(
  id bigint NOT NULL,
  url character varying(255),
  md5 character(32),
  CONSTRAINT pages_pkey PRIMARY KEY (id)
)

The faster lookup should mainly be possible through the shorter column length (and therefore smaller index). Actually I don’t know if the fixed width is good or bad here, but hashes usually don’t vary in length. After creating this table I added a B-Tree unique Index to the md5 column to enable a fast lookup.
After a while a noticed a rather high CPU load on lookups for this table so I tried to analyze the problem. First I tried the obvious through psql.

cloud=# explain analyze select * from pages where md5 ='abc';
                                                     QUERY PLAN
---------------------------------------------------------------------------------------------------------------------
 Index Scan using i_pages_md5 on pages  (cost=0.00..8.50 rows=1 width=166) (actual time=0.046..0.046 rows=0 loops=1)
   Index Cond: (md5 = 'abc'::bpchar)
 Total runtime: 0.157 ms
(3 rows)

As the explain shows all works perfectly fine and lookups shouldn’t be a problem. So there had to be something different what was going on.

After that I tried the same select with an actual md5.

cloud=# explain analyze select * from pages where md5 = md5('abc');
                                                  QUERY PLAN
--------------------------------------------------------------------------------------------------------------
 Seq Scan on pages  (cost=0.00..32017.63 rows=3994 width=166) (actual time=1203.699..1203.699 rows=0 loops=1)
   Filter: ((md5)::text = '900150983cd24fb0d6963f7d28e17f72'::text)
 Total runtime: 1203.769 ms
(3 rows)

Now you can see the plan does change quite a lot. I have a full table scan instead of an index scan. You can also see that the query time increases nearly by factor ten thousand.

The reason for this dramatic change is a simple type mismatch. For whatever reason the md5 function will be evaluated to a string of the type text. To create a match with the column md5 all values had to be casted to that type. The side effect of this is that the index can no longer be used, because it is of the wrong type.

To solve this I just had to cast the result of the md5 function back to something that is compatible with the index type. In my case I used a fixed width character field which is represented in the database as bpchar (blank padded character). So after modifying the query to the following I was back on index usage.

cloud=# explain analyze select * from pages where md5 = md5('abc')::bpchar;
                                                     QUERY PLAN
---------------------------------------------------------------------------------------------------------------------
 Index Scan using i_pages_md5 on pages  (cost=0.00..8.50 rows=1 width=166) (actual time=0.141..0.141 rows=0 loops=1)
   Index Cond: (md5 = '900150983cd24fb0d6963f7d28e17f72'::bpchar)
 Total runtime: 0.199 ms
(3 rows)

for the Administrator:

#!/bin/bash

DOMAIN=tibcoesb
USER=esb
TIBCO_HOME=/home/$USER/tibco
PATH=/bin:/usr/bin:/sbin:/usr/sbin
NAME=tibco-admin

start() {
# Start daemons.
echo "Starting Tibco Admin"
/bin/su $USER -c"cd $TIBCO_HOME/administrator/domain/$DOMAIN/bin;./tibcoadmin_$DOMAIN 2>&1 | /usr/bin/logger -t $NAME" &
echo "done"
}

stop() {
# Stop daemons.
echo "Shutting down Tibco Admin"
killall tibcoadmin_$DOMAIN
echo "done"
}

case "$1" in
  start)
	start
	    ;;
  stop)
	stop
    	    ;;
  *)
        echo $"Usage: $0 {start|stop}"
    	exit 2
esac

for the Hawk

#!/bin/bash

DOMAIN=tibcoesb
USER=esb
TIBCO_HOME=/home/$USER/tibco
PATH=/bin:/usr/bin:/sbin:/usr/sbin
NAME=tibco-hawk

start() {
# Start daemons.
echo "Starting Tibco Hawk"
/bin/su $USER -c"cd $TIBCO_HOME/tra/domain/$DOMAIN;./hawkagent_$DOMAIN 2>&1 | /usr/bin/logger -t $NAME" &
echo "done"
}

stop() {
# Stop daemons.
echo "Shutting down Tibco Hawk"
killall hawkagent_$DOMAIN
echo "done"
}

case "$1" in
  start)
	start
	    ;;
  stop)
	stop
    	    ;;
  *)
        echo $"Usage: $0 {start|stop}"
    	exit 2
esac

The easiest way to achieve this would be to change the authentication for this service. Unluckily this service is used by a lot of other clients, so I can’t change it out of the blue. Next thing that came to my mind was using the underlying tomcat to do some IP-based blocking. After some searching around I gave that one up. Tibco has a really complex deployment model for BusinessWorks so I wasn’t able to even find the process-specific section for the tomcat engine.
After some searching around in the documentation (actually several hours – hooray to a usable documentation) I found some parameter which allows you to block IPs with some TRA-file parameters. In case you want to look it up yourself it is in the BusinessWorks documentation under ‘Administration’ -> Chapter 8 ‘Custom Engine Properties’ -> ‘Available Custom Engine Properties’ (and another hooray to the excellent formatting – Note to Tibco: HTML has tags, so you can do more with the content then continuous text – even some breaks would enhance the experience).
So back to the point:

bw.plugin.http.server.allowIPAddresses
bw.plugin.http.server.restrictIPAddresses

Both tags are rather self explanatory so you can use right away. Tibco says you can use it with a single IP, a list of IPs (comma-separated) or regular expression. As stated before this highlight of a documentation doesn’t say what reg-exp syntax means. I guess Tibco means they use the Tomcat engine for evaluation of these values an that means they use the Java syntax described here.

A lot of people out there use services like delicious where you tag you favorite sites and make this available to other users. Now I started to grab that data and began to build a massive tag cloud. After some time the site collected hundreds of thousands of links with their corresponding tags. So now you can start on the site and search for tags which interests you. These search tags will then be correlated against the cloud database and you get the most active links for your tags. So here is an example.
Let’s say you are interested in a tomcat tutorial.
tags:

tutorial, tomcat

results:

howtoforge.com
howtogeek.com
java.sun.com

Of course those results will be a link to the concrete tutorial (not just the entry page).
So far to the official part. For me this is more of a fun project. I prefer to start with some random tag and then wander around. It’s more like browsing, cause you start at points that you don’t already know. You have the chance to break out of your existing network of most used sites and see something new.

So have fun with it.

www.cloudtheweb.com

PS: For the implementation part – if you have any questions, just ask. I’m planning to explain some details about how it works on some later posts.

So enjoy the new features and lets hope the next release will not take another year.