a00: ask price
b00: bid price
g00: day’s range low
h00: day’s range high
j10: market cap
v00: volume
a50: ask size
b60: bid size
b30: ecn bid
o50: ecn bid size
z03: ecn ext hr bid
z04: ecn ext hr bid size
b20: ecn ask
o40: ecn ask size
z05: ecn ext hr ask
z07: ecn ext hr ask size
h01: ecn day’s high
g01: ecn day’s low
h02: ecn ext hr day’s high
g11: ecn ext hr day’s low
t10: last trade time, will be in unix epoch format
t50: ecnQuote/last/time
t51: ecn ext hour time
t53: RTQuote/last/time
t54: RTExthourQuote/last/time
l10: last trade
l90: ecnQuote/last/value
l91: ecn ext hour price
l84: RTQuote/last/value
l86: RTExthourQuote/last/value
c10: quote/change/absolute
c81: ecnQuote/afterHourChange/absolute
c60: ecnQuote/change/absolute
z02: ecn ext hour change
z08: ecn ext hour change
c63: RTQuote/change/absolute
c85: RTExthourQuote/afterHourChange/absolute
c64: RTExthourQuote/change/absolute
p20: quote/change/percent
c82: ecnQuote/afterHourChange/percent
p40: ecnQuote/change/percent
p41: ecn ext hour percent change
z09: ecn ext hour percent change
p43: RTQuote/change/percent
c86: RTExtHourQuote/afterHourChange/percent
p44: RTExtHourQuote/change/percent

These are the ones I found in the javascript sources. For most of it I don’t have a clue what it is for so don’t ask me what it means.

Next thing I want to mention is the result. A general approach would be to request your data and then run it through a proper JSON parser so you can work with the data as an object. Luckily this doesn’t work. The JSON returned is not 100 percent compliant with the JSON definition. Yahoo packs these symbols (seen above) not into quotes, so the parser denies to read it. A simple way around it is to string replace all possible symbols with quotes.

That’s it so far. I will post more if I have new information to share.

• curl – to send and receive the http request

• transform – a primitive tool to do streaming operations within one line

• spidermonkey shell (a javascript shell which can parse and reformat the data)

The complete logic will be done in the javascript. So lets start with the curl command line:

curl -s -o - -N 'http://streamerapi.finance.yahoo.com/streamer/1.0?s=JAVA,MSFT&k=l10&callback=parent.yfs_u1f&mktmcb=parent.yfs_mktmcb&gencallback=parent.yfs_gencb'

Let’s see what we have here. First we call the yahoo streaming api and want the current price (l10) for the stocks of Sun ans Microsoft. The callback part cannot be changed. If you change this part the whole request will not succeed. Also important is to get the output to STDOUT so that we can pipe the output to the next application.

Second part of the work is just to call the transform application (further explanation here).

The third part is to pipe the output of the transform process into the javascript shell. I started the shell with the following command:

js -f script.js

The script script.js look like this:

yfs_u1f = function(tmp) {
try{
print("msft: "+tmp.MSFT.l10);
}catch(ex){}
try{
print("java: "+tmp.JAVA.l10);
}catch(ex){}
};

yfs_mktmcb = function(tmp) {
/*ignore timestamp */

};
var parent=this;
parent.yfs_u1f = yfs_u1f;
parent.yfs_mktmcb = yfs_mktmcb;

while(1==1){
var t = readline();
if(t.substr(0,3) == "try"){
eval(t);
}
}

First we have to implement the callback functions which will be called from the http response. Then we construct an object called parent where we map these functions into. Now we have a working construct to receive the data and are able to work with it in our shell. What we need now is a little while loop to continuously read from STDIN and wait for new data. By the way accessing the tmp variable in the callback function seems somewhat complicated to me. I’m sure there is an easier way to access it but I have no clue how. If you have an idea how to do it better please post it to the comments.

The complete bash statement would look like this:

curl -s -o - -N 'http://streamerapi.finance.yahoo.com/streamer/1.0?s=JAVA,MSFT&k=l10&callback=parent.yfs_u1f&mktmcb=parent.yfs_mktmcb&gencallback=parent.yfs_gencb' | /tmp/transform | js -f script.js

If you run this you should get this output:

msft: 17.83
java: 4.47
msft: 17.84
msft: 17.86
msft: 17.81
java: 4.46

Now you can use whatever tools you want to work with that data. For me this will be piped directly into my postgres db for further processing.

Recently I saw a new feature on the yahoo finance homepage. Now it is possible to enable a streaming option and then you get a live update on certain values via AJAX. So I thought why not get this feature and let yahoo stream the content to the client instead of polling the server from time to time.

So I tried to capture the javascript request with wireshark, so that I could reproduce it. This is what it looks like:

GET /streamer/1.0?s=^GDAXI,USD=X&o=^N225,LHS.F,JI4.F,EEX.F,HEI.F,CBK.F,PRE.F,NDX1.F&k=c10,g00,h00,l10,p20,t10&j=c10,l10,p20,t10&r=0&marketid=us_market&callback=parent.yfs_u1f&mktmcb=parent.yfs_mktmcb&gencallback=parent.yfs_gencb HTTP/1.1
Host: streamerapi.finance.yahoo.com

Here is what I got on this request so far:

parameter ‘s’ -> symbol on the main site (what you currently looking at)

parameter ‘o’ -> that is the ticker on the top

parameter ‘k’ and ‘j’ -> that are the values that are transfered

c10 -> unknown

g00 -> day low

h00 -> day high

l10 -> current price

p20 -> unknown

t10 -> timstamp

a00 -> ask

b00 -> bid

With that data you can build a Push-Client with your own custom data. Here is a sample Request for what I needed.

wget -Obla 'http://streamerapi.finance.yahoo.com/streamer/1.0?s=^GDAXI,USD=X&o=BEI.F,SIE.F,PRA.F&k=l10,a00,b00,g00,h00&j=l10,a00,b00,g00,h00&r=0&marketid=us_market&callback=parent.yfs_u1f&mktmcb=parent.yfs_mktmcb&gencallback=parent.yfs_gencb'

(Streaming the current ‘ask’, ‘bid’ and ‘price’ values for the stocks ‘BEI.F’,'SIE.F’ and ‘PRA.F’.)

The response for that looks like that:

<html>
<head>
<script type='text/javascript'> document.domain='finance.yahoo.com';</script>
</head>
<body></body>
<script>try{parent.yfs_mktmcb({"unixtime":1228213139,"open":1228228200,"close":1228251600});}catch(e){}</script>
<script>try{parent.yfs_u1f({"USD=X":{l10:"1.00",a00:"1.00",b00:"1.00",g00:"0.00",h00:"0.00"}});}catch(e){}</script>
<script>try{parent.yfs_u1f({"BEI.F":{l10:"42.31",a00:"41.70",b00:"41.53",g00:"42.31",h00:"43.95"}});}catch(e){}</script>
<script>try{parent.yfs_u1f({"SIE.F":{l10:"44.46",a00:"44.26",b00:"44.20",g00:"44.40",h00:"47.55"}});}catch(e){}</script>

No you can parse that document for the requested Data. But be carefull because the html structure is never closing (at least not as lang as the streaming goes on).