June 15, 2004
PubSub and Jabber's Network Topolgy 
The about the PubSub service is really exciting both for us and the rest of the Jabber community. We've been saying all along that IM and syndication belong together, and the PubSub service is the server side extension to that vision. The most exciting part for me about a PubSub like service, is the possibility to subscribe to a topic, and then get notified within in seconds of the actual post. It's basically like subscribing to Google results and being instantly notified when Google discovers a new webpage matching your query. Very exciting.
By the way, anyone reading the Jabber mailing lists could have put two and two together and realized that PubSub.com derived their names from Jabber's pubsub proposal which PubSub.com uses to expose their database to the outside world. Two months back I thought that this was just one big coincidence. [ Update: Turns out it was just a coincidence. Bob Wyman says they started using the name even before JEP-60 was first drafted. ]
Bob Wyman, PubSub's CTO, recently about PubSub's usage of Jabber on the Jabber Standard's mailing list, and asked for some feedback. I the obvious things that I thought would be needed to make PubSub's service behave like a first-class Jabber citizen. Justin Karneges also on the most relevant point that exposing PubSub's service to the rest of the Jabber network is essential to exploiting the full potential of PubSub's service. Bob raised some valid concerns regarding server karma which is used to regulate bandwidth so that other rogue servers/clients don't monopolize the available bandwidth.
After thinking about PubSub's potential problems with pushing a lot of updates to clients via various servers, I thought of a nice work around. Jabber has a really elegant 3 part protocol scheme for initiating file transfers. The first step, , is rather generic and is intended for use outside of just file transfers. Stream Initiation relies on "profiles" to fill in the implementation details for the service i.e. there's a . My idea is to have a stream initiation profile for initiating a direct end-to-end XMPP session.
Slightly strange, but not without merit. Ignoring the details for the moment, the end result is that I'll have a direct XMPP session ( direct TCP/IP connection) with the Jabber entity on the other end. The big advantage of this is that a service like PubSub can push as many updates as they like in my direction. My server will be really happy because it's not involved with routing each of the PubSub's Atom IQ packets.
Having clients support direct end-to-end XMPP sessions extends the Jabber topology. The current Jabber topology is for clients to connect to a server, and the servers talk among themselves to relay messages for clients on different servers. The technical terms for this is type of topology is federated. However, with direct e2e the Jabber topology would be more of a mesh network where the current Jabber topology is responsible for what could be consider the first tier for XMPP traffic, and the direct e2e would be the second tier for high bandwidth/low-latency traffic.
Another area aside from a Publish-Subscribe service that would benefit from direct e2e is . File sharing could require a good amount of data to be exchange especially if the directory-file structures being shared are very large.
These ideas need to be flushed out, but I think there's a lot of potential here.
The about the PubSub service is really exciting both for us and the rest of the Jabber community. We've been saying all along that IM and syndication belong together, and the PubSub service is the server side extension to that vision. The most exciting part for me about a PubSub like service, is the possibility to subscribe to a topic, and then get notified within in seconds of the actual post. It's basically like subscribing to Google results and being instantly notified when Google discovers a new webpage matching your query. Very exciting.By the way, anyone reading the Jabber mailing lists could have put two and two together and realized that PubSub.com derived their names from Jabber's pubsub proposal which PubSub.com uses to expose their database to the outside world. Two months back I thought that this was just one big coincidence. [ Update: Turns out it was just a coincidence. Bob Wyman says they started using the name even before JEP-60 was first drafted. ]
Bob Wyman, PubSub's CTO, recently about PubSub's usage of Jabber on the Jabber Standard's mailing list, and asked for some feedback. I the obvious things that I thought would be needed to make PubSub's service behave like a first-class Jabber citizen. Justin Karneges also on the most relevant point that exposing PubSub's service to the rest of the Jabber network is essential to exploiting the full potential of PubSub's service. Bob raised some valid concerns regarding server karma which is used to regulate bandwidth so that other rogue servers/clients don't monopolize the available bandwidth.
After thinking about PubSub's potential problems with pushing a lot of updates to clients via various servers, I thought of a nice work around. Jabber has a really elegant 3 part protocol scheme for initiating file transfers. The first step, , is rather generic and is intended for use outside of just file transfers. Stream Initiation relies on "profiles" to fill in the implementation details for the service i.e. there's a . My idea is to have a stream initiation profile for initiating a direct end-to-end XMPP session.
Slightly strange, but not without merit. Ignoring the details for the moment, the end result is that I'll have a direct XMPP session ( direct TCP/IP connection) with the Jabber entity on the other end. The big advantage of this is that a service like PubSub can push as many updates as they like in my direction. My server will be really happy because it's not involved with routing each of the PubSub's Atom IQ packets.
Having clients support direct end-to-end XMPP sessions extends the Jabber topology. The current Jabber topology is for clients to connect to a server, and the servers talk among themselves to relay messages for clients on different servers. The technical terms for this is type of topology is federated. However, with direct e2e the Jabber topology would be more of a mesh network where the current Jabber topology is responsible for what could be consider the first tier for XMPP traffic, and the direct e2e would be the second tier for high bandwidth/low-latency traffic.
Another area aside from a Publish-Subscribe service that would benefit from direct e2e is . File sharing could require a good amount of data to be exchange especially if the directory-file structures being shared are very large.
These ideas need to be flushed out, but I think there's a lot of potential here.
