You are here

Home › Content

Why Transports Matter

Mon, 17/09/2007 - 16:25 — sander

This is an open letter to all people related to the content of this article. So, you better read everything to see if that includes you ;-)

Instant messaging clients like Pidgin, Adium, Miranda and Kopete combine support for different instant messaging networks in one program. They often also support platforms that are not supported by the official client software. Therefore, these so-called multiprotocol clients are seen by many as a good thing for interoperability. However, I strongly disagree on this. Instead, my analysis sounds the opposite way:

  • Multiprotocol clients are a curse for interoperability.
  • Multiprotocol clients hinder adoption of open standards like XMPP and SIMPLE.
  • Multiprotocol clients even hinder their own adoption!
  • Multiprotocol clients explain why "the Firefox of instant messaging" does not yet exist.

In this article, I will explain these tough claims. Also, I will elaborate about a possible strategy for those who would like to get rid of closed non-interoperable instant messaging networks. In short this strategy is simple: we need to render the walled gardens pointless by destroying the reason they are more profitable than open networks.

Why a Curse and Why Transports?

In this section, I will tell you why multiprotocol clients do not help much to fight closed networks. Not much, as a lot of resources are wasted in these projects which could otherwise be used in a more effective way to get rid of closed networks in favour of open networks. Without closed networks there would be a very friendly and innovative environment suitable for the emerge of "the Firefox of instant messaging".

Ok, but what is the right way to fight closed networks? The right way is a combination of several dedicated XMPP clients with some darn good transports. Transports are server-side gateways that allow XMPP clients to connect to closed networks without any significant effort like reverse engineering or designing a multiprotocol architecture. To be more clear about this abstraction, transports translate the language of the XMPP client to the language of a client on a closed network without the XMPP client knowing that language.

Time for the core of this article...Why transports matter:

  • The quality of support for open standards like XMPP is lower in multiprotocol clients than in dedicated XMPP clients. This can be easily explained by the increased complexity of the software to support multiple protocols, but this is only a minor point that can be solved by having enough eye balls to catch bugs. More important is the fact that XMPP is less used by geeks compared to popular closed protocols. Hence, there are more eye balls looking for bugs in closed protocols support than to catch bugs in XMPP protocol support.

    Compare this with instant messaging clients dedicated to the open standard protocol XMPP. First, these clients don't need a complex multiprotocol software architecture. Second, their reason to exist is XMPP support. Thanks to this dedication, the quality of XMPP support is more likely to be much higher than that of unfocused multiprotocol clients. A bug that breaks XMPP support is a major issue in XMPP clients whilst it is a less major issue in multiprotocol clients as these clients will still be usable for other protocols.

    Transports matter because they force end users whom mostly use closed networks to contribute their "eye ball capacity" to XMPP support.

  • The 'instant messaging experience'(or innovativeness) of multiprotocol clients is lower than that of the official clients for the closed networks. They run behind because of the required reverse engineering, the complex software architecture, and the tendency to only support features common to the majority of supported popular networks. They tend to limit their feature set to the least common denominator of its constituents, which can be very small. They cannot influence the future of the underlying protocols as these protocols are controlled by the companies owning the closed networks. Where is the room for innovation? There is no room for this as multiprotocol clients simply can never take over the lead of the official clients in designing the future of the closed protocols.

    Dedicated XMPP client projects instead, fully focus on a single protocol that is extremely well documented and thus don't needs any reverse engineering. Moreover, XMPP client projects can have their say in the future of the underlying protocols and even can propose their own protocol extensions for standardisation. That's an open door for unlimited innovation!

    Transports matter because they allow XMPP client projects to fully focus on innovation without losing interoperability with closed networks and without being influenced or hindered by the closed networks. Transports allow XMPP clients to lead the instant messaging world, instead of following the closed networks owners, like multiprotocol clients tend to do. The later can be compared with Mozilla: no focus, complex, not sexy, following the leader(s), not popular, and so forth. The combination of dedicated XMPP clients and transports could become like Firefox: focused on open standards, innovative, taking over the leadership position, very popular, and so forth.

  • The image of open source multiprotocol clients hurts the broader open source image. Open source multiprotocol clients do not lead in their environment and are seen as poor extracts of the official clients. Unfortunately, this poor image of being a follower instead of a leader means the public will associate open source alternatives by definition with poor extracts of the real thing. Hence, people even may not try XMPP because of earlier unsatisfying experiences with multiprotocol clients.

  • Choice on closed networks is created by multiprotocol clients. Multiprotocol clients allow people who for some reason cannot or do not want to use the official client to connect to closed networks. This choice also reduces the vulnerability of closed networks to malware. You may think these are all good things, but in fact they hinder adoption of open standards like XMPP.

    Imagine a world without alternatives to official clients for closed networks. All people who are today contributing to the multiprotocol part of multiprotocol client projects would focus on XMPP and server-side transports. Client projects wouldn't be forced to do urge releases due to some compatibility change in the closed network protocol. Development would be totally independent from the closed network owners. Focus can be directed to innovative client features in the XMPP world. All end users using one of these clients in combination with transports would use XMPP. So, every user would contribute to the maturity XMPP, even if they only want to communicate with people on closed networks!

    Transports matter because they can lead to more and higher quality choice in the XMPP world. More choice because the existence of very good transports make it more interesting to create XMPP software. Higher quality because all end users will use the XMPP code, even if they only use their XMPP client to connect to closed networks.

  • Flexibility of multiprotocol clients is extremely low. Multiprotocol client projects are bloated. They need testers for all integrated protocols. The complexity of the multiprotocol architecture needs maintainance. Large changes in the support for one protocol are not easy if this could break support of other protocols, and if they make such changes they have to do this very carefully. Multiprotocol client projects tend to first add features common to all integrated protocols instead of adding innovative features.

    Compare multiprotocol client projects with XMPP client projects. Look at the number of innovative protocol features in multiprotocol clients compared to XMPP clients. Also consider how many coders are involved in the different types of projects. The following statistics from ohloh.net about contributors over the entire history of the project can help you. Of course, these numbers are not 100% accurate at all (e.g. some projects give translators commit rights whilst others don't), but still, the significant differences give an indication:

    Multiprotocol Clients XMPP Clients
    Pidgin (33), Kopete (100), Adium (52), Miranda (22) Coccinella (3),Tkabber (2), Psi (7), GOIM (5), Spark (10), Gajim (19), JWChat (3), Bombus (1), Freetalk (4)

    The fact that there are more active dedicated XMPP clients in the wild than there are active multiprotocol clients says a lot about the required community size to sustain a less complex but focused XMPP client.

    Transports matter because they move the maintenance burden and release burden (no forced releases) away from the client projects. They increase the projects' flexibility and development speed (less and shorter testing periods, less complexity). They move away the eventual legal issues in some countries from the client project to a server project which is more suitable to benefit from today's globalised world. A country eventually can forbid multiprotocol clients, but it never can forbid inhabitants to use a transport service hosted in another country without being punished with a trade war by other nations as this would be a trade barrier in services.

The Solution

Multiprotocol clients should become single protocol clients. To accomplish this goal I propose to "steal" community of multiprotocol client projects in favour of XMPP software so that their 'community resources' will become insufficient to sustain a complex design for multiple protocols. We also can ask them to drop support for closed protocols, but I'm afraid they will just laugh at us.

The Tactics

To "steal" community of multiprotocol client projects, he reasons why end users chose multiprotocol clients should be eliminated. Currently, I see two constructive ways to do this:

  • Help the Wine project to support the official clients. In this way, the hard task of reverse engineering and maintaining the proprietary protocol code becomes less urgent for people who contribute to multiprotocol clients, and they may stop doing their hard work. Also, end users will be more happy on their new platform as they can stay using the client they are used to. They can use all features of this closed network...including all misfeatures of the client. If this end user wants choice, he has to install an XMPP client.

    Note also that official instant messaging clients for closed networks are strategically an interesting target for the Wine project. Instant messaging clients are probably the most important application of younger people. These people like to experiment with new operating systems. However, they face huge switching costs as they cannot run the instant messaging client their friends use. The switching costs are higher if they need to use an alternative client as they need to learn a new interface for their most important application, and because the alternative multiprotocol client probably does not has all features of the official client.

    As a side note, you can see why the Mac OS X version of Windows Live Messenger can't be compared with the Windows version. You also see one of the reasons why Microsoft will not adopt XMPP soon: it hinders youngsters to switch to non-Microsoft platforms. But if the Windows version of Windows Live Messenger can fluently run under any platform thanks to Wine, Microsoft loses a good reason to not adopt XMPP. Note that I do not say they will per se adopt XMPP afterwards!

  • More and better transports are needed to allow XMPP clients to compete effectively with the key feature of multiprotocol clients: interoperability with multiple closed networks. Multiprotocol clients can't innovate as fast as XMPP clients can (see above), so we can tackle them on the interoperability feature which is XMPP's core feature.

Wishlist for Transports

Currently, there is lot's of room for improvement of transports in XMPP land. Because transports are a key feature in the success of Coccinella and other XMPP clients, we would ask the broader XMPP community to improve the current situation regarding transports.

Wishlist for transports:

  • Focus on reliability of service: uptime, hot code update, fail-over, clustering to bypass blocking, and so forth.

  • Focus on reliability of compatibility. Fast deployment of fixes for protocol updates by the closed network owner is what we need. The faster transports are updated, the more expensive and thus the less interesting it becomes for closed network owners to change their protocols to block alternative software.

  • Focus on basic interoperability features only. Text messaging and presence should be the focus. Other features are not important and only make it harder to maintain "reliability of compatibility".

  • More closed networks. Transports should cover all closed networks.

  • XMPP based ideology. Transport projects should favour the XMPP world in the first place. "What XMPP feature will I map to the closed network today?" instead of "What closed network feature will I map to the XMPP world today?" XMPP protocols should be written first and only after that we should look if they can be mapped to closed networks, we should not write XMPP protocols to get similar features as the closed networks have. Taking the lead means not copying features, instead it means innovating with unique protocol features.

    Also, transports should be friendly only to people on the XMPP side of the gateway. People on the other side of the gate should get what they have chosen for: no openness, no help, and a bad experience. For example, an error message should sound "You cannot send this file to your contact because your software is not interoperable. Please contact <name of network owner> for more information." instead of "You cannot send this file to your contact because your contact does uses an XMPP client. Please consider switching to XMPP to fix this issue.". When the XMPP user wants to send a file to his contact on the closed network it should sound: "We are sorry but this file cannot be send because your contact uses a chat system that is not interoperable. Please consider to ask your contact to switch to the open XMPP. This issue cannot be fixed without the help of the closed network owner. May we therefore also ask you to sign the petition on <URL>? By supporting this petition, you ask the owner to open its closed network and by this offering a better service to you. This interoperability issue can't be fixed without their help".

    As you can see, with a difference in approach we can forward all troublesome interoperability experiences to the real responsible instances (the owner of the closed networks). In this way end users will not associate the issues to XMPP and they may even understand open standards fanboys! This can mean a big thing for the image and popularity of XMPP.

  • No lock-in feature of a server. As is normal in good open standard communities, any lock-in is seen in the XMPP community as evil because it is bad for the emerge of the open standard. Lock-in also can be the root of a software monoculture. We also don't want that. Luckily, server projects are making their components compatible with other servers. And hopefully, this server will follow soon. But we are not there yet.

  • Shaped to die fast. Transports' main goal should be to die as fast as possible. If a transport project dies it may mean the related closed network became open or went offline! Transports should be tools of judo strategy:

    The central message of Judo Strategy is that [instances] facing more powerful opponents should avoid head-to-head struggles and other trials of strength. Instead, by employing speed, agility, and creative strategic thinking, they can shape the dynamics of competition in ways that prevent rivals from taking full advantage of their superior strength. At its most effective, judo strategy can even transform an opponent’s apparent advantages into strategic liabilities that undermine his or her ability to compete.

  • Sexy software. Transports should become sexy software projects with a nice website, good documentation, a good community, easy installation and configuration, and so forth. But this item is obvious :-)

About me

In case you want to argue on my thoughts, consider all next facts before you reply:

  • I am no real coder; my speciality is copy-paste coding :o) and other ways to contribute to open source projects
  • I like open standards
  • I like XMPP
  • I like unlimited choice in all XMPP software
  • I don't think XMPP software is so good it can't be improved
  • I don't believe Coccinella is so good it can't be improved
  • I also like other XMPP clients; monoculture is evil
  • I like successful open source projects like Firefox and Ubuntu because they increase popularity of the broader open source world
  • I don't want multiprotocol client projects to die, they just should become dedicated to only one open standard (preferably XMPP)
  • I prefer Konqueror over Firefox, but I also like Firefox
Story type: 
Sander's blog
Various

Comments

Mon, 17/09/2007 - 16:34 — ghudson

You're correct that the need for multi-protocol clients hinders client development. A project like Coccinella would be a huge undertaking if it tried to support all the commercial IM protocols as well as XMPP.

Unfortunately, transports are a horrible solution from a security perspective. They require handing over to some XMPP server your authentication credentials for the commercial IM services. If you run the server yourself, that's fine, but to do that you have to have an always-on machine with a stable DNS address and substantial sysadmin expertise, which does not describe most of the target audience for IM clients. If you don't run the server yourself, you've given away the car keys.

They're also not a very sustainable solution. Commercial IM providers don't like their users using non-approved clients, which means they dislike transports. It's very easy for them to recognize transport servers and block them at the IP level, and we've seen that happen in the past with popular transport servers.

So I'm never likely to recommend that MIT offer transport services on its XMPP servers. We'd be trafficking in passwords we have no business knowing and we'd be subject to being turned off by the commercial providers at any time, with no workaround.

I wish I did have a good solution, other than "hope and hope and hope that XMPP gets more adoption," either from users abandoning the commercial IM services or from the commercial IM services opening up their networks. But I'm sure that transports aren't it.

Mon, 17/09/2007 - 16:47 — ff

Transports, but installed locally together with the client (or on some trusted pc) and run as a daemon. Yes, there's some overhead and some managing complexity, since users must install one more sw package, but we avoid all the security risks, and in this way transports are seen as multiprotocol clients by legacy networks, with less risks of having their IPs blocked.

Mon, 17/09/2007 - 17:43 — sander

I already thought of that. Though, the problem is that it is less convenient to setup. Also, the end user needs to update the software himself when the closed network changes protocol. Transports are more friendly to end users. To reduce the risk to get the IPs of the transports blocked, it is IMO better to spread the connections using a clustered transport of which the nodes have different IPs. If you limit the maximum allowed number of connections per IP to let's say 100, the closed network owner may think you are a company using his client. It cannot consistently block IPs with this amount of connections. If they do, they risk to block enterprise users which may deploy their own XMPP server because of this. And that is probably not what the closed network owner wants ;-) So, he will be very careful with blocking, and if not, transports still help XMPP as also "good users" their IP gets blocked ;-)

--
Mvg, Sander Devrieze.

Mon, 17/09/2007 - 17:22 — sander

Unfortunately, transports are a horrible solution from a security perspective. They require handing over to some XMPP server your authentication credentials for the commercial IM services. If you run the server yourself, that's fine, but to do that you have to have an always-on machine with a stable DNS address and substantial sysadmin expertise, which does not describe most of the target audience for IM clients. If you don't run the server yourself, you've given away the car keys.

IMO this is a non-issue:

  • If you don't trust your server to store the passwords of your closed network accounts; why would you trust it to store you XMPP account? If you don't trust other servers, you should run your own XMPP server and don't use closed network accounts at all
  • There is no difference with webmail services fetching emails from other POP/IMAP accounts...most people don't care about storing the password for their other email account on the server (e.g. GMail). Why would these people refuse to store the passwords of their instant messaging accounts on Google Talk?
  • The average end user doesn't care about this

They're also not a very sustainable solution. Commercial IM providers don't like their users using non-approved clients, which means they dislike transports. It's very easy for them to recognize transport servers and block them at the IP level,

Why do you think this is easy? I'm not so sure about this. Remember that if they accidentally block the IP of a big company using an approved client for a closed network, this company will lose it's communication channel. I'm pretty sure they will not like this...and they may decide to deploy their own XMPP server because of this ;-) ...another way how transports can help adoption of XMPP. Oh yes, this quote is relevant again: "At its most effective, judo strategy can even transform an opponent’s apparent advantages into strategic liabilities that undermine his or her ability to compete." The opponent's advantage is his control over his network and thus his ability to block access. Undermining is that transports can make them also block the "good users" So, the more the opponent tries to block transports, the more "good users" may get blocked and get pissed about the closed network ;-)

and we've seen that happen in the past with popular transport servers.

AFAIK they only block IPs with too many connections...so popular transports sometimes get blocked in this way. See also the first wishlist for transports-->"clustering to bypass blocking"

and we'd be subject to being turned off by the commercial providers at any time, with no workaround.

With enough(*) transports in the wild, the closed network owner will spend all his time in trying to block all transports...thus less time left for innovation...

(*) I guess Google and others have an unlimited amount of IPv6 IPs ;-)

--
Mvg, Sander Devrieze.

Wed, 19/09/2007 - 11:05 — pihhan

Your ideas are very nice, but very unrealistic. Fact we are missing cool & stable application for XMPP like what firefox is for web is essential. But many things in XMPP is not ideal and we need to offer solid replacement of their proprietary network client before we force them from it.

Understand, what is gateway on XMPP and what is multiprotocol client in nature. XMPP gateway is great service sometimes, but it is very dificult to find good and stable gateways for foreign network. One thing is gateway, that can interconnect with foreign network as server to server. For example, email gateway. I think email gateway is vital for XMPP, as can connect open technologies together. But if you are looking for solid gateway, you will propably find it very limited in features. Biggest problem is of course how to eliminate spam. That can be improved easily.

But gateways connecting somewhere as client are different thing. There are not as many XMPP users as users of popular network like MSN, Yahoo or ICQ. Still, there are too much XMPP users for a little of avaiable and working transports to foreign network. For example, why does not jabber.org host any foreign network transport? It does need resources, it does need work to keep it running. Changes in closed protocol are usual, you need to be prepared for update as soon as possible. Users cannot help you, admin is only one who can fix it. In other words, you need people and resources to provide such transport. Problem arrives when you have big XMPP server with hundreds to thousands users connected at the same time. Then you will have also part of them using foreign transports to keep contact with their friends, who are not yet prepared to switch to XMPP. It is big difference writing single connection client, or 5 connection client. And it is different to write service for hundreds to thousands user online. Not every coder knows how to write such service, even if he is able to make multiprotocol client better. But it is not only about solid transport itself. Foreign networks will block everyone from single IP often, if he is doing too much connections in short time. If you have transport, you have hundreds connection from the same IP. It network does block your IP, you have serious problem, as all hundreds users lost their connection. Then more they try to reconnect, they make more connections and keep it blocked. This is often situation why ICQ transport does not work.

Big difference is also that provider of XMPP service needs to pay for server for foreign networks and for traffic it does generate. They sponsor XMPP to their users for free usually, but why should they sponsor foreign gateways? There might be also legal problems, as almost every network does not permit connection to their network from unofficial clients. Noone will take to court every single user of multiprotocol, but why not sue company providing reliable access to their network for serveral thousands users and has money to get for breaking the license?

Simply, multiprotocol clients are not great, but they does offer to user not primary on XMPP to simply add one icon to their program, and be connected to XMPP. Transports are not always able to fulfill such uptime and reliability, because not everytime it is only their fault. If you are providing service for free for some hundreds users, you would propably like multiprotocol clients, because it does not drain your resource for every single client on foreign network.

I think we are missing good libraries for foreign networks, which could be shared in XMPP gateway and several multiprotocol clients, so it would have better features and better reliability. Almost every client has its own code, which is problem.

I believe we need to make jingle working, before we will be able to get more users to XMPP.

Wed, 19/09/2007 - 12:20 — sander

Your ideas are very nice, but very unrealistic. Fact we are missing cool & stable application for XMPP like what firefox is for web is essential. But many things in XMPP is not ideal and we need to offer solid replacement of their proprietary network client before we force them from it.

Who said adding the missing "cool & stable application for XMPP" is impossible? ;-)

For example, why does not jabber.org host any foreign network transport? It does need resources, it does need work to keep it running. Changes in closed protocol are usual, you need to be prepared for update as soon as possible. Users cannot help you, admin is only one who can fix it. In other words, you need people and resources to provide such transport.

I agree with you that the current transports are not yet ideal. In the wishlist for transports I point out how ideal transport projects look like. What you say is exactly what is in this wishlist for future transports... E.g. who said maintaining and updating a transport can't be made so easy that it's just a matter of apt-get upgrade? ;-)

Problem arrives when you have big XMPP server with hundreds to thousands users connected at the same time. Then you will have also part of them using foreign transports to keep contact with their friends, who are not yet prepared to switch to XMPP. It is big difference writing single connection client, or 5 connection client. And it is different to write service for hundreds to thousands user online. Not every coder knows how to write such service, even if he is able to make multiprotocol client better. But it is not only about solid transport itself. Foreign networks will block everyone from single IP often, if he is doing too much connections in short time. If you have transport, you have hundreds connection from the same IP. It network does block your IP, you have serious problem, as all hundreds users lost their connection. Then more they try to reconnect, they make more connections and keep it blocked. This is often situation why ICQ transport does not work

Every time closed network owners block an IP, they risk to block big companies using their closed network (as they also have many connections). If something like that happens, this company may decide to deploy its internal XMPP server; something the closed network owner probably does not want. ;-) So, if we can make it very risky for the closed network owners to block IPs that may be either transports or otherwise companies, this would be judo strategy in its most effective form (remember the quote "At its most effective, judo strategy can even transform an opponent’s apparent advantages into strategic liabilities that undermine his or her ability to compete."...the advantage is the closed network owner's ability to control the access to his network and so to block IPs) B-)

Big difference is also that provider of XMPP service needs to pay for server for foreign networks and for traffic it does generate. They sponsor XMPP to their users for free usually, but why should they sponsor foreign gateways?

For the same reason they also provide the XMPP service for free I guess...a better user experience. Also, I had the Google Talk service in mind when I wrote this article...

There might be also legal problems, as almost every network does not permit connection to their network from unofficial clients.

As already noted in the article, this is the same possible legal problem in some countries that multiprotocol clients face. Though, the risk is less: if some judge in a country decides it is illegal and the software may not be distributed in that country any more, only the transport can't be distributed in that country whereas a whole multiprotocol client would be illegal in that country (which then needs a name change and remove the support for the forbidden network to be legal in that country again)

Noone will take to court every single user of multiprotocol, but why not sue company providing reliable access to their network for serveral thousands users and has money to get for breaking the license?

Only multiprotocol client projects have been pestered by closed network owners so far. Remember how the Gaim/Pidgin project suffered from a legal battle with AOL. With transports and XMPP clients, only transports may be the target for legal actions, the XMPP clients always stay on the safe side and thus development of XMPP clients can't be delayed/hindered by closed network owners.

Take also into account that in several countries the legal status of transports is 100% certain; perfectly legal. So, it will be very hard for closed network owners to take down all servers with transports....much harder than targeting a single multiprotocol client project...

If you are providing service for free for some hundreds users, you would propably like multiprotocol clients, because it does not drain your resource for every single client on foreign network.

Some hundreds of users is not much...a number transports can easily handle without risking being blocked by closed network owners. There are companies with lot's more concurrent connections on single IP to a closed network...

I think we are missing good libraries for foreign networks, which could be shared in XMPP gateway and several multiprotocol clients, so it would have better features and better reliability. Almost every client has its own code, which is problem.

I heard from coders this would be hard because the libraries of multiprotocol clients are not optimized for lot's of connections. This may be fixed of course.

I believe we need to make jingle working, before we will be able to get more users to XMPP.

Already tried the VoIP features of the client of this domain? ;-) (still lots of room for improvement though, like video conferencing support)

--
Mvg, Sander Devrieze.

Thu, 20/09/2007 - 09:08 — Wertigon

There's just one thing I disagree with you here, and that is that transports should be limited to messages and presence. I think, today, that people have come to expect two more things; File Transfers, and Avatar support.

These two are crucial to get working on transports. It's more or less the baseline of IM nowadays. So, having a transport that have avatar support and a reliable transfer mechanism, like Jingle File Transfers (even if it's only limited to, say, 10 MB large files) is crucial to get adoption going. But, anything more advanced than that...

I can say that in my country MSN is huge. Everyone have an account there, except for a few geeks like me who run Jabber-only. I know that most other countries are the same, just replace MSN with $BIGGEST_IM_NETWORK. And, sadly enough MSN/Live! Messenger is state-of-the-art when it comes to IM and fancy features... :(

Finally, I'd just like to say that you're spot on when it comes to being friendly on the XMPP side and unfriendly on the "other side". If people doesn't know it exists, how can they make the switch?

Thu, 20/09/2007 - 12:23 — sander

There's just one thing I disagree with you here, and that is that transports should be limited to messages and presence.

I agree with you on this; a "focus on basic interoperability features" does not mean other feature are not welcome, they just shouldn't be the priority.

I think, today, that people have come to expect two more things; File Transfers, and Avatar support.

I agree that these are nice, but nobody will die without these. Also, IMO it is better to wait until XMPP clients switch to a common avatar protocol (User Avatar using PEP probably) and a common file transfer protocol (I believe this is on the TODO list of the XSF). Oh yes, the item "XMPP based ideology" is valid here: better wait until the XMPP community deployed common and stable protocols before you map them using a transport.

And thanks for feedback (also other "feedback people"; thx :-) )

--
Mvg, Sander Devrieze.

Tue, 25/09/2007 - 18:28 — ghudson

I have a little more insight now into how commercial providers would see transports (specifically AOL; answers may vary for MSN and Yahoo):

* AOL has become more open to the idea of using non-approved clients to communicate with AIM (see developer.aim.com, for instance) as long as they don't perceive a major threat to the market share of their branded client. So AOL is unlikely to shut down your transport simply because they find out it's an XMPP transport.

* However, AOL does do rate-limiting by IP address in order to clamp down on spam and other forms of abuse. If someone starts spamming AOL users through a transport you use, you can expect a service interruption.

* A NAT gateway looks just like a transport to a commercial service provider... and it turns out that NATs with large numbers of users behind them do in fact create problems for the commercial service networks (and vice versa).

So, it's still not really desirable to be aggregating lots of users onto a single IP address via a transport. A transport with a big IP address cluster might have still have issues with spammers.

The idea of transports as locally deployed software has promise, though there are lot of logistical issues to solve there in order to get a friendly user experience.

Mon, 07/07/2008 - 03:18 — Chascon

I'll just keep it short, transports need to be integrated as plugins into a Jabber client.

Mon, 07/07/2008 - 04:46 — sander

When you integrate transports as plugins, you'll have a multi-protocol client instead of an XMPP client.

Mon, 07/07/2008 - 18:05 — Chascon

Not really. When you criticize multi-protocol clients you're referring to monolithic designs, else they would not require the larger numbers of developers, man power, and wasted time rewriting client code. Maybe I do propose a multi-protocol client, but a XMMP and a modular one, at that.

Let me elaborate a bit ...
On the issue of keeping transports separate from the client --for all the reasons already discussed (legal and ease of use) -- the answer is "easy". The transport-client model could be re-conceptualized as a modular one where proprietary transports are installable as plugins post-install . These plugins could be distributed via a side project not directly related to any client project, from a country where this is perfectly legal. The the case that any proprietary IM protocol changes, just update the plugin. This would take care of the legal issue, and security concerns of handing over the car keys to an unknown server admin.

This way, there also need not be the concern of setting up a jabber server, to side-step the handing over one's keys. As a side note, I'm thinking of setting up Bitlbee to work with Coccinella simply because it seems easy to set up, and because --in my experience-- it is vastly more reliable than Jabber transports for proprietary IM services. For this last reason, I haven't looked into setting up a Jabber server. My own Bitlbee server would allow me to continue using Coccinella, as I'm refusing to continue handing over my keys and I exclusively communicate with proprietary protocol end-users. Has anyone looked at the info that is accessible when one connects to an proprietary IM via a jabber server? Imagine handing this over to strangers. If Jabber transports were reliable and I could install them locally and easily, I would gladly use them again --especially if I could load them as client plugins. So in this regard, we agree, transports _are_ important.

Despite that my latest user experience suggests that transports haven't improved much in years, there is something I like about Jabber, and I only go back to Adium because of proprietary IM interoperability reliability which is totally missing from corresponding Jabber transports.

I agree with most everything said in the article, although I find it unrealistic. For instance, how are we to "steal" IM users from other projects? I think it would be much easier to provide proprietary interoperability as a standard service, post-install obviously for legal reasons, and essentially educate, inform, and tempt people to use XMMP based technologies already available via their client, perhaps via a clickable "Did you know" pop-up window that leads into an informative short lesson, if they should so choose. I'm pretty sure Gimp does this.

In terms of project resource efficiency, the modular approach I suggest stratifies resources making for an efficient use of coding, meaning that a change in proprietary IM would not necessitate a recoding of the client app, but only of the transport. After all, there should be not be any need to recode the client since transports are gateways or translators of XML to proprietary IM lingo and have the entire responsibility of acting as the go between. What does this mean? Well it means that changes in protocols only affect transports and not clients.

Although I can believe that multi-protocol clients are run inefficiently, you don't take into account the resources that go into coding transports. Of course, you could argue that these are coded only once for all Jabber clients to benefit, thus still being efficient. The plugins architecture I propose could be also be decided upon by consensus, thus making it operable with multiple clients, not withstanding porting to the native platform being run at that instance. But even in this last instance, a XML plugin should be arch independent, thus rendering the need for porting non-existent --"drag and drop into place and run client" as a last resort.

Lastly, it's not as if the built-in transport concept hasn't been done before. See, Talkonaut as an example.
"Talkonaut is a feature-rich IM chat client that is based on Jabber protocol ... with built-in transport support for MSN, Yahoo, AIM and ICQ protocols."

And yes, there isn't going to be a lot of inertia until VOIP gets off the ground as something stable. I haven't gotten it to work. But besides this, transports need to be easily accessible to new users --as in built-in-- if Jabber is going to get a chance at getting more users, as well as easy file transfers.

The other things missing from mass Jabber adoption is a concerned campaign, as what happened with the Spread Firefox project. For this to work, you need a recognizable end user product, but the diversity of Jabber clients is not conducive to this. Sure there's some diversity with Mozilla code, but these Mozilla based programs are minimal and don't have the backing that Firefox has, so they don't ruin Firefox's product recognizability. This element will probably always be missing, unless a huge for profit company adopts a Jabber client and starts a campaign.

As it stands, Jabber is like a pre-Christian landscape, where there are multiple deities inter-related somehow --often with endless local variations-- while the proprietary IM world is collectivised --to use a sociological term. So the question is, how do we collectivise (or "convert)" people from one to another? I propose that we use the other methods I write about here, but we'll probably always be missing a concerted campaign to proselytise. Perhaps we need a legitimising agent, as Constantine, this by way Google and their Google Talk, but that software (ie., their VOIP) is not even current with Jabber development (dare I say, "vapour-ware"?).

What's needed? Something like the following list.

-modular arch, hopefully on which there is community consensus-for security, legality, & efficient coding
-educational prompts
-better transports
-campaign/product recognizabilty
-finally get VOIP off the ground!
-file transfers

Maurice Cepeda
Chascon

Mon, 07/07/2008 - 21:57 — sander

This would take care of the legal issue,

If there are any legal problems, this will only be with software that the user actually has to download. Transports cannot be seen as a product; transports are for sure a service. A country can theoretically forbid its inhabitants to use third-party software to connect to proprietary service. On the other hand, they cannot forbid its inhabitants to use a service located in another country where running such a service is perfectly legal. If a government forbids to use such a service in another country, they risk a trade war with the country where this service is operated.

and security concerns of handing over the car keys to an unknown server admin.

I don't think this is an issue. If you use proprietary networks you also don't care about Microsoft/AOL/Yahoo/etc controlling the server. I believe some (all?) even have an EULA that says all your messages are their property! If you care about all this, you run your own XMPP server and don't use closed proprietary networks at all!

I agree with most everything said in the article, although I find it unrealistic. For instance, how are we to "steal" IM users from other projects?

You answer the question yourself: "Despite that my latest user experience suggests that transports haven't improved much in years, there is something I like about Jabber, and I only go back to Adium because of proprietary IM interoperability reliability which is totally missing from corresponding Jabber transports." ;-)

Although I can believe that multi-protocol clients are run inefficiently, you don't take into account the resources that go into coding transports.

I assume you are talking about the project? Well, coding transports is more efficiently because:

  • Transports are be separate projects that can fully focus on their key goal: compatibility between XMPP and a specific closed network. They can fully focus on this goal and they don't have to wait for the other transport projects. Also the XMPP clients can fully focus on their goal to provide a better IM experience for a specific niche. All this is no true for multi-protocol clients. Multi-protocol client projects can't fully focus on a better IM experience because their main goal is interoperability with closed networks. Also, the different protocol plugins interfere with each other in a way that one plugin can restrict others. For example, adding features to a specific protocol plugin may interfere with other plugins. This can lead to forks such as recently happened with the MSN protocol plugin for Pidgin. Obviously, forks are not efficient (though they are no necessarely bad!).
  • The fact that it will work in all clients.
  • The fact that it is easier to get people testing a new version very fast. When one server updates a transport to the newest version, all users using that transport will automatically test the new version without them requiring to download something. This is not true for multi-protocol clients. This makes transports more efficient because bug reports will be more actual. This makes debugging of the issue easier for the coder.
  • The fact that transports only need to run on a limited amount of different hardware and software configurations. Compare this with multi-protocol clients that are run by a lot of people which all have different hardware and software. The difference in their configuration may cause bugs. The chance you will see such problems on a server is much lower. Remember also that these issues may be difficult to solve for the coder as he does not has access to the specific hardware/software configuration of the user reporting the problem.

The other things missing from mass Jabber adoption is a concerned campaign, as what happened with the Spread Firefox project.

I don't think this is missing: there never was a campaign for mass adoption of for instance the Web or email. XMPP is like the Web or email, not like Firefox. Coccinella on the other hand is more similar to Firefox ;-)

Tue, 08/07/2008 - 01:13 — Chascon

Countries can't keep people from installing --for instance-- a for msn im plugin that is available from X country where it is legal to distribute. It's been done for years with CSS script on Debian that enabled the ability to see DVDs. And I truly doubt these governments would make it illegal to use a program that has plugin architecture that "could" be used to install a plugin that allows IM interoperability, among other things. This is the case of being judged guilty before committing the "crime". So transports _don't_ have to be kept separate from clients.

On the note of "stealing" users, let me clarify. You don't force people into using any product or standard and you certainly don't steal users because they'll use anything they want. Furthermore, trying to force people to use open standards is not optimal and certainly isn't in line with freedom to choose. Education is the key, and setting programs to educate, along with good documentation, and setting the playing field is the way. Perhaps getting governments to establish standards, as with the development of the net, would help but other than this "forcing" is not optimal and certainly not pro-choice.

About going back to Coccinella, I'm not going back to it if I can't get it up working with Bitlbee because of the lack of stable Jabber transports. As for the ability to benefit from the hordes of people using Jabber transports on pubic servers, I just don't see the benefit. I mean, the MSN Jabber transports just --to be blunt-- suck.

About the benefits of the modularity of Jabber clients and their separation from transports, I agree with you. I just think you frame your findings by not considering the transport developers with client developers. I also think that anything good you can say about this set-up, you should be able to say about a modular client+transport model. If Pidigm has had problems as of late, well maybe their architecture is ill-designed. It has been suggested that their code is a mess.

About handing over the keys to a public Jabber server admin not being a problem because you've already handed your keys over to Yahoo, Google, etc. I don't think it's the same. You don't know who a no-name Jabber server admin is, one that might be here one day and gone the other. On the other hand, large corporations need to keep a good public rep, whereas a basement Joe doesn't have to care because he has no public stock that would be affected by bad publicity.

It's also a matter of math (or probability), I can give my neighbour a set of my house keys, but am I not more at risk of being ransacked if I start handing out more copies of my house keys to other people? Especially people that have nothing to lose if they turn to a life of crime (a bum off the street)? Sure, I am. That's what running off of a public Jabber server is like. I can see tempers flaring, but it's only an analogy.
This sounds like a case for my old probabilities class, not that I was great at the stuff.

Oh, and on Coccinella being like Firefox, I think it could be but isn't. Why? Well, FF is modular in the sense that it is flexible. People can install extensions to their hearts content. Sure there are sometimes extension conflicts, but it is modular. In this way, FF is almost anything anyone wants it to be. My FF is a ftp client, webserver, html editor, even a "window manager", and more. To take this even farther, Mozilla code is also a music player, Songbird, a Suite, SeaMonkey, and even more. Coccinella is far from being anything like this. Why? Again, it's not modular.

I would like to reconsider what I said about about a large corporation, although this would be a good legitimizing factor, a grass roots campaign could be effective. I recall that the FF campaign tried to take advantage of this "word of mouth" approach.

Last thing and in reference to your point on conflicting plugins, I don't see why a simple protocol to deal with these conflicts couldn't be developed.

Oh and on "there never was a campaign for mass adoption of for instance the Web", no there wasn't. But there was a concerted push on the part of the American government to establish a standard protocol. If you remember, the net came out of the cold war and the American struggle on how work out a non-centralized network with huge redundancies, in the event that main centres got hit. It never just got mass acceptance because it was "good" or "better", as you suggest.

Chascon

Tue, 08/07/2008 - 09:14 — sander

Countries can't keep people from installing --for instance-- a for msn im plugin that is available from X country where it is legal to distribute. It's been done for years with CSS script on Debian that enabled the ability to see DVDs.

Linux distributions cannot include CSS into their distributions if they want to be able to distribute it into the US without fearing legal risks. This is also the reason why you see in distributions such as Ubuntu a warning dialog when you try to install the codec (I believe it says something that installing CSS is illegal some countries and that Ubuntu is not responsible for any legal risks of the user who installs it. Something similar could eventually happen with support for proprietary protocols, but if people install an XMPP client and use that to communicate using the XMPP protocol with a server in another country (where operating such a server is not illegal), the user can't be blamed for doing anything illegal: he is not connecting to the closed network himself; it is the service in the other country that is connecting to the closed network.

About going back to Coccinella, I'm not going back to it if I can't get it up working with Bitlbee because of the lack of stable Jabber transports. As for the ability to benefit from the hordes of people using Jabber transports on pubic servers, I just don't see the benefit. I mean, the MSN Jabber transports just --to be blunt-- suck.

You are definitely right! Transports should become rock-solid. More features in transports are IMO not as important as reliability.

Oh, and on Coccinella being like Firefox, I think it could be but isn't.

I guess was not clear enough. I tried to say that XMPP is a protocol such as HTTP and thus cannot be compared with Firefox. Both Firefox and Coccinella are using an open protocol and in that way they are similar.

Coccinella is far from being anything like this. Why? Again, it's not modular.

Actually Coccinella is modular. It's just that nobody cared about making a third-party module so far...are you interested in doing that? Then go ahead! ;-)

Last thing and in reference to your point on conflicting plugins, I don't see why a simple protocol to deal with these conflicts couldn't be developed.

Actually, there already exists something like that. But the problem is you will restrict yourself to the API of that system; XMPP is not the main channel as something new is used. Telepathy could be compared with an imaginary system in Web browsers to connect to different imaginary closed Web standards.

Oh and on "there never was a campaign for mass adoption of for instance the Web", no there wasn't. But there was a concerted push on the part of the American government to establish a standard protocol. If you remember, the net came out of the cold war and the American struggle on how work out a non-centralized network with huge redundancies, in the event that main centres got hit.

I was talking about the Web (WWW), not the Internet. Recommended reading is Tim-Berners Lee's book. In this book about the birth of the Web you will see there was a campaign to promote the Web! Initially there were even closed Web networks...very similar to today's situation with XMPP...I'm sure history will repeat! ;-)

Except where otherwise noted, this work is licensed under a Creative Commons License.