You are here

Bytestreams

File transfers has been a long standing problem within the jabber community and the present solution using the so called bytestreams protocol is far from perfect. A kind of improvement by using symmetric streamhosts called fast mode is being used by Psi and Coccinella, but this extension was rejected by the Jabber Council (?) for reasons unknown to me. I can guess why; it is a bit complicated to implement, especially with stream proxies.

The Coccinella now has support for stream proxies, and together with fast mode it should be fairly effective in handling various network configurations. Since proxy support is also fairly common in ejabberd, I think I've timed it fairly well. One further reason I added proxy support is that people often make complaints about whiteboard transfers not working, so I thought it wouldn't be too difficult to add a new file transfer profile to ftrans/si/bytestreams. I'll ping the jdev list when I have a proposal.

I was also thinking why the bytestreams+fastmode gets so complicated when the protocol seems so deceptively clean. Lets focus on the initiator alone. There are two different flows:

  • (1) iq query/response
  • (2) socks5 connections and negotiations, denoted s5 here

They interact and depend on each other. As seen from the initiator:

  • (a) iq-stream initiate (send)
  • (b) (f) iq-stream target provides streamhosts to initiator (recv)
  • (1) s5 socket to initiators server
  • (2) (f) s5 fast socket to targets streamhost
  • (3) s5 socket initiator to proxy

where (f) means fast mode. Iq-stream (a) controls (1) and (3), while iq-stream (b) controls (2). There are three possible s5 streams:

  • (A) s5 (server) initiator <--- s5 (client) target
  • (B) (f) s5 (client) initiator ---> s5 (server) target
  • (C) s5 (client) initiator ---> s5 (server) proxy

The first succesful stream wins and kills the other.

The iq-streams do the signaling job while the SOCKS5 essentially do the media transport (except for some simple negotiating). But in fast mode this is further complicated since a CR character is sent by the initiator in the s5 stream as a signal which to pick. Sounds messy. Try write a state diagram!

After extensive testing using Coccinella as initiator and Psi as a target, I was only able to find one case where it bailed out. Coccinella had picked the fast stream, sent the CR and started writing data to its socket when Psi suddenly closed its socket. I was never able to reproduce it despite extensive tries.

I also noted that the XEP says that the streamhosts shall be tried in turn, which is very important since the proxy host shall only be used if the p2p failed. It seems, however, that Psi tries them all at once and therefore may select the proxy even if the normal or the fast would have worked. But I may have mistaken.

Comments

Hi,

I was referred to your post here, and I just wanted to comment on a few things:

1) Connecting to the streamhosts in order (that is, waiting for failure before moving to the next one.. the XEP isn't clear on that) can result in very slow negotiations. Gaim, for example, is known to get stuck for many minutes if the first streamhost cannot be reached. A trickle-connect (like ICE) might make more sense than waiting for full failure, but again the XEP doesn't go into that kind of depth about what it means to connect in order.

2) Psi tries to prioritize the streamhosts by use of the <proxy> element. Basically all non-proxy streamhosts are tried first until failure, and then the proxy streamhosts are tried.

3) Fast mode is complex, but anything like this will be complex. Maybe it would have been slightly simpler to not use the extra CR character for activation, but the result would still be very complex. You should have seen the things the council used to reject: fallback from S5B to IBB in XEP-95/96 for example (where Jive has proposed their own extension), and also the original reverse connection text in XEP-65 (which was like fast mode, except not simultaneous. initiator did a full s5b negotiation, and if that fully failed, the target would initiate s5b in the other direction.. how is this complex?!). Fortunately, the council is different now and more accepting of complex protocols (see Jingle, which would have been rejected instantly in the old days).

Anyway, great to see another fast mode implementation!

-Justin

Hi Justin,

1) I agree that the XEP is vague about how connections should be made, and I wonder why. My "solution" to getting it fast was to set a short timeout for the SOCKS5 connections, presently 8 secs. My impression is that if it fails it does so within a very short period of time. I don't see situations where it would succeed after, say 30 secs. But I may be wrong.

2) I didn't (and don't) see a <proxy/> element in the XEP. Perhaps I should also add one when providing the streamhosts. Sounds like a good idea. My lack of a proxy element was probably the reason I saw Psi pick the proxy before the "normal" and the fast streams.

3) I was actually trying to understand the underlying structure why it was so complex, so I don't think the cause of the complexity is due to the protocol chosen, but due to its inherent structure. Fallback S5B to IBB would be a good thing, I think.

The fast mode has been in Coccinella for some time already:

Coccinella-0.95.12 (20060616):
...
 o added fast bytestreams protocol (Psi)

/Mats