When executing "cozsftp -B 1000000 ..." the following error is received:
100MB.dat 0% 0 0.0KB/s --:-- ETA[00799] Outbound message too long 1000025
When executing "cozsftp -B 200000 ..." cozsftp hangs without an error message.
IBM's man page for the -B option for sftp states:
" -B buffer_size Specifies the size of the buffer that sftp uses when
transferring files. Larger buffers requires fewer round trips at
the cost of higher memory consumption. The default is 32768
bytes. If specifying buffer_size > INT_MAX, sftp only allocates
INT_MAX at most. For more information, see ""Limitations"" in
topic 2.5.SFTP.4.
"
...
" The biggest buffer size that can be allocated is 2147483647(INT_MAX)
bytes. INT_MAX is defined in limits.h.
"
cozsftp get error or hangs when using large buffer sizes
1,000,000 Buffer Size gets same error Linux to Linux
When I try using just sftp from a linux system to another linux system using the one million buffer size, The same error occurs.
Good question - this is very interesting.
First, the maximum buffer size for OpenSSH 5.1 is 256K, which is why you get an error if larger. This is working as designed.
But, we confirmed that in Co:Z SFTP if you set the buffer size to over 100000 or so that the client may hang, as you report. If the progress meter is active, then it will interrupt every second and release the hang so that transfer continues, but the performance is severely degraded. This is what you might be seeing in your other post.
We have traced this down to something wrong with how Unix domain sockets work on z/OS. OpenSSH sftp has a compile option to use pipes rather than a Unix domain socket pair - when we rebuild with this option the problem goes away completely. We are going to do some more testing, but this may be the correct solution.
Also, we have found that when testing with different -B sizes that performance is not significantly affected (the default of 32k seems to be about as fast as larger). This is true with either Co:Z sftp or Ported Tools sftp. For now, we suggest that you don't set -B.
If you are interested in testing a patch that uses pipes (and avoids these problems), please contact us at info@dovetail.com.
Thanks for letting us know about this problem
First, the maximum buffer size for OpenSSH 5.1 is 256K, which is why you get an error if larger. This is working as designed.
But, we confirmed that in Co:Z SFTP if you set the buffer size to over 100000 or so that the client may hang, as you report. If the progress meter is active, then it will interrupt every second and release the hang so that transfer continues, but the performance is severely degraded. This is what you might be seeing in your other post.
We have traced this down to something wrong with how Unix domain sockets work on z/OS. OpenSSH sftp has a compile option to use pipes rather than a Unix domain socket pair - when we rebuild with this option the problem goes away completely. We are going to do some more testing, but this may be the correct solution.
Also, we have found that when testing with different -B sizes that performance is not significantly affected (the default of 32k seems to be about as fast as larger). This is true with either Co:Z sftp or Ported Tools sftp. For now, we suggest that you don't set -B.
If you are interested in testing a patch that uses pipes (and avoids these problems), please contact us at info@dovetail.com.
Thanks for letting us know about this problem