We are starting to use CoZ sftp server and noticed that the ascii and binary subcommands are invalid (looks like all other subcommands are available from help).
Is it possible for users (and our scripts) to continue to use the ascii and binary subcommands and have the CoZ sftp server process them them as if "ls /+mode=text" or "ls /+mode=binary" were used?
Are there other user changes we need to be aware of?
Thanks.
Co:Z sftp server with using ascii and binary subcommands
I'm not sure exactly what you mean, but the IBM sftp *server* has no support for ASCII text translation that I am aware of.
The "ascii" and "binary" are subcommands that IBM added to their port of the OpenSSH sftp *client* (the /bin/sftp command). They are not part of OpenSSH, and you will not find these commands on other platform OpenSSH implementations.
With other default settings, the equivalent commands in the "cozsftp" client are:
lzopts mode=text,servercp=ISO8859-1
and:
lzopts mode=binary
(Note: clientcp in the cozsftp client defaults to the local EBCDIC codepage, and servercp defaults to ISO8859-1, unless the server is also Co:Z SFTP in which case servercp defaults to its default clientcp. )
If you are using the Co:Z SFTP server, you can use the "ls" command from the remote sftp client to set the server's settings.
For example (from a remote OpenSSH (not z/OS) client):
So, it might be a good idea to add "ascii" and "binary" commands to the cozsftp command, that are aliases for the equivalent lzopts command. It is a simple thing to do, and would allow us to be more compatible with IBM's extension. I would imagine that the reason that no one has asked for this before is that users switch to Co:Z sftp so that they can use MVS datasets, and they end up changing their scripts and jobs anyway.
Finally, the Co:Z support for text translation is quite extensive - it not only supports the client and server but has additional settings for mode=text such as the "linerule", "trim", "overflow", and "pad" (as well as clientcp and servercp).
See for more information:
http://dovetail.com/docs/sftp/using.html#using_opts
http://dovetail.com/docs/sftp/client.html
The "ascii" and "binary" are subcommands that IBM added to their port of the OpenSSH sftp *client* (the /bin/sftp command). They are not part of OpenSSH, and you will not find these commands on other platform OpenSSH implementations.
With other default settings, the equivalent commands in the "cozsftp" client are:
lzopts mode=text,servercp=ISO8859-1
and:
lzopts mode=binary
(Note: clientcp in the cozsftp client defaults to the local EBCDIC codepage, and servercp defaults to ISO8859-1, unless the server is also Co:Z SFTP in which case servercp defaults to its default clientcp. )
If you are using the Co:Z SFTP server, you can use the "ls" command from the remote sftp client to set the server's settings.
For example (from a remote OpenSSH (not z/OS) client):
Code: Select all
sftp> ls /+mode=text
sftp> ls /+lrecl=80,recfm=fb,space=cyl.3.1
sftp> put local.txt //HLQ.MVS.DATA
sftp> put local.txt /path/to/zFS/file
sftp> lzopts mode=binary
sftp> put local.bin /path/x.bin Finally, the Co:Z support for text translation is quite extensive - it not only supports the client and server but has additional settings for mode=text such as the "linerule", "trim", "overflow", and "pad" (as well as clientcp and servercp).
See for more information:
http://dovetail.com/docs/sftp/using.html#using_opts
http://dovetail.com/docs/sftp/client.html
The issue we ran into was caused by us having the sftp-server.rc in /etc/ssh set with export SFTP_ZOS_OPTIONS="mode=text" as default.
When the user ran their SFTP client on Solaris, their binary transfer did not work because the binary subcommand wasn't available (file was transferred but the data was wrong).
To make this transparent to SFTP users who are already using (or expect to use) the ascii and binary subcommands), it would be great if they didn't need to use different subcommands to set the mode for a file transfer. This would make the transition much easier as well as a great path to exploiting all of the new function with the CoZ sftp server.
Thoughts?
Thanks.
When the user ran their SFTP client on Solaris, their binary transfer did not work because the binary subcommand wasn't available (file was transferred but the data was wrong).
To make this transparent to SFTP users who are already using (or expect to use) the ascii and binary subcommands), it would be great if they didn't need to use different subcommands to set the mode for a file transfer. This would make the transition much easier as well as a great path to exploiting all of the new function with the CoZ sftp server.
Thoughts?
Thanks.
I'm sorry, but I'm still confused.
What "binary" subcommand did they use on a Solaris SFTP client? There isn't one, as far as I know.
If you want to have the default for the Solaris client user be binary, then you could have a "mode=binary" setting in that users z/OS ~/.ssh/sftp-server.rc. Then, their transfers would be compatible with IBM Ported Tools sftp-server.
Then, when they use Co:Z SFTP server, the Solaris client can switch to text (ASCII codepage by default), using:
sftp> ls /+mode=text
But this wouldn't work with IBM's sftp-server, and there isn't an equivalent that I know of.
So, it sounds like your problem is this: if you change the system-wide default from mode=binary (the default) to mode=text, then you will be incompatible with remote clients who switch from IBM's sftp-server (that only supports binary).
What "binary" subcommand did they use on a Solaris SFTP client? There isn't one, as far as I know.
If you want to have the default for the Solaris client user be binary, then you could have a "mode=binary" setting in that users z/OS ~/.ssh/sftp-server.rc. Then, their transfers would be compatible with IBM Ported Tools sftp-server.
Then, when they use Co:Z SFTP server, the Solaris client can switch to text (ASCII codepage by default), using:
sftp> ls /+mode=text
But this wouldn't work with IBM's sftp-server, and there isn't an equivalent that I know of.
So, it sounds like your problem is this: if you change the system-wide default from mode=binary (the default) to mode=text, then you will be incompatible with remote clients who switch from IBM's sftp-server (that only supports binary).
Now I 'm confused too 
We already updated the CoZ mode to binary in the global sftp-server.rc file to correct it.
From IBM's Ported Tools User Guide:
OpenSSH supports sftp can treat files as binary or text. By default, sftp assumes that files are binary. Files transferred between EBCDIC and ASCII platforms are not converted.
For file transfers between z/OS and ASCII UNIX platforms, you might need to convert your files (treat them as text).
The sftp ascii subcommand can be used to transfer
files in ASCII between the local host and a remote UNIX host. This subcommand assumes that the file data on the network should be encoded in ISO/IEC 8859-1.
The sftp binary subcommand can be used to disable this conversion and return to performing binary file transfers.
scp treats files as text. By default, scp performs ASCII/EBCDIC conversion on files.
Example from mvs to mvs using IBM's sftp
$ sftp mvs1
Connecting to mvs1...
sftp> ascii
Sets the file transfer type to ASCII.
sftp> binary
Sets the file transfer type to binary.
sftp> quit
I'm guess I asking for support for subcommands of ascii and binary because that's what users already are familiar with.
We already updated the CoZ mode to binary in the global sftp-server.rc file to correct it.
From IBM's Ported Tools User Guide:
OpenSSH supports sftp can treat files as binary or text. By default, sftp assumes that files are binary. Files transferred between EBCDIC and ASCII platforms are not converted.
For file transfers between z/OS and ASCII UNIX platforms, you might need to convert your files (treat them as text).
The sftp ascii subcommand can be used to transfer
files in ASCII between the local host and a remote UNIX host. This subcommand assumes that the file data on the network should be encoded in ISO/IEC 8859-1.
The sftp binary subcommand can be used to disable this conversion and return to performing binary file transfers.
scp treats files as text. By default, scp performs ASCII/EBCDIC conversion on files.
Example from mvs to mvs using IBM's sftp
$ sftp mvs1
Connecting to mvs1...
sftp> ascii
Sets the file transfer type to ASCII.
sftp> binary
Sets the file transfer type to binary.
sftp> quit
I'm guess I asking for support for subcommands of ascii and binary because that's what users already are familiar with.
> I'm guess I asking for support for subcommands of ascii and binary because that's what users already are familiar with.
Your last example is using the IBM sftp client, which is where IBM has added support for the binary and ascii subcommands. If you have a z/OS SFTP client job that uses the IBM "/bin/sftp" client command, currently you will need to change the "binary" and "ascii" commands if you want to use the "cozsftp" command. But likely, you will switch to using the cozsftp command because you want to exploit other features, such as dataset support and you will be editing the client JCL/scripts anyway. It does sound like a reasonable suggestion that we somehow emulate the IBM "ascii" and "binary" subcommands in the cozsftp client, but its a little tricky to do the "ascii" command since cozsftp support for text translation supports many more settings that can affect text translation (even though 99% of the time, "lzopts mode=text" is sufficient).
Your other reference was to a Solaris client, switching from IBM's sftp-server to Co:Z sftp-server. There is no "ascii" or "binary" subcommand on the Solaris OpenSSH client, or any other OpenSSH client other than IBM's. There is no way to do ascii text translation to IBM's sftp-server from a non z/OS OpenSSH client - IBM's sftp-server only supports binary zFS/HFS files. Your problem was that your Solaris client was used to binary, but when you switched the server to run Co:Z sftp-server, you changed the default from mode=binary to mode=text. Changing it back to binary allowed that client to behave correctly as before. The is no way to avoid this - we can't add new subcommands to the Solaris sftp client (unless you change the Solaris script to start with "ls /+mode=binary").
IBM's manual is quite confusing - but it does say that "you might need to convert your files....". What they don't clearly say is that the IBM z/OS sftp client will convert text using the non-standard "ascii" subcommand. The IBM sftp-server only supports binary with non z/OS remote clients.
Your last example is using the IBM sftp client, which is where IBM has added support for the binary and ascii subcommands. If you have a z/OS SFTP client job that uses the IBM "/bin/sftp" client command, currently you will need to change the "binary" and "ascii" commands if you want to use the "cozsftp" command. But likely, you will switch to using the cozsftp command because you want to exploit other features, such as dataset support and you will be editing the client JCL/scripts anyway. It does sound like a reasonable suggestion that we somehow emulate the IBM "ascii" and "binary" subcommands in the cozsftp client, but its a little tricky to do the "ascii" command since cozsftp support for text translation supports many more settings that can affect text translation (even though 99% of the time, "lzopts mode=text" is sufficient).
Your other reference was to a Solaris client, switching from IBM's sftp-server to Co:Z sftp-server. There is no "ascii" or "binary" subcommand on the Solaris OpenSSH client, or any other OpenSSH client other than IBM's. There is no way to do ascii text translation to IBM's sftp-server from a non z/OS OpenSSH client - IBM's sftp-server only supports binary zFS/HFS files. Your problem was that your Solaris client was used to binary, but when you switched the server to run Co:Z sftp-server, you changed the default from mode=binary to mode=text. Changing it back to binary allowed that client to behave correctly as before. The is no way to avoid this - we can't add new subcommands to the Solaris sftp client (unless you change the Solaris script to start with "ls /+mode=binary").
IBM's manual is quite confusing - but it does say that "you might need to convert your files....". What they don't clearly say is that the IBM z/OS sftp client will convert text using the non-standard "ascii" subcommand. The IBM sftp-server only supports binary with non z/OS remote clients.
Hello, thank you for pointing me to this topic.
To second the OP, I also would consider the ASCII subcommand as useful, first of all for interactive use of the coz-client by users who rarely use cozsftp. If available, the ascii subcommand is much easier to remember. But affected are not only rare users of cozsftp: since I often use both CoZ and IBM sftp client, I also often try to instruct cozsftp to "ascii" instead of the correct lzopts
...
To second the OP, I also would consider the ASCII subcommand as useful, first of all for interactive use of the coz-client by users who rarely use cozsftp. If available, the ascii subcommand is much easier to remember. But affected are not only rare users of cozsftp: since I often use both CoZ and IBM sftp client, I also often try to instruct cozsftp to "ascii" instead of the correct lzopts
...Jeno,
I think that this is a good suggestion (an "ascii" command alias for the Co:Z SFTP client), and I was wondering why we haven't had this suggestion until now. The reason, I guess, is that P.T. sftp client users that switch to cozsftp normally do so to get dataset support, and when they switch they have to start using lzopts anyway.
I am in favor of adding "ascii" and "binary" subcommands as aliases, respectively, for:
lzopts mode=text
lzopts mode=binary
The problem with this is that in text mode there are several other settings that affect how conversion is done (code pages, trim, pad, linerule, etc). "lzopts mode=text" with the defaults is sufficient in 95% of the situations, but there would be exceptions.
This seems to be a case of perfect as the enemy of good enough. Your thoughts?
I think that this is a good suggestion (an "ascii" command alias for the Co:Z SFTP client), and I was wondering why we haven't had this suggestion until now. The reason, I guess, is that P.T. sftp client users that switch to cozsftp normally do so to get dataset support, and when they switch they have to start using lzopts anyway.
I am in favor of adding "ascii" and "binary" subcommands as aliases, respectively, for:
lzopts mode=text
lzopts mode=binary
The problem with this is that in text mode there are several other settings that affect how conversion is done (code pages, trim, pad, linerule, etc). "lzopts mode=text" with the defaults is sufficient in 95% of the situations, but there would be exceptions.
This seems to be a case of perfect as the enemy of good enough. Your thoughts?
Well, I agree with you in both points. However, I refer to the case where IBM sftp in text mode would be sufficient. I think that the cozsftp defaults can be pre-defined so that it is 100% (or more *) compatible with IBM sftp when switching to ascii mode. Anyway, even compatibility in only 80% of cases would already dramatically reduce the need to RTFM the forgotten lzopts keyword ...
*) "more compatible" in my case means here that, IBM sftp in ascii mode would use latin-1, while I would rather prefer latin-2 (IBM-1165 to ISO8859-2). Even the line terminator can be better defaulted ...
*) compatible with IBM sftp when switching to ascii mode. Anyway, even compatibility in only 80% of cases would already dramatically reduce the need to RTFM the forgotten lzopts keyword ...*) "more compatible" in my case means here that, IBM sftp in ascii mode would use latin-1, while I would rather prefer latin-2 (IBM-1165 to ISO8859-2). Even the line terminator can be better defaulted ...
Re: Co:Z sftp server with using ascii and binary subcommands
The problem with this is that in text mode there are several other settings that affect how conversion is done (code pages, trim, pad, linerule, etc).
== Solitaire ==