Unable to FTP file VIA COZSFTP with the correct format

[LeonBylsma]

We send files to a server from which Plastic Cards get's embossed. We have now converted the ftp to use secure COZSFTP but no matter which linerule or other format commands we use, we can not get the file to be in the same format it used to be when we use normal FTP. I have 2 Sample files available to show the format when sending with ftp and then also with COZSFTP. If you open these files with an editor (we use Notepad ++) Then we see a combination of CR, CRLF and LF termination but also some other special characters which we think is causing the problem. Can we send you the files to see if you can perhaps resolve the riddle?

[coz]

Leon,

Sure - send the files to info@dovetail.com and we'll take a look. If you can, please send:

1.) the original dataset in XMIT format
2.) the FTP result file
3.) the COZSFTP result file

Please don't send any confidential information!

[LeonBylsma]

Hi Steve, I have sent the files by mail as requested and thank you and these are pure test records so no confidential info is included.

[coz]

Leon,

OK, here's what's going on: First off, it's important to understand that when converting from a dataset to a stream (which will become a POSIX file), the Co:Z linerule setting is applied at the end of each record. Since your sample dataset only contains a single record, the Co:Z linerule is applied once, at the end of the stream. Therefore, the end of the resulting POSIX file will contain one occurance of the linerule (e.g. 0x0d0a if linerule=crlf).

However, what you are seeing is the conversion of characters within the single record itself, and in particular, the treatment of the EBCDIC line feed character (0x25). Co:Z uses Unicode Services for codepage conversion. If you haven't specified custom client and server codepages, you probably have a conversion something like IBM-1047->ISO8859-1. You can check this by issuing the "lzopts" command from cozsftp - look for the "clientcp" and "servercp" values.

Using those two codepages as a starting point, I can tell you that on most systems, Unicode Services will convert the 0x25 to 0x85 (which is what we're seeing in your sample). FTP (using the tables on your system), on the other hand is converting the 0x25 to 0x0A.

If you want Co:Z to behave (somewhat) like FTP without getting into complicated custom conversion techniques, you may be able to simply specify a different code page combination that Unicode Services will translate more to your liking. In particular, you may want to try setting "clientcp=850", which will likely convert the 0x25 to 0x0A that you want. I say "likely", because it depends on how the Unicode Services tables are set up on your machine, but most default installations will give you this result. Note that this set of codepages may not give you an exact duplicate of what you are seeing with FTP, but the differences might not be relevant in your particular case.

BTW, a useful tool in your Co:Z installation is the "showtrtab" utility. This program, located in <COZ_HOME>/bin takes a source and target code page and displays the translation table that Co:Z will use. So, for example on our system, we see the following results:

Code: Select all

/u/vendor/coz190/bin>showtrtab -s IBM-1047 -t ISO8859-1                                                                                      
00:  00 01 02 03   9C 09 86 7F   97 8D 8E 0B   0C 0D 0E 0F 
10:  10 11 12 13   9D 0A 08 87   18 19 92 8F   1C 1D 1E 1F 
20:  80 81 82 83   84 85 17 1B   88 89 8A 8B   8C 05 06 07 
                      ^^
                      ||
...

Code: Select all

/u/vendor/coz190/bin>showtrtab -s IBM-1047 -t 850                                                                                            
00:  00 01 02 03   DC 09 C3 1C   CA B2 D5 0B   0C 0D 0E 0F 
10:  10 11 12 13   DB 0A 08 C1   18 19 C8 F2   1A 1D 1E 1F 
20:  C4 B3 C0 D9   BF 0A 17 1B   B4 C2 C5 B0   B1 05 06 07 
                      ^^
                      ||
...

Note the difference with handling of the EBCDIC 0x25.