Bytes added to file transferred in binary mode

[rmogondovetailedtech]

Hi,

I have this JCL code:

//MOGRENRP JOB (410),'cozsftp test',MSGLEVEL=(1,1),
// CLASS=A,MSGCLASS=X,NOTIFY=&SYSUID,USER=&SYSUID,
// REGION=0M
//DLTSTEP EXEC PGM=IEFBR14
//SYSPRINT DD SYSOUT=*
//DFBNCCIT DD DSN=MOGREN.FBNCCIT,DISP=(OLD,DELETE)
//DFBNCCIB DD DSN=MOGREN.FBNCCIB,DISP=(OLD,DELETE)
/*
//******************************************************
//* USE COZSFTP TO PUSH AND PULL A FILE
//******************************************************
//COZBATCH EXEC PGM=COZBATCH
//STEPLIB DD DSN=SYS2.COZ.LOADLIB,DISP=SHR
//STDOUT DD SYSOUT=*
//STDERR DD SYSOUT=*
//TEXTFILE DD DSN=MOGREN.FBNCCIT,
// SPACE=(TRK,(15,2)),DISP=(NEW,KEEP),
// DCB=(RECFM=FB,LRECL=80)
//BINFILE DD DSN=MOGREN.FBNCCIB,
// SPACE=(TRK,(15,2)),DISP=(NEW,KEEP),
// DCB=(RECFM=FB,LRECL=80)
//STDIN DD *
#ulimit -a
#env
#echo $PATH
#coz_bin="/usr/lpp/coz/bin"
ruser="mogren"
server="ccentnim03u"
#Change the file name as required in the following:
rfile="fibonacci.c"
ssh_opts="-oBatchMode=no"
ssh_opts="$ssh_opts -oConnectTimeout=60"
ssh_opts="$ssh_opts -oServerAliveInterval=60"
ssh_opts="$ssh_opts -oStrictHostKeyChecking=no"
cozsftp $ssh_opts -b- $ruser@$server <<EOB
lzopts mode=text,linerule=lf,trim,recfm=fb
lzopts -a
#put //TEST.MOGREN.JCL(FBNCCI) $rfile
get $rfile //DD:TEXTFILE
put //DD:TEXTFILE ${rfile}_txt
EOB
cozsftp $ssh_opts -b- $ruser@$server <<EOB1
lzopts mode=binary,trim
lzopts -a
get $rfile //DD:BINFILE
put //DD:BINFILE ${rfile}_bin
EOB1
#fromdsn '//mogren.fbncci' > /u/mogren/fibonacci.c_20131105
#cat /u/mogren/tmp.txt | todsn -x shr '//TEST.MOGREN.JCL(TMP)'
//

that produces this log:

1 J E S 2 J O B L O G -- S Y S T E M S Y S T -- N O D E C L R K S Y S T
0
10.22.36 JOB08085 ---- THURSDAY, 17 APR 2014 ----
10.22.36 JOB08085 IRR010I USERID MOGREN IS ASSIGNED TO THIS JOB.
10.22.36 JOB08085 ICH70001I MOGREN LAST ACCESS AT 10:14:36 ON THURSDAY, APRIL 17, 2014
10.22.36 JOB08085 $HASP373 MOGRENRP STARTED - INIT B - CLASS A - SYS SYST
10.22.36 JOB08085 IEF403I MOGRENRP - STARTED - TIME=10.22.36
10.22.37 JOB08085 - -----TIMINGS (MINS.)------ -----PAGING COUNTS---
10.22.37 JOB08085 -STEPNAME PROCSTEP RC EXCP CONN TCB SRB CLOCK SERV WORKLOAD PAGE SWAP VIO SWAP
10.22.37 JOB08085 -DLTSTEP 00 8 8 .00 .00 .0 BATCH 0 0 0
10.22.46 JOB08085 -COZBATCH 00 4674 164 .02 .00 .1 25 BATCH 3 0 0
10.22.46 JOB08085 IEF404I MOGRENRP - ENDED - TIME=10.22.46
10.22.46 JOB08085 -MOGRENRP ENDED. NAME-cozsftp test TOTAL TCB CPU TIME= .02 TOTAL ELAPSED TIME= .1
10.22.46 JOB08085 $HASP395 MOGRENRP ENDED
0------ JES2 JOB STATISTICS ------
- 17 APR 2014 JOB EXECUTION DATE
- 49 CARDS READ
- 181 SYSOUT PRINT RECORDS
- 0 SYSOUT PUNCH RECORDS
- 11 SYSOUT SPOOL KBYTES
- 0.16 MINUTES EXECUTION TIME
1 //MOGRENRP JOB (410),'cozsftp test',MSGLEVEL=(1,1), JOB08085
// CLASS=A,MSGCLASS=X,NOTIFY=&SYSUID,USER=&SYSUID, 00020022
// REGION=0M 00030022
IEFC653I SUBSTITUTION JCL - (410),'cozsftp test',MSGLEVEL=(1,1),CLASS=A,MSGCLASS=X,NOTIFY=MOGREN,USER=MOGREN,
REGION=0M
2 //DLTSTEP EXEC PGM=IEFBR14 00031033
3 //SYSPRINT DD SYSOUT=* 00031134
4 //DFBNCCIT DD DSN=MOGREN.FBNCCIT,DISP=(OLD,DELETE) 00032033
5 //DFBNCCIB DD DSN=MOGREN.FBNCCIB,DISP=(OLD,DELETE) 00033034
/* 00034034
//****************************************************** 00040022
//* USE COZSFTP TO PUSH AND PULL A FILE 00050022
//****************************************************** 00060022
6 //COZBATCH EXEC PGM=COZBATCH 00070022
7 //STEPLIB DD DSN=SYS2.COZ.V2R4M4.LOADLIB,DISP=SHR 00080036
8 //STDOUT DD SYSOUT=* 00090022
9 //STDERR DD SYSOUT=* 00100022
10 //TEXTFILE DD DSN=MOGREN.FBNCCIT, 00110030
// SPACE=(TRK,(15,2)),DISP=(NEW,KEEP), 00120033
// DCB=(RECFM=FB,LRECL=80) 00130028
11 //BINFILE DD DSN=MOGREN.FBNCCIB, 00131030
// SPACE=(TRK,(15,2)),DISP=(NEW,KEEP), 00132033
// DCB=(RECFM=FB,LRECL=80) 00133030
12 //STDIN DD * 00140022
STMT NO. MESSAGE
-
10 IGD01007I DATACLAS=SRCFLIB
10 IGD01007I UNIT=
10 IGD01008I STORCLAS SET BY DEF_STORCLAS IN RACF
10 IGD01008I STORCLAS=SCBASE
10 IGD01008I ANYVOL=
10 IGD01009I MGMTCLAS=MCSTDNR
10 IGD01010I SIZE=0000033E
10 IGD01010I STORGRP=SGTSO
11 IGD01007I DATACLAS=SRCFLIB
11 IGD01007I UNIT=
11 IGD01008I STORCLAS SET BY DEF_STORCLAS IN RACF
11 IGD01008I STORCLAS=SCBASE
11 IGD01008I ANYVOL=
11 IGD01009I MGMTCLAS=MCSTDNR
11 IGD01010I SIZE=0000033E
11 IGD01010I STORGRP=SGTSO
ICH70001I MOGREN LAST ACCESS AT 10:14:36 ON THURSDAY, APRIL 17, 2014
IEF236I ALLOC. FOR MOGRENRP DLTSTEP
IEF237I JES2 ALLOCATED TO SYSPRINT
IGD103I SMS ALLOCATED TO DDNAME DFBNCCIT
IGD103I SMS ALLOCATED TO DDNAME DFBNCCIB
IEF142I MOGRENRP DLTSTEP - STEP WAS EXECUTED - COND CODE 0000
IEF285I MOGREN.MOGRENRP.JOB08085.D0000102.? SYSOUT
IGD105I MOGREN.FBNCCIT DELETED, DDNAME=DFBNCCIT
IGD105I MOGREN.FBNCCIB DELETED, DDNAME=DFBNCCIB
IEF373I STEP/DLTSTEP /START 2014107.1022
IEF032I STEP/DLTSTEP /STOP 2014107.1022
CPU: 0 HR 00 MIN 00.01 SEC SRB: 0 HR 00 MIN 00.00 SEC
VIRT: 4K SYS: 304K EXT: 4K SYS: 10096K
IEF236I ALLOC. FOR MOGRENRP COZBATCH
IGD103I SMS ALLOCATED TO DDNAME STEPLIB
IEF237I JES2 ALLOCATED TO STDOUT
IEF237I JES2 ALLOCATED TO STDERR
IGD101I SMS ALLOCATED TO DDNAME (TEXTFILE)
DSN (MOGREN.FBNCCIT )
STORCLAS (SCBASE) MGMTCLAS (MCSTDNR) DATACLAS (SRCFLIB)
VOL SER NOS= TSO010
IGD101I SMS ALLOCATED TO DDNAME (BINFILE )
DSN (MOGREN.FBNCCIB )
STORCLAS (SCBASE) MGMTCLAS (MCSTDNR) DATACLAS (SRCFLIB)
VOL SER NOS= TSO006
IEF237I JES2 ALLOCATED TO STDIN
IEF237I JES2 ALLOCATED TO SYSOUT
IGD103I SMS ALLOCATED TO DDNAME SYS00001
IGD104I HFS FILE WAS RETAINED, DDNAME IS (SYS00001)
FILENAME IS (/etc/resolv.conf)
IGD103I SMS ALLOCATED TO DDNAME SYS00003
IGD104I SYS1.SYST.TCPIP.STANDARD.TCPXLBIN RETAINED, DDNAME=SYS00003
IGD103I SMS ALLOCATED TO DDNAME SYS00004
IGD104I HFS FILE WAS RETAINED, DDNAME IS (SYS00004)
FILENAME IS (/etc/resolv.conf)
IGD103I SMS ALLOCATED TO DDNAME SYS00006
IGD104I SYS1.SYST.TCPIP.STANDARD.TCPXLBIN RETAINED, DDNAME=SYS00006
IEF285I MOGREN.MOGRENRP.JOB08085.D0000105.? SYSOUT
IEF142I MOGRENRP COZBATCH - STEP WAS EXECUTED - COND CODE 0000
IGD104I SYS2.COZ.V2R4M4.LOADLIB RETAINED, DDNAME=STEPLIB
IEF285I MOGREN.MOGRENRP.JOB08085.D0000103.? SYSOUT
IEF285I MOGREN.MOGRENRP.JOB08085.D0000104.? SYSOUT
IGD104I MOGREN.FBNCCIT RETAINED, DDNAME=TEXTFILE
IGD104I MOGREN.FBNCCIB RETAINED, DDNAME=BINFILE
IEF285I MOGREN.MOGRENRP.JOB08085.D0000101.? SYSIN
IEF373I STEP/COZBATCH/START 2014107.1022
IEF032I STEP/COZBATCH/STOP 2014107.1022
CPU: 0 HR 00 MIN 01.22 SEC SRB: 0 HR 00 MIN 00.01 SEC
VIRT: 236K SYS: 336K EXT: 9408K SYS: 10912K
IEF375I JOB/MOGRENRP/START 2014107.1022
IEF033I JOB/MOGRENRP/STOP 2014107.1022
CPU: 0 HR 00 MIN 01.23 SEC SRB: 0 HR 00 MIN 00.01 SEC
cozsftp> lzopts mode=text,linerule=lf,trim,recfm=fb
linerule=lf mode=text recfm=fb trim
cozsftp> lzopts -a
clientcp=IBM-1047 conddisp=catlg estsize jeslrecl=80
jesowner=MOGREN jesrecfm=f jesstatus=* linerule=lf
loglevel=I mode=text recfm=fb replace
servercp=ISO8859-1 smf trim NOblksize
NObufno NOcopies NOdataclas NOdest
NOdir NOdisp NOdsntype NOdsorg
NOforms NOgdgnt NOhold NOjesblksize
NOjesincsysin NOjesjobname NOjesjobwait NOlabel
NOlike NOlrecl NOmaxcscnt NOmaxdsndirlevels
NOmaxvol NOmgmtclas NOmount NOnorecall
NOoutdes NOoverflow NOpad NOrelease
NOreqexits NOreset NOretpd NOsequence
NOshowall NOspace NOspin NOstorclas
NOsysout NOtechnique NOtrtab NOtrtch
NOucount NOunit NOunixls NOvol
NOwriter
cozsftp> #put //TEST.MOGREN.JCL(FBNCCI) fibonacci.c
cozsftp> get fibonacci.c //DD:TEXTFILE
Fetching /home/mogren/fibonacci.c to //DD:TEXTFILE
cozsftp> put //DD:TEXTFILE fibonacci.c_txt
Uploading //DD:TEXTFILE to /home/mogren/fibonacci.c_txt
cozsftp> lzopts mode=binary,trim
mode=binary trim
cozsftp> lzopts -a
clientcp=IBM-1047 conddisp=catlg estsize jeslrecl=80
jesowner=MOGREN jesrecfm=f jesstatus=* loglevel=I
mode=binary replace servercp=ISO8859-1 smf
trim NOblksize NObufno NOcopies
NOdataclas NOdest NOdir NOdisp
NOdsntype NOdsorg NOforms NOgdgnt
NOhold NOjesblksize NOjesincsysin NOjesjobname
NOjesjobwait NOlabel NOlike NOlinerule
NOlrecl NOmaxcscnt NOmaxdsndirlevels NOmaxvol
NOmgmtclas NOmount NOnorecall NOoutdes
NOoverflow NOpad NOrecfm NOrelease
NOreqexits NOreset NOretpd NOsequence
NOshowall NOspace NOspin NOstorclas
NOsysout NOtechnique NOtrtab NOtrtch
NOucount NOunit NOunixls NOvol
NOwriter
cozsftp> get fibonacci.c //DD:BINFILE
Fetching /home/mogren/fibonacci.c to //DD:BINFILE
cozsftp> put //DD:BINFILE fibonacci.c_bin
Uploading //DD:BINFILE to /home/mogren/fibonacci.c_bin
stty: !FSUMB045 failed to get termios attributes: EDC5137I Inappropriate I/O control operation. (errno2=0x10150119)
Co:Z SFTP version: 2.4.4 (5.0p1) 2014-03-18
Copyright (C) Dovetailed Technologies, LLC. 2008-2013. All rights reserved.
Connection established, local_addr=172.20.100.138 local_port=3653 remote_addr=172.20.128.93 remote_port=22
ZosDatasetÝI¨: Opening dataset DD:TEXTFILE for write
ZosDatasetÝI¨: Closing dataset //MOGREN.FBNCCIT - 2430 bytes received, 30 records written
ZosDatasetÝI¨: Opening dataset DD:TEXTFILE for read
ZosDatasetÝI¨: Closing dataset //MOGREN.FBNCCIT - 30 records read, 572 bytes sent
Co:Z SFTP version: 2.4.4 (5.0p1) 2014-03-18
Copyright (C) Dovetailed Technologies, LLC. 2008-2013. All rights reserved.
Connection established, local_addr=172.20.100.138 local_port=3654 remote_addr=172.20.128.93 remote_port=22
ZosDatasetÝI¨: Opening dataset DD:BINFILE for write
ZosDatasetÝI¨: Closing dataset //MOGREN.FBNCCIB - 2430 bytes received, 31 records written
ZosDatasetÝI¨: Opening dataset DD:BINFILE for read
ZosDatasetÝI¨: Closing dataset //MOGREN.FBNCCIB - 31 records read, 2480 bytes sent

which shows that cozsftp transfers 2430 bytes in 30 records when operating in text mode, but it receives 2430 bytes in 31 records when operating in binary mode, with the put command sending 2480 bytes instead of 2430. The extra 50 bytes are all zeroes when displayed by "od -h fibonacci.c_bin" on AIX.

Any idea why the extra bytes appear in a binary transfer?


If it helps, here is the file that is transferred:


#include <stdio.h>


int main (int argc, const char * argvݨ) {

int numToPrint, i;
double num1, num2, ans;

num1 = 0;
num2 = 1;

numToPrint = 146;
/* printf("How many numbers would you like to print?\t");
scanf("%d", &numToPrint); */

printf("1\t%.0f\n2\t%.0lf\n", num1, num2);

for(i = 0; i < (numToPrint - 2); ++i){

ans = (num1 + num2);

printf("%d\t%.0lf\n", i+3, ans);

num1 = num2;
num2 = ans;
}


return 0;
}


Thanks for your help!


Roy Mogren
Systems Programmer II
Clark County IT Department
E-mail: mogren@ClarkCountyNV.gov

[coz]

I'll address the binary case only, as the mode=text case is working as you expect.

It appears that your source file "/home/mogren/fibonacci.c" is a file where all of the lines are padded with spaces through column 80. In addition, each line is terminated with a posix style line separator (\n).

Because you are transferring the file in binary mode, Co:Z SFTP does no inspection of the contents for line terminators or trailing spaces; specifically the fact that you have specified "trim" as an option is a red herring, it will not apply for binary mode transfers.

As the bytes are streamed to z/OS, Co:Z will write them as-is to your dataset which is FB=80. This means that the first row of the dataset will contain the first 80 character line, but will not have room for the line terminator. Therefore, the line terminator will be placed in column one of the second record. Then the first 79 bytes of the second line will be placed in the second record and the 80th byte and next line terminator will be placed in columns 1 and 2 of the third record (and so on). This is why there is an addition record in the binary case. Finally, this last record is padded with the default pad character of 0x00, explaining the additional 50 bytes that you are seeing.

You should verify this by performing the binary transfer and examining the dataset in hex mode.

The bottom line is that for this kind of transfer you probably only want to use mode=text.

[rmogondovetailedtech]

Thanks Steve!