Number of bytes vs Records when sending files

LeonBylsma · Post by **LeonBylsma** » Thu Apr 16, 2009 8:37 am

My question is this ... when receiving files on the mainframe into omvs, we get a bytes sent and bytes written message via the smf records (into the Mvs Log) as follows : 127 bytes read, 127 bytes sent ...
This is very usefull to verify whether the file sent has been received in full. However when files are sent directly to Zos then it records this on the mvs log : 1215 bytes received, 15 records written .... how do I verify that the file that was sent was sent in full ? I use this extensively for automation purposes and needs to find a way of verifying whether complete files have been sent

Post by **dovetail** » Thu Apr 16, 2009 10:35 am

Please clarify which program is writing the SMF records that you see: Co:Z SFTP or IBM FTP?

With Co:Z SFTP, if you get an SMF record without an error indication (replycode, abendinfo), then it completed successfully. The SFTP protocol has a handshake to confirm that all data was sent.

I do not understand what you mean "when files are sent directly to Zos then it records...". Sent how? Which protocol / server product? How are messages written to the MVS LOG? Co:Z SFTP does not write messages to SYSLOG.

LeonBylsma · Post by **LeonBylsma** » Fri Apr 17, 2009 1:29 am

We were given a module from yourselves (Dovetailed Tehcnologies) that allows us to write all ftp messages as SMF records and therefore we could re-route the ftp messages into the MVS logs, however that in itself is not really the issue. If you look at the support manual that you supply on this website for Coz : SFTP you mention for example on the topic 4.5 "Working with posix files" the following example :

$ sftp user@linux.com
Connecting to linux.com...
user@linux.com's password: *****
sftp> lzopts mode=text,servercp=UTF-8
mode=text servercp=UTF-8
sftp> lzopts
clientcp=IBM-1047 loglevel=I mode=text
servercp=UTF-8
sftp> pwd
Remote working directory: /tmp
sftp> get msgs.txt
Fetching /tmp/msgs.txt to msgs.txt
/tmp/msgs.txt 100% 19KB 19.0KB/s 00:00
ZosPosixFile: Closing file msgs.txt - 19488 bytes received, 19488 bytes written
sftp>

In here as the last line, the message indicates bytes received and bytes written ... however when working with datasets as in topic 4.4 then the example is as follows :

$ sftp user@linux.com
Connecting to linux.com...
user@linux.com's password:
sftp> lzopts mode=text
mode=text
sftp> lzopts
clientcp=IBM-1047 loglevel=I mode=text
servercp=ISO8859-1
sftp> get /tmp/GPGDSN //USER.GPGDSN
Fetching /tmp/GPGDSN to //USER.GPGDSN
ZosDataset: Opening dataset USER.GPGDSN for write with options: new catalog
/tmp/GPGDSN 100% 1215 1.2KB/s 00:00
ZosDataset: Closing dataset //USER.GPGDSN - 1215 bytes received, 15 records written

so here you have 1215 bytes received and 15 records written ....

how do I know from this that all bytes sent was actually written as it reflects records as opposed to bytes ?

Post by **dovetail** » Fri Apr 17, 2009 11:44 am

I don't recall giving out a module to do this. Are you referring to this? :
http://dovetail.com/forum/viewtopic.php?t=745

If so, then I don't believe that SMF has anything to do with this. If I understand what you are doing, then you are redirecting the Co:Z SFTP server messages to /dev/console so that they are going to your system log.

Let me try to answer your question -

In both cases, Posix files and datasets, these count messages give you the number of bytes received over the network versus the number of records or bytes written to the file.

In the case of Posix files - the number of bytes received may not be the same as the number of bytes written, since the server settings may have a different line terminator.

In both cases, this message is written when the entire file has been received over the wire (there is a protocol ack to indicate this) and the entire file has been written. So this message will *never* tell you if the file is incomplete.

I hope that this clarifies the message a little, but I assume that this is not the news that you are looking for. I will try to follow up with some suggestions to meet your requirement.

LeonBylsma · Post by **LeonBylsma** » Mon Apr 20, 2009 12:47 am

Thank you for the reply ... in fact this does resolve my problem, as I was unsure if the message would show if a file could for some reason not be sent in full. i.e. so this means that you would never get a partial file being sent as it should fail back on the sender's side and that also means that I can assume that once the message shows, that the file was sent in full as intended ... Thank you for clarifying this for me ...

Regards