SFTPIN DD concatenation

[Steve.Harman]

I have a job that concatenates a dataset with SFTPIN DD *

//SFTPPUT EXEC PROC=SFTPPROC
//SFTPIN DD *
user=srvuser
host=srvname
lzopts="mode=text"
lfile=//DD:OLDDD
rfile=/data/incoming/\
/*
// DD DISP=SHR,DSN=SRV.FILE.NAME
// DD *
. $script_dir/sftp_put.sh
/*
//OLDDD DD DSN=ZDATA.TO.SEND,DISP=SHR

The SRV.FILE.NAME contains one line with a name that contains a timestamp, like

acct.a324.d141208.t152231

The filename is unique on each iteration of the job, created in an earlier job step

It's been working fine for a while, but now the job never sees the . $script line:

CoZBatchÝN¨: Copyright (C) 2005-2013 Dovetailed Technologies LLC. All rights reserved.
CoZBatchÝN¨: version 2.4.5 2014-05-29
CoZBatchÝI¨: executing progname=login-shell="-/bin/sh"
CoZBatchÝI¨: returning rc=exitcode=0

If I use IEBGENER to create the control cards, it concatenates just fine and the SFTP step works correctly:

//STEP01 EXEC PGM=IEBGENER
//SYSPRINT DD SYSOUT=*
//SYSIN DD DUMMY
//SYSUT2 DD DSN=&&TEMP,DISP=(,PASS),UNIT=SYSDA
//SYSUT1 DD *
user=srvuser
host=srvname
lzopts="mode=text"
lfile=//DD:OLDDD
rfile=/data/incoming/\
/*
// DD DISP=SHR,DSN=SRV.FILE.NAME
// DD *
. $script_dir/sftp_put.sh
/*
//SFTPPUT EXEC PROC=SFTPPROC
//SFTPIN DD DSN=&&TEMP,DISP=(OLD,DELETE)
//OLDDD DD DSN=ZDATA.TO.SEND,DISP=SHR

This was working under release 2.4.3, seemed to stop after installing 2.4.5 - But, the really odd part is that the 1st method still works on our lab system. So that leads me to suspect a difference in the OMVS environment. I have a workaround, so this is more of a curiosity thing. Any ideas on why the SFTPIN concatenation doesn't work?

[dovetail]

I can't explain the difference, unless the concatenate file has a problem.


If you just want to pass a file name in from the proc, there is a much easier way:

// EXEC SFTPPROC,PARM='RF=&RFILE'

and then in your script:

rfile = $RF

[Steve.Harman]

Thanks for the response.

Won't &RFILE be an MVS dataset name?

Coming into the SFTPPROC, the MVS dataset name is known. It contains the file name part of rfile, which needs to be smushed together with the directory portion to create an absolute specification for rfile.

The MVS dataset name is SRV.FILE.NAME - but rfile needs to be /data/incoming/acct.a324.d141208.t152231

Something has to open SRV.FILE.NAME, read it, and add the /data/incoming/ to the front of its contents. I was using the continuation backslash followed by the concatenated DSN to form the complete rfile, using two lines.

Or maybe I have a misunderstanding of how to define &RFILE. It seems like some additional step is needed to formulate the absolute path. IEBGENER works, I could write a rexx or something, but each requires another job step. Concatenating within the SFTP step made it a little easier.

[dovetail]

&RFILE would be substituted via a PROC variable or SET variable in the JCL.
Like:

// SET RFILE='/data/incoming/acct.a324.d141208.t152231'

So maybe this is more like what you want?

Code: Select all

...
// SET FN=ACCT.A324.D141208.T152231
// EXEC SFTPPROC,ARGS='FN=&FN'
//FROMDD  DD DISP=SHR,DSN=&FN
//SFTPIN DD *
user=srvuser
host=srvname
lzopts="mode=text"
lfile=//DD:FROMDD
lowerFN=$(echo $FN | tr "[:upper:]" "[:lower:]")
rfile=/data/incoming/$lowerFN
. $script_dir/sftp_put.sh
//

Remember - our sample SFTPPROC executed PGM=COZBATCH, which is used to run a z/OS UNIX shell script in batch. So you can write any UNIX shell scripting that you need to populate the variables before you run the "sftp_put.sh" script, which finishes configuring stuf and invokes the "cozsftp" shell command.

[Steve.Harman]

Thanks, again. I appreciate you making the effort to provide a solution.

I guess I'm not very good at communicating my situation, and please don't spend any more time on this.

But, for the record...

On the SET command, the acct.a324.d141208.t152231 is not known. acct.a324.d141208.t152231 is a record in a dataset, and is unique for every execution. The dataset is known, SRV.FILE.NAME. A means of extracting data from SRV.FILE.NAME (reading the dataset to get the one-line record of acct.a324.d141208.t152231) and then creating the final rfile name of /data/incoming/acct.a324.d141208.t152231 is what my goal is.

So, when the job gets to the SFTP step the only name that's known is SRV.FILE.NAME. I don't know what's in the file, but I want that one-line record it contains to be used as the filename on the remote server.

The odd-looking concatenation works with IEBGENER. By adding the DSN into the control cards, it forces the interpreter to open SRV.FILE.NAME and use the contents in the concatenation.

So, the concatenation:

//SYSUT1 DD *
user=srvuser
host=srvname
lzopts="mode=text"
lfile=//DD:OLDDD
rfile=/data/incoming/\ <=== This is a continued line
/* <=== End this block of inline cards
// DD DISP=SHR,DSN=SRV.FILE.NAME <=== Add whatever's in this file to them
// DD * <=== Continue the concatenation
. $script_dir/sftp_put.sh
/*

creates a stream of cards that resolves to:

user=srvuser
host=srvname
lzopts="mode=text"
lfile=//DD:OLDDD
rfile=/data/incoming/\
acct.a324.d141208.t152231
. $script_dir/sftp_put.sh

That's what mysteriously stopped working in the SFTPPROC, but IEBGENER works as expected. And that's my permanent work-around, unless I can figure out what caused it to stop.

Thanks again for looking at this.

[dovetail]

I see.
I cannot explain why the concatenation doesn't work directly with SFTPIN DD *. Are the datasets all compatible for concatenating?

Also, you could do this:

Code: Select all

//SFTPIN DD *
...
# read a single-line dataset into variable srvfn
srvfn=$($cozbin_dir/fromdsn -x shr //SRV.FILE.NAME)

lfile=//DD:FROMDD
rfile=/data/incoming/$srvfn
. $script_dir/sftp_put.sh
//

[Steve.Harman]

Awesome! That's a much simpler solution and one less job step. Thank you!

[johson]

Coming into the SFTPPROC, the MVS dataset name is known. It contains the file name part of rfile, which needs to be smushed together with the directory portion to create an absolute specification for rfile.