upgrade from to 2.4.1 from 1.7.2 batch fails fots1373

Discussion of the COZBATCH utility for z/OS
Post Reply
dmbru1
Posts: 6
Joined: Thu Apr 17, 2014 8:17 am

upgrade from to 2.4.1 from 1.7.2 batch fails fots1373

Post by dmbru1 »

upgrading from 1.7.2 to 2.4.1 the batch fails but interative works with the same userids and passwords for our jobs referencing password dataset with ask_pass

There are no leading spaces, all are case sensitive and no line numbers in the password datasets

failure- FAILED FOTS1373 Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password). Message

we have added-vvv for debugging

same batch works on old version 1.7.2, fails when run on 2.4.1
example
//RUNSFTP EXEC PGM=COZBATCH,REGION=0M,PARM='_CEE_RUNOPTS=HEAP(,,,FREE)'
//STEPLIB DD DISP=SHR,DSN=COZ.V1R7M2.LOADLIB
//STDIN DD *
# CUSTOMIZE THESE ...
coz_bin="/usr/local/coz/bin"
remoteuser="ssrdftp"
server="107.115.202.41"
servercp="ISO8859-1"
sfile=XFR.UAT.ALC.SSR.DEPOSITS
rfile=stmtALC160420141041.940

export PASSWD_DSN="PASSDEF.UNIX.SSRDFTP(DEV)" - upper case
export SSH_ASKPASS=£coz_bin/read_sfs_passwd_dsn.sh
export DISPLAY=all

ssh_opts="-obatchmode=no"
ssh_opts="£ssh_opts -oConnectTimeout=10"
ssh_opts="£ssh_opts -oServerAliveInterval=10"
ssh_opts="£ssh_opts -oStrictHostKeyChecking=no"
ssh_opts="£ssh_opts -oUserKnownHostsFile=/dev/null"

£coz_bin/cozsftp £ssh_opts -b- -vvv £remoteuser@£server <<EOB
lzopts mode=text,servercp=£servercp,loglevel=I

CD IN
put //£sfile £rfile
EOB
//*
it still fails if the steplib to the old version is removed
does it make a difference that the new version 2.4.1
some batch points in the ask pass to
COZ_BIN=`dirname £0`
"£COZ_BIN/fromdsn" -x shr "£PASSWD_DSN" | awk 'NR==2 {print £2}'
and
"£COZ_BIN/fromdsn" -x shr "£PASSWD_DSN" | sed 's/¬ *\(.*\) *£/\1/'
but batch still fails

thank you
dovetail
Site Admin
Posts: 2022
Joined: Thu Jul 29, 2004 12:12 pm

Re: upgrade from to 2.4.1 from 1.7.2 batch fails fots1373

Post by dovetail »

If you would like to post the output from running with adding the following before the cozsftp command, we will revew. (Please remove any confidential information)

export COZ_LOG=F
ssh_opts="$ssh_opts -vvv"

I guess that the problem could be that your z/OS Unix codepage or locale setting is causing a problem with this in read_passwd_dsn.sh

"$COZ_BIN/fromdsn" -x shr "$PASSWD_DSN" | sed 's/^ *\(.*\) *$/\1/'

the sed command is attempting to remove leading blanks from the line.
If your password files don't have leading blanks, then you can remove the | sed stuff.

I suggest that you test this by adding the following lines to your script (before the cozsftp command) to see if the read_passwd_dsn.sh script is working properly in your environment:

# Test the read_passwd_dsn.sh script
export PASSWD_DSN="PASSDEF.UNIX.SSRDFTP(DEV)"
£coz_bin/read_passwd_dsn.sh

Also - on the export PASSWD_DSN line... do you actually have this?: - upper case
This would cause an error.


PS> You might want to look at the sample Co:Z SFTP batch PROC and scripts: http://dovetail.com/docs/sftp/client.html#client-batch
These will simplify your jobs and avoid most problems with locale settings since they do "export LANG=C" automatically.
dmbru1
Posts: 6
Joined: Thu Apr 17, 2014 8:17 am

Re: upgrade from to 2.4.1 from 1.7.2 batch fails fots1373

Post by dmbru1 »

CoZBatchÝN¨: Copyright (C) 2005-2013 Dovetailed Technologies LLC. All rights res
CoZBatchÝN¨: version 2.4.1 2013-06-24
CoZBatchÝI¨: executing progname=login-shell="-/bin/sh"
fromdsn(PASSDEF.UNIX.FORBTFTP(DEVT))ÝN¨: 2 records/160 bytes read; 31 bytes writ
KBytes/sec).
180.33.18.90
user password
ZosSettingsÝF¨: -> ZosSettings()
Co:Z SFTP version: 2.4.1 (5.0p1) 2013-06-24
Copyright (C) Dovetailed Technologies, LLC. 2008-2013. All rights reserved.
ZosSettingsÝF¨: clientCodePage set to IBM-1047 (default)
ZosSettingsÝF¨: serverCodePage set to ISO8859-1 (default)
ZosSettingsÝF¨: -> set(mode=binary)
ZosSettingsÝF¨: <- set()
ZosSettingsÝF¨: -> set(conddisp=catlg)
our batch colleague submitted this
ZosSettingsÝF¨: <- set()
ZosSettingsÝF¨: -> set(overflow=wrap)
ZosSettingsÝF¨: <- set()
ZosSettingsÝF¨: -> set(linerule=flexible)
ZosSettingsÝF¨: <- set()
ZosSettingsÝF¨: -> set(jesowner=JUNUXX)
ZosSettingsÝF¨: <- set()
ZosSettingsÝF¨: -> set(jeslrecl=80)
ZosSettingsÝF¨: <- set()
ZosSettingsÝF¨: -> set(jesrecfm=f)
ZosSettingsÝF¨: <- set()
ZosSettingsÝF¨: -> set(jesstatus=*)
ZosSettingsÝF¨: <- set()
ZosSettingsÝT¨: -> readConfiguration()
ZosSettingsÝT¨: -> processConfigurationFile(/u/junuxx/.ssh/cozsftp_config)
ZosSettingsÝD¨: no configuration file (/u/junuxx/.ssh/cozsftp_config) found.
ZosSettingsÝT¨: <- processConfigurationFile()
ZosSettingsÝT¨: -> processConfigurationFile(/etc/ssh/cozsftp_config)
ZosSettingsÝT¨: processConfigurationFile: Processing fixed: configuration fil
ZosSettingsÝT¨: processConfigurationFile: Processing default: configuration f
ZosSettingsÝT¨: processConfigurationFile: Processing configuration file optio
VAR("_CEE_REALLOC_CONTROL=256K,25")'
ZosSettingsÝF¨: -> parseOptionString(ssh-le-options=HEAP(12M,1M,,FREE),ENVAR("_C
ZosSettingsÝF¨: -> parseAndSetOption(ssh-le-options=HEAP(12M,1M,,FREE),ENVAR("_CZosSettingsÝF¨: -> set(jesowner=JUNUXX)
ZosSettingsÝF¨: <- set()
ZosSettingsÝF¨: -> set(jeslrecl=80)
ZosSettingsÝF¨: <- set()
ZosSettingsÝF¨: -> set(jesrecfm=f)
ZosSettingsÝF¨: <- set()
ZosSettingsÝF¨: -> set(jesstatus=*)
ZosSettingsÝF¨: <- set()
ZosSettingsÝT¨: -> readConfiguration()
ZosSettingsÝT¨: -> processConfigurationFile(/u/junuxx/.ssh/cozsftp_config)
ZosSettingsÝD¨: no configuration file (/u/junuxx/.ssh/cozsftp_config) found.
ZosSettingsÝT¨: <- processConfigurationFile()
ZosSettingsÝT¨: -> processConfigurationFile(/etc/ssh/cozsftp_config)
ZosSettingsÝT¨: processConfigurationFile: Processing fixed: configuration fil
ZosSettingsÝT¨: processConfigurationFile: Processing default: configuration f
ZosSettingsÝT¨: processConfigurationFile: Processing configuration file optio
VAR("_CEE_REALLOC_CONTROL=256K,25")'
ZosSettingsÝF¨: -> parseOptionString(ssh-le-options=HEAP(12M,1M,,FREE),ENVAR("_C
ZosSettingsÝF¨: -> parseAndSetOption(ssh-le-options=HEAP(12M,1M,,FREE),ENVAR("_CZosSettingsÝF¨: -> set(jesowner=JUNUXX)
ZosSettingsÝF¨: <- set()
ZosSettingsÝF¨: -> set(jeslrecl=80)
ZosSettingsÝF¨: <- set()
ZosSettingsÝF¨: -> set(jesrecfm=f)
ZosSettingsÝF¨: <- set()
ZosSettingsÝF¨: -> set(jesstatus=*)
ZosSettingsÝF¨: <- set()
ZosSettingsÝT¨: -> readConfiguration()
ZosSettingsÝT¨: -> processConfigurationFile(/u/junuxx/.ssh/cozsftp_config)
ZosSettingsÝD¨: no configuration file (/u/junuxx/.ssh/cozsftp_config) found.
ZosSettingsÝT¨: <- processConfigurationFile()
ZosSettingsÝT¨: -> processConfigurationFile(/etc/ssh/cozsftp_config)
ZosSettingsÝT¨: processConfigurationFile: Processing fixed: configuration fil
ZosSettingsÝT¨: processConfigurationFile: Processing default: configuration f
ZosSettingsÝT¨: processConfigurationFile: Processing configuration file optio
VAR("_CEE_REALLOC_CONTROL=256K,25")'
ZosSettingsÝF¨: -> parseOptionString(ssh-le-options=HEAP(12M,1M,,FREE),ENVAR("_C
ZosSettingsÝF¨: -> parseAndSetOption(ssh-le-options=HEAP(12M,1M,,FREE),ENVAR("_C
ZosSettingsÝF¨: -> set(ssh-le-options=HEAP(12M,1M,,FREE),ENVAR("_CEE_REALLOC_CON
ZosSettingsÝF¨: <- set()
ZosSettingsÝF¨: <- parseAndSetOption(1)
ZosSettingsÝF¨: <- parseOptionString()
ZosSettingsÝT¨: <- processConfigurationFile()
ZosSettingsÝT¨: <- readConfiguration()
ZosSettingsÝF¨: <- ZosSettings()
ZosSettingsÝF¨: -> parseOptionString()
ZosSettingsÝF¨: <- parseOptionString()
ZosUtilÝT¨: -> zos_log_region_size()
ZosUtilÝD¨: region size requested = 4096K, Actual below/above limit = 4160K / 32
ZosUtilÝT¨: <- zos_log_region_size()
usage: cozsftp Ý-1Cv¨ Ý-B buffer_size¨ Ý-b batchfile¨ Ý-F ssh_config¨
Ý-o ssh_option¨ Ý-P sftp_server_path¨ Ý-R num_requests¨
Ý-S program¨ Ý-s subsystem | sftp_server¨ host
Ý-k keyringÝ;label¨¨
cozsftp ÝÝuser@¨hostÝ:file Ýfile¨¨¨
cozsftp ÝÝuser@¨hostÝ:dirÝ/¨¨¨
cozsftp -b batchfile Ýuser@¨host
CoZBatchÝI¨: returning rc=exitcode=1
dmbru1
Posts: 6
Joined: Thu Apr 17, 2014 8:17 am

Re: upgrade from to 2.4.1 from 1.7.2 batch fails fots1373

Post by dmbru1 »

i edited the password dataset member to just have the password in it and not the username first and it looks like the job worked, using usr/local/coz/bin/read_passwd_dsn.sh, just asking the batch gys to confirm and concur they are happy, as the userid and password work manually when I putty to the box so I wondered why the password member had the userid and password in it on the same line separated by a space
dovetail
Site Admin
Posts: 2022
Joined: Thu Jul 29, 2004 12:12 pm

Re: upgrade from to 2.4.1 from 1.7.2 batch fails fots1373

Post by dovetail »

I understand now; I apologize that I didn't see it before. It appears that someone at your installation customized the read_passwd_dsn.sh script to use awk to pick off the password from a line that contained userid and password. This is not how we distributed it. When you upgraded, you got the standard version of the script, which only expects to find the password.
dmbru1
Posts: 6
Joined: Thu Apr 17, 2014 8:17 am

Re: upgrade from to 2.4.1 from 1.7.2 batch fails fots1373

Post by dmbru1 »

on the lpar concerned we have two files pointed to in their cozbatch
read_sfs_passwd_dsn.sh
COZ_BIN=`dirname £0`
"£COZ_BIN/fromdsn" -x shr "£PASSWD_DSN" | awk 'NR==2 {print £2}'
and
read_std_passwd_dsn.sh
COZ_BIN=`dirname £0`
"£COZ_BIN/fromdsn" -x shr "£PASSWD_DSN" | awk 'NR==1 {print £2}'

I got it working with both using this
//RUNSFTP EXEC PGM=COZBATCH
//STEPLIB DD DISP=SHR,DSN=COZ.V2R4M1.LOADLIB
//STDIN DD *

# Customize these ...
coz_bin="/usr/local/coz/bin"
remoteuser="forbtftp"
server="180.33.18.90"
servercp="ISO8859-1"
remotefile="//net.symbols.xfrplex"

# These can be used to read the ssh password from a (secured) dataset
# if you don't want to setup public/private keypairs
export PASSWD_DSN='//tsgdmb.jcl.cntl.xfr(pwd)'
export SSH_ASKPASS=£coz_bin/read_passwd_dsn.sh
export DISPLAY=none
export COZ_LOG=F
ssh_opts="-oBatchMode=no" # allows ssh to use SSH_ASKPASS program
ssh_opts="£ssh_opts -oConnectTimeout=60"
ssh_opts="£ssh_opts -vvv"
ssh_opts="£ssh_opts -oServerAliveInterval=60"
ssh_opts="£ssh_opts -oStrictHostKeyChecking=no" # accept initial host ke

# Invoke the Co:Z sftp client with an in-line batch of commands
# that downloads a remote file to a local DD.
# Note that "-oBatchMode=no" must be specified before "-b"
# since ssh opts are first-sticky

£coz_bin/cozsftp £ssh_opts -vvv -b- £remoteuser@£server <<EOB
£coz_bin/read_passwd_dsn.sh
cd /sasgendata1/general/mktg/_incoming/from_junction_uat
put £remotefile
EOB job 5138
and pointing to these
read_sfs_passwd_dsn.sh
COZ_BIN=`dirname £0`
"£COZ_BIN/fromdsn" -x shr "£PASSWD_DSN" | awk 'NR==2 {print £2}'
and
read_std_passwd_dsn.sh
COZ_BIN=`dirname £0`
"£COZ_BIN/fromdsn" -x shr "£PASSWD_DSN" | awk 'NR==1 {print £2}'
using the password set up they have already
walab1
Posts: 2
Joined: Wed Apr 23, 2014 10:12 am

Re: upgrade from to 2.4.1 from 1.7.2 batch fails fots1373

Post by walab1 »

Good afternoon

When we installed the new version we copied the read_passwd_dsn.sh from the old version to read_passwd_dsn.sh.old in the new version as well as the read_sfs_passwd_dsn.sh.

We have run the same job in control M on the LPar with version 2.4.1 pointing to all 3 versions of the read_passwd_dsn and all the jobs failed and then ran the same jobs on the LPar with Version 1.7.2 pointing to both version of the read_passwd_dsn (not the new one) and both jobs were successful.

Thank you
dovetail
Site Admin
Posts: 2022
Joined: Thu Jul 29, 2004 12:12 pm

Re: upgrade from to 2.4.1 from 1.7.2 batch fails fots1373

Post by dovetail »

Have you customized read_passwd_dsn.sh to change what was distributed?

Please run a test job that includes this before invoking cozsftp:

# Test the read_passwd_dsn.sh script
export PASSWD_DSN="YOUR.PASSWD.DSN(MEMBER)"
$coz_bin/read_passwd_dsn.sh

It should simply print out the password to stdout. Does it work?

Perhaps the problem is that when you copied the read_passwd_dsn.sh script that you did not make it executable?
It should be:

chmod 755 read_passwd_dsn.sh
walab1
Posts: 2
Joined: Wed Apr 23, 2014 10:12 am

Re: upgrade from to 2.4.1 from 1.7.2 batch fails fots1373

Post by walab1 »

Good Morning

Sorry for the delay in getting back to you.

We have fixed the issue. we were running the controll M batch Jobs with a REGION=4M in the jobcard and a REGION=0M in the SFTP step. We have now corrected the Job card to REGION=0M and all the jobs have now completed successfully.

Thank you for your help with this matter
Post Reply