Intermittent CoZ problem

General discussion of the Co:Z Toolkit
Post Reply
jacobsm
Posts: 37
Joined: Thu Apr 23, 2009 9:18 am
Location: Tampa, Florida

Intermittent CoZ problem

Post by jacobsm »

We seem to have Intermittent (first time) problems going from zOS to a zLinux server.

*-*-*-*-* JSTEP04 COZLNCH SYSOUT

CoZLauncherÝN¨: version: 1.3.0 2009-06-11
CoZLauncherÝN¨: Copyright (C) Dovetailed Technologies, LLC. 2006. All rights res
CoZLauncherÝW¨: Unable to read /etc/init.options, continuing without setting bas
cozagentÝN¨: version: 1.0.5 2008-03-01
todsn-clientÝE¨: transmitted byte counts do not match, server sent/received = 0/
todsn-clientÝE¨: Error: no exit code received from CoZServer
todsn-clientÝE¨: transmitted byte counts do not match, server sent/received = 0/
todsn-clientÝE¨: Error: no exit code received from CoZServer
cozagentÝE¨: Target Program(6293) ended with RC=1
cozagentÝE¨: Stderr DD Writer(6343) ended with RC=102
cozagentÝE¨: Stdout DD Writer(6295) ended with RC=102
CoZLauncherÝE¨: info@ifninfo target command '<default shell>' ended with RC=1

Re-running the same jobstep worked without a problem.
dovetail
Site Admin
Posts: 2022
Joined: Thu Jul 29, 2004 12:12 pm

Post by dovetail »

This might caused by some problem with one of the the cozserver or todsn/fromdsn "servers" that runs on z/OS. Perhaps there was some kind of dataset or I/O error. Is there any other information in the job or syslog?

If you can reproduce this, please enable trace-level logging in the launcher job and email your trace to info@dovetail.com.
//COZLG1 JOB (),'COZ'
//STEP1 EXEC PROC=COZPROC,
// ARGS='-LT,t myuid@linux1.myco.com'
...
jacobsm
Posts: 37
Joined: Thu Apr 23, 2009 9:18 am
Location: Tampa, Florida

Intermittent CoZ problem

Post by jacobsm »

Thanks. I've told the user to turn on the trace and recreate the problem. we also get this error every so often.


*-*-*-*-* DUMMYSTP COZLNCH SYSOUT

CoZLauncherÝN¨: version: 1.3.0 2009-06-11
CoZLauncherÝN¨: Copyright (C) Dovetailed Technologies, LLC. 2006. All rights res
CoZLauncherÝW¨: Unable to read /etc/init.options, continuing without setting bas
CoZLauncherÝE¨: EDC8115I Address already in use. (errno2=0x744C7247): listen() e

Would the same trace help here also?
dovetail
Site Admin
Posts: 2022
Joined: Thu Jul 29, 2004 12:12 pm

Post by dovetail »

You are getting a warning message and an error message:

[W] Unable to read /etc/init.options, continuing without setting bas

This is a warning about reading /etc/init.options, which Co:Z will use to set base system environment variables, like TZ. As recommended in the IBM USS init and tuning guide, this file should be readable by all.

[E] EDC8115I Address already in use. (errno2=0x744C7247): listen()

This probably means that there were no ports available in the range specified by the configuration property "server-ports". You need to configure a range of ports that is large enough to accommodate the number of concurrent jobs that are using that range (only one port is required for each concurrent Co:Z job, assuming that other programs aren't poaching them). Turning on tracing for the COZLauncher might help to diagnose:

// EXEC COZPROC,ARGS='-LT user@host'
jacobsm
Posts: 37
Joined: Thu Apr 23, 2009 9:18 am
Location: Tampa, Florida

Intermittent CoZ problem

Post by jacobsm »

Is there any reason not to turn on the ssh-tunnel=true option in the configuration file to get around the port limit?
dovetail
Site Admin
Posts: 2022
Joined: Thu Jul 29, 2004 12:12 pm

Post by dovetail »

ssh-tunnel=true/false will not solve port availability problems on z/OS, but if ssh-tunnel=false is used then you won't need to reserve the same port range on the remote server.

Here are the gory details:

With ssh-tunnel=true:
- the CoZ Launcher batch job will bind/listen on an available port in the server-ports range using the loopback interface (127.0.0.1)
- the remote cozagent will listen on the same port on the remote machine, also on the loopback interface
- the ssh connection will setup a reverse port-forward from the remote port to the z/OS port and this will be used to create tunneled sockets for all todsn/fromdsn commands

With ssh-tunnel=false:
- the CoZ Launcher batch job will bind/listen on an available port in the server-ports range using any interface (0.0.0.0)
- the remote cozagent will not listen on a port, and no ssh reverse port-forward will be setup
- remote todsn or fromdsn commands will open sockets independent of the ssh connection, directly to the z/OS listening port. These connections will therefore not be encrypted.

In both cases, the connections from todsn/from to the z/OS launching job will send a secure random token that was sent by launching job over the ssh session to the cozagent. This token will be used to double-check that any connections to the z/OS launching job are from the remote session. If a valid token is not presented to the z/OS listening port (socket), the connection will be rejected.
Post Reply