SSH CEE5206S SIGNIT timeout and RC=103

tsdjim · Post by **tsdjim** » Tue Feb 15, 2011 8:56 am

I am receiving a timeout when executing a program on zLinux from z/os using CoZLauncher. The message is CEE5206S followed by RC=103.

Here is a summary of the trace:

[12:03:09.819946] CoZLauncher[N]: version: 1.7.5 2010-12-13
[12:03:09.821456] CoZLauncher[N]: Copyright (C) Dovetailed Technologies, LLC. 2006. All rights reserved.
7:03:11.331055] CoZLauncher[D]: About to start target command: ''
[07:03:11.332063] CoZLauncher[T]: <- launchAgent()
[07:03:11.332087] CoZLauncher[T]: -> waitForChildren()
[07:03:11.332504] CoZLauncher[T]: -> agentStdinThread()
[07:03:11.333100] CoZLauncher[T]: -> agentStderrThread()
[07:03:11.333608] CoZLauncher[T]: -> agentStdoutThread()
[07:03:11.610601] CoZLauncher[D]: First agent message received:
OpenSSH_5.0p1, OpenSSL 0.9.8k 25 Mar 2009
debug1: Reading configuration data /etc/ssh/ssh_config

debug1: Reading configuration data /etc/ssh/zos_ssh_config

CEE5206S The signal SIGINT was received.

debug1: zsshSmfSetConnSmfStatus: SMF status is 0

ebug1: Sending command: /opt/dovetail/coz/bin/cozagent

debug2: channel 0: request exec confirm 0

debug2: fd 3 setting TCP_NODELAY

debug2: callback done

debug2: channel 0: open confirm rwindow 0 rmax 32768

debug2: channel 0: rcvd adjust 2097152

debug2: channel 0: rcvd ext data 40

cozagent[N]: version: 1.0.11 2010-10-27
debug2: channel 0: written 40 to efd 6

debug2: channel 0: rcvd ext data 182

todsn-client(1096)[W]: Timeout attempting to connect socket
todsn-client(1095)[W]: Timeout attempting to connect socket
fromdsn-client(1094)[W]: Timeout attempting to connect socket
debug2: channel 0: written 182 to efd 6

debug2: channel 0: rcvd ext data 358

todsn-client(1095)[E]: CoZClient.C(706) - write(1, 0x3ffffd4513d, 27) - [11] Resource temporarily unavailable
todsn-client(1095)[E]: Error: no exit code received from CoZServer
fromdsn-client(1094)[E]: CoZClient.C(706) - write(3, 0x3ffffc9513d, 27) - [11] Resource temporarily unavailable
fromdsn-client(1094)[E]: Error: no exit code received from CoZServer
debug2: channel 0: written 358 to efd 6

debug1: client_input_channel_req: channel 0 rtype exit-status reply 0

debug1: client_input_channel_req: channel 0 rtype eow@openssh.com reply 0

debug2: channel 0: rcvd ext data 338

debug2: channel 0: rcvd eof

debug2: channel 0: output open -> drain

debug2: channel 0: rcvd close

debug2: channel 0: close_read

debug2: channel 0: input open -> closed

todsn-client(1096)[E]: CoZClient.C(706) - write(1, 0x3fffff7313d, 27) - [11] Resource temporarily unavailable
todsn-client(1096)[E]: Error: no exit code received from CoZServer
cozagent[E]: Stdin DD Reader(1094) ended with RC=102
cozagent[E]: Stdout DD Writer(1095) ended with RC=102
cozagent[E]: Stderr DD Writer(1096) ended with RC=102
debug2: channel 0: written 338 to efd 6

debug2: channel 0: obuf empty

debug2: channel 0: close_write

debug2: channel 0: output drain -> closed

debug2: channel 0: almost dead

[07:04:17.299318] CoZLauncher[T]: -> waitStatusAgent()
debug2: channel 0: gc: notify user

[07:04:17.300671] CoZLauncher[D]: CoZAgent: completed with RC=103

Post by **dovetail** » Wed Feb 16, 2011 8:56 am

Can you provide more information -

1) Does this happen all of the time when connecting to this host?
2) What are your settings for ssh-tunnel?

We might need to see more of the trace, but what it looks like is that the target programs are unable to make a socket connection back to the originating job. If you are running with ssh-tunnel=true (the default), then this socket connection is made over the port that is setup as a reverse port forward via ssh. One cause of this kind of problem is that the target SSHD server is not configured to allow reverse port forwarding.

tsdjim · Post by **tsdjim** » Wed Feb 16, 2011 9:12 am

After the initial setup of SSH I tested with some basic commands to Linux and they worked fine,
so I presume that reverse port forwarding is not the problem, but I will check the setting tomorrow.

I believe that I tried both true/false for ssh-tunnel with the same problem, but I will verify this tomorrow.

Could it be a timeout problem with ssh-rnd-helper as some CEE5206S searches on the net seem to indicate?

Post by **dovetail** » Wed Feb 16, 2011 9:53 am

Its possible that ssh-rand-helper is the problem. I would have to see the entire log to know for sure.

Also, if port forwarding failed you would see a debug message in the log from the Ported Tools ssh client.

tsdjim · Post by **tsdjim** » Wed Feb 16, 2011 12:06 pm

I have tried to increase the timeout value for ssh_rnd_helper by specifying the following but without much effect:

//STDIN DD *
export _ZOS_SSH_PRNG_CMDS_TIMEOUT=19999999999

Is this the correct way to set the environment variable for SSH

Post by **dovetail** » Wed Feb 16, 2011 12:21 pm

1) I don't believe that your problem is the ssh_rand_helper since you can make a connection.

2) I don't think that you have a port forwarding error, since you are able to run fine before adding the named pipe

Some questions:

Q1): what version of the Co:Z Target system toolkit are you running. If you aren't running the latest version (1.0.11), I suggest you try that first since there are some fixes that may apply here.

Q2) Can you tell where your program is when this timeout occurs? Has your program finished reading from the named pipe? Try putting "set -x" at the beginning of your STDIN DD script to see if you can tell where things hang.

tsdjim · Post by **tsdjim** » Wed Feb 16, 2011 1:11 pm

Its the latest 1.0.11 Toolkit just recently downloaded.

I will try the set command and other suggestions tommorow.

tsdjim · Post by **tsdjim** » Sat Feb 19, 2011 11:58 am

Yes SSH-TUNNEL was TRUE, setting it to FALSE solved the problem.

Thanks for the help.