Parallel streams

Discussion of the Co:Z Toolkit Dataset Pipes utilities
Post Reply
sdean4
Posts: 25
Joined: Wed Nov 07, 2012 2:05 pm

Parallel streams

Post by sdean4 »

We are looking to maximize throughput when sending data between z/os and a remote machine. We can force the utilization of separate paths through 4 separate OSA cards by using a virtual IP address (1 address per OSA). Will dataset pipes be able to handle this parallelism without becoming the single-threaded bottleneck?
dovetail
Site Admin
Posts: 2022
Joined: Thu Jul 29, 2004 12:12 pm

Re: Parallel streams

Post by dovetail »

Assuming that you have set ssh-tunnel=false, then the remote/target system will open IP connections to the hostname/ipaddress specified by the "server-host" configuration property.
If this is a virtual IP, then it should work as you describe. Each connection is opened by a distinct process on the remote system, so there is no "single threading".

See: http://dovetail.com/docs/coz/config.htm ... r_optional
sdean4
Posts: 25
Joined: Wed Nov 07, 2012 2:05 pm

Re: Parallel streams

Post by sdean4 »

Thanks for your reply. I have discussed this with my systems architect, and he would like additional clarification to make sure we are understanding this configuration correctly:

Assuming the Co:Z "server" is on the Solaris x86 machine, and each LPAR is running a Co:Z "agent", we make the following assumptions:
•The Co:Z agents contact the server on a port defined in the server-host configuration property
•The Co:Z server will respond to agent requests from the same IP it is listening on

Expanding this idea, if our server has four 1Gb NICs and the mainframe has four 1Gb OSA ports, we should be able to maximize our throughput if each of the four target LPARs (agents) could contact the Co:Z server on an independent physical path. For this to work:
•Each LPAR (agent) would be assigned its own OSA interface and configured to contact co:Z on an independent IP address on the server
•Co:Z on the server would have to be able to listen on and manage four 1Gb NIC/IPaddresses on the server OR we would have to spin up 4 instances of Co:Z on the server, each listening on its own NIC/IPaddress

Please comment or expand on our assumptions and explain what additional/modified Co:Z configuration setup, if any, that we need to accomplish the last bullet item.
thanks,
Sandy
dovetail
Site Admin
Posts: 2022
Joined: Thu Jul 29, 2004 12:12 pm

Re: Parallel streams

Post by dovetail »

I'm not sure exactly what you are trying to accomplish. I can't comment on how you would set up your routing tables, etc., but I will try to clarify how things work so that your network engineers can implement to your requirements.

Some clarifications:
- Co:Z Launcher and Co:Z Server run on the initiating / launching z/OS
- Co:Z Agent runs on the target system.
- when Dataset Pipes commands are used on the target system, they will open sockets back to the launching system to Co:Z Server. This includes the STDOUT/STDERR/STDIN redirections, which are done implicitly by Co:Z Agent.

Assuming that you are using ssh-tunnel=false, then all of the connections are made independently according to the following:

- the Co:Z Launcher starts the ssh connection to start the target Co:Z Agent, using the target hostname or ip address given on the ARGS=user@host or target-host property.

- the Co:Z Launcher also starts a Co:Z Server listener on the first available port found in the server-ports range on the stack. By default it will listen on all stacks on a multi-stack z/OS system, but this can be controlled using the server-ip-stack property.

- the Co:Z Agent starts connections to the Co:Z Server at the name or ip address given in the server-host property (or the default z/OS host name), to the port selected above from the server-ports range.

So, what you want to do is to use routing tables to control is the last bullet so that you optimally use multiple adapters on both the target and launching system.
Post Reply