Idea: DLL for Dataset Pipes, invoking from a user program.

Discussion of the Co:Z Toolkit Dataset Pipes utilities
Post Reply
john.mckown
Posts: 48
Joined: Tue Jun 12, 2007 2:46 pm

Idea: DLL for Dataset Pipes, invoking from a user program.

Post by john.mckown »

I am wonder what ya'll might think of this idea. Taking the functionality of the todsn and fromdsn and wrapping it in a DLL. I think this would make it easier to write an application which could use the functionality of todsn and fromdsn more easily than using a language facility to fork/spawn a new process to perform a like function. And, thinking about it, a DLL wrapping the lsjes functionality might be very helpful too. This might not be needed on z/OS itself. But I am thinking more of on a remote system.

As an example. I run Fedora (FC-17) on my desktop. It would be nice to set up an SSH connection to the z/OS UNIX environment (which I do using the ControlMaster, ControlPath, and ControlPersist parameters in the ~/.ssh/config file). I could then have a "full screen" application (QT or GTK+ or ???) on my Linux/Intel box which could do things such use the lsjes capability to display entries in the JES SPOOL. The user could then direct the application to "select" a particular job. This would cause the application to use the fromdsn DLL to read the SPOOL data down to the Linux system (perhaps storing it in a temp file, or maybe just memory) and then displaying it (perhaps using "less") or maybe even putting the data into an editor's (vim/gvim/emacs/...) buffer. This would be basically like doing an SE on the job in SDSF. The only thing missing from this scenario is a simple way for the user to cancel or purge the job's output (which I can envision a rather complicated way of doing, using my own code to run a REXX program on z/OS).

Another application might be to write an equivalent of ISPF option 3.4 on Linux which somehow does a catsearch on z/OS to get a list of datasets. Once it has this, it displays it. The could could then, like with ISPF 3.4, do a member list. From there use the fromdsn DLL to copy the member to do the equivalent of browse, view, or edit. If edit, then it would even be possible to use the todsn DLL to write the changed data back to the member, or dataset (if it was sequential). Thinking about it, perhaps even a DLL to delete a member. I do recognize the issue of dataset / member integrity due to lack of the ENQs used by ISPF so that two people don't edit the same dataset / member concurrently. Perhaps this could be handled similar to how vim does, perhaps by reading the current data, comparing it to the original (or checking an MD5 hash), and warning the use if it has changed.

I will grant that all of this can be done by using the host language's functions such as system() or fork() or spawn() to run the commands just as they would be on the command line. I guess that I consider using a DLL to be more "elegant". I'm not sure, but perhaps the "run the external" command would even be more general because not all languages (especially scripting languages like awk and Perl) can use a DLL. Hum.

If this is considered, then possible enhancements on the z/OS side would be a way to run arbitrary z/OS UNIX commands. As in the example thoughts above, in order to run the "catsearch" program and the REXX UNIX program to purge job output.

Having been educated a bit by what you've accomplished with your sftp server (which might be a way to purge that job now that I think of it), I might even try to write my own z/OS ssh subsystem code to do some of these things. Would be interesting just to try, even if I don't succeed.

Well, that's it for now for me. See ya'll on MVS-OE or IBM-MAIN perhaps.
dovetail
Site Admin
Posts: 2022
Joined: Thu Jul 29, 2004 12:12 pm

Re: Idea: DLL for Dataset Pipes, invoking from a user progra

Post by dovetail »

Some very interesting ideas.

Under Linux, with ssh controlmaster session running, it would be pretty easy to use the equivalent of the "pipe()" C function to invoke fromdsn/todsn/fromfile/tofile/cozclient commands. (other languages have similar mechanisms - for example in perl you use "open (FH, "fromdsn .... |"); So, I'm not sure that a DLL is really necessary or adds much. So, a ISPF 3.4 GUI seems eminently feasible to me. Perhaps written in something like Python?

The problem with this is that under Windows you can't really do this ... OpenSSH controlmaster sessions don't work under Windows. There is too much overhead to start and authenticati a new ssh session for each command. So, for Windows, it might be cool to have a DLL that did the equivalent of cozclient-like functionality, but with the ability to manage multiple command channels under a single ssh connection. (fromdsn/todsn/fromfile/tofile are really just special cases of the cozclient command). So, a DLL that is sort of a re-implementation of the cozclient command, but one that setup multiple ssh command channels under a single ssh session would seem like a good idea. Something like libssh2 could probably be used to implement this.
john.mckown
Posts: 48
Joined: Tue Jun 12, 2007 2:46 pm

Re: Idea: DLL for Dataset Pipes, invoking from a user progra

Post by john.mckown »

The last point first: Lack of MS-Windows support. I never consider MS-Windows. I use it only at work, and only because I am forced to. I really don't like it. That's why I never think of it.

I guess that I was thinking of a DLL, especially on z/OS UNIX, because my work environment is "CPU poor" and management is constantly wanting to decrease our MSU footprint to reduce our software costs. If I really need to do this, I guess I'll go with the "more expensive" fork()-like process and use unnamed pipes for stdin/stdout/stderr
dovetail
Site Admin
Posts: 2022
Joined: Thu Jul 29, 2004 12:12 pm

Re: Idea: DLL for Dataset Pipes, invoking from a user progra

Post by dovetail »

I'm not sure exactly what you mean by ""more expensive" fork()-like process". If your Linux client uses pipe() or eqivalent, the forks/processes would be on Linux. With ControlMaster sessions, you will be a new OMVS job for each - is that what you are referring to?

One important thing to do if you are using SSH on z/OS is to get the latest PTFs so that you can offload cipher and mac processing to ICSF / CPACF (a co-processor card is not required). This will save more CPU than anything else I can think of.
john.mckown
Posts: 48
Joined: Tue Jun 12, 2007 2:46 pm

Re: Idea: DLL for Dataset Pipes, invoking from a user progra

Post by john.mckown »

I use fromdsn and todsn both on Linux/Intel and z/OS UNIX. I improperly intermixed them in my initial thoughts. The DLL is mainly for use in z/OS UNIX work, not Linux offloaded work.

I'll need to look at my PTF levels (I'm the main sysprog) for the OpenSSH PTF level. Does that require ICSF to be running? I've never gotten it to work. I don't seem to be able to initialize the VSAM datasets properly. I'll double check that too. We are constantly behind in maintenance because we are basically moribund on z/OS.
dovetail
Site Admin
Posts: 2022
Joined: Thu Jul 29, 2004 12:12 pm

Re: Idea: DLL for Dataset Pipes, invoking from a user progra

Post by dovetail »

Yes, ICSF needs to be running (FMID HCR7770 or later) for Ported Tools OpenSSH Cipher/Mac hardware support. You don't need a coprocessor card, but you need the (no charge) CPAC-F microcode feature installed on your system.

http://www-03.ibm.com/systems/z/os/zos/ ... index.html
.. the IBM Ported Tools page for info on the PTF (UA63842):
ICSF ciphers and MAC algorithms. OpenSSH can be set up to use Integrated Cryptographic Service Facility (ICSF) to implement certain ciphers and MAC (message authentication code) algorithms. This extension enables OpenSSH to use hardware support when applicable and is provided by way of the PTF for APAR OA37278.
Here's the APAR
http://www-01.ibm.com/support/docview.w ... sg1OA37278

Details for enabling this support are found in the latest IBM Ported Tools User's Guide.
Post Reply