Odd problem writing to multiple PDS members

orr94 · Post by **orr94** » Fri Mar 10, 2006 3:18 pm

I am using JZOS 1.2.2 and Java 1.4.2 on a z/OS 1.4 system. We aren't using the JZOS Batch Launcher for this particular problem; we are running from either WebSphere 5.1 or from a UNIX prompt. Hopefully this problem won't require an upgrade to JZOS 1.2.3, because we are on a tight schedule.

This is a very odd problem, so I'll try my best to describe it. We are using ZFile (and the stream returned by ZFile.getOutputStream()) to write data out to members of a PDS. We open each member with the following format:

Code: Select all

PrintWriter currentOut = new PrintWriter(new OutputStreamWriter(
 new ZFile("//'SYS4.MYPDS($AMEMBER)'", "wt").getOutputStream(), "CP1047"));

Our program gets through 33 members without any problems; however, on the 34th, the data we write out is repeated dozens of times until the PDS runs out of extents, then the program attempts to continue to a 35th member, and fails with the following error message:

Code: Select all

com.dovetail.jzos.ZFileException: //'SYS4.MYPDS($AMEMBER)': fopen failed - EDC5045I The operation attempted could not
be performed because the file was open. 
ERRNO=45 ERRNO2=0x594003d LAST_OP=43 ABEND_CODE=0x562 ABEND_RC=98

The problem doesn't appear to be related to the specific member, because when we write in a different order, it still fails on the 34th member. If we write fewer members, we don't have the problem.

As a test, I replaced the ZFile OutputStream with a StringWriter, then sent the StringWriter to stdout. This worked correctly, indicating that the problem is related to my use of ZFile.

I had no success opening the files in different modes: "w" failed in the same manner, "wb" gets through, but does not apply the line breaks, "wb,type=record" is just a mess (hard to describe, but I'll elaborate if needed).

We used to use JRIO for this, and it wasn't a problem. However, when we upgraded from WAS 3.5 (which ran in EBCDIC) to WAS 5.1 (which runs in ASCII), we needed to switch to JZOS's ZFile so that we could specify the codepage in which to write.

I know this is probably a confusing description of my problem, but it is a very confusing problem! Any insight anyone has would be greatly appreciated!

Post by **dovetail** » Sun Mar 12, 2006 12:02 pm

I'm at a loss... my best guess is that there is something going wrong in the C library routines.

- How are you closing the ZFile?
- Have you tried opening the file using a DD name? "//DD:MYDD($AMEMBER)"
- Do you get the same results if you open the file with:
BufferedWriter w =
FileFactory.newBufferedWriter("//'SYS4.MYPDS($AMEMBER)'",
"Cp1047");
- Can you send us a self-contained sample program that will reproduce the problem?
- I'm curious as to what is going on with "wb,type=record".... of course you couldn't use a PrintStream on it....

orr94 · Post by **orr94** » Sun Mar 12, 2006 7:02 pm

I'm not at work right now, but I'll answer the questions that I can:

- Closing ZFile - I close the PrintWriter, which I assume closes all the underlying output streams. If this is not the case, then that could clearly be the problem.

- Open using DD name - we aren't running this program in batch mode; it's actually run from a servlet (WebSphere 5.1), or from the command line. And the member names are determined at run-time, so we really couldn't run it in batch mode with DD's for each member.

- Using FileFactory - I can try that on Monday when I'm at work, if you think it might help.

- Sample program - I can try to put a self-contained program together on Monday that will reproduce the problem.

- wb,type=record - why couldn't you use a PrintStream (or in my case, PrintWriter) in that case? At any rate, when I try that method, I only get through 1 member, which only has 4 lines of garbage text. If you could tell me why PrintWriter wouldn't work, maybe I'd understand why I'm getting that result.

I appreciate your help... I'll try what you suggested on Monday. In the mean time, if you have any suggestions based on my response, I'd love to hear them!

orr94 · Post by **orr94** » Mon Mar 13, 2006 10:25 am

An update on my progress:

- Using FileFactory - I tried this, and I got the same result as before

- Sample program - I haven't been able to reproduce the problem with a self-contained program yet. As best I can tell, I'm pretty much doing the same thing in my sample program... I'll keep trying, but if you have any ideas in the mean time, I'd love to hear them!

Guest · Post by **Guest** » Mon Mar 13, 2006 11:30 am

The open fail is probably just a result of your initial problem. What value do you have for MAXFILEPROC on this system, and do you have a value set for FILEPROCMAX in the OMVS segment of the ID under which this is running?

orr94 · Post by **orr94** » Mon Mar 13, 2006 12:54 pm

Agree that the opening problem is related to the initial problem. MAXFILEPROC for both the system and my OMVS segment is set to 10000.

I'm beginning to wonder if the issue is related to the output buffers. Does the OutputStream returned by ZFile do any internal buffering? I have this suspicion because I was explicitly calling flush() in some areas of my code; when I removed the flush() commands, those particular sections stopped repeating. However, I am no longer calling flush(), and repeating is still occuring on a larger scale. In fact, it starts repeating in the middle of a line, which makes me wonder if the buffer is filling at that point, a flush() is called (somewhere), and then the repeating occurs. Am I on to something?

Also, I would like to repeat that this problem does NOT occur when I substitute a StringWriter for ZFile.getOutputStream(), which is what made me wonder if ZFile's OutputStream was doing any internal buffering.

Post by **dovetail** » Mon Mar 13, 2006 1:42 pm

No, the OutputStream returned by ZFile does no buffering. But, the C-library routines probably do.

So, ZFileOutputStream.write() just calls ZFile.write(), which just calls C - fwrite()

ZFile outputStream.flush() just calls ZFile.flush(), which just calls C - fflush().

Btw - all the Java source for JZOS is included in jzos_src.jar in the alphaworks distribution.

You can trace the calls to the C library functions called by ZFile if you do this:

ZUtil.setLoggingLevel(ZUtil.LOG_TRACE);

This might help figure out what is going wrong.

Also, you should check to see if you are getting any Write-to-Programmer messages in the joblog relating to IO abends on the dataset.

Finally, you can't use mode=wb,type=record with a OutputStream, since when in record mode each call to write is a new record.

orr94 · Post by **orr94** » Mon Mar 13, 2006 2:14 pm

There are no relevant messages (that I can find) in the joblogs. However, I did find something interesting when I used ZUtil.setLoggingLevel(ZUtil.LOG_TRACE). Here's the STDERR output (modified to hide actual names, of course):

Code: Select all

01:59:31 ZTOOLS(T): -> ~ZFileImpl()
01:59:31 ZTOOLS(T): <- ~ZFileImpl()
01:59:31 ZTOOLS(T):  -> ZFileImpl(//'SYS4.MYPDS($MEM1)', w)
01:59:31 ZTOOLS(T): <- ZFileImpl()
01:59:31 ZTOOLS(T): -> open()
01:59:31 ZTOOLS(T): -> initialize()
01:59:31 ZTOOLS(T): recfm=FB
01:59:31 ZTOOLS(T): <- initialize()
01:59:31 ZTOOLS(T): name=//'SYS4.MYPDS($MEM1)', mode=w, recordCount=0, fileLength=0
01:59:31 ZTOOLS(T): <- open()
01:59:31 ZTOOLS(T):  -> write(2335DBD8, 8192, 0, 8192)
01:59:31 ZTOOLS(T): name=//'SYS4.MYPDS($MEM1)', mode=w, recordCount=1, fileLength=8192
01:59:31 ZTOOLS(T): bytes written=8192
01:59:31 ZTOOLS(T): <- write()
01:59:31 ZTOOLS(T):  -> write(2335DBD8, 8192, 0, 8192)
01:59:31 ZTOOLS(T): name=//'SYS4.MYPDS($MEM1)', mode=w, recordCount=2, fileLength=16384
01:59:31 ZTOOLS(T): bytes written=8192
01:59:31 ZTOOLS(T): <- write()
01:59:31 ZTOOLS(T):  -> write(2335DBD8, 8192, 0, 1014)
01:59:31 ZTOOLS(T): name=//'SYS4.MYPDS($MEM1', mode=w, recordCount=3, fileLength=17398
01:59:31 ZTOOLS(T): bytes written=1014
01:59:31 ZTOOLS(T): <- write()
01:59:31 ZTOOLS(T): -> close()
01:59:31 ZTOOLS(T): <- close()
01:59:31 ZTOOLS(T): -> ~ZFileImpl()
01:59:31 ZTOOLS(T): <- ~ZFileImpl()
01:59:31 ZTOOLS(T):  -> ZFileImpl(//'SYS4.MYPDS($MEM2)', w)
01:59:31 ZTOOLS(T): <- ZFileImpl()
01:59:31 ZTOOLS(T): -> open()
01:59:31 ZTOOLS(T): -> initialize()
01:59:31 ZTOOLS(T): recfm=FB
01:59:31 ZTOOLS(T): <- initialize()
01:59:31 ZTOOLS(T): name=//'SYS4.MYPDS($MEM2)', mode=w, recordCount=0, fileLength=0
01:59:31 ZTOOLS(T): <- open()
01:59:31 ZTOOLS(T):  -> write(23418098, 8192, 0, 8192)
01:59:31 ZTOOLS(T): name=//'SYS4.MYPDS($MEM2)', mode=w, recordCount=1, fileLength=8192
01:59:31 ZTOOLS(T): bytes written=8192
01:59:31 ZTOOLS(T): <- write()
01:59:31 ZTOOLS(T):  -> write(23418098, 8192, 0, 8192)
01:59:31 ZTOOLS(T): name=//'SYS4.MYPDS($MEM2)', mode=w, recordCount=2, fileLength=16384
01:59:31 ZTOOLS(T): bytes written=8192
01:59:31 ZTOOLS(T): <- write()
01:59:31 ZTOOLS(T):  -> write(23418098, 8192, 0, 929)
01:59:31 ZTOOLS(T): name=//'SYS4.MYPDS($MEM2)', mode=w, recordCount=3, fileLength=17313
01:59:31 ZTOOLS(T): bytes written=929
01:59:31 ZTOOLS(T): <- write()
01:59:31 ZTOOLS(T): -> close()
01:59:31 ZTOOLS(T): <- close()
01:59:31 ZTOOLS(T): -> ~ZFileImpl()
01:59:31 ZTOOLS(T): <- ~ZFileImpl()
01:59:31 ZTOOLS(T):  -> ZFileImpl(//'SYS4.MYPDS($MEM3)', w)
01:59:31 ZTOOLS(T): <- ZFileImpl()
01:59:31 ZTOOLS(T): -> open()
01:59:31 ZTOOLS(T): -> initialize()
01:59:31 ZTOOLS(T): recfm=FB
01:59:31 ZTOOLS(T): <- initialize()
01:59:31 ZTOOLS(T): name=//'SYS4.MYPDS($MEM3)', mode=w, recordCount=0, fileLength=0
01:59:31 ZTOOLS(T): <- open()
01:59:31 ZTOOLS(T):  -> write(234C7178, 8192, 0, 8192)
01:59:32 ZTOOLS(T): <- write()
01:59:32 ZTOOLS(T):  -> write(234C7178, 8192, 0, 8192)
01:59:32 ZTOOLS(T): <- write()
01:59:32 ZTOOLS(T):  -> write(234C7178, 8192, 0, 8192)
01:59:32 ZTOOLS(T): <- write()
01:59:32 ZTOOLS(T):  -> write(234C7178, 8192, 0, 8192)
01:59:32 ZTOOLS(T): <- write()
01:59:32 ZTOOLS(T):  -> write(234C7178, 8192, 0, 8192)
01:59:32 ZTOOLS(T): <- write()
01:59:32 ZTOOLS(T):  -> write(234C7178, 8192, 0, 8192)
01:59:32 ZTOOLS(T): <- write()
01:59:32 ZTOOLS(T):  -> write(234C7178, 8192, 0, 8192)
01:59:32 ZTOOLS(T): <- write()
01:59:32 ZTOOLS(T):  -> write(234C7178, 8192, 0, 8192)
01:59:32 ZTOOLS(T): <- write()
01:59:32 ZTOOLS(T):  -> write(234C7178, 8192, 0, 8192)
01:59:32 ZTOOLS(T): <- write()
01:59:32 ZTOOLS(T):  -> write(234C7178, 8192, 0, 8192)
01:59:32 ZTOOLS(T): <- write()

(and so on for many pages....)

01:59:32 ZTOOLS(T):  -> write(234C7178, 8192, 0, 8192)
01:59:32 ZTOOLS(T): <- write()
01:59:32 ZTOOLS(T):  -> write(234C7178, 8192, 0, 8192)
01:59:32 ZTOOLS(T): <- write()
01:59:52 ZTOOLS(T):  -> ZFileImpl(//'SYS4.MYPDS($MEM4)', w)
01:59:52 ZTOOLS(T): <- ZFileImpl()
01:59:52 ZTOOLS(T): -> open()
01:59:52 ZTOOLS(T): <- open()
01:59:52 ZTOOLS(T): -> ~ZFileImpl()
01:59:52 ZTOOLS(T): <- ~ZFileImpl()
01:59:52 ZTOOLS(T):  -> write(234C7178, 8192, 0, 8192)
01:59:52 ZTOOLS(T): <- write()
com.dovetail.jzos.ZFileException: //'SYS4.MYPDS($MEM4)': fopen failed -
  EDC5045I The operation attempted could not be performed because the
  file was open. ERRNO=45 ERRNO2=0x594003d LAST_OP=604 
  ABEND_CODE=0x562 ABEND_RC=98
	at com.dovetail.jzos.ZFile.fopen(Native Method)
	at com.dovetail.jzos.ZFile.<init>(ZFile.java:243)

Notice that on the members that worked, after each write() statement, there were two lines showing the record count/file size and the bytes written. On the member that fails, however, those lines do not occur. Also, notice that the $MEM3 stream is never closed before continuing on to MEM4, which would explain the exception on fopen.

I don't know what this all means, since I can't see the native code (or at least I don't know where to look for it), but this certainly seems odd...

Post by **dovetail** » Mon Mar 13, 2006 2:42 pm

Ok, I think I know what is happening....

I assume that you are using java.io.PrintStream somewhere.
This class is really crap, since what it does when it gets an exception on writing is to retry the write and silently eat the exception. So, there is a buffer in whatever you use to wrap the PrintStream which is being written a second time when it gets an exception. If you look at the code in PrintStream, you'll see that it retries writing the buffer once, and then it just gives up and tosses the data away.

So, you are probably getting an exception from the underlying C -library.

If you use a BufferedWriter (wrapped on a OutputStreamWriter wrapped on the ZFile OutputStream) you will get the IOException which will tell you what is wrong.

As mentioned before, the easy way to do this is to use FileFactory.getBufferedWriter("//'PDS(MEM)'");

One possibility for the exception is that you are writing a line that is larger than the LRECL allows, which would throw a ZFileException (a subclass of IOException).

orr94 · Post by **orr94** » Mon Mar 13, 2006 2:51 pm

I'm actually using the PrintWriter, but the effect is the same; it eats its exceptions just like PrintStream.

I'll try using just the BufferedWriter (without wrapping it in a PrintWriter), though it'll take some time to convert the code, since I've been using println all over the place. I'll let you know what I come up with...

Thanks again for the help!

orr94 · Post by **orr94** » Tue Mar 14, 2006 8:42 am

Problem solved... I replaced the PrintWriter(s) with BufferedWriter(s), and sure enough, an IOException was being thrown. One of the lines I was printing out was longer than the LRECL, as you suspected. What's odd is that the line that was too long was one of the first ones printed out, and it continued to print out dozens of lines past that before it started repeating the whole block of text. I would expect that as soon as it tried to print out the line that was too long, the exception would be thrown and it would stop printing out new text (though I understand the loop when the PrintWriter was being used).

At any rate, I've got it working now, and I have sworn off all use of PrintWriters and PrintStreams! Thanks a lot for your help!