Query on UTF-8

Discussion of the Co:Z Toolkit Dataset Pipes utilities
Post Reply
marguccio
Posts: 18
Joined: Fri Feb 14, 2014 9:09 am

Query on UTF-8

Post by marguccio »

Hello there,
using fromdsn, we need to write the content of a dataset on z/OS (coded in UTF-8) to a Linux box.
Do we need to specify the source and target codepages using the options -s and -t on the fromdsn command ?

Kind regards,
marguccio
Posts: 18
Joined: Fri Feb 14, 2014 9:09 am

Re: Query on UTF-8

Post by marguccio »

Hello there,
let me elaborate my previous query ...
the UTF-8 dataset on zOS will be copied to a Linux box using 'fromdsn'. Once there,
a java class will produce a .xlsx file, the latter will then be ZIPed. The compressed file
is returned back to z/OS on its Unix System Services file system using 'tofile'.
We have been using the binary option on both commands, but the file on USS looks corrupted
and cannot be opened.
Are we using the correct option on both fromdsn/tofile or are we making something wrong ?

Kind regards,
dovetail
Site Admin
Posts: 2022
Joined: Thu Jul 29, 2004 12:12 pm

Re: Query on UTF-8

Post by dovetail »

If the data set is already UTF-8, then binary transfer (-b option) should work, but that will not cause record boundaries to be converted into newline characters.

For that, you would need "text" mode processing, but you can specify that the source and target codepages are identical. This will have the effect of disabling code-page translation, but the end-of-line character in the target codepage will still be used for record boundaries.

For example:

Code: Select all

fromdsn -s UTF-8 -t UTF-8 //HLQ.MY.XML  > /tmp/test.xml 
I suggest that you try this and verify that the output is as expected. Look at the source dataset and target file in hex if you need to. The only difference should be:

- ASCII/UTF-8 new-line (x0a) characters should be inserted at records boundaries
- ASCII/UTF-8 space (x20) characters should be trimmed at the end of each record IFF the dataset has fixed-length records.

BTW: if you are using Co:Z Launcher, you can use bash process substitution "<(command...)" like:

Code: Select all

//STEP1 EXEC PROC=COZPROC,ARGS='myid@mylinux.myco.com'
//XMLIN DD DISP=SHR,DSN=...   (UTF-8 encoding)
//ZIP     DD DISP=(NEW,CATLG,DELETE),DSN=....,
//               SPACE=(CYL,(10,10),RLSE),DCB=(RECFM=U,BLKSIZE=27998)
//XSLT  DD  DISP=SHR,DSN=....  (UTF-8 encoding)
//STDIN DD *

# probably need to first:  export CLASSPATH= 
java org.apache.xalan.xslt.Process \
    -IN <( fromdsn -s UTF-8 -t UTF-8 //DD:XMLIN )  \
    -XSL <( fromdsn -s UTF-8 -t UTF-8 //DD:XSLT ) \
    -OUT >( gzip | todsn -b //DD:ZIP)
//
This has the benefit of doing the transfer and processing in parallel, with no data at rest.
marguccio
Posts: 18
Joined: Fri Feb 14, 2014 9:09 am

Re: Query on UTF-8

Post by marguccio »

Hello dovetail (Steve?),
thanks for your reply. We found out why the file was corrupted: we wrongly used 'todsn'
(not 'tofile' as I posted) to store data from the Linux box to the z/OS USS file system.
To summarize, fromdsn and tofile, both with -b option, does accomplish what
we wanted to do.
However, I noticed the following:

//RUNCOZ EXEC PROC=COZPROC,ARGS='mylinux§mydomain
//STDIN DD *
# Linux --> zOS
tofile -b /tmp/zOSfile.zip < /tmp/Linuxfile.zip

works, rc=0, file is sored correctly.

//RUNCOZ EXEC PROC=COZPROC,ARGS='mylinux§mydomain
//OUTPUT DD PATH='/tmp/zOSfile.zip',
// PATHOPTS=(OWRONLY,OCREAT),
// PATHMODE=(SIRWXU,SIRWXG,SIRWXO)
//STDIN DD *
# Linux --> zOS
tofile -b dd:output < /tmp/Linuxfile.zip

does NOT work, rc=0, but file is empty. Anything wrong in PATHOPTS or PATHMODE ?
They look good to me.

Kind regards,
dovetail
Site Admin
Posts: 2022
Joined: Thu Jul 29, 2004 12:12 pm

Re: Query on UTF-8

Post by dovetail »

the tofile command does not support DDs, only z/OS Unix path names.
marguccio
Posts: 18
Joined: Fri Feb 14, 2014 9:09 am

Re: Query on UTF-8

Post by marguccio »

ouch! ... you are damn right.

Thanks again very much.
Sumer
Posts: 1
Joined: Wed Mar 11, 2015 2:01 am

Re: Query on UTF-8

Post by Sumer »

does NOT work, rc=0, but file is empty. Anything wrong in PATHOPTS or PATHMODE ?
Our Actual Exams and Testking how to prepare for mcat gre exam provide you 100% pass guarantee. You can get access to and exams. Our Testking juniper book is also very useful tool. olivet.edu for its easiness. Colorado Technical University
dovetail
Site Admin
Posts: 2022
Joined: Thu Jul 29, 2004 12:12 pm

Re: Query on UTF-8

Post by dovetail »

Please provide some context by posting sample job input for what you are trying to do.
Post Reply