Page 1 of 1

I can't open a file with large record even after I used iconv

Posted: Wed Jan 13, 2016 1:53 am
by demecarv
Around two months ago I created here a question (Why I can read a file via UDLIST but I can't via cat command) because I was getting the message "...ISRP521 ... record with length grater than 32750... not supported by ISPF". Then, someone answered my question informing to use iconv -f IBM-1047 -t ISO8859-1 ebcdic_file_name >ascii_file_name. It fixed my issue in that case. Now, I am facing same issue and, even after iconv I can't read the file. My first guess was that I could be using the wrong "from" format. Then I tried the unix command "file myfile.xml" to see what was the file format. I get the message "magic file /etc/magic: EDC5129I No such file or directory then I assume that the mainframe I am suing doesn't have such command. Now I am stuck. My real target is to open the file. Any workaround will be very appreciated. An alternative to figure out what is file format will be appreciated as well.

Re: I can't open a file with large record even after I used iconv

Posted: Thu Jan 14, 2016 7:04 pm
by demecarv
I am very interested on this question. Any clue will be appreciated. For instance, a suggestion on how to figure out which is the file format may help me.

Re: I can't open a file with large record even after I used iconv

Posted: Thu Jan 14, 2016 7:07 pm
by enrico-sorichetti
An alternative to figure out what is file format will be appreciated as well.
how do You expect us to tell ? :twisted:

Re: I can't open a file with large record even after I used iconv

Posted: Thu Jan 14, 2016 8:55 pm
by demecarv
Enrico-sorichetti, if there is no way to figure out which is the file format in Unix, just tell me this. Based on my research, file command would tell me this but as I pasted the print I get this message "magic file /etc/magic: EDC5129I No such file or directory". By the way, whenever I had trouble to open a file getting the message "Record too large", iconv command has helped. In this it didn`t help me at all. So, I am looking for alternatives and my first guess was that I am using the wrong "from format". Supposing you face the scenario where you want to openedit a file in a mainframe with oedit command and you get the message "Record too large", what would you try? Just tell me this.

Re: I can't open a file with large record even after I used iconv

Posted: Thu Jan 14, 2016 9:19 pm
by Robert Sample
Earlier, you got lucky. iconv will help with a large record IF AND ONLY IF the file code page is incorrect. Using iconv converts an ASCII file to an EBCDIC file and "works" to fix OEDIT large record issues because the line terminators are not recognized in ASCII and converting them to EBCDIC allows them to be recognized. So the file was treated as one long record before the iconv and as individual records after.

In no case will iconv help you if the actual record length is more than 32767 bytes -- such a file cannot be opened in OEDIT, period. OEDIT uses the ISPF editor and there is a limit of 32767 bytes per record with it. You could use a native Unix editor such as ed (or vi) to edit the file; neither of these editors has a user interface anywhere near the ISPF user interface so you will need to research the commands BEFORE you invoke them against your file.

If you just want to look at the data, you can use the cat Unix command to list the file.

Oh, and people on this forum -- like other forums -- respond if / when they have time as participation is entirely voluntary. Prompting for responses after just a few hours (or days) will NOT get you faster nor better responses, and often causes people who would have responded to decide not to respond so they won't be harassed.
Supposing you face the scenario where you want to openedit a file in a mainframe with oedit command and you get the message "Record too large", what would you try?
I'd use vi to look at it, or maybe ed. In order to use the file command on a z/OS Unix system, someone has to define the /etc/magic file -- and it appears this has not been done on your site.

Re: I can't open a file with large record even after I used iconv

Posted: Thu Jan 14, 2016 9:41 pm
by enrico-sorichetti
why not ask the application developers about it?
they should know.

Re: I can't open a file with large record even after I used iconv

Posted: Tue Jan 19, 2016 9:33 pm
by demecarv
I tried vi, ed and cat and none of them worked for me. Any other suggestion will be appreciated.

-rwxrwxrwx 1 ABC9999 XYZ 1486 Jan 19 09:24 users.xml
$ vi users.xml
FSUM9140 Terminal "dumb" has insufficient capabilities for Curses.

$ ed users.xml
1486
,p
--Ì_%-ÎÁÊËÑ?>-------Á>Ä?ÀÑ>Å--ñë|------------Ä-ÍËÁÊË-Ì_%>Ë-Ä--ÑÄÈÈ-----Ä-ÍËÁÊ-ÑÀ
--á--------_/Ñ%--ÀÁ_ÁÈÊÑ?¬Ä/ÊÎ/%Ç? _/ËÈÁÊÄ/ÊÀ-Ä?_--øÊÁÎ-%?Å?>------------------ø

...
w
1487
q

$ cat users.xml
--Ì_%-ÎÁÊËÑ?>-------Á>Ä?ÀÑ>Å--ñë|------------Ä-ÍËÁÊË-Ì_%>Ë-Ä--ÑÄÈÈ-----Ä-ÍËÁÊ-ÑÀ

Re: I can't open a file with large record even after I used iconv

Posted: Tue Jan 19, 2016 10:05 pm
by Robert Sample
I copied some of your data to my mainframe, converted it from ISO8859-1 to IBM-1047 and it became readable EBCDIC data. So your FIRST issue is to convert the file to EBCDIC.

However, the first few bytes indicate the file is XML. An XML file probably has one record that is longer than 32767 characters and hence can NEVER be opened by OEDIT. Since you don't normally need to edit the data before processing the XML, go ahead and process it without editing it. Note, however, that XML has an encoding as part of the specification and doing an iconv may very well render the file unusable as XML since the encoding is being changed without changing the XML specification. It is possible to handle UTF-8 or ASCII data on a z/OS system but you have to know plenty about z/OS and EBCDIC and XML to get the processing to work correctly.

It appears that the file was transferred to the mainframe as a binary transfer. This is incorrect for XML. XML does not support binary data in general (XML characters must all be valid for the code page) and hence should be transferred to a z/OS system as text. It is possible to use binary in an XML document, but then the processing changes from pure XML to an extension (which will be based upon the platform being used and hence will NOT be portable across systems).