Page 1 of 1

How to open a big xml file?

Posted: Sun Nov 08, 2015 1:57 am
by demecarv
I tried to see the logs from certain Jython command aimed to deploy my application to Websphere 8.5. Unfortunatelly, I can't read it via "z/OS UNIX Directory List" since when I typed ea (edit/ASCII mode) on left side of my file I get the message on right top of my screen: "Record too large". In order to workaround it, I copied it to other file named trace.xml and I tried to convert it in order to use oedit. Unfortunatelly, when I tried iconv I got a very strange message (strange for a new Unix user) and an empty outcome file created. I guess I am doing nothing wrong with iconv as this command works for small files? I rembenber that I have read somewhere that OS/390 has some issue known by IBM related to open a big file. Well, I don't consider 15 Megabytes a big xml file.
Would the file size be the issue in both moment (UdList and iconv)? If so, since I just want to read the file, is there a workaround maybe by spliting it in multiple smaller files?
Could a special character (Ý) be blocking me to open the file and informing the odd message "Record too large". If so, how can I open such file?

Code: Select all

$ iconv -f ISO8859-1 IBM-1047 trace.xml > traceCon.xml
Usage: iconv Ý-cs¨ Ý-M|-T¨ -f oldset -t newset Ýfile ...¨
       iconv Ý-cs¨ Ý-M|-T¨ -F Ý-f oldset¨ -t newset Ýfile ...¨
       iconv -l Ý-v¨

-rwxrwxrwx   1 MyUser1  SYSPROG  14195459 Nov  7 13:25 trace.xml
-rw-r--r--       1 MyUSer1  SYSPROG              0 Nov  7 13:46 traceCon.xml

Re: How to open a big xml file?

Posted: Mon Nov 09, 2015 4:06 am
by Robert Sample
I think you're confusing record size and file size. ISPF EDIT has a limit on how long a record can be handled (32760 is the longest a standard DASD record can be), and oedit is a record-based editor (just like ISPF EDIT). I'm not sure if the 32760 limit applies to oedit or not, but there is some limit to how long the records it can edit are. "Record too long" indicates that one (or more) records were longer than what the editor supports -- and may have NOTHING to do with the 15 million byte size of the file (then again, all 15 million bytes could be a single record). Special characters would not cause "Record too large" errors.

Unix System Services has a split command which could be used, or if your site has Perl installed you could use it to split your input file into pieces.

The error message you got is the syntax for the iconv command. Since your command does not include the -t between the oldest and the newest, the syntax diagram was given to help remind you of how to code up your command. The empty output file is because redirection will create the file even if nothing is copied into it.

Re: How to open a big xml file?

Posted: Mon Nov 09, 2015 8:04 pm
by demecarv
Robert, thanks.
After I added -t between the two formats, my file got converted. Then I tried oedit but I got the message "...a record length greater than 32750...". So, I used wc -l to see how many lines. I got 92080. So I did split command and now, after I got the file splitted in many smaller files (xaa, xab, xac... xdo) I can odit each one. Now I can move forward and read the log files.