Find the number of records in a Tape File without reading it.

[Vidya Kumari]

Hi,

We have couple of tape files, some of which are archieved and some of them are not. For an audit requirement, we need to see the number of records on these files. Can you please suggest a simple method of finding the number of records. We have two programs written in COBOL which can read the files and can get the number of records but with these tape files are having records around 16-18 million records each and cobol program as such takes long time to read it and generate a report on counts.

We also though of using SORT but with sortout as dummy it does not really help with the execution time. Can there be a way to count just records with less time used.

Thanks

[Robert Sample]

You have a couple of options:
1. Find out which application writes the tape file and check the record counts for the program doing the writes. If the program doesn't write record counts, or you cannot access the application, then see option 2.
2. Read the entire tape file and count the records. This can be done with your SORT product or a custom-written program. If you use a custom-written program, make sure you provide enough buffers (and memory for the buffers) -- typically 50 to 250 would be adequate -- for reading the tape.

cobol program as such takes long time to read it and generate a report on counts.

Statements like this are worthless to post. What is a "long time" to you may not be a long time to someone else. If you cannot quantify a value, don't put it in your post as it provides no benefit to people reading the post. Furthermore, are you aware that elapsed time for jobs on a z/OS system is HEAVILY dependent upon a great number of factors? These factors include number of address spaces, Workload Manager policy, CPU utilization of the LPAR, channel contention, etc. It is entirely possible that your job could run in half the time if the job class was changed (just as one example) -- but only someone working at your site, such as your site support group, could tell you for sure.

If your tape management system tells you the number of blocks of data, you can estimate the record count by dividing the block size by the record length to get records per block and then multiplying by the number of blocks. Whether or not an auditor would accept such an estimate, however, depends upon your auditors.

[Vidya Kumari]

First of all thanks for your reply.

1. Find out which application writes the tape file and check the record counts for the program doing the writes. If the program doesn't write record counts, or you cannot access the application, then see option 2.

There are so many applications which write these tape files.There is not one application we are looking at. So I shall go for option 2.

2. Read the entire tape file and count the records. This can be done with your SORT product or a custom-written program. If you use a custom-written program, make sure you provide enough buffers (and memory for the buffers) -- typically 50 to 250 would be adequate -- for reading the tape.

Thanks but reading all the records is the overkill of the process. As the files are huge having 20 million records each. It some times takes more than 6 hours to just read all the records.

How do you decide on the bufferes? Is there some formula for that?

[Vidya Kumari]

cobol program as such takes long time to read it and generate a report on counts.

Statements like this are worthless to post. What is a "long time" to you may not be a long time to someone else. If you cannot quantify a value, don't put it in your post as it provides no benefit to people reading the post. Furthermore, are you aware that elapsed time for jobs on a z/OS system is HEAVILY dependent upon a great number of factors? These factors include number of address spaces, Workload Manager policy, CPU utilization of the LPAR, channel contention, etc. It is entirely possible that your job could run in half the time if the job class was changed (just as one example) -- but only someone working at your site, such as your site support group, could tell you for sure.

The time goes from 3 hours to 6 hours. Mainly dependent on the number of records. I shall check on job class, thanks.

If your tape management system tells you the number of blocks of data, you can estimate the record count by dividing the block size by the record length to get records per block and then multiplying by the number of blocks. Whether or not an auditor would accept such an estimate, however, depends upon your auditors.

I have not explored this option, will check on it.

[Anuj Dhawan]

2. Read the entire tape file and count the records. This can be done with your SORT product or a custom-written program. If you use a custom-written program, make sure you provide enough buffers (and memory for the buffers) -- typically 50 to 250 would be adequate -- for reading the tape.

Thanks but reading all the records is the overkill of the process. As the files are huge having 20 million records each. It some times takes more than 6 hours to just read all the records.

Tape files are sequential-read, I'm not sure without reading the records, one way or other - how are you going to count them?

[Robert Sample]

As the files are huge having 20 million records each.

For your learning, a tape file with 20 million records is NOT "huge". In fact, I would classify it as small to medium size.

As far as taking 3 to 6 hours to read a 20-million-record data set, that is HOGWASH! Either there is more going on than you are telling us, or your read JCL is horribly inefficient (such as you accepting the default QSAM buffer allocation), or your job does not have high priority in Workload Manager. For comparison, two weeks ago I needed to read 12 weekly SMF tapes with 2.8 million to 4 million records per tape; the total was 40 million records or so. Including mounting the tapes, reading the SMF data, un-mounting the tapes -- the total elapsed job time was less than 30 minutes. If your job is taking 3 to 6 hours for 20 million records, you need to work with your site support group to find out what is slowing it down as you should be able to read at least 100 million to 500 million (or more) records in 3 to 6 hours.