Page 1 of 1
Replacing IEBGENER with ICEGENER.
Posted: Fri Feb 16, 2018 2:41 pm
by durga
Hi,
We are using SyncSort at our company. I was reading a post and it was said there that for straight copy jobs IEBGENER can be replaced with ICEGENER and it happens at system level. If we do so in our company how can I trace the CPU usage change, before and after, to see how much CPU usage difference is there? Could someone please share your idea to measure such a change.
Re: Replacing IEBGENER with ICEGENER.
Posted: Sun Feb 18, 2018 9:14 pm
by DB2 Guy
This is an interesting question. I am not sure how can we really measure such a thing apart from listing out some of the IEBGENER Jobs you've. Write a REXX or by any other means look for the CPU time from the JESMSGLG and compare it with equivalent ICEGENER Jobs and compare.
Have said that but will it be a true measure, I'm not convinced myself...
Re: Replacing IEBGENER with ICEGENER.
Posted: Sun Feb 18, 2018 10:10 pm
by Robert Sample
Contact your site support group and work with them. The best way to measure CPU usage is to look at the SMF records, and most sites do not allow application programmers to access the SMF data. The type 30 records have data about jobs and steps (cpu usage included), but they are not necessarily easy to use / summarize. Your site support group may already have jobs written to report on CPU usage by jobs / steps.
Re: Replacing IEBGENER with ICEGENER.
Posted: Wed Jul 04, 2018 8:58 am
by durga
Thanks
DB2 Guy wrote: ↑Sun Feb 18, 2018 9:14 pmThis is an interesting question. I am not sure how can we really measure such a thing apart from listing out some of the IEBGENER Jobs you've. Write a REXX or by any other means look for the CPU time from the JESMSGLG and compare it with equivalent ICEGENER Jobs and compare.
Have said that but will it be a true measure, I'm not convinced myself...
That is what I was also thinking but it would be a lot of work to get something which is still not very correct. So I thought if there can be an alternate way of doing it.
Re: Replacing IEBGENER with ICEGENER.
Posted: Wed Jul 04, 2018 9:05 am
by durga
Robert Sample wrote: ↑Sun Feb 18, 2018 10:10 pmContact your site support group and work with them. The best way to measure CPU usage is to look at the SMF records, and most sites do not allow application programmers to access the SMF data. The type 30 records have data about jobs and steps (cpu usage included), but they are not necessarily easy to use / summarize. Your site support group may already have jobs written to report on CPU usage by jobs / steps.
Hi Robert.
As SyncGener might impact all possible copy jobs then what is the best approach to measure the CPU use of all these jobs? I mean do I need to create a list of all jobs I think are impacted and work with site support with those jobs or they would know themselves that how much CPU they saved?
Re: Replacing IEBGENER with ICEGENER.
Posted: Wed Jul 04, 2018 2:32 pm
by nicc
Why not consult with them directly? They should be only too eager to help you reduce resources needed and the sooner you consult with them the sooner everyone concerned can benefit from any savings.
Re: Replacing IEBGENER with ICEGENER.
Posted: Wed Jul 04, 2018 3:57 pm
by durga
Because they are not very helpful and a lot of politics is here. The moment they'd know, they'd try to show it as their work, in case it works. If it does not work, it was my fault.
Re: Replacing IEBGENER with ICEGENER.
Posted: Thu Jul 05, 2018 9:29 pm
by Robert Sample
I just did a test. I copied 2,764,562 SMF records to a temporary data set. My JCL has two copy steps, one using ICEGENER and one using IEBGENER. We use DFSORT not Syncsort but I would not think the difference between DFSORT and Syncsort would be significant. I ran the job twice, once doing the IEBGENER first and ICEGENER second, then I swapped the two steps and ran it again. The statistics from the job summary (JES2) are:
Code: Select all
COPY EXCP TCB SRB SERVICE UNITS
ICE 1983 24.76 0.26 40,728,073
IEB 107K 24.13 2.13 100,620,564
IEB 107K 24.10 1.77 39,411,004
ICE 1978 25.28 0.49 105,648,101
In both jobs, the second step used more service units because it used VIO while the first step did not. ICEGENER used a total of 50.79 CPU seconds while IEBGENER used 52.13 CPU seconds, which is only 2.64% more CPU time. While ICEGENER did less than 4,000 EXCP and IEBGENER did 214,000 (approximately) EXCP, you did not ask about overall resource consumption -- just CPU time. On the basis of the CPU time difference my tests showed, I would say it is a very unprofitable change to convert jobs to ICEGENER from IEBGENER since someone has to find the jobs in the scheduler (or wherever they are) and update them; a 2.6% savings would be hard to justify unless you have MANY jobs sorting BILLIONS of records.
Re: Replacing IEBGENER with ICEGENER.
Posted: Thu Jul 05, 2018 10:11 pm
by durga
But this will done at system level. So all the qualified jobs will taken care by them selves and we don't need to identify the jobs manually.
Thanks for EXCP, I forgot to say that. I have also did the experiment the way you did but my records were only 1000. And with that CPU difference was not too high. I thought to gradually increase the records, which I shall do tomorrow. In my experiment, even when I coded ICEGENER or SYNCGNER, it looks that only IEBGENER was executing. I shall post the results in some time, not able to get to mainframes just now.
Re: Replacing IEBGENER with ICEGENER.
Posted: Fri Jul 06, 2018 6:22 pm
by durga
Hi,
This is the job which I have submitted:
Code: Select all
//S020 EXEC PGM=SYNCGENR
//SYSUT1 DD DSN=...TEST.FILE,
// DISP=SHR
//SYSUT2 DD SYSOUT=*
//SYSIN DD DUMMY,DCB=BLKSIZE=80
//SYSPRINT DD SYSOUT=*
//*
I have also submitted this job:
Code: Select all
//S020 EXEC PGM=ICEGENER
//SYSUT1 DD DSN=MISAD2.TEST.FILE,
// DISP=SHR
//SYSUT2 DD SYSOUT=*
//SYSIN DD DUMMY,DCB=BLKSIZE=80
//SYSPRINT DD SYSOUT=*
Both the Jobs executed successfully. And this is I saw in SYSPRINT:
Code: Select all
DATA SET UTILITY - GENERATE
IEB352I WARNING: ONE OR MORE OF THE OUTPUT DCB PARMS COPIED FROM INPUT
PROCESSING ENDED AT EOD
and
Code: Select all
DATA SET UTILITY - GENERATE
IEB352I WARNING: ONE OR MORE OF THE OUTPUT DCB PARMS COPIED FROM INPUT
PROCESSING ENDED AT EOD
What does it tell?
Re: Replacing IEBGENER with ICEGENER.
Posted: Fri Jul 06, 2018 6:24 pm
by durga
This is from JESYSMSG:
Code: Select all
STEP/S020 /STOP 2018186.0021
CPU: 0 HR 00 MIN 00.01 SEC SRB: 0 HR 00 MIN 00.00 SEC
VIRT: 420K SYS: 252K EXT: 8K SYS: 10260K
What is VTRT in this? I searched for it but could not find any info.
Re: Replacing IEBGENER with ICEGENER.
Posted: Fri Jul 06, 2018 6:34 pm
by enrico-sorichetti
read the DFSORT manuals to see what are the prerequisites ( input/output datasets attributes/specifications ) for using ICEGENER
if they are NOT satisfied ICEGENER will revert to IEBGENER
Re: Replacing IEBGENER with ICEGENER.
Posted: Fri Jul 06, 2018 6:40 pm
by Robert Sample
IEB352I WARNING: ONE OR MORE OF THE OUTPUT DCB PARMS COPIED FROM INPUT
This message means that IEBGENER (not ICEGENER -- the IEB message prefix indicates IEBGENER was used) copied (at least one of) RECFM, LRECL, or BLKSIZE for the output data set from the input data set.
VIRT: 420K SYS: 252K EXT: 8K SYS: 10260K
This line means your job step required 420K of virtual (VIRT) storage (as opposed to real memory), the system required 252K of virtual storage (all 672K came from the 24-bit address space, or below-the-line memory) while the job step needed 8K of extended (above the line) memory and the system used 10260K of above-the-line memory.
Re: Replacing IEBGENER with ICEGENER.
Posted: Fri Jul 06, 2018 8:36 pm
by durga
enrico-sorichetti wrote: ↑Fri Jul 06, 2018 6:34 pmread the DFSORT manuals to see what are the prerequisites ( input/output datasets attributes/specifications ) for using ICEGENER
if they are NOT satisfied ICEGENER will revert to IEBGENER
I have looked at this document:
http://publibz.boulder.ibm.com/cgi-bin/ ... 0528171007
And I took this example:
Following is an example of how an IEBGENER application can be changed to use ICEGENER by substituting the name ICEGENER for the name IEBGENER in the EXEC statement.
Code: Select all
//GENER JOB...
// EXEC PGM=ICEGENER
//SYSPRINT DD SYSOUT=*
//SYSUT1 DD DSN=CONTROL.MASTER,DISP=OLD,UNIT=3380,VOL=SER=MASTER
//SYSUT2 DD DSN=CONTROL.BACKUP,DISP=OLD,UNIT=3380,VOL=SER=BACKUP
//SYSIN DD DUMMY
Re: Replacing IEBGENER with ICEGENER.
Posted: Fri Jul 06, 2018 8:38 pm
by durga
I reread the document and it says this too: "The IEBGENER DD statements SYSUT1 (input), SYSUT2 (output), and SYSPRINT (messages) are used by DFSORT for SORTIN, SORTOUT, and SYSOUT, respectively. These DD statement names will be translated by using an extended parameter list to invoke the copy function. If DFSORT cannot be used (for example, because IEBGENER control statements are specified), control will be transferred to IEBGENER."
Why DFSORT can not be used? What does that mean? Does that mean that ICEGENER is actually is not enabled?
Re: Replacing IEBGENER with ICEGENER.
Posted: Fri Jul 06, 2018 10:09 pm
by Robert Sample
The quote you provided tells you one reason why SORT would not be used (emphasis added by me):
If DFSORT cannot be used (for example, because IEBGENER control statements are specified), control will be transferred to IEBGENER."
In other words, if you have //SYSIN DD * followed by a GENERATE or EXITS or RECORD statement (or any other utility control statement), that GUARANTEES you will not be using SORT but IEBGENER instead. If you have //SYSIN DD DUMMY then that is a precursor condition to having SORT be used; there may be other reasons why SORT would not be used (I haven't researched to find all the times SORT cannot be used). ICEGENER may be enabled, but if you have utility control statements in the job, then ICEGENER will not be used -- period.
And, by the way, based on my testing showing a difference of 1 CPU second per million records, you have ALREADY spent more time / money researching this than your organization is likely to save over the next TEN YEARS.
Re: Replacing IEBGENER with ICEGENER.
Posted: Mon Jul 09, 2018 12:16 pm
by durga
Thanks Robert. Actually I could not make my point clear. When I used this Job:
Code: Select all
//GENER JOB...
// EXEC PGM=ICEGENER
//SYSPRINT DD SYSOUT=*
//SYSUT1 DD DSN=CONTROL.MASTER,DISP=OLD,UNIT=3380,VOL=SER=MASTER
//SYSUT2 DD DSN=CONTROL.BACKUP,DISP=OLD,UNIT=3380,VOL=SER=BACKUP
//SYSIN DD DUMMY
I thought SuncSort will be used but it was not used, even with DUMMY in SYSIN. So can we say that we have SyncGNER but not used ever?
Re: Replacing IEBGENER with ICEGENER.
Posted: Mon Jul 09, 2018 2:23 pm
by nicc
Your post does not make sense. 'Suncsort'? 'SyncGNER'?
ICEGENER is part of the DFSort suite not SyncSort. Whereas there may be aliases set up for one to refer to other this does not always happen.
Note that IEBGENER is not part of the DFSort suite.
And why would you expect SyncSort to be called instead of ICEGENER? The whole thread has been about ICEGENER/IEBGENER.
Re: Replacing IEBGENER with ICEGENER.
Posted: Mon Jul 09, 2018 5:05 pm
by Robert Sample
If your site is using Syncsort, then you need to consult the Syncsort documentation about whether or not they have an ICEGENER equivalent, and if so how to invoke it. There may be site configuration required to use it -- I have not used Syncsort in more than three decades and hence have no idea of its capabilities.
Thread locked since you are not clear what you want, you don't know what your site is doing, and you seem unwilling to consult with your site support group.
I found this quote from a Syncsort support person:
SyncSort's BetterGener utility program offers improved performance over IEBGENER in straight copy tasks and is transparent to users when it is installed with SyncSort for z/OS. Implementing BetterGener will provide significant savings in system resources such as elapsed time, CPU time, and channel utilization through improved I/O handling. Once BetterGener is activated, eligible IEBGENER job streams are automatically processed by SyncSort. There is no need to replace working IEBGENER job streams with explicit requests for SyncSort’s more efficient copying. BetterGener’s SYNCGENR module will intercept calls to IEBGENER and determine whether the particular copy application should be processed by SyncSort. In situations supported by SyncSort, SYNCGENR will link to the sort, generating the environment of a program-initiated sort execution. In situations not supported by SyncSort, SYNCGENR will transfer control to IEBGENER.
So if you are not getting BetterGener (ICEGENER is the name of an IBM product and hence should never be used with Syncsort's product) when you run IEBGENER, either your site does not have it installed or your copy steps do not qualify for BetterGener execution. If it is not installed, ONLY YOUR SITE SUPPORT GROUP COULD INSTALL IT. Since you are not interested in getting them involved, there is nowhere else for this thread to go.