Replacing IEBGENER with ICEGENER.

SyncSort's Sort Product, SyncTool for z/OS, SYNCINIT and SYNCLIST.
Locked
durga
Registered Member
Posts: 53
Joined: Mon Jul 01, 2013 3:28 pm

Replacing IEBGENER with ICEGENER.

Post by durga »

Hi,

We are using SyncSort at our company. I was reading a post and it was said there that for straight copy jobs IEBGENER can be replaced with ICEGENER and it happens at system level. If we do so in our company how can I trace the CPU usage change, before and after, to see how much CPU usage difference is there? Could someone please share your idea to measure such a change.
User avatar
DB2 Guy
Forum Moderator
Forum Moderator
Posts: 119
Joined: Sun Apr 21, 2013 8:25 pm
India

Re: Replacing IEBGENER with ICEGENER.

Post by DB2 Guy »

This is an interesting question. I am not sure how can we really measure such a thing apart from listing out some of the IEBGENER Jobs you've. Write a REXX or by any other means look for the CPU time from the JESMSGLG and compare it with equivalent ICEGENER Jobs and compare.

Have said that but will it be a true measure, I'm not convinced myself... :?
User avatar
Robert Sample
Global Moderator
Global Moderator
Posts: 1886
Joined: Fri Jun 28, 2013 1:22 am
Location: Dubuque Iowa
United States of America

Re: Replacing IEBGENER with ICEGENER.

Post by Robert Sample »

Contact your site support group and work with them. The best way to measure CPU usage is to look at the SMF records, and most sites do not allow application programmers to access the SMF data. The type 30 records have data about jobs and steps (cpu usage included), but they are not necessarily easy to use / summarize. Your site support group may already have jobs written to report on CPU usage by jobs / steps.
durga
Registered Member
Posts: 53
Joined: Mon Jul 01, 2013 3:28 pm

Re: Replacing IEBGENER with ICEGENER.

Post by durga »

Thanks
DB2 Guy wrote: Sun Feb 18, 2018 9:14 pmThis is an interesting question. I am not sure how can we really measure such a thing apart from listing out some of the IEBGENER Jobs you've. Write a REXX or by any other means look for the CPU time from the JESMSGLG and compare it with equivalent ICEGENER Jobs and compare.

Have said that but will it be a true measure, I'm not convinced myself...
That is what I was also thinking but it would be a lot of work to get something which is still not very correct. So I thought if there can be an alternate way of doing it.
durga
Registered Member
Posts: 53
Joined: Mon Jul 01, 2013 3:28 pm

Re: Replacing IEBGENER with ICEGENER.

Post by durga »

Robert Sample wrote: Sun Feb 18, 2018 10:10 pmContact your site support group and work with them. The best way to measure CPU usage is to look at the SMF records, and most sites do not allow application programmers to access the SMF data. The type 30 records have data about jobs and steps (cpu usage included), but they are not necessarily easy to use / summarize. Your site support group may already have jobs written to report on CPU usage by jobs / steps.
Hi Robert.

As SyncGener might impact all possible copy jobs then what is the best approach to measure the CPU use of all these jobs? I mean do I need to create a list of all jobs I think are impacted and work with site support with those jobs or they would know themselves that how much CPU they saved?
nicc
Global Moderator
Global Moderator
Posts: 691
Joined: Wed Apr 23, 2014 8:45 pm

Re: Replacing IEBGENER with ICEGENER.

Post by nicc »

Why not consult with them directly? They should be only too eager to help you reduce resources needed and the sooner you consult with them the sooner everyone concerned can benefit from any savings.
Regards
Nic
durga
Registered Member
Posts: 53
Joined: Mon Jul 01, 2013 3:28 pm

Re: Replacing IEBGENER with ICEGENER.

Post by durga »

Because they are not very helpful and a lot of politics is here. The moment they'd know, they'd try to show it as their work, in case it works. If it does not work, it was my fault.
User avatar
Robert Sample
Global Moderator
Global Moderator
Posts: 1886
Joined: Fri Jun 28, 2013 1:22 am
Location: Dubuque Iowa
United States of America

Re: Replacing IEBGENER with ICEGENER.

Post by Robert Sample »

I just did a test. I copied 2,764,562 SMF records to a temporary data set. My JCL has two copy steps, one using ICEGENER and one using IEBGENER. We use DFSORT not Syncsort but I would not think the difference between DFSORT and Syncsort would be significant. I ran the job twice, once doing the IEBGENER first and ICEGENER second, then I swapped the two steps and ran it again. The statistics from the job summary (JES2) are:

Code: Select all

COPY EXCP     TCB    SRB   SERVICE UNITS
ICE  1983   24.76   0.26     40,728,073
IEB  107K   24.13   2.13    100,620,564

IEB  107K   24.10   1.77     39,411,004
ICE  1978   25.28   0.49    105,648,101 
In both jobs, the second step used more service units because it used VIO while the first step did not. ICEGENER used a total of 50.79 CPU seconds while IEBGENER used 52.13 CPU seconds, which is only 2.64% more CPU time. While ICEGENER did less than 4,000 EXCP and IEBGENER did 214,000 (approximately) EXCP, you did not ask about overall resource consumption -- just CPU time. On the basis of the CPU time difference my tests showed, I would say it is a very unprofitable change to convert jobs to ICEGENER from IEBGENER since someone has to find the jobs in the scheduler (or wherever they are) and update them; a 2.6% savings would be hard to justify unless you have MANY jobs sorting BILLIONS of records.
durga
Registered Member
Posts: 53
Joined: Mon Jul 01, 2013 3:28 pm

Re: Replacing IEBGENER with ICEGENER.

Post by durga »

But this will done at system level. So all the qualified jobs will taken care by them selves and we don't need to identify the jobs manually.

Thanks for EXCP, I forgot to say that. I have also did the experiment the way you did but my records were only 1000. And with that CPU difference was not too high. I thought to gradually increase the records, which I shall do tomorrow. In my experiment, even when I coded ICEGENER or SYNCGNER, it looks that only IEBGENER was executing. I shall post the results in some time, not able to get to mainframes just now.
durga
Registered Member
Posts: 53
Joined: Mon Jul 01, 2013 3:28 pm

Re: Replacing IEBGENER with ICEGENER.

Post by durga »

Hi,

This is the job which I have submitted:

Code: Select all

//S020    EXEC PGM=SYNCGENR          
//SYSUT1   DD  DSN=...TEST.FILE, 
//             DISP=SHR              
//SYSUT2   DD  SYSOUT=*              
//SYSIN    DD  DUMMY,DCB=BLKSIZE=80  
//SYSPRINT DD  SYSOUT=*              
//*                                  
I have also submitted this job:

Code: Select all

//S020    EXEC PGM=ICEGENER         
//SYSUT1   DD  DSN=MISAD2.TEST.FILE,
//             DISP=SHR             
//SYSUT2   DD  SYSOUT=*             
//SYSIN    DD  DUMMY,DCB=BLKSIZE=80 
//SYSPRINT DD  SYSOUT=*             
Both the Jobs executed successfully. And this is I saw in SYSPRINT:

Code: Select all

DATA SET UTILITY - GENERATE                                             
IEB352I WARNING: ONE OR MORE OF THE OUTPUT DCB PARMS COPIED FROM INPUT  
                                                                        
PROCESSING ENDED AT EOD                                                 
and

Code: Select all

DATA SET UTILITY - GENERATE                                           
IEB352I WARNING: ONE OR MORE OF THE OUTPUT DCB PARMS COPIED FROM INPUT
                                                                      
PROCESSING ENDED AT EOD                                               
What does it tell?
durga
Registered Member
Posts: 53
Joined: Mon Jul 01, 2013 3:28 pm

Re: Replacing IEBGENER with ICEGENER.

Post by durga »

This is from JESYSMSG:

Code: Select all

STEP/S020    /STOP  2018186.0021                                    
CPU:     0 HR  00 MIN  00.01 SEC    SRB:     0 HR  00 MIN  00.00 SEC
VIRT:   420K  SYS:   252K  EXT:        8K  SYS:    10260K           
What is VTRT in this? I searched for it but could not find any info.
enrico-sorichetti
Global Moderator
Global Moderator
Posts: 825
Joined: Wed Sep 11, 2013 3:57 pm

Re: Replacing IEBGENER with ICEGENER.

Post by enrico-sorichetti »

read the DFSORT manuals to see what are the prerequisites ( input/output datasets attributes/specifications ) for using ICEGENER
if they are NOT satisfied ICEGENER will revert to IEBGENER
cheers
enrico
When I tell somebody to RTFM or STFW I usually have the page open in another tab/window of my browser,
so that I am sure that the information requested can be reached with a very small effort 8-)
User avatar
Robert Sample
Global Moderator
Global Moderator
Posts: 1886
Joined: Fri Jun 28, 2013 1:22 am
Location: Dubuque Iowa
United States of America

Re: Replacing IEBGENER with ICEGENER.

Post by Robert Sample »

IEB352I WARNING: ONE OR MORE OF THE OUTPUT DCB PARMS COPIED FROM INPUT
This message means that IEBGENER (not ICEGENER -- the IEB message prefix indicates IEBGENER was used) copied (at least one of) RECFM, LRECL, or BLKSIZE for the output data set from the input data set.
VIRT: 420K SYS: 252K EXT: 8K SYS: 10260K
This line means your job step required 420K of virtual (VIRT) storage (as opposed to real memory), the system required 252K of virtual storage (all 672K came from the 24-bit address space, or below-the-line memory) while the job step needed 8K of extended (above the line) memory and the system used 10260K of above-the-line memory.
durga
Registered Member
Posts: 53
Joined: Mon Jul 01, 2013 3:28 pm

Re: Replacing IEBGENER with ICEGENER.

Post by durga »

enrico-sorichetti wrote: Fri Jul 06, 2018 6:34 pmread the DFSORT manuals to see what are the prerequisites ( input/output datasets attributes/specifications ) for using ICEGENER
if they are NOT satisfied ICEGENER will revert to IEBGENER
I have looked at this document: http://publibz.boulder.ibm.com/cgi-bin/ ... 0528171007

And I took this example:
Following is an example of how an IEBGENER application can be changed to use ICEGENER by substituting the name ICEGENER for the name IEBGENER in the EXEC statement.

Code: Select all

//GENER JOB...
// EXEC PGM=ICEGENER
//SYSPRINT DD SYSOUT=*
//SYSUT1 DD DSN=CONTROL.MASTER,DISP=OLD,UNIT=3380,VOL=SER=MASTER
//SYSUT2 DD DSN=CONTROL.BACKUP,DISP=OLD,UNIT=3380,VOL=SER=BACKUP
//SYSIN DD DUMMY
durga
Registered Member
Posts: 53
Joined: Mon Jul 01, 2013 3:28 pm

Re: Replacing IEBGENER with ICEGENER.

Post by durga »

I reread the document and it says this too: "The IEBGENER DD statements SYSUT1 (input), SYSUT2 (output), and SYSPRINT (messages) are used by DFSORT for SORTIN, SORTOUT, and SYSOUT, respectively. These DD statement names will be translated by using an extended parameter list to invoke the copy function. If DFSORT cannot be used (for example, because IEBGENER control statements are specified), control will be transferred to IEBGENER."

Why DFSORT can not be used? What does that mean? Does that mean that ICEGENER is actually is not enabled?
User avatar
Robert Sample
Global Moderator
Global Moderator
Posts: 1886
Joined: Fri Jun 28, 2013 1:22 am
Location: Dubuque Iowa
United States of America

Re: Replacing IEBGENER with ICEGENER.

Post by Robert Sample »

The quote you provided tells you one reason why SORT would not be used (emphasis added by me):
If DFSORT cannot be used (for example, because IEBGENER control statements are specified), control will be transferred to IEBGENER."
In other words, if you have //SYSIN DD * followed by a GENERATE or EXITS or RECORD statement (or any other utility control statement), that GUARANTEES you will not be using SORT but IEBGENER instead. If you have //SYSIN DD DUMMY then that is a precursor condition to having SORT be used; there may be other reasons why SORT would not be used (I haven't researched to find all the times SORT cannot be used). ICEGENER may be enabled, but if you have utility control statements in the job, then ICEGENER will not be used -- period.

And, by the way, based on my testing showing a difference of 1 CPU second per million records, you have ALREADY spent more time / money researching this than your organization is likely to save over the next TEN YEARS.
durga
Registered Member
Posts: 53
Joined: Mon Jul 01, 2013 3:28 pm

Re: Replacing IEBGENER with ICEGENER.

Post by durga »

Thanks Robert. Actually I could not make my point clear. When I used this Job:

Code: Select all

//GENER JOB...
// EXEC PGM=ICEGENER
//SYSPRINT DD SYSOUT=*
//SYSUT1 DD DSN=CONTROL.MASTER,DISP=OLD,UNIT=3380,VOL=SER=MASTER
//SYSUT2 DD DSN=CONTROL.BACKUP,DISP=OLD,UNIT=3380,VOL=SER=BACKUP
//SYSIN DD DUMMY
I thought SuncSort will be used but it was not used, even with DUMMY in SYSIN. So can we say that we have SyncGNER but not used ever?
nicc
Global Moderator
Global Moderator
Posts: 691
Joined: Wed Apr 23, 2014 8:45 pm

Re: Replacing IEBGENER with ICEGENER.

Post by nicc »

Your post does not make sense. 'Suncsort'? 'SyncGNER'?
ICEGENER is part of the DFSort suite not SyncSort. Whereas there may be aliases set up for one to refer to other this does not always happen.
Note that IEBGENER is not part of the DFSort suite.

And why would you expect SyncSort to be called instead of ICEGENER? The whole thread has been about ICEGENER/IEBGENER.
Regards
Nic
User avatar
Robert Sample
Global Moderator
Global Moderator
Posts: 1886
Joined: Fri Jun 28, 2013 1:22 am
Location: Dubuque Iowa
United States of America

Re: Replacing IEBGENER with ICEGENER.

Post by Robert Sample »

If your site is using Syncsort, then you need to consult the Syncsort documentation about whether or not they have an ICEGENER equivalent, and if so how to invoke it. There may be site configuration required to use it -- I have not used Syncsort in more than three decades and hence have no idea of its capabilities.

Thread locked since you are not clear what you want, you don't know what your site is doing, and you seem unwilling to consult with your site support group.

I found this quote from a Syncsort support person:
SyncSort's BetterGener utility program offers improved performance over IEBGENER in straight copy tasks and is transparent to users when it is installed with SyncSort for z/OS. Implementing BetterGener will provide significant savings in system resources such as elapsed time, CPU time, and channel utilization through improved I/O handling. Once BetterGener is activated, eligible IEBGENER job streams are automatically processed by SyncSort. There is no need to replace working IEBGENER job streams with explicit requests for SyncSort’s more efficient copying. BetterGener’s SYNCGENR module will intercept calls to IEBGENER and determine whether the particular copy application should be processed by SyncSort. In situations supported by SyncSort, SYNCGENR will link to the sort, generating the environment of a program-initiated sort execution. In situations not supported by SyncSort, SYNCGENR will transfer control to IEBGENER.
So if you are not getting BetterGener (ICEGENER is the name of an IBM product and hence should never be used with Syncsort's product) when you run IEBGENER, either your site does not have it installed or your copy steps do not qualify for BetterGener execution. If it is not installed, ONLY YOUR SITE SUPPORT GROUP COULD INSTALL IT. Since you are not interested in getting them involved, there is nowhere else for this thread to go.
Locked

Return to “SyncSort, SyncTool, SyncGener.”