Remove Duplicates and get output unsorted.

[Pragya]

Hi,

I have a file and I need to remove duplicates records from it.When the duplicates are removed, I need an output file that is unsorted, the order of input records should not change.

Is this possible using SORT?

[William Collins]

Do you only want duplicates which are consecutive removed, or if record number one is the same as the last record on the file, you want the last record removed? Either can be done, but the coding would be different, so need to know. Can you show representative sample input, and required output, please? Also let us know the RECFM and LRECL.

[Pragya]

I know it's very late to reply on this - but can i still follow up?

[William Collins]

Yes.

[Pragya]

Sample input is, RECFM=FB, LRECL=80:

Code: Select all

MOHANK 123456789012345 RAJES 
ANTAKS 123456789012345 MIRAJ 
MOHANK 123456789012345 NANAK 

[William Collins]

We need all the questions answered, and sample input and expected output which cover the all the conditions. In the sameple you have shown, there are no duplicate records even,

[zprogrammer]

For the given input

Code: Select all

MOHANK 123456789012345 RAJES 
ANTAKS 123456789012345 MIRAJ 
MOHANK 123456789012345 NANAK

Do you need output like

Code: Select all

MOHANK 123456789012345 RAJES 
ANTAKS 123456789012345 MIRAJ

or

Code: Select all

ANTAKS 123456789012345 MIRAJ 
MOHANK 123456789012345 NANAK

[Pragya]

The first one is I need, Pandora Box.

[zprogrammer]

You could achieve this by DFSORT

1.Step 1 => Add sequence number and sort to eliminate duplicates
2.Step 2 => Sort the sequence numbers and OUTREC the needed data