Migrating mainframe code to more modern platforms is substantially faster when done by East Agile. We use our AI expertise to transform these processes.

For example, we can convert Syncsort jobs to CoSort jobs rapidly and automatically. This is something even IRI, maker of CoSort, can not do.  

But why even do that? Moving out of a mainframe environment offers an opportunity to modernize your infrastructure. At this point you could consider more modern environments, such as using Python which could transform your hiring challenges and capabilities.  Using Polars is one option for achieving CoSort or Syncsort like sorting performance in a modern Python environment using tools such as Polars. I will provide an example of that below.

Here is an example of a Syncsort script and its equivalent in IRI CoSort.

Key syntax differences applied:

Note: CoSort/SortCL syntax can vary by version. You may need to adjust based on your specific IRI CoSort version. The exit module syntax and some advanced features like SPANINC may require additional configuration in your environment.

Consider this sample Syncsort script that demonstrates multiple advanced features in a realistic business scenario. The script demonstrates a monthly transaction reconciliation job with these features:

Data Integration: Merges POS and online transaction files, joins with customer master data for enrichment.

Processing Logic: Multi-condition INCLUDE/OMIT filtering, derived field calculations (date manipulation, arithmetic, conditional assignments), and primary/secondary sort keys.

Six Output Files:

1. Detailed transaction report with formatted headers/trailers
2. Region/store summary with section breaks and subtotals
3. High-value transaction exceptions (>$100K)
4. Payment type breakdown with conditional labels
5. Rejected records with error code translation
6. JSON-formatted statistics for downstream systems

Enterprise Features: Symbol substitution for runtime parameters, JOIN operations, SUM aggregation, SECTIONS for control breaks, EDIT masks for numeric formatting, user exit modules (E15/E35), and performance options.

Here is the Syncsort version:

*-----------------------------------------------------------------------*
*  SYNCSORT SCRIPT: MONTHLY_TRANSACTION_RECONCILIATION                  *
*  PURPOSE: Process daily transaction files, validate, aggregate,       *
*           and generate multiple output reports for reconciliation     *
*  AUTHOR:  Data Operations Team                                        *
*  VERSION: 2.4.1                                                       *
*-----------------------------------------------------------------------*

*-----------------------------------------------------------------------*
*  SYMBOL DEFINITIONS                                                   *
*-----------------------------------------------------------------------*
OPTION SYMBOLS
DEFINE SYM_PROCESS_DATE='20250115'
DEFINE SYM_REGION_FILTER='US'
DEFINE SYM_MIN_AMOUNT=1000
DEFINE SYM_MAX_AMOUNT=9999999999

*-----------------------------------------------------------------------*
*  INPUT FILE DEFINITIONS - MULTIPLE SOURCES                            *
*-----------------------------------------------------------------------*
INFILE TRANSIN1 
    FILE='/data/daily/transactions_pos_&SYM_PROCESS_DATE..dat'
    FORMAT=FB
    LRECL=500
    
INFILE TRANSIN2
    FILE='/data/daily/transactions_online_&SYM_PROCESS_DATE..dat'
    FORMAT=FB
    LRECL=500

INFILE CUSTMAST
    FILE='/data/master/customer_master.dat'
    FORMAT=FB
    LRECL=200

INFILE PRODREF
    FILE='/data/reference/product_codes.dat'
    FORMAT=FB
    LRECL=100

*-----------------------------------------------------------------------*
*  FIELD DEFINITIONS - TRANSACTION RECORD LAYOUT                        *
*-----------------------------------------------------------------------*
FIELDS TRANS_LAYOUT
    TRANS_ID:           1,16,CH
    TRANS_DATE:         17,8,CH
    TRANS_TIME:         25,6,CH
    CUST_ID:            31,12,CH
    CUST_TYPE:          43,2,CH
    PRODUCT_CODE:       45,10,CH
    PRODUCT_CAT:        55,4,CH
    QUANTITY:           59,7,ZD
    UNIT_PRICE:         66,11,PD
    GROSS_AMOUNT:       77,13,PD
    DISCOUNT_PCT:       90,5,PD
    DISCOUNT_AMT:       95,11,PD
    NET_AMOUNT:         106,13,PD
    TAX_RATE:           119,5,PD
    TAX_AMOUNT:         124,11,PD
    TOTAL_AMOUNT:       135,15,PD
    PAYMENT_TYPE:       150,2,CH
    CARD_TYPE:          152,4,CH
    AUTH_CODE:          156,12,CH
    TERMINAL_ID:        168,8,CH
    STORE_ID:           176,6,CH
    REGION_CODE:        182,4,CH
    COUNTRY_CODE:       186,2,CH
    CURRENCY_CODE:      188,3,CH
    FX_RATE:            191,9,PD
    BASE_CURRENCY_AMT:  200,15,PD
    STATUS_CODE:        215,2,CH
    ERROR_CODE:         217,4,CH
    BATCH_ID:           221,10,CH
    SOURCE_SYSTEM:      231,3,CH
    CREATED_TS:         234,26,CH
    FILLER:             260,241,CH

*-----------------------------------------------------------------------*
*  FIELD DEFINITIONS - CUSTOMER MASTER LAYOUT                           *
*-----------------------------------------------------------------------*
FIELDS CUST_LAYOUT
    CM_CUST_ID:         1,12,CH
    CM_CUST_NAME:       13,50,CH
    CM_CUST_TYPE:       63,2,CH
    CM_CREDIT_LIMIT:    65,13,PD
    CM_CURRENT_BAL:     78,13,PD
    CM_STATUS:          91,1,CH
    CM_REGION:          92,4,CH
    CM_TIER:            96,2,CH
    CM_ACCT_MGR:        98,8,CH
    CM_OPEN_DATE:       106,8,CH

*-----------------------------------------------------------------------*
*  MERGE TRANSACTION FILES AND APPLY INITIAL FILTERS                    *
*-----------------------------------------------------------------------*
MERGE FIELDS=(TRANS_DATE,TRANS_TIME,TRANS_ID)
    FROM(TRANSIN1,TRANSIN2)

*-----------------------------------------------------------------------*
*  VALIDATION AND FILTERING RULES                                       *
*-----------------------------------------------------------------------*
INCLUDE COND=(
    TRANS_DATE,GE,C'20250101',AND,
    TRANS_DATE,LE,C'&SYM_PROCESS_DATE',AND,
    COUNTRY_CODE,EQ,C'&SYM_REGION_FILTER',AND,
    TOTAL_AMOUNT,GE,&SYM_MIN_AMOUNT,AND,
    TOTAL_AMOUNT,LE,&SYM_MAX_AMOUNT,AND,
    STATUS_CODE,EQ,C'AP',AND,
    (PAYMENT_TYPE,EQ,C'CC',OR,
     PAYMENT_TYPE,EQ,C'DC',OR,
     PAYMENT_TYPE,EQ,C'EF'),AND,
    CUST_ID,NE,C'            '
)

OMIT COND=(
    ERROR_CODE,NE,C'0000',OR,
    TOTAL_AMOUNT,EQ,0,OR,
    CUST_TYPE,EQ,C'XX'
)

*-----------------------------------------------------------------------*
*  JOIN WITH CUSTOMER MASTER FOR ENRICHMENT                             *
*-----------------------------------------------------------------------*
JOIN UNPAIRED,F1,F2
    FILE=F1,INCLUDE=(TRANSIN1,TRANSIN2)
    FILE=F2,INCLUDE=(CUSTMAST)
    JOINKEYS FILE=F1,FIELDS=(CUST_ID,31,12,A)
    JOINKEYS FILE=F2,FIELDS=(CM_CUST_ID,1,12,A)

REFORMAT FIELDS=(
    F1:1,500,
    F2:CM_CUST_NAME,50,
    F2:CM_CREDIT_LIMIT,13,
    F2:CM_TIER,2,
    F2:CM_STATUS,1
)

*-----------------------------------------------------------------------*
*  DERIVED FIELD CALCULATIONS                                           *
*-----------------------------------------------------------------------*
INREC BUILD=(
    1,500,
    501:TRANS_DATE,Y4T,TOGREG=Y4T,ADDDAYS=+0,TOGREG=(-),TOJUL=4D,
    505:GROSS_AMOUNT,DIV,QUANTITY,TO=PD,LENGTH=11,
    516:NET_AMOUNT,MUL,+100,DIV,GROSS_AMOUNT,TO=PD,LENGTH=5,
    521:TOTAL_AMOUNT,SUB,NET_AMOUNT,TO=PD,LENGTH=13,
    534:SEQNUM,8,ZD,
    542:(CUST_TYPE,EQ,C'PR'),
        IFTHEN=(WHEN=(NET_AMOUNT,GT,+50000),
                BUILD=(1,541,C'HV')),
        IFTHEN=(WHEN=NONE,BUILD=(1,541,C'NV')),
    544:(PRODUCT_CAT,SS,C'ELEC'),
        IFTHEN=(WHEN=COND,BUILD=(1,543,C'Y')),
        IFTHEN=(WHEN=NONE,BUILD=(1,543,C'N'))
)

*-----------------------------------------------------------------------*
*  PRIMARY SORT KEYS                                                    *
*-----------------------------------------------------------------------*
SORT FIELDS=(
    REGION_CODE,4,CH,A,
    STORE_ID,6,CH,A,
    TRANS_DATE,8,CH,A,
    TOTAL_AMOUNT,15,PD,D
)

*-----------------------------------------------------------------------*
*  SUMMARIZATION FOR AGGREGATES                                         *
*-----------------------------------------------------------------------*
SUM FIELDS=(
    QUANTITY,
    GROSS_AMOUNT,
    DISCOUNT_AMT,
    NET_AMOUNT,
    TAX_AMOUNT,
    TOTAL_AMOUNT,
    BASE_CURRENCY_AMT
)

*-----------------------------------------------------------------------*
*  OUTPUT FILE 1: DETAILED TRANSACTION REPORT                           *
*-----------------------------------------------------------------------*
OUTFIL FNAMES=DETAILED
    FILE='/data/output/trans_detail_&SYM_PROCESS_DATE..dat'
    FORMAT=FB
    LRECL=600
    BUILD=(
        C'DTL|',
        TRANS_ID,
        C'|',
        TRANS_DATE,4,4,C'-',TRANS_DATE,6,2,C'-',TRANS_DATE,1,2,
        C'|',
        CUST_ID,
        C'|',
        501:CM_CUST_NAME,SQUEEZE,
        C'|',
        PRODUCT_CODE,
        C'|',
        QUANTITY,EDIT=(TTTTTT9),
        C'|',
        GROSS_AMOUNT,EDIT=($$$$,$$$,$$9.99),
        C'|',
        DISCOUNT_AMT,EDIT=($$$$,$$$,$$9.99),
        C'|',
        NET_AMOUNT,EDIT=($$$$,$$$,$$9.99),
        C'|',
        TAX_AMOUNT,EDIT=($$$,$$$,$$9.99),
        C'|',
        TOTAL_AMOUNT,EDIT=($$$$$,$$$,$$9.99),
        C'|',
        PAYMENT_TYPE,
        C'|',
        STORE_ID,
        C'|',
        REGION_CODE
    )
    HEADER1=(C'TRANSACTION DETAIL REPORT',5X,
             C'GENERATED: ',DATE=(4MD/),1X,TIME=(12:),
             80:C'PAGE: ',PAGE=(EDIT=(TTTTT9)))
    HEADER2=(130C'-')
    TRAILER1=(C'*** END OF REPORT ***',5X,
              C'TOTAL RECORDS: ',COUNT=(EDIT=(TTTTTTTTT9)))

*-----------------------------------------------------------------------*
*  OUTPUT FILE 2: SUMMARY BY REGION AND STORE                           *
*-----------------------------------------------------------------------*
OUTFIL FNAMES=SUMMARY
    FILE='/data/output/trans_summary_&SYM_PROCESS_DATE..dat'
    FORMAT=FB
    LRECL=200
    REMOVECC
    SECTIONS=(REGION_CODE,
        HEADER3=(C'REGION: ',REGION_CODE,/,60C'='),
        TRAILER3=(C'  REGION TOTAL: ',
                  TOT=(TOTAL_AMOUNT,EDIT=($B,$$$,$$$,$$$,$$9.99)),
                  C'  TRANS COUNT: ',
                  CNT=(EDIT=(TTTTTTTT9)),/)
    )
    BUILD=(
        C'  ',
        STORE_ID,
        C' | ',
        TRANS_DATE,
        C' | ',
        COUNT=(EDIT=(TTTTT9)),
        C' | ',
        TOTAL_AMOUNT,EDIT=($$$,$$$,$$$,$$9.99),
        C' | ',
        BASE_CURRENCY_AMT,EDIT=($$$,$$$,$$$,$$9.99)
    )
    TRAILER2=(
        /,C'GRAND TOTALS',/,60C'=',/,
        C'TOTAL TRANSACTIONS: ',CNT=(EDIT=(TTTTTTTTTT9)),/,
        C'TOTAL GROSS AMOUNT: ',TOT=(GROSS_AMOUNT,EDIT=($B,$$$,$$$,$$$,$$$,$$9.99)),/,
        C'TOTAL DISCOUNTS:    ',TOT=(DISCOUNT_AMT,EDIT=($B,$$$,$$$,$$$,$$$,$$9.99)),/,
        C'TOTAL NET AMOUNT:   ',TOT=(NET_AMOUNT,EDIT=($B,$$$,$$$,$$$,$$$,$$9.99)),/,
        C'TOTAL TAX:          ',TOT=(TAX_AMOUNT,EDIT=($B,$$$,$$$,$$$,$$$,$$9.99)),/,
        C'GRAND TOTAL:        ',TOT=(TOTAL_AMOUNT,EDIT=($B,$$$,$$$,$$$,$$$,$$9.99))
    )

*-----------------------------------------------------------------------*
*  OUTPUT FILE 3: EXCEPTION RECORDS (HIGH VALUE)                        *
*-----------------------------------------------------------------------*
OUTFIL FNAMES=HIGHVAL
    FILE='/data/output/trans_highvalue_&SYM_PROCESS_DATE..dat'
    FORMAT=FB
    LRECL=300
    INCLUDE=(TOTAL_AMOUNT,GT,+100000)
    BUILD=(
        C'HIGH_VALUE_ALERT|',
        TRANS_ID,C'|',
        TRANS_DATE,C'|',
        CUST_ID,C'|',
        CM_CUST_NAME,SQUEEZE,C'|',
        CM_TIER,C'|',
        TOTAL_AMOUNT,EDIT=($$$$$,$$$,$$$,$$9.99),C'|',
        CM_CREDIT_LIMIT,EDIT=($$$$$,$$$,$$$,$$9.99),C'|',
        STORE_ID,C'|',
        AUTH_CODE,C'|',
        CREATED_TS
    )

*-----------------------------------------------------------------------*
*  OUTPUT FILE 4: PAYMENT TYPE BREAKDOWN                                *
*-----------------------------------------------------------------------*
OUTFIL FNAMES=PAYTYPE
    FILE='/data/output/trans_by_payment_&SYM_PROCESS_DATE..dat'
    FORMAT=FB
    LRECL=150
    SECTIONS=(PAYMENT_TYPE,
        HEADER3=(C'PAYMENT METHOD: ',PAYMENT_TYPE,
                 IFTHEN=(WHEN=(PAYMENT_TYPE,EQ,C'CC'),BUILD=(C' (CREDIT CARD)')),
                 IFTHEN=(WHEN=(PAYMENT_TYPE,EQ,C'DC'),BUILD=(C' (DEBIT CARD)')),
                 IFTHEN=(WHEN=(PAYMENT_TYPE,EQ,C'EF'),BUILD=(C' (ELECTRONIC FUNDS)')),
                 /,40C'-'),
        TRAILER3=(C'SUBTOTAL: ',TOT=(TOTAL_AMOUNT,EDIT=($$$,$$$,$$$,$$9.99)),
                  C'  COUNT: ',CNT=(EDIT=(TTTTTT9)),/)
    )
    BUILD=(
        C'  ',CARD_TYPE,C' | ',
        COUNT=(EDIT=(TTTT9)),C' | ',
        TOTAL_AMOUNT,EDIT=($$$,$$$,$$$,$$9.99)
    )

*-----------------------------------------------------------------------*
*  OUTPUT FILE 5: REJECTED RECORDS FOR REPROCESSING                     *
*-----------------------------------------------------------------------*
OUTFIL FNAMES=REJECTS
    FILE='/data/output/trans_rejects_&SYM_PROCESS_DATE..dat'
    FORMAT=FB
    LRECL=550
    SAVE
    BUILD=(
        1,500,
        501:C'REJ',
        504:ERROR_CODE,
        508:IFTHEN=(WHEN=(ERROR_CODE,EQ,C'E001'),BUILD=(C'INVALID_CUSTOMER')),
            IFTHEN=(WHEN=(ERROR_CODE,EQ,C'E002'),BUILD=(C'AMOUNT_MISMATCH')),
            IFTHEN=(WHEN=(ERROR_CODE,EQ,C'E003'),BUILD=(C'AUTH_FAILED')),
            IFTHEN=(WHEN=(ERROR_CODE,EQ,C'E004'),BUILD=(C'DUPLICATE_TRANS')),
            IFTHEN=(WHEN=NONE,BUILD=(C'UNKNOWN_ERROR')),
        524:DATE=(4MD-),
        534:TIME=(12:)
    )

*-----------------------------------------------------------------------*
*  OUTPUT FILE 6: STATISTICAL ANALYSIS (VLSHRT FORMAT)                  *
*-----------------------------------------------------------------------*
OUTFIL FNAMES=STATS
    FILE='/data/output/trans_statistics_&SYM_PROCESS_DATE..dat'
    FORMAT=VB
    LRECL=400
    VLTRIM=C' '
    BUILD=(
        C'{"report_date":"&SYM_PROCESS_DATE",',
        C'"region":"',REGION_CODE,C'",',
        C'"store":"',STORE_ID,C'",',
        C'"trans_count":',COUNT=(EDIT=(TTTTTTTTTT9)),C',',
        C'"total_gross":',GROSS_AMOUNT,EDIT=(TTTTTTTTTTTT9.99),C',',
        C'"total_discount":',DISCOUNT_AMT,EDIT=(TTTTTTTTTTTT9.99),C',',
        C'"total_net":',NET_AMOUNT,EDIT=(TTTTTTTTTTTT9.99),C',',
        C'"total_tax":',TAX_AMOUNT,EDIT=(TTTTTTTTTTTT9.99),C',',
        C'"total_amount":',TOTAL_AMOUNT,EDIT=(TTTTTTTTTTTT9.99),C',',
        C'"avg_trans_value":',TOTAL_AMOUNT,DIV,COUNT,EDIT=(TTTTTTTT9.99),C',',
        C'"currency":"',CURRENCY_CODE,C'"}'
    )

*-----------------------------------------------------------------------*
*  PROCESSING OPTIONS                                                   *
*-----------------------------------------------------------------------*
OPTION COPY
OPTION VLSHRT
OPTION SPANINC=RC4
OPTION MAINSIZE=MAX
OPTION DYNALLOC=(SYSDA,5)
OPTION SORTDD=SYSDA
OPTION FILSZ=E500000
OPTION MSGPRT=ALL
OPTION LIST
OPTION VERIFY

*-----------------------------------------------------------------------*
*  ERROR HANDLING AND RETURN CODES                                      *
*-----------------------------------------------------------------------*
MODS E15=(VALIDEXIT,4096,EXITMOD,N),
     E35=(ENRICHEXIT,8192,EXITMOD,N)

RECORD TYPE=F,LENGTH=500

*-----------------------------------------------------------------------*
*  END OF SYNCSORT CONTROL STATEMENTS                                   *
*-----------------------------------------------------------------------*

Here is the Cosort version:

/*********************************************************************/
/*  COSORT SCRIPT: MONTHLY_TRANSACTION_RECONCILIATION                */
/*  PURPOSE: Process daily transaction files, validate, aggregate,   */
/*           and generate multiple output reports for reconciliation */
/*  VERSION: 2.4.1 (Converted from Syncsort)                         */
/*********************************************************************/

/*********************************************************************/
/*  SYMBOL DEFINITIONS                                               */
/*********************************************************************/
/SYMBOL NAME=SYM_PROCESS_DATE VALUE="20250115"
/SYMBOL NAME=SYM_REGION_FILTER VALUE="US"
/SYMBOL NAME=SYM_MIN_AMOUNT VALUE=1000
/SYMBOL NAME=SYM_MAX_AMOUNT VALUE=9999999999

/*********************************************************************/
/*  INPUT FILE 1: POS TRANSACTIONS                                   */
/*********************************************************************/
/INFILE=/data/daily/transactions_pos_${SYM_PROCESS_DATE}.dat
    /PROCESS=MERGE
    /RECORD_LENGTH=500
    /FIXED

/*********************************************************************/
/*  INPUT FILE 2: ONLINE TRANSACTIONS                                */
/*********************************************************************/
/INFILE=/data/daily/transactions_online_${SYM_PROCESS_DATE}.dat
    /PROCESS=MERGE
    /RECORD_LENGTH=500
    /FIXED

/*********************************************************************/
/*  INPUT FILE 3: CUSTOMER MASTER                                    */
/*********************************************************************/
/INFILE=/data/master/customer_master.dat NAME=CUSTMAST
    /PROCESS=JOIN
    /RECORD_LENGTH=200
    /FIXED

/*********************************************************************/
/*  INPUT FILE 4: PRODUCT REFERENCE                                  */
/*********************************************************************/
/INFILE=/data/reference/product_codes.dat NAME=PRODREF
    /RECORD_LENGTH=100
    /FIXED

/*********************************************************************/
/*  FIELD DEFINITIONS - TRANSACTION RECORD LAYOUT                    */
/*********************************************************************/
/FIELD=(TRANS_ID, POSITION=1, LENGTH=16, TYPE=ASCII)
/FIELD=(TRANS_DATE, POSITION=17, LENGTH=8, TYPE=ASCII)
/FIELD=(TRANS_TIME, POSITION=25, LENGTH=6, TYPE=ASCII)
/FIELD=(CUST_ID, POSITION=31, LENGTH=12, TYPE=ASCII)
/FIELD=(CUST_TYPE, POSITION=43, LENGTH=2, TYPE=ASCII)
/FIELD=(PRODUCT_CODE, POSITION=45, LENGTH=10, TYPE=ASCII)
/FIELD=(PRODUCT_CAT, POSITION=55, LENGTH=4, TYPE=ASCII)
/FIELD=(QUANTITY, POSITION=59, LENGTH=7, TYPE=ZONED_DECIMAL)
/FIELD=(UNIT_PRICE, POSITION=66, LENGTH=11, TYPE=PACKED_DECIMAL, PRECISION=2)
/FIELD=(GROSS_AMOUNT, POSITION=77, LENGTH=13, TYPE=PACKED_DECIMAL, PRECISION=2)
/FIELD=(DISCOUNT_PCT, POSITION=90, LENGTH=5, TYPE=PACKED_DECIMAL, PRECISION=2)
/FIELD=(DISCOUNT_AMT, POSITION=95, LENGTH=11, TYPE=PACKED_DECIMAL, PRECISION=2)
/FIELD=(NET_AMOUNT, POSITION=106, LENGTH=13, TYPE=PACKED_DECIMAL, PRECISION=2)
/FIELD=(TAX_RATE, POSITION=119, LENGTH=5, TYPE=PACKED_DECIMAL, PRECISION=2)
/FIELD=(TAX_AMOUNT, POSITION=124, LENGTH=11, TYPE=PACKED_DECIMAL, PRECISION=2)
/FIELD=(TOTAL_AMOUNT, POSITION=135, LENGTH=15, TYPE=PACKED_DECIMAL, PRECISION=2)
/FIELD=(PAYMENT_TYPE, POSITION=150, LENGTH=2, TYPE=ASCII)
/FIELD=(CARD_TYPE, POSITION=152, LENGTH=4, TYPE=ASCII)
/FIELD=(AUTH_CODE, POSITION=156, LENGTH=12, TYPE=ASCII)
/FIELD=(TERMINAL_ID, POSITION=168, LENGTH=8, TYPE=ASCII)
/FIELD=(STORE_ID, POSITION=176, LENGTH=6, TYPE=ASCII)
/FIELD=(REGION_CODE, POSITION=182, LENGTH=4, TYPE=ASCII)
/FIELD=(COUNTRY_CODE, POSITION=186, LENGTH=2, TYPE=ASCII)
/FIELD=(CURRENCY_CODE, POSITION=188, LENGTH=3, TYPE=ASCII)
/FIELD=(FX_RATE, POSITION=191, LENGTH=9, TYPE=PACKED_DECIMAL, PRECISION=6)
/FIELD=(BASE_CURRENCY_AMT, POSITION=200, LENGTH=15, TYPE=PACKED_DECIMAL, PRECISION=2)
/FIELD=(STATUS_CODE, POSITION=215, LENGTH=2, TYPE=ASCII)
/FIELD=(ERROR_CODE, POSITION=217, LENGTH=4, TYPE=ASCII)
/FIELD=(BATCH_ID, POSITION=221, LENGTH=10, TYPE=ASCII)
/FIELD=(SOURCE_SYSTEM, POSITION=231, LENGTH=3, TYPE=ASCII)
/FIELD=(CREATED_TS, POSITION=234, LENGTH=26, TYPE=ASCII)
/FIELD=(FILLER, POSITION=260, LENGTH=241, TYPE=ASCII)

/*********************************************************************/
/*  FIELD DEFINITIONS - CUSTOMER MASTER LAYOUT                       */
/*********************************************************************/
/FIELD=(CM_CUST_ID, POSITION=1, LENGTH=12, TYPE=ASCII) FILE=CUSTMAST
/FIELD=(CM_CUST_NAME, POSITION=13, LENGTH=50, TYPE=ASCII) FILE=CUSTMAST
/FIELD=(CM_CUST_TYPE, POSITION=63, LENGTH=2, TYPE=ASCII) FILE=CUSTMAST
/FIELD=(CM_CREDIT_LIMIT, POSITION=65, LENGTH=13, TYPE=PACKED_DECIMAL, PRECISION=2) FILE=CUSTMAST
/FIELD=(CM_CURRENT_BAL, POSITION=78, LENGTH=13, TYPE=PACKED_DECIMAL, PRECISION=2) FILE=CUSTMAST
/FIELD=(CM_STATUS, POSITION=91, LENGTH=1, TYPE=ASCII) FILE=CUSTMAST
/FIELD=(CM_REGION, POSITION=92, LENGTH=4, TYPE=ASCII) FILE=CUSTMAST
/FIELD=(CM_TIER, POSITION=96, LENGTH=2, TYPE=ASCII) FILE=CUSTMAST
/FIELD=(CM_ACCT_MGR, POSITION=98, LENGTH=8, TYPE=ASCII) FILE=CUSTMAST
/FIELD=(CM_OPEN_DATE, POSITION=106, LENGTH=8, TYPE=ASCII) FILE=CUSTMAST

/*********************************************************************/
/*  JOIN SPECIFICATION                                               */
/*********************************************************************/
/JOIN TYPE=LEFT_OUTER
    /KEY=(CUST_ID) FILE=INFILE
    /KEY=(CM_CUST_ID) FILE=CUSTMAST

/*********************************************************************/
/*  INCLUDE CONDITIONS (VALIDATION AND FILTERING)                    */
/*********************************************************************/
/INCLUDE=(
    (TRANS_DATE >= "20250101") AND
    (TRANS_DATE <= "${SYM_PROCESS_DATE}") AND
    (COUNTRY_CODE = "${SYM_REGION_FILTER}") AND
    (TOTAL_AMOUNT >= ${SYM_MIN_AMOUNT}) AND
    (TOTAL_AMOUNT <= ${SYM_MAX_AMOUNT}) AND
    (STATUS_CODE = "AP") AND
    (PAYMENT_TYPE = "CC" OR PAYMENT_TYPE = "DC" OR PAYMENT_TYPE = "EF") AND
    (CUST_ID != "            ")
)

/*********************************************************************/
/*  OMIT CONDITIONS                                                  */
/*********************************************************************/
/OMIT=(
    (ERROR_CODE != "0000") OR
    (TOTAL_AMOUNT = 0) OR
    (CUST_TYPE = "XX")
)

/*********************************************************************/
/*  DERIVED FIELD CALCULATIONS                                       */
/*********************************************************************/
/FIELD=(UNIT_PRICE_CALC, TYPE=NUMERIC, PRECISION=2)
    /COMPUTE=GROSS_AMOUNT / QUANTITY

/FIELD=(DISCOUNT_RATIO, TYPE=NUMERIC, PRECISION=2)
    /COMPUTE=(NET_AMOUNT * 100) / GROSS_AMOUNT

/FIELD=(TAX_TOTAL, TYPE=NUMERIC, PRECISION=2)
    /COMPUTE=TOTAL_AMOUNT - NET_AMOUNT

/FIELD=(SEQ_NUM, TYPE=NUMERIC, LENGTH=8)
    /COMPUTE=RECORD_NUMBER()

/FIELD=(VALUE_FLAG, TYPE=ASCII, LENGTH=2)
    /COMPUTE=IF(CUST_TYPE = "PR" AND NET_AMOUNT > 50000, "HV", "NV")

/FIELD=(ELEC_FLAG, TYPE=ASCII, LENGTH=1)
    /COMPUTE=IF(CONTAINS(PRODUCT_CAT, "ELEC"), "Y", "N")

/*********************************************************************/
/*  PRIMARY SORT KEYS                                                */
/*********************************************************************/
/SORT_FIELDS=(
    REGION_CODE, ASCENDING,
    STORE_ID, ASCENDING,
    TRANS_DATE, ASCENDING,
    TOTAL_AMOUNT, DESCENDING
)

/*********************************************************************/
/*  SUMMARIZATION FIELDS                                             */
/*********************************************************************/
/SUM_FIELDS=(
    QUANTITY,
    GROSS_AMOUNT,
    DISCOUNT_AMT,
    NET_AMOUNT,
    TAX_AMOUNT,
    TOTAL_AMOUNT,
    BASE_CURRENCY_AMT
)

/*********************************************************************/
/*  OUTPUT FILE 1: DETAILED TRANSACTION REPORT                       */
/*********************************************************************/
/OUTFILE=/data/output/trans_detail_${SYM_PROCESS_DATE}.dat NAME=DETAILED
    /RECORD_LENGTH=600
    /FIXED

    /REPORT
    /PAGE_HEADER=(
        "TRANSACTION DETAIL REPORT     GENERATED: " DATE("YYYY/MM/DD") " " TIME("HH:MM:SS")
        "                                                                PAGE: " PAGE_NUMBER
    )
    /PAGE_HEADER2=(FILL("-", 130))

    /FIELD=(OUT_DETAIL, TYPE=ASCII)
        /COMPUTE="DTL|" +
            TRANS_ID + "|" +
            SUBSTR(TRANS_DATE,1,4) + "-" + SUBSTR(TRANS_DATE,5,2) + "-" + SUBSTR(TRANS_DATE,7,2) + "|" +
            CUST_ID + "|" +
            TRIM(CM_CUST_NAME) + "|" +
            PRODUCT_CODE + "|" +
            FORMAT(QUANTITY, "9999999") + "|" +
            FORMAT(GROSS_AMOUNT, "$$$,$$$,$$9.99") + "|" +
            FORMAT(DISCOUNT_AMT, "$$$,$$$,$$9.99") + "|" +
            FORMAT(NET_AMOUNT, "$$$,$$$,$$9.99") + "|" +
            FORMAT(TAX_AMOUNT, "$$$,$$$,$$9.99") + "|" +
            FORMAT(TOTAL_AMOUNT, "$$,$$$,$$$,$$9.99") + "|" +
            PAYMENT_TYPE + "|" +
            STORE_ID + "|" +
            REGION_CODE

    /REPORT_FOOTER=("*** END OF REPORT ***     TOTAL RECORDS: " RECORD_COUNT())

/*********************************************************************/
/*  OUTPUT FILE 2: SUMMARY BY REGION AND STORE                       */
/*********************************************************************/
/OUTFILE=/data/output/trans_summary_${SYM_PROCESS_DATE}.dat NAME=SUMMARY
    /RECORD_LENGTH=200
    /FIXED

    /BREAK_FIELD=REGION_CODE
    /BREAK_HEADER=("REGION: " REGION_CODE, FILL("=", 60))
    /BREAK_FOOTER=(
        "  REGION TOTAL: " FORMAT(SUM(TOTAL_AMOUNT), "$,$$$,$$$,$$$,$$9.99")
        "  TRANS COUNT: " COUNT()
    )

    /FIELD=(OUT_SUMMARY, TYPE=ASCII)
        /COMPUTE="  " + STORE_ID + " | " +
            TRANS_DATE + " | " +
            FORMAT(COUNT(), "999999") + " | " +
            FORMAT(TOTAL_AMOUNT, "$$$,$$$,$$$,$$9.99") + " | " +
            FORMAT(BASE_CURRENCY_AMT, "$$$,$$$,$$$,$$9.99")

    /REPORT_FOOTER=(
        "",
        "GRAND TOTALS",
        FILL("=", 60),
        "TOTAL TRANSACTIONS: " FORMAT(TOTAL_COUNT(), "9999999999"),
        "TOTAL GROSS AMOUNT: " FORMAT(TOTAL_SUM(GROSS_AMOUNT), "$,$$$,$$$,$$$,$$$,$$9.99"),
        "TOTAL DISCOUNTS:    " FORMAT(TOTAL_SUM(DISCOUNT_AMT), "$,$$$,$$$,$$$,$$$,$$9.99"),
        "TOTAL NET AMOUNT:   " FORMAT(TOTAL_SUM(NET_AMOUNT), "$,$$$,$$$,$$$,$$$,$$9.99"),
        "TOTAL TAX:          " FORMAT(TOTAL_SUM(TAX_AMOUNT), "$,$$$,$$$,$$$,$$$,$$9.99"),
        "GRAND TOTAL:        " FORMAT(TOTAL_SUM(TOTAL_AMOUNT), "$,$$$,$$$,$$$,$$$,$$9.99")
    )

/*********************************************************************/
/*  OUTPUT FILE 3: EXCEPTION RECORDS (HIGH VALUE)                    */
/*********************************************************************/
/OUTFILE=/data/output/trans_highvalue_${SYM_PROCESS_DATE}.dat NAME=HIGHVAL
    /RECORD_LENGTH=300
    /FIXED
    /INCLUDE=(TOTAL_AMOUNT > 100000)

    /FIELD=(OUT_HIGHVAL, TYPE=ASCII)
        /COMPUTE="HIGH_VALUE_ALERT|" +
            TRANS_ID + "|" +
            TRANS_DATE + "|" +
            CUST_ID + "|" +
            TRIM(CM_CUST_NAME) + "|" +
            CM_TIER + "|" +
            FORMAT(TOTAL_AMOUNT, "$$$$$,$$$,$$$,$$9.99") + "|" +
            FORMAT(CM_CREDIT_LIMIT, "$$$$$,$$$,$$$,$$9.99") + "|" +
            STORE_ID + "|" +
            AUTH_CODE + "|" +
            CREATED_TS

/*********************************************************************/
/*  OUTPUT FILE 4: PAYMENT TYPE BREAKDOWN                            */
/*********************************************************************/
/OUTFILE=/data/output/trans_by_payment_${SYM_PROCESS_DATE}.dat NAME=PAYTYPE
    /RECORD_LENGTH=150
    /FIXED

    /BREAK_FIELD=PAYMENT_TYPE
    /BREAK_HEADER=(
        "PAYMENT METHOD: " PAYMENT_TYPE +
        IF(PAYMENT_TYPE = "CC", " (CREDIT CARD)",
           IF(PAYMENT_TYPE = "DC", " (DEBIT CARD)",
              IF(PAYMENT_TYPE = "EF", " (ELECTRONIC FUNDS)", ""))),
        FILL("-", 40)
    )
    /BREAK_FOOTER=(
        "SUBTOTAL: " FORMAT(SUM(TOTAL_AMOUNT), "$$$,$$$,$$$,$$9.99") +
        "  COUNT: " FORMAT(COUNT(), "9999999")
    )

    /FIELD=(OUT_PAYTYPE, TYPE=ASCII)
        /COMPUTE="  " + CARD_TYPE + " | " +
            FORMAT(COUNT(), "99999") + " | " +
            FORMAT(TOTAL_AMOUNT, "$$$,$$$,$$$,$$9.99")

/*********************************************************************/
/*  OUTPUT FILE 5: REJECTED RECORDS FOR REPROCESSING                 */
/*********************************************************************/
/OUTFILE=/data/output/trans_rejects_${SYM_PROCESS_DATE}.dat NAME=REJECTS
    /RECORD_LENGTH=550
    /FIXED
    /SAVE_REJECTS=YES

    /FIELD=(ERROR_DESC, TYPE=ASCII, LENGTH=16)
        /COMPUTE=IF(ERROR_CODE = "E001", "INVALID_CUSTOMER",
                 IF(ERROR_CODE = "E002", "AMOUNT_MISMATCH",
                 IF(ERROR_CODE = "E003", "AUTH_FAILED",
                 IF(ERROR_CODE = "E004", "DUPLICATE_TRANS", "UNKNOWN_ERROR"))))

    /FIELD=(OUT_REJECT, TYPE=ASCII)
        /COMPUTE=RECORD_DATA + "REJ" + ERROR_CODE + ERROR_DESC +
            DATE("YYYY-MM-DD") + TIME("HH:MM:SS")

/*********************************************************************/
/*  OUTPUT FILE 6: STATISTICAL ANALYSIS (JSON FORMAT)                */
/*********************************************************************/
/OUTFILE=/data/output/trans_statistics_${SYM_PROCESS_DATE}.dat NAME=STATS
    /VARIABLE_LENGTH
    /MAX_RECORD_LENGTH=400
    /TRIM_TRAILING_SPACES=YES

    /FIELD=(OUT_JSON, TYPE=ASCII)
        /COMPUTE='{"report_date":"${SYM_PROCESS_DATE}",' +
            '"region":"' + REGION_CODE + '",' +
            '"store":"' + STORE_ID + '",' +
            '"trans_count":' + FORMAT(COUNT(), "9999999999") + ',' +
            '"total_gross":' + FORMAT(GROSS_AMOUNT, "999999999999.99") + ',' +
            '"total_discount":' + FORMAT(DISCOUNT_AMT, "999999999999.99") + ',' +
            '"total_net":' + FORMAT(NET_AMOUNT, "999999999999.99") + ',' +
            '"total_tax":' + FORMAT(TAX_AMOUNT, "999999999999.99") + ',' +
            '"total_amount":' + FORMAT(TOTAL_AMOUNT, "999999999999.99") + ',' +
            '"avg_trans_value":' + FORMAT(TOTAL_AMOUNT / COUNT(), "99999999.99") + ',' +
            '"currency":"' + CURRENCY_CODE + '"}'

/*********************************************************************/
/*  PROCESSING OPTIONS                                               */
/*********************************************************************/
/OPTION MEMORY=MAX
/OPTION WORK_DIRECTORY=/tmp/cosort_work
/OPTION ESTIMATED_RECORDS=500000
/OPTION MESSAGE_LEVEL=ALL
/OPTION LIST=YES
/OPTION VERIFY=YES

/*********************************************************************/
/*  USER EXIT MODULES                                                */
/*********************************************************************/
/EXIT_MODULE NAME=VALIDEXIT ROUTINE=E15 SIZE=4096
/EXIT_MODULE NAME=ENRICHEXIT ROUTINE=E35 SIZE=8192

/*********************************************************************/
/*  END OF COSORT CONTROL STATEMENTS                                 */
/*********************************************************************/

Now consider mainframe modernization and a switch to a Python on Linux (in the Cloud or locally).

The direction you take depends on data volume, team skills, and how much you want to modernize versus just migrate.

Options ranked by modernization level

1. Lift-and-shift: CoSort/Syncsort on Linux

• Minimal code changes
• Proven performance for sort/merge workloads
• Licensing costs continue
• Still maintains "batch file" paradigm

This works, but you're paying for legacy tooling and not gaining much.

2. Modern single-node alternatives (my recommendation for most cases)

DuckDB — best if your team knows SQL:

• Handles larger-than-memory datasets efficiently
• Single binary, no infrastructure
• Columnar storage, very fast for aggregations
• Can read fixed-width files directly

Polars — best if your team prefers Python/dataframes:

• Rust-based, extremely fast
• Lazy evaluation optimizes query plans
• Handles large files with streaming
• More expressive than pandas for complex transforms

For a typical mainframe batch job (millions of records, GB-scale files), either of these will match or beat Syncsort performance on modern hardware without licensing costs.

3. Distributed processing: Spark

• Overkill unless you're processing TB+ daily
• Requires cluster infrastructure (or Databricks/EMR spend)
• Good if you're already in that ecosystem

Practical recommendation

For our example in Syncsort (or CoSort) script, one could use DuckDB or Polars. Here's a sketch in Polars:

import polars as pl
from datetime import datetime

# Configuration
PROCESS_DATE = "20250115"
REGION_FILTER = "US"
MIN_AMOUNT = 1000
MAX_AMOUNT = 9999999999

# Schema for fixed-width parsing
def read_transactions(path: str) -> pl.LazyFrame:
    """Read fixed-width transaction file."""
    return pl.read_csv(
        path,
        has_header=False,
        schema={
            "trans_id": pl.Utf8,
            "trans_date": pl.Utf8,
            "trans_time": pl.Utf8,
            "cust_id": pl.Utf8,
            "cust_type": pl.Utf8,
            "product_code": pl.Utf8,
            "product_cat": pl.Utf8,
            "quantity": pl.Int64,
            "unit_price": pl.Float64,
            "gross_amount": pl.Float64,
            "discount_pct": pl.Float64,
            "discount_amt": pl.Float64,
            "net_amount": pl.Float64,
            "tax_rate": pl.Float64,
            "tax_amount": pl.Float64,
            "total_amount": pl.Float64,
            "payment_type": pl.Utf8,
            "card_type": pl.Utf8,
            "auth_code": pl.Utf8,
            "terminal_id": pl.Utf8,
            "store_id": pl.Utf8,
            "region_code": pl.Utf8,
            "country_code": pl.Utf8,
            "currency_code": pl.Utf8,
            "fx_rate": pl.Float64,
            "base_currency_amt": pl.Float64,
            "status_code": pl.Utf8,
            "error_code": pl.Utf8,
            "batch_id": pl.Utf8,
            "source_system": pl.Utf8,
            "created_ts": pl.Utf8,
        }
    ).lazy()

# For true fixed-width, you'd use a custom parser or convert first
# This assumes pre-converted CSV or use polars' slice operations on raw bytes

def process_transactions():
    # Read and merge transaction sources
    trans_pos = read_transactions(f"/data/daily/transactions_pos_{PROCESS_DATE}.csv")
    trans_online = read_transactions(f"/data/daily/transactions_online_{PROCESS_DATE}.csv")
    
    transactions = pl.concat([trans_pos, trans_online])
    
    # Read customer master
    customers = pl.read_csv("/data/master/customer_master.csv").lazy()
    
    # Filter and validate
    valid_trans = (
        transactions
        .filter(
            (pl.col("trans_date") >= "20250101") &
            (pl.col("trans_date") <= PROCESS_DATE) &
            (pl.col("country_code") == REGION_FILTER) &
            (pl.col("total_amount") >= MIN_AMOUNT) &
            (pl.col("total_amount") <= MAX_AMOUNT) &
            (pl.col("status_code") == "AP") &
            (pl.col("payment_type").is_in(["CC", "DC", "EF"])) &
            (pl.col("cust_id").str.strip_chars() != "") &
            (pl.col("error_code") == "0000") &
            (pl.col("total_amount") != 0) &
            (pl.col("cust_type") != "XX")
        )
    )
    
    # Capture rejects
    rejects = (
        transactions
        .filter(
            (pl.col("error_code") != "0000") |
            (pl.col("total_amount") == 0) |
            (pl.col("cust_type") == "XX")
        )
        .with_columns(
            pl.when(pl.col("error_code") == "E001").then(pl.lit("INVALID_CUSTOMER"))
            .when(pl.col("error_code") == "E002").then(pl.lit("AMOUNT_MISMATCH"))
            .when(pl.col("error_code") == "E003").then(pl.lit("AUTH_FAILED"))
            .when(pl.col("error_code") == "E004").then(pl.lit("DUPLICATE_TRANS"))
            .otherwise(pl.lit("UNKNOWN_ERROR"))
            .alias("error_desc")
        )
    )
    
    # Join with customer master
    enriched = (
        valid_trans
        .join(
            customers,
            left_on="cust_id",
            right_on="cm_cust_id",
            how="left"
        )
        .with_columns([
            # Derived calculations
            (pl.col("gross_amount") / pl.col("quantity")).alias("unit_price_calc"),
            (pl.col("net_amount") * 100 / pl.col("gross_amount")).alias("discount_ratio"),
            (pl.col("total_amount") - pl.col("net_amount")).alias("tax_total"),
            # Value flag
            pl.when(
                (pl.col("cust_type") == "PR") & (pl.col("net_amount") > 50000)
            ).then(pl.lit("HV")).otherwise(pl.lit("NV")).alias("value_flag"),
            # Electronics flag
            pl.col("product_cat").str.contains("ELEC").alias("elec_flag"),
        ])
        .sort(["region_code", "store_id", "trans_date", "total_amount"], 
              descending=[False, False, False, True])
    )
    
    # Collect once, then write multiple outputs
    enriched_df = enriched.collect()
    
    # Output 1: Detail report
    enriched_df.write_csv(f"/data/output/trans_detail_{PROCESS_DATE}.csv")
    
    # Output 2: Summary by region/store
    summary = (
        enriched_df
        .group_by(["region_code", "store_id", "trans_date"])
        .agg([
            pl.count().alias("trans_count"),
            pl.sum("gross_amount").alias("sum_gross"),
            pl.sum("discount_amt").alias("sum_discount"),
            pl.sum("net_amount").alias("sum_net"),
            pl.sum("tax_amount").alias("sum_tax"),
            pl.sum("total_amount").alias("sum_total"),
            pl.sum("base_currency_amt").alias("sum_base_curr"),
        ])
        .sort(["region_code", "store_id", "trans_date"])
    )
    summary.write_csv(f"/data/output/trans_summary_{PROCESS_DATE}.csv")
    
    # Output 3: High value transactions
    high_value = enriched_df.filter(pl.col("total_amount") > 100000)
    high_value.write_csv(f"/data/output/trans_highvalue_{PROCESS_DATE}.csv")
    
    # Output 4: Payment type breakdown
    payment_summary = (
        enriched_df
        .group_by(["payment_type", "card_type"])
        .agg([
            pl.count().alias("trans_count"),
            pl.sum("total_amount").alias("sum_total"),
        ])
        .sort(["payment_type", "card_type"])
    )
    payment_summary.write_csv(f"/data/output/trans_by_payment_{PROCESS_DATE}.csv")
    
    # Output 5: Rejects
    rejects.collect().write_csv(f"/data/output/trans_rejects_{PROCESS_DATE}.csv")
    
    # Output 6: Statistics as JSON
    stats = (
        enriched_df
        .group_by(["region_code", "store_id", "currency_code"])
        .agg([
            pl.count().alias("trans_count"),
            pl.sum("gross_amount").alias("total_gross"),
            pl.sum("discount_amt").alias("total_discount"),
            pl.sum("net_amount").alias("total_net"),
            pl.sum("tax_amount").alias("total_tax"),
            pl.sum("total_amount").alias("total_amount"),
            (pl.sum("total_amount") / pl.count()).alias("avg_trans_value"),
        ])
    )
    stats.write_ndjson(f"/data/output/trans_statistics_{PROCESS_DATE}.json")
    
    return {
        "records_processed": len(enriched_df),
        "rejects": rejects.collect().height,
        "high_value_count": len(high_value),
    }

if __name__ == "__main__":
    result = process_transactions()
    print(f"Processing complete: {result}")

Why this approach beats CoSort on Linux 

What you would need to address

1. Fixed-width file parsing: Neither Polars nor DuckDB handles packed decimal or complex fixed-width natively. Options:
   • Pre-convert to CSV/Parquet on the mainframe before migration
   • Write a small Python parser using struct module
   • Use a tool like awk or miller for initial conversion
2. Report formatting: The fancy headers/footers/control breaks need separate handling—either a reporting library or post-process the CSVs.
3. Orchestration: Wrap in Airflow, Dagster, or even cron for scheduling.

Reach out to East Agile to learn more using our contact form.