Showing posts with label z OS DASD. Show all posts
Showing posts with label z OS DASD. Show all posts

Monday, 28 July 2014

How z/OS Stores COBOL Data Sets on DASD

A COBOL program writes an 80-byte logical record; QSAM can combine many such records into one block; z/OS places the block in CKD or ECKD track format inside an allocated extent. That path is more useful than picturing the application selecting a disk sector.

COBOL data storage on z OS DASD from logical records through blocks tracks and extents
COBOL sees logical records; z/OS access methods and storage services map them to DASD space.

The storage path from COBOL to DASD

  1. The program reads or writes a logical record described by its FD and record layout.
  2. An access method such as QSAM buffers records and can group them into blocks.
  3. The blocks are recorded on CKD or ECKD tracks assigned to the data set.
  4. Tracks belong to one or more extents on a DASD volume.
  5. The volume table of contents records data-set and extent information; a catalog can map the data-set name to its volume.
Application boundary: ordinary COBOL sequential I/O addresses records through an access method. The program does not calculate cylinder, head, and record addresses for each WRITE.

Key z/OS DASD terms

TermMeaningWhat the developer sees
Logical recordThe application data unit.LRECL, RECFM, and the COBOL record description.
BlockA non-VSAM recording unit that can hold one or more logical records.BLKSIZE; QSAM normally handles blocking and unblocking.
TrackA CKD/ECKD recording area containing record 0 and data records or blocks.Allocation and diagnostic tools report track use.
CylinderA fixed group of tracks on a CKD device.SPACE=(CYL,...) requests space in cylinder units.
ExtentA contiguous area allocated or reserved for one data set.Primary space forms the initial allocation; secondary requests add extents.
VolumeA logical DASD volume identified by a volume serial.VOLSER, storage class, catalog, and VTOC information.

CKD and ECKD track format

IBM documents DASD tracks in count-key-data or extended count-key-data format. Each track contains record 0 and data records. A count area identifies a block's cylinder, head, record number, and data length. A hardware key area is optional; most application data uses count-data format, while VSAM uses software keys inside its own structures.

Hardware documentation may call a unit on the track a record, while z/OS software documentation often calls it a block and reserves “record” for the logical application record. Confirm which layer a manual is describing before comparing sizes.

Logical records and blocks

For a fixed blocked sequential data set, RECFM=FB means each logical record has the same length and a block contains an integral number of records. With LRECL=80 and BLKSIZE=0, z/OS selects a suitable system-determined block size.

//OUTFILE DD DSN=USER1.DAILY.REPORT, // DISP=(NEW,CATLG,DELETE), // UNIT=SYSDA, // SPACE=(CYL,(5,2),RLSE), // DCB=(RECFM=FB,LRECL=80,BLKSIZE=0)

The job requests five cylinders initially and two cylinders for each secondary allocation. RLSE asks z/OS to release eligible unused space when the data set is closed. The blocked versus unblocked records guide explains blocking-factor calculations and system-determined BLKSIZE.

Primary space, secondary space, and extents

The primary quantity is the initial space request. When it fills, z/OS can obtain the secondary quantity as another extent, subject to volume space and the extent limit for that data-set type. Extents let a data set grow without requiring one preallocated continuous area for its maximum possible size.

Repeated small secondary allocations can create many extents and eventually hit a format limit. A failed allocation or extension should be diagnosed from allocation messages, volume free space, SMS class rules, and the data set's current extent count rather than from COBOL source alone.

What the VTOC and catalog do

The VTOC holds control-block information for data sets and free space on a volume, including extent locations and attributes such as DSORG, RECFM, LRECL, and BLKSIZE. The catalog serves a different lookup purpose: it associates a cataloged data-set name with the volume or storage information needed to locate it.

Do not merge the concepts: cataloging a data set does not store its records in the catalog. The VTOC and catalog contain metadata; the data blocks occupy allocated DASD extents.

VSAM uses control intervals and control areas

The record-to-block explanation applies directly to non-VSAM access methods such as QSAM. VSAM organizes records in control intervals, with control intervals grouped into control areas. A control interval can occupy one or more contiguous DASD blocks. Use IDCAMS DEFINE and LISTCAT for VSAM allocation and catalog inspection, and the COBOL indexed-file guide for application access.

PDS and PDSE storage differs

A PDS contains a fixed-size directory and members, and deleted member space is not automatically reclaimed until compression. A PDSE has a managed, expandable directory and automatically reuses space from deleted or replaced members. PDSE processing can also reblock logical records when writing to DASD, so the block size observed by an application need not be the original physical block size.

Modern DASD is often virtualized

z/OS applications continue to use logical CKD/ECKD volumes, tracks, cylinders, and extents. The storage controller can map those logical constructs onto RAID arrays, flash, caches, and replicated storage. Rotational speed and literal platter geometry are therefore poor application assumptions.

The separate COBOL disk-I/O guide follows a READ or WRITE through QSAM, buffers, access methods, the channel subsystem, and storage cache. This page owns the on-volume layout and allocation model.

How to inspect a data set

  • Use ISPF 3.4 or LISTDS to view DSORG, RECFM, LRECL, BLKSIZE, volume, and allocation information for non-VSAM data sets.
  • Use IDCAMS LISTCAT for VSAM cluster, component, control-interval, and allocation details.
  • Check job allocation messages when primary or secondary space cannot be obtained.
  • Match the data-set attributes to the COBOL FD and record layout; the fixed versus variable record guide covers FB and VB choices.
  • For field-level layout questions, use the COBOL record-description guide.

Common storage-model mistakes

  • Equating a COBOL record with a DASD block: several records can share one block, and a spanned record can cross blocks.
  • Calling DASD units PC sectors: supported z/OS interfaces describe CKD/ECKD tracks, records or blocks, cylinders, and extents.
  • Assuming BLKSIZE equals track capacity: device format and per-block overhead affect track use.
  • Treating the catalog as the data store: it supplies location metadata, not the application records.
  • Applying QSAM blocking terms directly to VSAM: VSAM uses control intervals and control areas.

Official IBM references

Frequently asked questions

Does COBOL write directly to a DASD track?

Usually no. A COBOL program works with logical records through an access method such as QSAM or VSAM. z/OS and the access method perform buffering, blocking, and device I/O.

What is the difference between a record and a block?

A logical record is the application unit described by LRECL and RECFM. A non-VSAM block is the physical recording and transfer unit that can contain one or more logical records.

What is a DASD extent?

An extent is a contiguous area of tracks, cylinders, or blocks allocated or reserved for a data set on a DASD volume. Secondary allocations add extents as the data set grows.

Are modern mainframe disks still physical CKD platters?

z/OS continues to use CKD or ECKD addressing and allocation concepts, but enterprise storage systems can virtualize the logical volumes over internal disk or flash. Applications should use z/OS data-set attributes rather than assume a physical platter layout.

Working rule: diagnose storage from the logical record, access method, block or control interval, allocation, and volume metadata layers—in that order.

COBOL Disk I/O: How z/OS Reads and Writes Data

READ CUSTOMER-FILE asks COBOL for one logical record. It does not tell a disk arm to find one sector. On z/OS, the COBOL run time and an access method such as QSAM manage buffers and blocks, the channel subsystem handles the I/O request, and the storage controller may satisfy that request from cache before any back-end media access is needed.

COBOL disk I O path through QSAM buffers, the channel subsystem, storage cache, and z OS DASD
A COBOL record request passes through QSAM, asynchronous I/O services, and cached DASD storage.

COBOL record I/O versus physical storage I/O

COBOL programs work with records defined under an FD. A sequential READ returns the next logical record; a WRITE supplies one logical record. The program normally does not address a track, cylinder, or sector.

QSAM places physical blocks in buffers. One block can contain one record or several records, depending on RECFM, LRECL, and BLKSIZE. If the requested record is already in an input buffer, QSAM can return it without starting a new device I/O. If an output buffer has room, a WRITE can add the record without immediately sending a block to the device.

Keep the units separate: the COBOL statement handles a logical record, QSAM transfers physical blocks, and CKD track geometry belongs below the application interface.

What happens during a COBOL READ

  1. The program issues READ CUSTOMER-FILE.
  2. The COBOL run time asks QSAM for the next logical record.
  3. If that record is already in a QSAM input buffer, it is moved or made available to the program.
  4. If another block is required, z/OS starts an I/O request through the channel subsystem.
  5. The storage controller checks its cache. A cache hit can satisfy the read without accessing back-end media.
  6. After the block reaches the QSAM buffer, QSAM returns the required logical record and retains other records in the block for later reads.

The COBOL sequential-file guide shows the surrounding OPEN INPUT, READ, end-of-file, and CLOSE logic.

What happens during a COBOL WRITE

  1. The program fills its output record and issues WRITE CUSTOMER-RECORD.
  2. QSAM places the logical record in an output buffer.
  3. When the block is ready to be written, the access method submits the I/O request.
  4. The channel subsystem carries the request to the control unit and storage system.
  5. A modern storage system can accept the write into protected cache and later destage the data to its managed media.
  6. CLOSE completes access-method processing, including the final partially filled output block.

Use the correct open mode for the operation. The COBOL file I/O modes guide distinguishes INPUT, OUTPUT, I-O, and EXTEND.

Why the channel subsystem matters

IBM Z does not require a central processor to control every byte transfer. The channel subsystem accepts the I/O request and communicates with control units and devices independently. While one task waits for its read or write, z/OS can dispatch other ready work on the processors.

A subchannel represents an I/O device to the operating system, and a channel path supplies a route between the processor complex and external equipment. Multiple paths and storage functions can affect availability and response time, but those details remain outside ordinary COBOL source code.

Tracks and cylinders are CKD address concepts

Traditional z/OS DASD uses count-key-data or extended count-key-data formatting. A cylinder contains a fixed number of tracks, and each track contains formatted records. z/OS still exposes CKD-compatible volumes and device types such as 3390 even when the storage system implements them on modern cached hardware.

The old post treated a sector, a physical head, and a COBOL record as if they formed one direct path. That physical-disk description is not a safe model for a current z/OS application. The companion DASD tracks, cylinders, and data-set layout article owns the storage-geometry intent; this page owns the runtime I/O path.

Why storage cache changes the read path

For a read, the controller can return data already present in cache. IBM calls this a cache hit. A cache miss requires the storage system to obtain the requested data from a lower storage layer, so response time can be higher.

For writes, enterprise storage uses cache and nonvolatile protection so the host request does not have to mirror the timing of a literal platter write. Storage tools therefore report metrics such as cache-hit percentage, I/O rate, bandwidth, response time, and write-cache delay.

Do not diagnose current DASD performance from actuator movement alone. Start with host response-time data and storage metrics, then determine whether the delay is in the access method, channel path, cache, control unit, or back-end storage.

Where blocks and buffers affect performance

For a large QSAM sequential pass, blocked records reduce the number of physical I/O requests compared with transferring one record per block. Additional QSAM buffers can also allow access-method processing to overlap more effectively with the program, at the cost of storage.

Use system-determined blocking for eligible new sequential data sets unless an interface requires a specific value. The detailed COBOL blocked-versus-unblocked records guide covers FB, VB, BLKSIZE, blocking factor, and BLOCK CONTAINS 0.

COBOL and JCL example

SELECT CUSTOMER-FILE ASSIGN TO CUSTIN
    ORGANIZATION IS SEQUENTIAL
    FILE STATUS IS WS-FILE-STATUS.

FD  CUSTOMER-FILE
    RECORDING MODE IS F
    BLOCK CONTAINS 0 RECORDS
    RECORD CONTAINS 100 CHARACTERS.
01  CUSTOMER-RECORD PIC X(100).

READ CUSTOMER-FILE
    AT END SET END-OF-FILE TO TRUE
END-READ
//CUSTIN  DD DSN=APP.CUSTOMER.MASTER,
//            DISP=SHR

For an existing cataloged input data set, the DD statement does not need to repeat its DCB attributes. At OPEN, the program description must be compatible with the actual data set. A mismatch can produce file status 39. The COBOL file-operations guide covers OPEN, READ, WRITE, REWRITE, and status handling.

How to investigate a slow I/O-bound step

  1. Confirm the DD name, data-set organization, RECFM, LRECL, and BLKSIZE.
  2. Separate sequential access from random access; they create different request patterns.
  3. Compare elapsed time with CPU time to see whether the step spends substantial time waiting.
  4. Review I/O count, average response time, transfer size, cache-hit percentage, and delay fields in the monitoring data available at your site.
  5. Check whether the access method and buffer count match the workload.
  6. Ask storage support to correlate host measurements with controller metrics before changing allocation rules.

EXCP counts are useful when the access method reports them, but one EXCP is not the same as one physical media access. Caching, channel programs, access-method behavior, and storage implementation break that one-to-one assumption.

Common misconceptions

MisconceptionBetter model
One COBOL READ equals one sector read.One READ returns one logical record; QSAM may already hold it in a buffered block.
One EXCP equals one physical disk movement.An EXCP reflects an access-method I/O request, while cache and storage implementation affect physical activity.
A cylinder always describes literal rotating platters.For z/OS work, cylinder and track are CKD-compatible address and allocation units presented by DASD.
COBOL controls the channel path.COBOL requests record I/O; z/OS and the channel subsystem manage device communication.
A faster CPU always fixes a slow batch step.An I/O-bound step can remain limited by response time, request pattern, buffering, or storage delay.

Related record and file definitions

Record format changes how a block is interpreted. Review fixed versus variable-length COBOL records for FB, VB, RDW, and FD examples. The sequential-file FD entries guide covers the file description clauses that establish the program view.

Official IBM references

COBOL disk I/O FAQ

Does a COBOL READ access one disk sector?

No. A COBOL READ requests one logical record. QSAM can return it from an existing input buffer or request another physical block. The storage controller can then satisfy that I/O from cache or obtain the required data from its managed storage.

What is the difference between a record and a block?

A logical record is the unit processed by the COBOL program. A physical block is the transfer unit used by a non-VSAM access method such as QSAM and can contain one or several logical records.

Why can COBOL continue while an I/O request is active?

IBM Z uses a channel subsystem that performs I/O independently of the central processors. The requesting task can wait, and the processors can dispatch other ready work while the device operation proceeds.

Which measurements help diagnose slow DASD I/O?

Check elapsed time, I/O response time, I/O rate, transfer size, cache-hit percentages, and delays reported by host and storage monitoring. EXCP counts can help for some access methods, but they are not a complete latency measure.

Read the measurements in the same layers as the request: COBOL record, QSAM buffer and block, channel-subsystem I/O, storage cache, then DASD volume.

New In-feed ads