SMF Decoded: Enhancing IT Data Accessibility with Zetaly's Self-Service Collector

In the current data-driven landscape, it is imperative to be able to fully utilise your data. But what if the data you need is locked away in formats that seem impenetrable, like IBM's SMF records? That's where Zetaly's Self-Service Collector steps in as a game-changer. Let's explore how to break free from data constraints and access valuable insights hidden within complex data sources, starting from the mainframe world.

The Brilliant Yet Complex Binary: the Language of SMF

SMF, short for System Management Facilities, began recording the activities of IBM mainframes during the 1960s, serving as a comprehensive framework for system administrators to collect data about various aspects of their mainframe environments. This data could include performance metrics, system logs, security information, and much more. The importance of SMF grew as businesses and organizations increasingly relied on mainframes to run their critical operations.

One of the distinguishing features of SMF is its use of binary encoding. Unlike distributed-like log records that utilize human-readable text/number formats, SMF records are written in binary, which is a base-2 numerical system of ones and zeros. This binary format allows for compact, efficient storage and processing of data, but it also adds a layer of complexity that may seem arcane in the age of user-friendly graphical interfaces.

The choice of binary encoding wasn't accidental. Back in the early days of computing, when mainframes were first taking shape, every bit of storage and processing power mattered. The binary format used in SMF allowed the mainframes to optimize data storage and access. Despite the challenges it presents for human interpretation, it's a testament to the ingenuity of early computer scientists.

But with current achievable processing power, why hold on binary? Firstly, historical data preservation has always been a key driver in IT decision-making. SMF records have been collected and stored for decades, providing a rich historical archive of mainframe system activities. This historical data is invaluable for auditing, troubleshooting, and analyzing long-term trends. Secondly, mainframes have a long lifespan, and many organizations still rely on older systems that generate SMF records. Maintaining the SMF format ensures that data collected on older mainframes can be seamlessly integrated with newer systems, allowing for a consistent and unified approach to system management and analysis.

As SMF records serve as a treasure trove of information about the mainframe's performance, security, and operations, they can not be under-exploited. Hopefully, ready-to-use solutions could help collect, transform, and make sense of this precious data.

Data Accessibility: the Core Mission of Zetaly's Self-Service Collector        

While tools designed to collect and interpret SMF records are invaluable for monitoring and managing mainframe systems, it's important to note that they typically focus on the most common or widely used SMF record types. For example, Zetaly offers out-of-the-box meaningful insights into system performance and resource usage, useful for day-to-day operations. 

However, the landscape of SMF records is vast and diverse, encompassing a wide range of data categories, from hardware events to application-specific information. And that's why a self-service collector is an invaluable asset: you are in charge of defining which data is important for your company's business.

Zetaly Self-Service Collector was designed to give users complete freedom over which additional and optional SMF records should be collected and transformed. As a result, any SMF data, even not natively supported, are easily parseable, without the need to understand the assembly language whatsoever. 

The example of SMF record 66

SMF record 66, also known as Type 66, identifies the entry being altered and the catalog in which the catalog record is written or deleted, and gives the new, updated, or deleted catalog record. This record indicates if the entry was renamed (function indicator = ‘R’) and, if so, gives the old and new names of the entry. It identifies the job by job log identification and user identification. 

Nevertheless, this record does not belong to the "holy" array of SMF records related to sensitive and business-critical programs such as CICS or DB2. Thus, it is likely not supported by default by nowadays log analytics software. That is where Zetaly's Self-Service Collector comes in handy. Its interface helps you define how to collect, store, and interpret a specific SMF record. 

Fortunately, IBM excels in providing comprehensive documentation, as you can see from this snippet from SMF 66 documentation for its section 58: https://www.ibm.com/docs/en/zos/3.1.0?topic=rm-headerself-defining-section-56

All you have to do from here is copy these field decoding instructions into the self-service collector interface, leading to the following result:

How do you get this outcome? Let's explore the decoding process in three easy steps.

• Let's take the first field SMF66LEN, which gives us the record length, as an example. Its position is set at the first byte (0) and spans two bytes. As a first step, we can easily describe the byte range and its localization on the array.

• The next step allows us to turn bytes into human-readable data. Here, we transform a binary into an integer.

• As a last step, you can test your parsing instructions and get a preview of the decoded data.

Beyond SMF decoding: resource optimization & data enrichment

Deciphering binary records is only the first step in harnessing the power of mainframe log data. Collecting and understanding SMF records presents other significant challenges:

• Data Diversity: SMF records cover a wide range of data categories, from system performance metrics to security information. Each category may have its own unique format and data structure and are stored in separate files.
• Data Volume: Mainframes can process large volumes of data, and SMF records can accumulate quickly. Managing, storing, and processing these records, especially in high-transaction environments, can be a significant logistical challenge.
• Interpretation and Analysis: Even when collected, interpreting and analyzing SMF records requires a deep understanding of mainframe operations, as well as the ability to correlate data from various records to gain meaningful insights.

Regarding data volume, Zetaly's self-service collector already provides a solution as you can selectively choose which field to collect. This selective approach empowers users to retain only the essential data while discarding the rest, effectively reducing the footprint on both storage resources and processing capabilities.

But all these challenges call for more than a standalone data decoder. That is why the true power of Zetaly's Self-Service Collector lies within its native integration with Zetaly's Data platform and all of its Data Analytics solutions.

Any IT Data, From Anywhere: Reaching the Big Picture

SMF records are indeed valuable as mainframes continue to hold a vital role for large businesses due to their enduring relevance in addressing specific business needs. After all, these powerful computing systems offer unmatched reliability, scalability, and security, making them ideal for mission-critical operations. 

However, IT evolves into increasingly complex environments marked by hybrid systems and a multitude of platforms. The interconnection of various technologies and the integration of heterogeneous systems mean that, for a comprehensive view of IT operations, it is now essential to aggregate and analyze data generated by all these systems. This is the only way to obtain a holistic picture of performance, security, and operations management, enabling businesses to make informed decisions in an increasingly complex IT landscape.

Zetaly's ambition behind its Self-Service Collector is the ability to collect, interpret, and correlate any IT data, from any source, regardless of format or origin.