technical: Tracking Software and Module Usage Without TADz
Last month we looked at IBMs Tivoli Asset Discovery for z/OS, and how it can discover software installed, and monitor its usage. It can also monitor module usage for local application programs. However not everyone has the luxury of TADz. In our previous article Discover the Software You Have, we looked at how to discover software installed without using TADz. But what about monitoring software or load module usage?
To monitor software products, you'll need to identify one or more modules that are always loaded when that product is used. For example, CICS always uses DFHSIP, FOCUS uses FOCUS, and CA Easytrieve usually uses EZTPA00. Then the process is the same as monitoring one of your own load modules.
If your program is always called from EXEC PGM=XXX JCL, then the logging is already happening in SMF Type 30 records. So processing of these records using Merrill's MXG or similar gets you a list of the user, job, time, date, step and more for that program. Similarly Type 110 records for CICS programs; IMS log records for IMS programs, and Type 101 for DB2 Stored Procedures.
If you want to monitor anyone accessing an entire load library (ie PDS or PDS/E) that is not in LPA or the Linklist, then that monitoring is also already happening. Dataset open records in SMF Type 42 Subtype 6 will tell you who and when a load library dataset was opened. However remember that it only logs when the dataset is opened. It won't identify the individual modules loaded, or how often.
If your program/module is in LLA, then LLA has some features to help.
The D LLA,STATS command will show each library and the number of fetches for each:
MEMBERS FETCHED: 0 MEMBERS IN VLF: 0
DASD FETCHES: 0 VLF RETRIEVES: 0
However no information on who accessed a module or when.
Some z/OS monitors such as the Pycroft Six IMON and Rocket MXI can also get and display these LLA statistics.
You can get better information by rolling up your sleeves and coding an LLA exit. This exit will get control whenever a program is loaded from LLA. From here you could write an SMF record to log the access. But remember to make it fast - it's going to be called a lot. Clark Jennings includes a sample in file 567 of www.cbttape.org.
Perhaps an easier way is to use your security product to do the monitoring for you. Most security products such as IBM RACF and CA ACF2 can secure programs; restricting access to sensitive programs such as AMASPZAP and ICKDSF. So the idea is to enable security for the module in question, but allow everyone access. The trick is to also enable logging, so anyone accessing the module will be logged. Then it's a matter of processing the RACF/CA ACF2 SMF records to find out the user and job executing the module. For RACF, it all starts with the RDEFINE command:
RDEFINE PROGRAM pgm001 ADDMEM('ibmuser.loadlib1'//NOPADCHK) UACCC(READ)
This allows anyone to access the program pgm001 from library ibmuser.loadlib1, and audits everything (i.e. writes an SMF Type 80 record). You can even use masks (e.g. PGM* for any program beginning with PGM) in this command. Don't forget to refresh the PROGRAM class (SETROPTS WHEN(PROGRAM) REFRESH), and read up carefully about program control to avoid affecting other security processing. For z/OS UNIX modules the command extattr +p does something similar, though you'll have to use the RACF Permit command to permit access, and audit everything. Once done, you can use the RACF SMF data unload utility to dump the SMF records, and then use something like DFSORT ICETOOL to wade through. Commercial SMF reporting tools such as Merrill MXG can also process the SMF Type 80s.
The drawback: won't work for CICS programs.
Other Software Products
For many years IBMs Peter Relson has offered a free 'as is, no warranty' tool to monitor program fetch activity: the Module Fetch Monitor. Contact Peter directly at firstname.lastname@example.org.
Those lucky enough to have CA PMO, the TSO administration utility PMOMON will show the number of load module loads for those libraries managed by PMO (i.e., modules regularly used). Similar functionality is provided by CA PDSMAN. Unfortunately, these two products will only show load libraries. Program objects in PDSEs aren't supported.
UBS Hainer's P-Tracker can be used to monitor program and load library usage. Similarly Data Processing Techniques' PDS Monitor also advertises tracking of load modules and program objects.
If none of the above options suits your needs, there are still some choices. An obvious one is the z/OS syslog. Many software products issue a message, or can be configured to do so every time they run. So looking for this message in the z/OS syslog may do the trick. Alternatively, some software products include exits that could be used. So a simple exit could issue a WTO or write an SMF record every time the product is called.
Another options is to write your own front-end to a module being called. This is how it could work:
Again, this isn't for the fainthearted, as you need to ensure that the software module is called correctly, and your front-ending module doesn't mess things up. But this is a good option when retiring a product. You could even write a message in your front-end program warning the user that this software will soon be retired.
- You write a small routine that first writes an SMF record when called, and then calls the module in question.
- You link this routine into the module in question, and set the entry point of the module to your routine.
- Now, anyone calling the module will call your routine first.
Options Not To Try
There are also a couple of options that would work, but you won't want to try. GTF can be used to trace module usage, but is far too expensive in CPU costs to consider using it for an extended period across a z/OS system. It is more ideally suited to monitoring of individual address spaces for a short period of time. For example, analysing programs used by IMS dependent regions when determining preload lists.
Another option that you won't want to try is the z/OS PER slip trap. This could be setup to log an entry to LOGREC or a trace every time a private or LPA module is entered. However SLIP traps can stop your z/OS system with their CPU overhead, and must not be used for a z/OS-wide hunt for module users.
Software products such as TADz and P-Tracker take the hard work out of tracking module and software program usage. However they're not the only solutions. With a good knowledge of z/OS, there are other options you can use to find out who is using a module, and how often.