Memory Usage Update

A previous post described some of the techniques that we have used to reduce the memory requirements of Gen applications. One of the approaches that we have been trialling is to delay the allocation of local view storage for generated C code.

The generated C code for an action block will define a structure for the local views and the memory for these structures will be allocated for all of the action blocks linked into the load module when the load module is loaded, even if the action blocks are not called. Note that this is only strictly true for action blocks that do not have any uninitialized local views.

We have developed a source code post-processor that re-defines the local view structure so that the memory is only allocated when the action block is called and is then freed on return from the action block.

This has dramatically reduced the memory usage of the load modules, especially for co-operative servers that remain loaded by the transaction enabler.

We have been running with the modified code for a month now with no unexpected results. The benefits are a big reduction in the overall memory requirement. This allows us to improve performance by a combination of having more load modules loaded into the TE thus reducing program load time, freeing up memory that can then be allocated to the DBMS and fewer page faults.

User Defined Functions

In my last post, I described how you can extend the Gen runtime with user defined functions.

The post described how the use of user defined functions can simplify action diagrams because you can nest functions and also the input and output parameters can be specified using any attribute view of the same domain, whereas imports and exports to action blocks have to use the same entity type or workset views.

It is worth pointing out that whilst this simplifies the action diagram code, the underlying generated code will still need to ‘map’ the input and output views to additional data items that have the same length as the function’s parameters, and hence the fact that the action diagram code appears not to have this extra ‘mapping’ does not mean that there isn’t some runtime overhead associated to this data mapping.

To add a function, you have to code the logic in the same way that you would for an external action block and place the code into a library so that it is available at link time. You also have to define the function to the model, specifying the input and output parameters. These are more restrictive than for a normal action block. You are limited to one output parameter and up to 5 input parameters, and group views are not allowed. Defining the function to the model is quite a complex process, especially if you are not familiar with the Gen meta-model, so you may want to take a look at the QAT Function Manager which automates much of the tedious tasks associated with defining the function. There are also various presentations and documents produced by CA on this subject, so I do not want to repeat them here.

Once you have added your function to the model and coded the logic in C, COBOL or your other target language, you are ready to start using it in the model.

Some of the functions that we have added are:

IET_FIND. This is a replacement for the standard Gen find function which allows you to search a string using wild cards (like %) and other search operators.

IET_GET_ROW_CLICKED. This is a function that returns the clicked row given an x & y co-ordinate returned by the Gen GUI right mouse functions and a list box control. Gen only provides the x & y co-ordinates and to get the row selected, you need to adjust for the scrolling of the list box, the font height and whether the user has changed the default DPI setting in Windows.

IETeGUI functions. For IETeGUI we have developed a range of functions that allow the developer to fine-tune the GUI at execution time, for example, dynamically changing a list box column position, visibility, headings, etc. Using functions greatly simplies the coding when you want to provide detailed control over the GUI runtime.

Extending the Gen Runtime

I thought it would be worthwhile describing how we have extended the Gen runtime in the development of our tools, but first we should clarify what we mean by the ‘runtime’.

A high level language (including Gen, but also languages like C, COBOL, Java, .Net, etc,) simplifies the developer’s task by providing functions that are inherent in the programming language. Examples include printf in C, STRING in COBOL and the Gen concat function.

The actual implementation of the function by the compiler (or code generator) might be via a macro, code generation (i.e. the COBOL or C compiler generating assembler instructions to implement the statement) or via a call to a language runtime module. To the developer, it does not matter how the function is implemented. For example, the CA Gen concat function is implemented in generated COBOL code whereas in C it is implemented via a Gen runtime function call.

So what is the difference between a language’s runtime functions and those that you might procure from a 3rd party vendor or develop in-house?

From a developer’s perspective, in some languages (like C) the code to use a language function will not differ much from the code to use a 3rd party function, whereas in others (like COBOL), the in-built functions are typically referenced via language statements (i.e. STRING) whereas other functions would be used via a CALL statement. Typically a function that is invoked via simple language statements is easier to use than having to implement a separate program call to invoke the function.

In CA Gen action diagramming, the same is true. Compare the following:

SET c to numtext(substr(concat(a,b),1,5))

With the following which uses alternative routines for numtext, substr and concat.

SET map_1 TO a
SET map_2 TO b
USE my_concat
WHICH IMPORTS: map_1, map_2
WHICH EXPORTS: map_3
SET map_4 TO map_3
SET start to 1
SET end to 5
USE my_substr
WHICH IMPORTS map_4,start,end
WHICH EXPORTS map_5
SET map_6 to map_5
USE my_numtext
WHICH IMPORTS map_6
WHICH EXPORTS map_7
SET c to map_7

In the first example, one statement could achieve the desired result, whereas if we replace the use of in-built functions with external action blocks that achieve the same result, we need a lot of additional code as well as special ‘mapper’ views that are of the same type as the import and export views of the EABs.

In the above example, I have shown an extreme case where each function has its own set of import and export vies that are not shared between the functions.

Gen allows you to develop your own functions, so you could then code the above as:

SET c to my_numtext(my_substr(my_concat(a,b),1,5))

We have made use of this capability to add our own functions into the Gen model and hence simplify our code, both in terms of the number of statements needed to implement a given task, but also reducing the number of ‘mapping’ views needed.

In a subsequent posting, I will describe some of the functions that we have developed.

Windows Timers in Gen GUI

Watching paint dry is not everyone’s idea of an interesting day at work, and for certain applications this is extra tedius when you have to click a ‘refresh’ button just to see how dry that paint has become.

Various functions in our applications submit background tasks and exhibit this mindless click-to-see-if-it-has-finished scenario. After much RSI and boredom we decided it was time to implement auto-refresh and timer events in the GUI so that at least you didn’t have to click the tired old mouse so much.

To achieve this in our Gen GUI we developed an EAB which we call to register and start a timer event in Windows (WM_TIMER messages). Basically this is a Windows event interception routine which processes the timer events we are interested in, and any other events get passed on to the default window event handler (the Gen GUI & Windows runtimes).

To make something actually ‘happen’ when the WM_TIMER event is received, TIREVENT is used to trigger an event in the client PStep, and the timer event is deactivated again, as far as Windows is concerned.

The event in the PStep can then do its' thing, and if it wants to repeat the timer again (like a regular auto-refresh) it simply calls the EAB again telling it to reactivate the timer.

Pretty simple stuff in principle, but a big improvement for the poor user and their aching click-finger.

Tabs Controls in Gen GUI – an easier approach

Let’s face it, designing neat but sophisticated applications using Gen’s GUI can be challenging. The temptation to resort to OCX controls to get a good-looking app is high, but at the expense of difficult and cumbersome coding in your client PStep.

In particular, a user-friendly way to simplify groups of activities or information in a GUI would be to use a Tab control. If you were coding any other language (VB, C++ etc.) this would be an easy choice, but in Gen the result is a huge client PStep which contains all the logic for every tab page.

Better would be to split each tab page’s logic into a separate client PStep, but this simply is not possible using OCX controls in Gen.

To get the look and feel of a tab control, we’ve developed some cheats which allow exactly this… very simply, each tab IS a separate client PStep, but the tabs are actually a combination of buttons and bitmaps, designed to look like tabs!

To reduce the complexity of the Dialog Flows required, a central non-screened PStep is created which has the Dialog Flows (links) from itself to each of the tab PSteps. There are no flows between the tabs themselves. If a tab button is pressed, it simply flows back to the non-screened PStep with instructions on where to flow to next.

There are of course a few tricks to make this look slicker, including positioning and sizing of the individual tab windows, to give the impression that they are in fact the same window. Originally we achieved this by simply getting the window x,y,w,h and passing it around on the flows, but now we take advantage of IETeGUI’s resize features specifically for this purpose.

Find out more about IETeGUI here... http://www.iet.co.uk/Products/IETeGUI

Also, the tab buttons tend to ‘float’ around a bit, compared to the designed position in the Gen Toolset meaning that the buttons wouldn't always be perfectly lined up, so we put logic into each client PStep to explicitly place them at specific positions so they line up nicely. Again, we now make use of some automatic functions in IETeGUI to help with this.

Another aspect of the illusion of tabs is the border of the tab control. Our original implementation simply used carefully designed background bitmaps for those windows, and the tab buttons were placed so that they sat along the top border of the tab. If your windows are not resizeable, then this works perfectly. However, for resizeable windows, you’ll want to resize the tab’s border as the window gets resized. To achieve this we originally wrote some C code which intercepted the window’s redraw/resize events and drew the border dynamically, but now use some built-in features of IETeGUI to achieve much the same thing.

Avoiding too much logic in Window Open events

When flowing between windows in a Gen GUI application, it is good to have the window pop-up quickly, even if the data is not populated immediately, as this gives the user the feeling that something is happening.

If you put too much logic inside the Open event of the window, this perception changes because the window does not get physically displayed until the end of the Open event, thus giving the illusion that the application has hung, crashed or simply disappeared.

We’ve found that a good practise is to minimise the amount of logic (especially server calls) performed in the Open event, and if necessary use a TIREVENT within the Open event to trigger another event where the longer processing is performed. The window will then get displayed after the Open event so the user can at least see it is alive.

Coping with the 32k view size limit

You are likely to have come across the 32k view size limit for procedure step import and export views unless you are generating for platforms that do not have the limit like pure windows clients, Java or .Net.

When the user needs to scroll through simple lists of data, a paging method that uses a ‘start from’ field is usually adequate, and if you want to display a lot of data on the client, you can still repeatedly call the server to retrieve batches of data, again using a hidden ‘start from’ field.

However there are occasionally situations where the above approach does not work. Examples that we have come across include impact analysis on the server which returns a set of object ids based on a complex trawl through the database. You need to return the complete set of data and the 32k view size limit can then result in truncation of the results.

We have therefore developed some techniques for handling this.

The first technique is to limit the export view to the identifiers of the returned rows. You can then call a different server to retrieve additional details, for example, the first server returns 1000 identifiers and then a second server is used to retrieve the additional details in batches of 100.

The second technique is to reduce the number of attribute views in the export view. For each attribute in a view, Gen adds 1 byte for the state flag and for generated C code there is an additional null terminator byte. Gen also adds an additional 6 bytes for each attribute in the export view. Consider the situation where you need to return ten 1 byte attributes in a group view. For C code, each row in the group view will occupy 90 bytes. However if you were to string the ten bytes together into a single 10 byte field, then each row would occupy 18 bytes, thus allowing a much larger group view size.

The third technique is to first call the server to populate the result set and return the first batch of data. Any remaining data is stored in a temporary location, for example a file. If the results set is incomplete, the client can then call the server again repeatedly to retrieve the remainder of the data from the temporary location.