Thursday, December 31, 2009

Doing my part for ER/Studio

There's not much out there on SAX Basic when used in ER/Studio.  Or there's not enough.  This is the best way to determine when there's a gap in the Internet.  Shock!  I know, hard to think of, but there are gaps.  I consider it my duty to fill in.

On the subject of incorrect documentation:

As taken from

Split Function
Syntax Split(Str, [Sep], [Max])
Group Miscellaneous
Description Return a string array containing substrings from the original string.

Parameter Description
Str Extract substrings from this string value.
Sep Look for this string value to separate the substrings. (Default: " ")
Max Create at most this many substrings. (Default -1, which means create as many as are found.)
See Also Join( ).
Example Sub Main
Debug.Print Split("1 2 3")(1) '"2"
End Sub

This does not function in ER/Studio Macro Editor.  This is the following excerpt:

Debug.Print Split("xx", yy", ",")(1)

This causes the error: Expecting an existing scalar var.

This is a major drag 'cause I need this function right now.  I'll write one.

In checking the SAX documentation that comes with ER/Studio, I see its not present in the documentation, nor is the Join function, but the Array function is present.  Note that the Array function cannot be used as a argument, which is not expected behavior for a function.

To fill in the function, I'll grab the code from  I suppose I could have written it, but that would be an annoying waste of time.  It does need adjustment to fit the SAX ER/Studio hybrid, though.

The signature from freevbcode is:

Public Function Split(ByVal Sin As String, sOut() As String, _
Optional sDelim As String, Optional nLimit As Long = -1, _
Optional bCompare As VbCompareMethod = vbBinaryCompare) As _

To fit in line with SAX documentation, the output is not a passed value but instead a return value.  I'm not sure SAX takes things as ByRef.  We'll try it, though I like it as a return value better.

Conversion error #1: InStr function in SAX does not take vbCompareMethod as an argument.  This is another difference between VB5 and ER/Studio SAX Basic. 

Wednesday, December 30, 2009

The Evolution of the Perfect Function

It's not perfect, it won't be, but in my efforts to model appropriate coding behavior, I'm taking this name transformation tool as a test case.  SAX Basic has many limitations, even beyond the limitations of Visual Basic.  But writing good code should be a skill outside the limits of the language used.  As legacy code gets older and stiffer, we need to become archeologists as well as experts.  We forget the advantages of older languages.  There is a great body of knowledge out there for older languages, and experienced developers.  The language is also dead, at least in terms of evolution, and has the advantages that Latin has in the use of scientific technical writing.  Interestingly, the usage of the older computer languages does not have to be dead, proper apps can be written, and trusted to be stable, not subjected to constantly evolving compilers and interfaces.  They are locked in.  This is always seen as a weakness by developers, but having a trustable fixed framework to work within is a freedom for exploring the higher levels of design.  If you are constantly adapting to new languages and interpreters and compilers and standards and technologies, when can you practice the fundamentals?  This is what I see in most developers.  No sense of risk, or exploration, and no interest in the "old" ways of doing things, and no recognition that programming requires experience and higher orders of thinking than just what can be learned in a book.

Soooo, anywhoo, The "Abbreviate Physical Names.bas" program has evolved.  The Dialog section has been converted almost completely to programmatic code.  Here's a sample:


This is required to be declarative within the limits of SAX Basic, but outside of that limitation I'm able to create flexible code by keeping all values in an array.  Correction: I have moved the array to a class.  So many developers are either old-fashioned VB3'ers and have no understanding of class design.  The don't see the point to it.  My main reason for using a class is state.  Statics only can live within subs and functions in VB.  They can't properly be shared, though I suppose a nesting of functions with control codes to direct control to the appropriate function, turning it into CICS.  That's not really desirable.

So a class has its collection of attributes which I see as really being shared statics.  Also by using a class some tricks can come in handy.  some methods can return this or self, or in VB Me.  This way a series of functions could be chained in a single function call, simulating a sort of stream.  I haven't had to use this yet, but its nice to know that its there.

Another clever trick to push a stream of methods into a function call is by having control methods return a 0 and summing the result, with the final method returning the proper value.  Ugly and system dependent, but remember we are dealing with old systems, and so there is no shame is writing code that is dependent on idiosyncratic behavior.  As long as a behavior is repeatable there is no expectation that that behavior will change.  So documented or undocumented, there is no difference.

 It's amazing what the ability to maintain state of a variable across several different methods does.  Philosophically the difference is significant, but lost to most older VB developers, even though class functionality has been around quite a while.

The code is much better.  The dictionary of terms loads automatically to reduce steps.  The macro will probably be called every time you change an object's name, so it is good for it to be a simple set of steps.  The dialog items have much more sophisticated placement and can flow vertically or horizontally.  It will take some improvements for it to be less cludgy.  The cancel and Ok button have been resized and placed according to Microsoft standards, other standards have been adopted for a slicker appearance.  The use of optional parameters is heavily depended on since only so many function calls per Dialog statement can be called.  By explicit naming of parameters in function calls, a function becomes a highly flexible command interface that would normally be many separate functions.  In typical OO programming it is considered desirable to break things out into 1 function per atomic action.  But when dealing with old interpreters there must be a different way.

Public Sub buildLayout( _
id As Integer, _
itemType As itemTypeEnum, _
w As Integer, _
h As Integer, _
title As String, _
TakeXYFromID As Integer, _
Optional OffsetXInd As Boolean = True, _
Optional OffsetYInd As Boolean = False, _
Optional stereotypeItem As StereotypeItemEnum = SIE_STANDARD _

I like to list the parameters vertically, silly me.  The last 3 parameters are optional, so calls, if explicitly named, can be made without any positional knowledge of the others, true loose coupling.

Here is an example of a call intended to create a text box and have it flow down the screen from the previous control:

lc.buildLayout STATUS_TXT, IT_TXT, 200,1, "Status", PREVFROMCURRGRP, OffsetXInd:= No, OffsetYInd := Yes

The last 2 arguments are ignorant of the other optional parameter stereotypeItem.  This is standard overloading behavior in C++ and Java.

But here is where it's different.  Here's an alternate use of the function:


This is an example of building a Cancel button at the bottom and far right, 2 buttons indented in from the right.  I do the same for the Ok button, except only 1 button's width indented from the right.  Notice that this function call is unaware of the Offset parameters.  No goofy dummy values have to be sent, and no commas have to be stuffed.

As a function evolves, we tend to want to tune it and tweak it, rather than make all knew functions.  This is according to natural human behavior where we develop a singular process, say at a manufacturing plant assembly line, and we forcibly evolve it.  Adjustments are made as a series of deltas, always referring back to the base design.  If the base design is good, it will lend itself to this sort of evolution.  It may look scary and a bit of a monstrosity, but we are not Gods and this is the best we humans can do.  This is far better than scrapping well-designed code in favor of totally new efforts, discarding all the experience embedded in the old.

This goes against all the training out there, but it aligns perfectly with the way human functionality works.  We do not reinvent the "walk" function every month.  When we seek to "run" we base our learning on the concepts in "walk".  Even "stop" is similar to a walk at 0 mph.  Whether we "walk" or "drive", we use the same search techniques for sign identification, follow the streets laid out for us, estimate according to the same distance calculations.

OO thinking would have us worry about "has a" and "is a".  Do you when you select a device or entity?  No.  Humans merely name objects and use them according to their nature.  If it does function A, we use it for that function.  If not, we seek out a widget which does function A.  We adapt our skills and behaviors around the limits of the function implementation, until such time as we find a better implementation, then we readapt.

This is how we should code.  Over use of inheritance to try and separate out what subtle differences makes for duplication of effort across classes, and then the creation of a zillion utility classes then referenced from these split classes.

I say, take the old tools, and beat them until they function according to the nature of men's minds.  Stop grabbing up new tools that are in some ways more primitive that those before it.

12/30/09 - Update:  This function doesn't work so well.  I ended up multiplying the output by 1.56.  But it still  frees me from manually coding sizes each time I enter new text.

Monday, December 28, 2009

Measuring Text in COM (SAX)

One of the big rules: No magic numbers.  In the code, I have string labels, and I have pixel widths for those labels.  This is an anathema.  The computer should know how much space its going to use, right?  I know from past experience with Actuate that exotic fonts don't measure properly in the win32 API, so take the WYSWYG and toss it out the window for Kanjuu font or whatever that was that blew chunks in Actuate.  Gave me fits.

So I got it to work and you can too.  Measuring in .NET is easy, just say MeasureText.  Great, but now for the rest of us, off supporting those apps that aren't so cutting edge:

Public Function GetStdWidth(Text As String) As Integer
GetStdWidth = GetTextWidth(Text , "MS Sans Serif" , 8, fontitalic := No, fontstrikethru := No, fontunderline := No, fontbold := No)
End Function

Public Function GetTextWidth(Text As String, fontname As String, fontsize As Integer, fontitalic As Boolean, fontstrikethru As Boolean, fontunderline As Boolean, fontbold As Boolean) As Integer
GetTextWidth = GetTextSize(Text , fontname , fontsize, fontitalic, fontstrikethru, fontunderline, fontbold).cx
End Function

Public Function GetTextSize(Text As String, fontname As String, fontsize As Integer, fontitalic As Boolean, fontstrikethru As Boolean, fontunderline As Boolean, fontbold As Boolean) As SIZE
    Dim tempDC As Long
    Dim tempBMP As Long
    Dim f As Long
    Dim lf As LOGFONT
    Dim textSize As SIZE
    ' Create a device context and a bitmap that can be used to store a
    ' temporary font object
    tempDC = CreateDC("DISPLAY", "", "", 0)
    tempBMP = CreateCompatibleBitmap(tempDC, 1, 1)
    ' Assign the bitmap to the device context
    DeleteObject SelectObject(tempDC, tempBMP)
    ' Set up the LOGFONT structure and create the font
    lf.lfFaceName = fontname & Chr$(0)
    lf.lfHeight = -MulDiv(fontsize, GetDeviceCaps(GetDC(0), LOGPIXELSY), 72)
    lf.lfItalic = fontitalic
    lf.lfStrikeOut = fontstrikethru
    lf.lfUnderline = fontunderline
    If fontbold Then lf.lfWeight = 800 Else lf.lfWeight = 400
    f = CreateFontIndirect(lf)
    ' Assign the font to the device context
    DeleteObject SelectObject(tempDC, f)
    ' Measure the text, and return it into the textSize SIZE structure
    GetTextExtentPoint32 tempDC, Text, Len(Text), textSize
    ' Clean up (very important to avoid memory leaks!)
    DeleteObject f
    DeleteObject tempBMP
    DeleteDC tempDC
    ' Return the measurements
    GetTextSize = textSize
End Function

This took a little bit to work because most examples where for VB6, which has a few global objects, like StdFont.  No such type exists in SAX Basic, and it doesn't appear to be a simple structure.  One script referenced IFont, an interface that definitely ain't in SAX.

Another interesting thing is that VB can reference types defined later than their reference, but SAX requires them to be listed first.  I prefer that types be listed first, and I don't think you can have complex interdependent types in Basic since you don't have the concept of pointers.

I was very surprised when it worked.  It measures very accurately.  For now it assumes you haven't changed the system default fonts, but I may add code to fetch the current system fonts.  Since its a message dialog, there isn't much I have to worry about, but I will try to improve it.

The use of GetStdWidth(Text) in pixels pulls several hard-coded widths out, making the entire system more maintainable.  BUT as I usually do, I allow manual widths to be passed and not restrict all function users to accept the calculated width.  Typical hubris in programmers is to accept fully that the system works perfectly, or if it doesn't, then its not their fault and you can call Microsoft.  This avoidance of responsibility to the user infests the developer community.  Instead, I always assume the caller of my function knows more than I do unless I know better (;).   More desirable is to allow hybridized usage, passing in codes that request the usage of the system metric in combination with a passed adjusting constant, for cases when the system almost is right.  Even better would be to add caveats and wherefores to the actual internal function behavior, to say, "Use the windows function, UNLESS font is Kanjee, then take font size * 4.3 or something".  See what I mean?

Enhancing the Name Transformation Utility

I'm not giving up.  As I swim deeper into the details I recognize the dangerous territory I am in.  There be dragons!  I started with the UML, found it too weak at database design, switched ERwin 4.1.4, got curious about ER/Studio, tried it, liked it(!), switched, found the Naming Utility menu option, loved it(!), got nervous that ERwin might have newer stuff, found out it was as goofy as ever (typical CA), discovered the renaming script out on Embarcadero's script site (, ran it after much angsting, found it surprisingly better than the one that comes with ER/Studio in a few ways.

Wew!  So as I fiddled with the naming macro (, I began to wonder what the limitations of SAX Basic might be.  I don't like to invest energy in a tool only to find it substandard after much work has been done.  The company, Polar Engineering and Consulting, is probably still around, though web artifacts are not always the safest way to determine recent activity.  The new version (.NET) of WinWrap has some nice enhancements, though somewhat esoteric to a simple developer like me.  I'm curious about it, but I doubt I can snarf it, and it doesn't appear to be embedded in any vendor apps out there yet.  The free SAX version 6 is heavily present in all kinds of apps, so that's exciting!  I gobbled a reference on the stuff, though it didn't cover all the features available - most, though.  I'm not sure this is public domain, so be sure and tell all your friends and swamp the site!

Well, the language looks pretty effective and based on functionality and not following some goofy standard that Microsoft made up.  I lost myself in converting the Dialog in the naming macro to be a Java Layout Manager.  I'm still in there, but I've come out the other end finally.  I have probed the limits of SAX and found that it cannot generate dynamically constructed dialogs without using external win32 APIs, and that may be going too far.  I was able to at least reduce the magic number count by 80% at least and still going.  some things like control names have to be declarative and there's no real way around it, though I just thought of an idea:  I could "guess" the control index number in a function from the declarative statement that creates the control and capture it in an array of named key pairs, perhaps a collection?  Then I could drop the declared field names and code would refer by actual string values in a lookup to an index #.  Crazy enough it just might work!  What I'm most concerned with is the the use of string constants to reference Dialog controls throughout the program.  These cannot be checked at edit time or compile time, and they will get unmanageable if I exert too much more time.  I could convert them to constants, but these would have to exactly match the declared name in the generator portion, which cannot be codified.  So now there's a nice thing to break!  Plus constant definitions have to sit outside the function and declarative dialog section, so you will have a fun time keeping things in sync.

You can see that I got lost again.  But the outcome of my adventure has been an adorable flexible Dialog creation section.  It is of course HUGE, but only small portions need editing, like the declarative name.  Much has been pushed into an array of group definitions.  It is a single dimensioned array in the same way that a Java collection would have a bag of objects to layout.  A Dialog screen is not really (though kinda) a 2-dimensional space.  I see it more as a collection of loosely related controls.  The controls may be anchored below or beside another control.  Almost always they are related to a single other control, and not several others, so the relationships you see in 2D mapping don't apply.

The form still generates instantly and is very uniform and self-consistent in terms of padding and alignment, which are the sort of things which make a screen less trusted.  As a developer I hate to drag and snudge and smidge objects and then close one eye and squint, and ask 5 other developers (each billing at 100K) if they think they are lined up, only to think of a better label.  You should never avoid making positive changes to a screen because your worn out with pixel pushing.  A good label is worth gold and can save man-years (well, maybe not man-years) of errors because a user didn't understand what a field was for.

Here's a pretty pic:

I only included the part I've tuned up with generation code.  The title of one group is clipped, and I may pull in win32 GetFontMetric (hopefully it works better than it did in WinXP) to measure and check for max sizes before assigning widths.  What's nice is if everything is defined first in arrays then the declarative portion just has to refer to the array items.  One problem is that since the declarative can't actually know what's in the array at edit time, It has to be delicate to some degree, BUT I may be able to create dummies!  So, Group1, Group2, Group3,...GroupN.  These could be listed out in between the Begin Dialog and End Dialog sections, each with a set of OptionGroups/OptionButtons, PushButtons, etc.  CheckBoxes and OptionButtons have to be physically beneath the option groups and group boxes they are linked to, or at least OptionButtons do.  I'm going to experiment with checkboxes, because they don't interact with each other.  A good reason maybe to get rid of radio buttons. Hmmm.  Tab order would be screwed up, though, if they groups were laid out differently than their actual screen order.  

But notice the alignment: neatly piled left to right with identical spacing.  The bottom of the Text box is a little tight since the constructor for group box height doesn't know what I'm putting in the group, only the count.  For consistency I'll just add a pixel to all of them, though I think where check boxes/radio buttons are the last item in a box they would look better a little tighter, but this would require 2 height dimensions:  1 represents the space it allocates, the other represents the placement of the bottom border of the box.  Objects beneath should be placed after the allocated space, not in relation to bottom border.  But then there would be inconsistent spacing vertically between groups, even though groups aligned across all the way down the page. To think about.  There's no way I can think of to account for the difference of behavior between text boxes and check boxes visually.

Other things I'm fixing:
  • Sorting the transformation map.  It makes sense to at least sort the Logical-to-Physical mappings so you can spot dups or misspellings.  Easier to visually search.  Since sorting in memory might be a little difficult, unless I can find a function to sort, for now I'll sort off the flat file using .NET file functions.  Since I have Visual Studio installed, these functions are there, and I think these libraries can be installed from Microsoft downloads.  It's a bit slow so I'll look for an in-memory win32 way to sort.
  • Misspellings are almost all gone.
  • Indenting is cleaned up.  Spacing, too.
  • Some names are made more representative, but since there is no global search and replace in the SAX editor I have to be careful.  I edit in TextPad, but I don't like switching back and forth.  If I could alter ER/Studio's editor or replace the dll it calls with a new SAX editor....  Crazy, I know.
  • Fix the "Save" button so it doesn't ask you, it just does it.  Add a Save As button.
  • Fix the "Open" button so it just opens the last one, or open it automagically.  I'm poppin in here all the time so I don't want to keep opening-closing.  Maybe make it modeless?

Sunday, December 27, 2009

iTunes: Another Steve Jobs Protection Racket

Grr!  It's crazy!  iTunes does everything in its power to make music cost money and make sure you don't use any other tools.  You can build lists, but you can't export them in any meaningful format.  You can write out files to a Data DVD, but its in some cryptic folder structure to avoid any simple pull into another product.  You can't delete things because they don't want to deal with the hassle of phone calls when people deleted their paid for songs.  No FLACs, if you please, EVER.  Probably because its free.  Time to escape.

I need macros and plugins for this beast.  I'm stuck using it because it has some cool features and looks nice, which sadly is important to me.  ONX?  Help!.

What ONX is that is so frickin new: Multi-return values.

Some simple things that other developers will never sink to.  For one, return 2 or more values.   For instance, in SAX Basic I have the following construct:

Begin Dialog
GroupBox _
"Object Types To Include"
End Dialog

Beauteous, right?  Shut up.  Now, in SAX Basic, you can't functionalize code in a Begin Dialog/End Dialog section.  It's not really code, its more like code generator instructions.  But, for some reason you can put functions in it, which themselves contain code.  Yet it ain't pretty or quick to code.  ONX will assist in getting around this by generating on the fly the return of sets of data.  The above lends itself to the RECT paradigm.  In Win32 you often send a RECT struct pointer that has all 4 corners of a rectangle.  But Basic often misses this concept - that you might want to send a set of coordinates as a single argument.  So...

%<def_rtn_set_call(arraygroup, groupbox,="" h,="" posx,="" posy,="" title)%="" w+gspx,="">

Begin Dialog
GroupBox %<get_rtn_set(>arrGrpLayout, ARRAYGROUP, GROUPBOX, TRANSFORMATION_GROUP_ID)%>
End Dialog

Which would expand to the first code set.  An important point to point out (pointedly) is that the goal is not to necessarily reduce code to some obfuscated, unreadable gibberish.  It is to introduce controllability and a sort of weird maintainability and to make it so the coder only changes the code they need, everything else is cut and paste.  Here's a way that pulls out the redundant stuff:

Begin Dialog
GroupBox %<get_rtn_set(>arrGrpLayout, ARRAYGROUP, GROUPBOX, %IN1%
End Dialog

Much better.  To add a section, add a line to the WITH section.  But where's the Type Safety, you screech!!?  Ok, we'll add an enumeration:

%<enum(maindiag_groups type(diaggroup))="">


The "%IN1%" is saying take the 1st value of each line in the WITH section.  A better way might be ":?" or ":1".  Sort of similar to parameterized queries.

Begin Dialog
GroupBox %arrGrpLayout, ARRAYGROUP, GROUPBOX, %IN1%
End Dialog

Now the USING section will only tolerate an ENUM of type DIAGGROUP.  DIAGGROUP is a user-defined name.  MAINDIAG_GROUPS is our custom name.  Maybe it should say "Name:=MAINDIAG_GROUPS".

I just made this up, it could use some work.  User defined names may need to be unique from ONX functions, then again maybe not.  Not sure.  I plan to make several "what ifs" until I see a pattern.

Almost all older language constructs like "GroupBox" take comma separated parameters so this is a useful feature.  MFC has a similar disaster for Form transfer,  I recall.

For the OptionGroup and OptionButton sections, an extra construct is needed, a way to take stored metadata for each group that is defined in the runtime and pasteable as if it were a field name instead of a text of function call.

OptionGroup                                             .optTransformation

The ".optionTransformation" is really a string constant(!), but SAX Basic is using a generator syntax and never considers that you might want to variabilize the value to reduce errors.  Referencing this "field" from the DialogFunc% looks like this:

If DlgValue("optTransformations") = 1 Then

Now SAX decides to treat it as a string constant!  Very frustrating.  One solution might be:


OptionGroup                                             %

If DlgValue(%<var(opttrans).asconstant%>) = 1 Then

I'd like to add OPTTRANS as a metadata imaginary extension to the real SAX array arrGrpLayout, perhaps renaming it.  So, part of the values in arrGrpLayout live in real-space in the code, the others live in a macrotized world overtop of the code visible to SAX until post-generation.  A big-ass precompiler.  But addition of the arrGrpLayout has to be synchronized with the metadata version or we can't get proper errors before actually running the code.  It would be awkward for the macro layer to actually simulate a run of the code.  Post code could be put in by the macrotizer to test item count in the real array and compare it to the macro view.  Or perhaps at the first reference to either the real layer object or overlaid object, code is generated to validate the count.  In sections like "Begin Dialog", it would be difficult, the functions called from, for example, the GroupBox section would have to be made to include verification code, which might be awkward.  These could all expand at QA time, perhaps there is a double pass QA.  One could be automated, testing counts, etc.

Very hairy stuff, but fun.

Why Basic is not always the inferior choice

Try this in almost any other language:

Private Function CalcGrpRow(Optional Reset As Boolean = False) As Integer
Static CurrRow As Integer

CalcGrpRow = CurrRow
CurrRow = CurrRow + 1
End Function

I'm not even sure what you call this.  In C++ you would do it this way:

int CalcGrpRow(int reset = FALSE) {
  static int currRow = 0;
  return currRow++;

The post-increment operator works here.  Another way:

Static CurrRow As Integer

CurrRow = CurrRow + 1
Return CurrRow - 1

That's just ridiculous.  But there may be a VB version that supports the return statement but not the static assignment.  So that's what you would get.

It's interesting that the documentation at is either incomplete or a different version.  No version is present in the doc (ALWAYS VERSION!)  But there are some nifty features I've only been able to find by experiment, like Optional parameters and ParamArray parameters.  I need to find better documentation.

Functions you won't see on MTV:
CallersLine: Tells you what function, macro and linenum and line text called you, as far back as the call stack goes.  Code that can identify its caller is (to me) one of the next elements of computer evolution.  People take into account their callers when responding.  Their response evolves as needed.  It anticipates needs, and recognizes when legacy support is needed for older callers.

CallByName:  I can't say more.  Tooooooo cool for school!  You won't find it in Computer Science!

Others with honorable mention: Choose, Clipboard, DDE___, Redim Preserve, Array, ParamArray, Eval, For Each, Like - does the In operator work? - , MacroRunThis, Select (mucho better than a switch), ShowPopupMenu, With (very useful), all the Dialog functionality, declare win32 functions, providing Window handles, conversion to and from Unicode to DOS.

Come'on, Basic is cool!  I know Lisp and Prolog have more mathematical constructs and complexity, but I can always count on Basic to take a common-sense approach to a very complicated idea.

How 'bout this little ditty?  Have you ever called a parameter by name?

i = CalcGrpRow(Reset:=True)

How flexible is that?  And make all the types variants or Any.  Though I'm not sure what the limits are on Any and arrays.  I wish the Overloads option was available, but not yet.

I'm looking on ER/Studio's SAX Basic Help for v6, and it still doesn't mention Optional or ParamArray!  Very strange.

In VB 2008, I see all the goodies: Lamdas , Short-circuit evaluation AndAlso/OrElse, Shared (class static), MustOverride (virtual = 0), Shadows (no C++ equivalent!), try and finally control (I think), the "return" statement (which is nice), overriding of + operator (much easier than C++), extension functions, partial methods for better generator support (a big step forward), declaration assignment to properties (stolen from the pages of C++) -

Dim product1 As New Product With {.Quantity = 100}

Let's see, what else....Conditional Compilation, the Filter function, Queries (!)

Check this out:

Dim customerList1 = Aggregate order In orders _
                    Into AllOrdersOver100 = All(order.Total >= 100)

Nice, but wouldn't analytics be AWESOME!  In Oracle this greatly enhances the flexibility of functions, the ability to look at data through a floating window, able to look backward and forward from the "here" point as it moves through the data.  Hierarchical, too.  and Pivots!  I want it all!

Delegates, for supporting the oldest C code in the book, function addresses.

Nullable types, Generic Classes (i.e., templates), these are nice.  I see indicators as three-type fields.

Dim ridesBusToWork3 As Nullable(Of Boolean)

The HasValue method is nice.  And LINQ is interesting.  Interfaces, inheritance.

Threads, fairy easy:

Dim TestThread As New System.Threading.Thread(AddressOf TestSub)

Which must mean some mutex support of some kind.

VB 2010, I'm peeking at.  Unions, looks very ugly.

Structure TestUnion
    Public i As Integer

    Public d As Double

    Public c As Char

    Public b As Byte
End Structure

Serialization - why did they make it so complex?  They're uglifying the code, turning it into the bastard child of C++.  Why?  So sad.

Covariance and Contravariance look like great ideas, but implementation is horrendous!

Interface ICovariant(Of Out R)
    Function GetSomething() As R
    ' The following statement generates a compiler error.
    ' Sub SetSomething(ByVal sampleArg As R)
End Interface

DynamicObject and ExpandoObject look great!  I love the name!

Events, Handles, I don't know when these came into the language, but I'll try to use them if I can understand them.

Anonymous types?  A little bizarre...

Dim sampleObject = 
    New With {Key .FirstProperty = "A", .SecondProperty = "B"}

I skipped to a What's New page:

Look at these Func(TResult) supports:

' Note that each lambda expression has no parameters.
      Dim lazyOne As New LazyValue(Of Integer)(Function() ExpensiveOne())
      Dim lazyTwo As New LazyValue(Of Long)(Function() ExpensiveTwo("apple")) 

Any clue??  I need english here.

Bit arrays with FlagAttribute.  Interesting.

Simpler initializers: Dim x() = {1, 2, 3}

Inline If operator: If(first, second)

Indexed Properties, not sure if just in C#, or if even necessary in VB.

And on it goes.  I think if set logic in the Query and LINQ sections improves or is extensible (!), then we may be on to something.  And would VB 2010 interface with ER/Studio?

Saturday, December 26, 2009

Lamba Functions: I just got it!

You ever have a Eureka experience?  I was flipping through VB support as it is currently.

Get this:  A lambda function is just a nameless function that you can pass about and declare inline and even return.  Its a new parameter type!  I'm such the big brain tonight.

Thank you.  Return to your TV sets.

Nice function: Max

Ok, my first publication!  This should be added to ALL languages immediately!

Public Function Max(x() As Variant) As Variant
Dim i As Integer
Dim highest As Variant
highest = Empty
For i = LBound(x) To UBound(x)
If highest = Empty Or x(i) > highest Then
highest = x(i)
End If
Next i

MaxOf = highest
End Function

Why "Max" for a name?  Well, Max is a standard SQL function that calculates the maximum across a data set.  Nulls are ignored, which is kind of the point of supporting Empty.  This allows negatives to be max tested, too, instead of trying to create some hypothetical minimum like -99,999,999,999.

Note that this works for variants, too.  The only addition I can think of right off is some "presorted" flag and a binary search, which for small sets is not too helpful.

In C++ this could be overridden to support other types, or templated.  It's probably already in the STL.  In C this should support some simple VARG stuff.

Support for stuff like MAX(int1, int2) and MAX(int1, int2, int3) would reduce the need to preform arrays.  I think VB6 supports inline arrays, as does C or C++.

I suppose a Min is in order.  The comparison operator could be overridden to support various other aggregate operations, perhaps, though I'm drawing a blank on what.  A dup search?  Averages? Means?

I must search the web for similar functions, and destroy them with my mind.  Hee hee!  Why this isn't a standard in languages like VB, I'll never know.  Do they take up that much space?  They could be in a math DLL, or part of a Uses clause.  There should be no limit to the number of functions out there available.  Perl seems to have that behavior, but I can't stand Perl.

SAX Basic; my redemption

This is a cool language. I've been out of the programming side of things for a long time, so this my first chance at a language so maybe I'm a bit overly excited.

Bugs so far:

  1. The editor doesn't have a Replace All function, so I'll settle for a text editor.  I'm look for an external editor more effective.
  2. The position of objects doesn't seem proportionally correct.  Option buttons aligned vertically are not the same numerical coordinates apart in order to properly appear equidistant.  They don't even match left to right.  The lower the x coordinate, the more compressed the coordinates become on the y-axis.  What kind of crazy vector system is this?
  3. Dialogs don't appear to be resizeable, but I can add a Win32 call for that ;)
Here's the cleanup I'm doing on the Dialog:

GroupBox 580,7,200,56,"Word Separation",              .GroupOfSeparationOptions
OptionGroup                                             .optSeparation
OptionButton 600,21,60,14,"No",                    .optSelSeparateWords
OptionButton 600,32,70,14,"Yes",                   .optSelDoNotSeparateWords

Doesn't quite look right here, but that's another project.  It may seem foolish to name the actual Option Buttons, but we increase code flexibility and increase natural documentationing (new word).  References will be as such: "If .optSeparation = .optSelSeparateWords..."  or some such.

I'm also rearranging the code block to map its order of object creation to the order in the display, for sh*ts and giggles.  I'm naming all the groups, even though they're never referenced.  Again, just crazy.  I'm correcting spelling as I go, this was written by some poor engrish student.  I refuse to publish code written in engrish.  For that we can thank Akira Iwata from J-SYS Sofware Co., Ltd., who wrote this April 27th, 2001.  For 9 years we have suffered words like Logacal and Phycial, and convertion (not so bad, actually), and  "PurseNamingList".  I just figured out he meant "Parse".  You know how many clock cycles my brain wasted trying to figure out what Purse meant??  Another favorite function name is "Listup".  Nice.  I'm changing it to UpdateList, for my own peace of mind.  Or "Separater".  I admit I like the idea of cleaning up English's foolish rules.  For the word "Separate", it should be "Separater", but it isn't, and they lost WWII, and they'll just have to accept it.

I've always hankered for the obfuscation of check boxes and radio buttons.  The fact that the interaction between them is different is a user GUI property, not a code property.  opt and chk should be the same indicative tests.  Though the selection of an opt excludes a set of other values is codish (new word), I'm going to pursue this line of thinking.  I may convert some of the opts to indicators.  To me an indicator can only be 3 values: True, NOT True, and unset.  The misuse of indicators to represent 2 distinct meanings is a design flaw.  For instance, the option "optSeparation" can be unset, "SeparateWords", or "Do not SeparateWords".  An indicator should never be between "SeparateWords" and "SeparateByVowel", or something.  These would be true options of a "SeparationType", unless you could do both(!).  Or "SeparateBySyllable", or "SeparateByUFOs".  I don't like the GUI interface to be too closely tied to the code, but I guess there's no way to completely separate.

Possibly, a group of radio buttons could be coded as separate ifs, or if elses.  As Ifs, it would be:

If SeparateWords then
End If

If SeparateByVowels then
   ..again separate
End If

and so forth...

Here the GUI could be changed to a series of check boxes, or a multi-select list, or a single-select combo with free-form text!  Any which way lets the code live on without changing, though all paths must be tested.  It could also be done as an If Else If, which will trap unimplemented options, a very important behavior:

If SeparateWords then

Else If SeparateByVowels then
   ..again separate

   .. Unrecognized separation option
End If

This is important because we may have added a GUI option and forgotten to code for it!  The QA team will mindlessly click on everything, and get the proper error message, instead of silent but deadly pass-thru.  With silent ignorance, the QA dude will search the output to see the result of the requested behavior change, never realizing that the code never did anything different.  And in the QA meeting, developers are puzzled as to why this is happening.  Another words, a big waste of time.

Still, you see that it makes the options truly ORs and not ANDs.  Which is probably a valid User rule that should pass down to the code.  I don't know.  If computers can evolve past OR and AND optionality, then the systems can be more self-managing.  Users change from single-select to multi-select, the code goes on.  Less maintenance!

We could have something like this:

If Not SeparateWords and Not SeparateVowels and NotSeparateOuterSpace Then
  -- error.

This requires that the code mirror of the GUI be updated for new options, or else an error condition results.  One way would be to have the GUI create a table of what it "means" and have the code section review that it did what the GUI "meant".

Here's what I mean:  If the GUI meant "User MUST select one and only one SeparationOption", then they would insert some codified logic in a table.

Each If match would seek the rules table from the GUI and delete where it met the rule.

At the end of the code segment honoring the GUI request, it would FOR Loop through the rules table, or just count the items.  If not a zero count, then throw an error and list the unmet obligations, like "SeparationOfOuterSpace" was not tested for.

Easy way I just thought of.  Each object on the Dialog must be processed by something.  Can I cycle through the objects on the frame?  There used to be a Tag attribute.  This could be set.  Or you could physically delete the objects!  Interesting, I will code, or try to.

I corrected the "Close" button to a proper "Cancel" button.  We must follow Microsoft's example.  It imbues trust to our users, for if we do not follow the rules, why should they?  And how much can they trust us?

And I'll convert the Execute button to an Ok button.  I like the clarity of "EXECUTE", but Ok is a standard, damn it!  Plus, the system close button works now.

I converted the Dialog to a function call, because functions are mathematically useful representations of objects.  The result of the function should be processed at the calling of the function, not within the buttons on the dialog.  A minor point about where control should be coded.

There is some capability to process the Dialog items.  The function "DlgCount()" gives me a count of items in the dialog from within the DialogFunc.  Items can be enabled by "DlgEnable()" referencing the name of the object.  Very metadata.  The "DlgEnd" statement lets us control the return value, cool.   I need to see if the DlgListBoxArray "list",lists$() works.  Currently the list of objects is contained in a Text box on the dialog, but maybe a list box is better?  But then we lose simple cut and paste and editing and commenting, ect.

To cycle through the objects, we can use 

For I = 0 To DlgCount()-1
    Debug.Print I;DlgType(I)
Next I

Interesting, WinWrap is version 6.3, localized by SBE6_32.  This may help extend the system.

Alright.  I've been enhancing the dialog portion to be extensible and work somewhat like a Bag Layout in Java.  Very silly, but fun.

Will update when done.

Friday, December 25, 2009

The Nature of Competition

We cannot expect vendors to compete with themselves, at least not wholeheartedly.  ER/Studio provides incredible macro support and by doing such they create some arena for self-competing behavior.  The naming macro they provide is a great example.  The desire of Embarcadero would naturally be that users use the Naming Standards Utility that they so graciously provided.  It is a selling point, I'm sure.  So to expect them to properly flesh out the naming macro would be foolish.

This is how the impetus for ONX came about.  There is a world of scripts and hacks and interfaces to be made that will never be in the interests of any proper vendor.  They politely provide the tools to customize their apps because it is a great way to sell to any level of customer.  Managers love to hear that the tool can be customized with Visual Basic macros if it is not to their liking.  It greases the palms and assuages the doubts considerably.

As I work on the naming macro I realize how powerful it is and how much it can be improved to support a really useful utility.  The macro already overcomes some of the limitations of the NSU that comes with it as a binary menu option.  It provides a list "unmatches" or failed matches, and allows correction at the time of discover and then you can save it back.  Since the interface is simply a multi-line text box I can edit and tweak and scroll through the pairs easily.  In the NSU the page up and down function is broken - such a simple thing!

I have already added a virtual run functionality so that I can see what would be renamed if I actually executed it. This is great.  I also added a parenthesized version of the "before" name so you can consider what you are doing.  Simple but great!  I'm going to add a logging function to save and date this activity.

By the nature of the simplicity of the CSV file I was able to put a note at the end of a new row explaining a possible issue with the FRM abbreviation, which could mean FROM or FORM.  It saved back to the map file without a blink.  That's the way computers are supposed to work!  And yet so much of systems block and deny any flexibility that might come from not restricting every keystroke of the user.  What is gained?

I'll add a sort button, and try and see if I can make the dialog sizeable, and smarter about the saving, reducing some keystrokes.  The mismatches should include some reference back to where they were found.  In an entity, an attribute, an index?  And what was the full string?

Next, it will check the FreezeName flag, and perhaps allow you to set it on the fly.  Who knows?

Also, the author, "JT" had originally put in a Merge check box, meaning after the naming was done, it would push the model to the physical, which may not be necessary in the version?  I'll check.

Vendors count on our laziness as developers and our disorganized approach to things to keep them in business.  Managers help by their abject fear and mistrust of their own developers, preaching "Buy-vs-Build", when so much crap comes from the vendor and never changes.  Like ERwin's macro language.  It hasn't changed one iota since inception nor has one line of documentation about it been provided.  Its also kept far from the designer, making it a search and rescue mission everytime you want to find the toolbox.  I suspect the macro language will be the difference that puts ER/Studio over the top.

Other features of great import: PK/AK renaming, group name behavior, validation of name order ("ed" words as suffix, "ing" words as prefix), name length checking, applying cleanup against rules/checks/triggers, sps, UDFs, compare to names in physical databases, rename database objects (!), verification of "ind" files that they are a proper datatype.

I'll put the macro up when I've got it perfect as a post.

Beginning the Autonaming fixup

Time for me to get off the pot.  We'll see how much development time my son gives me.  Here's the current screen:

It's actually better than the Naming Standards Utility in that it defaults to targeting physical.  I don't understand why the NSU defaults to targeting Logical.  Why?

I need to add some target objects: relations, PKs, and AKs.  For me, I always name the primary key PKONX_[entity abbreviation].  That way an error is more traceable simply from its name.  AKs are more difficult, but they should have a nice logical name?  Freezing may be a nice feature to somehow add, but no idea how.  Stuff in a control file, I guess.

I like the unmatch, too.  A quick add function here would be GREAT!  Just fill in a new value on the spot, have it verify that its not already mapped to.

For now I'm experiencing a lovely bug (8.0.3) where new macros don't attach to the My Macros folder.  I can only edit existing macros.  There must be a configuration or registry entry to control this.  First, where the heck does it save the macros I see in the Macro tab?  They can't be internal.
C:\ProgramData\Embarcadero\ERStudio_8.0\Macros\Sample Macros\Meta Data Management Macros.

Ah!  They have to exist in this physical folder structure:
- ProgramData\Embarcadero\ERStudio_8.0\Macros
       - My Macros
       - Sample Macros
             - Meta Data Management Macros
             - Model Layout Macros
             - Modeling Productivity Macros
             - Physical Modeling Macros

Not documented.  If you change the save target in the editor, your screwed until you figure it out.
I dropped my new object in "My Macros\Naming" and renamed the file "Abbreviate Physical Names.BAS".
I'l refresh...
Tada!  There she is!  Pretty as can be!

It already stores its information in the Registry which is cool.  I like INI files, but registry entries work too.
I wonder if there's any test runner documentation tools out there.

My first concern is performance; we'll see how well she performs as the system scales.  Basic is not known to be a powerhouse.  I may be able to tune it somewhat.  Traps would be good, for instance, upon saving an object, table/column names are updated if changed.  Can we trap a "with" event?  Hmmm.

Musical Database tools

ERwin 7.2 was a disaster.  It would not connect to SQL Server 2008 via db-lib, even though ERwin 4.1.4 does fine.  It also has no Physical layer.  It must have been a special version.  The readme said something about integrating with Visual Studio 2005 Team Edition.  I got ahold of 7.3 the full version so I can try that, but the interface didn't look so great.  Not much was new.  The macro toolbox was still its cramped little excuse, with no proper documentation.  History of changes was nice to see.  Undo functionality is great.  Scrolling works now too.  But the editors are still midgets and unmaximizable.  Descriptions can be enlarged, which is weird because they are a text box in a modal dialog.  The process manager looks cute, but I'm probably not going to be building a million models.  The tranforms are cool.  But I can't go back for so few improvements.  I can macrotize the naming on ER/Studio and thereby everything is fine, better than ERwin 7.2.

Still jonesin  for ER/Studio Enterprise.  It has multiple modeling tools.  I've got Embarcadero's RAD Studio 2010, which I may check out the c builder when I get that far.

Thursday, December 24, 2009


I'm stressed!  I have the latest (almost) ERwin version, with the Validator and the Model Manager (ModelMart).  But to not have the SAX Basic!  I'm in agony!  Though as I installed 7.2, I saw that it came with an API, which might be better in the long run.  Plus I have the Process Modeler and this is syncable with the Data Modeler.  It is almost too much capability (plus market share) to pass up.  Sigh.  I think I'm losing ER/Studio.

Hopefully, though, I'll get some additional targets with 7.3.  Not targeting Oracle and others is saddening.

Perhaps I'll install DB2 and test out ER/Studio's targeting....

Key features of AllFusion ERwin Data Modeler 7
* Synchronization of models / databases
* Automatic creation of database and reverse engineer
* Publishing of models
* Support notations: IDEF1x, IE, Dimensional
* Possible joint work group of designers (with the help of environment AllFusion Model Manager (ModelMart))
* Documenting the database structure
* Transferring the database structure (but not the actual data) from one database type to another

ERwin Validator 7.3:
Checking the structure of databases and models of CA ERwin Data Modeler
CA ERwin Data Model Validator - a tool for checking the structure of databases and models created in CA ERwin Data Modeler, to identify defects and

design errors. The flexibility of CA ERwin Data Model Validator is that you can make random tests, and analyze the separate tables. The product

complements the functionality of CA ERwin Data Modeler, automating laborious task of finding and correcting errors, while increasing the skills of

designers database, built through the training system.
Embedded functionality of quality assurance and testing of models make it possible to control semantics models at each stage of development and make the

necessary corrections, which helps developers to create high-quality models.
CA ERwin Data Model Validator allows you to analyze the data structure, keys, indexes, columns and relationships. Moreover, the decision will help to

display graphically the structure of the entire database, including columns with cross-references and lists of relations.