Several days ago I mentioned that I'm working on an idea that combines ANTLR and Windows Workflow to demonstrate "business process authoring for analysts".

In hindsight, that's a pretty bombastic thing to say. So instead, let's just say that the tool I'm working on could be used by semi-technical folk to author executable business processes. There are many other fine tools in this space; mine incorporates a few (IMHO) clever ideas, but I'm not breaking any monumental new ground.

I'm getting closer to having something for public consumption... I'm trying to walk that line between making it solid enough for folks to actually use of it, without crossing every T and dotting every I. I'm very eager for feedback and input, but I want a minimal level of utility before it goes out the door.

So if I've piqued your interest at all, stay tuned.

Oh, and by the way... .NET 3.5 expression trees are FANTASTIC.  Except that they don't serialize without writing your own surrogate. Bah.

In the interest of not scaring off employers of both the "current" and "potential future" variety, it's perhaps useful for me to follow up and clarify my comments originally made here. 

In general, I meant what I said... code reuse is usually more trouble than it's worth. Usually, you're better off not trying too hard to make it work, because you'll probably fail, and waste a lot of time. Even if you're smart, and certainly if you're not.

I come by this point of view honestly... I've personally failed more often than not in trying to build "The One Reusable API to Rule Them All". I don't think it's a big secret that most others fail at this, too. I like to think I've gotten better as I've aged... but it's still a tough nut to crack.

IMO, the characteristic most often desired in "software that shall be reusable" is FLEXIBILITY. Truly flexible code that is widely applicable across multiple disparate problem domains is a thing of beauty... and extremely rare. The fact that any such code exists isn't the telling point; the fact that it represents an infinitesimally small percentage of all code ever written is.

Most "flexible" code I've personally witnessed was anything but. I've concluded that flexibility (and the goal of wide-scale reuse) is wildly overestimated as a desirable (and achievable) code characteristic for most classes of software.

So I say all this to say, reusable software is hard. It has instant sex appeal for non-technical manager types (which makes it even more dangerous), which is probably one reason it perpetuates. And certainly, there exist many fine counter-examples in the wild (none of this really applies to SDK vendors, obviously).

But I think the world would be a better place if we software types focused on building for today, and not For All Time To Come. Try it, and use your extra free time to read up on refactoring and test-driven development... there's your flexibility, cowboy. 

I understand that "sorry it's been awhile since I posted on my blog" posts are considered bad form by the blogging elite... oh well.

I started a new job last October, working here. Very interesting company, very interesting work (rules engine technology)... but they were quite eager to throw me immediately into the deep end of the pool, so I haven't had time for much of anything since I started. Just now starting to find some energy for endeavors beyond, like this blog.

I've got an interesting new project idea percolating... I'll be posting more on that very soon. I also hope to have a CodePlex project up soon with some prototype code.

As a teaser, think ANTLR + Windows Workflow = "business process authoring for analysts".

ASP.NET as WF host is an important and useful scenario for WF adoption, but there are some interesting details that are worth understanding before you tackle this yourself.

The first thing to understand is that WF itself imposes no specific threading model; rather, it must be configured to conform to any specific threading requirements of its host process.  This means that WF can be hosted in a single-threaded console or WinForms app, and also can scale up to be hosted inside ASP.NET, with all of its multithreaded goodness (thankfully hidden under the covers from the average web developer).

But there's the rub... ASP.NET uses the .NET ThreadPool to dispatch request processing.  The default WF scheduler also uses the ThreadPool to execute individual workflows; in combination this would be bad, as it results in 2 ThreadPool threads consumed per web request.  What we need is a way to re-use the ASP.NET thread to also execute the workflow.

Enter ManualWorkflowSchedulerService.  As opposed to the DefaultWorkflowSchedulerService, the manual service grants explicit control over which thread is used to execute a given workflow.  Its usage is a bit goofy... there's an extra call to ManualWorkflowSchedulerService.StartWorkflow() that comes after calling Start() on the workflow object itself.  But it is the thread on which StartWorkflow() is invoked that executes the workflow... so, a necessary evil.

Okay, all of this is fine, and frankly already well documented.  The purpose of this post is to highlight a few subtleties.

When you create an instance of ManualWorkflowSchedulerService, you must specify whether you're using active timers, or not.  The idea here is that, for workflows that might go idle (as a result of DelayActivity, or any other activity that implements IEventActivity), the manual scheduler itself cannot know when to resume such workflows (as the default scheduler can do).  So you have two choices... choosing "active timers" means a separate thread will spin up, periodically check for expired activity wait timers, and resume workflows with expired timers.  Choosing "no active timers" means you must write extra code to check for expired timers, and manually resume them when necessary.

The primary issue with active timers is that, since two threads are now involved, it's not very difficult to create a race condition where your primary ASP.NET processing thread completes before the active timer thread wakes up your idled workflow and allows it to complete its work.  The resulting problems are implementation-specific, but not good regardless.

The way to solve this problem is naturally to use some sort of thread synchronization mechanism to ensure that the main ASP.NET thread waits on the active timer thread to restart your idled workflow.  See the sample code referenced at the end of this post for an illustration of the issue, and one potential solution using thread events.

An important observation here is that ManualWorkflowSchedulerService, used in this way, implies synchronous execution of workflows.  This is counter to the relative "fire-and-forget"-ness of the default scheduler.

On the other hand, if you don't use active timers with the manual scheduler, it is *your* responsibility to detect expired workflow timers, and to then resume the idled workflows (by calling WorkflowInstance.Resume() ).  This generally means spinning up your own secondary thread (might as well just use an active timer) or periodic polling from your main thread (ick).  Again, see the sample code for an illustration of the issue and one possible resolution.

A final note... given the synchronous nature of ASP.NET request processing (setting aside async ASP.NET handlers for the moment) I can't generally recommend the use of async activities in workflows used inside ASP.NET.  By all means, use WF to define processing logic for HTTP requests inside ASP.NET.  Just tread lightly with the use of DelayActivity, HandleExternalEventActivity, etc. inside such workflows.

Here's the sample code I refer to above, that illustrates what I'm describing here (requires VS.NET 2008 beta 2).

Enjoy!

http://www.infoq.com/news/2007/07/worthless-code

This parallels my general take on code reuse...

Josh's 1st Axiom of Code Reuse

In order to be reusable, code must first be good.  Since most code is not good, most code is not reusable.  This means you.

Josh's 2nd Axiom of Code Reuse

Most of the time, idea reuse is infinitely more beneficial and productive than code reuse.

Josh's 3rd Axiom of Code Reuse

Most of the time, don't bother... it's not worth the trouble.

...and it's incredible.

http://www.amazon.com/Essential-Workflow-Foundation-Microsoft-Development/dp/0321399838/ref=pd_bbs_sr_2/104-2057410-5880735?ie=UTF8&s=books&qid=1183209452&sr=8-2

A few Amazon reviewers poo-poo it for being too esoteric and not dealing with "how to get work done in WF", or something.  This is definitely not a WF cookbook.  But the explanations behind bookmarking and continuations as the underpinnings of the WF architecture are fabulous.  Quite simply, if you don't understand this stuff, you don't understand WF.

The more I dig into this technology, the more fascinated I am.  The service model, the support for arbitrary execution semantics (but lack of preference for any baked-in ones), the multiple abstraction levels at which you can participate, the simplicity and yet power of the bookmark metaphor... it's really excellent stuff.

If you have even a casual interest in "next generation programming models" (as the tagline goes), read this book.

Well now, this was a fine how-do-you-do on a Wednesday morning...

http://footheory.com/blogs/bennie/archive/2007/05/12/simple-application-extensibility-with-wf-rules.aspx

Holy cow!  Turns out the rules engine that ships with WF is usable in your own application, even if you're not using workflows at all!  This is incredibly cool.  A little background for the uninitiated...

Windows Workflow is a technology that ships with .NET 3.0.  It enables you to express program logic and control flow visually, in what looks a bit like a Visio diagram.  It directly enables things like continuations and passivation in your software.  It is, in general, a Very Big Deal... though it usually plays second fiddle to WPF and WCF when folks start discussing .NET 3.0.

One of the features of WF is a business rules engine that can be used to author and execute rules against workflows in your application.  The rules work against internal workflow state, and produce side effects during execution to modify that state in some meaningful way.  For example, your workflow might expose properties such as AccountName or OrderAmount, so a simple rule might be (in pseudo-code):

if this.OrderAmount > 100

then this.Discount = 10%

else this.Discount = 0%

The idea is that you express such rules out-of-band from your code, which provides really nice maintenance and visibility benefits.  Since rules tend to be the core value of a software system, they're best not buried inside layer after layer of C# code where only a programmer can maintain them.

You get an out-of-the-box editor with Intellisense for rule authoring.  In addition, there are standard hooks in WF (through the use of PolicyActivity, typically) for invoking the rules at runtime.  This all works great assuming you're authoring and using workflows in your application.

Okay... so here's the neat part.  You can use this rules engine even if you aren't using workflows.  There's a great sample on the WF community site that demonstrates how to re-host the rules designer in your own application, as well as how to use alternate storage mechanisms for rules (by default, rules are saved to a file in your workflow project, and the file is compiled as a resource).  But, that code is still relying on workflows and custom policy activities to execute rules.  So the code from this sample is used in the other sample found at the link I mentioned at the top of this post to demonstrate rule execution without workflow involved at all.  I also re-purposed the base sample code to whip up my own demo app.

Here's a screenshot of rule authoring against my own "User" type (which has properties like Name and Age):

Notice the full Intellisense support for my custom type... this makes rule editing a breeze.

Once you edit and save your rules (storage is just a single table in the SQL database of your choice), you can then reference the rules (sets of rules, actually) by name at runtime, for execution.  Here's a code snippet:

// create my user object...

User joe = new User();

joe.Name = "Joe";

joe.Age = 35;

joe.Valid = false;

// next several lines execute the ruleset called "User2"...

RuleSetService svc = new RuleSetService();

RuleSet ruleset = svc.GetRuleSet( new RuleSetInfo( "User2" ) );

RuleExecution exec

    = new RuleExecution( new RuleValidation( joe.GetType(), null ), joe );

ruleset.Execute( exec );

// okay, ruleset has executed, let's see what the outcome is...

Console.WriteLine( joe.Valid.ToString() );

And that's it... about 4 lines of code to invoke a ruleset at runtime.  And again... no workflow in sight.  Not that I have anything against workflows, it's just nice to know you can use the rules engine by itself if/when needed.

I've uploaded my own sample application here.  Grab it, unzip, restore the database backup, check your config file connection strings, and party on.  There are 5 VS.NET 2005 projects included... 4 of them are modified versions of the code found in the previously mentioned WF community site sample, the other (called "Tester") is my own sample that exercises some rules I created through the designer.  So just to be clear... most of this is not my own code.  But that also speaks to the beauty of this approach... most of the heavy lifting is done for us already, so it's easy to get up and running.

Enjoy!

In an earlier post, I discussed my interest in Silverlight as a means to off-load CPU-intensive processing to web client machines, which are often just sitting around parsing angle brackets and not much else.

The idea is to do some work on background thread(s), so your Silverlight-enabled browser UI remains responsive, but you still get the benefits of doing less work on the server.

Turns out you can indeed spin up background threads in the Silverlight CLR, but there's currently no good way to marshal UI updating behavior from a background thread to the UI thread... ala Control.Invoke() in Windows Forms.

The "best" solution for now seems to be to maintain some shared (non-UI) state between your threads.  You update the state from a background thread upon completion of some long-running task; meanwhile, from the main UI thread, you periodically check (through use of HtmlTimer, a decidedly low-rez timer component) if the state has changed and when it does, update your UI.  It's inelegant and tedious, but it more or less works. 

I worked up a quick demo... you can get it here.  Note that you'll need Orcas Beta 1, Silverlight 1.1 alpha, etc. to make this work.

I've heard that the Silverlight team plans to provide a high-resolution timer in a forthcoming release... this will help, but frankly I think they should go one step further and implement some type of thread dispatching behavior, so we can avoid these kinds of hacks.

Silverlight isn't just about UI glitz and glamour... we can get some Real Work Done with this thing, given the right tools.  C'mon, Microsoft... help us out!

...to my faithful readers (all 3 of you, including Mom ).  I've had to undertake some major blog surgery this weekend... switched hosting providers AND blog engines... now running on dasBlog.  Community Server is nice enough, but it was getting in the way of me getting stuff done.

Links and URLs and feeds are pretty hosed right now... we've got our best man on it, I promise.

Your patience is appreciated... and, assuming you're gonna stick around, mandatory.