Archive of published articles on April, 2011

Back home

Two keys to effective multithreaded programs


In my last post, I went over a mostly lock-free producer/consumer queue that worked entirely off the ThreadPool. This covers the two most important aspects to effective multithreaded programming: avoid creating new Threads, and work lock-free where possible.

Avoiding the creation of new threads is easy- you just need to schedule tasks on the ThreadPool (though QueueUserWorkItem, or System.Threading.Tasks usage), instead of using Thread.Start or new Thread. The goal is, you want the ThreadPool to manage threads for optimal performance, you don’t want to create other Threads that are going to push the ThreadPool threads off the CPU, and are going to need to be created/destroyed. A long-lived thread is usually a waste because it is inactive for much of the time. A short-lived thread is a waste because allocating and destroying a thread is an expensive operation! So just avoid creating threads- you almost never have to if you design your systems properly.

Lock-free programming is much more difficult. The reason lock-free programming is important (in this case) is, you don’t want to block ThreadPool threads. If a ThreadPool thread is blocked for a while, the ThreadPool will create a new thread- which means we’re basically creating new Threads as above, which is bad! Worst case scenario: a Parallel.ForEach loop that involves the launching of a Process/WaitForExit that blocks the thread. You’ll end up with almost every iteration of the ForEach loop having launched its own process, so you’ll have n threads in your main program, with n processes running, with their own threads. You will have a huge mess and everything will be context switching like crazy and performance across the board will suffer!

So this all basically requires breaking sequential work into discrete chunks, and linking them together- and actually this is exactly what Task Based Parallelism is, and you can do it effectively with System.Threading.Tasks in .NET 4.0 (or 3.5 with Reactive Extensions’ System.Threading.dll), and with .NET 5.0’s async and await keywords.  However, these are necessarily high-level concepts and systems, and you cannot just throw code into these systems and expect them to work correctly or predictably.  Managing threads effectively, as described here, is the first (and easiest) component of writing effective asynchronous/multithreaded systems.  The more difficult part is writing systems that can function correctly and predictably in a asynchronous environment.  I’ve found, though, that thinking about the (easier) thread management aspect can help inform the (difficult) systems design component.


Completely async file writer


I’ve been doing more and more asynchronous programming lately.  I needed to implement a logging system recently, but wanted to use asynchronous file IO.  There was also the possibility that log calls could come in faster than they could complete, so I needed to synchronize the logging calls, marking them finished when the async IO completed.  I wanted to make this as lock-free as possible.

The idea is, you have a ‘AsyncFileWriter’ field on a class, with a ‘QueueWrite’ method that threads can call to request a file write, passing the string/bytes, and a callback that fires when the IO is complete.

The ‘AsyncFileWriter’ has a Queue on it, and when calls to QueueWrite are made, a request (args and callback) is added to the Queue, and a call to a ‘WriteImpl’ is queued on the ThreadPool.  ‘WriteImpl’ has basically a semaphore, only allowing 1 thread to dequeue a request (args and callback) and begin the file IO.  This ‘semaphore’ is released in the async IO complete callback.  After the semaphore is released, another call to WriteImpl is queued on the ThreadPool, which will just return immediately if the request queue is empty.

The net effect is we have, basically, a producer-consumer queue, where producer threads can also be consumers (or queue up other ThreadPool threads to be consumers).  When a request is processed, the consumer thread that finished processing it queues up another consumer thread.  If a request comes in while the ‘semaphore’ is taken, the thread just drops its request into the request Queue and immediately returns- if the ‘semaphore’ is not taken, the thread dropping off the request then takes the lock and acts as a consumer thread.

I think the pseudocode is more clear than the actual code:

class AsyncFileWriter
   string filename #initialized from constructor
   Queue _requestQueue
   bool _insideWrite

   public void QueueWrite(byte[] bytes, Action callback)
      lock (_requestQueue) _requestQueue.Add(bytes, callback)

   private void WriteImpl()
      if (_insideWrite)
      lock (_requestQueue)
         request = _requestQueue.Dequeue()
         _insideWrite = true
      if request != null:
         fs = new FileStream(filename)
         fs.BeginWrite(request.Bytes, () => OnIOComplete(fs, request.Callback))

   private void InIOComplete(FileStream fs, Action callback)
      _insideWrite = false
      ThreadPool.QueueWork(WriteImpl) #Keep clearing the queue

So you can see that, we only take locks when adding or removing items from the queue. If 5 threads enter WriteImpl at the same time, only one will take the lock, and the rest will return. And when the IO is complete, it’ll queue up another ThreadPool request to keep clearing the queue.

I initially implemented this for async IO as mentioned above, but have abstracted the pattern for anything- it is just basically a way to throttle processing with a ThreadPool-managed producer/consumer queue. I’ll go over this pattern along with some code in a future post.

1 Comment

Do you have your Visual Studio WinForms Designer License?


I often joke at work that you should need a license to use the Visual Studio GUI Designer.  Not because it is hard to use- quite the contrary.  Because it is so goddamn easy to make a UI, people forget that UI classes should be written like any other piece of code, NOT as if they were created using a WYSIWYG editor.

Since when was it OK to have an object with 70 fields?  Why is OK to have a UserControl with 70 fields on it, including a context menu and top menu, a tab control and three full tab pages set up on it?

I’ve seen very talented programmers forget that they are talented programmers, and build UI’s with the designer that are absolutely atrocious.  Here’s how I’d suggest you break bad habits for using the Designer:

  1. Only use the Designer to set up UserControl subclasses of no more than 10 controls (both Controls and other subclassed UserControls).  If your UserControl/Form subclass has more than 10 designer-fields, it’s too big.
  2. All ToolStripItem/MenuItem classes should always be created dynamically (or at least in the main class file), NOT through the Designer.  Remember, you can build a ‘tree’ of ToolStripItems through code (think of Linq2Xml and how the code formally looks like the XML), not having to assign things to fields all over the place.  I’ll show an example in a future post.
  3. All events should be assigned through code, NOT the Designer (you can also avoid tons of event handler methods by just hooking up the events with lambdas that call the method you would have been calling).
  4. Use subclasses of Control types (ComboBox, ContextMenuStrip) to encapsulate logic.  Make them private classes inside of your UserControl if possible- the goal is to break up the logic out of a ton of methods on the UserControl.  If you have more than 4 control-specific methods, look at creating a subclass.
  5. Rarely if ever pass a reference to a Parent around- instead, raise events on your children and listen to them from the parent.
  6. If you can create your UI dynamically, you can get away with private parameterless constructors on UserControls, so you can actually have some data integrity and straightforward object use (no ‘must call Initialize/Bind method to assign data objects’ crap).
  7. Learn UI programming in a language without a WYSIWYG editor, like python.  You quickly learn small UI classes laid out in a dynamic manner are more flexible than monster static UI’s that the VS Designer is often used to create.

I can quickly gauge how passionate and experienced someone is about their craft by looking at how they build a complex UI.  If they build designer-driven uber-classes, you can be sure they either haven’t had enough experience building complex UI’s, or haven’t thought enough about what they’re doing to realize how fucked up the code they are writing is.

1 Comment

Get better at coding or get better at arguing


A while ago, one of the junior TA’s here was frustrated with all the code critiquing I was giving him, and exclaimed: “In 6 months, we haven’t had an argument about my code which I’ve won.”

I replied, “Well, get better at coding or get better at arguing.”

He got better at both.

No Comments

It is 2011. Start building responsive UIs.


In February, I finally figured out my New Year’s Resolution.  It was to build no more UIs that lock up, ever (except in exceptional circumstances where there is no other choice).  It spawned this post that I’ve finally gotten around to finishing (and some of the topics I have started queuing up posts for).

It is a well known customer-service mantra we have that, any response is better than no response. If you can or cannot help someone- or you are in the midst of doing so- you should still tell them what’s going on. It builds better relationships, communication, and confidence in your team.

So it goes with UIs. Nothing is so frustrating to users as a slow UI. Here’s a quick checklist to see whether your UI behaves as it should:

  1. A UI should always stay responsive and never lock up.
  2. A UI should give feedback about what it is doing, and the more detailed feedback the better.
  3. The user should be able to cancel something taking too long.

The frustrating thing to me is that so many of our UIs aren’t responsive, and the only reason is laziness or ignorance. I’ve written my fair share of unresponsive UIs, but I am smarter and more experienced and I say, no more! Threading APIs are sophisticated enough where any programmer really needs to know how to write asynchronous applications. Here’s what you need to learn to create a good UI (my advice is specific to .NET/Windows but the advice should apply to most platforms). I’m going to cover each topic in some detail in later posts (and maybe add/remove/modify topics).

The Windows Message Loop

To understand why your application locks up, you need to understand Windows Messages, the Main Message Loop, and other basic threading concepts. The gist is, your Main thread is what receives messages like user input and is responsible for redrawing, so it needs to be free to act on events. So that means 1) you need it to work asynchronously, so it can act on a message in the queue and go back to doing work, and 2) most processing needs to happen on a background thread. This is a highly discussed topic and much literature is available but I’ll try to give a good summary.

Offloading Work to Background Threads

Offloading work to backgrounds threads means more than just putting long-running tasks into a BackgroundWorker.  Understanding tricks for putting as much on the background as possible can make your UIs much more responsive.


This is the most basic way of keeping your UI responsive, and once you understand how to use it, can be incredibly effective and simple. I consider BackgroundWorkers almost trivial to set up after you create a few. They can be limiting, however, and they do incur some architectural overhead and you may have to make code design compromises. They need to be a tool in your toolbox, but you need to go further.

Tasks Parallel Library (TPL) and Task Parallelism

.NET 4.0 (and 3.5 with Reactive Extensions) has a System.Threading.Tasks namespace filled with all sorts of goodies. The Parallel class is excellent for data parallelism, so you can easy multithread foreach loops and LINQ queries. Task parallelism is much more sophisticated, however, and not too much has been written about it because it is complex. .NET 5.0 is going to be focused on asynchronous programming with the ‘await’ and ‘async’ keywords, which is mostly handled under the hood with the TPL. So you can find tons of good articles about task parallelism by searching for info about the .NET 5.0 CTP (community technology preview).


Lots of complexity comes with asynchrony, and especially multithreaded programming. What happens if something fails, or more pertinent to UI programming, the user wants to cancel (what’s the point of having a responsive UI if the user can’t do anything?). Having a situation where we need to be able to roll back/cancel any changes means we need a way to store how things were when the process was started. You can do this by storing state (cloning, essentially), which is difficult to implement and maintain well because it is so unclear (does everything in the object’s references need to clone? What if you have something you can’t clone), or more preferably, having data that cannot change state. To be a good asynchronous programmer, you need to be good at creating and working with immutable data. If your data is immutable, you are safe- cancellation, undos, exception recovery, etc., is trivial because you have data in a good known state that nothing can change.

No Comments

My time at BioWare in numbers


36 The number of months of been working at BWA

5 The number of job titles I’ve held

3 The number of teams I’ve been on/bosses I’ve reported to

2 The number of departments I’ve worked for

1 The number of projects I’ve worked on at BWA

3 The number of times I’ve spoken at GDC *

I started at BWA in April 2008 as an Associate Technical Animator. I was promoted to Tech Animator after several months. After several more months, I helped ‘found’ the Tech Art Team and disband the Tech Animation team (moved from Animation Team to Tech Art Team, and changing bosses). I was promoted to Senior Tech Artist after several more months. I was responsible for hiring 2 additional Tech Artists. I moved over to the Tools Team as a Senior Tools Programmer (moved from Tech Art Team/Art Dept, to Tools Team/Prog Dept, and changing bosses).

*Tech Art Techniques Panel (2009), The Old Republic’s Object Model Pipeline (2010), Tech Art Bootcamp: Ending the Culture War: Building a Better Pipeline by Uniting Tech Art and Tools Engineering (2011)

1 Comment

Augmented Coding


First, a shout-out for the inspiration for this post: Patrick Smacchia and the NDepend developers have given me my first free software license as a result of this blog. NDepend is a pretty awesome static code analysis tool. You can find more info here:

NDepend, or any static code analysis tool, is one of many tools in a toolbox that contribute to something I’m now calling ‘Augmented Coding ™’. We’re turning a corner with code development, where we need to get past the idea that coding is just typing text into a window. Joel Spolsky and numerous others have commented how a programmer’s job is more assembling components, rather than actually coding. The most productive programmers understand, and have learned to embrace, this (or they’ve gotten the fuck out to lower-level areas where they can just program closer to the metal).

We need to progress out of the age of the Text Editor.

Skills of ‘principle’ are essential, obviously. But actual ‘technical/trade’ skills will help you develop those skills of principle much more effectively. Great programmers need to be touch typists.  My naive college professors for 3D art used to say, they are supposed to teach you the principles, not how to use the programs. But somehow, myself, a middling artist at best, had some of the best work, because I could fly around Maya, and understood and could writer shaders, and knew all the shortcuts in Photoshop (oh and I taught myself ZBrush, which makes even crap look decent). And now somehow I, a middling typist, can create proper code in a fraction of the time, because I keep my hands off my mouse, because I have ReSharper and use it, and use shortcuts as much as possible (when I say ‘proper’, I mean, things readonly are marked readonly, I have truly immutable classes instead of private setters, things are named/laid out according to conventions, I don’t have unused or improperly scoped members, etc.).  And particularly with NDepend and code analysis tools, codebases are too big for a single person to intrinsically know areas of dead code, unused or heavily used code, non-conforming standards, etc.  If you’re working in a codebase with more than a few other devs, and you think you don’t need static code analysis in your codebase, or it isn’t worth the hassle of getting it setup, you’re wrong- do it and thank me later.

In future posts I may go over what exactly I do and look for when assessing the necessary technical attributes of the fully Augmented Coder- both technical aides like code generation and static code analysis, and better use of existing tools like your IDE or version control.

1 Comment

Lazy iteration, potential benefit


At this point, most people are familiar with the deferred evaluation model of LINQ, using ‘yield’ statements. Since this is new for some teams at work, there’s been a question of when is it appropriate. Deferred evaluation is of most benefit when there is some chance the entire Enumerable will not be evaluated. And it is of dubious use when the source of the enumeration may change. Those are the two extremes- where and when one uses ‘yield’ and deferred evaluation will of course vary according to the task. But regardless, anyone using IEnumerable in .NET 3.5+ needs to understand the deferred evaluation model of LINQ/yield or they will introduce any number of subtle bugs.

Sorry for the length between posts, there was a lot of work and career stuff going on that should be winding down now (and I’ve been doing a lot of travel lately so I have a few blog posts queued up).

No Comments