Ill

Being ill sucks

Last week I was really very ill – something flu-like – and this week I’m slowly recovering from whatever it was. Beside sapping all of my energy it’s also completely messed with my ability to actually think. Because of that, I haven’t really done anything particularly interesting – I’ve been using this time to do all of the boring / monotonous tasks that I’d been putting off – and haven’t had any particularly exciting insights into, well, anything really.

Never too ill for a monad

Since I’ve not really done much, and I’m not feeling up to writing about anything particularly involved, here’s a very simple error monad-like-structure I threw together this week for something I was working on.

public class Exceptional<TValue>
{
	#region Fields

	private readonly TValue value;
	private readonly Exception exception;
	private readonly bool hasResult;

	#endregion

	#region Constructors

	private Exceptional(TValue value)
	{
		this.value = value;
		hasResult = true;
	}

	private Exceptional(Exception exception)
	{
		this.exception = exception;
		hasResult = false;
	}

	#endregion
	
	#region Methods

	public Exceptional<T> Apply<T>(Func<TValue, T> func)
	{
		return UnWrap(
			v => Exceptional<T>.Wrap(() => func(v)),
			ex => new Exceptional<T>(ex));
	}

	public T UnWrap<T>(
		Func<TValue, T> whenResult,
		Func<Exception, T> whenException)
	{
		return hasResult
			? whenResult(value)
			: whenException(exception);
	}

	public static Exceptional<TValue> Wrap(Func<TValue> func)
	{
		try
		{
			return new Exceptional<TValue>(func());
		}
		catch (Exception exception)
		{
			return new Exceptional<TValue>(exception);
		}
	}

	#endregion
}

The brain is a muscle…

So, even though I really wasn’t feeling particularly up to it, I wrote something similar to the above in F#. I’m trying to slowly get a feel for the language after years of OOP indoctrination, and this is (sadly) probably the biggest thing I’ve written in F# so far. I love the concepts behind the language but it can be incredibly frustrating to move from being able to produce clean, concise, idiomatic code in C# to struggling to write even the simplest structures in F#.

type Exceptional<'a> = 
	| Value of 'a
	| Ex of Exception

// (a -> b) -> (a -> Exceptional<b>)
let Bind f = 
	fun a -> 
		try Value(f a) 
		with 
			| ex -> Ex(ex)

let incr x = 
	match x with 
		| 7 -> 	raise (Exception "Seven Eight Nine")
		| _ -> 	x + 1
		
let _incr =  Bind incr

_incr 3 |> Console.WriteLine
_incr 7 |> Console.WriteLine

Devlore

It’s the same old story…

Once again the top story on Hacker News was something I’ve read many times before – The 500-mile email – and I won’t lie, I read it again. There are a number of such stories, most developers who’ve been in the industry for some time undoubtedly know quite a few of them. Often about a hard-to-find bug or a laughably incompetent colleague or user, one thing that unites them all is that you can guarantee you’ll see them again; maybe not for a month, maybe not for a year, maybe in a slightly different retelling, but they’ll be back.

The first sentence of the Wikipedia article on lore says:

Folklore (or lore) consists of legends, music, oral history, proverbs, jokes, popular beliefs, fairy tales, stories, tall tales, and customs included in the traditions of a culture, subculture, or group.

Every item on that list applies to the developer community, from the myth of the 10x developer to musicforprogramming; from Falsehoods Programmers Believe About Names to Exploits of a Mom; we even have our own idiolect: Heisenbug, YAGNI, kludge; our own revered heroes: Donald Knuth, Marting Fowler, Kent Beck; our archetypal villains: business analysts, users, PHP.

There’s no real conclusion to be had here, I simply found it interesting to realise that we’re not just a group of people sharing a career, we’re an entire (sub)culture. How many other careers have this level of cultural identity? Is there something unique to development and developers that lends itself to this?

The recent StackOverflow Developer Survey suggests that:

Code is life for most devs, and for many coding is a lifetime pursuit.

Is coding more of a lifestyle than a career? Does our population being generally more introverted lead us to embrace shared experience more strongly? Or is it simply that our skills with – and constant use of – technology allow us to share more easily with each other?

Just a shower thought for the day, really…

Await-ing happiness

Introduction

There are currently two main patterns for asynchronous programming in .NET:

  • The ‘original’ asynchronous programming pattern: IAsyncResult, BeginInvoke, and EndInvoke
  • Event asynchronous programming (EAP): a Begin method and a Completed event

Using a Task-based asynchronous pattern has a number of advantages over the above, and in most cases will result in much easier to follow code. However, unless you’re already familiar with the concepts involved, the way it functions will almost certainly be quite different from what you’re used to and to your current mental model of asynchronous work. This means that while I would encourage you to get familiar with the patterns and practices involved, it can take some time working with Tasks to become comfortable with them and to think in their language.

Similar concepts exist in other languages / frameworks and may have more intuitive names, depending on what you believe is in a name: they are often called promises or futures. Linky.

Currently, if you want to offload some work from the main thread of an application, you’re likely to get hold of a background or threadpool thread and start it running. Now you either have no idea when it completes and so no way to return a value, or you have a callback for completion (something like BackgroundWorker). Working like this causes you to split your logic between two methods, one before starting the background work and one after it has completed; in more complex situations it can quickly become difficult to work with your code, to understand the flow of you application, and to reason about concurrency issues.

The basics

Slightly simplistically, a Task or Task represents a method that is in some state of execution, that is it is either about to execute, currently executing, or has already executed. Unlike previous ways of working with asynchronous code in .NET, this allows us to keep hold of a reference to the asynchronously executing code.

So how do you know when your Task has actually completed? You await it:

Task task = FrobnicateAsync();
await task;

This allows us to keep all of our logic in a single method, and makes the flow of our code much easier to follow. In fact the code reads and executes almost exactly the same as if there were nothing asynchronous happening at all; this is the unicorn – making code execute asynchronously without increasing complexity.

// logic before the async work
await FrobnicateAsync();
// logic after the async work completes

Rules to live by

I’m mostly thinking of the WPF world here, but much of the same applies no matter where you are.

Don’t think of async/await as a silver bullet
Like anything, it has it’s quirks and pitfalls but its advantages hugely outweigh these in almost all cases.

Never use async void
With async void you have no Task and so lose your context / reference to the asynchronously executing code and have no way to know when it completes; this puts us back into the ‘old-world’ of asynchronous programming.

… except when the context is already out of your hands
For example void Main() and WPF UI event handlers (`void Button_Click()’ &c) where it was not your code that invoked them in the first place.

Don’t be impatient, always await
Again, if you don’t await a Task, you’ve lost your context to the asynchronously executing code and we’ve gained nothing from using Tasks.

Never call .Wait() or .Result
This is more applicable to WPF as doing so is likely to cause a deadlock when done on the UI thread, but actually applies more widely. While these methods exist for good reason, calling them should be extremely rare, as this breaks the next rule…

Don’t mix and match
It is a seriously bad idea to just throw Tasks into the middle of an existing application hoping it will solve all of your problems, it’s more likely to cause unexpected behaviour than it is to actually solve anything. It’s often significantly simpler to introduce async ‘top-down’; that is, making your entry-points async (Button_Click &c) and spreading async down into your application from there. Other approaches inevitably lead to either async void or calls to .Result and the problems mentioned above.

await GetConclusionAsync();

There are a number of ‘work-arounds’ for various problems that people have had with TPL to be found, however I find that more often than not these are solutions not to problems in the framework but to problems caused by the misuse of it. Now, I’m not saying that there are no hard parts, but if you take the time to grok the TPL and get a feel for it, then you’ll encounter far fewer ‘problems’ with it.

Put in the time and respect it deserves and I promise, you’ll love it.

How do you call it?

Where we’re at

Following on from last time, we have our MethodInfo object and now need to actually invoke it. However we want to do this in a way that is as clean, safe and performant as possible.

Keeping it simple

We could simply call invoke on the MethodInfo object:

var foo = (Foo)methodInfo.Invoke(fooFactory, new object[] { 42 });

Last week, we put a lot of effort into trying to make what we were doing as safe as possible, so to throw away all the safety on the final hurdle seems rather silly. No type checking on the method’s parameters, an ugly cast of the return value to (Foo), and a fairly serious performance overhead make this solution less than ideal.

A nicer way

We can use System.Linq.Expressions to compile ourselves a Func<int, Foo> representing the method we wish to invoke:

var parameter = Expression.Parameter(typeof (int));
var lambda = Expression.Lambda<Func<int, Foo>>(
	Expression.Call(
		Expression.Constant(fooFactory), 
		methodInfo, 
		parameter),
	parameter);
var getFoo = lambda.Compile();
var foo = getFoo(42);

And this works… almost.

For reasons that I won’t go into deeply here, System.Linq.Expressions doesn’t deal with inheritance; it covers only a subset of language features and inheritance is not one of them. If we’re using an interface or base class for fooFactory then this will fail when we try to construct to Func.

The work around for this restriction is extremely simple though, we make the normal compiler do the work with inheritance by wrapping the call with our own concrete method.

private Foo GetFoo<T>(int id)
	where T : FlagBase
{
	return fooFactory.GetFoo<T>(id);
}

Since we’re now compiling this call to a Func<int, Foo> we can do this ahead of time and store the result, failing fast if anything is wrong and allowing us to invoke it repeatedly while avoiding the performance penalties associated with reflection. This leaves our full code looking like:

public abstract class FlagBase
{
	private readonly Func<int, Foo> getFoo;
	private readonly IFooFactory fooFactory = new FooFactory();
	
	public FlagBase()
	{
		var runtimeType = this.GetType();
		Func<int, Foo> methodGroup = GetFoo<FlagBase>;
		var methodInfo = methodGroup.Method
			.GetGenericMethodDefinition()
			.MakeGenericMethod(runtimeType);
		
		var parameter = Expression.Parameter(typeof (int));
		var lambda = Expression.Lambda<Func<int, Foo>>(
			Expression.Call(
				Expression.Constant(this),
				methodInfo, 
				parameter),
			parameter);
		
		this.getFoo = lambda.Compile();
	}
	
	public void Frobnicate()
	{
		var foo = getFoo(42);
		// ...
	}
	
	private Foo GetFoo<T>(int id)
		where T : FlagBase
	{
		return fooFactory.GetFoo<T>(id);
	}
}

Conclusion

What we were trying to do wasn’t particularly pretty and I certainly wouldn’t want to have to do it regularly, but as I mentioned last week this was about striking a balance between doing this strictly with the appropriate information at compile time, and making our API easier and cleaner to consume. Something like this is very much down to personal preference / style but I’m happy with the compromises that have been made, and with a good suite of tests I’m confident that the behaviour of this ‘hack’ is safe and reliable.

Not enough information to be generic…

This problem arose in a library I was writing last week, there were a number of ways to resolve it, all of them involving some amount of pain / ugliness in the code. The only question was where that pain would be.

The problem

We want to call a generic method with the exact runtime type of the current class.

public class FooFactory
{
	public Foo GetFoo<T>(int id) 
		where T : FlagBase
	{
		// ...
	}
}
public abstract class FlagBase
{
	private void Frobnicate()
	{
		var foo = fooFactory.GetFoo<???>(42);
		// ...
	}
}

So, our goal is to get the runtime type of the class in place of the ???.

The solution

It’s relatively simple to imagine ways we can get the deriving class to provide this information.

Since we’re using inheritance, an abstract method fits the bill:

public abstract class FlagBase
{
	protected abstract Foo GetFoo(FooFactory fooFactory, int id);
}

But now we must override this method in each and every deriving class, must get the type parameter correct each time, and must assume the deriving class is actually calling fooFactory correctly.

Another alternative could be to go for a CRTP pattern:

public abstract class FlagBase {}

public abstract class FlagBase<T> : FlagBase
	where T : FlagBase<T>
{
	private void Frobnicate()
	{
		var foo = fooFactory.GetFoo<T>(42);
		// ...
	}
}

This keeps the deriving classes clean but there’s a huge caveat, and one that we can’t enforce. If we create a derived class FlagA : FlagBase<FlagA> we can then create FlagB : FlagA. Unfortunately, we now have FlagB : FlagBase<FlagA> and the type parameter is not the actual runtime type of the class! To make this work we would have to restrict the inheritance from FlagBase<T> to be only one level deep, something that we cannot do.

There are other patterns that we can attempt to apply, but none of them allow us to keep the consuming code out of the picture. In this scenario at least, requiring the caller to provide this information is a bad idea:

  • It’s remarkably easy to provide the wrong information.
  • Every extra requirement you make of a consumer is another opportunity for a mistake.
  • It simply makes the library harder to consume.

Sometimes forcing the consumer to do the extra work is just something we have to live with for the type system we’re in, but here there’s a cleaner alternative – for the consumer at least – and a way that we can ‘internalise’ the pain.

What’s left in the toolbag?

this.GetType() and reflection to the rescue!
Not a sentence you often want to hear…

Searching Google or StackOverflow for anything reflection-based invariably leads to suggestions of invoking the required object by name and, particularly if you’re not very familiar with C#, it can be easy to be led down the garden path. In my opinion (and experience) hard coded object names scream of refactoring pain in the future. While any mistake should be picked up in your test suite, you want to strive to make refactoring as easy as possible, and why have a strongly typed language and a compiler if you’re not going to make them do at least some of the leg work?

For what we’re trying to achieve here, I prefer to get hold of a ‘strongly typed’ method group then change the generic type of that reference. It avoids strings and if applied carefully can be done safely.

public abstract class FlagBase
{
	private void Frobnicate()
	{
		var runtimeType = this.GetType();
		Func<int, Foo> methodGroup = fooFactory.GetFoo<FlagBase>;
		var methodInfo = methodGroup.Method
			.GetGenericMethodDefinition()
			.MakeGenericMethod(runtimeType);
		
		//var foo = fooFactory.GetFoo<???>(42);
		// ...
	}
}

The main type safety (yes, I’m using that term fast-and-loose) mistake we can make here is supplying the type parameters incorrectly, either the wrong number or types that don’t satisfy any type constraints. Fortunately, having the reference to the method right next to us in code helps identify the former at a glance. Satisfying T : FlagBase is guaranteed here, as we’re doing this work inside FlagBase and the runtime type of the object clearly satisfies the constraint.

Now we’re one step further along: we have a reference to the correctly-generically-typed method we want to call. But how do we actually go about calling it?

Using what we’ve built

We could simply call invoke on the MethodInfo object.

var foo = (Foo)methodInfo.Invoke(fooFactory, new object[] { 42 });

This leads to questions of type safety of the method’s parameters and any return value, as well as concerns about performance. Also, there’s only so much safety that I’m willing to throw away to make a library easier to consume.

The answer lies in System.Linq.Expressions for help in compiling this reference to something safely callable. Unfortunately for us, that has its own pitfalls to overcome…