Jinxing your application

If you ever wrote a multithreading application you should understand how hard is to get it right. If you don’t understand it then your application most probably isn’t written correctly.

You’ve written a multithreaded application and now what. How can you test it whether it is written correctly or not. Unit testing won’t be of great help because there are complex currency issues that might manifest in a bug only under certain circumstances. Imagine this piece of code:

class Program
{
    static int x;
    static Random rnd = new Random();

    static void Main(string[] args)
    {
        List<Thread> threads = new List<Thread>();
        for (int i = 0; i 4; i++)
        {
            Thread t = new Thread(Runner) { IsBackground = true };
            t.Start();
            threads.Add(t);
        }
        foreach (Thread t in threads)
        {
            t.Join();
        }
    }

    static void Runner()
    {
        for (int i = 0; i 1000000; i++)
        {
            Thread.Sleep(rnd.Next(10));
	   int orig = x;
            x += 5;
            Debug.Assert(x == orig+5);        
        }
    }
}

I am increasing a shared static variable from multiple threads without any synchronization. Will it work? It might or might not (try it!). It depends on when different threads are accessing the variable x. One thing is for sure, this code isn’t correct and it most probably won’t work and for sure it won’t work always. That is the biggest problem with multithreading – if something works it doesn’t mean that it it is correctly written and that it will work always. More about this later. If you use unit testing to test runner method the test will pass because unit testing doesn’t test multithreading, at least not easily.

So, how does one test such code and scenarios. One way is to use a static analysis tool. The other way is to put a jinx on your application. Once your application is jinxed it will be much more prone to displaying concurrency and other multithreading errors. And that’s exactly what Jinx does. Behind the scenes it makes your application fail more often that it would fail in normal circumstances. I mean that it shows faults in the application (if any) that would otherwise remain hidden and would occur only randomly here and there (you know, your user will find it after 2 minutes of running the application) – it doesn’t fail your application for no reason, it just emphasizes your bugs.

The most interesting aspect of Jinx is its the way it works. Jinx is a sort of hypervisor. You certainly know Hyper-V hypervisor whose task is to run guest operating systems. Jinx’s task is to make a clone of your OS and debugged application within and run multiple versions of it under various conditions. This is done so that any multithreading error is more likely to appear. Just by running more versions of the same application the error is more likely to manifest itself. But Jinx throws all sort of other jinxes to your application as well. That’s the shallow explanation. You’ll find more on official overview page and FAQ page.

So, let’s jinx the code above. Note that if you don’t want to you don’t need any modification of an existing code. Jinx can be set to do its task on any newly run application. However, some fine control is a better way to go. Here are the simple steps for selective jinxing:

1. Reference jinxinterface assembly. It contains a single static JinxInteface class with a bunch of static methods and serves as a communication bridge between application and Jinx.
2. Add jinx.cs file to your project. Again it contains a single static Jinx class that is a wrapper around JinxInteface class mentioned above. Its purpose is mostly to apply Conditional("DEBUG") over methods so they won’t get executed for non-debug version of the application.
3. Call Jinx.RegisterApplication(); method at start of the application. This way you’ll let Jinx know that your application should use some jinxing.
4. Replace Debug.Assert with Jinx.Assert. The only difference with both asserts is that the latter send statistical information to Jinx.

After the changes the code should look like this:

class Program
{
    static int x;
    static Random rnd = new Random();

    static void Main(string[] args)
    {
        Jinx.RegisterApplication();
        List<Thread> threads = new List<Thread>();
        for (int i = 0; i 4; i++)
        {
            Thread t = new Thread(Runner) { IsBackground = true };
            t.Start();
            threads.Add(t);
        }
        foreach (Thread t in threads)
        {
            t.Join();
        }
    }

    static void Runner()
    {
        for (int i = 0; i 1000000; i++)
        {
            Thread.Sleep(rnd.Next(10));
            int orig = x;
            x += 5;
            Jinx.Assert(x == orig + 5);
        }
    }
}

Before any analyzing takes place Jinx should be enabled and set – this is a system wide option. There are two ways to open the Jinx console – either through Tool/Jinx Visual Studio menu item or directly from All Programs via start menu. Either way you need Administrator privileges. Enabling is easy, just check Enable Jinx checkbox and that’s it. As per what programs are analyzed I’ll use “Analyze the most recent program I have registered.” option.

You can adjust some strategy settings on Strategy tab, I’ll skip this as it is an advanced option. And you can see the statistics on Status tab. Let’s run the application now. Jinx will kick in and the CPU will get under heavy load and the system might shutter due to Jinx running versions of the application in parallel. But the assert failure pops up almost immediately – the bug was caught for sure. If you run the application without jinxs there error would manifest much more later if at all and note that the test code in question is an extreme example. Here is the status tab page after a couple of errors caught:

The asserts observed counter is increased thanks to Jinx.Assert method call. Jinx analyzer isn’t limited to asserts and it might catch other type of errors as well according to PetraVM guys. Jinx is much more than this simple example.

So far I’ve run few examples like this and Jinx performed well and I think Jinx is a good weapon against multithreading bugs. However there might be a problem for testing. Since Jinx is a hypervisor on its own it won’t get along with other hypervisors such as Hyper-V. In other words forget about running Jinx on guest OS. A dedicated machine is required. Perhaps this issue will change in the future.

Also be careful when experimenting with Jinx as it is in beta phase right now (you can apply for testing over here). Running a beta hypervisor might result in a BSOD and all the consequences from BSOD such as non bootable Windows after. Which happened when I was writing this post. Perhaps this post is jinxed as well :-). Humor aside, I am sure guys behind Jinx will make it rock solid for the RTM. They obviously know very well the hypervisor craft.

Happy jinxing your applications!