On Windows 7 Microsoft has come up with yet another “assistant”.

The Program Compatibility Assistant may be a great invention for non-developers, but for Developers using Visual Studio 2008 on Windows 7 (x64) it fails to inspire any enthusiasm, because it may change unhandled exceptions into silent failures.

If a vshosted app throws an unhandled exception, then the PCA will detect it, report that VSHost.exe did something bad and prevent any further unhandled exceptions from appearing in VS for the particular vshost execution path. Most annoyingly, PCA reports that it has made the compatibility changes with no option to say “No thank you”.

From then on no Unhandled Exception may make it’s way through to the GUI (neither VS nor the app under development).

This is bad because it gives a false sense of security – the app runs with no problems reported … !

What happens seems to be exactly the same as if the OS automatically pushed the “Continue” button on the Unhandled Exception messagebox.

An example: Create a Windows Forms application, add a statement which may occasionally throw a DivideByZero exception in the Form Load handler and some UI update after the statement, and then run it a few times – sometimes the UI update will happen, sometimes it won’t – but the problem is not reported.

This is really bad – the app which looks like it works well … doesn’t !

As with most other problems this can be prevented with unit-testing, but if the unit-testing isn’t test-driven, but instead the tests are written after the app-code, then it may not be obvious that there is a problem. Which means that it’s now entirely up to the developer to think of every possible thing that could go wrong – instead of just starting the app from VS, and hopefully seeing some of them.

Unfortunately vshost doesn’t handle “unhandled exceptions” very well, which means that even when PCA is disabled, then VSHost will provoke the crash dialog – not a position in the code with the debugger handy.

The problem can’t be worked-around by not using VSHost (Project setting) – that will just provoke similar actions on the app-executable if run with PCA, and the app-crash isn’t caught if running without PCA.

For some unknown reason the unhandled exception won’t necessarily have the same repercussions when the app is run outside VS.

PCA persists the path of the “offensive” VSHost (…MyProject/bin/Debug/MyProject.vshost.exe) in

[HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers]

along with

[HKCU\Software\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Compatibility Assistant\Persisted]

VSHost will be marked with a “DISABLEUSERCALLBACKEXCEPTION” value.

PCA is a Service which can be disabled, but don’t forget to check and cleanup the registry associated with it, because the OS will keep using the PCA registry settings when running VSHost and other apps.

[The symptoms described were seen on Windows 7 (x64) running Visual Studio 2008 Professional Edition]