debugging

How Do You Fix An Impossible Bug?

Within the span of an hour, it had all gone to hell.

The first deployment went rather smoothly. It was a fix to an existing web service, and went out with no problems, or so we thought. Within ten minutes of the deployment, the users started complaining of a minor bug, one that was seemingly omnipresent but didn't really stop them from doing meaningful work. The team which had sent out the deployment immediately set to work figuring out what was going on.

Unrelated to that deployment and forty minutes later, my team was launching a major change to a web site that was consuming that other team's web service. When our change went out, the bug that the users had been complaining about from the web service deployment vanished, replaced by a major bug that caused them to be unable to do any work at all. Naturally we were a little concerned.

The major bug caused an error message that included this statement:

"Unknown server tag 'asp:ScriptManager'."

Now usually when I see an error message like that, I think the project is missing a reference. But no, the references the project needed did, in fact, exist. The second thing I think is that it was using the incorrect version of .NET (in this case, 2.0). Wrong again, it's using the proper version. So now I'm a bit stumped; I pull one of my developers off his project to work on this, and he and I go to our manager and the web service team to try to hash this out.

It took the five of us about an hour to work out where exactly the problems were. As so often happens with major problems like this, it wasn't one problem but several, all intersecting at one time. They snowballed like so:

  • The previous day, my group attempted to roll out the major change to the web site. The roll out didn't work, and another manager (who had previously owned the project, and unbeknownst to me) immediately copied the application to a different location on the same server. He figured this would solve the problem, as it had before with a different app; it didn't.
  • Before the web service change went out, the users had already been notified to use the new location. Consequently they started complaining about the major error.
  • When the web service change was deployed, a different set of users within the same group complained about the minor bug, as word had not reached them to use the new location.
  • When our web site change went out (to the original location), the users of that site noticed and now complained of a "new" bug at the old location, despite the fact that it was the same bug as the bug at the new location.
  • All of this taken together meaning that the fact that our web site and their web service were related was a coincidence. The fact that the two deployments went out so near to each other had nothing to do with what the actual problem was. Coincidences like this are the worst possible thing that can happen when trying to find a bug.

Got all that?

Ultimately we worked out the problem. Well, really, we stumbled onto it. How we got there was such blind luck that I'm not convinced we actually solved the problem so much as lucked into a solution.

A bit of googling for the error message revealed this StackOverflow answer which states that the reason we get the above error is that a piece of the web.config is missing. Here's what it should be.

<pages>  
    ...
    <controls>
        <add tagPrefix="asp" namespace="System.Web.UI" assembly="System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
    </controls>
</pages>  

We had previously confirmed that in our application configuration that line did, in fact, exist. Obviously this could not be the problem. (I apparently forgot about what happened the last time I assumed something was true.) Later, when we started getting more desperate to find the source problem, I had our server team give me a copy of the app running on the production servers. This is what I found:

<pages>  
    ...
    <controls>
        <!--<add tagPrefix="asp" namespace="System.Web.UI" assembly="System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>-->
    </controls>
</pages>  

Yes, you read that right. For some damn reason, the offending line of configuration code had been commented out.

And we had no idea how this happened.

We tried examining the config transforms; there was nothing that would result in that line being commented out. We tried looking at the server logs, the build logs, the source control history, anything that might give us a scrap of information as to how the hell this happened. We found nothing.

As you might imagine, this was a little frightening. This line got commented out, and we couldn't reproduce how. How can you fix a bug that should have never occurred in the first place? When we deployed the corrected configuration file, it worked, of course. But in the meantime we had wasted nearly an entire day looking for something that should have been impossible.

But was it impossible, or did we miss something? I'm inclined to believe the latter. One of the things I stress to my team when they come to me with bug reports is the important question of what changed? What changed between when the system worked and when it didn't? Was it business rules, data sources, the build process? If we can determine what changed, the time needed to pinpoint the actual bug shrinks dramatically. In this case, either we simply didn't know what had changed (the most likely scenario) or nothing had changed (the far scarier scenario). Either way, something was off and we couldn't determine what it was.

What was even more worrisome was that there had been a minor bug reported before the major bug showed up, the one that was annoying but not work-stopping. That minor bug was not reproducible now, so it's entirely possible it's still lurking out there, waiting for some unsuspecting user to click the wrong button or enter the wrong date and then blow up the whole system. We don't think it will be that serious, but we also don't really know that it won't. We're stuck in bug-induced purgatory.

That's a terrible feeling. Something went wrong, and you can't figure out how. You know what the solution is, but you don't know why.

I suppose I should just be happy we figured out the problem. I am, sort of. And yet I am forced to conclude that there exists a bug which caused a critical part of our application configuration to be commented out for no discernible reason. A part of me still wonders: how can I find and fix that bug, a bug that should have never existed in the first place?

And what if it happens again?

What about you, dear readers? What are some bugs you found that you couldn't source? How did you fix these "impossible" bugs? Share in the comments!

Happy Coding Debugging!

Post image is 2008 broken computer monitor from Wikimedia, used under license.

Debugger Windows in Visual Studio 2015 Quick Guide

Debugging is a critical skill that every programmer needs to become better at their jobs. Visual Studio provides several windows that help make debugging easier, but until now I hadn't really used them all that often. Well, in the spirit of not caring that I suck as long as I'm learning, I did a little research into the debugger windows that I'd never truly used before, and compiled a list of the most useful debugger windows in Visual Studio 2015. Here's hoping this helps some other debugging newbie as much as it has helped me.

We will cover the following windows in this post:

Without further ado, let's get started!

Breakpoints

Shortcut: Ctrl + Alt + B

The Breakpoints window is the simplest of the debugger windows; it just lists all the breakpoints in your application, and bolds those that are currently active. From this window, you can enable or disable specific breakpoints (by checking/unchecking them in the list), delete breakpoints, or edit their settings.

The Breakpoints window, showing two breakpoints, including one that is currently active

Output

Shortcut: Ctrl + Alt + O

The Output window displays status messages from the IDE. This window is basically the talkative younger cousin to the other windows; it spouts everything it has ever heard, and occasionally you'll hear something that interests you. It's the most verbose of the debugger windows, but if you need to really dig deep to find the bug you're looking for, it is invaluable.

The Output window, showing a collection of status messages.

Diagnostic Tools

Shortcut: Ctrl + Alt + F2

New in Visual Studio 2015 is the Diagnostic Tools window. This handy window allows you to monitor and analyze performance data even as you are debugging, and IMHO is a truly useful addition to the Visual Studio IDE.

The Diagnostic Tools window, showing Process Memory and CPU Utilization graphs

The Diagnostic Tools window displays the following information:

  • Events: Displays the time information related to breakpoint- and stepping-enabled events.
  • Process Memory: A graph showing the memory being used by the process over time.
  • CPU Utilization: A graph showing the CPU utilization over time (note that the unit is "% of all processors")

Autos

Shortcut: Ctrl + Alt + V, A

The Autos window shows the variables used in the current line of code and the preceding line of code of a set Breakpoint, listing their current values. This is a quick way to view the nearby state of the application at the set breakpoint. If you are only concerned about the immediate vicinity of the line being executed, the Autos window will show you all the current values.

The Autos window, showing four variables and their current values

Locals

Shortcut: Ctrl + Alt + V, L

The Locals window displays the variables (and their values) that are local to the current context or scope. Generally this means the procedure or function you are currently debugging. Both the Locals and Autos windows are populated automatically by Visual Studio's debugger.

The Locals window, showing five variables and their current values.

Immediate

Shortcut: Ctrl + Alt + I

The Immediate window, like Locals and Autos, is used to debug the values of properties and expressions. However, unlike those windows, the Immediate window is not automatically populated; it only shows value for properties and expressions that you enter into it. More importantly, the Immediate window can evaluate expressions using a command-like syntax, three examples of which are included in the screenshot below.

The Immediate window gives you more fine-grain control over what expressions or value you'd like to view at a given point in the execution, but you have to populate it manually.

The Immediate window, showing three different ways of printing an object.

Watch

Shortcut: Ctrl + Alt + W, 1 or Ctrl + Alt + W, 2 or Ctrl + Alt + W, 3 or Ctrl + Alt + W, 4

The Watch window allows you to watch values entered into it. This means that you could track the value of that particular variable throughout the execution of your method. Even better, you can track more than just variables; you can track any expression recognized by the debugger. If you're trying to find out what happened to your variable or object, and need to know where and when it changed, this window will help you find out. Visual Studio 2015 allows you to have up to 4 Watch windows open at a time (hence the multiple shortcuts).

Because the values in the Watch window are persistent, they remain in the window even if they're no longer in current execution scope. Here's the window with variables that are in scope:

The Watch Window, showing two properties in context

And here is the window with those same two variables, except that now the execution has moved beyond the appropriate scope:

The Watch Window, showing two properties that are out of the current context.

Call Stack

Shortcut: Ctrl + Alt + C

The Call Stack window simply displays each method or function that is currently on the stack. This is very useful if you are trying to track down which method called the method that contains the current line of code. Even more useful is the fact that this window will also display the language that each item on the stack was written in, so those of you working in multiple-language environments will get extra usage out of this window.

The CallStack window, showing a two-item stack

Exception Settings

Shortcut: Ctrl + Alt + E

The Exception Settings window allows you to specify which exceptions will cause the debugger to break, and where they should break. For example, you could specify that the debugger only break on ArgumentNullExceptions, or JavaScript OutOfMemory exceptions, or nothing at all (which I don't recommend, but you could do it). If you need fine-grain control over what exceptions do and do not cause the debugger to break, this window has you covered.

The Exception Settings window, showing a list of Common-Language Runtime exceptions.

Summary

These are the debugger windows in Visual Studio that I believe are the most useful. Now that I more thoroughly understand what each of these windows are meant to be used for, I will certainly be utilizing them more often. Hopefully, you will too.

Did I miss any debugger windows in Visual Studio 2015 that you use? Do you have another purpose for one of the windows listed? Share in the comments!

Happy Debugging!

What is the ModelState? - ASP.NET MVC Demystified

Ever wondered just what the ModelState was that keeps popping up in your ASP.NET MVC controllers? So did I. Let's break down what the ModelState is and why we use it.

What is ModelState?

ModelState is a property of a Controller, and can be accessed from those classes that inherit from System.Web.Mvc.Controller.

The ModelState represents a collection of name and value pairs that were submitted to the server during a POST. It also contains a collection of error messages for each value submitted. Despite its name, it doesn't actually know anything about any model classes, it only has names, values, and errors.

ModelState has two purposes: to store the value submitted to the server, and to store the validation errors associated with those values.

But that's the boring explanation. Show me the code!

The Setup

First, we have the AddUserVM view model:

ViewModels/Home/AddUserVM.cs
public class AddUserVM  
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string EmailAddress { get; set; }
}

Next, we have a simple view:

Views/Home/Add.cshtml
@model ModelStateDemo.ViewModels.Home.AddUserVM
<h2>Add</h2>

@using(Html.BeginForm())
{
    <div>
        <div>
            @Html.TextBoxFor(x => x.FirstName)
        </div>
        <div>
            @Html.TextBoxFor(x => x.LastName)
        </div>
        <div>
            @Html.TextBoxFor(x => x.EmailAddress)
        </div>
        <div>
            <input type="submit" value="Save" />
        </div>
    </div>
}

Finally, we have the controller actions:

Controllers/HomeController.cs
...
[HttpGet]
public ActionResult Add()  
{
    AddUserVM model = new AddUserVM();
    return View(model);
}

[HttpPost]
public ActionResult Add(AddUserVM model)  
{
    if(!ModelState.IsValid)
    {
        return View(model);
    }
    return RedirectToAction("Index");
}

When we submit the form to the POST action, all of the values we entered will show up in the AddUserVM instance. But how did they get there?

The ModelStateDictionary Class

Let's look at the rendered HTML form for the Add page:

<form action="/Home/Add" method="post">  
    <div>
        <div>
            <label for="FirstName">First Name:</label>
            <input id="FirstName" name="FirstName" type="text" value="">
        </div>
        <div>
            <label for="LastName">Last Name:</label>
            <input id="LastName" name="LastName" type="text" value="">
        </div>
        <div>
            <label for="EmailAddress">Email Address:</label>
            <input id="EmailAddress" name="EmailAddress" type="text" value="">
        </div>
        <div>
            <input type="submit" value="Save">
        </div>
    </div>
</form>  

In a POST, all values in <input> tags are submitted to the server as key-value pairs. When MVC receives a POST, it takes all of the post parameters and adds them to a ModelStateDictionary instance. When debugging the controller POST action in Visual Studio, we can use the Locals window to investigate this dictionary:

The Values property of the ModelStateDictionary contains instances that are of type System.Web.Mvc.ModelState. What does a ModelState actually contain?

What's in a ModelState?

Here's what those values look like, from the same debugger session:

Each of the properties has an instance of ValueProviderResult that contains the actual values submitted to the server. MVC creates all of these instances automatically for us when we submit a POST with data, and the POST action has inputs that map to the submitted values. Essentially, MVC is wrapping the user inputs into more server-friendly classes (ModelState and ValueProviderResult) for easier use.

There's still two important properties that we haven't discussed, though: the ModelState.Errors property and the ModelStateDictionary.IsValid property. They're used for the second function of ModelState: to store the errors found in the submitted values.

Validation Errors in ModelState

Let's change our AddUserVM class:

ViewModels/Home/AddUserVM.cs
public class AddUserVM  
{
    [Required(ErrorMessage = "Please enter the user's first name.")]
    [StringLength(50, ErrorMessage = "The First Name must be less than {1} characters.")]
    [Display(Name = "First Name:")]
    public string FirstName { get; set; }

    [Required(ErrorMessage = "Please enter the user's last name.")]
    [StringLength(50, ErrorMessage = "The Last Name must be less than {1} characters.")]
    [Display(Name = "Last Name:")]
    public string LastName { get; set; }

    [EmailAddress(ErrorMessage = "The Email Address is not valid")]
    [Required(ErrorMessage = "Please enter an email address.")]
    [Display(Name = "Email Address:")]
    public string EmailAddress { get; set; }
}

We've added validation attributes, specifically Required, StringLength, and EmailAddress. We've also set the error messages that are to be displayed if the corresponding validation errors occur.

With the above changes in place, let's modify the Add view to display the error messages if they occur:

Views/Home/Add.cshtml
@model ModelStateDemo.ViewModels.Home.AddUserVM

<h2>Add</h2>

@using(Html.BeginForm())
{
    @Html.ValidationSummary()
    <div>
        <div>
            @Html.LabelFor(x => x.FirstName)
            @Html.TextBoxFor(x => x.FirstName)
            @Html.ValidationMessageFor(x => x.FirstName)
        </div>
        <div>
            @Html.LabelFor(x => x.LastName)
            @Html.TextBoxFor(x => x.LastName)
            @Html.ValidationMessageFor(x => x.LastName)
        </div>
        <div>
            @Html.LabelFor(x => x.EmailAddress)
            @Html.TextBoxFor(x => x.EmailAddress)
            @Html.ValidationMessageFor(x => x.EmailAddress)
        </div>
        <div>
            <input type="submit" value="Save" />
        </div>
    </div>
}

Notice the two helpers we are using now, ValidationSummary and ValidationMessageFor.

  • ValidationSummary reads all errors from the model state and displays them in a bulleted list.
  • ValidationMessageFor displays only errors for to the property specified.

Let's see what happens when we attempt to submit an invalid POST that is missing the email address. When we get to the POST action while debugging, we have the following values in our ModelStateDictionary:

Note that the ModelState instance for the email address now has an error in the Errors collection. When MVC creates the model state for the submitted properties, it also goes through each property in the ViewModel and validates the property using attributes associated to it. If any errors are found, they are added to the Errors collection in the property's ModelState.

Also note that IsValid is false now. That's because an error exists; IsValid is false if any of the properties submitted have any error messages attached to them.

What all of this means is that by setting up the validation in this manner, we allow MVC to just work the way it was designed. ModelState stores the submitted values, allows them to be mapped to class properties (or just as parameters to the action) and keeps a collection of error messages for each property. In simple scenarios, this is all we need, and all of it is happening behind the scenes!

Custom Validation

But what if we needed to perform more complex validation than what is provided by attributes? Say we needed to validate that the first and last names are not identical, and display a particular error message when this happens.

We can actually add errors to the model state via the AddModelError method on ModelStateDictionary:

[HttpPost]
public ActionResult Add(AddUserVM model)  
{
    if(model.FirstName == model.LastName)
    {
        ModelState.AddModelError("LastName", "The last name cannot be the same as the first name.");
    }
    if(!ModelState.IsValid)
    {
        return View(model);
    }
    return RedirectToAction("Index");
}

The first parameter to the AddModelError method is the name of the property that the error applies to. In this case, we set it to LastName. You could also set it to nothing (or a fake name) if you just want it to appear in the ValidationSummary and not in a ValidationMessage.

Now the error will be displayed on the page:

Summary

The ModelState represents the submitted values and errors in said values during a POST. The validation process respects the attributes like Required and EmailAddress, and we can add custom errors to the validation if we so desire. ValidationSummary and ValidationMessageFor read directly from ModelState to display errors to the user.

For a bit more info, check out Professional ASP.NET MVC 5, specifically Chapter 6, which details validation methods using the ModelState.

I've also got a very simple sample project on Github that demonstrates how the ModelState works and provides all the code and markup in this post. Take a look!

Happy Coding!