bug-hunt

Solving the Stupid Bug

A few weeks back I wrote about a very unusual bug that returned 500 errors for routes that should have been valid. Recently my team and I, to our great relief, discovered the cause of this bug. Of course, that wasn't before we ended up yelling "WTF?!" a great many times. A great many times. A New Failing Route Most of the our collective WTFs occurred when we discovered the original problem. As I wrote about in the earlier post,... Read more >

A Stupid Bug and a Plea for Help

UPDATE (17 May 2017): We've fixed the stupid bug described in this post! Check it out! Bug hunts can take on many forms. Some are simple cleanup jobs, some take hours and end up being something stupid, and some look rather easy and turn out to be impossible. My group recently encountered a fourth kind, a bug where the solution is found but makes no sense, and we could use some help figuring out why this bug exists. Birth of... Read more >

How Do You Fix An Impossible Bug?

Within the span of an hour, it had all gone to hell. The first deployment went rather smoothly. It was a fix to an existing web service, and went out with no problems, or so we thought. Within ten minutes of the deployment, the users started complaining of a minor bug, one that was seemingly omnipresent but didn't really stop them from doing meaningful work. The team which had sent out the deployment immediately set to work figuring out what... Read more >

Ancient ORACLE, Modern IIS, and a Failure to RTFM

My coworkers Rajit and Dave and I have been spending an inordinate amount of time over the last few days looking for a really annoying error in one of our apps. The exception message reads like this: Could not load file or assembly 'Oracle.DataAccess' or one of its dependencies. An attempt was made to load a program with an incorrect format. Yep, that's right, we're using ORACLE as our database. I assure you, this is not by choice. We've... Read more >

The Bug Is In Your Code

Have you ever been hunting a bug and been absolutely sure that it was in someone else's code, only to find out that, nope, it was in yours all along? I sure did. Come along with me as we explore my latest minor failure and remind ourselves that, most of the time, the bug is in your code. Emperor Gum Moth from Wikimedia, used under license Well, hopefully that bug isn't in your code, but you never know. Bad Assumptions,... Read more >

The Bug Hunt From Hell (Or, Why You Shouldn't Forget About Caching)

We have an in-house media server that stores internal images, videos, etc. This project has many components, but the major ones are: A database, where information about the media items is stored. A service layer, which allows multiple apps to use the server and database. A private site (which talks to the database through the service layer) that allows certain individuals to upload media items. With lots of pieces, something was bound to break eventually. This week it did, and... Read more >