I choose you AUS2

Last week made for a very interesting and busy week. The high point of the week was getting the next update to Firefox 1.5/2 and Thunderbird 1.5 out the door. There was a lot of testing to be done to make sure that there would be as little negative user impact as possible. A few of us had to work into the night, but the hard work paid off. Ultimately, we were able to get Firefox 1.5.0.12 and Firefox 2.0.0.4 out to the user.

Everything didn't go completely smoothly, as things rarely do in the real world. Once users started getting their updates, we saw a 3-fold increase in bandwidth. We were able to work our way through the bandwidth issues however, and everything turned out pretty well. Once we got over that hurdle it was just a waiting game to see if there were any major problems reported by users. I think we can safely say that the minor update launch was a success.

What is involved in testing Firefox to ensure smooth updates? It is a fairly straight forward process. I will not go into all the details, but it generally involves downloading an older version of Firefox. Then, changing a config file so that when Firefox checks for updates from a test server. Once the update is installed, we ensure the browser is not "broken" by running a few checks.

For those who are curious, there are a couple terms you should be aware of when working on updates for the community. The first is a "partial" update. This is when you are updating from one version to the next version of said software. The second is a "complete" update. This is when you are updating from one version to the latest available version of said software. The third type of update check is known as a "fallback". A fallback is tested by updating to a new version of the software, but instead of accepting the update, we tell Firefox to update later. We then go into a config file and change the update status from "pending" to "failed". This makes Firefox think that an update was unsuccessful and it attempts to update to the latest version. While this process is fairly straightforward, there is a lot of work to do for update testing because Firefox has a lot of different versions, locales, and is in use on many different platforms by our users. But this hard work pays off in the end and gives you a feeling of self-satisfaction once the project is deemed a success.

As soon as we got that project out the door, we immediately switched gears and started working on the next major project. I am quickly learning that in the software business, there is a constant flow of work. As soon as one project is complete, the next project begins, if it hasn't already. This makes for a very interesting career, if you are up to the challenge. I can honestly say that my experience from other industries, mainly the automotive industry, has prepared me for this aspect of the software industry.

The next big project this week was starting some of the testing on Firefox 3, currently dubbed Minefield 3.0 Alpha 5 Pre. The extent of testing Firefox 3 this past week was running Basic Functionality Tests (BFTs) on each of the main platforms. These platforms are Windows XP, Windows Vista, Mac OSX and Linux. Running the BFTs themselves is again fairly straightforward, but ultimately there is a lot of work involved. The BFT series of tests consists of 196 tests ranging from installation to GUI and layout to security checks to software updates. Using a program called Litmus, we can track the progress of these tests. While running these tests, it is not uncommon to come across a bug that is unrelated to the test you are running.

While running the BFTs for Minefield on Friday, I came across five bugs. Some related to the test I was running, some completely unrelated. If you are ever in doubt, ask someone on irc to confirm what you are seeing and then file a bug.

After all of this testing is complete, we use another tool of the trade: the wiki. Using a wiki, we log what tests we ran, on what version, on what platform, who ran the tests, and any problems we noticed along the way. This allows everyone to get a snapshot of where the problems exist and where we were successful.

The third major task of the week was triaging bugs. Using, bugzilla, a tracking system used to log bugs or issues with our software, we go through and double check any bugs that are sitting as "unconfirmed". This allows us to identify any serious problems or items that were isolated incidents and have long since been fixed. This process involves finding bugs in bugzilla that need to be checked, try to reproduce the issue noted in the bug. If you want to get involved in this aspect of QA, I suggest you come to #testday on irc sometime. Be wary, some bugs in bugzilla can be pretty technical. Just seek out the ones that you think you can handle. No one is expected to know everything.

As you can see from my ever expanding post here, that Mozilla had quite a busy week. The way things are looking, there are going to be many busy weeks to come. I think I would be more worried if we weren't busy. So this is definitely a good sign.

On that note, I must bid you adieu.

See you next week!