While preparing a training class this past month on Effective Exception Handling for a client, I came up with a list of "Ten Commandments of Exception Handling."  Some of them were the basic, standard items ("Thou shalt not catch or throw java.lang.Throwable", "Thou shalt not throw raw java.lang.RuntimeExceptions--thou shalt throw a subclass instead", "Thou shalt not catch an Exception unless thou art going to truly handle it, or are going to wrap it in another Exception").  Other items on the list were specifically tailored for the client.  I liked one of the "commandments" better than the others, however:

"XIII: Thou shalt not catch any Exceptions in JUnit tests, unless testing that your code under test throws the Exceptions the API says it does."

Companies with mission critical web applications, especially with large number of users, can not afford to have poorly performing web sites. Typically, as the number of simultaneous users increases, the response time of a web site degrades. During the development phase of a project, unit-testing and regression testing would not be able to uncover all the performance related issues. Manually creating multiple user sessions in order to observe the system behavior under multi-user situations is unpractical or inefficient. Automated load testing tools would be the solution for such purposes. They provide facility to inject sustained load or burst load on the system in a controlled way, thereby being able to collect statistics about system performance. There are commercially available automated test tools. But OpenSTA (Open, System Testing Architecture) is an open source toolset for automated load testing of web applications.

In a previous blog entry I mentioned that I like emma for measuring code coverage in my JUnit test suites.  In this entry I will discuss several items that make some JUnit test suites stronger than others.  In this entry I am discussing JUnit 3.8.1.

How do you measure the quality of your JUnit tests and put a number
on it?  This is a very good question with no quick and easy answers,
I'm afraid. I can tell you two answers that are NOT measures of test quality:

A basic overview of Rational’s automated testing tool, Robot, is that it records the user’s actions in a VB-like script that can be played back whenever the user requests.  This recording is done through the UI – and so it can only be done after parts of the application have been developed.  Since it will be used as a baseline for regression testing in later release cycles – it shouldn’t be written until that part of the system has been tested.  Robot scripts can be quite powerful, and the user can create ‘verification points’ to test to make sure that the data in a certain drop down is accurate, business results are correct, or even compare screenshots of the application to a baseline to make sure that the UI hasn’t changed.  While great for regression testing when no changes have been made to parts of a system, it may not be the most efficient way to test in an iterative development environment.

While scripted testing serves an important purpose and is often the preferred strategy, exploratory testing offers some worthwhile benefits in certain situations. If circumstances permit, a combination of scripted and unscripted testing can be ideal. Some valuable advantages of exploratory testing are:

• Ability to focus resources on targeted areas, i.e. recently changed/ambiguous code
• Tests can be quickly created and modified to isolate bugs
• Risk mitigated by exposing and documenting more defects than might be discovered through traditional scripted testing in a short time frame
• Process of pin-pointing defects often helps uncover potential workarounds