Software QA 101: The Basics of Testing

Although it doesn't get a lot of press, testing is arguably the most important phase in any software development project. Certainly testing is the best-the only-way to know whether your project or product is destined for success or doomed to failure before it goes live. The U.S. Commerce Department has estimated that buggy software costs nearly $60 billion annually, and that at least $22.2 billion worth of those bugs could have been prevented with more thorough testing.

And although you might like to think that your newly coded application is virtually bug-free, we all know by now that it just ain't so. Sandra Tise of Empirix, a testing tool maker, notes that there's simply no such thing as an error-free application: "Frankly, I know of no software test that I've done that has come back perfect."

There are many, many different ways to test software, but most tend to fall into two broad categories: functional testing and performance testing.

Functional testing
Functional testing covers how well the system executes the functions it is supposed to execute-including user commands, data manipulation, searches and business processes, user screens, and integrations. Functional testing covers the obvious surface type of functions, as well as the back-end operations (such as security and how upgrades affect the system).

Arthur Povlot of Tescom Software Systems Testing, says:
"If you have an old Vantive [CRM] system, and you have to migrate all your information to a new [CRM] system, that is where a lot of the functional issues will occur. You have to do that migration and then verify that the information is there [in the new system]."

Although functional testing is often done toward the end of the development cycle, it can-and should, say experts-be started much earlier. Individual components and processes can be tested early on, even before it's possible to do functional testing on the entire system.

The Aberdeen Group, a Boston-based analyst firm, has noted that because today's applications require constant updating and redeployment, IT needs to think in terms of testing throughout the entire life-cycle of an application. In a 2003 InSight report, it noted that

"...cross-lifecycle software testing has a significant positive impact, not only on software quality but also on timely application delivery and ongoing maintenance costs, especially when it is integrated early in the application development cycle."

Although it doesn't get a lot of press, testing is arguably the most important phase in any software development project. Certainly testing is the best-the only-way to know whether your project or product is destined for success or doomed to failure before it goes live. The U.S. Commerce Department has estimated that buggy software costs nearly $60 billion annually, and that at least $22.2 billion worth of those bugs could have been prevented with more thorough testing.

And although you might like to think that your newly coded application is virtually bug-free, we all know by now that it just ain't so. Sandra Tise of Empirix, a testing tool maker, notes that there's simply no such thing as an error-free application: "Frankly, I know of no software test that I've done that has come back perfect."

There are many, many different ways to test software, but most tend to fall into two broad categories: functional testing and performance testing.

Testing tools and services
Naturally, there are products and services on the market to help developers test their products. Gartner, the IT research and consulting firm, defines the overall market of "automated software quality" as consisting of automated testing tools, software test management tools, and hosted testing services.

Hosted testing services, like any hosted offering, provide the hardware and software, and sometimes the labor as well. According to Gartner, load testing is the most service offered by hosted test vendors.

Software test management tools help in the planning and monitoring of the test effort and are particularly useful in establishing reusable test processes.

Testing tools allow developers to create scripts and run multiple scenarios, over and over again-something that would otherwise require a huge team of manual testers to accomplish. And if the application is one that must support thousands of concurrent users, an automated tool is virtually required. Companies such as Empirix, Mercury Interactive, Compuware, Segue, and IBM all make automated tools. They're not cheap, however; according to Povlot, performance testing tools run about $100,000, or $10,000 per testing user for a functional testing tool.

But the cost of testing is usually small compared with the potential expense of missing a deadline or having a key system fail on the first day of its rollout.

As Tise observes, "When you go and upgrade a mission critical application, you darn well better be thinking about testing."