Quality Control | Basic Concepts

Quality control describes the directed use of testing to measure the achievement of a specified standard. Quality control is a formal (as in structured, not as in wearing-a-tuxedo fancy) use of testing. Quality control is a superset of testing, although it often used synonymously with testing. Roughly, you test to see if something is broken, and with quality control you set limits that say, in effect, if this particular stuff is broken then whatever you're testing fails.

Yet another way of looking at the difference between testing and quality control is to consider the difference between a test as an event and a test as a part of a system. For example, let's say our test is the measurement of your ability to assemble a jigsaw puzzle in one hour. We test you today, and you complete the puzzle in fifty-eight minutes, so you pass. This seems pretty trivial, but say that there is some need in your life requiring you to solve puzzles quickly: we tested you once, but we must verify that you can meet this weird requirement continually over time. The solution is to test you at regular intervals, which will allow us to see if you can still be successful when under stress, when you haven't slept, when your workload is high -- our quality control approach to this issue says "you must finish puzzle in one hour or less" and "we will test this requirement periododically over time". And as a side effect of this quality control testing, you might find that you are more likely to improve in your puzzle solving skills because of the repeated practice; this is the beginning of a shift to quality assurance.

A structured quality control program becomes necessary when the stakes rise: your site is supposed to make money, or communicate a corporate message, or extend a brand, or disseminate important information. Quality control also becomes necessary when the team grows, or you partner with more companies, or your site gets more visitors. You need quality control as soon as it becomes important to prove, through the appearance and functionality of your web site, that your site has the legitimacy and professionalism to stand behind its message.

Or, if you prefer the short list, you need to establish a quality control program if:

1. you are working as part of a team that is building and/or maintaining a big web site, and your responsibility is for testing, quality control, or quality assurance
2. you are delivering a site to a customer, i.e., YOU are the contractor
3. you are receiving site code from a contractor, agency, or technology partner

Quality Control | The Test Plan

For quality control to be effective, you must test the same things the same way every time you test. When you change your tests, your results become inconsistent. You need a test plan.

A test plan is simply a high-level summary of the areas (functionality, elements, regions, etc.) you will test, how frequently you will test them, and where in the development or publication process you will test them. A test plan also needs an estimate of the duration of testing, and statement of any required resources.

The major phases of a web site need test plans, because the focus and emphasis of testing will change over time. Testing a new site in development is very different from testing a site that has been up and running for some time. Furthermore, any changes to the website code, incremental of major, requires regression test plans.

Clearly, you need to decide what you will test. Understand your site's mission statement, statement of goals, business plan -- whatever it is called, your site should have a concrete explanation of the "vision" behind its creation and the hoped-for "path" for its success. If your site has no such explicit statement of direction, then the codifying of such a statement should be your first goal.

To help define what you should test, ask yourself these types of questions:

• Why did I make this site? What is the site's purpose?
• What are the business goals, if any, behind this site?
• What has to work for this site to be effective? What has to work for this to even function as a web site?
• Who is the audience for this site? Can they use this site?
• What is core functionality offered by this site? Can all users at least access this core functionality?

Use the answers to these questions to decide what needs to be tested, then develop your testcases.

Quality Control | The Importance of Testcases

Much quality control testing for a web site involves testing the site-as-a-whole because HTML and HTTP must follow certain general protocols, standards, and rules of syntax; testing for broad patterns of behavior is a good first step. Most of the following items (on this non-all-inclusive list) can be tested with automated tools:

• basic adherence to an HTML DTD (document type definition)
• links are not broken
• links point at correct targets
• adherence to a coding style guide or standard; do graphics have ALT values? do graphics have HEIGHT and WIDTH values?
• adherence to a content style guide or standard
• correctness of graphics; does the graphic portray what it is supposed to portray?
• correctness of dynamic content or includes
• application functionality; if your site offers the ability to do something, like search or purchase online, does it work?
• basic compatibility; does your site work in all browsers? does it work in browsers that it was required to work with?
• basic performance; are the pages within a specified page-weight? does the site work acceptably with slower modem speeds?

Not all of these areas are easy to test with broad, automated tests. Firing up a link-checker won't tell you if your search function is working, or working as it is supposed to. For areas like application functionality, you must create specific, individual tests; you must devise scenarios based on expected user behavior, scenarios that describe how a user will interact with the functionality.

Use these scenarios to create testcases consisting of the specific steps a user would follow to accomplish these scenarios. For example, if a scenario calls for a user to add an item to a shopping cart in order to purchase it, your testcase should include every action a user would follow to complete the scenario. Yes, this is tedious, but the value is a test that can be assigned to inexperienced testers and repeated over and over and over.

[Learn about types of tests | learn more about testcases.]

Quality Control | Some Issues to Consider

The goals you set for your site will have a great deal to do with the kind of site you have. Commerce sites have distinctly different requirements than do personal home pages, or even corporate presence sites.

Quality control can be difficult when you find your testing resources limited or overextended. You will often find it impossible to test everything, or to verify every link, or to read every page for deviations from the style guide. You must develop some consistent testcases to check the major problem areas, automating tests if reasonable, and you must develop feel for your site -- as trite as that sounds --

Testing and creating testcases is always a learning experience. As you test and refine your testcases you will find a balance between not enough testing and just plain overkill, and between extremely detailed testcases and simple spot checks. I'm not sure there is a best way to do "quality control" (besides focusing on quality assurance) since every web site is different, but a target of zero defects is always a good thing; the trick is in revising your testcases to continually pinpoint any problem areas.

The quality control process has the major shortcoming of being reactive to problems. Quality control sets the standards for the web site and tests for web components that fall below those standards, but does nothing to improve the quality of web components. The strictest testing of the output will not necessarily improve the quality of the input -- this challenge is met by the quality assurance process.

[Learn about types of tests | learn more about testcases | expand into quality assurance.]