Other Software QA and Testing Resources 1

Top 5 List

comp.software.testing usenet news group - Via Google Groups web site (formerly the Deja News site), can be used to search through past postings; postings go back to 1995.

Stickyminds.com - Comprehensive software testing resource site associated with 'Better Software' Magazine, with articles, news, information on software testing and quality engineering, books, tools, conferences, message boards.

Cem Kaner's software testing site - Cem Kaner's site contains a large selection of his articles about software testing, legal issues, test management, and more (see the 'Publications' section of the site). Also see his 'badsoftware.com' website , a consumer and legal-issues orientation to software quality issues.

STORM - Software Testing Online Resources/MTSU - a well-organized site with listings of many links to software QA and testing-related web sites.

The comp.software.testing FAQ - The comp.software.testing FAQ; maintained by Raymond Rivest; excellent resource for testing-related conferences, mailing lists, books, periodicals, organizations, and links to other sites. Not maintained recently but still useful.

Software Testing Tutorial: Free Course - In this class, you will learn basic skills and concepts of Software Testing. Lessons are taught using REAL-LIFE SCENARIOS for improved learning. refer the tutorials sequentially one after the other.

This Software Testing Tutorials that took 400+ hours to create with beautifully annotated screenshot, VIDEOS and is very comprehensive. This tutorials are designed by a Thought Works veteran and I have personally edited them. The course covers:

• Testing Fundamentals
• Test Case Development
• Testing Techniques
• Test Management & Control
• Huge List of different Types of Testing
• Testing Certification Guide
• LIVE PROJECTS to simulate corporate test environments for beginners.

Return to top of Resources Listing

Software QA and Testing-related Organizations and Certifications

SEI - Software Engineering Institute web site; info about SEI technical programs, publications, bibliographies, some online documents, SEI courses and training, links to related sites.

American Society for Quality - American Society for Quality (formerly the American Society for Quality Control) web site; geared to quality issues in general, not just Software QA. ASQ is the largest quality organization in the world, with more than 100,000 members. Provides a wide variety of general quality-related certifications, as well as the CSQE (Certified Software Quality Engineer).

SPIN - Software Process Improvement Network, for those interested in improving software engineering practices. Organized into regional groups called "SPINs" that meet and share their experiences initiating and sustaining software process improvement programs. Annual meeting at the Software Engineering Process Group (SEPG) Conference, which is co-sponsored by the SEI and a regional SPIN. Web site lists links to regional SPINs worldwide.

IEEE Standards - IEEE web site; has Software Engineering Standards titles and prices; the topical areas for publications of interest would include listings under the 'Computer Engineering' section in the categories of Software Design/Development and Software Quality and Management.

Society for Software Quality - Has chapters in San Diego, Delaware, and Washington DC area; each with monthly meetings.

QAI - Quality Assurance Institute.

EOQ-SG - European Organization for Quality - Software Group, an independent not-for-profit organization founded in 1983. It is comprised of more than 30 national quality organizations and other institutions, enterprises and specialists.

Certification Information for Software QA and Test Engineers:

CSQE - ASQ (American Society for Quality) CSQE (Certified Software Quality Engineer) program - information on requirements, outline of required 'Body of Knowledge', listing of study references and more.

CSQA/CSTE - QAI (Quality Assurance Institute)'s program for CSQA (Certified Software Quality Analyst), CSTE (Certified Software Tester), and Certified Software Project Manager (CSPM) certifications.

ISEB Software Testing Certifications - The British Computer Society maintains a program of 2 levels of certifications - ISEB Foundation Certificate, Practitioner Certificate.

ISTQB Certified Tester - The International Software Testing Qualifications Board is a part of the European Organization for Quality - Software Group, based in Germany. Certifications and testing are admininstered by ISTQB organizations in each of a number of countries around the world. Two levels are available: Foundation and Advanced.

ASTQB - The American Software Testing Qualifications Board is a branch of the ISTQB Certifications are based on experience and a written test. Two levels are available: Foundation and Advanced.

Return to top of Resources Listing

Links to QA and Testing-related Magazines/Publications

Professional Tester - European-based quarterly English-language testing magazine. Web site includes archives with articles from past issues.

Software Test and Performance Magazine - New magazine started in 2004, from BZ Media. Table of contents for current and past contents available online; past issues are available for download.

Software Quality Professional Magazine - Published by the American Society for Quality; web site includes table of contents and abstracts of all articles, and full text of selected articles.

Better Software Magazine - Web site has full text of each print issue's featured article. Published by Software Quality Engineering, Inc.

Quality Techniques Newsletter - Monthly online QA and testing newsletter from Software Research, Inc.; includes current and past issues.

IT Metrics and Productivity Journal - Free on-line publication from Computer Aid, Inc. covering best practices in software development, maintenance, and management, process, metrics, estimation, IT governance, etc. See site's 'archive' link for large collection of past articles.

Methods and Tools - Software Methods and Tools e-newsletter web site by Martinig and Associates; regular articles are included on process improvement, testing, modeling, management, etc. Site includes current issues and past issues with full text of all articles; as well as extensive additional information and resources.

CMP Publications - Links to CMP's publications including Information Week, Software Development, TechNet, SysAdmin, Network Computing, C/C++ Users Journal, IT Architect,Secure Enterprise, Dr. Dobbs, MSDN Magazine, Wall Street and Technology, Byte, PERL Journal, and others. Most have searchable archives and some articles and reviews concerning software testing, web site test tools, and related issues.

Return to top of Resources Listing

General Software QA and Testing Resources

Agile Methodologies - Martin Fowler's online discussion of 'agile' methodologies (XP, Scrum, Crystal, FDD, DSDM, etc.) includes summaries of various approaches as well as reference information, and factors to consider in choosing these approaches.

XP Resources - Large collection of resources from Ron Jeffries about 'Extreme Programming' including a discussion of how QA fits into the XP approach, XP Magazine archives with articles such as 'Test-First Design', 'Incremental Requirements', 'Extreme Programming and the CMM', and more. Also see 'The Rules and Practices of Extreme Programming ' at the www.extremeprogramming.org web site.

Scrum - Web site of Advanced Development Methods, Inc and their Scrum methodology, a team-based agile approach to iteratively, incrementally develop software with rapidly changing requirements.

Crystal - Web site about Allistair Cockburn's collection of 'shrink-to-fit, human-powered software development methodologies' based on the premise that each project needs an approach appropriate to its particular member talents and people issues, and that efficiency is best maximized via frequent deliveries and improved communications. Methodology suggestions are grouped by team size (no larger than 50 as of 2004) and recommended for teams at a single geographic location working on non-life-critical projects.

WSR Consulting Group publications - Good collection of QA and Testing related articles with an emphasis on proper management of developer-customer relationships for software projects.

Practical Software and Systems Measurement - Web site with extensive information on software development metrics, sponsored by U.S. government. Site contains articles, reports, examples, and a free PC-based software tool to assiste in project-specific metrics development.

Software Estimation - December 2005 interview about software metrics and estimation from the IT Metrics and Productivity Journal .

Software Project Estimation - Good introductory article covering the basic issues of software project estimation, from the Software Productivity Centre in Vancouver.

Testing Education Articles - Collection of articles on software testing and the teaching of software testing from the Florida Institute of Technology funded by the U.S. National Science Foundation.

'Software Experts' site - Software engineering site oriented to microcontroller/embedded system environments, by Eberhard De Wille and Dana Vede. Site has sections on design, coding, refactoring, process, and a large section on testing.

ITIL - 'IT Infrastructure Library' - a set of best-practices guides on the management and provision of operational IT Services. From itSMF, a UK-based organization comprised of 1000 companies and government organizations worldwide that is developing and promoting best practice IT Service Management standards and qualifications. There is a related British Standards Institution BS15000 IT Service Management standard and certification scheme. An online organizational ITIL Service Management Self Assessment is available.

Big Ball of Mud - Outstanding essay on the 'de-facto standard software architecture', by Briane Foote and Joseph Yoder of the U. of Illinois at Urbana-Champaign. The 'Big Ball of Mud' architecture is defined as 'a casually, even haphazardly, structured system. Its organization, if one can call it that, is dictated more by expediency than design....The overall structure of the system may never have been well defined. If it was, it may have eroded beyond recognition.' They discuss why this architecture is so popular, advantages and disadvantages, and what can be done to improve such systems.

The Frameworks Quagmire - Old but still relevant article by Sarah A. Sheard summarizing and exploring the conflicts and complexities among various software process standards existing in the late 1990's - CMMi, SW-CMM, SDCE, Trillium, IEEE, FAA-iCMM, EIA-632, NATO-AQAP, MIL-STD-498, ISO/IEC-12207, ISO-9000, etc.

Satisfice.com Web Site - James Bach's Satisfice.com Web Site with a great collection of his articles on various aspects of software testing.

Bret Pettichord's Software Testing Hotlist - Web site with links to various test and QA-related info; not updated recently but still has a good list of test automation articles and other useful web sites.

BetaSoft Web Site - Wide variety of QA, testing, and automated testing discussion forums, sections for jobs and resumes, other resources.

SQATester.com - QA and Testing information, discussion forums, other resources.

Seven Steps to Test Automation Success - Good introductory article on how to approach automated testing; by Bret Pettichord.

Test Tools for Free Short article by Danny Faught discussing some free test tools, along with the basics of 'freeware', and information on a testing freeware newsletter. (Note: Many free web testing tools are also included among the web testing tools listed in the Softwareqatest.com 'Web Test Tools' page .)

Effective Performance Testing articles - Extensive collection of how-to and other information on performance testing at Scott Barber's web site.

Evaluating and Choosing the Right Tool - Elisabeth Hendrickson describes a five-step process for comparing, evaluating, and choosing the right test tool; from the Stickyminds.com web site.

Totally Data-Driven Automated Testing - Introductory article by Keith Zambelich on 'data-driven' and 'keyword-driven' automated testing. Article is from 1998 but information in the article is still valid and applicable.

JUnit.org - Site for test/development engineers using JUnit or one of the other XUnit testing frameworks. Has many useful articles and resources on automated Java regression testing and 'agile' testing processes in general.

Java GUI Testing - Short discussion of automated Java GUI testing issues, includes interesting discussion of methods of identifying a component in a GUI hierarchy for use in developing automated test scripts.

Function Point FAQ - Old but still useful article on the basics of the Function Point approach to estimating software development effort, based on an end-user/functional view of a software application's 'size'.

Illustrative Risks to the Public in the Use of Computer Systems - Enormous list of software, system, and related problems compiled by Peter Neumann/SRI International. Organized by categories such as space, defense, medical, stock market, elections, insurance, cryptography, etc. Includes related book list, other information. (Also see 'Risks Digest' listed below.)

Process Improvement Case Study Featuring Reviews and Inspections - Article titled 'Process Improvement: Case Study of an Improvement Program Featuring Reviews and Inspections' in Software Quality Professional magazine.

Will Bugs Eat Up the U.S. Lead in Software? - Business Week (International Edition) article comparing present state of U.S. software industry to U.S. automobile industry in the 1970's, when Japan took away huge market share with better and cheaper products by adopting Deming's and Juran's quality approaches. Discusses a possible repeat with the current US software industry losing software development market share to countries such as India. Entire issue and cover story reports on software problems - discussion of past problems, problems with software engineering, and possible fixes. Also see similar Infoweek article - 'The Big Picture: Killer Apps And Dead Bodies'.

ARIANE 5 Flight 501 Failure Report by the Inquiry Board - A rare and instructive detailed public analysis of a major software failure - the 1996 launch failure of the new Ariane 5 rocket. This is the official report of the inquiry board appointed by the French National Center for Space Studies and the European Space Agency. Also see the article 'Design by Contract: The Lessons of Ariane' which includes a discussion of the code reuse issues brought to light by the Ariane 5 failure.

Eiffel FAQ - FAQ site for a programming approach, based on the ideas of Bertrand Meyer, with the goal of improving software component reusability, extendibility and reliability using assertions, preconditions, and postconditions.

Internationalization Testing - Article at Sun's web site; includes guidelines and good check list of questions for testing an internationalized software product.

German-Language QA/Testing Web Site - German-language QA and testing web site, with articles, downloadable software and documentation, discussion forum, book reviews.

Risks Digest - Digest of the 'Forum on Risks to the Public in Computers and Related Systems'. Includes latest issue and archives covering software and system problems, vulnerabilities, disasters; based on the comp.risks newsgroup.

SEI Capability Maturity Models - SEI's CMM/CMMI web site, with info including the SW-CMM (Software Capability Maturity Model), P-CMM (People CMM) SA-CMM (Software Acquisition CMM), SE-CMM (Systems Engineering CMM), and IPD-CMM (Integrated Product Development CMM).

Construx Software Resources - Site with many useful resources such as CxOne, a lightweight, tailorable, modular, and scalable software engineering framework, estimation info and resources, various checklists, and Steve McConnell's 'Software Survival Guide' website.

CM Crossroads - Online community and resource center for configuration management, with library of resource links, interactive discussion forums, blogs, podcasts, monthly newsletter, Career Search and Jobs Board.

CM FAQ - Configuration Management FAQ edited by David Eaton; includes 'What is CM?', 'How should a CM system relate to process enforcement?', CM books and other resources, etc. Not updated recently but still a useful FAQ.

SR/Institute's Software Quality Hot List - Extensive collection of links to many QA and testing-related articles, resources, etc.

Uniform Computer Information Transactions Act (UCITA) - Text of controversial proposed U.S. laws (formerly Uniform Commercial Code Article 2B) concerning software quality. This would essentially implement new laws in all 50 states in the U.S. Additional info on UCITA controversies at Ralph Nader's CPT (Consumer Project on Technology) web site and Cem Kaner's BadSoftware.com web site. . As of late 2005, only the states of Maryland and Virginia had enacted UCITA since it was first proposed in 1999.

How To Build Reliable Code - Old but still-relevant article from a past issue of Byte Magazine.

NASA SW Engineering Lab site - U.S. National Aeronautics and Space Administration's Software Engineering Lab web site; includes publications such as "Recommended Approach to Software Development". Though this publication is several years old, it is a good online guide to the SW development process with extenisve sections on System Testing, Acceptance Testing, and other QA and testing issues. Other publications freely available at the site include "Cost and Schedule Estimation Study Report", "C Style Guide", "Software Measurement Guidebook", and "Improving the Software Testing Process...".

FDA Medical Device Software Validation Guidelines - U.S. Food and Drug Administration's 2002 guidelines for medical device software validation.

Negotiating Testing Resources - Excellent article by Cem Kaner about testing project planning and budgeting; from a 1996 software quality conference.

Software Engineering Resources - Large collection of useful information and links to many other sites and resources, all related to the SW engineering process including project planning and management, metrics, risk analysis, programming methods, OO SW engineering, testing, QA, CM. From R.S. Pressman, author of the book 'Software Engineering, A Practitioner's Approach'.

Software Technology Roadmap - Software Engineering Institute's technology descriptions listing - summaries of many software terms and technologies such as COM, OO Design, Function Point Analysis, etc

Software Test Coverage Analysis article Article containing a good discussion of test coverage analysis from Bullseye Testing Technology, maker of "C-Cover Test Coverage Analyzer" tool.

comp.object FAQ - Extensive FAQ for object oriented subjects; includes some info about object-oriented testing.

CMMI (Capability Maturity Model Integration) - A suite of process improvement models for product and service development and maintenance. The suite includes the CMMI-SW model, and there is a 'staged' and 'continuous' version. Each of the CMMI models can be coordinated with other CMMI models to enable enterprise-wide process improvement. CMMI-SW builds on the previous SW-CMM model which was 'sunsetted'.

Return to top of Resources Listing

Web QA and Testing Resources

Web Site Performance Testing - A collection of useful information on various aspects of performance testing, from Scott Barber's web site. Topics include: "Pinpointing and Exploiting Specific Performance Bottlenecks", "Common Performance Testing Challenges", "How Fast is Fast Enough", and "Introduction to Performance Testing". Although some of the information is not specifically oriented to web performance testing, it is still highly applicable.

Load Testing Of Web Sites - Article from IEEE Internet Computing about web load testing; useful overview from 2002.

Evaluating Web Sites for Accessibility - Article on the World Wide Web Consortium web site's 'Web Accessibility Initiative' section on how to assess and test web sites for accessibility issues.

Keynote Systems Resources Page - Useful collection of articles and information on web site performance testing. Also see the related site performance indices which lists a variety of business, consumer, government, and other web sites along with their 'performance index'.

CMSWatch - Web Content Management web site from CMSWorks Inc. with information, news, opinions, analysis, products, best-practices.

Handling and Avoiding Web Page Errors - Three part series from Microsoft site; covers sources of common Web page errors, how to handle run-time script errors, and techniques for avoiding preventable errors. Old but still useful.

Testing Database-Driven Web Sites - Old but still useful article on web testing from 'DBMS Magazine' (now 'Internet Systems' Magazine).

Return to top of Resources Listing

Web Security Testing Resources

Computer Audit FAQ - Good introductory information from IsecT Ltd. on 'Computer Audit', which refers to the analysis of computer systems and networks by examining the effectiveness of their technical and procedural controls (information security control systems) to minimise risks. Also has links to other resources, and some articles such as 'Strategic Approach to Information Security Management'

SANS Top 20 List - Web site of SANS (System Administration, Networking, and Security Institute), a cooperative research and education organization for sysadmins, security professionals, and network administrators for sharing lessons learned and solutions. List and descriptions of top Windows and UNIX internet security vulnerabilities, along with links to other resources.

CVE - Searchable, downloadable, and on-the-web 'Common Vulnerabilities and Exposures' list hosted by Mitre Corp. CVE goal is to standardize the names for all publicly known vulnerabilities and security exposures, so that security information can be efficiently shared and handled. Many security test tools are utilizing or planning on utilizing this standardized naming/numbering system.

W3 Security Resources - Large collection of information and resources on web security, including an FAQ, hosted by the W3C Consortium (the folks who set web standards/protocols, etc.)

Microsoft Security Site - Microsoft's web site for discussion of security issues for MS products, including their web server products. Includes security self-assessment information, home and business security information, etc.

Security Focus.Com - Site for news, forums, resources, vulnerability info, conference info, tools, etc. related to computer security including web and internet security issues. Search vulnerability database by keywords, date, vendor, version, etc.

CERIAS Security Archive - Former COAST site; Purdue University's computer security site; includes extensive collection of links organized by subject to security tools, info resources, etc. Tools list of more than 100 security tools includes many test tools such as CRACK, COPS, IPSend, Tiger, Secure Sun, etc.; all tools listed are available for download from the COAST site.

Computer Emergency Response Team site - CERT's internet security web site; includes web server security information; hosted by the Software Engineering Institute at Carnegie Mellon University.

Return to top of Resources Listing

Web Usability Resources

Useit.com - Jakob Nielsen's web usability website with such articles as 'How Users Read on the Web", 'Costs of User Testing', and 'Differences between Print Design and Web Design'.

IBM Ease of Use site - A wide variety of information from IBM about 'User Engineering', design guidelines, tools and other information, including web design suggestions, ideas, guidelines, etc.

Microsoft Usability Home Page - Microsoft's collection of info about their usability labs, related issues, listings of resources; includes info from the 'Human Factors and the Web' conference series. Also has a 'Usability Publications' section with a large collection of links to articles on usability issues.

Return to top of Resources Listing