Internet Test Plan Template

1. Introduction

The objective of this document is to provide a roadmap for the testing life cycle and a constant source of information for the strategy of testing.

2.2. Tasks

The tasks identified in this document are:

· Define objectives

· Identify risks

· Define functional, GUI, and usability testing

· Define hardware/software requirements

· Define environmental requirements

· Define resource requirements, identify resources

· Define test procedures

o Create functional list

o Identify test conditions

o Create test matrix

· Define problem management reporting process and documentation

o Create Defects by Developer Report

· Define acceptance criteria and approval identification

o Create final report

3. Risks

3.1 Identify and prioritize risk areas

The following are examples:

· Risks associated with not having documented Test Cases

Facilitating Regression Testing - The amount of time required to perform Regression Testing by person/s other than the original tester is increased because the steps to perform the tests are not explicitly defined.

Expected Results - The Expected Results are not explicitly defined which again increases the learning curve for retesting the product by tester/s other than the original tester.

Actual Results - The Actual Results are not captured for each step, thus there is no current means of determining if the results have changed when retesting is performed.

Þ Mitigating the risks

· Risks associated with fast turn around times

Functional Test Cases - As identified previously, there are certain risks associated with the lack of Functional Test Cases for product testing. However, it is also noted that these risks are minimized by the use of other test mechanisms.

Test Results documentation - As identified previously, Expected and Actual test results are not being documented. The amount of time to test each product and the expeditious requirements for testing outweighs the current need to document results other than those identified in the Defect Reports.

Identifying code fixes - In order to ensure that recurring issues can easily be re-addressed in the future, code fixes are typically documented by the developer within the defect-tracking tool. Again, in keeping with the timelines for each tool release, specific code changes are not being warehoused within the defect-reporting tool.

Full Regression Testing - Typically, a full Regression Test would be performed prior to releasing a product to the user community. Again, due to the need to release each product expediently, partial Regression Testing is performed and is usually specific to the retesting of known fixes.

Þ Mitigating the risks

· Risks associated with other untested browsers

Based on the constant influx of new browsers, it would be impossible to ensure that each browser has been tested. Currently, Internet Explorer and Netscape are used to test each product due to their popularity with the majority of users in the marketplace.

Þ Mitigating the risks

· Risks associated with code changes applied to the test environment during the testing process

Þ Mitigating the risks

3.2 Identify test dependencies on external milestones

3.3 Contingency plan for risks

4. Testing Strategy

4.1 Functional Testing

Procedure for Defining the Tests

To ensure that test conditions are defined in an expeditious yet comprehensive manner, the following procedure has been developed:

· Determine the intent of each function

· Determine any input and output for each function

· Define the inputs and outputs that will be used to test each function and document them in the test matrix

· Create positive and negative test conditions and corresponding test data

· Determine the steps to exercise the data

Defining Test Conditions

The following items will be included when defining test conditions as applicable:

· Required fields

· Boundary values

· Data types

Test Validation

Each test condition and its corresponding data must have an expected result to determine if the function behaves as defined in the specification. Any variance from the expected result will be documented in a problem report.

4.2 GUI Testing

The following items are common items to consider when performing GUI Testing:

Menus/Menu Items

· Meaningful menu bar labels (related to calling function)

· Order by frequency of use or importance

· Drop-down menu pulls down upon selection

Buttons

· Meaningful label names

· Consistency of label names

· Consistency of button placement and grouping

· Consistency of button size (width and height)

Toolbars

· Consistency of placement from page to page

· Graphic representation on buttons intuitively represent the function they are calling

· Tool tips used to assist user in learning associated functions

Check Boxes

· Descriptive labels

· Used only when multiple selections are an option (single options use radio buttons)

Color

· Sufficient contrast to reduce eye strain

· Avoid the use of red and green combinations due to color blindness differentiation problems

· Appropriate use for attracting attention

Form Labels

· Sufficient field length to accommodate common screen resolutions

Text Fields

· Field length matches database storage size or scrolls to accommodate

Messages and Message Boxes

· Error message text is consistent throughout the application

· Error messages are meaningful to the user and do not contain technical information (this information should be captured for the technician but unannounced to the user)

· Informational messages which interfere with the user's interaction with the product should be limited or allow the user the ability to disable it, once learned

Page Design/Layout

· Graphics and text are not cluttered and difficult to view

· Layout is logical and does not require the user to search for typical functions

Fonts

· Custom fonts should be interchangeable with system default fonts without degradation of the page being viewed

Items to consider when testing:

· Are the menu items descriptive?

· Do the buttons describe the appropriate action?

· Are the button sizes and alignment consistent?

· Are the button placements consistent, e.g., OK on the left and Cancel on the right when placed horizontally?

· What is the overall appearance?

· Are the colors appealing?

· Are colors used consistently when designating specific functionality, e.g., blue underscore to represent a hyperlink?

· Is the text easy to read? Are paragraphs formatted in small sections (four or five sentences) versus several large, lengthy sentences?

· Are the messages informative?

· If the fonts used are not supplied with the typical operating system, are they available dynamically through the application?

A checklist is provided in section 13.5.

4.3 Usability Testing

The following items are common items to consider when performing Usability Testing:

Navigation

The site should provide flexible navigation that enables the user to move freely through the site. Types of navigation include:

· Left-oriented

Buttons or links are vertically lined up on the left side of the page versus the right side of the page to ensure that variances in screen width do not result in unpredictable button visibility.

· Top-oriented

Buttons or links are horizontally placed on the top of the page, but must be placed in a manner which does not appear to conflict with the current page headings. Utilization of a separate frame is recommended for this type of button placement.

· Bottom-oriented

Buttons or links are horizontally placed at the bottom of the page. Most often users must scroll to see them unless they are placed in their own non-scrolling frame.

· Map or center-oriented

Graphics or designs dominate the page like a map, with the links or buttons placed with the appropriate graphic. It is important that these buttons appear distinguishable from the graphics or text surrounding them to ensure that their use is intuitive.

Site Map (Navigational Aid)

The site could include a site map that aids the user in finding his or her way around the site. A site map facilitates the user's ability to memorize navigation of the site.

Help

Context sensitive help, pop-up windows, tool-tips assist the user in explaining how to perform different tasks or use the application.

Links

Links bring you to another page or to another area on the current page and are consistently and easily identifiable.

Response Time/Performance

Response time is subjective to the user's environment as well as the environment controlling the application under test and any link between the 2 environments. Thus response and performance are difficult to monitor without the use of tools. Unacceptable response times should still be noted, however, as they may warrant the acquisition of tools.

Security

Security should be specific to the type of information it is protecting. It should not hinder the user's ability to easily navigate through the system.

Items to consider when testing:

· Is the application/product easy to use?

· Is the site easy to navigate through?

· Are 'clickable' items obvious to the user?

· Do you know where a button or link will lead? Are the buttons/links labeled properly?

· Is the most important content/information organized in a manner so that the user has it readily available?

· Do you need to scroll down to view important information on a page?

· Do the links bring you to the proper pages? Are you able to distinguish between links that have been visited and not visited?

· Is the navigation from page to page consistent?

· Can you move to other areas within the site without having to retrace your steps backwards? Is the same main navigation toolbar resident on all pages?

· Are navigational aids (site map, help information, documentation) available? Are they intuitive?

· Is the site secure? Can someone else get to your information without entering your username and password?

· What type of response time are you receiving? Is it taking an unacceptable amount of time to retrieve data? Is it taking an unacceptable amount of time to process data?

A checklist is provided in section 13.5.

5. Hardware and Software Requirements

Define the various browsers, TSR's (e.g., anti-virus software, system utilities, etc.), modems and other peripheral devices to be included with the application to be tested. Generally, Internet Explorer and Netscape are used to test each product on a dirty machine with at least one anti-virus TSR and a 56K modem.

5.1 Hardware

Recommended Minimum Requirements

5.2 Software

Browsers In Use for Testing

TSRs In Use for Testing

6. Environment Requirements

6.1 Client operating system/s

6.2 Configuration of hardware and software environment

6.3 Define clean and dirty machines

A Clean Machine is any workstation setup for the specific use of a product. Only software defined by the Developer is installed, and the environment is not compromised by any DLL’s or OCX’s installed with other software products unrelated to the application under test.

A Dirty Machine is any workstation being used which has not been expressly setup for the products being tested. This machine could have different versions of DLL’s and OCX’s that were installed with other products. This machine could be compromised by TSR applications that keep conflicting versions of dependent files in memory.

Clean Machines allow QA to identify dependency files by only installing files identified in the installation package. Dirty Machines allow QA to identify file dependency conflicts with other software. Internet products that do not require the downloading of software to the user's PC are not impacted by the use of either clean or dirty machines, per se. However, memory resource issues may be uncovered from the use of a dirty machine due to the amount of memory available to run an Internet application.

7. Test and Project Schedules

Each project will bring with it its own unique set of testing requirements. Estimates need to be based on the unique requirements of each test.

8. Resources

8.1 Contact Information

Name	Phone	Email

9. Test Procedures

9.1 Define process for test commencement

9.2 Step-by Step (Getting Started)

User Registration Setup

Installation

Application Testing

9.3 Define test maintenance

10. Control Procedures

10.1Problem Reporting

A Problem Tracking Report will be used to communicate issues found with each application under test. A sample of the Problem Tracking Report is provided in Section 13.3.

11. Retention Policy

11.1Test Results

The test results of the Functional, GUI and Usability testing are defined within the Problem Reports and Final Report. The Final Report contains the final copy of the Problem Report.

12. Approval

12.1 Acceptance criteria

12.2 Due Diligence defined

Due diligence includes the performance of Functional, GUI and Usability testing as defined within this Test Plan.

12.3 Approval identification

12.4 Reporting criteria

A Final Report is generated at the conclusion of testing. The information provided in this report includes:

· Tests performed

· User Interface Analysis

· Documentation Analysis

· Defects/Issues Found

· Summary

· Final Problem Tracking Report

13. Templates and Samples

The following pages contain the templates for the Functional List, the Test Matrix, and the Defects by Developer spreadsheet. Also included in this section are samples for the Problem Tracking Report and the Final Report.

13.1 Functional List

The following is the template for a Functional list.

NAME OF APPLICATION

Functional List

Function Name	Description	Risk Factor [1]	Time Estimate

13.2 Test Matrix

The following is the template for a Test Matrix.

Function Name	Test Conditions
Function	1.
	2.
	3.
	4.

Function	1.
	2.
	3.
	4.

Function	1.
	2.
	3.
	4.

Function	1.
	2.
	3.
	4.

13.3 Sample Problem Tracking Report

The following is a sample page of the Problem Tracking Report as described in Section 10.1, Problem Reporting.

Tester /Developer	Browser	Function	Problem Description /Status	Severity

13.4 GUI and Usability Checklist

Items	Questions	Answers
Menu Bars	Are the menu bar labels meaningful?	Yes	No	N/A
	Are the labels descriptive of the associated functions being performed?	Yes	No	N/A
	Are the menu bar labels ordered by frequency of use and/or importance?	Yes	No	N/A
Buttons	Do the buttons have meaningful label names? Do they describe the appropriate actions?	Yes	No	N/A
	Are the labels for the buttons consistent from page to page?	Yes	No	N/A
	Is the button placement and grouping consistent? (e.g., 'OK' on the left and 'Cancel' on the right when placed horizontally, 'OK on the top and 'Cancel' beneath when placed vertically)	Yes	No	N/A
	Are the button sizes consistent (width and height)?	Yes	No	N/A
Toolbars	Do the toolbars have consistent placement from page to page?	Yes	No	N/A
	Do the icons on the toolbar intuitively represent the function that they call?	Yes	No	N/A
	Are tool tips available to assist the user in learning the associated tools on the toolbar?	Yes	No	N/A
Check Boxes and Radio Buttons	Do the check boxes contain descriptive labels?	Yes	No	N/A
	Are check boxes used only for functions allowing multiple selections?	Yes	No	N/A
	Are radio buttons used for single selection functions?	Yes	No	N/A
Color	Is there sufficient contrast to reduce eye strain?	Yes	No	N/A
	Is the use of red and green color combinations minimized to avoid color blindness differentiation problems?	Yes	No	N/A
	Is there appropriate use of color for attracting attention?	Yes	No	N/A
	Are the colors appealing?	Yes	No	N/A
	Are colors used consistently when designating specific functionality, e.g., blue underscores to represent hyperlinks?	Yes	No	N/A
Form Labels	Are form labels' field length sufficient to accommodate common screen resolutions?	Yes	No	N/A
Text Fields	Do the field lengths of the input text fields match their corresponding database storage size? If not, are the text fields scrollable to accommodate the full length of the database field?	Yes	No	N/A
Messages	Is the error message text consistent throughout the application?	Yes	No	N/A
	Are the error messages meaningful to the user?	Yes	No	N/A
	Do the error messages contain non-technical information?	Yes	No	N/A
	Are informational messages that interfere with the user's interaction with the product limited? If not, is the ability to disable informational messages available once the product is learned?	Yes	No	N/A
Page Design	Are graphics and text arranged on the pages so that they are easy to view and not cluttered?	Yes	No	N/A
	Is the layout logical so that the user does not have to search for typical functions?	Yes	No	N/A
	Is the text easy to read?	Yes	No	N/A
	Are paragraphs formatted in small sections of four or five sentences instead of several large, lengthy sentences?	Yes	No	N/A
Fonts	Are custom fonts interchangeable with system default fonts without degradation of the page being viewed?	Yes	No	N/A
Fonts	If the fonts used are not supplied with the typical operating system, are they available dynamically through the application?	Yes	No	N/A
Navigation	Is the site easy to navigate through?	Yes	No	N/A
	Are 'clickable' items obvious to the user?	Yes	No	N/A
	Does the application adequately identify where a button or link will lead the user to? Are the buttons/links labeled properly?	Yes	No	N/A
	Is the most important content/information organized in a manner so that the user can readily access it?	Yes	No	N/A
	Is important information located on the page so that you don't have to scroll down to get to it or see that it is present?	Yes	No	N/A
	Do the links bring you to the proper pages?	Yes	No	N/A
Navigation (cont'd)	Are you able to distinguish between pages visited and pages not visited?	Yes	No	N/A
	Is the navigation from page to page consistent?	Yes	No	N/A
	Can you move to other areas within the site without having to retrace your steps backwards? (Is the main navigation toolbar resident on all pages?)	Yes	No	N/A
	Is help information/documentation available?	Yes	No	N/A
	Is there a site map available?	Yes	No	N/A
	Is the product easy to use?	Yes	No	N/A
Security	Is the site secure? Are your username and password required to ensure that there is restricted access to your information?	Yes	No	N/A
Response Time	Is the application taking an acceptable amount of time to retrieve data?	Yes	No	N/A
Response Time	Is the application taking an acceptable amount of time to process data?	Yes	No	N/A

[1] Risk Factors:

High: Any feature within the product which must be tested to ensure that the function works as defined in the Product Specification.

Medium: Recommended to be included in the product testing although this may be impacted by the release schedule.

Low: May or may not be tested based on the time available to test, but is not considered critical to the successful release of the product.