Internet Test Plan Template

 

1.      Introduction

 

2.      Objectives and Tasks

 

2.1.   Objectives

 

The objective of this document is to provide a roadmap for the testing life cycle and a constant source of information for the strategy of testing.

 

2.2.            Tasks

 

The tasks identified in this document are:

 

·        Define objectives

·        Identify risks

·        Define functional, GUI, and usability testing

·        Define hardware/software requirements

·        Define environmental requirements

·        Define resource requirements, identify resources

·        Define test procedures

o       Create functional list

o       Identify test conditions

o       Create test matrix

·        Define problem management reporting process and documentation

o       Create Defects by Developer Report

·        Define acceptance criteria and approval identification

o       Create final report

3.      Risks

 

3.1              Identify and prioritize risk areas

 

The following are examples:

 

·        Risks associated with not having documented Test Cases

 

Facilitating Regression Testing - The amount of time required to perform Regression Testing by person/s other than the original tester is increased because the steps to perform the tests are not explicitly defined.

 

Expected Results - The Expected Results are not explicitly defined which again increases the learning curve for retesting the product by tester/s other than the original tester.

 

Actual Results - The Actual Results are not captured for each step, thus there is no current means of determining if the results have changed when retesting is performed.

 

Þ    Mitigating the risks

 

·        Risks associated with fast turn around times

 

Functional Test Cases - As identified previously, there are certain risks associated with the lack of Functional Test Cases for product testing.  However, it is also noted that these risks are minimized by the use of other test mechanisms.

 

Test Results documentation - As identified previously, Expected and Actual test results are not being documented.  The amount of time to test each product and the expeditious requirements for testing outweighs the current need to document results other than those identified in the Defect Reports.

 

Identifying code fixes - In order to ensure that recurring issues can easily be re-addressed in the future, code fixes are typically documented by the developer within the defect-tracking tool.  Again, in keeping with the timelines for each tool release, specific code changes are not being warehoused within the defect-reporting tool. 

 

Full Regression Testing - Typically, a full Regression Test would be performed prior to releasing a product to the user community.  Again, due to the need to release each product expediently, partial Regression Testing is performed and is usually specific to the retesting of known fixes.

 

Þ    Mitigating the risks

 

·        Risks associated with other untested browsers

 

Based on the constant influx of new browsers, it would be impossible to ensure that each browser has been tested.  Currently, Internet Explorer and Netscape are used to test each product due to their popularity with the majority of users in the marketplace.

 

Þ    Mitigating the risks

 

·        Risks associated with code changes applied to the test environment during the testing process

 

Þ    Mitigating the risks

 

3.2              Identify test dependencies on external milestones

 

3.3              Contingency plan for risks

 

4.      Testing Strategy

 

4.1              Functional Testing

 

Procedure for Defining the Tests

 

To ensure that test conditions are defined in an expeditious yet comprehensive manner, the following procedure has been developed:

 

·        Determine the intent of each function

·        Determine any input and output for each function

·        Define the inputs and outputs that will be used to test each function and document them in the test matrix

·        Create positive and negative test conditions and corresponding test data

·        Determine the steps to exercise the data

 

Defining Test Conditions

 

The following items will be included when defining test conditions as applicable:

 

·        Required fields

·        Boundary values

·        Data types

 


Test Validation

 

Each test condition and its corresponding data must have an expected result to determine if the function behaves as defined in the specification.  Any variance from the expected result will be documented in a problem report.

 

4.2  GUI Testing

 

The following items are common items to consider when performing GUI Testing:

 

Menus/Menu Items

·        Meaningful menu bar labels (related to calling function)

·        Order by frequency of use or importance

·        Drop-down menu pulls down upon selection

 

Buttons

·        Meaningful label names

·        Consistency of label names

·        Consistency of button placement and grouping

·        Consistency of button size (width and height)

 

Toolbars

·        Consistency of placement from page to page

·        Graphic representation on buttons intuitively represent the function they are calling

·        Tool tips used to assist user in learning associated functions

 

Check Boxes

·        Descriptive labels

·        Used only when multiple selections are an option (single options use radio buttons)

 

Color

·        Sufficient contrast to reduce eye strain

·        Avoid the use of red and green combinations due to color blindness differentiation problems

·        Appropriate use for attracting attention

 

Form Labels

·        Sufficient field length to accommodate common screen resolutions

 

Text Fields

·        Field length matches database storage size or scrolls to accommodate

 

Messages and Message Boxes

·        Error message text is consistent throughout the application

·        Error messages are meaningful to the user and do not contain technical information (this information should be captured for the technician but unannounced to the user)

·        Informational messages which interfere with the user's interaction with the product should be limited or allow the user the ability to disable it, once learned

 

Page Design/Layout

·        Graphics and text are not cluttered and difficult to view

·        Layout is logical and does not require the user to search for typical functions

 

Fonts

·        Custom fonts should be interchangeable with system default fonts without degradation of the page being viewed

 

Items to consider when testing:

 

·        Are the menu items descriptive?

 

·        Do the buttons describe the appropriate action?

 

·        Are the button sizes and alignment consistent?

 

·        Are the button placements consistent, e.g., OK on the left and Cancel on the right when placed horizontally?

 

·        What is the overall appearance?

 

·        Are the colors appealing?

 

·        Are colors used consistently when designating specific functionality, e.g., blue underscore to represent a hyperlink?

 

·        Is the text easy to read?  Are paragraphs formatted in small sections (four or five sentences)  versus several large, lengthy sentences?

 

·        Are the messages informative?

 

·        If the fonts used are not supplied with the typical operating system, are they available dynamically through the application?

 

A checklist is provided in section 13.5.

 

4.3              Usability Testing

 

The following items are common items to consider when performing Usability Testing:

 

Navigation

The site should provide flexible navigation that enables the user to move freely through the site.  Types of navigation include:

·        Left-oriented

Buttons or links are vertically lined up on the left side of the page versus the right side of the page to ensure that variances in screen width do not result in unpredictable button visibility.

·        Top-oriented

Buttons or links are horizontally placed on the top of the page, but must be placed in a manner which does not appear to conflict with the current page headings.  Utilization of a separate frame is recommended for this type of button placement.

·        Bottom-oriented

Buttons or links are horizontally placed at the bottom of the page.  Most often users must scroll to see them unless they are placed in their own non-scrolling frame.

·        Map or center-oriented

Graphics or designs dominate the page like a map, with the links or buttons placed with the appropriate graphic.  It is important that these buttons appear distinguishable from the graphics or text surrounding them to ensure that their use is intuitive.

 


Site Map (Navigational Aid)

The site could include a site map that aids the user in finding his or her way around the site.  A site map facilitates the user's ability to memorize navigation of the site.

 

Help

Context sensitive help, pop-up windows, tool-tips assist the user in explaining how to perform different tasks or use the application.

 

Links

Links bring you to another page or to another area on the current page and are consistently and easily identifiable.

 

Response Time/Performance

Response time is subjective to the user's environment as well as the environment controlling the application under test and any link between the 2 environments.  Thus response and performance are difficult to monitor without the use of tools.  Unacceptable response times should still be noted, however, as they may warrant the acquisition of tools.

 

Security

Security should be specific to the type of information it is protecting.  It should not hinder the user's ability to easily navigate through the system.

 

Items to consider when testing:

 

·        Is the application/product easy to use?

 

·        Is the site easy to navigate through?

 

·        Are 'clickable' items obvious to the user?

 

·        Do you know where a button or link will lead?  Are the buttons/links labeled properly? 

 

·        Is the most important content/information organized in a manner so that the user has it readily available?

 

·        Do you need to scroll down to view important information on a page?

 

·        Do the links bring you to the proper pages?  Are you able to distinguish between links that have been visited and not visited?

 

·        Is the navigation from page to page consistent?

 

·        Can you move to other areas within the site without having to retrace your steps backwards?  Is the same main navigation toolbar resident on all pages?

 

·        Are navigational aids (site map, help information, documentation) available?  Are they intuitive?

 

·        Is the site secure?  Can someone else get to your information without entering your username and password?

 

·        What type of response time are you receiving?  Is it taking an unacceptable amount of time to retrieve data?  Is it taking an unacceptable amount of time to process data?

 

A checklist is provided in section 13.5.

 

5.      Hardware and Software Requirements

 

Define the various browsers, TSR's (e.g., anti-virus software, system utilities, etc.), modems and other peripheral devices to be included with the application to be tested.  Generally, Internet Explorer and Netscape are used to test each product on a dirty machine with at least one anti-virus TSR and a 56K modem. 

 

5.1              Hardware

 

Recommended Minimum Requirements

 

5.2              Software

 

Browsers In Use for Testing

 

TSRs In Use for Testing

 

6.      Environment Requirements

 

6.1              Client operating system/s

 

6.2              Configuration of hardware and software environment

 

6.3              Define clean and dirty machines

 

A Clean Machine is any workstation setup for the specific use of a product.  Only software defined by the Developer is installed, and the environment is not compromised by any DLL’s or OCX’s installed with other software products unrelated to the application under test.

 

A Dirty Machine is any workstation being used which has not been expressly setup for the products being tested.  This machine could have different versions of DLL’s and OCX’s that were installed with other products.  This machine could be compromised by TSR applications that keep conflicting versions of dependent files in memory. 

 

Clean Machines allow QA to identify dependency files by only installing files identified in the installation package.  Dirty Machines allow QA to identify file dependency conflicts with other software.  Internet products that do not require the downloading of software to the user's PC are not impacted by the use of either clean or dirty machines, per se.  However, memory resource issues may be uncovered from the use of a dirty machine due to the amount of memory available to run an Internet application.

 

7.      Test and Project Schedules

 

Each project will bring with it its own unique set of testing requirements.  Estimates need to be based on the unique requirements of each test.

 

8.      Resources

 

8.1              Contact Information

 

Name

Phone

Email

 

 

 

 

 

 

 

 

 

 

 

9.      Test Procedures

 

9.1              Define process for test commencement

 

9.2              Step-by Step (Getting Started)

 

 

User Registration Setup

 

Installation

 

Application Testing

 

9.3              Define test maintenance

 

10.  Control Procedures

 

10.1Problem Reporting

 

A Problem Tracking Report will be used to communicate issues found with each application under test.  A sample of the Problem Tracking Report is provided in Section 13.3. 

 

11.  Retention Policy

 

11.1Test Results

 

The test results of the Functional, GUI and Usability testing are defined within the Problem Reports and Final Report.  The Final Report contains the final copy of the Problem Report.

 

12.  Approval

 

12.1          Acceptance criteria

 

12.2          Due Diligence defined

 

Due diligence includes the performance of Functional, GUI and Usability testing as defined within this Test Plan.

 

12.3          Approval identification

 


12.4          Reporting criteria

 

A Final Report is generated at the conclusion of testing.  The information provided in this report includes:

 

·        Tests performed

·        User Interface Analysis

·        Documentation Analysis

·        Defects/Issues Found

·        Summary

·        Final Problem Tracking Report

 

13.  Templates and Samples

 

The  following pages contain the templates for the Functional List, the Test Matrix, and the Defects by Developer spreadsheet.  Also included in this section are samples for the Problem Tracking Report and the Final Report.


13.1     Functional List

 

The following is the template for a Functional list.

 

NAME OF APPLICATION

Functional List

 

Function

Name

 

Description

Risk

Factor [1]

Time

Estimate

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


13.2     Test Matrix

 

The following is the template for a Test Matrix.

 

 

Function

Name

Test

Conditions

Function

1.       

2.       

3.       

4.       

 

 

Function

1.       

2.       

3.       

4.       

 

 

Function

1.       

2.       

3.       

4.       

 

 

Function

1.                   

2.                   

3.                   

4.                   


13.3     Sample Problem Tracking Report

 

 

The following is a sample page of the Problem Tracking Report as described in Section 10.1,  Problem Reporting.

 

Tester

/Developer

Browser

Function

Problem Description

/Status

Severity

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


13.4     GUI and Usability Checklist

 

Items

Questions

Answers

Menu Bars

Are the menu bar labels meaningful?

 Yes

 No

N/A

Are the labels descriptive of the associated functions being performed?

 Yes

 No

N/A

Are the menu bar labels ordered by frequency of use and/or importance?

 Yes

 No

N/A

Buttons

Do the buttons have meaningful label names?  Do they describe the appropriate actions?

 Yes

 No

N/A

Are the labels for the buttons consistent from page to page?

 Yes

 No

N/A

Is the button placement and grouping consistent? (e.g., 'OK' on the left and 'Cancel' on the right when placed horizontally, 'OK on the top and 'Cancel' beneath when placed vertically)

 Yes

 No

N/A

Are the button sizes consistent (width and height)?

 Yes

 No

N/A

Toolbars

Do the toolbars have consistent placement from page to page?

 Yes

 No

N/A

Do the icons on the toolbar intuitively represent the function that they call?

 Yes

 No

N/A

Are tool tips available to assist the user in learning the associated tools on the toolbar?

 Yes

 No

N/A

Check Boxes and Radio Buttons

Do the check boxes contain descriptive labels?

 Yes

 No

N/A

Are check boxes used only for functions allowing multiple selections?

 Yes

 No

N/A

Are radio buttons used for single selection functions?

 Yes

 No

N/A

Color

Is there sufficient contrast to reduce eye strain?

 Yes

 No

N/A

Is the use of red and green color combinations minimized to avoid color blindness differentiation problems?

 Yes

 No

N/A

Is there appropriate use of color for attracting attention?

 Yes

 No

N/A

Are the colors appealing?

 

 Yes

 No

N/A

Are colors used consistently when designating specific functionality, e.g., blue underscores to represent hyperlinks?

 Yes

 No

N/A

Form Labels

Are form labels' field length sufficient to accommodate common screen resolutions?

 Yes

 No

N/A

Text

 Fields

Do the field lengths of the input text fields match their corresponding database storage size?  If not, are the text fields scrollable to accommodate the full length of the database field?

 Yes

 No

N/A

Messages

 

Is the error message text consistent throughout the application?

 Yes

 No

N/A

Are the error messages meaningful to the user?

 Yes

 No

N/A

Do the error messages contain non-technical information?

 Yes

 No

N/A

Are informational messages that interfere with the user's interaction with the product limited?  If not, is the ability to disable informational messages available once the product is learned?

 Yes

 No

N/A

Page Design

Are graphics and text arranged on the pages so that they are easy to view and not cluttered?

 Yes

 No

N/A

Is the layout logical so that the user does not have to search for typical functions?

 Yes

 No

N/A

Is the text easy to read?

 

 Yes

 No

N/A

Are paragraphs formatted in small sections of four or five sentences instead of several large, lengthy sentences?

 Yes

 No

N/A

Fonts

Are custom fonts interchangeable with system default fonts without degradation of the page being viewed?

 Yes

 No

N/A

If the fonts used are not supplied with the typical operating system, are they available dynamically through the application?

 Yes

 No

N/A

Navigation

Is the site easy to navigate through?

 Yes

 No

N/A

Are 'clickable' items obvious to the user?

 

 Yes

 No

N/A

Does the application adequately identify where a button or link will lead the user to?  Are the buttons/links labeled properly?

 Yes

 No

N/A

Is the most important content/information organized in a manner so that the user can readily access it?

 Yes

 No

N/A

Is important information located on the page so that you don't have to scroll down to get to it or see that it is present?

 Yes

 No

N/A

Do the links bring you to the proper pages?

 

 Yes

 No

N/A

Navigation (cont'd)

Are you able to distinguish between pages visited and pages not visited?

 Yes

 No

N/A

Is the navigation from page to page consistent?

 Yes

 No

N/A

Can you move to other areas within the site without having to retrace your steps backwards?  (Is the main navigation toolbar resident on all pages?)

 Yes

 No

N/A

Is help information/documentation available?

 

 Yes

 No

N/A

Is there a site map available?

 

 Yes

 No

N/A

Is the product easy to use?

 Yes

 No

N/A

Security

Is the site secure?  Are your username and password required to ensure that there is restricted access to your information?

 Yes

 No

N/A

Response Time

Is the application taking an acceptable amount of time to retrieve data?

 Yes

 No

N/A

Is the application taking an acceptable amount of time to process data?

 Yes

 No

N/A

 

 



[1] Risk Factors:  

High: Any feature within the product which must be tested to ensure that the function works as defined in the Product Specification.

Medium:  Recommended to be included in the product testing although this may be impacted by the release schedule.

Low:  May or may not be tested based on the time available to test, but is not considered critical to the successful release of the product.