1.
Introduction
The objective of this document is to provide a
roadmap for the testing life cycle and a constant source of information for the
strategy of testing.
The tasks identified in this document are:
·
Define
objectives
·
Identify
risks
·
Define
functional, GUI, and usability testing
·
Define
hardware/software requirements
·
Define
environmental requirements
·
Define
resource requirements, identify resources
·
Define
test procedures
o Create functional list
o Identify test conditions
o Create test matrix
·
Define
problem management reporting process and documentation
o Create Defects by Developer
Report
·
Define
acceptance criteria and approval identification
o Create final report
3.1
Identify and prioritize risk areas
The following are examples:
·
Risks
associated with not having documented Test Cases
Facilitating Regression Testing - The amount of time required to perform Regression
Testing by person/s other than the original tester is increased because the
steps to perform the tests are not explicitly defined.
Expected Results - The Expected Results are not explicitly defined which again
increases the learning curve for retesting the product by tester/s other than
the original tester.
Actual Results - The Actual Results are not captured for each step, thus there is no
current means of determining if the results have changed when retesting is
performed.
Þ
Mitigating
the risks
·
Risks
associated with fast turn around times
Functional Test Cases - As identified previously, there are certain risks
associated with the lack of Functional Test Cases for product testing. However, it is also noted that these risks
are minimized by the use of other test mechanisms.
Test Results documentation - As identified previously, Expected and Actual
test results are not being documented.
The amount of time to test each product and the expeditious requirements
for testing outweighs the current need to document results other than those
identified in the Defect Reports.
Identifying code fixes - In order to ensure that recurring issues can
easily be re-addressed in the future, code fixes are typically documented by
the developer within the defect-tracking tool.
Again, in keeping with the timelines for each tool release, specific
code changes are not being warehoused within the defect-reporting tool.
Full Regression Testing - Typically, a full Regression Test would be
performed prior to releasing a product to the user community. Again, due to the need to release each
product expediently, partial Regression Testing is performed and is usually
specific to the retesting of known fixes.
Þ
Mitigating
the risks
·
Risks
associated with other untested browsers
Based on the constant influx
of new browsers, it would be impossible to ensure that each browser has been
tested. Currently, Internet Explorer
and Netscape are used to test each product due to their popularity with the
majority of users in the marketplace.
Þ
Mitigating
the risks
·
Risks
associated with code changes applied to the test environment during the testing
process
Þ
Mitigating
the risks
3.2
Identify test dependencies on external milestones
3.3
Contingency plan for risks
Procedure for
Defining the Tests
To ensure that test conditions are defined in an
expeditious yet comprehensive manner, the following procedure has been
developed:
·
Determine
the intent of each function
·
Determine
any input and output for each function
·
Define
the inputs and outputs that will be used to test each function and document
them in the test matrix
·
Create
positive and negative test conditions and corresponding test data
·
Determine
the steps to exercise the data
Defining Test
Conditions
The following items will be included when defining
test conditions as applicable:
·
Required
fields
·
Boundary
values
·
Data
types
Test
Validation
Each test condition and its corresponding data must
have an expected result to determine if the function behaves as defined in the
specification. Any variance from the
expected result will be documented in a problem report.
The following items are common items to consider
when performing GUI Testing:
Menus/Menu
Items
·
Meaningful
menu bar labels (related to calling function)
·
Order
by frequency of use or importance
·
Drop-down
menu pulls down upon selection
Buttons
·
Meaningful
label names
·
Consistency
of label names
·
Consistency
of button placement and grouping
·
Consistency
of button size (width and height)
Toolbars
·
Consistency
of placement from page to page
·
Graphic
representation on buttons intuitively represent the function they are calling
·
Tool
tips used to assist user in learning associated functions
Check Boxes
·
Descriptive
labels
·
Used
only when multiple selections are an option (single options use radio buttons)
Color
·
Sufficient
contrast to reduce eye strain
·
Avoid
the use of red and green combinations due to color blindness differentiation
problems
·
Appropriate
use for attracting attention
Form Labels
·
Sufficient
field length to accommodate common screen resolutions
Text Fields
·
Field
length matches database storage size or scrolls to accommodate
Messages and
Message Boxes
·
Error
message text is consistent throughout the application
·
Error
messages are meaningful to the user and do not contain technical information
(this information should be captured for the technician but unannounced to the
user)
·
Informational
messages which interfere with the user's interaction with the product should be
limited or allow the user the ability to disable it, once learned
Page
Design/Layout
·
Graphics
and text are not cluttered and difficult to view
·
Layout
is logical and does not require the user to search for typical functions
Fonts
·
Custom
fonts should be interchangeable with system default fonts without degradation
of the page being viewed
Items to
consider when testing:
·
Are
the menu items descriptive?
·
Do
the buttons describe the appropriate action?
·
Are
the button sizes and alignment consistent?
·
Are
the button placements consistent, e.g., OK on the left and Cancel on the right
when placed horizontally?
·
What
is the overall appearance?
·
Are
the colors appealing?
·
Are
colors used consistently when designating specific functionality, e.g., blue
underscore to represent a hyperlink?
·
Is
the text easy to read? Are paragraphs
formatted in small sections (four or five sentences) versus several large, lengthy sentences?
·
Are
the messages informative?
·
If
the fonts used are not supplied with the typical operating system, are they
available dynamically through the application?
A checklist is provided in section 13.5.
The following items are common items to consider
when performing Usability Testing:
Navigation
The site should provide flexible navigation that
enables the user to move freely through the site. Types of navigation include:
·
Left-oriented
Buttons or links are vertically lined up on the left
side of the page versus the right side of the page to ensure that variances in
screen width do not result in unpredictable button visibility.
·
Top-oriented
Buttons or links are horizontally placed on the top
of the page, but must be placed in a manner which does not appear to conflict
with the current page headings.
Utilization of a separate frame is recommended for this type of button
placement.
·
Bottom-oriented
Buttons or links are horizontally placed at the
bottom of the page. Most often users
must scroll to see them unless they are placed in their own non-scrolling
frame.
·
Map
or center-oriented
Graphics or designs dominate the page like a map,
with the links or buttons placed with the appropriate graphic. It is important that these buttons appear
distinguishable from the graphics or text surrounding them to ensure that their
use is intuitive.
Site Map
(Navigational Aid)
The site could include a site map that aids the user
in finding his or her way around the site.
A site map facilitates the user's ability to memorize navigation of the
site.
Help
Context sensitive help, pop-up windows, tool-tips
assist the user in explaining how to perform different tasks or use the
application.
Links
Links bring you to another page or to another area
on the current page and are consistently and easily identifiable.
Response
Time/Performance
Response time is subjective to the user's
environment as well as the environment controlling the application under test
and any link between the 2 environments.
Thus response and performance are difficult to monitor without the use
of tools. Unacceptable response times
should still be noted, however, as they may warrant the acquisition of tools.
Security
Security should be specific to the type of
information it is protecting. It should
not hinder the user's ability to easily navigate through the system.
Items to
consider when testing:
·
Is
the application/product easy to use?
·
Is
the site easy to navigate through?
·
Are
'clickable' items obvious to the user?
·
Do
you know where a button or link will lead?
Are the buttons/links labeled properly?
·
Is
the most important content/information organized in a manner so that the user
has it readily available?
·
Do
you need to scroll down to view important information on a page?
·
Do
the links bring you to the proper pages?
Are you able to distinguish between links that have been visited and not
visited?
·
Is
the navigation from page to page consistent?
·
Can
you move to other areas within the site without having to retrace your steps backwards? Is the same main navigation toolbar resident
on all pages?
·
Are
navigational aids (site map, help information, documentation) available? Are they intuitive?
·
Is
the site secure? Can someone else get
to your information without entering your username and password?
·
What
type of response time are you receiving?
Is it taking an unacceptable amount of time to retrieve data? Is it taking an unacceptable amount of time
to process data?
A checklist is provided in section 13.5.
5.
Hardware and Software Requirements
Define the various browsers, TSR's (e.g., anti-virus
software, system utilities, etc.), modems and other peripheral devices to be
included with the application to be tested.
Generally, Internet Explorer and Netscape are used to test each product
on a dirty machine with at least one anti-virus TSR and a 56K modem.
Recommended Minimum Requirements
Browsers In
Use for Testing
TSRs In Use
for Testing
6.2
Configuration of hardware and software environment
6.3
Define clean and dirty machines
A Clean Machine is any workstation setup for the specific use of a
product. Only software defined by the
Developer is installed, and the environment is not compromised by any DLL’s or
OCX’s installed with other software products unrelated to the application under
test.
A Dirty Machine is any workstation being used which
has not been expressly setup for the products being tested. This machine could have different versions
of DLL’s and OCX’s that were installed with other products. This machine could be compromised by TSR
applications that keep conflicting versions of dependent files in memory.
Clean Machines allow QA to identify dependency files
by only installing files identified in the installation package. Dirty Machines allow QA to identify file
dependency conflicts with other software.
Internet products that do not require the downloading of software to the
user's PC are not impacted by the use of either clean or dirty machines,
per se. However, memory resource issues
may be uncovered from the use of a dirty machine due to the amount of memory
available to run an Internet application.
Each project will bring with it its own unique set
of testing requirements. Estimates need
to be based on the unique requirements of each test.
8.
Resources
|
Name |
Phone |
Email |
|
|
|
|
|
|
|
|
|
|
|
|
9.1
Define process for test commencement
9.2
Step-by Step (Getting Started)
User
Registration Setup
Installation
Application
Testing
A Problem Tracking Report will be used to
communicate issues found with each application under test. A sample of the Problem Tracking Report is
provided in Section 13.3.
The test results of the Functional, GUI and
Usability testing are defined within the Problem Reports and Final Report. The Final Report contains the final copy of
the Problem Report.