Testing Documentation
Testing documentation is the documentation of artifacts that are created during or before the testing of a software application. Documentation reflects the importance of processes for the customer, individual and organization.
Types of test document
In software testing, we have various types of test document, which are as follows:
Test scenarios
Test case
Test plan
Requirement traceability matrix(RTM)
Test strategy
Test data
Bug report
Test execution report
******************************************************************************
Test Scenarios
It is a document that defines the multiple ways or combinations of testing the application. Generally, it is prepared to understand the flow of an application. It does not consist of any inputs and navigation steps.
******************************************************************************
Test case
It is an in-details document that describes step by step procedure to test an application. It consists of the complete navigation steps and inputs and all the scenarios that need to be tested for the application. We will write the test case to maintain the consistency, or every tester will follow the same approach for organizing the test document.
******************************************************************************
Test plan
It is a document that is prepared by the managers or test lead. It consists of all information about the testing activities. The test plan consists of multiple components such as Objectives, Scope, Approach, Test Environments, Test methodology, Template, Role & Responsibility, Effort estimation, Entry and Exit criteria, Schedule, Tools, Defect tracking, Test Deliverable, Assumption, Risk, and Mitigation Plan or Contingency Plan.
******************************************************************************
Requirement Traceability Matrix (RTM)
The Requirement traceability matrix [RTM] is a document which ensures that all the test case has been covered. This document is created before the test execution process to verify that we did not miss writing any test case for the particular requirement.
******************************************************************************
Test strategy
The test strategy is a high-level document, which is used to verify the test types (levels) to be executed for the product and also describe that what kind of technique has to be used and which module is going to be tested. The Project Manager can approve it. It includes the multiple components such as documentation formats, objective, test processes, scope, and customer communication strategy, etc. we cannot modify the test strategy.
******************************************************************************
Test data
It is data that occurs before the test is executed. It is mainly used when we are implementing the test case. Mostly, we will have the test data in the Excel sheet format and entered manually while performing the test case.
The test data can be used to check the expected result, which means that when the test data is entered, the expected outcome will meet the actual result and also check the application performance by entering the in-correct input data.
******************************************************************************
Bug report
The bug report is a document where we maintain a summary of all the bugs which occurred during the testing process. This is a crucial document for both the developers and test engineers because, with the help of bug reports, they can easily track the defects, report the bug, change the status of bugs which are fixed successfully, and also avoid their repetition in further process.
******************************************************************************
Test execution report
It is the document prepared by test leads after the entire testing execution process is completed. The test summary report defines the constancy of the product, and it contains information like the modules, the number of written test cases, executed, pass, fail, and their percentage. And each module has a separate spreadsheet of their respective module.
******************************************************************************
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Test Scenario
The test scenario is a detailed document of test cases that cover end to end functionality of a software application in liner statements. The liner statement is considered as a scenario. The test scenario is a high-level classification of testable requirements. These requirements are grouped on the basis of the functionality of a module and obtained from the use cases.
**********************************************************************
How to write Test Scenarios
As a tester, follow the following steps to create Test Scenarios-
Read the requirement document such as BRS (Business Requirement Specification), SRS (System Requirement Specification) and FRS (Functional Requirement Specification) of the software which is under the test.
Determine all technical aspects and objectives for each requirement.
Find all the possible ways by which the user can operate the software.
Ascertain all the possible scenario due to which system can be misused and also detect the users who can be hackers.
After reading the requirement document and completion of the scheduled analysis make a list of various test scenarios to verify each function of the software.
Once you listed all the possible test scenarios, create a traceability matrix to find out whether each and every requirement has a corresponding test scenario or not.
Supervisor of the project reviews all scenarios. Later, they are evaluated by other stakeholders of the project.
**********************************************************************
Features of Test Scenario
The test scenario is a liner statement that guides testers for the testing sequence.
Test scenario reduces the complexity and repetition of the product.
Test scenario means talking and thinking about tests in detail but write them in liner statements.
It is a thread of operations.
Test scenario becomes more important when the tester does not have enough time to write test cases, and team members agree with a detailed liner scenario.
The test scenario is a time saver activity.
It provides easy maintenance because the addition and modification of test scenarios are easy and independent.
Example of Test scenarios
Here we are taking the Gmail application and writing test scenarios for different modules which are most commonly used such as Login, Compose, Inbox, and Trash
Test scenarios on the Login module
Enter the valid login details (Username, password), and check that the home page is displayed.
Enter the invalid Username and password and check for the home page.
Leave Username and password blank, and check for the error message displayed.
Enter the valid Login, and click on the cancel, and check for the fields reset.
Enter invalid Login, more than three times, and check that account blocked.
Enter valid Login, and check that the Username is displayed on the home screen.
Test scenarios on Compose module
Checks that all users can enter email ides in the To, Cc, and Bcc.
Check that the entire user can enter various email ids in To, Cc, and Bcc.
Compose a mail, send it, and check for the confirmation message.
Compose a mail, send it, and check in the sent item of the sender and the inbox.
Compose a mail, send it, and check for invalid and valid email id (valid format), check the mail in sender inbox.
Compose main, discard, and then check for conformation message and check-in draft.
Compose mail click on save as draft and check for the confirmation message
Compose mail click on close and check for conformation save as drafts.
**********************************************************************
Test scenarios on Inbox module
Click on the inbox, and verify all received mail are displayed and highlighted in the inbox.
Check that a latest received mail has been displayed to the sender email id correctly.
Select the mail, reply and forward send it; check in the sent item of sender and inbox of the receiver.
Check for any attached attachments to the mail that are downloaded or not.
Check that attachment is scanned correctly for any viruses before download.
Select the mail, reply and forward save as draft, and check for the confirmation message and checks in the Draft section.
Check all the emails are marked as read are not highlighted.
Check all mail recipients in Cc are visible to all users.
Checks all email recipients in Bcc are not visible to the users.
Select mail, delete it, and then check in the Trash section.
Test scenario on Trash module
Open trash, check all deleted mail present.
Restore mail from Trash; check-in the corresponding module.
Select mail from trash, delete it, and check mail is permanently deleted.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Test Case
The test case is defined as a group of conditions under which a tester determines whether a software application is working as per the customer's requirements or not. Test case designing includes preconditions, case name, input conditions, and expected result. A test case is a first level action and derived from test scenarios.
**********************************************************************
When do we write a test case?
We will write the test case when we get the following:
When the customer gives the business needs then, the developer starts developing and says that they need 3.5 months to build this product.
And In the meantime, the testing team will start writing the test cases.
Once it is done, it will send it to the Test Lead for the review process.
And when the developers finish developing the product, it is handed over to the testing team.
The test engineers never look at the requirement while testing the product document because testing is constant and does not depends on the mood of the person rather than the quality of the test engineer.
**********************************************************************
Why we write the test cases?
We will write the test for the following reasons:
To require consistency in the test case execution
To make sure a better test coverage
It depends on the process rather than on a person
To avoid training for every new test engineer on the product
**********************************************************************
Test case template
The primary purpose of writing a test case is to achieve the efficiency of the application.
As we know, the actual result is written after the test case execution, and most of the time, it would be same as the expected result. But if the test step will fail, it will be different. So, the actual result field can be skipped, and in the Comments section, we can write about the bugs.
And also, the Input field can be removed, and this information can be added to the Description field.
Step number
It is also essential because if step number 20 is failing, we can document the bug report and hence prioritize working and also decide if it’s a critical bug.
**********************************************************************
Test case type
It can be functional, integration or system test cases or positive or negative or positive and negative test cases.
Release
One release can contain many versions of the release.
Pre-condition
These are the necessary conditions that need to be satisfied by every test engineer before starting the test execution process. Or it is the data configuration or the data setup that needs to be created for the testing.
For example: In an application, we are writing test cases to add users, edit users, and delete users. The per-condition will be seen if user A is added before editing it and removing it.
Test data
These are the values or the input we need to create as per the per-condition.
For example, we will take the Gmail application and let us see the severity based on the modules:
Modules Severity
Login Critical
Help Minor
Compose mail Critical
Setting Minor
Inbox Critical
Sent items Major
Logout Critical
And for the banking application, the severity could be as follows:
Modules Severity
Amount transfer critical
Feedback minor
**********************************************************************
Types of test cases
We have a different kind of test cases, which are as follows:
Function test cases
Integration test cases
System test cases
The functional test cases
Firstly, we check for which field we will write test cases and then describe accordingly.
In functional testing or if the application is data-driven, we require the input column else; it is a bit time-consuming.
Rules to write functional test cases:
In the expected results column, try to use should be or must be.
Highlight the Object names.
We have to describe only those steps which we required the most; otherwise, we do not need to define all the steps.
To reduce the excess execution time, we will write steps correctly.
Write a generic test case; do not try to hard code it.
Let say it is the amount transfer module, so we are writing the functional test cases for it and then also specifies that it is not a login feature.
**********************************************************************
Integration test case
In this, we should not write something which we already covered in the functional test cases, and something we have written in the integration test case should not be written in the system test case again.
Rules to write integration test cases
Firstly, understand the product
Identify the possible scenarios
Write the test case based on the priority
When the test engineer writing the test cases, they may need to consider the following aspects:
If the test cases are in details:
They will try to achieve maximum test coverage.
All test case values or scenarios are correctly described.
They will try to think about the execution point of view.
The template which is used to write the test case must be unique.
**********************************************************************
System test cases
We will write the system test cases for the end-to-end business flows. And we have the entire modules ready to write the system test cases.
The process to write test cases
The method of writing a test case can be completed into the following steps, which are as below:
System study
In this, we will understand the application by looking at the requirements or the SRS, which is given by the customer.
Identify all scenarios:
When the product is launched, what are the possible ways the end-user may use the software to identify all the possible ways.
I have documented all possible scenarios in a document, which is called test design/high-level design.
The test design is a record having all the possible scenarios.
Write test cases
Convert all the identified scenarios to test claims and group the scenarios related to their features, prioritize the module, and write test cases by applying test case design techniques and use the standard test case template, which means that the one which is decided for the project.
Review the test cases
Review the test case by giving it to the head of the team and, after that, fix the review feedback given by the reviewer.
Test case approval
After fixing the test case based on the feedback, send it again for the approval.
Store in the test case repository
After the approval of the particular test case, store in the familiar place that is known as the test case repository.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Error Guessing Technique
The test case design technique ensures that all the possible values that are both positive and negative are required for the testing purposes. In software testing, we have three different test case design techniques which are as follows:
Error Guessing
Equivalence Partitioning
Boundary Value Analysis[BVA]
Error guessing is a technique in which there is no specific method for identifying the error. It is based on the experience of the test analyst, where the tester uses the experience to guess the problematic areas of the software. It is a type of black box testing technique which does not have any defined structure to find the error.
**********************************************************************
How does the error guessing technique be implemented?
The implementation of this technique depends on the experience of the tester or analyst having prior experience with similar applications. It requires only well-experienced testers with quick error guessing technique. This technique is used to find errors that may not be easily captured by formal black box testing techniques, and that is the reason, it is done after all formal techniques.
The main purpose of this technique is to identify common errors at any level of testing by exercising the following tasks:
Enter blank space into the text fields.
Null pointer exception.
Enter invalid parameters.
Divide by zero.
Use maximum limit of files to be uploaded.
Check buttons without entering values.
**********************************************************************
Purpose of Error guessing
The main purpose of the error guessing technique is to deal with all possible errors which cannot be identified as informal testing.
The main purpose of error guessing technique is to deal with all possible errors which cannot be identified informal testing.
It must contain the all-inclusive sets of test cases without skipping any problematic areas and without involving redundant test cases.
This technique accomplishes the characteristics left incomplete during the formal testing.
Depending on the tester's intuition and experience, all the defects cannot be corrected. There are some factors that can be used by the examiner while using their experience -
Tester's intuition
Historical learning
Review checklist
Risk reports of the software
Application UI
General testing rules
Previous test results
Defects occurred in the past
Variety of data which is used for testing
Knowledge of AUT
**********************************************************************
Examples of Error guessing method
Example1
A function of the application requires a mobile number which must be of 10 characters. Now, below are the techniques that can be applied to guess error in the mobile number field:
What will be the result, if the entered character is other than a number?
What will be the result, if entered characters are less than 10 digits?
What will be the result, if the mobile field is left blank?
After implementing these techniques, if the output is similar to the expected result, the function is considered to be bug-free, but if the output is not similar to the expected result, so it is sent to the development team to fix the defects.
However, error guessing is the key technique among all testing techniques as it depends on the experience of a tester, but it does not give surety of highest quality benchmark. It does not provide full coverage to the software. This technique can yield a better result if combined with other techniques of testing.
**********************************************************************
Example2
Suppose we have one bank account, and we have to deposit some money over there, but the amount will be accepted on a particular range of which is 5000-7000. So here, we will provide the different input's value until it covers the maximum test coverage based on the error guessing technique, and see whether it is accepted or give the error message:
value description
6000 Accept
5555 Accept
4000 Error message
8000 Error message
blank Error message
100$ Error message
---- ----
---- ----
Maximum test coverage
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Equivalence Partitioning Technique
Equivalence partitioning is a technique of software testing in which input data is divided into partitions of valid and invalid values, and it is mandatory that all partitions must exhibit the same behavior. If a condition of one partition is true, then the condition of another equal partition must also be true, and if a condition of one partition is false, then the condition of another equal partition must also be false. The principle of equivalence partitioning is, test cases should be designed to cover each partition at least once. Each value of every equal partition must exhibit the same behavior as other.
**********************************************************************
Examples of Equivalence Partitioning technique
Assume that there is a function of a software application that accepts a particular number of digits, not greater and less than that particular number. For example, an OTP number which contains only six digits, less or more than six digits will not be accepted, and the application will redirect the user to the error page.
**********************************************************************
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Boundary Value Analysis
Boundary value analysis is one of the widely used case design technique for black box testing. It is used to test boundary values because the input values near the boundary have higher chances of error.
Whenever we do the testing by boundary value analysis, the tester focuses on, while entering boundary value whether the software is producing correct output or not.
Boundary values are those that contain the upper and lower limit of a variable. Assume that, age is a variable of any function, and its minimum value is 18 and the maximum value is 30, both 18 and 30 will be considered as boundary values.
The basic assumption of boundary value analysis is, the test cases that are created using boundary values are most likely to cause an error.
There is 18 and 30 are the boundary values that's why tester pays more attention to these values, but this doesn't mean that the middle values like 19, 20, 21, 27, 29 are ignored. Test cases are developed for each and every value of the range.
Let's understand via practical:
Imagine, there is a function that accepts a number between 18 to 30, where 18 is the minimum and 30 is the maximum value of valid partition, the other values of this partition are 19, 20, 21, 22, 23, 24, 25, 26, 27, 28 and 29. The invalid partition consists of the numbers which are less than 18 such as 12, 14, 15, 16 and 17, and more than 30 such as 31, 32, 34, 36 and 40. Tester develops test cases for both valid and invalid partitions to capture the behavior of the system on different input conditions.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Test Plan
A test plan is a detailed document which describes software testing areas and activities. It outlines the test strategy, objectives, test schedule, required resources (human resources, software, and hardware), test estimation and test deliverables.
The test plan is a base of every software's testing. It is the most crucial activity which ensures availability of all the lists of planned activities in an appropriate sequence.
The test plan is a template for conducting software testing activities as a defined process that is fully monitored and controlled by the testing manager. The test plan is prepared by the Test Lead (60%), Test Manager(20%), and by the test engineer(20%)
**********************************************************************
How to write a Test Plan
Making a test plan is the most crucial task of the test management process. According to IEEE 829, follow the following seven steps to prepare a test plan.
First, analyze product structure and architecture.
Now design the test strategy.
Define all the test objectives.
Define the testing area.
Define all the useable resources.
Schedule all activities in an appropriate manner.
Determine all the Test Deliverables.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Traceability Matrix
Traceability matrix is a table type document that is used in the development of software application to trace requirements. It can be used for both forward (from Requirements to Design or Coding) and backward (from Coding to Requirements) tracing. It is also known as Requirement Traceability Matrix (RTM) or Cross Reference Matrix (CRM).
**********************************************************************
Goals of Traceability Matrix
It helps in tracing the documents that are developed during various phases of SDLC.
It ensures that the software completely meets the customer's requirements.
It helps in detecting the root cause of any bug.
**********************************************************************
Types of Traceability Test Matrix
The traceability matrix can be classified into three different types which are as follows:
Forward traceability
Backward or reverse traceability
Bi-directional traceability
**********************************************************************
Forward traceability
The forward traceability test matrix is used to ensure that every business's needs or requirements are executed correctly in the application and also tested rigorously. The main objective of this is to verify whether the product developments are going in the right direction.
**********************************************************************
Backward or reverse traceability
The reverse or backward traceability is used to check that we are not increasing the space of the product by enhancing the design elements, code, test other things which are not mentioned in the business needs. And the main objective of this that the existing project remains in the correct direction. In this, the requirements are mapped into the backward direction to the test cases.
**********************************************************************
Bi-directional traceability
It is a combination of forwarding and backward traceability matrix, which is used to make sure that all the business needs are executed in the test cases. It also evaluates the modification in the requirement which is occurring due to the bugs in the application.
**********************************************************************
Advantage of RTM
Following are the benefits of requirement traceability matrix:
With the help of the RTM document, we can display the complete test execution and bugs status based on requirements.
It is used to show the missing requirements or conflicts in documents.
In this, we can ensure the complete test coverage, which means all the modules are tested.
It will also consider the efforts of the testing teamwork towards reworking or reconsidering on the test cases.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Bug in Software Testing
we will learn about defect/bug in software testing and why it occurs, basic terminology of a defect, and bug tracking tool.
The Bug is the informal name of defects, which means that software or application is not working as per the requirement.
In software testing
, a software bug can also be issue, error, fault, or failure. The bug occurred when developers made any mistake or error while developing the product.
While testing the application or executing the test cases, the test engineer may not get the expected result as per the requirement. And the bug had various names in different companies such as error, issues, problem, fault, and mistake, etc.
**********************************************************************
Basic terminology of defect
Let see the different terminology of defect:
Defect
Bug
Error
Issue
Mistakev
Failurev
**********************************************************************
Why defect/bug occur?
In software testing, the bug can occur for the following reasons:
Wrong coding
Missing coding
Extra coding
Bug tracking tool
We have various types of bug tracking tools available in software testing that helps us to track the bug, which is related to the software or the application.
**********************************************************************
Some of the most commonly used bug tracking tools are as follows:
Jira
Bugzilla
Redmine
Mantis
Backlog
Jira
Jira is one of the most important bug tracking tools. Jira is an open-source tool that is used for bug tracking, project management, and issue tracking in manual testing
.
Jira includes different features like reporting, recording, and workflow. In Jira, we can track all kinds of bugs and issues, which are related to the software and generated by the test engineer.
