Test data may be in Database, so that we should connect our test to database and retrieve the data and use the same in test.
We should know 3 things while we dealing with database connections viz..
1. How to Connect
2. How to Establish connection
3. How to Retrieve and use the data
*** For connection , we need to provide two things
1.Driver / Provider : A third party software used for establishing the connection between front end and back end of the application.
2.Location : Location of database
Record Set : It is a temporarily location where we can store the retrieve data from data base at time. From that temporarily location we can use the data one by one or as per requirement in our testing.
Connection : Connects the application and database.
* We need to create Object Instances for both Record Set and Connection.
For Oracle
con.open “provider=oraoledb.1;server=locahost;uid=userID;pwd=password;database=database name”
For SQL
con.open “provider=sqloledb.1;server=locahost;uid=userID;pwd=password;database=database name”
Friday, October 31, 2008
Saturday, October 25, 2008
Automated Stress Testing Implementation
Automated stress testing implementations will be different depending on the interface to the product application. The types of interfaces available to the product drive the design of the automated stress test tool. The interfaces fall into two main categories:
1)Programmable Interfaces: Interfaces like command prompts, RS-232, Ethernet, General Purpose Interface Bus (GPIB), Universal Serial Bus (USB), etc. that accept strings representing command functions without regard to context or the current state of the device.
2)Graphical User Interfaces (GUI's): Interfaces that use the Windows model to allow the user direct control over the device, individual windows and controls may or may not be visible and/or active depending on the state of the device.
1)Programmable Interfaces:
These interfaces have allowed users to setup, control, and retrieve data in a variety of application areas like manufacturing, research and development, and service. To meet the needs of these customers, the products provide programmable interfaces, which generally support a large number of commands (1000+), and are required to operate for long periods of time, for example, on a manufacturing line where the product is used 24 hours a day, 7 days a week. Testing all possible combinations of commands on these products is practically impossible using manual testing methods.
Programmable interface stress testing is performed by randomly selecting from a list of individual commands and then sending these commands to the device under test (DUT) through the interface. If a command has parameters, then the parameters are also enumerated by randomly generating a unique command parameter. By using a pseudo-random number generator, each unique seed value will create the same sequence of commands with the same parameters each time the stress test is executed. Each command is also written to a log file which can be then used later to reproduce any defects that were uncovered.
For additional complexity, other variations of the automated stress test can be performed. For example, the stress test can vary the rate at which commands are sent to the interface, the stress test can send the commands across multiple interfaces simultaneously, (if the product supports it), or the stress test can send multiple commands at the same time.
2)Graphical User Interfaces:
In recent years, Graphical User Interfaces have become dominant and it became clear that we needed a means to test these user interfaces analogous to that which is used for programmable interfaces. However, since accessing the GUI is not as simple as sending streams of command line input to the product application, a new approach was needed. It is necessary to store not only the object recognition method for the control, but also information about its parent window and other information like its expected state, certain property values, etc. An example would be a ‘HELP’ menu item. There may be multiple windows open with a 'HELP' menu item, so it is not sufficient to simply store "click the 'HELP' menu item", but you have to store "click the 'HELP' menu item for the particular window". With this information it is possible to uniquely define all the possible product application operations (i.e. each control can be uniquely identified).
Additionally, the flow of each operation can be important. Many controls are not visible until several levels of modal windows have been opened and/or closed, for example, a typical confirm file overwrite dialog box for a 'File->Save As…' filename operation is not available until the following sequence has been executed:
1.Set Context to the Main Window
2.Select 'File->Save As…'
3.Select Target Directory from tree control
4.Type a valid filename into the edit-box
5.Click the 'SAVE' button
6.If the filename already exists, either confirm the file overwrite by clicking the 'OK' button in the confirmation dialog or click the cancel button.
In this case, you need to group these six operations together as one "big" operation in order to correctly exercise this particular 'OK' button.
1)Programmable Interfaces: Interfaces like command prompts, RS-232, Ethernet, General Purpose Interface Bus (GPIB), Universal Serial Bus (USB), etc. that accept strings representing command functions without regard to context or the current state of the device.
2)Graphical User Interfaces (GUI's): Interfaces that use the Windows model to allow the user direct control over the device, individual windows and controls may or may not be visible and/or active depending on the state of the device.
1)Programmable Interfaces:
These interfaces have allowed users to setup, control, and retrieve data in a variety of application areas like manufacturing, research and development, and service. To meet the needs of these customers, the products provide programmable interfaces, which generally support a large number of commands (1000+), and are required to operate for long periods of time, for example, on a manufacturing line where the product is used 24 hours a day, 7 days a week. Testing all possible combinations of commands on these products is practically impossible using manual testing methods.
Programmable interface stress testing is performed by randomly selecting from a list of individual commands and then sending these commands to the device under test (DUT) through the interface. If a command has parameters, then the parameters are also enumerated by randomly generating a unique command parameter. By using a pseudo-random number generator, each unique seed value will create the same sequence of commands with the same parameters each time the stress test is executed. Each command is also written to a log file which can be then used later to reproduce any defects that were uncovered.
For additional complexity, other variations of the automated stress test can be performed. For example, the stress test can vary the rate at which commands are sent to the interface, the stress test can send the commands across multiple interfaces simultaneously, (if the product supports it), or the stress test can send multiple commands at the same time.
2)Graphical User Interfaces:
In recent years, Graphical User Interfaces have become dominant and it became clear that we needed a means to test these user interfaces analogous to that which is used for programmable interfaces. However, since accessing the GUI is not as simple as sending streams of command line input to the product application, a new approach was needed. It is necessary to store not only the object recognition method for the control, but also information about its parent window and other information like its expected state, certain property values, etc. An example would be a ‘HELP’ menu item. There may be multiple windows open with a 'HELP' menu item, so it is not sufficient to simply store "click the 'HELP' menu item", but you have to store "click the 'HELP' menu item for the particular window". With this information it is possible to uniquely define all the possible product application operations (i.e. each control can be uniquely identified).
Additionally, the flow of each operation can be important. Many controls are not visible until several levels of modal windows have been opened and/or closed, for example, a typical confirm file overwrite dialog box for a 'File->Save As…' filename operation is not available until the following sequence has been executed:
1.Set Context to the Main Window
2.Select 'File->Save As…'
3.Select Target Directory from tree control
4.Type a valid filename into the edit-box
5.Click the 'SAVE' button
6.If the filename already exists, either confirm the file overwrite by clicking the 'OK' button in the confirmation dialog or click the cancel button.
In this case, you need to group these six operations together as one "big" operation in order to correctly exercise this particular 'OK' button.
Friday, October 24, 2008
Web Testing
Web based functionality on most applications is now a part of everyday life. As such the test tool should provide good web based test functionality in addition to its client/server functions.
In judging the rating for this category I looked at the tools native support for HTML tables, frames, DOM, various platforms for browsers, Web site maps and links.
Web testing can be riddled with problems if various considerations are not taken into account. Here are a few examples
• Are there functions to tell me when the page has finished loading?
• Can I tell the test tool to wait until an image appears?
• Can I test whether links are valid or not?
• Can I test web based objects functions like is it enabled, does it contain data, etc.
• Are there facilities that will allow me to programmatically look for objects of a certain type on a web page or locate a specific object?
• Can I extract data from the web page itself? E.g. the title? A hidden form element?
With Client server testing the target customer is usually well defined you know what network operating system you will be using, the applications and so on but on the web it is far different. A person may be connecting from the USA or Africa, they may be disabled, they may use various browsers, and the screen resolution on their computer will be different. They will speak different languages, will have fast connections and slow connections, connect using MAC, Linux or Windows, etc, etc. So the cost to set up a test environment is usually greater than for a client server test where the environment is fairly well defined.
In judging the rating for this category I looked at the tools native support for HTML tables, frames, DOM, various platforms for browsers, Web site maps and links.
Web testing can be riddled with problems if various considerations are not taken into account. Here are a few examples
• Are there functions to tell me when the page has finished loading?
• Can I tell the test tool to wait until an image appears?
• Can I test whether links are valid or not?
• Can I test web based objects functions like is it enabled, does it contain data, etc.
• Are there facilities that will allow me to programmatically look for objects of a certain type on a web page or locate a specific object?
• Can I extract data from the web page itself? E.g. the title? A hidden form element?
With Client server testing the target customer is usually well defined you know what network operating system you will be using, the applications and so on but on the web it is far different. A person may be connecting from the USA or Africa, they may be disabled, they may use various browsers, and the screen resolution on their computer will be different. They will speak different languages, will have fast connections and slow connections, connect using MAC, Linux or Windows, etc, etc. So the cost to set up a test environment is usually greater than for a client server test where the environment is fairly well defined.
Classification of Test Data Types
In the process of testing a system, many references are made to "The Data" or "Data Problems".
Although it is perhaps simpler to discuss data in these terms, it is useful to be able to classify the data according to the way it is used. The following broad categories allow data to be handled and discussed more easily.
Environmental data
Environmental data tells the system about its technical environment. It includes communications
addresses, directory trees and paths and environmental variables. The current date and time can be seen as environmental data.
Setup data
Setup data tells the system about the business rules. It might include a cross reference between country and delivery cost or method, or methods of debt collection from different kinds of customers.
Typically, setup data causes different functionality to apply to otherwise similar data. With an effective approach to setup data, business can offer new intangible products without developing new functionality - as can be seen in the mobile phone industry, where new billing products are supported and indeed created by additions to the setup data.
Input data
Input data is the information input by day-to-day system functions. Accounts, products, orders, actions, documents can all be input data. For the purposes of testing, it is useful to split the categorization once more:
FIXED INPUT DATA
Fixed input data is available before the start of the test, and can be seen as part of the test
conditions.
CONSUMABLE INPUT DATA
Consumable input data forms the test input
It can also be helpful to qualify data after the system has started to use it;
Transitional data
Transitional data is data that exists only within the program, during processing of input data.
Transitional data is not seen outside the system (arguably, test handles and instrumentation make it output data), but its state can be inferred from actions that the system has taken. Typically held in internal system variables, it is temporary and is lost at the end of processing.
Output data
Output data is all the data that a system outputs as a result of processing input data and events. It
generally has a correspondence with the input data (cf. Jackson's Structured Programming
methodology), and includes not only files, transmissions, reports and database updates, but can also include test measurements. A subset of the output data is generally compared with the expected results at the end of test execution. As such, it does not directly influence the quality of the tests.
Although it is perhaps simpler to discuss data in these terms, it is useful to be able to classify the data according to the way it is used. The following broad categories allow data to be handled and discussed more easily.
Environmental data
Environmental data tells the system about its technical environment. It includes communications
addresses, directory trees and paths and environmental variables. The current date and time can be seen as environmental data.
Setup data
Setup data tells the system about the business rules. It might include a cross reference between country and delivery cost or method, or methods of debt collection from different kinds of customers.
Typically, setup data causes different functionality to apply to otherwise similar data. With an effective approach to setup data, business can offer new intangible products without developing new functionality - as can be seen in the mobile phone industry, where new billing products are supported and indeed created by additions to the setup data.
Input data
Input data is the information input by day-to-day system functions. Accounts, products, orders, actions, documents can all be input data. For the purposes of testing, it is useful to split the categorization once more:
FIXED INPUT DATA
Fixed input data is available before the start of the test, and can be seen as part of the test
conditions.
CONSUMABLE INPUT DATA
Consumable input data forms the test input
It can also be helpful to qualify data after the system has started to use it;
Transitional data
Transitional data is data that exists only within the program, during processing of input data.
Transitional data is not seen outside the system (arguably, test handles and instrumentation make it output data), but its state can be inferred from actions that the system has taken. Typically held in internal system variables, it is temporary and is lost at the end of processing.
Output data
Output data is all the data that a system outputs as a result of processing input data and events. It
generally has a correspondence with the input data (cf. Jackson's Structured Programming
methodology), and includes not only files, transmissions, reports and database updates, but can also include test measurements. A subset of the output data is generally compared with the expected results at the end of test execution. As such, it does not directly influence the quality of the tests.
Software Testing Dictionary
The following definitions are taken from accepted and identified sources.
Ad Hoc Testing: Testing carried out using no recognised test case design technique. [BCS]
Assertion Testing: (NBS) A dynamic analysis technique which inserts assertions about the relationship between program variables into the program code. The truth of the assertions is determined as the program executes.
Background testing: is the execution of normal functional testing while the SUT is exercised by a realistic work load. This work load is being processed "in the background" as far as the functional testing is concerned.
Benchmarks: Programs that provide performance comparison for software, hardware, and systems.
Benchmarking: is specific type of performance test with the purpose of determining performance baselines for comparison.
Big-bang testing: Integration testing where no incremental testing takes place prior to all the system's components being combined to form the system
Breadth test: - A test suite that exercises the full scope of a system from a top-down perspective, but does not test any aspect in detail.
Clean test: A test whose primary purpose is validation; that is, tests designed to demonstrate the software`s correct working.(syn. positive test)
Code Inspection: A manual [formal] testing [error detection] technique where the programmer reads source code, statement by statement, to a group who ask questions analyzing the program logic, analyzing the code with respect to a checklist of historically common programming errors, and analyzing its compliance with coding standards. Contrast with code audit, code review, code walkthrough. This technique can also be applied to other software and configuration items. Syn: Fagan Inspection
Code Walkthrough: A manual testing [error detection] technique where program [source code] logic [structure] is traced manually [mentally] by a group with a small set of test cases, while the state of program variables is manually monitored, to analyze the programmer's logic and assumptions. Contrast with code audit, code inspection, code review.
Coexistence Testing: Coexistence isn’t enough. It also depends on load order, how virtual space is mapped at the moment, hardware and software configurations, and the history of what took place hours or days before. It’s probably an exponentially hard problem rather than a square-law problem.
Compatibility bug: A revision to the framework breaks a previously working feature: a new feature is inconsistent with an old feature, or a new feature breaks an unchanged application rebuilt with the new framework code.
Compatibility Testing: The process of determining the ability of two or more systems to exchange information. In a situation where the developed software replaces an already working program, an investigation should be conducted to assess possible comparability problems between the new software and other programs or systems.
Composability testing:Testing the ability of the interface to let users do more complex tasks by combining different sequences of simpler, easy-to-learn tasks.
Condition Coverage: A test coverage criteria requiring enough test cases such that each condition in a decision takes on all possible outcomes at least once, and each point of entry to a program or subroutine is invoked at least once. Contrast with branch coverage, decision coverage, multiple condition coverage, path coverage, statement coverage
Conformance directed testing: Testing that seeks to establish conformance to requirements or specification.
CRUD Testing: Build CRUD matrix and test all object creation, reads, updates, and deletion.
Data-Driven testing:An automation approach in which the navigation and functionality of the test script is directed through external data; this approach separates test and control data from the test script.
Data flow testing Testing in which test cases are designed based on variable usage within the code
Database testing. Check the integrity of database field values.
Defect Age. A measurement that describes the period of time from the introduction of a defect until its discovery. .
Defect Discovery Rate.A metric describing the number of defects discovered over a specified period of time, usually displayed in graphical form.
Defect Removal Efficiency (DRE).A measure of the number of defects discovered in an activity versus the number that could have been found. Often used as a measure of test effectiveness.
Defect Seeding.The process of intentionally adding known defects to those already in a computer program for the purpose of monitoring the rate of detection and removal, and estimating the number of defects still remaining. Also called Error Seeding.
Defect Masked.An existing defect that hasn't yet caused a failure because another defect has prevented that part of the code from being executed.
Depth test. A test case, that exercises some part of a system to a significant level of detail.
Decision Coverage: A test coverage criteria requiring enough test cases such that each decision has a true and false result at least once, and that each statement is executed at least once. Syn: branch coverage. Contrast with condition coverage, multiple condition coverage, path coverage, statement coverage Dirty testing Negative testing.
End-to-End testing: Similar to system testing; the 'macro' end of the test scale; involves testing of a complete application environment in a situation that mimics real-world use, such as interacting with a database, using network communications, or interacting with other hardware, applications, or systems if appropriate.
Equivalence Partitioning: An approach where classes of inputs are categorized for product or function validation. This usually does not include combinations of input, but rather a single state value based by class. For example, with a given function there may be several classes of input that may be used for positive testing. If function expects an integer and receives an integer as input, this would be considered as positive test assertion. On the other hand, if a character or any other input class other than integer is provided, this would be considered a negative test assertion or condition.
Error Guessing: Another common approach to black-box validation. Black-box testing is when everything else other than the source code may be used for testing. This is the most common approach to testing. Error guessing is when random inputs or conditions are used for testing. Random in this case includes a value either produced by a computerized random number generator, or an ad hoc value or test conditions provided by engineer.
Error guessing: A test case design technique where the experience of the tester is used to postulate what faults exist, and to design tests specially to expose them
Error seeding. The purposeful introduction of faults into a program to test effectiveness of a test suite or other quality assurance program.
Exception Testing. Identify error messages and exception handling processes an conditions that trigger them.
Exhaustive Testing.(NBS) Executing the program with all possible combinations of values for program variables. Feasible only for small, simple programs.
Exploratory Testing: An interactive process of concurrent product exploration, test design, and test execution. The heart of exploratory testing can be stated simply: The outcome of this test influences the design of the next test.
Ad Hoc Testing: Testing carried out using no recognised test case design technique. [BCS]
Assertion Testing: (NBS) A dynamic analysis technique which inserts assertions about the relationship between program variables into the program code. The truth of the assertions is determined as the program executes.
Background testing: is the execution of normal functional testing while the SUT is exercised by a realistic work load. This work load is being processed "in the background" as far as the functional testing is concerned.
Benchmarks: Programs that provide performance comparison for software, hardware, and systems.
Benchmarking: is specific type of performance test with the purpose of determining performance baselines for comparison.
Big-bang testing: Integration testing where no incremental testing takes place prior to all the system's components being combined to form the system
Breadth test: - A test suite that exercises the full scope of a system from a top-down perspective, but does not test any aspect in detail.
Clean test: A test whose primary purpose is validation; that is, tests designed to demonstrate the software`s correct working.(syn. positive test)
Code Inspection: A manual [formal] testing [error detection] technique where the programmer reads source code, statement by statement, to a group who ask questions analyzing the program logic, analyzing the code with respect to a checklist of historically common programming errors, and analyzing its compliance with coding standards. Contrast with code audit, code review, code walkthrough. This technique can also be applied to other software and configuration items. Syn: Fagan Inspection
Code Walkthrough: A manual testing [error detection] technique where program [source code] logic [structure] is traced manually [mentally] by a group with a small set of test cases, while the state of program variables is manually monitored, to analyze the programmer's logic and assumptions. Contrast with code audit, code inspection, code review.
Coexistence Testing: Coexistence isn’t enough. It also depends on load order, how virtual space is mapped at the moment, hardware and software configurations, and the history of what took place hours or days before. It’s probably an exponentially hard problem rather than a square-law problem.
Compatibility bug: A revision to the framework breaks a previously working feature: a new feature is inconsistent with an old feature, or a new feature breaks an unchanged application rebuilt with the new framework code.
Compatibility Testing: The process of determining the ability of two or more systems to exchange information. In a situation where the developed software replaces an already working program, an investigation should be conducted to assess possible comparability problems between the new software and other programs or systems.
Composability testing:Testing the ability of the interface to let users do more complex tasks by combining different sequences of simpler, easy-to-learn tasks.
Condition Coverage: A test coverage criteria requiring enough test cases such that each condition in a decision takes on all possible outcomes at least once, and each point of entry to a program or subroutine is invoked at least once. Contrast with branch coverage, decision coverage, multiple condition coverage, path coverage, statement coverage
Conformance directed testing: Testing that seeks to establish conformance to requirements or specification.
CRUD Testing: Build CRUD matrix and test all object creation, reads, updates, and deletion.
Data-Driven testing:An automation approach in which the navigation and functionality of the test script is directed through external data; this approach separates test and control data from the test script.
Data flow testing Testing in which test cases are designed based on variable usage within the code
Database testing. Check the integrity of database field values.
Defect Age. A measurement that describes the period of time from the introduction of a defect until its discovery. .
Defect Discovery Rate.A metric describing the number of defects discovered over a specified period of time, usually displayed in graphical form.
Defect Removal Efficiency (DRE).A measure of the number of defects discovered in an activity versus the number that could have been found. Often used as a measure of test effectiveness.
Defect Seeding.The process of intentionally adding known defects to those already in a computer program for the purpose of monitoring the rate of detection and removal, and estimating the number of defects still remaining. Also called Error Seeding.
Defect Masked.An existing defect that hasn't yet caused a failure because another defect has prevented that part of the code from being executed.
Depth test. A test case, that exercises some part of a system to a significant level of detail.
Decision Coverage: A test coverage criteria requiring enough test cases such that each decision has a true and false result at least once, and that each statement is executed at least once. Syn: branch coverage. Contrast with condition coverage, multiple condition coverage, path coverage, statement coverage Dirty testing Negative testing.
End-to-End testing: Similar to system testing; the 'macro' end of the test scale; involves testing of a complete application environment in a situation that mimics real-world use, such as interacting with a database, using network communications, or interacting with other hardware, applications, or systems if appropriate.
Equivalence Partitioning: An approach where classes of inputs are categorized for product or function validation. This usually does not include combinations of input, but rather a single state value based by class. For example, with a given function there may be several classes of input that may be used for positive testing. If function expects an integer and receives an integer as input, this would be considered as positive test assertion. On the other hand, if a character or any other input class other than integer is provided, this would be considered a negative test assertion or condition.
Error Guessing: Another common approach to black-box validation. Black-box testing is when everything else other than the source code may be used for testing. This is the most common approach to testing. Error guessing is when random inputs or conditions are used for testing. Random in this case includes a value either produced by a computerized random number generator, or an ad hoc value or test conditions provided by engineer.
Error guessing: A test case design technique where the experience of the tester is used to postulate what faults exist, and to design tests specially to expose them
Error seeding. The purposeful introduction of faults into a program to test effectiveness of a test suite or other quality assurance program.
Exception Testing. Identify error messages and exception handling processes an conditions that trigger them.
Exhaustive Testing.(NBS) Executing the program with all possible combinations of values for program variables. Feasible only for small, simple programs.
Exploratory Testing: An interactive process of concurrent product exploration, test design, and test execution. The heart of exploratory testing can be stated simply: The outcome of this test influences the design of the next test.
Tuesday, October 21, 2008
What is a Test Case Point (TCP)
TCP is a measure of estimating the complexity of an application. This is also used as an estimation technique to calculate the size and effort of a testing project.
The TCP counts are nothing but ranking the requirements and the test cases that are to be written for those requirements into simple, average and complex and quantifying the same into a measure of complexity.
In this courseware we shall give an overview about Test Case Points and not elaborate on using TCP as an estimation technique.
Calculating the Test Case Points:
Based on the Functional Requirement Document (FRD), the application is classified into various modules like say for a web application, we can have ‘Login and Authentication’ as a module and rank that particular module as Simple, Average and Complex based on the number and complexity of the requirements for that module. A Simple requirement is one, which can be given a value in the scale of 1 to3. An Average requirement is ranked between 4 and 7. A Complex requirement is ranked between 8 and 10.
The test cases for a particular requirement are classified into Simple, Average and Complex based on the following four factors.
• Test case complexity for that requirement OR
• Interface with other Test cases OR
• No. of verification points OR
• Baseline Test data
A sample guideline for classification of test cases is given below.
• Any verification point containing a calculation is considered 'Complex'
• Any verification point, which interfaces with or interacts with another application is classified as 'Complex'
• Any verification point consisting of report verification is considered as 'Complex'
• A verification point comprising Search functionality may be classified as 'Complex' or 'Average' depending on the complexity
Depending on the respective project, the complexity needs to be identified in a similar manner.
Based on the test case type an adjustment factor is assigned for simple, average and complex test cases. This adjustment factor has been calculated after a thorough study and analysis done on many testing projects.
The Adjustment Factor in the table mentioned below is pre-determined and must not be changed for every project.
The TCP counts are nothing but ranking the requirements and the test cases that are to be written for those requirements into simple, average and complex and quantifying the same into a measure of complexity.
In this courseware we shall give an overview about Test Case Points and not elaborate on using TCP as an estimation technique.
Calculating the Test Case Points:
Based on the Functional Requirement Document (FRD), the application is classified into various modules like say for a web application, we can have ‘Login and Authentication’ as a module and rank that particular module as Simple, Average and Complex based on the number and complexity of the requirements for that module. A Simple requirement is one, which can be given a value in the scale of 1 to3. An Average requirement is ranked between 4 and 7. A Complex requirement is ranked between 8 and 10.
The test cases for a particular requirement are classified into Simple, Average and Complex based on the following four factors.
• Test case complexity for that requirement OR
• Interface with other Test cases OR
• No. of verification points OR
• Baseline Test data
A sample guideline for classification of test cases is given below.
• Any verification point containing a calculation is considered 'Complex'
• Any verification point, which interfaces with or interacts with another application is classified as 'Complex'
• Any verification point consisting of report verification is considered as 'Complex'
• A verification point comprising Search functionality may be classified as 'Complex' or 'Average' depending on the complexity
Depending on the respective project, the complexity needs to be identified in a similar manner.
Based on the test case type an adjustment factor is assigned for simple, average and complex test cases. This adjustment factor has been calculated after a thorough study and analysis done on many testing projects.
The Adjustment Factor in the table mentioned below is pre-determined and must not be changed for every project.
Monday, October 13, 2008
Defect Management
Defect:
A mismatch in the application and its specification is a defect. A software error is present when the program does not do what its end user expects it to do.
Defect Fundamentals:
A Defect is a product anomaly or flaw. Defects include such things as omissions and
imperfections found during testing phases. Symptoms (flaws) of faults contained in software that is sufficiently mature for production will be considered as defects. Deviations from expectation that is to be tracked and resolved is also termed a defect.
An evaluation of defects discovered during testing provides the best indication of software quality. Quality is the indication of how well the system meets the requirements. So in this context defects are identified as any failure to meet the system requirements.
Defect evaluation is based on methods that range from simple number count to rigorous statistical modeling.
Rigorous evaluation uses assumptions about the arrival or discovery rates of defects during the testing process. The actual data about defect rates are then fit to the model. Such an evaluation estimates the current system reliability and predicts how the reliability will grow if testing and defect removal continue. This evaluation is described as system reliability growth modelling
Defect Tracking:
After a defect has been found, it must be reported to development so that it can be fixed.
1.The Initial State of a defect will be 'New'.
2.The Project Lead of the development team will review the defect and set it to one of the following statuses:
Open – Accepts the bug and assigns it to a developer.
Invalid Bug – The reported bug is not valid one as per the requirements/design
As Designed – This is an intended functionality as per the requirements/design
Deferred –This will be an enhancement.
Duplicate – The bug has already been reported.
Document – Once it is set to any of the above statuses apart from Open, and the testing team does not agree with the development team it is set to document status.
Once the development team has started working on the defect the status is set to WIP ((Work in Progress) or if the development team is waiting for a go ahead or some technical feedback, they will set to Dev Waiting
After the development team has fixed the defect, the status is set to FIXED, which means the defect is ready to re-test.
On re-testing the defect, and the defect still exists, the status is set to REOPENED, which will follow the same cycle as an open defect.
If the fixed defect satisfies the requirements/passes the test case, it is set to Closed.
Defect Classification:
The severity of bugs will be classified as follows:
1.Critical:
The problem prevents further processing and testing. The Development Team must be informed immediately and they need to take corrective action immediately.
2.High:
The problem affects selected processing to a significant degree, making it inoperable, Cause data loss, or could cause a user to make an incorrect decision or entry. The Development Team must be informed that day, and they need to take corrective action within 0 – 24 hours.
3.Medium:
The problem affects selected processing, but has a work-around that allows continued processing and testing. No data loss is suffered. These may be cosmetic problems that hamper usability or divulge client-specific information. The Development Team must be informed within 24 hours, and they need to take corrective action within 24 - 48 hours.
4.Low:
The problem is cosmetic, and/or does not affect further processing and testing. The Development Team must be informed within 48 hours, and they need to take corrective action within 48 - 96 hours.
Defect Reporting Guidelines:
The key to making a good report is providing the development staff with as much information as necessary to reproduce the bug. This can be broken down into 5 points:
The key to making a good report is providing the development staff with as much information as necessary to reproduce the bug. This can be broken down into 5 points:
1) Give a brief description of the problem
2) List the steps that are needed to reproduce the bug or problem
3) Supply all relevant information such as version, project and data used.
4) Supply a copy of all relevant reports and data including copies of the expected
results.
5) Summarize what you think the problem is.
When you are reporting a defect the more information you supply, the easier it will be for the developers to determine the problem and fix it.
A mismatch in the application and its specification is a defect. A software error is present when the program does not do what its end user expects it to do.
Defect Fundamentals:
A Defect is a product anomaly or flaw. Defects include such things as omissions and
imperfections found during testing phases. Symptoms (flaws) of faults contained in software that is sufficiently mature for production will be considered as defects. Deviations from expectation that is to be tracked and resolved is also termed a defect.
An evaluation of defects discovered during testing provides the best indication of software quality. Quality is the indication of how well the system meets the requirements. So in this context defects are identified as any failure to meet the system requirements.
Defect evaluation is based on methods that range from simple number count to rigorous statistical modeling.
Rigorous evaluation uses assumptions about the arrival or discovery rates of defects during the testing process. The actual data about defect rates are then fit to the model. Such an evaluation estimates the current system reliability and predicts how the reliability will grow if testing and defect removal continue. This evaluation is described as system reliability growth modelling
Defect Tracking:
After a defect has been found, it must be reported to development so that it can be fixed.
1.The Initial State of a defect will be 'New'.
2.The Project Lead of the development team will review the defect and set it to one of the following statuses:
Open – Accepts the bug and assigns it to a developer.
Invalid Bug – The reported bug is not valid one as per the requirements/design
As Designed – This is an intended functionality as per the requirements/design
Deferred –This will be an enhancement.
Duplicate – The bug has already been reported.
Document – Once it is set to any of the above statuses apart from Open, and the testing team does not agree with the development team it is set to document status.
Once the development team has started working on the defect the status is set to WIP ((Work in Progress) or if the development team is waiting for a go ahead or some technical feedback, they will set to Dev Waiting
After the development team has fixed the defect, the status is set to FIXED, which means the defect is ready to re-test.
On re-testing the defect, and the defect still exists, the status is set to REOPENED, which will follow the same cycle as an open defect.
If the fixed defect satisfies the requirements/passes the test case, it is set to Closed.
Defect Classification:
The severity of bugs will be classified as follows:
1.Critical:
The problem prevents further processing and testing. The Development Team must be informed immediately and they need to take corrective action immediately.
2.High:
The problem affects selected processing to a significant degree, making it inoperable, Cause data loss, or could cause a user to make an incorrect decision or entry. The Development Team must be informed that day, and they need to take corrective action within 0 – 24 hours.
3.Medium:
The problem affects selected processing, but has a work-around that allows continued processing and testing. No data loss is suffered. These may be cosmetic problems that hamper usability or divulge client-specific information. The Development Team must be informed within 24 hours, and they need to take corrective action within 24 - 48 hours.
4.Low:
The problem is cosmetic, and/or does not affect further processing and testing. The Development Team must be informed within 48 hours, and they need to take corrective action within 48 - 96 hours.
Defect Reporting Guidelines:
The key to making a good report is providing the development staff with as much information as necessary to reproduce the bug. This can be broken down into 5 points:
The key to making a good report is providing the development staff with as much information as necessary to reproduce the bug. This can be broken down into 5 points:
1) Give a brief description of the problem
2) List the steps that are needed to reproduce the bug or problem
3) Supply all relevant information such as version, project and data used.
4) Supply a copy of all relevant reports and data including copies of the expected
results.
5) Summarize what you think the problem is.
When you are reporting a defect the more information you supply, the easier it will be for the developers to determine the problem and fix it.
Subscribe to:
Posts (Atom)
