Database testing is new to many people, and as a result you are likely to face several challenges:
1. Insufficient testing skills.
2. Insufficient unit tests for existing databases.
3. Insufficient database testing tools.
4. Reticent DM groups.
Database Testing and Data Inspection:
A common quality technique is to use data inspection tools to examine existing data within a database. You might use something as simple as a SQL based Query tool such as DB Inspect to select a subset of the data within a database to visually inspect the results.
For Example: You may choose to view the unique values in a column to determine what values are stored in it, or compare the row count of a table with the count of the resulting rows from joining the table with another one. If the two counts are the same then you don't have an RI problem across the join.
As Richard Dallaway points out, the problem with data inspection is that it is often done manually and on an irregular basis. When you make changes later, sometimes months or years later, you need to redo your inspection efforts. This is costly, time consuming, and error prone.
Data inspection is more of a debugging technique than it is a testing technique. It is clearly an important technique, but it's not something that will greatly contribute to your efforts to ensure data quality within you organization.
Best Practices:
Use an in-memory database for regression testing: You can dramatically speed up your database tests by running them, or at least portions of them, against an in-memory database such as HSQLDB. The challenge with this approach is that because database methods are implemented differently across database vendors that any method tests will still need to run against the actual database server.
Start fresh each major test run: To ensure a clean database, a common strategy is that at the beginning of each test run you drop the database, then rebuild it from scratch taking into account all database re factorings and transformations to that point,then reload the test data, and then run your tests. Of course, you wouldn't do this to your production database.
Take a connection approach to regression testing: TDD approach to development is an incredibly effective way to work.
Train people in testing: Many developers and DBAs have not been trained in testing skills, and they almost certainly haven't been trained in database testing skills. Invest in your people, and give them the training and education they need to do their jobs.
Pair with novices with people that have database testing experience: One of the easiest ways to gain database testing skills is to pair program with someone who already has them.
Sunday, July 12, 2009
Saturday, June 6, 2009
Unstructured Loops
Whenever possible, this class of loops should be redesigned to reflect the use of the structural programming constructs.
Concatenated Loops
Concatenated Loops can be tested using the approach defined for simple loops,if each of the loops is independent of the other. However, if two loops are concatenated and the loop counter for loop 1 is used as the initial value for loop 2, then the loops are not independent.
Nested Loops
If we extend the test approach from simple loops to nested loops, the number of possible tests would grow geometrically as the level of nesting increases.
1. Start at the innermost loop. Set all other loops to minimum values.
2. Conduct simple loop tests for the innermost loop while holding the outer loops at their minimum iteration parameter values. Add other tests for out-of-range or exclude values.
3. Work outward, conducting tests for the next loop, but keep all other outer loops at minimum values and other nested loops to "typical" values.
4. Continue until all loops have been tested.
1. Start at the innermost loop. Set all other loops to minimum values.
2. Conduct simple loop tests for the innermost loop while holding the outer loops at their minimum iteration parameter values. Add other tests for out-of-range or exclude values.
3. Work outward, conducting tests for the next loop, but keep all other outer loops at minimum values and other nested loops to "typical" values.
4. Continue until all loops have been tested.
Control Structural Testing
There are some of the variations of Control Structural Testing is described below.
Conditional Testing: Conditional testing is a test case design method that exercises the logical conditions contained in a program module.
Data Flow Testing: The data flow testing method selects test paths of a program according to the locations of definitions and uses of variables in the program.
Loop Testing:It is a White Box Testing technique that focuses exclusively on the validity of loop constructs. Four classes of loops can be defined: Simple loops, Concatenated loops,nested loops, and unstructured loops.
Simple Loops: The following sets of tests can be applied to simple loops, where 'n' is the maximum number of allowable passes through the loop.
1. Skip the loop entirely.
2. Only one pass through the loop.
3. Two passes through the loop.
4. 'm' passes the loop where m < n
5. n-1, n, n+1 passes through the loop.
Conditional Testing: Conditional testing is a test case design method that exercises the logical conditions contained in a program module.
Data Flow Testing: The data flow testing method selects test paths of a program according to the locations of definitions and uses of variables in the program.
Loop Testing:It is a White Box Testing technique that focuses exclusively on the validity of loop constructs. Four classes of loops can be defined: Simple loops, Concatenated loops,nested loops, and unstructured loops.
Simple Loops: The following sets of tests can be applied to simple loops, where 'n' is the maximum number of allowable passes through the loop.
1. Skip the loop entirely.
2. Only one pass through the loop.
3. Two passes through the loop.
4. 'm' passes the loop where m < n
5. n-1, n, n+1 passes through the loop.
Graph Matrices
The procedure for deriving the flow graph and even determining a set of basis paths is amenable to mechanization. To develop a software tool that assists in basis path testing, a data structure, called a graph matrix can be quite useful.
A Graph Matrix is a square matrix whose size is equal to the number of nodes on the flow graph. Each row and column corresponds to an identified node, and matrix entries correspond to connections between nodes.
A Graph Matrix is a square matrix whose size is equal to the number of nodes on the flow graph. Each row and column corresponds to an identified node, and matrix entries correspond to connections between nodes.
Computing Cyclomatic Complexity
Cyclomatic Complexity has a foundation in graph theory and provides us with extremely useful software metric. Complexity is computed in one of the three ways:
1. The number of regions of the flow graph corresponds to the Cyclomatic Complexity.
2. Cyclomatic Complexity, V(G), for a flow graph, G is defined as
V(G)= E-N+2
Where E, is the number of flow graph edges, N is the number of flow graph nodes.
3. Cyclomatic Complexity, V(G) for a flow graph, G is also defined as: V(G)= P+1
Where P is the number of predicates nodes contained in the flow graph G.
1. The number of regions of the flow graph corresponds to the Cyclomatic Complexity.
2. Cyclomatic Complexity, V(G), for a flow graph, G is defined as
V(G)= E-N+2
Where E, is the number of flow graph edges, N is the number of flow graph nodes.
3. Cyclomatic Complexity, V(G) for a flow graph, G is also defined as: V(G)= P+1
Where P is the number of predicates nodes contained in the flow graph G.
Cyclomatic Complexity
Cyclomatic Complexity is a software metric that provides a quantitative measure of the logical complexity of a program. When used in the context of a basis path testing method,the value computed for Cyclomatic Complexity defines the number for independent paths in the basis set of program and provides us an upper bound for the number for independent paths must be conducted to ensure that all statements have been executed at least once.
An independent path is any path through the program that introduces at least one new set of processing statements or a new condition.
An independent path is any path through the program that introduces at least one new set of processing statements or a new condition.
Basic Path Testing
Basic Path Testing is a White Box Testing technique first proposed by Tom McCabe. The Basic Path method enables to derive a logical complexity measure of a procedural design and use this measure as a guide for defining a basis set of execution paths.
Test cases derived to exercise the basis set are guaranteed to execute every statement in the program at least one time during testing.
The flow graph depicts logical control flow using a diagrammatic notation. Each structured construct has a corresponding flow graph symbol.
Test cases derived to exercise the basis set are guaranteed to execute every statement in the program at least one time during testing.
The flow graph depicts logical control flow using a diagrammatic notation. Each structured construct has a corresponding flow graph symbol.
Limitations of White Box Testing?
Unfortunately in White Box Testing,exhaustive testing of a code presents certain logistical problems. Even for small programs,the number of possible logical paths can be very large. For instance,a 100 line C Language program that contains two nested loops executing 1 to 20 times depending upon some initial input after some basic data declaration.
Inside the interior loop four if-then-else constructs are required. Then there are approximately 10 to the power of 14 logical paths that are to be exercised to test the program exhaustively. Which means that a magic test processor developing a single test case,execute it and evaluate results in one millisecond would require 3170 years working continuously for this exhaustive testing which is certainly impractical.
Exhaustive White Box Testing is impossible for large software systems. But that doesn't mean White Box Testing should be considered as impractical.Limited White Box Testing in which a limited number of important logical paths are selected and exercised and important data structures are probed for validity,is both practical and White Box Testing.
It is suggested that White and Black Box testing techniques can be coupled to provide an approach that validate the software interface selectively ensuring the correction of internal working of the software.
Inside the interior loop four if-then-else constructs are required. Then there are approximately 10 to the power of 14 logical paths that are to be exercised to test the program exhaustively. Which means that a magic test processor developing a single test case,execute it and evaluate results in one millisecond would require 3170 years working continuously for this exhaustive testing which is certainly impractical.
Exhaustive White Box Testing is impossible for large software systems. But that doesn't mean White Box Testing should be considered as impractical.Limited White Box Testing in which a limited number of important logical paths are selected and exercised and important data structures are probed for validity,is both practical and White Box Testing.
It is suggested that White and Black Box testing techniques can be coupled to provide an approach that validate the software interface selectively ensuring the correction of internal working of the software.
Why White Box Testing?
We do White Box Testing because Black Box Testing is unlikely to uncover numerous sorts of defects in the program. These defects can be of the following nature.
1. Logic errors and incorrect assumptions are inversely proportional to the probability that a program path will be executed. Error tend to creep into our work when we design and implement functions,conditions or controls that are out of the program.
2. The logical flow of the program is sometimes counterintuitive,meaning that our unconscious assumptions about flow of control and data may lead to design errors that are uncovered only when path testing starts.
3. Typographical errors are random,some of which will be uncovered by syntax checking mechanisms but others will go undetected until testing begins.
1. Logic errors and incorrect assumptions are inversely proportional to the probability that a program path will be executed. Error tend to creep into our work when we design and implement functions,conditions or controls that are out of the program.
2. The logical flow of the program is sometimes counterintuitive,meaning that our unconscious assumptions about flow of control and data may lead to design errors that are uncovered only when path testing starts.
3. Typographical errors are random,some of which will be uncovered by syntax checking mechanisms but others will go undetected until testing begins.
What do we do in White Box Testing?
We use the control structure of the procedural design to derive test cases. Using White box testing methods a tester can derive the test cases that
1. Guarantee that all independent paths within a module have been exercised at least once.
2. Exercise all logical decision on their true and false values.
3. Execute all loops at their boundaries and within their operational bounds.
4. Exercise internal data structure to ensure their validity.
White Box Testing is also called Structural or Glass Box Testing
1. Guarantee that all independent paths within a module have been exercised at least once.
2. Exercise all logical decision on their true and false values.
3. Execute all loops at their boundaries and within their operational bounds.
4. Exercise internal data structure to ensure their validity.
White Box Testing is also called Structural or Glass Box Testing
Saturday, May 23, 2009
Stubs and Drivers
A software application is made up of a number of 'Units', where output of one 'Unit' goes as an 'Input' of another Unit.
Example: A 'Sales Order Printing' Program takes a 'Sales Order' as an input, which is actually an output of 'Sales Order Creation' Program.
Due to such interfaces,independent testing of a Unit becomes impossible. But that is what we want to do; we want to test a Unit in isolation! So here we use 'Stub' and 'Driver'.
A 'Driver' is a piece of software that drives the Unit being tested. A driver creates necessary 'Inputs' required for the Unit and then invokes the Unit.
A Unit may reference another Unit in its logic. A 'Stub' takes place of such subordinate unit during the Unit Testing. A 'Stub' is a piece of software that works similar to a unit which is referenced by the Unit being tested, but it is much similar that the actual unit.
A Stub works as a 'Stand-in' for the subordinate unit and provides the minimum required behavior for that unit.
Programmer needs to create such 'Drivers' and 'Stubs' for carrying out Unit Testing. Both the Driver and the Stub are kept at a minimum level of complexity, so that they do not include any errors while testing the Unit in question.
Example: For Unit Testing of 'Sales Order Printing' program, a 'Driver' program will have the code which will create Sales Order records using hardcoded data and then call 'Sales Order Printing' program. Suppose this printing program uses another unit which calculates Sales discounts by some complex calculations. Then call to this unit will be replaced by a 'Stub', which will simply return fix discount data.
Example: A 'Sales Order Printing' Program takes a 'Sales Order' as an input, which is actually an output of 'Sales Order Creation' Program.
Due to such interfaces,independent testing of a Unit becomes impossible. But that is what we want to do; we want to test a Unit in isolation! So here we use 'Stub' and 'Driver'.
A 'Driver' is a piece of software that drives the Unit being tested. A driver creates necessary 'Inputs' required for the Unit and then invokes the Unit.
A Unit may reference another Unit in its logic. A 'Stub' takes place of such subordinate unit during the Unit Testing. A 'Stub' is a piece of software that works similar to a unit which is referenced by the Unit being tested, but it is much similar that the actual unit.
A Stub works as a 'Stand-in' for the subordinate unit and provides the minimum required behavior for that unit.
Programmer needs to create such 'Drivers' and 'Stubs' for carrying out Unit Testing. Both the Driver and the Stub are kept at a minimum level of complexity, so that they do not include any errors while testing the Unit in question.
Example: For Unit Testing of 'Sales Order Printing' program, a 'Driver' program will have the code which will create Sales Order records using hardcoded data and then call 'Sales Order Printing' program. Suppose this printing program uses another unit which calculates Sales discounts by some complex calculations. Then call to this unit will be replaced by a 'Stub', which will simply return fix discount data.
Thursday, May 21, 2009
High Severity - Low Priority and Low Severity - High Priority
Examples for the defects with "High Severity - Low Priority" and "Low Severity - High Priority"?
Example1:
Severity Assumes: Testing Point of View(Group of Testers). Priority Assumes : Technical Point of View.( Or Business Point of View) . if you search in dictionary for both words . Priority says it is very important to deal with. Severity Says Causing some Problem or Pain,difficult in working. So both things have different meanings.
Low Priority & High severity:
1. Spelling Mistake : Low Severity for a Tester But if the Same Spell Mistake is in Company Name or Logo then high priority.
High Severity & Low Priority:
1. User Name is accepting Null characters it is high Severity for a Tester but if management says product has to be released after a miles then the priority is Low.
Hence: Severity = Defect = Tester = Failure of Application
Priority = Business= Mangaer= magnitude of Business
Example2:
Low Severity - High Priority:
Scenario:
If company logo or comapny name is not correct or spelling mistake in the application.Hence here we can say there no issue realted to the functionality of the application but as per business process priority of bug is high.
High Severity - Low Priority:
Scenario:
1st:
If in the application any optional or less frquently used option is having the issues or crash. for example we had the option called export to excel,pdf and note pad option.but if we export the reports in to the note pad system is hanging however our client was using most of the time export to excel and export to pdf.
2nd:
In our banking application we had option called 'calculater' but if we try to calculate some thing it was crashing.
Example3:
It depends on how your severity/priority system is set up. I feel it
is the job of QA to find bugs, and present their implications. It is
the job of Project and/or Development management to determine which
bugs to fix and when is acceptable or necessary to release the
project.
Therefore, I use this methodology -
Severity is "owned" by QA and is objective - for example: A crash is
High Severity, a typo is Low.
Priority is "owned" by Project Managment/Development and is set based
on Business Needs / Develeopment Cycle.
For example, a high severity bug found just prior to release may be
given a low priority since fixing it might destabilize the release.
Or, as others have stated, a typo (Low severity) on the Home page
might be given a high priority. Priority tends to be a judgement call
based on overall project needs - that's why it should not be set by
QA.
Another way to think of it is to consider the "risk" to the project -
if few users will be affected, then the priority would be low, if many
users would be adversely affected (either technically or
perceptually), then the priority would be high. If the project is
"market-driven" and must be available to users by a certain date then,
as that date approaches more and more bugs are likely to be
reclassified from higher to lower status.
There are no absolutes - judgement is everything.
Example1:
Severity Assumes: Testing Point of View(Group of Testers). Priority Assumes : Technical Point of View.( Or Business Point of View) . if you search in dictionary for both words . Priority says it is very important to deal with. Severity Says Causing some Problem or Pain,difficult in working. So both things have different meanings.
Low Priority & High severity:
1. Spelling Mistake : Low Severity for a Tester But if the Same Spell Mistake is in Company Name or Logo then high priority.
High Severity & Low Priority:
1. User Name is accepting Null characters it is high Severity for a Tester but if management says product has to be released after a miles then the priority is Low.
Hence: Severity = Defect = Tester = Failure of Application
Priority = Business= Mangaer= magnitude of Business
Example2:
Low Severity - High Priority:
Scenario:
If company logo or comapny name is not correct or spelling mistake in the application.Hence here we can say there no issue realted to the functionality of the application but as per business process priority of bug is high.
High Severity - Low Priority:
Scenario:
1st:
If in the application any optional or less frquently used option is having the issues or crash. for example we had the option called export to excel,pdf and note pad option.but if we export the reports in to the note pad system is hanging however our client was using most of the time export to excel and export to pdf.
2nd:
In our banking application we had option called 'calculater' but if we try to calculate some thing it was crashing.
Example3:
It depends on how your severity/priority system is set up. I feel it
is the job of QA to find bugs, and present their implications. It is
the job of Project and/or Development management to determine which
bugs to fix and when is acceptable or necessary to release the
project.
Therefore, I use this methodology -
Severity is "owned" by QA and is objective - for example: A crash is
High Severity, a typo is Low.
Priority is "owned" by Project Managment/Development and is set based
on Business Needs / Develeopment Cycle.
For example, a high severity bug found just prior to release may be
given a low priority since fixing it might destabilize the release.
Or, as others have stated, a typo (Low severity) on the Home page
might be given a high priority. Priority tends to be a judgement call
based on overall project needs - that's why it should not be set by
QA.
Another way to think of it is to consider the "risk" to the project -
if few users will be affected, then the priority would be low, if many
users would be adversely affected (either technically or
perceptually), then the priority would be high. If the project is
"market-driven" and must be available to users by a certain date then,
as that date approaches more and more bugs are likely to be
reclassified from higher to lower status.
There are no absolutes - judgement is everything.
Wednesday, May 20, 2009
Subscribe to:
Posts (Atom)
