Comments on: Foundation Series: Black Box and White Box Software Testing http://tynerblain.com/blog/2006/01/12/foundation-series-black-box-and-white-box-software-testing/ Software product success. Sun, 12 Feb 2012 19:10:25 +0000 hourly 1 http://wordpress.org/?v=3.0.4 By: Usability Testing vs Software Testing | brainmates - product management people http://tynerblain.com/blog/2006/01/12/foundation-series-black-box-and-white-box-software-testing/comment-page-1/#comment-483693 Usability Testing vs Software Testing | brainmates - product management people Tue, 17 Mar 2009 04:08:43 +0000 http://tynerblain.com/blog/2006/01/12/foundation-series-black-box-and-white-box-software-testing/#comment-483693 [...] Box method tests the product againsts requirements without any regard for the program structure. Tyner Blain writes that you don’t need to know anything about how the software works.  This means that [...] [...] Box method tests the product againsts requirements without any regard for the program structure. Tyner Blain writes that you don’t need to know anything about how the software works.  This means that [...]

]]> By: Software testing series: Organizing a test suite with tags part two -Tyner Blain http://tynerblain.com/blog/2006/01/12/foundation-series-black-box-and-white-box-software-testing/comment-page-1/#comment-245 Software testing series: Organizing a test suite with tags part two -Tyner Blain Wed, 08 Feb 2006 13:47:40 +0000 http://tynerblain.com/blog/2006/01/12/foundation-series-black-box-and-white-box-software-testing/#comment-245 [...] Test suite becomes less effective over time. Tests can grow irrelevant over time when the software they test is constantly changing (as in this project). This problem has been addressed to a large extent by using whitebox unit tests in the test suite. We are not targeting this as part of the current software solution. [...] [...] Test suite becomes less effective over time. Tests can grow irrelevant over time when the software they test is constantly changing (as in this project). This problem has been addressed to a large extent by using whitebox unit tests in the test suite. We are not targeting this as part of the current software solution. [...]

]]> By: Scott Sehlhorst http://tynerblain.com/blog/2006/01/12/foundation-series-black-box-and-white-box-software-testing/comment-page-1/#comment-235 Scott Sehlhorst Tue, 07 Feb 2006 20:34:02 +0000 http://tynerblain.com/blog/2006/01/12/foundation-series-black-box-and-white-box-software-testing/#comment-235 Kasama, thanks for the question. Here's my personal perspective, maybe other readers will share theirs... My personal experience has been over the last 10 years, and exclusively in enterprise software. In that realm, after working with about 40 different teams for fortune 100 (US) companies, I have never seen a group that didn't use manual blackbox testing. Different groups have had different approaches - either scripted "acceptance tests" or "current feature validation tests" or a form of UAT (user acceptance test). Roughly half of those teams have also had automated black-box tests - running predefined scripts through a solution to "smoke test" it at a minimum, and as "regression tests" more commonly. Every group I've worked with would argue that they use manual whitebox testing - e.g. the developer tests the functionality of code before checking it in. Fewer than 1/4 of the teams have used automated white-box unit tests. So, my personal experience is that 25% use automated graybox testing (white + black) 25% use automated blackbox testing 50% use manual blackbox testing I have not worked with a team that did no testing at all. Maybe some other readers would like to share their anecdotal data? Kasama, thanks for the question. Here’s my personal perspective, maybe other readers will share theirs…

My personal experience has been over the last 10 years, and exclusively in enterprise software. In that realm, after working with about 40 different teams for fortune 100 (US) companies, I have never seen a group that didn’t use manual blackbox testing.

Different groups have had different approaches – either scripted “acceptance tests” or “current feature validation tests” or a form of UAT (user acceptance test).

Roughly half of those teams have also had automated black-box tests – running predefined scripts through a solution to “smoke test” it at a minimum, and as “regression tests” more commonly.

Every group I’ve worked with would argue that they use manual whitebox testing – e.g. the developer tests the functionality of code before checking it in. Fewer than 1/4 of the teams have used automated white-box unit tests.

So, my personal experience is that
25% use automated graybox testing (white + black)
25% use automated blackbox testing
50% use manual blackbox testing

I have not worked with a team that did no testing at all. Maybe some other readers would like to share their anecdotal data?

]]> By: Kasama http://tynerblain.com/blog/2006/01/12/foundation-series-black-box-and-white-box-software-testing/comment-page-1/#comment-234 Kasama Tue, 07 Feb 2006 19:34:06 +0000 http://tynerblain.com/blog/2006/01/12/foundation-series-black-box-and-white-box-software-testing/#comment-234 Discuss the reason why Blackbox and whitebox testing is not widely used in today's software engineering business. If you disagree, discuss the reason Discuss the reason why Blackbox and whitebox testing is not widely used in today’s software engineering business. If you disagree, discuss the reason

]]> By: Tyner Blain » Software testing series: A case study http://tynerblain.com/blog/2006/01/12/foundation-series-black-box-and-white-box-software-testing/comment-page-1/#comment-129 Tyner Blain » Software testing series: A case study Sun, 22 Jan 2006 21:31:44 +0000 http://tynerblain.com/blog/2006/01/12/foundation-series-black-box-and-white-box-software-testing/#comment-129 [...] Educate the test team. No, we’re not automating you out of a job. A gray box testing strategy is comprehensive. Automating regression testing effectively allows manual testers to focus on system level testing and overall quality assurance. The time saved can be applied to testing that should be, but isn’t being done today. [...] [...] Educate the test team. No, we’re not automating you out of a job. A gray box testing strategy is comprehensive. Automating regression testing effectively allows manual testers to focus on system level testing and overall quality assurance. The time saved can be applied to testing that should be, but isn’t being done today. [...]

]]> By: Tyner Blain » Where bugs come from http://tynerblain.com/blog/2006/01/12/foundation-series-black-box-and-white-box-software-testing/comment-page-1/#comment-126 Tyner Blain » Where bugs come from Sun, 22 Jan 2006 21:20:50 +0000 http://tynerblain.com/blog/2006/01/12/foundation-series-black-box-and-white-box-software-testing/#comment-126 [...] On the right side of the diagram, developers will design and then implement the solution, and also define the whitebox and blackbox tests of their implementation. These tests confirm that the code is working “as designed”. [...] [...] On the right side of the diagram, developers will design and then implement the solution, and also define the whitebox and blackbox tests of their implementation. These tests confirm that the code is working “as designed”. [...]

]]> By: Tyner Blain » Foundation series: Unit testing software http://tynerblain.com/blog/2006/01/12/foundation-series-black-box-and-white-box-software-testing/comment-page-1/#comment-109 Tyner Blain » Foundation series: Unit testing software Fri, 20 Jan 2006 03:15:09 +0000 http://tynerblain.com/blog/2006/01/12/foundation-series-black-box-and-white-box-software-testing/#comment-109 [...] Testing software is more than just manually banging around (also called monkey testing) and trying to break different parts of the software application. Unit testing is testing a subset of the functionality of a piece of software. A unit test is different from a system test in that it provides information only about a particular subset of the software. In our previous Foundation series post on black box and white box testing, we used the inspections that come bundled with an oil change as examples of unit tests. [...] [...] Testing software is more than just manually banging around (also called monkey testing) and trying to break different parts of the software application. Unit testing is testing a subset of the functionality of a piece of software. A unit test is different from a system test in that it provides information only about a particular subset of the software. In our previous Foundation series post on black box and white box testing, we used the inspections that come bundled with an oil change as examples of unit tests. [...]

]]> By: Tyner Blain » Software testing series: Black box vs white box testing http://tynerblain.com/blog/2006/01/12/foundation-series-black-box-and-white-box-software-testing/comment-page-1/#comment-87 Tyner Blain » Software testing series: Black box vs white box testing Fri, 13 Jan 2006 13:04:45 +0000 http://tynerblain.com/blog/2006/01/12/foundation-series-black-box-and-white-box-software-testing/#comment-87 [...] You will hear three answers to this question - black, white, and gray. We recently published a foundation series post on black box and white box testing - which serves as a good background document. [...] [...] You will hear three answers to this question – black, white, and gray. We recently published a foundation series post on black box and white box testing – which serves as a good background document. [...]

]]> By: Scott Sehlhorst http://tynerblain.com/blog/2006/01/12/foundation-series-black-box-and-white-box-software-testing/comment-page-1/#comment-86 Scott Sehlhorst Thu, 12 Jan 2006 23:24:09 +0000 http://tynerblain.com/blog/2006/01/12/foundation-series-black-box-and-white-box-software-testing/#comment-86 Absolutely right! Thanks, Roger. Absolutely right! Thanks, Roger.

]]> By: Roger L. Cauvin http://tynerblain.com/blog/2006/01/12/foundation-series-black-box-and-white-box-software-testing/comment-page-1/#comment-85 Roger L. Cauvin Thu, 12 Jan 2006 23:01:37 +0000 http://tynerblain.com/blog/2006/01/12/foundation-series-black-box-and-white-box-software-testing/#comment-85 Another advantage of black-box testing is that is keeps your "eye on the ball". What ultimately matters is that the product fulfills the requirements, which are by definition independent of how the product is implemented. Paying sufficient attention to quality black-box testing forces your team to think in terms of the user experience, whereas you run the risk of neglecting what really matters if you perform only white-box testing. Another advantage of black-box testing is that is keeps your “eye on the ball”. What ultimately matters is that the product fulfills the requirements, which are by definition independent of how the product is implemented. Paying sufficient attention to quality black-box testing forces your team to think in terms of the user experience, whereas you run the risk of neglecting what really matters if you perform only white-box testing.

]]> By: Tyner Blain » Software testing series: Introduction http://tynerblain.com/blog/2006/01/12/foundation-series-black-box-and-white-box-software-testing/comment-page-1/#comment-84 Tyner Blain » Software testing series: Introduction Thu, 12 Jan 2006 20:47:19 +0000 http://tynerblain.com/blog/2006/01/12/foundation-series-black-box-and-white-box-software-testing/#comment-84 [...] There is also a Foundation series post on black box and white box testing. [...] [...] There is also a Foundation series post on black box and white box testing. [...]

]]>