The SQA2 Blog: BBT - Behavioral Based Testing
It’s hard not to compare BBT(behavior based testing), without raising some questions about BDD(Behavior driven development). In many aspects they do the same thing, but are very different in their own rights. BBT’s design and development relies heavily on BDD concepts, just like BDD relies on TDD (test driven development). Like all new tools, holes and mistakes create the need and desire for a more improved version. This is where BBT comes into play, using a lot of what works with some improvements to make it all its own. Throughout this blog I’ll be going over the differences with process, setup, and automation of both technologies.
Let’s start off with the process, for these two tools, they’re very different. For instance, BDD calls for the product owner to work with the developer as well as the QA resource to create feature files. Feature files are files that contain tests in a plain English format. This allows business to fully understand what’s being tested. From my experience I’ve noticed a huge problem with this approach. First, most product owners don’t like writing feature files. And the ones that do, their tests require some modification before any automation can use it. This is where I feel BBT is far superior. Product owners can continue to write acceptance criteria, while the QA engineer handles the rest. In the BBT process, you graph all the acceptance criteria. Which allows business to easily read a graph and identify additional pitfalls.
Keep in mind that these tools work very differently. In the BDD approach, the majority of the tests cases are just covering a specific acceptance criteria and that leaves a lot of room for error. With BBT, a truth table is devised from the graphed out criteria assisting the engineer to account for all test cases. Even with all these differences, a lot remains the same. For instance, BBT uses contexts and outcomes, in contrast to the given, when, and then steps. Not only can they be compared but they’re pretty much the same thing, each step is reserved for a specific set of commands in automation that can be reused over and over when using the same phrasing.
Now on to setup, once again these two tools are verify different. In this case let’s start off with what’s similar about the two. For instance, both tools have a manual approach that require no coding at all. BDD has feature files, which is only one part of the process. The other being step definition files which contain the actual code with a regex linking the two together.
BBT the open source manual approach starts with graphing and truth tables before any coding is done. SQAsquared created a BBT graphing tool that reads the graph and creates all possible positive and negative test cases for you. The major difference here is how the tests are setup. BDD requires the feature file, that looks for a test case title along with individual steps to go along with it. Whereas BBT looks only for the test case title which has key information about what’s needed to get to the appropriate outcome.
There are only a hand full of differences when it comes to the automation portion of these two tools. Like I mentioned earlier many new tools improved versions of existing tools that almost had it all right. For instance, BDD had the logic for automation down solid. It used regex to run human readable files which made it easy for anyone to add tests as well as provided excellent documentation. BBT readily uses the same logic. The code is directly related to all the contexts and outcomes. Adding more contexts and outcomes results in additional tests. The only real difference is how you call code and the associated terminology. As stated earlier, BDD – given, when, and then, BTT – context and outcome.
When it comes to automation, tests over time become stale and flat out useless. When working with the BBT graphing tool, it will assist with cleaning up all graphed tests. This is achieved by just adjusting the graph to fit your new acceptance needs and simply generating the test cases. Now you have the most updated list of tests without having to go through an modify each and every one individually.
Lets walk through an example, here is a basic BDD test written in the gherkin format(BDD human readable language).
Now from this example we already know this isn’t enough to cover testing. We would then have to add additional tests for all possibilities. This is where BBT thrives, here is an example of the same exact criteria in BBT.
In this graph I have each step laid out, all my given statements on the left which we call contexts. On the right I have the then statement, as well as the action or the when statement needed to get there. So as you probably noticed already, we are doing the same thing in both cases just in a different style. Lets now take a look at the results when we generate test cases from our graph.
The first thing you should see is that it wrote the test with the same exact steps. But BBT doesn’t stop there, like I mentioned before, the tool also generates negative tests as shown below.
Now let’s visualize that your company is using BDD or another automation framework and your test suite has over 2000+ tests. First off, that is awesome! Now think what would happen if you introduced another step to the product requiring an extra step. A step that you needed to add to about 20% of those tests. Sounds like a nightmare to me, but with the BBT tool you have a major win. All you would have to do is update the graph with that extra step and click generate tests. Just like that you have updated all your tests. Its as simple as that.
To sum things up, BDD is a great tool, but the majority companies, at least the ones I’ve worked at, don’t follow its intended use and it only cover what you can think of. BBT is an improved tool based on BDD, specifically in test case management, creation, and coverage; the main headaches of the QA industry. Since BDD heavily influences BBT the transition is pretty easy, not many growing pains. The biggest transition for me was the shift in process, the graphing approach verse the identifying and realizing tests approach. Like BDD, you can use BBT in many different testing environments, whether its front end, back end, service based testing, etc. You can find more information and other blogs from how BBT works to BBT on the testing frontlines with Mobile, Business Intelligence, Security, and many others here.
