The SQA2 Blog: BBT - Implemented
When it comes to testing web-services, tests are pretty black and white, submitting data and expecting data or a response back. However, there is so much to test, it regularly leaves many guessing if they got everything. And this is where switching to BBT alleviates a lot of stress. Keep in mind that BBT won’t replace any current development methodologies, only your current testing framework. This allows seamless integration with many current methodologies.
BBT is designed to ensure full test case coverage, in the shortest amount of time possible. Think how long it would take you to write test case permutations on a web-service with 25+ fields/values to verify, most likely more than you’d like. BBT will help you make sure that you capture all permutations. Throughout this post I’ll be taking you through how BBT can work for you, not only in the test case identification process but the automation process as well.
Gathering requirements, every web-service has requirements to properly outline each expectation and functionality. Depending on the structure, the success of a web-service can be dependent on many other services, which at times can be a nightmare. When gathering requirements, I felt it best to always get specifics, not ranges or guesses. Failing to do so will most likely lead you down a rabbit hole full of bad habits and production defects. Whether you’re a QA professional or a developer you need to gather all requirements in order to properly test anything. Every requirement has a specific outcome, and this is where we start.
The great thing about BBT is that we can use any tool to help us graph out the outcomes (what we expect, or the verification) and from there contexts (a specific behavior or action). In the BBT process we always start out with all the outcomes first, what we expect. In many, if not all cases, this will come in the form of acceptance criteria. From here it’s easy to map out which fields or values depend on or require one another. This will not only help in the automation process by creating a visual flow, but it will provide living documentation of how the web-service works and it will surface any additional questions that need answering.
Uncovering the test, if you don’t know already the manual approach to BBT has only a handful of steps:
- Identifying all possible outcomes
- Identifying all possible contexts
- Graphing out the outcomes and contexts
- Identifying constraints
- Generating a Truth table
- Identifying test cases based on the truth table
- Realizing test case steps
You can find more information about BBT as a whole by reading up on BBT for dummies. Which will thoroughly take you through each step.
Let’s go over something simple. If I submit a request with a valid user ID, first name and last name, then I expect that user’s account information. Now from this alone, there are a few unknowns. For instance, what else is required or even optional, and what exactly is being returned? Whether its required or optional, each value in the request is a context. The response would then be the outcome, what do we expect to see, whether it’s a positive or negative response. Graphing out the logic behind each value now becomes easier to manage when broken down into its simplest form. Example shown below.
Example:
The above example is easily created in paint. But from something as simple as this, you can then create the truth table, which will allow you to capture all possible positive and negative test cases. Leaving no stone unturned.
Here’s an example of the same test but with more information about the response. (Image provided by the SQA² BBT tool.)
Simply adding additions context, everything left of the successful response oval, can help modify the request made to a specific web-service. As well as adding additional outcomes to the successful response will ensure all possible values are verified. Typically adding more information to graphs comes hand in hand with updating or enhancing requirements. When doing so, it adds in additional layers to the truth table mentioned previously. This will in turn give more permutations for each value on the graph.
For example one additional outcome generates a minimum of two additional test cases; a “success” and a “fail” case. Now depending on the amount of contexts, it can multiply enormously, but it will ensure you have all possible test cases. Keep in mind this process can’t avoid human error, bad logic results in bad test cases. So be sure to review and double check your work before moving on to the truth tables and actual coding. Once coding starts, there is one very important understanding to keep in mind. All test cases are sharing context and outcomes, just in different combinations. Which means you only need to code each context and outcome once. This cuts down on automation time as well as making it easy to manage code.
When it comes down to it, BBT helps save time and money, creates living documentation, makes managing test cases easier, and provides peace of mind. As mentioned previously, you can use many graphing tools. However, I found that the SQA² graphing tool is by far the best and easiest way of test case creation and graphing. They can also auto-generate the test cases in the BBT framework, leaving you to only apply the steps. The tool actually reads the graphs and based off all the connections, it will write all the test cases for you, no more true tables needed. The BBT tool does make life easier and allows you to do the job faster. By no means is the tool required.
For more information about the open source BBT framework or the BBT tool SQA² has created please visit BBT.SQAsquared.com. Also, you can find more information and other blogs from how BBT works to BBT on the testing frontlines with Mobile, Business Intelligence, Security, and many others here
