The SQA2 Blog: Advice Center, Automation, BBT - Behavioral Based Testing
We’ve all seen it: The test automation that started out passing at 100 percent is now running at a 60 percent success rate…and somehow that is considered acceptable. But how can a 60 percent success rate—i.e. a 40 percent failure rate—be acceptable? If 40 percent of your tests are either getting executed manually or not executing at all, your return on investment in software development testing automation has taken a nosedive as well.
How does software development testing automation get to this point? What causes so many tests to fail over time? While every situation is different, the underlying issue often looks like this…
A couple of analysts on a software development team are tasked with creating software development testing automation for an application. Then the development team starts to quickly and iteratively build new features into that application. While the application’s feature set progresses at a rapid pace, keeping the automation up to date does not.
For example, say the underlying application is an e-commerce website. As the product and engineering teams add and change features such as a product page, a checkout cart, a nice signup page, etc., someone needs to ensure there are automation scripts for creating test accounts, test orders, test purchases, and, more importantly, they must update tests when features change. Whether or not this happens in a timely fashion often depends on how the automation was built. If it is difficult to maintain, and the skill set of the team isn’t up to the task, the daily automation tasks quickly overwhelm the QA team—and, predictably, the test automation failure rate starts to skyrocket.
Preventing the problem of failed testing automation
Luckily, maintaining software development testing automation doesn’t have to be difficult. Recent developments in this area, such as Behavior Based Testing (BBT), has made it possible to prevent these problems. BBT ensures the automation is easily maintainable, and therefore an effective tool for the team to use over the long term. This also ensures the big investment in automation continues to pay off in the long run.
Using Behavior Based Testing to ensure maintainability
At its core, BBT is a visual approach to mapping requirements to test cases. Instead of hard-coding tests for each individual requirement, you map out context, events and outcomes, and then translate these to Gherkin-style test cases. The BBT approach is similar to Behavior Driven Development (BDD), but visual and with a higher level of coverage.
It is these Gherkin-style test cases that are automated, although not in the way you might expect. While most organizations would take these test cases and automate the test methods one case at a time, with BBT you don’t. The BBT approach is based on automating the individual steps needed to complete multiple test cases. Each context, event and outcome is automated in easy-to-maintain code snippets. The automation then pulls in whichever context, event and outcome it needs to create the test.
While each step is therefore maintainable code, the big benefit of this approach is that you do not have to “recreate the wheel” each time a change is made. Once you’ve built out a framework for testing a particular aspect of your application, BBT will reutilize it. When you introduce a completely new feature into the application you’ll need to have your Automation Engineer modify the testing code. But when a new field is added to an existing feature, updating the automation is as simple as adding a new context—and only minor coding is required. BBT therefore makes maintaining your software development testing automation over time easy.
Example #1: Modifying a registration form
To illustrate how this works, let’s say your application has an existing registration form that captures the user’s name, email, address and password. You thought things were going well with this registration form, but now your company’s Customer Service department has alerted you to a problem. When people are setting their password they sometimes “fat finger” it on their mobile device and fail to type in the password they had in mind. Then when they try to log in next time, the password that they thought they had set doesn’t work. Customer Service has asked your team to add in a “confirm password” field so that users would be forced to type the password in twice.
With Behavior Based Testing all you need to do is add a confirm password context to the already-existing method in the test case and complete a tiny bit of coding.
That’s it!
Example #2: Adding a new feature to the login form
Here’s another example. You want to add a “remember me” check box to an existing login form. Instead of rewriting all your automation test cases, once again with BBT you simply add a new context. You create a method that’s responsible for checking that check box, add it to the BBT map as a context titled “Remember Me checked” and then all the test cases that have already been built are automatically updated. This include any other test that uses this test as a prerequisite. That’s it! No massive recoding effort, and no need to hunt for other instances in the automation that might be affected by this change.
Conclusion
As these examples illustrate, the beauty of BBT is that it makes it fast and easy to update your software development testing automation as the application changes. BBT eliminates the need to find every location where something is done or interfaces with a form. When requirements shift, your test cases and automation are easily updated.
If you’re still creating testing automation based on methodologies that don’t allow for easy maintainability as your development project grows, and are tired of watching your automation efforts fail over time, Behavior Based Testing can provide the easy maintainability you need. With automation that works both initially and over time, you’ll enjoy a solid return on your testing automation investment.
Stay tuned for the next article in this series, when we’ll explain how BBT provides quantitative coverage of requirements.
