The SQA2 Blog:
Automation and where to start?
With a vast amount of frameworks and tools for automation, how do we decide which is best? And how to we decide the best automation testing tool and/or framework to implement for our project? Repetitive tasks such as identifying, realizing, and executing test cases while automating a full suite of regression test cases can become cumbersome. The need for a strong automation tool to alleviate pain-points such as maintainability and usability has become a necessity.
What are the criteria you need to consider when selecting an automation tool? Does this tool satisfy your need for coverage? Does it support all applications; from desktop to mobile? What is the learning curve and is it acceptable for your goal? We have to consider these questions when investing time in a automation tool/framework. This will determine the long-term success for the organization.
Many organizations implement automation for its obvious benefits. It is a quick and easy solution providing coverage for critical paths, regressions, and smoke tests. However, there is a steep learning curve when it comes to automation. In addition, the path is narrow and relies heavily on others to “jump on the bandwagon.”
Planning towards a manual approach is often simpler. It does require developing skills and a dedication to process, best practices, and principles. This assures the success of a manual approach; as creating a framework for an ease of use often faces push backs from colleagues for many reasons.
The Case for BBT
What makes an automation framework successful? Usability. Adaptability. Simple learning curve. To be simple enough for any professional with little to no programming experience to generate, manage, and automate test cases. All would be factors for many when decided if a framework is successful. The Behavior Based Testing approach to automation is that solution! Providing QA, developers, and stakeholders a visual representation of test coverage for any component in their application and having confidence in the test coverage; with minimal work.
Utilizing BBT visual cause and effect graph, businesses can quickly identify the behavior and coverage for their application. Below is a sample of a simple pixel tracking web services:
Compared to a manual approach, from a perspective of a user without any programming experience, we are required to understand the language to determine the coverage of the tests. However, with BBT, it provides a quicker means of updating the values directly in the code and debugging instead of the user interface debugging mechanics BBT provides.
We sacrifice ease of use for an individual over coverage and visibility to our test cases. This can become cumbersome when presenting the amount of coverage of a component to assure it meets the acceptance criteria. This happens because programming languages require basic syntax knowledge to understand the coverage. Cause and effect graphs are understandable universally as QA, developers, and stakeholders can identify the components needed for the behavior. Another benefit BBT provides is generating permutations of test cases and ensuring full test coverage for the behavior. In comparison, the manual approach will take a lot more effort to provide additional coverage, assuring all areas of the behaviors were not missed.
As an overview, we compared manual approach to Behavior Based Testing and identified the pros and cons for each. Manual provides an easier means of debugging and modifying the code at the code level, without the maintaining of any UI components. BBT provides an easier approach in communicating with QA, business, and developers in terms of coverage. It also allows maintaining of graphs instead of code, which is much easier to understand by non-programmers, and generates permutations of test coverage.
In our next blog, we will dive deeper into the comparison between manual and BBT automation.