The Difference Between RBT and BBT

The SQA² Blog: BBT - Behavioral Based Testing

Most professionals involved in the world of software testing have likely come across the term Cause-effect graphing. You might have also heard of the two testing methodologies that utilize Cause-effect graphing, Requirements Based Testing (RBT), and the more recent addition, Behavior Based Testing (BBT). The purpose of this post is to explain the approach these two methodologies take toward designing, and developing tests, as well as, provide examples, and contrast the two. As a disclaimer, I work for SQAsquared, the company that develops and pioneers the Behavior Based Testing framework, but I will try to be unbiased during this comparison.

Requirements Based Testing (RBT)

Just as the name implies, Requirements Based Testing has a strong focus on the system requirements. It is a testing methodology supported by Bender RBT Inc. This concept is very straight forward. The following 12-step process sums up the RBT process:

1. Validate requirements against objectives
2. Apply scenarios against requirements
3. Perform initial ambiguity review
4. Perform domain expert reviews
5. Create cause-effect graph
6. Logical consistency check by BenderRBT
7. Review of test cases by specification writers
8. Then review of test cases by users
9. And review of test cases by developers
10. Walk test cases through design
11. Walk test cases through code
12. Execute test cases against code

For more detail on each individual step you can take a look at the Bender RBT Requirements Based Testing Process Overview document

Testable requirements within the RBT methodology must go through a checklist of items. The checklist, made up of sixteen items, with the most critical characteristics being, Deterministic and Unambiguous. There is even a provided list for the analyst to check a requirement for ambiguity.

As a result, the checklist of items act as a way for the Analyst or Tester to assess the validity and quality of the requirement. Thus, if the requirements fail any of the items, a domain expert reviews the requirements.

The expectation is that once the “requirements” pass all the checklist of requirements, then the system is testable. Then you can continue with test design and creation by way of cause-effect graphing.

Example

Let’s take a look at an example of this in action. For the purpose of this example, I’ll be acting as all roles (domain expert, tester, developer) for a simple system that allows a manager to track all their employees.

Let’s say that company XYZ is in need of a new system that allows them to enter and save employee information. For simplicity let’s say all they need to store is the first name, last name, and social security number. Let’s narrow down the scope for this example, and take a look at the requirements revolving around adding employees to the system.

• New employees can be added to the system by specific users
• Form with employee information must be valid
• Employee is added to the system once form is submitted

At this point the Analyst, or tester will look at these requirements and go through the checklist. Just checking for the two most critical characteristics, Deterministic and Unambiguous, we can see a few of the requirements fail. Upon completing the initial check, we will review with the domain expert and business, and come to the redefined requirements:

• Only System Administrator, and Manager roles can submit an employee form
• Employee First Name must be valid
• Employee Last Name must be valid
• Social Security Number must be valid
• Employee is added once a valid form is submitted

For the sake of this example, let’s assume that everyone on the project understands what the definition of “valid” is. Now that we have our new and improved requirements, we go through our checklist again and verify we can test all requirements.

After that, we can move on to creating a cause and effect graph. Below I’ve created a caused-effect graph based on the requirements. The graph shows an OR relationship between “User is System Administrator” and “User is Manager”. It also shows all the conditions must be true for the “Form Passes Validation” effect to happen. We also apply constraints to the “Form Submitted” and “Employee is Added” effects.

Figure 1: Cause-effect graph created on SQAsquared’s BBT graphing tool

The next step in the process would be to create a truth table from the graph, and from that you would derive your test cases. Once you derive the test cases, you begin reviewing the test cases with others on the team and then eventually the execution of the test cases.

Now that we’ve seen RBT in action, let’s take a look at Behavior Based Testing.

Behavior Based Testing (BBT)

Behavior Based Testing takes a behavioral approach to requirement analysis, test design, and creation. BBT is a relatively new testing frameworkthat is being pioneered by SQAsquared. It takes a behavioral approach to describing a system in the same sense as Behavior Driven Development (BDD), and marries it with Cause-effect graphing. BBT aims to describe the behavior of the system under test in terms of its Outcomes, Contexts, and Constraints. I will go into the definition of these terms a little later in the post. The BBT process is as follows:

1. Review system requirements
2. Identify Outcomes
3. Identify Contexts
4. Create Graph of Outcomes and Contexts
5. Identify system Constraints
6. Add Constraints to graph
7. Generate a Truth Table
8. Identify Test Cases with Truth Table

Focus on behavior

As you can see from the process steps, BBT focuses on describing the system behavior through the graphing process. BBT also changes the language used while creating a Cause-effect graph. BBT encourages the Analyst, or Tester to think in a certain way, by utilizing a specific language. The BBT framework uses system requirements as a starting point to describe the system behavior. It also relies on domain knowledge to assist in the creation of Contexts and Outcomes, and works well alongside Domain Driven Design (DDD).

Once the behavior of a system is described and graphed, this would then be reviewed with the development team. The goal of having a graph, in the language of Contexts, and Outcomes, is to have a single, understandable, talking point, for a development team.

An Outcome is defined as an expected end result of a step or series of steps based on certain input conditions. The BBT framework also defines several best practices when defining Outcomes, such as always write then in the past tense. A Context is defined as a condition that must be true in order to result in an Outcome. A Constraint is used to indicate logical relationships between Contexts, or between Outcomes. Constraints along with some best practices are put in place to keep the graph readable, and the downstream test cases precise.

BBT also tries to force the Analyst, or Tester to take an outside-in approach, by first creating the Outcomes first, and encourages them to think about what are the Stakeholder’s goals. This key point is what makes BBT a behavioral framework.

In a nutshell, the goal of BBT is to leverage the advantages of BDD, gained by using a ubiquitous language to describe the behavior of a system, and combining it with the quantifiable test coverage gained through Cause-effect graphing.

Example

Let’s bring back company XYZ and apply a Behavior Based Testing approach to the problem. First step, let’s take a look at the provided requirements again:

• New employees can be added to the system by specific users
• Form with employee information must be valid
• Employee is added to the system once form is submitted

Step 1

Let’s apply the first step and identify Outcomes using some behavioral terms.

• The “Employee” Information Should Be Validated
• An “Employee” Form Should Be Submitted
• And “Employee” Should Be Added to the System

Notice the language that is being used when defining Outcomes. The fact that Outcomes are expressed using the term “Should Be” leaves open the question “Should it be?” Much like in BDD, the simple change of language, causes a shift in the way someone thinks. The fact that Outcomes are not expressed in concrete terms like, “must”, or “is” allows the Analyst, or Tester to question the validity of the system behavior. Therefore, BBT works toward developing robust requirements and tests.

Step 2

Moving on with the Context identification steps:

• User is System Administrator
• User is Manager
• First Name is valid
• Last Name is valid
• SSN is valid

As you can see the Contexts identified here are identical to the cause identified in the previous RBT example. I’ll touch a little more on the reason for this in the next section. Next I will also add Constraints to the Outcomes specifying that “Form Should have Passed Validation” before “Form Should Be Submitted”, and “Form Should Be Submitted” before “Employee Should Be Added”.

Figure 2: Cause-effect graph created on SQAsquared’s BBT graphing tool

We derive a Truth Table from the completed graph. We can then create test cases directly from the Truth Table. These steps are similar to any implementation of Cause-effect graphing.

RBT vs BBT

It could be said that the relationship and differences between RBT and BBT are similar to those between TDD and BDD. While both methodologies make use of Cause-effect graphing, the main difference between RBT and BBT is the language used, and the general mindset when implementing either RBT methodology or BBT framework.

One focuses on defining the quality of system requirements with the use of very detailed, and defined characteristics, while the other focuses on the use of a ubiquitous language and encourages you to think in behavioral terms.

Final Thoughts

Behavior Based Testing as a whole is still relatively new, but it is an adaptation/combination of a couple tried and tested methodologies.

I am excited to see BBT grow with time and use. I personally know the plans we have for BBT at SQAsquared, and I am excited to see how it grows.

If you have any feedback, corrections, or experiences with either of the testing methodologies, I would love to hear them. Thanks for reading!