The SQA2 Blog: General
Most QA vendor renewals get signed on autopilot. You know the vendor, the team is in place, and switching feels expensive. The contract rolls over, and the questions that would tell you something about the next twelve months never get asked.
The standard evaluation measures the wrong thing. Rate, skills, and coverage tell you whether a vendor is good today. They tell you nothing about what happens after the things that always happen, happen. People leave. Teams change. A project suddenly needs more capacity than the vendor planned for. Whether your QA function absorbs those events or gets knocked over by them depends on how the vendor is built, and that almost never shows up on a scorecard.
I wrote recently about why QA continuity is an asset, and an overlooked one. The short version is that the most valuable thing a QA team develops is knowledge of your system, and in most vendor models that knowledge lives in individual people. When they go, it goes, and your team pays to rebuild it.
This post is the practical follow-up. Below are seven questions that measure whether your QA vendor is built for continuity or just staffed for the moment. Answer them honestly about your current vendor. It takes about two minutes, your score shows up at the end, and nothing you enter gets collected.
The QA Vendor Continuity Score
Seven questions about your current QA vendor. Answer honestly. Your score appears at the end, and nothing you enter is collected or sent anywhere.
These are questions any vendor built for continuity can answer easily. If you want to see how we answer them, start a conversation.
Reading your score
If you scored 12 to 14, your vendor built for continuity, and that is rare. Renewals are low-risk for you. The points you dropped are still worth a conversation, but you're asking for improvements, not fixing problems.
If you scored 7 to 11, you're in the most common band, and the most uncomfortable one. Everything works right now because the same people happen to be on your account. Nothing protects you when that changes, and something always changes. The useful part is that your zeros and ones give you the agenda for your next vendor meeting. Ask your vendor to answer each one specifically. A vendor with real answers will give them. A vendor without them will give you reassurance, and you'll hear the difference.
If you scored 0 to 6, your QA function depends on individuals and staffing conditions your vendor doesn't control. That doesn't mean the relationship is bad or the people are weak. It means nothing is holding it up except the people currently on it, and you find out what that costs at the worst possible time, usually mid-release. Have the renewal conversation early and on your terms, and lead with the questions you scored zero on.
One more exercise worth running
Take the same idea into a meeting as a scenario. Assume your vendor loses a fifth of its bench over the next six months because people quit or move on. Ask which of your releases slip first. Ask who picks up what those people knew. Ask how long it takes replacements to become productive on your system instead of just assigned to it.
If that discussion produces confident answers, write them down and hold the vendor to them. If it produces silence, you've learned something more useful than any reference check would have told you.
None of these questions are "gotchas". A vendor built for continuity answers all seven easily, because the answers describe how they already operate. If your vendor can't answer them, better to know that before you sign for another year than after.
