No matter where you are in your career, you will inevitably be asked for your opinion on the quality of something. Usually, the person inquiring is your boss, manager, or some other highly important person, as if being put on the spot isn’t bad enough. Shy of giving them the default “let me look into it and get back to you later” there doesn’t seem like a lot you can do, especially if your immediate opinion is negative. For a Test Manager or QA Lead, this can be a nightmare. QA engineers deal with facts. They may ultimately have opinions on the matter, but no matter what the opinion may be, they will have to answer for it.
The best thing I’ve found to do is throw down your ninja smoke bomb, drop the status report on the managers, and roll the heck away. I can show them the facts and tell you what I think, but as the manager should have the complete view on the project (not just the QA portion), they, in theory, have a much better view of the health of the entire project. Hopefully, the information you provide can help those powers that be make the ultimate call on the projects.
What is in this mystical status report? Hopefully, you are utilizing a testing tool that allows you to plug into some form of cube for tracking purposes. If so, you are in luck. Setting up an Excel or SSRS report and connecting direct is pretty easy. You will just need to create a new connection, and map it to your cube. If you are one of the unfortunate souls that do not have a cube of some sorts, you may have more difficulty, but take heart in that it is still possible!
After I’ve got my connection established, I start pulling down statistics, including:
Taxonomy – to identify the main causes of defects
Defect Areas – to identify defect clusters on sections/areas of the project
Defect Count by Severity and Status – to identify how many defects we still have and are the numbers decreasing over time. Below, this graph is filtered by a single build.
Even though those are my standard metrics for a basic report, there are many more statistics you can pull down to help managers form opinions. For higher risk projects, I like to drill down into the reason for defect status. Was the defect deferred, fixed, or rejected? If there are a high number of rejections, we probably need a meeting between the QA Lead and Technical Lead to figure out why. It could be something as simple as miscommunication, or lack of QA engineer skill. No matter the reason, the statistic is valuable.
By providing this information to the entire team on a regular basis, you can help keep the project on track. If everyone knows the status, then the likelihood of being cornered about the status drops, allowing you to focus all your ninja skills on finding defects and increasing quality instead of avoiding people.