Tuesday, April 26, 2011

Defect Severity

I now have a huge backlog of blog posts, but here is one that has me losing some sleep.

Why do developers seem to get their feathers ruffled when we talk about defect severity?  I am not implying that developers are ducks, chickens, or geese.  I could have used the phrase why do developers get their panties in a wad when we talk about defect severity, but that did not seem PC.

First I will give a little back ground.  I was asked to inspect and access team quality.  Hmm, that is a monumental task.  Where shall I start?  Lets take a look at how many severity one defects are in production.  If there are any severity one defects in production, then the quality status is RED.  If there are no severity one defects in production then the quality status is GREEN.  Seemed like a simple assessment, which would require some follow up conversations if the values were red.  By no means do I mean to imply that this is the only indicator of quality, but it was the first flash light used to look into the cave.

But wait we (development) do not care about severity we focus all of our efforts on Priority.  This has been echoed by at least three development leads.

My two brain cells are telling me that severity is the perception of how bad a defect is and priority is the process around getting a defect resolved.  Hmm, do I have this all wrong?  Is priority the only thing of importance.  I read Perfect Software by Jerry Weinberg a few months ago.  Did I not understand his writing?  Well I guess I had better do whatever any internet geek would do, "Google It"!

I found a great post by my friend and colleague Dr. Stan Taylor.  I did not see to much in that post that I could disagree with.  Next I stumbled on a Bug Advocacy video by Professor Cem Kaner.  I did not find a definitive answer there.  I read at least 10 other blog posts and all seemed to have a slightly different spin, but there did seem to be some consistency that severity is the perception of the user and priority is the decision process on when to fix a defect.

I am still perplexed as to why inspecting severity gets such a negative response by development.  Let me attempt to noodle out a couple of basic definitions.  Please note that I did not come up with these definitions, but I tend to agree with these.


What is Quality? -  “Quality is value to some person (that matters)”

What is a bug? – Something that bothers (bugs) someone who matters.

Severity - defines the impact the defect has on the customer and the likelihood of occurrence. 

S1 – Product is on fire.  We are getting sued.  We cannot take money
S2 – Breaking but we can offer a workaround
S3 – Minor functional defect (pain in the butt or poor user experience)
S4 – Cosmetic (we can live with it)

Priority – determines the order in which defects will be fixed/resolved and retested

            What do we fix?
            When do we fix it?

Priority decisions are based on impact on the customer (severity) and resolution difficulty (time & resources)

P1 – Resolve Immediately
P2 – High Attention
P3 – Normal
P4 – low

Those seem reasonable to me and some of the definition came from a Power Point presentation by Dave Whalen called "The Matrix". 

So now I am thinking that priority cannot be established without taking into consideration the severity.

I am still puzzled as to why developers do not want to track defects based on severity.  Somebody perceived the defect as having a negative impact on someone.

I am concluding that I need to have more conversations with developers!  Here is a reasonable post provided by one of the development leads.  One point made in our conversation was that when I used the phrase perception of the customer it inserts emotion into the concern.  The developer preferred impact on the customer. 

I can certainly understand with the pressures of Rapid Software development of today there is little time for fixing defects. So the balancing act of meeting milestones and building great features is extremely priority centric.

However persons documenting the defect have no input with respect to priority at the time of discovery, so they must focus on setting a proper severity.  A well-documented defect can be leveraged as a sales tool to influence priority and the person submitting the defect must be an advocate of the defect in order to see it through to resolution.

When severity and priority do not align, I would suggest it is extremely important to communicate the factors used to set the priority to the person who established the severity.

Hmm, have developers been burned in the past by incorrect values of severity being placed on the defects? Could that be the reason for the ruffled feathers?  I think this might be a good possibility.

Perhaps feathers are not ruffled at all, because based on all of the blog posts the ambiguity of Severity and Priority have been around for a long long time.

Time to read "Perfect Software" again!


Friday, April 01, 2011

Goodwill for Automation Code

I had an interesting dream, thought, or inspiration last night.  It could have been codeine induced based on the horrible cough I have been battling, but no the less it was an important thought.

What happens to legacy code?  Does it simply get deleted and gone eternally?  Does it fall into an abyss?

No!  We take it to Goodwill for Automation Code.

It occurred to me that one of the key takeaways from STP conference was mentoring.  The "Ah Ha moment" was that we could leverage legacy code to educate others.  We have some Software Engineers in test that are struggling with TDD, test driven development.  So my concept is to look at our code repository and find code that has matured over time.  We can then share this evolution of code with the testers that struggle with TDD.

I also struggled a bit with TDD, until I saw the light.  Even for test automation it is important to unit test your code.  How did I learn?  What turned on the light?

Mentoring by my colleague certainly was key. The knowledge of knowing my new methods would potentially be used by many tests forced me to write better code, that could easily and repeatably be tested.  I still struggle because I simply want to get a test implemented quickly, so I admit sometimes I write my unit tests after the fact.  One rule of thumb I do have is that if I struggle at all to write a piece of code, I immediately stop and write the unit tests.  Breaking the complexity into simpler chunks helps me to get to a more robust solution.

So I am going to go into the office today, then search for learning examples of code.  These legacy jewels will then be moved into Goodwill for future education opportunities.

Cast off code will be used to assist others.  Amazing the thoughts that come to you during sleep and I do not recommend the codeine!

Keep on testing!