Looking in a Mirror

I initially was going to add a post about checking versus testing.  I started to reread some of the posts starting in 2009 to the present.  I quickly concluded that the topic truly is not controversial at all.  In general I concluded that both are important if you do care to make a distinction.

So I decided to walk through recent blog posts and I came a across a gem from Peter Walen.  I have meet Mr. Walen at I believe two conferences and I always learn something, so enjoy his post.  Where his post struck a nerve with me is the idea of self reflection or introspection.

I believe people tend to blame others before they look in the mirror.  I recommend that everyone do some introspection before they seek external cause.  So I would like to point anyone who happens upon this blog to check out "The Leadership Gift".  Christopher Avery educates communities on the "Responsibility Tree".  You can see the tree on the right hand side of the link.  I think very few people are able to work up the ladder to the final rung to take responsibility for their own actions.

When I read Peter's post I can relate.  The hiring process is hard.  Letting go of control is hard.  Trust is hard.  Achieving success hard.  So in combining the post on personality traits and the responsibility tree, I think you should be looking in the mirror at oneself first in order to continuously improve.  Introspection will allow you to grow and perhaps help others.

To Many Bugs

I was a bit disappointed that I never saw a response to my questions on the Atlassian blog posts in order to gain a bit more context.  However I do realize the comment ping pong can go on and on.

I read quite a few blog posts this week, but nothing really got me really thinking.  I did come across a tweet that was interesting by Michael Kelly.  He tweeted "There is nothing more depressing to me then logging bug after bug after bug. Feels like a waste, but it's clearly needed. Sad."

I recently experienced a situation where we had to stop testing for a similar reason.  Two many defects were being found in one specific section of a website. It was not the highest risk section so we felt the value of the testing although important was not the best use of the time available.

Mike's tweet made me wonder about when is it appropriate to stop testing if there are too many defects. If this occurred in a single session, I might recommend the team spend another iteration before the next test session is scheduled.  Another angle is that I might call a test dojo where the team is all in a room testing together.  Perhaps we can find and fix as we go.  Developers typically take a lot of pride in their work, so they will be responsive to rapidly fixing things or openly admit they need more time or assistance.

I believe I also read a tweet from Lisa Crispin that illustrated a paired story deliver approach.  I like the approach but the paradigm may not always work.  In the case of finding two bugs it might be a great time for a tester to offer to pair.

To many bugs from my point of view is most likely hurried work.  I guess it could be due to poor design, lack of functional detail, or something else that was poorly communicated.  Regardless of the cause I would recommend that testers find the most professional and collaborative way to adjust the situation.

I agree with Michael that it can be a huge time sink having to document a large volume of bugs.  Personally I would rather see them get fixed than create an object in some system that now needs to be managed to resolution.

I guess find the bug nest early and get creative on a rapid solution.

Happy Testing!

Quality Assistance

Recently there were a couple of blog posts by Atlassian testers.  The first article I read was called “Inside Atlassian - Introducing  Atlassian QA” , which was developed by Andrew Prentice.  The second article was titled “The Jira QA Process”, which was written by Penny Wyatt.

Both are well-crafted articles and I believe they did the job of causing me to think.

First I would like to explore a statement was made by Andrew.  He said, “To be fair, a large number of people claiming to be professional testers can’t test either.”  I cannot say I disagree with the statement, but I ponder how do you determine if someone can test or not.  I would suggest that it is through demonstration of skill and others having the trust in that skill.

In Penny’s article she explains, “During this process, a QA engineer has multiple points at which he or she provides input into the way the story is developed and tested – providing every form of quality improvement except actually testing the story themselves.”  This thought seems to slightly contradict Andrew’s position by stating their testers do not even test.  I agree with her premise that they are involved in the entire process injecting quality into every story.  Where I disagree is the testers not actually testing.   I certainly may be missing something, but I have found that through the active act of testing a vast amount of knowledge is gained.  Typically great testing leads to more great testing.  Perhaps Penny was only referencing testing at the story level.

I agree that everyone should own quality.  I agree that experienced testers should assist in the education of others.  I do not like the term Quality Assurance.  But I think great testers should always test.

I am leaning toward using the term Quality Engineering, but I know that will be controversial so I had better prepare.

Should Testers Learn Automation?

 I have read various articles that debate this topic, but I never really have formed my own opinion.  The more I think about the future of testing, I conclude the answer is yes.  Testers should learn automation.

One of the most influential testers today is James Bach.  He knows code.  His brother admitted in an article recently that code is not is forte, but if pressed I bet he knows code.  Writing code might not be absolutely necessary to be a great tester, but I think it provides context.

I did a reflection on my career as a tester. 

Stan Taylor hired me at Excite@Home.  Stan gave me the start in the field and was a terrific mentor.  At first I learned how to set up multiple test environments, which gave me context about operating systems and browsers.  Next challenge was performance testing with Silk Performer.  I learned a proprietary language, 4Test and dabbled in regular expressions.  I was then able to extend the 4test language to functional testing, Silk Test.  The next phase was interesting when I got moved to a development team to do UI work.  Sad part is again I learned a proprietary language called Dynamic Content Generation, DCG.   My current boss, Jack Yang, was my development mentor.  He educated me on development basics: loops, logical statements, repository branching, tagging, & command line executions.  He gave me coding assignments that would challenge my skills, pushing me beyond my comfort zone.  Soon I was making production ready changes.

Next job was at a start up again with Stan Taylor.  Stan had built a beautiful JavaScript library and leveraged Webload for functional test execution.  Not only did I learn JavaScript, but also refactoring and reusable methods became important.  Code reviews and collaboration were great practices.  One important lesson to share about this experience was it was the first time I got to do “white box” testing.  I got to pair with developers and inspect java code. I was permitted to make suggestions on how to enhance the unit tests.  The ability for me to understand code structure made this possible.

Next adventure I got to learn some sound testing processes with a company that build complex telecom oriented software.  Guy Lipof and Joseph Griffin leveraged efficient testing techniques and I was exposed to collaborative testing in the form of test fests..  Eventually I ended up leading seven remote testers.  All of the testing was manual and it took the eight of us five business days to execute 1500 regression tests.  I came into work one day and I was informed that we had to let the seven testers go for budgetary reasons.  Holy cow, this is a great team how can I regression test this by myself.  We are talking eight weeks of busting hump.  The conclusion was automation.  I turned to WATIR and ruby.  WATIR Library was very education friendly.  The forums and people were amazing.  The result was in one month I had automated 70% of the existing tests, tossed out 20%, and the remaining tests were manual.  In the end I could do the complete regression plus test new features in five business days.  Was it the prettiest code in the world?  It definitely was not.  I was able to refactor some common methods and modules.  I attended AWTA(2007) where I met an amazing group of testers.  I met Brett Pettichord, Paul Rogers, Elisabeth Hendrickson, Brian Marick, Chris McMahon, Charley Baker, Andy Tinkham, Bob Cotton, Jim Mathews and many more great testing minds.  It was at this conference I learned the power of pair programming and collaborative thinking.  I was inspired to learn more by all of these people sharing their expertise.

In 2009, I moved to my current company.  The mission was to help set up an automation framework.  We settled on Ruby, because it was the language I was most comfortable with and there were tons of examples available.  We selected Selenium for the potential of cross-browser automation.  Building out automation is definitely a fun adventure from my point of view.  Some things I learned during this adventure were paired programming, factory patterns, page object patterns, mocks, and test driven development.

Now as a Director of Testing, I do not find myself writing much code.  Recently I recognized how valuable this journey has been.  I also learned that if you do not practice you get left behind a bit.  So I am now trying to learn some new aspects of software development during my copious spare time.

I did not share this short journey to highlight myself.  I wanted to share the experiences because I think some aspects of this journey are important to learning as a tester to constantly improve.  So my message to any tester that happens to stumble on this blog post.  Reach for the stars and add an understanding of code to your tool kit.  The majority of the lessons are at your fingertips free or at a very low cost.  You never know what you might learn reviewing someone else’s code and once others review your code you learn even more.

Happy Testing!

Computing for Data Analysis

I am attempting to sharpen the saw.  I am taking a course online from Coursera, which I believe is associated with John Hopkins University called Computing for Data Analysis.  The course is turning out to be HARD!

https://class.coursera.org/compdata-003/class/index

The course assumes that I remember math from 30 plus years ago, which obviously is a bad assumption.  The course also assumes I have been exposed to statistics, which is also false.

I am learning the basics of the R programming language.  I am getting to learn RStudio.

Although it is hard, I think in the end I will have learned a tiny bit.  Let me share with you two lessons I learned today.

The runif command does not mean "run if" but "r uniforms".

Quote - "less typing is always better because good programmers are always lazy"

Please note I thought the quote was funny.  I do not believe developers are lazy.  I do support the premise that developers try to write code as simple as possible.

I will continue to tough out this course because in the end I believe I will have learned something useful and applicable.

Quality Artifacts Everywhere

Recently I came across a situation where I observed defects, tasks, and even stories that were documented in multiple places (Google Doc, Issue Tracking System, Complex Stories, Wiki...).  How can a team evaluate quality when there are so many lists?  I looked a little deeper and even found single defects in the issue tracking system that were a list of defects.

I am right there with the next person for not entering an object into an issue tracking system if I do not have to.   Once an issue artifact is created it must be managed through to a resolution.  My guess is you have no clue with respect to quality of you have lists buried within lists within other lists.

If you have 100 defects and 100 tasks left to complete in an iteration, then you can evaluate when you are near done.  If you have 5 lists buried within the 100 defects , 5 lists buried in 100 tasks, and a Google spreadsheet with 75 more ideas, how do you ever know you are nearing done.

As much of a pain in the tail as it is I recommend two approaches.

One if you find an issue and do not want to put it into the tracking system, then fix the issue immediately and verify that it is fixed to your satisfaction.

The second is to enter the issue into the issue tracking system.

My final recommendation is to settle into a specific process, follow the process, iterate on the process, but do not create numerous processes within processes.

Keeping it simple helps to keep the team on the same page.

Happy Testing!

Do you have what it takes to argue?

I watched Jon Bach's keynote at CAST 2013 this morning.

As usual he has a fantastic way to deliver information and the topic was on the money.  I agree that we should have more arguments.  The one challenge I have is that I may not have all of the skills to facilitate a sound argument.

I have a spouse who typically cannot lose an argument.  Her brother who has a law degree is equally as acute.  Between the two of them they help sharpen my argument skills, but I am lacking the ninja tools to win consistently.  The Software Industry is full of extremely sharp people and many have the chops to win an argument.  As Jon did for his keynote I thought I had better do some research.

The first site stated that the first thing you should do is select the strongest side of the argument.  I am not sure this is the right advice unless I was wanting to be a debate champion.  Normally the arguments I find myself involved in are because I believe in a certain concept.  So for starters my position may not be the strongest side.  So I think my take away from this suggestion is that I need to always be prepared to persuade the other side that I have a very compelling position.  So I need to reflect more frequently and build out the key list of bullets on my position.  I need to have these points stored in the part of my brain for rapid recall.

Another site talked about sneaky tactics.  The points on this link were pertinent, but I am not sure I am clever enough to be sneaky.  The two points I think I need to add to my skill set is not diluting my position with weak points and consciously concede valid points to the opponent.

Let's face it; the best way to win an argument is to avoid it altogether.  This position is not what Jon was advocating in his key note.  What I take away from this statement is that if you do not have acute points to defend your position perhaps it is time to agree to disagree then go fill your arsenal with more context.  Live to fight another day may be more applicable.

In a couple of weeks I may have the pleasure of visiting Rice University.  I stumbled on this gem.  The first bullet point is "Drink Liquor".  Jon Bach probably would not support this position since he kindly provided me his drink tickets at STP in Nashville.  Thanks Jon!  I got a bit of a chuckle when I read this point, but I think the underlying tenant is that you need to have an element of confidence when stepping into the argument.  I also concluded from this post that humor probably can also help in a good argument.  I think for me my confidence grows by having more information, "context".

I am the type of person who just puts stuff out on the table without necessarily thinking first.  I think I should learn to take my time, stay calm, and apply logic.  I am certainly not afraid of a great argument as long as the TRUST is there.  Jon referred to this as being in a safe environment.  I will probably continue more thinking and research on this topic.  I think I have some arguments coming my way, so preparation is probably a good thing.

Thanks Jon for sparking thought on this topic.  The next time I gather with testers I think we should have some exercises that improve our arguments skills.  I am going to have to give that concept a bit more thought.  Stay tuned ...

Happy Testing!

Yet another round with defect Severity and Priority

Let me start by a quote from a developer.  “We only focus on priority.”

I have had so many conversations around severity and priority.  Rather than banter about the difference and the usages, I came up with a new concept relative to defects.

Equality to all defects!

I would like to see teams implement TDD practices and a mantra of “We found it. We will fix it!”  Imagine a world where developers, testers, and product managers all treat every defect equally.   We cannot release new code because it is defective.

In the Agile development world the best teams fix their defects as they build the product.  Testers are catching them early in the process, so why not fix them.  It is true that there is no such thing as perfect software; however, if you happen to find an imperfection should you ignore it.  The answer is no.

Do you ever hear “We are do busy to fix defects”?  Or  “that is just cosmetic, so we will fix it later”?  The statements indicate that defects are not important. 

All defects should be treated with equal importance no matter where they are found.  In fact if a customer finds a defect it should take on greater importance.  The defect should be fixed immediately in the next sprint.  If the team uses Kanban then push the defect to the top of the queue.

If we treated all defects as equals then we can throw out severity and priority.  Most importantly we testers no longer have to explain that there is truly a difference.

What work is left is harder!

I am reading Slack by Tom DeMarco.  There is a chapter where he is talking about process obsession.  I have always been against process for the sake of process.  At one of my previous jobs it took at least 8 hours a day just to get through the CCMI daily bull shit.  My point is not to rant about how inefficient institutionalized process is, but to share a quote from the book.  I found this statement interesting.

"When the new automation is in place, there is less total work to be done by the human worker, but work is left is harder."

As an experienced tester I believe test automation is important.  I promote automation so that we have more time for cognitive testing.  I cherish the time available to execute well designed test sessions.  What never occurred to me is that the cognitive testing just might be harder than the automation.

From my experience automation is pretty darn hard.  Once I completed automation I did feel a great sense of satisfaction, but I never pondered that I had freed up some of my time to do stuff that was harder.  I viewed automation as giving me freedom.  I now had the freedom not to do scripted testing, but the confidence to explore.

The phrase "less total work" is also interesting and from my point of view somewhat misleading.  In theory the more software automation you have the more time you have to innovate new concepts and ideas, so in essence the scope of work increases.  With more automation I believe that maintenance increases.  Automation frameworks are constantly advising as a technology.  At some point the team must refactor to stay current.

I am looking forward to the day when automation equals less work!  For some reason I think I am going to be waiting a very long time.  Does software automation truly give us slack?

Vanishing Defects

I know I watch to much television, but the Allstate commercial during the Seahawk versus Redskin game gave me an idea.  Allstate has a concept called the vanishing deductible for safe drivers.  Can we use a similar idea with defects?

I would like to propose that if a defect is more than 120 days old without a resolution, we should make it vanish, Vanishing Defects.

Development teams typically are investing the majority of their energy building new features.  From my 12 years of experience there is little time dedicated to fixing the defects.  The problem is compounded deeper by a defect ranking system.  If severity of defects is marked annoyance, cosmetic or non-essential, defects become destined for the eternal defect pile.  Teams triage defects and set a priority.  If defect priority is marked normal or perhaps some day, then those defects are also destined for the eternal defect pile.  The vanishing defect model would automatically reduce the size of the defect pile.

When customer support inquires about a defect that is older than 120 days, the response would be simple.  Your defect vanished!

Some software development teams should be on the A&E TV show, Hoarders.  I have seen backlogs with defects greater than 2 years old.  Should defects be hoarded?  With the vanishing defect policy intervention would not be required.

Ok!  I will make a compromise. Instead of vanishing defects perhaps the defects should be archived after 120 days.  

I think a better model to consider would be Defect Regeneration.  This would be similar to a gecko growing a new tail.  All defects must be fixed within 120 days.   Some geckos regenerate a tail in 2 days to 2 months, so 120 days should be plenty of time.

That concept will probably not fly either, so we are destined to see the giant landfill of defects forever!

Lister's Law

I am not making any resolutions to blog more although I should, but I am ending 2012 reading a very enlightening book called Slack by Tom DeMarco.

I am probably a third of the way through the book.  On page 50 there is the mention of something referred to as Lister's Law.

"People under time pressure do not think faster." - Tim Lister

When I read that it had a familiar ring.  Perhaps Michael Bolton and James Bach have echoed similar phrases.  I know these two pioneers encourage us testers to think.  In fact there actions challenge us to think.  I never stopped to consider that thinking happens at a fixed rate.  I have never actively set aside time to think nor have I pondered the rate of thought.

I do not know about other testers, but I do not think my brain every stops.  I actually believe that is why I have trouble sleeping, but that is another story altogether.

Despite the wisdom in this book I feel testers are always going to face an extreme amount of pressure.  Under the burden of pressure and rapid decisions how can we afford the time to think.  I do not yet know how this concept of a fixed rate of thinking will influence my day to day actions.  But I do know that I need to strive for a balance and inject some slack into my day for deep and explicit thought.

I am a reactionary type of person and very free to offer opinions.  Many time my opinions land me in trouble especially when I do not take the time to think about the context of my points and the perspectives of the audience.

A colleague the other night joined me at a concert and we were talking about the open mouth insert foot phenomenon.  He stated he now considers three things:

• Should what I am thinking be said
• Should it be said now, and
• Should it be said by me

I really think he is spot on.  We especially need to give ourselves the time to think if we get to the third bullet in this thought process.

I am not sure about all testers, but I feel constantly under pressure.  In 2013, I am going to try and find ways to relieve some of that pressure.  I do not know how yet, but I assure you I will be giving it some more thought.

Slack is written with a business focus, but I believe the lessons expand to our society.  We simply put to much pressure on ourselves and those around us.  

Add some slack to your day to day operations and add Slack to your 2013 must read list.

Appreciations

Here it is Thanksgiving morning 2012.  Of course I woke up at 5:00 AM when I had an opportunity to sleep in.  I meandered through all of my RSS feeds.  And of course I stumbled upon this gem of a post by Christopher Avery.  By the way Mr. Avery is an awesome person to chat with.  I had the absolute pleasure of talking with him at the Rally On conference this year.

On Friday November 16 I attended Keep Austin Agile Conference put together by Agile Austin.  I learned quite a few things at this conference, but there was one thing I had not heard before regarding retrospectives.  Earl Everett in his talk on "Retrospective Tips, Tricks, and Traps" he mentioned the use of appreciations during retrospectives.

Honestly I think it is a great idea.  What I immediately struggled with is how many appreciations or the quality of the appreciations.  Also I had some thoughts on the emotions of appreciations.  I concluded that I do not show my appreciation enough, so I will incorporate appreciations more into my daily routine both at work and at home.

Christopher Avery added more controversy to my thought process by stating "Reflect on your ratio of potential vs. actual acknowledgment".  I completely agree that each day we should reflect and I need to add a list of appreciations to that reflection, but how many of your appreciations should truly be acknowledged.  I have been thinking all morning as to what ratio would be good for me.  I keep thinking my ratio of actual appreciations should be low and strategically delivered.  I am leaning toward attempting to do something like take action on my top two appreciations for each day regardless of how many potential appreciations I can come up with.  Is that enough appreciations?  Maybe not, but it is certainly a start.

Another thought I have is that in my list of potential appreciations there may be reoccurring recipients.  Perhaps one of my daily appreciations should be directed toward someone or something new on my list each day.  I kind of see this as a twist on "Paying it Forward", but it is absolutely practical to show daily appreciations.

I am a facilitator of meetings.  As a facilitator I easily fall into the trap of speaking to much.  People who know me, know I have an opinion on everything.  I am hoping by limiting my appreciations to two per day, appreciations will become contagious.  In a collaborative agile software world we could all use a bit more appreciation.

It is Thanksgiving after all, so I have certainly a great deal to be thankful for.  On Monday show your appreciations to your colleagues, especially a Tester.

Happy Turkey Day!

Time to think

I truly dislike the fact that I do not blog more.  I still do a lot of reading and I am participating what seems to be a great course on-line with Stanford University called "A Crash Course on Creativity".  I simply do not do enough writing.

This morning I read Michael Bolton's blog post.   As usual his post got me thinking on various topics but primarily on "Where does my time go"?

As Mr. Bolton describes I am going to go through the exercise of using graph paper to illustrate where my time disappears to, but more importantly I am going to think more clearly as to where my time should be spent as a Senior QA Engineering Manager.

I need to spend more time marketing the value of testing.

I need to spend more time marketing test innovations.

I need to spend more time on educating and mentoring the team.

I need to spend more time educating myself.

I need to spend more time learning to become a forward thinker or visionary.

Ideally I would like to spend more time testing, but apparently that is no longer in my job description as a Senior Manager.

So just in this very short blog post of thinking out loud.  I have come up with two key areas that I must find time for, Marketing and Education.

Hmmm!  Now I need to go spend time thinking more about why do I have to market testing at all and why I feel compelled to educate others or myself.

First I guess I need to steal some graph paper from our daughter and map out where my time goes.  Then I might be able to add a slot for additional blog posts.

Happy Testing!

Focus on the Customer

Hello Testers!  It has been way to long since I posted something on this blog.  I find it interesting that it took a vacation to inspire me to post again.

While planning the vacation I visited numerous websites.  As a tester, I was somewhat  alarmed how many defects I observed on these websites.  The defects did not really bother me too much because they are websites.  What really bothered me was how non-userfriendly many of these sites are.

When booking airfare it was unclear as to whether the prices were round trip or each way.  When booking trains it was not easy to determine was it a direct route or a slow route.  Key attributes such as family passes were not obvious nor routinely recommended.  You purchase your ticket for Virgin trains, but you never ride on a Virgin train.

The craziest thing I found wrong with almost every website is that it was not obvious how to contact customer support.  Many times through exhaustive searching of a website I could not find the answers to my questions.

So here is the one example that set me over the edge.  Our daughter has become a vegetarian, so she wanted me to contact United Airlines to make sure she got the vegetarian meal on the plane.  I thought this would be a simple radio button or checkbox on the user profile.  Hell, we had to add passport details and other personal information.  When I went in to adjust her profile I could not find anywhere to change her meal selection.  I finally ended up calling customer support.

When we were preparing to leave my daughter asked me to confirm that she had a vegetarian option for the trip home.  Again nothing on the website was obvious.  The only number available to call customer support was a 1-800 number.  No where on the site could I find out an efficient way to contact customer service from an international location.  They flew us to a foreign country, so they should be prepared to service, me, the CUSTOMER.  Well, calling from London on a cell phone, I incurred phone charges at an international rate.  I have not got the bill yet, but I can just imagine the cost of this inquiry.  As it turned out they set the vegetarian option for the entire round trip flight, but how was I to know.

I will not even get started on my experience with airport kiosks.

Bottom line for me as a tester is that I need to pay extremely close attention to what the customer should experience.  I need to focus on not only the functionality that is on the web page, but what functionality simply might be missing.

Be the USER!

Unicorns, Fairies, and Squatches

Over the past few weeks I have seen numerous references to use cases accompanied by questions. 

Why do your tests not cover every use case? 

Why would you not test that?

Why do you test like that?

My inner ear rings with the thoughts of “Perfect Software” and “Complete Coverage”.

It is pretty easy to address the concept of perfect software by just handing someone Gerald Weinberg’s great book.  Let me quickly simplify the great message Mr. Weinberg presents in this book:  “THERE IS NO SUCH THING AS PERFECT SOFTWARE!”  Please accept my apologies for the cyber shouting.

Matt Heusser and Peter Walen did a great job at STP Conference in Dallas debunking the concept of “Complete Testing”, but you had to be there.  For everyone not in attendance of this wonderful and interact presentation let me quickly share the message:  “THERE IS NO SUCH THING AS COMPLETE TESTING!”

While I pondered why so many people still believe in unicorns, fairies, and squatches, I came to another conclusion.  Because of the belief in Unicorns, Fairies, and Squatches, testers realistically only get to actually test software 25% – 40% of their working hours.  The remainder of the time is spent chasing unicorns, fairies, and squatches.  I think this may factor in well with respect to another great book which I am still reading, "How to Reduce the Cost of Software Testing."  I need to calculate how much it costs to chase unicorns, fairies, and squatches.

I am so glad that with good testing technique in that limited testing time we typically find most critical defects.  I think it is truly OK to let the smaller fairies go.  Capturing the Squatches is the most important task at hand!

Do you believe in unicorns, fairies, and squatches?

Happy Testing!