Sunday, September 21, 2014

ISO 29119 - What not to do!

This post is inspired by Michael Bolton's post here.

I am a software tester.  I am an advocate for rapid and creative testing.  Think and do not follow!

I am formally a chemist where I managed an Ambient Air Analysis laboratory.  Our laboratory had many other divisions analyzing water, soil, and other tests that had to comply to EPA protocols.  All of these protocols were based on a documented government standard.  The irony of analyzing environmental standards relative to protocols was that if you found a better way to test for something, you were WRONG!  You could get the governing body to draft an amendment to the protocol or actually convert it to a new standard, but it most likely would take years.

Because some of us do things differently in testing software are we WRONG?

One funny story is that to do environmental analysis you had to have "certified" reference standards.  I discovered on a audit/tour of a gas standards company that the standards they were selling were certified against an expired standard.  The further irony was that the some gas standard companies certify their newly generated standards against standards they themselves had prepared.  I created the certification standard and I sell "certified" standards to the public.  Sure seems like a wolf in the hen house.

One of the most frustrating things about working in the environmental laboratory were the government audits.  If you did not follow protocol to the letter you risk large fines and even loss of business.  Something as simple as failing to put your initials in a laboratory logbook or an expired training record could result in a fine.

I believe I was a good chemist solving real world environmental problems.  When the lab got bought and I was told to only run samples of this kind in compliance with this standard to maximize profit.  I changed careers!  I was no longer permitted to innovate and solve environmental challenges.

When I hear debates like the one on ISO 29119, all of those laboratory frustrations resurface.  Today I lead a fantastic team that test a family of web sites designed to help create fun vacations for families and groups of friends.  Does an ever changing website really need to comply with some standard?  I think not.  Do we want a quality product that delights our customers.  Absolutely!

Now there may be software that requires a high degree of rigor and I get that. Just because you follow some guidance does not mean the software complies with a standard.

In the business of analyzing gas samples the oracle was a certified reference standard.  What is the oracle for "Perfect Software"?

I also worked for a software company that was attempting to achieve a high level of CMMI certification. The work became sit in review meetings 8 hours a day, then find time to actually test the software.    This routine involved test plans in word documents, test matrices, change control on test cases, and so on.  I do not want to go back to that routine.

My vote is test software with a high degree of technical creativity, find the bugs, and never follow any mandated guidance.

Honestly if I get a copy of ISO 29119, I will probably read it as a reference of what not to do!

Happy Testing!

2 comments:

Barton Wilson said...

That's true. ISO 29119 helped rise a lot of debates regarding standardized testing. Of course, the credibility of a standardized test can be questionable, since achieving consensus or even agreeing to reach a common decision is not easy. In any case, it was great to hear of your opinion regarding the matter as a tester yourself. Thanks for sharing that, Carl! More power to you!

Barton Wilson @ International Standards Authority, Inc

Kent Gregory said...

Despite all the unfortunate commotion, it’s good to know that you’ll use the ISO 29119 as a basis on what to implement and avoid during tests. It will definitely help you in a lot of ways. Anyway, thanks for sharing this with us. Have a great day!


Kent Gregory @ ARMATURE Corporation