Friday, November 21, 2008

Kung Fu Testing

“Colloquially, kung fu (or gung fu) alludes to any individual accomplishment or cultivated skill obtained by long and hard work.”1 What in the world does Kung Fu have to do with Software Testing? Last week we had a software release due to our client and it was one of those weeks, if something would or could go wrong, it did. The database became corrupted, the testing lab network became ultra-slow, and connectivity to our client site was intermittent. With all of these challenges we managed to battle through and get the release out on schedule, although with a couple of qualifications. During this crazy fray the lead developer coined our style of testing as “Kung Fu Testing”.

Not giving the phrase much thought at first, I liked it. After more thought I was pondering why would she draw the analogy of our testing style to that of Kung Fu. Was it because we did a good job of dissecting and targeting the critical risk areas to test or perhaps the long hours and dedication toward meeting the delivery date? Was it because out of self defense we effectively adjusted our testing strategy or was it the precision of the weapons selected to accomplish the task at hand?

I conclude that it was a little bit of everything stated above.

Dissection - We carefully chose to execute test cases and test plans in a precise order of risk.

Dedication – We worked into the wee hours of the morning for several nights in a row resurrecting our test environment.

Self Defense - We completed testing of all vital functions. We then executed regression test suites based on efficiency.

Weapons – Our WATIR framework allowed for some minor adjustments to the methods and we could readily execute the system regression tests.

We executed almost 600 test cases in only three days. Typically we allocate 5 days, but several days were needed to resurrect the testing lab. The precision in which the test plans were executed gave us the confidence to deliver the release on schedule. One test plan could not be executed, but we explained the risk in the release notes. After the delivery of the code we were able to execute this test plan further mitigating any risk.

Having recently completed the AST course Black Box Software Testing Foundations, I hope to expand upon this concept of Kung Fu Testing to be oracle and heuristic focused. There is always room for improvement, but accomplishments happen because of cultivating testing skills and hard work.



1 Wikipedia

Wednesday, July 23, 2008

To act or Not to Act

Quick question for all test enthusiasts.

1. You are browsing the web.
2. You find a site, perhaps a major casino or an airline
3. The site is not user-friendly
4. Site has a major defect

Do you act an attempt to report the issue?

Or

No act and accept the flaw?

I am finding myself more and more documenting my experience and submitting it to the site. Which brings me to another point in that often it is difficult to find on the site the appropriate link to submit a defect.

Recently I sent a browser compatibility observation to a company and the CEO replied with a Thanks!

Comment and let me know what you would do.

Long live WATIR!

Friday, April 25, 2008

WATIR Fall

“Open-source tools will not apply to our traditional water fall approach to software development”

The most alarming aspect of this statement was that it was made as we were politely debating the use of QTP versus open-source testing tools. My first thought was, what does an old traditional development process have to do with tool selection?

It has been awhile since a single statement resonated with me over night.

In my view this statement implies that testing must wait until development is complete.

Approach to using QTP is record some GUI interactions and play them back. Next step might be to change values for some of the variables and repeat the test. Ideally the variables are pushed into a data table where the values can be read and a single test executed multiple times. If a tester or scripter is really curious they will look under the hood and find ways to improve the code generated by the record playback

In today’s world we are constantly hearing from management the key to success is time to market. I have seen numerous power points where these words appear. If testing waits until development is complete, then are we doing everything we can to get our products faster to the market?

Test automation should be considered throughout the process. The role of testing teams is to encourage testing from start to finish. Ideally test automation is being implemented in parallel to the development. Quality teams should be encouraging test driven development. As unit tests are being created, WATIR tests can be developed in advance of ever seeing the final code. Conversations take place with the developers and you can determine what an object on the UI is going to look like. Is it a radio button? If so ask the developer what tag name that radio button will have. Go write a test case in Ruby to test that radio button.

Really all you need to know that a requirement is going to be met on page X by implementing a radio button and you write:

assert(@ie.radio(:id, "requirement1").exists?)

@ie.radio(:id, "requirement1").set

Or

@ie.radio(:id, "requirement1").isSet


What else is on that page? There is a link. Is the link an image or something else? What is the name of the link? Where should the link take me? This can clearly be done in parallel with development.

@ie.link(:id, "loginButton").click

Client has a requirement that on this page there should be a certain warning message, so you write:

assert(@ie.contains_text('Danger Danger Will Robinson'))

These test cases took just a few seconds to write. Eventually these tests will need to be placed into a flow and executed, but the foundations can quickly be made from the requirements. Will this help get the software faster to the market? I certainly think so!

Large corporations need to shift the paradigm of long process water fall mentality to Smarter Faster Cheaper, Agile. Companies need to make their processed more agile and meet the demands of their clients by delivery better software fast.

Are there more complex tests than the few illustrated above? Absolutely!

This is where a team of testers collaborating can write extremely robust multi-purpose sets in Ruby. A quick example would be team A already has written some code to validate the content of an XML response. Team B needs to do the same thing, but for a different application. So the next step is to borrow the code and implement.

To be continued…

Thursday, March 06, 2008

Have you completed your Unit Tests?

Perhaps this is isolated to my world, but more often than not the perception of Software Quality Assurance has come to the point of "Did you complete the unit testing?" Large corporations take this view when it comes to software quality. Unit tests are wonderful and necessary. I have participated on one project where we had to rely on JUnit for test automation for our regression testing. But unit tests are not the only game.

Unit tests typically are written by the same developer who wrote the code. Perhaps corporations today expect that the software company conduct unit testing, then delivers the code to their contracted testing team.

Faster and cheaper certainly does not translate to a quality software product. There are many types of testing that should be conducted by experienced testing teams before deliverying code to the clients. Exploratory testing cannot be executed by a JUnit test. Integration testing or performance testing and GUI testing cannot be easily implemented by JUnit.

When companies now ask "Did you complete your unit testing?", my answer is always yes we have tested our software.