Top Ten STP Conference Experiences

I plan to write up individual posts on some of the sessions attended at the STP Conference.  In the mean time I thought I would rank the sessions and experiences in order of value to me. 

1. Session 103 - Testers! Get out of the Quality Assurance business - Michael Bolton
2. Keynote - G Forces in the Organization - Kent Beck
3. Keynote - Nice Bike: Fueling Performance with Passion - Mark Scharenbroich
4. Breakfast bytes - Agile Testing Ninjas - Lanette Creamer
5. Breakfast bytes - Pirates are better than Ninjas Even in the Agile World - Adam Goucher
6. Workshop - Hands on Performance Testing Dan Downing and Goranka Bjedov
7. Session 403 - Ever Been fooled by Performance Testing Results? - Mieke Gevers
8. Session 305 - From Start to Success with Web Automation - Adam Goucher
9. Keynote - Top Ten Leadership Skills: Surviving and thriving in 21st Century - Kelli Vria
10. Party on the Strip - Christian Audigier at Treasure Island (met many great Testers!)

If you attended, what were your top ten?

Better Late than never

I finally realized that there are many great blogs that I follow, so I had better add a blog list to my tiny space of free thought on testing.

What really inspired me to do this was this post:

http://cartoontester.blogspot.com/2010/10/monotonous-context-and-certification.html

The cartoon absolutely illustrates that testers can get caught in a rut.  Collectively we need to continuously remind ourselves to get out of the rut. 

Think, experiment, and do!

I wish I had the creative mind to do great cartoons like Andy Glover.

AWK and GREP are your friends!

I definitely plan to write up some lessons learned from the STP Conference, but I thought I would share this quick tidbit for performance testers.

Our website profile is constantly changing so we inspect our log files to determine the correct mix for our performance testing.  There are many ways to do this and you can get as complex as you want.  Typically we choose our peak traffic day.

You have an access log and you want to know ratio of the number of GETS to the number of POSTS.

This command should give you the total number of requests.

cat | awk '{ print $7 }' | wc

This command should give you the total number of GETS.

 cat |grep \"GET | awk '{ print $7 }' | wc

This command should give you the number of POSTS.

 cat |grep \"POST | awk '{ print $7 }' | wc 

You can substitute "less" for the "wc" and you will see on your screen all of the GET requests.

You can use multiple grep segments to further refine your data. 

cat |grep \"GET | grep " 200 " | awk '{ print $7 }' | wc 

So my specific test I am only interesting in GETS with response code of "200".  I could write these results to a file.

cat |grep \"GET | grep " 200 " | awk '{ print $7 }' | wc >> foo.txt 

Yes some of you already know this stuff, but my main point is that you can learn a ton about your site traffic simply by dissecting the access logs.  You can generate a flat file that can be used by your performance tool of choice to randomly send requests to your system.  You can write a simple shell script that can take any access log and generate a quick breakdown on the request profile.  This profile can be used to proportion your performance test traffic in a similar pattern as your current user base.


Go ahead and karate chop your access logs!

 

Short Post on STP Conference

It is pouring down rain in Las Vegas.  Yes I said pouring down rain.  The rain and long taxi line is preventing us from going back to our condo.  So we are hanging out waiting to go to the big party at Treasure Island.  We will be true geeks and take our computer backpacks to the party.

So here are my short thoughts.

• Great people
• Extremely engaging
• Some material unfortunately is not applicable to the things we are currently focused on
• Some speakers I could listen to all day long
• I am eating way to much food
• I am doing tons of walking, but not off setting the amount of food and mmmm alcohol
• I am having a great time
• And the organizers of the event are doing a FABULOUS job!
• I hope I can be a presenter in the near future.
• I hope I can continue to collaborate with some of these smart people.
• And final note - I wish I had a large volume of cash

Hopefully I will find the time to post future blogs on all of the things that are currently floating around in my two brain cells.

Have fun!

STP Conference are you ready for me?

On Sunday I depart with 2 colleagues for Las Vegas to attend the STP Conference.  I am pumped up.  I am ready to learn what is happening in the wonderful world of testing.

On Monday I am attending an all day hands workshop about Performance Testing.  I do know how to use JMeter, but I am hoping to learn more tricks and gain insight as to how others approach performance testing.  Perhaps I have a few tricks up my sleeve to share!

Here are some of the other sessions I am hoping to actively participate in.

1. Testers!  Get Out of the Quality Assurance Business by Michael Bolton
2. Becoming an Automation Entrepreneur by Linda Hayes
3. From Start to Success with Web Automation by Adam Goucher
4. Ever Been Fooled by Performance Test Results? by Mieke Gevers
5. Testing in an Agile Environment by Rob Walsh
6. Test Faster: Model Your Test Process to Test Faster by John Ruberto
7. Using Open Source Testing at Ford by Frank Cohen
8. What You Need to Know About Performance Testing by Michael Czeiszperger

I may switch gears and attend other sessions, depending on the value.  There are so many excellent topics to choose from.  I am hoping to meet and network with as many testing gurus as possible.

There are a couple of key note speakers I am curious about Kelli Vrla and Kent Beck.  I am sure they will all be interesting.   The real key is to keep me away from poker tables.  :o)

Stay tuned for future posts on various topics spinning in my two brain cells after the conference.

Happy Testing!

Page Generator Example

Last time I posted I promised some code.  I finally got around to compiling the code into a format that could be placed on this blog.  I borrowed some of these methods from our framework library, but I think this code illustrates how efficient a generator can be for formatting page locators.  Page object pattern is a valuable approach to an automation framework.

Note that this code calls a test helper file.  I have not included that here, but basically it is a file that specifies the gems that are necessary.

require 'rubygems'
gem "rspec", "=1.1.12"
require "spec"
gem "selenium-client", ">=1.2.18"
require "selenium/client"

In the code below there are two generators.  One can be used to get all of the links on the page and the other is used for other html elements.

Once these generators are executed the locators can be placed in a page file.  At the bottom of the output the labels are formatted into the correct accessor format for a page object file.

Below is a ton of code so enjoy!  Let me know if you have any questions.

One final comment in that not of all the links on Google's home page get formatted nicely.  I did not have time to fix the formats.

Hopefully on the next post I will provide the page file and an example of using the page file.

Enjoy!


require File.dirname(__FILE__) + "/../../spec/test_helper"

describe "Generate home page locators for google" do

  append_after(:each) do
    browser.close_current_browser_session
  end

  it "should generate locators for check boxes, text boxes, text areas, input images,and select boxes for the google home page" do

    my_setup("http://www.google.com/")

    browser.open ("/")

    get_source_and_print_elements(browser)
  end

  it "should generate and format the links from the google home page" do

    my_setup("http://www.google.com/")

    browser.open ("/")

    html = browser.get_html_source

    generate_link_instance_variables_from_html(html)
  end


  #  Below are the methods that help to extract out the components from the html and format them into locators and accessors

  def my_setup(target)

    @browser = Selenium::Client::Driver.new \
    :host => ENV['Host'] || "localhost",
    :port => 4444,
    :browser => ENV['SELENIUM_RC_BROWSER'] || "*firefox",
    :url => target,
    :timeout_in_second => 60

    browser.start_new_browser_session

  end

  def browser
    return @browser
  end

  def get_source_and_print_elements(browser)
    html =browser.get_html_source
    html_elements_check_boxes = GooglePage.generate_instance_variables_from_html(:html =>html, :locator_type => "css", :locator => "input[type='checkbox']")
    html_elements_text = GooglePage.generate_instance_variables_from_html(:html =>html, :locator_type => "css", :locator => "input[type='text']")
    html_elements_selectGooglePage.generate_instance_variables_from_html(:html =>html, :locator_type => "css", :locator => "select")
    html_elements_text_area=GooglePage.generate_instance_variables_from_html(:html =>html, :locator_type => "css", :locator => "textarea")
    html_elements_image = GoogleBasePage.generate_instance_variables_from_html(:html =>html, :locator_type => "css", :locator => "input[type='image']")
    html_elements_radio = GooglePage.generate_instance_variables_from_html(:html =>html, :locator_type => "css", :locator => "input[type='radio']")
    html_elements_form = GooglePage.generate_instance_variables_from_html(:html =>html, :locator_type => "css", :locator => "form")
    merge_and_print_elements([html_elements_check_boxes, html_elements_select, html_elements_text,
                              html_elements_text_area, html_elements_image, html_elements_radio, html_elements_form])
  end


  def generate_instance_variables_from_html(options)
    if options.kind_of?(Hash)
      @html = options[:html]
      @locator_type = options[:locator_type]
      @locator = options[:locator]
    end
    doc = Nokogiri::HTML(@html)
    html_elements = {}
    if (@locator_type=="css")
      doc.css(@locator).each do |html_element|
        attribute_name =  html_element.get_attribute("id")
        attribute_value = html_element.get_attribute("id")
        if !attribute_name.nil?
          attribute_name.gsub!('input-', '')
          attribute_name.gsub!('select-', '')
#          attribute_name.gsub!(/([A-Z]+)/, '_\1')
          attribute_name.gsub!('\\', '')
          attribute_name.gsub!(' ', '_')
          attribute_name.gsub!('.', '_')
          attribute_name.gsub!('-', '_')
          attribute_name.gsub!('__', '_')
          attribute_name = attribute_name.to_s.downcase
          puts "@#{attribute_name} = \"#{attribute_value}\"" if $debug
          html_elements[attribute_name]= attribute_value
        end
      end
    end
    return html_elements
  end

  def merge_and_print_elements(page_elements_types)
    html_elements={}
    page_elements_types.each do |element_type|
      html_elements.merge!(element_type)
    end

    puts "found (#{html_elements.length} elements)"

    html_elements.keys.sort.each do |key|
      puts "@#{key} = \"#{html_elements[key]}\""
    end
    html_elements.keys.sort.each do |key|
      print ":#{key}, "
    end
  end

  def generate_link_instance_variables_from_html(html)

    doc = Nokogiri::HTML(html)
    links = {}
    doc.css("a").each do |link|
    #  puts link
      link.text

      links =  format_link_text_from_label(link, links)
    end

    links.each_pair { |key, value| puts "@#{key} =  \"css=a:contains(\\\"#{value}\\\")\"" }
    links.each_key { |key| print ":#{key}, " }

    return links
  end

  def format_link_text(link, links)
    link_content=link.content.to_s.downcase
    link_content = link_content.strip
    link_content.gsub!(' ', '_')
    link_content.gsub!('"', '')
    link_content.gsub!('-', '_')
    link_content.gsub!('+', '_')
    link_content.gsub!('*', '')
    link_content.gsub!('\'', '')
    link_content.chomp
    #puts "#{link_content} --->     #{link.get_attribute("href")} "
    links[link_content] = link.get_attribute("href")
    return links
  end

  def format_link_text_from_label(link, links)
    link_text=link.text.strip
    link_text=link_text.chomp
    link_content=link.content.to_s.downcase
    link_content = link_content.strip
    link_content.gsub!(' ', '_')
    link_content.gsub!('"', '')
    link_content.gsub!('-', '_')
    link_content.gsub!('+', '_')
    link_content.gsub!('*', '')
    link_content.gsub!('\'', '')
    link_content.gsub!('_&_', '_')
    link_content.gsub!('(', '')
    link_content.gsub!(')', '')
    link_content.gsub!(':', '')
    link_content.gsub!('.', '_')
    link_content.gsub!('@', '_')
    link_content.chomp
    links[link_content] = link_text
    return links
  end

end

Pages, Generators, & Validators

We have been working on a framework that leverages RSpec, Selenium GRID, and Team City.  As we wrote code for various projects we settled on the Page Object Pattern.  I am not going to provide code for this post but I would like to share a few definitions.

Pages -  A ruby class that contains the locators and custom methods relative to a single web page.  The class is named similar to the page title or the common term used by the team to identify the page.  The class is initialized with locators which are either XPath or CSS that map to the objects on a page.  The locators are set as attribute accessors.  Also included in this class are any custom methods that add efficiencies to navigating through the page objects.

Generators - A method or a series of methods that can grab and format the objects from a page.  One approach is to simply copy the page source and save it as an html file, which can subsequently be parsed using Nokogiri.  This approach may be necessary if you cannot readily navigate to a page due to SSL or some other restrictions.  The second approach is to navigate directly to the target page and get the source html.  Once you store the source html parse the html using Nokogiri.  The simplest form of a generator locates all of the ids on a page and formats the ids into a standard ruby format.  Here is one example for a link.

@location_link = "css=li.'location-link' a"

A generator can generate the locators names and associated css styles for links, radio buttons, buttons, check boxes, and select boxes.  All of the magic happens using Nokogiri to parse the HTML and format the locators.

Validators - This is a RSpec file that confirms the accuracy of the locators that are made by the generators.  A good validator can navigate to a page and execute browser.element? on all locators in the page class.  Any missing locators will be identified.  Validators are very helpful as we do test driven development, but they are extremely important if you have a rapidly changing UI.  You can rebuild the page files as frequently as needed.  Your automated tests key off these locators so keeping the locators current keep you tests current.

This has been a very effective pattern for our framework.  There are two important elements.  The first is a well constructed UI.  The second is Nokogiri.  Hopefully in my next post I will provide some real and usable code.

Happy Testing!

Did the test really pass?

I thought I would share a quick experience since I had not posted in a couple of weeks.  Last Friday I had my head phones on and fingers flying on the key board.  I have been really pumped to write code because my coworker has shifted my paradigm from hacking to test driven development.  I am still learning this approach but I feel it is the right approach and more importantly FUN!

So I created some factories that managed the data.  I created some tests against the factories to validate that the tests would get the correct data from the factories.  I then wrote a data driven script.  The script iterates through numerous key words and performs a comparative test between the UI and the back-end XML generation.

When I was done every test passed with flying colors.  I was stoked.  The next morning it occurred to me that perhaps I should test a negative condition and force the test to fail.  I fed the script all kinds of data that should have resulted in a failure.  The test passed every single time.  What the heck is going on?

After a long pause I recognized that the URLs and http requests I was building were flawed.  It was time to consult the client.  After a couple of quick questions the requests were corrected and the test script worked as expected.

So the lesson I share with you is always and I mean always do a karate chop on your test in order to make sure it fails when it is expected to fail.

Continue to chop the code!

Simple But Effective

Thought I would toss out a quick post about a few simple lines of code that came to save the day.

Have no fear If Else is here to save the day!

We do product line development where we have twelve unique products whose foundation is a common code base, but each product has variant points and are updated incrementally.

Recently we deployed 4 of the 12 products and numerous automation scripts broke.  With a little investigation there was a difference in a single page flow.

To the rescue was:

if browser.element? "some_css_locator"
    browser.click "some_css_locator", :wait_for => :page
else
end

All automation tests are now passing!

The really cool part is this code will work even if the new page flow is deployed to the other 8 products.

Long live css locators and the If Else!

Do your Homework!

Good morning!  This is a quick post to my blog rather than making a tacky post on a users forum.  I believe we all gain very valuable information and assistance from user forums.  I pay close attention to a couple of forums and groups, but here is a snippet of a post on the JMeter forum that gets my feathers ruffled.

"I wnat to perform load,performance and capacity testing for a web site. can
you guide me how to do that."

Do you see anything wrong with this post?

OK, there are typing errors, but what I see is an individual who has not even attempted to learn about the topic.  These individuals need to install the tools.  Follow the examples learn what they can then post intelligent questions.

So far the forum has explained to this individual the differences between load and performance testing and provided this individual with links to all the starting material.  Now with this post they expect the forum to do the work for them.

Please people put in a little bit of effort!

Code Etiquette

Valuable lessons learned this week in the wonderful world of test coding.

Here are some of my hopefully old habits!

1.  Write code
2.  Re-factor and compress repeat code into useful methods
3.  Comment out old code to validate new methods

Sounds reasonable, right?

Then I do the following:

1.  Fail to check in the code
2.  Get in a hurry to check in the code
3.  Leave commented out code when checking the code in
4.  Fail to execute the tests as a unit prior to checking in code
5.  Fail to do an update to the code base prior to developing new code

A couple of major flaws in my actions or non-actions are that I check in new code that is already outdated because I failed to do an update.  I check in ugly crusty code full of meaningless commented out lines.  My team has absolutely no clue as to where I left off or what demons I was battling.

So I need to start a new set of developer habits as a test developer.

1.  Start each effort updating the target project
2.  Execute a "Diff" on all files that have differences
3.  Closely examine the differences and make smart choices on which file is more current or when to merge
4.  Continue to write code, re-factor and comment out old lines of code, BUT clean all of this up prior to checking in the code
5.  If I have demons I am battling during development (examples - Java Script timing issues, or page builders) I plan to clearly annotate these areas with comments and TO DO lists so team members can at least have a sense as to where I left off
6.  Seek help if I am stuck for to long a small area of code
7.  Test the code, Test the code, Test the code -  What this means is test it locally in isolation, test the suite of code locally, and test the code on our production system
8.  Checking in any code at the end of each day regardless if I am complete or not.  The code should include clear concise documentation if there are non-functional areas and not crusty commented out lines of code that server absolutely no purpose.

I definitely have to find a way to have better developer habits.  The last thing we need to do is cause unnecessary tension within a great team.  I am not a developer by nature so I really have to work at it.  But I do promise to always strive for continuous improvement.

If anyone has other suggestions for skills improvement please leave a comment!

Happy coding!

Time to Refactor

It is a rainy day in Austin Texas and I decided it was high time to re-factored my blog.  My  test automation adventure started off learning OTC tools Silk Performer, Silk Test, Webload, and QTP.  Eventually I recognized the advantages of open-source testing tools and the communities that support them.  My first adventure was using WATIR.  I loved using WATIR and the community. Unfortunately when I changed companies the ramp time was quick and WATIR at the time did not have all of the weapons we needed to accomplish our test automation task.

I have the absolute pleasure of working with a very sharp developer, Scott Sims.  In part I owe the knew title for my blog to him.  One day he had described a scene to me where his daughter "went all Kung Fu panda" on his wife.  I also get a kick out of when he hits some keys on his keyboard and "Hiyaaa" a zillion application windows open for battle.  In my blog archives I also had a previous post regarding Kung Fu testing when I worked with Cari Spruiell on a project.  We had to be nimble and fast on our feet to get the high quality project out the door.

All in all I think it is an appropriate title for today's testing world.  If you do not have an arsenal of tools and tricks you will not keep pace with the rapid development.

Over the past six months I have collaborated with Scott and we have ended up with a framework that I will describe like this "powered by Ruby, RSpec, Selenium GRID and supercharged with testing by brand, finely formatted results stored in Rally, page driven test execution via Team City.  I will add posts describing our choices and the adventure as we advance this framework.

Over the past few months we had the challenge of preparing our site for the traffic generated by a Superbowl ad.  We succeeded, but would not have done it with out the help of JMeter and Browser Mob.  Along the way I will add posts on performance testing.  I hope you will stop by this blog occasionally and share your thoughts.

Stay tuned!  Lets see if I can actually post weekly.

Rapid TPS Results using JMeter

Performance testing can certainly be an art. Or a journey into a deep dark cavern. But sometimes there are very simple ways to get a result. Here is the simplest way ever to determine transactions per second, TPS, if you use JMeter and save your results to a .csv file.

On the command line navigate to the directory where your file exists. Sorry but this magic assumes a unix like operating system.

The magic -

type wc -l {filename}
divide this result by the duration of your test in seconds

Potential magic using Windows OS (Untested by me, but found via a Google search)

dir /b c:\temp | find /c /v "~~~"

I like this approach because it allows you to quickly evaluate changes in your results. As you are optimizing an application configuration for performance you can very quickly approximate the throughput as TPS.

Stay tuned for two future posts:

How to quickly move results to MySQL
How to use Rake tasks to manage your results

FYI - I stole this magic from the magician Scott Sims.

Happy Testing!

Rubymine Rocks

Of course I missed another post last week. Amazing how soccer, work, life in that order controls your time. Any rate I was not sure what to post this week. I have been extremely busy learning more about JMeter, BrowserMon and performance testing and creating new automated regression tests.

I decided to pass on writing about performance testing and make a couple statements as to why Rubymine ROCKS.

Number one reason Rubymine ROCKS is that when you are working with Ruby and XPath you can pause the script at the point of failure. Then you highlight the XPath code that is failing and hit F11. Now edit the day lights out of that XPath until you get the XPath to work.

Typically I have a copy of the target page up in another Firefox instance and viewing the HTML closely.

A second reason I like Rubymine is the ability to quickly see what methods are available and command-B to see what the methods are doing (is the correct term intellisense?).

Probably not the last reason I find, but when you want to refactor a term you can find it and replace it for the entire project. Most IDE do this, but I like how Rubymine does it.

I also like how Rubymine tells me gems are not attached. I like the fact that you can can place code in test mode and test one method at a time. And I also really like the fact that they fixed the rename functionality so the name actually appears and I can make minor changes versus having to type in an entire new name.

I have definitely become a fan of Rubymine. Go Horns! I mean RubyMine!

Rocking the Ruby code!

Selenium Brittleness

This will be a quick post, but I wanted to share a little tip. When using the Selenium IDE you often get xpath that looks similar to this.

browser.click "//div[@id='location-bar']/div[2]/div[4]/a"

Well when attempting to do cross-browser testing this Xpath may not work for all browsers and relying on an index value that may change is a bit brittle.

One way to adjust this Xpath would be to make it look like this.

browser.click "//div[@id='location-bar']/div[@class='this']/div[@class='that']/a"

Of course everything is dependent on how the object on the page is represented. Inspect the HTML closely.

Here are another helpful hints on clicking links.

browser.click "//a[contains(@href,'stuff')]"
or
browser.click "//a[contains(text(),'otherstuff')]"

I hope you find these pointers helpful.

Happy coding!

Using Selenium GRID

For the past 3 months I have been on a journey with a team of extremely sharp developers and test engineers to build out a test automation framework. We settled on using Selenium and the Selenium GRID.

There are so many framework options, but I have to say at this point Selenium GRID rocks.

Here are the main pieces to the puzzle.

• MacBook Pro
• Use Rubymine for development
• Now using Selenium GRID 1.04
• Code base is Ruby leveraging the use of Rake
• VMware Fusion
• Team City
• Rally

Locally we can now develop a script and execute it against Safari on MacOSX, Firefox 3.0, Firefox 3.5, IE 6.0, IE 7.0 & IE 8.0. We can execute tests in parallel. I love watching 6 - 10 firefox icons bouncing up and down on my MacBook.

Tests are launched by the test teams in production environment using a job configured in Team City. One click of a button kicks of the test suite. Soon teams can choose to run only smoke test, only regression tests, or all. In production the tests are executed in parallel for each browser configured, which us currently IE 7.0, IE 8.0, Safari on Mac Mini, Firefox 3.0 and Firefox 3.5. After the test execution the test leads can see results input directly into Rally test cases or they can view a nice HTML report. They can also view the results in Team City.

If you cannot tell, I love this stuff! It is really fun to develop in this environment because you can see immediate cross browser results.

Yes I still struggle as a developer and create some brittle tests. But the good news is collaborating with a great team we quickly re-factor brittle code.

Our automation targets a complex and ever changing environment, but having a dedicated automation team is the key to success.

Happy Testing!

JMeter to the Rescue

Recently I was involved in a high priority performance testing effort. The tools at hand were Webload and WAPT. I have previous experience with Webload so that is the initial path we took. The testing effort is relatively simple. The goal was to have concurrent users doing a series of gets and posts. Clarification the posts are actually http puts. I quickly could not find a way to get Webload to emulate an http put. Further complicating things was the system under test, SUT, requires secure header keys. Information on the internet on Webload seemed very limited and I was not able to rapidly find the answers to these two concerns.

The time line was limited so there was not bandwidth for a coding effort nor to purchase any new tool.

We quickly turned to JMeter. This is going to be a short post, but there are a couple of key advantages we had with JMeter.

Observed advantages:

• JMeter out of the box could handle puts
• JMeter worked well with RAKE
• Fast installation and short learning curve
• Easily extensible
• Ability to run headless (no GUI) via RAKE

Observed dis-advantantages:

• Executing tests with the GUI dropped threads as load increased due to utilization of machine java processes
• graphic listeners are of extremely poor quality
• A single instance appears to be limited to approximately 300 concurrent threads
  

The bottom line to this post is that JMeter came to the rescue and has sound potential for future performance projects.

There are some key action items for me.

• Evaluate the ability to distribute load across several machines
• Evaluate the ability of JMeter to interact with Web-based applications (ajax stuff etc...)
• Develop Ruby scripts to calculate and report out data metrics as the number of transactions increase.
• Understand the limitations of this tool
• Evaluate other open source plugins developed for JMeter

I am looking forward to working more closely with the development teams to expand the use of jMeter for performance testing.

Happy Testing!

Selenium RC plus RSpec

It has been an extremely long time since I posted to this humble blog. I have currently changed jobs and I am newly energized to learn, implement, and build out a test automation framework.

There are a zillion frameworks to choose from. I have been using the WATIR framework for several years as I had bandwidth. I never had the opportunity to learn and execute a full solution. I recently interviewed several teams at the new company and I found one common theme with respect to test automation. Every team has selected Selenium RC as the foundation.

Initially I attempted WATIR to execute the tasks required by the team. With the complexity of the web pages and solutions, I quickly learned that WATIR may not be the best solution.

One of the teams built a Selenium & RSpec framework. Although there is some learning curve for me, I am liking what I see.

As I go on this adventure I will attempt to do a weekly post on the good, the bad, and the ugly.

Keep on Testing!

Kung Fu Testing

“Colloquially, kung fu (or gung fu) alludes to any individual accomplishment or cultivated skill obtained by long and hard work.”1 What in the world does Kung Fu have to do with Software Testing? Last week we had a software release due to our client and it was one of those weeks, if something would or could go wrong, it did. The database became corrupted, the testing lab network became ultra-slow, and connectivity to our client site was intermittent. With all of these challenges we managed to battle through and get the release out on schedule, although with a couple of qualifications. During this crazy fray the lead developer coined our style of testing as “Kung Fu Testing”.

Not giving the phrase much thought at first, I liked it. After more thought I was pondering why would she draw the analogy of our testing style to that of Kung Fu. Was it because we did a good job of dissecting and targeting the critical risk areas to test or perhaps the long hours and dedication toward meeting the delivery date? Was it because out of self defense we effectively adjusted our testing strategy or was it the precision of the weapons selected to accomplish the task at hand?

I conclude that it was a little bit of everything stated above.

Dissection - We carefully chose to execute test cases and test plans in a precise order of risk.

Dedication – We worked into the wee hours of the morning for several nights in a row resurrecting our test environment.

Self Defense - We completed testing of all vital functions. We then executed regression test suites based on efficiency.

Weapons – Our WATIR framework allowed for some minor adjustments to the methods and we could readily execute the system regression tests.

We executed almost 600 test cases in only three days. Typically we allocate 5 days, but several days were needed to resurrect the testing lab. The precision in which the test plans were executed gave us the confidence to deliver the release on schedule. One test plan could not be executed, but we explained the risk in the release notes. After the delivery of the code we were able to execute this test plan further mitigating any risk.

Having recently completed the AST course Black Box Software Testing Foundations, I hope to expand upon this concept of Kung Fu Testing to be oracle and heuristic focused. There is always room for improvement, but accomplishments happen because of cultivating testing skills and hard work.



1 Wikipedia

To act or Not to Act

Quick question for all test enthusiasts.

1. You are browsing the web.
2. You find a site, perhaps a major casino or an airline
3. The site is not user-friendly
4. Site has a major defect

Do you act an attempt to report the issue?

Or

No act and accept the flaw?

I am finding myself more and more documenting my experience and submitting it to the site. Which brings me to another point in that often it is difficult to find on the site the appropriate link to submit a defect.

Recently I sent a browser compatibility observation to a company and the CEO replied with a Thanks!

Comment and let me know what you would do.

Long live WATIR!