Tuesday, September 04, 2007

Fantastic Community

Hello WATIR World! It has been a long time since I posted to this blog. I have once again been afforded the opportunity to play with WATIR code and improve my development skills. I recently was stuck on refactoring the previous code, so I posted to the WATIR forum.


The community is fantastic and quickly responded with several approaches. I will be honest in that I did not understand some of them, but somehow the responses inspired me to get the code working. Brett asked that I share some snippets of code, so I thought I would post them here.


Before I do that I also need to mention my mistake. When I posted to the forum I really did not include enough information. I was a bit concerned about the proprietary nature of the code, so I only posted a tiny portion of our code. I should have reworked the code and posted the entire script. Please when you post include as much information as possible so these brilliant people can respond appropriately.

Warning there is a lot of code.

Here is an example of what the original code looked like:


# Script is designed to execute Smoke test test cases
#
#
#Initials Date Summary
#CLS 10/04/05 Checked in first draft
#CLS 12/02/05 Clean up/Site Selection code
#List of to do's or ideas for improvement
#

# BEGIN -
#includes:
require 'watir' # the watir controller
require "rexml/document"
require "date"
require '/apps/ruby/lib/ruby/1.8/test/unit'
require '/apps/ruby/lib/ruby/1.8/test/unit/ui/console/testrunner'
require 'watir/testUnitAddons'
require 'watir/WindowHelper'
include REXML
include Watir
puts "## Begin Test - DN Advanced flow"
class SmokeTest < Test::Unit::TestCase
include Watir
def setup
@ie = IE.new
@ie.goto($URL)
end
def startClicker( button , waitTime = 3)
w = WinClicker.new
longName = @ie.dir.gsub("/" , "\\" )
shortName = w.getShortFileName(longName)
c = "start rubyw #{shortName }\\watir\\clickJSDialog.rb #{button } #{ waitTime} "
puts "Starting #{c}"
w.winsystem(c)
w=nil
#assert_same(expected, actual, [message] ) #Expects expected.equal?(actual)
end
def test1
# User enters login information and test validates assertions
assert(@ie.text_field(:id, "Person.username").exists?)
assert(@ie.text_field(:id, "Person.password").exists?)
assert(@ie.link(:id, "loginButton").exists?)
@ie.text_field(:name, "Person.username").set($Username)
@ie.text_field(:name, "Person.password").set($Password)
# User submits login information
@ie.link(:id, "loginButton").click
# User chooses application flow
@ie.image(:src , /ps_header_module_solution_builder_iconUp.gif/).click
# User selects an Account
@ie.selectBox( :name , "NS_Account").select("Basic Training")
# User Sets specific solution name
@ie.text_field(:name, "NS_Title").set("Test1")
# User selects Long Distance Voice check box
@ie.checkBox(:id, "DN").set
assert(@ie.checkbox(:id, "DN").isSet?)
# User selects LD advanced mode
@ie.radio(:id, "advDN_mode").set
assert(@ie.radio(:id, "advDN_mode").isSet?)
# User advances to Solution Status Page
@ie.link(:id, "next").click
# User moves to Site Selection Page
@ie.link(:id, "next").click
# User clears the selection of all sites
@ie.image(:id, "selectAllButton").click
# Selects specific sites
assert( @ie.cell(:id, '001-TX-AUS').exists? )
@ie.cell(:id, '001-TX-AUS').checkbox(:index,1).set
@ie.cell(:id, '002-TX-DAL').checkbox(:index,1).set
# User moves to Sol. Status Page
@ie.link(:id, "next").click
# User deselects technologies
@ie.checkBox(:id, "Frame RelayTECHNOLOGIES").click
# @ie.checkBox(:id, "IVPN CPETECHNOLOGIES").click
@ie.radio(:id, "2_answer").set
# User moves to AME Page
@ie.link(:id, "next").click
# User adds a Local Access
@ie.link(:text, "ADD NEW...").click
@ie.link(:text, "LOCAL ACCESS").click
ie2 = IE.attach(:title, "Local Access")
ie2.selectBox( :name , "loopType").select("Access Link")
ie2.selectBox( :name , "loopSitesAvailable").select("001-TX-AUS")
ie2.selectBox( :name , "loopSitesAvailable").select("002-TX-DAL")
ie2.image(:src , /ps_transferto_button.gif/).click
ie2.image(:src , /ps_save_button.gif/).click
assert(@ie.contains_text('Technology: Access Link - 1.5 Mbps '))
assert( @ie.image(:src , /ps_icon_local_loop.gif/).exists? )
# User adds a port
@ie.link(:text, "ADD NEW...").click
@ie.link(:text, "PORT").click
ie2 = IE.attach(:title, "Port")
ie2.selectBox( :name , "portLoop").select("local access 0")
ie2.selectBox( :name , "portType").select("myport")
ie2.selectBox( :name , "portSitesAvailable").select("001-TX-AUS")
ie2.selectBox( :name , "portSitesAvailable").select("002-TX-DAL")
ie2.image(:src , /ps_transferto_button.gif/).click
ie2.image(:src , /ps_save_button.gif/).click
assert(@ie.contains_text('myPort '))
assert( @ie.image(:src , /ps_icon_local_loop_with_port.gif/).exists? )
# User adds a connection
@ie.link(:text, "ADD NEW...").click
@ie.link(:text, "CONNECTION").click
ie2 = IE.attach(:title, "Connection")
sleep 2
ie2.selectBox( :name , "connPort1").select("port 0")
ie2.selectBox( :name , "connSitesAvailable1").select("001-TX-AUS")
ie2.image(:src , /ps_transferto_button/).click
ie2.selectBox( :name , "connPort2").select("port 0")
ie2.selectBox( :name , "connSitesAvailable2").select("002-TX-DAL")
ie2.link(:url, /connSitesAvailable2/).click
ie2.selectBox( :name , "Bandwidth").select("256.0 kbps")
ie2.selectBox( :name , "connType").select("Standard 256kbps")
ie2.image(:src , /ps_save_button.gif/).click
assert(@ie.contains_text('Standard 256kbps'))
assert( @ie.image(:src , /ps_icon_local_loop_with_port_and_connection.gif/).exists? )
# User moves to Solution Status Page
@ie.link(:id, "next").click
# User moves to Recommedations Page
@ie.link(:id, "next").click
# User moves to Compare Services Page
@ie.link(:id, "next").click
# User moves to General options Page
@ie.link(:id, "next").click
# User moves to Per Site Options Page
@ie.link(:id, "next").click
# User moves to Proposed Solution Page
@ie.link(:id, "next").click
# User saves the solution
@ie.link(:title, "Suspend and save this solution...").click
# # User logouts out of the application
startClicker("OK" , 3) # click the OK button after 3 seconds
sleep 3
@ie.link(:text, "Logout").click
# Closing Browser instance
@ie.close
#
end
end


Here is what the new code looks like:


# Script is designed to execute ATM Smoke test test cases
#
#
#Initials Date Summary
#CLS 10/04/05 Checked in first draft
#CLS 12/02/05 Clean up/Site Selection code
#CLS 08/31/07 Refactored code
#List of to do's or ideas for improvement
#
# BEGIN -
#includes:
require 'watir' # the watir controller
require "rexml/document"
require "date"
require '/apps/ruby/lib/ruby/1.8/test/unit'
require '/apps/ruby/lib/ruby/1.8/test/unit/ui/console/testrunner'
require 'watir/testUnitAddons'
require 'watir/WindowHelper'
require '/SCPTTest/test/Functional/Project1'
include REXML
include Watir
include Project1
include Navigation
puts "## Begin Test - Test1"
puts "## Executes Test cases on SmokeTest Plan"
class Refactor < Test::Unit::TestCase
def test1
@ie = IE.new
@ie.goto("http://localhost:8080/scp/index.jsp")
Login("42","42")
# Prime modes - DN, LD, DIA
# Seconday modes - guidedDIA_mode, advDIA_mode, guidedLD_mode, advLD_mode, guidedDN_mode, advDN_mode
selectAccountModules("Ace Marketing","DNX_ATM_XX","DN","advDN_mode")
selectSites("Dallas Office","Houston Office")
deselectTechnology("Frame RelayTECHNOLOGIES","2_answer")
addAccess("Access Link - 1.5 Mbps","Dallas Office","Houston Office","BOB")
addPort("Port -.536 Mbps:UNI","Dallas Office","Houston Office","BOB")
addConnection("Dallas Office","Houston Office","256.0 kbps","Standard 256kbps","BOB")
quickNavigation()
logout()
end
end

Here are the methods in the module:


# This module provides access to the various data sources used by RPA
# It is designed to make it easier to write RPA functional tests

# Most of the information we need to test RPA is in XML files,
# This module uses REXML to parse the XML.
# REXML is available at http://www.germane-software.com/software/rexml/
require 'watir' # the watir controller
require "rexml/document"
require "date"
require '/apps/ruby/lib/ruby/1.8/test/unit'
require '/apps/ruby/lib/ruby/1.8/test/unit/ui/console/testrunner'
require 'watir/testUnitAddons'
require 'watir/WindowHelper'
require '/projectpath/test/Functional/Project1'
include REXML
include Watir
# This is a module, which can be used Ruby applications and tests.
# To make this module available to your test, add two lines to the file:
# require "Project2"
# include Project2
# There is no need for the test to require or include REXML.
module Project2
require 'watir'
include Watir
# The CatalogData class loads, parses and provides access to the
# PowerSeller catalog XML file, which is how out-of-the-box stores
# product information
class CatalogData
#The constructor for CatalogData takes two strings:
# - scp_dir: the directory where Project2 is installed
# - file: the name of the catalog file, usually 'devCatalogSupplement.xml'
#It creates a REXML Document and assigns it to the instance variable @catalogDoc
def initialize(scp_dir, file)
catalogFile = File.new(scp_dir + "/data/" + file)
@catalogDoc = Document.new catalogFile
end
#Takes a string id, and returns the catalog item with that id, if it
#exists - nil otherwise.
def getItemById(id)
@catalogDoc.elements["catalog/catalogData/item[@id=" + id + "]"]
end
#Returns the string value of the name of the catalog
def getItemName(item)
item.elements["name"].text
end
#Takes a plan element and returns an attribute by Id
def getItemAttributeById(item,id)
item.elements["attributeValue[@attributeId=" + id + "]"]
end
#Takes a plan element and returns the contained price element
def getPlanPriceData(plan)
priceInfo = plan.elements["attributeValue[@attributeId='108']"].cdatas[0].to_s
priceDoc = Document.new priceInfo
end
#Takes a plan and term, and returns a Hash of the pricing info for that plan
#at that term (in months) for the default price sheet
def getTermPrice(plan,term)
priceDoc = getPlanPriceData(plan)
termPriceElement = priceDoc.elements["price/price/dateRange/term[@length=" + term.to_s + "]/discountable"]
termPrice = {"mrc" => termPriceElement.attributes["mrc"].to_f,"nrc" => termPriceElement.attributes["nrc"].to_f }
end
#Takes a string id and returns the string name of that provider
def getProviderById(id)
@catalogDoc.elements["catalog/catalogData/provider[@id=" + id + "]"]
end
end
class Currency
def initialize(float)
@value = float
end
def to_s
currency = @value.to_s
s = sprintf( "%3.2f", currency )
return "$" + s
end
end
# The Table class provides convenience methods for accessing data in
# WATIR tables
class Table
#The constructor for Table takes a WATIR table and
# assigns it to the instance variable @table
def initialize(table)
@table = table
end
#Takes a string and two integers. Returns the contents of
# the table cell at the index of the second integer for the
# row where the string is in the cell at the index of the
# first integer.
# Example: myTable.assoc("findMe",2,4) finds the row of
# the table where the text "findMe" is in the second column,
# and returns the contents of the 4th column
def assoc(text,rowFinder,targetCell)
@table.each { row
if row[rowFinder].to_s.include? text
return row[targetCell].to_s
end
}
return nil
end
def printPretty()
rows = 0
@table.each { row
rows += 1
puts "row " + rows.to_s + ":"
print "\t"
cells = 0
row.each { cell
cells += 1
print "cell " + cells.to_s + ": [" + cell.to_s + "]\t"
}
print "\n"
}
puts "-----------------------------------------------------------------------------------------"
end
end
# This module is intended to provide generic methods for navigation through the PowerSeller application
module Navigation
def startClicker( button , waitTime = 3)
w = WinClicker.new
longName = @ie.dir.gsub("/" , "\\" )
shortName = w.getShortFileName(longName)
c = "start rubyw #{shortName }\\watir\\clickJSDialog.rb #{button } #{ waitTime} "
puts "Starting #{c}"
w.winsystem(c)
w=nil
#assert_same(expected, actual, [message] ) #Expects expected.equal?(actual)
end
#Login for any PowerSeller application
def scpLogin(user,password)
assert(@ie.text_field(:id, "Person.username").exists?)
assert(@ie.text_field(:id, "Person.password").exists?)
# puts "my first variable is " + user
# puts "my second variable is " + password
@ie.text_field(:name, "Person.username").set(user)
@ie.text_field(:name, "Person.password").set(password)
@ie.link(:id, "loginButton").click
end
# Select account and modules
def selectAccountModules(account,solutionName,primemode,secmode)
@ie.image(:src , /ps_header_module_solution_builder_iconUp.gif/).click
@ie.selectBox( :name , "NS_Account").select(account)
@ie.text_field(:name, "NS_Title").set(solutionName)
@ie.checkBox(:id, primemode).set
assert(@ie.checkbox(:id, primemode).isSet?)
@ie.radio(:id, secmode).set
assert(@ie.radio(:id, secmode).isSet?)
@ie.link(:id, "next").click
assert(@ie.contains_text(solutionName))
@ie.link(:id, "next").click
end
#Select sites
def selectSites(site1,site2)
startClicker("OK" , 5)
@ie.image(:id, "selectAllButton").click
assert( @ie.cell(:id, site1).exists? )
@ie.cell(:id, site1).checkbox(:index,1).set
@ie.cell(:id, site2).checkbox(:index,1).set
@ie.link(:id, "next").click
end
def selectFiveSites()
mydata.length
startClicker("OK" , 5)
@ie.image(:id, "selectAllButton").click
assert( @ie.cell(:id, site1).exists? )
@ie.cell(:id, site1).checkbox(:index,1).set
@ie.cell(:id, site2).checkbox(:index,1).set
@ie.cell(:id, site3).checkbox(:index,1).set
@ie.cell(:id, site4).checkbox(:index,1).set
@ie.cell(:id, site5).checkbox(:index,1).set
@ie.link(:id, "next").click
end
def deselectSites(site1,site2)
@ie.link(:id, "back").click
assert(@ie.cell(:id, site1).checkbox(:index,1).isSet?)
assert(@ie.cell(:id, site2).checkbox(:index,1).isSet?)
@ie.cell(:id, site1).checkbox(:index,1).clear
assert_false(@ie.cell(:id, site1).checkbox(:index,1).isSet?)
@ie.cell(:id, site2).checkbox(:index,1).clear
assert_false(@ie.cell(:id, site2).checkbox(:index,1).isSet?)
end
def toggleTermButtons()
assert(@ie.radio(:id, "36ContractTerm").isSet?)
@ie.radio(:id, "12ContractTerm").set
assert(@ie.radio(:id, "12ContractTerm").isSet?)
@ie.radio(:id, "1ContractTerm").set
assert(@ie.radio(:id, "1ContractTerm").isSet?)
assert_false(@ie.radio(:id, "24ContractTerm").isSet?)
@ie.radio(:id, "24ContractTerm").set
assert(@ie.radio(:id, "24ContractTerm").isSet?)
assert_false(@ie.radio(:id, "12ContractTerm").isSet?)
end
# DIA Needs Analysis pages for guided mode
def diaAnalysis()
@ie.link(:id, "next").click
@ie.link(:id, "next").click
end
def diaPreferences(tech1)
# @ie.checkBox(:id, tech1).click
@ie.link(:id, "next").click
end
def deselectTechnology(tech1,premier)
@ie.checkBox(:id, tech1).click
@ie.radio(:id, premier).set
@ie.link(:id, "next").click
end
def selectNVPN(tech1)
@ie.checkBox(:id, tech1).click
@ie.link(:id, "next").click
end
def selectDefaultTechnologies()
@ie.link(:id, "next").click
end
def addAccess(access,site1,site2,assert)
@ie.link(:text, "ADD NEW...").click
sleep 2
@ie.link(:text, "LOCAL ACCESS").click
sleep 2
ie2 = IE.attach(:title, "Local Access")
ie2.selectBox( :name , "loopType").select(access)
ie2.selectBox( :name , "loopSitesAvailable").select(site1)
ie2.selectBox( :name , "loopSitesAvailable").select(site2)
ie2.image(:src , /ps_transferto_button.gif/).click
ie2.image(:src , /ps_save_button.gif/).click
sleep 4
assert(@ie.contains_text(assert))
# assert( @ie.image(:src , /ps_icon_local_loop.gif/).exists? )
end
def addPort(port,site1,site2,assert)
@ie.link(:text, "ADD NEW...").click
sleep 2
@ie.link(:text, "PORT").click
sleep 2
ie2 = IE.attach(:title, "Port")
ie2.selectBox( :name , "portLoop").select("local access 0")
ie2.selectBox( :name , "portType").select(port)
ie2.selectBox( :name , "portSitesAvailable").select(site1)
ie2.selectBox( :name , "portSitesAvailable").select(site2)
ie2.image(:src , /ps_transferto_button.gif/).click
ie2.image(:src , /ps_save_button.gif/).click
sleep 2
assert(@ie.contains_text(assert))
assert( @ie.image(:src , /ps_icon_local_loop_with_port.gif/).exists? )
end
def addServicePlan(domestic,mac,international,site1,site2,assert1,assert2)
@ie.link(:text, "ADD NEW...").click
@ie.link(:text, "SERVICE PLAN").click
ie2 = IE.attach(:title, "Service Plan")
ie2.selectBox( :name , "servicePort").select("port 0")
ie2.selectBox( :name , "domesticServiceType").select(domestic)
ie2.selectBox( :name , "domesticServiceMacLevel").select(mac)
ie2.selectBox( :name , "internationalServiceType").select(international)
ie2.selectBox( :name , "internationalServiceMacLevel").select(">= 0")
ie2.selectBox( :name , "serviceSitesAvailable").select(site1)
ie2.selectBox( :name , "serviceSitesAvailable").select(site2)
ie2.image(:src , /ps_transferto_button.gif/).click
ie2.image(:src , /ps_save_button.gif/).click
assert(@ie.contains_text(assert1))
assert(@ie.contains_text(assert2))
# assert( @ie.image(:src , /ps_icon_local_loop_with_port.gif/).exists? )# User moves to Solution Status Page
end
def addConnection(site1,site2,speed,connection,assert)
@ie.link(:text, "ADD NEW...").click
sleep 2
@ie.link(:text, "CONNECTION").click
sleep 2
ie2 = IE.attach(:title, "Connection")
ie2.selectBox( :name , "connPort1").select("port 0")
ie2.selectBox( :name , "connSitesAvailable1").select(site1)
ie2.image(:src , /ps_transferto_button/).click
ie2.selectBox( :name , "connPort2").select("port 0")
ie2.selectBox( :name , "connSitesAvailable2").select(site2)
ie2.link(:url, /connSitesAvailable2/).click
ie2.selectBox( :name , "Bandwidth").select(speed)
ie2.selectBox( :name , "connType").select(connection)
ie2.image(:src , /ps_save_button.gif/).click
sleep 2
assert(@ie.contains_text(assert))
assert( @ie.image(:src , /ps_icon_local_loop_with_port_and_connection.gif/).exists? )
end
def addProductBundle(bundletype,bundle,site1,site2,assert)
@ie.link(:text, "ADD NEW...").click
@ie.link(:text, "PRODUCT BUNDLE").click
sleep 10
ie2 = IE.attach(:title, "Product Bundle")
assert(ie2.selectBox(:name, "bundleType").exists?)
assert(ie2.selectBox(:name, "bundleType").enabled?)
ie2.selectBox( :name , "bundleType").select(bundletype)
ie2.selectBox( :name , "bundleItem").select(bundle)
ie2.selectBox( :name , "bundleSitesAvailable").select(site1)
ie2.selectBox( :name , "bundleSitesAvailable").select(site2)
ie2.image(:src , /ps_transferto_button.gif/).click
ie2.image(:src , /ps_save_button.gif/).click
sleep 4
assert(@ie.contains_text(assert))
assert( @ie.image(:src , /ps_icon_local_loop_with_port_and_connection.gif/).exists? )
end
def quickNavigation()
@ie.link(:id, "next").click
@ie.link(:id, "next").click
@ie.link(:id, "next").click
@ie.link(:id, "next").click
@ie.link(:id, "next").click
@ie.link(:id, "next").click
@ie.link(:title, "Suspend and save this solution...").click
end
def diaNavigation()
@ie.link(:id, "next").click
@ie.link(:id, "next").click
sleep 2
@ie.checkBox(:id, "sol_0").click
@ie.link(:id, "next").click
@ie.link(:id, "next").click
@ie.link(:id, "next").click
@ie.link(:id, "next").click
@ie.link(:id, "next").click
@ie.link(:title, "Suspend and save this solution...").click
end
def logout()
startClicker("OK" , 3) # click the OK button after 3 seconds
sleep 3
@ie.link(:text, "Logout").click
@ie.close
end
end
end

If you have questions please let me know.

Wednesday, February 07, 2007

Viewing data in a Table

I have been meaning to post this code for awhile. There have been some recent posts on the WATIR forum about looking data in a table cell, so I thought I would share this code. I can not promise it meets all table needs, but it has worked well for us.

I typically evaluate are tables in this manner.

@ie.show_tables

The I pick the table I believe contains the data I am attempting to access and call this method:

@Table.new(@ie.table(:index, 40)).printPretty

Print Pretty code is placed in a library file and I require it for the scripts that need to evaluate tables.

Here is an example as to how you could use this.

Create a file called MSNTest.rb with this code:

# CLS February 7,2007 Example for finding table data
#includes:
require 'watir' # the watir controller
require "rexml/document"
require "date"
require '/apps/ruby/lib/ruby/1.8/test/unit'
require '/apps/ruby/lib/ruby/1.8/test/unit/ui/console/testrunner'
require 'watir/testUnitAddons'
require 'watir/WindowHelper'
require '/Apps/watir_bonus/examples/Stuff.rb'
include REXML
include Watir
include Stuff

puts "## Begin Test - MSN Table Test"



class MSN_Test < Test::Unit::TestCase
include Watir


def setup
@ie = IE.new
@ie.goto("http://www.msn.com")

end

def test1

assert(@ie.link(:text, "Sports").exists?)

# @ie.show_all_objects

@ie.show_tables

Table.new(@ie.table(:index, 1)).printPretty
end
end

#

#
# END - MSN test

In the same directory create a file called Stuff.rb with this code:


# This module uses REXML to parse the XML.
# REXML is available at http://www.germane-software.com/software/rexml/
require "rexml/document"
require "date"
include REXML

# There is no need for the test to require or include REXML.
module Stuff

# The Table class provides convenience methods for accessing data in
# WATIR tables
class Table
#The constructor for Table takes a WATIR table and
# assigns it to the instance variable @table
def initialize(table)
@table = table
end

#Takes a string and two integers. Returns the contents of
# the table cell at the index of the second integer for the
# row where the string is in the cell at the index of the
# first integer.
# Example: myTable.assoc("findMe",2,4) finds the row of
# the table where the text "findMe" is in the second column,
# and returns the contents of the 4th column
def assoc(text,rowFinder,targetCell)
@table.each { |row|
if row[rowFinder].to_s.include? text
return row[targetCell].to_s
end
}
return nil
end

def printPretty()
rows = 0
@table.each { |row|
rows += 1
puts "row " + rows.to_s + ":"
print "\t"
cells = 0
row.each { |cell|
cells += 1
print "cell " + cells.to_s + ": [" + cell.to_s + "]\t"
}
print "\n"
}
puts "-----------------------------------------------------------------------------------------"
end

end
end

Execute the MSNTest..rb script to see the results of the content of the table on the MSN home page. This site happens to only have one table, but if there were multiple tables just use the index value of the table your target data resides (Table.new(@ie.table(:index, ??)).printPretty).

Enjoy!

Friday, January 26, 2007

Trip Report for AWTA

The Austin Workshop for Test Automation, AWTA, took place in Austin Texas January 12 -14, 2007.

The information below outlined italics is intended to be from my perspective, including tangent thoughts during the conference.

The first think worth note is the meeting format. The meeting took on its own flavor of LAWST meeting style.

Meeting began with brief introductions round table, stating who you are and what you hope to gain from workshop.

Next a participant gave an experience report. Each person at the workshop was given a green, purple, and pink index card. During the experience report the only questions permitted are clarifying questions. If it is not a clarifying question the facilitator will move the report forward and table the question. After the experience report it becomes “Open Season”. Here is where the cards come into play. If you have a question or comment regarding the report you hold up your green card and the facilitator puts your name on a list. If a discussion is going on and you want to add input you hold up the purple card. If you think a topic has deviated from the topic or you think it has been beat to death you raise the pink card. This format is a very good means to keep a meeting on topic and permit everyone to participate.


The first experience report was presented by Jean McAuliffe. She previously had worked for Rational and had gotten tired of the rigorous process. She went to work for a start up that used Agile techniques. She fell in love with the approach and began using an open source tool called Fitnesse. This tool is based on an open-source project called Fit.

This is a tool that can allow the team to test without the GUI. Column based data from a Wiki page are fed into fixtures created by the developers. This is a cool way to test, but I am not sure it is appropriate for us. In my opinion we need to have Unit Test coverage and a more robust UI testing suite.


Jean explained how she left this agile team to lead a start up. All of the effort was sweat equity and she found it difficult to implement Fitnesse, because the developers did not want to spend time writing the fixtures. These tools are unique in that there are no tools like it on the open market. Sometimes test work fine in Fitnesse, but then fail at the GUI level. The Fit discussions lead Paul Roger’s and others to begin talking about the difficulties of open source tools. “What would make open-source tools easier to use? Bulleted ListSome responses included:

  • More concrete or real world examples
  • More Peer help
  • Better documentation
  • Documented in different ways to serve a wider audience
  • Identify and bring in Tool Smiths

Brian Marick added that consumer of tools needed

  • Direct communication to the experts
  • Developers to have a good attitude toward Newbies
  • Acknowledgement to members who submit quality defects
  • Perhaps documentation similar to Wikapedia, but can only add content and not edit the original content. A place where clarifications, struggles, and resolutions can be entered.
  • Concept of having a "WATIR buddy" – paired test programming
Brian added that Selenium was geared more toward developers and WATIR received a wider audience.

Danny Faught the context school of testing and the importance of Agile.

It is a real puzzle to the testing community as to what tools should be used. One example is WATIR versus Selenium. Potentially Selenium RC has offered an improvement. Selenium uses HTML tables and is not re-factorable. Selenium RC comes with API’s to plug into using Ruby. There was a ton of conversation around how to make open-source tools better and I unfortunately did not capture it all.

A point of clarification or realization for me was that WATIR was developed for IE only. Teams were leaning toward Selenium where multi-browser support is required. There are new projects Fire WATIR and Safari WATIR to assist with other browsers.


Another observation about this workshop format was the use of GIANT Post it pads. Concepts were written down by scribes and posted all over the room for future elaboration and discussion. This was a pretty cool way to capture the concepts.

The next portion of the meeting was very interesting. They referred to it as “Brain Storming”. This was an hour long whirlwind of problems and ideas around the topic “Costs, Limitations, and Improving Test Frameworks”.


Next experience report came from Bob Cotton of Rally. Bob showed some of the work they were doing using RSpec, which is behavior driven development versus test-driven development. He referenced Mike Cohn’s testing pyramid.







Fit or Fitnesse is typically used to test the middle layer. JUnit is typically used for the Unit Test. And WATIR for the UI portion. I got the sense that the Acceptance portion is expected to be executed without the UI and consists of direct testing of business objects. Testing should be done proportionally with the most being Unit Testing, less Acceptance and even less UI. The other interesting comment was that he drew a tiny circle above the period indicating the amount of manual testing that should take place.

Bob talked about an approach where the things you do not have time to do should be doe first. How are you going to test should be the focus and not how are you going to build. He talked about doing functional test and acceptance testing through the GUI via a layer of abstraction.

He used Ruby to build a set of new methods that would be leveraged in Selenium (i.e. – attr_reader and attr_writer). Test suites then run nightly and they typically take about 3 hours.

He talked about behavior-driven development using RSpec which followed the pattern of state the setup/context (state of the world), specify (action), and teardown/end (close the test).

His code was based on Ruby (which out of the box has an XML parser), Selenium RC, RSpec, Test::Unit, and proprietary code.

There agile approach is to understand the story, then determine how to test the story.

  • What are the acceptance tests?
  • Unit tests should be XYZ
  • Functional tests should be ABC
During open season a question was asked why did the team throw away several frameworks. They first used HTTPUnit and could not efficiently use it because it could not do AJAX type stuff. They tried something called JIFFY, which crashed frequently and provided false positives. They finally settled on Selenium and planned to extend the functionality to meet the needs of the team.

I really liked structuring the test cases with the RSpec format: set up, do, teardown format.

Example:
context "BDD framework" do
setup do
@bdd_framework = BddFramework.new
end

specify "should be adopted quickly" do
@bdd_framework.should_be_adopted_quickly
end

specify "should be intuitive" do
@bdd_framework.should_be_intuitive
end
end

Fit has the ability to test the acceptance layer of the Testing Pyramid. What would replace Fit?

New tools need to be developed to exercise the middle layer (e.g. – JRuby). Brian had some comments. He stated that he agrees that Domain Specific Language (DSL) is important. DSL is a programming language designed to be useful for a specific set of tBulleted Listasks. DSLs focus on doing one sort of task well (e.g. – Ruby for Test Automation).

Brian discussed Business facing tests.
  • Tests of Business logic
  • Workflows through the system
  • Presentation tests - UI- How does it look to a person?

A Fit table does this by describing the way the system works. He proposed pick up the look and feel of a page and turn it into an automated test using Ruby. He gave a brief demonstration of code and it was a very unique way to test page views.

At this point it occurred to me how important it would be to test PowerSeller in IE7 before our clients ask for it. I set this as an action item for myself.

Next another interesting format to this workshop took place called “Open Spaces”. Basically everyone in the room could toss out ideas of things they wanted more information on or things they wanted to present to those interested. Many topics are place on cards and taped to the wall. After the topic session was complete someone would grab a topic and head to the corner of a room. Essentially people would vote with there feet and go to the topic they were interested in.

I chose to participate in the discussion of creating a Common Browser driver that supported both WATIR and Selenium. The other discussions in the room focused on What it means to be a Tool Smith and a Custom JavaScript project.

Bret Pettichord and Bob Cotton did most of the work and discussion on the common driver. The group split into two groups when Paul Rogers decided to focus on his new approach to test automation called RUMBA.

RUMBA in a nutshell is a framework that places a tiny piece of flash code on every web page, then flash is used as the testing framework. Unfortunately I do not understand the specifics, but I think it has great potential.

Developing the common driver was an interesting topic. From my perspective I wondered why bother since other WATIR projects are moving ahead full steam to test other browsers. The collaboration between Bret and Bob was fun to watch. My take away was a style of coding and two books were recommended, Ruby Cookbook and Programming Ruby. As they were collaborating, Brain Marick stopped by the group and suggested the name for the project, Mineral WATIR.

Miscellaneous tools were mentioned:

SYSTIR – I believe this is a domain specific language using Ruby

WATIR::Simple – I believe this is also intended as a DSL.

Tclockexe – Can change the format of your machine clock. Good for dating screen shots.

PrintDesktop.com – Allows you to send a snapshot of your desktop directly to a printer. One advantage would be if your machine freezes and you want to show IT the state.

XPlanner – open source project management tool

We received a summary report of the Tool smith discussion.

Someone mentioned we should read an article by Eric Raymond. Look to works by James Bach on the definition of a good test. Also check out an innovative company called Atomic Object. Carl Erickson and David Crosby were at the AWTA.

Next was another really cool format used during the workshop, “Lightning Talks”. Basic format is that anyone can speak, but they only have 5 minutes to make their point.

Chris McMahon took the lead with his talk on Better Marketing for Open Source tools. High energy pitch on how software should be represented today. Software development is not a factory and not as described by structuralists. Software is not engineering or manufacturing. Software involves unity, complexity, and variety. Today software is a craft and a discipline. Using the language of art, music, and culture is the best way to talk about software.

Al Snow then talk about “User Groups Internal to a Company” introducing FIT at his company. They formed internal user groups and held routine meet ups to discuss FIT. The important take away for me was to get others involved. Al was able to collaborate with other users of FIT within his organization.

Rick Hower talked about How to Find Useful Information about a Tool. Today it is difficult to choose the correct testing tool. Do you do thorough research or do you just go for it and pick one. There are 400 – 500 different tools to choose from. In summary the documentation needs to improve and there needs to be more comparisons across tools. Tools should clearly list their strengths and even their weaknesses

A possible tool for testing web services can be found at soapui.org.

Danny Faught spoke on Tool Review and the OpenSTA exceptions. Is an open source project healthy? He talked about looking at activity on user forums, number of downloads or the number of releases for a product. However even those statistics could be misleading. One example was SCLC which is a valuable tool, but really has had no active development since 2003. Another one mentioned was MWSnap. He talked about the health of JUnit. He ran out of time to contrast OpenSTA. I spoke with him later and he stated that on the surface it appears to be a healthy product, but apparently the tool is not maturing and contains complex defects.

Other possible open source load testing tools for consideration are JMeter and TheGrinder

Martin Taylor illustrated his testing framework that interacts with test director called EPDAV (Environment Preconditions Data Actions Verification. This is a proprietary framework and Martin is attempting to make it open source.

David Whitesell gave a lightning talk on Distributed Execution Monitoring and Resource Management. He hated Winrunner because he could not use his machine while winrunner was executing and he could not run multiple browsers. David switched to selenium and uses a Java based structure to drive and monitor distributed test executions.

Jeff Fry demonstrated some really cool WATIR code. The code essentially generated methods on the fly as it conducted google searches. The code was used to reproduce a random defect in google search algorithm that he discovered.

Jim Mathews (from Austin) gave a talk on a Fitnesse implementation. He quickly illustrated how good testing is fostered by good architecture. He could easily structure Fitnesse to execute object level tests.

Chris McMahon gave another talk, but I did not take good notes. The only thing I noted was that I needed to get a copy of a Better Software article from August 2006 called Old School Meets New Wave.

Martin has a follow up presentation on how EPDAV was leveraged as a self load balancing client. It uses a combination of python and XML.

Al Snow expressed his thoughts on developing a Hybrid framework or putting the glue between all the testing pieces.

David Crosby presented System-Test Driven Development be leveraged at Atomic Object. He stressed the importance of starting with the simplest thing that could possible fail. He sighted an approach called 37 signals, which I need to learn more about.

Brian Marick gave presentation and my two brain cells have no clue what it was about. This could have been his position on how the latest version of Fit broke his new testing framework.

Elisabeth Hendrickson demonstrated here JavaScript site called WebStickies and how to test it using JSUnit and Selenium “Square head” addition.

It will be interesting to see if Easy Test can handle testing an application such as this (Web 2.0).

Brian Marick then gave an experience report with Fit. Fit development was led by Ward Cunningham who made the first Wiki. There were three types of ways to execute Fit: Column based data, Row based data, and Action based data.

SageCRM is a workflow tool. It was tossed out in conversation and I wanted to investigate it. I am not sure if it is even open source. Someone also mentioned tools from Green Pepper and thought they were similar t Fitnesse.

We had an experience report from the founders of WATIR, Brett Pettichord, Paul Rogers, and Charley Baker (new Project Manager). The project started out as IE Controller and the foundation was the test harness. WATIR was only intended to be a library. There was discussion on how to handle the growth of an open source project, manage defects, and manage contributions. It was pointed out that you rarely have the opportunity to hear the story first hand from all of the original stake holders.

In my opinion the project has been successful, because the owners insist on having unit tests for all methods.

In the Open Spaces section I had the opportunity to discuss Chris McMahon’s upcoming article in Better Software, Power of Three – A Trio of Techniques for Testing Databases. Article includes some excellent approaches using Ruby to validate data in a database. Code needs to require ODBC, which are standard drivers in Ruby.

I then looked over the shoulders of Elisabeth and Paul as they worked on test automation for the WebStickies site. My take away was a sound approach to troubleshooting and organizing code. I still do not fully understand JSUnit and Selenium, but it was cool to see the tools in action.

Domain Specific language for the web is Ruby on Rails and for system testing it is WATIR.

Buy the “Everyday Scripting in Ruby” by Brian Marick

Carl and David from Atomic object talked about the concept of Developer Testers and Tester Developers. Chris McMahon offers his perspective on his blog.

We went into a final session of Lightning Talks.

Paul Rogers – Demonstrated his work on finding all objects on a page using Ruby.

I need to ask if this code is available. It is similar to our “print pretty” method for tables.

Bret Pettichord – Demonstrated how he warps time using Ruby.

Pretty cool code if you need to test timeouts and things of that nature and do not want to wait.

Martin Taylor – Discussed how he re-factored his framework to not be tightly coupled to Test Director and has the potential to integrate with other applications including open source.

Bret Pettichord – demonstrated direct interaction with Ruby and Excel. As the tests executed you could visually see the cells in Excel change color based on the result.

I would love to understand how to do this. This would probably be my Mt. Everest.

Paul Rogers – demonstration on why relying on spreadsheets might not be the best solution. Data-driven testing can become a maintenance issue as the number of spreadsheets and test cases increase. His team has 500 spreadsheets and 3500 test cases. One change in the application could result in changing numerous spreadsheets. The challenges are how do you know which spreadsheets are impacted and how do you quickly change the data.

Brian Marick – Follow up demonstration on how to use the Ruby library to change time. Mentioned that he had an example where he used Ruby to redefine plus to be minus.

Andy Tinkham – mentioned that he was working on his thesis. The objective was to build a testing tool kit that could live on an external drive and be carried anywhere by a Tool smith.

Here are my final notes. I need to read, Enterprise Integration Patterns by Hohpe. Check out StoryTeller (note there may be a better link). And in the near future I should check out SmartMonkeyTesting.com for a Java controlled testing framework.

Here are some key perspectives and action items.

It was unbelievable to be submersed in the conversation with brilliant open-source personalities who were purpose-driven. The group openly shared there thoughts and opinions while respecting everyone. There was laughter and subtle humor throughout the workshop. The atmosphere was kept light but educational. I left the workshop each evening with my two brain cells going full steam ahead. I was surprised I even got any sleep. There is a wealth of knowledge out there regarding test automation and it is there for the taking. I now find myself every night reading articles, projects, and yes even reviewing code.


Open Source
My action items are to become more involved in the WATIR community, specifically assisting with defect management. Eventually reviewing the methods and assisting with the RDocs. I am determined to focus on learning Ruby and WATIR. I plan to re-factor all of our current WATIR code. I see many flaws in they manner in which I implemented it.


Current Project
I am enlightened as to how little testing we actually do on our customization releases and I am determined to find ways to be smart and improve our testing practices. Our team is not actively engaged in developing unit tests. I hope the team will embrace Unit Tests and extend our test coverage. I think we have been extremely lucky that we have not had any major defects discovered by the client in the past 6 or 7 releases. I want to find ways to test the business objects without the UI and also validate the database.

Corporate Community
I am challenged to get involved on a corporate level with a few things. Reach out to other teams and understand their testing practices. Learn more complicated aspects of our internal tool, Easy Test. How does Easy Test compare to WATIR. I would also like to organize a meeting of testers within our company and hold a brief workshop similar to AWTA. Definitely a challenge but something I plan to explore.