Agile Testing with Python Test Frameworks.notes
Wednesday, March 23, 2005
TITLE OF PAPER: Agile Testing with Python Test Frameworks
URL OF PRESENTATION: _URL_of_powerpoint_presentation_
PRESENTED BY: Grig Gheorghiu
REPRESENTING: Avamar
CONFERENCE: PyCon 2005
DATE: Wednesday, March 23, 2005
LOCATION: GWU Cafritz Conference Center, Grand Ballroom
--------------------------------------------------------------------------
REAL-TIME NOTES / ANNOTATIONS OF THE PAPER:
{If you've contributed, add your name, e-mail & URL at the bottom}
Agile Testing with Python Test Frameworks
What is agile testing
Practice TDD (test-driven-development)
Short feedback loop between developers and testers
find bugs that matter early rather than late
collaborate with programmers to improve testability
shift from last line of defense to service provider
focus on providing value
fluid roles: do whaever job needs to be done
toolsmith role: automate what you can , delivery in less than 40 hours
spend time and resources only on things that provide information
Involve customers early in the writing business-facing tests
Deploy automated regression testing
Conduct sessions of exploratory testing
Python As An Agile Language
Simplicity
clean and simple syntax reads like pseudo-code
Buitl in high level data types
natrually object-oriented, encourages code reuse
Communication
powerful yet simple idons
lets you code and oesn't get iyour way
standard coding style enforced by significantwhitepspace
execuatable docuemtnation
FeedBack
dynamci interpreted: short dev cycle, fast feedback loop
interactive shell session procices instantaniuosu feeback
unit test frameowkrs (unittest, doctest, py.test) enble frequent feedback
courage
agile testing practice: Practice TDD
port of jUnit to python (un-pythonic)
easy to use by people familiar with the xUnit frmeworkds
test classes need to inhereit from unitest. TestCase
api can get in the way
Unit testing with Doctest
executable documentation or literate testing
no api to remember
perfect way to keep documention in sync with the code
can be combined with epydoc to produce "agile documentation"
no provisions for test fixture management or test suite organization
no api
test functino /methods start with test and test claees with test
save test code in files
use normal assert statment
setup[method| class| module] hooks provide test fixture mangement and method,
pylint: pthon source code analyzier
coverage
accumulates vcoverage data
buildbot
port of MikeClark's jUnitPef
adds performance testing capabilities to existing unittest tests
provides 2 types of performance tests
timedtest: runs an existing test case by imposing a limit to the time
load test
main functionality my team needs to test
backup a set of files/directories/mail messages/ databases
restore them
verify that they are restored properly
we encapsulated the functionality in a BRV class (Backup-Restore-Validate)
test reulst management encapsulated in a test class
logs results to stdout, logfiles, databawe
http post mechanism: apache web server with firebird db backend
generation of lagre data sets for load testing
100s of AD users/ exchange mailboxes (win32com, win32net)
1000s of Exchange folders/messages (CDO/MAPI/ADO
STAF/STAX
our goal: automated regression test on many clients/platforms
from IBM: automated test distrubution, execution and reporting
STAF agent runs on all the machines STAF testbed
STAX service machine cordinate agents, gathers all results and logs
commands are sent to STAX service host vial XML "job" files
Jython code can be embedded in script elements inside XML
command are executed on agents via rocess elements
function definitions and calls
iterate, paralleliterate, timer constructs
logging and notification constructs
Global Python variables vis SCRIPTFILE and SCRIPT commands
Java GUI tool
Used by Linux Text Project (ltp.sf.net)
Nightly build completeion message sent to distrubtion lists
.procmailrc file triggers Python script whichs STAX job file to the STAX service machine
Script queries central log file for the new job
agile testing practices
involve customers in writing business-facing tests
automated acceptance testing
FitNesse is based on Ward Cunninghams' FIT
tests as HTML tables
FtNesse specific: Web server/Wiki, Test Suites, Setup/teardown pages
FIT/FitNesse: test the business logic below the GUI
enforce good code design practicies (MVC)
create test interfaces for better code
Two categoreis of test tools
tools that simulate browsers (web protocol drivers)
implement http request/resonse protocol
some parse html into dom
examples, mechanize, webunit, pbp, twill, maxQ
tools that automate brosers (web browser drivesr)
examples: selenium pamie, jssh, watir
maxQ uses roxy to cature http requests/rpelies
tool is written in java, but generates jython scripts
record/layback capability
captures http resonse codes and raw html
can verify that a given test or URI exists in the http
selenium
uses real browser to play back testing scripts
based on javascript: cross-platform and cross-browser
tests can be written as html tables, similiar to fitnesse
browser can also be driven via scripts (python ruby, perl)
testrunner mode: static html, js, css pages on web site under test
driven mode: standalone server (twisted-based)
bypasses javascript
showing an example of using selenium actualy testing a plone registration site in real-time
jython
number one contender in the dynamic java arena (scripting the jvm)
combines agility of python with easy acccess to java libraries
gvr: "testing (popular area for Jython)"
java teast tools can be easily driven from within jython
web app testing with htpunit from jython
many test tools use jython as the test scripting language
bret pettichord: "homebrew test automation"
agile toolsmith: delivery something usefule in less han 40 hours
huge variety of open source test tools available
use high-quality scripting langnuage
holistic testing
unit testing with unitest doctest, py.test
nightly regression testing with STAF/STAX
business logic acceptance testing with PyFIT/FitNess
python/agile testing blog: agiletesting.blogspot.com
Q. does selenium support web apps that use frames
A. it does - from what he knows there are some subtle issues with IFrames
Q. how does selenium verify the test ran correctly
A. it uses a DOM parser to parse the page and identify the elements to be verified
if your links use id's then all is easy, otherwise you have to use xpath and the like
--------------------------------------------------------------------------
REFERENCES: {as documents / sites are referenced add them below}
ward cunningham's FIT: http://fit.c2.com/wiki.cgi?WelcomeVisitors
selenium: http://selenium.thoughtworks.com/index.html
author blog: http://agiletesting.blogspot.com/
--------------------------------------------------------------------------
QUOTES:
--------------------------------------------------------------------------
CONTRIBUTORS: {add your name, e-mail address and URL below}
Mike Taylor
Nick Bastin (nick.bastin@gmail.com) <http://www.nickbastin.com>
--------------------------------------------------------------------------
E-MAIL BOUNCEBACK: {add your e-mail address separated by commas }
--------------------------------------------------------------------------
NOTES ON / KEY TO THIS TEMPLATE:
A headline (like a field in a database) will be CAPITALISED
This differentiates from the text that follows
A variable that you can change will be surrounded by _underscores_
Spaces in variables are also replaced with under_scores
This allows people to select the whole variable with a simple double-click
A tool-tip is lower case and surrounded by {curly brackets / parentheses}
These supply helpful contextual information.
--------------------------------------------------------------------------
Copyright shared between all the participants unless otherwise stated...