LESSON #1: If your tests don't run fast enough, they won't be able to run frequently, i.e. they won't be useful
LESSON #2: Mock out layers below what you want to test, to make tests run faster. Now you've got a lot more "small" tests that just test 1 layer at a time, e.g. web client separately, application code separately. Caveat: watch out for no integration between modules
LESSON #3: Write smaller number of end-to-end tests, maybe 15 or 20, even for a large project
Some useful things I learned:
- heatmap tools (where can I get one?) show graphically which code modules have the highest cyclomatic number. These are the ones you want to focus on!
- STAF Automation Framework (XML, Python, Java) http://staf.sourceforge.net/
- Data generators http://www.generatedata.com, http://www.mathworks.com/matlabcentral/
- freshmeat.net, search projects for software testing, logs, etc.
- Functional UI test tools: Selenium (html), Canoo Webtest (htmlUnit), FEST (java swing)
- Behavioral testing: easyb (Groovy, RSpec)
- Load & performance testing, read paper for ideas http://edms.cern.ch/file/925013/3/EGEE-Grid-Cloud.pdf
- Designing the Software for testability will help both developers & testers. Guideline is SOCKS (Simpler components, Observable states, Control, Knowledge of expected result)
- Code review tool - http://www.review-board.org/
- Testing equipment? Try http://www.fanfaresoftware.com/products/howItWorks/index.aspx
- Model-based testing http://en.wikipedia.org/wiki/Model-based_testing
- Mock objects: check each language for implementation details (java, ruby http://mocha.rubyforge.org/)
two Hendriks from Sweden