For business or tester facing tests on projects (not unit tests), I\'ve typicall

Question

For business or tester facing tests on projects (not unit tests), I've typically used FitNesse, but I've dabbled with RSpec and I'm interested in learning Selenium and Robot.

Do you have experience in these (or other) acceptance testing tools? What are the pros and cons?

For FitNesse, I can think of the following pros and cons initially.

PROS:

Wiki based editing, which most people understand
Lots of support and plugins for different languages
Completely self contained, you just download and go
CONS:

Refactoring the tests can be a little clunky
You need to know (or document) the various properties that each method in your test fixture expects. There's no "intellisense" or IDE to do this for you that I'm aware of.

Explanation / Answer

We use Robot Framework extensively and Fitnesse a little.

Fitnesse I think has an edge if you like the wiki interface that it exposes to people writing the tests - it's particularly good if your data consists of tables of primitives with expected inputs and outputs. Once you get beyond the basics though it gets clunkier and clunkier.

Robot Framework is extremely good if you're testing workflows either directly or indirectly (e.g. if you're testing through the GUI and need to reach a particular position in a system with no test hooks).

It works around the idea of keywords which can be composed from other keywords - which allows you to organise your tests into layers of abstraction, but in a way which people from a non-programming can grasp and leverage. There are a number of libraries out there (for example a wrapper around Selenium for web testing and others for Java and Windows apps, telnet, etc).

I've found that we could get testers with no previous test automation experience editing and maintaining tests with no or little developer support with a couple days (aside from initial setup) and within a week or so they were creating new tests in a maintainable manner by themselves. I don't think any of the other frameworks I've seen really allow that without a lot of developer support.

I don't have direct experience of the other acceptance testing frameworks out there - but from what I've seen from brief dabbles plus experience reports at conferences I think if it's primarily developers (or testers/analysts with moderate to good coding skills) who will be creating and maintaining the tests then probably one of the other frameworks could be a good choice. Cucumber (or its variations) is talked about a lot for example. I really liked the look of something I saw at CITCON London 2010 by YouDevise which has apparently just been released as open source - I think this is it.

I don't think there's a one-size-fits-all solution - different tools are good for different contexts and mixes of users.

Related Questions

Navigate

• Browse (All)
• Browse F
• Subjects

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.