Introduction
It’s been a while since I’ve posted about some BDD work, but I’ve recently gotten back into it using Behave. Over the next few months, I’ll be posting more and more, as I get deeper into the tool, but I figure I’ll start with a comparison based on my initial work. This post will focus on a comparison between Cucumber and Behave; while both use the same Given When Then (GWT) format, there are some major differences in the implementation and tooling, and I’ll start by pointing out some of those. Expect some best practices, and a deeper dive into areas that I’ve call out in this post in my future posts.
Packaging
To start off, the biggest difference I’ve noticed is the way you install Behave versus Cucumber. Your cucumber tests can be packaged, and then deployed and run on any system that has a jdk installed (if using Cucumber ported for Java). Alternatively, if using true Cucumber, the only thing needed on the machine is ruby to be installed. This means all of your tests, code, libraries, etc, can be packaged and deployed to any test or development machine, unpacked, and then run.
Behave, alternatively, needs to be natively installed on any machine that you choose to run it on. While this isn’t necessarily a huge deal, it may limit or complicate some deployments and executions of the tests, depending on the nature of the application under test. If running back end testing, where the machine is important to the tests, or running in a dynamic environment, this additional installation can make testing/deployment more complicated. Ideally, we want to keep things as uncomplicated as possible, and unfortunately, Behave isn’t a quick/easy thing to install as it currently exists
Logging
Unfortunately, by default, Behave does not provide a lot of logging. All tests that run write out the GWT test steps to the command line, but no log file is generated. It is relatively simply to turn on additional logging, but this goes more towards debugging, not towards creating a true documentation of the test steps that were run, provided in a separate output file. Cucumber by default generates either a JSON or HTML file, which makes is simple to deliver/store the results to third parties. There are a few hacks within Python and Behave that will allow us to put these test steps into an output file instead of the command line, but these are ugly, and not ideal
Hooks
Behave actually offers some better hooks than Cucumber does. Obviously, this can change slightly because of the language chosen for Cucumber, but by default, Cucumber only offers before and after hooks. RSpec and Test::Unit or JUnit and TestNG (for Ruby and Java respectively), provide some additional before and after methods, but even those aren’t as good as the ones offered by Behave. The before_all hook is unique to Behave, and is extremely useful when settings up large functional test suites. Additionally, making use of Python’s extensible classes, those hooks become even more powerful.
Data Parameterization
Unfortunately, Python doesn’t support data parameterization. Unit testing can be parameterized, but this uses some clunky, ugly code. Utilizing the default Gherkin support for data is the best method to compensate for this, but unlike languages like Ruby and Java, additional parameterization isn’t supported, meaning that testing over multiple browsers, or interfaces isn’t supported without coding this into the test steps. Also, because Gherkin is limited to a single set of data parameters (no multidimensional data sets), you can’t loop multiple data sets over multiple browsers. I will be looking into further elegant ways to overcome this limited, so definitely stay tuned.
Some Similarities
Behave even touts being very similar to Cucumber, especially when Cucumber is used with rubypython. There are a lot of great tools and techniques that can be re-used. Further posts will also expand on these further, but just a few similarities are below.
The GWT language is the same. Behave actually uses Gherkin, which makes porting over your tests from Cucumber, or cross training your testers/SAs a breeze. This also means that all of the data structures you are used to from Cucumber can also be used. Additionally, that wonderful Cucumber Parser can be re-used.
Tagging in Behave is setup similarly to tagging in Cucumber, even making use of the same @WIP to indicate which tests are not yet ready to be executed.
