Introduction
The more I have gotten into developing testing frameworks based on Cucumber, the more I’ve seen how important writing effective cucumber tests are. The feature files provided earlier are not a good example of cucumber tests, and below we will cover why, and how to turn these into effectively written test cases.
A badly written test informs little of the system, cannot be tested against multiple interfaces, and most painfully requires more upkeep. A smartly formed test can not only be re-used, but can also directly inform anyone reading the test exactly what the system is capable of. Cucumber is a tool that supports Behavior Driven Development, BDD. People think that the only place where a system has behavior is in the user interface and especially in the graphical user interface. This is not the case. All systems have behavior at different places and different levels. Most applications have multiple interfaces, and each of these can be tested by examining their behavior. An important point is that interfaces generally behave the same. This demonstrates how Cucumber and BDD is not about testing GUIs. It is about systems behavior.
Cucumber is Declarative
BDD and Cucumber should be written in a declarative fashion. Define properties that should hold true for a particular solution. A set of boundaries the solution should stay within should be outlined; however the steps to arriving at the solution should not be included. You don’t know, and should not care, about the implementation when you define your feature files. The implementation comes later and may be someone else’s responsibility. These tests should be written from the point of view of an architect, not of a developer. The focus needs to stay on what, not how.
How To Write Tests
The whole point of BDD is that it is vitally important to write each test/feature step
- One at a time
- With a domain expert
- In plain Language
The use of plain language in the feature file is crucial to successful use of Cucumber. State the result that you wish to obtain while avoiding specifying how you expect to get it.
Test Step Amgibuity
Because there must only be one method (step) which matches the regular expression in the classpath, it is important to view the system at a high level. These methods are global; the reasoning being that they describe a part of the system, and just as a system with redundant systems might need a design, so should tests. If two different parts are described the same way, then they are the same and the same step definition can be used in both cases. If you have described two different parts of the system with the exact same wording, then you have an issue with ambiguity. Take some time and understand why you describe different parts of a system the same way. Chances are that they actually are the same thing and that you have some refactoring to do.
A Quick Example
When looking at a test to retrieve a quote from a website, a test statement should look like
When request a rate quote for 54481 for 1 month out
And not
Given I visit the page www.deltadentalcoversme.com
When I enter 54481 in the zip field
And I click GetQuote
And I select 1 month out
And I click GetQuote
Then I see a rate quote
What is important about the difference in styles? The first example, When request a rate quote for 54481 for 1 month out, is a functional requirement. The second, much longer, example is a procedural reference. Functional requirements are features and procedures belong in the implementation details. You should concern yourselves with that which has to happen and not how you expect it to happen. That way when somebody later decides to change the rate workflow, or add another interface, you then simply change the implementation of the steps behind the scenes.
Fixing our Previous Cucumber Tests
When trying to write tests thinking about the what and not the how, it helps to envision the system as if you were performing black box testing. Determine the initial setup you require, look at the inputs you have, ignore what specifically happens to the data, and then again look at the expected output. Let’s take the example of some of our previous login tests.
Feature: Testing for login page
Scenario: Login without password
Given I want to use the browser Firefox
When I type testuser1 in the username input field
And I click the login button
Then I see the login error message "Please provide a password."
And I am on the login page
Scenario: Login with bad password
Given I want to use the browser Firefox
When I type testuser1 in the username input field
When I type testuser2 in the password input field
And I click the login button
Then I see the login error message "The password provided does not match the username entered."
And I am on the login page
Scenario Outline: Successful login
Given I want to use the browser \[browser\]
When I type testuser1 in the username input field
When I type testuser1 in the password input field
And I click the login button
Then I am on the launcher page
Examples:
| browser |
| Firefox |
| Chrome |
| InternetExplorer |
Let’s pull out our interested data into the below table
| Test Case | Prerequisites | Inputs | Outputs |
|---|---|---|---|
| No password | Browser=Firefox | username=testuser1 | URL=login, error message=Please provide a password. |
| Bad password | Browser=Firefox | username=testuser1, password=testuser2 | URL=login, error message=The password provided does not match the username entered |
| Successful | Browser=[variable] | username=testuser1, password=testuser1 | URL=launcher |
Using this data, let’s rebuild our test cases more generically
Feature: Testing for login page
Scenario: Login without password
Given I want to use the browser Firefox
When I set the username to testuser1
When I login to CosmicComix
Then I see the error message "Please provide a password."
And I am on the login page
Scenario: Login with bad password
Given I want to use the browser Firefox
When I set the username to testuser1
And I set the password to testuser2
When I login to CosmicComix
Then I see the error message "The password provided does not match the username entered."
And I am on the login page
Scenario Outline: Successful login
Given I want to use the browser \[browser\]
When I set the username to testuser1
And I set the password to testuser1
When I login to CosmicComix
Then I am on the launcher page
Examples:
| browser |
| Firefox |
| Chrome |
| InternetExplorer |
We can see how these test steps are more generic guidelines for what to do to login to any system, and not at this level specific to our Cosmic Comix application design. If I changed the layout, or fields that were present on the login page, these tests would not have to change, just the back-end step controlling these test steps. For this small subset of three tests, that means changing one method, instead of changing three. The more tests we have, obviously the less rework will be required. Additionally, a user with little information about how our system behaves can understand these tests with more ease than the previous ones.
