Tagging Basics
Cucumber provides a simple method to organize features and scenarios by user determined classifications. This is implemented using the convention that any space delimited string found in a feature file that is prefaced with the commercial at (@) symbol is considered a tag. Any string may be used as a tag and any scenario or entire feature can have multiple tags associated with it. Be aware that tags are heritable within Feature files.
- Scenarios inherit tags from the Feature statement
- Examples inherit tags from the Feature and Scenario statements
Utilizing Tagging
If multiple databases or environments are expected to be exercised with Cucumber tests, tagging is an excellent way to make reuse of test code. It’s easiest and simplest to design each feature file (think test suite) with at least one specific data-environment in mind. If multiple data-environments share common functionality, design the tests to run over as many as possible. The purpose is to indicate valid tests for each of the desired sets of data or environments. Each feature file should then be tagged with the intended data-environment. Including these tags utilizes Cucumber’s built in feature for narrowing down tests, and if the Java code can also be setup to switch data-environments based on the tag. If scenarios in the feature file are specific to different data-environments, then it should be considered to break them into different features. If not broken into features, then include tagging as the scenario level, instead of the feature level.
In addition to indicating tags for data-environments, tags should also be utilized for testing types. Scenario Outlines contain data tables to run over, and it’s an effective use of these tables to divide the tests run into Smoke and Regression tests. Create smaller subsets of the data for a Smoke test, and tag the larger tables as Regression. Additional testing types can also be rolling into this structure, with little effect on the tests. Consider adding Acceptance or Integration tests.
Now let’s take a look at our previous cucumber tests, and roll in some useful tagging. In our below example, we have two different environments to test over: our CosmicComix.org website, and our CosmicComix.net website, each with slightly different functionality. Our login test applies to both sites, and so we apply tags for both at the Feature level. Additionally, we want to indicate which of our test run for the full regression, and which is appropriate for a smoke test.
@CCOrg @CCNet
Feature: Testing for login page
Scenario Outline: Bad login
Given I want to use the browser Firefox
When I set the username to [username]
When I set the password to [password]
When I login to CosmicComix
Then I see the error message "[message]"
And I am on the login page
@Regression
Examples:
| username | password | message |
| testuser1 | | Please provide a password. |
| | testuser1 | Please provide a username. |
| testuser | testuser | That username does not match anything in our records. |
| testuser1 | testuser2 | The password provided does not match the username entered. |
Scenario Outline: Successful login
Given I want to use the browser [browser]
When I set the username to testuser1
And I set the password to testuser1
When I login to CosmicComix
Then I am on the launcher page
@Regression
Examples:
| browser |
| Firefox |
| Chrome |
| InternetExplorer |
@Smoke
Examples:
| browser |
| Firefox |
In the above example, if we ran providing the tag @Smoke we would only run the Successful login test for the Firefox browser. If instead we ran providing the tag @Regression we would instead run all of the Bad login tests, and the Successful login test over all three browsers. Providing no tag would run all tests, both Smoke and Regression. Alternatively a tilde (~) could be added before the desired tag to negate the tag. E.g. if we ran with the tag ~@Regression all tests NOT tagged as Regression would be executed.
Java Code Tie In
As we discussed previously, we can run our Cucumber tests with a multitude of runners. If we use something like Ant as our JUnit runner, we can simply pass in our tags, and then further access them from the java code. We just need to add a few arguments under our java run target. For example:
…
<arg value="--tags"/>
<arg value="${dataenvironment}"/>
<arg value="--tags"/>
<arg value="${testtype}"/>
<sysproperty key="dataenvironment" value="${dataenvironment}"/>
…
Passing the dataenvironment as a tag allows Cucumber to recognize it as a tag to be run against. Additionally you’ll notice the –tags value in their twice. That is to ensure we get the proper logic selected. This means we’ll run anything tagged as dataenvironment AND testtype. If we had placed them in the same tag, we’d be running anything tagged as dataenvironment OR testtype. Finally, by adding the value as a sysproperty, our Java code can access the value, and make the desired changes in the code based on the database being run against. To access this value, setup your JUnit framework to read in this value before any of the tests execute as below.
@Before //performed before anything is done
public void setup() throws SAXException, IOException {
if( System.getProperty( "dataenvironment") != null ) {
database = Databases.valueOf( System.getProperty( "dataenvironment" ).substring(1) );
}
}
And then later determine how to connect to the specific data-environment within the database code set.
Finally, to properly execute this code, you just kick off Ant with the below command.
ant -Dtesttype=@Smoke -Ddataenvironment=@CCOrg
