Introduction

As a software tester with Java experience, generally working with Java applications, the release of the cucumber-jvm really excited me. I have been thinking about trying out Behavioral Driven Development for some time, and this tool (along with a new project) finally gave me the push I needed to try it out. I find myself often having to interpret acceptance criteria into workflows and user stories, and being able to write tests directly using these criteria is not only a huge time saver, but also allows for an easier distribution of work, along with more dispersed knowledge of how the tests will run, and what they are running in the automation realm.

Unfortunately there are a lack of sources that have all the information in one place regarding how to build a project from scratch using these tools. After scouring many corners of the interwebs and getting feedback from several forums, I was able to build a testing framework using cucumber-jvm, and get it running in a continuous integration environment.

This is the first post of a 3 4 part sequence that will take you through each step in detail. Specifically, we will be building a set of tests using Selenium Webdriver, getting those tests to run from Cucumber features, discussing some best practices for coding up the tests, and finally how to run those tests both locally, and in Hudson.

This post will go through the process of setting up the Cucumber testing framework, integrating Selenium Webdriver into it, and writing a few stories. For this example, I will be using Eclipse as my editing tool, however any IDE will suffice, and even using something as simple as notepad will work.

The first step is to setup our project. This can be done in Eclipse by adding a new Java Project, or by just creating a new folder in your desired working location.

Jar Setup

After our project is setup, we want to ensure we have the appropriate jars. For me this took a lot of digging and research, so I’ll simply spell it out below what and detail what each one is, what it is used for, and where to find it. A list of all associated jars for cucumber can also be found here. A folder entitled ‘lib’ should be created in the project directory, and all of the below files should be added to this folder. Additionally, I suggest you grab the source and java doc files, as they will provide more insight into running and debugging your tests when problems arise.

Jar Description Location
cucumber-core-XXX.jar In addition to all of the base (Ruby) runtime features, this module contains all the story terminology utilized from writing scenarios and implemented in the defining methods http://repo1.maven.org/maven2/info/cukes/cucumber-core/1.1.1/
cucumber-html-XXX.jar This provides a nice output in html format upon test completion. The stories are matched to the jUnit results, pulling in any associated errors http://repo1.maven.org/maven2/info/cukes/cucumber-html/0.2.2/
cucumber-java-XXX.jar This provides the Java runtime environment for all tests http://repo1.maven.org/maven2/info/cukes/cucumber-java/1.1.1/
cucumber-junit.XXX.jar This module contains a customized jUnit framework for cucumber http://repo1.maven.org/maven2/info/cukes/cucumber-junit/1.1.1/
junit-4.10.jar This module contains the basic Java jUnit framework http://sourceforge.net/projects/junit/files/latest/download
gherkin-2.11.5.jar gherkin is the language cucumber’s feature stories are written in, and this module contains parsing instructions for the features http://repo1.maven.org/maven2/info/cukes/gherkin/2.11.5/
gherkin-jvm-deps-1.0.2.jar This module provides the Java parser for gherkin, and a direct hook for the java runtime for scenario outlines http://repo1.maven.org/maven2/info/cukes/gherkin-jvm-deps/
selenium-server-2.31.0.jar This module provides drivers for multiple different browsers, and the runtime management for them https://github.com/samtingleff/jchronic
selenium-server-standalone-2.31.0.jar This module provides the specific functionalities for running selenium commands, and how to interact with our browser drivers http://selenium.googlecode.com/files/selenium-server-standalone-2.31.0.jar

Once all of these are downloaded, restart Eclipse, and refresh the project. This restart should automagically add all of the jars to your build path.

We now have all of our initial setup completed. Gathering everything together is the most challenging part, the rest is simply getting familiar with the gherkin, regular expressions, and the cucumber framework.

Package Setup

To start off our coding, we’ll need a src folder. If using Eclipse, this folder was automatically created when you started a new Java project. Under our src folder, we’ll need to create a directory, so that Java has a package to associate with. This is important, as the cucumber runtime won’t recognize tests or features without this package.

The test application we’ll be running against is called ‘Cosmic Comix’ so let’s create two folders, following general Java convention. Under the src folder create the folder ‘Comix,’ and under that one, create the folder ‘Cosmic.’ Here we will create the three files needed to run our initial tests.

Generic Test Runner

The first file we’ll be looking at creating is our generic cucumber runner. This is the file that runs. It interprets the feature stories, calls our test definitions, and prepares the reports. Under the Cosmic folder, create a new Java file titled ‘GenericTest.java.’ This needs to look like below, and you’ll notice lots of comments dictating what each line is used for.

	package comix.cosmic;		//this is our package declaration

	//we need to specific which framework to use. By default junit built for cucumber is a simple way to start
	import cucumber.api.junit.Cucumber;
	//as with most junit frameworks, we need to specify the junit runner
	import org.junit.runner.RunWith;

	//this is a cucumber annotation dictating the cucumber runner
	@RunWith(Cucumber.class)
	//this is how we want the results formatted. The only customizable line, different formats can be added or removed. I’ve included the most common ones for convenience
	@Cucumber.Options(format = {"pretty", "html:target/cucumber-html-report", "json-pretty:target/cucumber-report.json"})
	//this is an empty class to run with. This needs to remain empty
	public class GenericTest {
	}

The above file shouldn’t be changed too much. It is standard for running any cucumber tests. It doesn’t matter what the file is named, but must be a java file.

Feature Files

The second file we will start looking at is our feature file. The test application mentioned previously has a login page for users. We want to add some simple tests to examine this functionality. Under the Cosmic folder, create a new Java file titled ‘Login.features.’ This file obviously has lots of room for changes. I have left comments out of this file, but will add some comments below.

	Feature: Testing for login page

	Scenario: Login without password

		Given I want to use the browser Firefox
		When I type testuser1 in the username input field
		And I click the login button
		Then I see the login error message "Please provide a password."
		And I am on the login page

	Scenario: Login without username

		Given I want to use the browser Firefox
		When I type testuser1 in the password input field
		And I click the login button
		Then I see the login error message "Please provide a username."
		And I am on the login page

	Scenario: Login with bad username

		Given I want to use the browser Firefox
		When I type testuser in the username input field
		When I type testuser in the password input field
		And I click the login button
		Then I see the login error message "That username does not match anything in our records."
		And I am on the login page

	Scenario: Login with bad password

		Given I want to use the browser Firefox
		When I type testuser1 in the username input field
		When I type testuser2 in the password input field
		And I click the login button
		Then I see the login error message "The password provided does not match the username entered."
		And I am on the login page

	Scenario Outline: Successful login

		Given I want to use the browser [browser]
		When I type testuser1 in the username input field
		When I type testuser1 in the password input field
		And I click the login button
		Then I am on the launcher page

		Examples:
				|	    browser	 |
				|	    Firefox	 |
				|	    Chrome       |
				|     InternetExplorer   |

It doesn’t matter what the file is named, but must have a ‘feature’ extension. Additionally these scenarios can be broken into multiple feature files; all that is required is that they are in the same folder as the test runner.

Now for some explanation for the above code. Each feature file is written in gherkin and contains multiple scenarios. Each scenario can be treated as a story. Cucumber scenarios follow the format of acceptance criteria; Givens, Whens, Thens. Cucumber also allows for Ands, which simply repeats the above statement type. These statements should be written so that they are generic enough to allow multiple steps to use the same statement with different values, but specific enough to get the desired outcome from the step.

Additionally gherkin allows for scenarios to be written passing in multiple inputs in one scenario. This is defined as an Outline, and Examples should be listed below. Following the example above, if multiple values want to run using only one test, place the desired input location(s) in square ([]) or angled (<>) brackets and then put the desired inputs in a table labeled as Examples.

Test Implementation

Finally we can move onto our last file, the implementation of each of the steps in the feature files done in Java. Let’s create a new file under the same folder, and call it ‘Tests.java.’ This file will contain all of our implementations for our tests. This file can take on almost any form, but the basic structure should be preserved. You’ll notice lots of comments indicating what each line is used for.

	package comix.cosmic;

	import static junit.framework.Assert.assertEquals;
	import static junit.framework.Assert.assertTrue;

	import java.util.HashMap;

	import org.openqa.selenium.By;
	import org.openqa.selenium.WebDriver;
	import org.openqa.selenium.WebElement;
	import org.openqa.selenium.android.AndroidDriver;
	import org.openqa.selenium.chrome.ChromeDriver;
	import org.openqa.selenium.firefox.FirefoxDriver;
	import org.openqa.selenium.ie.InternetExplorerDriver;
	import org.openqa.selenium.interactions.Actions;
	import org.openqa.selenium.iphone.IPhoneDriver;
	import org.openqa.selenium.safari.SafariDriver;

	import cucumber.api.java.After;
	import cucumber.api.java.Before;
	import cucumber.api.java.en.Given;
	import cucumber.api.java.en.Then;
	import cucumber.api.java.en.When;

	public class Tests {
		//these are the different browsers we are willing to run against
		public enum Browsers		{ Firefox, Chrome, InternetExplorer, Android, Ipad, Iphone, Opera, Safari };
		//this hashmap will keep our users that are active in the system
		private HashMap<String,String> 		users = new HashMap<String,String>();
		//this is our selenium webdriver controlling our browsers
		private WebDriver		driver;

		@Before		//any steps we want to perform before we start our tests
		public void setup() {
			//initializing our system by adding our users
			users.put("testuser1","testuser1");
			users.put("testuser2","testuser2");
			users.put("testuser3","testuser3");
		}

		@After		//any steps we want to perform after our tests
		public void cleanUp() {
			//close our browser, and finalize our driver instance
			driver.quit();
		}

		//our statement for choosing a browser to test in
		@Given("^I want to use the browser (.*)$")
		public void chooseBrowser(Browsers browser) throws Exception {
			//instantiate a new browser based on the choice of browsers
			switch ( browser ) {
				case Firefox:				{ driver = new FirefoxDriver(); 		break; }
				case Chrome:				{ driver = new ChromeDriver();			break; }
				case InternetExplorer:			{ driver = new InternetExplorerDriver(); 	break; }
				case Android:				{ driver = new AndroidDriver();			break; }
				case Iphone:				{ driver = new IPhoneDriver();			break; }
				case Safari:				{ driver = new SafariDriver();			break; }
				default:				{ throw new Exception();	}
			}
			//open our test site's URL
			driver.get( "http://cosmiccomix.appspot.com/index.html" );
		}

		//which user have we already logged in as
		@Given("^I have logged in as (.*)$")
		public void loginAs(String user) throws Exception {
			//webdrivers select element by id functionality
			By byElement = By.id("username");
			//locate our element
			WebElement webElement = driver.findElement( byElement );
			//setup an action
			Actions selAction = new Actions(driver);
			//send keys to the element selected
			selAction.sendKeys( webElement, user ).perform();
			//webdrivers select element By.id functionality
			byElement = By.id("password");
			//locate our element
			webElement = driver.findElement( byElement );
			//setup an action
			Actions selAction = new Actions(driver);
			//send keys to the element selected
			selAction.sendKeys( webElement, password ).perform();
			//webdrivers select element by id functionality
			byElement = By.id("login");
			//locate our element
			webElement = driver.findElement( byElement );
			//setup an action
			Actions selAction = new Actions(driver);
			//click the element selected
			selAction.click( webElement ).perform();
		}

		//////////////////////////////////
		// Login Definitions
		//////////////////////////////////

		//type in our username
		@When("^I type (.*) in the username input field$")
		public void enterUsername(String user) throws Exception {
			//webdrivers select element by id functionality
			By byElement = By.id("username");
			//locate our element
			WebElement webElement = driver.findElement( byElement );
			//setup an action
			Actions selAction = new Actions(driver);
			//send keys to the element selected
			selAction.sendKeys( webElement, user ).perform();
		}

		//type in our password
		@When("^I type (.*) in the password input field$")
		public void enterPassword(String password) throws Exception {
			//webdrivers select element by id functionality
			By byElement = By.id("password");
			//locate our element
			WebElement webElement = driver.findElement( byElement );
			//setup an action
			Actions selAction = new Actions(driver);
			//send keys to the element selected
			selAction.sendKeys( webElement, password ).perform();
		}

		//click the login button
		@When("^I click the login button$")
		public void clickLogin() throws Exception {
			//webdrivers select element by id functionality
			By byElement = By.id("login");
			//locate our element
			WebElement webElement = driver.findElement( byElement );
			//setup an action
			Actions selAction = new Actions(driver);
			//click the element selected
			selAction.click( webElement ).perform();
		}

		//check our error messages
		@Then("^I see the login error message \"(.*)\"$")
		public void checkLoginErrorMessage(String errorMessage) throws Exception {
			//webdrivers select element by id functionality
			By byElement = By.id("overError");
			WebElement errorElement = null;
			//wait for up to 5 seconds for our error message
			long end = System.currentTimeMillis() + 5000;
			while (System.currentTimeMillis() < end) {
				errorElement = driver.findElement( byElement );
				// If results have been returned, the results are displayed in a drop down.
				if (!errorElement.getText().equals("")) {
					break;
				}
			}
			//ensure we got our expected error message
			assertEquals( errorMessage, errorElement.getText() );
			//if we have a bad username
			if ( errorMessage.contains( "username" ) ) {
				byElement = By.id("userError");
				errorElement = driver.findElement( byElement );
				//ensure username is marked as the problem
				assertEquals( "*", errorElement.getText() );
			}
			//if we got a bad password
			if ( errorMessage.contains( "password" ) ) {
				byElement = By.id("passError");
				errorElement = driver.findElement( byElement );
				//ensure password is marked as the problem
				assertEquals(  "*", errorElement.getText() );
			}
		}

		//check the page we are on
		@Then("^I am on the (.*) page$")
		public void checkPage(String page) throws Exception {
			String title = null;	//the page title
			String url = null;		//the page url
			if ( page.equalsIgnoreCase( "login" ) ) {	//settings for the login page
				title = "Login To Cosmic Comics";
				url = "index.html";		
			}
			if ( page.equalsIgnoreCase( "launcher" ) ) {//settings for the launcher page
				title = "Choose A Comic To View";
				url = "launcher.html";
			}
			//ensure we have the expected title
			assertEquals( title, driver.getTitle() );
			//ensure we are on the correct page
			assertTrue( driver.getCurrentUrl().endsWith( url ) );
		}
	}

You’ll notice in the above code that all of the selenium, cucumber, and junit assertions are rolled into one file. This is done mainly for simplicity in this posting, as you’ll see lots of repeated code, and little error checking. In the following post, we’ll dive deeper into this page and discuss some best practices of coding.

Running In Eclipse

The final step of this is to get it all running. For this post, we will just use Eclipse’s debugging and running tools. To run this series of tests, open the ‘GenericTest.java’ file. From the Run menu, choose either Run, or Debug. You should shortly see some Firefox windows opening up, running a test, then closing after about 30 seconds. When these tests have completed, navigate to the ‘target’ folder, and browser through all of the different generated test results. The next post, along with best practices, will also go into interpreting results, and how best to go through building/designing these tests. As mentioned above, Selenium is sometimes tricky to get working perfectly, especially on dynamic webpages, or those that use ajax.

The last post of the series will give details on how to run this both from the command line (on any local machine), and how to set this up in a continuous integration environment, specifically utilizing our open source tool SecureCI.

9 thoughts to “Cucumber-JVM Setup

  • Pingback: Cucumber-Webdriver Best Practices » Coveros

  • Pingback: Running Cucumber-JVM Locally » Coveros

  • Pingback: Writing Effective Cucumber Tests » Coveros

  • Avatar
    lakshmipathy

    hi i am starting working on cucumber with java , i have one doubt , i will be having many feature file , for these feature files i will be having implemented java files and configuration files, this is my one application but for whole this application i should maintain one browser instance it should not close until the application close and it should be accessible by all implemented java files i tried but i am facing little difficulty can you please help me in this issue

    Reply
  • Max Saperstone
    Max Saperstone

    To do that, you will want to look more into hooks available to cucumber, and ensure that your tests all have access to the driver element. If you have multiple classes implementing tests (as best practice dictates), then ensure all of those tests extend a base test where you define your driver.
    To start a browser before any of your tests are run, try using the hack I suggested below, to start your browser. To kill your browser, do something similar with the @after annotation, keeping track of the number of tests run before executing driver.quit().
    http://www.coveros.com/background-and-hooks-for-cucumber-jvm/

    Reply
  • Avatar
    Sunil

    Hi in my project we have around feature files, to complete entire test suite execution its taking around 15hours, to optimize the execution time, we are planning to split our features files in to two suites, Suite 1 execution planning from 1 machine, Suite 2 from 2 machine

    Here my question is how to splite the features:
    In cucumber runner file how to specify suite 1 have to consider x, y, z features and rest of a, b, c, features from suite 2

    Reply
    • Max Saperstone
      Max Saperstone

      Hi Sunil,
      How are your feature files broken up? If x, y, and z are all different feature files, you could try differentiating with a tag. Since tags are heritable, I would place an @Suite1 tag at the top of all your files you want for Suite 1, and an @Suite2 tag at the top of all of your Suite 2 feature files. Then in your runner, specify the suite you want to run with the tag. I discuss tagging here. Also, depending on how you kick off your tests, you may want to consider passing in this value from the command line via ant or maven. More information about passing in tags via ant can be found on this post

      Reply
      • Avatar
        suneel

        Max,
        what your are saying its clear to me. the issue what i have is, i have complete test suite package with me.
        splitted all our feature files in to 2 suites, ie., (Suite1 and Suite 2). Each suite have to run on different machines. Suite1 – Machine1 and Suite -2 Machine2. Entire test pacakge is available in both the machines.

        Here the question is in cucumbermain.java how to call Suite1 have to run only on machine1 and Suite -2 have to run on only Machine2 only, i dont want to do any manual changes after taking the test suite package from CI.

        Reply
  • Pingback: Running Cucumber with Maven - Coveros

Leave a comment

Your email address will not be published. Required fields are marked *

X