Running Selenium Tests with Maven

One of the most important things in software testing is integrating tests with the build tool that your project uses. Developers need to be able to run your tests easily, otherwise, they’re probably not going to run them. Another reason for integrating tests is that it encourages clearly defining your project’s build process. In the case of Maven, each step is associated with a goal, which comes from the default lifecycle. Integration tests should be run after your project is built and deployed to ensure it functions as intended. The ‘integration-test’ phase of the default lifecycle is executed after your project is compiled, tested, and packaged. This is where Selenium tests can be run to ensure that the project is working as expected.
This guide is best paired with the SecureCI Test Framework, written by Max Saperstone. You can read more about the framework here

To have Maven run your Selenium tests, in the build settings, include the following for Maven to know where to compile your tests from.

 <testSourceDirectory> src/test/java/ </testSourceDirectory>

Anything in the ‘src/test/java’ directory will be compiled to /target/test-classes, check for this folder after compiling to ensure your path is correct.

Assuming that unit tests are already being run with the Maven surefire plugin, be sure to exclude your Selenium tests by name with the following in the Maven surefire configuration settings. Of course, make sure to follow these naming conventions for your Selenium tests to be excluded.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.18.1</version>
    <configuration>
        <excludes>
          <exclude>**/*UI.java</exclude>
          <exclude>**/*API.java</exclude>
        </excludes>
    </configuration>
</plugin>

Integration tests should be run with the Maven failsafe plugin, instead of surefire. The difference is that the surefire plugin will fail the build on any test failure and failsafe will continue with the build, showing failure after completion. If you’re going to be running other integration tests that don’t use Selenium, you’ll want to use profiles to allow two different failsafe configurations. To use the Maven failsafe plugin, include the following in the build plugins

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-failsafe-plugin</artifactId>
  <version>2.8.1</version>
  <configuration>
    <suiteXmlFiles>
      <suiteXmlFile>selenium.xml</suiteXmlFile>
    </suiteXmlFiles>
  </configuration>
  <executions>
    <execution>
      <id>integration-test</id>
      <goals>
        <goal>integration-test</goal>
      </goals>
    </execution>
    <execution>
      <id>verify</id>
      <goals>
        <goal>verify</goal>
      </goals>
    </execution>
  </executions>
</plugin>

There are a few different ways to specify which tests to run, I chose to use a TestNG suite XML file, which looks like this:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="MRCOE Selenium Suite" parallel="methods" thread-count="20" verbose="3" >
  <test name="MRCOE Selenium Tests" >
    <packages>
      <package name="com.bytecubed.mrcoe" />
    </packages>
 </test>
</suite>

The package name tag should be set to the package name of your tests, optionally you can specify single test methods or classes.

Before you can run your tests, the dependencies should be uploaded to Nexus or your choice of artifact repository manager. The dependencies look like this in the pom.xml:

<dependencies>
  <dependency>
    <groupId>org.seleniumhq.selenium</groupId>
    <artifactId>selenium-java</artifactId>
    <version>2.52.0</version>
    <scope>test</scope>
  </dependency>
  <dependency>
    <groupId>org.testng</groupId>
    <artifactId>testng</artifactId>
    <version>6.8</version>
    <scope>test</scope>
  </dependency>
  <dependency>
    <groupId>secureci-testing-framework</groupId>
    <artifactId>ssl</artifactId>
    <version>1.0</version>
    <scope>test</scope>
  </dependency>
  <dependency>
    <groupId>secureci-testing-framework</groupId>
    <artifactId>selenium-java</artifactId>
    <version>2.52.0</version>
    <scope>test</scope>
  </dependency>
  <dependency>
    <groupId>secureci-testing-framework</groupId>
    <artifactId>stf</artifactId>
    <version>1.0.1</version>
    <scope>test</scope>
  </dependency>
  <dependency>
    <groupId>secureci-testing-framework</groupId>
    <artifactId>jackson-all</artifactId>
    <version>1.9.0</version>
    <scope>test</scope>
  </dependency>
  <dependency>
    <groupId>secureci-testing-framework</groupId>
    <artifactId>selenium-server-standalone</artifactId>
    <scope>test</scope>
    <version>2.52.0</version>
  </dependency>
</dependencies>

To run the tests, use the verify goal like so

mvn clean verify -Dbrowser=Firefox -DappURL=http://example.com

The -Dbrowser option is used to specify the browser, but if none is included HtmlUnit is used by default. Next, the -DappURL option is used to provide the test site to STF. Assuming you’ll be running API tests as well as UI tests with STF, I found it easiest to use different groups to separate the tests. The -Dgroups option can be set to a single test group or any number of test groups separated by commas.