Compatibility Testing – Local Simulation Tricks For Mobile Web Applications

Introduction

After giving a talk last week with Sauce Labs about compatibility testing for mobile web apps, I got several questions about simulation using Selenium Webdriver. I had written back in June about compatibility testing using Selenium IDE, but the talk expanded on this concept, allowing tests run in client languages to perform similar actions. For those interested, here are some written down steps to follow for setting up your machine to locally simulate other browsers. Alternatively, this setup could be used on a CI machine for simulation, before going into the cloud, performing this testing on emulators.

Workflow

My overall workflow is as follows below, when writing tests. There is no need to start with Selenium IDE, but as I’ve noticed that the majority of tests are not strong coders, it can be helpful to workout some of the larger kinks by starting in Selenium IDE. If you are comfortable enough working with Selenium Webdriver, then I would suggest skipping it entirely (as Webdriver offers more functionality), and go directly to my second step.

1. Start in Selenium IDE
   1. Write/Record tests in Selenium IDE
   2. Execute tests in Selenium IDE
   3. Note issues, debug, and fix
   4. Resize browser to smaller window
   5. Re-run tests in Selenium IDE
   6. Note issues, debug, and fix
   7. Set user-agent switcher to mobile device
   8. Re-run tests in Selenium IDE
   9. Note issues, debug, and fix
   10. Export tests to language of choice
2. Test from IDE
   1. Execute tests in IDE
   2. Note issues, debug, and fix
   3. Resize browser to smaller window
   4. Re-run tests in IDE
   5. Note issues, debug, and fix
   6. Set to profile with user-agent switcher set for mobile device
   7. Re-run tests in IDE
   8. Note issues, debug, and fix
3. Test in the Cloud
   1. Emulators can be more simply used than simulators
   2. Utilize frameworks already established for cloud provider

This is ideal for getting your same test to run on multiple devices, at different sizes. Ideally, we will have the same test run all multiple devices. This can’t always be accomplished however. Sometimes you want different workflows to exists for mobile vs desktop web applications. Sometimes, different information should be displayed on different devices, and this is as designed. When this is case, be sure to note this specifically in the tests, and setup your code to run different tests on different devices. I have found that different test framework have better or worse support for differing parameterized testing.

As I mentioned previously, I have covered manipulating the browsers with Selenium IDE to cover different user-agent switchers and screen sizes, but doing this in Selenium Webdriver is not as strait-forward. I will be using Java for the below examples, but these same processes can be applied to any language that Selenium supports. So, how can we accomplish this simply in Webdriver?

Varying Screen Size

In my @Before class, I like to setup my screen size. By doing it this way, I only need to change the screen size once, and all tests will then be executed at that new size. This is a simple two line command (or could even be combined into one), that looks like this:

@Before
public void setUp() throws Exception {
    //Resize the current window to the given dimension
    Dimension d = new Dimension(420,600);
    this.driver.manage().window().setSize(d);
}

This particular snippet of code sets the screen size to a width of 420 and height of 600. I would suggest looking up common devices, and setting your screen size accordingly. You can change these numbers, and re-run the test multiple times, to vary that the tests run without issue at multiple sizes.

Varying User Agent

This is definitely a bit trickier than changing the screen size, and takes some initial setup as well. I will cover how to do this using a Firefox browser, as I have found this simplest. First steps are to ensure that Firefox is installed with a version that is compatible with the version of Selenium Webdriver that you are running. As of today’s date, Firefox 40 works fine with Selenium Webdriver 2.47.

Next, you need to create a new Firefox profile that you can use/run for these automated tests. Open a command prompt on your system, and launch Firefox using the -p option. It will look something like

firefox.exe -p

or

firefox -p

depending on your operating system. You will want to create a new profile, and un-check the use selected profile without asking at startup option. I like to name my profiles something explicit and descriptive, something like IPHONE or ANDROID. Launch this new profile, and install a user-agent switcher on it. Any switcher will work just fine. Now activate the UA switcher on that browser.

Finally, all we need to do is modify the profile that we want to run over within our Selenium Webdriver code. Similar to setting the screen size, I like to do this in my @Before class, that way I can do it once, and it will apply to all of my tests. This, again, is a simple three line command (or could even be combined into one), that looks like this:

@Before
public void setUp() throws Exception {
    //Load our local iphone profile for firefox
    ProfilesIni profile = new ProfilesIni();
    FirefoxProfile ffprofile = profile.getProfile("IPHONE");
    this.driver = new FirefoxDriver(ffprofile);
}

This particular snippet of code grabs the new Firefox profile that I had setup and named IPHONE, and that I had set the UA to simulate an iphone device. Depending on the UA switcher installed, I have had varying success with getting this to run consistently, without initially loading the profile in a Firefox browser, and setting the UA to an iphone device. Similarly to switching screen sizes, it’s nice to run this over multiple devices, maybe iphones, androids, windows devices, and maybe more.

Making Things More Efficient

All of the test steps I have put in are helpful for debugging, but what is you want to start automating these, and running them consistently on your machine? Well, then we don’t want to be changing our devices constantly, we want something nicely setup that will simulate our tests running over a multitude of devices. To do that, I would actually use parameterization to pass in multiple devices to my tests, instead of looking at one at a time. The below code is again in Java, using the JUnit testing framework, but there is no reason not to use these same techniques for other languages or frameworks.

The first snippet from my LocalSimulationTest class is setting up our variables that we will needing to resize our device, and set our user agent switcher for simulation.

/**
 * The {@link device} name that we will be simulating
 */
 private String device;

/**
 * The {@link width} of the device that we will be simulating
 */
 private String width;

/**
 * The {@link height} of the device that we will be simulating
 */
 private String height;

/**
 * The Firefox {@link profile} that we will be loading for the simulating of the device
 */
 private String profile;
 
 /**
  * Constructs a new instance of the test. The constructor requires four
  * string parameters, which represent the device, device size, and browser
  * with the user agent switcher. 
  * @param device
  * @param width
  * @param height
  * @param profile
  */
 public LocalSimulationTest(String device, String width, String height, String profile) {
     super();
     this.device = device;
     this.width = width;
     this.height = height;
     this.profile = profile;
 }

Since we’re using JUnit for our parameterization, we need to setup each of these variables we want to pass in via our constructor. Once we have the class properly setup, we can look at setting up our actual parameters. We can start adding in different devices to be testing over.

/**
 * @return a LinkedList containing arrays representing the browser
 * combinations the test should be run against. The values in the array are used 
 * as part of the invocation of the test constructor
 */
@SuppressWarnings({ "rawtypes", "unchecked" })
@ConcurrentParameterized.Parameters
public static LinkedList browsers() {
     LinkedList browsers = new LinkedList();
     browsers.add(new String[]{"desktop", "1920", "1080", "DESKTOP"});
     browsers.add(new String[]{"iphone4", "320", "480", "IPHONE"});
     browsers.add(new String[]{"iphone6p", "414", "736", "IPHONE"});
     browsers.add(new String[]{"galaxys4", "480", "640", "ANDROID"});
    return browsers;
}

We can add many more, but there probably isn’t a point in testing over ALL devices, we want a good range of sizes, and some Android, iOS, and desktop. Depending on our consumer, we might want to windows phones, or blackberry devices. One thing that is glaringly lacking is our tablets, and so we should definitely add in some Android and iOS devices with larger screen sizes. Finally, we want to add in all of this information to our @Before method. This will look very similar to what we saw in the previous examples, but this time, instead of hard coding in the different device information, we will pull it from our parameters,

/**
 * Constructs a new {@link FirefoxDriver} instance which is configured to use 
 * the capabilities defined by the {@link #device}. {@link #width}, {@link 
 * #height} and {@link #profile} instance variables will determine how to 
 * configure our simulator to have everything run smoothly locally.
 * @throws Exception if an error occurs during the creation of the {@link
 * FirefoxDriver} instance.
 */
@Before
public void setUp() throws Exception {
    //Load our local profile for firefox browser
    ProfilesIni profileIni = new ProfilesIni();
    FirefoxProfile ffprofile = profileIni.getProfile(profile);
    this.driver = new FirefoxDriver(ffprofile);
    //Resize the current window to the given dimension
    Dimension d = new Dimension(Integer.valueOf(width),Integer.valueOf(height));
    this.driver.manage().window().setSize(d);
}

The profile, width, and height variables are obtained from our passed in parameters. In this way, we can simply run the same tests over multiple devices. Remember, in order to use these new Firefox profiles, ensure you have one instance of the profile opened up before you launch these tests, and that the browser thas the UA set to what you desire for that profile.

Conclusion

That’s it! Good luck incorporating in all of these changes to your current code base to expand out your testing being executed. As always, remember to drop some comments or questions if you have them!