Apr 18, 2013

Multi-Browser JavaScript Unit Testing with Sauce

[banner id=”java”]
“The frontend code”. It’s often prefixed with “A fix to the…” and suffixed with “… due to a bug in Internet Explorer”. Why is this? JavaScript unit testing is easy due to a plethora of testing frameworks. But how do we catch bugs in our JavaScript code due to browser differences? It’s too manual of a process to open the tests in every browser before each commit.

Enter Sauce, which provides Remove WebDriver Browsers as a Service, or RWDBaaS. I may have just made up that acronym, but I am not making up this service. It’s a great way to run automated Selenium-based tests against a large number of browsers. But automated cross-browser unit tests? Well, that’s just one step removed and I would like to show you that.

Jasmine unit tests

Managing Expectations

Like any good software project, let’s first define what this article is NOT about:

With that out of the way, let’s move on to the real meat of this thought experiment: using Sauce to run our JavaScript tests on a large variety of browsers.

The full source code for this article can be found in the sauce-unit-test-example Github repository.

Step 1: Write your unit test

Here is a simple Jasmine test to verify the splice() function works as expected:

describe("JavaScript Array", function() {
  it("should be able to splice", function() {
    myArray = ['a', 'b', 'c'];


Not very complicated. Let’s move on.

Step 2: Write your browser test

We need a way to quickly verify that the unit tests passed. Let’s use Geb:

class JasmineSpec extends GebReportingSpec {
    def "all Jasmine specs should pass"() {
        go '/SpecRunner.html'

        title == 'Jasmine Spec Runner'

This is also quite simple – verify that the page title is “Jasmine Spec Runner” and that all the tests passed.

Here is a sample GebConfig you could use with this:

baseUrl = 'http://localhost:8080'

sauceUsername = System.getProperty('sauce.username')
sauceApiKey = System.getProperty('sauce.apiKey')

driver = { new FirefoxDriver() }
environments {

    'windows-ie-8' {

        driver = { 
            DesiredCapabilities capabilities = DesiredCapabilities.internetExplorer()

            new RemoteWebDriver(new URL("http://${sauceUsername}:${sauceApiKey}@ondemand.saucelabs.com:80/wd/hub"), capabilities) 

    'windows-ie-9' {

        driver = { 
            DesiredCapabilities capabilities = DesiredCapabilities.internetExplorer()
            capabilities.setCapability('platform', 'Windows 2008')

            new RemoteWebDriver(new URL("http://${sauceUsername}:${sauceApiKey}@ondemand.saucelabs.com:80/wd/hub"), capabilities) 


This allows us to run the tests using a local Firefox browser, or optionally using a specific version of IE on Sauce.

Step 3: Setup the build environment
We will be using Gradle to start up a local server and run the Geb tests. Here is what I whipped up:

apply plugin: "groovy"
apply plugin: "idea"
apply plugin: "war"
apply plugin: "jetty"

buildscript {
    repositories {
        maven {
            url "https://repository-saucelabs.forge.cloudbees.com/release"
    dependencies {
        classpath "com.saucelabs:sauce-connect:3.0.24"
        classpath("com.saucelabs:ci-sauce:1.29") {
            transitive = false

        classpath "commons-io:commons-io:1.4"
        classpath "commons-lang:commons-lang:2.6"

def sauce = [
    username: System.getProperty('sauce.username'),
    apiKey: System.getProperty('sauce.apiKey')

jettyRun.contextPath = '/'
jettyRun.webAppSourceDirectory = new File('src/test/javascript')
stopPort = 8090
stopKey = '123'

repositories {

dependencies {
    groovy "org.codehaus.groovy:groovy-all:2.1.1"

    testCompile "org.gebish:geb-spock:0.9.0-RC-1"
    testCompile "org.spockframework:spock-core:0.7-groovy-2.0"

    testCompile "org.seleniumhq.selenium:selenium-firefox-driver:2.26.0"

task gebTest(type: Test) {
    testReportDir = reporting.file("$name/tests")
    testResultsDir = file("$buildDir/test-results/$name")

    systemProperty 'geb.build.reportsDir', reporting.file("$name/geb")
    systemProperty 'geb.env', System.getProperty('geb.env')
    systemProperty 'sauce.username', sauce.username
    systemProperty 'sauce.apiKey', sauce.apiKey

task test(overwrite: true, dependsOn: [tasks['gebTest']])

task wrapper(type: Wrapper) {
    gradleVersion = '1.4'

gebTest.doFirst {
  jettyRun.daemon = true

  if(sauce.username && sauce.apiKey) {
    sauce.connect = new com.saucelabs.ci.sauceconnect.SauceConnectTwoManager()
    sauce.connect.openConnection(sauce.username, sauce.apiKey, 4445, null, null, null, null)

gebTest.doLast {
  sauce.connect?.closeTunnelsForPlan(sauce.username, null)

This Gradle file does three main things:

  1. Launch a server running locally.
  2. Run Geb tests using either Firefox or another browser, specified using the geb.env property.
  3. Set up a secure tunnel to Sauce, using Sauce Connect. This is the secret sauce that allows the web browser running on a remote virtual machine to access your local web server.

Step 4: Run the tests

To run the tests with Firefox:
./gradlew clean test

This should briefly open Firefox on your machine, run the Jasmine specifications, and verify that everything passed. You should see “BUILD SUCCESSFUL” on the command line.

That’s great, but what about other browsers (Internet Explorer)? Go get yourself a free account on Sauce, and copy down your API key. Then run:
./gradlew clean test -Dgeb.env=windows-ie-8 -Dsauce.username=your_username -Dsauce.apiKey=your_api_key

With any luck, you should see BUILD FAILED on the command line. Congratulations – Sauce just caught my rookie JavaScript mistake!


Sauce tests in Jenkins

In this example, we wrote a simple Jasmine specification, wrote a Geb test to verify that the Jasmine tests passed, used Sauce Connect to establish a secure tunnel with Sauce, and then used Sauce itself as a remote WebDriver to run the tests against Internet Explorer.

Where could you go from here?

  • Add more browsers to GebConfig.
  • Set up a Jenkins job to run the tests against a large number of browsers.

I have an example Jenkins job, also in the sauce-unit-test-example Github repo. Go clone the repository and try it out for yourself! Good luck!

About the Author

David Norton profile.

David Norton

Director, Platform Engineering

Passionate about continuous delivery, cloud-native architecture, DevOps, and test-driven development.Passionate about continuous delivery, cloud-native architecture, DevOps, and test-driven development.

  • Experienced in cloud infrastructure technologies such as Terraform, Kubernetes, Docker, AWS, and GCP.
  • Background heavy in enterprise JVM technologies such as Groovy, Spring, Spock, Gradle, JPA, Jenkins.
  • Focus on platform transformation, continuous delivery, building agile teams and high-scale applications.
Leave a Reply

Your email address will not be published. Required fields are marked *

Related Blog Posts
Android Development for iOS Developers
Android development has greatly improved since the early days. Maybe you tried it out when Android development was done in Eclipse, emulators were slow and buggy, and Java was the required language. Things have changed […]
Add a custom object to your Liquibase diff
Adding a custom object to your liquibase diff is a pretty simple two step process. Create an implementation of DatabaseObject Create an implementation of SnapshotGenerator In my case I wanted to add tracking of Stored […]
Keeping Secrets Out of Terraform State
There are many instances where you will want to create resources via Terraform with secrets that you just don’t want anyone to see. These could be IAM credentials, certificates, RDS DB credentials, etc. One problem […]
Validating Terraform Plans using Open Policy Agent
When developing infrastructure as code using terraform, it can be difficult to test and validate changes without executing the code against a real environment. The feedback loop between writing a line of code and understanding […]