Testing CoffeeScript apps with Selenium

Published: 2011-10-25

This blog entry describes how to write functional tests for CoffeeScript web apps using Selenium. By way of example I’ll describe how I added Selenium tests to the Routeless Backbone Contacts tutorial app and automated it’s execution with a Cakefile.

The source is on Github, the tests described in this post are at the 3.0 tag.

Unit testing frameworks such as QUnit work well for testing libraries, APIs and computational logic but fall short when it comes to testing user interface intensive browser applications, this is where Selenium comes in. The Selenium IDE provides an easy way to functionally test an application user interface in a real browser with exactly the same mouse and keyboard input that a user would enter.

At it’s core “Selenium automates browsers” with a macro-like scripting language called Selenese and a Firefox IDE for recording and playing Selenese scripts. Selenium tests the actual user interface so you record your tests after the user interface has been implemented.

The primary purpose of functional user interface tests is to catch future user interface regressions. The likelihood of user interface regressions at each release increases with the size of the application.

Selenese

Selenese is the Selenium IDE scripting language, it is surprisingly powerful:

  • Variables can be stored and manipulated.
  • You can run JavaScript snippets with the runScript and waitForCondition commands.
  • The Selenese native format is an HTML table but it can also be exported to languages with Selenium binding (Python, Java, C#).

Requisites

To use the Selenium IDE you will need to install:

This post is based on Firefox 7.0.1 (Ubuntu) with Selenium IDE 1.3.0.


Creating a test

Creating a Selenese test script is a two step process:

  1. Record a sequence of user interface interactions (mouse and keyboard inputs).
  2. Add asserts and verifications to the test sequence. This is the step that transforms the browser automation sequence into a test.

The Selenium IDE documentation also explains how to run, save and edit tests.

Here’s the test for the tutorial application.

Note

To run the tests you will need to open the application in a web server (file based URLs won’t work). I use the Mongoose web server for app testing.

Mongoose web server

The Mongoose web server is a neat little configurationless cross-platform web server that’s ideal for testing web apps.

To install Mongoose on Linux:

  1. Unpack and make the source distribution:
    tar -xzf mongoose-3.0.tgz
    cd mongoose
    make linux
  2. Copy the mongoose executable to somewhere in your PATH.

To use:

  1. Change to your app directory.
  2. Run mongoose:
    mongoose [-i <index-files>]
  3. Open your browser at localhost:8080, if an index file exists it will be displayed else you’ll be presented with a file browser interface.

Running tests from the command-line

There’s no easy way to open the Selenium IDE and get it to execute a test from the command-line (at least I couldn’t find one), but you can export the tests to Python (or Java or C#) and then run them from the command-line. Apparently a JavaScript driver is in the works but not here yet, so I opted for Python.

I like being able to run Python tests from the command-line but I prefer developing the tests using the IDE. Keeping both in sync can be a challenge but being able to copy commands from the Selenium IDE to the Python source is a real help (set the IDE Options→Clipboard Format→Python 2 (WebDriver) option and you will be able to copy-and-paste IDE commands as Python code).

  1. In addition to the Selenium IDE you will need to install the Python bindings for Selenium:
    sudo pip install selenium
  2. Now export the test from the Selenium IDE to a Python test file using the IDE File→Export Test Case As→Python 2 (WebDriver) menu command.

The exported Python scripts make use of the Python unit testing framework.

Export anomalies and omissions

The export of Selenese to Python is not perfect, here are some of the manual edits I had to do:

  • Replace:
    # ERROR: Caught exception [ERROR: Unsupported command [getConfirmation]]

    with this:

    driver.switch_to_alert().accept()

    See also the Selenium FAQ.

  • Moved the focus from the search box to fire the search box change event (used click command as focus is not supported by Python WebDriver API):
    driver.find_element_by_css_selector("input.search").send_keys("sm")
    driver.find_element_by_name("name").click()   # Fire search 'change' event.

    See Dynamic UI controls below.

Here’s the Python test for the tutorial.

Dynamic user interface controls

Incremental search boxes and auto-completing drop-down lists typically emit keystroke events (up/down/press) to dynamically update other elements in response to user keyboard entry. The Selenese type command will not work in this situation.

  • The typeKeys command is designed for these situations but I was unable to get it to work.
  • Nor did the fireEvent command on keyup work.

The only reliable work-around I could find for testing the incremental Search was to add the change event to the search box and use:

driver.find_element_by_css_selector("input.search").send_keys("sm")
driver.find_element_by_name("name").click()

The click command moves the focus from the search box to trigger a search box DOM change event.


Setup and Teardown

At the start of testing you will typically create test fixtures to put the application in a known state. For data oriented apps this means means populating the database with fixed data, the Routeless Backbone Contacts application has built-in Clear All and Import Data commands that serve this purpose.

Built-in data fixture commands are not normally part of an application but there’s no reason not to implement hidden or debug only fixture commands. Alternatively, if you have moved exclusively to Python scripts for Selenium testing then, you could setup and teardown fixtures in the traditional fashion using Python unittest class and module fixtures.


Cakefile automation

The Routeless Backbone Contacts application is written in CoffeeScript so it made sense to automate the tests with a Cakefile. Because cake runs under Node.js you have a wealth of modules available.

The tutorial Cakefile implements two tasks:

cake server
Start the Mongoose server.
cake test
Run the Python Selenium test.

Cakefile

{exec,spawn} = require 'child_process'

task 'server', 'Start mongoose server', ->
    console.log 'starting mongoose web server...'
    server = spawn 'mongoose', [],
        customFds: [process.stdin, process.stdout, process.stderr]
    process.exit()

task 'test', 'Run tests', ->
    console.log 'starting mongoose web server...'
    server = spawn 'mongoose'
    console.log 'starting tests...'
    exec 'python test/test.py', (err, stdout, stderr) ->
        throw err if err
        console.log stdout + stderr
        server.kill()   # and exit cake.
Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: