Tool Review: Selenium

Overview

Technique:

Selenium (every tool of the Selenium suite) interacts with the ui elements of a web site/application using JavaScript injection. Therefore a custom proxy server is used to inject the JavaScript code into the html source. When using the Selenium IDE this magic is done automatically. If not, Selenium RC is used to manipulate the html source to be automate-able. This method has some disadvantages:

• Injecting Code into the html source of the web site/application to test could lead to unpredictable behaviour (even though this never occurred during our tests)
• When injecting JavaScript only one base-URL is possible (for example http://selenium.org/) because of The Same Origin Policy. Testing different base-URLs requires different test cases.

Documentation:

The tools of Selenium are very well documented. A user guide and a complete documentation is can be found here. A full functional reference is included in the Selenium IDE.

Support (Community/Commercial):

Community support is available through the offical user group, an own IRC channel (#selenium at irc.freenode.net) and a bug tracker. Furthermore some companies are listed to provide commercial support.

License:

Selenium is available under the Apache License 2.0

Selenium IDE

Installation:

The start into automated browser tests is easy using the Firefox (version 2 and higher) add-on Selenium IDE. The installation is similar to other add-ons and works almost automatically, just use the download link and allow Firefox to install the add-on.

After restarting Firefox the Selenium IDE add-on is fully installed and ready to run. To use the add-on you can either start the Selenium IDE in a popup using Tools/Selenium IDE (recommended) or in the sidebar using View/Sidebar/Selenium IDE. When using the Selenium IDE in the sidebar it is not possible to record user interaction in other browser windows. So if the web site/web application uses popups you have to start the Selenium IDE as a popup.

Handling & Usability:

After opening the Selenium IDE (either as sidebar or as popup) the next step is to create a new test case, using the capture engine. Just enter the base-URL (for example www.google.com) into the field under the file menu. Then click the red "capture" button and navigate to that URL using the navigation bar of Firefox. Enter "Selenium" in the search field and click on the "Google-Search" button. After the search results are displayed, click on the link "Selenium web application testing system". Finally stop the recording process by clicking the capture button again. The result should look like this:

A look at the recored steps shows what selenium was able to capture:

• open "/": opens the base-URL
• type "q" "Selenium": types the value "Selenium" into the field which is identified by "q" (the search fields value of the tag name is "q")
• click "btnG": clicks the "Google Search" button
• click "link=Selenium web application testing system" clicks on the specified link

If you run the test by clicking the "Play current test case" button     , the test will probably fail because the Selenium IDE is faster than google displays the results. After clicking on buttons/links/... which reloads the current site, you have to use the command clickAndWait. This command synchronizes automatically whith the browser so, Selenium IDE waits automatically until the site has been loaded before it continues with the next step. After changing both click commands to clickAndWait commands (by selecting the desired step and in the table and changing command using the drop down list) the Selenium IDE is able to execute the test case without problems. This behaviour is already well documented on the Selenium documentation site

Predefined Funcionality:

The Selenium IDE is not able to read or write external files or open popups for user interaction. Selenium IDE is only focused on browser interaction (goBack, maximize,...), element interaction (click, type, fireevent,...) and element verification (assertVisible, verifyValue,...). Synchronization gets done automatically via clickAndWait or different waitForXXX functions.

Data-Driven-Testing:

Since the Selenium IDE is very limited in its possibilities according to its functionality beside the browser interaction, dynamic testing like Data-Driven-Testing is not possible.

Compatibility:

The Selenium IDE is available for Firefox on every platform, so if you can run Firefox, you are able to run the Selenium IDE.

Conclusion - Selenium IDE:

Selenium IDE is an easy and quick solution to create an run browser tests inside Mozilla Firefox. The functionality is limited but covers the most important requirements.

Selenium RC

Installation:

The installation of Selenium RC is more complex. The ZIP-File contains client drivers for several programming languages (.Net, java, perl, php, python and ruby) and the Selenium Server (a simple Java jar file, Java 1.5 needed). To start the server, extract and navigate to the "selenium-server-1.X.X" directory and type "java -jar selenium-server.jar". Now you can comminucate via one of the client drivers with the server running on port 4444.

Handling & Usability:

Selenium RC does not provide any help to create test cases. Therefore you have to write them on your own, using one of the programming languages and the appropriate client drivers. Another further possibility is to capture tests using the Selenium IDE. Captured tests can be exported to scripts using every supported programming language. So you can create at least test case sceletons using the Selenium IDE. For example exporting a captured test case to Java creates a complete Unit Test which is ready to execute. Selenium RC only acts as the missing link between the programming language and the browser. Therefore you have to take care of the framework to run the tests on your own. In case of Java you could use Eclipse to create and run the Java Unit Tests. Regardless which programming language you have chosen, you have to integrate the client driver somehow. The client driver adds the Selenium interaction functionality (functions to interact with browser elements) as well as the functionality to communicate with the Selenium Server. 

With Selenium RC you can choose the browser to tests. For example in this test "*firefox" tells Selenium RC to use Firefox (*iexplore is for using Internet Explorer, furthermore with the *custom keyword you could use any browser you want specified by the location of its executable). When running the test, Selenium opens to browser windows, one with the application under test (in this case www.google.com) and one with a test overview. When the test is finished, the browser quits automatically and the result is shown as red or green depending on the status of the result.

Predefined Functionality:

Selenium RC offers the same functionality as Selenium IDE but with the ability to use the whole functionality of the current client programming language, so you can create mighty tests.

Data-Driven-Testing:

Again Selenium RC on its own is not able to run dynamic tests. If used with the client programming language it is possible to implement Data-Driven Testing. Furthermore some self made wrappers for FitNesse are available.

Compatibility:

Selenium RC runs on every platform supporting Java 1.5+. With its wide support for programming languages it fits into most existing environments. Also the great support for almost every JavaScript compatible browser is quite almost unique.

Conclusion - Selenium RC:

If tests need to run in different browsers, every night or every build, Selenium RC is the right product. The setup of Selenium RC is more complex than the simple installation of the Firefox add-on Selenium IDE, but you get the capabilities to create more complex and dynamic tests. To use Selenium RC you have to be familiar with at least one of the programming languages of a supported client driver.

Important links

• Selenium home
• Selenium documentation
• Selenium downloads (IDE V1.0.7, RC V1.0.3)
• Cubic Test home (Selenium Eclipse Plugin)