17 KiB
UI Tests with Selenium
[[TOC]]
Quick Start: How to run tests locally in Eclipse
There are two ways to run the Selenium tests locally on your computer. Either, you compile the GWT UI using our build script and run the tests based on the compiled UI, or you run the tests using the GWT hosted mode.
Firefox Prerequisites
You have to ensure that your Firefox browser has a profile called "Selenium" and that in this profile the latest version of the GWT plugin is installed. To ensure this, launch the server by choosing the "Sailing Server (Proxy)" or "Sailing Server (No Proxy)" launch config. Then, run the "SailingGWT" launch to start the GWT UI in hosted / development mode. Afterwards you can launch Firefox from the command line with the -p option. On Windows machines, you can do this by pressing the Windows key, then typing "firefox.exe -p". In the profile manager create a profile called Selenium and start Firefow with that profile. Hit the entry page of the AdminConsole by entering http://127.0.0.1:8888/gwt/AdminConsole.html?gwt.codesvr=127.0.0.1:9997 into the address bar. This will ask you to install the GWT plugin into your Selenium profile. When done, exit the browser. You may use the profile manager again to set your default profile to your original profile.
Running the tests with GWT hosted mode
Launch the server by choosing the "Sailing Server (Proxy, winddbTest)" or "Sailing Server (No Proxy, winddbTest)" launch config. Then, run the "SailingGWT" launch to start the GWT UI in hosted / development mode.
When the GWT development mode has finished its initialization as indicated by the "Development Mode" view showing the entry point URLs, launch the "com.sap.sailing.senelium.test (Proxy, GWT Codesvr)" or "com.sap.sailing.senelium.test (No Proxy, GWT Codesvr)" launch. This will then pop up Firefox windows using the "Selenium" profile and run the tests.
Running the tests after a successful local GWT compile
Stop any SailingServers currently running on your local machine (if you fail to do so, the Maven build will be unable to write some of the build artifacts in the com.sap.sailing.gwt.ui project). Build using the git-managed build script from configuration/buildAndUpdateProduct.sh. To lower round-trip times, we introduced the -b option that only builds one GWT permuation (Chrome, English). You may use the -t option to keep the Maven build from running the tests. If inside the SAP VPN, additionally use the -p option to ensure the HTTP proxy is being used. This makes for a command line like this:
./configuration/buildAndUpdateProduct.sh -p -b -t build
After the build has completed (hopefully successfully), refresh the com.sap.sailing.gwt.ui project in your Eclipse workspace. Start the SailingServer launch configuration (with / without proxy, depending on your VPN state). Then, launch the JUnit launch configuration com.sap.sailing.selenium.test (with / without proxy). It will run the Selenium tests against your local server, running on port 8888. Make sure you have a "Selenium" firefox profile installed, see above.
General
To validate the web application from the user perspective we use a combination consisting of Selenium and JUnit, which allows us to simulate user interactions via a real browser instance. Therefore we provide a special JUnit-runner as well as different classes in the package com.sap.sailing.selenium.core to execute tests, using the Selenium WebDriver API, and to keep the execution of the tests as flexible as possible at the same time.
To avoid duplicated and error prone code in test cases, we prefer the usage of the Page Object design pattern when writing automated UI tests. A page object is an object-oriented class that serves as an interface to the UI. The tests then use the methods of the page object class whenever they need to interact with the UI. The benefit is that if the UI changes, the tests themselves don’t need to be changed. Only the code within the page object needs to be changed. Subsequently all changes to support the new UI are located in one place.
Selenium Runner
The Selenium runner is used to execute tests, using the Selenium WebDriver API. The runner takes a XML configuration file, which has to be provided via the system property selenium.test.environment.configuration, and runs the tests for each defined browser. The configuration file is needed to define the browsers with which the tests should be executed as well as to provide all the information to Selenium regarding to the web drivers. To avoid unnecessary errors we also provide a documented XML Schema for the configuration file.
At runtime, the instance of the web driver along with additional information from the configuration is encapsulated by the interface TestEnvironment. The Selenium runner provides an instance to the tests by injecting it to fields of the correspondent type, annotated with Managed. With the class AbstractSeleniumTest we provide a skeletal implementation for tests, which already has all the necessary annotations applied. Furthermore this class provides a mechanism to automatically capture a screenshot in the case that a test fails.
Additional Classes
Since GWT applications are heavily AJAX driven, one of the challenges in performing UI tests is to be able to tell when the application is in a state you expect. In simple cases you can wait until a loading animation disappears or until a well-known element has changed the state (e.g. visibility). However, since things are usually not that simple, especially in the context of GWT, it is more stable if we are able to tell exactly when an asynchronous request has finished. For this reason we use a counter for pending request which is incremented if an asynchronous callback is created and decremented when the callback completes, regardless of a success or a failure. With this counter a test which triggers a request (which can cause additional request) can wait until the counter reaches 0 again.
The necessary JavaScript code for the counter is automatically injected in the host page by the provided class AbstractEntryPoint. With the class MarkedAsyncCallback we provide an abstract implementation of the AsyncCallback interface for asynchrony requests, which should be marked as pending until they complete using the introduced counter. Since the counter is incremented as soon as an instance is created, MarkedAsyncCallback is an object of single-use. Therefore it is strictly forbidden to create an instance of this class which is never used or which is used multiple times. Otherwise the counter will have an unpredictable value.
Page Object Pattern
Within a web app's UI there are areas that tests interact with. A page object simply models these as objects within the test code and abstracts from the actual structure (DOM) of a website. This reduces the amount of duplicated code and means that if the UI changes, the fix need only be applied in one place.
Page objects can be thought of as facing in two directions simultaneously. Facing towards the developer of a test, they represent the services offered by a particular page. Facing away from the developer, they should be the only thing that has a deep knowledge of the structure of a page (or part of a page). It's simplest to think of the methods on a page object as offering the "services" that a page offers rather than exposing the details and mechanics of the page.
Because the developer of a test should think about the services that they're interacting with rather than the implementation, page objects should seldom expose the underlying web driver or elements. To facilitate this, methods on a page object should return other page objects.
Since we prefer the usage of the Page Object design pattern when writing automated UI tests, we provide a base class PageObject as well as the two specializations HostPage and PageArea. The class HostPage represents a GWT entry point which waits for the completion of the GWT bootstrap process before it is initialized. The PageArea is used to represent a component or a specific area on a page, which is useful to improve maintainability for complex websites and reusable parts.
There is a lot of flexibility in how the page objects may be designed, but there are a few basic rules for getting the desired maintainability of the test code. Page objects themselves should never be make verifications or assertions. This is part of the test and should always be within the test’s code, never in a page object. The page object will contain the representation of the page, and the services the page provides via methods but no code related to what is being tested should be within the page object. There is one, single, verification which can, and should, be within the page object and that is to verify that the page or the area, and possibly critical elements, were loaded correctly. This verification should be done while instantiating the page object.
To make the development of page objects as simple and easy as possible, we also support a factory for this pattern which helps to remove some boiler-plate code from the page objects by using annotations. We provide the annotations FindBy and FindBys for fields on a page object to specify a mechanism for locating an element or a list of elements. The factory uses the annotations to lazily locate and wait for the element or the element list to appear, by polling the UI on a regular basis, and initializes the field with the element or the list of elements.
Since we use GWT to build the UI, we also provide a specific mechanism to locate elements by the value of the GWT debug identifier. By convention we use the attribute selenium-id for the debug identifier which is implemented by the class AbstractEntryPoint. The class BySeleniumId defines the corresponding mechanism and locates an element or a list of elements by the value of the selenium-id attribute.
Writing new Tests
If you start to write new UI tests, some preparations are needed, depending on what you work on. The first thing you should verify is, that the used entry point extends the provided class AbstractEntryPoint, since this class automatically injects the JavaScript code for the counter of pending request into the host page. In addition it configures the attribute used for the debug identifier and marks the GWT bootstrap using the counter. Since the counter is initialized to be 1 and decremented as soon as the bootstrap process has finished, tests as well as page objects are able to wait until the bootstrap process completes.
In the next step you should set a debug identifier for all relevant widgets the user interacts with, by calling ensureDebugId(String id) on the instance of the widget. This will set the selenium-id attribute of the widget to the specified value, so you can easily look up it in the resulting HTML document. You should also consider setting a debug id on logical or self-contained parts, like a form or field sets. By convention we always use the type of the widget as a suffix (e.g. LoginButton or NameTextField) to give a hint to the available interactions.
After preparing the UI for testing, you have to implement new page objects or to extend existing ones, if needed. Therefore we provide the two classes HostPage and PageArea which also share common functionality. The class HostPage represents a whole page and in the context of GWT it is used for page objects representing an EntryPoint. The second class PageArea in contrast should be used to represent complex widgets (e.g. tables and tab panels) or logical parts like field sets. While implementing your page objects, you should always think in services instead of single widgets. So, for example you should provide a method login(String name, String password) for a simple login form, which sets the username, the password and clicks the login button instead of the 3 methods setName(String name), setPassword(String password) and login().
To get access to the HTML-elements representing the widgets the user interacts with, you should use the provided factory by creating instance fields of the type WebElement or List<WebElement> and annotating them with FindBy or FindBys as well as CacheLookup, if the element is static and never changes. Usually you use the annotation FindBy which needs you to provide the mechanism as well as the value to use for locating the element. Since we use the debug identifiers of GWT we also provide the corresponding mechanism with the class BySeleniumId and a typical example would look like following:
public MyPageObject extends PageArea {
@FindBy(how = BySeleniumId.class, using = ”NameTextField”)
private WebElement nameTextField;
...
}
Since GWT widgets are usually represented by more complex HTML constructs it is also recommended to implement a page object for the used widgets, if none exists. A CheckBox for example is modeled by the following HTML fragment:
<span class=”gwt-CheckBox” style=”white-space: nowrap”>
<input id=”gtw-uid-1” type=”checkbox” value=”on”/>
<label for=”gwt-uid-1”>Track Wind</label>
</span>
The corresponding page object should abstract from this by only providing methods to select and deselect the checkbox as well as to retrieve the selection state. The knowledge about the lookup of the input field should be hidden by the implementation. This allows a reuse in all tests and other page objects and keeps necessary changes in one place for the case GWT modifies the representation in later releases.
To write a test case you should extend the class AbstractSeleniumTest which has all necessary annotations already applied for running the test with the special JUnit-Runner and to have access to the TestEnvironment which provides the WebDriver instance as well as some other configuration settings. The concrete test is then written as usual, using the services of the page objects and making assertions.
Execution of Tests
For the execution of the tests you have several options. In either case you have to provide a configuration file according to your environment, since our tests are run by the special JUnit-Runner. You can use the file ci-test-environment.xml as a starting point for your own configuration. Within the configuration file you define the context root to specify the server and the path of the web application. Furthermore you have to list all browsers via driver-definition elements you want to test with.
The easiest way to execute the tests is to perform a full Maven build. Here you have to provide the configuration file in the property with the name parameters.integration-tests as a command line argument in your user settings.
<properties>
<parameters.integration-tests>
-Dselenium.test.environment.configuration=[path-to-your-file]
</parameters.integration-tests>
</properties>
After the build, Tycho will start a server instance and runs all tests against the deployed application.
Since a full Maven build needs some time, you can also execute the tests in the Eclipse IDE. Here you have to start a server manually via an appropriated launch configuration (e.g. SailingServer (Proxy, winddbTest)). From there you can either run all UI-Tests with the JUnit launch configuration com.sap.sailing.selenium.test (with/without proxy) which expects the configuration file under the name local-test-environment.xml or you can run a single test by selecting Run As -> JUnit Test for your test class and specifying the configuration file in the VM Arguments section of the run configuration.
For a more practical example of how to write page objects and test you should take a look at the tutorial.
Updating Selenium
While our build environment is stable regarding to the used browser version, this may not be the case in your development environment, where you have the newest browser version installed probably. The short release cycles of the browsers often bring changes in the implementation, which are incompatible with Selenium. Therefore you have to update the used Selenium version by performing the following steps to be able to run the tests local.
- Download the latest Client & WebDriver Bindings for Java from the official Selenium website
- Delete the selenium-java-.jar in the root directory of the project org.openqa.selenium.osgi as well as all libraries in the lib directory and copy the new versions from the downloaded file in the appropriate folders
- Open the MANIFEST.MF with the Plug-in Manifest Editor and switch to the Runtime tab
- Remove all the old libraries from the Classpath section and add all new versions
- Add all new packages that start with org.openqa.selenium to the Exported Packages section in the case there were packages added
- Updated the version number in the MANIFEST.MF and in the pom.xml