Archive for October, 2010

Writing Functional Tests on Groovy on Grails: Experiences from CollabNet Subversion Edge.

October 30, 2010 2 comments

I first wrote this technical document for the open-source project CollabNet Subversion Edge on how to design and implement Functional Tests using Groovy on Grails. Therefore, this documentation can also be reached at “

Introduction and Setup

The CollabNet Subversion Edge Functional Tests are based on the Groovy on Grails plugin “Functional Tests“. But, before you get started with them, make sure you have covered the following required steps:

Besides Unit and Integration tests used during development, the source-code under development already contains the functional tests plugin support and some developed classes, as shown in the Eclipse view “Project Explorer”. In the file system, the files are located at CSVN_DEV/test/functional, where the directory CSVN_DEV is where you have previously checked out the source-code. Those set of test cases are the last one being run in our internal Continuous Integration server (Hudson) and it’s usually a good place to find bugs related to the user-facing features during development.


Functional Tests Basics

This section covers the basics of the functional tests infranstruction on Subversion Edge and assumes you are already familiar with the Grails Functional Tests plugin documentation. The plugin is already installed in the Subversion Edge development project, as you can use the commands to run a functional test and visualize the test results and report. The test cases are run as RESTful calls to the controllers defined by the application, but it can also use a URL. For instance:

After the execution of an HTTP method wrapper such as “get()” or “post()”, any test case has the access to the response object “this.response” with the HTML code payload. Grails uses this object to execute any of the “assert*” methods documented.

Another important piece of configuration is the CSVN_DEV/grails-app/conf/Config.groovy. Althoug Grails grails uses the closure “environment.test”, SvnEdge uses the general closure “svnedge” during development and test phases and, therefore, values from configuration of that closure are accessable from the test classes.

Functional tests infrastructure

Considering you have your development infrastructure set up, you will find the current implementation of functional tests at the directory “CSVN_DEV/tests/functional”. Notice that the directory structure follows the Java convention for declaring packages, and has already been configured to be included as source-code directories in the current Eclipse configuration artifact “.classpath” on trunk.


We have created the following convention for defining the packages and functional test classes:

  • com.collabnet.svnedge

The package containing major abstract classes to embrace code reuse while aggregating reusable patterns throughout the entire Test infrastructure. The reusable utility methods were extracted during the first iteration of the development of the SvnEdge functional tests. For instance, the access to the configuration keys from “CSVN_DEV/grails-app/conf/Config.groovy” can be easily accessed from the test cases using the method “getConfig()” or just “config”. Similarly, the access to the i18n keys can be performed by calling “getMessage(‘key’)”, as the value of “key” is one of the keys located at the “”, which renders strings displayed in the User Interface. Note that the English version of the i18n messages are used in the functional tests. Moreover, the abstract classes have their own intent for the scenarios found on Subversion Edge functionalities:

  1. AdminLoggedInAbstractSvnEdgeFunctionalTests: test class that sets up the test case with the user “admin” already logged in (“/csvn/status/index”).
  2. LoggedOffAbstractSvnEdgeFunctionalTests: test class that starts the application in the index auth page where a user can login (“/csvn/auth/index”).
  • com.collabnet.svnedge.console

The test cases related to the web console, or Subversion Edge itself. Different components must have its own package. For instance, take the name of the controllers to as the name of the component to be tested such as “user” and “repo”, as they should have their own test packages as “com.collabnet.svnedge.console.ui.user” and “com.collabnet.svnedge.console.ui.repo”, respectively. The only classes implemented at this time are the login

  • com.collabnet.svnedge.teamforge

The test cases related to the teamforge integration. As you will see, there are only one abstract class and two functional classes covering the functional tests of the conversion process when the server has repositories to be imported (Full Conversion) and when the server does not have any repository created (Fresh Conversion). The latter case is a bit tricky as the SvnEdge environment defines a fresh conversion when its database does not have any repository defined. In this way, test cases related to repositories need to make sure to “discover” repositories if the intent is to verify the existence of repositories in the file-system.

Running Functional Tests

As described in Grails functional tests “mini bible”, the only thing needed to run a functional test case is the following command under the directory CSVN_DEV:

grails test-app -functional OPTIONAL_CLASS_NAME

The command will start the current version of Grails installed using the functional tests environment. If you don’t provide the optional parameter “OPTIONAL_CLASS_NAME”, grails executes all the functional tests defined. However, since the execution of all current implementation of test classes takes more than 10 minutes, use the complete name of the test class (package name + name of the class – sufix “Tests”). For instance, the following command executes the functional Tests implemented in the class LoginFunctionalTests:

grails test-app -functional com.collabnet.svnedge.console.ui.LoginFunctional

The command selects the test suite class “CSVN_DEV/tests/functional/com/collabnet/svnedge/console/ui/LoginFunctionalTests.groovy” to be executed, as the output of the execution of the test cases identify the environment and the location where the test reports will be saved. The recommendation here is to keep using the Eclipse STS infrastructure to save your commands execution as shown below.


As shown below, the functional tests execution output is the same from executing the tests using the command line or the Eclipse command as shown in the output view. The tests are prepared to be executed and save the output logs and reports in the directory “CSVN_DEV/target/test-reports”.

Welcome to Grails 1.3.4 -
Licensed under Apache Standard License 2.0
Grails home is set to: /u1/svnedge/replica_admin/grails/grails-1.3.4/

Base Directory: /u1/development/workspaces/collabnet/svnedge-1.3.4/console
Resolving dependencies...
Dependencies resolved in 1565ms.
Running script /u1/svnedge/replica_admin/grails/grails-1.3.4/scripts/TestApp.groovy
Environment set to test
    [mkdir] Created dir: /u1/development/workspaces/collabnet/svnedge-1.3.4/console/target/test-reports/html
    [mkdir] Created dir: /u1/development/workspaces/collabnet/svnedge-1.3.4/console/target/test-reports/plain

Starting functional test phase ...

Once the functional tests execution finishes the execution, the test reports are written and can be accessed using a web browser. The following snippet shows the result of running the test case started above, which shows how long it took Grails to execute the 4 test cases defined in theLoginFunctionalTests test suite, the stats of how many tests passed or failed, as well as the location where the test reports are located along with the final result of PASSED or FAILED. Note that the directory “target/test-reports” is relative to the directory “CSVN_DEV” as described above.

Tests Completed in 12654ms ...
Tests passed: 4
Tests failed: 0
2010-09-28 12:19:11,334 [main] INFO  /csvn  - Destroying Spring FrameworkServlet 'gsp'
2010-09-28 12:19:11,350 [main] INFO  bootstrap.BootStrap  - Releasing resources from the discovery service.
2010-09-28 12:19:11,350 [main] INFO  bootstrap.BootStrap  - Releasing resources from the Operating System service.
2010-09-28 12:19:11,352 [main] INFO  /csvn  - Destroying Spring FrameworkServlet 'grails'
Server stopped
[junitreport] Processing /u1/development/workspaces/collabnet/svnedge-1.3.4/console/target/test-reports/TESTS-TestSuites.xml
                  to /tmp/null1620273079
[junitreport] Loading stylesheet jar:file:/home/mdesales/.ivy2/cache/org.apache.ant/ant-junit/jars/ant-junit-1.7.1.jar
[junitreport] Transform time: 2339ms
[junitreport] Deleting: /tmp/null1620273079

Tests PASSED - view reports in target/test-reports
Application context shutting down...
Application context shutdown.

Accessing the Test Results Report

Once the execution terminates, you can have access to the test reports. This is where you will find all the answers to the test results, including the detailed information of the entire HTTP payload transmitted between SvnEdge server and the Browser emulator that the Functional Tests use. As shown below, the location of the test cases reports is highlighted as a hyper-link to the index page of the test reports. Clicking on it results on opening the Eclipe’s built-in browser view with the reports.


This report is generated per execution, and therefore, they are deleted before each new execution. In case you need keep information of a test run, copy the contents of the directory “CSVN_DEV/target/test-reports”, as you will find reports in both HTML and XML. The report for each test suite includes the list of each test case run, the status and time of execution. The report includes 3 main output:

  • Properties: system properties used.
  • System.out: the output of the standout output of the process; same output print in the grails output, but more organized.
  • System.err: the output of the standard error of the process.

The most used output is the System.out. Clicking on this hyper-link takes you to the organized output of the traffic, highlighting the HTTP Headers, HTTP Body, redirects, test assersions and test results.

Identifying Test cases report scope

The link to the System.out output is the most important and used throughout the development of the test case, as the output of the execution of each test case is displayed in this area.


Each test case has its own test result scope, and you can easily identify the initialization of the execution of a test case by the key “Output from TEST_CASE_NAME”, where “TEST_CASE_NAME” is the name of the method name that defines the test case. For instance, the log for the execution of the test cases for the LoginFunctionalTests includes the following strings:

--Output from testRootLogin--
--Output from testRegularLogin--
--Output from testDotsLogin--
--Output from testFailLogin--

The output of the execution of the HTTP Request Header of a test case is started with “>>>>>” shown as follows:

>>>>>>>>>>>>>>>>>>>> Making request to / using method GET >>>>>>>>>>>>>>>>>>>>
Initializing web request settings for http://localhost:8080/csvn/
Request parameters:
Request headers:
Accept-Language: en
Accept: */*

On the other hand, the output of the HTTP Response Header of a test case is started with “<<<<<<” as shown below. The HTTP Reponse Header Parameters are output for verification of anything used by the test cases. Note that following access to the “/csvn/” “root” context results in an HTTP Forward to the “Login Page” identified by the context “/csvn” controller “/login/auth” and, therefore, there is not “Content” available.

<<<<<<<<<<<<<<<<<<<< Received response from GET http://localhost:8080/csvn/ <<<<<<<<<<<<<<<<<<<<
Response was a redirect to
  http://localhost:8080/csvn/login/auth;jsessionid=hueqpw5eaq32 <<<<<<<<<<<<<<<<<<<<
Response was 302 'Found', headers:
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Set-Cookie: JSESSIONID=hueqpw5eaq32;Path=/csvn
Location: http://localhost:8080/csvn/login/auth;jsessionid=hueqpw5eaq32
Content-Length: 0
Server: Jetty(6.1.21)


#Following redirect to http://localhost:8080/csvn/login/auth;jsessionid=hueqpw5eaq32
>>>>>>>>>>>>>>>>>>>> Making request to http://localhost:8080/csvn/login/auth;jsessionid=hueqpw5eaq32
 using method GET >>>>>>>>>>>>>>>>>>>>

If the HTTP Response contains the body payload, it will be output as is in the Content section:

<<<<<<<<<<<<<<<<<<<< Received response from
  GET http://localhost:8080/csvn/login/auth;jsessionid=hueqpw5eaq32 <<<<<<<<<<<<<<<<<<<<
Response was 200 'OK', headers:
Expires: -1
Cache-Control: no-cache
max-age: Thu, 01 Jan 1970 00:00:00 GMT
Cache-Control: private
Content-Type: text/html; charset=utf-8
Content-Language: en
Content-Length: 4663
Server: Jetty(6.1.21)
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
<html xmlns="" lang="en" xml:lang="en">

    <title>CollabNet Subversion Edge Login</title>
    <link rel="stylesheet" href="/csvn/css/styles_new.css"
    <link rel="stylesheet" href="/csvn/css/svnedge.css"
    <link rel="shortcut icon"

Whenever a test case fails, the error message is output as follows:

"functionaltestplugin.FunctionalTestException: Expected content to loosely contain [] but it didn't"

Looking deeper in the raw output for the string “Expected content to loosely contain but it didn’t”, you see what HTML output was used for the evaluation of the test case. Sometimes an error case is related to the current UI or to an external test verification. This specific one is related to teamforge integration as the test server did not have a specific user named “” located in the list of users.

Failed: Expected content to loosely contain [] but it didn't

Writing New Functional Test Cases Suite

This section describes how to create test suites using the Functional Tests plugin. In order to maximize code-reuse, we defined a set of Abstract classes that can be used in specific type of tests as shown in the diagram below. Instead of each test case extend the regular class “functionaltestplugin.FunctionalTestCase“, we created a more general abstract class “AbstractSubversionEdgeFunctionalTests” to define general access to configuration artifact, internationalization (i18n) message keys, among others. In addition to the infrastructural utility methods, the main abstract SvnEdge test class contains a set of “often used” method executions such as “protected void login(username, password)”, which is responsible for trying to perform the login to SvnEdge for a given “username” and “password”. The result of the command can then be verified in the body of the implementing class. More details later in this section. First, any test will be implementing one of the test scenario classes: “AdminLoggedInAbstractSvnEdgeFunctionalTests” or “LoggedOutAbstractSvnEdgeFunctionalTests“. However, the test cases for the conversion process needed a specialized Abstract class “AbstractConversionFunctionalTests“, which is of type “AdminLoggedInAbstractSvnEdgeFunctionalTests” because only the admin user can perform the conversion process.


As it is shown in the UML Class Diagram above, the AbstractSvnEdgeFunctionalTests extends from the Grails Functional Test class. In this way, it will inherit all the basic method calls for assertions from JUnit and grails. The class is shown in RED because it is a “PROHIBITED” class. That is, no other classes but the GREEN ones should directly extend from the RED class. The fact is that the test cases implementation in Subversion Edge only has 2 different types of tests and, therefore, new test cases should only inherit from “AdminLoggedInAbstractSvnEdgeFunctionalTests” or “LoggedOutAbstractSvnEdgeFunctionalTests“. Similarly, additional functional tests to verify other scenarios from the conversion process has to inherit the behavior of the abstract class “AbstractConversionFunctionalTests“.

Basic Abstract Classes

As described in the previous sections, the two major types of test cases are related to when the Admin user is logged in and when there is no user logged in. That is, tests that require different users to login can use the latter test class to perform the login and navigate through the UI. Before continuing, It is important to note that the Functional Tests implementation are based on JUnit using the 3x methods name conversions. For instance, the methods “protected void setUp()” and “protected void tearDown()” are called before and after running each test case defined in a test class. Furthermore, it is also important to to call the super implementation of each of the methods because of the dependency on the Grails infrastructure. Take a look at the following JavaDocs to have an idea of the basic utility methods implemented on each of them.

Just as a reminder, upon executing the test cases defined in a class, JUnit executes the method “setUp()”. If any failure occurs in this test, Grails will fail not only the first test case, but ALL the test cases defined in the Test Suite. This is related to the fact that the method “setUp()” is executed before the execution of each test case. Once the execution of a given test case is finished, the execution of the method “tearDown()” is performed. Any failure on this method also results in ALL test cases to fail.

The test cases defined in the abstract classes are defined to give the implementing concrete classes the access to all the important features for the test cases. As mentioned earlier, utility methods to access the configuration properties and internationalization (i18n) messages are provided. In addition, convenient test cases for performing assertions are also implemented in the Abstract classes. The next sections will provide in-depth details in the implementation of the test suites.

Concrete Functional Tests Suites Implementation

The simplest implementation of Functional Tests is the LoginFunctionalTests used as an example before. However, executing the scenario to be implemented using the production version is the first recommended step before writing any piece of code. You need to collect information about the scenario to be executed, choose UI elements to use in your test case, etc. For instance, consider the execution of the Login scenario of a user with wrong username. By default, the development and test environments of Subversion Edge will be bootstrapped with different users such as “admin”, “user” and “”. Considering a scenario where the attempt to login with a wrong username called “marcello” is performed as the result is shown in the screenshot below:


The test case shows that by entering a wrong username and password, an error message is shown as the server responded with a complete and correct page (HTTP Response Code 200), although an error occurred during the execution of the test case. Based on those information, the automated tests can be written in the test suite to verify the possible test cases for the different users in SvnEdge, including the implementation of the wrong input. Note that the implementation of each test case have the procedures to be verified in the super class through the call to a method “testUserLogin” whereas the implementation of the testFailLogin() is the only implementation that is located in the LoginFunctionalTests. Other abstract and concrete test classes are shown in the UML Class diagram below. Note that the YELLOW classes are the concrete classes that extends the functionality from the abstract classes.


  • LoginFunctionalTests.html: The concret functional tests class suite that verify the login for each of the different usernames, as well as the failure tests.
package com.collabnet.svnedge.console.ui

import com.collabnet.svnedge.LoggedOutAbstractSvnEdgeFunctionalTests;

class LoginFunctionalTests extends LoggedOutAbstractSvnEdgeFunctionalTests {

    protected void setUp() {

    protected void tearDown() {

    void testRootLogin() {

    void testRegularLogin() {

    void testDotsLogin() {

    void testFailLogin() {
        this.login("marcello", "xyzt")
        assertContentContains getMessage("user.credential.incorrect",
            ["marcello"] as String[])

The fact is that the methods “loginAdmin()”, “loginUser()”, etc, are implemented in the AbstractSvnEdgeFunctionalTests to allow code reuse in other test classes, and therefore, the test case “testFailLogin()” uses the basic method “AbstractSvnEdgeFunctionalTests.login(username, password)” for the verification of a user that does not exist. Also, note that the verification of the login scenario is as simple as verifying if the a given String exists in the resulting HTTP Response output. For instance, when attempting to login with a user that does not exist, the error message “Wrong username/password provided for the user “marcello”. This is due to the fact that the String is located in the messages bundle “user.credential.incorrect” and the method “getMessage()” is the helper method implemented in the class AbstractSvnEdgeFunctionalTests.

Another important thing to keep in mind is about code convention. The name of test cases are defined as cameCase, prefixed by the keyword “test”. The name of the test cases can be as long as “AbstractConversionFunctionalTests.html“. The most important point here is that the name of the method must be coherient to the steps being performed. Also, note that Groovy accepts a more relaxed code notation, which makes it easy to read:

        // JAVA method invocation Notation
        this.login("marcello", "xyzt")

        // GROOVY method invocation Notation
        this.login "marcello", "xyzt"

When it comes to the real implementation of a given scenario, you have to constantly refer to the Grails Functional Tests documentation and that’s where you will find your “best friends”. Yes!!! Your best friends! The assert methods that will help you verify the results of the HTTP Response. But first, let’s take a look at the implementation of the basic method that performs “login” and “logout”. As we know from the definition of the abstract classes, each time a method from a class that extends “LoggedOutAbstractSvnEdgeFunctionalTests” is executed, the method setUp() inherited from this class is executed first.

public abstract class LoggedOutAbstractSvnEdgeFunctionalTests extends AbstractSvnEdgeFunctionalTests {

    protected void setUp() {
        //The web framework must be initialized.


        if (this.response.contentAsString.contains(
                getMessage(""))) {

    protected void tearDown() {
        //Stop Svn Server in case it is running


        //The tear down method terminates all the web-related objects, and
        //therefore, must be performed in the end of the operation.

Note that the implementation of the concrete classes MUST make a call to the super.setUp() first, so it executes the depending steps. As you can see in the class implementation below, the method setUp() will first make a request to “/”, that is, “http://localhost:8080/csvn/&#8221; since the RESTful method “get()” uses the base URL + the context name “/csvn”. Then, the first assertion is important to verify that the Server is up and running, as well as verifying that the request did not return any error in the UI. Bookmark the RFC2616 and use the HTTP Response Status Codes as required. The default one to verify is “200”, even though the scenario results in an error message as the test case “LoginFunctionalTests.testFailLogin()”. Finally, after verifying if the status code is as expected, the test uses the object “response” to verify if the HTML content contains the string identified by the key “” in the the i18n artifact “CSVN_DEV/grails-app/i18n/”. For this case, the method is verifying for the key:

Following the way JUnit implements the test execution cycle, the method “tearDown()” is executed right after each method “testXYZ()” is executed. In our case, there are a few steps to be verified before terminating the test case. As it might be necessary, the HTTP server might have been started during a test case, and therefore, the method “stopSvnServer()” is called. This is specially placed in the “highest” abstract class because all types of test cases might want to start the HTTP server from the status page. After an HTTP reques to “/” is performed, the verification to the output is performed to and in case is necessary, the method “logout()” is executed as implemented in the abstrac class “AbstractSvnEdgeFunctionalTests“. That is, if the HTML code from the response object contains the string identified by the key “” (LOGOUT), then click in the link “LOGOUT”. Then, assert if the HTTP response status was equals to 200 and that the content contains the header string “Login” identified by the key “”.

     * Performs the logout by clicking on the link.
    protected void logout() {
        def logout = getMessage("")
        if (this.response.contentAsString.contains(logout)) {
            click logout

Similarly, test cases that perform login will essentially fill out the login form and click on the button “Log in”. The basic implementation of the method “login(username, password) is shown below. The HTTP GET Request to the page “/login/auth” is performed followed by the assert of the status code. Then, if the test environment keeps the user Logged In as a result of a failure of any previous test case, the verification if the user is logged in is performed so that the call to the method “logout()”, as shown above, is performed. Finally, when the user s in the front page, the login form is filled out with the correct values. Please refer to the “Grails Functional Tests Documentation” for details on how to fill out and submit form fields, but it should be straightforward. The only detail needed is to capture the name of the form defined in the HTML code. A good helper way is to use Google Chrome or Firefox “Web Developer” plugin to capture the UI element “ids”. Specifically for the form submission, the ID of the form and the “id”s from the form fields are necessary. Then, the label value of the SUBMIT button is necessary, and as shown in the code below, that string is located in the string with the key “”.


    protected void login(username, password) {

        if (this.response.contentAsString.contains(
                getMessage(""))) {
        def login = getMessage("")
        form('loginForm') {
            j_username = username
            j_password = password
            click login

It is extremely important to note here a very hard problem when it comes to “clickable” items in the UI. Since we are using a mix of the Grails GSP tags and some CSS styles from TeamForge, Grails creates the buttons in a different way for Forms and Places without the HTML form entity. Whenever a form was generated by Grails, the Submit button like the “login” one showed above will only respond to the command “click LABEL” inside of the form() closure. On the other hand, the command “click LABEL” will only perform its action when declared outside of the form() closure. Different examples of these GOTCHAS have been found while the Conversion tests were being written.

To summarize the steps to automate manual tests with corresponding Functional Tests, the suggested steps are as follows:

  1. . Perform the test scenario manually and gather necessary information about the User Interface, choosing unique elements that are present in the resulting action. For the case of login, the verification of the string “Logged in as:” is perfomed. For tests exploring failures and error messages, choose to assert about the existence of these error messages.
  2. . Once you are familiar about how the scenario behaves, create the main Test Case Suite by extending from one of the GREEN abstrac classes in the UML Class Diagram shown above. Choose the names related to the component.
  3. . Propose code reuse by implementing new methods in the AbstractSvnEdgeFunctionalTests if necessary, or if other components will use the same implementation. If not, keep the implementation in the test class developed.
  4. . Add JavaDocs to the methods that are going to be inherited or are difficult to understand. Try documenting the method execution before writing the test case as you will understand the scope of the test better. Next section will provide a good understanding on how to write those supporting documentation.

Advanced Functional Tests Techniques

Once you get used to the way to write automated test cases, you should be able to implement complex test cases that involves not only the local Subversion Edge server, but also external servers such as the TeamForge server used during the tests of conversion process. Don’t forget to document the steps in a structured way inside the JavaDocs, as documentation later makes it easy to understand the purpose of the tests.

Note that the JavaDocs of the classes contain a more detailed specification of the execution of the test cases. For example, the sentences starting with “Verify” are related to the assertions necessary to verify the test case, while “Go to” are related to the HTTP Request method “get()”. Each of the sections are identified so that the implementation of the methods setUp(), tearDown(), and the actual method are explicitly written using Groovy. The source code has more detailed implementation of the test cases.

Test Case 1: Successful conversion to TeamForge Mode

   * SetUp
        * Login to SvnEdge
        * Revert to Standalone Mode in case on TeamForge Mode

   * Steps to reproduce
         * Go to the Credentials Form
         * Enter correct credentials and existing CTF URL and try to convert;

   * Expected Results
         * Successful conversion message is shown
         * Login -> Logout as admin
         * Verify that the server is on TeamForge mode;
         * Login to CTF server and verify that the system ID
            from the SvnEdge server is listed on the list of integration servers

   * Tear Down
         * Revert conversion if necessary
         * Logout from the SvnEdge server

The implementation of complex test cases might require verification of different properties of local and external resources. The example of the conversion process was the first challenge of this nature we had to implement. The following code snippet is the implementation of assertions of the conversion as the Expected results. Note that the method custom assertion methods were written to support this implementation (“assertProhibitedAccessToStandaloneModeLinksWorks()” and “assertConversionSucceededOnCtfServer()”.

     * Verify that the state of the conversion is persisted:
     * <li>The local server shows the TeamForge URL
     * <li>The CTF server shows the link to the server. This can be verified
     * by the current system ID on the list of integration servers.
    protected void assertConversionSucceeded() {
        // Step 1: verify that the conversion is persistent
        assertStatus 200

        assertStatus 200

        // verify that the software version is still shown

        assertStatus 200

        // verify that prohibited links work

        // Step 2: verify that the CTF server DOES list the system ID

Using the response object

As seen in some of the examples, the assertions are the way to verify that a given expected value exists in the HTTP Response payload received from the Server. However, whenever the test case needs to make a decision based on the contents of the response object, you can use the direct access to the response object. For instance, instead of failing a test that needs to have the user logged out, this code snippet verifies if the user is logged in and then performs the logout procedure. The same logic can be applied in different scenarios such as verifying if the server is started/stopped by verifying the status page button. Similarly, the test can verify if there are any created repositories in the file-system before creating a new test repository.

        if (this.response.contentAsString.contains(getMessage(""))) {

Dealing with external resources

The nature of Subversion Edge requires the integration with TeamForge, and how about testing the state of both systems in the same test case? Considering the Grails Plugin allows external HTTP requests during tests, why not performing the same steps an Admin would do to verify the state of the server? This was a bit tricky, but works like a charm. As we had designed before, reusing the configuration was the first step to define which remote TeamForge to use during tests. Then, the Test case could take care of automating the ways to generate the URL for the CTF server based on the configuration parameters during the tests of conversion. Here’s the closure in the file “CSVN_DEV/grails-app/conf/Config.groovy” that one can change which TeamForge server to use (svnedge.ctfMaster).

    ctfMaster {
        ssl = false
        domainName = ""
        username = "admin"
        password = "admin"
        port = 80
        systemId = "exsy1002"

Taking a closer look of what we needed, this is related to the assertions for the last expected result “Login to CTF server and verify that the system ID from the SvnEdge server is listed on the list of integration servers”. So, the translation of this sentence into Groovy code originated the method call “AbstractConversionFunctionalTests.assertConversionSucceededOnCtfServer()”, as the steps to perform this assertion are used by all different scenarios. As implemented, the first step requires that the login to TeamForge take the user to the Administration page “List Integrations” using the method ” this.goToCtfListIntegrationsPage()” before verifying if the system ID saved by the conversion process exists in that page. However, observations on how the HTTP Request flow in TeamForge works was necessary to understand the forwards after the user is logged in. After building the necessary parameters in the method “loginToCtfServerIfNecessary()” was implemented with all the needed values from both the Grails Config.groovy and from the environment. As warned before, the clickable elements of forms can differ from Subversion Edge and TeamForm, and therefore, the grails element “click LABEL” was used here outside the form closure. Finally, don’t be tempted to verify strings in TeamForge using i18n as they are different and Subversion Edge does not have direct access to them. Prefer validating steps using form elements or IDs produced by TeamForge as the UI can change on the remote server.

     * Verifies that the CTF server lists the current ctf server system ID.
    protected void assertConversionSucceededOnCtfServer() {

        assertContentContains("Site Administration")
        assertContentContains("SCM Integrations")
        def appServerPort = System.getProperty("jetty.port", "8080")
        def csvnHostAndPort = server.hostname + ":" + appServerPort

        // TeamForge removes any double-quotes (") submitted via the SOAP API.
        assertContentContains("This is a CollabNet Subversion Edge server in " +
            "managed mode from ${csvnHostAndPort}.")

     * Goes to the list of integrations on the CTF server
    private void goToCtfListIntegrationsPage() {
        // Goes to the list integrations page
        get(this.makeCtfUrl() + "/sf/sfmain/do/listSystems")

    * Makes login to CTF server from a given point that connects to the server.
    * In case the response content DOES NOT contains the string "Logged in as",
    * then make the login. The resulting page is the redirected page requested
    * earlier.
    private void loginToCtfServerIfNecessary() {
        if (!this.response.contentAsString.contains("Logged in as")) {
            assertStatus 200
            def ctfUsername = config.svnedge.ctfMaster.username
            def ctfPassword = config.svnedge.ctfMaster.password
            form("login") {
                username = ctfUsername
                password = ctfPassword
            // the button is a link instead of a form button. Use it outside
            // the form closure.
            click "Log In"
            assertStatus 200

Test Case Suites Needed

A few test cases have been written for specific functionalities of the application. However, here’s some of the test cases that can be developed.

* User Functional Tests
- Create User of each type
  - Login/Logout
  - Verify access to prohibited URLs
  - Access SVN and ViewVC Pages
- List Users
- Delete User
- Change User password
  - Logout and login with the new password.
  - Access SVN and ViewVC pages with new password
- View Self page
- Try changing the server settings, accessing other admin sections

* Repos Functional Tests
- Create Repo
- Discover Repos
- List Repos
- Edit Access Rules
  - Login with users without access to specific repos without access

* Statistics Functional Tests
- Access the pages for statistics

* Administration Functional Tests
- Changing server settings as Admin
- Changing the server Authentication settings
  - Login / Logout and verify changes.
  - Restart server after changing settings

* Server Logs Functional Tests
- Change log level
- View log Files
- View non-existing file
- View existing file
- View log files

* Packages Update Functional Tests
- Update the software packages
- Convert the server and try to update the server to new version

If you have any questions regarding the Functional Tests specification, please don’t hesitate to send an email to

Marcello de Sales – Software Engineer – CollabNet, Inc.

Categories: java, Subversion

BlackBeltFactory: If you are a teacher at heart and love technology, this is your place…

While studying Java for fun and to take the Java certification from the Sun Microsystems back in 2004, I used to hang out in different tutorial websites with reviews for the exams. I was still living in Brazil, where I grew up, when I first started studying Java at the University and seeing passionate about the “Write Once, Run Anywhere” premise… When I found JavaBlackBelt in 2006, I joined it to try perfecting my Java skills and keep up-to-date with the language. Given the transformation of how Social Networking took the Internet, everything changed since then, as they changed their branding and name to BlackBeltFactory, as well as have added social interaction capabilities and a market place for developers, technologists and the ones who love to teach and learn.

My previous experience was just related to my own learning experience: practice/learn the fundamentals of the Java Programming Language. It was essentially a website where users could go and take exams in different subjects related not only with Java, but also with relating technologies such as XML, Web Services, Hibernate, etc. However, I must confess that it is hard to keep up with the exams when you have your day-to-day job, school obligations, etc. I had conquered the Java Blue Belt and I was facing a lot of changes starting with moving from New York to California for the dream of the Silicon Valley and then having the opportunity to engage on another 2 years of my dreamed MS and work with what I love: Java and Computer Science. The academic world can take all of your time with research papers to read (ACM was my browser’s start page) and exams/finals. On the other hand, the only place where I could focus on practicing Java was my research projects: (my thesis and conferences). So, I never became a JavaBlackBelt per say and I was cleaning my mailbox when the name BlackBeltFactory was showing up on older and older emails. Yes, JavaBlackBelt had evolved and “taken the Social Networking train”. There are a list of changes listed on their website here.

The very first basic change BlackBeltFactory did was to take advantage of their infrastructure and start thinking on a more “language/vendor-agnostic” approach: why not offering training in other languages? I saw C# another programming language listed on their website and I must say that the BlackBeltFactory was a cool place to hang out and take exams prepared and reviewed by peers in the community. It is definitely a place to challenge your skills set on a given track. You can only take exams when you provide contributions: review questions, add comments, etc. This approach requires the user to be active in the learning community.

In my opinion, BlackBeltFactory’s natural progression could not have been different: take advantage of the Social Networking capabilities that we are currently live to provide the user’s better learning experience. Teaching is one of my passions and I must say that BlackBeltFactory did a great job in adding features like “Become a coach”. After you have passed the exam to be a coach, it seems that you can either create a free or a paid training to someone. Similarly, users interested about learning can ask others about services of teaching a specific topic. This marketplace is healthy and very interesting to me in the sense that I don’t need to drive anywhere to teach someone something I’m passionate about. As far as I could see, BlackBeltFactory offers the process for both participants to engage on a program. Hummm… Now I don’t need to think about going for PhD and teach! 😀

Another great feature is the translation capability. Although the previous version of the website branded as JavaBlackBelt was awesome for English Speaking users, the platform could not capture users of other nationalities and without knowledge of English. As a Brazilian, I can say that it is difficult, in general, to the ones who are starting with our field of technology/science to properly “bootstrap” their career because of the restricted access to content in Portuguese. That’s why I made sure I had Portuguese as one of the first translated versions of CollabNet Subversion Edge as I’m working in the project. BlackBeltFactory just gave me yet another reason to stick around and contribute to their community as I have a passion for learning and sharing knowledge.

All in all, I think I have to squeeze more of my time to play around in the BlackBeltFactory! For the love of teaching, I have already joined 2 Brazilian groups for the translations and I will make time to review exams and try to get my Java BlackBelt 😀 I could not get even the yellow in Kung Fu when I was 15, but I think I might have potential for Java. I have liked my LinkedIn and Twitter accounts, which are nice as a linking resource.

%d bloggers like this: