Functional iOS Testing in Swift

One of the projects we are currently working on at Shutl involves building an iOS application. The application is essentially quite simple; it acts as a client for our API, adding animations, visuals, and notifications.

Testing is a key part of our development process, so when we started developing the application, one of the first steps was to find a testing framework that suited our needs. XCode provides XCTest as a testing framework that works good for unit testing. Unfortunately, if you want to test the behavior of your app from a user perspective, XCTest’s abilities are very limited.

Because we are mostly a Ruby shop, we’re familiar with using cucumber.

That’s how we came across Frank, a handy framework which enables you to write functional tests for your iOS applications using cucumber.

Frank

The way Frank works is that you “frankify” your iOS app, which then lets you use the accessibility features of iOS to emulate a user using an iOS device. You can launch the app, rotate the device, and interact with the screen in most of the ways a real user can.

If you’re familiar with CSS selectors, interacting with elements on the screen should look very familiar, albeit with a slightly different syntax. Frank also provides custom selectors and predefined steps for some of the most common interactions.

For instance if you want to select a label with the content “I am a label” you could use this:

check_element_exists('label marked:"I am a label"')

There are also predefined steps provided for more complex instructions like clicking a button with the content “Click me”:

When I touch the button marked "Click me"

At first we considered testing against a live QA server but soon experienced problems with this setup. We needed predictable data for our tests, and this is difficult to achieve as the data stored in a live QA environment changes all the time. Combine this with availability issues and you’ve got yourself an unworkable solution.

After some thought, the route we decided to take was to mock these services and return fixtures.

How we are doing it

The idea is to keep all the logic that directly interacts with the server inside one unique class or struct. It will provide necessary functions such as fetchUser and updateResource that can be invoked from wherever they’re needed. This allows us to easily implement alternate versions of these functions without affecting the rest of the code.

In the example code below, we have two different implementations. The first one, shown here, uses our remote API to retrieve data from the server.

static func requestSuperHeroName(name: String, gender: String, completionHandler: (String) -> Void) {
    let url = NSURL(string: "http://localhost:4567?name=\(name)&gender=\(gender)")
    let request = NSURLRequest(URL: url!)
    NSURLConnection.sendAsynchronousRequest(request, queue: NSOperationQueue.mainQueue()) {(response, data, error) in
        if let name = NSString(data: data, encoding: NSUTF8StringEncoding) {
            completionHandler(name)
        }
    }
}

The second implementation – our test mock – is simply returning a hard-coded value with the same structure as the ones returned by the server.

static func requestSuperHeroName(name: String, gender: String, completionHandler: (String) -> Void) {
    completionHandler("Super \(name)")
}

Next we’ll define two different targets, one using the real client and another one using the test mock client, and we’ll use the second – mocked – target to create our frankified app.

Step by step

Here is a walk-through of how such an app could be implemented by using our sample app “What superhero are you?”. You provide the app with your name and gender, and it uses a highly advanced algorithm to determine which superhero you are.

1. Set your app up with two targets. One will be using the real backend, and the other one will be using the mocked backend.
   
2. Frankify your app.
   
3. Write your first test. Our first feature looks like this:

   Feature:
     As a user
     I want to use the app
     So I can determine which superhero I am
    
   Scenario:
     Put in my name and gender and have my superhero have
     it return which superhero I am
    
   Given I launch the app
   When  I enter my name
   And   I choose my gender
   And   I touch the button marked "Which superhero am I?"
   Then  I want to see which superhero I am
   

   And the related steps:

   When(/^I enter my name$/) do
     fill_in('Name', with: 'Jon')
   end
    
   When(/^I choose my sex$/) do
     touch("view:'UISegmentLabel' marked:'Male'")
   end
    
   Then(/^I want to see which superhero I am$/) do
     sleep 1
     check_element_exists("view:'UILabel' marked:'Super Jon'")
   end
   

4. Make the tests pass!
   
5. Done!
   

   Conclusions

   This solution works, but it’s not without its limitations. The most significant one being that you need to return some sensible data in your mocks. In our test app we work with very simple logic, and it did the trick. We return fixed responses, which means that there is no way of testing more complex interactions. These can and should be covered by unit and integration tests, which come with their own problems.

   It can also be hard to test certain user actions, like swiping something on the screen. The more customized your app’s interface is, the harder it will be to test it with Frank. Almost anything can be done, but the solution will most likely feel hacked. Also we have yet to find a way of testing web UIs.

   Frank is not a magic bullet for functional testing in Swift, but so far we’ve found it a useful addition to our codebase, and we’re liking it!

   Links

   What Superhero are you? on Github
   Testing with Frank
   (CC image by Chris Harrison)
   (CC image by Esther Vargas)