Juan Delgado

How can you _not_ automate testing?

16 Dec 2013

At the studio we have just helped a client release their new mobile app, Android and iOS. What makes it really special for me is that it is the first project that we fully BDD from day 1. And this has had a massive impact on the way we work.

Just as a quick recap, the main 2 benefits we are after with BDD are:

One has worked a treat, the other one not so much.

Collaboration and communication

This is the one benefit that has worked really well. We made a few mistakes at the beginning of the project, but we were quick enough to resolve them. This is how the process looks like now:

I’m pretty sure this process has saved us an enormous amount of time. Doubts are solved quickly and, crucially, ahead of development. Decisions are also taken on the spot, avoiding unnecessary emails, posts in Basecamp, etc.

Test automation

Sadly, this is the part that has not worked so well. In fairness, it has worked better for Android than for iOS because of the tools we chose.


For Android we are using a mix of Cucumber JVM, JUnit, Android instrumentation and Robotium. The good thing about this setup is that the tests are run on the actual device providing a lot of control, for example with access to the full Robotium API.


For iOS, however, we chose Calabash. The good thing about Calabash is that the tests are written in Ruby, which integrates very nicely with Cucumber itself. It has also allowed us to use Sinatra as our mocking server for API requests.

The downside of things can be resumed in:


All in all we still think BDD is the way to go. Tools can only get better (we are looking into Appium) so our whole set up can only get better too.

We are still finding the limits of what’s possible and sensible to automate (automated analytics integration testing anyone?), but we are getting there.

BTW, we are hiring if you want to be part of the devster team.


Archive About

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.