Testing Command-line Applications with Aruba

25 Oct 2011 ~ 2 minute read

Cucumber is often used to test web applications. Many developers hook it into their Rails projects to integration test site features. Wouldn’t it be great if there were a way to test command-line applications in a similar fashion? You can with Aruba.

Aruba

Aruba is a Cucumber extension for testing command-line applications written in any language. Passing arguments, interacting with the file system, capturing exit codes, and mimicking interactive usage are all features provided out of the box. Below is a basic test for the mv command that passes:

Scenario: Backing up test.conf
  When I run `mv test.conf test.conf.bak`
  Then the output should contain:
  """
  mv: rename test.conf to test.conf.bak: No such file or directory
  """

Now let’s showoff a few of Aruba’s built-in steps to prevent the command from failing:

Scenario: Backing up test.conf
  Given an empty file named "test.conf"
  When I run `mv test.conf test.conf.bak`
  Then the exit status should be 0
  And the following files should exist:
    | test.conf.bak |
  And the following files should not exist:
    | test.conf     |

The first step creates an empty file and executes mv inside of Aruba’s sandbox directory. After the mv command is executed, its exit status is compared to 0 and the existence of test.conf.bak (and non-existence of test.conf) is confirmed.

It’s also worth noting that after each scenario Aruba clears out its sandbox — a temporary directory that becomes the current working directory for your command-line tool — unless you explicitly tag the scenario with @no-clobber. This tag preserves the previous scenario’s final state. Tying this back to the example above, the next scenario would begin with only test.conf.bak in the sandbox. Additional Aruba-specific tags can be found in the README.

Extending the Aruba API

As a command-line application evolves, other conditions not available in Aruba’s built-in API will require testing. For example, say you need to assert a file’s user and group attributes. Because Aruba’s API was built using Ruby modules, it can be reopened inside of Cucumber’s env.rb:

module Aruba
  module Api
    def check_file_owner_and_group(paths_and_users_and_groups)
      prep_for_fs_check do # Lower-level function provided by Aruba
        paths_and_users_and_groups.each do |path, user, group|
          stat = File.stat(path)

          Etc.getpwuid(stat.uid).name.should == user
          Etc.getgrgid(stat.gid).name.should == group
        end
      end
    end
  end
end

Then create a matcher:

Then /^the following files should have username "([^"]*)" and group "([^"]*)":$/ do |user, group, files|
  check_file_owner_and_group(files.raw.map { |file_row| (file_row << user) << group })
end

Now that step can be included to test the user and group attributes of files:

Scenario: Backing up test.conf
  Given an empty file named "test.conf"
  When I run `mv test.conf test.conf.bak`
  And the exit status should be 0
  And the following files should exist:
    | test.conf.bak |
  And the following files should not exist:
    | test.conf     |
  And the following files should have username "hector" and group "staff":
    | test.conf.bak |

Conclusion

Using a behavior-driven development approach for building command-line applications with Cucumber and Aruba was a pleasure. Aruba’s API covers a decent amount of ground and was easily expandable. The source code was straightforward and after skimming its internals, I was able to expand the API to meet my needs. Hopefully reading this will help you do the same.