6.19. Behavior Driven Testing

Table of Contents

6.19.1. Defining Step Implementations using Step
6.19.2. Using Step Patterns with Placeholders
6.19.3. Using Step Patterns with Regular Expressions
6.19.4. The BDD context Object
6.19.5. context.userData: Passing Data Between Steps
6.19.6. Accessing Tables and Multi-Line Text
6.19.7. Defining Step Implementations using Given/When/Then
6.19.8. Step Lookup Order & Overriding Shared Step Implementations
6.19.9. Influencing Scenario Execution from Within Step Implementations
6.19.10. Performing Actions During Test Execution Via Hooks
6.19.11. The Anatomy of a BDD Test Case

In order to automate executing a .feature file, step implementations need to be defined which associate a piece of script code with a pattern. Every time a step is encountered in a .feature file, Squish will try to find a pattern matching the step text. If a match is found, the associated code is executed.

6.19.1. Defining Step Implementations using Step

A step implementation is defined by calling a pre-defined Step function. A simple definition would look like this:

Python
@Step("user starts the addressbook application")
def step(context):
    startApplication("addressbook")
JavaScript
Step("user starts the addressbook application", function(context) {
    startApplication("addressbook");
});
Tcl
Step {user starts the addressbook application} {context} {
    startApplication addressbook
}
Ruby
Step("user starts the addressbook application") do |context|
    Squish::startApplication("addressbook")
end
Perl
use Squish::BDD;

Step "user starts the addressbook application", sub {
    startApplication("addressbook");
};

In this example, a step matching the pattern “user starts the addressbook application” will cause Squish to execute

Python
startApplication("addressbook")
JavaScript
startApplication("addressbook");
Tcl
startApplication addressbook
Ruby
Squish::startApplication("addressbook")
Perl
startApplication("addressbook");
[Note]Note

The step text (after stripping any keywords from the text, such as 'Given') has to match the pattern exactly (the match is "anchored" to the start and end of the text). The table below shows which step texts would match the pattern “user starts the addressbook application”:

Step textPattern matches?
When a user starts the addressbook applicationYes
When a user starts the addressbook application we talk aboutNo
Given a user starts the addressbook applicationYes
Given we assume that a user starts the addressbook applicationNo

Note how, leaving aside the initial keywords such as 'Given' or 'When', the step text has to match the pattern exactly: there must not be any text before or after it.

Defining a step via Step will always succeed, except in the following cases:

  • The pattern given is empty or otherwise malformed — the latter may occur when using regular expressions.

  • There is already an existing definition for the given pattern.

  • For script languages in which Step takes a 'signature' argument: the signature must at least contain one argument for the context.

6.19.2. Using Step Patterns with Placeholders

Patterns can make use of certain placeholders to extract data from the step text and pass it to the script code. This allows making step-definitions more reusable since it avoids having to hardcode specific values. For instance, here's an improvement to the above step definition which avoids hardcoding the name of the application to start:

Python
@Step("user starts the |word| application")
def step(context, appName):
    startApplication(appName)
JavaScript
Step("user starts the |word| application", function(context, appName) {
    startApplication(appName);
});
Tcl
Step {user starts the |word| application} {context appName} {
    startApplication $appName
}
Ruby
Step("user starts the |word| application") do |context, appName|
    Squish::startApplication(appName)
end
Perl
use Squish::BDD;

Step "user starts the |word| application", sub {
    my %context = %{shift()};
    my $appName = shift();
    startApplication($appName);
};

The application name addressbook is no longer hardcoded. Instead, the |word| will match the word and pass it via the new appName argument.

The following placeholders can be used:

PlaceholderMeaning
|word| Matches a single word. A word is one or more characters in length and consists of letters, digits and underscore (_) characters.
|integer| Matches a positive or negative integer value, i.e. a number without a fractional component.
|any| Matches a sequence of one or more arbitrary characters.

6.19.3. Using Step Patterns with Regular Expressions

Sometimes, you may find that the above mentioned placeholders are insufficient, that you need more control over which portions of a step text gets extracted. This can be achieved by not passing a plain string to the Step function but rather a regular expression. Regular expressions are vastly more powerful, but also require being more careful - they can easily become somewhat cryptic to the reader.

Regular expressions not only allow much finer control over the text being matched. They can make use of capturing groups to extract data from the step text and pass it to the step code.

[Note]Python-specific

In Python scripts, you can pass a regular expression object directly (i.e. the object returned by Python's re.compile function). But if you don't want to import the re module, you can also pass a string and the optional argument regexp set to True, as in

@Step(r"user starts the (\w+) application", regexp=True)
def step(context, appName):
    ...

We also recommend to use Python's raw string syntax (r"..." resp. r'...') for specifying the strings to avoid escaping of the backslash character in the regular expression.

[Note]Tcl-specific

In Tcl scripts, there are no regular expression values. Instead, the Step command is invoked with the -rx option, as in

Step -rx "This is a reg.lar expression$" {context} { ... }

For example, the above examples can be expressed using regular expressions as follows:

Python
@Step(r"user starts the (\w+) application", regexp=True)
def step(context, appName):
    startApplication(appName)
JavaScript
Step(/user starts the (\w+) application/, function(context, appName) {
    startApplication(appName);
});
Tcl
Step -rx {user starts the (\w+) application} {context appName} {
    startApplication $appName
}
Ruby
Step(/user starts the (\w+) application/) do |context, appName|
    Squish::startApplication(appName)
end
Perl
use Squish::BDD;

Step qr/user starts the (\w+) application/, sub {
    my %context = %{shift()};
    my $appName = shift();
    startApplication($appName);
};

6.19.4.  The BDD context Object

Each BDD implementation function has a context singleton object passed in as the first argument. The properties it can have are:

  • context.text - the title of a step in step implementations

  • context.title - the title of a scenario, step or feature, but only in hook functions. See Performing Actions During Test Execution Via Hooks (Section 6.19.10) for details.

  • context.userData - An extra property that can be used for passing arbitrary data (typically as a list, map or dictionary) between steps and hooks.

  • context.table - stores table data if it is being passed into the step.

  • context.multiLineText - stores multi-line text if it is being passed into the step.

Examples using userData, table and multiLineText are in the following sections.

6.19.5. context.userData: Passing Data Between Steps

The context argument passed to all steps (and hooks) features a userData field which can be used to pass data between steps. This property is initially empty but can be written to from within a step. Consider e.g. a feature file like:

Feature: Shopping Cart

  Scenario: Adding three items to the shopping cart

    Given I have an empty shopping cart
    When I add bananas to the shopping cart
    And I add apples to the shopping cart
    And I add milk to the shopping cart
    Then I have 3 items in the shopping cart

The implicit state in this scenario (the current contents of the shopping cart) can be modeled using userData:

Python
@Given("I have an empty shopping cart")
def step(context):
    context.userData = []

@When("I add |word| to the shopping cart")
def step(context, itemName):
    context.userData.append(itemName)

@Then("I have |integer| items in the shopping cart")
def step(context, expectedAmount):
    test.compare(len(context.userData), expectedAmount)
JavaScript
Given("I have an empty shopping cart", function(context) {
    context.userData = [];
});

When("I add |word| to the shopping cart", function(context, itemName) {
    context.userData.push(itemName);
});

Then("I have |integer| items in the shopping cart", function(context, expectedAmount) {
    test.compare(context.userData.length, expectedAmount);
});
Perl
Given("I have an empty shopping cart", sub {
    my $context = shift();
    $context->{userData} = [];
});

When("I add |word| to the shopping cart", sub {
    my($context, $itemName) = @_;
    push @{$context->{userData}}, $itemName;
});

Then("I have |integer| items in the shopping cart", sub {
    my($context, $expectedAmount) = @_;
    test::compare(scalar(@{$context->{userData}}), $expectedAmount);
});
Tcl
Given "I have an empty shopping cart" {context} {
    $context userData {}
}

When "I add |word| to the shopping cart" {context itemName} {
    set items [$context userData]
    lappend items $itemName
    $context userData $items
}

Then "I have |integer| items in the shopping cart" {context expectedAmount} {
    set items [$context userData]
    test compare [llength $items] $expectedAmount
}
Ruby
Given("I have an empty shopping cart") do |context|
  context.userData = []
end

When("I add |word| to the shopping cart") do |context, itemName|
  context.userData.push itemName
end

Then("I have |integer| items in the shopping cart") do |context, expectedAmount|
  Test.compare context.userData.count, expectedAmount
end

Here, the userData field keeps track of the items added to the shopping cart, because this is extra data related to the context of our test. It could also be used to keep track of active Application Context (Section 6.3.12) objects in multiple-AUT test cases, for example.

[Note]Note

The userData is never cleared; you can however do so explicitly using e.g. a OnScenarioStart hook.

6.19.6. Accessing Tables and Multi-Line Text

The context object exposes optional extra arguments to the step with these two properties:

multiLineText

Returns an optional multi-line text argument to a step. The text is returned as a list of strings, each of which being a single line. For example, a step such as

Given I create a file containing
"""
[General]
Mode=Advanced
"""

The multi-line text can be accessed via

Python
@Step("I create a file containing")
def step(context):
    text = context.multiLineText
    f = open("somefile.txt", "w")
    f.write("\n".join(text))
    f.close()
JavaScript
Step("I create a file containing", function(context) {
  var text = context.multiLineText;
  var file = File.open("somefile.txt", "w");
  file.write(text.join("\n"));
  file.close();
});
Tcl
Step {I create a file containing} {context} {
  set text [$context multiLineText]
  set f [open "somefile.txt" "w"]
  puts $f [join $text "\n"]
  close $f
}
Ruby
Step("I create a file containing") do |context|
    File.open("somefile.txt", 'w') { |file|
        file.write(context.multiLineText.join("\n"))
    }
end
Perl
use Squish::BDD;

Step "I create a file containing", sub {
    my %context = %{shift()};
    my @multiLineText = @{$context{'multiLineText'}};

    my $text = join("\n", @multiLineText);

    open(my $fh, '>', 'somefile.txt') or die;
    print $fh $text;
    close $fh;
};
table

Returns an optional text argument to a step. The table is returned as a list of lists: each inner list represents a row, and individual list elements represent cells. For example, a step such as

Given I enter the records
  | firstName | lastName | age |
  | Bob       | Smith    | 42  |
  | Alice     | Thomson  | 27  |
  | John      | Martin   | 33  |

The table argument can be accessed via

Python
@Step("I enter the records")
def step(context):
    table = context.table

    # Drop initial row with column headers
    for row in table[1:]:
        first = row[0]
        last = row[1]
        age = row[2]
        # ... handle first/last/age
JavaScript
Step("I enter the records", function(context) {
    var table = context.table;

    // Skip initial row with column headers by starting at index 1
    for (var i = 1; i < table.length; ++i) {
        var first = table[i][0];
        var last = table[i][1];
        var age = table[i][2];
        // ... handle first/last/age
    }
});
Tcl
Step {I enter the records} {context} {
  set table [$context table]

  # Drop initial row with column headers
  foreach row [lreplace $table 0 0] {
    foreach {first last age} $row break
    # ... handle $first/$last/$age
  }
}
Ruby
Step("I enter the records") do |context|
    table = context.table

    # Drop initial row with column headers
    table.shift

    for first,last,age in table do
        # ... handle first/last/age
    end
end
Perl
use Squish::BDD;

Step "I enter the records", sub {
    my %context = %{shift()};
    my @table = @{$context{'table'}};

    # Drop initial row with column headers
    shift(@table);

    for my $row (@table) {
        my ($first, $last, $age) = @{$row};
        # ... handle $first/$last/$age
    }
};

6.19.7. Defining Step Implementations using Given/When/Then

In addition to Step, Squish supports defining step implementations using three more functions, called Given, When and Then. These three functions follow the same syntax as Step but cause a slightly different behavior when executing steps since step implementations registered using Step match more step texts than those registered with any of the other three functions. The following table illustrates the difference:

Step textStep("...", ..) matchesGiven("...", ..) matchesWhen("...", ..) matchesThen("...", ..) matches
Given I say helloYesYesNoNo
When I say helloYesNoYesNo
Then I say helloYesNoNoYes
And I say helloYesMaybeMaybeMaybe
But I say helloYesMaybeMaybeMaybe

Note how patterns registered via Step always match, no matter what keyword the step at hand starts with. Given, When and Then only match if the step starts with the matching keyword (or a synonym such as "And" or "But").

[Tip]Tip

It is generally preferable to use Given/When/Then instead of Step for registering patterns since that also improves the efficiency of the auto-completion offered by the Squish IDE.

Whether patterns registered via Given/When/Then match steps starting with And or But depends on the keyword preceding the And/But. Consider:

Feature: Some Feature
  Scenario: Some Scenario
    Given I am hungry
      And I'm in the mood for 20000 of something
      But I do not like peas
     Then I will enjoy rice.

In this feature description, the lines starting with “And” and “But” succeed a line starting with “Given”, they are synonymous. This means that only patterns registered via Step or Given would match the step “And I'm in the mood for 20000 of something”.

6.19.8. Step Lookup Order & Overriding Shared Step Implementations

When executing a BDD test, Squish will respect a specific order when deciding which step implementation matches a given step text. Step implementations are considered based on the order in which source files containing step implementations are loaded when starting a BDD test. By default, this order is as follows:

  1. steps subdirectory of current test case.

  2. shared/scripts/steps directory of current test suite.

This order causes steps defined in the local test case to be preferred over steps defined in the shared scripts directory of the test suite.

Actually, Squish will not only load step implementations stored in above-mentioned directories but also those stored in subdirectories. For instance, you can register step implementations in files like steps/basic/basicsteps.py or steps/uisteps/mainwindowsteps.js.

[Tip]Tip

The list of directories from which step implementations are loaded is not hard-wired into Squish. Instead, it's a list of directories specified in the test.* file stored in the test case directory (e.g. test.js for JavaScript tests. You can adjust this order for individual tests, or for all newly created tests by editing the file scriptmodules/*/bdt_driver_template.*, e.g. scriptmodules/javascript/bdt_driver_template.js.

Even though registering two step implementations with the same pattern will yield an error, registering two step implementations which are defined in different directories will not trigger an error. This means that you can 'override' shared steps by using the exact same pattern in a testcase-specific file.

6.19.9. Influencing Scenario Execution from Within Step Implementations

It is possible to define step implementations such that the current scenario is aborted and all subsequent steps (if any) are skipped. This is useful in case subsequent steps rely on certain conditions (e.g. some file exists) but a preceding step which is supposed to establish/verify that condition fails.

To skip subsequent steps in the current scenario, step implementations can return a special value - the exact name of which depending on the respective scripting language:

Python
@Step("I create a file containing")
def step(context):
    try:
        text = context.multiLineText
        f = open("somefile.txt", "w")
        f.write("\n".join(text))
        f.close()
    except:
        # Failed to create file; skip subsequent steps in current scenario
        return AbortScenario
JavaScript
Step("I create a file containing", function(context) {
  try {
    var text = context.multiLineText;
    var file = File.open("somefile.txt", "w");
    file.write(text);
    file.close();
  } catch (e) {
    // Failed to create file; skip subsequent steps in current scenario
    return AbortScenario;
  }
});
Tcl
Step {I create a file containing} {context} {
  if {[catch {
    set text [$context multiLineText]
    set f [open "somefile.txt" "w"]
    puts $f [join $text "\n"]
    close $f
  }]} then {
    # Failed to create file; skip subsequent steps in current scenario
    return AbortScenario
  }
}
Ruby
Step("I create a file containing") do |context|
    begin
        File.open("somefile.txt", 'w') { |file|
            file.write(context.multiLineText.join("\n"))
        }
    rescue Exception => e
        # Failed to create file; skip subsequent steps in current scenario
        next ABORT_SCENARIO
    end
end
Perl
use Squish::BDD;

Step "I create a file containing", sub {
    my %context = %{shift()};
    my @multiLineText = @{$context{'multiLineText'}};

    my $text = join("\n", @multiLineText);

    if (open(my $fh, '>', 'somefile.txt')) {
        print $fh $text;
        close $fh;
    } else {
        # Failed to create file; skip subsequent steps in current scenario
        return ABORT_SCENARIO;
    }
};

6.19.10. Performing Actions During Test Execution Via Hooks

It is common that some actions need to be performed before or after some event occurs. This need arises in cases like

  • Global variables should be initialized before test execution starts.

  • An application needs to be started before a Feature is executed.

  • Temporary files should be removed after a Scenario is executed.

Squish allows defining functions which are hooked into the test execution sequence. The functions can be registered with certain events and are executed before or after an event occurs. You can register as many functions for an event as you like: the hook functions will be called in the same order as they were defined. Functions can be associated with any of the following events:

OnFeatureStart/OnFeatureEnd

These events are raised before/after the first/last scenario in a specific Feature is executed. The context argument passed to functions associated with either of these events provides a title field which yields the title of the feature which is about to get executed (resp. just finished executing).

OnScenarioStart/OnScenarioEnd

These events are raised before/after the first/last step in a specific Scenario is executed. The OnScenarioStart event is also raised before any Background is executed. The context argument passed to functions associated with either of these events provides a title field which yields the title of the scenario which is about to get executed (resp. just finished executing).

OnStepStart/OnStepEnd

These events are raised before/after the code in a specific Step is executed. The context argument passed to functions associated with either of these events provides a text field which yields the text of the step which is about to get executed (resp. just finished executing).

You can associate code to be executed in case any of these events occur using language-specific API. In general, the name of the function to sign up for an event equals the name of the event except for Python, where the name of the function doesn't matter since a decorator is used instead. These functions are registered in the same script files in which you would use the Step function to register step implementations.

Here are a few examples:

Setting up a OnFeatureStart hook to setup global variables:

JavaScript
OnFeatureStart(function(context) {
    counter = 0;
    inputFileName = "sample.txt";
});
Perl
my $inputFileName;
my $counter;

OnFeatureStart sub {
    $counter = 0;
    $inputFileName = 'sample.txt';
};
Python
@OnFeatureStart
def hook(context):
    global counter
    global inputFileName
    counter = 0
    inputFileName = "sample.txt"
Ruby
OnFeatureStart do
    @counter = 0
    @inputFileName = 'sample.txt'
end
Tcl
OnFeatureStart {context} {
  global counter
  global inputFileName
  set $counter 0
  set $inputFileName "sample.txt"
}

Registering OnScenarioStart & OnScenarioEnd event handlers to start & stop an AUT:

JavaScript
OnScenarioStart(function(context) {
    startApplication("addressbook");
});

OnScenarioEnd(function(context) {
    currentApplicationContext().detach();
});
Perl
OnScenarioStart sub {
    ::startApplication('addressbook');
};

OnScenarioEnd sub {
    ::currentApplicationContext.detach();
};
Python
@OnScenarioStart
def hook(context):
    startApplication("addressbook")

@OnScenarioEnd
def hook(context):
    currentApplicationContext().detach()
Ruby
OnScenarioStart do |context|
    Squish::startApplication 'addressbook'
end

OnScenarioEnd do |context|
    Squish::currentApplicationContext.detach
end
Tcl
OnScenarioStart {context} {
  startApplication "addressbook"
}

OnScenarioEnd {context} {
    applicationContext [currentApplicationContext] detach
}

Generating extra warning log output whenever a step which mentions the word delete is about to get executed:

JavaScript
OnStepStart(function(context) {
    var text = context["text"];
    if (text.search("delete") > -1) {
        test.warning("About to execute dangerous step: " + text);
    }
});
Perl
OnStepStart sub {
    my %context = %{shift()};
    if (index( $context{text}, 'delete') != -1) {
        ::test::warning("About to execute dangerous step: $context{text}");
    }
};
Python
@OnStepStart
def hook(context):
    text = context.text
    if text.find("delete") > -1:
        test.warning("About to execute dangerous step: %s" % text)
Ruby
OnStepStart do |context|
    if context['text'].include? 'delete'
        Squish::Test.warning "About to execute dangerous step: #{context['text']}"
    end
end
Tcl
OnStepStart {context} {
  set text [$context text]
  if {[string first "delete" $text] > -1} {
    test warning "About to execute dangerous step: $text"
  }
}

6.19.11.  The Anatomy of a BDD Test Case

All BDD test cases start by running a regular script in your chosen scripting language. This script is auto-generated, can be found in the same directory as the corresponding test.feature file, and is called test.xy, where xy = (pl|py|rb|js|tcl). It looks like this:

Python

source(findFile('scripts', 'python/bdd.py'))

setupHooks('../shared/scripts/bdd_hooks.py')
collectStepDefinitions('./steps', '../shared/steps')

def main():
    testSettings.throwOnFailure = True
    runFeatureFile('test.feature')

JavaScript

source(findFile('scripts', 'javascript/bdd.js'));

setupHooks(['../shared/scripts/bdd_hooks.js']);
collectStepDefinitions(['./steps', '../shared/steps']);

function main()
{
    testSettings.throwOnFailure = true;
    runFeatureFile("test.feature");
}
Perl

use warnings;
use strict;
use Squish::BDD;

setupHooks("../shared/scripts/bdd_hooks.pl");
collectStepDefinitions("./steps", "../shared/steps");

sub main {
    testSettings->throwOnFailure(1);
    runFeatureFile("test.feature");
}
Ruby

require 'squish'
require 'squish/bdd'

include Squish::BDD
setupHooks "../shared/scripts/bdd_hooks.rb"
collectStepDefinitions "./steps", "../shared/steps"

def main
    Squish::TestSettings.throwOnFailure = true
    Squish::runFeatureFile "test.feature"
end
Tcl

source [findFile "scripts" "tcl/bdd.tcl"]

Squish::BDD::setupHooks "../shared/scripts/bdd_hooks.tcl"
Squish::BDD::collectStepDefinitions "./steps" "../shared/steps"

proc main {} {
    testSettings set throwOnFailure true
    runFeatureFile "test.feature"
}

There are some functions that are called from this script which are part of the Squish BDD API, but are rarely used in regular testcases. This section will explain what those functions do, in case you wish to reuse BDD steps or other BDD features in script-based tests.

First, setupHooks() is called at the start of each BDD test case to scan for and set up the hook functions described in Performing Actions During Test Execution Via Hooks (Section 6.19.10).

Next, collectStepDefinitions() is used to scan for and import step definitions found in the directories specified as arguments to the function. One or more arguments can be provided, and the earlier ones have priority over the later ones. Step implementations are typically located in files called steps.xy, where xy = (pl|py|rb|js|tcl).

Python
collectStepDefinitions('./steps', '../shared/steps')
JavaScript
collectStepDefinitions('./steps', '../shared/steps');
Perl
use Squish::BDD;
collectStepDefinitions("./steps", "../shared/steps");
Ruby
include Squish::BDD
collectStepDefinitions "./steps", "../shared/steps"
Tcl
source [findFile "scripts" "tcl/bdd.tcl"]
Squish::BDD::collectStepDefinitions "./steps" "../shared/steps"

Hence in above example, the step definition from the current Test Case Resources will be used in favor of one with the same name, if it is also found in Test Suite Resources.

Finally, runFeatureFile() can be called on a BDD .feature file after hooks are set up and steps are collected, to run all the Scenarios in the specified Gherkin feature file.