4.8. Squish for Mac OS X Tutorials

Table of Contents

4.8.1. Tutorial: Starting to Test macOS Applications
4.8.2. Tutorial: Designing Behavior Driven Development (BDD) Tests
4.8.3. Tutorial: Migration of Existing Tests to BDD

Learn how to test native Mac OS X applications.

4.8.1. Tutorial: Starting to Test macOS Applications

This tutorial will show you how to create, run, and modify tests for an example macOS Cocoa application. In the process, you will learn about Squish's most frequently used features so that by the end of the tutorial you will be able to start writing your own tests for your own applications.

This chapter presents most of the major concepts behind Squish and provides the information you need to get started using Squish for testing your own applications. This tutorial does not discuss all of Squish's features, and those that it does cover are not covered in full detail. After reading this tutorial we recommend reading the User Guide (Chapter 5), and at least skimming the API Reference Manual (Chapter 6) and the Tools Reference Manual (Chapter 7), so that you are familiar with all the features that Squish has to offer, even if you don't need to use them all straight away.

This tutorial is divided into several sections. If you are new to Squish it is best to read all of them. If you are already using Squish you might want to just skim the tutorial, stopping only to read those sections that cover any new features that you haven't used before—or you could just skip straight to the User Guide (Chapter 5).

When we show how to achieve something using the IDE we will also follow with an explanation of how to do the same thing using the command line tools. Using an IDE is the easiest and best way to start, but once you build up lots of tests, you will want to automate them, (e.g., doing nightly runs of your regression test suite), so it is worth knowing how to use the command line tools since they can be run from batch files or shell scripts.

For this chapter we will use a simple address book application as our AUT. The application is shipped with Squish in SQUISHDIR/examples/mac/addressbook. This is a very basic application that allows users to load an existing address book or create a new one, add, edit, and remove entries, and save (or save as), the new or modified addressbook. Despite the application's simplicity, it has all the key features that most standard applications have: a menu bar with pull down menus, a toolbar, and a central area—in this case showing a table. It supports in-place editing and also has a sheet for adding items. All the ideas and practices that you learn to test this application can easily be adapted to your own applications. And naturally, the User Guide (Chapter 5) has many more examples and shows how to test various Cocoa-specific features.

The screenshot shows the application with a newly created, empty address book.

The macOS SquishAddressBook.app example.

[Note]Using the Examples

The first time you try running a test for one of the example AUTs you might get a fatal error that begins “Squish couldn't find the AUT to start...”. If this occurs, click the Test Suite Settings toolbar button, and in the Application Under Test (AUT) section choose the AUT from the combobox if it is available, or click the Browse... button and navigate to the AUT's executable via the finder dialog that pops up. (Some versions of Squish will automatically pop up this dialog if no AUT is specified.) This only needs to be done once per example AUT. (This doesn't arise when testing your own AUTs.)

In the following sections we will create a test suite and then create some tests, but first we will very briefly review some key Squish concepts.

4.8.1.1. Squish Concepts

To perform testing, two things are required:

  1. an application to test—known as the Application Under Test (AUT), and

  2. a test script that exercises the AUT.

One fundamental aspect of Squish's approach is that the AUT and the test script that exercises it are always executed in two separate processes. This ensures that even if the AUT crashes, it should not crash Squish. (In such cases the test script will fail gracefully and log an error message.) In addition to insulating Squish and test scripts from AUT crashes, running the AUT and the test script in separate processes brings other benefits. For example, it makes it easier to store the test scripts in a central location, and it also makes it possible to perform remote testing on different machines and platforms. The ability to do remote testing is particularly useful for testing AUTs that run on multiple platforms, and also when testing AUTs that run on embedded devices.

Squish runs a small server (squishserver) that handles the communication between the AUT and the test script. The test script is executed by the squishrunner tool, which in turn connects to the squishserver. The squishserver starts the AUT and injects the Squish hook into it. The hook is a small library that makes the AUT's live running objects accessible and that can communicate with the squishserver. With the hook in place, the squishserver can query AUT objects regarding their state and can execute commands—all on behalf of the squishrunner. And the squishrunner itself requests that the AUT performs whatever actions the test script specifies. All the communication takes place using network sockets which means that everything can be done on a single machine, or the test script can be executed on one machine and the AUT can be tested over the network on another machine.

The following diagram illustrates how the individual Squish tools work together.

From the test engineer's perspective this separation is not noticeable, since all the communication is handled transparently behind the scenes.

Tests can be written and executed using the Squish IDE, in which case the squishserver is started and stopped automatically, and the test results are displayed in the Squish IDE's Test Results view (Section 8.2.17). The following diagram illustrates what happens behind the scenes when the Squish IDE is used.

The Squish tools can also be used from the command line without the Squish IDE—this is useful for those testers who prefer to use their own tools (for example, their favorite editor), and also for performing automatic batch testing (for example, when running regression tests overnight). In these cases, the squishserver must be started manually, and stopped when all the testing is complete (or, if preferred, started and stopped for each test).

[Note]Terminology

The Squish documentation mostly uses the term widget when referring to GUI objects (i.e., buttons, menus, menu items, labels, table controls, etc). Windows users might be more familiar with the terms control and container, but here we use the term widget for both. Similarly, macOS users may be used to the term view; again, we use the term widget for this concept.

[Note]The Squish Directory

Throughout the manual, we often refer to the SQUISHDIR directory. This means the directory where Squish is installed, which might be /Applications/Squish, /usr/local/squish, ~/Squish, or somewhere else, depending on where you installed it. The exact location doesn't matter, so long as you mentally translate the SQUISHDIR directory to whatever the directory really is when you see paths and filenames in this manual.

4.8.1.2. Creating a Test Suite

A test suite is a collection of one or more test cases (tests). Using a test suite is convenient since it makes it easy to share tests scripts and test data between tests.

To begin with start up the Squish IDE, either by clicking or double-clicking the squishide icon, or by launching squishide from the taskbar menu or by executing open squishide.app on the command line—whichever you prefer and that is suitable for the platform you are using. Once Squish starts up you might be greeted with a Welcome Page in case you're starting the squishide for the first time. Click the Workbench button in the upper right to dismiss it. Then, the squishide will look similar to the screenshot—but probably slightly different depending on the windowing system, colors, fonts, and theme that you use, and so on.

The Squish IDE with no Test Suites

Once Squish has started click File|New Test Suite... to pop-up the New Test Suite wizard shown below.

The New Test Suite wizard's Name and Directory page

Enter a name for your test suite and choose the folder where you want the test suite to be stored. In the screenshot we have called the test suite suite_py and will put it inside the addressbook folder. (For your own tests you might use a more meaningful name such as "suite_addressbook"; we chose "suite_py" because for the sake of the tutorial we will create several suites, one for each scripting language that Squish supports.) Naturally, you can choose whatever name and folder you prefer. Once the details are complete, click Next to go on to the Toolkit (or Scripting Language) page.

The New Test Suite wizard's Toolkit page

If you get this wizard page, click the toolkit your AUT uses. For this example, we must click Mac since we are testing a macOS application. Then click Next to go to the Scripting Language page.

The New Test Suite wizard's Scripting Language page

Choose whichever scripting language you want—the only constraint is that you can only use one scripting language per test suite. (So if you want to use multiple scripting languages, just create multiple test suites, one for each scripting language you want to use.) The functionality offered by Squish is the same for all languages. Having chosen a scripting language, click Next once more to get to the wizard's last page.

The New Test Suite wizard's AUT page

If you are creating a new test suite for an AUT that Squish already knows about, simply click the combobox to drop-down the list of AUTs and choose the one you want. If the combobox is empty or your AUT isn't listed, click the Browse button to the right of the combobox—this will pop-up a file open dialog from which you can choose your AUT. In the case of Cocoa programs, the AUT is the application's executable (e.g., SquishAddressBook on macOS). Once you have chosen the AUT, click Finish and Squish will create a sub-folder with the same name as the test suite, and will create a file inside that folder called suite.conf that contains the test suite's configuration details. Squish will also register the AUT with the squishserver. The wizard will then close and Squish's IDE will look similar to the screenshot below.

The Squish IDE with the suite_py test suite

We are now ready to start creating tests. Read on to learn how to create test suites without using the IDE, or skip ahead to Recording Tests and Verification Points (Section 4.8.1.3) if you prefer.

[Note]For command-line users

To create a new test suite from the command line, three steps are necessary: first, create a directory for the test suite; second, register the AUT with squishserver and third, create a test suite configuration file.

  1. Create a new directory to hold the test suite—the directory's name should begin with suite_. In this example we have created the SQUISHDIR/examples/mac/addressbook/suite_py directory for Python tests. (We also have similar subdirectories for other languages but this is purely for the sake of example, since normally we only use one language for all our tests.)

  2. Register the AUT with the squishserver. [12] This is done by executing the squishserver on the command line with the --config option and the addAUT command. For example, assuming we are in the SQUISHDIR directory on macOS:

    squishserver --config addAUT SquishAddressBook \
    SQUISHDIR/examples/mac/addressbook
    

    We must give the addAUT command the name of the AUT's executable and—separately—the AUT's path. In this case the path is to the executable that was added as the AUT in the test suite configuration file. (For more information about application paths, see AUTs and Settings (Section 7.3) in the User Guide (Chapter 5), and for more about the squishserver's command line options see squishserver (Section 7.4.4) in the Tools Reference Manual (Chapter 7).)

  3. Create a plain text file (ASCII or UTF-8 encoding) called suite.conf in the suite subdirectory. This is the test suite's configuration file, and at the minimum it must identify the AUT, the scripting language used for the tests, and the wrappers (i.e., the GUI toolkit or library) that the AUT uses. The format of the file is key = value, with one key–value pair per line. For example:

    AUT            = SquishAddressBook
    LANGUAGE       = Python
    WRAPPERS       = Mac
    OBJECTMAPSTYLE = script
    

    The AUT is the Cocoa executable registered during the previous step. The LANGUAGE can be set to whichever one you prefer—currently Squish is capable of supporting JavaScript, Python, Perl, Ruby, and Tcl. The WRAPPERS should be set to Mac.

We are now ready to record our first test.

4.8.1.3. Recording Tests and Verification Points

Squish records tests using the scripting language that was specified for the test suite, rather than using a proprietary language. Once a test has been recorded we can run the test and Squish will faithfully repeat all the actions that we performed when recording the test, but without all the pauses that humans are prone to but which computers don't need. It is also possible—and very common—to edit recorded tests, or to copy parts of recorded tests into manually created tests, as we will see later on in the tutorial.

Recordings are made into existing test cases. We begin by creating a New Script Test Case. There are two ways we can do this. One way is to click File|New Test Case.... This will pop up the New Squish Test Case wizard (Section 8.3.10)—simply enter the name for the test case and then click Finish. Another way is to click the New Test Case () toolbar button (to the right of the Test Cases label in the Test Suites view); this will create a new test case with a default name (which you can easily change). Use one of these methods and give the new test case the name “tst_general”. Squish automatically creates a sub-folder inside the test suite's folder with this name and also a test file, for example test.py. (If we had chosen JavaScript as our scripting language the file would be called test.js, and correspondingly for Perl, Ruby, or Tcl.)

The Squish IDE with the tst_general test case
[Note]Note

If you get a sample .feature file instead of an empty script file, then click on the arrow left to the Run Test Suite button and select New Script Test Case.

To make the test script file (e.g., test.py) appear in an Editor view (Section 8.2.6), click—or double-click depending on the Preferences|General|Open mode setting—the test case. (Incidentally, the checkboxes are used to control which test cases are run when the Run Test Suite toolbar button is clicked; we can always run a single test case by clicking its Run Test button.) Initially, the script is empty. If we were to create a test manually (as we will do later on in the tutorial), we must define a main function. The name "main" is special to Squish—tests may contain as many functions and other code as we like (providing it is legal in the scripting language), but when the test is executed (i.e., run), Squish executes the main() function. This is actually very convenient since it means we are free to create other functions, import libraries, and so on, without problems. It is also possible to share commonly used code between test scripts—this is covered in the User Guide (Chapter 5). (In fact, two other function names are special to Squish, cleanup and init; see Tester-Created Special Functions (Section 6.1) for details.)

Once the new empty test case has been created, we are free to write test code manually, or to record a test. If we choose to record we can either replace all the test's code with the recorded code, or insert recorded code into the middle of some existing test code. We will only concern ourselves with recording and replacing in the tutorial.

[Note]For command-line users

Creating a new test case from the command line is a two-step process: first, create a test case directory; and second, create an empty test case script.

  1. Create a new subdirectory inside the test suite directory. For example, inside the SQUISHDIR/examples/mac/addressbook/suite_py directory, create the tst_general directory.

  2. Inside the test case's directory create an empty file called test.py (or test.js if you are using the JavaScript scripting language, and similarly for the other languages).

Before we dive into recording let's briefly review our very simple test scenario:

  1. Open the MyAddresses.adr address file.

  2. Navigate to the second address and then add a new name and address.

  3. Navigate to the fourth address (that was the third address) and change the surname field.

  4. Navigate to the first address and remove it.

  5. Verify that the first address is now the new one that was added.

We are now ready to record our first test. Click the Record () to the right of the tst_general test case shown in the Test Suites view (Section 8.2.18)'s Test Cases list. This will cause Squish to run the AUT so that we can interact with it. Once the AUT is running perform the following actions—and don't worry about how long it takes since Squish doesn't record idle time:

  1. Click File|Open, and once the file dialog appears, press Shift+Command+g and enter the MyAddresses.adr filename in the line edit that appears, then click the Go button, then click the Open button.

  2. Click the first row, then click the Add toolbar button, then in the Add sheet's first line edit type in "Jane". Now click (or tab to) the second line edit and type in "Doe". Continue similarly, to set an email address of "jane.doe@nowhere.com" and a phone number of "555 123 4567". Don't worry about typing mistakes—just backspace delete as normal and fix them. Finally, click the Add button. There should now be a new second address with the details you typed in.

  3. Double-click the fourth row's second (surname) column, delete its text and replace it with "Doe". (You can do this simply by overtyping, then pressing Enter.)

  4. Click the first row, then click the Remove toolbar button and then click the Remove button in the message box. The first row should be gone, so your "Jane Doe" entry should now be the first one.

  5. In the Squish Control Bar, click the Verify toolbar button and select Properties from the drop-down menu.

    This will make the Squish IDE appear, and the central widget should show you a Verification Point Creator view (Section 8.2.21). Before selecting any properties, make sure the Type of the VP is Scriptified Property, as shown in the following screenshots.

    In the Application Objects view, expand the Address Book - MyAddresses.adr_NSWindow_0 object, then the NSView_0 object, then the NSScrollView_0 object, then the NSClipView_0 object, and then the NSTableView_0 object. Click the Jane_NSCFString_0 object to make its properties appear in the Properties view (Section 8.2.12), and then check the stringValue property's checkbox. Now scroll down and click the Doe_NSCFString_127 object and check its stringValue property. Finally, click the Save and Insert Verifications button (at the bottom of the Verification Point Creator) to have the forename and surname verifications for the first row inserted into the recorded test script. (See the screenshot below.) Once the verification points are inserted, the Squish IDE's window will be hidden again and the Control Bar window and the AUT will be back in view.

  6. We've now completed the test, so in the AUT click SquishAddressBook|Quit, then click Don't Save in the message box, since we don't want to save any changes.

The Squish IDE showing two verification points about to be inserted

Once we quit the AUT, the recorded test will appear in Squish's IDE as the screenshot illustrates. (Note that the exact code that is recorded will vary depending on how you interact. For example, you might invoke menu options by clicking them or by using key sequences—it doesn't matter which you use, but since they are different, Squish will record them differently.)

The Squish IDE showing the recorded tst_general test

If the recorded test doesn't appear, click (or double-click depending on your platform and settings) the tst_general test case; this will make Squish show the test's test.py file in an editor window as shown in the screenshot.

Now that we've recorded the test we are able to play it back, i.e., run it. This in itself is useful in that if the play back failed it might mean that the application has been broken. Furthermore, the two verifications we put in will be checked on play back as the screenshot shows.

Inserting verification points during test recording is very convenient. Here we inserted two in one go, but we can insert as many as we like as often as we like during the test recording process. However, sometimes we might forget to insert a verification, or later on we might want to insert a new verification. We can easily insert additional verifications into a recorded test script as we will see in the next section, Inserting Additional Verification Points (Section 4.1.1.4).

Before going further we will look at how to record a test from the command line. Then we will see how to run a test, and we will also look at some of the code that Squish generated to record the test and discuss some of its features.

[Note]For command-line users

First and foremost, the squishserver must always be running when recording or running a test. This is handled automatically by the Squish IDE, but for command line users the squishserver must be started manually. (See squishserver (Section 7.4.4) for further details.)

To record a test from the command line we execute the squishrunner program and specify the test suite we want to record inside and the name we want to give to the test case. For example (assuming we are in the directory that contains the test suite's directory):

squishrunner --testsuite suite_py --record tst_general --useWaitFor

It is always best to record using the --useWaitFor option since this records calls to the waitForObject function which is more reliable than using the snooze function which for historical reasons is the default. (Note that the Squish IDE automatically uses the waitForObject function.)

To run a test case in the IDE just click the Run Test Case toolbar button (the green right-pointing triangle that appears when the test case is selected in the Test Suites view (Section 8.2.18)). This will cause Squish to run the AUT and replay every action (omitting human idle time, but allowing just enough time for the GUI toolkit to keep up). It is worth trying out since it has quite an impressive effect, especially if you haven't seen it in action before.

When we have two or more test cases we can run them individually by clicking the test case we want to run to select it and then clicking the Run Test button, or we can run them all (one after the other) by clicking the Run Test Suite toolbar button (which is above and slightly to the left of the Run Test button. (Actually, only those test cases that are checked are run by clicking the Run Test Suite toolbar button, so we can easily run a particular group of tests.)

[Note]For command-line users

As noted earlier, the squishserver must always be running when recording or running a test. (See squishserver (Section 7.4.4) for further details.)

To play back a recorded test from the command line we execute the squishrunner program and specify the test suite our recorded script is in and the test case we want to play. For example (assuming we are in the directory that contains the test suite's directory):

squishrunner --testsuite suite_py --testcase tst_general

If you look at the code in the screenshot (or the code snippet shown below) you will see that it consists of lots of waitForObject calls as parameters to various other calls such as activateItem, mouseClick, and type. The waitForObject function waits until a GUI object is ready to be interacted with (i.e., becomes visible and enabled), and is then followed by some function that interacts with the object. The typical interactions are activate (pop-up) a menu, click a menu option or a button, or type in some text. (For a complete overview of Squish's script commands see the User Guide (Chapter 5), the API Reference Manual (Chapter 6), and the Tools Reference Manual (Chapter 7). Objects are identified by names that Squish generates. (See How to Identify and Access Objects (Section 5.1) for full details.)

The generated code is about 35 lines of code. Here's an extract that just shows how Squish records clicking the Add toolbar button, typing in Jane Doe's details into the Add sheet, and clicking Add at the end to close the sheet and update the table.

[Note]Scripting Language Support

Although the screenshots only show the Python test suite in action, for the code snippets quoted here and throughout the tutorial, we show the code for all the scripting languages that Squish supports. In practice you would normally only use one of them of course, so feel free to just look at the snippets in the language you are interested in and skip the others. (In the HTML version of this manual you can use the combobox at the top of the page to select the language you use—this will hide the code snippets in other languages.)

Python

    mouseClick(waitForObject(names.address_Book_MyAddresses_adr_Add_NSToolbarItem))
    type(waitForObject(names.add_Address_NSTextField), "Jane")
    type(waitForObject(names.add_Address_NSTextField), "<Tab>")
    type(waitForObject(names.add_Address_NSTextField_2), "Doe")
    type(waitForObject(names.add_Address_NSTextField_2), "<Tab>")
    type(waitForObject(names.add_Address_NSTextField_3), "jane.doe@nowhere.com")
    type(waitForObject(names.add_Address_NSTextField_3), "<Tab>")
    type(waitForObject(names.add_Address_NSTextField_4), "555 123 4567")
    mouseClick(waitForObject(names.add_Address_Add_NSButton))
JavaScript

    mouseClick(waitForObject(names.addressBookMyAddressesAdrAddNSToolbarItem));
    type(waitForObject(names.addAddressNSTextField), "Jane");
    type(waitForObject(names.addAddressNSTextField), "<Tab>");
    type(waitForObject(names.addAddressNSTextField_2), "Doe");
    type(waitForObject(names.addAddressNSTextField_2), "<Tab>");
    type(waitForObject(names.addAddressNSTextField_3), "jane.doe@nowhere.com");
    type(waitForObject(names.addAddressNSTextField_3), "<Tab>");
    type(waitForObject(names.addAddressNSTextField_4), "555 123 4567");
    mouseClick(waitForObject(names.addAddressAddNSButton));
Perl

    mouseClick(waitForObject($Names::address_book_myaddresses_adr_add_nstoolbaritem));
    type(waitForObject($Names::add_address_nstextfield), "Jane");
    type(waitForObject($Names::add_address_nstextfield), "<Tab>");
    type(waitForObject($Names::add_address_nstextfield_2), "Doe");
    type(waitForObject($Names::add_address_nstextfield_2), "<Tab>");
    type(waitForObject($Names::add_address_nstextfield_3), "jane.doe\@nowhere.com");
    type(waitForObject($Names::add_address_nstextfield_3), "<Tab>");
    type(waitForObject($Names::add_address_nstextfield_4), "555 123 4567");
    mouseClick(waitForObject($Names::add_address_add_nsbutton));
Ruby

    mouseClick(waitForObject(Names::Address_Book_MyAddresses_adr_Add_NSToolbarItem))
    type(waitForObject(Names::Add_Address_NSTextField), "Jane")
    type(waitForObject(Names::Add_Address_NSTextField), "<Tab>")
    type(waitForObject(Names::Add_Address_NSTextField_2), "Doe")
    type(waitForObject(Names::Add_Address_NSTextField_2), "<Tab>")
    type(waitForObject(Names::Add_Address_NSTextField_3), "jane.doe@nowhere.com")
    type(waitForObject(Names::Add_Address_NSTextField_3), "<Tab>")
    type(waitForObject(Names::Add_Address_NSTextField_4), "555 123 4567")
    mouseClick(waitForObject(Names::Add_Address_Add_NSButton))
Tcl

    invoke mouseClick [waitForObject $names::Address_Book_MyAddresses_adr_Add_NSToolbarItem]
    invoke type [waitForObject $names::Add_Address_NSTextField] "Jane"
    invoke type [waitForObject $names::Add_Address_NSTextField] "<Tab>"
    invoke type [waitForObject $names::Add_Address_NSTextField_2] "Doe"
    invoke type [waitForObject $names::Add_Address_NSTextField_2] "<Tab>"
    invoke type [waitForObject $names::Add_Address_NSTextField_3] "jane.doe@nowhere.com"
    invoke type [waitForObject $names::Add_Address_NSTextField_3] "<Tab>"
    invoke type [waitForObject $names::Add_Address_NSTextField_4] "555 123 4567"
    invoke mouseClick [waitForObject $names::Add_Address_Add_NSButton]

As you can see the tester used the keyboard to tab from one text field to another and clicked the Add button using the mouse rather than with a key press. If the tester had clicked the button any other way (for example, by pressing Enter), the outcome would be the same, but of course Squish will have recorded the actual actions that were taken.

[Note] Object Names

Squish recordings refer to objects using variables that begin with a names. prefix. This identifies them as Symbolic Names. Each variable contains, as a value, the corresponding Real Name, which can be string-based, or implemented as a key-value mapping of properties to values. Squish supports several naming schemes, all of which can be used—and mixed—in scripts. The advantage of using Symbolic Names is that if the application changes in a way that results in different names being needed, we can simply update Squish's Object Map (which relates symbolic names to real names), and thereby avoid the need to change our test scripts. (See the Object Map (Section 7.11) and the Object Map view (Section 8.2.10) for more about the Object Map.)

[Tip] Editor Context Menu

When a Symbolic Name is under the cursor, the editor's context menu allows you to Open Symbolic Name, showing its entry in the Object Map, or Convert to Real Name, which places its inline key-value representation in your desired script language at the cursor, allowing you to hand-edit the values and properties in your script.

Now that we have seen how to record and play back a test and have seen the code that Squish generates, let's go a step further and make sure that at particular points in the test's execution certain conditions hold.

4.8.1.4. Inserting Additional Verification Points

In the previous section we saw how easy it is to insert scriptified property verification points during the recording of test scripts. Verification points can also be inserted into existing test scripts, either by setting a breakpoint and using the Squish IDE, or simply by editing a test script and putting in calls to Squish's test functions such as test.compare and test.verify.

Squish supports four kinds of verification points: those that verify that object properties have particular values—known as "Object Property Verifications"; those that verify that an entire table has the contents we expect—known as "Table Verifications"; those that verify that two images match—known as "Screenshot Verifications"; and a hybrid verification type that includes properties and screenshots from multiple objects, known as "Visual Verifications". The most commonly used kind is object property verifications, and it is these that we will cover in the tutorial. For further reading, see How to Create and Use Verification Points (Section 5.22)).

Regular (non-scriptified) property verification points are stored as XML files in the test case or test suite resources, and contain the value(s) that need to be passed to test.compare(). These verification points can be reused across test cases, and can verify many values in a single line of script code.

Scriptified property verification points are direct calls to the test.compare function, with two arguments—the value of a particular property for a particular object, and an expected value. We can manually insert calls to the test.compare function in a recorded or hand written script, or we can get Squish to insert them for us using scriptified verification points. In the previous section we showed how to use the Squish IDE to insert verifications during recording. Here we will first show how to use the Squish IDE to insert verifications into an existing test script, and then we will show how to insert a verification by hand.

Before asking Squish to insert verification points, it is best to make sure that we have a list of what we want to verify and when. There are many potential verifications we could add to the tst_general test case, but since our concern here is simply to show how to do it, we will only do two—we will verify that the "Jane Doe" entry's email address and phone number match the ones entered, and put the verifications immediately after the ones we inserted during recording.

To insert a verification point using the IDE we start by putting a break point in the script (whether recorded or manually written—it does not matter to Squish), at the point where we want to verify.

The Squish IDE showing the tst_general test case with a breakpoint

As the above screenshot shows, we have set a breakpoint near the end of the script. This is done simply by double-clicking, or selecting Add Breakpoint after bringing up the context menu from the gutter (next to the line number in the editor). We chose this line because it follows the script lines where the first address is removed, so at this point (just before invoking the File menu to close the application), the first address should be that of "Jane Doe". The screenshot shows the verifications that were entered using the Squish IDE during recording. Our additional verifications will follow them. (Note that your line number may be different if you recorded the test in a different way, for example, using keyboard shortcuts rather than clicking menu items.)

Having set the breakpoint, we now run the test as usual by clicking the Run Test button, or by clicking the Run|Run Test Case menu option. Unlike a normal test run the test will stop when the breakpoint is reached, and Squish's main window will reappear (which will probably obscure the AUT). At this point the Squish IDE will automatically switch to the Squish Test Debugging Perspective (Section 8.1.2.3).

[Note]Perspectives and Views

The Squish IDE works just like the Eclipse IDE. If you aren't used to Eclipse it is crucial to understand one key concept: Views and Perspectives. In Eclipse (and therefore in the Squish IDE), a View is essentially a child window (perhaps a dock window, or a tab in an existing window). And a Perspective is a collection of Views arranged together. Both are accessible through the Window menu.

The Squish IDE is supplied with three Perspectives—the Squish Test Management Perspective (Section 8.1.2.2) (which is the Perspective that the Squish IDE starts with, and the one we have seen in all previous screenshots), Squish Test Debugging Perspective (Section 8.1.2.3), and Squish Spy Perspective (Section 8.1.2.1). You can change these Perspectives to include additional Views (or to get rid of any Views that you don't want), and you can create your own Perspectives with exactly the Views you want. So if your windows change dramatically it just means that the Perspective changed; you can always use the Window menu to change back to the Perspective you want. In practice, Squish will automatically change perspective to reflect the current situation, so it isn't really necessary to change perspective manually.

As the screenshot below shows, when Squish stops at a breakpoint the Squish IDE automatically changes to the Squish Test Debugging Perspective (Section 8.1.2.3). The perspective shows the Variables view (Section 8.2.20), the Editor view (Section 8.2.6), the Debug view (Section 8.2.5), the Application Objects view (Section 8.2.1), and the Properties view (Section 8.2.12), Methods view (Section 8.2.9), and Test Results view (Section 8.2.17).

To insert a verification point we can expand items in the Application Objects view until we find the object we want to verify, or we can use the Object Picker () to visually select it. In this example we want to verify the NSTableView's first row's texts, so we expand the “Address Book - MyAddresses_adr_NSWindow_0” item, and its child items until we find the NSTableView, and within that the item we are interested in. Once we click the item object its properties are shown in the Properties view (Section 8.2.12) as the screenshot shows.

Picking an object to verify in the Application Objects view

The normal Squish Test Management Perspective (Section 8.1.2.2) can be returned to at any time by choosing it from the Window menu (or by clicking its toolbar button), although the Squish IDE will automatically return to it if you stop the script or run it to completion.

Here, we can see that the stringValue property of the first item in the table has the value “Jane”; we already have a verification for this that we inserted during recording. Scroll down so that you can see the corresponding (i.e., first) email address (the jane.doe@nowhere.com_NSCFString_250 item). To make sure that this is verified every time the test is run, select (without checking the box) this object in the Application Objects view (Section 8.2.1) to make its properties appear in the Properties view (Section 8.2.12), and then click the stringValue property to check its check box. After it is checked, the Verification Point Creator view (Section 8.2.21) appears as shown in the screenshot.

Choosing a property value to verify

After the property is selected, the verification point has not yet been added to the test script. We could easily add it by clicking the Save and Insert Verifications button, but before doing that, set the Type of the verification to Property, since the combobox remembers the previously selected VP type. Choosing a meaningful name for the VP is also a good idea at this time. Otherwise we get names like VP1 and VP2, etc.

We'll add one more thing to be verified before we insert it into the script.

Scroll down and click the first phone number item (the 555 123 4567_NSCFString_375 item in the Application Objects view (Section 8.2.1); then check its stringValue property in the Properties view (Section 8.2.12). Now both verifications will appear in the Verification Point Creator view (Section 8.2.21) as the next screenshot shows.

Choosing several property values to verify

We have now said that we expect these properties to have the values shown, that is, an email address of “jane.doe@nowhere.com” and phone number of “555 123 4567”. We must click the Save and Insert Verifications button to actually insert the verification point, so do that now.

We don't need to continue running the test now, so we can either stop running the test at this point (by clicking the Stop toolbar button), or we can continue (by clicking the Resume button).

Once we have finished inserting verifications and stopped or finished running the test we should now disable the break point. Just Ctrl+Click the break point and click the Disable Breakpoint menu option in the context menu. We are now ready to run the test without any breakpoints but with the verification points in place. Click the Run Test button. This time we will get some test results—as the screenshot shows—one of which we have expanded to show its details. (We have also selected the line of code that Squish inserted to perform the verification—notice that the name you picked for the VP is used here).

The newly inserted verification point in action

This particular verification point is stored as an XML file, containing two property comparisons for email, and phone number of the newly inserted entry. You can see it in the Test Case Resources window in the VPs tab, as well as open it in an editor from there.

Another way to insert verification points is to insert them in code form. In theory we can just add our own calls to Squish's test functions such as test.compare and test.verify anywhere we like in an existing script. In practice it is best to make sure that Squish knows about the objects we want to verify first so that it can find them when the test is run. This involves a very similar procedure as using the Squish IDE. First we set a breakpoint where we intend adding our verifications. Then we run the test script until it stops. Next we navigate in the Application Objects view (Section 8.2.1) until we find the object we want to verify. At this point it is wise to Ctrl+Click the object we are interested in and click the Add to Object Map context menu option. This will ensure that Squish can access the object. Then Ctrl+Click again and click the Copy to clipboard (Symbolic Name) context menu option—this gives us the name of the object that Squish will use to identify it. Now we can edit the test script to add in our own verification and finish or stop the execution. (Don't forget to disable the break point once it isn't needed any more.)

Although we can write our test script code to be exactly the same style as the automatically generated code, it is usually clearer and easier to do things in a slightly different style, as we will explain in a moment.

For our manual verifications we want to check the number of addresses present in the NSTableView after reading in the MyAddresses.adr file, then after the new address is added, and finally after the first address is removed. The screenshot shows two of the lines of code we entered to get one of these three verifications, plus the results of running the test script.

Manually entered verification points in action

When writing scripts by hand, we use Squish's test module's functions to verify conditions at certain points during our test script's execution. As the screenshot (and the code snippets below) show, we begin by retrieving a reference to the object we are interested in. Using the waitForObject function is standard practice for manually written test scripts. This function waits for the object to be available (i.e., visible and enabled), and then returns a reference to it. (Otherwise it times out and raises a catchable exception.) We then use this reference to access the item's properties and methods—in this case the NSTableView's numberOfRows method—and verify that the value is what we expect it to be using the test.verify function. (Incidentally, we got the name for the object from a later line so we didn't need to set a breakpoint and manually add the table's name to the Object Map to ensure that Squish would remember it in this particular case because Squish had already added it during the test recording.)

Here is the code we entered manually for the first verification for all the scripting languages that Squish supports. Naturally, you only need to look at the code for the language that you will be using for your own tests. (For the other verifications we just did calls to the test.verify function—or to the test.compare function for Tcl since it's more convenient—reusing the table object reference we obtained in the code shown below.)

Python

    table = waitForObject(names.address_Book_MyAddresses_adr_NSTableView)
    test.compare(table.numberOfRows(), 125)
JavaScript

    var table = waitForObject(names.addressBookMyAddressesAdrNSTableView);
    test.compare(table.numberOfRows(), 125);
Perl

    my $table = waitForObject($Names::address_book_myaddresses_adr_nstableview);
    test::compare($table->numberOfRows(), 125);
Ruby

    table = waitForObject(Names::Address_Book_MyAddresses_adr_NSTableView)
    Test.compare(table.numberOfRows(), 125)
Tcl

    set table [waitForObject $names::Address_Book_MyAddresses_adr_NSTableView]
    test compare [invoke $table numberOfRows] 125

The coding pattern is very simple: we retrieve a reference to the object we are interested in and then verify its properties using one of Squish's verification functions. And we can, of course, call methods on the object to interact with it if we wish.

For the Qt toolkit, we have more examples of manually written tests in the Creating Tests by Hand (Section 4.1.1.5) section, and further examples are in the User Guide (Chapter 5).

For complete coverage of verification points, see How to Create and Use Verification Points (Section 5.22) in the User Guide (Chapter 5).

4.8.1.4.1. Test Results

After each test run finishes, the test results—including those for the verification points—are shown in the Test Results view at the bottom of the Squish IDE.

This is a detailed report of the test run and would also contain details of any failures or errors, etc. If you click on a Test Results item, the Squish IDE highlights the script line which generated the test result. And if you expand a Test Results item, you can see additional details of the test.

4.8.1.5. Creating Tests by Hand

Now that we have seen how to record a test and modify it by inserting verification points, we are ready to see how to create tests manually. The easiest way to do this is to modify and refactor recorded tests, although it is also perfectly possible to create manual tests from scratch.

Potentially the most challenging part of writing manual tests is to use the right object names, but in practice, this is rarely a problem. We can either copy the symbolic names that Squish has already added to the Object Map when recording previous tests, or we can copy object names directly from recorded tests. And if we haven't recorded any tests and are starting from scratch we can use the Spy. We do this by clicking the Launch AUT toolbar button. This starts the AUT and switches to the Squish Spy Perspective (Section 8.1.2.1). We can then interact with the AUT until the object we are interested in is visible. Then, inside the Squish IDE we can navigate to the object in the Application Objects view and use the context menu to both add the object to the Object Map (so that Squish will remember it) and to the clipboard (so that we can paste it into our test script). And at the end we can click the Quit AUT toolbar button to terminate the AUT and return Squish to the Squish Test Management Perspective (Section 8.1.2.2). (See How to Use the Spy (Section 5.21.3) in the User Guide (Chapter 5) and the Squish Spy Perspective (Section 8.1.2.1) for more details on using the Spy.)

We can view the Object Map by clicking the Object Map toolbar button (see also, the Object Map view (Section 8.2.10)). Every application object that Squish interacts with is listed here, either as a top-level object, or as a child object (the view is a tree view). We can retrieve the symbolic name used by Squish in recorded scripts by Ctrl+Clicking the object we are interested in and then clicking the context menu's Copy item. This is useful for when we want to modify existing test scripts or when we want to create test scripts from scratch, as we will see later on in the tutorial.

Squish's Object Map

4.8.1.5.1. Modifying and Refactoring Recorded Tests

Suppose we want to test the AUT's Add functionality by adding three new names and addresses. We could of course record such a test but it is just as easy to do everything in code. The steps we need the test script to do are: first click File|New to create a new address book, then for each new name and address, click the Add toolbar button, then fill in the details, and click Add. And finally, Quit without saving. We also want to verify at the start that there are no rows of data and at the end that there are three rows. We will also refactor as we go, to make our code as neat and modular as possible.

First we must create a new empty test case. Click File|New Test Case... and set the test case's name to be tst_adding. Squish will automatically create an empty test.py (or test.js, and so on) file.

Command line users can simply create a tst_adding directory inside the test suite's directory and create and edit the test.py file (or test.js and so on) within that directory.

The first thing we need is a way to start the AUT and then invoke a menu option. Here are the first few lines from the recorded tst_general script:

Python

import names

def main():
    startApplication("SquishAddressBook")
    waitForObject(names.file_NSMenuItem)
    activateItem(names.file_NSMenuItem)
    waitForObject(names.open_NSMenuItem)
    activateItem(names.open_NSMenuItem)
JavaScript

import * as names from 'names.js';

function main()
{
    startApplication("SquishAddressBook");
    waitForObject(names.fileNSMenuItem);
    activateItem(names.fileNSMenuItem);
    waitForObject(names.openNSMenuItem);
    activateItem(names.openNSMenuItem);
Perl

require 'names.pl';

sub main
{
    startApplication("SquishAddressBook");
    waitForObject($Names::file_nsmenuitem);
    activateItem($Names::file_nsmenuitem);
    waitForObject($Names::open_nsmenuitem);
    activateItem($Names::open_nsmenuitem);
Ruby

require 'squish'
require 'names'
include Squish

def main
    startApplication("SquishAddressBook")
    waitForObject(Names::File_NSMenuItem)
    activateItem(Names::File_NSMenuItem)
    waitForObject(Names::Open_NSMenuItem)
    activateItem(Names::Open_NSMenuItem)
Tcl

source [findFile "scripts" "names.tcl"]

proc main {} {
    startApplication "SquishAddressBook"
    waitForObject $names::File_NSMenuItem
    invoke activateItem $names::File_NSMenuItem
    waitForObject $names::Open_NSMenuItem
    invoke activateItem $names::Open_NSMenuItem

Notice that the pattern in the code is simple: start the AUT, then wait for the File menu to appear, then activate the File menu; wait for the menu item, then activate the menu item. In both cases we have used the waitForObject and activateItem functions that we've already seen.

[Note]Note

It may seem a waste to put our functions in tst_adding because we could also use them in tst_general and in other test cases. However, to keep the tutorial simple we will put the code in the tst_adding test case. It is of course very easy to create shared scripts, but we defer coverage of that to the user guide. (See How to Create and Use Shared Data and Shared Scripts (Section 5.23) for how to share scripts.)

If you look at the recorded test (tst_general) or in the Object Map you will see that Squish sometimes uses different names for the same things. For example, the window is identified in two different ways, initially as AddressBook_Untitled_NSWindow, but if the user clicks File|Open and opens the MyAddresses.adr file, the window is then identified as AddressBook_MyAddresses_adr_NSWindow. The reason for this is that Squish needs to uniquely identify every object in a given context, and it uses whatever information it has to hand. So in the case of identifying Main Windows (and their children), Squish uses the window title text to give it some context. (For example, an application's File or Edit menus may have different options depending on whether a file is loaded and what state the application is in.)

Naturally, when we write test scripts we don't want to have to know or care which particular variation of a name to use, and Squish supports this need by providing alternative naming schemes, as we will see shortly.

Sometimes the AUT will appear to freeze during test execution. When this happens, just wait for Squish to time out the AUT (about 20 seconds), and then it will pop up an Object Not Found dialog (Section 8.3.14), indicating an error like this:

Object Not Found Dialog

Don't worry! It just means that Squish doesn't have an object with the given name in the Object Map. From here, we can Pick New Object, Debug, Throw Error, or after picking a new object, Retry. Picking a new object will update the object map entry for the symbolic name. In addition to the Object Picker (), we can also use the Spy's Application Objects view (Section 8.2.1) to locate the objects we are interested in and use the context menu to Add To Object Map. However, recording a dummy test is often quicker for adding lots of objects to the Object Map, providing we interact with all the AUT objects we are interested in.

We've spent a bit of time on the issue of naming since it is probably the part of writing scripts that leads to the most error messages (usually of the "object ... not found" kind shown above.) Once we have identified the objects we are going to access in our tests, writing test scripts using Squish is very straightforward. And of course you can almost certainly use the scripting language you are most familiar with since Squish supports the most popular ones available.

We are now almost ready to write our own test script. It is probably easiest to begin by recording a dummy test. So click File|New Test Case... and set the test case's name to be tst_dummy. Then click the dummy test case's Record (). Once the AUT starts, click File|New, then click the (empty) table, then click Add and add an item, then press Return or click OK. Finally, click SquishAddressBook|Quit Squish Address Book to finish, and say "No" to saving changes. Then replay this test just to confirm that everything works okay. The sole purpose of this is to make sure that Squish adds the necessary names to the Object Map since it is probably quicker to do it this way than to use the Spy for every object of interest. After replaying the dummy test you can delete it if you want to.

With all the object names we need in the Object Map we can now write our own test script completely from scratch. We will start with the main function, and then we will look at the supporting functions that the main function uses.

Python

import names

def main():
    startApplication("SquishAddressBook")
    table = waitForObject(names.address_Book_Untitled_NSTableView)
    mouseClick(waitForObject(names.address_Book_Untitled_New_NSToolbarItem))
    test.verify(table.numberOfRows() == 0)
    data = [("Andy", "Beach", "andy.beach@nowhere.com", "555 123 6786"),
            ("Candy", "Deane", "candy.deane@nowhere.com", "555 234 8765"),
            ("Ed", "Fernleaf", "ed.fernleaf@nowhere.com", "555 876 4654")]
    for oneNameAndAddress in data:
        addNameAndAddress(oneNameAndAddress)
    test.compare(table.numberOfRows(), len(data))
    closeWithoutSaving()
JavaScript

import * as names from 'names.js';

function main()
{
    startApplication("SquishAddressBook");
    var table = waitForObject(names.addressBookUntitledNSTableView);
    mouseClick(waitForObject(names.addressBookUntitledNewNSToolbarItem));
    test.verify(table.numberOfRows() == 0);
    var data = new Array(
        new Array("Andy", "Beach", "andy.beach@nowhere.com", "555 123 6786"),
        new Array("Candy", "Deane", "candy.deane@nowhere.com", "555 234 8765"),
        new Array("Ed", "Fernleaf", "ed.fernleaf@nowhere.com", "555 876 4654"));
    for (var row = 0; row < data.length; ++row)
        addNameAndAddress(data[row]);
    test.compare(table.numberOfRows(), data.length);
    closeWithoutSaving();
}
Perl

require 'names.pl';

sub main
{
    startApplication("SquishAddressBook");
    my $table = waitForObject($Names::address_book_untitled_nstableview);
    mouseClick(waitForObject($Names::address_book_untitled_new_nstoolbaritem));
    test::verify($table->numberOfRows() == 0);
    my @data = (["Andy", "Beach", "andy.beach\@nowhere.com", "555 123 6786"],
                ["Candy", "Deane", "candy.deane\@nowhere.com", "555 234 8765"],
                ["Ed", "Fernleaf", "ed.fernleaf\@nowhere.com", "555 876 4654"]);
    foreach $oneNameAndAddress (@data) {
        addNameAndAddress(@{$oneNameAndAddress});
    }
    test::compare($table->numberOfRows(), scalar(@data));
    closeWithoutSaving();
}
Ruby

require 'squish'
require 'names'
include Squish

def main
    startApplication("SquishAddressBook")
    table = waitForObject(Names::Address_Book_Untitled_NSTableView)
    mouseClick(waitForObject(Names::Address_Book_Untitled_New_NSToolbarItem))
    Test.verify(table.numberOfRows() == 0)
    data = [["Andy", "Beach", "andy.beach@nowhere.com", "555 123 6786"],
          ["Candy", "Deane", "candy.deane@nowhere.com", "555 234 8765"],
          ["Ed", "Fernleaf", "ed.fernleaf@nowhere.com", "555 876 4654"]]
    data.each do |oneNameAndAddress|
        addNameAndAddress(oneNameAndAddress)
    end
    Test.compare(table.numberOfRows(), data.length)
    closeWithoutSaving
end
Tcl

source [findFile "scripts" "names.tcl"]

proc main {} {
    startApplication "SquishAddressBook"
    set table [waitForObject $names::Address_Book_Untitled_NSTableView]
    invoke mouseClick [waitForObject $names::Address_Book_Untitled_New_NSToolbarItem]
    test compare [invoke $table numberOfRows] 0
    set data [list \
        [list "Andy" "Beach" "andy.beach@nowhere.com" "555 123 6786"] \
        [list "Candy" "Deane" "candy.deane@nowhere.com" "555 234 8765"] \
        [list "Ed" "Fernleaf" "ed.fernleaf@nowhere.com" "555 876 4654"] ]
    for {set i 0} {$i < [llength $data]} {incr i} {
        addNameAndAddress [lindex $data $i]
    }
    test compare [invoke $table numberOfRows] [llength $data]
    closeWithoutSaving
}

We begin by starting the application with a call to the startApplication function. The name we pass as a string is the name registered with Squish (normally the name of the executable). Then we obtain a reference to the NSTableView. The object name we used was not put in the Object Map when the tst_general test case was recorded, so we recorded a dummy test to make sure the name was added—we could just as easily have used the Spy of course. We then copied the name from the Object Map into our code. The waitForObject function waits until an object is ready (visible and enabled) and returns a reference to it—or it times out and raises a catchable exception. We have used a symbolic name to access the table—these are the names that Squish uses when recording tests—rather than a real/multi-property name. It is best to use symbolic names where possible because if any AUT object name changes, with symbolic names we just have to update the Object Map (Section 7.11), without needing to change our test code. Once we have the table reference we can use it to access any of the NSTableView's public methods and properties.

Once we have started the AUT and obtained a reference to the NSTableView, we tell Squish to click the New toolbar button to create a new empty address book. Next, we verify that the table's row count is 0. The test.verify function is useful when we simply want to verify that a condition is true rather than compare two different values. (For Tcl we usually use the test.compare function rather than the test.verify function simply because it is slightly simpler to use in Tcl.)

Next, we create some sample data and call a custom addNameAndAddress() function to populate the table with the data using the AUT's Add sheet. Then we again compare the table's row count, this time to the number of rows in our sample data. And finally we call a custom closeWithoutSaving function to terminate the application.

We will now review both of the supporting functions, so as to cover all the code in the tst_adding test case, starting with addNameAndAddress().

Python

def addNameAndAddress(oneNameAndAddress):
    mouseClick(waitForObject(names.address_Book_Untitled_Add_NSToolbarItem))
    type(waitForObject(names.add_Address_NSTextField), oneNameAndAddress[0])
    type(waitForObject(names.add_Address_NSTextField_2), oneNameAndAddress[1])
    type(waitForObject(names.add_Address_NSTextField_3), oneNameAndAddress[2])
    type(waitForObject(names.add_Address_NSTextField_4), oneNameAndAddress[3])
    mouseClick(waitForObject(names.add_Address_Add_NSButton));
JavaScript

function addNameAndAddress(oneNameAndAddress)
{
    mouseClick(waitForObject(names.addressBookUntitledAddNSToolbarItem));
    type(waitForObject(names.addAddressNSTextField), oneNameAndAddress[0]);
    type(waitForObject(names.addAddressNSTextField_2), oneNameAndAddress[1]);
    type(waitForObject(names.addAddressNSTextField_3), oneNameAndAddress[2]);
    type(waitForObject(names.addAddressNSTextField_4), oneNameAndAddress[3]);
    mouseClick(waitForObject(names.addAddressAddNSButton));
}
Perl

sub addNameAndAddress
{
    my (@oneNameAndAddress) = @_;
    mouseClick(waitForObject($Names::address_book_untitled_add_nstoolbaritem));
    type(waitForObject($Names::add_address_nstextfield), $oneNameAndAddress[0]);
    type(waitForObject($Names::add_address_nstextfield_2), $oneNameAndAddress[1]);
    type(waitForObject($Names::add_address_nstextfield_3), $oneNameAndAddress[2]);
    type(waitForObject($Names::add_address_nstextfield_4), $oneNameAndAddress[3]);
    mouseClick(waitForObject($Names::add_address_add_nsbutton));
}
Ruby

def addNameAndAddress(oneNameAndAddress)
    mouseClick(waitForObject(Names::Address_Book_Untitled_Add_NSToolbarItem))
    type(waitForObject(Names::Add_Address_NSTextField), oneNameAndAddress[0])
    type(waitForObject(Names::Add_Address_NSTextField_2), oneNameAndAddress[1])
    type(waitForObject(Names::Add_Address_NSTextField_3), oneNameAndAddress[2])
    type(waitForObject(Names::Add_Address_NSTextField_4), oneNameAndAddress[3])
    mouseClick(waitForObject(Names::Add_Address_Add_NSButton))
end
Tcl

proc addNameAndAddress {oneNameAndAddress} {
    invoke mouseClick [waitForObject $names::Address_Book_Untitled_Add_NSToolbarItem]
    invoke type [waitForObject $names::Add_Address_NSTextField] [lindex $oneNameAndAddress 0]
    invoke type [waitForObject $names::Add_Address_NSTextField_2] [lindex $oneNameAndAddress 1]
    invoke type [waitForObject $names::Add_Address_NSTextField_3] [lindex $oneNameAndAddress 2]
    invoke type [waitForObject $names::Add_Address_NSTextField_4] [lindex $oneNameAndAddress 3]
    invoke mouseClick [waitForObject $names::Add_Address_Add_NSButton]
}

For each set of name and address data we click Add to pop up the Add sheet. Then, for each value received, we populate the appropriate field by waiting for the relevant NSTextField to be ready and then typing in the text using the type function. Finally, we click the sheet's Add button. We got the line at the heart of the function by copying it from the recorded tst_general test and simply parametrizing it by the field name and text. Similarly, we copied the code for clicking the Add button from the tst_general test case's code.

Python

def closeWithoutSaving():
    type(waitForObject(names.address_Book_Untitled_NSTableView), "<Command+q>")
    mouseClick(waitForObject(names.save_Changes_Don_t_Save_NSButton))
JavaScript

function closeWithoutSaving()
{
    type(waitForObject(names.addressBookUntitledNSTableView), "<Command+q>");
    mouseClick(waitForObject(names.saveChangesDonTSaveNSButton));
}
Perl

sub closeWithoutSaving
{
    type(waitForObject($Names::address_book_untitled_nstableview), "<Command+q>");
    mouseClick(waitForObject($Names::save_changes_don_t_save_nsbutton));
}
Ruby

def closeWithoutSaving
    type(waitForObject(Names::Address_Book_Untitled_NSTableView), "<Command+q>")
    mouseClick(waitForObject(Names::Save_Changes_Don_t_Save_NSButton))
end
Tcl

proc closeWithoutSaving {} {
    invoke type [waitForObject $names::Address_Book_Untitled_NSTableView] "<Command+q>"
    invoke mouseClick [waitForObject $names::Save_Changes_Don_t_Save_NSButton]
}

Here we quit the application by pressing Command+Q and then click the "save changes" dialog's Don't Save button. The last line was copied from the recorded test.

The entire test is about 25 lines of code—and would be even less if we put some of the common functions (such as closeWithoutSaving()) in a shared script. And much of the code was copied directly from the recorded test, and in some cases parametrized.

This should be sufficient to give a flavor of writing test scripts for an AUT. Keep in mind that Squish provides far more functionality than we used here. (All of which is covered in the API Reference Manual (Chapter 6) and the Tools Reference Manual (Chapter 7).) And Squish also provides access to the entire public APIs of the AUT's objects.

However, one aspect of the test case is not very satisfactory. Although embedding test data as we did here is sensible for small amounts, it is rather limiting, especially when we want to use a lot of test data. Also, we didn't test any of the data that was added to see if it correctly ended up in the NSTableView. In the next section, we will create a new version of this test, only this time we will pull in the data from an external data source, and check that the data we add to the NSTableView is correct.

4.8.1.5.2. Creating Data Driven Tests

In the previous section we put three hard-coded names and addresses in our test. But what if we want to test lots of data? Or what if we want to change the data without having to change our test script's source code? One approach is to import a dataset into Squish and use the dataset as the source of the values we insert into our tests. Squish can import data in .tsv (tab-separated values format), .csv (comma-separated values format), .xls, or .xlsx (Microsoft® Excel™ spreadsheet formats). [13]

Test data can either be imported using the Squish IDE, or manually using a file manager or console commands. We will describe both approaches, starting with using the Squish IDE.

For the SquishAddressBook application we want to import the MyAddresses.tsv data file. To do this we must start by clicking File|Import Test Resource to pop-up the Import Squish Resource dialog (Section 8.3.6). Inside the dialog click the Browse button to choose the file to import—in this case MyAddresses.tsv. Make sure that the Import As combobox is set to “TestData”. By default the Squish IDE will import the test data just for the current test case, but we want the test data to be available to all the test suite's test cases: to do this check the Copy to Test Suite for Sharing radio button. Now click the Finish button. You can now see the file listed in the Test Suite Resources view (in the Test Data tab), and if you click the file's name it will be shown in an Editor view (Section 8.2.6). The screenshot shows Squish after the test data has been added.

[Note]For command-line users

It is also possible to import test data outside the Squish IDE using a file manager (such as Finder) or console commands. To do this, create a directory inside the test suite's directory called shared. Now make a directory inside the shared directory called testdata. Now copy the data file (in this example, MyAddresses.tsv) into the shared/testdata directory. Now quit the Squish IDE if it is running and start it up again. If you click the Test Suite Resources view's Test Data tab you should see the data file. Click the file's name to see it in an Editor view (Section 8.2.6).

Squish with some imported test data

Although in real life we would modify our tst_adding test case to use the test data, for the purpose of the tutorial we will make a new test case called tst_adding_data that is a copy of tst_adding and which we will modify to make use of the test data.

The only function we have to change is main, where instead of iterating over hard-coded items of data, we iterate over all the records in the dataset. We also need to update the expected row count at the end since we are adding a lot more records now, and we will also add a function to verify each record that's added.

Python

import names

def main():
    startApplication("SquishAddressBook")
    table = waitForObject(names.address_Book_Untitled_NSTableView)
    mouseClick(waitForObject(names.address_Book_Untitled_New_NSToolbarItem))
    test.verify(table.numberOfRows() == 0)
    limit = 10 # To avoid testing 100s of rows
    for row, record in enumerate(testData.dataset("MyAddresses.tsv")):
        forename = testData.field(record, "Forename")
        surname = testData.field(record, "Surname")
        email = testData.field(record, "Email")
        phone = testData.field(record, "Phone")
        addNameAndAddress((forename, surname, email, phone)) # pass a single tuple
        checkNameAndAddress(row, table, record)
        if row > limit:
            break
    test.compare(table.numberOfRows(), row + 1)
    closeWithoutSaving()
JavaScript

import * as names from 'names.js';

function main()
{
    startApplication("SquishAddressBook");
    var table = waitForObject(names.addressBookUntitledNSTableView);
    mouseClick(waitForObject(names.addressBookUntitledNewNSToolbarItem));
    test.verify(table.numberOfRows() == 0);
    var limit = 10; // To avoid testing 100s of rows since that would be boring
    var records = testData.dataset("MyAddresses.tsv");
    for (var row = 0; row < records.length; ++row) {
        var record = records[row];
        var forename = testData.field(record, "Forename");
        var surname = testData.field(record, "Surname");
        var email = testData.field(record, "Email");
        var phone = testData.field(record, "Phone");
        addNameAndAddress(new Array(forename, surname, email, phone));
        checkNameAndAddress(row, table, record);
        if (row > limit)
            break;
    }
    test.compare(table.numberOfRows(), row + 1);
    closeWithoutSaving();
}
Perl

require 'names.pl';

sub main
{
    startApplication("SquishAddressBook");
    my $table = waitForObject($Names::address_book_untitled_nstableview);
    mouseClick(waitForObject($Names::address_book_untitled_new_nstoolbaritem));
    test::verify($table->numberOfRows() == 0);
    my $limit = 10; # To avoid testing 100s of rows since that would be boring
    my @records = testData::dataset("MyAddresses.tsv");
    my $row = 0;
    for (; $row < scalar(@records); ++$row) {
        my $record = $records[$row];
        my $forename = testData::field($record, "Forename");
        my $surname = testData::field($record, "Surname");
        my $email = testData::field($record, "Email");
        my $phone = testData::field($record, "Phone");
        addNameAndAddress($forename, $surname, $email, $phone);
        checkNameAndAddress($row, $table, $record);
        if ($row > $limit) {
            last;
        }
    }
    test::compare($table->numberOfRows(), $row + 1);
    closeWithoutSaving();
}
Ruby

require 'squish'
require 'names'
include Squish

def main
    startApplication("SquishAddressBook")
    table = waitForObject(Names::Address_Book_Untitled_NSTableView)
    mouseClick(waitForObject(Names::Address_Book_Untitled_New_NSToolbarItem))
    Test.verify(table.numberOfRows() == 0)
    limit = 10 # To avoid testing 100s of rows
    rows = 0
    TestData.dataset("MyAddresses.tsv").each_with_index do
        |record, row|
        forename = TestData.field(record, "Forename")
        surname = TestData.field(record, "Surname")
        email = TestData.field(record, "Email")
        phone = TestData.field(record, "Phone")
        addNameAndAddress([forename, surname, email, phone]) # pass as a single Array
        checkNameAndAddress(row, table, record)
        break if row > limit
        rows += 1
    end
    Test.verify(table.numberOfRows() == rows + 1)
    closeWithoutSaving
end
Tcl

source [findFile "scripts" "names.tcl"]

proc main {} {
    startApplication "SquishAddressBook"
    set table [waitForObject $names::Address_Book_Untitled_NSTableView]
    invoke mouseClick [waitForObject $names::Address_Book_Untitled_New_NSToolbarItem]
    test compare [invoke $table numberOfRows] 0
    set limit 10
    set data [testData dataset "MyAddresses.tsv"]
    set columns [llength [testData fieldNames [lindex $data 0]]]
    set row 0
    for {} {$row < [llength $data]} {incr row} {
        set record [lindex $data $row]
        set forename [testData field $record "Forename"]
        set surname [testData field $record "Surname"]
        set email [testData field $record "Email"]
        set phone [testData field $record "Phone"]
        set details [list $forename $surname $email $phone]
        addNameAndAddress $details
        checkNameAndAddress $row $table $record
        if {$row > $limit} {
            break
        }
    }
    test compare [invoke $table numberOfRows] [expr $row + 1]
    closeWithoutSaving
}

Squish provides access to test data through its testData module's functions—here we used the testData.dataset function to access the data file and make its records available, and the testData.field function to retrieve each record's individual fields.

Having used the test data to populate the NSTableView we want to be confident that the data in the table is the same as what we have added, so that's why we added the checkNameAndAddress function. We also added a limit to how many records we would compare, just to make the test run faster.

Python

def checkNameAndAddress(row, table, record):
    for column in range(len(testData.fieldNames(record))):
        cellName = {'rowNumber': row, 'columnNumber': column, 'tableView': names.address_Book_Untitled_NSTableView, 'type': 'NSString'}
        cell = waitForObject(cellName)
        test.compare(cell.stringValue, testData.field(record, column))
JavaScript

function checkNameAndAddress(row, table, record)
{
    for (var column = 0; column < testData.fieldNames(record).length; ++column) {
        var cellName = {'rowNumber': row, 'columnNumber': column, 'tableView': names.addressBookUntitledNSTableView, 'type': 'NSString'};
        var cell = waitForObject(cellName);
        test.compare(cell.stringValue, testData.field(record, column));
    }
}
Perl

def checkNameAndAddress(row, table, record)
    for column in 0...TestData.fieldNames(record).length
        cellName = {:rowNumber => row, :columnNumber => column, :tableView => Names::Address_Book_Untitled_NSTableView, :type => 'NSString'}
        cell = waitForObject(cellName)
        Test.compare(cell.stringValue, TestData.field(record, column))
    end
end
Tcl

proc checkNameAndAddress {row table record} {
    set columns [llength [testData fieldNames $record]]
    for {set column 0} {$column < $columns} {incr column} {
        set cellName [::Squish::ObjectName rowNumber $row columnNumber $column tableView $names::Address_Book_Untitled_NSTableView type NSString]
        set cell [waitForObject $cellName]
        test compare [property get $cell stringValue] [testData field $record $column]
    }
}

This function accesses the give NSTableView row and extracts each of its columns' values. We use Squish's testData.fieldNames function to get a column count and then use the test.compare function to check that each value in the table is the same as the value in the test data we used.

As we mentioned earlier, the symbolic names Squish uses for menus and menu items (and other objects) can vary depending on the context, and often with the start of the name derived from the window's title. For applications that put the current filename in the title—such as the Address Book example—names will include the filename, and we must account for this.

In the case of the Address Book example, the main window's title is “Address Book” (at startup), or “Address Book - Unnamed” (after File|New , but before File|Save or File |Save As), or “Address Book - filename” where the filename can of course vary. Our code accounts for all these cases by making use of real (multi-property) names.

Symbolic names embed various bits of information about an object and its type. Real names are represented by a brace enclosed list of space-separated key–value pairs. Every real name must specify the type property and at least one other property. Here we've used the rowNumber, columnNumber, tableView, and type properties to uniquely identify each cell in the table so that we can compare each cell's stringValue with the data used to populate it.

The screenshot show Squish's Test Summary log after the data-driven tests have been run.

Squish after a successful data-driven test run

Squish can also do keyword-driven testing. This is a bit more sophisticated than data-driven testing. See How to Do Keyword-Driven Testing (Section 5.16).

4.8.1.6. Learning More

We have now completed the tutorial! Squish can of course do much more than we have shown here, but the aim has been to get you started with basic testing as quickly and easily as possible. The User Guide (Chapter 5) provides many more examples, including those that show how tests can interact with particular widgets such as spinboxes, comboboxes, date/time editors, and line editors, and of course with list, table, and tree views.

The API Reference Manual (Chapter 6) and Tools Reference Manual (Chapter 7) give full details of Squish's testing API and the numerous functions it offers to make testing as easy and efficient as possible. It is well worth reading the User Guide (Chapter 5) and at least skimming the API Reference Manual (Chapter 6) and Tools Reference Manual (Chapter 7)—especially since the time invested will be repaid because you'll know what functionality Squish provides out of the box and can avoid reinventing things that are already available.

If you are interested in testing iPhone Apps, further details are given here: Tutorial: Starting to Test iOS Applications (Section 4.9.1).

4.8.2. Tutorial: Designing Behavior Driven Development (BDD) Tests

This tutorial will show you how to create, run, and modify Behavior Driven Development (BDD) tests for an example application. You will learn about Squish's most frequently used features. By the end of the tutorial you will be able to write tests for your own applications.

For this chapter we will use a simple address book application as our Application Under Test (AUT). It is called SquishAddressBook.app in your examples/addressbook folder. This is a very basic application that allows users to load an existing address book or create a new one, add, edit, and remove entries. The screenshot shows the application with a newly created, empty address book.

The Mac OS X SquishAddressBook.app example.

4.8.2.1. Introduction to Behavior Driven Development

Behavior-Driven Development (BDD) is an extension of the Test-Driven Development approach which puts the definition of acceptance criteria at the beginning of the development process, as opposed to writing tests after the software has been developed. This process also allows for possible cycles of code changes after testing.

BDD process

Behavior Driven Tests are built out of a set of Feature files, which describe product features through the expected application behavior in one or many Scenarios. Each Scenario is built out of a sequence of steps which represent actions or verifications that need to be tested for that Scenario.

BDD focuses on expected application behavior, not on implementation details. Therefore BDD tests are described in a human-readable Domain Specific Language (DSL). As this language is not technical, such tests can be created not only by programmers, but also by product owners, testers or business analysts. Additionally, during the product development, such tests serve as living product documentation. For Squish usage, BDD tests shall be created using Gherkin syntax. The previously written product specification (BDD tests) can be turned into executable tests. This step by step tutorial presents automating BDD tests with Squish IDE support.

4.8.2.2. Gherkin Syntax

Gherkin files describe product features through the expected application behavior in one or many Scenarios. An example showing the "Filling of the address book" feature of the address book example application.


Feature: Filling of the address book
    As a user I want to fill the address book with entries

    Scenario: Initial state of the newly created address book
        Given the address book application is running
        When I create a new address book
        Then the address book should be empty

    Scenario: State after adding one entry
        Given the address book application is running
        When I create a new address book
        And I add a new person 'John','Doe','john@m.com','500600700' to the address book
        Then '1' entries should be present

    Scenario: State after adding two entries
        Given the address book application is running
        When I create a new address book
        And I add new persons to the address book
            | forename  | surname  | email        | phone  |
            | John      | Smith    | john@m.com   | 123123 |
            | Alice     | Thomson  | alice@m.com  | 234234 |
        Then '2' entries should be present

    Scenario: Forename and surname is added to table
        Given the address book application is running
        When I create a new address book
        And I add a new person 'Bob','Doe','Bob@m.com','123321231' to the address book
        Then previously entered forename and surname shall be at the top

Most of the above is free form text (does not have to be English). It's just the Feature/Scenario structure and the leading keywords like Given, And, When and Then that are fixed. Each of those keywords marks a step defining preconditions, user actions and expected results. The application behavior description above can be passed between software developers to implement the features, while at the same time, the same description can be passed to software testers to implement automated tests.

4.8.2.3. Test Implementation

4.8.2.3.1. Creating a Test Suite

First, we need to create a test suite, which is a container for all test cases. Start the squishide and select File|New Test Suite.... Please follow the New Test Suite wizard, provide a test suite name, choose the Mac toolkit and a scripting language of your choice and finally register the SquishAddressBook application as AUT. Please refer to Creating a Test Suite (Section 4.8.1.2) for more details about creating new test suites.

4.8.2.3.2. Creating a Test Case

Squish offers two types of test cases: "Script Test Case" and "BDD Test Case". As "Script Test Case" is the default one, in order to create a new BDD Test Case we need to use the drop-down menu by clicking on the expander next to New Test Case () and choosing New BDD Test Case. The Squish IDE will remember your choice and "BDD Test Case" will become the default when clicking on the button in the future.

Creating a new BDD Test Case

The newly created BDD test case consists of a test.feature file (filled with a Gherkin template while creating a new BDD test case), a file named test.(py|js|pl|rb|tcl) which will drive the execution (there is no need to edit this file), and a test suite resources file named shared/steps/steps.(py|js|pl|rb|tcl) where step implementation code will be placed.

We need to replace the Gherkin template with a Feature for the example application. To do this, copy the Feature description below and paste it into the Feature file.


Feature: Filling of the address book
    As a user I want to fill the address book with entries

    Scenario: Initial state of the newly created address book
        Given the address book application is running
        When I create a new address book
        Then the address book should be empty

When editing the test.feature file, a Feature file warning No implementation found is displayed for each undefined step. The implementations are in the steps subdirectory, in Test Case Resources, or in Test Suite Resources. Running our Feature test now will currently fail at the first step with a No Matching Step Definition and the following steps will be skipped.

4.8.2.3.3. Recording Step Implementation

In order to record the Scenario, press the Record () next to the respective Scenario that is listed in the Scenarios tab in the Test Case Resources view.

Record Scenario

This will cause Squish to run the AUT so that we can interact with it. Additionally, the Control Bar is displayed with a list of all steps that need to be recorded. Now all interaction with the AUT or any verification points added to the script will be recorded under the currently recording step, Given the address book application is running (which is bolded in the Step list on the Control Bar). In order to verify that this precondition is met, we will add a Verification Point. To do this, click on Verify button in the Control Bar and then select Properties.

Control Bar

As a result the Squish IDE is put into Spy mode which displays all Application Objects and their Properties in dockable Views. In the Application Objects view, select the main window item (Address Book - Untitled_NSWindow). Selecting it will update the Properties view on the right side. Next click on the checkbox in front of the property isVisible in the Properties View. Finally, click on the button Save and Insert Verifications. The Squish IDE disappears and the Control Bar is shown again.

Inserting a Verification Point

When we are done with each Step, we can move to the next undefined step (playing back the ones that were previously defined) by clicking on the Finish Recording Step... () arrow button in the Control Bar that is located to the left of the current step.

Next, for the Step When I create a new address book, click on the New button () in the toolbar of the addressbook application and click on the Finish Recording Step... () in the Squish control bar, to finish recording and go to the next step.

Finally, for the step the address book should be empty, verify that the table containing the address entries is empty. To record this verification, click on Verify while recording and select Properties. Now, in Application Objects, navigate or use the Object Picker () to select the table containing the address book entries (in our case this table is empty). You will notice that the Table object does not provide a property for the number of rows. This information is not directly available. Therefore we need to modify the recorded code later on. To let Squish generate some test.compare() code for our step, insert a verification for a random property, for example alignment. Finally, click on the last Finish Recording Step... () arrow button in the Control Bar.

As a result, Squish will generate the following step definitions in the steps.* file (in the Steps tab of the Test Suite Resources view):

Python

@Given("the address book application is running")
def step(context):
    startApplication("SquishAddressBook")
    test.compare(waitForObjectExists(names.o_NSWindow).isVisible, 0)

@When("I create a new address book")
def step(context):
    mouseClick(waitForObject(names.address_Book_Untitled_New_NSToolbarItem))

@Then("the address book should be empty")
def step(context):
    #test.compare(waitForObjectExists(names.address_Book_Untitled_NSTableView).alignment, 0)
    test.compare(waitForObjectExists(names.address_Book_Untitled_NSTableView).numberOfRows(), 0)

JavaScript

Given("addressbook application is running", function(context) {
    startApplication("SquishAddressBook");
    test.compare(waitForObjectExists(names.addressBookUntitledNSWindow).isVisible, 1);
});

When("I create a new addressbook", function(context) {
    mouseClick(waitForObject(names.addressBookUntitledNewNSToolbarItem));
});

Then("addressbook should have zero entries", function(context) {
    //test.compare((waitForObject(names.addressBookUntitledNSTableView)).alignment, 0);
    test.compare((waitForObject(names.addressBookUntitledNSTableView)).numberOfRows(), 0);
});

Perl

Given("the address book application is running", sub {
    my $context = shift;
    startApplication("SquishAddressBook");
    test::compare(waitForObjectExists($Names::address_book_untitled_nswindow)->isVisible, 1);
});

When("I create a new address book", sub {
    my $context = shift;
    mouseClick(waitForObject($Names::address_book_untitled_new_nstoolbaritem));
});

Then("the address book should be empty", sub {
    my $context = shift;
    #test::compare(waitForObjectExists($Names::address_book_untitled_nstableview)->alignment, 1);
    test::compare(waitForObjectExists($Names::address_book_untitled_nstableview)->numberOfRows, 0);
});

Ruby

Given("the address book application is running") do |context|
    startApplication("SquishAddressBook")
    Test.compare(waitForObjectExists(Names::O_NSWindow).isVisible, 0)
end

When("I create a new address book") do |context|
    mouseClick(waitForObject(Names::Address_Book_Untitled_New_NSToolbarItem))
end

Then("the address book should be empty") do |context|
    #Test.compare(waitForObjectExists(Names::Address_Book_Untitled_NSTableView).alignment, 0)
    Test.compare(waitForObjectExists(Names::Address_Book_Untitled_NSTableView).numberOfRows(), 0)
end

Tcl

Given "the address book application is running" {context} {
    startApplication "SquishAddressBook"
    test compare [property get [waitForObjectExists $names::NSWindow] isVisible] 0
}

When "I create a new address book" {context} {
    invoke mouseClick [waitForObject $names::Address_Book_Untitled_New_NSToolbarItem]
}

Then "the address book should be empty" {context} {
    test compare [invoke [waitForObjectExists $names::Address_Book_Untitled_NSTableView] numberOfRows] 0
}

In order to get the number of rows, we have to call a method of the underlying NSTableView object.

The application is automatically started at the beginning of the first step due to the recorded startApplication() call. At the end of each Scenario, the onScenarioEnd hook is called, causing detach() to be called on the application context. Because the AUT was started with startApplication(), this causes it to terminate. This hook function is found in the file bdd_hooks.(py|js|pl|rb|tcl), which is located in the Scripts tab of the Test Suite Resources view. You can define additional hook functions there. For a list of all available hooks, please refer to Performing Actions During Test Execution Via Hooks (Section 6.19.10).

Python
@OnScenarioEnd
def OnScenarioEnd():
    for ctx in applicationContextList():
        ctx.detach()

JavaScript
OnScenarioEnd(function(context) {
    applicationContextList().forEach(function(ctx) { ctx.detach(); });
});
Perl
OnScenarioEnd(sub {
    foreach (applicationContextList()) {
        $_->detach();
    }
});
Ruby
OnScenarioEnd do |context|
    applicationContextList().each { |ctx| ctx.detach() }
end
Tcl
OnScenarioEnd {context} {
    foreach ctx [applicationContextList] {
        applicationContext $ctx detach
    }
}
4.8.2.3.4. Step Parametrization

So far, our steps did not use any parameters and all values were hardcoded. Squish has different types of parameters like any, integer or word, allowing our step definitions to be more reusable. Let us add a new Scenario to our Feature file which will provide step parameters for both the Test Data and the expected results. Copy the below section into your Feature file.


    Scenario: State after adding one entry
        Given the address book application is running
        When I create a new address book
        And I add a new person 'John','Doe','john@m.com','500600700' to the address book
        Then '1' entries should be present

After auto-saving the Feature file, the Squish IDE provides a hint that only 2 steps need to be implemented: When I add a new person 'John', 'Doe','john@m.com','500600700' to the address book and Then '1' entries should be present. The remaining steps already have a matching implementation.

To record the missing Steps, hit the Record () next to the test case name in the Test Suites view. The script will play until it gets to the missing step and then prompt you to implement it. If you select the Add button, then you can type in the information for a new entry. Click on Finish Recording Step... () to move to the next step. For the second missing step, we could record an object property verification like we did with the step, the address book should be empty. Or we could copy that step's implementation in the steps.(py|js|pl|rb|tcl) file and increment the number at the end of the test.compare() line. Instead of testing for zero items, we are testing for one item.

Now we parametrize the generated step implementation by replacing the values with parameter types. Since we want to be able to add different names, replace John with '|word|'. Note that each parameter will be passed to the step implementation function in the order of appearance in the descriptive name of the step. Finish parametrizing by editing the typed values into keywords, to look like this example step, When I add a new person 'John', 'Doe', 'john@m.com', '500600700' to the address book:

Python

@When("I add a new person '|word|','|word|','|any|','|integer|' to the address book")
def step(context, forename, surname, email, phone):
    mouseClick(waitForObject(names.address_Book_Untitled_Add_NSToolbarItem))
    mouseClick(waitForObject(names.add_Address_NSTextField), 42.6172, 5.125, 0, 0)
    type(waitForObject(names.add_Address_NSTextField), forename)
    mouseClick(waitForObject(names.add_Address_NSTextField_2), 95.7852, 14.4336, 0, 0)
    type(waitForObject(names.add_Address_NSTextField_2), surname)
    mouseClick(waitForObject(names.add_Address_NSTextField_3), 93.7422, 9.19531, 0, 0)
    type(waitForObject(names.add_Address_NSTextField_3), email)
    mouseClick(waitForObject(names.add_Address_NSTextField_4), 98.6094, 15.3789, 0, 0)
    type(waitForObject(names.add_Address_NSTextField_4), phone)
    mouseClick(waitForObject(names.add_Address_Add_NSButton))
    context.userData = {'forename': forename, 'surname': surname}
JavaScript

When("I add a new person '|word|','|word|','|any|','|integer|' to address book", function(context, forename, surname, email, phone) {
    mouseClick(waitForObject(names.addressBookUntitledAddNSToolbarItem));
    mouseClick(waitForObject(names.addAddressNSTextField), 54.3047, 9.10156, 0, 0);
    type(waitForObject(names.addAddressNSTextField), forename);
    mouseClick(waitForObject(names.addAddressNSTextField_2), 45.3555, 10.9453, 0, 0);
    type(waitForObject(names.addAddressNSTextField_2), surname);
    mouseClick(waitForObject(names.addAddressNSTextField_3), 49.4805, 8.01562, 0, 0);
    type(waitForObject(names.addAddressNSTextField_3), email);
    mouseClick(waitForObject(names.addAddressNSTextField_4), 46.3125, 12.1641, 0, 0);
    type(waitForObject(names.addAddressNSTextField_4), phone);
    mouseClick(waitForObject(names.addAddressAddNSButton));
    context.userData["forename"] = forename;
    context.userData["surname"] = surname;
});

Perl

When("I add a new person '|word|','|word|','|any|','|integer|' to the address book", sub {
    my $context = shift;
    my $forename = shift;
    my $surname = shift;
    my $email = shift;
    my $phone = shift;
    mouseClick( waitForObject($Names::address_book_untitled_add_nstoolbaritem) );
    type( waitForObject($Names::add_address_nstextfield), $forename );
    mouseClick( waitForObject($Names::add_address_nstextfield_2), 62.4609, 14.2227, 0, 0 );
    type( waitForObject($Names::add_address_nstextfield_2), $surname );
    mouseClick( waitForObject($Names::add_address_nstextfield_3), 60.375, 15.5, 0, 0 );
    type( waitForObject($Names::add_address_nstextfield_3), $email );
    mouseClick( waitForObject($Names::add_address_nstextfield_4), 108.117, 15.7734, 0, 0 );
    type( waitForObject($Names::add_address_nstextfield_4), $phone );
    mouseClick( waitForObject($Names::add_address_add_nsbutton) );
    $context->{userData}{'forename'} = $forename;
    $context->{userData}{'surname'} = $surname;
});

Ruby

When("I add a new person '|word|','|word|','|any|','|integer|' to the address book") do |context, forename, surname, email, phone|
    mouseClick(waitForObject(Names::Address_Book_Untitled_Add_NSToolbarItem))
    mouseClick(waitForObject(Names::Add_Address_NSTextField), 57.1602, 4.91016, 0, 0)
    type(waitForObject(Names::Add_Address_NSTextField), forename)
    mouseClick(waitForObject(Names::Add_Address_NSTextField_2), 71.3672, 17.5703, 0, 0)
    type(waitForObject(Names::Add_Address_NSTextField_2), surname)
    mouseClick(waitForObject(Names::Add_Address_NSTextField_3), 71.3672, 16.5, 0, 0)
    type(waitForObject(Names::Add_Address_NSTextField_3), email)
    mouseClick(waitForObject(Names::Add_Address_NSTextField_4), 79.707, 16.7734, 0, 0)
    type(waitForObject(Names::Add_Address_NSTextField_4), phone)
    mouseClick(waitForObject(Names::Add_Address_Add_NSButton))
    context.userData = Hash.new
    context.userData[:forename] = forename
    context.userData[:surname] = surname
end

Tcl

When "I add a new person '|word|','|word|','|any|','|integer|' to the address book" {context forename surname email phone} {
    invoke mouseClick [waitForObject $names::Address_Book_Untitled_Add_NSToolbarItem]
    invoke mouseClick [waitForObject $names::Add_Address_NSTextField] 118.832 21.0234 0 0
    invoke type [waitForObject $names::Add_Address_NSTextField] $forename
    invoke mouseClick [waitForObject $names::Add_Address_NSTextField_2] 1 1 0 0
    invoke type [waitForObject $names::Add_Address_NSTextField_2] $surname
    invoke mouseClick [waitForObject $names::Add_Address_NSTextField_3] 101.105 9.63281 0 0
    invoke type [waitForObject $names::Add_Address_NSTextField_3] $email
    invoke mouseClick [waitForObject $names::Add_Address_NSView] 208.312 62.9688 0 0
    invoke type [waitForObject $names::Add_Address_NSTextField_4] $phone
    invoke mouseClick [waitForObject $names::Add_Address_Add_NSButton]
    $context userData [dict create forename $forename surname $surname]
}

4.8.2.3.5. Provide parameters for Step in table

The next Scenario will test adding multiple entries to the address book. We could use step When I add a new person John','Doe','john@m.com','500600700' to the address book multiple times just with different data. But lets instead define a new step called And I add new persons to the address book which will handle data from a table.


    Scenario: State after adding two entries
        Given the address book application is running
        When I create a new address book
        And I add new persons to the address book
            | forename  | surname  | email        | phone  |
            | John      | Smith    | john@m.com   | 123123 |
            | Alice     | Thomson  | alice@m.com  | 234234 |
        Then '2' entries should be present

The step implementation to handle such tables looks like this:

Python

@When("I add new persons to the address book")
def step(context):
    table = context.table
    # Drop initial row with column headers
    for row in table[1:]:
        forename = row[0]
        surname = row[1]
        email = row[2]
        phone = row[3]
        mouseClick(waitForObject(names.address_Book_Untitled_Add_NSToolbarItem))
        mouseClick(waitForObject(names.add_Address_NSTextField), 42.6172, 5.125, 0, 0)
        type(waitForObject(names.add_Address_NSTextField), forename)
        mouseClick(waitForObject(names.add_Address_NSTextField_2), 95.7852, 14.4336, 0, 0)
        type(waitForObject(names.add_Address_NSTextField_2), surname)
        mouseClick(waitForObject(names.add_Address_NSTextField_3), 93.7422, 9.19531, 0, 0)
        type(waitForObject(names.add_Address_NSTextField_3), email)
        mouseClick(waitForObject(names.add_Address_NSTextField_4), 98.6094, 15.3789, 0, 0)
        type(waitForObject(names.add_Address_NSTextField_4), phone)
        mouseClick(waitForObject(names.add_Address_Add_NSButton))

JavaScript

When("I add new persons to address book", function(context) {
    var table = context.table;
    // Drop initial row with column headers
    for (var i = 1; i < table.length; ++i) {
        var forename = table[i][0];
        var surname = table[i][1];
        var email = table[i][2];
        var phone = table[i][3];
        mouseClick(waitForObject(names.addressBookUntitledAddNSToolbarItem));
        mouseClick(waitForObject(names.addAddressNSTextField), 54.3047, 9.10156, 0, 0);
        type(waitForObject(names.addAddressNSTextField), forename);
        mouseClick(waitForObject(names.addAddressNSTextField_2), 45.3555, 10.9453, 0, 0);
        type(waitForObject(names.addAddressNSTextField_2), surname);
        mouseClick(waitForObject(names.addAddressNSTextField_3), 49.4805, 8.01562, 0, 0);
        type(waitForObject(names.addAddressNSTextField_3), email);
        mouseClick(waitForObject(names.addAddressNSTextField_4), 46.3125, 12.1641, 0, 0);
        type(waitForObject(names.addAddressNSTextField_4), phone);
        mouseClick(waitForObject(names.addAddressAddNSButton));
    }
});

Perl

When("I add new persons to the address book", sub {
    my %context = %{shift()};
    my @table = @{$context{'table'}};

    # Drop initial row with column headers
    shift(@table);

    for my $row (@table) {
        my ($forename, $surname, $email, $phone) = @{$row};
        mouseClick( waitForObject($Names::address_book_untitled_add_nstoolbaritem) );
        type( waitForObject($Names::add_address_nstextfield), $forename );
        mouseClick( waitForObject($Names::add_address_nstextfield_2), 62.4609, 14.2227, 0, 0 );
        type( waitForObject($Names::add_address_nstextfield_2), $surname );
        mouseClick( waitForObject($Names::add_address_nstextfield_3), 60.375, 15.5, 0, 0 );
        type( waitForObject($Names::add_address_nstextfield_3), $email );
        mouseClick( waitForObject($Names::add_address_nstextfield_4), 108.117, 15.7734, 0, 0 );
        type( waitForObject($Names::add_address_nstextfield_4), $phone );
        mouseClick( waitForObject($Names::add_address_add_nsbutton) );
    }
});

Ruby

When("I add new persons to the address book") do |context|
    table = context.table
    # Drop initial row with column headers
    table.shift
    for forename,surname,email,phone in table do
        mouseClick(waitForObject(Names::Address_Book_Untitled_Add_NSToolbarItem))
        mouseClick(waitForObject(Names::Add_Address_NSTextField), 57.1602, 4.91016, 0, 0)
        type(waitForObject(Names::Add_Address_NSTextField), forename)
        mouseClick(waitForObject(Names::Add_Address_NSTextField_2), 71.3672, 17.5703, 0, 0)
        type(waitForObject(Names::Add_Address_NSTextField_2), surname)
        mouseClick(waitForObject(Names::Add_Address_NSTextField_3), 71.3672, 16.5, 0, 0)
        type(waitForObject(Names::Add_Address_NSTextField_3), email)
        mouseClick(waitForObject(Names::Add_Address_NSTextField_4), 79.707, 16.7734, 0, 0)
        type(waitForObject(Names::Add_Address_NSTextField_4), phone)
        mouseClick(waitForObject(Names::Add_Address_Add_NSButton))
    end
end

Tcl

When "I add new persons to the address book" {context} {
    set table [$context table]
    # Drop initial row with column headers
    foreach row [lreplace $table 0 0] {
        foreach {forename surname email phone} $row break
        invoke mouseClick [waitForObject $names::Address_Book_Untitled_Add_NSToolbarItem]
        invoke mouseClick [waitForObject $names::Add_Address_NSTextField] 118.832 21.0234 0 0
        invoke type [waitForObject $names::Add_Address_NSTextField] $forename
        invoke mouseClick [waitForObject $names::Add_Address_NSTextField_2] 1 1 0 0
        invoke type [waitForObject $names::Add_Address_NSTextField_2] $surname
        invoke mouseClick [waitForObject $names::Add_Address_NSTextField_3] 101.105 9.63281 0 0
        invoke type [waitForObject $names::Add_Address_NSTextField_3] $email
        invoke mouseClick [waitForObject $names::Add_Address_NSView] 208.312 62.9688 0 0
        invoke type [waitForObject $names::Add_Address_NSTextField_4] $phone
        invoke mouseClick [waitForObject $names::Add_Address_Add_NSButton]
    }
}

4.8.2.3.6. Sharing Data Between Steps and Scenarios

Lets add a new Scenario to the Feature file. This time we would like to check not the number of entries in the address book list, but if this list contains proper data. Because we enter data into the address book in one step and verify them in another, we must somehow share information about entered data between those steps in order to perform a verification.


    Scenario: Forename and surname is added to table
        Given the address book application is running
        When I create a new address book
        And I add a new person 'Bob','Doe','Bob@m.com','123321231' to the address book
        Then previously entered forename and surname shall be at the top

To share this data, context.userData can be used.

Python

@When("I add a new person '|word|','|word|','|any|','|integer|' to the address book")
def step(context, forename, surname, email, phone):
    mouseClick(waitForObject(names.address_Book_Untitled_Add_NSToolbarItem))
    mouseClick(waitForObject(names.add_Address_NSTextField), 42.6172, 5.125, 0, 0)
    type(waitForObject(names.add_Address_NSTextField), forename)
    mouseClick(waitForObject(names.add_Address_NSTextField_2), 95.7852, 14.4336, 0, 0)
    type(waitForObject(names.add_Address_NSTextField_2), surname)
    mouseClick(waitForObject(names.add_Address_NSTextField_3), 93.7422, 9.19531, 0, 0)
    type(waitForObject(names.add_Address_NSTextField_3), email)
    mouseClick(waitForObject(names.add_Address_NSTextField_4), 98.6094, 15.3789, 0, 0)
    type(waitForObject(names.add_Address_NSTextField_4), phone)
    mouseClick(waitForObject(names.add_Address_Add_NSButton))
    context.userData = {'forename': forename, 'surname': surname}

JavaScript

When("I add a new person '|word|','|word|','|any|','|integer|' to address book", function(context, forename, surname, email, phone) {
    mouseClick(waitForObject(names.addressBookUntitledAddNSToolbarItem));
    mouseClick(waitForObject(names.addAddressNSTextField), 54.3047, 9.10156, 0, 0);
    type(waitForObject(names.addAddressNSTextField), forename);
    mouseClick(waitForObject(names.addAddressNSTextField_2), 45.3555, 10.9453, 0, 0);
    type(waitForObject(names.addAddressNSTextField_2), surname);
    mouseClick(waitForObject(names.addAddressNSTextField_3), 49.4805, 8.01562, 0, 0);
    type(waitForObject(names.addAddressNSTextField_3), email);
    mouseClick(waitForObject(names.addAddressNSTextField_4), 46.3125, 12.1641, 0, 0);
    type(waitForObject(names.addAddressNSTextField_4), phone);
    mouseClick(waitForObject(names.addAddressAddNSButton));
    context.userData["forename"] = forename;
    context.userData["surname"] = surname;
});

Perl

When("I add a new person '|word|','|word|','|any|','|integer|' to the address book", sub {
    my $context = shift;
    my $forename = shift;
    my $surname = shift;
    my $email = shift;
    my $phone = shift;
    mouseClick( waitForObject($Names::address_book_untitled_add_nstoolbaritem) );
    type( waitForObject($Names::add_address_nstextfield), $forename );
    mouseClick( waitForObject($Names::add_address_nstextfield_2), 62.4609, 14.2227, 0, 0 );
    type( waitForObject($Names::add_address_nstextfield_2), $surname );
    mouseClick( waitForObject($Names::add_address_nstextfield_3), 60.375, 15.5, 0, 0 );
    type( waitForObject($Names::add_address_nstextfield_3), $email );
    mouseClick( waitForObject($Names::add_address_nstextfield_4), 108.117, 15.7734, 0, 0 );
    type( waitForObject($Names::add_address_nstextfield_4), $phone );
    mouseClick( waitForObject($Names::add_address_add_nsbutton) );
    $context->{userData}{'forename'} = $forename;
    $context->{userData}{'surname'} = $surname;
});

Ruby

When("I add a new person '|word|','|word|','|any|','|integer|' to the address book") do |context, forename, surname, email, phone|
    mouseClick(waitForObject(Names::Address_Book_Untitled_Add_NSToolbarItem))
    mouseClick(waitForObject(Names::Add_Address_NSTextField), 57.1602, 4.91016, 0, 0)
    type(waitForObject(Names::Add_Address_NSTextField), forename)
    mouseClick(waitForObject(Names::Add_Address_NSTextField_2), 71.3672, 17.5703, 0, 0)
    type(waitForObject(Names::Add_Address_NSTextField_2), surname)
    mouseClick(waitForObject(Names::Add_Address_NSTextField_3), 71.3672, 16.5, 0, 0)
    type(waitForObject(Names::Add_Address_NSTextField_3), email)
    mouseClick(waitForObject(Names::Add_Address_NSTextField_4), 79.707, 16.7734, 0, 0)
    type(waitForObject(Names::Add_Address_NSTextField_4), phone)
    mouseClick(waitForObject(Names::Add_Address_Add_NSButton))
    context.userData = Hash.new
    context.userData[:forename] = forename
    context.userData[:surname] = surname
end

Tcl

When "I add a new person '|word|','|word|','|any|','|integer|' to the address book" {context forename surname email phone} {
    invoke mouseClick [waitForObject $names::Address_Book_Untitled_Add_NSToolbarItem]
    invoke mouseClick [waitForObject $names::Add_Address_NSTextField] 118.832 21.0234 0 0
    invoke type [waitForObject $names::Add_Address_NSTextField] $forename
    invoke mouseClick [waitForObject $names::Add_Address_NSTextField_2] 1 1 0 0
    invoke type [waitForObject $names::Add_Address_NSTextField_2] $surname
    invoke mouseClick [waitForObject $names::Add_Address_NSTextField_3] 101.105 9.63281 0 0
    invoke type [waitForObject $names::Add_Address_NSTextField_3] $email
    invoke mouseClick [waitForObject $names::Add_Address_NSView] 208.312 62.9688 0 0
    invoke type [waitForObject $names::Add_Address_NSTextField_4] $phone
    invoke mouseClick [waitForObject $names::Add_Address_Add_NSButton]
    $context userData [dict create forename $forename surname $surname]
}

All data stored in context.userData can be accessed from steps implementations and Hooks in all Scenarios of the given Feature. Finally, we need to implement the step, Then previously entered forename and surname shall be at the top.

Python

@Then("previously entered forename and surname shall be at the top")
def step(context):
    table = waitForObject(names.address_Book_Untitled_NSTableView)
    children = object.children(table)
    forename = children[0]
    surname = children[1]
    test.compare(forename.stringValue, context.userData["forename"])
    test.compare(surname.stringValue, context.userData["surname"])

JavaScript

Then("previously entered forename and surname shall be at the top", function(context) {
    test.compare(object.children(waitForObjectExists(names.addressBookUntitledNSTableView))[0], "Bob");
    test.compare(object.children(waitForObjectExists(names.addressBookUntitledNSTableView))[1], "Doe");
});
Perl

Then("previously entered forename and surname shall be at the top", sub {
    my $context = shift;
    my $table = waitForObject($Names::address_book_untitled_nstableview);
    my @children = object::children($table);
    my $forename = $children[0];
    my $surname = $children[1];
    test::compare( $forename->stringValue, $context->{userData}{'forename'} );
    test::compare( $surname->stringValue, $context->{userData}{'surname'} );
});
Ruby

Then("previously entered forename and surname shall be at the top") do |context|
    table = waitForObject(Names::Address_Book_Untitled_NSTableView)
    children = Squish::Object.children(table)
    forename = children[0]
    surname = children[1]
    Test.compare(forename.stringValue, context.userData[:forename])
    Test.compare(surname.stringValue, context.userData[:surname])
end
Tcl

Then "previously entered forename and surname shall be at the top" {context} {

    set obj [waitForObject $names::Address_Book_Untitled_NSTableView]
    set children [object children $obj]
    set forename [lindex $children 0]
    set surname [lindex $children 1]

    test compare [property get $forename stringValue] [dict get [$context userData] forename]
    test compare [property get $surname stringValue] [dict get [$context userData] surname]
}

If you copied the step implementation from here or wrote it yourself instead of recording, you might encounter an error during replay: Object ':Bob_NSString' not found. Object name not found in Object Map.

This happens because we introduced new GUI objects without accessing them through Squish first which would add them to the object map. To resolve the issue, press the Pick New Object button in the error dialog and then click on the missing object. Now select Retry to continue replaying.

4.8.2.3.7. Scenario Outline

Assume our Feature contains the following two Scenarios:

Scenario: State after adding one entry
    Given the address book application is running
    When I create a new address book
    And I add a new person 'John','Doe','john@m.com','500600700' to the address book
    Then "1" entries should be present

Scenario: State after adding one entry
    Given the address book application is running
    When I create a new address book
    And I add a new person 'Bob','Koo','bob@m.com','500600800' to the address book
    Then "1" entries should be present

As we can see, those Scenarios perform the same actions using different test data. The same can be achieved by using a Scenario Outline (a Scenario template with placeholders) and Examples (a table with parameters).

Scenario Outline: Adding single entries multiple times
    Given the address book application is running
    When I create a new address book
    And I add a new person '<forename>','<surname>','<email>','<phone>' to the address book
    Then '1' entries should be present
    Examples:
        | forename | surname  | email       | phone     |
        | John     | Doe      | john@m.com  | 500600700 |
        | Bob      | Koo      | bob@m.com   | 500600800 |

Please note that the OnScenarioEnd hook will be executed at the end of each loop iteration in a Scenario Outline.

4.8.2.4. Test Execution

In the Squish IDE, users can execute all Scenarios in a Feature, or execute only one selected Scenario. In order to execute all Scenarios, the proper test case has to be executed by clicking on the Play button in the Test Suites view.

Execute all Scenarios from Feature

In order to execute only one Scenario, you need to open the Feature file, right-click on the given Scenario and choose Run Scenario. An alternative approach is to click on the Play button next to the respective Scenario in the Scenarios tab in Test Case Resources.

Execute one Scenario from Feature

After a Scenario is executed, the Feature file is colored according to the execution results. More detailed information (like logs) can be found in the Test Results view.

Execution results in Feature file

4.8.2.5. Test Debugging

Squish offers the possibility to pause an execution of a Test Case at any point in order to check script variables, spy application objects or run custom code in the Squish script console. To do this, a breakpoint has to be placed before starting the execution, either in the Feature file at any line containing a step or at any line of executed code (i.e. in middle of step definition code).

Breakpoint in Feature file

After the breakpoint is reached, you can inspect all application objects and their properties. If a breakpoint is placed at a step definition or a hook is reached, then you can additionally add Verification Points or record code snippets.

4.8.2.6. Re-using Step Definitions

BDD test maintainability can be increased by reusing step definitions in test cases located in another directory. For more information, see collectStepDefinitions()

4.8.3. Tutorial: Migration of Existing Tests to BDD

This chapter is aimed at users that have existing Squish tests and who would like to introduce Behavior Driven Testing. The first section describes how to keep the existing tests and add new tests with the BDD approach. The second section describes how to convert existing tests to BDD.

4.8.3.1. Extending Existing Tests to BDD

The first option is to keep any existing Squish script-based tests and extend them by adding new BDD tests. It's possible to have a Test Suite containing both script-based test cases and BDD tests. Simply open an existing Test Suite with test cases and choose New BDD Test Case from the drop down list.

Creating a new BDD test case

Assuming your existing test cases make use of a library and you are calling shared functions to interact with the AUT, those functions can also be used from BDD test cases. In the example below, a function is used in multiple script-based test cases:

Python
def createNewAddressBook():
    mouseClick(waitForObject(names.address_Book_Untitled_New_NSToolbarItem))
JavaScript
function createNewAddressBook(){
    mouseClick(waitForObject(names.addressBookUntitledNewNSToolbarItem));
}
Perl
sub createNewAddressBook{
    mouseClick(waitForObject($Names::address_book_untitled_new_nstoolbaritem));
}
Ruby
def createNewAddressBook
    mouseClick(waitForObject(Names::Address_Book_Untitled_New_NSToolbarItem))
end
Tcl
proc createNewAddressBook {} {
    invoke mouseClick [waitForObject $names::Address_Book_Untitled_New_NSToolbarItem]
}

New BDD test cases can easily use the same function:

Python
@When("I create a new address book")
def step(context):
    createNewAddressBook()
JavaScript
When("I create a new address book", function(context){
    createNewAddressBook()
});
Perl
When("I create a new address book", sub {
    createNewAddressBook();
});
Ruby
When("I create a new address book") do |context|
  createNewAddressBook
end
Tcl
When "I create a new address book" {context} {
    createNewAddressBook
}

4.8.3.2. Converting Existing Tests to BDD

The second option is to convert an existing Test Suite that contains script-based tests into behavior driven tests. Since a Test Suite can contain both, migration can be done gradually. A Test Suite containing a mix of both test case types can be executed and results analyzed without any extra effort required.

The first step is to review all test cases of the existing Test Suite and group them by the Feature they test. Each script-based test case will be transformed into a Scenario, which is a part of a Feature. For example, assume we have 5 script-based tests. After review, we realize that they examine two Features. Therefore, when migration is completed, our Test Suite will contain two BDD feature files, each of them containing one Feature. Each Feature will contain multiple Scenarios. In our example the first Feature contains three Scenarios and the second Feature contains two Scenarios.

Conversion Chart

At the beginning, open a Test Suite in the Squish IDE that contains script-based tests to be migrated to BDD. Next, create a new test case by choosing New BDD Test Case from the context menu. Each BDD test corresponds to a test.feature file containing one Feature. Next, open the test.feature file to describe the Feature using the Gherkin language. Following the syntax from the template, edit the Feature name and optionally provide a short description. Next, analyze which actions and verifications in the test case need to be migrated. This is how an example test case for the address book application may look:

Python
def main():
    startApplication("SquishAddressBook")
    mouseClick(waitForObject(names.address_Book_Untitled_New_NSToolbarItem))
    test.compare(waitForObjectExists(names.address_Book_Untitled_NSTableView).numberOfRows(), 0)
JavaScript
function main() {
    startApplication("SquishAddressBook");
    mouseClick(waitForObject(names.addressBookUntitledNewNSToolbarItem));
    test.compare((waitForObjectExists(names.addressBookUntitledNSTableView)).numberOfRows(), 0);
}
Perl
sub main {
    startApplication("SquishAddressBook");
    mouseClick(waitForObject($Names::address_book_untitled_new_nstoolbaritem));
    test.compare(waitForObjectExists($Names::address_book_untitled_nstableview).numberOfRows(), 0)
}
Ruby
def main
    startApplication("SquishAddressBook")
    mouseClick(waitForObject(Names::Address_Book_Untitled_New_NSToolbarItem))
    Test.compare(waitForObjectExists(Names::Address_Book_Untitled_NSTableView).numberOfRows(), 0)
Tcl
proc main {} {
    startApplication "SquishAddressBook"
    invoke mouseClick [waitForObject $names::Address_Book_Untitled_New_NSToolbarItem]
    test compare [invoke [waitForObjectExists $names::Address_Book_Untitled_NSTableView] numberOfRows] 0
}

After analyzing the above script-based test we can create the following Scenario and add it to test.feature:

Scenario: Initial state of the created address book
    Given the address book application is running
    When I create a new address book
    Then the address book should be empty

Next, right-click on the Scenario and choose the option Create Missing Step Implementations from the context menu. This will create a skeleton of Steps definitions:

Python
@Given("the address book application is running")
def step(context):
    test.warning("TODO implement the address book application is running")

@When("I create a new address book")
def step(context):
    test.warning("TODO implement I create a new address book")

@Then("the address book should be empty")
def step(context):
    test.warning("TODO implement the address book should be empty")
JavaScript
Given("the address book application is running", function(context) {
    test.warning("TODO implement the address book application is running");
});

When("I create a new address book", function(context) {
    test.warning("TODO implement I create a new address book");
});

Then("the address book should be empty", function(context) {
    test.warning("TODO implement the address book should be empty");
});
Perl
Given("the address book application is running", sub {
    my $context = shift;
    test::warning("TODO implement the address book application is running");
});

When("I create a new address book", sub {
    my $context = shift;
    test::warning("TODO implement I create a new address book");
});

Then("the address book should be empty", sub {
    my $context = shift;
    test::warning("TODO implement the address book should be empty");
});
Ruby
Given("the address book application is running") do |context|
    Test.warning "TODO implement the address book application is running"
end

When("I create a new address book") do |context|
    Test.warning "TODO implement I create a new address book"
end

Then("the address book should be empty") do |context|
    Test.warning "TODO implement the address book should be empty"
end
Tcl
Given "the address book application is running" {context} {
    test warning "TODO implement the address book application is running"
}

When "I create a new address book" {context} {
    test warning "TODO implement I create a new address book"
}

Then "the address book should be empty" {context} {
    test warning "TODO implement the address book should be empty"
}

Now we put code snippets from the script-based test case into respective step definitions and remove the lines containing test.warning. If your script-based tests make use of shared scripts, you can call those functions inside the step definition as well. For example, the final result could look like this:

Python
@Given("the address book application is running")
def step(context):
    startApplication("SquishAddressBook")
    test.compare(waitForObjectExists(names.o_NSWindow).isVisible, 0)

@When("I create a new address book")
def step(context):
    mouseClick(waitForObject(names.address_Book_Untitled_New_NSToolbarItem))

@Then("the address book should be empty")
def step(context):
    #test.compare(waitForObjectExists(names.address_Book_Untitled_NSTableView).alignment, 0)
    test.compare(waitForObjectExists(names.address_Book_Untitled_NSTableView).numberOfRows(), 0)

JavaScript
Given("addressbook application is running", function(context) {
    startApplication("SquishAddressBook");
    test.compare(waitForObjectExists(names.addressBookUntitledNSWindow).isVisible, 1);
});

When("I create a new addressbook", function(context) {
    mouseClick(waitForObject(names.addressBookUntitledNewNSToolbarItem));
});

Then("addressbook should have zero entries", function(context) {
    //test.compare((waitForObject(names.addressBookUntitledNSTableView)).alignment, 0);
    test.compare((waitForObject(names.addressBookUntitledNSTableView)).numberOfRows(), 0);
});

Perl
Given("the address book application is running", sub {
    my $context = shift;
    startApplication("SquishAddressBook");
    test::compare(waitForObjectExists($Names::address_book_untitled_nswindow)->isVisible, 1);
});

When("I create a new address book", sub {
    my $context = shift;
    mouseClick(waitForObject($Names::address_book_untitled_new_nstoolbaritem));
});

Then("the address book should be empty", sub {
    my $context = shift;
    #test::compare(waitForObjectExists($Names::address_book_untitled_nstableview)->alignment, 1);
    test::compare(waitForObjectExists($Names::address_book_untitled_nstableview)->numberOfRows, 0);
});

Ruby
Given("the address book application is running") do |context|
    startApplication("SquishAddressBook")
    Test.compare(waitForObjectExists(Names::O_NSWindow).isVisible, 0)
end

When("I create a new address book") do |context|
    mouseClick(waitForObject(Names::Address_Book_Untitled_New_NSToolbarItem))
end

Then("the address book should be empty") do |context|
    #Test.compare(waitForObjectExists(Names::Address_Book_Untitled_NSTableView).alignment, 0)
    Test.compare(waitForObjectExists(Names::Address_Book_Untitled_NSTableView).numberOfRows(), 0)
end

Tcl
Given "the address book application is running" {context} {
    startApplication "SquishAddressBook"
    test compare [property get [waitForObjectExists $names::NSWindow] isVisible] 0
}

When "I create a new address book" {context} {
    invoke mouseClick [waitForObject $names::Address_Book_Untitled_New_NSToolbarItem]
}

Then "the address book should be empty" {context} {
    test compare [invoke [waitForObjectExists $names::Address_Book_Untitled_NSTableView] numberOfRows] 0
}

Note that the test.log("Create a new address book") got removed while migrating this script-based test to BDD. When the step I create a new address book is executed, the step name will be logged into Test Results, so the test.log call would have been redundant.

When the test case execution ends, Squish terminates the AUT. Squish also ensures that the AUT is terminated at the end of each Scenario. This is done from the auto-generated OnScenarioEnd hook.

Python
@OnScenarioEnd
def hook(context):
    for ctx in applicationContextList():
        ctx.detach()
JavaScript
OnScenarioEnd(function(context) {
    applicationContextList().forEach(function(ctx) { ctx.detach(); });
});
Perl
# Detach (i.e. potentially terminate) all AUTs at the end of a scenario
OnScenarioEnd(sub {
    foreach (applicationContextList()) {
        $_->detach();
    }
});
Ruby
OnScenarioEnd do |context|
    applicationContextList().each { |ctx| ctx.detach() }
end
Tcl
OnScenarioEnd { context } {
    applicationContext [currentApplicationContext] detach
}

The above example was simplified for this tutorial. In order to take full advantage of Behavior Driven Testing in Squish, please familiarize yourself with the section Behavior Driven Testing (Section 6.19) in API Reference Manual (Chapter 6).




[12] Each AUT must be registered with the squishserver so that test scripts do not need to include the AUT's path, thus making the tests platform-independent. Another benefit of registering is that AUTs can be tested without the Squish IDE—for example, when doing regression testing.

[13] Both .csv and .tsv files are assumed to use the Unicode UTF-8 encoding—the same encoding used for all test scripts.