5.3. Creating a Data-Driven Test

After our first complete test with a verification point, we now want to make it more flexible and want to test different data on the addressbook without having to modify the test case itself every time. This approach is called data-driven testing. It means that the test data is read from a file which contains data records. Certain parts or the entire test is run for each data record. So we have to separate the test script and test data.

In this tutorial we modify our first test case. The examples directory in your package, however, uses a separate test case for this (tst_add_address_datadriven.).

First we add a testdata file containing the data we want to use. To do this, right-click on the testdata item of the new test case and choose New Testdata. This creates a new item in Squish's list view on which the name has to be changed. Change the name of the file to addresses.tsv.

Alternatively it is also possible to import existing test data files (Excel and TSV files). You can either import the file using the context menu by right-clicking on the testdata item in the Squish IDE's test suite view or by just copying the file into the test data directoy.

The window to the right will now display an empty table, wherein data can be added in a more convenient way. The table header has a context menu that can be activated with a click of the right mouse button. Use the Insert column and Rename Column to add and rename colums. Now set the columns and add data as seen here:

In the last column we put the number of the record since we will need this data later on.

[Note]Note

The file will have the column names as the first line separated by tabs. These names identify the fields. The other lines are records containing addresses, one address per line and each field is separated by a tab. The test data file parser ignores lines which start with a hash symbol (#) and treats them as comments.

We want to run the test case for every address in the file. This means that we must modify the test script to use the test data rather than hardcoded values. Squish provides a special script API for using test data. In the test script, we add the following code before the line which sets the focus on the First Name line edit by pressing Alt-N:

    for address_record in testData.dataset("addresses.tsv"):
        firstName = testData.field(address_record, "First Name")
        lastName = testData.field(address_record, "Last Name")
        address = testData.field(address_record, "Address")
        email = testData.field(address_record, "E-Mail")

The first line loops over all the records in the test data file addresses.tsv and reads the current record into the variable address_record. The following four lines read the fields (First Name, Last Name, etc.), from the current record and assign them to variables.

Now we must replace the parts in the script where it uses hardcoded data and change it to use the variables mentioned above. For example we must change this line:

        type(":Addressbook.Add/Change Entry.First Name_QLineEdit", "Max")

to the following:

        type(":Addressbook.Add/Change Entry.First Name_QLineEdit", firstName)

We must do similar changes to the the remaining lines which input data into the line edits.

Now we need to adjust the verification point to also use the test data for the comparisons instead of the hardcoded values: first pass the test data record (stored in address_record) to the test.vp("VP1") statement; i.e. simply add address_record as second argument to the call:

test.vp("VP1", address_record)

Now we need to edit the verification point itself to use the test data. So we right-click on the script code for the verification point and choose Open Verification Point "VP1" in the context menu. Now the verification point editor opens and we can choose to use test data fields instead of hardcoded values by changing the values in the Expected Value column. For Qt 3 this looks like this:

In Qt 4 it looks like this (after the changes):

[Note]Note

You can also select whether a test is currently expected to fail or succeed in the Expected Failure column.

Now we go back to the test script to make sure that the loop's body has the correct indentation (see also the sidebar about scoping in Python). This is important when using Python. The editor of the Squish IDE helps you doing the job: select the whole loop (starting at the for statement until the test.vp() statement) and then press the TAB key.

Here is the complete Python code after all changes for Qt 3:

def main():
    snooze(1.5)

    for address_record in testData.dataset("addresses.tsv"):
        firstName = testData.field(address_record, "First Name")
        lastName = testData.field(address_record, "Last Name")
        address = testData.field(address_record, "Address")
        email = testData.field(address_record, "E-Mail")
    
        type(":Addressbook.tab control_QTabBar", "<<Alt+N>>")
        snooze(2.0)
        type(":Addressbook.Add/Change Entry.First Name_QLineEdit", firstName)
        snooze(0.2)
        type(":Addressbook.Add/Change Entry.First Name_QLineEdit", "<Tab>")
        snooze(2.0)
        type(":Addressbook.Add/Change Entry.Last Name_QLineEdit", lastName)
        snooze(0.1)
        type(":Addressbook.Add/Change Entry.Last Name_QLineEdit", "<Tab>")
        snooze(3.4)
        type(":Addressbook.Add/Change Entry.Address_QLineEdit", address)
        snooze(0.3)
        type(":Addressbook.Add/Change Entry.Address_QLineEdit", "<Tab>")
        snooze(3.6)
        type(":Addressbook.Add/Change Entry.E-Mail_QLineEdit", email)
        snooze(1.4)
        clickButton(":Addressbook.Add/Change Entry.Add_QPushButton")
        snooze(2.2)
        test.vp("VP1", address_record)
        
    activateItem(":Addressbook.automatic menu bar_QMenuBar", "File")
    activateItem(":Addressbook.File_QPopupMenu", "Quit")

The same script for Qt 4 looks like this:

def main():
    snooze(1.5)
    
    for address_record in testData.dataset("addresses.tsv"):
        firstName = testData.field(address_record, "First Name")	
        lastName = testData.field(address_record, "Last Name")
        address = testData.field(address_record, "Address")
        email = testData.field(address_record, "E-Mail")

        type(":Addressbook_QTabBar", "<Alt+N>")
        snooze(1.1)
        type(":Addressbook.Add/Change Entry.First Name_QLineEdit", firstName)
        snooze(0.3)
        type(":Addressbook.Add/Change Entry.First Name_QLineEdit", "<Tab>")
        snooze(2.0)
        type(":Addressbook.Add/Change Entry.Last Name_QLineEdit", lastName)
        snooze(0.7)
        type(":Addressbook.Add/Change Entry.Last Name_QLineEdit", "<Tab>")
        snooze(2.8)
        type(":Addressbook.Add/Change Entry.Address_QLineEdit", address)
        snooze(0.2)
        type(":Addressbook.Add/Change Entry.Address_QLineEdit", "<Tab>")
        snooze(3.4)
        type(":Addressbook.Add/Change Entry.E-Mail_QLineEdit", email)
        snooze(1.4)
        clickButton(":Addressbook.Add/Change Entry.Add_QPushButton")
        snooze(1.0)
        test.vp("VP1", address_record)
        
    activateItem(":Addressbook_QMenuBar", "File")
    snooze(0.9)
    activateItem(":Addressbook.File_QMenu", "Quit")

Now we extended our simple first test to a data driven test where we can easily add more tests adding more addresses to our addresses.tsv file without changing the script code or verification points. Summing it up: This data-driven approach makes it much easier to create flexible, maintainable and extensible test scripts.

If you run the test again, you see that the test adds addresses for all records of the data file and that the generated test log also contains information about the test data file and record which has been used to generate the test log: