13.10. How to Test Multiple AUTs from a Single Test Script, Using ApplicationContext

13.10.1. How to Start and Access Multiple Applications Under Test
13.10.2. How to Use ApplicationContext Objects

Usually, a single application under test is specified for each test suite. This AUT is then executed and accessed by each test case. All the tutorials show this one test suite/one AUT approach, but in fact it is possible to start multiple applications and access and test all of them from within a single test suite. This makes it possible to test the interaction between different applications or between multiple instances of the same application. For example, being able to test multiple applications is essential for testing client/server systems.

Whenever an AUT is started a corresponding Application Context (Section 14.3.11) object is created, and it is this object that is used by Squish to provide access to the AUT. Squish allows us to access the ApplicationContext object directly in our code, and this means that we can query the AUT for information such as the command line it was launched with, its current state, and so on. This information can also be accessed by making use of the context object returned by the currentApplicationContext function.

13.10.1. How to Start and Access Multiple Applications Under Test

When testing multiple applications from a single test script, the first step is to ensure that no application is set to be automatically started. Using the Squish IDE, click the Test Suite Settings toolbar button (in the Test Suites view (Section 16.2.16)) to make the test suite's Settings view (Section 16.2.14) visible. Now, in the editor's "Application Under Test (AUT)" section, make sure that the Automatically start the AUT checkbox is unchecked.

The function used to start an application is startApplication. This function starts the given application (assuming it is located in an application path—see AUTs and Settings (Section 15.3)) using the given command line arguments and returns a corresponding ApplicationContext object. The application context object is a handle that refers to the application.

Optionally, as the second and third parameters, a host and port can be passed to the startApplication function. This way, the startApplication function will connect to the squishserver on the specified host and listen to the specified port, instead of using the default host and port (as specified in the Squish IDE's settings or on the squishrunner's command line). This allows us to control multiple applications on multiple computers from a single test script.

Special care must be taken if the application is using a different GUI toolkit than the test suite's default toolkit. The global testSettings Object (Section 14.3.14) object allows us to set the configuration of the toolkit wrapper on a per-AUT basis. See the testSettings.setWrappersForApplication function for details on how to do this.

If we run two or more AUTs within a test script, which one should test code apply to? We can make one of the AUTs the “active” application by using the setApplicationContext function, passing an ApplicationContext as the sole parameter. Once the call is made, all script code applies to the active application—unless another setApplicationContext call is made to change the active application. Note that whenever we call the startApplication function, not only is the application's ApplicationContext object returned, but the application is automatically set to be the active application.

We can obtain a list of all the currently running AUTs' ApplicationContext objects, by calling the applicationContextList function. And we can retrieve the context object of the active application by calling the currentApplicationContext function.

[Note]AUT Sub-processes

If you want to record and access applications which are started by the AUT itself, and not by Squish, see the Recording the Sub-Processes started by the AUT (Section 15.7.1) section.

We will now look at some examples that show how to start multiple AUTs and how to use ApplicationContext objects to query them.

We will take as an example a client/server chat system. The system has a chat server called chatserver which must be running for communication to take place, and two chat clients, one written in Qt called chatclientqt, and the other written in Java called chatclientjava.

In the test we will first start the chat server. Then we start two clients; these automatically connect to the chat server at startup. We will then type something into the message editor of the first client and check that the second client received the message.

Python
startApplication("chatserver")
client1 = startApplication("chatclientqt", "Qt")
client2 = startApplication("chatclientjava", "Java")

setApplicationContext(client1)
editor = waitForObject("ChatWindow.messageEditor")
type(editor, "Message for client #2")

setApplicationContext(client2)
msgView = waitForObject("ChatWindow.messageView")
test.compare(msgView.text, "Message for client #2")
JavaScript
startApplication("chatserver");
var client1 = startApplication("chatclientqt", "Qt");
var client2 = startApplication("chatclientjava", "Java");

setApplicationContext(client1);
var editor = waitForObject("ChatWindow.messageEditor");
type(editor, "Message for client #2");

setApplicationContext(client2);
var msgView = waitForObject("ChatWindow.messageView");
test.compare(msgView.text, "Message for client #2");
Perl
startApplication("chatserver");
my $client1 = startApplication("chatclientqt", "Qt");
my $client2 = startApplication("chatclientjava", "Java");

setApplicationContext($client1);
my $editor = waitForObject("ChatWindow.messageEditor");
type($editor, "Message for client #2");

setApplicationContext($client2);
my $msgView = waitForObject("ChatWindow.messageView");
test::compare($msgView->text, "Message for client #2");
Tcl
startApplication "chatserver"
set client1 [startApplication "chatclientqt" "Qt"]
set client2 [startApplication "chatclientjava" "Java"]

setApplicationContext $client1
set editor [waitForObject "ChatWindow.messageEditor"]
invoke type $editor "Message for client #2"

setApplicationContext $client2
set msgView [waitForObject "ChatWindow.messageView"]
test compare [property get $msgView text] "Message for client #2"

We begin by starting each of the applications in turn, although we only keep references to the client AUTs' ApplicationContext objects since we don't directly access the server in the test. Once the applications are running we make the first client the active AUT since the active AUT is currently client2 since that was the AUT started by the most recent startApplication call. Then we get a reference to the client's chat editor and type some text into it. And at the end, we make the second client the active AUT, get a reference to its chat editor (a different widget this time since the toolkit is different—Java rather than Qt), and we compare the second client's editor's text with the text we sent from the first client.

13.10.2. How to Use ApplicationContext Objects

It is possible to use an ApplicationContext object to retieve information about the AUT it refers to. The application context of the AUT defined in the test suite settings can be retrieved using the defaultApplicationContext function, and of the currently running AUT by the currentApplicationContext function. When multiple AUTs are started there should not be any AUT defined in the test suite settings—each AUT's context object can be retrieved as the return value of the call to the startApplication function which is used to start the AUT, or from the applicationContextList function which returns all the AUTs' context objects.

The Application Context (Section 14.3.11) section details the properties and functions that are accessible from ApplicationContext objects. Here are some examples.

Python
ctx = currentApplicationContext()
test.log(ctx.commandLine)
test.log(ctx.cwd)
JavaScript
var ctx = currentApplicationContext();
test.log(ctx.commandLine);
test.log(ctx.cwd);
Perl
my $ctx = currentApplicationContext();
test::log($ctx->commandLine);
test::log($ctx->cwd);
Tcl
set ctx [currentApplicationContext]
test log [applicationContext $ctx commandLine]
test log [applicationContext $ctx cwd]

Here we print the command line the AUT was invoked with and its current working directory—both are properties.

Python
ctx = currentApplicationContext()
peakMemory = 0
while ctx.isRunning:
    peakMemory = max(ctx.usedMemory, peakMemory)
    if not ctx.isFrozen(20):
	break
test.log(peakMemory)
JavaScript
var ctx = currentApplicationContext();
var peakMemory = 0;
while (ctx.isRunning) {
    peakMemory = Math.max(ctx.usedMemory, peakMemory);
    if (!ctx.isFrozen(20))
	break;
}
test.log(peakMemory);
Perl
my $ctx = currentApplicationContext();
my $peakMemory = 0;
while ($ctx->isRunning) {
    if ($ctx->usedMemory > $peakMemory) {
	$peakMemory = $ctx->usedMemory;
    }
    if (!$ctx->isFrozen(20)) {
	last;
    }
}
test::log($peakMemory)
Tcl
set ctx [currentApplicationContext]
set peakMemory 0
while {[applicationContext $ctx isRunning] == 1} {
    if {[applicationContext $ctx usedMemory] > $peakMemory} {
	set peakMemory [applicationContext $ctx usedMemory]
    }
    if {![applicationContext $ctx isFrozen 20]} {
	break
    }
}
test log $peakMemory

Here we access the currently running AUT and keep track of the maximum amount of memory it is using. We break out of the loop if the application stops running (in which case isRunning will be false), or if the application becomes unresponsive (frozen), after waiting 20 seconds.

Python
ctx = currentApplicationContext()
test.log("STDOUT", ctx.readStdout())
test.warning("STDERR", ctx.readStderr())
JavaScript
var ctx = currentApplicationContext();
test.log("STDOUT", ctx.readStdout());
test.warning("STDERR", ctx.readStderr());
Perl
my $ctx = currentApplicationContext();
test::log("STDOUT", $ctx->readStdout());
test::warning("STDERR", $ctx->readStderr());
Tcl
set ctx [currentApplicationContext]
test log "STDOUT" [applicationContext $ctx readStdout]
test warning "STDERR" [applicationContext $ctx readStderr]

Here we have added everything that the AUT has written to stdout and stderr to the test log, classifying all stderr messages as warnings.