5.12. How to Test Multiple AUTs from a Single Test Script, Using ApplicationContext

Table of Contents

5.12.1. How to Start and Access Multiple Applications Under Test
5.12.2. Notes on using startApplication with Android
5.12.3. How to Use ApplicationContext Objects

Usually, a single application under test is specified for each test suite. This AUT is then executed and accessed by each test case. All the tutorials show this one test suite/one AUT approach, but in fact it is possible to start multiple applications and access and test all of them from within a single test suite. This makes it possible to test the interaction between different applications or between multiple instances of the same application. For example, being able to test multiple applications is essential for testing client/server systems.

Whenever an AUT is started a corresponding Application Context (Section 6.3.12) object is created, and it is this object that is used by Squish to provide access to the AUT. Squish allows us to access the ApplicationContext object directly in our code, and this means that we can query the AUT for information such as the command line it was launched with, its current state, and so on. This information can also be accessed by making use of the context object returned by the currentApplicationContext function.

5.12.1. How to Start and Access Multiple Applications Under Test

When testing multiple applications from a single test script, the first step is to ensure that no application is set to be automatically started. Using the Squish IDE, click the Test Suite Settings toolbar button (in the Test Suites view (Section 8.2.17)) to make the test suite's Settings view (Section 8.2.15) visible. Now, in the editor's "Application Under Test (AUT)" section, make sure that the Automatically start the AUT checkbox is unchecked.

The function used to start an application is startApplication. This function starts the given application (assuming it is located in an application path—see AUTs and Settings (Section 7.3)) using the given command line arguments and returns a corresponding ApplicationContext object. The application context object is a handle that refers to the application.

Optionally, as the second and third parameters, a host and port can be passed to the startApplication function. This way, the startApplication function will connect to the squishserver on the specified host and listen to the specified port, instead of using the default host and port (as specified in the Squish IDE's settings or on the squishrunner's command line). This allows us to control multiple applications on multiple computers from a single test script.

Special care must be taken if the application is using a different GUI toolkit than the test suite's default toolkit. The global testSettings Object (Section 6.3.18) object allows us to set the configuration of the toolkit wrapper on a per-AUT basis. See the testSettings.setWrappersForApplication function for details on how to do this.

If we run two or more AUTs within a test script, which one should test code apply to? We can make one of the AUTs the “active” application by using the setApplicationContext function, passing an ApplicationContext as the sole parameter. Once the call is made, all script code applies to the active application—unless another setApplicationContext call is made to change the active application. Note that whenever we call the startApplication function, not only is the application's ApplicationContext object returned, but the application is automatically set to be the active application.

We can obtain a list of all the currently running AUTs' ApplicationContext objects, by calling the applicationContextList function. And we can retrieve the context object of the active application by calling the currentApplicationContext function.

[Note]AUT Sub-processes

If you want to record and access applications which are started by the AUT itself, and not by Squish, see the Record / Replay on Sub-Processes started by the AUT (Section 7.7) section.

We will now look at some examples that show how to start multiple AUTs and how to use ApplicationContext objects to query them.

We will take as an example a client/server chat system. The system has a chat server called chatserver, written in Qt, which must be running for communication to take place, and two chat clients, one written in Qt called chatclientqt, and the other written in Windows called chatclientwin. We will use a Squish for Qt package, which includes the wrappers for Qt and native Windows applications, and a Qt test suite.

In the test we will first start the chat server. Then we start two clients; these automatically connect to the chat server at startup. We will then type something into the message editor of the first client and check that the second client received the message.

Python
startApplication("chatserver")
client1 = startApplication("chatclientqt")
testSettings.setWrappersForApplication("chatclientwin", ("Windows"))
client2 = startApplication("chatclientwin")

setApplicationContext(client1)
editor = waitForObject("ChatWindow.messageEditor")
type(editor, "Message for client #2")

setApplicationContext(client2)
msgView = waitForObject("ChatWindow.messageView")
test.compare(msgView.text, "Message for client #2")
JavaScript
startApplication("chatserver");
var client1 = startApplication("chatclientqt");
testSettings.setWrappersForApplication("chatclientwin", ["Windows"]);
var client2 = startApplication("chatclientwin");

setApplicationContext(client1);
var editor = waitForObject("ChatWindow.messageEditor");
type(editor, "Message for client #2");

setApplicationContext(client2);
var msgView = waitForObject("ChatWindow.messageView");
test.compare(msgView.text, "Message for client #2");
Perl
startApplication("chatserver");
my $client1 = startApplication("chatclientqt");
testSettings->setWrappersForApplication("chatclientwin", ("Windows"));
my $client2 = startApplication("chatclientwin");

setApplicationContext($client1);
my $editor = waitForObject("ChatWindow.messageEditor");
type($editor, "Message for client #2");

setApplicationContext($client2);
my $msgView = waitForObject("ChatWindow.messageView");
test::compare($msgView->text, "Message for client #2");
Ruby
startApplication("chatserver")
client1 = startApplication("chatclientqt")
testSettings.setWrappersForApplication("chatclientwin", ("Windows"))
client2 = startApplication("chatclientwin")

setApplicationContext(client1)
editor = waitForObject("ChatWindow.messageEditor")
type(editor, "Message for client #2")

setApplicationContext(client2)
msgView = waitForObject("ChatWindow.messageView")
Test.compare(msgView.text, "Message for client #2")
Tcl
startApplication "chatserver"
set client1 [startApplication "chatclientqt"]
testSettings setWrappersForApplication chatclientwin { Windows }
set client2 [startApplication "chatclientwin"]

setApplicationContext $client1
set editor [waitForObject "ChatWindow.messageEditor"]
invoke type $editor "Message for client #2"

setApplicationContext $client2
set msgView [waitForObject "ChatWindow.messageView"]
test compare [property get $msgView text] "Message for client #2"

We begin by starting each of the applications in turn, although we only keep references to the client AUTs' ApplicationContext objects since we don't directly access the server in the test. For the Windows client we set the toolkit wrapper to use the Windows wrapper instead of the default Qt wrapper. Once the applications are running we make the first client the active AUT since the active AUT is currently client2 since that was the AUT started by the most recent startApplication call. Then we get a reference to the client's chat editor and type some text into it. And at the end, we make the second client the active AUT, get a reference to its chat editor (a different widget this time since the toolkit is different—Java rather than Qt), and we compare the second client's editor's text with the text we sent from the first client.

5.12.2. Notes on using startApplication with Android

For Android the autName in startApplication function is the name of the package. It may be followed by slash plus activity. Without an activity, the package main activity is started, otherwise the one given.

When the activity is a dash, then the application is started but no activity is started. Use this when your AUT uses activities from other packages.

Finally, the package name may be prefixed with a device string, a set of launcher argument settins plus colon. So when specifying everything, the autName looks like device{option1,...}:package/activity and without options device:package/activity. Likewise device maybe omitted when options are wanted but the device is the default one (provided by the IDE or squishrunner).

If for example only one of multiple started apps should have their settings cleared when started by Squish, the startApplication command can use the --clear-app-settings (Section 7.4.7.2.2) launcher argument in the application string. E.g. startApplication("{clear-app-settings}:com.froglogic.addressbook").

[Note]Note

All packages started by Squish must be set as an instrumentation target using the squishandroid.jar library, found in the lib directory.

Using the New... in Squish testsuite settings page, Squish automatically creates and installs such an instrumentation package.

Also see Installing Squish for Android (Section 3.1.14)

We will take as example a package com.example.android.tools from which a package com.example.android.carpenter launches an activity when tapping a button. A tool selection will automatically close the activity and we are back at our main activity.

Python
ctx0 = startApplication("com.example.android.tools/-")
ctx1 = startApplication("com.example.android.carpenter")
tapObject(waitForObject(":Choose Tool_Button")
setApplicationContext(ctx0)
tapObject(waitForObjectItem("_List", "Hammer"))
setApplicationContext(ctx1)
JavaScript
var ctx0 = startApplication("com.example.android.tools/-");
var ctx1 = startApplication("com.example.android.carpenter");
tapObject(waitForObject(":Choose Tool_Button");
setApplicationContext(ctx0);
tapObject(waitForObjectItem("_List", "Hammer"));
setApplicationContext(ctx1);
Perl
my $ctx0 = startApplication("com.example.android.tools/-");
my $ctx1 = startApplication("com.example.android.carpenter");
tapObject(waitForObject(":Choose Tool_Button");
setApplicationContext(ctx0);
tapObject(waitForObjectItem("_List", "Hammer"));
setApplicationContext(ctx1);
Ruby
ctx0 = startApplication("com.example.android.tools/-")
ctx1 = startApplication("com.example.android.carpenter")
tapObject(waitForObject(":Choose Tool_Button")
setApplicationContext(ctx0)
tapObject(waitForObjectItem("_List", "Hammer"))
setApplicationContext(ctx1)
Tcl
set ctx0 [startApplication "com.example.android.tools/-"]
set ctx1 [startApplication "com.example.android.carpenter"]
invoke tapObject [waitForObject ":Choose Tool_Button"]
setApplicationContext $ctx0
invoke tapObject [waitForObjectItem "_List", "Hammer"]
setApplicationContext $ctx1

5.12.3. How to Use ApplicationContext Objects

It is possible to use an ApplicationContext object to retieve information about the AUT it refers to. The application context of the AUT defined in the test suite settings can be retrieved using the defaultApplicationContext function, and of the currently running AUT by the currentApplicationContext function. When multiple AUTs are started there should not be any AUT defined in the test suite settings—each AUT's context object can be retrieved as the return value of the call to the startApplication function which is used to start the AUT, or from the applicationContextList function which returns all the AUTs' context objects.

The Application Context (Section 6.3.12) section details the properties and functions that are accessible from ApplicationContext objects. Here are some examples.

Python
ctx = currentApplicationContext()
test.log(ctx.commandLine)
test.log(ctx.cwd)
JavaScript
var ctx = currentApplicationContext();
test.log(ctx.commandLine);
test.log(ctx.cwd);
Perl
my $ctx = currentApplicationContext();
test::log($ctx->commandLine);
test::log($ctx->cwd);
Ruby
ctx = currentApplicationContext()
Test.log(ctx.commandLine)
Test.log(ctx.cwd)
Tcl
set ctx [currentApplicationContext]
test log [applicationContext $ctx commandLine]
test log [applicationContext $ctx cwd]

Here we print the command line the AUT was invoked with and its current working directory—both are properties.

Python
ctx = currentApplicationContext()
peakMemory = 0
while ctx.isRunning:
    peakMemory = max(ctx.usedMemory, peakMemory)
    if not ctx.isFrozen(20):
	break
test.log("Peak Memory: %d" % peakMemory)
JavaScript
var ctx = currentApplicationContext();
var peakMemory = 0;
while (ctx.isRunning) {
    peakMemory = Math.max(ctx.usedMemory, peakMemory);
    if (!ctx.isFrozen(20))
	break;
}
test.log("Peak Memory: " + peakMemory);
Perl
my $ctx = currentApplicationContext();
my $peakMemory = 0;
while ($ctx->isRunning) {
    if ($ctx->usedMemory > $peakMemory) {
	$peakMemory = $ctx->usedMemory;
    }
    if (!$ctx->isFrozen(20)) {
	last;
    }
}
test::log("Peak Memory: $peakMemory")
Ruby
ctx = currentApplicationContext()
peakMemory = 0
while ctx.isRunning
    peakMemory = ctx.usedMemory > peakMemory ? ctx.usedMemory : peakMemory
    if !ctx.isFrozen(20)
	break
    end
end
Test.log("Peak Memory: #{peakMemory}")
Tcl
set ctx [currentApplicationContext]
set peakMemory 0
while {[applicationContext $ctx isRunning] == 1} {
    if {[applicationContext $ctx usedMemory] > $peakMemory} {
	set peakMemory [applicationContext $ctx usedMemory]
    }
    if {![applicationContext $ctx isFrozen 20]} {
	break
    }
}
test log "Peak Memory: $peakMemory"

Here we access the currently running AUT and keep track of the maximum amount of memory it is using. We break out of the loop if the application stops running (in which case isRunning will be false), or if the application becomes unresponsive (frozen), after waiting 20 seconds.

Python
ctx = currentApplicationContext()
test.log("STDOUT", ctx.readStdout())
test.warning("STDERR", ctx.readStderr())
JavaScript
var ctx = currentApplicationContext();
test.log("STDOUT", ctx.readStdout());
test.warning("STDERR", ctx.readStderr());
Perl
my $ctx = currentApplicationContext();
test::log("STDOUT", $ctx->readStdout());
test::warning("STDERR", $ctx->readStderr());
Ruby
ctx = currentApplicationContext()
Test.log("STDOUT", ctx.readStdout())
Test.warning("STDERR", ctx.readStderr())
Tcl
set ctx [currentApplicationContext]
test log "STDOUT" [applicationContext $ctx readStdout]
test warning "STDERR" [applicationContext $ctx readStderr]

Here we have added everything that the AUT has written to stdout and stderr to the test log, classifying all stderr messages as warnings.