UI Testing in Xcode

Wil Turner Developer Tools Brooke

Callahan Developer Tools

Overview

UI testing
- Find and interact with UI elements
- Validate UI properties and state
UI recording
Test reports

XCTest

Xcode’s testing framework

Test case subclasses
Test methods
Assertions
Integrated with Xcode
CI via Xcode Server and xcodebuild
Swift and Objective-C

Testing Matrix

Accessibility

Rich semantic data about
UI UIKit and AppKit integration
APIs for fine tuning
UI tests interact with the app the way a user does

Requirements

UI testing depends on new OS features
- iOS 9
- OS X 10.11
Privacy protection
- iOS devices
  - Enabled for development
  - Connected to a trusted host running Xcode
- OS X must grant permission to Xcode Helper
  - Prompted on first run

Getting Started

Xcode target type
APIs
UI recording

UI Testing Xcode Targets

UI tests have special requirements
- Execute in a separate process
- Permission to use Accessibility
New Xcode target templates
- Cocoa Touch UI Testing Bundle (iOS)
- Cocoa UI Testing Bundle (OS X)
“Target to be Tested” setting

APIs

Three new classes

XCUIApplication
XCUIElement
XCUIElementQuery

UI Recording

Interact with your app
Recording generates the code
- Create new tests
- Expand existing tests

What Did You See?

Adding a UI testing target
Using recording
- Finding UI elements
- Synthesizing user events
Adding validation with XCTAssert

UI Testing API

Example

Testing the Add button

// application:
let app = XCUIApplication()
app.launch()

// element and query:
let addButton = app.buttons["Add"]
addButton.tap()

// assertion:
XCTAssertEqual(app.tables.cells.count, 1)

XCUIApplication

Proxy for the tested application
- Tests run in a separate process
Launch
- Always spawns a new process
- Implicitly terminates any preexisting instance
Starting point for finding elements

XCUIElement

Proxy for elements in application
Types
- Button, Cell, Window, etc.
Identifiers
- Accessibility identifier, label, title, etc.
Most elements are found by combining type and identifier

Element Hierarchy

Element Uniqueness

Every XCUIElement is backed by a query
Query must resolve to exactly one match
- No matches or multiple matches cause test failure
- Failure raised when element resolves query
Exception
- exists property

Event Synthesis

Simulate user interaction on elements
APIs are platform-specific

button.click() // OS X
button.tap() // iOS
textField.typeText("Hello, World!") // iOS & OS X

XCUIElementQuery

API for specifying elements

Queries resolve to collections of accessible elements
- Number of matches:count
- Specify by identifier: subscripting
- Specify by index:elementAtIndex()

How do queries work?

Relationships
Filtering

Expressing relationships

Descendants
Children
Containment

Filtering

Element type
- Button, table, menu, etc.
Identifiers
- Accessibility identifier, label, title, etc.
Predicates
- Value, partial matching, etc.

Combining Relationships and Filtering

descendantsMatchingType()

So common, we provide convenience API for each type

let allButtons = app.descendantsMatchingType(.Button)
let allCellsInTable = table.descendantsMatchingType(.Cell)
let allMenuItemsInMenu = menu.descendantsMatchingType(.MenuItem)

can also:

let allButtons = app.buttons
let allCellsInTable = table.cells
let allMenuItemsInMenu = menu.menuItems

childrenMatchingType()

Differentiates between any descendant and a direct child relationship

let allButtons = app.buttons // descendantsMatchingType(.Button)
let childButtons = navBar.childrenMatchingType(.Button)

containingType()

Find elements by describing their descendants

let cellQuery = cells.containingType(.StaticText, identifier:"Groceries")

Predicate variant also available

XCUIElementQuery

Combining relationships and filtering

descendantsMatchingType()
childrenMatchingType()
containingType()

Combining Queries

Queries can be “chained” together Output of each query is the input of the next query

let labelsInTable = app.tables.staticTexts

Getting Elements from Queries

Subscripting
- table.staticTexts[“Groceries”]
Index
- table.staticTexts.elementAtIndex(0)
Unique
- app.navigationBars.element

Evaluating Queries

Queries are evaluated on demand
XCUIElement
- Synthesizing events
- Reading property values
XCUIElementQuery
- Getting number of matches (.count)
- Getting all matches (.allElementsBoundByAccessibilityElement)
Re-evaluated when UI changes

Queries and Elements

Similar to URLs

Creating a URL does not fetch a resource
- URL could be invalid, error raised when requested
Queries and elements
- Just a specification for accessible elements in the tested application
- Not resolved until needed

API Recap

Accessibility and UI Testing

Debugging tips

Not accessible
- Custom view subclasses
- Layers, sprites, and other graphics objects
Poor accessibility data
Tools
- UI recording
- Accessibility inspectors

Improving data

Interface Builder inspector
API
- UIAccessibility (iOS)
- NSAccessibility (OS X)

What Did You See?

Advanced UI testing
Correcting queries
Looping over elements
Improving accessibility

Test Reports

UI Refresh
Show results for all tests
- Pass/fail
- Failure reason
- Performance metrics
Same UI in Xcode and in Xcode Server
Per-device results for Xcode Server

UI testing additions

New data
Screenshots
Nested activities

Nested activities

UI testing APIs have several steps
Typing into a textfield
- Wait for the app to idle
- Evaluate the textfield query
- Synthesize the text input
- Wait for the app to idle
QuickLook for screenshots