Automate iOS App Launch Time Measurement with XCTest Performance Tests

June 30, 2025

Recently, I've been working on automating app launch time measurements. Manual testing with Instruments works, but it's time-consuming and produces inconsistent results due to varying system/device conditions. After building an automated solution using XCTest performance tests and shell scripts, I wanted to share the approach that's now running in our CI pipeline.

Why Automate Launch Time Measurement?

Manual testing has several limitations:

Inconsistent environment conditions between test runs
No historical data for tracking trends
Time-consuming process that doesn't scale
Human error in recording and analyzing results

The good news is that Apple provides all the tools we need - XCTest performance tests for measurements, xcodebuild for building & running the required tests, and xcresulttool for extracting metrics from .xcresult file. You'll notice there's quite a few "xc" prefixed tools we'll be using - it will all make sense at the end! Let's connect these pieces to build a complete automation solution.

The Building Blocks

Step 1: Writing the Performance Test

First, we need a performance test that measures app launch time:

@MainActor
func testLaunchPerformance() throws {
    let metrics: [XCTMetric] = [XCTApplicationLaunchMetric()]
    let measureOptions = XCTMeasureOptions.default
    measureOptions.iterationCount = 5
    
    measure(metrics: metrics, options: measureOptions) {
        XCUIApplication().launch()
    }
}

Key points about this test:

XCTApplicationLaunchMetric measures application launch duration
The framework runs multiple iterations to remove any bias
The first iteration is typically discarded as a "warm-up" run
Setting iterationCount to 5 provides stable measurements for our use case

Step 2: Running a Single Test with xcodebuild

Now that we have our test, let's run it using xcodebuild. The -only-testing flag lets us run just our launch performance test:

xcodebuild test \
    -project "YourApp.xcodeproj" \
    -scheme "YourApp" \
    -destination "platform=iOS Simulator,name=iPhone 16 Pro" \
    -only-testing:YourAppUITests/YourAppUITests/testLaunchPerformance \
    -resultBundlePath "./LaunchMetrics.xcresult"

The format for -only-testing is:

-only-testing:TARGET_NAME/CLASS_NAME/METHOD_NAME

This significantly reduces test execution time by running only the specific test we need.

Step 3: Optimizing with Build-for-Testing

While the previous approach works, we can optimize our workflow by separating the build and test phases. This is especially useful when running tests multiple times or in CI/CD pipelines:

build-for-testing: Compiles your app and test bundles without running tests. Creates all artifacts needed for testing, including the .xctestrun file with test metadata.

test-without-building: Runs tests using previously built artifacts. Requires the .xctestrun file from the build phase.

This enables building once and running tests multiple times without recompilation.

# Build once
xcodebuild build-for-testing \
    -project "YourApp.xcodeproj" \
    -scheme "YourApp" \
    -destination "platform=iOS Simulator,name=iPhone 16 Pro,OS=18.2" \
    -derivedDataPath "./DerivedData"

# Find the generated .xctestrun file
XCTESTRUN_FILE=$(find ./DerivedData -name "*.xctestrun" | head -1)

# Run tests without rebuilding
xcodebuild test-without-building \
    -xctestrun "$XCTESTRUN_FILE" \
    -destination "platform=iOS Simulator,name=iPhone 16 Pro,OS=18.2" \
    -only-testing:YourAppUITests/YourAppUITests/testLaunchPerformance \
    -derivedDataPath "./DerivedData"

The .xctestrun file is a plist containing test bundle paths, environment variables, and all configuration needed for test-without-building to execute tests.

When the test runs successfully, xcodebuild automatically creates an .xcresult bundle in the DerivedData/Logs/Test directory. This bundle contains all test results, including our performance metrics, logs, and any additional artifacts generated during the test execution. We'll use this bundle in the next step to extract our launch time measurements.

Step 4: Extracting Metrics with xcresulttool

Now we need to extract our performance data from the .xcresult bundle that was created in Step 3. The xcresulttool command-line utility helps us navigate through the bundle's hierarchical structure in three sequential steps:

Step 1: Get the testsRef ID The .xcresult bundle contains multiple actions. We need to find the test action and get its testsRef ID, which points to the test results.

Step 2: Get the summaryRef ID Using the testsRef ID, we navigate to our specific test case and extract the summaryRef ID, which contains the performance metrics.

Step 3: Extract the performance metrics Using the summaryRef ID, we filter the performance metrics to find the specific launch duration measurements using the identifier com.apple.dt.XCTMetric_ApplicationLaunch-AppLaunch.duration.

#!/bin/bash
# Extract launch metrics directly from xcresult

# Find the xcresult file in DerivedData
XCRESULT_PATH=$(find "./DerivedData/Logs/Test" -name "*.xcresult" -type d | head -1)

# Step 1: Get the testsRef ID from the root action
TESTS_REF_ID=$(xcrun xcresulttool get \
    --legacy \
    --format json \
    --path "$XCRESULT_PATH" \
    | jq -r '.actions._values[0].actionResult.testsRef.id._value')

# Step 2: Navigate to our test and get the summaryRef ID
SUMMARY_REF_ID=$(xcrun xcresulttool get \
    --legacy \
    --format json \
    --path "$XCRESULT_PATH" \
    --id "$TESTS_REF_ID" \
    | jq -r '.summaries._values[0].testableSummaries._values[0]
             .tests._values[0].subtests._values[0].summaryRef.id._value')

# Step 3: Extract the launch time measurements
LAUNCH_TIMES=$(xcrun xcresulttool get \
    --legacy \
    --format json \
    --path "$XCRESULT_PATH" \
    --id "$SUMMARY_REF_ID" \
    | jq -r '.performanceMetrics._values[] 
             | select(.identifier._value == "com.apple.dt.XCTMetric_ApplicationLaunch-AppLaunch.duration")
             .measurements._values[]._value')

echo "Raw launch times (in seconds):"
echo "$LAUNCH_TIMES"

The key identifier for launch metrics is com.apple.dt.XCTMetric_ApplicationLaunch-AppLaunch.duration. This identifier remains consistent across different Xcode versions, making our script reliable for long-term use.

Step 5: Statistical Analysis with AWK

With our raw launch times extracted, let's calculate meaningful statistics. AWK is perfect for this - it's available on every macOS system and requires no additional dependencies:

echo "$LAUNCH_TIMES" | awk '
    BEGIN { 
        sum = 0
        n = 0 
    }
    NF { 
        sum += $1
        sumsq += $1 * $1
        n++
        times[n] = $1
    }
    END {
        if (n > 0) {
            avg = sum / n
            
            # Calculate standard deviation
            if (n > 1) {
                variance = (sumsq - (sum * sum / n)) / (n - 1)
                stddev = sqrt(variance)
                cv = (stddev / avg) * 100
                
                printf "\n📊 Launch Performance Summary\n"
                printf "=============================\n"
                printf "Iterations: %d\n", n
                printf "Average: %.3f seconds (%.0f ms)\n", avg, avg * 1000
                printf "Std Dev: %.3f seconds (%.1f%%)\n", stddev, cv
                printf "\nIndividual runs:\n"
                
                for (i = 1; i <= n; i++) {
                    printf "  Run %d: %.3f seconds\n", i, times[i]
                }
            }
        }
    }
'

This AWK script processes our raw measurements to calculate average, standard deviation, and coefficient of variation - giving us a complete statistical picture of our app's launch performance.

Putting It All Together

Now let's combine everything we've built into a complete automation script. Here's what our production-ready script does:

Dependency Check: Verifies that xcodebuild, xcrun, and jq are installed
Build Phase: Uses build-for-testing to compile the app and test bundles
Test Execution: Runs only our launch performance test using test-without-building
Metric Extraction: Navigates through the .xcresult bundle hierarchy to extract launch times
Statistical Analysis: Processes the data with AWK to calculate average, standard deviation, and coefficient of variation
Error Handling: Includes proper error checks and cleanup procedures

The complete script handles all these steps automatically, making it perfect for CI/CD integration. You can customize it by setting environment variables like PROJECT, SCHEME, and DEVICE.

Here's what a typical output looks like:

🚀 iOS App Launch Performance Analyzer
=====================================
Building for testing...
✓ Build complete
Running performance test...
✓ Test complete
Extracting launch metrics...

📊 Launch Performance Summary
=============================
Iterations: 5
Average: 0.662 seconds (662 ms)
Std Dev: 0.021 seconds (3.2%)

Individual runs:
  Run 1: 0.642 seconds
  Run 2: 0.659 seconds
  Run 3: 0.694 seconds
  Run 4: 0.651 seconds
  Run 5: 0.664 seconds
✅ Analysis complete!

Next Steps: Taking It Further

Now that you have the foundation, here's where to go next:

1. Real Device Testing

Simulators are great for getting things up & running and maybe to compare the delta, but real devices tell the truth. Consider:

Device Farms: AWS Device Farm, Firebase Test Lab, BrowserStack
In-House Device Lab: Set up dedicated devices on your network
Device Matrix: Test on different device generations (iPhone 12 vs iPhone 16)

2. Advanced Metrics

Don't stop at launch time:

Time to Interactive (TTI): When can users actually interact?
First Meaningful Paint: When does useful content appear?
Memory Usage at Launch: Track memory spikes during startup
CPU Usage Patterns: Identify performance bottlenecks

Conclusion

We've built a complete automation solution for iOS app launch time measurement. The approach combines XCTest performance tests, xcodebuild automation, and shell script processing to create a CI-ready pipeline for tracking launch performance.

Key benefits of this solution:

Automated and consistent measurements
Statistical analysis for tracking performance trends
CI/CD integration for continuous monitoring
No external dependencies beyond standard macOS tools

This automated approach provides reliable data for making informed decisions about app performance and catching regressions early in the development cycle.

ritesh.hh