Alexey Alter-Pesotskiy

3 minute read

Gaining access to Command-line from XCTest

What’s this?

The command line is quite a powerful tool. It can be used to automate a bunch of things here and there. Obviously, it’s not only about Shell scripts, you can run any other tool from it as well. But using it directly is not always possible.

XCTest, for instance, works inside a sandbox, which means you literally can’t even stick your nose out of the box. Well, almost. Let’s bypass this limitation and learn how to gain access to the command line from tests.

And this is useful because…?

  1. This uncovers a whole new world of simulator-related features that are available through simctl and idb but not through XCTest API yet. For example:
  2. This opens up the possibility to create and use a standalone mock server that can completely replace the real backend for the testing purposes
  3. This allows executing any macOS UI actions through Applescript from your XCTests
  4. This gives the option to automate web-related actions via playwright or selenium instead of opening Safari on iOS
  5. This offers a pretty flexible way to export anything you want from the test to the host machine (screenshots, mocks, test data, etc.)
  6. And much much more 🤠

Cool, what’s the trick?

We need to create an HTTP server that will be listening for requests. Then we can send a request from XCTest to the server and run whatever we want. The server can even return the output back to XCTest if required.

Okay, show me how to do this!

Proof of concept

GIVEN we have a server running on localhost:4567
AND we sent a request to /terminal endpoint from XCTest with the desired command
WHEN server receives the request
AND server executes the command
THEN server returns the result back to XCTest
AND XCTest verifies the result

Walkthrough

  1. Choose any web framework of your choice (they are pretty much all the same for our use case)

     {
   "js": "express", // https://github.com/expressjs/express
   "rb": "sinatra", // https://github.com/sinatra/sinatra
   "py": "flask", // https://github.com/pallets/flask
   "go": "gin", // https://github.com/gin-gonic/gin
   "..": "..." // ...
 }

  2. Create an HTTP server (I picked up { "js": "express" } as an example)

    • Command-line:

        npm install express
  touch server.js
  open server.js

    • Source code:

        // server.js

  const process = require("child_process");
  const express = require("express");
  const app = express();
  const port = 4567;

  app.use(express.json());

  app.listen(port);

  app.post("/terminal", (req, res) => {
    const output = exec(req.body.command, req.query.async).toString("utf8").trim();
    res.send(output);
  });

  function exec(command, async) {
    if (async === "true") {
      return process.exec(command);
    } else {
      return process.execSync(command);
    }
  }

  3. Run the server

     node server.js

  4. Create the test file

     import XCTest

 class SampleUITests: XCTestCase {
     let host = "http://localhost"
     let port: UInt16 = 4567

     func testPlatformName() {
         let platformName = exec("sw_vers -productName")
         XCTAssertEqual(platformName, "macOS")
     }

     func exec(_ command: String, async: Bool = false) -> String {
         let urlString = "\(host):\(port)/terminal?async=\(async)"
         guard let url = URL(string: urlString) else { return "" }

         var request = URLRequest(url: url)
         request.httpMethod = "POST"
         request.setValue("application/json", forHTTPHeaderField: "Content-Type")
         request.httpBody = try? JSONSerialization.data(withJSONObject: ["command": command], options: [])
         var output = ""

         if async {
             // Do not wait for the command to complete
             URLSession.shared.dataTask(with: request).resume()
         } else {
             // Wait for the command to complete
             let semaphore = DispatchSemaphore(value: 0)
             let task = URLSession.shared.dataTask(with: request) { data, response, error in
                 if let data = data, let string = String(data: data, encoding: .utf8) {
                     output = string
                     semaphore.signal()
                 }
             }
             task.resume()
             semaphore.wait()
         }
         return output
     }
 }
  5. Run the test ✅
  6. Adjust the scripts to your needs and enjoy hacking!

Does this work on real devices?

Emm, yeah, but not out of the box. You need to make sure that the server is accessible from device.

There are two options to achieve this:

  • Connect to Wi-Fi and send requests to the server using the desktop IP address instead of localhost

or

  • Expose your local server to the internet (e.g.: via ngrok)

Alrighty, does this work on CI?

Sure thing, works like a charm!

Just make sure to run the server in the background on CI:

node server.js &

Thanks a million!

You’re more than welcome! Check out this GitHub repo for more details 🙂

Updated: