Deciphering the Messages of Apple’s T2 Coprocessor

00. Introduction

In 2018, we released two whitepapers exploring Apple’s T2 coprocessor. The first paper explored the new system architecture of the late 2017 iMac Pro and 2018 MacBook Pro and how the inclusion of the T2 coprocessor enabled the secure boot and encrypted storage capabilities of this new platform. The second paper performed a deep-dive into the Secure Boot process and raised the concern that the T2 coprocessor, running a full version of BridgeOS, may expose a large attack surface. In this article, we explore the exposed services, identify the communications transport and decipher the protocols macOS uses to communicate with the T2 coprocessor.

It will shock nobody that the T2 coprocessor communicates with macOS using Apple’s XPC interprocess communication mechanism. However, since the low-level workings of this communication mechanism are documented sparsely or not at all, this article aims to record not only the standard message format, but also how the T2’s use of XPC messaging appears to differ from conventional use of XPC. Building upon this understanding of the low-level communication channel, we demonstrate how one may analyze the network traffic between a macOS client and a T2 server and use this to exercise additional T2 functionality.

Our exploration of Apple’s T2 chip is based on our observations from two late 2017 iMac Pros and a 2018 MacBook Pro.

01. RemoteXPC

XPC is an interprocess communication mechanism designed by Apple which utilizes serialized property lists for messages. Apple has extended this mechanism to support a new range of functionality. Built on top of Network.framework, the RemoteXPC facility allows for XPC message passing between networked endpoints. The T2 coprocessor utilizes RemoteXPC to communicate to the host macOS installation. Currently, T2 services utilize this facility to pass messages back and forth as well as perform more complicated actions such as file transfers between the two domains, but we may very well see RemoteXPC used more broadly for cross-product communication in the future as the foundation is there.

T2 services that wish to utilize this transport register with the subsystem using the RemoteXPC library located at /System/Library/PrivateFrameworks/RemoteXPC.framework/RemoteXPC on macOS and baked into the dyld shared cache on BridgeOS. Services advertise their endpoints by invoking the xpc_remote_connection_create_remote_service_listener function with their reverse domain name notation endpoint identifier (e.g. com.apple.sysdiagnose.remote) that will be used to look up the service and set up an event handler for incoming connections to be routed to. The services then can then parse and reply to incoming messages through the use of strongly typed XPC dictionaries.

02. Exposed T2 Services

Located in /usr/libexec, Apple ships a feature-rich utility to interrogate and interface with the RemoteXPC facility called remotectl:

$ remotectl
usage: remotectl list
usage: remotectl show (name|uuid)
usage: remotectl get-property (name|uuid) [service] property
usage: remotectl dumpstate
usage: remotectl browse
usage: remotectl echo [-v service_version] [-d (name|uuid)]
usage: remotectl eos-echo
usage: remotectl netcat (name|uuid) service
usage: remotectl relay (name|uuid) service
usage: remotectl loopback (attach|connect|detach|suspend|resume)
usage: remotectl convert-bridge-version plist-in-path bin-out-path
usage: remotectl heartbeat (name|uuid)
usage: remotectl trampoline [-2 fd] service_name command args ... [ -- [-2 fd] service_name command args ... ]

A list command will display the connected devices and some basic information about them:

$ /usr/libexec/remotectl list
4525CE68-3808-49CB-89A5-9DAE6E329B39 localbridge      iBridge2,1   J137AP   2.0 (15P2064/15.16.2064.0.0,0)

A show command followed by a device name will display detailed information about the connected device as well as a list of advertised service endpoints:

$ ./remotectl show localbridge
Found localbridge (bridge)
    State: connected (connectable)
    UUID: 8366C34C-12B7-4BAD-9BFB-9896FFEC6A37
    Product Type: iBridge2,3
    OS Build: 2.4.1 (15P6613)
    Messaging Protocol Version: 1
    Heartbeat:
        Last successful heartbeat sent 9.892s ago, received 9.888s ago (took 0.004s)
        504 heartbeats sent, 0 received
    Properties: {
        AppleInternal => false
        ChipID => 32786
        EffectiveProductionStatusSEP => true
        HWModel => J680AP
        HasSEP => true
        LocationID => 2148532224
        RegionInfo => LL/A
        EffectiveSecurityModeAp => true
        FDRSealingStatus => true
        SigningFuse => true
        BuildVersion => 15P6613
        OSVersion => 2.4.1
        BridgeVersion => 15.16.6613.0.0,0
        SensitivePropertiesVisible => true
        ProductType => iBridge2,3
        Image4CryptoHashMethod => sha2-384
        SerialNumber => C02X60YUJGH5
        BootSessionUUID => EF9BCE55-BF25-4ADA-8034-FA46AE6DCEEF
        BoardId => 11
        EffectiveProductionStatusAp => true
        EffectiveSecurityModeSEP => true
        UniqueChipID => 1689691881472038
        UniqueDeviceID => 00008012-000600C40C600026
        RemoteXPCVersionFlags => 72057594037927942
        CertificateSecurityMode => true
        CertificateProductionStatus => true
        ModelNumber => Z0V00007Z
        RegionCode => LL
        InterfaceIndex => 8
        HardwarePlatform => t8012
        Image4Supported => true
    }
    Services:
        com.apple.sysdiagnose.stackshot.remote
        com.apple.eos.BiometricKit
        com.apple.nfcd.relay.control
        com.apple.logd.remote-daemon
        com.apple.CSCRemoteSupportd
        com.apple.aveservice
        com.apple.osanalytics.logTransfer
        com.apple.multiverse.remote.bridgetime
        com.apple.nfcd.relay.uart
        com.apple.private.avvc.xpc.remote
        com.apple.sysdiagnose.remote
        com.apple.corespeech.xpc.remote.control
        com.apple.corespeech.xpc.remote.record
        com.apple.powerchime.remote
        com.apple.xpc.remote.multiboot
        com.apple.bridgeOSUpdated
        com.apple.eos.LASecureIO

Two of the most interesting commands are netcat and relay, which establish a raw connection to a specified service and either redirect stdin/stdout to the connection or fork-off a listen socket for use with third-party applications. Available services vary by platform (i.e. MacBook Pro versus iMac Pro) and cover a wide range of functionality. While the implementation and functionality of these services is outside the scope of this article, we do strongly encourage you to look for yourselves and to share your findings.

Of particular note, remotectl does not require the invoking user to have root privileges to leverage these commands. This means that a non-privileged user may directly interface with T2 services at a very low level and access functionality that might not be exposed at the host macOS. They just need to understand the underlying transport.

03. Communication Transport

To discover and communicate with advertised services, the T2 exposes itself to macOS as a network interface, assigned as en6 on our lab machines. This macOS interface is configured for IPv6 with a universally static local address of fe80::aede:48ff:fe00:1122. The T2 exposes itself at a fixed IPv6 address of fe80::aede:48ff:fe33:4455.

$ ifconfig
...
VHC128: flags=101<UP,PROMISC> mtu 0
...
en6: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    ether ac:de:48:00:11:22
    inet6 fe80::aede:48ff:fe00:1122%en6 prefixlen 64 scopeid 0x9
    nd6 options=201<PERFORMNUD,DAD>
    media: autoselect (100baseTX <full-duplex>)
    status: active

The T2 network interface (en6) is privileged and gated by the com.apple.private.RemoteServiceDiscovery.device-admin and com.apple.private.network.intcoproc.restricted entitlements. Routing and capturing traffic through this interface is not normally possible. However, if SIP is disabled and VHC128 interface is brought up with the ifconfig VHC128 up command, traffic can be captured and analyzed with Wireshark on the VHC128 interface.

04. Decoding Message Layers

MacOS and the T2 communicate over a typical network stack, with a few notable exceptions. Ethernet frames are encapsulated within Mobile Broadband Interface Model (MBIM) packets for transmission over what we can therefore infer is a USB-based interface. For simple application-level messages, one MBIM frame will typically contain a single message, but for larger data transfers, multiple Ethernet frames will be encapsulated within each packet. This is somewhat ironic, as often a data transfer segment will be split into MTU-sized chunks at the TCP layer, only to be combined into a single packet at the MBIM layer.

Above the TCP/IP layer, macOS and the T2 use the HTTP/2 protocol to open, and often maintain, persistent connections between different applications. For the uninitiated, the following is a very cursory description of the protocol:

Conventionally, HTTP/2 is used as a way of effectively bundling the multiple HTTP/1 connections two endpoints might form over a short period of time into a single connection to reduce overhead. An HTTP/2-compliant HEADERS frame would also contain standard HTTP header parameters that describe the payload. However, Apple appears to be using the HTTP/2 protocol only as a mechanism to maintain persistent connections, passing empty HEADERS frames to open new streams. This breaks section 8.2.1 of the HTTP/2 specification. We work around this by modifying the h2 module to allow empty HEADERS frames.

Apple’s use of the HTTP/2 protocol is also irregular for two other reasons. First, the HTTP/2 specification mandates that stream IDs must be created in a monotonically increasing order. However, Apple’s application endpoints appear to hardcode a handful of particular stream IDs for certain communications, which when opened do not conform to this monotonic ordering requirement. Second, client-server communication patterns often use multiple streams in unusual ways. For instance, heartbeat messages send requests over stream 1 and responses on stream 3.

Above the HTTP/2 layer, the observed message format is similar to standard XPC, with a few differences. The largest is the encapsulation of each XPC message within what we call the XPC wrapper, for lack of a better term. Every XPC object is encapsulated within an XPC wrapper. However, it is not uncommon to see XPC headers sent with missing XPC payloads, most likely as control signals. File transfers will send raw data encapsulated directly in the HTTP/2 DATA frame (i.e. no XPC), and an empty HTTP/2 DATA frame appears to always signal the endpoint to close the stream, although this behavior is likely application-specific.

Note: for exceptionally large data payloads, such as file transfers, the data will be split across multiple MBIM packets.

05. XPC Wrapper Fields and Behavior

The XPC wrapper has the following structure, with fields in little-endian order:

struct XPC_Wrapper
{
    uint32_t magic;
    uint32_t flags;
    uint64_t body_len;
    uint64_t msg_id;
}

Its magic bytes are 0x29B00B92. We have not fully-reverse engineered the flags field, but observed protocols give some indication as to the meaning of certain bits. The body_len field describes the length of the following payload. The msg_id field is used to identify messages. It is used in slightly different ways depending on the protocol. As examples, we will look at the heartbeat and sysdiagnose protocols.

The heartbeat protocol is not exposed as a discrete RemoteXPC service but is implemented at the service discovery layer. Every 20 seconds or so, macOS will send a heartbeat request to the T2 and the T2 will immediately respond with a heartbeat reply. The reply’s msg_id field matches the request’s msg_id and the msg_id increments by 2 for every request/reply pair. The 17th bit of the flags field appears to designate a heartbeat request and the 18th bit of the flags field appears to designate a heartbeat reply.

Heartbeat request:

Flags: 0b 00000000 00000001 00000001 00000001 (0x10101)
MessageId: 0x23f89

Heartbeat reply:

Flags: 0b 00000000 00000010 00000001 00000001 (0x20101)
MessageId: 0x23f89

The sysdiagnose service protocol, exposed as com.apple.sysdiagnose.remote, is more complex and yields more clues about the flags field. The first bit of the flags field appears to always be set. Also, it appears that no other bit in the right-most octet is ever set. This may indicate its use as a protocol version field. The 9th bit appears to indicate that an XPC object will be present in the payload, but is not set when the XPC object contains only an empty dictionary. The 21st and 22nd bits appear to signal the opening of a new stream to perform a file transfer. When a new stream is opened to transfer a file from the T2 to macOS, the T2 will send an XPC wrapper containing the 21st bit and no payload. macOS will then reply with an XPC wrapper containing the 22nd bit and no payload. Then the T2 will send raw data over the HTTP/2 stream until the file has been transferred. The 23rd flags bit appears to be used during an initial handshake at the start of the sysdiagnose protocol. This handshake is initiated on an otherwise-unused HTTP/2 stream in which macOS will send an empty XPC wrapper with this 23rd bit set and the T2 will reply with an identical empty XPC wrapper.

Flag bits:
00000000 00000000 00000000 00000001 - Always set
00000000 00000000 00000001 00000000 - Data present
00000000 00000001 00000000 00000000 - Heartbeat request
00000000 00000010 00000000 00000000 - Heartbeat reply
00000000 00010000 00000000 00000000 - Opening a new file_tx stream
00000000 00100000 00000000 00000000 - Reply from file_tx stream
00000000 01000000 00000000 00000000 - Sysdiagnose init handshake

The msg_id field helps link file transfer metadata to the file transfer data itself. When sending a file, the T2 will send a description of the file on HTTP/2 stream 1 in the XPC payload, including a msg_id to be used in the future. It will then open a new stream for the file transfer and send an empty XPC wrapper with that msg_id before sending the data. The msg_id field also appears to have different semantic meaning depending on the context of the messages sent. For instance, in the sysdiagnose protocol, macOS sends the T2 a message “REQUEST_TYPE=1” with a msg_id of 0x1 and the T2 later replies with a message “RESPONSE_TYPE=1” with a msg_id of 0x2. Other file transfers in the middle of the sysdiagnose protocol, which are part of a different long-running connection, appear to increment the msg_id field by 2 between messages.

Further details of the sysdiagnose protocol will be covered below.

06. XPC Object Decoding

The XPC object format itself is undocumented. Apple publishes an application-level API for XPC messaging, but reserves the right to change the XPC object format at any time. Despite its being the de-facto standard message-passing format on Apple systems, there is relatively little third-party documentation of this object format. Two notable third-party resources were instrumental in learning to parse the XPC object format. The first is Jonathan Levin’s *OS Internals: Vol 1 book, which is an excellent resource for understanding the service and messaging frameworks of Apple’s operating systems. The second was a slide deck from Ian Beer’s talk at the Jailbreak Security Summit.

XPC objects are 4-byte aligned. A magic number and version number compose the XPC header. As mentioned in Levin’s book, the XPC header changes periodically, with previous magic bytes values of 0x58504321 (“XPC!”) and 0x40585043 (“@XPC”). In our testing on 10.13.3 “High Sierra”, we saw magic bytes of 0x42133742 (no longer a string containing “XPC”), followed by a version number 0x5, which interestingly matches the version number in Levin’s book for post-Darwin 16 despite the magic bytes update. These two fields (magic and version number) make up the entirety of the XPC object header. However, if the header is present, it is always followed by a dictionary object that contains any message contents to be transmitted, even if the dictionary is empty.

XPC objects are always prefixed with a 4-byte type field. What follows the type field is dependent on the type of the XPC object to be encoded. Many objects have similar formats, so rather than provide an exhaustive description of each XPC type, we will explain the formats of several categories of object types. xpc_types.py can be used as a more complete reference, although we have not deciphered the formats of every XPC type.

Types:
XPC_NULL              = 0x00001000
XPC_BOOL              = 0x00002000
XPC_INT64             = 0x00003000
XPC_UINT64            = 0x00004000
XPC_DOUBLE            = 0x00005000
XPC_POINTER           = 0x00006000
XPC_DATE              = 0x00007000
XPC_DATA              = 0x00008000
XPC_STRING            = 0x00009000
XPC_UUID              = 0x0000a000
XPC_FD                = 0x0000b000
XPC_SHMEM             = 0x0000c000
XPC_MACH_SEND         = 0x0000d000
XPC_ARRAY             = 0x0000e000
XPC_DICTIONARY        = 0x0000f000
XPC_ERROR             = 0x00010000
XPC_CONNECTION        = 0x00011000
XPC_ENDPOINT          = 0x00012000
XPC_SERIALIZER        = 0x00013000
XPC_PIPE              = 0x00014000
XPC_MACH_RECV         = 0x00015000
XPC_BUNDLE            = 0x00016000
XPC_SERVICE           = 0x00017000
XPC_SERVICE_INSTANCE  = 0x00018000
XPC_ACTIVITY          = 0x00019000
XPC_FILE_TRANSFER     = 0x0001a000

Fixed-sized objects, such as uint64, tend to have a simple fixed format:

Example: a uint64 with value 0x5:

00 40 00 00 05 00 00 00 00 00 00 00
|___type__| |________value________|

The value has a known size, so there is no length field present.

Variable-length objects, such as strings, may also specify a length:

Strings are null-terminated and also padded to the 4-byte alignment.

Example: a string with value “duolabs!”:

00 90 00 00 09 00 00 00 64 75 6f 6c 61 62 73 21 00 00 00 00
|___type__| |__length_| |d__u__o__l__a__b__s__!_\0_padding|

Note that even though the printable characters alone would fit the 4-byte alignment, the required null terminator requires padding out to 12 bytes.

Compound objects, such as dictionaries, are more complex. A dictionary has the following format:

The length field specifies the size of the dictionary in bytes, excluding the type and length fields, but including the num_entries field.

Note that the string-like dictionary keys do not contain a 4-byte length field, as the string type does. They are still null-terminated and 4-byte aligned.

Example: a dictionary containing two uint64s with values of 0x5 and 0x6 and keys of “five” and “six”:

00 f0 00 00 28 00 00 00 02 00 00 00 ...
|___type__| |__length_| |num_entry|
66 69 76 65 00 00 00 00 00 40 00 00 05 00 00 00 00 00 00 00 ...
|f__i__v__e_\0_padding| |_type| |value|
73 69 78 00 00 40 00 00 06 00 00 00 00 00 00 00
|s__i_x_\0| |_type| |value|

In addition to these three general categories of XPC objects, the file_transfer type is particularly noteworthy since it is used extensively in the sysdiagnose client, which we discuss below. The file_transfer object has the format:

The file_transfer type has an embedded dictionary with a fixed format. The only two fields of note are the embedded uint64 msg_id and embedded dictionary field “s” corresponding to the file_transfer_size. All other fields appear to be static. The file_transfer object is used to inform macOS that the T2 will subsequently start transferring a file of file_transfer_size bytes on a new stream identified by XPC wrapper parameter msg_id. Note that msg_id does not refer to the HTTP/2 stream ID that will be used.

Example: a file_transfer object specifying a subsequent file transfer of size 0x1b981 (113025 bytes) on a stream identified by message ID 0xe:

00 a0 01 00 0e 00 00 00 00 00 00 00 ...
|___type__| |______message_id_____|
00 f0 00 00 14 00 00 00 01 00 00 00 ...
|_type| |_length| |num_entry|
73 00 00 00 00 40 00 00 81 b9 01 00 00 00 00 00
|s_\0_pad_| |_type| |_file_transfer_size|

07. Exploring The Sysdiagnose Server

With our newfound understanding of the communications between the T2 chip and macOS, we wished to not only read messages, but also write messages and interact directly with the T2 chip. During our initial investigation of the XPC message format, we extensively used the sysdiagnose -c command to generate sample network traffic. Consequently, the sysdiagnose server on the T2 made a natural target with which to interact.

The sysdiagnose protocol can be thought of as having three semi-distinct parts. In Phase 1, the macOS sysdiagnose client sends a request to the T2 sysdiagnose server. The T2 chip begins collecting diagnostics and creates a gzipped tar archive. In Phase 2, the T2 chip uses a preexisting connection to the macOS SubmitDiagInfo service to send two metadata files with filenames of the format “stacks-<date>.ips”. Approximately 10 to 15 seconds later (on our lab machine), Phase 3 begins and the T2 chip responds to the macOS sysdiagnose client with the archive, named “bridge_sysdiagnose_<date>_Bridge_OS_Bridge_<build-version>.tar.gz”.

Interestingly, although no data appears to be exchanged over HTTP/2 stream 3 (opened during Phase 1), the T2’s sysdiagnose server will stop responding if it is not opened and an empty XPC wrapper is not sent over the stream. This appears to be entirely superfluous, although it clearly sends some signal to the server.

Sysdiagnose request:

New XPC Packet imac->t2 on HTTP/2 stream 1 TCP port 49155
XPC Wrapper: {
    Magic: 0x29b00b92
    Flags: 0b 00000000 00000000 00000001 00000001 (0x101)
    BodyLength: 0x30
    MessageId: 0x1
}
{
    "REQUEST_TYPE":
        uint64 0x0000000000000001: 1
}

Sysdiagnose response:

New XPC Packet t2->imac on HTTP/2 stream 1 TCP port 49155
XPC Wrapper: {
    Magic: 0x29b00b92
    Flags: 0b 00000000 00000000 00000001 00000001 (0x101)
    BodyLength: 0xc0
    MessageId: 0x2
}
{
    "RESPONSE_TYPE":
        uint64 0x0000000000000001: 1
    "FILE_TX":
        MessageId: 0x5
        File transfer size: 0x00000000005b49d7 5982679
    "FILE_NAME":
        "bridge_sysdiagnose_2019.01.18_16-57-46+0000_Bridge_OS_Bridge_16P375.tar.gz"
}

To implement our own sysdiagnose client, we need a way to connect to and communicate with the T2 chip. As mentioned earlier, the remotectl utility provides a relay command that will open a local port on macOS and forward any traffic directed at it to a service running on the T2. To initiate a new connection with the sysdiagnose server then, we ran: /usr/libexec/remotectl relay localbridge com.apple.sysdiagnose.remote and connected to the newly opened port. Using the twisted framework and h2 library, we were able to write a responsive sysdiagnose client application, generating raw XPC objects as needed using xpc_types.py. After understanding the messaging format and the base sysdiagnose protocol, we turned to better understanding the inputs the sysdiagnose server accepts.

The macOS sysdiagnose client has many different options about what data to collect, and some of these options modify the request made to the T2’s sysdiagnose server.

sudo sysdiagnose -cup

{
    "disableUIFeedback": True
    "shouldRunOSLogArchive": False
    "shouldRunLoggingTasks": False
    "shouldDisplayTarBall": False
    "shouldRunTimeSensitiveTasks": True
    "REQUEST_TYPE": uint64 0x0000000000000001: 1
}

From inspection of the T2’s sysdiagnose server binary, we can collect a list of parameters that the sysdiagnose server will accept:

Some of these parameters are self-explanatory, such as archiveName.

Request:

{
    "REQUEST_TYPE":
        uint64 0x0000000000000001: 1
    "archiveName":
        "duolabs"
}

Response:

{
    "RESPONSE_TYPE":
        uint64 0x0000000000000001: 1
    "MSG_TYPE":
        uint64 0x0000000000000002: 2
    "FILE_TX":
        MessageId: 0x58
        File transfer size: 0x00000000004a22b6 4858550
    "FILE_NAME":
        "duolabs.tar.gz"
}

Others are less clear. For instance, setting baseDirectory appears to cause the sysdiagnose server to hang. Passing baseDirectory sets an internal variable on the sysdiagnose server, but it is unclear from our manual analysis how this variable is used.

There are nominally 10 sysdiagnose request types:

switch ( REQUEST_TYPE )
{
    case 1u:
        sd_ops_sysdiagnose(...);
    case 2u:
        sd_ops_stackshot(...);
    case 4u:
        sd_ops_cancel(...);
    case 5u:
        sd_ops_cancelAll(...);
    case 6u:
        sd_ops_userinterrupt(...);
    case 7u:
        sd_ops_statusPoll(...);
    case 8u:
        sd_ops_airdrop(...);
    case 9u:
        sd_ops_watchList(...);
    case 10u:
        sd_ops_deleteArchive(...);

Sadly, most appear to be unimplemented, although their names may hint at Apple’s plans for the future.

void sd_ops_airdrop(...)
{
    if ( (unsigned int)os_log_type_enabled(...) )
    {
        _os_log_impl(..., "Airdrop not implemented", ...);

08. Conclusion

Apple’s T2 chip only continues to demonstrate that Apple is pushing the frontiers of secure computing. However, like much of the *OS ecosystem, the communication channels and XPC object format are opaque and can therefore end up receiving less third-party scrutiny than those of more open platforms. In this work, we aim to shine a light upon macOS and T2 communications. By exploring this facet of the T2 chip’s operations, we intend to contribute to the ongoing dialogue within the security community about how secure boot and trusted hardware play a critical role in providing a foundation for trusted computing. This foundation should continue to be extensively explored if it is to provide the bedrock upon which we build our future.

09. Tooling

Following our mission to democratize security, we are publishing the tooling we produced along the way so that researchers everywhere can take advantage of this work. As far as we are aware, the only other publicly-available tooling to analyze XPC objects is Jonathan Levin’s XPoCe tool.

https://github.com/duo-labs/apple-t2-xpc

Our tooling contributions include:

• sysdiagnose_client.py - a sysdiagnose client that can be used as a template for building future applications to communicate with the T2 chip
• sniffer.py - a scapy-based sniffer that reassembles and decodes streams of communications between macOS and the T2. It can be used to sniff the traffic on VHC128 or can read packets from a standard pcap file
• mbim.py - helper module for decoding the MBIM packet structure and extracting Ethernet frames for further processing
• xpc_wrapper.py and xpc_types.py - helper modules for encoding and decoding XPC wrappers and XPC objects
• h2/ - a modified copy of the hyper-h2 library that supports header-less HEADERS frames (which break the HTTP/2 specification)

Below, we demonstrate the usage of the xpc_types, xpc_wrapper, and mbim modules to encode and decode XPC objects. These helper modules are used in the sniffer and the sysdiagnose client, which can serve as working usage references.

Note that xpc_types aims to be as complete as possible, but we were unable to observe many of the XPC types that we datamined, particularly those that are not listed in Apple’s XPC Services API. Some types are implemented by referencing in-memory objects, but untested as we did not observe network traffic using them. However, all the main XPC object types are accounted for.

We can parse an XPC payload and turn it into XPC objects very easily:

$ python3
>>> from xpc_types import *
>>> raw_payload = b"\x42\x37\x13\x42\x05\x00\x00\x00\x00\xf0\x00\x00\x14\x00\x00\x00\x01\x00\x00\x00\x44\x55\x4f\x00\x00\x90\x00\x00\x04\x00\x00\x00\x64\x75\x6f\x00"
>>> stream = XPCByteStream(raw_payload)
>>> xpc_obj = XPC_Root(stream)
>>> print(xpc_obj)
{
    "DUO":
        "duo"
}

The above example converts raw bytes into an XPCByteStream object which has helper functions to pop fields out of the bytestream. It feeds the stream to the XPC_Root constructor, which decodes the outermost header layer and begins the potentially-recursive decoding process of the inner objects. Any individual object can be decoded as well, if you know the starting type:

>>> raw_payload = b"\x00\xf0\x00\x00\x14\x00\x00\x00\x01\x00\x00\x00\x6e\x75\x6d\x00\x00\x40\x00\x00\xce\xfa\xee\xff\xc0\xff\xca\xde"
>>> stream = XPCByteStream(raw_payload)
>>> xpc_dict = XPC_Dictionary(stream)
>>> print(xpc_dict)
{
    "num":
        uint64 0xdecaffc0ffeeface: 16053925026108209870
}

We can serialize these objects back out to bytes as well:

>>> from hexdump import hexdump
>>> payload = xpc_obj.to_bytes()
>>> hexdump(payload)
00000000: 42 37 13 42 05 00 00 00  00 F0 00 00 14 00 00 00  B7.B............
00000010: 01 00 00 00 44 55 4F 00  00 90 00 00 04 00 00 00  ....DUO.........
00000020: 64 75 6F 00                                       duo.

Finally, we can generate our own XPC objects from scratch:

>>> new_xpc_object = XPC_Root(
...     XPC_Dictionary({
...         "DUO": XPC_String("duo"),
...         "CISCO": XPC_String("cisco"),
...     }))
>>> print(new_xpc_object)
{
    "DUO":
        "duo"
    "CISCO":
        "cisco"
}
>>> hexdump(new_xpc_object.to_bytes())
00000000: 42 37 13 42 05 00 00 00  00 F0 00 00 2C 00 00 00  B7.B........,...
00000010: 02 00 00 00 44 55 4F 00  00 90 00 00 04 00 00 00  ....DUO.........
00000020: 64 75 6F 00 43 49 53 43  4F 00 00 00 00 90 00 00  duo.CISCO.......
00000030: 06 00 00 00 63 69 73 63  6F 00 00 00              ....cisco...

The constructor for each XPC object accepts either an XPCByteStream, or an appropriate representation for that object type. Note: recursive types, such as Dictionaries, do not convert non-XPC objects into XPC objects, nor do they perform a “deep” validation step of user-supplied values.

Wrapping an XPC object in an XPC wrapper is simply a matter of generating an XPC wrapper and then concatenating the XPC object bytes to it. The XPC wrapper just takes the four struct fields as ordered arguments:

>>> from xpc_wrapper import XpcWrapper
>>> xpc_bytes = new_xpc_object.to_bytes()
>>> magic = XpcWrapper.magic_bytes
>>> flags = 0x101
>>> msg_id = 0xe
>>> wrapper = XpcWrapper(magic, flags, len(xpc_bytes), msg_id)
>>> payload = wrapper.to_bytes() + xpc_bytes

Since the constructor is a bit different, we use a from_bytes classmethod to construct a new XpcWrapper from raw bytes:

>>> data = b'\x92\x0b\xb0\x29\x01\x01\x02\x00\x30\x00\x00\x00\x00\x00\x00\x00\x13\xf8\x0d\x00\x00\x00\x00\x00\x42\x37\x13\x42\x05\x00\x00\x00\x00\xf0\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00'
>>> wrapper, payload = XpcWrapper.from_bytes(data)
>>> stream = XPCByteStream(payload)
>>> xpc_obj = XPC_Root(stream)
>>> print(wrapper)
XPC Wrapper: {
    Magic: 0x29b00b92
    Flags: 0b 00000000 00000010 00000001 00000001 (0x20101)
    BodyLength: 0x30
    MessageId: 0xdf813
}
>>> print(xpc_obj)
{
}

Finally, MBIM packets can be decoded into scapy Ethernet frames as follows:

>>> from scapy.packet import Packet
>>> from mbim import MBIM
>>> pkt = Packet(b'\x00\x01\x20\x01\xe1\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x5c\x54\x59\x00\x00\x00\x00\x00\x00\x00\x10\x80\x02\x04\x82\x02\x4e\x43\x4d\x48\x0c\x00\x69\x6e\xe1\x00\x0c\x00\x4e\x43\x4d\x30\x2c\x00\x00\x00\x3a\x00\xa7\x00\x00\x00\x00\x00\x12\x0c\xea\x05\xfe\x11\x8b\x02\x00\x00\x00\x00\xd6\x1d\xea\x05\xc2\x23\xea\x05\x00\x00\x00\x00\x00\x00\x00\x00\x39\x42\xac\xde\x48\x00\x11\x22\xac\xde\x48\x33\x44\x55\x86\xdd\x60\x04\xb8\x6a\x00\x71\x06\x40\xfe\x80\x00\x00\x00\x00\x00\x00\xae\xde\x48\xff\xfe\x33\x44\x55\xfe\x80\x00\x00\x00\x00\x00\x00\xae\xde\x48\xff\xfe\x00\x11\x22\xe8\xd2\xc0\x00\x5e\x24\x01\xa6\xbb\x7e\x2a\x22\x80\x18\x08\x00\x85\xfb\x00\x00\x01\x01\x08\x0a\x26\x19\xbc\xfb\x53\x6d\x6c\xb2\x00\x00\x48\x00\x00\x00\x00\x00\x03\x92\x0b\xb0\x29\x01\x01\x02\x00\x30\x00\x00\x00\x00\x00\x00\x00\x8f\xf8\x0d\x00\x00\x00\x00\x00\x42\x37\x13\x42\x05\x00\x00\x00\x00\xf0\x00\x00\x20\x00\x00\x00\x01\x00\x00\x00\x53\x65\x71\x75\x65\x6e\x63\x65\x4e\x75\x6d\x62\x65\x72\x00\x00\x00\x40\x00\x00\x47\xfc\x06\x00\x00\x00\x00\x00')
>>> m = MBIM(pkt)
>>> for eth in m:
...     print(eth.show())
###[ Ethernet ]###
  dst       = ac:de:48:00:11:22
  src       = ac:de:48:33:44:55
  type      = IPv6
###[ IPv6 ]###
     version   = 6
     tc        = 0
     fl        = 309354
     plen      = 113
     nh        = TCP
     hlim      = 64
     src       = fe80::aede:48ff:fe33:4455
     dst       = fe80::aede:48ff:fe00:1122
###[ TCP ]###
        sport     = 59602
        dport     = 49152
        seq       = 1579418022
        ack       = 3145607714
        dataofs   = 8
        reserved  = 0
        flags     = PA
        window    = 2048
        chksum    = 0x85fb
        urgptr    = 0
        options   = [('NOP', None), ('NOP', None), ('Timestamp', (639220987, 1399680178))]
###[ Raw ]###
           load      = '\x00\x00H\x00\x00\x00\x00\x00\x03\x92\x0b\xb0)\x01\x01\x02\x000\x00\x00\x00\x00\x00\x00\x00\x8f\xf8\r\x00\x00\x00\x00\x00B7\x13B\x05\x00\x00\x00\x00\xf0\x00\x00 \x00\x00\x00\x01\x00\x00\x00SequenceNumber\x00\x00\x00@\x00\x00G\xfc\x06\x00\x00\x00\x00\x00'

Since one MBIM packet may contain many Ethernet frames (or none), we use a Python generator to allow looping over the encapsulated frames.