Building high-performance tracking SDKs for iOS

Sift Science’s iOS SDK enables mobile applications to send us device properties and application lifecycle events for use in fraud detection. There are special considerations to be taken in designing data-intensive tracking SDKs for mobile devices with limited bandwidth, battery, and permissions. Here are some techniques and principles we utilized in making our SDK simple, robust, and performant:

1. Upload data in batches

If the events your SDK is tracking do not have real-time implications, then you should batch multiple events together in a single network request to reduce the HTTP connection overhead.

2. Don’t block the main thread

When performing any non-trivial task (compression, networking, archiving), use GCD or your favorite threading library to avoid bogging down the main UI thread.

3. Use reasonable retry and back-off logic

We use exponential back-off for retrying upload on batches that fail due to server-side rate-limiting. For bad request errors, we don’t retry and instead drop the batch.

4. Compress everything

GZIP-ing all NSMutableURLRequests, particularly when dealing with events comprised of largely non-volatile content, yields immense savings in payload size.

5. Optimize collection based on how your backend will use the data

Our SDK tracks several types of mobile events. For events tracking the user’s in-app behavior, the time-series succession of events is an important signal that our systems can learn on. For other events tracking device properties, we only care about unique events. Because redundant device properties events do not provide our system with extra information, collecting and uploading them would be a waste of bandwidth. As such, we have heuristics for suppressing device properties updates unless they change or become stale in excess of a long timeout period – our next tip details how we implement this in a modular and reusable manner.

6. Use highly-configurable data structures

Because our SDK tracks different types of events with different frequencies, timeouts, and heuristics for redundancy, we use our own configurable queue to simplify parameterization that captures these factors based on collection frequency, TTL, and enqueueing logic. As an example, our queue configuration for collecting device properties events is shown below:

static const SFQueueConfig SFIosDevicePropertiesCollectorQueueConfig = {
    .appendEventOnlyWhenDifferent = YES,  // Don't append redundant events normally
    .acceptSameEventAfter = 3600,  // Only accept redundant events after an hour
    .uploadWhenMoreThan = 8,  // Upload when queue depth exceeds 8 events
    .uploadWhenOlderThan = 60,  // Upload when queue staleness exceeds 1 minute
};

7. Add an upper-bound rate limiter

We implemented a client-side token bucket as a precautionary upper-bound for suppressing unreasonably spammy events. We don’t expect this limit to be reached under normal use, but it serves as a safeguard against buggy applications or programmatically-triggered events.

8. Devise a system for connecting pre-auth to post-auth events

We want to associate mobile activities that occur before and after login for a given user. To accomplish this, we send along the IFV or IFA with each event, and our events backend can reconcile “anonymous” events once a user id is set.

9. Archive and unarchive data and configurations

Our SDK makes extensive use of NSKeyedArchiver/NSKeyedUnarchiver to make sure that our unprocessed events queue is persisted through backgrounding or unexpected termination, and also that we don’t leak memory by creating new allocations for configurations or data structures that already exist in the archive.

10. Make expensive data collection configurable

Let the clients of your SDK decide whether to allow collection of location and motion data, since these are expensive and potentially sensitive for clients or their end-users. Additionally, make sure to stop all expensive collection when the application is backgrounded and never prompt users for permissions (but do use granted permissions if the user has enabled them through the client application).