iOS Architecture at Lyft

June 30, 2014 was my first day at Lyft as the first iOS hire on the ~3 person team. The app was written in Objective-C, and the architecture was a 5000-line nested switch statement.

Since then, the team has grown to about 70 people and the codebase to 1.5M lines of code. This required some major changes to how we architect our code, and since it had been a while since we've given an update like this, now seems as good a time as any.

Requirements

The effort to overhaul and modernize the architecture began around mid-2017. We started to reach the limits of the patterns we established in the 2015 rewrite of the app, and it was clear the codebase and the team would continue to grow and probably more rapidly than it had in the past.

The primary problems that the lack of a more mature architecture presented and that we wanted to solve were:

There was not going to be one solution that would solve all of this inherently, but over the course of a few years we developed a number of processes and technical solutions to reduce these problems.

Modules

First, to provide better feature separation, we introduced modules. Every feature had its own module, with its own test suite, that could be developed in isolation from other modules. This forced us to think more about public APIs and hiding implementation details behind them. Compile times improved, and it required much less collaboration with other teams to make changes.

We also introduced an ownership model that ensured each module has at least one team that's responsible for that module's tech debt, documentation, etc.

Module types

After fully modularizing the app and having 700 modules worth of code, we took this a step further and introduced a number of module types that each module would follow.

Breaking modules down this way enabled us to implement dependency validators: we can validate that certain modules can't depend on others. For example, a logic module can't depend on a UI module, and a Service module can't import UIKit.

This module structure also prevents complicated circular dependencies, e.g. a Coupons module depending on Payments and vice versa. Instead, the Payments module can now import CouponsUI without needing to import the full Coupons feature. It's led to micromodules in some areas, but we've generally been able to provide good tooling to make this easier to deal with.

All in all we now have almost 2000 modules total for all Lyft apps.

Dependency Injection

Module types solved many of our dependency tree problems at the module level, but we also needed something more scalable than singletons at the code level.

For that we've built a lightweight dependency injection framework which we detailed in a SLUG talk. It resembles a service locator pattern, with a basic dictionary mapping protocols to instantiations:

1
2
let getNetworkCommunicator: NetworkCommunicating =
    bind(NetworkCommunicating.self, to: { NetworkCommunicator() })

The implementation of bind() doesn't immediately return NetworkCommunicator, but requires the object be mocked if we're in a testing environment:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
let productionInstantiators: [ObjectIdentifier: () -> Any] = [:]
let mockedInstantiators: [ObjectIdentifier: () -> Any] = [:]

func bind<T>(protocol: T.Type, instantiator: () -> T) -> T {
    let identifier = ObjectIdentifier(T.self)

    if NSClassFromString("XCTestCase") == nil {
        return productionInstantiators[identifier] ?? instantiator()
    } else {
        return mockedInstantiators[identifier]!
    }
}

In tests, the mock is required or the test will crash:

1
2
3
4
5
6
7
8
9
final class NetworkingTests: XCTestCase {
    private var communicator = MockNetworkCommunicator()

    func testNetworkCommunications() {
        mock(NetworkCommunicating.self) { self.communicator }

        // ...
    }
}

This brings two benefits:

  1. It forces developers to mock objects in tests, avoiding production side effects like making network requests
  2. It provided a gradual adoption path rather than updating the entire app at once through some more advanced system

Although this framework has some of the same problems as other Service Locator implementations, it works well enough for us and the limitations are generally acceptable.

Flows

Flows, inspired by Square's Workflow, are the backbone of all Lyft apps. Flows define the navigation rules around a number of related screens the user can navigate to. The term flow was already common in everyday communications ("after finishing the in-ride flow we present the user with the payments flow") so this terminology mapped nicely to familiar terminology.

Flows rely on state-driven routers that can either show a screen, or route to other routers that driven by different state. This makes them easy to compose, which promoted the goal of feature isolation.

At the core of flows lies the Routable protocol:

1
2
3
protocol Routable {
    let viewController: UIViewController
}

It just has to be able to produce a view controller. The (simplified) router part of a flow is implemented like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
final class Router<State> {
    private let routes: [(condition: (State) -> Bool, routable: Routable?)]

    func addRoute(routable: Routable?, _ condition: @escaping (State) -> Bool) {
        self.routes.append((condition, routable))
    }

    func route(for state: State) -> Routable? {
        self.routes.first { $0.condition(state) }
    }
}

In other words: it takes a bunch of rules where if the condition is true (accepting the flow's state as input) it provides a Routable. Each flow defines its own possible routes and matches those to a Routable:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
struct OnboardingState {
    let phoneNumber: String?
    let verificationCode: String?
    let email: String?
}

final class OnboardingFlow {
    private let router = Router<OnboardingState>
    private let state = OnboardingState()

    init() {
        self.router.addRoute({ $0.phoneNumber == nil }, EnterPhoneNumberViewController())
        self.router.addRoute({ $0.verificationCode == nil }, VerifyPhoneViewController())
        self.router.addRoute({ $0.email == nil }, EnterEmailViewController())

        // If all login details are provided, return  `nil` to indicate this flow has
        // no (other) Routable to provide and should be exited
        self.router.addRoute({ _ in }, nil)
    }

    func currentRoutable() -> Routable {
        return self.router.route(for: state)
    }
}

We're then composing flows by adding Routable conformance to each flow and have it provide a view controller, adding its current Routables view controller as a child:

1
2
3
4
5
6
7
extension Flow: Routable {
    var rootViewController: UIViewController {
        let parent = UIViewController()
        parent.addChild(self.currentRoutable().viewController)
        return parent
    }
}

Now a flow can also route to another flow by adding an entry to its router:

1
self.router.addRoute({ $0.needsOnboarding }, OnboardingFlow())

This pattern could let you build entire trees of flows:

Simplified flow diagram

When we first conceptualized flows we imagined having a tree of about 20 flows total; today we have more than 80. Flows have become the "unit of development" of our apps: developers no longer need to care about the full application or a single module, but can build an ad-hoc app with just the flow they're working on.

Plugins

Although flows simplify state management and navigation, the logic of the individual screens within a flow could still be very intertwined. To mitigate that problem, we've introduced plugins. Plugins allow for attaching functionality to a flow without the flow even knowing that the plugin exists.

For example, to add more screens to the OnboardingFlow from above, we can expose a method on it that would call into its router:

1
2
3
4
5
6
7
8
extension OnboardingFlow {
    public func addRoutingPlugin(
        routable: Routable?,
        _ condition: @escaping (OnboardingState) -> Bool)
    {
        self.router.addRoute((condition, routable))
    }
}

Since this method is public, any plugin that imports it can add a new screen. The flow doesn't know anything about this plugin, so the entire dependency tree is inverted with plugins. Instead of a flow depending on all the functionalities of all of its plugins, it provides a simple interface that lets plugins extend this functionality in isolation by having them depend on the flow.

Simplified plugin setup

Since all Lyft apps operate on a tree of flows, the overall dependency graph changes from a tree shape to a "bubble" shape:

Bubble dependency graph

This setup provides feature isolation at the compiler level which makes it much harder to accidentally intertwine features. Each plugin also has its own feature flag, making it very easy to disable a feature if necessary.

In addition to routing plugins, we also provide interfaces to add additional views to any view controller, deep link plugins to deep link to any arbitrary part of the app, list plugins to build lists with custom content, and a few others very unique to Lyft's use cases.

Unidirectional Data Flow

More recently we introduced a redux-like unidirectional data flow (UDF) for screens and views within flows. Flows were optimized for state management within a collection of screens, the UDF brings the same benefits we saw there to individual screens.

A typical redux implementation has state flowing into the UI and actions that modify state coming out of the UI. Influenced by The Composable Architecture, our implementation of redux actions also includes executing side effects to interact with the environment (network, disk, notifications, etc.).

Declarative UI

In 2018, we began building out our Design System. At the time, it was a layer on top of UIKit, often with a slightly modernized API, that would provide UI elements with common defaults like fonts, colors, icons, dimensions, etc.

When Apple introduced SwiftUI in mid-2019, it required a deployment target of iOS 13. At the time, we still supported iOS 10 and even today we still support iOS 12 so we still can't use it.

However, we did write an internal library called DeclarativeUI, which provides the same declarative APIs that SwiftUI brings but leveraging the Design System we had already built. Even better, we've built binding conveniences into both DeclarativeUI and our UDF Store types to make them work together seamlessly:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
import DeclarativeUI
import Unidirectional

final class QuestionView: DeclarativeUI.View {
    private let viewStore: Store<QuestionState>

    init(store: Store<QuestionState>) {
        self.viewStore = store
    }

    var body: DeclarativeUI.View {
        return VStackView(spacing: .three) {
            HeaderView(store: self.store)
            Label(text: viewStore.bind(\.header))
                .textStyle(.titleF1)
                .textAlignment(.center)
                .lineLimit(nil)
                .accessibility(postScreenChanged: viewStore.bind(\.header))
            VStackView(viewStore.bind(\.choices), spacing: .two) { choice in
                TwoChoiceButton(choice: choice).onEvent(
                    .touchUpInside,
                    action: viewStore.send(.choiseSelected(index: choice.index)))
            }
            .hidden(viewStore.bind(\.choices.isEmpty))

            if viewStore.currentState.model.usesButtonToIncrementQuestion {
                NextQuestionButton(store: self.store)
                    .hidden(viewStore.bind(\.choices.isEmpty))
            }
        }
    }
}

Putting it all together

All these technologies combined make for a completely different developer experience now than five years ago. Doing the right thing is easy, doing the wrong thing is difficult. Features are isolated from each other, and even feature components are separated from each other in different modules.

Testing was never easier: unit tests for modules with pure business logic, snapshot tests for UI modules, and for integration tests it takes little effort to sping up a standalone app with just the flow you're interested in.

State is easy to track with debug conveniences built into the architectures, building UI is more enjoyable than it was with plain UIKit, and adding a feature from 1 app into another is often just a matter of attaching the plugin to a second flow without detangling it from all other features on that screen.

It's amazing to look back at where the codebase started some 6 years ago, and where it is now. Who knows where it will be in another 6 years!

Note: If you're interested in hearing more, I also talked about many of these technologies on the Lyft Mobile Podcast!

Re-binding self: the debugger's break(ing) point

Update 07-29-2019: The bug described below is fixed in Xcode 11 so this blog post has become irrelevant. I'm leaving it up for historical purposes.

For the Objective-C veterans in the audience, the strong-self-weak-self dance is a practice mastered early on and one that is used very frequently. There are a lot of different incantations, but the most basic one goes something like this:

1
2
3
4
__weak typeof(self) weakSelf = self;
dispatch_group_async(dispatch_get_main_queue(), ^{
    [weakSelf doSomething];
});

Then, if you needed a strong reference to self again inside the block, you'd change it to this:

1
2
3
4
5
__weak typeof(self) weakSelf = self;
dispatch_group_async(dispatch_get_main_queue(), ^{
    typeof(weakSelf) strongSelf = weakSelf;
    [strongSelf.someOtherObject doSomethingWith:strongSelf];
});

Fortunately, this was much easier on day 1 of Swift when using the [weak self] directive:

1
2
3
4
5
DispatchQueue.main.async { [weak self] in
    if let strongSelf = self {
        strongSelf.someOtherObject.doSomething(with: strongSelf)
    }
}

self is now weak inside the closure, making it an optional. Unwrapping it into strongSelf makes it a non-optional while still avoiding a retain cycle. It doesn't feel very Swifty, but it's not terrible.

More recently, it's become known that Swift supports re-binding self if you wrap it in backticks. That makes for an arguably much nicer syntax:

1
2
3
4
DispatchQueue.main.async { [weak self] in
    guard let `self` = self else { return }
    self.someOtherObject.doSomething(with: self)
}

This was long considered, and confirmed to be, a hack that worked due to a bug in the compiler, but since it worked and there weren't plans to remove it, people (including us at Lyft) started treating it as a feature.

However, there is one big caveat: the debugger is entirely hosed for anything you do in that closure. Ever seen an error like this in your Xcode console?

1
2
3
error: warning: <EXPR>:12:9: warning: initialization of variable '$__lldb_error_result' was never used; consider replacing with assignment to '_' or removing it
    var $__lldb_error_result = __lldb_tmp_error
        ~~~~^~~~~~~~~~~~~~~~~~~~

That's because self was re-bound. This is easy to reproduce: create a new Xcode project and add the following snippet to viewDidLoad():

1
2
3
4
5
6
DispatchQueue.main.async { [weak self] in
    guard let `self` = self else { return }

    let description = self.description
    print(description) // set a breakpoint here
}

When the breakpoint hits, execute (lldb) po description and you'll see the error from above. Note that you're not even using self - merely re-binding it makes the debugger entirely useless inside that scope.

People with way more knowledge of LLDB than I do can explain this in more detail (and have), but the gist is that the debugger doesn't like self's type changing. At the beginning of the closure scope, the debugging context assumes that self's type is Optional, but it is then re-bound to a non-optional, which the debugger doesn't know how to handle. It's actually pretty surprising the compiler supports changing a variable's type at all.

Because of this problem, at Lyft we have decided to eliminate this pattern entirely in our codebases, and instead re-bind self to a variable named this.

If you do continue to use this pattern, note that in a discussion on the Swift forums many people agreed that re-binding self should be supported by the language without the need for backticks. The pull request was merged shortly after and with the release of Swift 4.2 in the fall, you'll be able to use guard let self = self else { return } (at the cost of losing debugging capabilities!)

Using Interface Builder at Lyft

Last week people realized that Xcode 8.3 by default uses storyboards in new projects without a checkbox to turn this off. This of course sparked the Interface Builder vs. programmatic UI discussion again, so I wanted to give some insight in our experience using Interface Builder in building the Lyft app. This is not intended as hard "you should also use Interface Builder" advice, but rather to show that IB can work at a larger scale.

First, some stats about the Lyft app:

With the rewrite of our app we moved to using IB for about 95% of our UI.

The #1 complaint about using Interface Builder for a project with more than 1 developer is that it's impossible to resolve merge conflicts. We never have this problem. Everybody on the team can attest that they have never run into major conflicts they couldn't reasonably resolve.

With that concern out of the way, what about some of the other common criticisms Interface Builder regularly gets?

Improving the workflow

Out of the box, IB has a number of shortcomings that could make working with it more painful than it needs to be. For example, referencing IB objects from code still can only be done with string identifiers. There is also no easy way to embed custom views (designed in IB) in other custom views.

Over time we have improved the workflow for our developers to mitigate some of these shortcomings, either by writing some tools or by writing a little bit of code that can be used project-wide.

storyboarder script

To solve the issue of stringly-typed view controller identifiers, we wrote a script that, just before compiling the app, generates a struct with static properties that exposes all view controllers from the app in a strongly-typed manner. This means that now we can instantiate a view controller in code like this:

1
let viewController = Onboarding.SignUp.instantiate()

Not only is viewController now guaranteed to be there at runtime (if something is wrong in the setup of IB the code won't even compile), but it's also recognized as a SignUpViewController and not a generic UIViewController.

Strongly-typed segues

All our view controllers have a base view controller named ViewController. This base controller implements prepare(for:sender:) like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
open override func prepare(for segue: UIStoryboardSegue, sender: Any?) {
    guard let identifier = segue.identifier else {
        return
    }

    let segueName = identifier.firstLetterUpperString()
    let selector = Selector("prepareFor\(segueName):sender:")
    if self.responds(to: selector) {
        self.perform(selector, with: segue.destination, with: sender)
    }
}

This means that a view controller that has a segue to TermsOfServiceViewController can now do this:

1
2
3
4
@objc
private func prepareForTermsOfService(_ viewController: TermsOfServiceViewController, sender: Any?) {
    viewController.onTermsAccepted = { [weak self] self?.proceed() }
}

We no longer have to implement prepareForSegue and then switch on the segue's identifier or destination controller, but we can implement a separate method for every segue from this view controller instead which makes the code much more readable.

NibView

We wrote a NibView class to make it more convenient to embed custom views in other views from IB. We marked this class with @IBDesignable so that it knows to render itself in IB. All we have to do is drag out a regular UIView from the object library and change its class. If there is a XIB with the same name as the class, NibView will automatically instantiate it and render it in the canvas at design time and on screen at runtime.

Every standalone view we design in IB (which effectively means every view in our app) inherits from NibView so we can have an "unlimited" number of nested views show up and see the final result.

Basic @IBDesignables

Since a lot of our views have corner radii and borders, we have created this UIView extension:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
public extension UIView {
    @IBInspectable public var cornerRadius: CGFloat {
        get { return self.layer.cornerRadius }
        set { self.layer.cornerRadius = newValue }
    }

    @IBInspectable public var borderWidth: CGFloat {
        get { return self.layer.borderWidth }
        set { self.layer.borderWidth = newValue }
    }

    @IBInspectable public var borderColor: UIColor {
        get { return UIColor(cgColor: self.layer.borderColor!) }
        set { self.layer.borderColor = newValue.cgColor }
    }
}

This lets us easily set these properties on any view (including the ones from UIKit) from Interface Builder.

Linter

We wrote a linter to make sure views are not misplaced, have accessibility labels, trait variations are disabled (since we only officially support portrait mode on iPhone), etc.

ibunfuck

A bug impacting developers that use Interface Builder on both Retina and non-Retina screens (which at Lyft is every developer) has caused us enough grief to write ibunfuck - a tool to remove unwanted changes from IB files.

Color palette

We created a custom color palette with the commonly used colors in our app so it's easy to select these colors when building a new UI. The color names in the palette follow the same names designers use when they give us new designs, so it's easy to refer to and use without having to copy RGB or hex values.

Our approach

In addition to these tools and project-level improvements, we have a number of "rules" around our use of IB to keep things sane:

Of course, even with these improvements everything is not peaches and cream. There are definitely still problems. New versions of Xcode often change the XML representation which leads to a noisy diff. Some properties can simply not be set in IB meaning we're forced to break our "do everything in IB" rule. Interface Builder has bugs we can't always work around.

However, with our improved infrastructure and the points from above, we are happy with how IB works for us. We don't have to write tons of Auto Layout code (which would be incredibly painful due to the nature of our UIs), get a visual representation of how a view looks without having to run the app after every minor change, and maybe one day we can get our designers make changes to our UI without developers' help.

Silencing NSLog

When your app has a lot of third-party dependencies, what often happens is that those libraries log a bunch of things to the Xcode console to help their own debugging. Unfortunately, a lot of these logs are useful only to the developers of the library, but not the developers of apps that integrate the library. For example, they log things like <SomeLibrary> (version 1.2.3) initialized, or <SomeLibrary> started <primary functionality>, sometimes with a long list of parameters or input sources that are irrelevant to you.

Finding your own log statements in a jungle of other logs can then be very difficult and adds to the frustration of not being able to work the debugger as you would like to.

If a library is open source you can suggest a change by removing the log or otherwise make it less obtrusive. However, if your change gets accepted at all, that doesn't solve the immediate problem of being able to debug your own code using the console.

Meet _NSSetLogCStringFunction(). This C function has been around in Foundation for a long time, and while there is some documentation on it, it's still a private method. However, that doesn't mean you can't use it in debug mode when your logs are the most valuable!

In short, this function lets you set a function pointer that can log C strings, which NSLog then uses instead of the normal implementation. You can do this in two ways.

The first one is by adding this to your Objective-C bridging header:

1
2
3
#if DEBUG
extern void _NSSetLogCStringFunction(void(*)(const char*, unsigned, BOOL));
#endif

and then use it like this:

1
2
3
4
5
6
func disableNSLog() {
#if DEBUG
    _NSSetLogCStringFunction { message, _, _ in
        // no actual logging, message just gets lost
    }
#endif

If you want to stick to pure Swift, you can do so by adding this to your code somewhere:

1
2
@_silgen_name("_NSSetLogCStringFunction")
func setNSLogFunction(_: @convention(c) (UnsafePointer<Int8>, UInt32, ObjCBool) -> Void)

and then use it like this:

1
2
3
4
5
6
7
func disableNSLog() {
#if DEBUG
    setNSLogFunction { message, _, _ in
        // no actual logging, message just gets lost
    }
#endif
}

Obviously, you can do anything you want inside the closure, including writing to a file, annotating the message with a date/time, passing it to your custom logging library, etc.

One downside of this is that Apple's frameworks use NSLog extensively as well, so in the above case of completely disabling logging, helpful messages get lost as well. You won't be able to use NSLog yourself either anymore, so I suggest you use print() or a custom logging framework that's not NSLog based.

If you're not afraid of doing (more) horrible things in your codebase, you can avoid losing Apple frameworks' messages by parsing the stack trace and looking at the framework that called this function and see if it's something you want to let through:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
func disableNSLog() {
#if DEBUG
    _NSSetLogCStringFunction { message, _, _ in
        // message is of type UnsafePointer<Int8> so first see if we can get a
        // normal String from that. Safety first!

        guard let message = String.fromCString(message) else {
            return
        }

        let callStack = NSThread.callStackSymbols()
        let sourceString = callStack[6]
        let separatorSet = NSCharacterSet(charactersInString: " -[]+?.,")
        let stackFrame = sourceString.componentsSeparatedByCharactersInSet(separatorSet)
        let frameworkName = stackFrame[3]

        if frameworkName == "UIKit" || frameworkName == "Foundation" {
            MyCustomLogger.log(message)
        }
    }
#endif
}

This discards all logs, except if they're coming from UIKit or Foundation. The stack trace parsing is by no means safe (its format could change, for example), but since it's wrapped in #if DEBUG directives it won't mess with anything in the App Store build.

Note that static libraries are part of your main app's target, which means you have to filter out logs from your own target to hide those.

You could even go a bit farther and check the message for keywords you like or don't like and make a decision on whether you want to log or not. Keep in mind, though, that any work you do here needs to be fast as you don't always know just how much is being logged.

Outlets: strong! or weak?

There are a lot of styles out there when it comes to using Interface Builder outlets in Swift. Even Apple's documentation and sample code isn't always consistent. The most common one, the one Apple uses in its sample code, follows this pattern:

@IBOutlet private weak var someLabel: UILabel!

Let's break this down by keyword:

While at first this seems like a solid approach, at Lyft we quickly realized we weren't fans of this one-size-fits-all way of defining outlets. Instead, the behavior and consequences of the different elements should define the outlet's exact syntax, just like any other variable.

For example, if there is a code path that removes an outlet from its superview, or the outlet is (intentionally) not hooked up in the storyboard, it needs to be an optional because the outlet is not guaranteed to be there when it's accessed.

@IBOutlet private var someLabel: UILabel?

If there is no code path that re-adds the outlet to the view hierarchy, it would also be good to make it weak to not hold on to it unnecessarily when it gets removed:

@IBOutlet private weak var someLabel: UILabel?

This ensures that if the label is removed from the superview, it's not being kept in memory by the strong reference in the view controller. In the most common case, where there is an outlet that will always be there, a strong, implicitly unwrapped optional is appropriate:

@IBOutlet private var someLabel: UILabel!

The outlet isn't weak in case the code ever changes so that there is a code path that removes the view from the view hierarchy but you forget to update the optionality of the property. The object will stay in memory and using it won't crash your app.

These examples all follow 3 simple rules:

  1. ! needs a guarantee that the view exists, so always use strong to provide that guarantee
  2. If it's possible the view isn't part of the view hierarchy, use ? and appropriate optional-handling (optional binding/chaining) for safety
  3. If you don't need a view anymore after removing it from the view hierarchy, use weak so it gets removed from memory.

Applying these three rules means you properly use the optional semantics. After all, using ! for a view that may not exist is no different than defining any other property as an implicitly unwrapped optional that may not exist.