Unsafe Swift: For Fun & Profit

One of the main stated goals of Swift is safety. You can see it reflected in the design of String, or the use of Optionals. For better or worse we still live in a C-based world, and there are times when you simply cannot survive without pointer arithmetic. With that world in mind, Russ Bishop shows us how to subvert the safety of Swift, for pleasure, and survival.

You can read an edited transcript below.

---

Why Safety? (1:19)

Swift adopts safe programming patterns.
- Apple

Why bother with safety? In many cases, we’re talking about memory safety. There’s a long list of mistakes that have been made. Sir Charles Antony Richard Hoare says that null reference pointers were his billion dollar mistake, and they were also involved in major bugs like Heartbleed, Sasser, and Code Red. In some cases, bugs like these have caused many billions of dollars worth of damage.

We can also consider productivity. There’s an experience that I call the “isa oops”, where your .ZIP file vended a different class than the one you declared in the code. You labor on with random errors and unrecognized selectors until you realize that it was not the class you thought it was. Dealing with undefined behavior is another reason, for which retainCount was removed.

Bridging (3:01)

A lot of existing code bases, as well as basic OS atomic functions, are written in C. Why not just write wrappers in Objective-C and call that from Swift code? This happened before with C#, and in theory, it shouldn’t matter. In theory, we can call a C library and it should be the same as using an unsafe block. But in reality, it really does matter. In this instance, doing the code in C# vastly limits the scope of potentially memory-unsafe operations. When there are three functions in the whole project to audit, everybody gathers, looks at the PRs, and contributes. The odds of having a problem are much lower. But when there are three hundred files to audit, people will say they don’t have time, and move on. Swift also has the benefit that definitely safe code is very distinct from potentially unsafe code. By moving as much as possible into Swift, and using the unsafe functionality that Swift provides, you have a very clear distinction between the parts of your code that are provably memory safe, and the parts that you need to pay much closer attention to. This makes operations more explicit and easier to reason about. If you’re lucky, you see a warning when you make an error. If you’re unlucky, in C or Objective-C you overwrite memory because the compiler doesn’t care and doesn’t give you errors or warnings. That’s an important difference.

The Toolbox: Safety Scissors

Strings (5:57)

Swift gives us things like strings, which are basic and used by everyone.

static func fromCString(cs: UnsafePointer<CChar>) -> String?

func withCString<Result>(@noescape f:UnsafePointer<Int8> -> Result) -> Result

In the function fromCString, so long as the pointer cs is NULL-terminated, things are perfectly safe. Another function withCString takes a closure, but why doesn’t it just return the pointer?

Lifetimes & Ownership (6:37)

Most of this talk will be about lifetimes and ownership: who owns the object, and for how long does it stay alive? Even though enums and structs are value types, Swift uses ARC for some enums and structs. They’re not reference counted. You can include strong object references in them, which is something ARC explicitly disallows in Objective-C. A function expecting a C string needs the storage to be contiguous, which is something that is not necessarily guaranteed by string or array. A C function also needs that storage to be alive, but pointers don’t say anything about lifetime. Thus, functions like withCString promise that the pointer will be contiguous and alive for the duration of the closure. This is why it is restricted to that scope, as well as declared as @noescape. If the closure outlived the lifetime of the call, the pointer could become a dangling pointer if the string were mutated.

Arrays look like structs and have value semantics, but in reality the structs are just headers, and the item storage is on the heap. They’re shared when they can be, and copied when they’re mutated. Swift hides things like that from us, so we can pretend that they’re always value types. The reality is that Swift won’t copy unless it absolutely has to. That raises the question, does adding two arrays copy them? Or does Swift just use a linked list of slices? Again, that doesn’t matter to us because it’s invisible.

Unmanaged (9:26)

We’ll start looking at actual types and functions that Swift provides us with. Unmanaged is a very useful one that is only for reference types, things that inherit NSObject or Swift objects. It also lets us get into opaque pointers and COpaquePointers, which I will discuss later in this talk. The more important parts are again around managing lifetime. Unmanaged is essentially giving us a way to manipulate the reference count of an object on our own. Thus passRetained and passUnretained allow us to take an object and turn it into an Unmanaged reference. takeRetainedValue and takeUnretainedValue then allow us to get objects back out of an Unmanaged reference. Then retain, release, and autorelease are a nice way to relive the glory days of Manual Reference Counting.

let o = OhMy()

//Unbalanced retain +1
let context = Unmanaged.passRetained(o).toOpaque()

//UnsafeMutablePointer<Void> won't manage any memory
let contextPointer = UnsafeMutablePointer<Void>(context)

dispatch_queue_set_specific(queue,
    &Keys.SomeKey,
    contextPointer,
    nil)

In the class OhMy, we create an unbalanced retain in the second line of code. That retains the object, but when this context goes out of scope, Swift is not going to automatically release the reference. It’s completely unbalanced, and we can turn this into an UnsafeMutablePointer. Lastly, we can use this to call various APIs like dispatch_queue_set_specific. If we just pass the object in, we can end up with a dangling pointer, because the object would be reallocated when it went out of scope. Unfortunately, Swift doesn’t have support for C function pointers now, so you can receive them and pass them along but you cannot provide or call them.

UnsafePointer (12:36)

In many cases, Swift will automatically convert for you, so when in doubt, try passing with the address of the operator. Arrays, ints, and structs all work. If for some reason you have a need to cast a pointer, you just pass an UnsafePointer to another one in the initializer, and Swift will cast that for you. How do we use UnsafePointers? Pointers in Swift are in one of three states. They can be unallocated, which means the pointer is null, or allocated, which means the pointer points to some memory that’s been malloced, but the location is uninitialized. When initialized, they reach the third state where the memory it’s pointing at actually has a valid value.

let nullPtr = UnsafeMutablePointer<Int>(nil) // nil is the null pointer
println(nullPtr.memory) // EXC_BAD_ACCESS

let ptr = UnsafeMutablePointer<Int>.alloc(1)
ptr.initialize(5)
ptr.destroy() // destination uninitialized, memory allocated
ptr.dealloc(1) // equivalent to free()

In this example, we have a pointer to an int that is initialized to null. If we try to access it, we’ll get a crash of BAD_ACCESS. Swift will not protect you from that. You are entirely on your own when you use the UnsafePointer classes. .alloc(1) is the equivalent to malloc(sizeof(Int)). The number 1 is the number of instances of that type that you want storage space for. The memory has been allocated with malloc, but has garbage in it. After we initialize it, the destination of the pointer has a valid value. If we call destroy to destroy the object that the pointer is pointing at, the memory has still been allocated. Then we can call dealloc, or the equivalent of free. If you think about this from the C point of view, this is the equivalent of malloc and assigning a value, then deleting what the pointer is pointing at and freeing the memory itself.

Back to Lifetimes & Ownership (15:50)

Again, why do we care? Returning to ARC, lifetimes, and ownership, Swift respects the types even inside of an UnsafePointer. UnsafePointer<T> and UnsafeMutablePointer<T> participate in ARC lifetime rules if T supports it. It’s as if pointers had __strong attributes by default. Unlike in Objective-C, where the compiler will complain if you declare a strong pointer to something that’s not a valid object pointer, Swift handles it automatically. This also means that assigning to the memory property releases any existing object and retains the new one, including nested references.

In the following example, we have a class OhMy and a struct that is a value type. Any time the struct is assigned, I have a copy of it, but through compiler magic, that reference is handled. When I create a pointer, I allocate, initialize, and pass it to the struct. Inside of the struct, I have a reference to an object. If that’s all I do, then deinit will never run because I have leaked an object. That is what destroy is for - you have a pointer to an int and you call destroy. Swift is smart enough to understand that the struct itself contains an object reference. When we call destroy, it will dig into the struct and release the object that we’re pointing to. Of course, we can’t forget dealloc, or we will leak memory. At run time, UnsafeMutablePointer has no storage. It has zero run time cost, so we have to remember to tell dealloc the size, because it has nowhere to store the size that we allocated. The difference between initializing an object and assigning it to the memory property is that the UnsafeMutablePointer doesn’t know that the memory is still uninitialized.
```swift class OhMy { deinit { println(“I got deinit’d!”) } } struct Fancy { var objectRef:AnyObject? }

let ptr = UnsafeMutablePointer.alloc(1) ptr.initialize(Fancy(objectRef: OhMy())) //Oops, deinit never runs… ever

ptr.destroy() //deinit runs here! ptr.dealloc(1) //don’t leak memory ```

You may also notice that with UnsafePointer, Swift treats them similarly to C. If I pass a pointer into a C function, then I can use brackets to index into it, and C will now recognize that you’re indexing five in, multiply its size by five, then jump to that point in memory. You can pretend that they’re an array by subscripting them, but only you know if it’s actually just a single item, or an array of ten thousand elements. Swift will not help you there, and you will run right off into the end of memory.

A few more quick things about UnsafePointer and UnsafeMutablePointer. Again, it’s all about ownership and lifetimes. move() transfers ownership of the current pointer. It takes the value that the pointer is pointing at, moves it out as the object, and calls release on it so that pointer is no longer holding onto it. The source pointer become uninitialized in the process. There’s also moveInitializeFrom that can transfer ownership from one pointer to another pointer. Again, if the destination pointer is not empty, you could leak. moveAssignFrom does the same, but it destroys any objects that already exist in the current pointer. We can do pointer arithmetic, or similarly to C, just array subscript.

AutoreleasingUnsafeMutablePointer (21:59)

Another type that is somewhat related is AutoreleasingUnsafeMutablePointer. Anything that takes NSError are all declared as this. Nil turns into a NULL pointer automatically, so you can just call with a nil as the parameter, and Swift will automatically make that its type. This is true for regular pointers as well. UnsafeMutablePointer’s are passed as-is, but object references are different. In this case, a temporary copy of the reference is allocated, and on return, the reference is copied back into myObject and retained. We usually don’t have to deal with these because Swift is usually good about automatically taking care of them, but I just wanted you to be aware that they exist.

To show you an example of something real, here is a DNS query. We declare the struct addrinfo and an UnsafeMutablePointer. It’s not allocated because we don’t need to. The last parameter to getaddrinfo is a pointer to a pointer, so we can pass in the address of the pointer and Swift will automatically handle it. The API being used says that you have to allocate a buffer to fill in, so this is how we would allocate a buffer to pass some C code. Then we create an array, give it the count, and initialize to null. We pass in the buffer, use the fromCString method, print, and then free it. The important part is that most of this is very similar to what you would write in C. ```swift let host = “www.apple.com” let port = “80”

var hints = addrinfo() hints.ai_family = PF_UNSPEC hints.ai_socktype = SOCK_STREAM hints.ai_protocol = IPPROTO_TCP

var firstResponse = UnsafeMutablePointer() if getaddrinfo(host, port, &hints, &firstResponse) == 0 { for var response = firstResponse; response != nil; response = response.memory.ai_next { if response.memory.ai_family == AF_INET || response.memory.ai_family == AF_INET6 { var buffer = [CChar](count:Int(INET6_ADDRSTRLEN), repeatedValue: 0) if inet_ntop(response.memory.ai_family, response.memory.ai_addr, &buffer, socklen_t(buffer.count)) != nil { let ipString = String.fromCString(buffer) println("\(ipString)") } } } freeaddrinfo(firstResponse) } ```

ManagedBuffer / ManagedBufferPointer (25:52)

These two things are fairly easy to use and not too dangerous. If you want to implement your own version of an array or another data structure while having it behave the exact same way that Swift structures do, this is where you start. They allow you to define your own copy-on-write struct. Struct internally references a ManagedBuffer. If it’s unique, we can mutate in place, otherwise we make a copy. This following snippet checks that the struct is uniquely referenced and copies only if necessary. That’s the exact same way that Swift arrays, dictionaries, and sets behave.

mutating set {
    if !isUniquelyReferenced(&_buffer) {
        _buffer = _buffer.copy()
    }
    _buffer[index] = newValue
}

Structs don’t have the initializers, so there isn’t a way to hook in and free the elements it was holding; that’s why we need this buffer class behind the scenes. In the array buffer, we can first see in deinit how to free our memory. We clean up the objects that this instance is holding, and then we have the subscript in the buffer that just accesses the pointer. This is a working implementation of the array, but it doesn’t do anything interesting, or anything more than Swift already does.
```swift deinit { withUnsafeMutablePointers { vptr, eptr -> Void in eptr.destroy(self.count) vptr.destroy() } }

subscript(index:Int) -> Element { get { return withUnsafeMutablePointerToElements { $0.advancedBy(index).memory } } set { withUnsafeMutablePointerToElements { $0.advancedBy(index).memory = newValue } } } ```

The Toolbox: Handle with Caution

COpaquePointer & Variadic Arguments (29:38)

There isn’t much to say about COpaquePointer, it’s just a bag-o-bits. However, it is useful for shoe-horning an UnmanagedPointer into an UnsafePointer, and vice-versa if you really need to. If you need to use variadic arguments, you can call C functions that take va_list. The life of CVaListPointer is tied to the closure, and the elements must implement the CVarArgType. Then there is getVaList, but I haven’t run into a use case for that yet.

In the following code, a C function takes the va_list, which takes count. Following that, there is code for how we would handle it in Swift. We tell Swift that the struct can be passed as a VarArgType, and then implement the encode method. In this case, I am mashing together the representations because this is what the platform ABI rules say to do. I can then go ahead, create the structs, and pass them to withVaList. The p that I get back will be the CVarArg pointer that I can then pass to my C function, and to the variadic arguments. I don’t recommend necessarily adopting the CVarArgType, as you would have to pay attention to the ABI rules.

// C code
typedef struct some_struct {
    NSInteger a;
    NSInteger b;
    NSInteger c;
} some_struct_t;
void valistFunction(int count, va_list args);

// Swift Code:
extension some_struct_t : CVarArgType {
    public func encode() -> [Word] {
        return self.a.encode() + self.b.encode() + self.c.encode()
    }
}

let s1 = some_struct_t(a: 42, b: 42, c: 42)
let s2 = some_struct_t(a: 99, b: 99, c: 99)

withVaList([s1, s2]) { p -> Void in
    valistFunction(2, p)
}

unsafeUnwrap / unsafeDowncast (31:40)

Other functions that exist include unsafeUnwrap and unsafeDowncast. In debug builds, unsafeUnwrap works exactly like !, the implicitly unwrapped optional operator. But in release builds, it doesn’t bother checking for nil, so if you’re wrong, prepare to see crashes. Profile first, and if this is a performance problem, it’s a tight loop somewhere; if it’s really an issue, then use it. For unsafeDowncast, it works in debug builds exactly like an as. In release builds, it doesn’t check the type, so it’s the wrong isa all over again. You can have a completely different class at runtime, and this type will not care. In Objective-C you may get an unrecognized selector or other weird behavior; in Swift, you might get unrecognized behavior or even memory corruption. Again, profile first, and then take a look at it.

The Toolbox: Handle with Extreme Caution (32:53)

For the following, use with extreme caution. The use of any of these in production should be an automatic red flag, and a trigger for the most stringent code review possible.

unsafeBitCast is brutal, seriously! It blindly treats the bits of its argument as whatever type you say it is. The beta documentation actually described it as “brutal”; it really does not care. Though it may be extremely dangerous, it can be lifesaving when you need it, so please use unsafeBitCast extremely carefully!

Another type in this category is unsafeAddressOf. It gets the address of whatever storage is holding that type, and has absolutely no guarantees about lifetime or anything else. However, you can accomplish just about anything with UnsafeMutablePointer and Unmanaged, so most of the time there isn’t a use for this. Personally, I haven’t found a legitimate use for this, but if you want to get the address and start looking around in memory, you can use this.

The Toolbox: Here there be dragons (34:20)

I would not ship this in a production application under any circumstances. But if you’re feeling adventurous, here they are.

@asmname is an attribute to decorate a function. You definitely need to understand the ABI to use this one. Swift will not help you marshall the parameters very much. The compiler is not going to be very forgiving, so you have to make sure that you’ve manipulated your parameters into a format it’s compatible with. The following code shows how to declare it. Give the function attribute, @asmname, and the string symbol; then function its arguments in return type. The compiler will not complain if you get the arguments wrong or return type incorrectly. It only expects that that symbol exists, and when it jumps to it, it better take those arguments and have that return type.
c @asmname("dispatch_get_current_queue") func _get_current_queue() -> dispatch_queue_t

Tips (35:43)

This last note is just one tip that I keep running into, and it’s something that keeps bothering me. Lots of APIs want a key or to void *. In Swift, just declare a static variable initialized to a unique string and then pass its address. That will work as long as it’s declared a static.

Thank you!

Resources (36:51)

• Apple’s Swift blog
• Mike Ash’s blog
• NSHipster
• objc.io
• Russ Bishop’s blog

Q&A (37:08)

Q: unsafeBitCast actually shows up the most in my code, mainly with enums since APIs don’t expose enums properly. Is there a better way to deal with enums without touching that?
Russ: This is a really frustrating one. A lot of the time I find that the parameter is declared as an unsigned int, and the enum is declared as a signed int. If you just take the enum and pass it to the unsigned int constructor, then that will pass it through. But it still checks that the value is within range. Going from a signed int to an unsigned int is not so much of an issue, but the reverse direction I’ve also found on other APIs. The other thing is that the type aliases can be used as the constructor as well.

Q: Obviously not all of these are documented by Apple, so what’s your process of discovering all these behaviors?
Russ: I’ll look at the documentation first, but you’re right that there is a lot of stuff that isn’t in there. If you go to the Swift REPL and use the flag — I think it’s something like -deprecated-integrated-repl — you can ask it to print the Swift module and all the bits that Xcode doesn’t show you. If you dig into the toolchain directory Xcode, you can also find stuff in libswiftCore and libswiftRuntime.
JP: If you’re interested in doing some more unsafe things with Swift, you can do a code search in GitHub for @asmname and language “Swift”, and you’ll see a bunch of really bad but interesting stuff.

Q: What would you like to see from Apple moving forward, or even hope to see removed if anything?
Russ: For me, I don’t want anything removed. I actually wish they would bring back __conversion. I found a few cases where it’s legitimate, and it’s really annoying to have to keep casting things all over the place. But I understand the arguments against it and why they took it away, it would be really easy for people to just start using it everywhere, which wouldn’t be good.

---