Caramel for Swift

Swift is going open source, opening up potential to build apps for servers and embedded devices. But the Swift standard library is surprisingly limited, with no support for common tasks like dealing with files or network connections. Caramel is a project in development that brings powerful, expressive I/O to Swift, and will enable a whole new generation of apps for Swift on any platform. In this talk, Steve Streza introduces the framework and its core concepts, talks about getting it to production-readiness, and asks for your help to build an inclusive community around cross-platform Swift.

---

Introduction (0:00)

My name is Steve Streza, and I’m an iOS developer at Tumblr. For a couple months, I’ve been working on a project called Caramel, an I/O framework for pure Swift applications running on servers and embedded devices. Before I delve into talking about it, I should mention something about Swift.

Swift is open-source now. We can bring it to other platforms and be just as productive with Swift on Linux as we are on iOS and Mac. However, to do that, we need to look at what’s actually being open-sourced: the standard library. If you look at the standard library in Swift, you realize that it’s actually kind of small, excluding the new foundation stuff. Mostly, it’s pure types that can be purely represented within the run times, meaning things like string, int, array, etc. There is no built-in I/O. To find stuff like you would get from POSIX, you would have to look in the Darwin module, which isn’t a part of the official API. There is a glibc module that has a lot of that stuff, but the downside is that those APIs generally aren’t very Swifty. You have unsafe mutable pointers all over the place, and figuring out how to make those work is not fun.

In short, we need a new foundation (pardon the pun) upon which to build applications, one that has first class support for Swift features, like strong types and error handling. We need to isolate the unsafety of I/O-based operations, and we need a modern, clean, and composable API design. Putting it all together, Swift needs a really strong I/O framework, and that’s what I’ve been trying to do with Caramel.

Design Principles of Caramel (2:20)

First and foremost, Caramel is a framework designed to be very Swift-friendly. It has typesafety, so that you don’t have to deal with things like type unsafe, unsafe mutable pointers, and all that stuff. You get strong types throughout all the APIs, and you get what you expect when you expect it. It should be simple to use, but it should also have of extensibility. It should be able to take an object that the framework gives you and extend it and compose it with other objects. All in all, it should embrace the ideas that make swift great and lovable.

On top of that, we can have a really great developer experience compared to other server platforms when we build our applications in Xcode. It’s a great IDE on day one for building server apps. That includes a full-featured debugger and instrumentation, which you would never see from a platform like Ruby, Node.js or Scala. Those languages have evolved to include those features, but you don’t have them on the first day. Also with Swift (via Xcode), we have LLDB, Instruments, etc. We have full autocomplete and type inference (and it works!) and that means that we don’t have to deal with things like not knowing what kind of types we’re dealing with. The framework and the developer tools give you all that and it works great.

Bedrock (4:02)

My mission statement for what I’m trying to do with Caramel is this: Caramel wants to be the friendliest and most productive platform for building applications for servers and embedded devices.

That is a pretty lofty goal, but fortunately, we have a bedrock upon which to build. Years of development have been built into cross-platform C libraries which we can use. The most important of these is a library called “libuv”. This is the core of Node.js, and it provides asynchronous I/O for dealing with files, network connections, DNS requests, HTTP, all that kind of stuff.

On top of that, there’s http_parser, an HTTP parsing library written in C for Node.js. Then, for cryptography, we’ve got OpenSSL or LibreSSL for doing encryption, HMAC, and hasing. The good news about all this stuff, if you’ve seen and have tried to use this before, is that you don’t have to care that any of it’s there. You’ll never ever have to go down to this level unless you absolutely want to.

There are Swift APIs in Caramel for doing all this stuff. We’re not going to expose unsafe APIs unless we absolutely have to. If that’s the case, then we’re probably designing it wrong and we should look at what I’m doing here. The other benefit that you get with these frameworks is that they’re cross-platform for free; they’re designed to run not just on Macs, but on Linux machines, FreeBSD, weird platforms like Solaris, and even platforms like Windows.

With that said, there are countless years of development and testing that have gone into the very foundational levels of the framework and that’s what Caramel’s building on top of. The role of Caramel is to encapsulate the inherent unsafeness of I/O and do the best we can to minimize failures and make what we expose as API as safe and robust as possible.

Data (6:07)

Enough talk of ideas, here’s a little bit of code:

var data = Data()
data.append([0xDE, 0xAD, 0xBE, 0xEF]) 
print("Data: \(data)") // Data: DEADBEEF

The foundational element behind dealing with I/O is dealing with binary data. We’re doing that with an array of bytes. Here, bytes are just typealias to UInt8. This means we have a typesafe value type for dealing with binary data, and it has a simple API which you can interact with as you would with arrays; you can concat arrays into it, you can append, you can access things by byte, and it works great. However, because it’s a Swift array, it’s not an unsafe, so you can use Swift semantics for it; being a var type, this means you can mutate it, whereas if it was a let type, you couldn’t. It comes with the safety you expect from Swift operations.

Data objects can interoperate well with the built-in Swift strings and Unicode.

let string = "Hello World" 

let utf8Data: Data = string.utf8.data // utf8 provided by Swift 

let utf16Data: Data = string.utf16.data // same with utfl6 

do { 
    let utf8String: String = try utf8Data.UTF8String
    let utf16String: String = try utf16Data.UTF16String
    guard string == utf8String, string == utfl6String else { 
        fatalError("This won't happen!")
    }
} catch let error { 
    print("Couldn't create string from data: \(error)") 
}

We have an extension on the string UTF-8 view and UTF-16 view to basically get data values out of those, and similarly, we can take those UTF data and UTF-16 data and convert them back to strings. That’s an error-prone operation, so we have to wrap it with Swift’s error-handling try catch semantics.

We have primitives for dealing with files and directory paths. There’s the value type, there’s convenience constructors for getting to the root directory and the home directory. Similar to NSString and NSURL, you can use fileByAppendingPathComponent or pathExtension, get the path components, and get the raw path if you’d like. We try to condense this behind a cross-platform API, so this will still work on Windows for example, even though Windows uses “" instead of “/”. I didn’t have to write it, libuv provides this for us.

File (8:22)

let etc = File.rootDirectory/"etc"
let hostsFile = etc.fileByAppendingPathComponent("hosts")

etc.isDirectory // true

hostsFile.isFile // true
hostsFile.isDirectory // false

hostsFile.path // "/etc/hosts"
hostsFile.pathComponents // ["/", "etc", "hosts"]

Files obviously work great with reading and writing data, so you can try to call hostsFile.data and it will return data to you if it can. Then, because it’s data, we have some extensions for doing fun things like SHA512. If we get that into a data structure, we can turn that into a string then create a new file later on that has a new path extension and write that out to a file. Because these are all error prone again, we’re using Swift error-handling to try catch and report errors as they come.

do {
    let hostsData = try hostsFile.data()
    let encodedData = hostsData.SHA512
    let encodedString = encodedData.hexString
    print( "Hosts file,  SHA512:  \(encodedString)")

    let sha512File = hostsFile.fileByAppendingPathExtension("sha512")
    try sha512File.createWithData(encodedString.utf8)
}  catch let error {
    print( "Read/write failed:  \(error)")
}

This works, but it may not be the most optimal way of writing code in Caramel. Let’s take a bit of a closer look at this statement:

try hostsFile.data().SHA512.hexString

So, hostsFile.data takes that data, converts it through SHA512, and then converts that into a string.

This isn’t the most efficient because I/O can sometimes delay for weird reasons. Sometimes when you’re reading data, you don’t know where the data’s coming from. If it’s coming from disk or from the network, there will be these punctuated delays throughout your read where you’re reading chunks of data, but it may not all coming in at the exact same time. You end up with weird gaps.

Ideally, during this kind of operation, you want to minimize how much time you’re spending and how much memory you’re allocating by basically running each part of the process through, like a waterfall, where it’s reading a little bit of data, doing the SHA512, reading a little bit more data, doing the SHA512, etc. This is good for doing data that is the size of a hosts file, and it’s also going to work great at the scale of a 200GB file.

However, what actually happens is that all of those operations get bunched up towards the end. You have to wait for the last thing to load from your read before the very first part of your SHA512 operation can happen. If you’re allocating memory for all of this stuff, you could be allocating tons of memory, which could create virtual memory thrashing and who knows what else. It’s not really an optimal solution.

Extrapolating Design Patterns (10:49)

What’s actually happening here? Well, we’re going to read the data from disk. In order to do that, we have to open a file descriptor, which is a little bit of a setup process. Then, we have to loop over that file and read the data in chunks (either 0, 1, or more than 1 times). When we’re done, we need to do some clean up (close the file descriptor), and then deal with errors in the appropriate way.

From this process, we can extrapolate a design pattern: setup, process, clean up, and handle errors. We can apply this kind of four-step process to other types of operations that you might do with I/O. Obviously, other file system operations map really cleanly to this: Writing the file, listing directories, etc. But this also is applicable to things like TCP connections, DNS requests, HTTP requests, timers. If you needed to have a timer on a loop, you’ve got a sequence that’s going over and over again. If the OS needs to send you a signal, you will get a signal handling pipeline that goes along with that. It also works if you need to process audio, or convert it into other types of audio. It’s also good for hardware devices, like signals coming in from USB. On and on and on, this pattern is very applicable.

Let’s take another look at the next part of that, where we take that data and process it through a SHA512 algorithm, then process that through a hexstring algorithm.

let encodedData = data.SHA512
let encodedString = encodedData.hexString

Again, we have to set up some initial state. Then we have to take an input and transform it to something else, and we do that over the loop of every piece of the operation we get. We need to finalize the output and hand it off, and just as before, we have to deal with errors that may come up during that process. In short: setup, transform input, finalize output, handle errors.

Here is another design pattern we can apply when it comes to things like string encoding and decoding. Parsing files through XML, JSON, and other kinds of things, like serializing model object to and from binary data types, encryption, decryption, hashing, and HMAC-ing. We can also look at things like HTTP routing as a means of transforming from one HTTP request state to another. There’s middleware that goes into all kinds of applications, at HTTP and other kinds of levels. It’s also gret for audio, video, and image processing, if you need to convert stuff.

Streams (13:25)

With that in mind, I want to present the core concept of what we’re trying to do in Caramel. It is a concept that comes from the world of functional reactive programming: streams.

In Caramel, streams are strongly typed one-way sequences of “things”. Things can be whatever you want. They’re results delivered over time. Some operations may be all at once, but a lot of them are not going to be. They can be asynchronously delivered to you at different times. If you throw an error in one of these streams, we’re going to catch the results and deliver them to you, no matter which step of the process they come in.

There is a synchronous version, which I call “pullable”, and an asynchronous version, which I call “pushable”, and those are used for different things. Many of the framework’s APIs provide streams and, in most cases, streams are underlying all of the convenience APIs, so that file.data function we looked at earlier is actually just creating a stream under the hood and pulling all of the data through it.

Crucially, streams can be transformed into other streams and this is a very powerful concept.There are plenty of convenience APIs that are provided by extensions, and you can also build your own streams that do whatever you needed them to.

Before, we were looking at code that looks like this:

do {
    let data: Data = try hostsFile.data()
    print("Data: \(data)")
} catch let error {
    print("Couldn't read: \(error)")
}

With a synchronous stream, this is now just a little bit longer:

do {
    let data: Data = try hostsFile.readPullStream.drain()
    print("Data: \(data)")
} catch let error {
    print("Couldn't read: \(error)")
}

This .drain just says, “Give me everything once the stream has closed,” but there is a looping variant you could easily use as well. What’s more interesting is the asynchronous version of this:

hostsFile.readPushStream.drain { (result: Result<Data>) in
    do {
        let data = try result.result()
        print("Data: \(data)")
    } catch let error {
        print("Couldn't read: \(error)")
    }
}

Here, we actually are draining asynchronously into a result object that contains either the error or the result, we’ve moved that logic of capturing errors into this block.

Transformers (15:42)

Transformers are the other really powerful concept in Caramel. Transformers turn things into other things of the same type or of another type. Transformers are not directly streams, but they get used by streams. That means you can take all the logic of a transformer, put it into a “thing”, and then either use that from a synchronous or an asynchronous variant by writing the code once. This means you can write a logic once and get streams anywhere you need them to be.

For every “thing” that comes in, you can turn it into 0, 1, or more than 1 thing coming out. Again, these can be the same type as what you got in, so maybe a data, or if it’s a string parsing, it can be a different type. And, if there’s an error that comes in from the stream ahead of you, you can pass it straight through. If you throw an error, you can pass that along and that will end up coming out the stream on the other end.

There are convenience APIs exposed on these streams that match against types, which means that you can transform not against a very specific stream, but against all streams that emit a certain type. Tht means you can match against any stream that emits data, or any stream that emits strings, or any stream that emits your model object. Its a very powerful concept for designing the data flows of your application. Additionally, you can write your own, so you could build things that map JSON to your model objects or that map image data to image objects, for example.

So again, we have this example from before:

do {
    let string: String = try hostsFile.data().UTF8String
    print("String: \(string)")
} catch let error {
    print("Couldn't read: \(error)")
}

It was a little bit different. It gets the data from a file, and then we are calling UTF8String on it. After we put in transformers, I’m going to break this out into two lines:

do {
    let stream = hostsFile.readPullStream.UTF8String
    let string: String = try stream.drain()
    print("String: \(string)")
} catch let error {
    print("Couldn't read: \(error)")
}

In the first line, we’re generating the stream object itself. This way, we’re just calling readPullStream.UTF8String. All of the transform APIs are just the name of whatever the thing is, it’s not suffixed with stream or anything like that. Then, just as before, you call stream.drain. Really, the only difference is we’re adding that UTF8String, just as we were doing before.

And, just as we were doing before with the asynchronous:

let stream = hostsFile.readPushStream.UTF8String
stream.drain { (result: Result<[String]>) in
    do {
        let string: String = try result.result().first
        print("String: \(string)")
    } catch let error {
        print("Couldn't read: \(error)")
    }
}

We are just putting .UTF8String. Interestingly, Caramel tries to be very typesafe and type aware, so the result that comes out of here is not data; it’s an array of strings. That’s the typesafetyness of streams at work.

With that in mind try not to worry about the string objects themselves because they can look a little scary. Fortunately, you don’t have to worry about that. Xcode and Swift takes a lot of the hard work out of that for you. The important thing is not so much the type of the stream, but what the stream emits. If you were to option click on that string variable in Xcode, it’s going to say “string”, it’s not going to say some big, long, convoluted thing.

Sequences of Things (19:09)

I talked a lot about streams and how they are sequences of things. What does that mean? Well, it’s any object that conforms to this protocol, StreamBuffer, can be passed through.

protocol StreamBuffer: SequenceType {
    init()
    mutating func appendContentsOf(elements: Self)
    mutating func append(element: Self.Generator.Element)
}

StreamBuffer is just a sequence type, so whatever you can build to conform to this, it implements these methods. This is really about creating an empty initialized version of a StreamBuffer and then gluing stuff into it.

It’s designed to make it very easy for streams and transformers to buffer things. That makes it possible to do things like parsing up to a certain bit and then returning all the rest back into a buffer so you can loop through it again. For example, let’s say you’re parsing strings by line. You can parse data up to a certain point, and then you have a slice of string data left that is not necessarily a part of a new line. You can just put it back in the buffer and wait for the next cycle to come through. StreamBuffer is also very easy to conform to because it’s just sequence type plus those three methods.

Caramel gives you a few of these out of the box. The most important one is Array. Any array can be passed through streams. It doesn’t matter what the value type or the thing is inside the stream, you can pass those through. There’s also String.UnicodeScalarView, which is basically there to handle converting data to UTF-8, UTF-16, and back. Finally, there’s Data, which refers to data objects themselves, not arrays of data. This means you can chunk data and pass it through chunk by chunk, as opposed to needing an entire slice of it. You can also return data back into being buffers that way.

Unfortunately, we can’t stream raw strings by themselves because streams go out of their way to not conform to sequence type because they don’t want to hide the details of Unicode. That’s just the way that they decided to approach that in Swift. We will pass arrays of strings because you can pass any kind of array that you want.

Extensions (21:37)

With that in mind, let’s talk a little bit about extensions. Extensions exist on streams whose StreamBuffer’s sequences are of certain types. We’re not extending on any given type of stream, we are extending on what the stream emits. That means it doesn’t matter where you get a stream from or what type of stream it is, or if it is being transformed from something. You can still extend it with convenience APIs.

For example, for any stream that emits binary data, you have things like Base64 encoding, cryptographic hashing and HMAC, compression, TLS, and string parsing. You can write that data to a file or to a socket, and you can build your own data extensions to parse things into structures that you like. Streams make it very easy to create a composable structure in your application, with pieces that you can then chain together as you will.

Servers (22:57)

There is a certain type called listenable, and all servers implement this. Basically, servers listen for incoming connections, they act on data that comes in, and they return data to them. It’s basically a loop. A connection can be modeled as a Pushable Stream (a single stream of data that goes around), and your application logic can be implemented as Transformers.

The listenable protocol deals with that. You asynchronously emit pushable streams and you compose them with transformers.

Hello World server (23:41)

For example, here’s some sample code for Hello World Server. When it gets a new connection, it’s going to write out “Hello World” and close the connection. Here’s that program running on Caramel:

try! TCPServer(port: 8080).listen { connection in
    connection.outgoing.write("Hello World\n".utf8.data)
    connection.outgoing.close()
}

We set up the TCP server on a port and we listen, and whenever a new connection comes in, we get a connection object for it. The connection object has an incoming stream and an outgoing stream. You write to the outgoing stream, and you listen to the incoming stream. We’re just going to write “Hello World”, convert it to data, and close it.

You can model that conceptually as this kind of loop, where you’ve got the client creating a connection going into the TCP, the TCP sends it along with type data, you return some data back, convert that back onto the wire, and send it back on its way.

IRC server (24:33)

A more complicated example is an IRC server. IRC is a new line-based string protocol, which means you can derive commands by converting the data into strings and then splitting those strings on new lines. There is sample IRC server written in Caramel up on GitHub. I’m not going to list out the code for it because that would be very long.

Each step in the process is like a transformer, so the client comes in and makes the connection, the TCP server hands that data off, the transformer converts it into string data, then that gets passed to the next thing, which splits those strings by a new line, and then it gets passed to your app. Your app can implement that functionality, but it’s going to be receiving an array of strings, not binary data. That means that it’s doing all of that work in a composed way, where you’ve got this data flowing step-by-step through the system.

HTTP server (25:39)

Now, let’s talk about the meat and potatoes, the HTTP server. You have a transformer that will convert binary data into HTTP requests objects, so you can think of HTTP as just a transformer of binary data to a model, and that’s what we’re going to do. The HTTP transformer receives the data, converts that into a request of type data, and passes that to your app. Your app can then do whatever it wants with it, and you pass that back.

Now, you can start thinking about middleware to implement on top of HTTP, for example GZIP data. You just pop in that transformer and any data that comes in over the wire; if it matches the GZIP header, it can do a decompression on it, pass that to your app (now freshly uncompressed), your app can pass uncompressed data back, and the GZIP module will handle transforming it back into the type of data that it needs. HTTP will serialize it, and then pass it back over the wire.

You can add more to this. Now that we have uncompressed data, let’s JSON parse it, and then we’ll pass back a typed JSON dictionary to your app. Your app does what it needs to do, passes it back, JSON serializes it, and GZIPs it. HTTP converts it to data, and passes it back to the wire. You can build your app in these composable modules, and then chain them together. This leads to really powerful functionality, and really isolated, testable logic in your app.

One way that you might implement this is through a router, and the router is a terminator on your code. However, the router itself might be able to handle its own routes, with its own chains, so you have your get user requests. Well, maybe you want to parse that data into a user model. This could be a piece of code that you write for the application yourself. It parses the user model into an object and now your app is dealing with a user model object. You write that back out, it serializes that to JSON, the JSON to data, the data to GZIP, the GZIP data to the HTTP request, etc.

Then, you might have a different thing for the post-request where it’s a totally different type of model. You might have a totally different transformer, and you serialize, deserialize it, etc.

By the way, if there’s an error in parsing or a failure at any point, it will short circuit that route and send an error back, so your application can isolate error handling to where it makes sense.

Moving Forward (28:22)

I want to stress none of this is production ready, and given that Swift just open-sourced, who knows what will happen with this. All the code is up on GitHub, and it is MIT-licensed.I’ve got an IRC room (#CaramelForSwift on Freenode), but it’s currently just me sitting in there. Come join! I want to stress that it has an inclusive code of conduct, because I want to make sure that everybody who wants to come participate in this can feel welcome to do so without having to worry about being harassed or threatened.

Where are we going to go from here? Right now, Caramel needs tooling, a command line runner app, unit tests, and API feedback. If you guys want to play with it and tell me what you think sucks about it, please do! It also needs more error handling. If anybody wants to submit pull requests and tell me ways to make this better, that would be awesome. I also need to document it, and get a pretty website for it. We aim to make the friendliest, most productive framework for building servers and embedded apps that we can make.

Q&A (30:05)

Q: How are you planning on marketing Caramel if it’s primarily going to be around Xcode at the moment? Doesn’t that mean everyone who wants to develop on it has to be on a Mac?

Steve: You can develop through Xcode, but there is no requirement to. This talk was prepared for before everything was open-source, and there’s no setup for deploying on Linux, or deploying to a Raspberry Pi, or anything like that. When that stuff comes to be, and we get things like support in Vim and Emacs, then it becomes a matter of just writing the code and hitting deploy. There’s no dependency on Xcode, nowhere in the framework is Foundation imported, or any of the Apple-provided frameworks. They are not in this code at all, and that was intentional, because we knew, or I knew, that we would have to deploy this on Linux. As far as marketing goes, I don’t have a good answer, but the goal is to make it widespread if people like it.

Q: I might’ve missed it, but you said there’s a package manager. Are there plans for a build tool of some sort?

Steve: Yeah, Apple gave us our package manager that we’ve all wanted for 10 years now, so… The build tools are primarily going to be Swift-based. The one trick right now is that libuv, which provides asynchronous I/O, relies on the concept of an event loop, and you need to actually say eventLoop.run somewhere in your code. Ideally, we can move to a standalone binary, where it’s sort of like the Swift REPL, where you can just type Swift and the name of a Swift file and it starts running it, but instead of Swift, it might be Caramel. So that would handle running the event loop after your application is set up, and then you could actually do stuff with it.

So there’s all kinds of things that I think would be interesting to talk about in terms of packaging. One of the crazy ideas that I had was something like bitcode for deploying applications into Docker instances, where you don’t know necessarily what kind of environment you’re going to be running on, but maybe that you can do some LLVM trick at run time to figure that out. So there’s all kinds of stuff that you can do, but right now we’re just scratching the service. Again, right now we’re just trying to get to the point where this runs and you can build web apps with it.

Q: It looks like this could really benefit from a dedicated FRP library for all the stream-based stuff. Have you considered it?

Steve: Yeah, I looked at some of the reactive libraries. First and foremost, they all have some sort of dependency on Foundation. That’s going to go away, so I’m not worried about it, but that was a consideration. Really, a lot of the reactive stuff today that you see as generic reactive libraries, they tend to sort of kill your stream as soon as an error happens. You can work around that, but it’s not something that I wanted to make a hard and fast rule. Certain things like HTTP connections, that are long-lived, that maybe used the same TCP connection under the hood for multiple HTTP requests, if an error killed that stream at that moment, that would break other assumptions, so there was a little bit of worry that I had about making sure that that didn’t happen. That’s not necessarily a hard and fast thing. I’m totally open to look at putting more structural reactive libraries in there because the nice thing about things like RxSwift is that they give you this massive set of operations that you can do. I have no problem. I love RxSwift, I use it in other projects, and ReactiveCocoa too, so that’s totally something that we can look into. Just for the sake of getting to this point, where we actually had some code that ran, I sort of needed to own the whole enchilada and we’ll figure out how to fragment that later.

Q: If I have something like a block cipher or an IRC server where you’re waiting for the entire newline-terminated string, what does that look like? Is that all going to be handled under the hood or how do I wait for an amount of data in my stream?

Steve: You would implement a transformer to do that. Basically, every time the transformer gets a new thing, it’s going to return everything that has been buffered up to that point and then whatever the new stuff is.

Q: So the transformer would buffer it and—

Steve: Yeah, and then that’s handled by the API for you. If you don’t need all of that, or you want to take a slice of that, and do something with it and then return the rest to the buffer, you could do that. That’s part of the transformer API. For example, if you have one string that contains one full line and then half of the next line, potentially, you would parse up to that first string and pass it along, and take the rest and put it back in the buffer. By doing that, you would be able to store the full string and only return full strings when you get them and then just wait for the next bit. There’s actually a built-in transformer to do that because splitting strings seemed like a pretty good thing to have kind of a baked component for it, so if you’re building that IRC server, then that’s how you’d do that.

Q: With the open-sourcing, and I can’t speak for Apple, but I would imagine the route they’re wanting people to go is to build off of Core Foundation, rather than bridging into C libraries that are cross-platform. And Core Foundation has support for streams, does this add beyond that, or what does that look like?

Steve: Obviously all the stuff that was announced is super new, I haven’t really done a full dive into it, but my understanding is that it’s going to be a pretty close port to what Foundation is today. If you look at what Foundation is today, it’s a 20- or 30-year-old framework that is in Objective‑C. It uses Objective‑C semantics. That’s what it was designed and built to be. We’ve got a totally different language now, with a totally different set of semantics, so hopefully there is an opportunity to build something else.

That’s what I’m hoping Caramel is. I’ve been using this to prototype some simple apps, like the IRC server, and it’s really powerful. It’s really cool to be able to just chain these things together and then deal with the right types as it comes out, whereas with Foundation, the streaming that comes built-in is based on what asynchronous file streaming was 20 years ago. Things like NSStream and NSInputStream have super weird APIs to use today. That may have made sense 20 years ago, when you had very, very limited system resources, but nowadays people just say file.read, and expect the whole thing. It doesn’t matter, systems are fast enough now. I/O is fast enough now. With that being said, the streaming that’s in there is all binary data–based. It’s all based on file I/O. It’s not generic. This tries to be a different approach, and if there is room, I would like to see people use it. If not, if people decide they don’t want to use it, if they want to use Foundation, then the people have spoken.

Q: You seemed pretty discouraged this morning, and I hope you do continue working on this. Is there any bigger project that you’re trying to develop with this as you go along developing Caramel?

Steve: I would like nothing more than to build my web apps, my servers, and my APIs with Swift. I think the first commit on this project was three or four days after WWDC, when they announced that Swift was going open-source. It became very clear right away, what does that mean? It means we have the standard library and that’s about it. That’s not enough to build a web server. My first attempt at this was to just build the web server. That’s what I wanted. But you start looking at other things, like building on embedded devices with Swift. I think being able to build on little devices myself, hacker projects, is super cool and Swift is an awesome language to do that in.

I don’t have a specific thing right now, I have a ton of ideas. I look at this kind of architecture and I want to build a music mixing app. There’s all kinds of stuff you can do with these typesafe streams of data and streams of objects. I think it’s like really cool, just playing with this in the limited time that I’ve had, while I’ve been scaffolding the whole thing. It is a very different programming model than what you get from Foundation and iOS apps.

This morning, Apple announces they’re open-sourcing Foundation. That’s what this is, right? It’s not Foundation, but it’s a Foundation-like developer platform. And when Apple comes along and says, “we’re going to build this,” that means that it gets the pedigree of coming from Apple. That means that developers are going to target it and use it. Open-source Foundation was never a part of what Apple said they were going to do. I think most people in the community never thought they were going to do anything like this. It was a shot in the gut right out the gate, but looking at what I’ve had time to look into today, it seems like our two approaches are going to be very different. If there’s an opportunity for coexistence, then this seems like a really powerful programming model. Similar programming models have existed in the world of Java and Scala and Haskell and others, not exactly like this, but similar, for a long time. There’s a general mindset that says this is an interesting model for programming, so that’s what I want to see.