Extending Xcode 8

on Oct 8 2016

Xcode 8 introduces a new mechanism for extending the source editor with app extensions. In this talk, you will learn more about the practical implications of developing Xcode extensions: how they are distributed, positive and negative tradeoffs of their design, and how to keep an extension’s host app efficiently in sync with the extension itself.

Introduction (0:00)

I am going to talk about extending Xcode 8, but before you start preparing your pillows to fall asleep, I promise you there will be no source code in this talk. Which is kind of an unusual thing to promise for a technical talk, but I just thought I’d take a different tack and talk more conceptually about Xcode extensions. I’ll bring you up to date with what they are, and if you haven’t been paying attention, what they can do.

A good developer (0:35)

I’m going to adapt an old cliché. A good developer is only as good as their tools. I think many of us feel this way. Especially when our tools are working for us, and we maybe even feel that way doubly so when our tools are not working for us. But we’re always trying to push the limit. We’re always trying to see what else we can do. What can we get out of these tools?

We get great tools from Apple, as Mac and iOS developers, but I don’t think any one of us is satisfied with them. There has been a tradition of plugins, extensions, custom tools, custom compilers sometimes even, and preprocessors. Whatever, you name it, there’s something out there. Finally, as of this past summer, at WWDC, Apple is officially on board as well with Xcode source editor extensions, and I want to talk a little bit about what they are.

They’re not here yet. They are coming in Xcode 8 and they are fairly limited. They are named, appropriately, “source editor extensions” because they only affect the source editor in Xcode, but, on the other hand, they are the only Apple supported mechanism for general purpose plugins for Xcode. If you’re interested in extending Xcode, this is what we have to work with for now.

I want to get some bad news out of the way. How many people here use Alcatraz? You don’t anymore. As of Xcode 8, you will not be using Alcatraz because Apple has added Xcode to the system integrity protection list which prevents all manner of things, including attaching to and debugging the app and injecting code into the app. That means the mechanism for loading plugins that was sort of informally supported over many years is not going to be there in Xcode 8.

The other bad news is that the scope of these extensions is pretty limited. You can’t do much outside of the source editor. And then another kind of bad news, depending on how you look at it is, you have to have an app carry this extension to your customers. You think of a plugin or an extension historically as being a standalone thing you can drag around and pop into a special directory and then it magically augments an app’s behavior. You’re going to have to build a Mac app to go with whatever extension you ship for Xcode.

Now some good news. Again I would like to appreciate that these are Apple approved. That means the likelihood of them stopping suddenly to work completely overnight - the way, actually, every Alcatraz plugin does whenever Apple updates Xcode - is greatly diminished. It’s a good sign that Apple sees that there’s a need for extending their tools. It’s a way that Apple’s been evolving a little bit over the past several years to indulge developers, acknowledging that they haven’t provided us everything we need, and to give us approved ways of doing more interesting things.

Apple’s focus on security and stability has become famous, especially the security stuff, and there have been a few scares over the past few years about plugins and Xcode and whether they might maliciously alter your app. Whether they might ship with it some kind of code that could compromise a system and your app would thereby be sort of complicit in this security violation of a user. The fact is, these being Apple approved also implies that they are designed for security and stability, and they in fact are. That leads to a lot of their limitations, but it’s actually a pretty good thing as a starting point. To know that you can install these things and use them and not feel as though they have the ability to completely change what you ship to customers.

Finally, I think it’s a good sign to see Apple doing anything on this front and there’s cause for optimism. You remember the first iPhone didn’t even have copy and paste. It was miserably slow. It had a terrible camera. When Apple gets excited about something and they do iterate on it, we do see continuous improvements year after year, and I hope that that’s a indication here that this is the tip of the iceberg for extending Xcode.

World of possibilities (5:05)

I want to talk a little bit about what extensions can do. I said already, they’re limited to the source editor, and you might think, okay well, that’s it. That’s all they can do. They can manipulate text. I tried to put myself in a mindset to imagine more about what they might do. It’s true they are limited to the source editor, but the way they’re implemented, they’re actually an independent process.

This is part of the security model of app extensions. They’re an independent process that has its own set of sandboxing privileges, and so you can essentially have this extension work on your behalf inside of Xcode doing a lot of different things that aren’t necessarily literally the process of changing text. Of course, they can change text. One thing that you might not have thought about is they can also access the web. You could take some text in a document and send it out to a web service, have it be transformed and come back and then substitute the text. You can also access NSWorkspace on the Mac. Which if you’re not a Mac developer, it’s like UIWorkspace. You can do things like open URLs with it, so you can have an action in an Xcode editor trigger a context specific opening of a web URL. Maybe jump to a GitHub page. Maybe go to a reference pertinent to the source that you’re editing. And you can also display rudimentary UI.

This thing has access to the whole Cocoa framework, so if you want to implement an extension that does something useful, but you don’t want to change the source that the user is using. Let’s say you had some kind of statistics panel you wanted to put up. You could put that together in an extension and then display a Cocoa panel that conveys information to the user.

Finally, the compromise of having to use an app to deliver these things means you also get to have an app on the user’s Mac. If you can think of things that make more sense in an app. The canonical thing is to have configuration preferences for your extension in an app and those would be communicated to the extension probably through a shared preference file. Apple recommends that you use Darwin notifications to communicate between the extension and the host app, but apart from that, you could then have them share stuff either through shared files or through shared memory.

The bad news, again: I already mentioned Alcatraz is gone. And any dreams you have of using an Xcode extension to rework the structure of a project - for example, I might like to have an extension that said, I point at a folder in my SourceTree and it says alphabetize all these files for me, or something like that - that’s not possible with this mechanism. Also, anything that we sort of appreciate about Xcode knowing so much about our source code, it has its entire semantic tree of our sources and our project. It would be cool to be able to access that, but we can’t access that directly through this extension model.

Finally, as part of the security limitations, the only way that your extension will be invoked is by direct user action. Essentially when you make an extension, you define commands that then show up in the editor menu of Xcode. The user has to select those for your code to run, and the only concession to convenience that Apple provides is that users can use Xcode’s built-in custom keyboard shortcuts to assign a keyboard shortcut to your extension command.

Build and run (8:44)

When I look at a new technology like this, I usually think, that looks like it could be cool. I know nothing about it. I should probably build something and try it out, and then I go down this, kind of unfortunately, error-prone road of trying to figure out how to make the first project. If you can’t find a sample project out there that happens to be a good skeleton for it then, in my experience anyway, I waste a lot of the enthusiasm I have trying to just get something working.

I would like to walk you through the process of making your own new source editor extension from scratch. It will be very painless, and I’m trying to show how it is painless, but you have to know the right steps. As an overview to remind you, they’re built on app extensions, so if you’ve done anything with app extensions on iOS or Mac, it’s the same kind of process. That means you use Xcode to build an app extension which gets bundled in a host app, so you always start with a host app when you’re working with app extensions.

The working model for how you debug and build these things is you have a scheme in Xcode which is for your extension and when you build and run that extension it asks you, it prompts you: which app do you want to run it against? For example, with Share extensions it makes a lot of sense. You choose an app that you want to invoke it. You want to run the app in order to use your Share extension.

In Xcode they’ve adopted the same model, so you have to choose Xcode even though there’s no other sensible choice for what you would debug against. The overview of what we’re going to do is, again, we’re going to create a new Mac application. If you set out to make a new Xcode source editor extension, you will not find anything in Xcode’s project templates to help you because making a new project for this is not something that you can do. You make a new Mac application first then you have to add a new extension target and then finally, when you have both of those targets set up you can build and run the extension scheme and select Xcode and you’ll be in business debugging your extension.

We’ll walk through it with visuals here just to make it painfully obvious. This is the new project template. You’re probably familiar with this except for yours probably doesn’t have a red sweater application template item. You select Cocoa application. Name your app whatever you like. I chose Xcode Extender. That’s the app that’s going to have to live on the user’s Mac somewhere, but it doesn’t have to do anything to possess this extension. Then, you look in the targets list and you’ll find at the bottom, there’s a plus button there. That’s how you add new targets to an existing project. In this case, when we click that another template browser comes up which has what we were looking for, the Xcode source editor extension. Name that one. I called it My Extension.

Then, Xcode will prompt you to activate the scheme that it’s automatically creating for you. This is a good idea because it will set you up to be ready to build and run your source editor extension and have it work. Now we have two targets in Xcode on our project. This was pretty fast once you know what to do. You’re at this point in about a minute or so and when you build and run it’ll prompt you, as I said, to choose an app. All of the ones except for Xcode don’t make sense, so we pick Xcode. Run.

This gloriously desaturated Xcode icon indicates that you are running a second copy of Xcode which is tailor-made to distinguish itself from the main copy. You run this to debug your source editor extension. After having done this, I go to any source file, and if you’re in a source file that’s suitable to extensions, and the editor menu will find the name of the extension, My Extension, and then any commands which it implements underneath.

This is the default you get when you build from their template but, of course, it’s pretty easy to change it after you get the setup. If I run that right now it won’t do anything, but you could, at this point, set a break point in the main version of Xcode and then be stuck there at the point where it’s calling you to invoke whatever functionality it is you do.

If you decide to take my encouragement and get started with this, I want to give you some advice that will, hopefully, speed you up through the pitfalls you may run into. If you’re on OS X 10.11, you have to go through some other steps. You can look in the developer release notes from Apple. They have a special line item for 10.11 users of Xcode 8. You do need Xcode 8 for this for a start, but then on top of that, if you’re using 10.11 and are not on the betas of Sierra, then you’ll need to do some extra steps.

You have to sign your app and extension. If you just build, I actually left that out of the last of the visuals because this is a pretty recent change as of developer preview or developer beta 6 of Xcode 8, the IDE will not load any plugins if they’re not signed. If both the app and the extension are not signed, and the way it shows that it’s not signed is it just doesn’t show up. There’s no console log error. There’s nothing, so take it from me, sign your app and your extension.

These are all reasons that your extension won’t show up in that little menu I showed you. Sometimes, and I think they’ve improved this in recent betas, it wouldn’t show up because the system didn’t notice that your extension existed yet, and I found that running the host app scheme from Xcode manually - so you actually run your app which, as you noticed, we didn’t actually do - that activates something in the system that says “Oh, let’s see if this thing has extensions.” I saw something in a release note about maybe improving that, but I have still run into problems where, for example, if I change something about the extension in the info.plist it won’t get updated sometimes until I run the host app.

Finally, Xcode doesn’t slow anything down to load these extensions. If you run Xcode and you’re excited to try your extension, you jump to the editor menu and there’s nothing there. It may be that you’ve done everything right and it just hasn’t finished loading extensions, so wait a couple seconds and click the menu again and see if your stuff shows up.

Hot tips (15:32)

I guess you could’ve called those hot tips, but those were caveats. They’re much different from hot tips. Caveats are things you have to understand because otherwise you might not even be able to proceed. At least that’s how I see it. These hot tips are things that are going to make you faster, and it’ll feel less tedious, and you’ll feel more productive developing and debugging these things.

The first thing you should do is select Xcode as the default debug target so you never ever again have to run and then select Xcode from that stupid app list. You do this from the Scheme editor. If you look in the Scheme editor under the run tab, there’s an executable; “ask on launch” is the default, and this is where I think they probably should have made the default Xcode, but we can fix it. We’ll fix it once and then we’ll never think about it again.

Another thing I noticed when I was building and trying to test my extensions, I’d have to go through this laborious process of: okay, do I have a project open? Now find the source file in here, click into it, okay now, test my extension and, something that’s maybe not obvious is, many apps on the Mac you can pass a full path to it when you invoke it, as if from the command line, and it will open that file - and Xcode is no exception.

If you go to the arguments tab in the Scheme Editor for the extension, you can add a full path. I recommend making a canonical source file that exhibits all of the features you want to test your extension against. Give it a keyboard shortcut. You have to do this through Xcode’s own preferences UI, but once you get into the groove of build, run, test, debug, you’re going to want to be able to invoke your shortcut as quickly as possible, and you can do that from Xcode’s preferences.

You can change the keyboard shortcut for everything - almost everything if not everything - in a menu in Xcode. That’s kind of handy to know. There are a lot of commands in there, by the way, that don’t even have menu items, so look in there if you’re ever wondering like wouldn’t it be cool if there was a keyboard shortcut for selecting the previous line but only the first word and the third word? It might be in there because there are all these things in there that, historically, have been part of Xcode way back to NeXT days, and they leave the command in there often even if it’s been taken out of the UI.

Finally, something cool I noticed was Apple has actually started embracing these within Xcode itself, so I’m inspired now to keep my eye on Xcode’s internal plugins folder because some of the stuff that’s in there. They have XC plugin bundle types, some of them have changed to .appx for app extension, and so the ones I’ve noticed are simple stuff. Like the command to comment and uncomment text. The IDE is now implemented as one of these extensions, so I’m going to keep my eye on Apple’s own extensions.

It sort of serves as a model for what is possible. They could always be cheating, but if you see something they’re doing, like for example, one thing they’re clearly cheating on is that their menu items show up wherever they want them to and yours will only show up at the bottom of that editor menu.

Conclusion (19:01)

Finally, I guess the point of this talk was to implore you to think about source editor extensions. Think about whether you have any clever ideas. They are fairly disappointing in their scope and what you can do with them, but I think Apple developers, of all developers in the world, are the ones who can do amazing things with a lot of constraints. The iPhone is an example of that. If anybody can come up with clever ways to make Xcode source editor extensions miraculously helpful, I think it’s this crowd.

The source code, which I mercifully left out of this presentation, is all pretty simple. It’s stuff I developed while I was playing around with this, but some of the canonical examples for things that you might think, “Oh yeah, I would like to put a panel up or I would like to open a URL on the website, on the browser,” or a cool one I put together pretty recently was inserting placeholders into the text the same way that Apple’s own completion stuff works. You can put placeholders into your text that you put in. It’s just a special text format that does that. That GitHub project will serve as an example of some of the ideas I had to try to think outside the box with these source editor extensions and maybe you’ll get inspired by some of that.

To close up here, I do believe we are often only as good as our tools and the good news though is that by working as a community we help each other. Carthage and CocoaPods are great examples of us all coming together and helping each other to do better work. I hope that this will get you thinking about this new technology that’s available to all of us and check out my GitHub stuff. Get your own brain working on what you think is possible or not and hopefully share some great tools to help us do better work.

Thank you very much for your time. You can get in touch with me on Twitter. I’m @danielpunkass and my site is red-sweater.com. Thank you very much.