Developing Apps for Apple TV

What is the Apple TV? What frameworks are available? What design differences should you think about? Sally’s talk from MBLTDev in Moscow covers making a tvOS client-server app, porting an iOS app to tvOS, and managing dependencies, the focus engine, on-demand resources, parallax images, and universal purchases.

---

Introduction (0:00)

I’m Sally Shepherd (@mostgood on Twitter), and I’m mainly an iOS developer. I also do some side projects working on fitness trackers for cats. It’s not very related, but very fun! I’ve wanted to talk about Apple TV forever, and I’m so excited that they finally brought it out for developers.

The Apple TV (0:33)

What is the Apple TV? The fourth generation was announced in September 2015, but this is the first time we’ve actually had access to writing software for it directly.

What kind of stuff can this little box do? It’s a 64-bit A8 processor with 2 GB of RAM, Ethernet, WiFi, Bluetooth, all the fun stuff. You can get it in 32 or 64 GB, and it’s about 10 cm², so it’s very little. It also comes with a fancy remote with a glass touch screen. It’s not as souped up as some video game consoles, but it’s capable of doing some pretty awesome stuff.

Its OS is called “tvOS”, and it provides access to a lot of different frameworks. If you’ve done iOS or Mac, you’re probably familiar with most of them already. You can write in Objective-C or Swift, and they’ve also added some special frameworks just for tvOS in JavaScript. There’s also a markup language so you can make lightweight template-driven apps.

Basics of Interaction (2:30)

Apple has pioneered a lot in the way of touch interfaces, and they carry that over to the TV with a cool remote. You’ve got the glass touch surface, which actually recognizes gestures; you can swipe, tap, and press on it. The Menu button usually serves two functions: either it will take you back to the previous screen in a navigation hierarchy, or, if it’s in a game, it will generally pause the game and then give you some contextual options. The Home button takes you home, the Siri button will activate Siri (in countries that support it), and then you’ve got the Play and Pause button.

It sounds lame, but one of my favorite features is the Volume Up and Down buttons. The Apple TV supports HDMI-CEC, so it can control the volume on your TV. It’s amazing to only need one remote. If your television doesn’t support HDMI-CEC, you can also pair it via infrared, so you can still use it. Finally, the remote also has an accelerometer and a gyroscope which you can leverage.

There are also third party controllers you can get, including the very popular SteelSeries Nimbus Controller, which is great for gaming. There is a special Apple TV version of Guitar Hero coming, and Disney Infinity is launching something as well.

When you are developing apps for the Apple TV, you’ll probably end up using the Simulator at one point in time, because it’s a lot faster than trying to build onto hardware every time. There is a software version of the Apple TV Remote and you use it kind of in combination with the keyboard as well. If you want to navigate through the items on the screen, you can use the arrow keys on your keyboard, and if you hold down the Option key, you can kind of mimic swiping and tapping and all sorts of fun things.

Basics of Design (5:15)

If your background is solely on mobile devices where interaction is driven through touch screens, you’ll probably have to modify some of the ways that you approach design. The biggest difference is how people experience TV.

On mobile and web, we pretty much design things to be a very exclusive kind of single-user experience. We only worry about what that one person wants to do. They are very much in control of what’s happening, and in these situations, we show them information that’s particular and personal to them. However, if you’re in a room with with lots of other people, you need to be careful about what to show. For example, bank details may not be great to broadcast to everyone in a room, and you probably don’t want your text messages showing up on the TV either. It’s important to think about the context, and to consider that other people are probably going to be around.

Another important difference is distance. People can be anywhere from 1-4 meters away from their TV, so don’t use impossibly small text. TV is a cinematic, dynamic experience, and you should try to play that up in your apps. However, don’t needlessly fill the screen with millions of elements. You should provide clear, concise layouts with shallow navigation hierarchies.

With web, we tend to put lots on the screen because users can use a cursor, so precision is fine. When we go to mobile, we simplify it; we have to make elements bigger, we have to put space around them so that users still have that precision, even on a touch screen. On the TV, we simplify it even another level. We create a grid layout, grouping things to speed up navigation.

Another thing to test is the color. When you build your app, especially on your computer, the color will probably look great. Then, you run it on a TV and it might look a bit scary. It is important to test it on different TVs, and try to test it in different color profiles as well, if your TV supports it. They’ve also opened up TestFlight distribution, so you can get beta testers for your Apple TV apps. Make sure that it doesn’t look weird on other people’s TVs.

Something to watch out for is overscan. This is a legacy from old CRT displays, where it would trim the edges of the picture and scale it up. Some modern TVs still support this so that old content doesn’t look bad. As you’re designing, make sure you always have some padding on the edge to account for this. Don’t put anything too important on the sides or the bottom.

I’m not going to go through everything about design because there is a lot, but if you are interested in it, or if you are going to do an Apple TV app, please read the Human Interface Guidelines for tvOS. It’s a really well written document, and it will save you lots of time.

Starting From Scratch (8:45)

Let’s look at what’s available for making a tvOS app. There are two main ways to approach it: the Client-Server approach and the Traditional approach.

tvOS provides two ways that Apple have called the TVML, or Television Markup Language, and the TVJS classes to show content from network resources. TVMLKit calls to a server-based JavaScript file which provides content in the TVML, and then this file includes Data, Assets, Media, and anything else that it needs on the server. The return files are rendered through templates, and all the user interaction is handled through the JavaScript file. You can add UIKit code or anything else you want. If you want to pursue it more, there is some great sample code called “TVML Catalog”, and it shows you all the different templates and what it can do.

For the more traditional approach, you can use UIKit, which is probably what most people are using on iOS. You can use things like Metal and OpenGLES, or if you want to do 2D gaming, SpriteKit, or for 3D games, SceneKit.

However, not everything is quite available on tvOS yet. A lot of frameworks are available, but several are not. The missing one that everybody seems to want is WebKit. You can’t show web views on the TV, but this is great, because you don’t get all these annoying pop-up ads while you are playing games. It’s important to note that these frameworks are tvOS-specific versions, so they are not necessarily as full-featured as their equivalents on iOS or Mac. Check the documentation to see what might be missing.

Porting an iOS App (11:12)

What if you had an existing iOS app, and you wanted to port it onto the TV?

There are two ways you could approach this: you can start with a new, tvOS-specific project, or you can just add a new target to your existing project.

When you do a new project and select tvOS, you get to choose between a Game, a Single View Application, or a Tabbed Application. You get the same three options if you add a new target. Which one you choose totally depends on your project itself, but if the TV one is going to be very, very different from the iOS one, it might be worth doing a separate project. If they are quite tightly linked, a new target is probably a better way to go.

I’ve been working on porting a game called “Qiktionary” to the TV. Qiktionary is a little word game where ou have to guess what the four letter word is and then it unlocks a fact. You guess a word, and it tells you if any of the letters are correct. If you’re wrong, you have to guess another word and eventually, through the process of elimination, you figure out which word is right.

When we started porting it, we decided to add a new target. The first step was to make it compile, which was a long step. For any Apple-provided frameworks or third party frameworks, it all has to be built for tvOS. This means if you added a new target, you’ll have basically two copies of the framework, because one will be for iPhone and one will be for TV.

We were using CocoaPods to manage dependencies, and although CocoaPods supports tvOS, it’s still down to whoever writes those libraries and frameworks to add tvOS support. None of the libraries that we were using had added support for this, so we changed everything over to get sub-modules to give ourselves more flexibility on being able to change the code and keep track of that.

Then, we had to do things like enable BitCode, which is an intermediate representation of a compiled program. By including Bitcode, it allows Apple to re-optimize your app binary without the need to submit a new version of your app.

After about a day of getting all of our dependencies up and running on tvOS, we had to wrap a lot of code to make sure it was only compiled under iOS, and some other modifications that only ran under TV. We finally got it to run, which was so exciting, but on first launch, it was SO small on the TV. The main problem we had was you couldn’t actually interact with anything on the screen, which is a big problem for a game. You couldn’t focus on anything, swiping did nothing, and pressing the button did nothing. So we decided to spend some more time and make it a bit more native.

Make It Native (15:21)

tvOS has this thing called the “Focus Engine” that handles, manages and controls the focus on the screen. It listens for any incoming focus movements from your Remote or Control, and when an event comes in, it automatically determines where the focus should update, and notifies your app.

Programmatically, you can ask for focus updates, but you can’t set focus or move focus in a certain direction. UIKit items such as Buttons and Controls, SeachBars, TableViews, and CollectionViews will work automatically because they have canBecomeFocused. For example, a UIButton will focus by default but a UILabel won’t, but you can make it focus just by overriding canBecomeFocused.

By default, all classes in UIView and UIViewcontroller that directly or indirectly control the viewer’s screen conform to the UIFocus environment protocol. If you are doing anything custom, it’s important to implement the protocol to handle the focus for these things too: preferredFocusView, setNeedsFocusUpdate, updateFocusIfNeeded, and shouldUpdateFocusInContext. You can also do didUpdateFocusInContext:withAnimationCoordinator to have nice animations happen.

Then, Focus Engine + SpriteKit = 🙁. It’s just not supported at the moment. Even if they did support it, I don’t know quite how it would work. It’s a little complicated. We basically had to write our own way of handling these events.

To start, we looked at the keyboard for Qiktionary, which we based off some sample code from Apple. If you imagine the keyboard, and each key is a KeyNode, basically, in the “keyboard” class, we create an object to store for each key, which keys are Up, Left, Right and Down from it. Then in the KeyNode class, we add a variable to track whether or not it’s focused, and some behavior on how it should look if it is focused, and if isn’t focused, and when the “keyboard” class launches, we make sure that we set the focus to the top key.

Then, we create a class that sits on top of the responder chain, and this captures any input device events and forwards them on to the currently focused KeyNode. When the user inputs a touch event on the remote, Apple TV captures the input into the UIresponder chain, forwards it on to the currently focused KeyNode, and the KeyNode then asks its parent, “What’s the thing to the right of this?” It returns the key that’s to the right, and then performs the update of the state. It works pretty well!

If you are having any problems where focus isn’t working, there is this whyIsThisViewNotFocusable for debugging, and it will actually tell you why it isn’t focused. In this case, it was returning NO from canBecomeFocused (although returning YES doesn’t actually fix it).

Storage and Other Concerns (19:20)

Another challenge is that the app bundle itself is limited to 200 MB. You can only read items from your bundle, you can’t actually write to them. To write data, you need to either write to the tmp directory or the cache directory, or store something in NSUserDefaults. The tmp directory’s contents are only actually alive until somebody presses the Home button, so it’s not very persistent. The cache folder basically will be there until the system decides it needs to clear it out. NSUserDefaults is great, but it’s only 500 KB. Apple recommends strongly to use something like iCloud or some other syncing service. You can also use on-demand resources for an additional 20 GB of data.

With iCloud syncing, there are three options: the Key Value Store, which can hold upto 2 MB and 1024 keys; CloudKit, private and public, so upto to 10 TB in public and private; and you can also store documents, but that depends on how much the user is paying for.

On-demand resources are a relatively new thing, where basically the resources are hosted on the App Store, and are separate from the app bundle that you download. The app can request sets of these on-demand resources and the operating system manages the downloading and storage of them. In Xcode, you basically create some tags and attach them to the resources, and then when your app requests the resources associated with the tag, it just downloads those assets.

We didn’t find too many things that we were personally missing from the frameworks, but I’ve worked with a lot of other people who have missed them. MultipeerConnectivity doesn’t exist yet for tvOS, for example, and a simple solution to that is to drop down to a lower level API and use something like NSNetService. For other things, you can use a third party REST API. It’s important to file a radar if there is something missing and you can’t find a way around it. That’s the only way that Apple will fix things.

We also have parallax images, where there are multiple layers to the image, and as you move around, they move at slightly different rates to create the effect. It gives us great sense of depth and vitality that helps emphasize what’s focused on the screen.

There is also the “Top Shelf” feature. Basically, users can put up to five apps in the Top Shelf, and when the app is focused, it shows related content. You get to define what these things are that it shows, and it’s surprisingly easy. Not that many people are doing it yet. Basically, you add a Top Shelf extension and it creates a class that conforms to the TVTopShelfProvider Protocol, and there are two things that you specify: the topShelfItemStyle and topShelfItems.

Finally, there’s “Universal Purchase”, which is quite cool. If you have an app on iOS you can bundle it with the TV app. If a customer bought it on iOS, they also get it for free on TV. To do this, all you need to do is make sure it has the same Bundle ID. It’s easy to do on iTunes Connect, but once have enabled this, there is no way to disable it, so make sure you made the right decision.

Lessons Learned (25:29)

We’ve only worked on porting Qiktionary for the past couple weeks, but to wrap up, I wanted to go through a few of the things that we’ve learned so far.

Architecture Choices

We definitely need to make things a lot more modular. Especially when you’re just doing iOS, it seems harmless to just say, “Yeah, we’ll just do this and fix it later, because we’ll totally have time.” By doing that, you end up with messy code.

If there’s too much of a separation between what you’re doing on tvOS vs. iOS, your class becomes thousands of lines long. Qiktionary wasn’t written so that the keyboard and the gameboard were all in the same class. The keyboard has its own thing, and we tried to make everything as reusable as possible so it’s easier if you need to swap it out for TV. Once you start porting, you’ll see all the bad ideas in your code. After this experience, writing iOS apps has given me a better perspective on how to approach things.

Strict Code Dependencies

I like CocoaPods, but I do worry about how people have all these dependencies that they don’t know about. We got rid of quite a few, and we’ve now decided to only use ones that support tvOS and iOS. If you make any frameworks for iOS, make sure that you add support for tvOS.

Design Changes

One thing we’ve learned is that keyboard layout is not very fun to play with. Keyboards on the Apple TV are generally not very fun, so we’ve been working on ways to improve the experience around that.

Ultimately, this is just the beginning. Despite the name “tvOS 9”, it is still just the beginning. More frameworks will be added, but please file radars if anything is broken or missing. And don’t make any more fireplace apps; there are already too many! Go make something awesome, because although it’s still a relatively quiet platform, people are spending a lot of money on it. 💰

Resources (28:47)

• If you want to get started with tvOS, the App Programming Guide for tvOS is excellent, in addition to the previously mentioned Human Interface Guidelines.

• The DemoBots sample code is great too. It’s a cross-platform game that runs on Mac, iOS and tvOS, so it shows how to handle a lot of that.

• There is also the TVML sample code as well as a UIKit sample code that catalogs everything.

Q&A (29:18)

Q: I’m not sure if you app is already available in the store, but do you see any impact on your user base, or users integrated to your OS or not?

Sally: It’s not in the store, we are still trying to make it better. But I have friends who’ve done tvOS versions of their apps and have seen huge increases in downloads.

Q: How much time do you spend to prepare tvOS version of your app?

Sally: We spent about a day just making it compile on its lowest level, and then we’ve spent probably another few days just trying to get the design bits to work. The Focus Engine stuff took about a day as well. It’s been a few weeks. I think we probably have another few weeks to go before we are ready to test, then another few weeks to submit.

Q: Have you had a chance to test it on a real device, and if so, how did you organize that?

Sally: Yep. Apple was really awesome and actually sent out developer hardware. I got an Apple TV for £1, which is great value. It’s great value at it’s normal price, but it’s tremendously good value at £1. It connects via a USB-C to your computer, and you can basically just select that as a device and run it straight from your computer.