Smoke & Mirrors: The Magic Behind Wonderful UI in Android

Download Slides

Introduction (00:00)

My name is Israel Ferrer Camacho. I am an Android developer at Twitter. I work on Moments, which is a nice experience: fullscreen scrolling where you can add a list of tweets, and it is a story–it does not have to follow a chronological order.

For those who are not an English native speaker as me, “smoke and mirrors” is an expression used to describe something that is obscuring or embellishing the truth. It is usually used for magicians because they do magic, which is simply not real. In the last Google I/O, Adam Powell and Yigit Boyar started the RecyclerView talk by saying UI developers have many things in common with magicians. If you think about some of the animations that we do, and some of the components that we use in the framework, that is the idea of this talk.

I’ll show some examples in the Android framework where we are using “smoke and mirrors” to make an application great, flawless, and smooth. Then I’ll talk about one of my favorite apps in Android, and how that experience is built, and I will do a demo.

Android Framework (02:17)

This is a Timeline in Twitter. Theoretically, anytime you have infinite tweets, even when scrolling really fast, performance is really good. The problem is we have an infinite amount of views that we want to show, but we have limitations. We cannot just instantiate thousands of views at the same time.

Actually, what we have to do is this: We are showing one, two, three, four views in the screen. We do not need to show any or instantiate any order view. Just five at the same time, seven, eight tops if the tweets are really small. How do we do that? We use this component called RecyclerView (the name will give you a hint of what it is doing, which is recycling).

RecyclerView (02:37)

Internally, RecyclerView has the linear LayoutManager (which recycles), responsible for measuring and positioning the item views, as well as determining when to recycle those views that are not shown anymore. When you try to show the next position, the LayoutManager is going to go to the RecyclerView and invoke the method getViewForPosition. When that is called, RecyclerView will go to the Adapter, and will ask for the type of that view because you can have many types of views in your RecyclerView; for example, if you want to have different headers, or you want to have a cutout for showing ads.

We get the type the RecyclerView is going to go to the RecyclePool, which is where all the ViewHolders that we can recycle. It will get a ViewHolder for that type. If there is none, it will go to the Adapter and then create one. But if there is one, it would send it back to the Adapter and say, can you bind this ViewHolder with new data, with the data for that new position that we want to show, the one that was showing on the animation.

Once it holds the information, it returns it back to the RecyclerView, and then the LayoutManager will lay out that item on the screen. This is a simple explanation of how the RecyclerView recycles views. You can learn more in the Google I/O talk called “RecyclerView Ins and Outs”, which shows how to animate items, and do transitions in between the states.

Shared Element Transitions (05:29)

Another example in the Android framework is this one that I am showing in the Twitter app. As you can see, there is a list of images, and when you click one it transitions to fullscreen. Those are called Shared Element Transitions.

Shared Element Transitions have smoke and mirrors too, but first let’s dig in on how it works. The activity has an activity transition state which persists a state of that transition, and the most important part is the shared views that are going to be transitioning from one screen to another. When activity life cycle is invoked, for example, it goes through onDestroy or onPause–it is disappearing, and then we are going to start a new activity, so it will call the activity transition state to start a new transition, which will call the activity transition coordinator.

ActivityTransitionCoordinator is a base class. There are two coordinators: one handles the enter transition, and one the exit transition, and that has a transition manager, and it will begin the transition. By the default the transition is a translation but you can change that on your theme, and you can customize your own transitions between activities.

What is the actual smoke and mirrors in transition in the shared elements? When an activity is going to hide or go to the background, and another one comes in the top, how are they re-enabled to do that transition from one to another? They hide the target activity, and they have a list of the views that are transitioning from what position they are transitioned. In any activity they do that transition. They use ViewOverlay, which is used by the default transition in the framework.

ViewOverlay (07:29)

ViewOverlay provides a transparent layer on top of a view, to which you can add any type of visual content, and that does not affect the layer on the top. That means that you can animate anything; it does not mess with the layer hierarchy. It is going to be your best friend forever in animations.

There are two layers: the LinearLayout and the ViewOverlay. I am going to add an animation to the ViewOverlay. How do I do that?

With LinearLayout (any view has that), you do getOverlay, and then you add to that overlay. Suddenly that ImageView is not going to be part of the LinearLayout anymore. It is going to be in this temporary ViewOverlay that we are going to use for animations. When you do that, the rect of that ImageView is invalidated in the parent, and that forces a re-layout. You have to be careful if you depend on that view, otherwise you will affect the whole hierarchy.

But the good thing is any type of touch event or animation gets delegated to that ViewOverlay with that ImageView. Imagine you have a reference to this ImageView, and you try to do animation. It will animate even if it is not part of the LinearLayout anymore, which in this case is part of the ViewOverlay.

Shared element transitions are nice, but there are limitations. The first one is that the user cannot control the transition with a touch. It’s just a simple animation from one place to another, so the eye of the user gets caught and does not lose the context between screens, but it does not allow the user to control the transition with a pinch and zoom. That is a limitation.

Another limitation is that the transition does not track the target destination, so if you do not eat all the touch events while the transition is running, it looks terrible. We can fix that by using this TransitionListener.

/**
  * A transition listener receives notifications from a transition.
  * Notifications indicate transition lifecycle events.  
  */
public static interface TransitionListener {

    void onTransitionStart(Transition transition);  
    void onTransitionEnd(Transition transition);  
    void onTransitionCancel(Transition transition);  
    void onTransitionPause(Transition transition);  
    void onTransitionResume(Transition transition);
}

Thankfully, we can hear all the life cycle events of the transition. What we can do is onTransitionStart, we are going to eat all the touch events. Then when the transition finishes, we will set the touch listener to the right place.

private View.OnTouchListener touchEater = new 
View.OnTouchListener() {
    @Override
    public boolean onTouch(View view, MotionEvent motionEvent) {
        return true;
    }
};

Important Attributes (10:39)

Let’s talk about two important attributes for an image. Consistently these are the attributes that are might be rendering outside the parent. First of all, the ClipChildren.

ClipChildren (10:52)

ClipChildren is an attribute of a ViewGroup. By default it’s set to true, which makes sense because it clips the children to the bounds of the parent ViewGroup and that is exactly what we want.

Here’s an example (see video). My ImageView is not allowed to draw outside the parent view unless I set ClipChildren to false. In that case, I am going to be able to draw outside the first parent but not the second, following parent. What I have to do is set that to false in all the parents.

<?xml version="1.0" encoding="utf-8"?>  
<FrameLayout
    android:id="@+id/parent"
    xmlns:android="http://schemas.android.com/apk/res/android"  
    android:layout_width="match_parent"  
    android:layout_height="match_parent"
    android:clipChildren="false"
    android:padding="@dimen/activity_vertical_margin">  
    <FrameLayout
        android:layout_width="300dp"  
        android:layout_height="300dp"  
        android:background="@color/colorAccent"  
        android:clipChildren="false">  
        <FrameLayout
            android:layout_width="200dp"  
            android:layout_height="200dp"  
            android:background="@color/colorPrimary">  
            <ImageView
                android:id="@+id/imageview"  
                android:layout_width="100dp"  
                android:layout_height="100dp"  
                android:src="@drawable/profile"/>
        </FrameLayout>  
    </FrameLayout>
</FrameLayout>

ClipPadding (11:49)

ClipPadding is the same. If you have the ClipChildren set to false but you have padding, the view is still going to clip to that padding (that happens especially with RecyclerViews). If suddenly a view gets cut in a RecyclerView, maybe it is because there is a padding and you did not set that to false.

Utilities (12:12)

A long time ago, I created these utilities, which allow me to go through all the parents of a view and set everything to false.

public static void disableParentsClip(@NonNull View view) {  
    while (view.getParent() != null &&
            view.getParent() instanceof ViewGroup) {  
        ViewGroup viewGroup = (ViewGroup) view.getParent();  
        viewGroup.setClipChildren(false);  
        viewGroup.setClipToPadding(false);
        view = viewGroup;
    }  
}

public static void enableParentsClip(@NonNull View view) {
    while (view.getParent() != null &&
            view.getParent() instanceof ViewGroup) {
        ViewGroup viewGroup = (ViewGroup) view.getParent();  
        viewGroup.setClipChildren(true);  
        viewGroup.setClipToPadding(true);
        view = viewGroup;
    }

Lesson: Almost anything fast enough will look good (12:30)

One of my favorite apps is Google Photos. I never had such a great UI experience in a gallery. It is top-notch. There is one layer, two layers and then fullscreen. It has three levels of zoom, and then it goes to fullscreen. The user is able to control the transition to fullscreen, which we were saying that Shared Element Transition cannot do. It is a limitation.

The way you figure out how something works in the UI is with these developer options: set both “Transition animation scale” and “Animator duration scale” to “10x”. Then the duration and animation are scaled 10 times slower, and you are able to see what is going on.

Another developer option that you can enable is “Show layout boundaries”. This is really useful for developers. It will show blue lines, which are the clip bounds, and red lines, which are the optical bounds. Pink areas are the margins. These help us see the alignment of the whole screen.

Here’s the Google Photos application with layout boundaries (see video). These are pictures from Google I/O. I pinch and zoom, that was the concert. Three levels, zoom scrolling, going back. We did not see that before because it was fast enough. Picture zoom is more difficult once you put it at scale 10x. You have to do a huge span in order to go to fullscreen.

You can see there are two different RecyclerViews. There is another invisible RecyclerView with a bigger destination size. They are cross-fading between one and another. It is all smoke and mirrors. The lesson is, anything fast enough will look good enough. If you try for 24 hours to make all your animations 10 times slower, you will see how the application suddenly does not make any sense any more. It is fast transition/animation that hides all the problems in implementation.

When you try to zoom in to the fullscreen photo here, it is the same image. They are faking it; they are copying the same drawable (maybe), and they are using it in order to do the transition to fullscreen. If you do it fast enough, and the scale is not enough (10 times slower), then it just works. More magic. These Google Photos guys are magicians. Fake it until you make it. That’s what we do all the time in development.

Demo (18:13)

I created a demo, which is a simplification of Google Photos. It has two zoom levels, and the last one goes to fullscreen. You can see all these smoke and mirrors, and check the code.

These are the layers. I have a container, a mediumRecyclerView, a smallRecyclerView, and a fullScreenImageContainer. The container is going to be the root of the whole hierarchy layout, and of course it is a FrameLayout. It is an obvious choice because we need to stack all those layers on the top of each other, and they are going to take up the full screen.

I need to put a fullscreen container on that screen because the shared element transition does not allow the user to control the transition to fullscreen. But I want to do it because that’s a nice experience. You can do that by creating your own NavigationManager. Depending on the state, you have a stack, and depending on the stack, you can instantiate the presenters and the view delegates for each of the layers.

But let’s talk about the important part: the medium and small RecyclerViews. From now on, I am going to call the smallRecyclerView “images”. The mediumRecyclerView will be the higher level of zoom before going into fullscreen. We are using two RecyclerViews because that is the trick that allow us to do that nice zoom-in effect on the gallery.

The pivot by default sits in the center of the view. When you try to scale something it scales from center, and it goes in all directions. That is not what we want; we want is to scale right down. The first thing we have to do is set the pivot to that top left point of the view, which is the RecyclerView because it is the one that we are scaling. That is important for scaling and rotating too.

/**
* Sets the x location of the point around which the view * is {@link #setRotation(float) rotated} and
* {@link #setScaleX(float) scaled}.
* By default, the pivot point is centered on the object.
*/
smallRecyclerView.setPivotX(0);
smallRecyclerView.setPivotY(0);
mediumRecyclerView.setPivotX(0);  
mediumRecyclerView.setPivotY(0);

We have two RecyclerViews, and my approach is to use two Adapters bound with the same collection back in the map. Because they are using the same data–they are the same image, they just have different sizes. It is important to set the medium one to invisible because we only want to show one first (the small one), the one that is going to be zooming in the first time.

smallRecyclerView.setAdapter(smallAdapter); 
mediumRecyclerView.setAdapter(mediumAdapter);  
mediumRecyclerView.setVisibility(INVISIBLE);

Let’s talk about touch events.

ItemTouchListenerDispatcher dispatcher =
      new ItemTouchListenerDispatcher(this,
      galleryGestureDetector, fullScreenGestureDetector);

smallRecyclerView.addOnItemTouchListener(onItemTouchListener);

mediumRecyclerView.addOnItemTouchListener(onItemTouchListener);

There are different approaches. I like to create this ItemTouchListenerDispatcher, which is a class that gets all the touch events. Depending on the state of that view, we delegate that touch to one or another class.

public class ItemTouchListenerDispatcher implements RecyclerView.OnItemTouchListener {
  ...
      @Override
      public void onTouchEvent(RecyclerView rv, MotionEvent e) {
          currentSpan = getSpan(e);  
          switch (rv.getId()) {
              case R.id.mediumRecyclerView: {  
                  if (currentSpan < 0) {
                      galleryGestureDetector.onTouchEvent(e);  
                  } else if (currentSpan == 0) {
                      final View childViewUnder = rv.findChildViewUnder(e.getX(), e.getY());  
                      if (childViewUnder != null) {
                          childViewUnder.performClick();  
                      }
                  }
                  break;  
              }
              case R.id.smallRecyclerView: {      
                  galleryGestureDetector.onTouchEvent(e);
                  break;  
              }
              default: {
                  break;  
              }
        }
    }  
...

I created this class, and added that class to both Adapters. It has more stuff but the touch event is probably the most important part. Since both the RecyclerViews are using the same, we can get the ID of RecyclerView. We know which RecyclerView is getting the touch event.

We have this galleryGestureDetector that we will use for scaling the RecyclerViews. In case the smallRecyclerView is getting touched, there is only one way: it can only go the mediumRecyclerView, it can only zoom in. We delegate that to the galleryGestureDetector which will do that zooming.

In case of the mediumRecyclerView, we have the same. The galleryGestureDetector will know how to do it. We have zoom out, which goes from the medium to the small; and we have zoom in or–in this example I was lazy, and I just did click–when you click, you go to the fullscreen.

We use the span. The span is the delta between the last span and the current span. A span is the distance between your fingers; that it is now pinch and zoom in, it is just clicking. With that, I ask the RecyclerView, which in this case is the mediumRecyclerView, give me the view on those coordinates, and then I just perform a click. That is how the zoom in, zoom out between this medium / small RecyclerView works.

Scaling with gesture (24:32)

But we were not scaling, we were sending the touch events. For scaling, we have OnScaleGestureListener, and it has onScaleBegin and onScaleEnd. onScaleBegin, you set up everything for starting the scaling. The scale itself is a constant stream of the scale gesture. onScaleEnd is the user just stopped touching the screen, it resets the position to the state that you want.

public interface OnScaleGestureListener {  
  /**
    * Responds to scaling events for a gesture in progress.
    * Reported by pointer motion.
    */
  public boolean onScale(ScaleGestureDetector detector);

  /**
    * Responds to the beginning of a scaling gesture. Reported by  
    * new pointers going down.
    */
  public boolean onScaleBegin(ScaleGestureDetector detector);

  /**
    * Responds to the end of a scale gesture. Reported by existing  
    * pointers going up.

    * @param detector The detector reporting the event - use this to  
    *          retrieve extended info about event state.
    */
    public void onScaleEnd(ScaleGestureDetector detector);  
}

Scale begin

onScaleBegin we want to show both small and medium RecyclerViews. We want to add the middle of that zoom in or zoom out. We are going to do a fade in, fade out in between both RecyclerViews. Both have to be visual. You can do it differently, but this approach works because the smoke and mirrors, it comes out at onScaleEnd.

@Override
public boolean onScaleBegin(@NonNull ScaleGestureDetector detector) { 
    mediumRecyclerView.setVisibility(View.VISIBLE);  
    smallRecyclerView.setVisibility(View.VISIBLE);  
    return true;
}

During Scale

onScale has all the maths of the talk. While there is a gestureTolerance that applies a low pass filter to avoid flakiness (because our fingers are not that accurate), the screen is getting some events there is a constant signal that can produce small increments and decrements on the scale. We want to apply a low pass filter; we know that a minimum span was met, and it is a span on that same duration that we were scaling before.

@Override
public boolean onScale(@NonNull ScaleGestureDetector detector) {
    if (gestureTolerance(detector)) {  
        //small
        scaleFactor *= detector.getScaleFactor();
        scaleFactor = Math.max(1f, Math.min(scaleFactor, SMALL_MAX_SCALE_FACTOR));  
        isInProgress = scaleFactor > 1;
        smallRecyclerView.setScaleX(scaleFactor);  
        smallRecyclerView.setScaleY(scaleFactor);

        //medium
        scaleFactorMedium *= detector.getScaleFactor();
        scaleFactorMedium = Math.max(0.8f, Math.min(scaleFactorMedium, 1f));
        mediumRecyclerView.setScaleX(scaleFactorMedium);  
        mediumRecyclerView.setScaleY(scaleFactorMedium);

        //alpha
        mediumRecyclerView.setAlpha((scaleFactor - 1) / (0.25f));  
        smallRecyclerView.setAlpha(1 - (scaleFactor - 1) / (0.25f));
    }
    return true;  
}

We have scaleFactor and scaleFactorMedium. ScaleGestureDetector gives us a scaleFactor, and we use that scaleFactor with a clamp function to limit the minimum and the maximum scale of both RecyclerViews. Of course, one is the inverse of the other, and the minimum and maximum have to match. Otherwise in the middle of the transition they would not have the same size, and then the fade in / fade out will look terrible, because it will not match. You can modify the numbers and make it as complex as you want.

The alpha is the same; we want that alpha to be in the middle of the transition, half of both of the RecyclerViews. At the beginning, set the smallRecyclerView to one, and the medium to zero, and at the end, the other way around.

What happens if suddenly the user stops touching in the middle of the transition? If we do not do anything else, if we leave it as it is, it is just going to look like two RecyclerViews and maybe half of both of them. You do not know, it depends, but it will look weird.

Scale ends

onScaleEnd, I want to hide this implementation with smoke and mirrors. I want to finish the transition automatically to one of the states.

@Override
public void onScaleEnd(@NonNull ScaleGestureDetector detector)
{
    if (IsScaleInProgress()) {
        if (scaleFactor < TRANSITION_BOUNDARY) {  
            transitionFromMediumToSmall();  
            scaleFactor = 0;
            scaleFactorMedium = 0;
        } else {
            transitionFromSmallToMedium();  
            scaleFactor = SMALL_MAX_SCALE_FACTOR;  scaleFactorMedium = 1f;
        }  
    }
}

For this, I use a TRANSITION_BOUNDARY. I calculate a delta, and I know how much is missing for the transition. If it is below 20 or 30%, (you can change it for whatever you want), I will assume that the user did not want to go fullscreen and we go back to this small image.

Otherwise, if it’s above that TRANSITION_BOUNDARY, he wants to go to fullscreen but he was just doing a really quick move. If you do not do that, everything gets stuck in one place. We do not want that because that will show the trick to the user.

fullScreenImageContainer (29:07)

The fullscreen is a black background that shows the image in full width. To do that there is this requirement by a RecyclerView. When you add an item to the RecyclerView, you add one item and then another item. The second item is going to be in a higher level than the first one. If you try to scale that first item, it is going to be drawing below the second item; it would just not look right. Each item has an elevation in a RecyclerView.

Recycler View elevation (29:29)

There are different approaches to addressing this:

1. The most difficult is to create a custom LayoutManager that knows when we are clicking that item. When we are clicking one single item, we want that item to be on top of everything, and to draw above all other items.

2. RecyclerView.ChildDrawingOrderCallback is a middle ground. When you get that callback, you set the elevations to the rest of the items we want to scale. It is going to be on the top of all of them.

3. Fake it with ViewOverlay! We can just use the ViewOverlay on top of the RecyclerView for the transition for the pinch to zoom to fullscreen. Then we just add that item, that view to the frame layout, which is the fullScreenContainer. Why make it more complicated when you can do it easily?

Transition to full screen (31:07)

@Override
public void onClick(@NonNull final View itemView) {
    ViewGroupOverlay overlay = fullScreenContainer.getOverlay();  
    overlay.clear();
      overlay.add(itemView);  
      fullScreenContainer.setBackgroundColor(TRANSPARENT);  
      fullScreenContainer.setVisibility(View.VISIBLE);  
      itemView.animate()
              .x(DELTA_TO_CENTER_X).y(DELTA_TO_CENTER_Y)  
              .scaleX(DELTA_SCALE).scaleY(DELTA_SCALE) 
              .withEndAction(setTransitionToRecyclerView()).start();
          }  
}

You get the overlay from the fullScreenContainer. You are going to clear the state of that overlay because you do not know if previously another item was zoomed in. Then you are going to add a new item. This parameter itemView was in onClick; that is the perform click that we were doing before on the itemTouchListenerDispatcher. We animate that itemView, and then animate to fullscreen. withEndAction is going to be setting the click listener so we can go back to the gallery, to the mediumRecyclerView.

Let’s take a look at that method.

private Runnable setTransitionToRecyclerView() {  
    return new Runnable() {
        @Override
        public void run() {
            fullScreenContainer.setBackgroundColor(BLACK);  
            fullScreenContainer.setOnClickListener(new View.OnClickListener() {
                @Override
                public void onClick(View v) {
                    overlay.add(itemView);  
                    fullScreenContainer.setBackgroundColor(TRANSPARENT);  
                    itemView.animate().x(originX).y(originY)
                            .scaleY(1).scaleX(1).withEndAction(  
                            new Runnable() {
                                @Override
                                public void run() {
                                    overlay.remove(itemView);
                                    fullScreenContainer.setVisibility(View.GONE);  
                                }
                            }).start();
                    }  
                });
            }  
        };
}

That method has more things, like setting the background to BLACK, setting the background to TRANSPARENT, and setting the visibility to GONE. The idea is you are just using the overlay to do your transitions. At the end of the state, when the transition is done, you add that view to the real parent where we will add all the logic to do whatever a gallery application does on a fullscreen image.

When somebody clicks on that item on that fullscreen image again, we transition back to the mediumRecyclerView and add that item again on the overlay. Then we animate it, set the originX, originY and then scale it to 1 (which is the original size for that image). Finally, do not forget to remove the item from overlay. Otherwise it will still be there until something on the framework decides to recycle that.

That is how easily you can allow the user to interact with a transition without using Shared Element Transition. Just by creating layers, layers on top of it, and just one activity. Then you get this: zoom in, zoom out, click listener fullscreen. Again, this is all magic.

Magic Tricks (34:08)

Magic tricks that you can use for animations:

• ClipPadding and ClipChildren to draw over parents and paddings. If you see something is cutting your views, it is probably one of those are not set to false.
• The ViewOverlay, best friend forever for animations. It allows you to draw over the whole layer hierarchy, and to do any type of things without messing with it.
• Shared element transition does not allow the user to control the transition without events. You can allow the user to do that by just creating a single activity, and then creating your own navigation, which does not have to be complex. It should be just show, hide, and maybe something like animate to next, or enter, or exit the screen.
• Finally, fast animations hide any problem with implementation. If you do it fast enough, it will look nice and smooth.

Remember: make it fast enough the eyes are not able to see the trick, which is pretty much what magicians do.

If you want more examples of smoke and mirrors, check out these great applications by Nick Butcher’s Plaid.