Skip to main content

Android UI Internal : UI Composition with Hardware Composer (HWC)

We ended last article with the statement that to reduce  power and increase  performance, surfaceFlinger will offload layer composition to display hardware. This is the so called Hardware Composer. How this offloading works and what is the benifit can be illustrated with following diagram.

composition using GLES and hardware composer

In case 1, all the layers are composited using openGL. The composited image goes to display controller and finally the phone screen.

In case 2, layer 1 goes to display controller directly while layer 2 & 3 are composited by openGL; the display hardware will then blend layer 1 and the composition result of layer 2 & 3 and finally output it to phone screen. 

Apparently, to enable case 2 more sophiscated display controller was needed so as to be able to accept two input and to blend them. What's the benefit to make this added complexity pays off? 

In the first case, every time layer 1 updates, openGL composition will be trigged for all three layers even layer 2/3 does not change; However, in the second case, if only layer 1 updates, it will goes directly to display controller and no openGL composition will happen. Keeping the openGL/3D hardware idle as much as possible is THE most important thing to ensure a long batter life which our customer will be happy with. In addition to this power benefit, composition using display controller is usually more performant than using openGL since display controller can do it parallel with dedicated hardware. To sum up, using hardware composer, we have decreased power consumption and increased performance. 

Whether a layer could be composited using hardware composer depend on the hardware capability and layer's attributions. Nowadays, smart phone's display hardwares support at least 2 simultaneous input, or overlay - a more jargon word.  Google suggests phone should at lease support 4 overlays to cover status bar, system bar, application and live wallpaper effectively. Layer attributions are things like color format(RGB or YUV), alpha value and angle (portrait or landscape). Only when there is available overlay and the capability of the overlay match the the layer attribution, this layer will go to hardware composer.  Usually, video playback using SurfaceView always goes to overlay since its performance and power  is an important KPI (Key Performance Index)  each and every phone strive to get it looks good. Full screen game is another case. 

Check out other articles in Android UI Internal series 

Popular posts from this blog

Android Camera2 API Explained

Compared with the old camera API, the Camera2 API introduced in the L is a lot more complex: more than ten classes are involved, calls (almost always) are asynchronized, plus lots of capture controls and meta data that you feel confused about.

No worries. Let me help you out. Whenever facing a complex system need a little bit effort to understand, I usually turns to the UML class diagram to capture the big picture.

So, here is the class diagram for Camera2 API.

You are encouraged to read this Android document first and then come back to this article, with your questions. I'll expand what is said there, and list the typical steps of using camera2 API. 

1. Start from CameraManager. We use it to iterate all the cameras that are available in the system, each with a designated cameraId. Using the cameraId, we can get the properties of the specified camera device. Those properties are represented by class CameraCharacteristics. Things like "is it front or back camera", "outpu…

Java Collections Framework Cheat Sheet

Java Collections Framework (JCF) implements the Abstract Data Type  for Java platform. Every serious Java programmer should familiar himself on this topic and be able to choose the right class for specific need.  A thorough introduction to JCF is not the target of this small article and to achieve that goal you can start with this excellent tutorial . 

Instead, I'd like to
1) Provide an overview of JCF's classes ,   2) Provide a cheat sheet you can post in your cubicel for daily reference, 3) Underline the relationship between JCF's implementation and the data structure and algorithm you learned in your undergraduate course

With these goals in mind, I came up following diagram - Java Collection Cheat Sheet. You can click it to zoom in. There is no necessity for more explanation once your familiar with UML class diagram and have a basic understanding of common data structures.

Android Security: An Overview Of Application Sandbox

The Problem: Define a policy to control how various clients can access different resources. A solution: Each resource has an owner and belongs to a group.Each client has an owner but can belongs to multiple groups.Each resource has a mode stating the access permissions allowed for its owner, group members and others, respectively. In the context of operating system, or Linux specifically, the resources can be files, sockets, etc; the clients are actually processes; and we have three access permissions:read, write and execute.