Editor's note: To view a PDF version of this article, click here.
To date, no protocol has been widely adopted for remote graphics rendering on multimedia-capable embedded applications such as mobile devices. These devices face several challenges that have delayed making such technology possible. Because they are power-limited and resource-constrained, they take too long to respond to user-interface requests. The low bandwidth and unreliable nature of network transport in these environments have also contributed to the low response time. While the challenges are real, they are not insurmountable. A number of solutions have been proposed that look to make thin-client user interfaces possible.
One characteristic all current protocols share, such as X Windows and RDP for desktop PCs, is that the display client requires abundant system resources for display, obtaining user input and communicating with the display server. Another commonality is that the protocols require state to be maintained on the client side. Both features require a "fat" client that works on powerful workstations communicating over a static medium. These protocols fail when the client must execute on a system with limited resources and must communicate over an unreliable medium (e.g., on a mobile network).
To address those issues, thin client protocols have been developed as both proprietary products (e.g., Cyrix's ICA) and offerings in the public domain (e.g., Virtual Network Computing, or VNC). Although implementation of thin clients has been attempted on certain resource-constrained devices, such as cellular phones, their most prominent use has been in PCs and workstations. Thin-client protocols face a number of challenges in the realm of low-power multi-media computing.
High bandwidth is required to allow large amounts of image data to be transported to the remote device, depending on the display resolution. Even with image compression, the volume of transported data can be substantial, as shown in Figure 1. Today's mobile and handset devices support display resolutions up to QVGA dimensions and 16-bit color depth (65,536 colors). As display technology advances, however, the requirement will increase.
How can this increasing requirement for bandwidth be addressed? One strategy is to adopt a protocol that uses image compression (see Figure 2). To allow fast screen updates for real-time remote display, a system implementation of this protocol requires a fast and good compression engine on the client side, a fast decompression engine on the server side and a low-overhead transport protocol on both the client and server. Although some multimedia processors possess fast image decompression engines, many processors are limited by the I/O bandwidth necessary to move frame data from the screen capture to the codec and then from the codec to the transport software.
To overcome that problem, some designers are turning to primitive-based protocols (see Figure 3), which take an entirely different approach for remote displays. One such protocol, the Telematic Interface to Graphics Rendering Engine (Tigress), leverages the video capabilities of the target processor. With Tigress, the server transforms display objects on the screen into a list of primitive graphics requests that are sent as packets over the communications medium. For example, if the screen contains GUI windows with text, the graphics requests may consist of commands to draw rectangles, render fonts and the like. The client receives the request packets, parses the packet data for the commands, executes the commands by invoking its graphics engine and returns a response packet to the server.
Since the transported data only consists of encoded commands and arguments, primitive-based protocols consume significantly lower bandwidth than the image-based approach.
Although architecturally different, both primitive-based and image-based thin-client protocols must choose from a common set of algorithmic features:
Compression scheme. Image-based protocols may choose from various compression schemes depending on network conditions; primitive-based protocols may apply compression to bitmap data.
Can you see me now?
Since the client is required to reproduce an exact rendition of the server's display, thin-client computing requires error-free communication. This requirement is even more challenging in mobile and wireless communications, where noisy communications channels are the norm. For image-based protocols, bit errors that occur during unreliable transport may result in rendering errors ranging from insignificant, with mistakes in certain pixel values, to catastrophic, involving the loss of entire image frames. The effect of bit errors might be even worse when image compression is used.
Reliability is even more critical for primitive-based protocols. Data loss can lead to commands not being executed, causing incomplete display on the client. Data corruption is easy to spot. It can lead to bad execution of commands, resulting in a remote display that does not match the server's display.
So, how can reliability be incorporated into the communications protocol? On IP-based networks, the Transmission Control Protocol (TCP) is used to provide end-to-end, reliable transport. The main advantage of TCP for thin clients is that it is self-clocking; i.e., the server may issue requests as fast as acknowledgments are received. On an error-free communications medium, that would result in a fast update rate. On an error-prone channel, however, the remote display is refreshed more slowly.
The main disadvantage of TCP for a multimedia processor is that it may be too "heavyweight," requiring several kbytes of memory and consuming a significant load of the processor. The primary shortcoming of using TCP over mobile networks is that disconnects may be frequent and the cost of reconnection high, since the three-way handshake must be repeated for each reconnect.
To support "lightweight" communication, Tigress uses a much simpler protocol than TCP, called the "Go-back-N ARQ with sliding window" protocol (see Figure 4). The protocol, similar to the V42.bis modem protocol, allows the server to transmit a burst of requests, of window size N, to the client without waiting for an acknowledgment response (ACK). In the last (N - 1) request, the server must solicit an ACK from the client.
The server starts a retransmission timer. If the timer expires without the server's receiving an ACK, it retransmits all N packets; otherwise, the server continues transmitting subsequent requests. If the client receives an out-of-sequence request (as is the case when data is lost), it transmits a response packet with negative acknowledgment (NAK), indicating that the server must retransmit all the packets in the window since the last acknowledged packet.
The Tigress protocol eliminates buffering on the client side, only requiring a simple state variable to be maintained for the last ACK. Like TCP, this protocol will result in fast screen updates on error-free communications channels, lesser on those that are error-prone. An improvement over TCP is that the cost of reconnection is much lower, only requiring that the ACK state variable be re-initialized. The end result is a remote display that matches the local display.
Thirst for power
Because of tight resource constraints, low-power multimedia devices must be treated as ultrathin clients for remote graphics rendering. Mobile devices do not have the luxury of large memories and must conserve power by operating the entire system at low clock speeds.
It has been shown that simultaneous usage of many system functional units results in greater power consumption. The contra-positive statement is also true: Lower power consumption can be achieved with fewer functional units being active at the same time. In addition, processor pipeline stalls result in an inefficient use of power.
Another observation is that the use of direct memory access (DMA) can result in significant power saving over programmed I/O. Because the processor is not involved in performing I/O, DMA has been observed to reduce power consumption to as much as 20 percent.
So, how do these tenets of low power consumption apply to thin-client applications? As detailed earlier, image-based protocols perform I/O to move data between the communications peripheral, the decompression engine and the display peripheral. To achieve lower power usage, the multimedia processor should use DMA to copy the data between its peripherals and memories. On most platforms, DMA is a no-brainer, since it also performs I/O faster than programmed I/O. Not all peripheral interfaces (especially network interfaces), however, are equipped with DMA.
Image-based protocols also require either a software or hardware decompression engine and, so, also incur a power penalty. In addition, image-based, thin-client software must strike a balance between the speed of image decompression and processor clock speed.
Primitive-based protocols tend to be computationally intensive, since they require the execution of graphics requests. As with the image codec, a graphics software engine yields far fewer milliwatts than its hardware counterpart, again at the cost of processor consumption. The crucial element in the primitive-based protocols is optimal execution of the graphics engine.
In the end, the decision as to which approach to take boils down to factors such as system efficiency, the characteristics of the communications channel, the display parameter and, of course, power.
Related Article
"Understanding The MAC Impact of 802.11e—Part 1"; www.commsdesign.com/design_corner/OEG20031029S0009
About the Author
Rajesh Mishra (rajesh.mishra@analog.com) is a senior software engineer at the Digital Media Technology Center of Analog Devices Inc. (Norwood, Mass.). He holds a BS from Rensselaer Polytechnic Institute.
