- Introduction into the UPnP Standard
- Coherence Framework
- Appendix A: Abbreviations
- Appendix B: Full interoperability test protocol
- Appendix C: Source code
The UPnP standard is already around for many years. It was developed by Microsoft to reduce the effort that is needed to configure devices attached to a network. But it never became popular until the Media Center PC's and the first UPnP enabled Set-Top-Boxes for TV's started to be sold a few years ago.
Nowadays one can find UPnP functionality in many places. The most popular implementation (besides the Internet Gateway Device used in DSL/Broadband-Routers) is the UPnP Audio/Video standard. It enables the user to share pictures, audio and video files on one device using an UPnP A/V server and view them on another device using an UPnP A/V renderer. Every major Multimedia product today features UPnP capabilities. Even though there are many different libraries and implementations of the UPnP A/V specification, they are not guaranteed to work together. Therefore interoperability is a major problem when using devices from different vendors.
The aim of this thesis is to implement a Test Suite that is able to test UPnP devices against the specification of the UPnP Forum. This is not limited only to test UPnP Audio/Video devices but to test UPnP devices of any flavour. Having the ability of being extended it can be of use even for developing a new kind of UPnP device and test it before the specification is handed in to be tested by the UPnP Forum to become a standardized official device specification.
I always found it comfortable to have a network that connects all the computers at home so everybody can share a music track for example. However the others who want to listen to that track always have to know exactly, on which computer and in which directory the file resides and it takes a lot of clicks to get there. Of course, you can ease the search process a little by using clear and understandable directory names but still, all the users have their own idea of a “good” directory structure hence it still remains somewhat uncomfortable.
When I first heard about UPnP and its capabilities, I was impressed. With UPnP A/V infrastructure everything would be a lot easier. Everybody that is connected to the network is able to run a server that offers all the files that should be shared. Now if someone else on the network is looking for images, music or videos (media files in general), this user just needs to install an UPnP A/V client software or has to buy a hardware box that is able to “speak” UPnP respectively (this software and those boxes usually consist of a control point for browsing and issuing commands to the server and a media renderer which actually shows the requested content). When the client is started it searches for an A/V server that offers content. If it finds one, this server is queried for the available content and displays the results on the users screen. There is no need to know who is offering those files and where they are located. This works even without having any meta data available in the shared files, but the navigation especially in a large music collection is a lot easier if you can browse it by artist, album or title.
Knowing this and being still curious I tried to find an UPnP A/V server that I could install to share my media files with everybody connected to my network in an easy way. I found some open source software for Linux, a server as well as a client. After some testing it worked, but some strange errors appeared while using this setup. So I tried another server and the previously seen errors disappeared in favour of other errors. After trying a few combinations of servers and clients I was disappointed. If all the devices would obey the according UPnP specification they should work together without any problems. Well, obviously they do not. Hence the only solution at this time was to find a reasonably working combination of server and client hoping it will get better.
Out of this interest I had the chance to get a traineeship at Nokia for three months in the late summer of 2006. I tested UPnP interoperability of the N800 against all the UPnP media servers I could find. The results were disappointing as well. Initially the goal was to change the source code of the client to speak to all the servers but I soon realized that it will be very difficult to make the client as flexible as possible to be able to talk to as many different servers as possible. Also the other way round it would not be easy to program a flexible server that is able to speak to all the clients.
So my first idea after this experience was to program a fully standard compliant media server. It should have no exception for any non-standard-conform client so every client can be tested against it. However, would it change the situation, if there is a server that only speaks to less than a handful of standard compliant clients? Would developers use it to test their devices and programs respectively? I guess it would not change anything and nobody would use it. So why not help the developers of these devices to validate their clients or servers against the specification instead of competing with their products? Therefore I discarded the idea of programming a fully standard compliant media server and came up with the idea to program a test suite to enable all developers to test their devices, be it an UPnP A/V client, an UPnP A/V server or another UPnP device.
The general idea is to make the test suite able to speak to all possible UPnP devices. This can be done by reading the XML file that defines the structure of the device and the services it offers. By doing this the program also knows which commands it can send and which answers and status changes it can expect after the device finished processing this command. Using this approach it can also deal with devices that are not specified or standardized yet. On the other hand, the test suite has to be able to perfectly simulate another device, at least to certain extend, so it cannot only test the responses of a tested device but also if the device itself is issuing correct commands and how it acts if the responses sent are invalid.
This is the point where my test suite is going to be more advanced than the Intel Tools for example. The Device Spy (that is part of the Intel Tools) can also search for UPnP devices in the network, retrieve the description and issue queries on the functions offered by the device. That is very useful for the first testing of a new device, of course. But what it cannot test is a complete infrastructure with devices interacting with each other. To test a new implemented control point, for instance, it needs a media server and a renderer because a control point is talking to both of them, in a way interconnecting them. Instead of having two programs running and capturing the traffic with a network analysing tool which is not very comfortable, the test suite can emulate both. Even more it can simulate uncommon cases where for instance one of the devices disappears while the control point is talking to it.
To ease the testing of these more complex scenarios I planned to enable the test suite to execute scripts. Since the suite is going to be implemented in Python itself it should not be difficult to create an interface that an external Python script can use to automate the testing procedure. In a way this scripting can be seen as unit testing that spans over multiple devices.
Concluding my motivation I also want to mention that the test suite I am developing during this thesis is going to be released as Open Source Software (OSS) afterwards. Thus it will be free of charge and therefore I hope the developers are more motivated to use it as they would be if they or their companies have to pay for it.
Introduction into the UPnP Standard
This chapter introduces the UPnP history, its technology and the ideas behind. It starts with a general overview of UPnP followed by a listing of all the techniques that are utilized to make an UPnP environment work and to conclude the introduction, there is a listing of all the devices that are already defined by the UPnP working committee.
What is UPnP™ and what is the UPnP™ Forum?
As written in [UPNPForum03], “UPnP™ technology defines an architecture for pervasive peer-to-peer network connectivity of intelligent appliances, wireless devices, and PCs of all form factors. It is designed to bring easy-to-use, flexible, standards-based connectivity to ad-hoc or unmanaged networks whether in the home, in a small business, public spaces, or attached to the Internet. UPnP technology provides a distributed, open networking architecture that leverages TCP/IP and the Web technologies to enable seamless proximity networking in addition to control and data transfer among networked devices.”
The UPnP Device Architecture (UDA) offers means to integrate a device into a network, to automatically discover other predefined device types that are connected to the same network, to learn about their abilities, to propagate the devices own abilities and finally to leave the network after telling the other devices it is doing so.
The technologies that are used for creating this powerful architecture are well known and well proven Internet protocols like IP, TCP, UDP and HTTP alongside the newer but rarely used protocols SSDP (Simple Service Discovery Protocol) and GENA (General Event Notification Architecture). For the data transport between the devices the data is encoded in XML using the SOAP format and transported over an HTTP connection. All those techniques were designed to span across different physical media and across system borders so they fit perfectly in the UDA which is designed to accommodate home and office networks. The “universal” in the name also shows this because instead of using special drivers for connecting those devices common protocols are used making the implementation independent of programming language and operating systems. Also are vendors not limited to the given APIs but they can extend the standardized ones or even develop their own APIs that suits their customers’ needs best.
The standardized device specifications are developed by the UPnP Forum. Its members are companies that use UPnP in their products and seek to develop standards for describing protocols and XML-schemas for devices to enable interoperability between devices based on those standards. For this working committees are built consisting of employees of the member companies that have expertise in designing and specifying a certain kind of device. Such a committee then has to create a specification, build sample implementations and build appropriate test suites. UPnP vendors now can build devices based on that specification and be confident of the interoperability with devices from other vendors that are also built following that specification. In addition the device is allowed to bear an UPnP logo if it passes a series of tests and if the test results were checked by a Test Reviewer. This enables the customer to recognize standard compliant devices.
Techniques used by the UDA
The UPnP device architecture is based on a number of protocols and formats that need to be known to the reader of this thesis. The following part gives a short introduction into each of them and mentions how this protocol or format is used in the UDA. Further information can be found in [Internet_protocols].
The Internet Protocol defines a globally unique address for each host. Using this address as destination address other hosts can send data packets to this host. If the communicating hosts are located in different networks, routers can determine the way to send the packet to deliver it to the destination host.
All devices in an UPnP network will first try to get an IP address by different means (i.e. using the DHCP, see below). As soon as an address was acquired the device can start communicating to other UPnP devices.
The Dynamic Host Configuration Protocol enables a device that gets connected to a network to automatically obtain an IP address from a DHCP server so that the user who is using the devices doesn’t have to bother about a correct configuration. Besides the IP address a device can also receive information about the subnet mask, the default router and the IP addresses of the DNS servers.
For an UPnP device this is the preferred way to get an IP address. However if a DHCP server is not available the device falls back to ZeroConf? protocol.
The Zero Configuration Networking is a way towards just using a network without having to configure it. It is mainly a set of rules how to use normal IP traffic to achieve the goal. When a ZeroConf enabled computer is attached to a network without a DHCP server it assigns itself an IP address based on the rules.
ZeroConf is only used by the UDA as a last resort for getting an IP address. As soon as a DHCP server becomes available the device has to release the IP address and use the one assigned by the server.
The Transmission Control Protocol enables a host to distinguish between several simultaneous connections established to it using a port number so it can run several services using only one IP address. This protocol also keeps track of the order in which the packets were send and if all of them reached the destination. If not the sender will be notified or if no notification arrives the sender just resends all the data that was send after the last acknowledgment. The packet size is chosen based on the amount of data that is to be transferred but is usually delineated by the maximum transmission unit (MTU) of the network the device is attached to.
The TCP is used in combination with the IP (therefore often called TCP/IP). In this combination UPnP devices use the protocol to interact with other UPnP devices.
The User Datagram Protocol works almost the same like the TCP but it does neither guarantee that the data packets will reach its destination nor that the packets will arrive in order. Since it does not have the control overhead it is small and fast hence suitable for purposes where many but short messages have to be transferred without the need of an acknowledgment for the arrival.
Together with the HTTP UPnP devices use the UDP for the transmission of SSDP messages (see below).
The Hyper Text Transfer Protocol is probably the most well known amongst all the used protocols, initially just designed for delivering HTML pages but in the meantime methods for sending data back to the server were introduced.
In the UDA it is mainly used as underlying protocol for SSDP, SOAP and GENA. They extend the standard set of commands with their own commands to perform different tasks. It is also used for the streaming of media content even though it was not developed for that purpose.
The Simple Service Discovery Protocol offers a means to discover network services and enables the servers to notify other devices about their services. It uses unicast or multicast UDP packets to achieve its functionality since the needed information is very small and the delivery information is not necessarily needed.
The UDA uses SSDP for all discovery related communication and is the only widely used implementation of this protocol.
The General Event Notification Architecture defines an HTTP enhancement that is used to subscribe to events and to transmit event notification between HTTP resources whereby a resource can be any device or service that needs to send or receive event notifications.
Like SSDP, GENA is actually used by the UDA only. UPnP devices use it to subscribe for events and to inform subscribed devices about state changes.
With the SOAP protocol XML-based messages can be transmitted in a network, usually using HTTP but other protocols like SMTP are also usable. It is mostly used to define an API for Remote Procedure Calls (RPC) that can be used by other devices in the network. A drawback of using XML as transport format is the size of messages: even though they are human readable, it needs some computing power to parse and process them which makes it not suitable for very small devices like microcontrollers. Hence it is really difficult to build a SOAP enabled device with such limited resources.
UPnP devices use SOAP to invoke actions on a service a certain UPnP device is offering.
The Extensible Markup Language is used to represent data in elements that are embedded in a tree hierarchy. A schema defines rules on how the elements have to be structured and what kind of data an element can hold. The advantage of this approach is that the data become readable for humans while remaining readable for a program.
The XML format is used for all the data transport in SOAP communication.
- Basic Device V1.0
- Home Automation
- Digital Security Camera V1.0
- HVAC V1.0
- Lighting Controls V1.0
- Internet Gateway V1.0
- WLAN Access Point V1.0
- Printer Enhanced V1.0
- Printer Basic V1.0
- Remote UI Client V1.0 and Remote UI Server V1.0
- Scanner V1.0
- Add-on services
- Device Security V1.0 and Security Console V1.0
- Quality of Service V1.0
- Quality of Service V2.0
This chapter covers the three UPnP Audio/Video devices specified by the UPnP Forum. Together they build an infrastructure for distributing media content throughout a local network at home without the need of a complicated setup.
That is the place where all the media files are stored that are going to be published in the network. After the activation of the MediaServer it scans all resources (usually the file system) for media that should be shared. To ease the selection of media for the client/user the server has to provide some metadata about the shared files. Hence it needs to read all the shared files and index them according to the data found in ID3 tags, AVI containers etc. The collected data can be browsed or searched by the client/user. If a file matches the criteria of the client/user it is requested from the MediaServer and it starts to stream it to the device rendering the content.
A MediaServer can be a computer with MediaServer software installed (i.e. [TVersity], see page 15), an intelligent network storage with a tiny MediaServer built-in (i.e. [TwonkyMedia?], see page 14) or another device that offers its data via an UPnP interface (i.e. [Nokia N80]).
The renderer takes the data that is streamed by the server and outputs it according to the media format. After activation and announcing itself in the network a MediaRenderer is waiting for a ControlPoint that is issuing commands to request and play a file. If such a command is received the renderer connects to the given MediaServer and retrieves the file. The data is decoded and sent to the output device, screen and/or speakers. While the playback is running the ControlPoint can still affect it and change for example the volume for audio playback or the contrast for video playback.
MediaRenderers are rarely sold on their own. Typically renderer and ControlPoint are integrated within one device. They are available as computer programs, often in form of a MediaCenter? (i.e. [NeroHome?], see page 16), as audio clients having a WLAN connection (i.e. [NOXON 2 audio]) or as fully featured client that is playing every media type (i.e. [Buffalo LinkTheater? PC-P3LWG/DVD]).
The ControlPoint connects MediaServer and MediaRenderer. It usually has a graphical interface that shows all available servers in a tree structure and when the user selects one the content of this server is displayed. Now all the content can be browsed, typically by file name, artist, song name, etc. If a file was chosen and more than one MediaRenderer is available the user has to choose where to play the file. After making the choice the URL of the chosen file is sent to the chosen renderer who in turn requests the actual file using this URL. When the URL is sent to the renderer it does not automatically start to play. The ControlPoint initiates the playback using a control command. As said before in the MediaRenderer section, the user can control all common parameters like volume, brightness or colour temperature using the interface of the ControlPoint.
ControlPoint software without an attached renderer is pretty rare. The only two I know, are Intel AV Media Controller (see page 16) and Cidero MediaController? (see page 16). These two are only ControlPoints without the ability to render any content. An interesting use case is the MediaStreamer? for the Nokia N800. It is a full featured ControlPoint with an internal renderer. While other combined devices can just play the content on the integrated renderer, MediaStreamer? allows the user to control another renderer that is playing some music track while listening to the next track using the internal renderer.
Today’s multimedia market offers lots of devices that feature UPnP capabilities. There are Broadband-Routers with a built-in hard drive that offer the media content on that drive via UPnP or programs like Windows Media Player which include UPnP MediaServer functionality as well. On the other end you can buy DVD players that cannot only play DVDs but that are also able connect to your UPnP MediaServer through a wireless LAN connection. Last but not least there are simple clients that just serve the purpose as mobile music player while they retrieve their music form an UPnP server as well.
It would be interesting to test all these devices against each other but my resources in time and money are limited. Therefore the following tests will only cover software servers, control points and renderer.
Software for testing
All the software that is going to be tested is either open source and hence free available or it is a trial version of a commercial product.
UPnP A/V MediaServer software
- uShare is a small UPnP media server that runs from the command line or as a daemon in a Linux or Unix environment. It just publishes the directory structure of the directory that was passed on the command line when starting uShare. It does not scan for meta data. Hence it is basically like sharing files with shared folders in the network but it eases the discovery of those files utilizing the UPnP architecture.
- GMediaServer is the software uShare is based on. Therefore it behaves pretty much the same as uShare does except that it only shares audio files. Video and image files are ignored when scanning the shared directories. As uShare it runs form the command line of a Linux or Unix system.
- MediaTomb is a bit more advanced. It scans the given files and creates a folder structure that can be browsed by the client. The four top folders are Audio, Video, Photos and PC Directory, with the latter showing the files in the folder structure like they are stored on the server (similar to the uShare behaviour). To be more compatible with UPnP clients it offers some switches to change the behaviour when sending responses to a requesting client. It also runs from the command line of a Linux or Unix system.
- TwonkyMedia is the software that is used on most embedded devices such as Network Attached Storages for the home network. Therefore it is especially interesting to test the interoperability of this software because it shows as well how interoperable these devices are going to be. TwonkyMedia runs on a number of different architectures. My tests were conducted on a Windows system.
- Intel AV Media Server is part of the Intel® Tools for UPnP Technologies. It is also pretty simple and makes audio, video and images files available on the network. The goal for developing this was to have an easy to use Media Server for testing and therefore features like transcoding and so on were not implemented.
- SimpleCenter is another Windows program for sharing files via UPnP. However, this functionality is only available in the Premium version which is also shipped with Nokia N80 phones. With SimpleCenter you can organize and sort your media files and this sorting is also exposed in the UPnP browsing tree along with the common metadata structure.
- Windows Media Connect was available separately until version 10 of Windows Media Player. Now the user is able to share directly all the files which are managed in the library of WMP.
- TVersity is one of the most versatile MediaCenter one could imagine. It shares all media files in the most common formats and it even transcodes the media if the player that accesses the content is not able to play in the format the file is stored in. Also it manages URLs for media on the internet or RSS feeds for podcasts or videos for example. All this is offered to the clients via UPnP and served by a Windows Service that runs in the background.
- Nero MediaHome is part of the Nero 7 software suite and an officially certified UPnP™ AV (Audio / Video) Media Server. Like most of the other programs it shares all types of media files and it also features a transcoding function. An interesting feature is the TV streaming. If the installed TV card is supported it records the selected TV stations and offers the recorded broadcasts via UPnP.
UPnP A/V ControlPoint/MediaRenderer software
- Cidero MediaController is an UPnP A/V ControlPoint written in Java. It discovers media servers and media renderer and shows them in the user interface. The user can select a media server, browse it and pass the URL of the selected file to a renderer for playing. Developed in Java it runs on every platform where a Java Runtime Environment is available. My Tests were conducted on a Windows PC.
- Djmount is a pretty simple UPnP client. It allows the user to mount the directory tree offered by a media server in a certain directory in the file system tree of Unix or Linux system. The metadata offered by the server is used to populate the file properties. Thus djmount acts as a bridge between the UPnP network and local programs that do not understand how to access resources shared via UPnP.
- The MediaPlayer on the Nokia N800 has a media library which shows all the available audio and video files on the device. Images are not listed. This library is also able to retrieve content from UPnP A/V servers. Since this device runs Linux, the MediaPlayer is a Linux program as well.
- Nero ShowTime is the player application of the Nero 7 software suite. It plays several different audio and video formats but it does not show images. Besides opening local files the user can browse a UPnP media server and add files from there to the play list.
- Nero Home is a MediaCenter application and also part of the Nero 7 software suite. As a MediaCenter it can be used with a remote control and offers an easy-to-use interface. Using the category Devices in the main screen shows all available devices of the computer it runs on. Additionally it shows an entry called “MediaHome Network”. This offers an interface to all available UPnP A/V servers connected to the network and lets the user browse through the offered content. Nero Home plays numerous formats of video and audio files. Images can be viewed in a slide show.
- MediaStreamer is another application running on the Nokia N800. It is focused on the UPnP architecture of server, control point and renderer. The intended usage is to transform the N800 into some sort of remote control that browses the content of a media server and controls an external renderer that is showing the selected content. While this external renderer is playing a music file for instance the user can switch back to the built-in renderer, look for the next song and pre-listen to it. Then it can be added to the play list and will be played next on the external renderer. This also works for images. Videos can only be played on an external renderer. The built-in renderer does not support video playback yet.
Network traffic analysis tool
Wireshark (formerly known as Ethereal) is a network protocol analyser and according to the website “Wireshark's powerful features make it the tool of choice for network troubleshooting, protocol development, and education worldwide.” It allows pretty advanced filtering of the captured data packets. Thus the user can inspect only the relevant ones without being distracted by the huge amount of other packets that are usually exchanged over the same Ethernet connection. Besides that Wireshark is Open Source hence free software and several precompiled binaries exist for the most commonly used operating systems. So I can use it for testing on both platforms, Linux and Windows. Because of these reasons and because I already used Wireshark before, I chose it for the traffic analysis.
|Cidero MediaController (Windows PC)||Intel AV Media Controller (Windows PC)||Djmount (Linux PC)||FileManager (N800)||MediaPlayer (N800)||Nero ShowTime (Windows PC)||Nero Home (Windows PC)||MediaStreamer (N800)|
|Intel AV Media Server||X||X||X||X||X||--||X||X|
- MediaPlayer does not show files, only directories. Request and responses look ok, also HTTP HEAD requests and responses, but maybe it is missing some information so it skips those files.
- Nero ShowTime works, but it caches the whole file before playing it. This results in a pretty long latency before the file actually starts to be played after telling Showtime to play it.
- MediaPlayer experiences the same problems as with uShare. Because uShare and GMediaServer share the same code base this is most likely the same problem.
- Nero ShowTime browses the shares but since it sends a slash after the object ID, the file cannot be played.
- MediaPlayer plays the videos, but does not list the images and the music in the library.
- ShowTime browses until first level below Root and then starts again at Root, and when it reaches second level again, it stops searching and removes MediaTomb from the MediaHome node in the dialogue.
Intel AV Media Server:
- Nero ShowTime browses the shares but since it sends a slash after the object ID, the file cannot be played.
- Intel AV Media Controller cannot handle more than one resource item in the browsing result. It will always use the URI of the first resource item when instructing the renderer.
- NeroHome does not seem to be able to speak to SimpleCenter even at root level.
- Nero ShowTime browses the shares but since it sends a slash after the object ID, the actual files cannot be played.
The interoperability problems that have been found in the last chapter are showing that most of the problems occur in the second part of the communication between the UPnP devices. Discovery and Description are working pretty well meaning all the devices can “see” each other. Just one test showed problems during this phase. But it seems to be very difficult to implement the application specific communication (Control and Eventing) correctly so that the programs quit service in 10 test cases. Especially the browsing for media content was problematic. Therefore this second part of communication is the most interesting one that should be easy to monitor and that should be checked for errors by the test suite.
Mode of operation
As written in chapter 3.3.2 the UPnP A/V architecture consists of three devices, MediaServer, MediaRenderer and ControlPoint whereas ControlPoint and MediaRenderer are often tied together in one device. This leads to four different test cases plus one general test case for all UPnP devices:
- The test suite checks if the tested device obeys all rules for the Discovery, Description and Presentation process.
- A MediaServer is tested and the test suite offers a MediaRenderer and a ControlPoint.
- A MediaRenderer is tested and the test suite offers a MediaServer and a ControlPoint.
- A ControlPoint is tested and the test suite offers a MediaServer and a MediaRenderer.
- A device that combines a ControlPoint and a MediaRenderer is tested and the test suite offers only the MediaServer.
The first test case is very simple to implement. For the Discovery part the test suite just needs to listen to the network. When a device announcement is received, the sending device is asked for the description of its root device. The received XML data will be parsed and if a known (i.e. a specified) device type is recognized the description will be validated against the official KES-450A schema. Also if sub devices are found during the parsing their descriptions are also retrieved and treated the same way as the root device description until the descriptions for all devices have been read. After parsing and checking all the received descriptions the test suite will complain about the ones that could not be parsed or validated against the according schema.
During the introduction into UPnP I wrote there are two ways for a device of getting known of the other UPnP devices connected to the network. The one described above is the first. The second way is also going to be tested. The test suite is broadcasting a request to all UPnP devices connected to the network. If a device is answering it is queried the same way as mentioned above. Actually the device has to answer that request as well as it has to announce itself when connecting to the network. This way it fulfils the requirements of the standard. Otherwise the test suite will complain about that. But to test both ways separately the test suite only recognises connection attempts of the one flavour it is currently testing.
Within the description there is the option to send a presentation URL for the device. Testing the content of this URL would be too difficult or even impossible because this presentation is usually done using HTML. So the only reasonable test is checking the availability of the document at the given URL. This is done at the end of the basic device check.
The second and third case is a bit more complicated. In both cases the test suite offers the two missing devices to complete the UPnP A/V setup. MediaServer and MediaRenderer are passive devices and only offer some services that can be controlled by the ControlPoint. Hence for the third part of UPnP communication, Control, the ControlPoint has to invoke all methods a MediaServer/MediaRenderer has to offer. This includes for instance browsing the content of a MediaServer, transferring the URI to the renderer and start to play the content. If all these functions are available, the answers are parseable and valid then the device is compliant to the standard. After this basic check the user of the test suite is able to start a more comprehensive check that also tests all the other functions the MediaServer/MediaRenderer offered in its description. This could be for example the optional PrepareForConnection(), SetContrast() or SetVolume() function. If they all work as expected the device is also compliant to the implemented optional UPnP functions respectively to the vendor-defined functions that the device sent within the description.
The Eventing, the fourth part of UPnP communication, needs to be checked as well. For doing this, the ControlPoint first subscribes for the events that are required to control the other two devices. That is the LastChange event of the AVTransport component for instance. For being standard compliant a device implementing the AVTransport component has to publish all changes of the internal state variables using this event. If the ControlPoint is ordered to invoke a method that results in a state change, the test suite can check afterwards if the tested device shows that change by sending according information with the next event message. This would be the basic test similar the one for Control mentioned above. In the advanced testing mode the test suite also allows checking events for state variables that are not required to be implemented.
Even though both functional parts were described separately, it absolutely makes sense to combine the tests for Control and Eventing. The functions have to be called anyway for checking Control behaviour and they often result in state changes that can be used for checking for correct Eventing behaviour as well.
The last two cases are even more difficult to test because the ControlPoint is the active part in the UPnP A/V setup. That leads to the only option of Black-Box-Testing . The test suite offers a MediaServer and, depending on the implementation of the other device, a MediaRenderer. Furthermore it reads all traffic that the ControlPoint sends to the other devices and checks the information for errors. The drawback of this method is the need for manual interaction with the user interface of the ControlPoint. The user of the test suite needs to utilise all the functions the user interface offers to maximise the chance that all functions the ControlPoint needs for operation have been invoked. The test suite cannot determine if all functions offered by the MediaServer and the MediaRenderer have been invoked because it does not know if the ControlPoint is capable of invoking all the offered methods. To still give the test suite user a hint of what has been checked, the test suite lists all method calls that have been logged. This way the developer of the ControlPoint can determine whether a certain method has been called or not.
Another problem is the Eventing. The test suite may write them to the log right between all the function calls. But if the ControlPoint acts appropriately to the event messages sent cannot be determined form the outside. So again it is up to the user to look through the log the test suite generated and see if the ControlPoint behaved as designed by the programmers.
According to [Coherence] the Coherence framework aims to help developers to enable their applications programmed in Python to participate in a digital living network. This makes it the perfect choice for using it as the base to build the test suite upon. Currently it is focused on UPnP but it is going to be expanded to other networks as well.
The framework is divided into three parts, the core, the devices and the services that connect the devices to the core. As a fourth part I included the backends even though they are not actually part of the framework they are essential to understand how the framework is intended to be used.
The core combines a SSDP server, an M-SEARCH server, a HTTP server and a HTTP client. This is the typical setup needed for an UPnP device to be operational. As shown in the figure above the core has an interface where all these services are connected to. This way other technologies like DAAP for instance can be integrated into the framework.
Having all the basic functionality in one place allows a lightweight implementation of the other parts of the framework. All the needed basic functions are shared between all connected services and devices so that their implementation is focused on the specific functions.
An UPnP device usually utilises several services to connect to other UPnP devices in the network. A MediaServer for instance needs to offer at least the ContentDirectory and ConnectionManager service (green boxes in Figure 8.2) to be accessible for a ControlPoint and a MediaRenderer. But the other devices in turn have to use these services to access the MediaServer. Therefore this functionality is encapsulated in several service classes with interfaces to the core and to the device classes. This way the functionality encapsulated in a service class can be used by both sides.
A device in the coherence framework represents an UPnP device (like the light orange box representing the MediaServer in Figure 8.2). It knows which services it has to offer and which ones it can offer additionally. During initialisation it takes the services it needs and connects to the core using these services. Together with the basic functions from the core this forms a fully functional device.
The backend connects the virtual UPnP device to the resources that actually show received data or read requested data. To use an Audio/Video example again: a MediaServer needs to have a storage for the files it offers. The easiest way would be to have all the files stored somewhere in the file system. This is what the Filesystem Backend is doing (see Figure 8.1). However this backend is just an example backend and is also used for testing the framework. These backends are not actually a part of the framework because it is only the base to build upon. It is completely up to the user of the framework where the files for the MediaServer reside, how the interface should look like that is driving the ControlPoint and which device is used to output the content the MediaRenderer received.
Appendix A: Abbreviations
a lot of abbreviations will be used, so they are going to be listed and explained here
Appendix B: Full interoperability test protocol
the full protocol of the interoperability testing
Appendix C: Source code
useful parts of the source code