Testing Information and Support

HbbTV Test Suite

The HbbTV Test Suite provides a set of test material to enable implementers to test receivers against the HbbTV specifications and other associated specifications (such as OIPF). It is suitable for suppliers of complete receivers and also suppliers of hardware and software components.

There are 3 releases each year of the HbbTV Test Suite, -1, -2 and -3.

-1 is when most changes are expected and is not to be used for certification. New tests can be added in this release.
-2 is the most important of the year and is intended for certifying devices entering the market the following year. New tests can be added in this release.
-3 fixes bugs and no new tests can be added in this release. It can also be used for certifying devices entering the market in the following year.

Currently the Test Suite includes tests for ETSI Specifications ETSI TS 102 796 v1.1.1, v1.2.1, v.1.4.1 (which includes v1.3.1) and Tests for HbbTV Specification v.2.0.2.

Anyone can access the HbbTV Test Suite through an HbbTV Registered Test Center. A list of these can be found in the following section.

HbbTV members who have the necessary expertise and equipment may also license the test suite for use in their own facility. The test suite license covers a whole company, including associated companies, and a list of those who have signed it can be found here. The test suite is distributed with details on how to obtain a new test harness which provides minimum required functionality to execute approved test materials in a manual way. Commercial test tools which implement the test harness specification are also available.

This file contains an indication of the contents of the test suite. The precise contents at any particular time may differ due to tests being removed and/or reinstated through the Test Material Challenge Process.

Additionally, as explained in this Presentation HbbTV also publishes a list of only the approved core HbbTV Test Suite coverage in the “HbbTV Test Assertion Specification”.

The precise contents of these lists at any particular time may differ due to tests being removed and/or reinstated through the Test Material Challenge Process. In case of any conflict, the list in the Test Assertion Specification takes precedence.

HbbTV Members wishing to license the HbbTV Test Suite should leave contact details here:

HbbTV Registered Test Centers

HbbTV Test Center
HbbTV Registered Test Centers offer HbbTV testing services to the market, for those manufacturers, operators or integrators who do not wish to do HbbTV receiver testing in-house. The process for a test center to register with HbbTV is lightweight with relatively few obligations, in return for which the test center can be added to the list below.

Manufacturers do not need to become an HbbTV Registered Test Center if they are only testing their own HbbTV products.

Selecting a Registered Test Center

HbbTV Registered Test Centers offer HbbTV testing services to the market, for those manufacturers, operators or integrators who do not wish to do HbbTV receiver testing in-house.
The process for a test center to register with HbbTV is lightweight with relatively few obligations, in return for which the test center can be added to the list below.
Manufacturers do not need to become an HbbTV Registered Test Center if they are only testing their own HbbTV products.

HbbTV cannot recommend a particular test center nor does it audit or accredit test centers for regulatory compliance. The following criteria may be considered when selecting a test center. It does not include other criteria covering any commercial relationship that would required with the test center.

Reputation	Is the organisation impartial and trusted in the industry? For example you may wish to request references or evidence of previous relevant work.

Technical Expertise	Does the test center employ testers with HbbTV knowledge; and appropriate qualifications?

Confidentiality	How does the test center ensure confidentiality of test reports and test samples?

Efficiency	Does the test center have good procedures for managing bookings and conduct testing efficiently and quickly?

Responsiveness	How will the test center provide swift support and answer questions in a timely manner?

Facilities	Does the test center have all necessary equipment to perform the range of testing required?

Awareness	How does the test center track changes in published specifications and test suite releases? A list of the approved tests is published above; are they using the latest version of the test suite?

Deliverability	Ask for an example test report, to determine if this meets the requirements of the client and any 3rd party regulatory body. Can this report be verified for authenticity?

Support	Will the test center assist in debugging any test failures? What is their re-test policy? Will they help with challenging any tests that are felt to be in error?

Requirements and procedure to become a Registered Test Center
- Candidate Registered Test Center must be a current HbbTV member and must have been member of HbbTV for at least six months prior to their application to qualify for Registered Test Center status.
- Candidate Registered Test Center must have signed the HbbTV Test Suite License Agreement.
- Candidate Registered Test Center must complete a questionnaire which will be made available on request. This request must be sent to:
  - HbbTV c/o European Broadcasting Union, L’Ancienne-Route 17A, CH-1218 Le Grand-Saconnex, Switzerland
  - Email: info@HbbTV.org
- Upon receipt of the request to become a Registered Test Center, the questionnaire as well as a reference copy of the Test Center Agreement (TCA) will be sent.
- Registered Test Center will be listed on this website.
List of Registered Test Centers
DTG Testing Limited
DTV Accredited Compliance Lab (part of Samsung Electronics)
Kineton SRL
Ocean Blue Software Ltd.
Resillion
Samsung Electronics HQ
Sofia Digital Ltd.
Sony Europe B.V.
Vestel Elektronik A.S.

HbbTV Testing Publications

The attached schema files have been developed by the HbbTV Association to capture test case requirements, to document the development and approval process for each test case, and to enable implementation of test cases using a HbbTV conformant test harness. These schema are published to enable other groups to use the same schema for development of their own test cases in a way that will maximise interoperability of implementations.

This work is licensed under the Creative Commons Attribution-NoDerivs3.0 Unported License. Creative Commons can be reached at 444 Castro Street, Suite 900, Mountain View, California, 94041, USA

Test Specification Test Case Test Case Results Test Implementation Playout Set Definition

HbbTV Test Repository

HbbTV Test Review
HbbTV welcomes test reviewers to help qualify whether submitted test material is fit to be included in an official Test Suite.

HbbTV and OIPF test material is collected and tracked in a common HbbTV Test Repository powered by SVN. The test material is not for testing or validating devices and such use is prohibited. The HbbTV Test Suite is provided for testing devices against the HbbTV specification and the OIPF test suite is provided for testing devices against the OIPF specification.

During the “Test Review” process, any issues found with submitted test materials are tracked using Redmine. HbbTV and OIPF have anticipated that issues may be encountered even after an official Test Suite is released, and a “Test Material Challenge Process” exists, again tracked by Redmine.
Who can be a Test Reviewer?
The process for developing and reviewing test materials is shared by two standards bodies, namely HbbTV and the Open IPTV Forum, since test specifications have significant overlap. A test reviewer can be a member of either body (or both), a subcontractor of said member or an approved test material provider.
The details of how test material is reviewed and approved, as well as how test reviewers are selected for particular tests is given in the Test Material Approval Procedure document which is available to companies that become test reviewers.
How to become a Test Reviewer
In order to become a test reviewer, your company will need to conclude the Test Repository Access Agreement (TRAA). To start this process, please send an e-mail to info@hbbtv.org, clearly stating your company wishes to get access to the HbbTV Test Repository.
In case your company is a member of HbbTV and/or OIPF, please clearly state this in the e-mail. Please note that such membership will be checked as part of the application procedure.

After the TRAA is signed, individual user accounts will be created to obtain access to the SVN Test Repository and the Redmine issue tracker.
Getting access to the SVN Test Repository
Access to the HbbTV test repository will first be approved per company. Presently only one SVN account per member company is permitted, however multiple individuals at that member company can use SVN with that account.

To request access to SVN send an email to support@hbbtv.org stating the name of the member company and the email that you would like the SVN account invitation email to be sent to. This invite will give you a link to complete your account creation.

Further details on how to connect to the repo will be given following a successful account creation
Getting access to the Redmine Issue tracker
If you do not have an account on the Redmine server, you can register for one by going to the following URL: https://redmine.hbbtv.org/account/register

When registering with Redmine, as with the email lists, please only use email addresses clearly linked to your company details given in the TRAA application questionnaire. Yahoo, Google, Hotmail or e-mail addresses that have no company affiliation are not acceptable.

When you have registered with Redmine, your account will be activated manually by the Redmine administrator. In case you have any questions around your account please contact support@hbbtv.org.

Testing Information and Support