What is a statistical metadata system?
Metadata can be defined as "data that define and describe other data" whereas statistical metadata are "data about statistical data, and comprise data and other documentation that describe objects in a formalised way" (both definitions come from the 2009 edition of the SDMX Metadata Common Vocabulary). Some metadata, however, describe other resources that are relevant to the collection, processing and dissemination of data (for example, questionnaires and publications) rather than relating directly to data themselves, so a slightly wider definition of metadata may need to be considered. With this in mind, the SDMX Metadata Common Vocabulary definition of a Statistical Metadata System (SMS) can be used: "A data processing system that uses, stores and produces statistical metadata". The term system refers to the people, processes and technology involved in managing statistical metadata.
Metadata have two basic functions. The first is to uniquely and formally define the content and links between objects and processes in the statistical information system. The second is to determine all related technical parameters.
With rapidly developing technologies, it can be challenging to implement efficient strategies for the production and dissemination of statistics. The growing use of the Internet has caused significant changes in the priorities for managing statistical metadata. In the past, priority was often given to technical metadata and information technology challenges, whereas now there has been a clear shift to focusing on content and methodological issues. When designing the SMS, priority should be given to the content of the system. The primary focus should be achieving business outcomes and ensuring the expected benefits can be realised through effective governance and processes.
Due to these changes, transparent and integrated descriptions of information flows within and outside the statistical office are vital. The use of technology for data collection, interactive communication with users, and dissemination of statistics, calls for a coherent and well functioning metadata system.
The implementation of an SMS should be independent of the technology used for statistical data processing. However, the links between the SMS and electronic processing systems must be ensured. Processing of statistical data should be driven by metadata stored in the SMS.
What is the role of the statistical metadata system?
The success of an SMS can be measured by the extent to which the needs of diverse groups of statistical metadata users are satisfied. The need for metadata is defined by the various activities, tasks and processes carried out in a statistical organization. Therefore, the role of SMS should be understood in the context of these processes and activities.
A basic framework for the role of the SMS in statistical organizations is defined in:
- The Fundamental Principles of Official Statistics
(adopted by the United Nations Statistical Commission in April 1994); and
- The Principles Governing International Statistical Activities
(endorsed by the Committee for Coordination of Statistical Activities in September 2005).
The principles relevant to metadata management are highlighted below:
Back to top
The SMS should be a tool enabling a statistical organization to effectively perform the following functions:
- Planning, designing, implementing and evaluating statistical production processes.
- Managing, unifying and standardizing workflows and processes.
- Documenting data collection, storage, evaluation and dissemination.
- Managing methodological activities, standardizing and documenting concept definitions and classifications.
- Managing communication with end-users of statistical outputs and gathering of user feedback.
- Improving the quality of statistical data and transparency of methodologies. Ensuring and evaluating the quality of statistical data is one of the most important activities. To this end, national and international statistical organizations have adopted a set of criteria (relevance and completeness, comparability and coherence of statistical concepts, accuracy of statistical estimations, timeliness and punctuality of delivered statistical information, its accessibility and clarity). The SMS should offer a relevant set of metadata for all of these criteria.
- Managing statistical data sources and cooperation with respondents.
- Improving discovery and exchange of data between the statistical organization and its users.
- Improving integration of statistical information systems with other national information systems. Growing demands to use administrative data for statistical purposes require better integration and sharing of metadata between statistical and administrative bodies, to ensure coherence and consistency of exchanged information.
- Disseminating statistical information to end users. End users need reliable metadata for searching, navigation, and interpretation of data. Metadata should also be available to assist postprocessing of statistical data.
- Improving integration between national and international organizations. International organizations are increasingly requiring integration of their own metadata with metadata of national statistical organizations in order to make statistical information more comparable and compatible, and to monitor the use of agreed standards.
- Developing a knowledge base on the processes of statistical information systems, to share knowledge among staff and to minimize the risks related to knowledge loss when staff leave or change functions.
- Improving administration of statistical information systems, including administration of responsibilities, compliance with legislation, performance and user satisfaction.
- Facilitating the evaluation of costs and revenues for the statistical organization.
- Unifying statistical terminology as a vehicle for better communication and understanding between managers, designers, subject-matter statisticians, methodologists, respondents and users of statistical information systems.
Back to top