1.1 Metadata strategy
Official statistics in Germany is characterised by a decentralised political system that divides the functions of government between the central government and 16 regional (Länder) governments. As a result, each region (Land) may have a statistical office of its own. Because some Länder merged their statistical offices, there are now 14 Länder offices in total. These offices are organisationally entirely independent institutions.
For the purpose of compiling most federal statistics (those ordered by a federal law), there is a predefined work sharing scheme between the offices. The Federal Statistical Office (Destatis) as the central authority is responsible for the methodological and technical preparation of the surveys and for the compilation and publication of nationwide results. The statistical offices of the Länder are responsible for the core processes of statistical production. Despite that, there are also several cases in which Destatis collects and processes data itself, e.g. in the area of foreign trade.
It is an important feature of the statistical system in Germany that for every major statistical activity* a law has first to be passed by parliament. For those statistical activities that involve federal and Länder offices, both the federal parliament and the parliaments of the Länder have to be consulted. The laws usually specify who is legally obliged to provide the information and what information is sought.
Apart from the statistics compiled by the statistical offices, there are also some statistics that are produced by other official agencies. For example, some of the data on unemployment comes from the Federal Employment Agency (Bundesagentur fuer Arbeit).
To organise the work in the "Verbund" - as the network of federal and Länder offices is known in German - a number of working groups exist with several governing committees on top. Senior governing committees are staffed with the heads of offices. Work sharing also extends to the area of IT-development. IT-systems that are to be used by all offices are put out to tender among the offices in the Verbund on the basis of an agreed business case (dubbed the "one for all"-policy).
Destatis itself is mostly organised according to subject areas and therefore resembles the classic case of a stovepipe organisation. There are currently six subject matter divisions and three central divisions. The central divisions deal with administration and legal questions (Division A), information technology (Division C) and management, dissemination and coordination issues (Division B). Methodological issues are split between Division C (mathematical methods) and Division B (research and development). More details can be found in the organizational chart.
* see 5.3 on the definition of statistical activity in official statistics in Germany
Metadata management has been an issue in the statistical system in Germany for many years. Maybe typical for a federal system, solutions have been found and implemented in isolated areas but they have not been coordinated through a common strategy. The current situation therefore resembles a "bottom-up" approach rather than a unified "top-down" solution.
The experience at Destatis and in the Verbund, however, shows that there is a strong need for a more coherent approach to handling metadata in the future. Several key projects in the Verbund - like standardisation of production or quality management - depend on standardised structures and concepts to understand the content of the different statistical activities in a coherent and uniform way. A metadata strategy would also help to provide a framework for the different projects.
Any future metadata strategy would need to be formulated in accordance with at least the most important stakeholders and it would need to be approved by the responsible committees. Therefore, it is not likely to take shape and become formally adopted in the near future. In the near past there were several projects - independently planned and implemented - that involved a centralised metadata management. The task is to combine the projects in a way that at least the outline of a common metadata strategy starts to emerge.
1.2 Current situation
There are currently two major projects that involve centralised metadata management.
A. Census metadata
The Census 2011 in Germany is carried out by the Verbund. It is based on a method that combines administrative sources with survey data. To deal with content from so many different sources a strong metadata management is needed. Hence, a working group has been instituted to deal with metadata issues and especially with the development of an IT-system for the management of census metadata. The working group is staffed with methodologists, IT-experts and subject matter statisticians from some Länder offices and Destatis.
The Census 2011 is currently the most important project and certainly the most pressing issue for Destatis in terms of metadata. To deal with the complexity of the project, it has been broken down into several sub-projects for which business cases are being written.
B. SteP - Standardisation of Production
SteP is a joint initiative of the Verbund to standardise production. A major objective of SteP is the design and deployment of generic IT-tools as building blocks of a standardised IT-landscape. Although SteP currently deals predominantly with IT-issues, a stronger involvement of subject matter experts should strengthen its outreach in the future.
SteP is organised around a simple process model that names the basic processes mainly in the collection and processing stages (see Section 2.3, fig. 2). There are sub working groups (called "steps") dealing with individual aspects of the statistical value chain (see here). A sub working group for metadata - called "step 12-metadata" has recently been established.
The idea behind this sub working group was to develop a metadata portal. This web portal shall allow users to access the metadata stored in already existing IT-systems. Basically, every system that stores metadata could become part of this project. At first, the portal is only intended for internal users. Since there is no general metadata model that standardises and explains the content of the underlying systems, the results would only be confusing for any outside reader.
In general, SteP has so far been a successful project for Destatis and the Verbund. In several of the most urgent areas, production was streamlined and economies of scale could be exploited. There is now a centralised storage facility for finalised micro data (accessible only to Länder offices) and a data editing tool that can be integrated into existing environments. Apart from the metadata portal, important ongoing projects within SteP include a database for incoming data.
A drawback of SteP is that there is no underlying, generally accepted metadata model guiding the project. While data can be passed on along the production chain, metadata is left behind resulting amongst other things in a redundant storage of metadata. As the standardisation process continues, this could become a problem of greater concern. Internal users might be confused with different IT-systems each using a different structure and a different terminology. The interoperability of the systems might also suffer because a metadata model usually embodies a generalised understanding of the way statistical activities are structured.
The task for step 12-metadata is to find a way to harmonise the different IT-systems in a way that the metadata stored can first be accessed and understood by users and secondly be shared by all IT-systems along the value chain.
Apart from these major projects, there are several other activities that involve centralized metadata management issues at Destatis. There is for example a close cooperation between quality reporting and metadata management since they overlap in many ways.