|5. System and design issues (Statistics Portugal)||Statistics Portugal||7. Lessons learned (Statistics Portugal)|
6.1 Overview of roles and responsibilities
The metadata system user profiles that interact with the life cycle of surveys are as follows:
Metadata system manager
This job has thus far been done by the metadata system manager, whose duties are:
- Coordinating managers of each subsystem in the system;
- Ensuring that the different subsystems' conceptual models are properly integrated;
- Defining the general harmonisation rules applicable to all subsystems, in cooperation with the subsystem managers;
- Planning training courses, subsystem revisions, etc. in cooperation with the subsystem managers.
Metadata subsystem manager (central metadata unit)
Each metadata, concepts, classifications, variables, methodological documents and data collection instruments subsystem has a manager in the Metadata Unit who guarantees the application of standardisation and harmonisation rules in each subsystem. These managers hold talks whenever necessary to articulate coherence and integrity between the different subsystems. They also have discussions with the survey managers and the Dissemination Database manager.
The concepts subsystem manager manages the concepts database, guarantees the application of the terminological rules in the formation of concepts (in the allocation of names to concepts and the construction of definitions), and decides in which thematic area and glossary each concept should be classified. S/he standardises the source of concepts under standard NP 405 and provides support for the SC working groups and Production Departments when drawing up new concepts and in the periodical revision of the concepts in each thematic area, organise the concepts into a conceptual system, for each thematic area, provides the translation of concepts to English and manages the system's decoding tables. Also s/he interacts with the IT technicians in the implementation and maintenance of the subsystem and prepares the necessary documentation for sending concepts to the SC for approval.
The classifications subsystem manager manages the classifications database. S/he ensures that there are no redundant classifications in the subsystem and that the names and versions of classifications and rules for classification and coding are harmonised. S/he arranges for classifications registered in the subsystem to be translated into English and, whenever possible, into French. S/he manages the system's own decoding tables and interacts with the IT technicians in the implementation and maintenance of the subsystem. S/he holds meetings with the managers of each classification to ensure the coherence and harmonisation of the subsystem and prepares the necessary documentation for sending classifications to the SC for approval. S/he interacts with the managers of the different classifications.
The variables subsystem manager ensures that the rules governing this subsystem are obeyed, so s/he checks proposed properties, object classes, representation classes and value domains to make sure that they are not duplicated. S/he also ensures that the names given to variables abide by the subsystem's rules. After conducting these checks, s/he approves or rejects the variables proposed by the SMs. S/he manages the system's own decoding system and interacts with the IT technicians in the implementation and maintenance of the subsystem.
The methodological document subsystem manager ensures that the format in Word of the document received from the Production Departments complies with the standard format approved by the SC and that the contents of each topic are in agreement with the expected content. At the moment, the subsystem is a prototype and not available to the SMs, so the metadata unit that enters them into the database and publishes them. In the final version of the subsystem, the SMs will enter the contents of methodological documents into the subsystem, where they will have "for approval" status until approved. The manager of this subsystem will change their status to "in force". S/he meets with the SMs to ensure the coherence and harmonisation of the contents of these documents and prepares the necessary documentation for sending surveys to the Board for approval.
The data collection instruments subsystem manager registers data collection instruments (questionnaires or files) in the database and allocates them periods of validity as requested by the SMs. S/he manages the subsystem's own decoding tables and interacts with the IT technicians in the implementation and maintenance of the subsystem.
Survey manager (SM)
This title is given to the statisticians (subject matter) in charge of each survey or to experts appointed by them. Their responsibilities in the system are as follows:
- At the end of the year and on an annual basis, the SM enters the survey plan for the following year into the planning system.
- S/he drafts the feasibility study.
The design phase of a survey begins with a feasibility study that must be approved by the Board. This study is drafted by the SM.
- The SM proposes the concepts, classifications and variables to be used in the survey. As there are SC-approved concepts and classifications for use in the NSS, they are the ones that should always be used unless they are not suited to the survey in question, in which case appropriate concepts and classifications must be proposed. When the variables to be observed or disseminated in each survey are being defined, those already defined should be taken into account and reused whenever possible.
- S/he drafts the methodological document in accordance with the SC-approved format. The methodological characterisation of surveys is usually carried out by methodologists in close cooperation with the survey managers.
- At the end of the design phase, as set forth in the rules, the SM distributes the methodological document and its questionnaire(s) to the units that will use the information produced and to the planning, methodology, metadata and information systems units for their opinion. On the basis of these opinions, s/he makes any appropriate changes and responds to each opinion, indicating the suggestions that have and have not been included with explanations. S/he alters the methodological document and questionnaire appropriately and sends their final versions to the metadata unit for the questionnaire to be registered and the methodological document to be published as the version in force.
- S/he re-plans surveys whenever necessary.
Classification manager (central metadata group or subject matter)
Each classification has a unit and a specialist in charge of its content. Classifications that are used throughout all statistical activity are managed in the Metadata Unit, while classifications specific to one system are managed in their respective Production Department. It is this specialist who manages the content of the different versions of a classification for which s/he is responsible. S/he interacts with the classifications subsystem manager.
This is the IT technician who coordinates the development of applications in all the metadata subsystems, ensures that they are included in the technical plan and maintains the subsystems.
Access to consultation applications is open to any user.
Responsible of implementing the SDMX standard
This is a specialist who plays an active part in the Eurostat Task Force to revise the Content Oriented Guidelines. S/he is responsible for studying the SDMX standard so that the metadata system can be adapted to its requirements.
6.2 Metadata management team
The Methodology and Information System Department at the Statistics Portugal (SP) has a Metadata Unit. Its main duties are:
- Design, coordination of development and permanent management of all aspects of the NSS metadata system;
- Coordination of the technical approval process of Surveys;
- Management of classifications that are used throughout the NSS.
The metadata subsystem managers and some classifications managers belong to this unit. There are other classifications managers in the Production Departments.
The specialised team attached to this section comprises 17 technicians and the head unit:
- Technicians with a more general profile, who participate in devising and testing the different metadata subsystems, manage them and assist in-house and external entities in using the system; currently they are 12 technicians with this profile.
- Nomenclaturists, who normally have degrees in economics and study and devise national classifications, monitor EU and international work on studying and devising statistical classifications, assist in-house and external entities in using classifications and give expert
- Terminologists, who have language and literature qualifications and belong to the concepts subsystem. They assist the Production Departments and SC working groups in designing conceptual systems, drafting definitions, arranging for the translation of concepts into English and giving expert opinions; currently the Metadata Unit has 1 technician with this profile.
Some of these specialists represent the SP in intra- and extra-community bodies and participate in statistical cooperation programmes.
The Metadata Unit does not have its own IT specialists. Experts from the Application Development Unit and also the Methodology and Information System Department provide IT services.
6.3 Training and knowledge management
The metadata system has been introduced to the SP in presentations of some of its systems, such as the classifications, methodological documents and variables systems. Some training courses on the classification system and data collection instruments (version 1) have been given to dissemination practitioners in order to answer users' questionnaires.
A presentation was also given of a project undertaken in 2006 with the Linguistics Centre at University Nova, Lisbon, defining a method for constructing conceptual systems. This new way of analysing concepts will mean that the concepts system will have to be altered to allow the necessary types of relationship to be defined so that concepts can be presented in this way.
In 2007, we began implementing a training plan in the metadata system. Two training courses were given on the variables system and the rest of the training plan is scheduled to be implemented in 2008, with the exception of the course on methodological documents, which will only take place in 2009. The plan includes not only training on consultation and management applications in the different subsystems but also on their underlying concepts, conceptual models and terminology. User manuals are being prepared for the training courses.
The intranet has a glossary of metadata terminology containing the concepts used by the integrated metadata system.
The current training plan that will be held every year consists of five courses:
1. Integrated metadata system
Fig. 16. Integrated Metadata System
This course is for all senior and assistant statisticians and includes the following subjects:
- The integrated metadata system's different components and its role as a technical statistical coordination tool;
- The basic concepts and terminology underlying this system;
- The role of each subsystem in the life cycle of surveys;
- Improving the statisticians' competences in the use of this system as a tool in designing the information subsystems in which they work;
This course is designed to help senior statisticians develop the following competences:
- Designing conceptual systems;
- Preparing definitions;
- Identifying problematic definitions in the concepts database, that need a revision and improving according to the terminology criteria.
To achieve this, the course includes:
- Terminology theory;
- The notion of conceptual system and the application of a methodology in its design;
- Semantic relationships in terminology;
3. Classifications systems
This course is for senior and assistant statisticians and includes the following subjects:
- The main international, EU and national classifications systems used in statistics, the classifications comprising them and the relationships between them;
- The approval process for classifications at different levels and harmonisation in the use of classifications in the NSS;
- Improving competences in the use of the classification subsystem.
4. Variables subsystem
This course is for senior and assistant statisticians and includes the following subjects:
- Recognition and designation of entities making up the conceptual model;
- The rules and principles in the name, standardisation and harmonisation of variables;
- How this subsystem relates to other subsystems in the integrated metadata system and other outside subsystems, such as the Dissemination Database and WebInq (electronic data collection);
- Use of the consultation applications and the survey manager (SM) profile of the management application;
- Defining indicators to be disseminated on the Official Statistics website.
5. Methodological documentation
This course is scheduled to begin in 2009 and is aimed at senior and assistant statisticians. Its subjects are:
- The SC-approved methodological document format for documenting the NSS's surveys;
- Use of the consultation applications and the survey manager (SM) profile of the management application.
The statisticians in the Metadata Unit participate in the above courses, EU-level courses and international conferences and practise high-level self-study.
6.4 Partnerships and cooperation
Statistics Canada's IMDB project was the main source of reference in developing the SP's integrated metadata system. When SP first began developing the system in 2003, some of its members visited Statistics Canada, where they followed a three-day programme set up by that agency:
- Brief, generic approach to the IMDB (Integrated Metadatabase) project;
- IMDB - Phase 2 (Description of a surveys, methods and quality metadata).
- COR (Common Object Repository);
- IMDB - Phase 3 (Defining variables).
- IMDB - Phase 3 (continued);
- Meeting with IT technicians involved in the project.
These were three days of highly useful work in which an in-depth analysis was made of some extremely relevant aspects that were addressed more generally in the project documentation available.
The following were also very important references for the definition of the system:
- The Corporate Metadata Repository model by Dan Gillman;
- The Neuchâtel model, which supports the classification subsystem;
- Documents on metadata systems, documentation and quality by Bo Sundgren:
- Documentation and Quality in Official Statistics, Statistics Sweden, 2001;
- Objects and their Classifications, Relations, and Life Histories - as Reflected by Official Statistics, Stockholm, Sweden: Statistics Sweden, 2004;
- Statistical Metadata - A tutorial;
- The αβγτ-model: A theory of multidimensional structures of statistics, Statistics Sweden, 2001;
- The Swedish Statistical Metadata System, Statistics Sweden, 2000;
- The Contents of a Statistical System as a Whole, Stockholm, Sweden: Statistics Sweden, 2004;
More recently, the Statistical Office of the Republic of Slovenia contacted the SP with a view to learning about its variables subsystem in detail. After analysing possible forms of cooperation, the SP provided the Slovenian agency with the subsystem's data model and has also responded to later requests.
As part of a statistical cooperation project with the Portuguese-speaking African countries, one of the Metadata Unit statisticians has been working with them on a project entitled "Classifications, Concepts and Nomenclatures", which coordinates the five countries' economic classifications. Consultancy services have also been provided to a project to develop a common integrated economic nomenclature system for the five countries.
The SP also belongs to the Eurostat Metadata Task Force, which is analysing the main components of the SDMX Content-Oriented Guidelines: framework, inter-domain concepts and associated codification, vocabulary and statistical domains.