The focus of this section is on the preparation of a corporate SMS vision, related planning and on the major characteristics of a metadata management framework and management strategy.
The vision should clearly state the goals or aims of the SMS. It should apply across the entire statistical information system and be realistic and within the capabilities of the statistical organization. It should also include a statement about scope: what is included in the SMS and what is not.
The SMS vision should support and clearly demonstrate its alignment with, and relevance to, the overall corporate vision and priorities of the organization. The SMS will be an essential enabler to achieve broader business, statistical and technical outcomes within the organization. In other words, it serves as a means to many ends, rather than being an end in its own right.
An important prerequisite for successful design, implementation and functioning of the SMS is the development of a corporate vision of the SMS in the statistical organization. The functions of the SMS, centred on metadata and data users, should be oriented towards the diverse processes and activities of the statistical information system. Operational units within a statistical organization, respondents, end users and other stakeholders should all have an input to the SMS vision, and, where practical, should be involved in the preparation, implementation, use and evaluation of the components of the SMS.
The vision should be an integral part of the strategic direction of the statistical organization, so should be developed with the direct involvement of senior management. It is an important task for the SMS managers to ensure that not only the development of the vision but also the SMS design, implementation and further development will be actively supported by senior managers. For this purpose a suitable SMS management and reporting structure should be established. Feedback and evaluation, supported by metadata accumulated in statistical production cycles, should be an integral part of the SMS design.
The vision should define major goals and functions of the SMS for the statistical organization and attribute priorities for their implementation. This includes clearly linking the goals of the SMS with strategic business drivers from within the organization and beyond.
The metadata requirements associated with each part of the statistical business process should be identified. All points of contact between the SMS and business processes, in terms of creation, update and use of metadata should be described. It is important that the high level picture can be discerned, along with the details. The Generic Statistical Business Process Model provides a tool for identifying at least higher-level links. The points of contact and relationships between the metadata model and business processes should be pictured both as they currently exist and as is envisaged in the future. This will summarize for the organization what would change, both at the strategic and the day-to-day level, as a result of realizing the vision. This, in turn, identifies the benefits more clearly from a business perspective.
An important part of the vision should be an analysis of the existing statistical metadata objects and services, finishing with a clear specification of which existing metadata and services can be used in the corporate SMS, which should be updated and which should not be used at all.
It is advisable, that the SMS is not developed as a purely technical project. It is still quite often the case in statistical organizations that the subject-matter departments do not understand fully the requests formulated by information technology specialists. When developing the vision, it is essential to express clearly that the first priority in the SMS is to safeguard the content and logical integration of statistical data and metadata.
To make the SMS a success story, the vision and its implications should be based on what is really possible for the organization. Effective management of the statistical information system and the integration of information flows, on both the national and international levels, should remain one of the major goals of SMS.
The vision should also encompass cost estimates for the SMS project. Wherever possible, tangible benefits should be identified and weighed against these costs. It is also recommended that likely medium to longer term costs and risks of inaction are estimated (e.g. ongoing duplication of effort and information, ongoing complexity and inconsistency, ongoing cost and risk of inertia to evolution to keep pace with changing statistical requirements). These should be broadly compared with the costs and risks of taking action. It should be ensured that the cost to producers of metadata is justified by the benefits to the metadata users. Such proposals should be very pragmatic, reflecting ultimate needs and metadata priorities. Experience shows that the human and financial costs of SMS developments can be quite demanding.
Experience also shows that many statistical organizations implemented some functional blocks of metadata without having a complete SMS vision at the beginning of the process. This includes lack of a practical strategy for how different "functional blocks" delivered at different times, possibly by different teams, will knit together into a coherent SMS capability and how this capability will be harnessed in practice by all the different aspects of the statistical information system. It is especially true for the objects dealing with the description of statistical data. It can be observed, that the following blocks of metadata have been frequently implemented: statistical variables and values sets, statistical surveys, socio-economic classifications and nomenclatures, time series, statistical publications, statistical population, economic subjects, statistical units, aggregation and statistical evaluation methods, output tables and others.
Without having a coherent vision there is very often a lack of coordination among individual metadata blocks. This causes many inconsistencies, duplications and, last but not least, the low efficiency of metadata tools in terms of both costs and staff capacities needed. The end users could, because of lack of coordination, struggle with unnecessary diversity of users' roles and related diversity (and possibly inconsistency) of terms and concepts being used in different existing metadata blocks. Such a situation certainly does not encourage cooperation of users with metadata implementation.
The vision should contain a metadata model complying with the SMS functions. Such a model should encompass metadata about data and processes behind them as well as metadata about other objects and processes of the statistical information system relevant to the SMS functions. Metadata needed for the management and administration of the statistical system such as metadata about costs and benefits, cost-effectiveness, satisfaction and complaints should also be a part of such a model. Metadata objects and links between them should be identified.
An agreed conceptual metadata model should be linked to the standard business processes that are the part of the statistical business process. This linkage is used to determine which metadata should be collected, and at which point in the process. Metadata models should take account of and use international standards where possible.
Figure 2 below provides an overview of the components of the SMS vision.
Figure 2: Schematic view of the SMS vision and its components (click to enlarge)
Metadata objects and metadata resources
Metadata should be structured according to the objects they describe. There are three main categories of objects to be considered when preparing the vision:
i. statistical data
ii. statistical processes
iii. tools enabling production and use of statistics.
Metadata related to statistical data are important tools supporting the production and final use of statistical information. These metadata include descriptions of statistical concepts, characteristics, variables, units, populations, classifications, registers, observation templates, statistical surveys, time series, observations, aggregations, methods, micro data, macro data, final outputs, statistical publications, statistical databases and archives. Information about respondents, end users, statistical websites and other metadata objects related to statistical data, also belong to this group.
Statistical processes can be divided into two major groups:
i. those associated with statistical production (data collection, data storage, data evaluation, data dissemination); and
ii. those associated with the statistical information system and statistical organization (planning and evaluation processes, supply processes, usage processes, total quality management and other management processes).
All processes are associated with important metadata, such as information on costs, performance measures, errors and error rates, diverse benchmarking indicators, etc. Process metadata are sometimes also referred to as paradata.
Processes are related to metadata in three ways:
i. they are carriers of metadata, passing them on to subsequent processes;
ii. they use metadata;
iii. they produce metadata.
Figure 3 illustrates the links between the metadata objects described above.
Figure 3: Example of metadata objects for statistical production processes (click to enlarge)
Tools enabling the production and use of statistics
The SMS should provide tools to enable the production and use of statistics including: (\i) search and retrieval tools; (ii) tools supporting statistical production, and; (iii) knowledge resources supporting the "intellectual processes" related to the statistical system, such as corporate management, planning and evaluation, research and development. These tools should be sharable by multiple processes. They need to be systematized and organized collectively in order to be easy to find and make use of, both by people and by automated processes within the statistical information system.
In this respect, the vision should promote the following:
i. Development of common terminology for metadata elements across all processes in the statistical life-cycle;
ii. Development of a common and consistent description of metadata elements allowing easy location, retrieval and exchange of data and metadata;
iii. Development of standard interchange formats and tools allowing sharing of metadata and data between organizations;
iv. Implementation of consolidated metadata repositories facilitating reuse of metadata;
v. Introduction of a registration processes to promote the use of standard metadata elements and increase knowledge on metadata flows and statistical integration;
vi. Improvement of metadata quality;
vii. That the production process will be metadata driven;
viii. Clear policies on how organizational and technical processes outside the SMS will change once it is agreed that the SMS is fit for purpose, and capabilities that allow these changes to take place. What constitutes "fit for purpose" (i.e. not necessarily perfect) should be agreed beforehand.
Preparation of the strategic plan
The aim of this subsection is to advise on the preparation of a corporate strategic plan for the development of an SMS. A strategic plan should be an integral part of the SMS vision, reflecting the goals and functions specified in this document. As a part of the vision, the senior management of the statistical organization should approve the strategic plan.
The development of a strategic plan needs to be a flexible and adaptive process, possibly with several iterations. The plan should give visibility, clarity and stability to the development efforts, but aspects are likely to change during its implementation, which may take several years. Certain parts may never be implemented; other parts may be implemented in a different way than originally assumed. Completely new components may appear as a result of new needs, new methodological and technical developments or changes of some other basic conditions for SMS development. Therefore, the plan should be regularly reviewed and revised.
Detailed plans should be developed and approved later on for the design and implementation phases of the SMS development. Such plans should reflect agreed priorities for the development of individual components of the SMS. Last but not least, specific plans should be prepared for the use and evaluation of the SMS components.
When preparing a strategic plan, the number of activities, sensitivity of their solution and their priorities for the statistical organization should be taken into consideration. Links between individual activities and importance of their contribution to the SMS strategic goals should be thoroughly analyzed. Conditions under which the goals could be met should be clearly specified.
A key part of the plan should be the establishment of an organizational framework and management strategy.
The strategic plan should be developed and approved by all actors involved in the design, implementation and maintenance of the SMS. It is therefore indispensable that such plan is prepared in close dialogue and cooperation with all actors involved in the process of the SMS development. The planning should be made explicit, so that the whole organization can discuss the strategies to be used and the choices to be made in the step-by-step development of the SMS. At the same time as the step-by-step development of the SMS is being planned, the subsequent uptake and use of the capabilities to be delivered should also be planned.
Recommendations for establishing the strategic plan
The recommendations below are based on the experiences of several national and international statistical organizations:
i. When preparing the plan, the organization should consider its current capabilities. Available human and financial resources, as well as organizational and technical feasibility, should be carefully analyzed in order to make the plan realistic.
ii. Goals defined in the vision should be transformed into practical steps to which priorities are then assigned.
iii. Practice shows that different organizations often have similar priorities. This is especially true for the development of databases on statistical classifications and nomenclatures, aggregated output databases, and metadata models for the websites. Some organizations give priority to metadata models for microdata. Discussions and sharing of experiences between statistical organizations are therefore strongly recommended.
iv. Quality of data and metadata should be considered a high priority.
v. External cooperation should be clearly defined; categorization and priority setting for external users should be specified. The plan should take the existing working plans of all external partners into consideration.
vi. The plan should be prepared in such detail that all partners will be able to commit their participation.
vii. External projects to establish data and metadata warehouses, both on the national and international level, should be considered for potential impact on the SMS.
viii. External activities on data security and data confidentiality related to the SMS should be considered.
ix. An integral part of the plan should be activities dealing with the development and implementation of international standards.
x. The plan should also consider activities to promote the SMS and create an atmosphere of cooperation with all stakeholders. To this end, prototypes for demonstration of SMS functions could be useful.
xi. Research activities on feasibility studies and analysis of user feedback should be also taken into the consideration when preparing an SMS plan.
xii. Transfer of know-how and training for participants in the SMS business case should be incorporated in the plan.
Management strategies for corporate SMS
A framework for a corporate metadata management strategy should be specified in the vision. The senior management should play a lead role in a corporate management model.
Responsibility for development of metadata policies and procedures and for providing training and advice to developers should be clearly assigned.
An important part of the SMS management strategy should be a systematic cooperation with major metadata stakeholders.
Implementation of the metadata management strategy should follow two broad approaches. They are:
i. User orientation - focusing on information relevant to usage such as finding and accessing data, understanding their structure and meaning, assessing their quality and relevancy, and using them correctly. This focus is dissemination oriented; and
ii. Producer orientation - metadata driven approach focusing on the needs of information systems and electronic processing.
There are two major dimensions to be taken into consideration when deciding on SMS management strategy:
(\i) the crosscutting nature of the SMS role and its functions in statistical organizations and, (ii) the requirement of corporate management during all phases of SMS development and use.
SMS management across the whole statistical organization
The SMS is an integral part of a statistical organization's strategic direction. It implies that the SMS management strategy should be integrated into the management strategy of the organization.
Diverse organizational units and external bodies participate in the SMS business case. Managers, subjectmatter statisticians, methodologists, information technology experts, researchers, respondents and end users are all SMS partners. Their functions, needs and obligations differ according to whether they participate in the SMS as metadata users, metadata suppliers, designers, developers, producers, administrators and/or evaluators. Clearly, the SMS does have a cross-cutting nature. The management strategy for the SMS business case should correspond to those needs.
It is strongly recommended that the top management of the organization is directly involved in the SMS and its management, but experience shows that this can be hard to achieve.
Some recommendations for the SMS management strategy across the whole organization:
i. Metadata management is a part of every project and should be considered alongside resource allocation and accountabilities, in the same way as business processes and data flows are considered.
ii. The SMS management strategy should be specified in close alliance with the existing
managerial structure of the organization. With the lead role of the senior management in the SMS management model, clear links should also be defined in the middle management level and in the experts' level (methodologists, subject-matter statisticians, information technology experts).
iii. Roles and responsibilities of all partners should be clearly defined, understood and followed. Where possible, automated workflows can be used to enforce agreed roles and responsibilities.
iv. An SMS management board should be established. This board will take an ultimate, corporate view on all decisions dealing with the SMS development.
v. A multidisciplinary team should be the major organizational form for the development of the SMS project. The "ideal" SMS Team(s) will include: statistical methodologists; subject-matter statisticians, dissemination specialists, end users, standards' experts, researchers, and information technology specialists in data modelling, business process design, architecture and applications development.
vi. Implementation of the SMS management strategy may highlight some needs for changes in the job description of some experts (namely methodologists and subject-matter statisticians) as well as in the organization of statistical work. This will often be the case in statistical organizations where a corporate SMS did not previously exist. Many critical issues could appear. Such issues should be foreseen and reflected in the MS vision and strategic plans as far as possible.
Corporate management of the SMS development life cycle
Figure 4 presents a model for managing the phases of an SMS development life cycle, being design, implementation, maintenance, use and evaluation. The governance of metadata management and the monitoring of outcomes should be made clear in the SMS management strategy.
Figure 4: Model for integrated management of SMS (click to enlarge)
The most important management activities in each phase of the SMS development life-cycle are outlined below.
Management of SMS design
The role of the design phase is to develop the SMS vision and global architecture, and to establish a management and implementation strategy for the project. The most important functions, tasks and activities to be considered by management are as follows:
i. Development of the SMS vision.
ii. A global plan for SMS development should be established and approved by all participants.
iii. To ensure the efficiency and value of metadata-related work, an SMS Global Architecture should be developed. This should encompass all processes that will work with metadata. An inventory of all such processes and existing metadata tools should be prepared. This inventory should be developed in close cooperation with major stakeholders. The order of priorities should then be decided.
iv. Specification of common components: Analyzing the inventory should reveal common metadata components. The possibility for these components to communicate with different parts of the SMS via a single interface should be explored. This should be considered a high priority.
v. The impact of the corporate SMS on existing statistical production systems should be clear from the SMS vision. Any necessary re-engineering of processes should be considered and planned at this stage.
vi. The metadata requirements associated with standard business processes are articulated, i.e. all the points of contact between the SMS metadata model and business processes, in terms of creation, update, and use activities, are described. For example, metadata associated with understanding user needs, frameworks and standards is acquired and used to inform later phases. To the greatest extent possible, the necessary input and output metadata should be captured in the collection strategy stage so that we know well in advance that the desired outputs are obtainable and fit for purpose.
vii. Major partners in the design phase are the users (both, inside and outside the organization), methodologists, subject-matter statisticians and information technology experts.
viii. Feedback and evaluation is an integral part of the design process and is supported by metadata, accumulated in all phases of the SMS development life cycle.
ix. During the transition period, some "legacy" metadata structures and processes may need to be supported by the SMS to meet requirements of legacy components of the statistical information system. The SMS architecture needs to clearly and consistently differentiate such legacy aspects from the aspects designed for the future to ensure that the latter are always applied in preference to the former.
x. Financial requirements for implementation phase should be detailed.
Management of SMS implementation
The role of this phase is to implement the SMS so that it is ready for use. The implementation of all SMS subprojects can be a long process. Depending on dependencies and priorities, some subprojects may be implemented in parallel and others sequentially.
The following major activities should be considered when preparing a strategy for this phase of development:
i. An agreed set of definitions and terminology should be developed. Consideration of national and international terminology standards, such as the Metadata Common Vocabulary published by the Statistical Data and Metadata eXchange (SDMX) initiative, is of high importance.
ii. Detailed and coordinated plans for all stages of SMS implementation should be prepared and approved by all partners at the beginning of the implementation phase. The basic framework of the SMS plan is defined in the vision.
iii. Existing processes using statistical metadata should have been reengineered. If complete reengineering cannot be achieved in advance, then firm plans and project management for achieving it in a timely manner must be in place.
iv. Outsourcing possibilities for the SMS implementation should be considered.
v. It is recommended to implement an SMS as a technically coherent project. This will allow standard links between metadata objects and processes, standard metadata tools for searching, retrieval, exporting and downloading metadata and harmonization of technical administration. Standard operations for administration of diverse metadata can then be developed.
vi. A crucial task in the implementation phase is to set up a corporate metadata repository. This is the physical implementation of the metadata model defined in the vision. The concept of the corporate metadata repository should be developed, although there could be a number of physical repositories. To develop an appropriate architecture is a demanding task and there is no blueprint for such an exercise. However, good practices exist that may be a useful guide.
vii. Physical loading of metadata into the corporate metadata repository should be done by metadata owners. This is a resource-consuming task and the impact on subject-matter staff should be recognised. For many, capturing metadata is a tedious extra task that brings them no perceived benefit. A prerequisite of the 'system' therefore, is that as many metadata as possible are captured automatically, as a result of a computer process or as a result of a required business process undertaken by a person. Thorough management and planning of those activities is imperative. Management support and highlighting benefits to users (refer to Section 4) can help motivate them to provide the necessary metadata.
viii. Regular monitoring of progress in implementation from the view of completeness and cost effectiveness is necessary.
ix. Tools and processes specified in the vision should be developed and tested by all target user groups. User manuals and documentation should be developed. Testing should be conducted before making the SMS available. Training for all metadata users should be organized.
Management of SMS use
The role of this phase is to ensure efficient use of metadata and metadata tools by all users specified in the vision. Effort should be made by management to monitor and coordinate activities and processes dealing with metadata usage by the broad range of users (refer to Sections 3 and 4). The metadata strategy in this
phase should encompass the following functions:
i. Prepare, maintain and coordinate detailed plans of metadata use by all users to ensure the required metadata quality within the required deadlines. The coordination of plans developed for individual users is a major goal of the management.
ii. The units responsible for statistical production should be accountable for the preparation and maintenance of plans related to the activities dealing with the production process. In this case, the SMS management should ensure that all activities dealing with the use of statistical metadata and metadata tools are planned and defined.
iii. Oversee the availability of metadata and metadata tools and ensure the links between metadata maintenance and use.
iv. Inform metadata users about all changes to metadata contents.
v. Arrange for ongoing feedback from users about metadata quality and the availability and efficiency of metadata tools. Feedback mechanisms could be integrated into regular activities of metadata use. Specially organized surveys on user satisfaction are useful, but should be complemented by these ongoing mechanisms.
vi. SMS management (in close cooperation with the SMS technical administrator) should be aware of the software and technological environment related to the use of metadata and metadata tools. As mentioned previously, metadata and metadata tools should be platform independent. However, it could be useful to maintain information about changes in the users' software environment (e.g. updates to web browsers, etc).
vii. Statistical websites are an integral part of SMS implementation and the use of metadata.
Furthermore, they are a regular part of the dissemination strategy of a statistical organization. The structure and quality of metadata presented on the website contribute to the satisfaction of metadata users. The need for statistical metadata on websites varies according to the needs of the individual users groups. The use of statistical metadata on websites should be monitored in order to keep track of users' satisfaction and evolution in their needs.
Management of SMS maintenance
The role of the SMS maintenance phase is to ensure that all metadata stored in the corporate metadata repository are up-to-date for ongoing use. To keep metadata up-to-date is of primary importance for all metadata users.
The following recommendations should be taken into consideration when preparing a strategy for the management of this phase:
i. The major functions to be considered are those relating to the administration of metadata content.
ii. Planning is an important instrument for managing the maintenance phase. Everyone participating in maintenance processes should approve a detailed plan of maintenance activities, which meets required timelines. Such a plan is an indispensable instrument for management for smooth and coherent monitoring of the phase of metadata maintenance.
iii. Ensure timeliness and coherence of maintenance activities.
iv. The concept of registration of metadata objects, ownership of metadata, what is the 'standard' for a particular classification or data item, what are the permitted variations from the 'standard' etc. should be all clearly defined, agreed and used.
v. SMS management should oversee the definition and maintenance of all metadata stored in corporate metadata repository, although other units in the statistical organization will also contribute to its ongoing enhancement.
vi. SMS management is responsible for definition of policies, procedures and protocols for the maintenance of the corporate metadata repository. A 'registration authority' manages all metadata entities in corporate metadata repository. The major partners for the SMS management are the "custodians" of metadata. The custodians are, according to the concepts of registration of metadata object specified in the vision, authorized to keep the metadata they are responsible for up-to-date.
vii. Rules and guidelines should be developed for the maintenance of each metadata entity in the corporate metadata repository, identifying the responsible metadata owner. It is recommended that these rules and guidelines are approved by senior management and become official documents of the statistical organization.
viii. Preparation of rules and guidelines requires joint work with metadata owners. Methodologists are also important partners in this process.
ix. Training of metadata owners in the rules and guidelines prepared for maintenance activities will be required.
x. SMS management should ensure maintenance of metadata history and updating of links between metadata in the corporate metadata repository.
xi. All maintenance functions performed by metadata administrators and metadata owners should use a coherent/standard set of metadata tools. Such tools should be available especially for the following maintenance functions: search and retrieval, inserting and deleting of metadata objects and related parameters, changes and corrections, presentations and exports, metadata editing and consistency controls, checking and updating of metadata links, maintenance of metadata history.
Management of SMS evaluation
The goal of the evaluation phase is to determine the efficiency of existing SMS functions and make proposals for improvement or further development of the SMS. There are clear links to the knowledge and experiences accumulated in the earlier phase of the SMS development life cycle, namely in the SMS use phase. By preparing proposals for further SMS development, the SMS evaluation phase makes a loop between the use and design phase of the SMS.
The management strategy of the SMS evaluation phase should include the following procedures and tasks:
i. Specify the major targets of SMS evaluation and prepare a plan of evaluation activities and procedures based on these. It should be clear which functions and aspects of the SMS are to be evaluated.
ii. Evaluation of user satisfaction should be an ongoing part of the SMS development life cycle. The most important object of evaluation is ultimately the external user of data and metadata produced by the statistical organization. For these users, it may not be appropriate to measure satisfaction with the SMS in isolation, as they are likely to see data and metadata as a single package. Incorporating questions on metadata and the external interfaces of the SMS alongside more general questions on satisfaction with data outputs may be more effective, and increases the potential for additional insights into user needs and satisfaction through cross-tabulation of results for different questions. It should be ensured however, that the satisfaction of other user groups, particularly internal users would also be evaluated.
iii. Other important aspects for evaluation are cost efficiency, implementation of standards, organization of work, maintenance procedures and technological implementation.
iv. In principle, there could be three major forms of evaluation: (\i) regular long-term evaluations (e.g. at 3 year intervals) that examine overall effectiveness of SMS functionality; (ii) regular short-term evaluations (e.g. annually) that primarily assess user satisfaction; and (iii) ad hoc evaluations as deemed necessary.
v. Benchmarks should be established for all defined targets. Benchmarking parameters and evaluation methods should be specified and agreed. For some cases an efficient benchmarking method is to compare experiences and plans with those of a similar organization. International cooperation can be highly efficient in this respect.
vi. Appoint evaluators for planned evaluation activities. The team of evaluators should include both staff from the statistical organization and metadata users. For evaluation of the project's efficiency and the overall technological solution, it may be useful to hire external evaluators to provide an independent view.
vii. Document information on the user feedback collected in the phase of the SMS use.
viii. Organize specific surveys on user satisfaction.
ix. Report to the senior management of the statistical organization on the evaluation outcomes, including prioritized recommendations. Based on the conclusions made by senior management, organize improvement of and/or further development of the SMS.
Evaluations should focus both on the effectiveness of the SMS as an output (and an element of statistical infrastructure), and on the effectiveness of the broader business outcomes being achieved. The latter should be compared with the original vision, but should also take into account any changes to business directions and priorities which have occurred in the meantime. As well as changes to the SMS itself, changes may need to be considered to implementation objectives, planning and governance.