The ABS Information Management Transformation Program: A statistical metadata perspective
This document provides a brief introduction to the Information Management Transformation Program (IMTP) which is currently being undertaken by the Australian Bureau of Statistics (ABS).
While IMTP designates a specific program within the ABS, including a specific top level unit within the organisation chart, the aim is for the ABS to achieve IMT (Information Management Transformation). All staff within the ABS have a role in achieving IMT.
It is intended this document will be updated from time to time as IMT proceeds within the ABS. The IMT Program is developing rapidly, however, and the information contained in this document is not guaranteed to be up to date in every regard at every point in time.
As noted in the subtitle, this document focuses on IMTP from a statistical metadata perspective. It is primarily intended as a supporting document to the ABS Metadata Case Study within the METIS (Joint UNECE/Eurostat/OECD Work Session on Statistical Metadata) wiki. From a more general perspective, IMT may be seen most prominently as an ambitious and far reaching program related to business transformation, or business re-engineering. From a more general perspective, in other words, the emphasis is on the "T" in IMT. "IM" reflects the fact that (statistical) information management (in which statistical metadata has a crucial role) is the "core business" of the ABS as a producer of official statistics, and that statistical information is at the core of the products and services we deliver for governments and the community.
It should also be noted that, as with all content in the METIS wiki maintained by ABS practitioners, this is an informal working document shared with colleagues in the field of statistical information management. Unless unambiguously indicated otherwise, no content accessed via these wiki pages should be considered to represent a formal statement on behalf of the Australian Bureau of Statistics.
In October 2009, the ABS Executive formally agreed on Statistical Data and Metadata Exchange (SDMX) and Data Documentation Initiative (DDI) as the standards that will form the core of the ABS's future directions and developments with regard to statistical information management.
Following on from this decision, in February 2010 the ABS formally established the ABS Information Management Transformation Program (IMTP). The IMTP has the potential to be the most significant transformation of our information infrastructure in over 30 years. It will seek to modernise the information management systems within the ABS, and to support its data management and business process standardisation strategies.
Our aspiration within the ABS is to have metadata driven end-to-end business processes that are conducted within a common data management framework. IMTP has an internal objective to deliver a coherent and integrated environment for the life-cycle management of our data.
More broadly, the main outcome of IMTP is for the Australian Government and the community to have access to domestic and international statistical information that is easily found and used, relatable and comparable to support evidence based policy and decision-making. To achieve this goal, the ABS needs to not only modernise its internal infrastructure but also redefine the way it communicates and disseminates data to clients.
In concert with the IMTP, the ABS has been instrumental in forming a network of a small number of other international statistics agencies with the combined mission to 'work together with pace and passion to better meet their societies' information needs while driving down costs'. The other members of the network are the National Statistical Institutes (NSIs) of Sweden, Norway, Canada, New Zealand and the United Kingdom.
In addition to performing the role of network secretariat for the first year, the ABS will lead a collaborative project to operationalise a common information management framework including the use of agreed metadata standards. Four other key collaboration projects were agreed upon, for which individual network members have agreed to take the lead. These projects are integral to the success of developing an end-to-end data management framework. These projects aim at increasing NSIs' capacity to pursue innovation, and improving their ability to meet the needs of the statistical community, through the collaborative development of the next generation of statistical infrastructure.
Drivers for IMTP can be summarised from external (eg client) and internal (eg NSI) perspectives.
From an external perspective
For statistical agencies to remain relevant they must respond to a changing environment where:
From an internal perspective
Drivers in an international context
IMTP can be viewed as the primary ABS response at this time to challenges which are widely recognised as facing producers of official statistics more generally.
For example, many of the drivers behind IMTP are discussed in the paper The case for an international statistical innovation program - Transforming national and international statistics systems. This paper, together with an associated presentation, is available through the following link.
There is also a high degree of correlation with the challenges and strategies identified in the Strategic vision of the High-level group for strategic developments in business architecture in statistics .
Broad response to drivers
Key themes in the ABS response to these challenges include
IMTP aims to deliver
Five broad phases are envisaged for IMTP, the first two of which have been completed.
Phase 1: Assess need and build momentum for a transformation program
Phase 2: Seek and establish support for an international collaboration
Phase 3: Develop a Business Case and Roadmap
Phase 4: Implement a prioritised migration strategy
Phase 5: Extend facilities to support the discovery of and access to data within the NSS
NSS refers to Australia's National Statistical Service, the community of government agencies, led by the ABS as Australia's national statistical organisation, building a rich statistical picture for a better informed Australia. Phase 5 denotes that, once ABS statistical business processes are successfully operating on a transformed basis, the potential for elements of the enabling infrastructure and generic toolsets, possibly with extensions, to be harnessed by other producers of statistics within the NSS will be explored.
The following diagram illustrates anticipated outcomes and high level outputs from IMTP.
The ten outcomes are grouped into three sets
The outputs likewise span
The above model of outcomes and outputs should be considered indicative at this time. Details will be refined as the business case and roadmap for IMT overall are refined through cycles of engagement with stakeholders external and internal to the ABS that will guide progress from a high level vision to a detailed and fully co-ordinated implementation plan for the transformation.
Further explanation of this diagram is provided in the ABS presentation to the SDMX Global Conference in May 2011.
The challenges that provide the drivers for IMT must be addressed in one form or another. In order to achieve the transformation in a timely manner (eg in well under a decade), and realise maximum benefits for users of ABS (and other NSS) statistics, significant resources in addition to those allocated to undertaking and supporting current "business as usual" activities within the ABS will be required. A key activity within IMTP is identifying and progressing strategic options for securing relevant resources. This includes further developing the business case to detail the nature of the investment required and the evidence base that demonstrates the anticipated benefits will be realised in practice from this investment to the extent currently envisaged.
A transformation such as IMT requires a planned and managed, coherent and consistent approach to communication with senior management, key partners and stakeholders (within and beyond the ABS) and staff in general.
As noted in the discussion of initiation of IMTP, and drivers in an international context, collaboration in responding to fundamental challenges facing producers of official statistics around the world is a primary strategy for the ABS and many other agencies. This includes
As noted in the discussion of initiation of IMTP, the ABS currently provides the Secretariat for the Statistical Network, a key international group focused on collaborative development of new solutions. (The Statistical Network, during its foundation stage, is currently comprised of six NSIs). ABS also leads the specific Statistical Network collaboration related to Operationalising a Common Metadata/Information Management Framework (OCMIMF). The first major deliverable from OCMIMF is the Generic Statistical Information Model (GSIM). Both of these ABS roles related to international collaboration are delivered through the IMTP team.
A number of major developments within the ABS, several of which commenced prior to formal initiation of IMTP, are at the forefront of realising IMT. These relate to new capabilities for phases and sub processes such as data collection, editing and dissemination. (They could be considered an element of the "Generic Tool Sets" high level output.) These Pathfinders
Metadata Registry/Repository (MRR) and Statistical Workflow Management (SWM)
These represent the primary IT elements of Enabling Infrastructure. The MRR supports discovery and (re)use of data and metadata across the ABS and beyond. It is intended to support the information requirements in terms of inputs, guides, outputs and enablers of automated, semi-automated and manual business processes. The definition of "information object types" supported by the MRR, and the relationships between these "information object types", will align with the information models associated with standards such as SDMX and DDI. (SDMX and DDI do not yet span all detailed, structured information requirements associated with statistical business processes, with one likely outcome of the MRR work being requests to consider extensions to the existing standards.)
The SWM component can be viewed as a form of Business Process Management System (BPMS). Integration of MRR and SWM is required, for example, to ensure
The following diagram provides more information about MRR and SWM and the relationship between them.
A Proof Of Concept related to MRR and SWM was completed on 30 June 2011 and is currently being evaluated within the ABS. Assessment of this Proof Of Concept will provide learnings that shape staged development of "production strength" MRR and SWM Enabling Infrastructure.
Statistical Information Management Framework (SIMF)
This work includes
Further information in regard to IMTP was provided in slides 5 to 17 and slides 21 to 25 of the ABS presentation to the SDMX Global Conference in May 2011.
A paper http://live.unece.org/fileadmin/DAM/stats/documents/ece/ces/ge.50/2011/wp.10.e.pdf and presentation on IMTP was also delivered to MSIS (Meeting on the Management of Statistical Information Systems) in the same month.
For any general enquiries in regard to IMTP, the leader of the Communications Team within IMTP is Kylie Saines. Kylie can be contacted via email (firstname.lastname@example.org) or phone (+61 2 6252 7848).
For any enquiries specific to the Statistical Information Management Framework aspect of IMTP, including use of standards such as SDMX and DDI, the leader of the Information Management Team within IMTP is Alistair Hamilton. Al can be contacted via email (email@example.com) or phone (+61 2 6252 5416).