This is in response to the meeting about potential collaboration between the DDI and SDMX initiatives that was held in the margins of the December 2010 European DDI Users Group meeting in Utrecht. The response focuses on the broader idea of collaboration, the objectives of such work, and possible next steps.
The DDI Alliance strongly endorses collaboration between the two groups for several reasons. First, we share the overarching goal of providing rich, standardized metadata to support effective data use and exchange. Second, while our original sets of stakeholders may have differed, there is increasing overlap between the communities served by the two standards. Third, a key guiding principle behind DDI is to facilitate interoperability with existing and emerging metadata standards.
The objectives set forth in the December meeting are compelling.
1. To avoid duplication of effort by the two standards, and thus avoid confusion about which standards should be used for specific types of applications. This is important because there has been a fair amount of confusion about which standard to apply for which purpose, partly as a result of the increasingly confusing metadata standards landscape more generally. People are inundated with information about standards and do not have enough guidance about how they relate to one another. The more we can dispel the confusion regarding DDI and SDMX and provide best practice guidelines based on typical use cases, the better. Note that there are similar efforts in the metadata community with respect to using standards together, e.g., using the METS packaging standard with the PREMIS preservation metadata standard .
2. To provide reassurance to the user communities of both DDI and SDMX that the end-to-end statistical process can be managed, and that the standards bodies are both considering the needs of users in this area. DDI 3 was developed with this end-to-end life cycle model in mind; a recent paper on Metadata-Driven Survey Design illustrates this concept very well. As Steven Vale points out in his paper on the relationships among GSBPM, SDMX, and DDI , in the academic community it is more common for the end-to-end process to include multiple hand-offs to multiple parties. This has resulted in the loss of metadata over the life course of research data, and it is important to remedy this situation.
3. To provide specific technical guidance about the use cases and implementation of the standards for specific purposes. This follows logically from the above and should be a key outcome of collaboration.
Continuing the dialogue already begun seems critical to success. With the upcoming March meeting in Lisbon and the May SDMX meeting in Washington, DC, there are more opportunities to interact, which is welcomed. Beyond these meetings, the DDI Alliance endorses the idea of a working group in this area that would draw from the DDI, SDMX, and GSBPM communities. A means of virtual communication would also be helpful.
Mary Vardigan, Director, DDI Alliance
Respectfully submitted 2011-03-24 on behalf of the DDI Alliance Steering Committee