|VI. Managing tensions between national statistical offices and researchers||Managing Statistical Confidentiality and Microdata Access||VIII. Some special issues|
Managing decision-making on confidentiality
83. There is always some chance of identification, even if very small. Software now exists which can estimate the proportion of records which are unique and therefore at some risk of identification.
84. It is the Chief Statistician or his or her delegate who needs to make the decision on the release of a microdata file, whether it be by an anonymised microdata file (public-use or licensed), through a remote access facility, or through a data laboratory. In order to make that decision, the Chief Statistician needs advice on whether, for example:
- the risk of identification is sufficiently small;
- the adjustments made to the data items have not unduly damaged the microdata file for research purposes; and
- the variables that have been collapsed are the most appropriate, taking into account both the needs of researchers and the identification risk.
85. As an illustration of the last point, choices can be made between the amount of detail provided on identifying variables such as geography, age and household structure.
86. Appropriate arrangements should be put in place to provide this advice on a consistent basis. It often needs to be supported by a research capability and could be located in a methodology area. Case studies 17 and 18 describe the arrangements in Slovenia and Australia respectively.
87. If users are to make effective use of microdata, they must have access to the appropriate metadata. This would include:
- (i) a description of the survey including any information on quality;
- (ii) a list of the data items and the classifications used (sometimes referred to as a 'data dictionary'); and
- (iii) definitions of the data items.
Provision of (i) will help ensure that the microdata are not used if the data are not really fit for the intended purpose.
88. As microdata are provided electronically, the metadata must be provided in a way that is accessible. If possible, the metadata should be released in association with the microdata. Printed copy may still be an effective means although the NSO web site is becoming increasingly useful for these purposes.
Managing breaches by the researcher
89. Efforts should be made to reduce the likelihood of breaches as outlined in the previous chapter. Nevertheless, breaches may occur and procedures for dealing with the breaches should be determined.
90. Breaches must be treated seriously. If this is not done, public confidence in the arrangements will erode. Also, breaches will occur more often if they are not treated seriously.
91. There are a number of ways of dealing with breaches. For example, if a legal offence has occurred, legal action should be considered. This is expensive but is essential in more severe cases to demonstrate the importance the NSO places on confidentiality, and reduce the likelihood of future offences.
92. Also, the researcher should be prevented from further access to microdata. This should be the minimum step that is undertaken.
93. Consideration should also be given to stopping further release to the institution of the researcher, at least until:
- (i) the institution has taken appropriate steps in dealing with the offence committed by the researcher; and
- (ii) the NSO is confident that the appropriate arrangements are in place within the institution to minimise the chance of further breaches.
94. The research community will generally be supportive of taking strong action against the relatively small number of offenders who may give the research community a bad name. It is in their long-term interest.
95. For minor breaches, a warning may be sufficient.