Message-ID: <1206665320.14396.1417133813750.JavaMail.confluence@ece-vmapps> Subject: Exported From Confluence MIME-Version: 1.0 Content-Type: multipart/related; boundary="----=_Part_14395_1948101526.1417133813749" ------=_Part_14395_1948101526.1417133813749 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Content-Location: file:///C:/exported.html
The process 5.3 Review, Validate and Edit should be made more inclusive = of aggregate or macro level input data sources that need to be validated pr= ior to transforming or compiling into an output (e.g. those inputting to a = national account). The current description uses the term 'microdata',= and sounds too much like it only applies to micro level survey or administ= rative data (e.g. business, household, person level records). In addi= tion, imputation could fall within this editing/validation process as it is= really just a method (often complex) of treating missing value type anomal= ies in the data. The title of the process could also be changed to so= mething like 'Validate inputs' to include what is being validated/edited, t= o avoid using similar terms, and to be more consistent with '6.2 Validate o= utputs'.
Eurostat: Wilhelmus Kloek
The split in 5.3 (Review, validate & edit) and 5.4 (Impute) fe= els unnatural to me. The more logical steps are detection of errors and cor= rection of errors. Imputation is just one approach to correction. My feelin= g is strenghtened by the fact that Eurostat will spend efforts on detecting= errors, but will usually report them back to the Member States and not sta= rt correcting the data originally received in order to avoid incoherence.= p>
By the way, the term data editing is confusing to anyone not used to thi= s terminology, and especilly to persons with an IT background. It gives the= impression of opening a text editor and do undocumented changes to the dat= a file. Data editing is not allowed!
My solution in point 4 on error detection and error correction is somewh= at simplistic in formulation. Imputation methods can also be used for missi= ng values and by extention as a modelling technique.The distinction between= 5.3 (review, validate & edit) and 6.2 (validate outputs) is unclear an= d perhaps not relevant.
Istat suggests to join former sub-processes 5.3 and 5.4 into the followi= ng sub-process 5.3.
5.3. Data validation - This sub-process applie= s to collected micro-data, and looks at each record to try to identify (and= where necessary correct) potential problems, errors and discrepancies such= as outliers, item non-response and miscoding. It can also be referred to a= s input data validation. It may be run iteratively, validating data against= predefined edit rules, usually in a set order. It may apply automatic edit= s, or raise alerts for manual inspection and correction of the data. Review= ing, validating and editing can apply to unit records both from surveys and= administrative sources, before and after integration. In certain cases, im= putation may be used as a form of editing.
Where data are missing or unreliable, estimates may be imputed. Specific= steps typically include:
If Istat suggestion is accepted, it is necessary to renumber all= the following sub-processes.
Please indicate your support for this change using the stars and legend = below