Need to produce DDI metadata or export ASCII out of a proprietary data file? Want to generate an import script for ingesting text raw data into a statistical package or database? Have some standard XML file that needs validation, enhancement, or conversion? Need to convert an Excel file into SDMX? How about generating summary statistics for inclusion in DDI or a report? Managing and coping with myriads of data and metadata files across formats and version can be daunting. While various tools and utilities are embedded in statistical and data management software or available as stand alone commercial packages to support some of these operations, none offer a comprehensive solution and stitching such tasks together is often challenging and frustrating.
The OpenMetadata File Manager aims to address these needs by providing an online service center for the management of statistical data files and related metadata. Users have the ability to upload their data/metadata files for exploration, processing, conversion, quality assurance, sharing, and other purposes. The long term vision of the the service is to deliver a comprehensive platform that delivers complex workflow processing, supports a broad range of data and metadata file formats, facilitates publication in online data services or big data engines, and facilitates harmonization or the production of linked data.
The service is currently in its initial development stages and available for private beta-preview to OpenMetadata registered users who can request access by contact us. The short term objectives of the service is to support the following features:
- Read in SPSS, Stata, and other common statistical package formats
- Production of DDI-C, DDI-C 2.5,DDI-L, Triple-S metadata from data files
- Export data to ASCII text formats
- Generate scrips/setup files for ingestion of ASCII+DDI data info common packages (SAS, Stata, SPSS, R) or database (SQL, BigData)
- Basic workflow processing
The environment leverages metadata standards such a DDI and SDMX and builds upon libraries and tools from Metadata Technology as well as numerous open source packages.