4.4 Dataset Subscription Service

Dataset Subscription Service now is available on Dtwin.city

The subscription service includes the following regular (quarterly) works:

data preparation
obtaining programmatic access to external data sources with indicators of socio-economic and spatial development
profiling primary data sources
downloading and recognizing raw data
confirming the completeness of primary data from sources
maintenance of directories (indicators, analytical measurements, scenarios, versions, stages of processing), data models of sources and recipients, as well as correspondence tables, taking into account changes over time
documentation of changes in the composition and structure of data sources and recipients
documenting changes in accounting methods
data processing
elimination of technical errors in primary data formats, including field naming, value formats, shifts in data series, and gaps in analytical data segmentation
aggregation of downloaded primary data and reduction to a reference structure
detection of changes in primary data, including retrospectively, including in the data structure, directories, and actual values
recovery of gaps, elimination of duplication of data
validation and proposals for adjusting the values of indicators by econometric methods according to a set of rules, including
meeting balance ratios
being in the range of acceptable intervals
satisfying the ratios of the principal components (eigenvectors)
providing data
creation and support of a programming interface (REST API) to directories, sets of primary and analytically verified data
uploading value sets and reference data in csv, xlsx, parquet, qs, fst data formats
documentation of collection, processing, and provision processes
putting down scenarios (fact assessment, forecast, plan, scenario, and goal), stages (initial, amended, corrected, and stage number), versions, and methods of accounting indicator values, which is necessary for the correct interpretation and tracing of indicator values
maintaining a library of verification methods, rules for verification and validation of indicator values
preparation of an interactive report on the scope, completeness, and identified errors