feat(datasets): periods (start_date/end_date), dg_id column, date filters; rename observation dates
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
29
README.md
29
README.md
@@ -72,10 +72,10 @@ the KATO column and the year columns:
|
||||
`parse_error`).
|
||||
3. The client polls `GET /datasets/{id}` until `awaiting_mapping`, then submits
|
||||
`POST /datasets/{id}/mapping` with the chosen `kato_column` and a
|
||||
`year_columns` map (each `{column, date}`). Validated against the detected
|
||||
columns; `status` → `extracting`.
|
||||
`year_columns` map (each `{column, start_date}`, optionally `end_date` for
|
||||
a period). Validated against the detected columns; `status` → `extracting`.
|
||||
4. A second worker job **unpivots** the attribute table into long-format
|
||||
`dataset_observations` — one row per `(kato_code, date)` with a numeric
|
||||
`dataset_observations` — one row per `(kato_code, start_date)` with a numeric
|
||||
`value` (or `value_text` for non-numeric cells); `status` → `ready`. Read
|
||||
them via `GET /datasets/{id}/observations` (paginated, optional
|
||||
`?kato_code=`).
|
||||
@@ -84,12 +84,25 @@ the KATO column and the year columns:
|
||||
curl -X POST https://dssgis.dwh.kz/datasets/<id>/mapping -H 'Content-Type: application/json' -d '{
|
||||
"kato_column": "като",
|
||||
"year_columns": [
|
||||
{"column": "F_2023", "date": "2023-01-01"},
|
||||
{"column": "D_2025", "date": "2025-01-01"}
|
||||
{"column": "F_2023", "start_date": "2023-01-01"},
|
||||
{"column": "D_2025", "start_date": "2025-01-01"}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
### Dataset periods
|
||||
|
||||
A dataset carries an optional period — `start_date` and `end_date`
|
||||
(`YYYY-MM-DD`, both nullable, `end_date` inclusive and never before
|
||||
`start_date`) — plus an optional `dg_id` linking it to a *dataset group*: the
|
||||
"одноимённые" datasets that describe the same thing for different periods. All
|
||||
three are accepted on `POST /datasets` (form fields), `PUT /datasets/{id}` and
|
||||
`PATCH /datasets/{id}` (an explicit `null`, or omission on PUT, clears them).
|
||||
|
||||
`GET /datasets` filters on them with `?dg_id=`, and with the inclusive bounds
|
||||
`?start_date_from=`, `?start_date_to=`, `?end_date_from=` and `?end_date_to=`
|
||||
(each `YYYY-MM-DD`).
|
||||
|
||||
## Getting started
|
||||
|
||||
```sh
|
||||
@@ -117,13 +130,13 @@ server runs it is served at `/openapi.yaml`, with an interactive **Redoc** UI at
|
||||
| GET | `/categories/{id}` | get |
|
||||
| PUT | `/categories/{id}` | update |
|
||||
| DELETE | `/categories/{id}` | delete |
|
||||
| GET | `/datasets` | paginated list of summaries (`?page=`, `?page_size=`, `?category_id=`) |
|
||||
| POST | `/datasets` | upload (multipart: `file`, `file_type`, `category_id`, `code`, `name`, `description?`, `unit?`, `meta?` (JSON), `automated?` (bool)) |
|
||||
| GET | `/datasets` | paginated list of summaries (`?page=`, `?page_size=`, `?category_id=`, `?dg_id=`, `?start_date_from=`, `?start_date_to=`, `?end_date_from=`, `?end_date_to=`) |
|
||||
| POST | `/datasets` | upload (multipart: `file`, `file_type`, `category_id`, `code`, `name`, `description?`, `unit?`, `meta?` (JSON), `automated?` (bool), `dg_id?`, `start_date?`, `end_date?`) |
|
||||
| GET | `/datasets/{id}` | full dataset (geometry as GeoJSON, `bbox` for rasters) |
|
||||
| GET | `/datasets/{id}.geojson` | GeoJSON `FeatureCollection`; plain `vector` returns its geometry as a single feature with the extracted attribute table as top-level properties; `vector_with_kato` always ignores its own geometry and joins the `districts` table by KATO, mapping observations onto each polygon |
|
||||
| GET | `/datasets/{id}/status` | processing status; long-polls with `?current=<status>` (holds up to `?wait=` secs, default 25, max 60) |
|
||||
| GET | `/datasets/{id}/download` | download the stored file |
|
||||
| POST | `/datasets/{id}/mapping` | set KATO column + year→date map (vector_with_kato) |
|
||||
| POST | `/datasets/{id}/mapping` | set KATO column + year→period map (vector_with_kato) |
|
||||
| GET | `/datasets/{id}/observations` | paginated unpivoted values (`?kato_code=`, `?page=`, `?page_size=`) |
|
||||
| DELETE | `/datasets/{id}` | delete (row + object) |
|
||||
|
||||
|
||||
Reference in New Issue
Block a user