Files
gis/docs/superpowers/plans/2026-09-05-landscape-program.md
2026-09-05 13:08:25 +05:00

2662 lines
108 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Landscape Program + Dataset Groups Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add dataset groups + dataset periods (with the `date`→`start_date` rename), and the Landscape Program raster pipeline (godal-backed, incremental, results stored as raster datasets).
**Architecture:** Part 1 extends the existing dataset stack (domain → postgres repo → service → chi handler) with a `dataset_groups` entity and period columns. Part 2 adds a pure-Go compute engine (`internal/landscape`: graph, per-window kernels, `Compute`) behind a swappable `raster.IO` interface (godal implementation), and a `LandscapeService` that plans runs (fingerprint-based skip/reuse), resolves inputs from dataset groups, downloads inputs, computes, and publishes outputs as raster datasets that the existing COG conversion picks up. A RabbitMQ consumer executes runs in the worker.
**Tech Stack:** Go 1.26 (toolchain go1.26.2 pinned), chi, pgx, goose, MinIO, RabbitMQ, GDAL via `github.com/airbusgeo/godal` v0.0.18 (cgo) + existing GDAL CLI tools.
**Spec:** `docs/superpowers/specs/2026-09-05-landscape-program-design.md`
## Global Constraints
- Build with `CGO_ENABLED=1`; local GDAL is Homebrew 3.13.3 (`pkg-config --modversion gdal`). The local Go SDK is incomplete; `go.mod` pins `toolchain go1.26.2`, so plain `go test ./...` works.
- Follow existing layering: `internal/domain` (no internal deps) → `internal/repository/postgres` → `internal/service` (interfaces declared in service) → `internal/transport/http`. Wiring only in `internal/app/app.go`.
- Errors: repositories return `domain.ErrNotFound` / `domain.ErrConflict` / `domain.ErrValidation` (wrapped with `%w`); handlers use `respondDomainError`.
- Dates in JSON are strings `YYYY-MM-DD` (`time.Parse("2006-01-02", …)`).
- Output rasters: `int16`, nodata `-9999`, `TILED=YES`, `BLOCKXSIZE=512`, `BLOCKYSIZE=512`, `COMPRESS=DEFLATE`, `PREDICTOR=2`, `BIGTIFF=IF_SAFER`.
- Pixel validity: a value is valid iff `1 <= v <= 5`. Anything else (including nodata) is invalid. Never resample; grids must match exactly or the node fails.
- Group codes (slots and outputs) are fixed strings listed in Task 4; they are the single source of truth for both the graph and the seed migration.
- Tests: table-driven `testing` package, stubs like `internal/service/dataset_test.go`. Run only your own packages while other agents work in parallel (`go test ./internal/landscape/...`), not `go test ./...`, unless you are the integration task.
- **Parallel execution rule:** several agents edit the same working tree on branch `feature/landscape-program`. Only touch the files listed in your task. Never run `git add -A`/`git add .`; add explicit paths. If `git commit` fails with `index.lock`, wait 2 s and retry.
## Execution waves
| Wave | Tasks (parallel within a wave) | Depends on |
|---|---|---|
| 1 | Task 1 → Task 2 (same lane, sequential); Task 3; Task 4; Task 5 | — |
| 2 | Task 6; Task 7 | 3, 4 (Task 6); 2, 4, 5 (Task 7) |
| 3 | Task 8 | 6, 7 |
---
## File structure
**Part 1**
- `migrations/00010_dataset_groups_and_dates.sql` — new table, dataset columns, observation rename, year_columns key rename.
- `internal/domain/dataset.go` — modify: rename fields, add group/date fields, `DatasetFields`, filter fields.
- `internal/domain/dataset_group.go` — create: `DatasetGroup`.
- `internal/repository/postgres/dataset.go` — modify: columns, filter clause, `UpdateFields(DatasetFields)`, `LatestInGroup`, observation columns.
- `internal/repository/postgres/dataset_group.go` — create: group repository.
- `internal/service/dataset.go` — modify: rename, upload/update inputs, `CreateRasterFromFile`, SLD fallback.
- `internal/service/dataset_group.go` — create: group service.
- `internal/service/dataset_test.go` — modify: stubs + tests. `internal/service/dataset_group_test.go` — create.
- `internal/transport/http/dataset_handler.go` — modify. `internal/transport/http/dataset_group_handler.go` — create. `internal/transport/http/router.go`, `internal/app/app.go` — modify (wiring).
- `api/openapi.yaml`, `README.md` — modify.
**Part 2**
- `internal/raster/io.go` (interfaces), `internal/raster/godal.go` (+ `godal_test.go`), `build/package/Dockerfile`, `go.mod`.
- `internal/landscape/graph.go`, `params.go`, `kernels.go` (+ tests), `compute.go` (+ `compute_test.go` with a fake IO).
- `internal/domain/landscape.go`, `migrations/00011_landscape.sql`, `internal/repository/postgres/landscape.go`.
- `internal/service/landscape.go` (+ test), `internal/messaging/rabbitmq/landscape_consumer.go`.
- `internal/transport/http/landscape_handler.go`, `router.go`, `app.go`, `api/openapi.yaml`, `README.md`.
---
### Task 1: Dataset periods — rename `date`→`start_date`, add dataset `start_date`/`end_date`, list filters, upload/PUT/PATCH fields
**Files:**
- Create: `migrations/00010_dataset_groups_and_dates.sql` (the *whole* migration incl. the `dataset_groups` table so Task 2 does not need another migration)
- Modify: `internal/domain/dataset.go`, `internal/repository/postgres/dataset.go`, `internal/service/dataset.go`, `internal/service/dataset_test.go`, `internal/transport/http/dataset_handler.go`, `api/openapi.yaml`, `README.md`
**Interfaces:**
- Produces: `domain.Observation{StartDate string; EndDate *string}`, `domain.YearColumn{Column, StartDate string; EndDate *string}`, `domain.Dataset/DatasetSummary{DatasetGroupID *uuid.UUID; StartDate, EndDate *string}`, `domain.DatasetFilter{…, DatasetGroupID *uuid.UUID; StartDateFrom, StartDateTo, EndDateFrom, EndDateTo *string}`, `domain.DatasetFields`, repo `UpdateFields(ctx, id uuid.UUID, f domain.DatasetFields) (domain.Dataset, error)`, service `UploadInput{…; DatasetGroupID *uuid.UUID; StartDate, EndDate *string}`, `DatasetUpdateInput`/`DatasetPatchInput` with the same three fields (+ `DatasetGroupIDSet`, `StartDateSet`, `EndDateSet` on patch).
- Task 2 relies on `dg_id` column + `DatasetGroupID` field existing after this task, and on `domain.ValidateDatePeriod`.
- [ ] **Step 1: Write the migration**
```sql
-- migrations/00010_dataset_groups_and_dates.sql
-- +goose Up
-- Dataset groups: "одноимённые" datasets versioned by period. A group may carry a
-- shared SLD used as a fallback for its datasets.
CREATE TABLE dataset_groups (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
code VARCHAR(255) NOT NULL UNIQUE,
name VARCHAR(255) NOT NULL,
description TEXT NOT NULL DEFAULT '',
sld_storage_key TEXT,
sld_filename VARCHAR(255),
sld_content_type VARCHAR(255),
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
ALTER TABLE datasets ADD COLUMN dg_id UUID REFERENCES dataset_groups (id) ON DELETE RESTRICT;
ALTER TABLE datasets ADD COLUMN start_date DATE;
ALTER TABLE datasets ADD COLUMN end_date DATE;
ALTER TABLE datasets ADD CONSTRAINT chk_datasets_period
CHECK (start_date IS NULL OR end_date IS NULL OR end_date >= start_date);
CREATE INDEX idx_datasets_dg_id_end_date ON datasets (dg_id, end_date DESC NULLS LAST, created_at DESC);
CREATE INDEX idx_datasets_start_date ON datasets (start_date);
CREATE INDEX idx_datasets_end_date ON datasets (end_date);
-- Period columns are renamed everywhere: observations, and the year_columns
-- mapping JSON ({column, date, date_to} -> {column, start_date, end_date}).
ALTER TABLE dataset_observations RENAME COLUMN date TO start_date;
ALTER TABLE dataset_observations RENAME COLUMN date_to TO end_date;
DROP INDEX uq_dataset_observations;
DROP INDEX idx_dataset_observations_kato_date;
CREATE UNIQUE INDEX uq_dataset_observations ON dataset_observations (dataset_id, kato_code, start_date, COALESCE(end_date, start_date));
CREATE INDEX idx_dataset_observations_kato_date ON dataset_observations (kato_code, start_date);
UPDATE datasets
SET year_columns = (
SELECT jsonb_agg(
(elem - 'date' - 'date_to')
|| jsonb_build_object('start_date', elem->'date')
|| CASE WHEN elem ? 'date_to' AND elem->'date_to' IS NOT NULL
THEN jsonb_build_object('end_date', elem->'date_to') ELSE '{}'::jsonb END)
FROM jsonb_array_elements(year_columns) AS elem)
WHERE year_columns IS NOT NULL AND jsonb_typeof(year_columns) = 'array' AND jsonb_array_length(year_columns) > 0;
-- +goose Down
UPDATE datasets
SET year_columns = (
SELECT jsonb_agg(
(elem - 'start_date' - 'end_date')
|| jsonb_build_object('date', elem->'start_date')
|| CASE WHEN elem ? 'end_date' AND elem->'end_date' IS NOT NULL
THEN jsonb_build_object('date_to', elem->'end_date') ELSE '{}'::jsonb END)
FROM jsonb_array_elements(year_columns) AS elem)
WHERE year_columns IS NOT NULL AND jsonb_typeof(year_columns) = 'array' AND jsonb_array_length(year_columns) > 0;
DROP INDEX uq_dataset_observations;
DROP INDEX idx_dataset_observations_kato_date;
ALTER TABLE dataset_observations RENAME COLUMN start_date TO date;
ALTER TABLE dataset_observations RENAME COLUMN end_date TO date_to;
CREATE UNIQUE INDEX uq_dataset_observations ON dataset_observations (dataset_id, kato_code, date, COALESCE(date_to, date));
CREATE INDEX idx_dataset_observations_kato_date ON dataset_observations (kato_code, date);
DROP INDEX idx_datasets_dg_id_end_date;
DROP INDEX idx_datasets_start_date;
DROP INDEX idx_datasets_end_date;
ALTER TABLE datasets DROP CONSTRAINT chk_datasets_period;
ALTER TABLE datasets DROP COLUMN dg_id;
ALTER TABLE datasets DROP COLUMN start_date;
ALTER TABLE datasets DROP COLUMN end_date;
DROP TABLE dataset_groups;
```
- [ ] **Step 2: Domain — rename and add fields** (`internal/domain/dataset.go`)
Rename `Observation.Date`→`StartDate` (`json:"start_date"`), `Observation.DateTo`→`EndDate` (`json:"end_date,omitempty"`), `YearColumn.Date`→`StartDate` (`json:"start_date"`), `YearColumn.DateTo`→`EndDate` (`json:"end_date,omitempty"`). Update doc comments accordingly. Add to both `Dataset` and `DatasetSummary` (after `CategoryID`):
```go
// DatasetGroupID links the dataset to its dataset group ("одноимённые"
// datasets versioned by period). Nullable.
DatasetGroupID *uuid.UUID `json:"dg_id"`
// StartDate / EndDate is the period the dataset describes (YYYY-MM-DD).
// Both nullable; EndDate drives the "freshest in group" rule.
StartDate *string `json:"start_date"`
EndDate *string `json:"end_date"`
```
Add to `DatasetFilter` (document: dates are inclusive `YYYY-MM-DD` bounds):
```go
DatasetGroupID *uuid.UUID
StartDateFrom *string
StartDateTo *string
EndDateFrom *string
EndDateTo *string
```
Add:
```go
// DatasetFields is the set of editable metadata columns written by
// UpdateFields. Nil pointers clear the corresponding nullable column.
type DatasetFields struct {
CategoryID uuid.UUID
Name string
Description *string
Unit *string
DatasetGroupID *uuid.UUID
StartDate *string
EndDate *string
}
// ValidateDatePeriod checks that start/end (YYYY-MM-DD, either may be nil)
// parse and that end is not before start. Returns an ErrValidation-wrapped
// error naming the offending field.
func ValidateDatePeriod(start, end *string) error {
var from, to time.Time
var err error
if start != nil {
if from, err = time.Parse("2006-01-02", *start); err != nil {
return fmt.Errorf("%w: start_date %q is not YYYY-MM-DD", ErrValidation, *start)
}
}
if end != nil {
if to, err = time.Parse("2006-01-02", *end); err != nil {
return fmt.Errorf("%w: end_date %q is not YYYY-MM-DD", ErrValidation, *end)
}
}
if start != nil && end != nil && to.Before(from) {
return fmt.Errorf("%w: end_date %q is before start_date %q", ErrValidation, *end, *start)
}
return nil
}
```
- [ ] **Step 3: Domain test** (`internal/domain/dataset_test.go`, append)
```go
func TestValidateDatePeriod(t *testing.T) {
s := func(v string) *string { return &v }
cases := []struct {
name string
start, end *string
wantErr bool
}{
{"both nil", nil, nil, false},
{"start only", s("2020-01-01"), nil, false},
{"ordered", s("2020-01-01"), s("2020-12-31"), false},
{"same day", s("2020-01-01"), s("2020-01-01"), false},
{"reversed", s("2020-12-31"), s("2020-01-01"), true},
{"bad start", s("01.01.2020"), nil, true},
{"bad end", nil, s("2020-13-01"), true},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
err := ValidateDatePeriod(tc.start, tc.end)
if (err != nil) != tc.wantErr {
t.Fatalf("err=%v wantErr=%v", err, tc.wantErr)
}
if err != nil && !errors.Is(err, ErrValidation) {
t.Fatalf("expected ErrValidation, got %v", err)
}
})
}
}
```
Run: `go test ./internal/domain/ -run TestValidateDatePeriod -v` → PASS after Step 2.
- [ ] **Step 4: Repository** (`internal/repository/postgres/dataset.go`)
`datasetColumns`: insert `dg_id, to_char(start_date, 'YYYY-MM-DD') AS start_date, to_char(end_date, 'YYYY-MM-DD') AS end_date,` right after `category_id,`; `scanDataset` scans `&d.DatasetGroupID, &d.StartDate, &d.EndDate` in the same position. Same for `datasetSummaryColumns`/`scanDatasetSummary`.
`Create`: insert `dg_id, start_date, end_date` (`$14,$15,$16`, values `d.DatasetGroupID, d.StartDate, d.EndDate`; pgx maps `*string` → `date` text fine, use `$15::date, $16::date` casts).
Replace `UpdateFields` with:
```go
// UpdateFields overwrites a dataset's editable metadata columns. Nil pointers
// clear the nullable columns. Returns domain.ErrNotFound if the dataset does
// not exist, or domain.ErrConflict if the category/group does not exist.
func (r *DatasetRepository) UpdateFields(ctx context.Context, id uuid.UUID, f domain.DatasetFields) (domain.Dataset, error) {
row := r.pool.QueryRow(ctx,
`UPDATE datasets
SET category_id = $2, name = $3, description = $4, unit = $5,
dg_id = $6, start_date = $7::date, end_date = $8::date, updated_at = now()
WHERE id = $1
RETURNING `+datasetColumns,
id, f.CategoryID, f.Name, f.Description, f.Unit, f.DatasetGroupID, f.StartDate, f.EndDate,
)
out, err := scanDataset(row)
return out, mapError(err)
}
```
`datasetFilterClause`: append
```go
if f.DatasetGroupID != nil {
args = append(args, *f.DatasetGroupID)
conds = append(conds, fmt.Sprintf("dg_id = $%d", len(args)))
}
for _, b := range []struct {
col, op string
val *string
}{
{"start_date", ">=", f.StartDateFrom}, {"start_date", "<=", f.StartDateTo},
{"end_date", ">=", f.EndDateFrom}, {"end_date", "<=", f.EndDateTo},
} {
if b.val != nil {
args = append(args, *b.val)
conds = append(conds, fmt.Sprintf("%s %s $%d::date", b.col, b.op, len(args)))
}
}
```
Add:
```go
// LatestInGroup returns the freshest dataset of a group: greatest end_date
// (NULL end_date sorts last), ties broken by created_at DESC. status and
// fileType optionally restrict candidates. Returns domain.ErrNotFound when no
// dataset matches.
func (r *DatasetRepository) LatestInGroup(ctx context.Context, groupID uuid.UUID, status *string, fileType *domain.FileType) (domain.Dataset, error) {
where, args := datasetFilterClause(domain.DatasetFilter{DatasetGroupID: &groupID, Status: status, FileType: fileType}, nil)
row := r.pool.QueryRow(ctx,
`SELECT `+datasetColumns+` FROM datasets`+where+
` ORDER BY end_date DESC NULLS LAST, created_at DESC LIMIT 1`, args...)
out, err := scanDataset(row)
return out, mapError(err)
}
```
Observations: `ReplaceObservations` uses `o.StartDate`/`o.EndDate`, copies into columns `"start_date", "end_date"`; `observationColumns` becomes `… to_char(start_date, 'YYYY-MM-DD') AS start_date, to_char(end_date, 'YYYY-MM-DD') AS end_date, …`; `scanObservation` scans `&o.StartDate, &o.EndDate`; `ORDER BY kato_code, date` → `ORDER BY kato_code, start_date` (three places: ListObservations ×2, ListAllObservations in `district.go`).
- [ ] **Step 5: Service** (`internal/service/dataset.go`)
- Interface `DatasetRepository`: `UpdateFields(ctx, id uuid.UUID, f domain.DatasetFields) (domain.Dataset, error)`; add `LatestInGroup(ctx context.Context, groupID uuid.UUID, status *string, fileType *domain.FileType) (domain.Dataset, error)`.
- `UploadInput`: add `DatasetGroupID *uuid.UUID; StartDate *string; EndDate *string`. In `Upload`, after category check: `if err := domain.ValidateDatePeriod(in.StartDate, in.EndDate); err != nil { return domain.Dataset{}, err }` and pass the three fields into `repo.Create`.
- `DatasetUpdateInput`: add the three fields; `Update` validates the period and calls `s.repo.UpdateFields(ctx, id, domain.DatasetFields{CategoryID: in.CategoryID, Name: in.Name, Description: in.Description, Unit: in.Unit, DatasetGroupID: in.DatasetGroupID, StartDate: in.StartDate, EndDate: in.EndDate})`.
- `DatasetPatchInput`: add `DatasetGroupID *uuid.UUID; DatasetGroupIDSet bool; StartDate *string; StartDateSet bool; EndDate *string; EndDateSet bool`; `Patch` merges like description/unit, validates the merged period, calls `UpdateFields` with `DatasetFields`.
- Rename all `yc.Date`/`yc.DateTo`/`o.Date`/`o.DateTo` to `StartDate`/`EndDate`; error messages say `start_date`/`end_date`. GeoJSON `data`/`periods` maps are keyed by `o.StartDate` and the period value is `*o.EndDate` (behaviour unchanged).
- [ ] **Step 6: Handler** (`internal/transport/http/dataset_handler.go`)
- `yearColumnInput`: `StartDate string \`json:"start_date" validate:"required,datetime=2006-01-02"\``, `EndDate *string \`json:"end_date" validate:"omitempty,datetime=2006-01-02"\``; build `domain.YearColumn{Column, StartDate, EndDate}`.
- `upload`: parse optional form values `dg_id` (UUID → 422 "dg_id must be a valid UUID"), `start_date`, `end_date` (blank → nil; format checked by the service) and pass into `UploadInput`.
- `list`: parse `dg_id` with `parseOptionalUUIDQuery`; for each of `start_date_from`, `start_date_to`, `end_date_from`, `end_date_to`: trim; if non-empty, `time.Parse("2006-01-02", v)` else 400 `"<name> must be YYYY-MM-DD"`; set the filter pointer.
- `datasetPutRequest`: add `DgID *string \`json:"dg_id" validate:"omitempty,uuid"\``, `StartDate *string \`json:"start_date" validate:"omitempty,datetime=2006-01-02"\``, `EndDate *string \`json:"end_date" validate:"omitempty,datetime=2006-01-02"\``; map into `DatasetUpdateInput` (parse `dg_id` UUID).
- `patch`: extend the raw-map handling used for description/unit: keys `dg_id`, `start_date`, `end_date`; `null` → `Set=true, nil`; string → parsed (`dg_id` must be UUID → 422; dates `datetime` → 422 "start_date must be YYYY-MM-DD").
- [ ] **Step 7: Update tests** (`internal/service/dataset_test.go`)
Update the stub repo: `UpdateFields(_ context.Context, id uuid.UUID, f domain.DatasetFields)` assigns all seven fields; add `LatestInGroup` to the stub:
```go
func (r *stubDatasetRepo) LatestInGroup(_ context.Context, groupID uuid.UUID, status *string, fileType *domain.FileType) (domain.Dataset, error) {
var best *domain.Dataset
for _, d := range r.store {
d := d
if d.DatasetGroupID == nil || *d.DatasetGroupID != groupID {
continue
}
if status != nil && d.Status != *status {
continue
}
if fileType != nil && d.FileType != *fileType {
continue
}
if best == nil || laterPeriod(d, *best) {
best = &d
}
}
if best == nil {
return domain.Dataset{}, domain.ErrNotFound
}
return *best, nil
}
// laterPeriod reports whether a is fresher than b: greater end_date, NULL
// end_date last, then newer created_at.
func laterPeriod(a, b domain.Dataset) bool {
switch {
case a.EndDate != nil && b.EndDate == nil:
return true
case a.EndDate == nil && b.EndDate != nil:
return false
case a.EndDate != nil && b.EndDate != nil && *a.EndDate != *b.EndDate:
return *a.EndDate > *b.EndDate
default:
return a.CreatedAt.After(b.CreatedAt)
}
}
```
Rename every `Date:`/`DateTo:` literal and `.Date`/`.DateTo` access to `StartDate`/`EndDate`. Add tests:
```go
func TestDatasetService_Upload_StoresPeriodAndGroup(t *testing.T) {
repo := newStubDatasetRepo()
svc := newTestService(repo) // use the existing helper in this file that builds a service with stub store/categories/jobs
gid := uuid.New()
start, end := "2020-01-01", "2020-12-31"
ds, err := svc.Upload(context.Background(), validRasterUpload(t, func(in *UploadInput) {
in.DatasetGroupID = &gid
in.StartDate = &start
in.EndDate = &end
}))
if err != nil {
t.Fatal(err)
}
if ds.DatasetGroupID == nil || *ds.DatasetGroupID != gid || *ds.StartDate != start || *ds.EndDate != end {
t.Fatalf("period/group not stored: %+v", ds)
}
}
func TestDatasetService_Upload_RejectsReversedPeriod(t *testing.T) {
svc := newTestService(newStubDatasetRepo())
start, end := "2021-01-01", "2020-12-31"
_, err := svc.Upload(context.Background(), validRasterUpload(t, func(in *UploadInput) {
in.StartDate = &start
in.EndDate = &end
}))
if !errors.Is(err, domain.ErrValidation) {
t.Fatalf("want ErrValidation, got %v", err)
}
}
```
(If no `newTestService`/`validRasterUpload` helper exists yet, add them next to the existing upload tests by extracting the setup the existing `TestDatasetService_Upload_Success` uses; the helpers take a mutator `func(*UploadInput)`.)
- [ ] **Step 8: Run tests** — `go test ./internal/domain/ ./internal/service/ ./internal/transport/... ./internal/repository/...` → PASS; `go vet ./internal/...` clean.
- [ ] **Step 9: OpenAPI + README**
`api/openapi.yaml`: `YearColumn` (`start_date` required, `end_date`), `Observation` (`start_date`, `end_date`), `MappingInput` example, `/datasets/{id}/mapping` description; `Dataset` and `DatasetSummary` gain `dg_id` (uuid, nullable), `start_date`, `end_date` (`format: date`, nullable); `DatasetPut`/`DatasetPatch` gain the same; `POST /datasets` form gains `dg_id`, `start_date`, `end_date`; `GET /datasets` parameters `dg_id`, `start_date_from`, `start_date_to`, `end_date_from`, `end_date_to`. `README.md`: mapping example uses `start_date`; add a "Dataset periods" paragraph describing the fields and filters.
- [ ] **Step 10: Commit** (lane A only; skip if the orchestrator commits)
```bash
git add migrations/00010_dataset_groups_and_dates.sql internal/domain/dataset.go internal/domain/dataset_test.go internal/repository/postgres/dataset.go internal/repository/postgres/district.go internal/service/dataset.go internal/service/dataset_test.go internal/transport/http/dataset_handler.go api/openapi.yaml README.md
git commit -m "feat(datasets): periods (start_date/end_date), dg_id column, date filters; rename observation dates"
```
---
### Task 2: Dataset groups — CRUD, latest, group SLD, dataset SLD fallback, `CreateRasterFromFile`
**Files:**
- Create: `internal/domain/dataset_group.go`, `internal/repository/postgres/dataset_group.go`, `internal/service/dataset_group.go`, `internal/service/dataset_group_test.go`, `internal/transport/http/dataset_group_handler.go`
- Modify: `internal/service/dataset.go`, `internal/service/dataset_test.go`, `internal/transport/http/router.go`, `internal/app/app.go`, `api/openapi.yaml`, `README.md`
**Interfaces:**
- Consumes: Task 1 (`DatasetGroupID`, `LatestInGroup`, `DatasetFields`).
- Produces (used by Task 7):
- `domain.DatasetGroup`
- `postgres.DatasetGroupRepository` with `GetByCode(ctx, code string) (domain.DatasetGroup, error)`, `GetByID`.
- `service.DatasetGroupRepository` interface (below), `service.NewDatasetGroupService(repo DatasetGroupRepository, datasets DatasetRepository, store ObjectStore)`.
- `service.DatasetService.CreateRasterFromFile(ctx, in CreateRasterInput) (domain.Dataset, error)` and `service.CreateRasterInput{CategoryID uuid.UUID; DatasetGroupID *uuid.UUID; Name, Filename, Path string; StartDate, EndDate *string; Meta json.RawMessage; Automated bool}`.
- `service.NewDatasetService(repo, store, categories, groups groupReader, jobs, parseColumns, parseRows, converter)` — **new 4th parameter** `groups groupReader` where `type groupReader interface{ GetByID(ctx context.Context, id uuid.UUID) (domain.DatasetGroup, error) }`.
- [ ] **Step 1: Domain** (`internal/domain/dataset_group.go`)
```go
package domain
import (
"time"
"github.com/google/uuid"
)
// DatasetGroup groups "одноимённые" datasets (the same indicator delivered for
// different periods). The freshest dataset of a group is the one with the
// greatest end_date. A group may carry a shared SLD style used as a fallback by
// datasets that have none of their own.
type DatasetGroup struct {
ID uuid.UUID `json:"id"`
Code string `json:"code"`
Name string `json:"name"`
Description string `json:"description"`
SldStorageKey *string `json:"sld_storage_key"`
SldFilename *string `json:"sld_filename"`
SldContentType *string `json:"sld_content_type"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
```
- [ ] **Step 2: Repository** (`internal/repository/postgres/dataset_group.go`)
```go
package postgres
import (
"context"
"fmt"
"strings"
"gis/internal/domain"
"github.com/google/uuid"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
)
// DatasetGroupRepository persists dataset groups in Postgres.
type DatasetGroupRepository struct{ pool *pgxpool.Pool }
// NewDatasetGroupRepository returns a DatasetGroupRepository backed by pool.
func NewDatasetGroupRepository(pool *pgxpool.Pool) *DatasetGroupRepository {
return &DatasetGroupRepository{pool: pool}
}
const datasetGroupColumns = `id, code, name, description, sld_storage_key, sld_filename, sld_content_type, created_at, updated_at`
func scanDatasetGroup(row pgx.Row) (domain.DatasetGroup, error) {
var g domain.DatasetGroup
err := row.Scan(&g.ID, &g.Code, &g.Name, &g.Description, &g.SldStorageKey, &g.SldFilename, &g.SldContentType, &g.CreatedAt, &g.UpdatedAt)
return g, err
}
func (r *DatasetGroupRepository) Create(ctx context.Context, g domain.DatasetGroup) (domain.DatasetGroup, error) {
row := r.pool.QueryRow(ctx,
`INSERT INTO dataset_groups (code, name, description) VALUES ($1, $2, $3) RETURNING `+datasetGroupColumns,
g.Code, g.Name, g.Description)
out, err := scanDatasetGroup(row)
return out, mapError(err)
}
func (r *DatasetGroupRepository) GetByID(ctx context.Context, id uuid.UUID) (domain.DatasetGroup, error) {
out, err := scanDatasetGroup(r.pool.QueryRow(ctx, `SELECT `+datasetGroupColumns+` FROM dataset_groups WHERE id = $1`, id))
return out, mapError(err)
}
func (r *DatasetGroupRepository) GetByCode(ctx context.Context, code string) (domain.DatasetGroup, error) {
out, err := scanDatasetGroup(r.pool.QueryRow(ctx, `SELECT `+datasetGroupColumns+` FROM dataset_groups WHERE code = $1`, code))
return out, mapError(err)
}
// List returns a page of groups ordered by code; code (exact) is optional.
func (r *DatasetGroupRepository) List(ctx context.Context, code *string, limit, offset int) ([]domain.DatasetGroup, error) {
where, args := groupWhere(code)
args = append(args, limit, offset)
rows, err := r.pool.Query(ctx, fmt.Sprintf(`SELECT %s FROM dataset_groups%s ORDER BY code LIMIT $%d OFFSET $%d`,
datasetGroupColumns, where, len(args)-1, len(args)), args...)
if err != nil {
return nil, mapError(err)
}
defer rows.Close()
out := make([]domain.DatasetGroup, 0)
for rows.Next() {
g, err := scanDatasetGroup(rows)
if err != nil {
return nil, mapError(err)
}
out = append(out, g)
}
return out, mapError(rows.Err())
}
func (r *DatasetGroupRepository) Count(ctx context.Context, code *string) (int, error) {
where, args := groupWhere(code)
var n int
err := r.pool.QueryRow(ctx, `SELECT count(*) FROM dataset_groups`+where, args...).Scan(&n)
return n, mapError(err)
}
func groupWhere(code *string) (string, []any) {
if code == nil {
return "", nil
}
return " WHERE code = $1", []any{*code}
}
func (r *DatasetGroupRepository) Update(ctx context.Context, g domain.DatasetGroup) (domain.DatasetGroup, error) {
row := r.pool.QueryRow(ctx,
`UPDATE dataset_groups SET code = $2, name = $3, description = $4, updated_at = now() WHERE id = $1 RETURNING `+datasetGroupColumns,
g.ID, g.Code, g.Name, g.Description)
out, err := scanDatasetGroup(row)
return out, mapError(err)
}
// Delete removes a group. Returns domain.ErrNotFound if absent and
// domain.ErrConflict (via the FK) if datasets still reference it.
func (r *DatasetGroupRepository) Delete(ctx context.Context, id uuid.UUID) error {
tag, err := r.pool.Exec(ctx, `DELETE FROM dataset_groups WHERE id = $1`, id)
if err != nil {
return mapError(err)
}
if tag.RowsAffected() == 0 {
return domain.ErrNotFound
}
return nil
}
func (r *DatasetGroupRepository) SetSLD(ctx context.Context, id uuid.UUID, key, filename, contentType string) (domain.DatasetGroup, error) {
row := r.pool.QueryRow(ctx,
`UPDATE dataset_groups SET sld_storage_key = $2, sld_filename = $3, sld_content_type = $4, updated_at = now() WHERE id = $1 RETURNING `+datasetGroupColumns,
id, key, filename, contentType)
out, err := scanDatasetGroup(row)
return out, mapError(err)
}
var _ = strings.TrimSpace // keep import if unused after edits; remove if not needed
```
(Remove the trailing `strings` line and import if you do not use `strings`.)
- [ ] **Step 3: Service test first** (`internal/service/dataset_group_test.go`)
```go
package service
import (
"context"
"errors"
"strings"
"testing"
"gis/internal/domain"
"github.com/google/uuid"
)
type stubGroupRepo struct {
store map[uuid.UUID]domain.DatasetGroup
}
func newStubGroupRepo() *stubGroupRepo { return &stubGroupRepo{store: map[uuid.UUID]domain.DatasetGroup{}} }
func (r *stubGroupRepo) Create(_ context.Context, g domain.DatasetGroup) (domain.DatasetGroup, error) {
for _, e := range r.store {
if e.Code == g.Code {
return domain.DatasetGroup{}, domain.ErrConflict
}
}
g.ID = uuid.New()
r.store[g.ID] = g
return g, nil
}
func (r *stubGroupRepo) GetByID(_ context.Context, id uuid.UUID) (domain.DatasetGroup, error) {
g, ok := r.store[id]
if !ok {
return domain.DatasetGroup{}, domain.ErrNotFound
}
return g, nil
}
func (r *stubGroupRepo) GetByCode(_ context.Context, code string) (domain.DatasetGroup, error) {
for _, g := range r.store {
if g.Code == code {
return g, nil
}
}
return domain.DatasetGroup{}, domain.ErrNotFound
}
func (r *stubGroupRepo) List(_ context.Context, code *string, limit, offset int) ([]domain.DatasetGroup, error) {
out := []domain.DatasetGroup{}
for _, g := range r.store {
if code == nil || g.Code == *code {
out = append(out, g)
}
}
return out, nil
}
func (r *stubGroupRepo) Count(_ context.Context, code *string) (int, error) {
l, _ := r.List(context.Background(), code, 0, 0)
return len(l), nil
}
func (r *stubGroupRepo) Update(_ context.Context, g domain.DatasetGroup) (domain.DatasetGroup, error) {
if _, ok := r.store[g.ID]; !ok {
return domain.DatasetGroup{}, domain.ErrNotFound
}
r.store[g.ID] = g
return g, nil
}
func (r *stubGroupRepo) Delete(_ context.Context, id uuid.UUID) error {
if _, ok := r.store[id]; !ok {
return domain.ErrNotFound
}
delete(r.store, id)
return nil
}
func (r *stubGroupRepo) SetSLD(_ context.Context, id uuid.UUID, key, filename, contentType string) (domain.DatasetGroup, error) {
g, ok := r.store[id]
if !ok {
return domain.DatasetGroup{}, domain.ErrNotFound
}
g.SldStorageKey, g.SldFilename, g.SldContentType = &key, &filename, &contentType
r.store[id] = g
return g, nil
}
func TestDatasetGroupService_Create_RequiresSlug(t *testing.T) {
svc := NewDatasetGroupService(newStubGroupRepo(), newStubDatasetRepo(), newStubStore())
_, err := svc.Create(context.Background(), DatasetGroupInput{Code: "Bad Code", Name: "x"})
if !errors.Is(err, domain.ErrValidation) {
t.Fatalf("want ErrValidation, got %v", err)
}
}
func TestDatasetGroupService_Latest_PrefersGreatestEndDateThenCreatedAt(t *testing.T) {
groups := newStubGroupRepo()
datasets := newStubDatasetRepo()
svc := NewDatasetGroupService(groups, datasets, newStubStore())
g, _ := svc.Create(context.Background(), DatasetGroupInput{Code: "pop", Name: "Population"})
s := func(v string) *string { return &v }
old := domain.Dataset{ID: uuid.New(), DatasetGroupID: &g.ID, EndDate: s("2019-12-31"), Status: domain.DatasetStatusReady}
fresh := domain.Dataset{ID: uuid.New(), DatasetGroupID: &g.ID, EndDate: s("2020-12-31"), Status: domain.DatasetStatusReady}
undated := domain.Dataset{ID: uuid.New(), DatasetGroupID: &g.ID, Status: domain.DatasetStatusReady}
for _, d := range []domain.Dataset{old, fresh, undated} {
datasets.store[d.ID] = d
}
got, err := svc.Latest(context.Background(), g.ID, nil, nil)
if err != nil {
t.Fatal(err)
}
if got.ID != fresh.ID {
t.Fatalf("want %s, got %s", fresh.ID, got.ID)
}
}
func TestDatasetGroupService_Latest_NotFoundWhenEmpty(t *testing.T) {
groups := newStubGroupRepo()
svc := NewDatasetGroupService(groups, newStubDatasetRepo(), newStubStore())
g, _ := svc.Create(context.Background(), DatasetGroupInput{Code: "pop", Name: "Population"})
if _, err := svc.Latest(context.Background(), g.ID, nil, nil); !errors.Is(err, domain.ErrNotFound) {
t.Fatalf("want ErrNotFound, got %v", err)
}
}
func TestDatasetGroupService_UploadSLD_RejectsBadExtension(t *testing.T) {
groups := newStubGroupRepo()
svc := NewDatasetGroupService(groups, newStubDatasetRepo(), newStubStore())
g, _ := svc.Create(context.Background(), DatasetGroupInput{Code: "pop", Name: "Population"})
_, err := svc.UploadSLD(context.Background(), g.ID, UploadSLDInput{Filename: "style.txt", ContentType: "text/plain", Size: 3, Reader: strings.NewReader("<x>")})
if !errors.Is(err, domain.ErrValidation) {
t.Fatalf("want ErrValidation, got %v", err)
}
}
```
`newStubStore()` — the in-memory ObjectStore stub already used in `dataset_test.go` (reuse its constructor name; if it is named differently, use that name).
- [ ] **Step 4: Run** `go test ./internal/service/ -run DatasetGroupService` → FAIL (undefined).
- [ ] **Step 5: Service** (`internal/service/dataset_group.go`)
```go
package service
import (
"bytes"
"context"
"fmt"
"io"
"path"
"path/filepath"
"strings"
"gis/internal/domain"
"github.com/google/uuid"
)
// DatasetGroupRepository is the persistence behaviour DatasetGroupService needs.
type DatasetGroupRepository interface {
Create(ctx context.Context, g domain.DatasetGroup) (domain.DatasetGroup, error)
GetByID(ctx context.Context, id uuid.UUID) (domain.DatasetGroup, error)
GetByCode(ctx context.Context, code string) (domain.DatasetGroup, error)
List(ctx context.Context, code *string, limit, offset int) ([]domain.DatasetGroup, error)
Count(ctx context.Context, code *string) (int, error)
Update(ctx context.Context, g domain.DatasetGroup) (domain.DatasetGroup, error)
Delete(ctx context.Context, id uuid.UUID) error
SetSLD(ctx context.Context, id uuid.UUID, key, filename, contentType string) (domain.DatasetGroup, error)
}
// DatasetGroupInput carries the mutable fields of a group.
type DatasetGroupInput struct {
Code string
Name string
Description string
}
// DatasetGroupPage is a page of groups.
type DatasetGroupPage struct {
Items []domain.DatasetGroup
Page int
PageSize int
Total int
}
// DatasetGroupService implements dataset-group business rules.
type DatasetGroupService struct {
repo DatasetGroupRepository
datasets DatasetRepository
store ObjectStore
}
// NewDatasetGroupService wires the group repository, the dataset repository
// (for the "latest" lookup) and the object store (for the group SLD).
func NewDatasetGroupService(repo DatasetGroupRepository, datasets DatasetRepository, store ObjectStore) *DatasetGroupService {
return &DatasetGroupService{repo: repo, datasets: datasets, store: store}
}
func (s *DatasetGroupService) Create(ctx context.Context, in DatasetGroupInput) (domain.DatasetGroup, error) {
if !domain.ValidSlug(in.Code) {
return domain.DatasetGroup{}, fmt.Errorf("%w: code must be a slug (lowercase latin letters, digits, and dashes)", domain.ErrValidation)
}
return s.repo.Create(ctx, domain.DatasetGroup{Code: in.Code, Name: in.Name, Description: in.Description})
}
func (s *DatasetGroupService) Get(ctx context.Context, id uuid.UUID) (domain.DatasetGroup, error) {
return s.repo.GetByID(ctx, id)
}
// List returns a page of groups; page/pageSize are clamped like datasets.
func (s *DatasetGroupService) List(ctx context.Context, code *string, page, pageSize int) (DatasetGroupPage, error) {
if page < 1 {
page = 1
}
if pageSize < 1 {
pageSize = DefaultPageSize
}
if pageSize > MaxPageSize {
pageSize = MaxPageSize
}
items, err := s.repo.List(ctx, code, pageSize, (page-1)*pageSize)
if err != nil {
return DatasetGroupPage{}, err
}
total, err := s.repo.Count(ctx, code)
if err != nil {
return DatasetGroupPage{}, err
}
return DatasetGroupPage{Items: items, Page: page, PageSize: pageSize, Total: total}, nil
}
func (s *DatasetGroupService) Update(ctx context.Context, id uuid.UUID, in DatasetGroupInput) (domain.DatasetGroup, error) {
if !domain.ValidSlug(in.Code) {
return domain.DatasetGroup{}, fmt.Errorf("%w: code must be a slug (lowercase latin letters, digits, and dashes)", domain.ErrValidation)
}
return s.repo.Update(ctx, domain.DatasetGroup{ID: id, Code: in.Code, Name: in.Name, Description: in.Description})
}
// Delete removes a group (409 via ErrConflict while datasets reference it) and
// then best-effort removes its SLD object.
func (s *DatasetGroupService) Delete(ctx context.Context, id uuid.UUID) error {
g, err := s.repo.GetByID(ctx, id)
if err != nil {
return err
}
if err := s.repo.Delete(ctx, id); err != nil {
return err
}
if g.SldStorageKey != nil {
_ = s.store.Remove(ctx, *g.SldStorageKey)
}
return nil
}
// Latest returns the freshest dataset of the group (greatest end_date, NULLs
// last, then newest), optionally restricted by status and file type.
func (s *DatasetGroupService) Latest(ctx context.Context, id uuid.UUID, status *string, fileType *domain.FileType) (domain.Dataset, error) {
if _, err := s.repo.GetByID(ctx, id); err != nil {
return domain.Dataset{}, err
}
return s.datasets.LatestInGroup(ctx, id, status, fileType)
}
// UploadSLD validates and stores the group's shared SLD, replacing any previous one.
func (s *DatasetGroupService) UploadSLD(ctx context.Context, id uuid.UUID, in UploadSLDInput) (domain.DatasetGroup, error) {
g, err := s.repo.GetByID(ctx, id)
if err != nil {
return domain.DatasetGroup{}, err
}
ext := strings.ToLower(filepath.Ext(in.Filename))
if !domain.ExtensionAllowedForSLD(ext) {
return domain.DatasetGroup{}, fmt.Errorf("%w: sld file must have one of these extensions: %s",
domain.ErrValidation, strings.Join(domain.AllowedSLDExtensions, ", "))
}
head := make([]byte, 512)
n, err := io.ReadFull(in.Reader, head)
if err != nil && err != io.EOF && err != io.ErrUnexpectedEOF {
return domain.DatasetGroup{}, fmt.Errorf("read sld upload: %w", err)
}
head = head[:n]
if err := domain.ValidateFileContent(ext, head); err != nil {
return domain.DatasetGroup{}, fmt.Errorf("%w: %s", domain.ErrValidation, err)
}
content := io.MultiReader(bytes.NewReader(head), in.Reader)
key := path.Join("groups", g.ID.String(), "sld", in.Filename)
if err := s.store.Put(ctx, key, content, in.Size, in.ContentType); err != nil {
return domain.DatasetGroup{}, err
}
updated, err := s.repo.SetSLD(ctx, id, key, in.Filename, in.ContentType)
if err != nil {
_ = s.store.Remove(ctx, key)
return domain.DatasetGroup{}, err
}
if g.SldStorageKey != nil && *g.SldStorageKey != key {
_ = s.store.Remove(ctx, *g.SldStorageKey)
}
return updated, nil
}
// DownloadSLD returns the group and a reader for its SLD; ErrSLDNotFound when none.
func (s *DatasetGroupService) DownloadSLD(ctx context.Context, id uuid.UUID) (domain.DatasetGroup, io.ReadCloser, error) {
g, err := s.repo.GetByID(ctx, id)
if err != nil {
return domain.DatasetGroup{}, nil, err
}
if g.SldStorageKey == nil {
return domain.DatasetGroup{}, nil, ErrSLDNotFound
}
obj, err := s.store.Get(ctx, *g.SldStorageKey)
if err != nil {
return domain.DatasetGroup{}, nil, err
}
return g, obj, nil
}
```
- [ ] **Step 6: DatasetService changes** (`internal/service/dataset.go`)
1. Add `groupReader` interface and the `groups groupReader` field + **4th constructor parameter** (update every `NewDatasetService(` call: `internal/app/app.go`, tests).
2. `ensureGroupExists(ctx, id *uuid.UUID) error` — nil → nil; not found → `fmt.Errorf("%w: dataset group %s not found", domain.ErrValidation, id)`. Call it in `Upload`, `Update`, and `Patch` (when the group changes).
3. `DownloadSLD` fallback: when `dataset.SldStorageKey == nil` and `dataset.DatasetGroupID != nil`, load the group; if it has an SLD, return a dataset copy whose `SldStorageKey/SldFilename/SldContentType` are the group's, and stream the group's object. Otherwise `ErrSLDNotFound`.
4. `CreateRasterFromFile`:
```go
// CreateRasterInput describes a raster file already on local disk that should
// become a dataset (used by the Landscape Program to publish results).
type CreateRasterInput struct {
CategoryID uuid.UUID
DatasetGroupID *uuid.UUID
Name string
Filename string // stored filename, e.g. "lp-z-integrated.tif"
Path string // local path to read from
StartDate *string
EndDate *string
Meta json.RawMessage
Automated bool
}
// CreateRasterFromFile stores a local GeoTIFF as a new raster dataset and
// enqueues the standard COG conversion, exactly like an uploaded raster.
func (s *DatasetService) CreateRasterFromFile(ctx context.Context, in CreateRasterInput) (domain.Dataset, error) {
f, err := os.Open(in.Path)
if err != nil {
return domain.Dataset{}, err
}
defer f.Close()
info, err := f.Stat()
if err != nil {
return domain.Dataset{}, err
}
return s.Upload(ctx, UploadInput{
CategoryID: in.CategoryID,
DatasetGroupID: in.DatasetGroupID,
Name: in.Name,
Meta: in.Meta,
Automated: in.Automated,
Filename: in.Filename,
FileType: domain.FileTypeRaster,
ContentType: "image/tiff",
Size: info.Size(),
Reader: f,
StartDate: in.StartDate,
EndDate: in.EndDate,
})
}
```
Add tests in `dataset_test.go`: `TestDatasetService_DownloadSLD_FallsBackToGroup` (dataset without SLD, group with SLD in stub store → reader returns group content) and `TestDatasetService_CreateRasterFromFile` (temp file with `II*\x00` header → dataset created with `file_type raster`, group id, convert job enqueued). The stub group reader is `stubGroupRepo` from Step 3.
- [ ] **Step 7: Handler** (`internal/transport/http/dataset_group_handler.go`)
```go
package http
import (
"io"
"net/http"
"strings"
"gis/internal/domain"
"gis/internal/service"
"gis/pkg/httputil"
"github.com/go-chi/chi/v5"
"github.com/go-playground/validator/v10"
)
// DatasetGroupHandler serves the /dataset-groups routes.
type DatasetGroupHandler struct {
svc *service.DatasetGroupService
validate *validator.Validate
}
func NewDatasetGroupHandler(svc *service.DatasetGroupService, validate *validator.Validate) *DatasetGroupHandler {
return &DatasetGroupHandler{svc: svc, validate: validate}
}
func (h *DatasetGroupHandler) Register(r chi.Router) {
r.Get("/", h.list)
r.Post("/", h.create)
r.Get("/{id}", h.get)
r.Put("/{id}", h.update)
r.Delete("/{id}", h.delete)
r.Get("/{id}/latest", h.latest)
r.Post("/{id}/sld", h.uploadSLD)
r.Get("/{id}/sld", h.downloadSLD)
}
type datasetGroupRequest struct {
Code string `json:"code" validate:"required,max=255,slug"`
Name string `json:"name" validate:"required,max=255"`
Description string `json:"description" validate:"max=2000"`
}
```
Implement `create` (201), `list` (`?code=`, `page`, `page_size` via `parsePositiveIntQuery`, `newPaginated`), `get`, `update`, `delete` (204), `latest` (`?status=` validated with `domain.ValidDatasetStatus`, `?file_type=` validated with `FileType.Valid()`; 404 via `respondDomainError`), `uploadSLD`/`downloadSLD` mirroring `DatasetHandler.uploadSLD/downloadSLD` (multipart `file`, default content type `application/xml`, `Content-Disposition: attachment`). Follow `category_handler.go` for decode/validate/error flow.
- [ ] **Step 8: Wiring** — `router.go`: add `DatasetGroups *DatasetGroupHandler` to `RouterDeps` and `r.Route("/dataset-groups", deps.DatasetGroups.Register)`. `app.go`: `groupRepo := postgres.NewDatasetGroupRepository(pool)`, pass `groupRepo` as the 4th arg to `NewDatasetService`, add field `groups *service.DatasetGroupService` = `service.NewDatasetGroupService(groupRepo, datasetRepo, store)`, and `DatasetGroups: transporthttp.NewDatasetGroupHandler(a.groups, validate)`.
- [ ] **Step 9: Run** `go test ./internal/service/ ./internal/transport/... && go build ./...` → PASS.
- [ ] **Step 10: OpenAPI + README** — add tag `Dataset Groups`, paths `/dataset-groups`, `/dataset-groups/{id}`, `/dataset-groups/{id}/latest`, `/dataset-groups/{id}/sld`, schemas `DatasetGroup`, `DatasetGroupInput`, `DatasetGroupPage`; note the SLD fallback on `GET /datasets/{id}/sld`. README: "Dataset groups" section (what a group is, freshness rule, SLD fallback).
- [ ] **Step 11: Commit**
```bash
git add internal/domain/dataset_group.go internal/repository/postgres/dataset_group.go internal/service/dataset_group.go internal/service/dataset_group_test.go internal/service/dataset.go internal/service/dataset_test.go internal/transport/http/dataset_group_handler.go internal/transport/http/router.go internal/app/app.go api/openapi.yaml README.md
git commit -m "feat(dataset-groups): CRUD, latest-in-group, group SLD with dataset fallback, CreateRasterFromFile"
```
---
### Task 3: Raster I/O abstraction + godal implementation + Dockerfile
**Files:**
- Create: `internal/raster/io.go`, `internal/raster/godal.go`, `internal/raster/godal_test.go`
- Modify: `go.mod`, `go.sum` (via `go get github.com/airbusgeo/godal@v0.0.18`), `build/package/Dockerfile`, `.golangci.yml` only if cgo needs a setting (usually not)
**Interfaces:**
- Produces (used by Task 6):
```go
package raster
type Grid struct { Width, Height int; GeoTransform [6]float64; ProjectionWKT string }
type Window struct { X, Y, Width, Height int }
type Reader interface { Grid() Grid; NoData() (float64, bool); ReadFloat64(w Window, dst []float64) error; Close() error }
type WriterOptions struct { NoData int16; TileSize int; Compression string }
type Writer interface { WriteInt16(w Window, src []int16) error; Close() error }
type IO interface {
Open(ctx context.Context, path string) (Reader, error)
Create(ctx context.Context, path string, g Grid, o WriterOptions) (Writer, error)
SameGrid(a, b Grid) bool
}
func DefaultWriterOptions() WriterOptions // {NoData: -9999, TileSize: 512, Compression: "DEFLATE"}
func NewGodalIO() *GodalIO // implements IO
```
- [ ] **Step 1: Interfaces** (`internal/raster/io.go`) — write the block above with doc comments. Add:
```go
// Windows tiles a w×h raster into row-major windows of at most size×size.
func Windows(w, h, size int) []Window {
var out []Window
for y := 0; y < h; y += size {
wh := min(size, h-y)
for x := 0; x < w; x += size {
out = append(out, Window{X: x, Y: y, Width: min(size, w-x), Height: wh})
}
}
return out
}
```
Test (`internal/raster/io_test.go`): `Windows(10, 10, 4)` → 9 windows, last is `{8,8,2,2}`; `Windows(5, 3, 8)` → one window `{0,0,5,3}`.
- [ ] **Step 2: Add dependency** — `go get github.com/airbusgeo/godal@v0.0.18`. Verify `CGO_ENABLED=1 go build ./internal/raster/` compiles (Homebrew GDAL provides `gdal.pc` in `/opt/homebrew/lib/pkgconfig`; if `pkg-config` is missing run `brew install pkgconf`).
- [ ] **Step 3: Failing round-trip test** (`internal/raster/godal_test.go`)
```go
package raster
import (
"context"
"path/filepath"
"testing"
)
func TestGodalIO_RoundTrip(t *testing.T) {
rio := NewGodalIO()
ctx := context.Background()
path := filepath.Join(t.TempDir(), "out.tif")
grid := Grid{Width: 10, Height: 7, GeoTransform: [6]float64{100, 30, 0, 200, 0, -30}, ProjectionWKT: "EPSG:32642"}
w, err := rio.Create(ctx, path, grid, DefaultWriterOptions())
if err != nil {
t.Fatal(err)
}
full := make([]int16, 10*7)
for i := range full {
full[i] = int16(i%6) - 1 // -1..4 (-1 stands in for garbage, not nodata)
}
full[0] = -9999
if err := w.WriteInt16(Window{0, 0, 10, 7}, full); err != nil {
t.Fatal(err)
}
if err := w.Close(); err != nil {
t.Fatal(err)
}
r, err := rio.Open(ctx, path)
if err != nil {
t.Fatal(err)
}
defer r.Close()
g := r.Grid()
if g.Width != 10 || g.Height != 7 || g.GeoTransform != grid.GeoTransform || g.ProjectionWKT == "" {
t.Fatalf("grid mismatch: %+v", g)
}
if nd, ok := r.NoData(); !ok || nd != -9999 {
t.Fatalf("nodata = %v %v", nd, ok)
}
dst := make([]float64, 3*2)
if err := r.ReadFloat64(Window{X: 8, Y: 5, Width: 2, Height: 2}, dst[:4]); err != nil {
t.Fatal(err)
}
// pixel (8,5) has index 58 -> 58%6-1 = 3 ; (9,5)=59 -> 4 ; (8,6)=68 -> 1 ; (9,6)=69 -> 2
want := []float64{3, 4, 1, 2}
for i, v := range want {
if dst[i] != v {
t.Fatalf("pixel %d = %v want %v", i, dst[i], v)
}
}
if !rio.SameGrid(g, grid) {
t.Fatal("SameGrid should be true for identical grids")
}
other := grid
other.GeoTransform[0] += 1
if rio.SameGrid(g, other) {
t.Fatal("SameGrid should be false for shifted origin")
}
}
```
Run: `go test ./internal/raster/ -run TestGodalIO -v` → FAIL (undefined `NewGodalIO`).
- [ ] **Step 4: Implementation** (`internal/raster/godal.go`)
```go
package raster
import (
"context"
"fmt"
"math"
"sync"
"github.com/airbusgeo/godal"
)
var registerOnce sync.Once
// GodalIO implements IO with the GDAL bindings (github.com/airbusgeo/godal).
// GDAL datasets are not goroutine-safe: callers must not share a Reader or
// Writer between goroutines without external synchronisation.
type GodalIO struct{}
// NewGodalIO registers GDAL drivers (once) and returns a GodalIO.
func NewGodalIO() *GodalIO {
registerOnce.Do(godal.RegisterAll)
return &GodalIO{}
}
// DefaultWriterOptions returns the Landscape Program output profile.
func DefaultWriterOptions() WriterOptions {
return WriterOptions{NoData: -9999, TileSize: 512, Compression: "DEFLATE"}
}
type godalReader struct {
ds *godal.Dataset
grid Grid
}
func (g *GodalIO) Open(_ context.Context, path string) (Reader, error) {
ds, err := godal.Open(path)
if err != nil {
return nil, fmt.Errorf("open raster %s: %w", path, err)
}
st := ds.Structure()
if st.NBands < 1 {
ds.Close()
return nil, fmt.Errorf("open raster %s: no bands", path)
}
gt, err := ds.GeoTransform()
if err != nil {
ds.Close()
return nil, fmt.Errorf("open raster %s: geotransform: %w", path, err)
}
return &godalReader{ds: ds, grid: Grid{Width: st.SizeX, Height: st.SizeY, GeoTransform: gt, ProjectionWKT: ds.Projection()}}, nil
}
func (r *godalReader) Grid() Grid { return r.grid }
func (r *godalReader) NoData() (float64, bool) { return r.ds.Bands()[0].NoData() }
func (r *godalReader) ReadFloat64(w Window, dst []float64) error {
if len(dst) < w.Width*w.Height {
return fmt.Errorf("read window: dst has %d elements, need %d", len(dst), w.Width*w.Height)
}
return r.ds.Bands()[0].Read(w.X, w.Y, dst[:w.Width*w.Height], w.Width, w.Height)
}
func (r *godalReader) Close() error { return r.ds.Close() }
type godalWriter struct{ ds *godal.Dataset }
func (g *GodalIO) Create(_ context.Context, path string, grid Grid, o WriterOptions) (Writer, error) {
tile := fmt.Sprint(o.TileSize)
ds, err := godal.Create(godal.GTiff, path, 1, godal.Int16, grid.Width, grid.Height,
godal.CreationOption("TILED=YES", "BLOCKXSIZE="+tile, "BLOCKYSIZE="+tile,
"COMPRESS="+o.Compression, "PREDICTOR=2", "BIGTIFF=IF_SAFER"))
if err != nil {
return nil, fmt.Errorf("create raster %s: %w", path, err)
}
if err := ds.SetGeoTransform(grid.GeoTransform); err != nil {
ds.Close()
return nil, fmt.Errorf("create raster %s: geotransform: %w", path, err)
}
if grid.ProjectionWKT != "" {
if err := ds.SetProjection(grid.ProjectionWKT); err != nil {
ds.Close()
return nil, fmt.Errorf("create raster %s: projection: %w", path, err)
}
}
if err := ds.Bands()[0].SetNoData(float64(o.NoData)); err != nil {
ds.Close()
return nil, fmt.Errorf("create raster %s: nodata: %w", path, err)
}
return &godalWriter{ds: ds}, nil
}
func (w *godalWriter) WriteInt16(win Window, src []int16) error {
if len(src) < win.Width*win.Height {
return fmt.Errorf("write window: src has %d elements, need %d", len(src), win.Width*win.Height)
}
return w.ds.Bands()[0].Write(win.X, win.Y, src[:win.Width*win.Height], win.Width, win.Height)
}
func (w *godalWriter) Close() error { return w.ds.Close() }
// SameGrid reports whether two grids describe the same pixels: equal size,
// geotransform within 1e-9 (absolute), and the same CRS (GDAL IsSame; an
// empty projection only matches an empty projection).
func (g *GodalIO) SameGrid(a, b Grid) bool {
if a.Width != b.Width || a.Height != b.Height {
return false
}
for i := range a.GeoTransform {
if math.Abs(a.GeoTransform[i]-b.GeoTransform[i]) > 1e-9 {
return false
}
}
if a.ProjectionWKT == "" || b.ProjectionWKT == "" {
return a.ProjectionWKT == b.ProjectionWKT
}
sa, err := godal.NewSpatialRefFromWKT(a.ProjectionWKT)
if err != nil {
return false
}
defer sa.Close()
sb, err := godal.NewSpatialRefFromWKT(b.ProjectionWKT)
if err != nil {
return false
}
defer sb.Close()
return sa.IsSame(sb)
}
```
Check godal's actual API names while implementing (`ds.Structure()`, `Bands()`, `Band.Read/Write` generic over slice types, `NewSpatialRefFromWKT`, `SpatialRef.IsSame`, `godal.CreationOption`); adjust if a name differs in v0.0.18 (`go doc github.com/airbusgeo/godal Dataset`).
- [ ] **Step 5: Run** `go test ./internal/raster/ -v` → PASS.
- [ ] **Step 6: Dockerfile** (`build/package/Dockerfile`)
```dockerfile
# Build context is the repo root: docker build -f build/package/Dockerfile .
FROM golang:1.26-alpine AS builder
# godal (GDAL bindings) needs cgo, a C toolchain and the GDAL headers.
RUN apk add --no-cache gdal-dev gcc musl-dev pkgconfig
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=1 GOOS=linux go build -trimpath -o /out/gis ./cmd/gis
FROM alpine:3.22
# gdal: shared library for godal (raster math in the worker).
# gdal-tools: gdal_translate / gdalinfo (COG conversion) and ogr2ogr.
RUN apk add --no-cache ca-certificates tzdata gdal gdal-tools \
&& adduser -D -u 10001 app
WORKDIR /app
COPY --from=builder /out/gis /usr/local/bin/gis
USER app
EXPOSE 8080
ENTRYPOINT ["gis"]
CMD ["serve"]
```
If Docker is available, verify with `docker build -f build/package/Dockerfile -t gis:lp .`; otherwise note in the report that the image build was not verified.
- [ ] **Step 7: Commit**
```bash
git add internal/raster/io.go internal/raster/io_test.go internal/raster/godal.go internal/raster/godal_test.go go.mod go.sum build/package/Dockerfile
git commit -m "feat(raster): IO abstraction with godal implementation; cgo Dockerfile"
```
---
### Task 4: Landscape graph, params and pixel kernels (pure Go)
**Files:**
- Create: `internal/landscape/graph.go`, `internal/landscape/params.go`, `internal/landscape/kernels.go`, `internal/landscape/kernels_test.go`, `internal/landscape/graph_test.go`
**Interfaces (produced, used by Tasks 5–8):**
```go
package landscape
const (
NoData int16 = -9999
CategoryCode = "landscape" // dataset category for every LP dataset
)
type Slot struct { Code, Title string } // Code is the dataset_groups.code
type Output struct { Code, Title string }
type Node struct { Code, Name string; Stage int; Inputs []string; Outputs []string; Kernel Kernel }
// Kernel computes one window: in[i] follows Node.Inputs order, out[j] follows Node.Outputs order;
// all slices have len == window pixels. Every out value must be set (NoData or class).
type Kernel func(in [][]float64, out [][]int16, p Params)
var Slots []Slot; var Outputs []Output; var Nodes []Node
func NodeByCode(code string) (Node, bool)
func Stages() [][]Node // nodes grouped by Stage ascending, each group ordered by Code
func OutputByCode(code string) (Output, bool)
func SlotByCode(code string) (Slot, bool)
func NodeForOutput(group string) (Node, bool) // which node produces a group
type Params struct {
WindowSize int `json:"window_size"`
TiePriority []int `json:"tie_priority"`
V1MinSE12 int `json:"v1_min_s_e12"`
V1MinSE3 int `json:"v1_min_s_e3"`
V2MinSE3 int `json:"v2_min_s_e3"`
V2MinSE4 int `json:"v2_min_s_e4"`
}
func DefaultParams() Params
func (p Params) WithDefaults() Params // zero fields -> defaults
func (p Params) Validate() error // window 256..8192; thresholds 1..5; tie_priority permutation of 1..5
func (p Params) ForNode(code string) Params // only fields the node uses (window_size never included); used in fingerprints
```
- [ ] **Step 1: graph.go**
```go
// Package landscape defines the Landscape Program (ЛП) computation graph — the
// per-pixel formulas that turn ecological (Z), vulnerability (U) and
// suitability (P) class rasters into constraints (E), conflicts (C) and
// functional zoning (V1–V3) — together with a window-based executor.
package landscape
import "sort"
const (
// NoData is the output nodata value of every LP raster.
NoData int16 = -9999
// CategoryCode is the dataset category holding every LP dataset.
CategoryCode = "landscape"
)
// Group codes. Slots are leaf inputs users upload; the rest are node outputs.
const (
SlotZBiodiversity = "lp-z-biodiversity"
SlotZLCR = "lp-z-lcr"
SlotZSoilProtection = "lp-z-soil-protection"
SlotZWR = "lp-z-wr"
SlotUSurfWater = "lp-u-surf-water"
SlotUSoil = "lp-u-soil"
SlotUCrop = "lp-u-crop"
SlotUVeg = "lp-u-veg"
SlotPIrrigation = "lp-p-irrigation"
SlotPSnow = "lp-p-snow"
SlotPCrop = "lp-p-crop"
SlotPPasture = "lp-p-pasture"
SlotPTourism = "lp-p-tourism"
OutZIntegrated = "lp-z-integrated"
OutUIntegrated = "lp-u-integrated"
OutZMax = "lp-z-max"
OutUMax = "lp-u-max"
OutE = "lp-e"
OutCIrrigation = "lp-c-irrigation"
OutCSnow = "lp-c-snow"
OutCCrop = "lp-c-crop"
OutCPasture = "lp-c-pasture"
OutCTourism = "lp-c-tourism"
OutPMax = "lp-p-max"
OutV1 = "lp-v1-conservative"
OutV2 = "lp-v2-balanced"
OutV3 = "lp-v3-development"
OutCMax = "lp-c-max"
)
type Slot struct{ Code, Title string }
type Output struct{ Code, Title string }
// Kernel computes one window. in[i] follows Node.Inputs order and out[j]
// follows Node.Outputs order; every slice has exactly window-pixel length.
type Kernel func(in [][]float64, out [][]int16, p Params)
// Node is one function of the graph. Stage = first digit of Code.
type Node struct {
Code, Name string
Stage int
Inputs []string
Outputs []string
Kernel Kernel
}
var Slots = []Slot{
{SlotZBiodiversity, "Z Biodiversity"}, {SlotZLCR, "Z LCR"}, {SlotZSoilProtection, "Z Soil Protection"}, {SlotZWR, "Z Water Resources"},
{SlotUSurfWater, "U Surface Water"}, {SlotUSoil, "U Soil"}, {SlotUCrop, "U Crop"}, {SlotUVeg, "U Vegetation"},
{SlotPIrrigation, "P Irrigation"}, {SlotPSnow, "P Snow"}, {SlotPCrop, "P Crop"}, {SlotPPasture, "P Pasture"}, {SlotPTourism, "P Tourism"},
}
var Outputs = []Output{
{OutZIntegrated, "Z Integrated"}, {OutUIntegrated, "U Integrated"}, {OutZMax, "Z Max"}, {OutUMax, "U Max"},
{OutE, "E Ecological Constraint"},
{OutCIrrigation, "C Irrigation"}, {OutCSnow, "C Snow"}, {OutCCrop, "C Crop"}, {OutCPasture, "C Pasture"}, {OutCTourism, "C Tourism"},
{OutPMax, "P Max"},
{OutV1, "V1 Conservative"}, {OutV2, "V2 Balanced"}, {OutV3, "V3 Development"}, {OutCMax, "C Max"},
}
var zInputs = []string{SlotZBiodiversity, SlotZLCR, SlotZSoilProtection, SlotZWR}
var uInputs = []string{SlotUSurfWater, SlotUSoil, SlotUCrop, SlotUVeg} // order matters: water, soil, crop, veg
var pInputs = []string{SlotPIrrigation, SlotPSnow, SlotPCrop, SlotPPasture, SlotPTourism} // sector order 1..5
var cOutputs = []string{OutCIrrigation, OutCSnow, OutCCrop, OutCPasture, OutCTourism}
var Nodes = []Node{
{Code: "00", Name: "z_integrated", Stage: 0, Inputs: zInputs, Outputs: []string{OutZIntegrated}, Kernel: kernelZIntegrated},
{Code: "01", Name: "u_integrated", Stage: 0, Inputs: uInputs, Outputs: []string{OutUIntegrated}, Kernel: kernelUIntegrated},
{Code: "02", Name: "z_max", Stage: 0, Inputs: zInputs, Outputs: []string{OutZMax}, Kernel: kernelZMax},
{Code: "03", Name: "u_max", Stage: 0, Inputs: uInputs, Outputs: []string{OutUMax}, Kernel: kernelUMax},
{Code: "10", Name: "e_ecological_constraint", Stage: 1, Inputs: []string{OutZIntegrated, OutUIntegrated}, Outputs: []string{OutE}, Kernel: kernelE},
{Code: "20", Name: "c_conflicts", Stage: 2, Inputs: append(append([]string{}, pInputs...), OutE), Outputs: cOutputs, Kernel: kernelCConflicts},
{Code: "21", Name: "p_max", Stage: 2, Inputs: pInputs, Outputs: []string{OutPMax}, Kernel: kernelMaxAnyValid},
{Code: "30", Name: "func_zoning", Stage: 3, Inputs: append([]string{OutE}, cOutputs...), Outputs: []string{OutV1, OutV2, OutV3}, Kernel: kernelFuncZoning},
{Code: "31", Name: "c_max", Stage: 3, Inputs: cOutputs, Outputs: []string{OutCMax}, Kernel: kernelMaxAnyValid},
}
func NodeByCode(code string) (Node, bool) {
for _, n := range Nodes {
if n.Code == code {
return n, true
}
}
return Node{}, false
}
func SlotByCode(code string) (Slot, bool) {
for _, s := range Slots {
if s.Code == code {
return s, true
}
}
return Slot{}, false
}
func OutputByCode(code string) (Output, bool) {
for _, o := range Outputs {
if o.Code == code {
return o, true
}
}
return Output{}, false
}
// NodeForOutput returns the node producing the given output group.
func NodeForOutput(group string) (Node, bool) {
for _, n := range Nodes {
for _, o := range n.Outputs {
if o == group {
return n, true
}
}
}
return Node{}, false
}
// Stages groups nodes by stage, ascending; within a stage nodes are sorted by code.
func Stages() [][]Node {
byStage := map[int][]Node{}
for _, n := range Nodes {
byStage[n.Stage] = append(byStage[n.Stage], n)
}
keys := make([]int, 0, len(byStage))
for k := range byStage {
keys = append(keys, k)
}
sort.Ints(keys)
out := make([][]Node, 0, len(keys))
for _, k := range keys {
ns := byStage[k]
sort.Slice(ns, func(i, j int) bool { return ns[i].Code < ns[j].Code })
out = append(out, ns)
}
return out
}
```
- [ ] **Step 2: graph_test.go** — assert: every node input is either a slot or an output of a node with a *lower* stage; every output has exactly one producer; `Stages()` returns 4 stages with codes `[00 01 02 03] [10] [20 21] [30 31]`; all group codes are valid slugs (`^[a-z0-9]+(?:-[a-z0-9]+)*$`). Run → PASS once graph.go compiles (kernels may be stubs until Step 4; add empty kernel funcs first so it compiles).
- [ ] **Step 3: params.go**
```go
package landscape
import (
"errors"
"fmt"
)
// Params are the run-time knobs of the graph (TZ §6): window size, tie
// priority for functional zoning, and the V1/V2 suitability thresholds.
type Params struct {
WindowSize int `json:"window_size"`
TiePriority []int `json:"tie_priority"`
V1MinSE12 int `json:"v1_min_s_e12"`
V1MinSE3 int `json:"v1_min_s_e3"`
V2MinSE3 int `json:"v2_min_s_e3"`
V2MinSE4 int `json:"v2_min_s_e4"`
}
// DefaultParams returns the TZ defaults.
func DefaultParams() Params {
return Params{WindowSize: 2048, TiePriority: []int{1, 2, 3, 4, 5}, V1MinSE12: 3, V1MinSE3: 4, V2MinSE3: 3, V2MinSE4: 5}
}
// WithDefaults fills zero-valued fields from DefaultParams.
func (p Params) WithDefaults() Params {
d := DefaultParams()
if p.WindowSize == 0 {
p.WindowSize = d.WindowSize
}
if len(p.TiePriority) == 0 {
p.TiePriority = d.TiePriority
}
if p.V1MinSE12 == 0 {
p.V1MinSE12 = d.V1MinSE12
}
if p.V1MinSE3 == 0 {
p.V1MinSE3 = d.V1MinSE3
}
if p.V2MinSE3 == 0 {
p.V2MinSE3 = d.V2MinSE3
}
if p.V2MinSE4 == 0 {
p.V2MinSE4 = d.V2MinSE4
}
return p
}
// Validate checks ranges: window 256..8192, thresholds 1..5, tie_priority a
// permutation of 1..5.
func (p Params) Validate() error {
var errs []error
if p.WindowSize < 256 || p.WindowSize > 8192 {
errs = append(errs, fmt.Errorf("window_size must be in 256..8192"))
}
for name, v := range map[string]int{"v1_min_s_e12": p.V1MinSE12, "v1_min_s_e3": p.V1MinSE3, "v2_min_s_e3": p.V2MinSE3, "v2_min_s_e4": p.V2MinSE4} {
if v < 1 || v > 5 {
errs = append(errs, fmt.Errorf("%s must be in 1..5", name))
}
}
seen := map[int]bool{}
for _, k := range p.TiePriority {
if k < 1 || k > 5 || seen[k] {
errs = append(errs, fmt.Errorf("tie_priority must be a permutation of 1..5"))
break
}
seen[k] = true
}
if len(p.TiePriority) != 5 && len(seen) == len(p.TiePriority) {
errs = append(errs, fmt.Errorf("tie_priority must list all of 1..5"))
}
return errors.Join(errs...)
}
// ForNode keeps only the fields that influence the node's result (window size
// never does), so fingerprints do not change when irrelevant knobs change.
func (p Params) ForNode(code string) Params {
if code == "30" {
return Params{TiePriority: p.TiePriority, V1MinSE12: p.V1MinSE12, V1MinSE3: p.V1MinSE3, V2MinSE3: p.V2MinSE3, V2MinSE4: p.V2MinSE4}
}
return Params{}
}
```
Tests (`params_test.go`): defaults validate; `Params{}.WithDefaults()` equals `DefaultParams()`; `TiePriority: {1,1,2,3,4}` invalid; `WindowSize: 100` invalid; `ForNode("00")` is zero; `ForNode("30")` has no WindowSize.
- [ ] **Step 4: Failing kernel tests** (`kernels_test.go`) — write these before the kernels:
```go
package landscape
import (
"reflect"
"testing"
)
func run(k Kernel, nOut int, in ...[]float64) [][]int16 {
out := make([][]int16, nOut)
for j := range out {
out[j] = make([]int16, len(in[0]))
}
k(in, out, DefaultParams())
return out
}
func TestClassify(t *testing.T) {
cases := map[float32]int16{1.0: 1, 1.8: 1, 1.81: 2, 2.6: 2, 2.75: 3, 3.4: 3, 3.5: 4, 4.2: 4, 4.25: 5, 5.0: 5}
for x, want := range cases {
if got := classify(x); got != want {
t.Errorf("classify(%v)=%d want %d", x, got, want)
}
}
if classify(0.9) != NoData || classify(5.1) != NoData {
t.Error("out of range must be NoData")
}
}
func TestKernelZIntegrated(t *testing.T) {
// pixel0: all valid (1,2,3,4) -> 2.5 -> class 2 ; pixel1: one nodata -> NoData ; pixel2: all 5 -> 5 -> class 5
got := run(kernelZIntegrated, 1,
[]float64{1, -9999, 5}, []float64{2, 2, 5}, []float64{3, 3, 5}, []float64{4, 4, 5})
if want := []int16{2, NoData, 5}; !reflect.DeepEqual(got[0], want) {
t.Fatalf("got %v want %v", got[0], want)
}
}
func TestKernelUIntegrated_Applicability(t *testing.T) {
// order: water, soil, crop, veg
// p0: all valid 1,1,5,5 -> (0.25+0.25+1.25+1.25)/1.0 = 3.0 -> class 3
// p1: crop invalid: 1,1,-,5 -> (0.25+0.25+1.25)/0.75 = 2.333 -> class 2
// p2: crop and veg invalid -> NoData
// p3: water invalid -> NoData
got := run(kernelUIntegrated, 1,
[]float64{1, 1, 1, -9999}, []float64{1, 1, 1, 1}, []float64{5, 0, 0, 5}, []float64{5, 5, 0, 5})
if want := []int16{3, 2, NoData, NoData}; !reflect.DeepEqual(got[0], want) {
t.Fatalf("got %v want %v", got[0], want)
}
}
func TestKernelZMax_RequiresAllValid(t *testing.T) {
got := run(kernelZMax, 1, []float64{1, 1}, []float64{4, 4}, []float64{2, -9999}, []float64{3, 3})
if want := []int16{4, NoData}; !reflect.DeepEqual(got[0], want) {
t.Fatalf("got %v want %v", got[0], want)
}
}
func TestKernelUMax_IgnoresInvalidCropVeg(t *testing.T) {
// water, soil, crop, veg ; p0: 1,2,-,5 -> 5 ; p1: 3,2,0,0 -> NoData (needs crop or veg) ; p2: 3,2,1,0 -> 3
got := run(kernelUMax, 1, []float64{1, 3, 3}, []float64{2, 2, 2}, []float64{0, 0, 1}, []float64{5, 0, 0})
if want := []int16{5, NoData, 3}; !reflect.DeepEqual(got[0], want) {
t.Fatalf("got %v want %v", got[0], want)
}
}
func TestKernelE(t *testing.T) {
got := run(kernelE, 1, []float64{2, 5, -9999}, []float64{4, 1, 3})
if want := []int16{4, 5, NoData}; !reflect.DeepEqual(got[0], want) {
t.Fatalf("got %v want %v", got[0], want)
}
}
func TestKernelCConflicts(t *testing.T) {
// 5 P then E; p0: E=3 -> min(P,3); p1: E invalid -> all NoData; p2: P2 invalid
e := []float64{3, 0, 4}
got := run(kernelCConflicts, 5,
[]float64{5, 5, 5}, []float64{1, 1, 7}, []float64{3, 3, 3}, []float64{4, 4, 4}, []float64{2, 2, 2}, e)
want := [][]int16{{3, NoData, 4}, {1, NoData, NoData}, {3, NoData, 3}, {3, NoData, 4}, {2, NoData, 2}}
if !reflect.DeepEqual(got, want) {
t.Fatalf("got %v want %v", got, want)
}
}
func TestKernelMaxAnyValid(t *testing.T) {
got := run(kernelMaxAnyValid, 1, []float64{1, 0, 0}, []float64{4, 0, 2}, []float64{-9999, 0, 0})
if want := []int16{4, NoData, 2}; !reflect.DeepEqual(got[0], want) {
t.Fatalf("got %v want %v", got[0], want)
}
}
// TZ §7 edge cases. Inputs: E, then C1..C5 (irrigation, snow, crop, pasture, tourism).
func TestKernelFuncZoning_TZEdgeCases(t *testing.T) {
e := []float64{0, 3, 5, 4, 2, 2, 3}
s1 := []float64{3, 0, 3, 5, 3, 2, 4}
s2 := []float64{3, 0, 3, 5, 3, 2, 4}
s3 := []float64{3, 0, 3, 2, 3, 2, 5}
s4 := []float64{3, 0, 3, 2, 3, 2, 1}
s5 := []float64{3, 0, 3, 2, 3, 2, 1}
got := run(kernelFuncZoning, 3, e, s1, s2, s3, s4, s5)
// pixel: 0 E invalid -> NoData ; 1 all S invalid -> NoData ; 2 E=5 -> regulated(1) everywhere ;
// 3 E=4, S=5 for sectors 1,2 -> V1 regulated, V2/V3 sector 1 -> class 2 ;
// 4 all S equal (3), E=2 -> irrigation (2) in all ; 5 E=2, S=2 -> V1 regulated (S<3), V2 best=irrigation (2), V3 irrigation (2)
// 6 E=3: S=[4,4,5,1,1] -> V1 (S>=4): winners 1,2,3 max 5 -> crop (4); V2 (S>=3) -> crop (4); V3 -> crop (4)
want := [][]int16{
{NoData, NoData, 1, 1, 2, 1, 4}, // V1
{NoData, NoData, 1, 2, 2, 2, 4}, // V2
{NoData, NoData, 1, 2, 2, 2, 4}, // V3
}
if !reflect.DeepEqual(got, want) {
t.Fatalf("got %v want %v", got, want)
}
}
func TestKernelFuncZoning_TiePriorityIsConfigurable(t *testing.T) {
in := [][]float64{{2}, {3}, {3}, {3}, {3}, {3}}
out := [][]int16{{0}, {0}, {0}}
p := DefaultParams()
p.TiePriority = []int{5, 4, 3, 2, 1}
kernelFuncZoning(in, out, p)
if out[0][0] != 6 || out[1][0] != 6 || out[2][0] != 6 {
t.Fatalf("expected tourism (6) everywhere, got %v", out)
}
}
```
Run: `go test ./internal/landscape/ -run Kernel -v` → FAIL (undefined).
- [ ] **Step 5: kernels.go**
```go
package landscape
// valid reports whether a class value is in the analysis domain (1..5). Any
// other value, including nodata and NaN, is invalid.
func valid(v float64) bool { return v >= 1 && v <= 5 }
// classify maps a float32 score onto classes 1..5 with the fixed intervals
// used by the reference notebook (comparisons in float32, like numpy on a
// float32 array). Scores outside [1, 5] yield NoData.
func classify(x float32) int16 {
switch {
case x < 1.0:
return NoData
case x <= 1.8:
return 1
case x <= 2.6:
return 2
case x <= 3.4:
return 3
case x <= 4.2:
return 4
case x <= 5.0:
return 5
default:
return NoData
}
}
// kernelZIntegrated: all four Z valid -> classify(0.25*Z1 + 0.25*Z2 + 0.25*Z3 + 0.25*Z4).
func kernelZIntegrated(in [][]float64, out [][]int16, _ Params) {
for i := range out[0] {
z1, z2, z3, z4 := in[0][i], in[1][i], in[2][i], in[3][i]
if !(valid(z1) && valid(z2) && valid(z3) && valid(z4)) {
out[0][i] = NoData
continue
}
score := float32(0.25)*float32(z1) + float32(0.25)*float32(z2) + float32(0.25)*float32(z3) + float32(0.25)*float32(z4)
out[0][i] = classify(score)
}
}
// kernelUIntegrated: water and soil required, at least one of crop/veg;
// U = (0.25*water + 0.25*soil + [0.25*crop] + [0.25*veg]) / (0.5 + [0.25] + [0.25]).
func kernelUIntegrated(in [][]float64, out [][]int16, _ Params) {
for i := range out[0] {
water, soil, crop, veg := in[0][i], in[1][i], in[2][i], in[3][i]
cropOK, vegOK := valid(crop), valid(veg)
if !(valid(water) && valid(soil) && (cropOK || vegOK)) {
out[0][i] = NoData
continue
}
num := float32(0.25)*float32(water) + float32(0.25)*float32(soil)
den := float32(0.5)
if cropOK {
num += float32(0.25) * float32(crop)
den += 0.25
}
if vegOK {
num += float32(0.25) * float32(veg)
den += 0.25
}
out[0][i] = classify(num / den)
}
}
// kernelZMax: all four valid -> max.
func kernelZMax(in [][]float64, out [][]int16, _ Params) {
for i := range out[0] {
m, ok := int16(0), true
for _, layer := range in {
v := layer[i]
if !valid(v) {
ok = false
break
}
if int16(v) > m {
m = int16(v)
}
}
if !ok {
m = NoData
}
out[0][i] = m
}
}
// kernelUMax: water and soil required plus one of crop/veg; max over valid layers.
func kernelUMax(in [][]float64, out [][]int16, _ Params) {
for i := range out[0] {
water, soil, crop, veg := in[0][i], in[1][i], in[2][i], in[3][i]
cropOK, vegOK := valid(crop), valid(veg)
if !(valid(water) && valid(soil) && (cropOK || vegOK)) {
out[0][i] = NoData
continue
}
m := max(int16(water), int16(soil))
if cropOK {
m = max(m, int16(crop))
}
if vegOK {
m = max(m, int16(veg))
}
out[0][i] = m
}
}
// kernelE: Z and U valid -> max(Z, U).
func kernelE(in [][]float64, out [][]int16, _ Params) {
for i := range out[0] {
z, u := in[0][i], in[1][i]
if !(valid(z) && valid(u)) {
out[0][i] = NoData
continue
}
out[0][i] = max(int16(z), int16(u))
}
}
// kernelCConflicts: inputs P1..P5, E; output j = min(Pj, E) where both valid.
func kernelCConflicts(in [][]float64, out [][]int16, _ Params) {
e := in[5]
for j := 0; j < 5; j++ {
p := in[j]
for i := range out[j] {
if !(valid(p[i]) && valid(e[i])) {
out[j][i] = NoData
continue
}
out[j][i] = min(int16(p[i]), int16(e[i]))
}
}
}
// kernelMaxAnyValid: max over the valid inputs; NoData when none is valid.
// Used for P_max and C_max.
func kernelMaxAnyValid(in [][]float64, out [][]int16, _ Params) {
for i := range out[0] {
m, any := int16(0), false
for _, layer := range in {
if v := layer[i]; valid(v) {
any = true
m = max(m, int16(v))
}
}
if !any {
m = NoData
}
out[0][i] = m
}
}
// kernelFuncZoning implements TZ_functional_zoning_go.md §4. Inputs: E, then
// S1..S5 (irrigation, snow, crop, pasture, tourism). Outputs: V1, V2, V3.
func kernelFuncZoning(in [][]float64, out [][]int16, p Params) {
e := in[0]
rules := [3]func(e, s int16) bool{
func(e, s int16) bool { return (e <= 2 && s >= int16(p.V1MinSE12)) || (e == 3 && s >= int16(p.V1MinSE3)) },
func(e, s int16) bool { return e <= 2 || (e == 3 && s >= int16(p.V2MinSE3)) || (e == 4 && s >= int16(p.V2MinSE4)) },
func(e, s int16) bool { return e <= 4 },
}
var s [5]int16
var base [5]bool
for i := range out[0] {
if !valid(e[i]) {
out[0][i], out[1][i], out[2][i] = NoData, NoData, NoData
continue
}
ev := int16(e[i])
anyBase := false
for k := 0; k < 5; k++ {
base[k] = valid(in[k+1][i])
if base[k] {
s[k] = int16(in[k+1][i])
anyBase = true
}
}
if !anyBase {
out[0][i], out[1][i], out[2][i] = NoData, NoData, NoData
continue
}
for v := 0; v < 3; v++ {
var cand [5]bool
maxS, anyCand := int16(0), false
for k := 0; k < 5; k++ {
cand[k] = base[k] && rules[v](ev, s[k])
if cand[k] {
anyCand = true
if s[k] > maxS {
maxS = s[k]
}
}
}
if !anyCand {
out[v][i] = 1 // regulated
continue
}
sector := int16(0)
for _, code := range p.TiePriority {
k := code - 1
if cand[k] && s[k] == maxS {
sector = int16(code)
break
}
}
out[v][i] = sector + 1
}
}
}
```
- [ ] **Step 6: Run** `go test ./internal/landscape/ -v` → PASS. `go vet ./internal/landscape/` clean.
- [ ] **Step 7: Commit**
```bash
git add internal/landscape/graph.go internal/landscape/graph_test.go internal/landscape/params.go internal/landscape/params_test.go internal/landscape/kernels.go internal/landscape/kernels_test.go
git commit -m "feat(landscape): computation graph, params and per-pixel kernels"
```
---
### Task 5: Landscape domain, migration 00011 (seed + run tables), repository
**Files:**
- Create: `internal/domain/landscape.go`, `migrations/00011_landscape.sql`, `internal/repository/postgres/landscape.go`
**Interfaces (produced, used by Task 7):**
```go
package domain
type LPRunStatus string // "queued" "running" "succeeded" "failed"
type LPNodeStatus string // "pending" "running" "skipped" "succeeded" "failed"
type LPDatasetRef struct { Group string `json:"group"`; DatasetID uuid.UUID `json:"dataset_id"` }
type LPRun struct { ID uuid.UUID; Status LPRunStatus; FromStage *int; Params json.RawMessage; Overrides map[string]uuid.UUID; Error *string; CreatedAt time.Time; StartedAt, FinishedAt *time.Time; Nodes []LPRunNode }
type LPRunNode struct { ID, RunID uuid.UUID; NodeCode string; Stage int; Status LPNodeStatus; Fingerprint string; Inputs []LPDatasetRef; Outputs []LPDatasetRef; Error *string; StartedAt, FinishedAt *time.Time }
package postgres
type LandscapeRepository — methods:
CreateRun(ctx, run domain.LPRun) (domain.LPRun, error) // inserts run + its Nodes (status pending) in one tx; returns with ids
GetRun(ctx, id uuid.UUID) (domain.LPRun, error) // with nodes ordered by node_code
ListRuns(ctx, limit, offset int) ([]domain.LPRun, error) // without nodes, newest first
CountRuns(ctx) (int, error)
SetRunStatus(ctx, id uuid.UUID, status domain.LPRunStatus, errMsg *string) error // running -> started_at=now(); succeeded/failed -> finished_at=now()
SetNodeStarted(ctx, runID uuid.UUID, code string, fingerprint string, inputs []domain.LPDatasetRef) error // status running, started_at
SetNodeFinished(ctx, runID uuid.UUID, code string, status domain.LPNodeStatus, outputs []domain.LPDatasetRef, errMsg *string) error // finished_at
FindReusableOutputs(ctx, code, fingerprint string, excludeRun uuid.UUID) ([]domain.LPDatasetRef, error) // latest succeeded node with same code+fingerprint; ErrNotFound if none
```
- [ ] **Step 1: Domain** (`internal/domain/landscape.go`) — write the types above with JSON tags: `id, status, from_stage, params, overrides, error, created_at, started_at, finished_at, nodes` and `id, run_id, node_code, stage, status, fingerprint, inputs, outputs, error, started_at, finished_at`. Add constants `LPRunQueued/Running/Succeeded/Failed` and `LPNodePending/Running/Skipped/Succeeded/Failed`.
- [ ] **Step 2: Migration** (`migrations/00011_landscape.sql`)
```sql
-- +goose Up
-- Landscape Program: seed the category and dataset groups (input slots and
-- node outputs; codes must match internal/landscape/graph.go), and store runs.
INSERT INTO categories (code, name, description)
VALUES ('landscape', 'Ландшафтная программа', 'Входные и результирующие растры Ландшафтной программы')
ON CONFLICT (code) DO NOTHING;
INSERT INTO dataset_groups (code, name) VALUES
('lp-z-biodiversity', 'Z Biodiversity'), ('lp-z-lcr', 'Z LCR'), ('lp-z-soil-protection', 'Z Soil Protection'), ('lp-z-wr', 'Z Water Resources'),
('lp-u-surf-water', 'U Surface Water'), ('lp-u-soil', 'U Soil'), ('lp-u-crop', 'U Crop'), ('lp-u-veg', 'U Vegetation'),
('lp-p-irrigation', 'P Irrigation'), ('lp-p-snow', 'P Snow'), ('lp-p-crop', 'P Crop'), ('lp-p-pasture', 'P Pasture'), ('lp-p-tourism', 'P Tourism'),
('lp-z-integrated', 'Z Integrated'), ('lp-u-integrated', 'U Integrated'), ('lp-z-max', 'Z Max'), ('lp-u-max', 'U Max'),
('lp-e', 'E Ecological Constraint'),
('lp-c-irrigation', 'C Irrigation'), ('lp-c-snow', 'C Snow'), ('lp-c-crop', 'C Crop'), ('lp-c-pasture', 'C Pasture'), ('lp-c-tourism', 'C Tourism'),
('lp-p-max', 'P Max'),
('lp-v1-conservative', 'V1 Conservative'), ('lp-v2-balanced', 'V2 Balanced'), ('lp-v3-development', 'V3 Development'), ('lp-c-max', 'C Max')
ON CONFLICT (code) DO NOTHING;
CREATE TABLE lp_runs (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
status VARCHAR(32) NOT NULL,
from_stage SMALLINT,
params JSONB NOT NULL,
overrides JSONB NOT NULL DEFAULT '{}'::jsonb,
error TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
started_at TIMESTAMPTZ,
finished_at TIMESTAMPTZ
);
CREATE INDEX idx_lp_runs_created_at ON lp_runs (created_at DESC);
CREATE TABLE lp_run_nodes (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
run_id UUID NOT NULL REFERENCES lp_runs (id) ON DELETE CASCADE,
node_code VARCHAR(8) NOT NULL,
stage SMALLINT NOT NULL,
status VARCHAR(32) NOT NULL,
fingerprint TEXT NOT NULL DEFAULT '',
input_datasets JSONB NOT NULL DEFAULT '[]'::jsonb,
output_datasets JSONB,
error TEXT,
started_at TIMESTAMPTZ,
finished_at TIMESTAMPTZ,
UNIQUE (run_id, node_code)
);
CREATE INDEX idx_lp_run_nodes_lookup ON lp_run_nodes (node_code, fingerprint, status, finished_at DESC);
-- +goose Down
DROP TABLE lp_run_nodes;
DROP TABLE lp_runs;
-- Seeded category/groups are left in place (datasets may reference them).
```
- [ ] **Step 3: Repository** (`internal/repository/postgres/landscape.go`) — implement the methods listed in Interfaces. Key SQL:
```sql
-- CreateRun (tx):
INSERT INTO lp_runs (status, from_stage, params, overrides) VALUES ($1,$2,$3,$4) RETURNING <runColumns>
INSERT INTO lp_run_nodes (run_id, node_code, stage, status) VALUES ($1,$2,$3,'pending') RETURNING <nodeColumns> -- per node
-- GetRun: SELECT <runColumns> FROM lp_runs WHERE id=$1 ; SELECT <nodeColumns> FROM lp_run_nodes WHERE run_id=$1 ORDER BY node_code
-- ListRuns: ... ORDER BY created_at DESC LIMIT $1 OFFSET $2
-- SetRunStatus:
UPDATE lp_runs SET status=$2, error=$3,
started_at = CASE WHEN $2 = 'running' THEN now() ELSE started_at END,
finished_at = CASE WHEN $2 IN ('succeeded','failed') THEN now() ELSE finished_at END
WHERE id=$1
-- SetNodeStarted:
UPDATE lp_run_nodes SET status='running', fingerprint=$3, input_datasets=$4, started_at=now(), error=NULL WHERE run_id=$1 AND node_code=$2
-- SetNodeFinished:
UPDATE lp_run_nodes SET status=$3, output_datasets=$4, error=$5, finished_at=now() WHERE run_id=$1 AND node_code=$2
-- FindReusableOutputs:
SELECT output_datasets FROM lp_run_nodes
WHERE node_code=$1 AND fingerprint=$2 AND status='succeeded' AND run_id <> $3 AND output_datasets IS NOT NULL
ORDER BY finished_at DESC LIMIT 1
```
`runColumns = id, status, from_stage, params, overrides, error, created_at, started_at, finished_at`; scan `overrides` into `map[string]uuid.UUID` via a `[]byte` + `json.Unmarshal` (empty → empty map). `nodeColumns = id, run_id, node_code, stage, status, fingerprint, input_datasets, output_datasets, error, started_at, finished_at`; JSON arrays scanned via pgx into `[]domain.LPDatasetRef` (pgx handles `jsonb` → struct slice; `output_datasets` NULL → nil). Every `Exec` with `RowsAffected()==0` returns `domain.ErrNotFound`; wrap errors with `mapError`.
- [ ] **Step 4: Compile** — `go build ./internal/domain/ ./internal/repository/postgres/`; `go vet` both. (No DB in CI; SQL is exercised by the integration task.)
- [ ] **Step 5: Commit**
```bash
git add internal/domain/landscape.go migrations/00011_landscape.sql internal/repository/postgres/landscape.go
git commit -m "feat(landscape): run/node domain, seed migration, repository"
```
---
### Task 6: `Compute` — window executor over `raster.IO`
**Files:**
- Create: `internal/landscape/compute.go`, `internal/landscape/compute_test.go`, `internal/landscape/fakeio_test.go`
**Interfaces:**
- Consumes: Task 3 `raster.IO/Reader/Writer/Grid/Window/Windows/DefaultWriterOptions`; Task 4 `Node`, `Params`.
- Produces (used by Task 7 via `service.ComputeFunc`):
```go
// Compute runs one node. inputs maps every Node.Inputs group code to a local
// file; outputs are written to filepath.Join(outDir, group+".tif") and returned
// keyed by group code. Grids of all inputs must be identical (raster.IO.SameGrid)
// or an error is returned before any output is created.
func Compute(ctx context.Context, rio raster.IO, node Node, inputs map[string]string, outDir string, p Params) (map[string]string, error)
```
- [ ] **Step 1: Fake IO for tests** (`fakeio_test.go`)
```go
package landscape
import (
"context"
"fmt"
"sync"
"gis/internal/raster"
)
// fakeIO keeps rasters in memory keyed by path.
type fakeIO struct {
mu sync.Mutex
files map[string]*fakeRaster
}
type fakeRaster struct {
grid raster.Grid
data []float64 // row-major
}
func newFakeIO() *fakeIO { return &fakeIO{files: map[string]*fakeRaster{}} }
func (f *fakeIO) put(path string, g raster.Grid, data []float64) {
f.files[path] = &fakeRaster{grid: g, data: data}
}
type fakeReader struct{ r *fakeRaster }
func (r *fakeReader) Grid() raster.Grid { return r.r.grid }
func (r *fakeReader) NoData() (float64, bool) { return -9999, true }
func (r *fakeReader) Close() error { return nil }
func (r *fakeReader) ReadFloat64(w raster.Window, dst []float64) error {
for y := 0; y < w.Height; y++ {
copy(dst[y*w.Width:(y+1)*w.Width], r.r.data[(w.Y+y)*r.r.grid.Width+w.X:(w.Y+y)*r.r.grid.Width+w.X+w.Width])
}
return nil
}
type fakeWriter struct {
f *fakeIO
r *fakeRaster
}
func (w *fakeWriter) WriteInt16(win raster.Window, src []int16) error {
w.f.mu.Lock()
defer w.f.mu.Unlock()
for y := 0; y < win.Height; y++ {
for x := 0; x < win.Width; x++ {
w.r.data[(win.Y+y)*w.r.grid.Width+win.X+x] = float64(src[y*win.Width+x])
}
}
return nil
}
func (w *fakeWriter) Close() error { return nil }
func (f *fakeIO) Open(_ context.Context, path string) (raster.Reader, error) {
f.mu.Lock()
defer f.mu.Unlock()
r, ok := f.files[path]
if !ok {
return nil, fmt.Errorf("fake: %s not found", path)
}
return &fakeReader{r: r}, nil
}
func (f *fakeIO) Create(_ context.Context, path string, g raster.Grid, _ raster.WriterOptions) (raster.Writer, error) {
f.mu.Lock()
defer f.mu.Unlock()
r := &fakeRaster{grid: g, data: make([]float64, g.Width*g.Height)}
f.files[path] = r
return &fakeWriter{f: f, r: r}, nil
}
func (f *fakeIO) SameGrid(a, b raster.Grid) bool { return a == b }
```
- [ ] **Step 2: Failing tests** (`compute_test.go`)
```go
package landscape
import (
"context"
"path/filepath"
"testing"
"gis/internal/raster"
)
func grid(w, h int) raster.Grid {
return raster.Grid{Width: w, Height: h, GeoTransform: [6]float64{0, 30, 0, 0, 0, -30}, ProjectionWKT: "EPSG:32642"}
}
func constRaster(w, h int, v float64) []float64 {
d := make([]float64, w*h)
for i := range d {
d[i] = v
}
return d
}
func TestCompute_E_TilesWindowsAndWritesEveryPixel(t *testing.T) {
fio := newFakeIO()
g := grid(10, 7)
z := constRaster(10, 7, 2)
u := constRaster(10, 7, 4)
z[0] = -9999 // pixel 0 invalid
fio.put("z.tif", g, z)
fio.put("u.tif", g, u)
node, _ := NodeByCode("10")
p := DefaultParams()
p.WindowSize = 4 // forces partial windows (10x7 with 4x4 tiles)
out, err := Compute(context.Background(), fio, node, map[string]string{OutZIntegrated: "z.tif", OutUIntegrated: "u.tif"}, "/out", p)
if err != nil {
t.Fatal(err)
}
path := out[OutE]
if path != filepath.Join("/out", OutE+".tif") {
t.Fatalf("unexpected output path %q", path)
}
res := fio.files[path]
if res.grid != g {
t.Fatal("output grid must copy the first input's grid")
}
for i, v := range res.data {
want := 4.0
if i == 0 {
want = -9999
}
if v != want {
t.Fatalf("pixel %d = %v want %v", i, v, want)
}
}
}
func TestCompute_FailsOnGridMismatch(t *testing.T) {
fio := newFakeIO()
fio.put("z.tif", grid(10, 7), constRaster(10, 7, 2))
fio.put("u.tif", grid(9, 7), constRaster(9, 7, 4))
node, _ := NodeByCode("10")
_, err := Compute(context.Background(), fio, node, map[string]string{OutZIntegrated: "z.tif", OutUIntegrated: "u.tif"}, "/out", DefaultParams())
if err == nil {
t.Fatal("expected grid mismatch error")
}
if _, created := fio.files[filepath.Join("/out", OutE+".tif")]; created {
t.Fatal("no output must be created on mismatch")
}
}
func TestCompute_FailsOnMissingInput(t *testing.T) {
fio := newFakeIO()
fio.put("z.tif", grid(2, 2), constRaster(2, 2, 2))
node, _ := NodeByCode("10")
if _, err := Compute(context.Background(), fio, node, map[string]string{OutZIntegrated: "z.tif"}, "/out", DefaultParams()); err == nil {
t.Fatal("expected error for missing input group")
}
}
func TestCompute_MultiOutputNode(t *testing.T) {
fio := newFakeIO()
g := grid(3, 1)
paths := map[string]string{}
for i, s := range []string{SlotPIrrigation, SlotPSnow, SlotPCrop, SlotPPasture, SlotPTourism} {
paths[s] = s + ".tif"
fio.put(paths[s], g, constRaster(3, 1, float64(i+1)))
}
paths[OutE] = "e.tif"
fio.put("e.tif", g, constRaster(3, 1, 3))
node, _ := NodeByCode("20")
out, err := Compute(context.Background(), fio, node, paths, "/o", DefaultParams())
if err != nil {
t.Fatal(err)
}
if len(out) != 5 {
t.Fatalf("want 5 outputs, got %d", len(out))
}
if fio.files[out[OutCTourism]].data[0] != 3 || fio.files[out[OutCIrrigation]].data[0] != 1 {
t.Fatal("min(P, E) not applied per output")
}
}
```
Run: `go test ./internal/landscape/ -run Compute -v` → FAIL.
- [ ] **Step 3: compute.go**
```go
package landscape
import (
"context"
"fmt"
"path/filepath"
"runtime"
"sync"
"gis/internal/raster"
"golang.org/x/sync/errgroup"
)
// Compute runs one node over its input rasters. See the package doc for the
// concurrency model: windows are processed by a worker pool; each worker owns
// its own Reader handles (GDAL datasets are not goroutine-safe) and writes are
// serialised per output. Window order does not affect the result.
func Compute(ctx context.Context, rio raster.IO, node Node, inputs map[string]string, outDir string, p Params) (map[string]string, error) {
p = p.WithDefaults()
paths := make([]string, len(node.Inputs))
for i, g := range node.Inputs {
path, ok := inputs[g]
if !ok || path == "" {
return nil, fmt.Errorf("node %s (%s): missing input %q", node.Code, node.Name, g)
}
paths[i] = path
}
// Validate grids with one set of handles before creating anything.
grid, err := checkGrids(ctx, rio, node, paths)
if err != nil {
return nil, err
}
outPaths := make(map[string]string, len(node.Outputs))
writers := make([]raster.Writer, len(node.Outputs))
for j, g := range node.Outputs {
path := filepath.Join(outDir, g+".tif")
w, err := rio.Create(ctx, path, grid, raster.DefaultWriterOptions())
if err != nil {
closeWriters(writers[:j])
return nil, fmt.Errorf("node %s: create output %s: %w", node.Code, g, err)
}
writers[j] = w
outPaths[g] = path
}
windows := raster.Windows(grid.Width, grid.Height, p.WindowSize)
jobs := make(chan raster.Window)
var writeMu sync.Mutex
eg, gctx := errgroup.WithContext(ctx)
workers := min(runtime.GOMAXPROCS(0), len(windows))
for w := 0; w < workers; w++ {
eg.Go(func() error {
readers, err := openAll(gctx, rio, paths)
if err != nil {
return err
}
defer closeReaders(readers)
size := p.WindowSize * p.WindowSize
in := make([][]float64, len(paths))
for i := range in {
in[i] = make([]float64, size)
}
out := make([][]int16, len(writers))
for j := range out {
out[j] = make([]int16, size)
}
for win := range jobs {
n := win.Width * win.Height
inView := make([][]float64, len(in))
for i, r := range readers {
if err := r.ReadFloat64(win, in[i][:n]); err != nil {
return fmt.Errorf("node %s: read %s window %+v: %w", node.Code, node.Inputs[i], win, err)
}
inView[i] = in[i][:n]
}
outView := make([][]int16, len(out))
for j := range out {
outView[j] = out[j][:n]
}
node.Kernel(inView, outView, p)
writeMu.Lock()
for j, wr := range writers {
if err := wr.WriteInt16(win, outView[j]); err != nil {
writeMu.Unlock()
return fmt.Errorf("node %s: write %s window %+v: %w", node.Code, node.Outputs[j], win, err)
}
}
writeMu.Unlock()
}
return nil
})
}
eg.Go(func() error {
defer close(jobs)
for _, win := range windows {
select {
case jobs <- win:
case <-gctx.Done():
return gctx.Err()
}
}
return nil
})
runErr := eg.Wait()
if err := closeWriters(writers); err != nil && runErr == nil {
runErr = err
}
if runErr != nil {
return nil, runErr
}
return outPaths, nil
}
// checkGrids opens every input once, verifies all grids match the first, and
// returns that grid.
func checkGrids(ctx context.Context, rio raster.IO, node Node, paths []string) (raster.Grid, error) {
readers, err := openAll(ctx, rio, paths)
if err != nil {
return raster.Grid{}, err
}
defer closeReaders(readers)
ref := readers[0].Grid()
for i := 1; i < len(readers); i++ {
if g := readers[i].Grid(); !rio.SameGrid(ref, g) {
return raster.Grid{}, fmt.Errorf("node %s: input %s grid (%dx%d, %v) differs from %s (%dx%d, %v); inputs must be aligned",
node.Code, node.Inputs[i], g.Width, g.Height, g.GeoTransform, node.Inputs[0], ref.Width, ref.Height, ref.GeoTransform)
}
}
return ref, nil
}
func openAll(ctx context.Context, rio raster.IO, paths []string) ([]raster.Reader, error) {
readers := make([]raster.Reader, 0, len(paths))
for _, p := range paths {
r, err := rio.Open(ctx, p)
if err != nil {
closeReaders(readers)
return nil, err
}
readers = append(readers, r)
}
return readers, nil
}
func closeReaders(rs []raster.Reader) {
for _, r := range rs {
if r != nil {
_ = r.Close()
}
}
}
func closeWriters(ws []raster.Writer) error {
var first error
for _, w := range ws {
if w == nil {
continue
}
if err := w.Close(); err != nil && first == nil {
first = err
}
}
return first
}
```
- [ ] **Step 4: Run** `go test ./internal/landscape/ -race -v` → PASS (race detector must be clean).
- [ ] **Step 5: Commit**
```bash
git add internal/landscape/compute.go internal/landscape/compute_test.go internal/landscape/fakeio_test.go
git commit -m "feat(landscape): window executor Compute with worker pool and grid checks"
```
---
### Task 7: LandscapeService (runs, planning, input resolution, execution) + RabbitMQ job
**Files:**
- Create: `internal/service/landscape.go`, `internal/service/landscape_test.go`, `internal/messaging/rabbitmq/landscape_consumer.go`
- Modify: none outside these (wiring is Task 8)
**Interfaces:**
- Consumes: Task 2 (`DatasetRepository.LatestInGroup`, `DatasetGroupRepository.GetByCode`, `DatasetService.Upload/CreateRasterFromFile`, `ObjectStore`), Task 4 (`landscape.Nodes/Stages/Slots/Outputs/Params/…`), Task 5 (`domain.LPRun*`, repo methods), `raster.IO` (Task 3).
- Produces (used by Task 8):
```go
package service
type ComputeFunc func(ctx context.Context, rio raster.IO, node landscape.Node, inputs map[string]string, outDir string, p landscape.Params) (map[string]string, error)
type LandscapeRepository interface { /* the 8 methods of postgres.LandscapeRepository, same signatures */ }
type LandscapeJobEnqueuer interface { EnqueueRun(ctx context.Context, runID uuid.UUID) error }
type datasetPublisher interface {
Upload(ctx context.Context, in UploadInput) (domain.Dataset, error)
CreateRasterFromFile(ctx context.Context, in CreateRasterInput) (domain.Dataset, error)
}
func NewLandscapeService(runs LandscapeRepository, datasets DatasetRepository, groups DatasetGroupRepository, categories categoryReader, publisher datasetPublisher, store ObjectStore, jobs LandscapeJobEnqueuer, rio raster.IO, compute ComputeFunc, log *slog.Logger) *LandscapeService
type SlotInfo struct { Code string `json:"code"`; Title string `json:"title"`; GroupID uuid.UUID `json:"group_id"`; Latest *domain.DatasetSummary `json:"latest"` }
type ResultInfo struct { Group string `json:"group"`; Title string `json:"title"`; Node string `json:"node"`; GroupID uuid.UUID `json:"group_id"`; Latest *domain.DatasetSummary `json:"latest"`; CogURL *string `json:"cog_url"`; SldURL *string `json:"sld_url"` }
type SlotUploadInput struct { Filename, ContentType string; Size int64; Reader io.Reader; Name string; StartDate, EndDate *string }
type RunInput struct { FromStage *int; Overrides map[string]uuid.UUID; Params landscape.Params }
type LPRunPage struct { Items []domain.LPRun; Page, PageSize, Total int }
func (s *LandscapeService) Slots(ctx) ([]SlotInfo, error)
func (s *LandscapeService) UploadToSlot(ctx, code string, in SlotUploadInput) (domain.Dataset, error) // ErrNotFound for unknown slot
func (s *LandscapeService) CreateRun(ctx, in RunInput) (domain.LPRun, error)
func (s *LandscapeService) GetRun(ctx, id uuid.UUID) (domain.LPRun, error)
func (s *LandscapeService) ListRuns(ctx, page, pageSize int) (LPRunPage, error)
func (s *LandscapeService) Results(ctx) ([]ResultInfo, error)
func (s *LandscapeService) Execute(ctx, runID uuid.UUID) error // worker entry point
func (s *LandscapeService) MarkRunFailed(ctx, runID uuid.UUID, reason string) error
package rabbitmq
const LandscapeRunRoutingKey = "landscape.run"; const LandscapeRunQueue = "gis.landscape.run"
type LandscapeJob struct { RunID uuid.UUID `json:"run_id"` }
type LandscapeJobPublisher struct{ pub *Publisher }; func NewLandscapeJobPublisher(pub *Publisher) *LandscapeJobPublisher; func (p *LandscapeJobPublisher) EnqueueRun(ctx, runID uuid.UUID) error
type LandscapeProcessor interface { Execute(ctx, runID uuid.UUID) error; MarkRunFailed(ctx, runID uuid.UUID, reason string) error }
func NewLandscapeRunHandler(p LandscapeProcessor, log *slog.Logger) Handler
```
- [ ] **Step 1: Failing service tests** (`internal/service/landscape_test.go`). Build stubs: `stubLPRepo` (in-memory runs/nodes, records `FindReusableOutputs` hits), reuse `stubDatasetRepo`, `stubGroupRepo` (seed all `landscape.Slots`/`Outputs` codes as groups), `stubCategoryRepo` (from `category_test.go`, seed code `landscape`), a `stubPublisher` implementing `datasetPublisher` that records `CreateRasterFromFile` calls and inserts a ready raster dataset with the given group into `stubDatasetRepo`, `stubStore` with `Get` returning bytes, `stubLPJobs`, `fakeIO`-free approach: `compute` is a `ComputeFunc` stub that writes an empty file per output and records the `inputs` map it received.
Tests:
```go
func TestLandscapeService_CreateRun_ValidatesParamsAndOverrides(t *testing.T)
// invalid tie priority -> ErrValidation; override for unknown group -> ErrValidation;
// override dataset not ready/not raster -> ErrValidation; valid -> run queued with 9 pending nodes and job enqueued.
func TestLandscapeService_Execute_ResolvesInputsAndPublishesOutputs(t *testing.T)
// seed ready raster datasets for all 13 slots; Execute; expect: run succeeded, 9 nodes succeeded,
// compute stub called 9 times, node "10" received inputs produced by nodes "00" and "01" in this run
// (dataset ids equal to the publisher's created ids), each published dataset has Automated=true,
// DatasetGroupID = its output group, Meta contains "lp_run_id".
func TestLandscapeService_Execute_ReusesFingerprintMatches(t *testing.T)
// run once (all computed); run again with unchanged inputs -> every node skipped, compute stub not called,
// outputs equal to first run's outputs.
func TestLandscapeService_Execute_RecomputesDownstreamOfChangedInput(t *testing.T)
// after first run, upload a newer dataset to lp-p-crop (later end_date); second run: nodes 00,01,02,03,10 skipped;
// 20,21 computed (inputs changed); 30,31 computed (their inputs are new C datasets).
func TestLandscapeService_Execute_FromStageForcesRecompute(t *testing.T)
// second run with FromStage=2 and unchanged inputs -> stages 0,1 skipped, stages 2,3 computed.
func TestLandscapeService_Execute_MissingSlotFailsRun(t *testing.T)
// no dataset in lp-z-wr -> node 00 and 02 failed with message containing "lp-z-wr"; run failed; stage 1 never started.
func TestLandscapeService_UploadToSlot_UnknownSlot(t *testing.T) // ErrNotFound
func TestLandscapeService_Results_ListsLatestPerOutput(t *testing.T) // cog_url = "/datasets/<id>.cog"; sld_url only when group has SLD
```
Write the test bodies fully (arrange/act/assert) following `dataset_test.go` style. Run → FAIL (undefined).
- [ ] **Step 2: Service implementation** (`internal/service/landscape.go`) — implement per spec §3.5–3.7. Core pieces:
```go
// fingerprint is a stable hash of the node, its resolved inputs and the
// node-relevant params.
func fingerprint(node landscape.Node, inputs []domain.LPDatasetRef, p landscape.Params) string {
sorted := append([]domain.LPDatasetRef(nil), inputs...)
sort.Slice(sorted, func(i, j int) bool { return sorted[i].Group < sorted[j].Group })
payload, _ := json.Marshal(struct {
Node string `json:"node"`
Inputs []domain.LPDatasetRef `json:"inputs"`
Params landscape.Params `json:"params"`
}{node.Code, sorted, p.ForNode(node.Code)})
sum := sha256.Sum256(payload)
return hex.EncodeToString(sum[:])
}
```
`Execute(ctx, runID)`:
1. `run := runs.GetRun`; parse `run.Params` into `landscape.Params` (`WithDefaults`); `runs.SetRunStatus(running)`.
2. `produced := map[string]uuid.UUID{}` guarded by a mutex (outputs created in this run).
3. For each stage from `landscape.Stages()`: `errgroup` over nodes → `s.runNode(ctx, run, node, params, produced)`; on error: `SetRunStatus(failed, err)` and `return err`.
4. `SetRunStatus(succeeded, nil)`.
`runNode`:
1. Resolve inputs: for each group `g` in `node.Inputs`: `run.Overrides[g]` → else `produced[g]` → else `datasets.LatestInGroup(ctx, groupID(g), &ready, &raster)`; `groupID` via `groups.GetByCode`. Missing → `fmt.Errorf("node %s: no ready raster dataset in group %q", …)`.
2. `fp := fingerprint(node, refs, params)`; `runs.SetNodeStarted(run.ID, node.Code, fp, refs)`.
3. If `run.FromStage == nil || node.Stage < *run.FromStage`: `outs, err := runs.FindReusableOutputs(ctx, node.Code, fp, run.ID)`; if found and every `outs[i].DatasetID` exists with `Status == ready` (`datasets.GetByID`) → record into `produced`, `SetNodeFinished(skipped, outs, nil)`, return.
4. Temp dir `os.MkdirTemp("", "lp-"+node.Code+"-*")` (deferred `RemoveAll`); download each input object (`store.Get(dataset.StorageKey)` → file `<tmp>/in-<group>.tif`).
5. `outPaths, err := s.compute(ctx, s.rio, node, inputPaths, tmp, params)`.
6. For each output group: `publisher.CreateRasterFromFile(ctx, CreateRasterInput{CategoryID: landscapeCategory.ID, DatasetGroupID: &gid, Name: title + " " + run.StartedAt(or now).Format("2006-01-02"), Filename: group + ".tif", Path: outPaths[group], StartDate: minStart(inputs), EndDate: maxEnd(inputs), Meta: {"lp_run_id", "lp_node", "inputs"}, Automated: true})`; collect refs; `produced[group] = ds.ID`.
7. `SetNodeFinished(succeeded, outRefs, nil)`. On any error: `SetNodeFinished(failed, nil, &msg)` and return the error.
`CreateRun`: `params := in.Params.WithDefaults(); params.Validate()` → wrap into `ErrValidation`; `FromStage` in 0..3; each override: group code must be a known group (`landscape.SlotByCode` or `OutputByCode`) and the dataset must exist, be `raster` and `ready` (else `ErrValidation`); marshal params; build `domain.LPRun{Status: queued, FromStage, Params, Overrides, Nodes: one pending node per landscape.Nodes}`; `runs.CreateRun`; `jobs.EnqueueRun(run.ID)` (on failure `SetRunStatus(failed, "failed to enqueue: …")` and return error).
`UploadToSlot`: `landscape.SlotByCode` (else `ErrNotFound`); `groups.GetByCode(code)`; `categories.GetByCode(landscape.CategoryCode)`; `publisher.Upload(UploadInput{CategoryID, DatasetGroupID: &g.ID, Name (default filename), Filename, FileType: raster, ContentType, Size, Reader, StartDate, EndDate})`.
`Slots` / `Results`: iterate `landscape.Slots` / `landscape.Outputs`, look up group by code, `LatestInGroup` (`Slots`: no status filter; `Results`: status ready) → `DatasetSummary` built from the `Dataset` (write a small `summaryOf(d domain.Dataset) domain.DatasetSummary` helper), `cog_url = "/datasets/" + id + ".cog"`, `sld_url = "/dataset-groups/" + gid + "/sld"` when the group has `SldStorageKey`.
`MarkRunFailed` → `runs.SetRunStatus(failed, &reason)`.
- [ ] **Step 3: RabbitMQ** (`internal/messaging/rabbitmq/landscape_consumer.go`) — mirror `parse_consumer.go`: constants, `LandscapeJob`, `LandscapeJobPublisher.EnqueueRun` (marshal `{run_id}` and `pub.Publish(ctx, LandscapeRunRoutingKey, body)`), `LandscapeProcessor`, `landscapeHandler.Handle` (decode, log, call `Execute`; on error call `MarkRunFailed` best-effort and return the error so the delivery is nacked), `NewLandscapeRunHandler`.
- [ ] **Step 4: Run** `go test ./internal/service/ ./internal/messaging/... -race` → PASS.
- [ ] **Step 5: Commit**
```bash
git add internal/service/landscape.go internal/service/landscape_test.go internal/messaging/rabbitmq/landscape_consumer.go
git commit -m "feat(landscape): run planning/execution service with fingerprint reuse; RabbitMQ job"
```
---
### Task 8: HTTP API, wiring, OpenAPI, README, integration check
**Files:**
- Create: `internal/transport/http/landscape_handler.go`, `internal/transport/http/landscape_handler_test.go`
- Modify: `internal/transport/http/router.go`, `internal/app/app.go`, `api/openapi.yaml`, `README.md`, `deployments/docker-compose.yml` (only if the worker needs an extra env; likely none)
**Interfaces:** consumes Task 7's `LandscapeService` API and Task 3's `raster.NewGodalIO()`; Task 6's `landscape.Compute` is passed as `service.ComputeFunc`.
- [ ] **Step 1: Handler** (`internal/transport/http/landscape_handler.go`)
```go
// LandscapeHandler serves the /landscape routes.
type LandscapeHandler struct {
svc *service.LandscapeService
validate *validator.Validate
}
func NewLandscapeHandler(svc *service.LandscapeService, validate *validator.Validate) *LandscapeHandler
func (h *LandscapeHandler) Register(r chi.Router) {
r.Get("/slots", h.slots)
r.Post("/slots/{code}", h.uploadSlot)
r.Get("/runs", h.listRuns)
r.Post("/runs", h.createRun)
r.Get("/runs/{id}", h.getRun)
r.Get("/results", h.results)
}
type runRequest struct {
FromStage *int `json:"from_stage" validate:"omitempty,gte=0,lte=3"`
Overrides map[string]string `json:"overrides" validate:"omitempty,dive,uuid"`
Params *landscape.Params `json:"params"`
}
```
`uploadSlot`: `r.ParseMultipartForm(maxUploadBytes)`, `FormFile("file")`, optional `name`, `start_date`, `end_date`; content type default `image/tiff`; 201 with dataset; unknown slot → 404 via `respondDomainError`. `createRun`: decode, validate, parse override UUIDs, `Params` nil → `landscape.Params{}`; 202 with run. `getRun`/`listRuns` (`page`, `page_size`, `newPaginated`)/`slots`/`results` → 200 JSON.
Handler tests (`httptest` + a `LandscapeService` built on the stubs from `landscape_test.go` is in another package — instead test only request parsing paths that fail before the service is needed: invalid JSON → 400, `from_stage: 7` → 422, override not a UUID → 422; construct the handler with a nil service since those paths return early).
- [ ] **Step 2: Wiring**
`router.go`: `Landscape *LandscapeHandler` in `RouterDeps`; `r.Route("/landscape", deps.Landscape.Register)`.
`app.go`:
```go
lpRepo := postgres.NewLandscapeRepository(pool)
lpJobs := rabbitmq.NewLandscapeJobPublisher(publisher)
rio := raster.NewGodalIO()
landscapeSvc := service.NewLandscapeService(lpRepo, datasetRepo, groupRepo, categoryRepo, datasetsSvc, store, lpJobs, rio, landscape.Compute, log)
```
Add `landscape *service.LandscapeService` to `App`; `Landscape: transporthttp.NewLandscapeHandler(a.landscape, validate)`; add `LandscapeConsumer()` returning `rabbitmq.NewConsumer(a.rabbit, rabbitmq.LandscapeRunQueue, rabbitmq.LandscapeRunRoutingKey, "gis-landscape-runner", rabbitmq.NewLandscapeRunHandler(a.landscape, a.Log), a.Log)` and include it in `Consumers()`. Update `workerCmd.Short` to mention landscape runs.
- [ ] **Step 3: OpenAPI + README** — tag `Landscape`; paths `/landscape/slots`, `/landscape/slots/{code}`, `/landscape/runs`, `/landscape/runs/{id}`, `/landscape/results`; schemas `LandscapeSlot`, `LandscapeRunInput`, `LandscapeParams`, `LandscapeRun`, `LandscapeRunNode`, `LandscapeDatasetRef`, `LandscapeResult`, `LandscapeRunPage`. README: new section "Landscape Program (ЛП)" — graph table (copy from spec §3.1), pixel rules summary, how to upload inputs, start a run, incremental recompute (`from_stage`), where results live, build requirements (cgo + GDAL: `brew install gdal` locally; Dockerfile handles the image).
- [ ] **Step 4: Full verification**
```bash
go build ./... && go vet ./... && go test ./... -race
```
Expected: all PASS. If Docker is available: `docker compose -f deployments/docker-compose.yml build` and `docker compose -f deployments/docker-compose.yml up -d postgres` + `go run ./cmd/gis migrate up` against it to prove migrations 00010/00011 apply (then `migrate status`). Report exactly what was and was not verified.
- [ ] **Step 5: Commit**
```bash
git add internal/transport/http/landscape_handler.go internal/transport/http/landscape_handler_test.go internal/transport/http/router.go internal/app/app.go internal/cli/worker.go api/openapi.yaml README.md
git commit -m "feat(landscape): HTTP API, worker consumer, wiring, docs"
```
---
## Self-review notes
- Spec §2.1–2.4 → Tasks 1–2. §3.1–3.2 → Task 4. §3.3 → Task 3. §3.4 → Task 6. §3.5–3.6 → Tasks 5, 7 (+ `CreateRasterFromFile` in Task 2). §3.7 → Task 8. §3.8 error handling is embedded in Tasks 6–7. §3.9 tests are spread across tasks.
- Name consistency: `LatestInGroup(ctx, groupID, status *string, fileType *domain.FileType)` (Tasks 1, 2, 7); `CreateRasterInput` / `CreateRasterFromFile` (Tasks 2, 7); `raster.Windows`, `DefaultWriterOptions` (Tasks 3, 6); `landscape.Stages`, `NodeByCode`, `SlotByCode`, `OutputByCode`, `Params.WithDefaults/Validate/ForNode` (Tasks 4, 6, 7); `domain.LPDatasetRef` (Tasks 5, 7).