diff --git a/README.md b/README.md index 98b5609..b37aefb 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,8 @@ internal/ transport/http/ chi router, middleware, handlers storage/s3/ MinIO/S3 object storage messaging/rabbitmq/ connection, publisher, consumer + landscape/ Landscape Program graph, kernels, window executor + raster/ GDAL raster I/O (COG conversion + godal engine) platform/logger/ slog setup pkg/httputil/ generic JSON/validation HTTP helpers migrations/ embedded goose SQL migrations @@ -126,6 +128,126 @@ which acts as the fallback style for every dataset in the group: otherwise falls back to its group's — 404 only when neither exists. Deleting a group also removes its stored SLD object. +## Landscape Program (ЛП) + +The **Landscape Program** is a DAG of per-pixel raster computations that turns +ecological value (Z), vulnerability (U) and sector suitability (P) class rasters +into an ecological constraint (E), per-sector conflicts (C) and three functional +zoning scenarios (V1–V3). Every input and output of every node is a *dataset +group*; the leaf inputs are the **slots** users upload to, and every result is an +ordinary `raster` dataset (COG + SLD) served by the existing dataset endpoints. + +The graph lives in Go (`internal/landscape/graph.go`); the groups and the +`landscape` category are seeded by migration `00011_landscape.sql`. A node code +`XY` reads as `X` = stage, `Y` = function within the stage: a stage starts only +after every node of all lower stages finished, and nodes of one stage run +concurrently. + +| Node | Name | Inputs (group codes) | Outputs (group codes) | +|---|---|---|---| +| 00 | z_integrated | `lp-z-biodiversity`, `lp-z-lcr`, `lp-z-soil-protection`, `lp-z-wr` | `lp-z-integrated` | +| 01 | u_integrated | `lp-u-surf-water`, `lp-u-soil`, `lp-u-crop`, `lp-u-veg` | `lp-u-integrated` | +| 02 | z_max | Z1..Z4 as above | `lp-z-max` | +| 03 | u_max | U as above | `lp-u-max` | +| 10 | e_ecological_constraint | `lp-z-integrated`, `lp-u-integrated` | `lp-e` | +| 20 | c_conflicts | `lp-p-irrigation`, `lp-p-snow`, `lp-p-crop`, `lp-p-pasture`, `lp-p-tourism`, `lp-e` | `lp-c-irrigation`, `lp-c-snow`, `lp-c-crop`, `lp-c-pasture`, `lp-c-tourism` | +| 21 | p_max | five P groups | `lp-p-max` | +| 30 | func_zoning | `lp-e`, five C groups | `lp-v1-conservative`, `lp-v2-balanced`, `lp-v3-development` | +| 31 | c_max | five C groups | `lp-c-max` | + +Sector order everywhere: 1 irrigation, 2 snow, 3 crop, 4 pasture, 5 tourism. + +### Pixel rules + +A pixel value is **valid** iff `1 <= v <= 5`; anything else (nodata included) is +invalid. Output rasters are `int16` with nodata `-9999`. `classify(x)` maps a +float32 score to a class: `<= 1.8 → 1`, `<= 2.6 → 2`, `<= 3.4 → 3`, `<= 4.2 → 4`, +`<= 5.0 → 5` (and requires `x >= 1.0`). + +| Node | Valid domain (pixel) | Value | +|---|---|---| +| z_integrated | all four Z valid | `classify(0.25*Z1 + 0.25*Z2 + 0.25*Z3 + 0.25*Z4)` | +| u_integrated | water ∧ soil ∧ (crop ∨ veg) | `classify(num/den)` with crop/veg contributing `0.25` each only when valid | +| z_max / u_max | as above | `max` over the valid inputs | +| e_ecological_constraint | Z ∧ U valid | `max(Z, U)` | +| c_conflicts | `P_j` ∧ E valid | `min(P_j, E)` per sector | +| p_max / c_max | at least one input valid | `max` over the valid ones | +| func_zoning | E valid ∧ at least one `C_k` valid | winning sector + 1, else 1 | + +Functional zoning takes `S_k = C_k`. A sector is a candidate when `S_k` and `E` +are valid and the scenario rule holds — V1: `E <= 2 && S >= v1_min_s_e12` or +`E == 3 && S >= v1_min_s_e3`; V2: `E <= 2`, or `E == 3 && S >= v2_min_s_e3`, or +`E == 4 && S >= v2_min_s_e4`; V3: `E <= 4`. With no candidate the pixel is class +1 (regulated); otherwise the candidate with the greatest `S` wins, ties broken by +`tie_priority`, and the class is `sector + 1` (2 irrigation, 3 snow, 4 crop, +5 pasture, 6 tourism). + +Inputs are **not** resampled or reprojected: a node whose inputs do not share one +grid (size, geotransform, CRS) fails with a clear message, and so does the run. + +### Uploading inputs, running, reading results + +```sh +# 1. fill a slot (repeat per input group; see GET /landscape/slots for the codes) +curl -X POST https://dssgis.dwh.kz/landscape/slots/lp-z-biodiversity \ + -F file=@z_biodiversity.tif -F name="Z Biodiversity 2025" \ + -F start_date=2025-01-01 -F end_date=2025-12-31 + +# 2. queue a run (202); the worker executes the graph asynchronously +curl -X POST https://dssgis.dwh.kz/landscape/runs \ + -H 'Content-Type: application/json' -d '{"from_stage": 0}' + +# 3. follow it, then read the layers +curl https://dssgis.dwh.kz/landscape/runs/ +curl https://dssgis.dwh.kz/landscape/results +``` + +`GET /landscape/slots` lists every leaf slot with the dataset currently filling +it (any status), so the UI can show what is still missing. The upload is the +regular raster path: `.tif`/`.tiff` with a TIFF magic-byte check, stored in the +slot's group under the `landscape` category, COG conversion enqueued. + +**Input resolution.** For each input group a node takes, in order: the run's +`overrides[group]`, else whatever this run already produced for that group, else +the freshest `ready` raster of the group (greatest `end_date`, NULLs last, newest +`created_at` first). A missing input fails the node and the run. + +**Incremental recompute.** Each node is fingerprinted over its code, its resolved +inputs and the params that actually affect it. Without `from_stage`, a node whose +fingerprint matches an earlier `succeeded` node — with all of its output datasets +still present and `ready` — is marked `skipped` and its outputs reused. With +`from_stage: N`, every node of stage `N` and above is recomputed regardless; +`from_stage: 0` recomputes everything. `overrides` pins a specific dataset +(existing, `raster`, `ready`) for a group instead of resolving the freshest one. +Run params: `window_size` (256..8192, default 2048), `tie_priority` (a +permutation of 1..5, default `[1,2,3,4,5]`), `v1_min_s_e12`/`v1_min_s_e3` +(defaults 3/4) and `v2_min_s_e3`/`v2_min_s_e4` (defaults 3/5). + +**Results.** Every output is a normal dataset in its group, named +` `, `automated = true`, with +`meta = {lp_run_id, lp_node, inputs}` and the min/max period of its inputs. +`GET /landscape/results` returns one entry per output group with its freshest +`ready` dataset, a `cog_url` (`/datasets/{id}.cog`) and, when the group carries +one, an `sld_url` (`/dataset-groups/{id}/sld`). A node failure fails its stage; +later stages are not started and the run ends `failed` with the message. Jobs are +nacked without requeue, so there is no automatic retry — start a new run. + +### Build requirements + +The raster math uses [godal](https://github.com/airbusgeo/godal), a **cgo** +binding to GDAL, so the binary must be built with `CGO_ENABLED=1` against GDAL +headers and libraries: + +```sh +brew install gdal # macOS; Linux: libgdal-dev / gdal-dev +go build ./... # cgo is on by default for a native build +``` + +`build/package/Dockerfile` already handles the image: the builder stage adds +`gdal-dev gcc musl-dev pkgconfig` and builds with `CGO_ENABLED=1`, and the +runtime stage installs `gdal` (the shared library godal needs) plus `gdal-tools` +(`gdal_translate`/`gdalinfo` for the COG conversion). + ## Getting started ```sh @@ -172,6 +294,12 @@ server runs it is served at `/openapi.yaml`, with an interactive **Redoc** UI at | GET | `/dataset-groups/{id}/latest` | freshest dataset of the group (`?status=`, `?file_type=`) | | POST | `/dataset-groups/{id}/sld` | upload/replace the group's shared SLD | | GET | `/dataset-groups/{id}/sld` | download the group's shared SLD | +| GET | `/landscape/slots` | ЛП input slots + the dataset filling each | +| POST | `/landscape/slots/{code}` | upload a GeoTIFF into a slot (multipart: `file`, `name?`, `start_date?`, `end_date?`) | +| GET | `/landscape/runs` | paginated runs, newest first (`?page=`, `?page_size=`) | +| POST | `/landscape/runs` | queue a run (`from_stage?`, `overrides?`, `params?`); 202 | +| GET | `/landscape/runs/{id}` | run with per-node status, inputs, outputs, timings | +| GET | `/landscape/results` | per output group: latest layer, `cog_url`, `sld_url` | Example upload: diff --git a/api/openapi.yaml b/api/openapi.yaml index 23c9b1a..eb9d563 100644 --- a/api/openapi.yaml +++ b/api/openapi.yaml @@ -35,6 +35,9 @@ tags: description: Geo dataset upload, processing, and retrieval - name: Dataset Groups description: Groups of same-named datasets versioned by period + - name: Landscape + description: >- + Landscape Program (ЛП): raster inputs, graph runs, and computed results paths: /healthz: @@ -750,6 +753,141 @@ paths: "400": { $ref: "#/components/responses/BadRequest" } "404": { $ref: "#/components/responses/NotFound" } + /landscape/slots: + get: + tags: [Landscape] + summary: List the Landscape Program input slots + description: | + Every leaf input group of the computation graph, in graph order, with + the dataset currently filling it (any status) or `null` when nothing + has been uploaded yet. + responses: + "200": + description: The input slots + content: + application/json: + schema: + type: array + items: { $ref: "#/components/schemas/LandscapeSlot" } + + /landscape/slots/{code}: + parameters: + - name: code + in: path + required: true + description: Slot (dataset group) code, e.g. `lp-z-biodiversity`. + schema: + type: string + post: + tags: [Landscape] + summary: Upload a raster into an input slot + description: | + Multipart upload of a GeoTIFF (`.tif`/`.tiff`, TIFF magic bytes checked) + into the slot's dataset group under the `landscape` category. It goes + through the regular raster upload path, so the COG conversion is + enqueued as usual. An unknown slot code yields 404. + requestBody: + required: true + content: + multipart/form-data: + schema: + type: object + required: [file] + properties: + file: + type: string + format: binary + description: The GeoTIFF to store in the slot. + name: + type: string + description: Dataset name; defaults to the uploaded file name. + start_date: + type: string + format: date + description: Start of the period the raster describes. + end_date: + type: string + format: date + description: >- + End of the period (inclusive); drives the "freshest in + group" rule used to resolve run inputs. + responses: + "201": + description: The created dataset + content: + application/json: + schema: { $ref: "#/components/schemas/Dataset" } + "400": { $ref: "#/components/responses/BadRequest" } + "404": { $ref: "#/components/responses/NotFound" } + "422": { $ref: "#/components/responses/ValidationError" } + + /landscape/runs: + get: + tags: [Landscape] + summary: List Landscape Program runs (paginated, newest first) + parameters: + - $ref: "#/components/parameters/PageParam" + - $ref: "#/components/parameters/PageSizeParam" + responses: + "200": + description: A page of runs + content: + application/json: + schema: { $ref: "#/components/schemas/LandscapeRunPage" } + "400": { $ref: "#/components/responses/BadRequest" } + post: + tags: [Landscape] + summary: Queue a Landscape Program run + description: | + Validates the params and overrides, persists the run with one pending + node per graph node, and publishes the worker job. Returns 202: the + graph is executed asynchronously, poll `GET /landscape/runs/{id}`. + requestBody: + required: false + content: + application/json: + schema: { $ref: "#/components/schemas/LandscapeRunInput" } + responses: + "202": + description: Run queued + content: + application/json: + schema: { $ref: "#/components/schemas/LandscapeRun" } + "400": { $ref: "#/components/responses/BadRequest" } + "422": { $ref: "#/components/responses/ValidationError" } + + /landscape/runs/{id}: + parameters: + - $ref: "#/components/parameters/IdParam" + get: + tags: [Landscape] + summary: Get a Landscape Program run with its nodes + responses: + "200": + description: The run + content: + application/json: + schema: { $ref: "#/components/schemas/LandscapeRun" } + "400": { $ref: "#/components/responses/BadRequest" } + "404": { $ref: "#/components/responses/NotFound" } + + /landscape/results: + get: + tags: [Landscape] + summary: List the Landscape Program result layers + description: | + One entry per output group of the graph with its freshest `ready` + raster, the COG URL to render it, and the group's SLD URL when one has + been uploaded. `latest` is `null` while nothing has been computed. + responses: + "200": + description: The result layers + content: + application/json: + schema: + type: array + items: { $ref: "#/components/schemas/LandscapeResult" } + components: parameters: IdParam: @@ -1309,3 +1447,239 @@ components: type: integer total_pages: type: integer + + LandscapeSlot: + type: object + description: A leaf input group of the Landscape Program graph. + required: [code, title, group_id, latest] + properties: + code: + type: string + description: Dataset group code, e.g. `lp-z-biodiversity`. + title: + type: string + group_id: + type: string + format: uuid + latest: + oneOf: + - $ref: "#/components/schemas/DatasetSummary" + - type: "null" + description: The freshest dataset of the slot's group, any status. + + LandscapeParams: + type: object + description: | + Run-time knobs of the graph. Every field is optional; omitted fields + take the defaults shown below. + properties: + window_size: + type: integer + minimum: 256 + maximum: 8192 + default: 2048 + description: Side of the square window the executor tiles rasters into. + tie_priority: + type: array + description: >- + Sector preference when several sectors tie in functional zoning; a + permutation of 1..5 (1 irrigation, 2 snow, 3 crop, 4 pasture, + 5 tourism). + items: + type: integer + minimum: 1 + maximum: 5 + minItems: 5 + maxItems: 5 + default: [1, 2, 3, 4, 5] + v1_min_s_e12: + type: integer + minimum: 1 + maximum: 5 + default: 3 + description: V1 minimum suitability where E <= 2. + v1_min_s_e3: + type: integer + minimum: 1 + maximum: 5 + default: 4 + description: V1 minimum suitability where E == 3. + v2_min_s_e3: + type: integer + minimum: 1 + maximum: 5 + default: 3 + description: V2 minimum suitability where E == 3. + v2_min_s_e4: + type: integer + minimum: 1 + maximum: 5 + default: 5 + description: V2 minimum suitability where E == 4. + + LandscapeRunInput: + type: object + description: Options of a new Landscape Program run. All fields optional. + properties: + from_stage: + type: [integer, "null"] + minimum: 0 + maximum: 3 + description: >- + Force recomputation from this stage on. Omitted (or null) reuses + every node whose inputs and params are unchanged; `0` recomputes + everything. + overrides: + type: object + description: >- + Pins a specific dataset for a group code instead of resolving the + freshest ready raster. The dataset must exist, be a raster, and be + `ready`. + additionalProperties: + type: string + format: uuid + params: + $ref: "#/components/schemas/LandscapeParams" + examples: + - from_stage: 0 + overrides: { "lp-z-lcr": "3f7c1f5e-9a1a-4a54-8b2e-2d0f5c9d1a11" } + params: { window_size: 1024 } + + LandscapeDatasetRef: + type: object + description: A dataset bound to one group slot of a node. + required: [group, dataset_id] + properties: + group: + type: string + description: Dataset group code. + dataset_id: + type: string + format: uuid + + LandscapeRunNode: + type: object + description: Execution state of one graph node within a run. + required: [id, run_id, node_code, stage, status, fingerprint] + properties: + id: + type: string + format: uuid + run_id: + type: string + format: uuid + node_code: + type: string + description: "Node code `XY`: X = stage, Y = function within the stage." + stage: + type: integer + status: + type: string + enum: [pending, running, skipped, succeeded, failed] + description: >- + `skipped` means a previous run's identical result was reused. + fingerprint: + type: string + description: >- + sha256 over the node code, its resolved inputs and the + node-relevant params; equal fingerprints allow reuse. + inputs: + type: [array, "null"] + items: + $ref: "#/components/schemas/LandscapeDatasetRef" + outputs: + type: [array, "null"] + items: + $ref: "#/components/schemas/LandscapeDatasetRef" + error: + type: [string, "null"] + started_at: + type: [string, "null"] + format: date-time + finished_at: + type: [string, "null"] + format: date-time + + LandscapeRun: + type: object + description: One execution of the Landscape Program graph. + required: [id, status, params, overrides, created_at] + properties: + id: + type: string + format: uuid + status: + type: string + enum: [queued, running, succeeded, failed] + from_stage: + type: [integer, "null"] + params: + $ref: "#/components/schemas/LandscapeParams" + overrides: + type: object + additionalProperties: + type: string + format: uuid + error: + type: [string, "null"] + description: Failure reason when status is `failed`. + created_at: + type: string + format: date-time + started_at: + type: [string, "null"] + format: date-time + finished_at: + type: [string, "null"] + format: date-time + nodes: + type: [array, "null"] + items: + $ref: "#/components/schemas/LandscapeRunNode" + + LandscapeRunPage: + type: object + required: [data, page, page_size, total, total_pages] + properties: + data: + type: array + items: + $ref: "#/components/schemas/LandscapeRun" + page: + type: integer + page_size: + type: integer + total: + type: integer + total_pages: + type: integer + + LandscapeResult: + type: object + description: One computed output group of the graph and its latest layer. + required: [group, title, node, group_id, latest, cog_url, sld_url] + properties: + group: + type: string + description: Dataset group code, e.g. `lp-v1-conservative`. + title: + type: string + node: + type: string + description: Code of the graph node producing this group. + group_id: + type: string + format: uuid + latest: + oneOf: + - $ref: "#/components/schemas/DatasetSummary" + - type: "null" + description: The freshest `ready` raster of the group, if any. + cog_url: + type: [string, "null"] + description: "`/datasets/{id}.cog` for `latest`, or null." + sld_url: + type: [string, "null"] + description: >- + `/dataset-groups/{group_id}/sld` when the group has an SLD, else + null. diff --git a/build/package/Dockerfile b/build/package/Dockerfile index 7de75fe..56cec4b 100644 --- a/build/package/Dockerfile +++ b/build/package/Dockerfile @@ -1,8 +1,9 @@ # Build context is the repo root: docker build -f build/package/Dockerfile . FROM golang:1.26-alpine AS builder -# godal (GDAL bindings) needs cgo, a C toolchain and the GDAL headers. -RUN apk add --no-cache gdal-dev gcc musl-dev pkgconfig +# godal (GDAL bindings) needs cgo, a C/C++ toolchain and the GDAL headers. +# g++ is required: parts of the binding compile as C++ against GDAL's headers. +RUN apk add --no-cache gdal-dev gcc g++ musl-dev pkgconfig WORKDIR /src @@ -12,7 +13,9 @@ RUN go mod download COPY . . RUN CGO_ENABLED=1 GOOS=linux go build -trimpath -o /out/gis ./cmd/gis -FROM alpine:3.22 +# Must match the builder's Alpine release (golang:1.26-alpine is Alpine 3.24): +# an older base ships a different libgdal soname and the binary will not start. +FROM alpine:3.24 # gdal: shared library for godal (raster math in the worker). # gdal-tools: gdal_translate / gdalinfo (COG conversion) and ogr2ogr. diff --git a/internal/app/app.go b/internal/app/app.go index 281eff2..1965f04 100644 --- a/internal/app/app.go +++ b/internal/app/app.go @@ -12,6 +12,7 @@ import ( "gis/api" "gis/internal/config" "gis/internal/domain" + "gis/internal/landscape" "gis/internal/messaging/rabbitmq" "gis/internal/parser" "gis/internal/platform/logger" @@ -39,6 +40,7 @@ type App struct { categories *service.CategoryService datasets *service.DatasetService groups *service.DatasetGroupService + landscape *service.LandscapeService eventRepo *postgres.EventRepository } @@ -72,8 +74,13 @@ func New(ctx context.Context) (*App, error) { groupRepo := postgres.NewDatasetGroupRepository(pool) eventRepo := postgres.NewEventRepository(pool) + lpRepo := postgres.NewLandscapeRepository(pool) + publisher := rabbitmq.NewPublisher(rabbit) jobPublisher := rabbitmq.NewDatasetJobPublisher(publisher) + lpJobs := rabbitmq.NewLandscapeJobPublisher(publisher) + + datasetsSvc := service.NewDatasetService(datasetRepo, store, categoryRepo, groupRepo, jobPublisher, parser.Columns, parser.Rows, raster.NewGDALConverter()) return &App{ Cfg: cfg, @@ -83,9 +90,13 @@ func New(ctx context.Context) (*App, error) { rabbit: rabbit, publisher: publisher, categories: service.NewCategoryService(categoryRepo), - datasets: service.NewDatasetService(datasetRepo, store, categoryRepo, groupRepo, jobPublisher, parser.Columns, parser.Rows, raster.NewGDALConverter()), + datasets: datasetsSvc, groups: service.NewDatasetGroupService(groupRepo, datasetRepo, store), - eventRepo: eventRepo, + landscape: service.NewLandscapeService( + lpRepo, datasetRepo, groupRepo, categoryRepo, datasetsSvc, store, + lpJobs, raster.NewGodalIO(), landscape.Compute, log, + ), + eventRepo: eventRepo, }, nil } @@ -108,6 +119,7 @@ func (a *App) Handler() stdhttp.Handler { Categories: transporthttp.NewCategoryHandler(a.categories, validate), Datasets: transporthttp.NewDatasetHandler(a.datasets, validate), DatasetGroups: transporthttp.NewDatasetGroupHandler(a.groups, validate), + Landscape: transporthttp.NewLandscapeHandler(a.landscape, validate), OpenAPISpec: api.Spec, }) } @@ -124,10 +136,20 @@ func (a *App) Consumers() []*rabbitmq.Consumer { a.PropertiesConsumer(), a.ExtractConsumer(), a.ConvertConsumer(), + a.LandscapeConsumer(), a.ExampleConsumer(), } } +// LandscapeConsumer builds the Landscape Program run consumer. +func (a *App) LandscapeConsumer() *rabbitmq.Consumer { + handler := rabbitmq.NewLandscapeRunHandler(a.landscape, a.Log) + return rabbitmq.NewConsumer( + a.rabbit, rabbitmq.LandscapeRunQueue, rabbitmq.LandscapeRunRoutingKey, + "gis-landscape-runner", handler, a.Log, + ) +} + // PropertiesConsumer builds the plain-vector properties-extraction consumer. func (a *App) PropertiesConsumer() *rabbitmq.Consumer { handler := rabbitmq.NewPropertiesHandler(a.datasets, a.Log) diff --git a/internal/cli/worker.go b/internal/cli/worker.go index 7fde693..9fcac98 100644 --- a/internal/cli/worker.go +++ b/internal/cli/worker.go @@ -15,7 +15,7 @@ var publishExample bool var workerCmd = &cobra.Command{ Use: "worker", - Short: "Run the RabbitMQ consumers (dataset parsing + example)", + Short: "Run the RabbitMQ consumers (dataset processing, landscape runs + example)", RunE: func(cmd *cobra.Command, args []string) error { ctx, cancel := signalContext() defer cancel() diff --git a/internal/transport/http/landscape_handler.go b/internal/transport/http/landscape_handler.go new file mode 100644 index 0000000..180df9f --- /dev/null +++ b/internal/transport/http/landscape_handler.go @@ -0,0 +1,170 @@ +package http + +import ( + "net/http" + + "gis/internal/landscape" + "gis/internal/service" + "gis/pkg/httputil" + + "github.com/go-chi/chi/v5" + "github.com/go-playground/validator/v10" + "github.com/google/uuid" +) + +// LandscapeHandler serves the /landscape routes. +type LandscapeHandler struct { + svc *service.LandscapeService + validate *validator.Validate +} + +// NewLandscapeHandler returns a LandscapeHandler. +func NewLandscapeHandler(svc *service.LandscapeService, validate *validator.Validate) *LandscapeHandler { + return &LandscapeHandler{svc: svc, validate: validate} +} + +// Register mounts the Landscape Program routes on r. +func (h *LandscapeHandler) Register(r chi.Router) { + r.Get("/slots", h.slots) + r.Post("/slots/{code}", h.uploadSlot) + r.Get("/runs", h.listRuns) + r.Post("/runs", h.createRun) + r.Get("/runs/{id}", h.getRun) + r.Get("/results", h.results) +} + +// runRequest is the body of POST /landscape/runs. Overrides are decoded as +// strings so a malformed id is a 422 instead of a 400 on the whole body. +type runRequest struct { + FromStage *int `json:"from_stage" validate:"omitempty,gte=0,lte=3"` + Overrides map[string]string `json:"overrides" validate:"omitempty,dive,uuid"` + Params *landscape.Params `json:"params"` +} + +// slots lists every leaf input slot with the dataset currently filling it. +func (h *LandscapeHandler) slots(w http.ResponseWriter, r *http.Request) { + slots, err := h.svc.Slots(r.Context()) + if err != nil { + respondDomainError(w, err) + return + } + httputil.WriteJSON(w, http.StatusOK, slots) +} + +// uploadSlot stores a raster in the slot's dataset group under the landscape +// category. An unknown slot code yields 404. +func (h *LandscapeHandler) uploadSlot(w http.ResponseWriter, r *http.Request) { + code := chi.URLParam(r, "code") + + if err := r.ParseMultipartForm(maxUploadBytes); err != nil { + httputil.WriteError(w, http.StatusBadRequest, "request must be multipart/form-data") + return + } + + file, header, err := r.FormFile("file") + if err != nil { + httputil.WriteError(w, http.StatusBadRequest, "file is required") + return + } + defer file.Close() + + contentType := header.Header.Get("Content-Type") + if contentType == "" { + contentType = "image/tiff" + } + + dataset, err := h.svc.UploadToSlot(r.Context(), code, service.SlotUploadInput{ + Filename: header.Filename, + ContentType: contentType, + Size: header.Size, + Reader: file, + Name: r.FormValue("name"), + StartDate: optionalFormValue(r, "start_date"), + EndDate: optionalFormValue(r, "end_date"), + }) + if err != nil { + respondDomainError(w, err) + return + } + httputil.WriteJSON(w, http.StatusCreated, dataset) +} + +// createRun queues a Landscape Program run and returns it with 202 Accepted; +// the worker executes it asynchronously. +func (h *LandscapeHandler) createRun(w http.ResponseWriter, r *http.Request) { + req, err := httputil.DecodeJSON[runRequest](w, r) + if err != nil { + httputil.WriteError(w, http.StatusBadRequest, "invalid request body") + return + } + if err := h.validate.Struct(req); err != nil { + httputil.WriteValidationErrors(w, err) + return + } + + overrides := make(map[string]uuid.UUID, len(req.Overrides)) + for group, raw := range req.Overrides { + id, err := uuid.Parse(raw) + if err != nil { + httputil.WriteError(w, http.StatusUnprocessableEntity, "overrides["+group+"] must be a valid UUID") + return + } + overrides[group] = id + } + + in := service.RunInput{FromStage: req.FromStage, Overrides: overrides} + if req.Params != nil { + in.Params = *req.Params + } + + run, err := h.svc.CreateRun(r.Context(), in) + if err != nil { + respondDomainError(w, err) + return + } + httputil.WriteJSON(w, http.StatusAccepted, run) +} + +// listRuns returns a page of runs, newest first. +func (h *LandscapeHandler) listRuns(w http.ResponseWriter, r *http.Request) { + page, ok := parsePositiveIntQuery(w, r, "page", 1) + if !ok { + return + } + pageSize, ok := parsePositiveIntQuery(w, r, "page_size", service.DefaultPageSize) + if !ok { + return + } + + result, err := h.svc.ListRuns(r.Context(), page, pageSize) + if err != nil { + respondDomainError(w, err) + return + } + httputil.WriteJSON(w, http.StatusOK, newPaginated(result.Items, result.Page, result.PageSize, result.Total)) +} + +// getRun returns one run with its per-node execution state. +func (h *LandscapeHandler) getRun(w http.ResponseWriter, r *http.Request) { + id, ok := parseUUIDParam(w, r, "id") + if !ok { + return + } + run, err := h.svc.GetRun(r.Context(), id) + if err != nil { + respondDomainError(w, err) + return + } + httputil.WriteJSON(w, http.StatusOK, run) +} + +// results lists every output group with its freshest ready dataset and the URLs +// to render it. +func (h *LandscapeHandler) results(w http.ResponseWriter, r *http.Request) { + results, err := h.svc.Results(r.Context()) + if err != nil { + respondDomainError(w, err) + return + } + httputil.WriteJSON(w, http.StatusOK, results) +} diff --git a/internal/transport/http/landscape_handler_test.go b/internal/transport/http/landscape_handler_test.go new file mode 100644 index 0000000..6ed979c --- /dev/null +++ b/internal/transport/http/landscape_handler_test.go @@ -0,0 +1,120 @@ +package http + +import ( + "net/http" + "net/http/httptest" + "strings" + "testing" + + "github.com/go-chi/chi/v5" + "github.com/go-playground/validator/v10" +) + +// newLandscapeTestHandler builds a handler with a nil service: every case below +// is rejected while parsing or validating the request, before the service is +// reached. +func newLandscapeTestHandler() *LandscapeHandler { + return NewLandscapeHandler(nil, validator.New(validator.WithRequiredStructEnabled())) +} + +func postRun(t *testing.T, body string) *httptest.ResponseRecorder { + t.Helper() + req := httptest.NewRequest(http.MethodPost, "/landscape/runs", strings.NewReader(body)) + req.Header.Set("Content-Type", "application/json") + rec := httptest.NewRecorder() + newLandscapeTestHandler().createRun(rec, req) + return rec +} + +func TestCreateRunInvalidJSON(t *testing.T) { + rec := postRun(t, `{"from_stage":`) + if rec.Code != http.StatusBadRequest { + t.Fatalf("status = %d, want %d (body %s)", rec.Code, http.StatusBadRequest, rec.Body.String()) + } +} + +func TestCreateRunUnknownField(t *testing.T) { + // DecodeJSON rejects unknown fields, so a typo is a 400 rather than a + // silently ignored option. + rec := postRun(t, `{"from_stages":1}`) + if rec.Code != http.StatusBadRequest { + t.Fatalf("status = %d, want %d (body %s)", rec.Code, http.StatusBadRequest, rec.Body.String()) + } +} + +func TestCreateRunFromStageOutOfRange(t *testing.T) { + rec := postRun(t, `{"from_stage":7}`) + if rec.Code != http.StatusUnprocessableEntity { + t.Fatalf("status = %d, want %d (body %s)", rec.Code, http.StatusUnprocessableEntity, rec.Body.String()) + } + if !strings.Contains(rec.Body.String(), "FromStage") { + t.Fatalf("body %s does not name the offending field", rec.Body.String()) + } +} + +func TestCreateRunOverrideNotUUID(t *testing.T) { + rec := postRun(t, `{"overrides":{"lp-z-lcr":"not-a-uuid"}}`) + if rec.Code != http.StatusUnprocessableEntity { + t.Fatalf("status = %d, want %d (body %s)", rec.Code, http.StatusUnprocessableEntity, rec.Body.String()) + } +} + +func TestUploadSlotRequiresMultipart(t *testing.T) { + req := httptest.NewRequest(http.MethodPost, "/landscape/slots/lp-z-lcr", strings.NewReader(`{}`)) + req.Header.Set("Content-Type", "application/json") + rec := httptest.NewRecorder() + newLandscapeTestHandler().uploadSlot(rec, req) + + if rec.Code != http.StatusBadRequest { + t.Fatalf("status = %d, want %d (body %s)", rec.Code, http.StatusBadRequest, rec.Body.String()) + } +} + +func TestListRunsInvalidPage(t *testing.T) { + req := httptest.NewRequest(http.MethodGet, "/landscape/runs?page=0", nil) + rec := httptest.NewRecorder() + newLandscapeTestHandler().listRuns(rec, req) + + if rec.Code != http.StatusBadRequest { + t.Fatalf("status = %d, want %d (body %s)", rec.Code, http.StatusBadRequest, rec.Body.String()) + } +} + +func TestGetRunInvalidID(t *testing.T) { + req := httptest.NewRequest(http.MethodGet, "/landscape/runs/nope", nil) + rec := httptest.NewRecorder() + newLandscapeTestHandler().getRun(rec, req) + + if rec.Code != http.StatusBadRequest { + t.Fatalf("status = %d, want %d (body %s)", rec.Code, http.StatusBadRequest, rec.Body.String()) + } +} + +// TestRegisterMountsRoutes checks the route table itself: every path reaches a +// handler (405/404 would mean it is not mounted). Each request is one that is +// rejected before the service is used. +func TestRegisterMountsRoutes(t *testing.T) { + r := chi.NewRouter() + r.Route("/landscape", newLandscapeTestHandler().Register) + + cases := []struct { + method, path, body, contentType string + want int + }{ + {http.MethodPost, "/landscape/runs", `{`, "application/json", http.StatusBadRequest}, + {http.MethodPost, "/landscape/slots/lp-z-lcr", `{}`, "application/json", http.StatusBadRequest}, + {http.MethodGet, "/landscape/runs?page=0", "", "", http.StatusBadRequest}, + {http.MethodGet, "/landscape/runs/nope", "", "", http.StatusBadRequest}, + } + for _, c := range cases { + req := httptest.NewRequest(c.method, c.path, strings.NewReader(c.body)) + if c.contentType != "" { + req.Header.Set("Content-Type", c.contentType) + } + rec := httptest.NewRecorder() + r.ServeHTTP(rec, req) + if rec.Code != c.want { + t.Errorf("%s %s: status = %d, want %d (body %s)", c.method, c.path, rec.Code, c.want, rec.Body.String()) + } + } +} diff --git a/internal/transport/http/router.go b/internal/transport/http/router.go index 713a590..af714d7 100644 --- a/internal/transport/http/router.go +++ b/internal/transport/http/router.go @@ -16,7 +16,9 @@ type RouterDeps struct { Datasets *DatasetHandler // DatasetGroups serves the /dataset-groups routes. DatasetGroups *DatasetGroupHandler - OpenAPISpec []byte + // Landscape serves the /landscape routes. + Landscape *LandscapeHandler + OpenAPISpec []byte } // NewRouter builds the application's HTTP handler with middleware and routes. @@ -40,6 +42,7 @@ func NewRouter(deps RouterDeps) http.Handler { r.Route("/categories", deps.Categories.Register) r.Route("/datasets", deps.Datasets.Register) r.Route("/dataset-groups", deps.DatasetGroups.Register) + r.Route("/landscape", deps.Landscape.Register) return r }