fix(landscape): guard redelivery, detach terminal writes, bound workers
F2: Execute had no status guard, so a redelivered or stale message reran a finished run. Runs that are neither queued nor running are logged and acked without work; "running" still executes so a worker that crashed mid-run recovers. F4: failRun, finishNodeFailed and MarkRunFailed wrote the terminal status on the caller's context, which is exactly the context that is already cancelled when the run failed because of a shutdown or a sibling node — the write was refused and the run stayed stuck in "running". They now write on a context.WithoutCancel copy with a 10s timeout; the consumer's best-effort MarkRunFailed does the same. F5: peak memory scaled with the host's core count. LP_MAX_WORKERS (default 8) is now the total window-worker budget of a run and LP_MAX_NODES (default 2) the node fan-out; each node gets max(1, workers/nodes) through the new landscape.ComputeWithWorkers. Compute and service.ComputeFunc keep their signatures: Compute delegates with MaxWorkersPerNode, and a legacy ComputeFunc handed to NewLandscapeService is wrapped. F6: the stage loop used a plain errgroup, so a failing node let its siblings run to completion. It now uses errgroup.WithContext and passes the group context into runNode, cancelling siblings mid-compute. F7: wrap every float32 product in an explicit conversion so the compiler cannot fuse it into an FMA, keeping the kernels bit-identical to numpy's per-op rounding. F8: README's builder package list now matches the Dockerfile (adds g++), and the Landscape section documents the memory model and the two new knobs. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
@@ -12,10 +12,11 @@ import (
|
||||
// Config holds all configuration for the application. Values are read from
|
||||
// environment variables; a local .env file (if present) is loaded first.
|
||||
type Config struct {
|
||||
HTTP HTTPConfig
|
||||
DB DBConfig
|
||||
S3 S3Config
|
||||
RabbitMQ RabbitMQConfig
|
||||
HTTP HTTPConfig
|
||||
DB DBConfig
|
||||
S3 S3Config
|
||||
RabbitMQ RabbitMQConfig
|
||||
Landscape LandscapeConfig
|
||||
}
|
||||
|
||||
// HTTPConfig configures the HTTP server.
|
||||
@@ -58,6 +59,19 @@ type RabbitMQConfig struct {
|
||||
Queue string `env:"RABBITMQ_QUEUE" envDefault:"gis.events.example"`
|
||||
}
|
||||
|
||||
// LandscapeConfig bounds the Landscape Program's parallelism, and with it the
|
||||
// worker's peak memory: a run holds up to MaxConcurrentNodes nodes in flight,
|
||||
// each running a pool of MaxWorkers/MaxConcurrentNodes goroutines that own one
|
||||
// window buffer per input and output. The defaults (8 total workers spread
|
||||
// over 2 nodes) keep that product bounded regardless of how many cores the
|
||||
// host has.
|
||||
type LandscapeConfig struct {
|
||||
// MaxWorkers is the total window-worker budget shared by all nodes of a run.
|
||||
MaxWorkers int `env:"LP_MAX_WORKERS" envDefault:"8"`
|
||||
// MaxConcurrentNodes is how many graph nodes of one stage run at a time.
|
||||
MaxConcurrentNodes int `env:"LP_MAX_NODES" envDefault:"2"`
|
||||
}
|
||||
|
||||
// Load reads configuration from the environment, loading an optional .env file
|
||||
// from the current working directory first.
|
||||
func Load() (*Config, error) {
|
||||
|
||||
Reference in New Issue
Block a user