GAN-Based Spatial Data Generation

Analytic generators place features by rule — a Poisson intensity, a copula, a diffusion kernel — and that transparency is exactly their ceiling: they can only reproduce the structure you can write down. Deep generative models learn the structure instead, which lets them capture spatial texture no closed-form process expresses, and costs you the guarantees the analytic models gave away for free. This page is part of Spatial Distribution & Pattern Generation: where point process simulation models emit patterns from a prescribed intensity function, a generative adversarial network learns the intensity — and the correlations, the clustering, the local morphology — from real data and samples new realizations that share it. The trade is precise and it defines this page: you gain expressive power over irregular, high-order spatial structure, and you take on training instability, mode collapse, reproducibility fragility, and a privacy surface that only formal methods can close. When the analytic route suffices, take it; the comparison in Poisson process vs. GAN-based point generation frames that decision directly.

Problem Framing: Learned Power, Learned Failure Modes

The reason to reach for a deep generative model is structure that no analytic process captures: the irregular street morphology of an informal settlement, the coupled clustering of amenities and residences, the way real point patterns are neither uniform nor cleanly Poisson. A GAN or diffusion model learns that structure from examples. But the same expressiveness that captures it also opens four failure modes that analytic generators simply do not have, and every one of them is silent until you test for it.

• Mode collapse. The generator discovers a narrow slice of outputs that reliably fools the discriminator and stops exploring the rest. The samples look individually realistic and the training loss looks healthy, yet whole regions of the real distribution — a land-use class, a density regime, an entire city’s morphology — are absent. A model that emits only dense urban blocks when the training set spanned rural to urban has collapsed, and marginal plots will not reveal it.
• Training non-convergence. The adversarial game oscillates rather than settling; discriminator and generator losses swing without the sample quality improving, and there is no loss value that reliably signals “done.” Stopping at the wrong step ships a half-trained model.
• Reproducibility fragility. Two training runs with the same data and hyperparameters diverge because GPU kernel nondeterminism, unpinned seeds, and data-loader shuffling all inject uncontrolled entropy. Without deliberate control, “regenerate the dataset” produces a different dataset, which breaks every audit trail.
• Memorization and privacy leakage. A high-capacity model can memorize training examples and reproduce a real trajectory or parcel near-verbatim in its output. Visual realism gives no privacy assurance whatsoever; only a formal mechanism bounds what an adversary can infer, which is why differential privacy is not optional for sensitive source data.

This page treats those four as first-class engineering problems with gates, not as tuning folklore. The architecture above exists to make each one observable and boundable before a single synthetic sample is released.

Prerequisites & Toolchain

Deep spatial generation adds a training stack on top of the standard geospatial one. Pin the deep-learning and privacy libraries to a major, and the geospatial libraries as the rest of the site does:

Copy
python >= 3.10
numpy == 1.26.*
torch == 2.*           # generator/discriminator, autograd, CUDA kernels
opacus == 1.*          # DP-SGD: per-sample grad clipping + accountant
geopandas == 0.14.*    # real-data ingest, attribute join
pyproj == 3.*          # normalize coordinates to a metric CRS before training
scipy == 1.11.*        # evaluation statistics, distances

Two environment decisions dominate whether the model is trainable and auditable. First, normalize coordinates into a bounded metric space before training, never feed raw longitude/latitude. A network fed degrees learns a distorted metric — a degree of longitude is a different distance at every latitude — and gradients scale inconsistently across the domain. Reproject to a metric CRS, then standardize coordinates to a fixed range so the generator’s output space is isotropic. Second, decide reproducibility policy up front, because retrofitting it is painful: pin every seed (Python, NumPy, Torch CPU and CUDA), enable deterministic algorithms, and disable non-deterministic cuDNN kernels, accepting the throughput cost. A generator you cannot reproduce is a generator you cannot audit.

Core Concept: Conditioning, Spatial Embeddings, and Decoupling

A GAN pits a generator that maps latent noise to synthetic samples against a discriminator that learns to separate real from synthetic; their competition drives the generator toward the real distribution. For spatial data, three design choices turn that generic recipe into something that produces geographically coherent output.

Conditioning makes the generator a function of context, not just noise. A conditional GAN takes a label — land-use class, region, time-of-day — alongside the latent vector, so you can sample “an industrial district” or “a rural night” on demand rather than drawing blindly from the marginal. Diffusion models reach the same conditional target by a different route, iteratively denoising toward a sample consistent with the condition, and are often more stable to train at the cost of slower sampling; the architectural machinery below applies to either.

Spatial embeddings are how location enters the network without the degree-distortion trap. Rather than feeding a raw coordinate, encode position as a set of continuous features — Fourier features of the normalized coordinate, or a learned embedding indexed by a spatial grid cell — so nearby locations map to nearby representations and the network can express smooth spatial variation. This is the deep-learning analogue of the covariate coupling used in attribute correlation modeling: both make attributes co-vary with location, one by learned embedding, the other by an explicit copula shift.

Decoupling geometry from attribute synthesis is the design decision that most often separates a maintainable pipeline from an unmaintainable one. Training one monolithic network to emit both a feature’s shape and its full attribute vector entangles two very different learning problems and multiplies the collapse surface. It is frequently better to let the generative model produce the hard-to-express geometry and spatial layout, then populate attributes with the explicit, gate-able copula machinery — you get learned spatial texture where you need it and auditable attribute correlations where you can get them for free.

The privacy dimension is inseparable from the architecture. Because the discriminator’s gradient is the channel through which information about individual training records flows into the generator, the place to bound leakage is that gradient. DP-SGD clips each per-sample gradient to a fixed norm and adds calibrated Gaussian noise before the optimizer step, so no single training record can move the weights beyond a bounded amount; a privacy accountant tracks the cumulative $(\varepsilon, \delta)$ spend across steps. The budget composes as roughly $\varepsilon_\text{total} \approx \sqrt{T}\,\varepsilon_\text{step}$ over $T$ steps, so training longer costs privacy — the same budget arithmetic the privacy-preserving generation frameworks stage governs across an entire pipeline. The drill-down on stabilizing GAN training with DP-SGD for coordinates covers how the clipping norm and noise multiplier interact with coordinate scale to keep a DP-constrained model trainable.

Step-by-Step Implementation

The loop runs as four auditable stages. Blocks use torch and assume coordinates are already normalized to a metric, standardized space.

Step 1 — Fully seed the run and enable deterministic execution. Reproducibility is a prerequisite, not a postscript; set it before any layer is constructed.

pythonCopy
import os, random
import numpy as np
import torch

def make_reproducible(seed: int) -> None:
    os.environ["PYTHONHASHSEED"] = str(seed)
    random.seed(seed)
    np.random.seed(seed)
    torch.manual_seed(seed)
    torch.cuda.manual_seed_all(seed)
    torch.use_deterministic_algorithms(True)   # error on nondeterministic kernels
    torch.backends.cudnn.benchmark = False      # disable autotuner nondeterminism

Step 2 — Define a conditional generator with a spatial embedding. Concatenate latent noise, a Fourier-feature spatial embedding, and the conditioning label as the generator input.

pythonCopy
import torch
from torch import nn

def fourier_features(coords: torch.Tensor, n_freq: int = 8) -> torch.Tensor:
    # coords: (batch, 2) standardized x/y; expand to smooth periodic features.
    freqs = 2.0 ** torch.arange(n_freq, device=coords.device)
    proj = coords.unsqueeze(-1) * freqs            # (batch, 2, n_freq)
    feats = torch.cat([proj.sin(), proj.cos()], dim=-1)
    return feats.flatten(start_dim=1)              # (batch, 2 * 2 * n_freq)

class SpatialGenerator(nn.Module):
    def __init__(self, z_dim: int, n_class: int, out_dim: int, n_freq: int = 8):
        super().__init__()
        in_dim = z_dim + n_class + 4 * n_freq      # noise + label + spatial embed
        self.net = nn.Sequential(
            nn.Linear(in_dim, 256), nn.ReLU(),
            nn.Linear(256, 256), nn.ReLU(),
            nn.Linear(256, out_dim),
        )

    def forward(self, z, label_onehot, coords):
        s = fourier_features(coords)
        return self.net(torch.cat([z, label_onehot, s], dim=1))

Step 3 — Wrap the optimizer in DP-SGD. Opacus attaches per-sample gradient clipping, noise injection, and an accountant to the discriminator — the network that touches real data — leaving the generator to learn only from the protected gradient.

pythonCopy
from opacus import PrivacyEngine

def attach_dp(discriminator, optimizer, data_loader, *, noise_multiplier: float,
              max_grad_norm: float):
    engine = PrivacyEngine()
    d, opt, loader = engine.make_private(
        module=discriminator,
        optimizer=optimizer,
        data_loader=data_loader,
        noise_multiplier=noise_multiplier,   # more noise -> stronger privacy
        max_grad_norm=max_grad_norm,         # per-sample clip norm, coord-scaled
    )
    return engine, d, opt, loader

Step 4 — Train, tracking the privacy budget as a stopping condition. Query the accountant each epoch and halt when the spent budget reaches the ceiling, rather than at a fixed step count.

pythonCopy
def train(engine, gen, disc, opt_g, opt_d, loader, *, epsilon_ceiling: float,
          delta: float, epochs: int, device):
    bce = torch.nn.BCEWithLogitsLoss()
    for _ in range(epochs):
        for real, label in loader:
            real, label = real.to(device), label.to(device)
            # --- discriminator step (DP-protected) ---
            z = torch.randn(real.size(0), gen.z_dim, device=device)
            fake = gen(z, label, real[:, :2]).detach()
            opt_d.zero_grad()
            loss_d = bce(disc(real), torch.ones(real.size(0), 1, device=device)) + \
                     bce(disc(fake), torch.zeros(real.size(0), 1, device=device))
            loss_d.backward(); opt_d.step()
            # --- generator step ---
            opt_g.zero_grad()
            gen_out = gen(z, label, real[:, :2])
            loss_g = bce(disc(gen_out), torch.ones(real.size(0), 1, device=device))
            loss_g.backward(); opt_g.step()
        eps = engine.get_epsilon(delta)          # cumulative spend so far
        if eps >= epsilon_ceiling:
            break                                # budget is the stopping rule

Validation & Testing

A learned generator is only releasable once gates prove it did not collapse, that its output is statistically faithful, and that the privacy budget it claims is the budget it spent. These are the checks that catch the four failure modes.

pythonCopy
import numpy as np
import torch

def gate_mode_coverage(real: np.ndarray, synth: np.ndarray,
                       min_coverage: float = 0.8, k: int = 5) -> None:
    """Fraction of real samples with a synthetic neighbour inside their own
    k-NN radius. Low coverage => the generator ignores parts of the support."""
    from scipy.spatial import cKDTree
    real_tree = cKDTree(real)
    radius = real_tree.query(real, k=k + 1)[0][:, k]   # each real point's k-NN radius
    synth_tree = cKDTree(synth)
    covered = np.array([
        len(synth_tree.query_ball_point(r, rad)) > 0
        for r, rad in zip(real, radius)
    ])
    assert covered.mean() >= min_coverage, f"mode collapse: coverage={covered.mean():.2f}"

def gate_privacy(engine, delta: float, epsilon_ceiling: float) -> None:
    eps = engine.get_epsilon(delta)
    assert eps <= epsilon_ceiling, f"spent epsilon {eps:.2f} exceeds ceiling"

The coverage gate is the load-bearing anti-collapse check: it fails precisely when the synthetic samples cluster into a slice of the real support, which the marginal distributions cannot reveal. The diagnostic detail behind it — precision-recall curves for generative models, nearest-neighbour ratios, and per-class coverage — lives in diagnosing mode collapse in spatial GANs. Pair coverage with a distributional-distance check on second-order spatial statistics from the realism metrics evaluation stage, and gate the accountant’s reported budget so a run that overspent privacy can never be promoted. Wire all three as build-failing checks through the CI/CD integration pipeline.

Performance & Scale Considerations

Deep generation is GPU-bound and its cost profile differs sharply from the analytic generators.

• DP-SGD is a throughput tax. Per-sample gradient clipping forbids the batched gradient reductions that make ordinary training fast, so a private step can be several times slower and needs more memory. Budget for it, and consider vectorized per-sample gradient support rather than a Python loop.
• Determinism costs speed. Deterministic algorithms and a disabled cuDNN autotuner remove kernel-level parallelism gains; the run is slower but reproducible. This is a deliberate trade — never disable determinism to hit a throughput target for a dataset that must be auditable.
• Diffusion trades training stability for sampling cost. A diffusion model is often easier to train to convergence than a GAN but needs many denoising steps per sample, so generating a large synthetic set is expensive. Amortize with batched sampling and, where acceptable, distillation to fewer steps.
• Coordinate scale drives the clip norm. The DP-SGD clipping norm interacts with the magnitude of coordinate gradients; if coordinates are not standardized, the clip either annihilates the signal or lets it through unprotected. Standardize first, then tune the clip norm.
• Evaluation is not free. Mode-coverage and distributional gates over large synthetic sets are themselves compute-heavy; run them on representative stratified subsamples in CI and the full set only on release candidates.

Failure Modes & Troubleshooting

Frequently Asked Questions

Related

• Point Process Simulation Models — the analytic generators a GAN replaces when structure exceeds what a closed-form process expresses.
• Poisson Process vs. GAN-Based Point Generation — the decision framework for when the learned route is worth its costs.
• Privacy-Preserving Generation Frameworks — the budget accounting DP-SGD spends within across a whole pipeline.
• Diagnosing Mode Collapse in Spatial GANs — precision-recall, coverage, and per-class diagnostics behind the anti-collapse gate.
• Stabilizing GAN Training with DP-SGD for Coordinates — tuning the clip norm and noise multiplier against coordinate scale.
• Realism Metrics & Evaluation — the distributional distances that gate spatial fidelity beside coverage.