Configuration#

A project’s settings live in config.yaml in the project folder. They say which meteorology to use, what footprint to make, and how to run HYSPLIT. This page covers the settings most projects need. Variants explains how to run the same receptors under several settings, and Configuration lists every option.

A typical config.yaml#

mets:
  hrrr:
    directory: /data/arl/hrrr
    file_format: "%Y%m%d_%H"
    file_tres: 6h

grid:
  xmin: -114.0
  xmax: -111.0
  ymin: 39.0
  ymax: 42.0
  xres: 0.01
  yres: 0.01

n_hours: -24
numpar: 500

stilt init writes a starter file with comments. If you build a Model in Python instead, the settings you pass are written to config.yaml each time you run it. The written file includes a variants section that lists the variants that run (see Variants), so it reads the same either way.

The settings most people change#

Setting	What it does	Typical value
`mets`	Where the meteorology files are and how they are named. See Meteorology.	one entry
`grid`	The map grid for the footprint: the longitude range (`xmin`/`xmax`), the latitude range (`ymin`/`ymax`), and the cell size in degrees (`xres`/`yres`). Leave it out, or set it to `null`, to keep only the particle trajectories.	0.01° (about 1 km) for a city, 0.1° for a region
`n_hours`	How many hours to follow the particles. Negative values run backward in time, which is what footprints need. Positive values run forward from the receptor (see Plume Background).	`-24` to `-72`
`numpar`	Particles released per simulation. More particles give a smoother footprint and take longer. The default is 200.	`200` to `1000`
`execution`	Where to run. Leave it out to run on your own computer. See Where To Run.	(omit)

Footprint options#

These settings sit next to grid at the top level of config.yaml.

smooth_factor: Scales the Gaussian smoothing applied to each particle’s influence. The default, 1.0, is standard STILT. Smaller values smooth less.
time_integrate: true sums the footprint over time into a single map, which makes smaller files. The default, false, keeps one layer per hour.
transforms: Steps that weight the particles before the footprint is made, mainly for column measurements. See Particle Weighting (Transforms).

Variants#

Every receptor is run once per variant. A variant has a name, a meteorology source, and any settings that differ from the top level of config.yaml. The top-level settings are the defaults.

With no variants section there is one variant per meteorology source, named after it. The typical config above runs each receptor once, as hrrr.

Once you write a variants section, only the variants in it run. Keep an entry with no overrides (hrrr: {} below) to run the defaults unchanged. You can give it any name. Add more entries to run the same receptors under other settings:

variants:
  hrrr: {}                     # the defaults with the hrrr met
  hrrr-zi08: {ziscale: 0.8}    # a mixed-layer sensitivity
  hrrr-np3k: {numpar: 3000}    # a particle-count check
  hrrr-err:                    # a wind-error run (see transport_error)
    siguverr: 2.6
    tluverr: 260
    zcoruverr: 450
    horcoruverr: 14
    grid: null                 # particles only

Each variant of each receptor is one simulation, stored in simulations/by-id/<receptor>/<variant>/ (see Project Folders And Reruns). Every variant runs the same list of receptors. A variant may set:

met: Which meteorology source to use. A variant named after a source uses that source. Otherwise you need met when mets has more than one entry.
realizations: Run the variant N times, as <name>-0 to <name>-(N-1). This is how you declare a transport-error ensemble (see Transport Error). The default krand: 4 gives each run different turbulence. For ensembles you can reproduce, set krand: 2 and a seed. Run k then uses seed + k. The names are numbered even with realizations: 1, so raising the count later only adds runs.
Any other setting: Transport settings (numpar, ziscale, turbulence, wind errors) and footprint settings (grid, smooth_factor, time_integrate, transforms) override the defaults for that variant only.

Variant names may use lowercase letters, digits, and hyphens, because they become folder names. The same rule applies to the names in mets.

A second footprint from the same particles#

Running HYSPLIT is the slow part of a simulation. Making a footprint from stored particles is quick. A variant with from: reuses another variant’s particles and changes only the footprint:

variants:
  hrrr: {}
  hrrr-regional:
    from: hrrr
    grid: {xmin: -125.0, xmax: -100.0, ymin: 30.0, ymax: 50.0, xres: 0.1, yres: 0.1}
  hrrr-ak:
    from: hrrr
    transforms: [{kind: averaging_kernel, table: kernels.parquet}]

A from: variant does not run HYSPLIT, and it may set only footprint settings. If you add one to a finished project and run again, PYSTILT makes just the new footprints from the stored particles.

A grid in a variant changes only the fields it names. A coarser version of the default domain is just grid: {xres: 0.1, yres: 0.1}.

You may not need a variant at all. stilt.Footprint.aggregate() sums a fine footprint onto coarser cells or irregular areas after the run.

Changing a variant that has run#

The first time a project runs, PYSTILT saves the full settings of every variant in simulations/variants.yaml. That file records what produced the outputs. On later runs, PYSTILT compares each variant against it. If a variant’s settings have changed, the run stops with an error that names them. This includes a changed default that the variant inherits, or a changed setting of its meteorology source. The error looks like this:

ConfigChangedError: These settings already ran under their name in ./my_project
(hrrr: ziscale). Declare a new variant for the new settings, or remove the
old outputs first (Model.remove / stilt rm --variant).

The run stops because the finished outputs would no longer match their name. You can go on in one of two ways:

Give the new settings a new name. Add hrrr-zi08: {ziscale: 0.8} and put hrrr back as it was. Only the new variant runs.
Remove the old outputs. stilt rm --variant hrrr (or model.remove("hrrr")) deletes every simulation of that variant and its saved settings. The variant then runs again as new. Variants declared with from: hrrr are removed too, since their footprints came from its particles. You can repeat --variant to remove several variants at once, for example after changing a default they all inherit.

Suppose a campaign raises the default numpar halfway through and keeps the old runs. The base run then has two names, one for each numpar. Select both when you load the results, for example sel(variant=["hrrr", "hrrr-np1k"]). simulations/variants.yaml shows what each name ran with.

Some settings do not change a result, so you can change them at any time:

execution
timeout, rm_dat, and exe_dir
where the meteorology files are (directory and subgrid_dir)

Taking a variant out of config.yaml does not delete its outputs. stilt run warns about such variants, and stilt status lists them, until you remove them with stilt rm.

Footprints for shapefiles, hexagons, or point sources#

You may plan to add footprints up over irregular areas, such as counties from a shapefile, H3 hexagons, or small windows around point sources. You can then name those areas in geometry instead of giving a grid. PYSTILT picks a grid fine enough to resolve them (see stilt.Grid.from_geometry()). This needs the geometry extra (pip install "pystilt[geometry]").

geometry:
  kind: file          # a shapefile, GeoPackage, or GeoJSON
  path: counties.shp
  ids: NAME           # attribute column used as cell ids

The other kinds are h3 hexagons and windows around point sources. To make footprints for several geometries from the same particles, use from: variants:

variants:
  hrrr: {}
  hrrr-hexes:
    from: hrrr
    geometry:
      kind: h3
      resolution: 8
      bounds: {xmin: -112.3, xmax: -111.6, ymin: 40.4, ymax: 41.0}
    cells_per_target: 4   # grid cells across the smallest hexagon (default)
  hrrr-sources:
    from: hrrr
    geometry:
      kind: windows
      coords: [[-111.97, 40.515], [-112.015, 40.779]]
      size: 0.01
      ids: [landfill, wwtp]

The grid PYSTILT picks is saved in simulations/variants.yaml and in each footprint file. You do not need the shapefile again to read a footprint.

If you give both grid and geometry, the grid is used as given. The geometry is kept with the settings, and sim.config.geometry.build() returns the stilt.Mesh to aggregate onto.

Each footprint file also stores a hash of the geometry it was made for. stilt.Footprint.aggregate() warns if the mesh you pass no longer matches, for example because the shapefile was edited after the run.

Note

stilt.Zones (groups of grid cells) cannot be written in YAML. Build them in code with Zones.from_labels(base, labels).

Running on a cluster#

To run on Slurm instead of your own computer, add an execution section:

execution:
  backend: slurm
  n_workers: 100          # number of Slurm array tasks
  account: my-account
  partition: my-partition
  time: "02:00:00"
  mem: 8G

PYSTILT uses backend, n_workers, cpus_per_task, array_parallelism, and setup itself. Every other key is passed to sbatch. See On An HPC Cluster (Slurm).

The same settings in Python#

You can pass any config.yaml key to Model directly, as plain dictionaries:

import stilt

model = stilt.Model(
    project="./my_project",
    mets={
        "hrrr": {
            "directory": "/data/arl/hrrr",
            "file_format": "%Y%m%d_%H",
            "file_tres": "6h",
        }
    },
    grid={
        "xmin": -114.0, "xmax": -111.0,
        "ymin": 39.0, "ymax": 42.0,
        "xres": 0.01, "yres": 0.01,
    },
    variants={"hrrr": {}, "hrrr-zi08": {"ziscale": 0.8}},
    n_hours=-24,
    numpar=500,
)

You can also use the config classes, which your editor can autocomplete and check:

config = stilt.ModelConfig(
    mets={
        "hrrr": stilt.MetConfig(
            directory="/data/arl/hrrr",
            file_format="%Y%m%d_%H",
            file_tres="6h",
        )
    },
    grid=stilt.Grid(
        xmin=-114.0, xmax=-111.0,
        ymin=39.0, ymax=42.0,
        xres=0.01, yres=0.01,
    ),
    n_hours=-24,
    numpar=500,
)
model = stilt.Model(project="./my_project", config=config)
model.variants          # {"hrrr": VariantConfig(...)}, each with its full settings

Settings given in Python replace config.yaml when the model runs. To keep editing the file by hand, open the project with stilt.Model(project="./my_project") and no settings.

Advanced HYSPLIT settings#

config.yaml also accepts the lower-level HYSPLIT and STILT settings, such as turbulence, time step, and output variables. They use the same names as in STILT-R. Most projects never change them. Configuration describes each one.

PYSTILT checks config.yaml when it loads the file, before anything runs. A misspelled setting is an error. The one exception is inside a mets entry. Extra keys there are passed on as download options (like domain for nams), so check their spelling yourself.