Meteorology#

STILT moves particles with gridded fields from a weather model: winds, temperature, turbulence, and boundary-layer height. The files must be in NOAA’s ARL format. NOAA publishes ARL files for HRRR, NAM, GDAS, GFS, and other models.

You can give PYSTILT meteorology in two ways:

Point it at files you already have, such as your group’s archive.
Let it download the files from NOAA. PYSTILT fetches the files each simulation needs and keeps them for later runs.

Each meteorology source has a name in config.yaml (hrrr in the examples below). Variants refer to a source by this name. With no variants section, each source runs as a variant of the same name (see Configuration), so you can run the same receptors with several sources and compare them.

Use files you already have#

Give the folder, the pattern of the file names, and how many hours each file covers:

mets:
  hrrr:
    directory: /data/met/hrrr
    file_format: "%Y%m%d_%H"
    file_tres: 6h

directory: The folder that holds the files. Subfolders are searched too.
file_format: The file names, with the date written as strftime codes: %Y for the year, %m month, %d day, and %H hour. Files named 20230715_18 match "%Y%m%d_%H". Files named hysplit.20230715.18z.hrrra match "hysplit.%Y%m%d.%Hz.hrrra". A file matches when its name starts with the pattern.
file_tres: How much time each file covers, such as 1h, 3h, or 6h.
n_min: The fewest files a simulation needs (default 1).

For each simulation, PYSTILT works out which files cover the hours the run spans (n_hours from the receptor time) and looks for those. If it finds fewer than n_min, the simulation fails with an error. If some are missing but at least n_min are found, PYSTILT logs a warning and runs with the files it has. Raise n_min to turn gaps into errors. A 24-hour backward run with 6-hour HRRR files needs 5 or 6 files, depending on the receptor hour, so n_min: 5 is a good choice there.

In Python, the same settings are a dictionary or a MetConfig:

hrrr = stilt.MetConfig(
    directory="/data/met/hrrr",
    file_format="%Y%m%d_%H",
    file_tres="6h",
)

You can list several sources:

mets:
  hrrr:
    directory: /data/met/hrrr
    file_format: "%Y%m%d_%H"
    file_tres: 6h
  gfs:
    directory: /data/met/gfs
    file_format: "gfs_%Y%m%d_%H"
    file_tres: 3h

Download from NOAA#

Set source to one of NOAA’s products and directory to where the downloads should go. You don’t need file_format or file_tres.

mets:
  hrrr:
    source: hrrr
    directory: /data/met/hrrr     # downloads are kept here

Downloading needs the cloud extra (pip install "pystilt[cloud]"). The arl-met package does the downloading.

Warning

NOAA’s files cover a continent or the whole globe, so each one is large, often several gigabytes. The whole file is downloaded before it is cropped. Crop to your region (see below) so that what is kept is small, and download on a machine with a fast connection and plenty of disk space.

These sources are available:

Name	Product	Domain	Period
`hrrr`	HRRR 3 km analysis	CONUS	Jun 2019–present
`hrrr.v1`	HRRR 3 km analysis v1	CONUS	Jun 2015–2019
`nam12`	NAM 12 km analysis	North America	May 2007–present
`nams`	NAMS hybrid sigma-pressure	CONUS / AK / HI	Jan 2010–present
`gdas1`	GDAS 1-degree global	Global	Dec 2004–present
`gdas0p5`	GDAS 0.5-degree global	Global	Sep 2007–mid 2019
`gfs0p25`	GFS 0.25-degree global	Global	Jun 2019–present
`reanalysis`	NCEP/NCAR Reanalysis 2.5-degree	Global	1948–present
`narr`	NCEP North American Regional Reanalysis 32 km	North America	1979–2019

Some sources take extra options, which you write next to the other fields. For example, nams takes a domain:

nams_ak = stilt.MetConfig(
    source="nams",
    domain="ak",            # "conus" (default), "ak", or "hi"
    directory="/data/met/nams_ak",
)

backend picks where to download from. The default, "s3", is NOAA’s archive on AWS. The others are "ftp" and "http".

stilt.MetConfig(source="gdas1", directory="/data/met/gdas1", backend="ftp")

Files already in directory are not downloaded again.

Cropping to your region#

Set subgrid_enable and subgrid_bounds to crop the meteorology to a box around your region before HYSPLIT reads it. Smaller files make runs faster and use less memory. Cropping matters most for global products (GFS, GDAS, Reanalysis) and on cluster nodes with limited memory.

from stilt import Bounds, MetConfig

hrrr = MetConfig(
    source="hrrr",
    directory="/data/met/hrrr",
    subgrid_enable=True,
    subgrid_bounds=Bounds(xmin=-114, xmax=-110, ymin=39, ymax=42),
    subgrid_buffer=0.5,   # degrees added on each side (default 0.2)
)

When downloading, each file is cropped right after it arrives and only the cropped copy is kept.

With your own files, each file is cropped the first time a simulation needs it. The cropped copies are saved in subgrid_dir, which defaults to <directory>/subgrid, and every later simulation reuses them. If your archive is read-only, or you want several projects to share one set of crops, set subgrid_dir yourself:

MetConfig(
    directory="/data/met/hrrr",
    file_format="%Y%m%d_%H",
    file_tres="6h",
    subgrid_enable=True,
    subgrid_bounds=Bounds(xmin=-114, xmax=-110, ymin=39, ymax=42),
    subgrid_dir="/scratch/met_subgrid/hrrr_slv",
)

Saved crops are found by file name only. If you change subgrid_bounds, subgrid_buffer, or subgrid_levels, use a new subgrid_dir. Otherwise PYSTILT reuses the old crops.

With your own files, subgrid_levels also drops the upper vertical levels. It has no effect on downloaded files.

MetConfig(
    ...,
    subgrid_levels=20,   # keep the lowest 20 levels
)

Where HYSPLIT reads the files#

Before each simulation, PYSTILT links the meteorology files it needs into that simulation’s working folder, or copies them if it cannot link. HYSPLIT reads them from there. PYSTILT does not change your files. On a cluster, this lets HYSPLIT read from fast local scratch space (see compute_root in Project Folders And Reruns).