arlmet.archives.Archive.fetch#

Archive.fetch(start, end, *, dest_dir, mirror='s3', bbox=None, levels=None, overwrite=False)[source]#

Download ARL files covering [start, end] to dest_dir.

Parameters:

  • start (Timestamp | str) – Time range (inclusive). Backward trajectories (start > end) are handled automatically. A time without a time zone is UTC; a time-zone-aware one is converted to UTC.

  • end (Timestamp | str) – Time range (inclusive). Backward trajectories (start > end) are handled automatically. A time without a time zone is UTC; a time-zone-aware one is converted to UTC.

  • dest_dir (Path | str) – Directory to save downloaded files. Created if absent.

  • mirror (Literal['s3', 'ftp', 'http']) – Which copy of the archive to download from — "s3" (default, AWS), "ftp" (NOAA ARL FTP), or "http" (NOAA READY web).

  • bbox (tuple[float, float, float, float] | None) – (west, south, east, north) in degrees. When provided, each file is cropped with arlmet.extract_subset() before caching (on a projected grid, to the smallest rectangle of cells that holds the box’s part on the grid). Strongly recommended for global products (GFS, GDAS).

  • levels (Iterable[int] | None) – ARL level indices to keep, counted from 0 at the surface. When provided, each file keeps only these levels, cropped with arlmet.extract_subset() like bbox. All levels are kept by default.

  • overwrite (bool) – Re-download even if a matching local file already exists.

Returns:

Local paths to the downloaded (and optionally cropped) files, in chronological order.

Return type:

list[Path]

Raises:

  • ImportError – If fsspec is not installed.

  • ValueError – If the range begins before the archive’s start_date.

Notes

Files are written to a hidden .<name>.<random>.partial file in dest_dir and renamed into place only once complete, so an interrupted fetch never leaves a truncated file that a later call would reuse. When cropping, the full download is also staged in dest_dir (not the system temp directory) and removed afterwards.

Concurrent calls (threads, processes, or jobs on other nodes sharing dest_dir) download each file once: the first makes a hidden folder .<name>.lock and the others wait for the finished file. If nothing grows for about five minutes (the first was killed, or its download hung) a waiter takes over and removes what the first left. A finished file is replaced only with overwrite=True, where each caller downloads again in turn: replacing a file breaks a reader on another NFS client.

Examples

>>> from arlmet.archives import HRRRArchive
>>> archive = HRRRArchive()
>>> archive.fetch(
...     "2024-07-18", "2024-07-19", dest_dir="./met"
... )