uataq.filesystem#

Access to the UATAQ filesystem through DataFile and GroupSpace objects.

The UATAQ subpackage operates on the idea that instrument data is collected differently by each research group, despite potentially being from the same site or instrument. Furthermore, the format in which data is written, and therefore read, is dependent on the system that logged the data rather than the instrument itself. To address this, UATAQ introduces a filesystem module which consists of GroupSpace and DataFile objects. The GroupSpace objects provide group-specific methods for working with data from that group, while the DataFile objects handle the actual parsing of the data files.

Each research group has its own module in groupspaces where group-specific code is stored. Each group module must contain a subclass of GroupSpace and define DataFile subclasses for each file format that the group uses.

All GroupSpace objects are stored in the groups dictionary with the group name as the key.

The default research group can be changed at runtime via uataq.filesystem.DEFAULT_GROUP or permanently (until the next update) by changing its line in uataq/filesystem/__init__.py.

Attributes

`DEFAULT_GROUP`	Default group to read data from.
`groups`	Groups dictionary to store GroupSpace objects.
`lvls`	Levels of data processing.

Functions

get_group

Get the group name.

Exported from submodules

`DataFile`	Abstract base class for data files.
`GroupSpace`	Abstract base class for group spaces.
`cpu_count`	Return the number of CPUs this process may run on.
`list_files`	Return a list of files in the specified directory that match the specified pattern.
`filter_datafiles`	Filter a list of files by a given time range.
`parse_datafiles`	Read and parse data files, in parallel, using multiple processes.

Modules

`core`	Classes and functions for working in the CHPC UATAQ filesystem.
`groupspaces`	Group space adapters, each describing one research group's on-disk layout.