StorageConfig

class fseval.config.StorageConfig(
    load_dir: Optional[str]=None,
    save_dir: Optional[str]=None,
)

Allows you to define a storage for loading and saving cached estimators, among other files, like the hydra and fseval configuration in YAML.

Attributes:

load_dir : Optional[str]	Defines a path to load files from. Must point to exactly the directory containing the files, i.e. you should not point to a higher-level directory than where the files are. Path can be relative or absolute, but an absolute path is recommended.
save_dir : Optional[str]	The directory to save files to. Can be relative or absolute.

Available storages

Local

class fseval.config.storage.LocalStorageConfig(
    load_dir: Optional[str]=None, 
    save_dir: Optional[str]=None,
)

Saves files to a local directory.

Assuming your rankers and validators save_cache is configured to CacheUsage.allow, the fitted estimators will be saved in the local hydra directory. That is, they will be saved as .pickle files, with an appropriate name. In a different run, you can set load_dir to this directory to load the fitted estimators again. Example:

Attributes:

load_dir : str	The directory to load files from
save_dir : str	The directory to save files to

Use with storage=local on the commandline, or.

conf/my_config.yaml

defaults:
  - base_pipeline_config
  - _self_
  - override storage: local

Weights and Biases

class fseval.config.storage.WandbStorageConfig(
    load_dir: Optional[str]=None, 
    save_dir: Optional[str]=None, 
    entity: Optional[str]=None, 
    project: Optional[str]=None, 
    run_id: Optional[str]=None, 
    save_policy: Optional[str]="live",
)

Storage for Weights and Biases (wandb), allowing users to save- and restore files to the service.

Like local storage, estimator caches are saved, as well as the entire YAML config of the run. Example:

Attributes:

load_dir : Optional[str]	when set, an attempt is made to load from the designated local directory first, before downloading the data off of wandb. Can be used to perform faster loads or prevent being rate-limited on wandb.
save_dir : Optional[str]	when set, uses this directory to save files, instead of the usual wandb run directory, under the files subdirectory.
entity : Optional[str]	allows you to recover from a specific entity, instead of using the entity that is set for the 'current' run.
project : Optional[str]	recover from a specific project.
run_id : Optional[str]	recover from a specific run id.
save_policy : str	policy for wandb.save. Can be 'live', 'now' or 'end'. Determines at which point of the run the file is uploaded. Defaults to "live".

Use with storage=wandb on the commandline, or.

Alternatively, in your config:

conf/my_config.yaml

defaults:
  - base_pipeline_config
  - _self_
  - override storage: wandb

Mock

class fseval.config.storage.MockStorageConfig(
    load_dir: Optional[str]=None, 
    save_dir: Optional[str]=None,
)

Disables storage.

Attributes:

load_dir : str	The directory to load files from
save_dir : str	The directory to save files to

Use with storage=mock on the commandline, or.

conf/my_config.yaml

defaults:
  - base_pipeline_config
  - _self_
  - override storage: mock