AsdfFile

class asdf.AsdfFile(tree=None, uri=None, extensions=None, version=None, ignore_version_mismatch=True, ignore_unrecognized_tag=False, copy_arrays=False, custom_schema=None)[source]

Bases: asdf.versioning.VersionedMixin

The main class that represents an ASDF file object.

Parameters:

tree : dict or AsdfFile, optional

The main tree data in the ASDF file. Must conform to the ASDF schema.

uri : str, optional

The URI for this ASDF file. Used to resolve relative references against. If not provided, will be automatically determined from the associated file object, if possible and if created from AsdfFile.open.

extensions : list of AsdfExtension

A list of extensions to use when reading and writing ASDF files. See AsdfExtension for more information.

version : str, optional

The ASDF version to use when writing out. If not provided, it will write out in the latest version supported by asdf.

ignore_version_mismatch : bool, optional

When True, do not raise warnings for mismatched schema versions. Set to True by default.

ignore_unrecognized_tag : bool, optional

When True, do not raise warnings for unrecognized tags. Set to False by default.

copy_arrays : bool, optional

When False, when reading files, attempt to memmap underlying data arrays when possible.

custom_schema : str, optional

Path to a custom schema file that will be used for a secondary validation pass. This can be used to ensure that particular ASDF files follow custom conventions beyond those enforced by the standard.

Attributes Summary

blocks Get the block manager associated with the AsdfFile.
comments Get the comments after the header, before the tree.
file_format_version
tag_mapping
tag_to_schema_resolver
tree Get/set the tree of data in the ASDF file.
type_index
uri Get the URI associated with the AsdfFile.
url_mapping

Methods Summary

add_history_entry(description[, software]) Add an entry to the history list.
close() Close the file handles associated with the AsdfFile.
copy()
fill_defaults() Fill in any values that are missing in the tree using default values from the schema.
find_references() Finds all external “JSON References” in the tree and converts them to reference.Reference objects.
get_array_compression(arr) Get the compression type for the given array data.
get_array_storage(arr) Get the block type for the given array data.
make_reference([path]) Make a new reference to a part of this file’s tree, that can be assigned as a reference to another tree.
open(fd[, uri, mode, validate_checksums, …]) Open an existing ASDF file.
open_external(uri[, do_not_fill_defaults]) Open an external ASDF file, from the given (possibly relative) URI.
remove_defaults() Remove any values in the tree that are the same as the default values in the schema
resolve_and_inline() Resolves all external references and inlines all data.
resolve_references([do_not_fill_defaults]) Finds all external “JSON References” in the tree, loads the external content, and places it directly in the tree.
resolve_uri(uri) Resolve a (possibly relative) URI against the URI of this ASDF file.
resolver(uri)
run_hook(hookname) Run a “hook” for each custom type found in the tree.
run_modifying_hook(hookname[, validate]) Run a “hook” for each custom type found in the tree.
set_array_compression(arr, compression) Set the compression to use for the given array data.
set_array_storage(arr, array_storage) Set the block type to use for the given array data.
update([all_array_storage, …]) Update the file on disk in place.
validate() Validate the current state of the tree against the ASDF schema.
write_to(fd[, all_array_storage, …]) Write the ASDF file to the given file-like object.

Attributes Documentation

blocks

Get the block manager associated with the AsdfFile.

comments

Get the comments after the header, before the tree.

file_format_version
tag_mapping
tag_to_schema_resolver
tree

Get/set the tree of data in the ASDF file.

When set, the tree will be validated against the ASDF schema.

type_index
uri

Get the URI associated with the AsdfFile.

In many cases, it is automatically determined from the file handle used to read or write the file.

url_mapping

Methods Documentation

add_history_entry(description, software=None)[source]

Add an entry to the history list.

Parameters:

description : str

A description of the change.

software : dict or list of dict

A description of the software used. It should not include asdf itself, as that is automatically notated in the asdf_library entry.

Each dict must have the following keys:

  • name: The name of the software
  • author: The author or institution that produced the software
  • homepage: A URI to the homepage of the software
  • version: The version of the software
close()[source]

Close the file handles associated with the AsdfFile.

copy()[source]
fill_defaults()[source]

Fill in any values that are missing in the tree using default values from the schema.

find_references()[source]

Finds all external “JSON References” in the tree and converts them to reference.Reference objects.

get_array_compression(arr)[source]

Get the compression type for the given array data.

Parameters:arr : numpy.ndarray
Returns:compression : str or None
get_array_storage(arr)[source]

Get the block type for the given array data.

Parameters:arr : numpy.ndarray
make_reference(path=[])[source]

Make a new reference to a part of this file’s tree, that can be assigned as a reference to another tree.

Parameters:

path : list of str and int, optional

The parts of the path pointing to an item in this tree. If omitted, points to the root of the tree.

Returns:

reference : reference.Reference

A reference object.

Examples

For the given AsdfFile ff, add an external reference to the data in an external file:

>>> import asdf
>>> flat = asdf.open("http://stsci.edu/reference_files/flat.asdf")  
>>> ff.tree['flat_field'] = flat.make_reference(['data'])  
classmethod open(fd, uri=None, mode='r', validate_checksums=False, extensions=None, do_not_fill_defaults=False, ignore_version_mismatch=True, ignore_unrecognized_tag=False, _force_raw_types=False, copy_arrays=False, custom_schema=None, strict_extension_check=False, ignore_missing_extensions=False)[source]

Open an existing ASDF file.

Parameters:

fd : string or file-like object

May be a string file or http URI, or a Python file-like object.

uri : string, optional

The URI of the file. Only required if the URI can not be automatically determined from fd.

mode : string, optional

The mode to open the file in. Must be r (default) or rw.

validate_checksums : bool, optional

If True, validate the blocks against their checksums. Requires reading the entire file, so disabled by default.

extensions : list of AsdfExtension

A list of extensions to use when reading and writing ASDF files. See AsdfExtension for more information.

do_not_fill_defaults : bool, optional

When True, do not fill in missing default values.

ignore_version_mismatch : bool, optional

When True, do not raise warnings for mismatched schema versions. Set to True by default.

ignore_unrecognized_tag : bool, optional

When True, do not raise warnings for unrecognized tags. Set to False by default.

copy_arrays : bool, optional

When False, when reading files, attempt to memmap underlying data arrays when possible.

custom_schema : str, optional

Path to a custom schema file that will be used for a secondary validation pass. This can be used to ensure that particular ASDF files follow custom conventions beyond those enforced by the standard.

strict_extension_check : bool, optional

When True, if the given ASDF file contains metadata about the extensions used to create it, and if those extensions are not installed, opening the file will fail. When False, opening a file under such conditions will cause only a warning. Defaults to False.

ignore_missing_extensions : bool, optional

When True, do not raise warnings when a file is read that contains metadata about extensions that are not available. Defaults to False.

Returns:

asdffile : AsdfFile

The new AsdfFile object.

open_external(uri, do_not_fill_defaults=False)[source]

Open an external ASDF file, from the given (possibly relative) URI. There is a cache (internal to this ASDF file) that ensures each external ASDF file is loaded only once.

Parameters:

uri : str

An absolute or relative URI to resolve against the URI of this ASDF file.

do_not_fill_defaults : bool, optional

When True, do not fill in missing default values.

Returns:

asdffile : AsdfFile

The external ASDF file.

remove_defaults()[source]

Remove any values in the tree that are the same as the default values in the schema

resolve_and_inline()[source]

Resolves all external references and inlines all data. This produces something that, when saved, is a 100% valid YAML file.

resolve_references(do_not_fill_defaults=False)[source]

Finds all external “JSON References” in the tree, loads the external content, and places it directly in the tree. Saving a ASDF file after this operation means it will have no external references, and will be completely self-contained.

resolve_uri(uri)[source]

Resolve a (possibly relative) URI against the URI of this ASDF file. May be overridden by base classes to change how URIs are resolved. This does not apply any uri_mapping that was passed to the constructor.

Parameters:

uri : str

An absolute or relative URI to resolve against the URI of this ASDF file.

Returns:

uri : str

The resolved URI.

resolver(uri)[source]
run_hook(hookname)[source]

Run a “hook” for each custom type found in the tree.

Parameters:

hookname : str

The name of the hook. If a AsdfType is found with a method with this name, it will be called for every instance of the corresponding custom type in the tree.

run_modifying_hook(hookname, validate=True)[source]

Run a “hook” for each custom type found in the tree. The hook is free to return a different object in order to modify the tree.

Parameters:

hookname : str

The name of the hook. If a AsdfType is found with a method with this name, it will be called for every instance of the corresponding custom type in the tree.

validate : bool

When True (default) validate the resulting tree.

set_array_compression(arr, compression)[source]

Set the compression to use for the given array data.

Parameters:

arr : numpy.ndarray

The array to set. If multiple views of the array are in the tree, only the most recent compression setting will be used, since all views share a single block.

compression : str or None

Must be one of:

  • '' or None: no compression
  • zlib: Use zlib compression
  • bzp2: Use bzip2 compression
  • lz4: Use lz4 compression
  • '' or None: no compression
  • input: Use the same compression as in the file read. If there is no prior file, acts as None.
set_array_storage(arr, array_storage)[source]

Set the block type to use for the given array data.

Parameters:

arr : numpy.ndarray

The array to set. If multiple views of the array are in the tree, only the most recent block type setting will be used, since all views share a single block.

array_storage : str

Must be one of:

  • internal: The default. The array data will be stored in a binary block in the same ASDF file.
  • external: Store the data in a binary block in a separate ASDF file.
  • inline: Store the data as YAML inline in the tree.
update(all_array_storage=None, all_array_compression='input', auto_inline=None, pad_blocks=False, include_block_index=True, version=None)[source]

Update the file on disk in place.

Parameters:

all_array_storage : string, optional

If provided, override the array storage type of all blocks in the file immediately before writing. Must be one of:

  • internal: The default. The array data will be stored in a binary block in the same ASDF file.
  • external: Store the data in a binary block in a separate ASDF file.
  • inline: Store the data as YAML inline in the tree.

all_array_compression : string, optional

If provided, set the compression type on all binary blocks in the file. Must be one of:

  • '' or None: No compression.
  • zlib: Use zlib compression.
  • bzp2: Use bzip2 compression.
  • lz4: Use lz4 compression.
  • input: Use the same compression as in the file read. If there is no prior file, acts as None

auto_inline : int, optional

When the number of elements in an array is less than this threshold, store the array as inline YAML, rather than a binary block. This only works on arrays that do not share data with other arrays. Default is 0.

pad_blocks : float or bool, optional

Add extra space between blocks to allow for updating of the file. If False (default), add no padding (always return 0). If True, add a default amount of padding of 10% If a float, it is a factor to multiple content_size by to get the new total size.

include_block_index : bool, optional

If False, don’t include a block index at the end of the file. (Default: True) A block index is never written if the file has a streamed block.

version : str, optional

The ASDF version to write out. If not provided, it will write out in the latest version supported by asdf.

validate()[source]

Validate the current state of the tree against the ASDF schema.

write_to(fd, all_array_storage=None, all_array_compression='input', auto_inline=None, pad_blocks=False, include_block_index=True, version=None)[source]

Write the ASDF file to the given file-like object.

write_to does not change the underlying file descriptor in the AsdfFile object, but merely copies the content to a new file.

Parameters:

fd : string or file-like object

May be a string path to a file, or a Python file-like object. If a string path, the file is automatically closed after writing. If not a string path,

all_array_storage : string, optional

If provided, override the array storage type of all blocks in the file immediately before writing. Must be one of:

  • internal: The default. The array data will be stored in a binary block in the same ASDF file.
  • external: Store the data in a binary block in a separate ASDF file.
  • inline: Store the data as YAML inline in the tree.

all_array_compression : string, optional

If provided, set the compression type on all binary blocks in the file. Must be one of:

  • '' or None: No compression.
  • zlib: Use zlib compression.
  • bzp2: Use bzip2 compression.
  • lz4: Use lz4 compression.
  • input: Use the same compression as in the file read. If there is no prior file, acts as None.

auto_inline : int, optional

When the number of elements in an array is less than this threshold, store the array as inline YAML, rather than a binary block. This only works on arrays that do not share data with other arrays. Default is 0.

pad_blocks : float or bool, optional

Add extra space between blocks to allow for updating of the file. If False (default), add no padding (always return 0). If True, add a default amount of padding of 10% If a float, it is a factor to multiple content_size by to get the new total size.

include_block_index : bool, optional

If False, don’t include a block index at the end of the file. (Default: True) A block index is never written if the file has a streamed block.

version : str, optional

The ASDF version to write out. If not provided, it will write out in the latest version supported by asdf.