AsdfFile
- class asdf.AsdfFile(tree=None, uri=None, extensions=None, version=None, ignore_version_mismatch=True, ignore_unrecognized_tag=False, ignore_implicit_conversion=NotSet, copy_arrays=False, memmap=NotSet, lazy_load=True, custom_schema=None)[source]
Bases:
object
The main class that represents an ASDF file object.
- Parameters:
- treedict or AsdfFile, optional
The main tree data in the ASDF file. Must conform to the ASDF schema.
- uristr, optional
The URI for this ASDF file. Used to resolve relative references against. If not provided, will be automatically determined from the associated file object, if possible and if created from
asdf.open
.- extensionsobject, optional
Additional extensions to use when reading and writing the file. May be an
asdf.extension.Extension
or alist
of extensions.- versionstr, optional
The ASDF Standard version. If not provided, defaults to the configured default version. See
asdf.config.AsdfConfig.default_version
.- ignore_version_mismatchbool, optional
When
True
, do not raise warnings for mismatched schema versions. Set toTrue
by default.- ignore_unrecognized_tagbool, optional
When
True
, do not raise warnings for unrecognized tags. Set toFalse
by default.- ignore_implicit_conversionbool
DEPRECATED When
True
, do not raise warnings when types in the tree are implicitly converted into a serializable object. The motivating case for this is currentlynamedtuple
, which cannot be serialized as-is.- copy_arraysbool, optional
When
False
, when reading files, attempt to memmap underlying data arrays when possible.- memmapbool, optional
When
True
, when reading files, attempt to memmap underlying data arrays when possible. When set, this argument will overridecopy_arrays
. When not set, thecopy_arrays
will determine if arrays are memory mapped or copied.copy_arrays
will be deprecated and the default will change in an upcoming asdf version which by default will not memory map arrays.- lazy_loadbool, optional
When
True
and the underlying file handle is seekable, data arrays will only be loaded lazily: i.e. when they are accessed for the first time. In this case the underlying file must stay open during the lifetime of the tree. Setting to False causes all data arrays to be loaded up front, which means that they can be accessed even after the underlying file is closed. Note: even iflazy_load
isFalse
,copy_arrays
is still taken into account.- custom_schemastr, optional
Path to a custom schema file that will be used for a secondary validation pass. This can be used to ensure that particular ASDF files follow custom conventions beyond those enforced by the standard.
Attributes Summary
Get the comments after the header, before the tree.
Get the ExtensionManager for this AsdfFile.
Get the list of user extensions that are enabled for use with this AsdfFile.
Get/set the tree of data in the ASDF file.
Get the URI associated with the
AsdfFile
.Get this AsdfFile's ASDF Standard version.
Get this AsdfFile's ASDF Standard version as a string.
Methods Summary
add_history_entry
(description[, software])Add an entry to the history list.
close
()Close the file handles associated with the
asdf.AsdfFile
.copy
()Fill in any values that are missing in the tree using default values from the schema.
find_references
([_warning_msg])Finds all external "JSON References" in the tree and converts them to
reference.Reference
objects.Get the compression type for the given array data.
get_array_save_base
(arr)Returns the
save_base
option forarr
.get_array_storage
(arr)Get the block type for the given array data.
Get a list of history entries from the file object.
info
([max_rows, max_cols, show_values, ...])Print a rendering of this file's tree to stdout.
keys
()make_reference
([path])Make a new reference to a part of this file's tree, that can be assigned as a reference to another tree.
open_external
(uri, **kwargs)Open an external ASDF file, from the given (possibly relative) URI.
Remove any values in the tree that are the same as the default values in the schema
Resolves all external references and inlines all data.
resolve_references
(**kwargs)Finds all external "JSON References" in the tree, loads the external content, and places it directly in the tree.
resolve_uri
(uri)Resolve a (possibly relative) URI against the URI of this ASDF file.
schema_info
([key, path, preserve_list, ...])Get a nested dictionary of the schema information for a given key, relative to the path.
search
([key, type_, value, filter_])Search this file's tree.
set_array_compression
(arr, compression, ...)Set the compression to use for the given array data.
set_array_save_base
(arr, save_base)Set the
save_base
option forarr
.set_array_storage
(arr, array_storage)Set the block type to use for the given array data.
update
([all_array_storage, ...])Update the file on disk in place.
validate
()Validate the current state of the tree against the ASDF schema.
write_to
(fd[, all_array_storage, ...])Write the ASDF file to the given file-like object.
Attributes Documentation
- comments
Get the comments after the header, before the tree.
- extension_manager
Get the ExtensionManager for this AsdfFile.
- Returns:
- asdf.extension.ExtensionManager
- extensions
Get the list of user extensions that are enabled for use with this AsdfFile.
- Returns:
- list of asdf.extension.ExtensionProxy
- file_format_version
- tree
Get/set the tree of data in the ASDF file.
When set, the tree will be validated against the ASDF schema.
- uri
Get the URI associated with the
AsdfFile
.In many cases, it is automatically determined from the file handle used to read or write the file.
- version
Get this AsdfFile’s ASDF Standard version.
- Returns:
- asdf.versioning.AsdfVersion
- version_map
- version_string
Get this AsdfFile’s ASDF Standard version as a string.
- Returns:
- str
Methods Documentation
- add_history_entry(description, software=None)[source]
Add an entry to the history list.
- Parameters:
- descriptionstr
A description of the change.
- softwaredict or list of dict
A description of the software used. It should not include asdf itself, as that is automatically notated in the
asdf_library
entry.Each dict must have the following keys:
name
: The name of the softwareauthor
: The author or institution that produced the softwarehomepage
: A URI to the homepage of the softwareversion
: The version of the software
- close()[source]
Close the file handles associated with the
asdf.AsdfFile
.
- fill_defaults()[source]
Fill in any values that are missing in the tree using default values from the schema.
- find_references(_warning_msg=False)[source]
Finds all external “JSON References” in the tree and converts them to
reference.Reference
objects.
- get_array_compression(arr)[source]
Get the compression type for the given array data.
- Parameters:
- arrnumpy.ndarray
- Returns:
- compressionstr or None
- get_array_save_base(arr)[source]
Returns the
save_base
option forarr
. Whenarr
is written to a file, ifsave_base
isTrue
the base array forarr
will be saved.- Parameters:
- arrnumpy.ndarray
- Returns:
- save_basebool or None
- get_array_storage(arr)[source]
Get the block type for the given array data.
- Parameters:
- arrnumpy.ndarray
- get_history_entries()[source]
Get a list of history entries from the file object.
- Returns:
- entrieslist
A list of history entries.
- info(max_rows=24, max_cols=120, show_values=True, refresh_extension_manager=False)[source]
Print a rendering of this file’s tree to stdout.
- Parameters:
- max_rowsint, tuple, or None, optional
Maximum number of lines to print. Nodes that cannot be displayed will be elided with a message. If int, constrain total number of displayed lines. If tuple, constrain lines per node at the depth corresponding to the tuple index. If None, display all lines.
- max_colsint or None, optional
Maximum length of line to print. Nodes that cannot be fully displayed will be truncated with a message. If int, constrain length of displayed lines. If None, line length is unconstrained.
- show_valuesbool, optional
Set to False to disable display of primitive values in the rendered tree.
- make_reference(path=None)[source]
Make a new reference to a part of this file’s tree, that can be assigned as a reference to another tree.
- Parameters:
- pathlist of str and int, optional
The parts of the path pointing to an item in this tree. If omitted, points to the root of the tree.
- Returns:
- reference
A reference object.
Examples
For the given AsdfFile
ff
, add an external reference to the data in an external file:>>> import asdf >>> flat = asdf.open("http://stsci.edu/reference_files/flat.asdf") >>> ff.tree['flat_field'] = flat.make_reference(['data'])
- open_external(uri, **kwargs)[source]
Open an external ASDF file, from the given (possibly relative) URI. There is a cache (internal to this ASDF file) that ensures each external ASDF file is loaded only once.
- Parameters:
- uristr
An absolute or relative URI to resolve against the URI of this ASDF file.
- Returns:
- asdffileAsdfFile
The external ASDF file.
- remove_defaults()[source]
Remove any values in the tree that are the same as the default values in the schema
- resolve_and_inline()[source]
Resolves all external references and inlines all data. This produces something that, when saved, is a 100% valid YAML file.
- resolve_references(**kwargs)[source]
Finds all external “JSON References” in the tree, loads the external content, and places it directly in the tree. Saving a ASDF file after this operation means it will have no external references, and will be completely self-contained.
- resolve_uri(uri)[source]
Resolve a (possibly relative) URI against the URI of this ASDF file. May be overridden by base classes to change how URIs are resolved. This does not apply any
uri_mapping
that was passed to the constructor.- Parameters:
- uristr
An absolute or relative URI to resolve against the URI of this ASDF file.
- Returns:
- uristr
The resolved URI.
- schema_info(key='description', path=None, preserve_list=True, refresh_extension_manager=False)[source]
Get a nested dictionary of the schema information for a given key, relative to the path.
- Parameters:
- keystr
The key to look up. Default: “description”
- pathstr or asdf.search.AsdfSearchResult
A dot-separated path to the parameter to find the key information on or an
asdf.search.AsdfSearchResult
object. Default = None (full dictionary).- preserve_listbool
If True, then lists are preserved. Otherwise, they are turned into dicts.
- refresh_extension_managerbool
If
True
, refresh the extension manager before looking up the key. This is useful if you want to make sure that the schema data for a given key is up to date.
- search(key=NotSet, type_=NotSet, value=NotSet, filter_=None)[source]
Search this file’s tree.
- Parameters:
- keyNotSet, str, or any other object
Search query that selects nodes by dict key or list index. If NotSet, the node key is unconstrained. If str, the input is searched among keys/indexes as a regular expression pattern. If any other object, node’s key or index must equal the queried key.
- type_NotSet, str, or builtins.type
Search query that selects nodes by type. If NotSet, the node type is unconstrained. If str, the input is searched among (fully qualified) node type names as a regular expression pattern. If builtins.type, the node must be an instance of the input.
- valueNotSet, str, or any other object
Search query that selects nodes by value. If NotSet, the node value is unconstrained. If str, the input is searched among values as a regular expression pattern. If any other object, node’s value must equal the queried value.
- filter_callable
Callable that filters nodes by arbitrary criteria. The callable accepts one or two arguments:
the node
the node’s list index or dict key (optional)
and returns True to retain the node, or False to remove it from the search results.
- Returns:
- asdf.search.AsdfSearchResult
the result of the search
- set_array_compression(arr, compression, **compression_kwargs)[source]
Set the compression to use for the given array data.
- Parameters:
- arrnumpy.ndarray
The array to set. If multiple views of the array are in the tree, only the most recent compression setting will be used, since all views share a single block.
- compressionstr or None
Must be one of:
''
orNone
: no compressionzlib
: Use zlib compressionbzp2
: Use bzip2 compressionlz4
: Use lz4 compressioninput
: Use the same compression as in the file read. If there is no prior file, acts as None.
- set_array_save_base(arr, save_base)[source]
Set the
save_base
option forarr
. Whenarr
is written to a file, ifsave_base
isTrue
the base array forarr
will be saved.Note that similar to other array options this setting is linked to the base array if
arr
is a view.- Parameters:
- arrnumpy.ndarray
- save_basebool or None
if
None
thedefault_array_save_base
value from asdf config will be used
- set_array_storage(arr, array_storage)[source]
Set the block type to use for the given array data.
- Parameters:
- arrnumpy.ndarray
The array to set. If multiple views of the array are in the tree, only the most recent block type setting will be used, since all views share a single block.
- array_storagestr
Must be one of:
internal
: The default. The array data will be stored in a binary block in the same ASDF file.external
: Store the data in a binary block in a separate ASDF file.inline
: Store the data as YAML inline in the tree.
- update(all_array_storage=NotSet, all_array_compression=NotSet, compression_kwargs=NotSet, pad_blocks=False, include_block_index=True, version=None)[source]
Update the file on disk in place.
- Parameters:
- all_array_storagestring, optional
If provided, override the array storage type of all blocks in the file immediately before writing. Must be one of:
internal
: The default. The array data will be stored in a binary block in the same ASDF file.external
: Store the data in a binary block in a separate ASDF file.inline
: Store the data as YAML inline in the tree.
- all_array_compressionstring, optional
If provided, set the compression type on all binary blocks in the file. Must be one of:
''
orNone
: No compression.zlib
: Use zlib compression.bzp2
: Use bzip2 compression.lz4
: Use lz4 compression.input
: Use the same compression as in the file read. If there is no prior file, acts as None
- compression_kwargsdict, optional
If provided, set this as the compression keyword arguments for all binary blocks in the file.
- pad_blocksfloat or bool, optional
Add extra space between blocks to allow for updating of the file. If
False
(default), add no padding (always return 0). IfTrue
, add a default amount of padding of 10% If a float, it is a factor to multiple content_size by to get the new total size.- include_block_indexbool, optional
If
False
, don’t include a block index at the end of the file. (Default:True
) A block index is never written if the file has a streamed block.- versionstr, optional
Update the ASDF Standard version of this AsdfFile before writing.
- write_to(fd, all_array_storage=NotSet, all_array_compression=NotSet, compression_kwargs=NotSet, pad_blocks=False, include_block_index=True, version=None)[source]
Write the ASDF file to the given file-like object.
write_to
does not change the underlying file descriptor in theasdf.AsdfFile
object, but merely copies the content to a new file.- Parameters:
- fdstring or file-like object
May be a string path to a file, or a Python file-like object. If a string path, the file is automatically closed after writing. If not a string path, it is the caller’s responsibility to close the object.
- all_array_storagestring, optional
If provided, override the array storage type of all blocks in the file immediately before writing. Must be one of:
internal
: The default. The array data will be stored in a binary block in the same ASDF file.external
: Store the data in a binary block in a separate ASDF file.inline
: Store the data as YAML inline in the tree.
- all_array_compressionstring, optional
If provided, set the compression type on all binary blocks in the file. Must be one of:
''
orNone
: No compression.zlib
: Use zlib compression.bzp2
: Use bzip2 compression.lz4
: Use lz4 compression.input
: Use the same compression as in the file read. If there is no prior file, acts as None.
- compression_kwargsdict, optional
If provided, set this as the compression keyword arguments for all binary blocks in the file.
- pad_blocksfloat or bool, optional
Add extra space between blocks to allow for updating of the file. If
False
(default), add no padding (always return 0). IfTrue
, add a default amount of padding of 10% If a float, it is a factor to multiple content_size by to get the new total size.- include_block_indexbool, optional
If
False
, don’t include a block index at the end of the file. (Default:True
) A block index is never written if the file has a streamed block.- versionstr, optional
Update the ASDF Standard version of this AsdfFile before writing.