loki.tools.util

Functions

`as_tuple`(item[, type, length])	Force item to a tuple, even if None is provided.
`auto_post_mortem_debugger`(type, value, tb)	Exception hook that automatically attaches a debugger
`binary_insertion_sort`(items[, lt])	Sort the given list of items using binary insertion sort.
`binary_search`(items, val, start, end[, lt])	Search for the insertion position of a value in a given range of items.
`cached_func`(func)	Decorator that memoizes (caches) the result of a function
`chunks`(l, n)	Yield successive n-sized chunks from l.
`dict_override`(base, override)	Contextmanager to temporarily override a set of dictionary values.
`execute`(command[, silent])	Execute a single command within a given directory or environment
`filter_ordered`(elements[, key])	Filter elements in a list while preserving order.
`flatten`(l[, is_leaf])	Flatten a hierarchy of nested lists into a plain list.
`graphviz_present`()	Test if graphviz is present and works The import will work as long as the graphviz python wrapper is available, but the underlying binaries may be missing.
`group_by_class`(iterable, klass)	Find groups of consecutive instances of the same type with more than one element.
`is_iterable`(o)	Checks if an item is truly iterable using duck typing.
`is_subset`(a, b[, ordered, subsequent])	Check if all items in iterable `a` are contained in iterable `b`.
`optional`(condition, context_manager, *args, ...)	Apply the context manager only when a condition is fulfilled.
`replace_windowed`(iterable, group, subs)	Replace a set of consecutive elements in a larger iterable.
`set_excepthook`([hook])	Set an exception hook that is called for uncaught exceptions
`stdchannel_is_captured`(capsys)	Utility function to verify if pytest captures stdout/stderr.
`stdchannel_redirected`(stdchannel, dest_filename)	A context manager to temporarily redirect stdout or stderr
`strip_inline_comments`(source[, ...])	Strip inline comments from a source string and return the modified string.
`timeout`(time_in_s[, message])	Context manager that specifies a timeout for the code section in its body
`yaml_include_constructor`(loader, node)	Add support for `!include` tags to YAML load

Classes

`CaseInsensitiveDict`	Dict that ignores the casing of string keys.
`LazyNodeLookup`(anchor, query)	Utility class for indirect, `weakref`-style lookups
`WeakrefProperty`(*[, default, frozen])	Descriptor object that stores a weakref to the encapsulated object.

as_tuple(item, type=None, length=None): Force item to a tuple, even if None is provided.

is_iterable(o)

Checks if an item is truly iterable using duck typing.

This was added because pymbolic.primitives.Expression provide an __iter__ method that throws an exception to avoid being iterable. However, with that method defined it is identified as a collections.Iterable and thus this is a much more reliable test than isinstance(obj, collections.Iterable).

is_subset(a, b, ordered=True, subsequent=False)

Check if all items in iterable a are contained in iterable b.

Parameters:

a (iterable) – The iterable whose elements are searched in b.
b (iterable) – The iterable of which a is tested to be a subset.
ordered (bool, optional) – Require elements to appear in the same order in a and b.
subsequent (bool, optional) – If set to False, then other elements are allowed to sit in b in-between the elements of a. Only relevant when using ordered.

Returns:

True if all elements of a are found in b, False otherwise.

Return type:

bool

flatten(l, is_leaf=None)

Flatten a hierarchy of nested lists into a plain list.

Parameters:: is_leaf (callable) – Optional function that gets called for each iterable element to decide if it is to be considered as a leaf that does not need further flattening.

chunks(l, n): Yield successive n-sized chunks from l.

execute(command, silent=True, **kwargs)

Execute a single command within a given directory or environment

Parameters:

command` (str or list of str) – The command to execute
silent (bool, optional) – Suppress output by redirecting stdout/stderr (default: True)
stdout (file object, optional) – Redirect stdout to this file object (Note: silent overwrites this)
stderr (file object, optional) – Redirect stdout to this file object (Note: silent overwrites this)
cwd (str or pathlib.Path) – Directory in which to execute command (will be stringified)

class CaseInsensitiveDict

Bases: OrderedDict

Dict that ignores the casing of string keys.

Basic idea from: https://stackoverflow.com/questions/2082152/case-insensitive-dictionary

get(key, default=None): Return the value for key if key is in the dictionary, else default.

strip_inline_comments(source, comment_char='!', str_delim='"\'')

Strip inline comments from a source string and return the modified string.

Note: this does only work reliably for Fortran strings at the moment (where quotation marks are escaped by double quotes and thus the string status is kept correct automatically).

Parameters:

source (str) – the source line(s) to be stripped.
comment_char (str) – the character that marks the beginning of a comment.
str_delim (str) – one or multiple characters that are valid string delimiters.

binary_insertion_sort(items, lt=<built-in function lt>)

Sort the given list of items using binary insertion sort.

In the best case (already sorted) this has linear running time O(n) and on average and in the worst case (sorted in reverse order) a quadratic running time O(n*n).

A binary search is used to find the insertion position, which reduces the number of required comparison operations. Hence, this sorting function is particularly useful when comparisons are expensive.

Parameters:

items (list) – the items to be sorted.
lt – the “less than” comparison operator to use. Default is the standard < operator (operator.lt).

Returns:

the list of items sorted in ascending order.

This implementation was adapted from https://www.geeksforgeeks.org/binary-insertion-sort/.

cached_func(func): Decorator that memoizes (caches) the result of a function

optional(condition, context_manager, *args, **kwargs)

Apply the context manager only when a condition is fulfilled.

Based on https://stackoverflow.com/a/41251962.

Parameters:

condition (bool) – The condition that needs to be fulfilled to apply the context manager.
context_manager – The context manager to apply.

class LazyNodeLookup(anchor, query)

Bases: object

Utility class for indirect, weakref-style lookups

References to IR nodes are usually not stable as the IR may be rebuilt at any time. This class offers a way to refer to a node in an IR by encoding how it can be found instead.

Note

Example: Reference a declaration node that contains variable “a”

from loki import LazyNodeLookup, FindNodes, Declaration
# Assume this has been initialized before
# routine = ...

# Create the reference
query = lambda x: [d for d in FindNodes(VariableDeclaration).visit(x.spec) if 'a' in d.symbols][0]
decl_ref = LazyNodeLookup(routine, query)

# Use the reference (this carries out the query)
decl = decl_ref()

Parameters:

anchor – The “stable” anchor object to which query is applied to find the object. This is stored internally as a weakref.
query – A function object that accepts a single argument and should return the lookup result. To perform the lookup, query is called with anchor as argument.

property anchor

yaml_include_constructor(loader, node)

Add support for !include tags to YAML load

Activate via yaml.add_constructor("!include", yaml_include_constructor) or yaml.add_constructor("!include", yaml_include_constructor, yaml.SafeLoader) (for use with yaml.safe_load).

Adapted from JUBE2 (https://fz-juelich.de/jsc/jube) and http://code.activestate.com/recipes/577612-yaml-include-support/

This allows to include other YAML files or parts of them inside a YAML file:

# include.yml
tag0:
  foo: bar

tag1:
  baz: bar

# main.yml
nested: !include include.yml

nested_filtered: !include include.yml:["tag0"]

which is equivalent to the following:

..code-block:: yaml

nested:

tag0:
foo: bar

tag1:
baz: bar

nested_filtered:
baz: bar

auto_post_mortem_debugger(type, value, tb)

Exception hook that automatically attaches a debugger

Activate by calling set_excepthook(hook=auto_post_mortem_debugger).

Adapted from https://code.activestate.com/recipes/65287/

set_excepthook(hook=None)

Set an exception hook that is called for uncaught exceptions

This can be called with auto_post_mortem_debugger() to automatically attach a debugger (Pdb or, if installed, Pdb++) when exceptions occur.

With hook set to None, this will restore the default exception hook sys.__excepthook.

timeout(time_in_s, message=None)

Context manager that specifies a timeout for the code section in its body

This is implemented by installing a signal handler for signal.SIGALRM and scheduling that signal for time_in_s in the future. For that reason, this context manager cannot be nested.

A value of 0 for time_in_s will not install any timeout.

The following example illustrates the usage, which will result in a RuntimeError being raised.

with timeout(5):
    sleep(10)

Parameters:

time_in_s (int) – Timeout in seconds after which to interrupt the code
message (str) – A custom error message to use if a timeout occurs

class WeakrefProperty(*, default=None, frozen=False)

Bases: object

Descriptor object that stores a weakref to the encapsulated object.

group_by_class(iterable, klass)

Find groups of consecutive instances of the same type with more than one element.

Parameters:

iterable (iterable) – Input iterable from which to extract groups
klass (type) – Type by which to group elements in the given iterable

replace_windowed(iterable, group, subs)

Replace a set of consecutive elements in a larger iterable.

Parameters:

iterable (iterable) – Input iterable in which to replace elements
group (iterable) – Group of elements to replace in iterable
subs (any) – Replacement for group in iterable

dict_override(base, override)

Contextmanager to temporarily override a set of dictionary values.

Parameters:

base (dict) – The base dictionary in which to overide values
replace (dict) – Replacement mapping to temporarily insert

stdchannel_redirected(stdchannel, dest_filename)

A context manager to temporarily redirect stdout or stderr

e.g.:

with stdchannel_redirected(sys.stderr, os.devnull):
    if compiler.has_function('clock_gettime', libraries=['rt']):
        libraries.append('rt')

Source: https://stackoverflow.com/a/17753573

Note, that this only works when pytest is invoked with ‘–show-capture’ (or ‘-s’). This can be checked using stdchannel_is_captured(capsys). Additionally, capturing of sys.stdout/sys.stderr needs to be disabled explicitly, i.e., use the fixture capsys and wrap the above:

with capsys.disabled():
    with stdchannel_redirected(sys.stdout, 'stdout.log'):
        function()

stdchannel_is_captured(capsys)

Utility function to verify if pytest captures stdout/stderr.

This hinders redirecting stdout/stderr for f2py/f90wrap functions.

Parameters:: capsys – The capsys fixture of the test.
Returns:: True if pytest captures output, otherwise False.
Return type:: bool

graphviz_present(): Test if graphviz is present and works The import will work as long as the graphviz python wrapper is available, but the underlying binaries may be missing.