Classes and decorator¶
Class definitions supporting the computation graphs.
-
class
paragraph.types.
Variable
(name=None, op=None, args=NOTHING, dependencies=NOTHING)¶ Bases:
object
A generic variable.
This class is the return type of all operations in a computation graph. Independent variables can be instantiated directly as follows:
>>> my_var = Variable(name="my_var")
Note
Type annotations are missing for attributes op and args, for Sphinx does not seem to cope with forward references.
-
name
¶ a string used to represent the variable. The attribute is mandatory for independent variables and None (the default) otherwise.
-
op
¶ the operation producing the variable, of type Op.
-
args
¶ a dictionary mapping arguments of the above op onto their values, of type Dict[Variable, Any].
-
dependencies
¶ a dictionary mapping arguments of the above callable onto other variables, of type Dict[str, Variable].
-
isinput
()¶ Return type: bool
-
isdependent
()¶ Return type: bool
-
-
class
paragraph.types.
Requirement
¶ Bases:
abc.ABC
Base class for defining a requirement mixin
A Requirement class defines one or several attributes storing the actual requirement being expressed, and takes complete responsibility of these attributes. Therefore, a concrete requirement class should:
- define default values or factory callbacks for all the attributes it introduces,
- redefine the
merge()
method below appropriately, respecting the prescribed constraints, - expose convenience methods to manipulate their attributes, these methods should operate in place to avoid unwanted interference with other Requirement mixins,
- provide an implementation of
__deepcopy__()
whenever relevant .
Deriving a compound requirements class boils down to putting together requirement mixins:
>>> @attr.s >>> class MyRequirements(DateRangeRequirement, DatasetContentsRequirement): ... pass
assuming DateRangeRequirement and DatasetContentsRequirement derive from the present class.
-
merge
(other)¶ Merges other into self in-place, must be implemented by all cooperating mixin classes.
The requirement mixins should define the
merge()
method for the attributes they define (and therefore take responsibility for). The computation implemented must be both:- commutative
- the result of [self.merge(usg) for usg in usages[self]] does not depend on the order of the usages in the list
- idempotent
- if usg1 = usg2, invoking self.update(usg2) after self.update(usg1) leaves self unchanged
Only under these conditions is it guaranteed that all backward traversals of the computation graph result in the same input requirements.
In addition, every implementation of this method must be cooperative and end with the following line:
super().merge(other)
Parameters: other – the requirement to merge into self, must be of the same concrete type as self.
-
new
()¶ Return an empty requirement of the same type as self.
-
class
paragraph.types.
Op
(*, thread_safe=True)¶ Bases:
object
Class of all computation graph operations.
A concrete Op class should redefine the
_run()
method, which fully specifies its behavior. Calling an Operation instance results in the following:- if no argument is of Variable type, the return value of the
_run()
method is returned, - otherwise, a Variable is returned, which represents the result of the operation applied to its arguments.
Additional requirements can be introduced for variable arguments, globally or individually, by redefining the
_arg_requirements()
method appropriately.-
thread_safe
¶ If False, the op is always executed in the main thread. Defaults to True.
-
static
split_args
(args)¶ Separate positional from keyword arguments in a dict.
This method extracts args entries with a key of type integer, and collates them into a list
Return type: Tuple
[List
[Any
],Dict
[str
,Any
]]
-
arg_requirements
(req, arg=None)¶ Compute the requirements on the input value for argument arg from the requirements req bearing on the output variable.
The base implementation below returns an empty requirement for every argument, preserving the type. Concrete classes should redefine this method whenever applicable.
Warning
This method should never update requirements in-place, as this would result in adversary side-effects. The recommended practice is for this method to: - instantiate a new Requirement instance or deep-copy the passed in req if information should be retained, - update the new instance using in-place methods exposed by the requirement class, and return it.
Should the output requirement be returned unmodified, it is however safe to just return its reference as-is.
Return type: Requirement
-
op
(*args, **kwargs)¶ Define a variable as the result of applying the op to the arguments provided.
- While this method always returns a Variable instance, it accepts:
- _concrete_ arguments of the type expected by
_run
at the same position/for the same keyword, - Variable instances that resolve to a value of the expected type.
- _concrete_ arguments of the type expected by
Return type: Variable
- if no argument is of Variable type, the return value of the
-
paragraph.types.
op
(func)¶ Wraps a function within an Op object.
The returned function accepts arguments of type Variable everywhere in its signature, in addition to the types accepted by the decorated function. In presence of such arguments, it returns a Variable object symbolizing the result of the operation with the arguments passed in. In absence of variable arguments, the function returned is just equivalent to the function passed in, in particular it returns a value of the pristine return type.
Warning
Operations returned by this decorator are marked thread-safe by default. It is the user’s responsibility to set Op.thread_safe to False where appropriate.
Parameters: func ( Callable
) – the function to transform into an OpReturn type: Op