auto_diff.op package

Submodules

auto_diff.op.op_constant module

class auto_diff.op.op_constant.OpConstant(x: Union[int, float, list, numpy.ndarray], **kwargs)

Bases: auto_diff.op.operation.Operation

Contains a constant.

__init__(x: Union[int, float, list, numpy.ndarray], **kwargs)

Parameters:

• x – The constant value.
• kwargs –

_get_name() → str

Get the name for display.

_get_op_name() → str

Get the name for indexing.

_forward(feed_dict: Mapping[Union[str, auto_diff.op.op_placeholder.OpPlaceholder], numpy.ndarray]) → numpy.ndarray

Returns the constant.

_backward(gradient: auto_diff.op.operation.Operation) → None

No backward operation needed.

auto_diff.op.op_flatten module

class auto_diff.op.op_flatten.OpFlatten(x: auto_diff.op.operation.Operation, **kwargs)

Bases: auto_diff.op.op_reshape.OpReshape

Flatten the tensor to 1-D array.

__init__(x: auto_diff.op.operation.Operation, **kwargs)

Initialize self. See help(type(self)) for accurate signature.

_get_name() → str

Get the name for display.

_get_op_name() → str

Get the name for indexing.

auto_diff.op.op_placeholder module

class auto_diff.op.op_placeholder.OpPlaceholder(shape: Sequence[int], **kwargs)

Bases: auto_diff.op.operation.Operation

The placeholder that represents values to be feed.

__init__(shape: Sequence[int], **kwargs)

Parameters:

• shape – Shape of the value.
• kwargs –

_get_name() → str

Get the name for display.

_get_op_name() → str

Get the name for indexing.

_forward(feed_dict: Mapping[Union[str, OpPlaceholder], numpy.ndarray])

Finds and returns the value in feed dictionary.

_backward(gradient: auto_diff.op.operation.Operation) → None

No backward operation needed.

auto_diff.op.op_reshape module

class auto_diff.op.op_reshape.OpReshape(x: auto_diff.op.operation.Operation, shape: Sequence[int], **kwargs)

Bases: auto_diff.op.operation.Operation

Reshape the tensor to a given shape.

__init__(x: auto_diff.op.operation.Operation, shape: Sequence[int], **kwargs)

Initialize self. See help(type(self)) for accurate signature.

_get_name() → str

Get the name for display.

_get_op_name() → str

Get the name for indexing.

_forward(feed_dict: Mapping[Union[str, auto_diff.op.op_placeholder.OpPlaceholder], numpy.ndarray]) → numpy.ndarray

Reshape the tensor.

_backward(gradient: auto_diff.op.operation.Operation) → None

Reshape the gradient to its old shape.

auto_diff.op.op_transpose module

class auto_diff.op.op_transpose.OpTranspose(x: auto_diff.op.operation.Operation, axes: Optional[Sequence[int]] = None, **kwargs)

Bases: auto_diff.op.operation.Operation

Transpose the tensor.

Basic operation without axes:

\[Y = X^T\]

Partial derivative of a single element:

\[\begin{split}\begin{array}{rcl} \displaystyle \frac{\partial L}{\partial x_{ij}} &=& \displaystyle \sum_{i,j} \frac{\partial L}{\partial y_{ij}} \cdot \frac{\partial y_{ij}}{\partial x_{ij}} \\ &=& \displaystyle \frac{\partial L}{\partial y_{ji}} \cdot \frac{\partial y_{ji}}{\partial x_{ij}} \\ &=& \displaystyle \frac{\partial L}{\partial y_{ji}} \cdot \frac{\partial x_{ij}}{\partial x_{ij}} \\ &=& \displaystyle \frac{\partial L}{\partial y_{ji}} \\ \end{array}\end{split}\]

Matrix derivative:

\[\frac{\partial L}{\partial X} = \left ( \frac{\partial L}{\partial Y} \right )^T\]

Generally, axes should be a permutation of the dimensions, suppose there is a function \(f\) that maps from \((0, 1, \dots, k)\) to the new permutation, then this transpose operation would be:

\[y_{i_1, i_2, \dots, i_k} = x_{f(i_1), f(i_2), \dots, f(i_k)}\]

The partial derivative of \(x_{i_1, i_2, \dots, i_k}\) is 1 only with \(y_{f(i_1)^{-1}, f(i_2)^{-1}, \dots, f(i_k)^{-1}}\). Therefore the derivative should be another transpose operation with inverse mapping function \(f^{-1}\).

__init__(x: auto_diff.op.operation.Operation, axes: Optional[Sequence[int]] = None, **kwargs)

Parameters:

• x – Input operation.
• axes – A permutation of dimensions. The dimensions will be reversed if it is None.
• kwargs – Arguments for parent.

_get_name() → str

Get the name for display.

_get_op_name() → str

Get the name for indexing.

_forward(feed_dict: Mapping[Union[str, auto_diff.op.op_placeholder.OpPlaceholder], numpy.ndarray]) → numpy.ndarray

Transpose the tensor.

_backward(gradient: auto_diff.op.operation.Operation) → None

Transpose the gradients to its old shape.

auto_diff.op.op_variable module

class auto_diff.op.op_variable.OpVariable(initializer: Union[Callable, int, float, list, numpy.ndarray], shape: tuple = None, **kwargs)

Bases: auto_diff.op.operation.Operation

Contains weights that could be updated.

__init__(initializer: Union[Callable, int, float, list, numpy.ndarray], shape: tuple = None, **kwargs)

Parameters:

• initializer – A function that accepts shape as its arguments or a numpy array.
• shape – Must be provided if the initializer is a function.
• kwargs –

update(value: Union[int, float, list, numpy.ndarray]) → None

_get_name() → str

Get the name for display.

_get_op_name() → str

Get the name for indexing.

_forward(feed_dict: Mapping[Union[str, auto_diff.op.op_placeholder.OpPlaceholder], numpy.ndarray]) → numpy.ndarray

Returns the contained weights.

_backward(gradient: auto_diff.op.operation.Operation) → None

No backward operation needed.

auto_diff.op.operation module

class auto_diff.op.operation.Operation(**kwargs)

Bases: object

Abstract operation for building computing graph.

__op_counter = None

The counter for giving each operation a unique index.

__op_collection = None

Collection of existing operations.

STEP_KEY = '__step__'

The key for extracting step information from session.

__init__(**kwargs)

Initialize self. See help(type(self)) for accurate signature.

_get_name() → str

Get the name for display.

_get_op_name() → str

Get the name for indexing.

dim

isscalar() → bool

forward(feed_dict: Mapping[Union[str, Operation], numpy.ndarray] = None) → numpy.ndarray

Do the calculations to get the output of the operations.

Parameters:feed_dict – Contains the real values of placeholders, see OpPlaceholder. Returns:A numpy array.

_forward(feed_dict: Mapping[Union[str, Operation], numpy.ndarray]) → numpy.ndarray

Forward operation to be implemented.

backward(gradient: Optional[auto_diff.op.operation.Operation] = None) → None

Update gradients recursively.

Parameters:gradient – Current gradient.

_backward(gradient: auto_diff.op.operation.Operation) → None

Backward operation to be implemented.

_broadcast_shape(x: auto_diff.op.operation.Operation, y: auto_diff.op.operation.Operation)

_broadcast_backward(gradient: auto_diff.op.operation.Operation)

transpose(axes: Optional[Sequence[int]] = None, **kwargs) → auto_diff.op.operation.Operation

See OpTranspose.

reshape(shape: Sequence[int], **kwargs) → auto_diff.op.operation.Operation

See OpReshape.

flatten(**kwargs) → auto_diff.op.operation.Operation

See OpFlatten.

expand_dims(axis: Optional[int] = None, **kwargs) → auto_diff.op.operation.Operation

See OpExpandDims.

squeeze(axis=None, **kwargs) → auto_diff.op.operation.Operation

See OpSqueeze.

sum(axis: Union[int, Sequence[int], None] = None, keepdims: bool = False, **kwargs) → auto_diff.op.operation.Operation

See OpSum.

dot(x: auto_diff.op.operation.Operation, **kwargs) → auto_diff.op.operation.Operation

See OpDot.

simplify() → auto_diff.op.operation.Operation

_Operation__op_collection = {}

_Operation__op_counter = [0]

Module contents