Module degann.networks.layers.tf_dense

Classes

class TensorflowDense (input_dim=32, units=32, activation_func: str = 'linear', weight_initializer=<tensorflow.python.ops.init_ops_v2.RandomNormal object>, bias_initializer=<tensorflow.python.ops.init_ops_v2.RandomNormal object>, is_debug=False, **kwargs)

This is the class from which all layers inherit.

A layer is a callable object that takes as input one or more tensors and that outputs one or more tensors. It involves computation, defined in the call() method, and a state (weight variables). State can be created:

• in __init__(), for instance via self.add_weight();
• in the optional build() method, which is invoked by the first __call__() to the layer, and supplies the shape(s) of the input(s), which may not have been known at initialization time.

Layers are recursively composable: If you assign a Layer instance as an attribute of another Layer, the outer layer will start tracking the weights created by the inner layer. Nested layers should be instantiated in the __init__() method or build() method.

Users will just instantiate a layer and then treat it as a callable.

Args

trainable

Boolean, whether the layer's variables should be trainable.

name

String name of the layer.

dtype

The dtype of the layer's computations and weights. Can also be a keras.DTypePolicy, which allows the computation and weight dtype to differ. Defaults to None. None means to use keras.config.dtype_policy(), which is a float32 policy unless set to different value (via keras.config.set_dtype_policy()).

Attributes

name

The name of the layer (string).

dtype

Dtype of the layer's weights. Alias of layer.variable_dtype.

variable_dtype

Dtype of the layer's weights.

compute_dtype

The dtype of the layer's computations. Layers automatically cast inputs to this dtype, which causes the computations and output to also be in this dtype. When mixed precision is used with a keras.DTypePolicy, this will be different than variable_dtype.

trainable_weights

List of variables to be included in backprop.

non_trainable_weights

List of variables that should not be included in backprop.

weights

The concatenation of the lists trainable_weights and non_trainable_weights (in this order).

trainable

Whether the layer should be trained (boolean), i.e. whether its potentially-trainable weights should be returned as part of layer.trainable_weights.

input_spec

Optional (list of) InputSpec object(s) specifying the constraints on inputs that can be accepted by the layer.

We recommend that descendants of Layer implement the following methods:

• __init__(): Defines custom layer attributes, and creates layer weights that do not depend on input shapes, using add_weight(), or other state.
• build(self, input_shape): This method can be used to create weights that depend on the shape(s) of the input(s), using add_weight(), or other state. __call__() will automatically build the layer (if it has not been built yet) by calling build().
• call(self, *args, **kwargs): Called in __call__ after making sure build() has been called. call() performs the logic of applying the layer to the input arguments. Two reserved keyword arguments you can optionally use in call() are: 1. training (boolean, whether the call is in inference mode or training mode). 2. mask (boolean tensor encoding masked timesteps in the input, used e.g. in RNN layers). A typical signature for this method is call(self, inputs), and user could optionally add training and mask if the layer need them.
• get_config(self): Returns a dictionary containing the configuration used to initialize this layer. If the keys differ from the arguments in __init__(), then override from_config(self) as well. This method is used when saving the layer or a model that contains this layer.

Examples:

Here's a basic example: a layer with two variables, w and b, that returns y = w . x + b. It shows how to implement build() and call(). Variables set as attributes of a layer are tracked as weights of the layers (in layer.weights).

class SimpleDense(Layer):
    def __init__(self, units=32):
        super().__init__()
        self.units = units

    # Create the state of the layer (weights)
    def build(self, input_shape):
        self.kernel = self.add_weight(
            shape=(input_shape[-1], self.units),
            initializer="glorot_uniform",
            trainable=True,
            name="kernel",
        )
        self.bias = self.add_weight(
            shape=(self.units,),
            initializer="zeros",
            trainable=True,
            name="bias",
        )

    # Defines the computation
    def call(self, inputs):
        return ops.matmul(inputs, self.kernel) + self.bias

# Instantiates the layer.
linear_layer = SimpleDense(4)

# This will also call `build(input_shape)` and create the weights.
y = linear_layer(ops.ones((2, 2)))
assert len(linear_layer.weights) == 2

# These weights are trainable, so they're listed in `trainable_weights`:
assert len(linear_layer.trainable_weights) == 2

Besides trainable weights, updated via backpropagation during training, layers can also have non-trainable weights. These weights are meant to be updated manually during call(). Here's a example layer that computes the running sum of its inputs:

class ComputeSum(Layer):

  def __init__(self, input_dim):
      super(ComputeSum, self).__init__()
      # Create a non-trainable weight.
      self.total = self.add_weight(
        shape=(),
        initializer="zeros",
        trainable=False,
        name="total",
      )

  def call(self, inputs):
      self.total.assign(self.total + ops.sum(inputs))
      return self.total

my_sum = ComputeSum(2)
x = ops.ones((2, 2))
y = my_sum(x)

assert my_sum.weights == [my_sum.total]
assert my_sum.non_trainable_weights == [my_sum.total]
assert my_sum.trainable_weights == []

Expand source code

class TensorflowDense(keras.layers.Layer):
    def __init__(
        self,
        input_dim=32,
        units=32,
        activation_func: str = "linear",
        weight_initializer=tf.random_normal_initializer(),
        bias_initializer=tf.random_normal_initializer(),
        is_debug=False,
        **kwargs,
    ):
        decorator_params = None

        if "decorator_params" in kwargs.keys():
            decorator_params = kwargs.get("decorator_params")
            kwargs.pop("decorator_params")

        if not isinstance(decorator_params, dict) and decorator_params is not None:
            raise "Additional parameters for activation function must be dictionary"

        if input_dim == 0 or units == 0:
            raise "Layer cannot have zero inputs or zero size"

        super(TensorflowDense, self).__init__(**kwargs)
        w_init = weight_initializer
        self.w = self.add_weight(
            shape=(input_dim, units),
            initializer=w_init,
            dtype="float32",
            # initial_value=w_init(shape=(input_dim, units), dtype="float32"),
            name=f"Var_w_{self.name}",
            trainable=True,
        )
        b_init = bias_initializer
        self.b = self.add_weight(
            shape=(units,),
            initializer=b_init,
            dtype="float32",
            # initial_value=b_init(shape=(units,), dtype="float32"),
            name=f"Var_b_{self.name}",
            trainable=True,
        )

        self.units = units
        self.input_dim = input_dim
        self._is_debug = is_debug
        self.activation_func = activations.get(activation_func)
        self.activation_name = activation_func
        self.weight_initializer = weight_initializer
        self.bias_initializer = bias_initializer
        self.decorator_params: Optional[dict] = decorator_params

    def call(self, inputs, **kwargs):
        """
        Obtaining a layer response on the input data vector
        Parameters
        ----------
        inputs
        kwargs

        Returns
        -------

        """
        if self.decorator_params is None:
            return self.activation_func(tf.matmul(inputs, self.w) + self.b)
        else:
            return self.activation_func(
                tf.matmul(inputs, self.w) + self.b, **self.decorator_params
            )

    def __str__(self):
        res = f"Layer {self.name}\n"
        res += f"weights shape = {self.w.shape}\n"
        if self._is_debug:
            # res += f"weights = {self.w.numpy()}\n"
            # res += f"biases = {self.b.numpy()}\n"
            res += f"activation = {self.activation_name}\n"
        return res

    def to_dict(self) -> dict:
        """
        Export layer to dictionary of parameters
        Returns
        -------
        config: dict
            dictionary of parameters
        """
        w = self.w.value.numpy()
        b = self.b.value.numpy()
        res = {
            LAYER_DICT_NAMES["shape"]: self.units,
            LAYER_DICT_NAMES["inp_size"]: self.input_dim,
            LAYER_DICT_NAMES["weights"]: w.tolist(),
            LAYER_DICT_NAMES["biases"]: b.tolist(),
            LAYER_DICT_NAMES["layer_type"]: type(self).__name__,
            LAYER_DICT_NAMES["dtype"]: w.dtype.name,
            LAYER_DICT_NAMES["activation"]: self.activation_name
            if self.activation_name is None
            else self.activation_name,
            LAYER_DICT_NAMES["decorator_params"]: _dec_params_to_list(
                self.decorator_params
            ),
        }

        return res

    def from_dict(self, config):
        """
        Restore layer from dictionary of parameters

        Parameters
        ----------
        config

        Returns
        -------

        """
        w = np.array(config[LAYER_DICT_NAMES["weights"]])
        b = np.array(config[LAYER_DICT_NAMES["biases"]])
        act = config[LAYER_DICT_NAMES["activation"]]
        dec_params = _dec_params_from_list(config[LAYER_DICT_NAMES["decorator_params"]])
        self.set_weights([w, b])
        # self.b = tf.Variable(
        #     initial_value=b, dtype=config[LAYER_DICT_NAMES["dtype"]], trainable=True
        # )
        self.activation_func = activations.get(act)
        self.activation_name = act
        self.decorator_params = dec_params

    @classmethod
    def from_config(cls, config):
        return cls(**config)

    @property
    def get_activation(self) -> str:
        return self.activation_name

Ancestors

• keras.src.layers.layer.Layer
• keras.src.backend.tensorflow.layer.TFLayer
• keras.src.backend.tensorflow.trackable.KerasAutoTrackable
• tensorflow.python.trackable.autotrackable.AutoTrackable
• tensorflow.python.trackable.base.Trackable
• keras.src.ops.operation.Operation
• keras.src.saving.keras_saveable.KerasSaveable

Static methods

def from_config(config)

Creates a layer from its config.

This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).

Args

config

A Python dictionary, typically the output of get_config.

Returns

A layer instance.

Expand source code

@classmethod
def from_config(cls, config):
    return cls(**config)

Instance variables

var get_activation : str

Expand source code

@property
def get_activation(self) -> str:
    return self.activation_name

Methods

def call(self, inputs, **kwargs)

Obtaining a layer response on the input data vector Parameters

---

inputs

 

kwargs

 

Returns

Expand source code

def call(self, inputs, **kwargs):
    """
    Obtaining a layer response on the input data vector
    Parameters
    ----------
    inputs
    kwargs

    Returns
    -------

    """
    if self.decorator_params is None:
        return self.activation_func(tf.matmul(inputs, self.w) + self.b)
    else:
        return self.activation_func(
            tf.matmul(inputs, self.w) + self.b, **self.decorator_params
        )

def from_dict(self, config)

Restore layer from dictionary of parameters

Parameters

config

 

Returns

Expand source code

def from_dict(self, config):
    """
    Restore layer from dictionary of parameters

    Parameters
    ----------
    config

    Returns
    -------

    """
    w = np.array(config[LAYER_DICT_NAMES["weights"]])
    b = np.array(config[LAYER_DICT_NAMES["biases"]])
    act = config[LAYER_DICT_NAMES["activation"]]
    dec_params = _dec_params_from_list(config[LAYER_DICT_NAMES["decorator_params"]])
    self.set_weights([w, b])
    # self.b = tf.Variable(
    #     initial_value=b, dtype=config[LAYER_DICT_NAMES["dtype"]], trainable=True
    # )
    self.activation_func = activations.get(act)
    self.activation_name = act
    self.decorator_params = dec_params

def to_dict(self) ‑> dict

Export layer to dictionary of parameters Returns

---

config : dict

dictionary of parameters

Expand source code

def to_dict(self) -> dict:
    """
    Export layer to dictionary of parameters
    Returns
    -------
    config: dict
        dictionary of parameters
    """
    w = self.w.value.numpy()
    b = self.b.value.numpy()
    res = {
        LAYER_DICT_NAMES["shape"]: self.units,
        LAYER_DICT_NAMES["inp_size"]: self.input_dim,
        LAYER_DICT_NAMES["weights"]: w.tolist(),
        LAYER_DICT_NAMES["biases"]: b.tolist(),
        LAYER_DICT_NAMES["layer_type"]: type(self).__name__,
        LAYER_DICT_NAMES["dtype"]: w.dtype.name,
        LAYER_DICT_NAMES["activation"]: self.activation_name
        if self.activation_name is None
        else self.activation_name,
        LAYER_DICT_NAMES["decorator_params"]: _dec_params_to_list(
            self.decorator_params
        ),
    }

    return res