Category ""
ADVERTISING
This function returns two items: the activation value “a” and a “cache” that contains “Z” (it’s what we will feed in to the corresponding backward function).
We calculate RELU function for L-1 layers and Sigmoid function for the Lth layer. We store Z as a cache to be used in while calculating gradients in the backward propagation. Compute the loss. Implement the backward propagation module to compute the gradients of activation function and parameters.
# GRADED FUNCTION: linear_backward def linear_backward(dZ, cache): """ Implement the linear portion of backward propagation for a single layer (layer l) Arguments: dZ -- Gradient of the cost with respect to the linear output (of current layer l) cache -- tuple of values (A_prev, W, b) coming from the forward propagation in the current layer ...
The function first calls linear_forward to obtain the logits z and the linear cache. Then, depending on the specified activation function, it calls either sigmoid_activation or relu_activation to compute the activation matrix a_next and the activation cache.
If you do so, you will find that Z is stored in cache. Now, coming back to the linear_activation_forward function, you can see that the former one (containing A, W and b) is the linear_cache and the later one (containing Z) is the activation_cache, both of which are stored as a tuple in cache.
cache - a python dictionary containing "linear_cache" and "activation_cache" stored efficiently for computing the backward pass. def linear_activation_forward(A_prev, W, b, activation): if activation == "sigmoid": # Inputs: "A_prev, W, b". Outputs: "A, activation_cache". Z, linear_cache = linear_forward(A_prev,W,b)
Z,Linear_cache = linear_forward (W,A_prev,b) A,activation_cache = sigmoid (Z) cache = (Linear_cache, activation_cache) return A, cache def L_model_forward (X, parameters): """ Implement forward propagation for the [LINEAR->RELU]* (L-1)->LINEAR->SIGMOID computation Arguments: X -- data, numpy array of shape (input size, number of examples)
First, in the lib.rs file we will define two structs - LinearCache and ActivationCache. pub a: Array2, pub w: Array2, pub b: Array2, pub z: Array2, The LinearCache struct...
Z, linear_cache = linear_forward (A_prev, W, b) A, activation_cache = sigmoid (Z) elif activation == "relu": # Inputs: "A_prev, W, b". Outputs: "A, activation_cache".
linear_activation_forward () produces the next cache – a length -2 tuple. For a single layer it calls linear_forward () to produce (Z, linear cache) and (A, activation_cache) and then, cache = (linear_cache, activation_cache).