TensorFlow and Keras-2

 Q1. Sigmoid and softmax functions

Which of the following statements is true for a neural network having more than one output neuron ?

Choose the correct answer from below:

A.     In a neural network where the output neurons have the sigmoid activation, the sum of all the outputs from the neurons is always 1.

B.     In a neural network where the output neurons have the sigmoid activation, the sum of all the outputs from the neurons is 1 if and only if we have just two output neurons.

C.     In a neural network where the output neurons have the softmax activation, the sum of all the outputs from the neurons is always 1.

D.     The softmax function is a special case of the sigmoid function

Ans: C

Correct option : In a neural network where the output neurons have the softmax activation, the sum of all the outputs from the neurons is always 1.

Explanation :

• For the sigmoid activation, when we have more than one neuron, it is possible to have the sum of outputs from the neurons to have any value.

• The softmax classifier outputs the probability distribution for each class, and the sum of the probabilities is always 1.

• The Sigmoid function is the special case of the Softmax function where the number of classes is 2.

Q2. Type of loss

We want to classify credit card transactions as fraudulent or normal, which loss type is appropriate for this use case?

 

Choose the correct answer from below, please note that this question may have multiple correct answers

A.     Categorical crossentropy

B.     Binary crossentropy

C.     Adam

D.     SGD

Ans: A, B

Correct Option:

• Categorical crossentropy
• Binary crossentropy

Explanation:
If you have one neuron at the end in the classification NN, then you need to use sigmoid. In that case, it will be binary cross-entropy. If you take 2 neurons at the end, then you have to one-hot encode the target and then you need to use softmax with CCE.

 

Q3. Callbacks in tensorflow

Which method gets called after each epoch in tensorflow callback?

Choose the correct answer from below:

A.     on_epoch_end

B.     on_epoch_finished

C.     on_end

D.     on_training_complete

Ans: A

Correct Option: on_epoch_end

Explanation:

• tensorflow callback method on_epoch_end contain functionalities that can be called at the end of each epoch.
• tf.keras.callbacks.Callback can be inherited by custom classes in which methods like on_train_begin, on_epoch_begin can be

Q4. Avoid overfitting

Jack was asked to create a classifier for a two-class non-linearly separable dataset consisting of 100 observations. He did not know the complexity of the non-linearity of separation therefore he created a model with 500 nodes in the only hidden layer with ReLU activation and used sigmoid in the output layer.

Jack was aware that his model can overfit the data so he implemented a function that can stop the training as soon as the model starts overfitting.

from keras.callbacks import EarlyStopping
es = EarlyStopping(monitor = 'val_loss',
 min_delta = 0,
 patience = 3,
 restore_best_weights = True)

Now Ryan also wanted to implement such a function and he made the observations given in the options. Which of Ryan's observation(s) are incorrect?

 

Choose the correct answer from below, please note that this question may have multiple correct answers

 

A.     The training process will be monitored according to the validation loss.

B.     The training process will stop as soon as the difference between validation loss of two consecutive epochs is greater than 0.

C.     The training process will be stopped if there are more than 3 epochs having val_loss smaller than the latest minimum val_loss value.

D.     The best model weights according to val_loss, will be saved after training.

 

Ans: B, C

Correct options :

• The training process will stop if the difference between validation loss of two consecutive epochs is greater than 0.

• The training process will be stopped if there are more than 3 epochs having val_loss smaller than the latest minimum val_loss value.

Explanation :

• Actually min_delta is set to 0 implying that if the val_loss decreases by any value greater than 0 it will be counted as an improvement.

• Actually patience is set to 3 implying that the training will be stopped if there are more than three consecutive epochs with no improvement according to min_delta. (i.e. all val_loss were increasing) Two epochs are said to be improving if the monitored value improves (i.e. here val_loss decreases by a value greater than equal to min_delta).

• Monitor: The 'monitor' argument takes the value or the metric based on which the training is evaluated.

• min_delta: The 'min_delta' argument takes the value which represents the absolute minimum difference between the monitored value in two consecutive epochs for which the training is stopped, that is the minimum change required to qualify as an improvement.

• patience: The 'patience' argument takes the maximum number of epochs for which no improvement was made.

• restore_best_weights: The 'restore_best_weights' parameter is set to true if we want to save the best model during training process according to the monitored metric.

Q5. Adding callbacks

We are trying to train a model on a training dataset for 20 epochs.

model.fit(x_train, y_train, epochs=20,callbacks = callback)

Add callbacks to the above model based on the conditions given below:

Cond1. If the validation accuracy at an epoch is less than the previous epoch's accuracy, we have to decrease the learning rate by 10%.

The options for Cond1 are:

a. reduce_lr = ReduceLROnPlateau(monitor='val_acc', factor=0.9,
                              patience=1)
   callback=[reduce_lr]

b. reduce_lr = ReduceLROnPlateau(monitor='val_acc', factor=0.9,
                              patience=0)
   callback=[reduce_lr]

Cond2. For every 3rd epoch, decay the learning rate by 5%.

The options for Cond2 are:

c.  def step_decay(epoch):
      initial_lrate = 0.1
      drop = 0.95
      epochs_drop = 3
      lrate = initial_lrate * math.pow(drop,math.floor((epoch)/epochs_drop))
      return lrate

   lrate = LearningRateScheduler(step_decay)
   callback = [lrate]

d. initial_learning_rate = 0.1

   lr_schedule = tf.keras.optimizers.schedules.ExponentialDecay(
   initial_learning_rate,
   decay_steps = 3,
   decay_rate = 0.95,
   staircase = True)

   model.compile(optimizer = tf.keras.optimizers.SGD(learning_rate = lr_schedule),
              loss ='sparse_categorical_crossentropy',
              metrics = ['accuracy'])

Which of the above options will be correct for requirements in Cond1 and Cond2?

Choose the correct answer from below:

A.     a, c and d

B.     b, c and d

C.     a, b and c

D.     a, b and d

Ans: B

Correct option : b, c and d

Explanation :

• If we set the patience = 1, the model will wait once more to get a lower accuracy again before decreasing the learning rate. Therefore setting the patience = 0, will decrease the learning rate as soon as the accuracy drops.
• If patience=1, would have been applied to metric `loss` then the model would have waited one more time to get a higher loss than the minimum encountered loss before decreasing the learning rate. 

• Both c and d can be used for updating the learning rate using optimizer and callbacks respectively.