UNIT V
Transfer Learning
Deep Learning: UNIT 4 : Autoencoders
UNIT IV
Autoencoders
1. Denoising
Autoencoders
2. Sparse
Autoencoders
3. Deep
Autoencoders
4. Variational
Autoencoders
5. GANS
Deep Learning : UNIT 5: Transfer Learning Notes
UNIT V
Transfer Learning
1. Types
2. Methodologies
3. Diving into Transfer Learning
4. Challenges
Reference: Dipanjan Sarkar, Raghav Bali, “Transfer Learning in Action”, Manning Publications, 2021
Deep Learning: UNIT-4 : Autoencoders Notes
UNIT IV
Autoencoders
1. Denoising Autoencoders
2. Sparse Autoencoders
3. Deep Autoencoders
4. Variational Autoencoders
5. GANS
Reference: Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow Concepts, Tools, and Techniques to Build Intelligent Systems, Aurélien Géron, SECOND EDITION, O’Reilly
Deep Learning UNIT- 4 Autoencoders PPTs
UNIT IV
Autoencoders
1. Denoising Autoencoders2. Sparse Autoencoders3. Deep Autoencoders4. Variational Autoencoders5. GANS
Reference: Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow Concepts, Tools, and Techniques to Build Intelligent Systems, Aurélien Géron, SECOND EDITION, O’Reilly
NN : Forward and Back Propagation MCQs & Program
NN : Forward and Back Propagation
Q1. Sigmoid and
softmax functions
Which of the following
statements is true for a neural network having more than one output neuron ?
Choose the correct answer
from below:
A. In a neural network where the output neurons have the sigmoid activation, the sum of all the outputs from the neurons is always 1.
B. In a neural network where the output neurons have the sigmoid activation, the sum of all the outputs from the neurons is 1 if and only if we have just two output neurons.
C. In a neural network where the output neurons have the softmax activation, the sum of all the outputs from the neurons is always 1.
D. The softmax function is a special case of the sigmoid function
Ans: C
Correct option : In
a neural network where the output neurons have the softmax activation, the sum
of all the outputs from the neurons is always 1.
Explanation :
- For the sigmoid activation, when we
have more than one neuron, it is possible to have the sum of outputs from
the neurons to have any value.
- The softmax classifier outputs the
probability distribution for each class, and the sum of the probabilities
is always 1.
- The Sigmoid function is
the special case of the Softmax function where the number
of classes is 2.
Q2. Forward propagation
Given the independent and
dependent variables in X and y, complete the code
to calculate the results of the forward propagation for
a single neuron on each observation of the dataset.
The code should print the
calculated labels for each observation of the given dataset i.e. X.
Input Format:
Two lists are taken as
the inputs. First list should be the independent variable(X) and the second
list should be the dependent variable(y)
Output Format:
A numpy array consisting
of labels for each observation.
Sample Input:
X
= [[100, 129, 157, 133], [168, 150, 30, 19], [4, 148, 106, 74], [123, 195, 60,
93], [169, 40, 188, 179], [40, 59, 29, 94], [165, 126, 16, 99], [167, 157, 65,
23], [128, 87, 37, 111], [191, 154, 89, 134], [101, 41, 145, 112], [43, 110,
197, 118], [147, 22, 109, 139], [11, 161, 135, 119], [26, 48, 199, 182], [96,
100, 82, 87], [149, 2, 8, 10], [5, 38, 166, 100], [193, 117, 59, 164], [133, 5,
38, 163], [88, 177, 84, 114], [9, 132, 177, 24], [94, 130, 83, 131], [77, 11,
141, 81], [154, 198, 175, 98], [21, 148, 170, 122], [185, 145, 101, 183], [100,
196, 111, 11], [97, 147, 112, 11], [25, 97, 95, 45], [6, 89, 88, 38], [51, 16,
151, 3], [90, 174, 122, 157], [2, 133, 121, 199], [15, 78, 163, 180], [103,
118, 7, 179], [102, 179, 157, 183], [113, 139, 195, 122], [55, 88, 68, 117],
[115, 185, 93, 102], [139, 82, 3, 165], [135, 29, 78, 11], [11, 16, 60, 123],
[103, 191, 187, 129], [146, 181, 28, 192], [85, 73, 136, 139], [117, 179, 81,
183], [15, 131, 106, 28], [58, 78, 111, 65], [76, 11, 25, 103], [11, 90, 162,
129], [144, 1, 16, 33], [33, 172, 40, 72], [106, 83, 160, 151], [68, 159, 150,
64], [31, 79, 83, 15], [51, 140, 173, 10], [105, 80, 70, 21], [195, 80, 64,
129], [50, 96, 107, 82], [185, 150, 15, 143], [28, 71, 27, 57], [58, 13, 146,
78], [20, 71, 183, 44], [91, 44, 15, 87], [77, 157, 95, 110], [132, 28, 193,
49], [177, 87, 57, 41], [194, 175, 17, 20], [166, 64, 134, 150], [79, 74, 162,
168], [166, 149, 34, 117], [160, 170, 127, 44], [99, 41, 103, 155], [48, 127,
138, 68], [17, 3, 101, 94], [29, 102, 123, 158], [194, 60, 135, 179], [73, 192,
145, 168], [21, 94, 154, 143], [17, 10, 145, 131], [73, 29, 195, 199], [132,
189, 90, 100], [134, 32, 81, 119], [118, 37, 119, 27], [51, 78, 187, 86], [95,
8, 56, 29], [156, 162, 186, 127], [126, 111, 144, 59], [7, 140, 32, 75], [40,
0, 109, 92], [165, 175, 61, 103], [178, 68, 185, 119], [132, 105, 36, 80],
[165, 117, 35, 176], [128, 49, 185, 9], [50, 176, 12, 198], [124, 164, 99,
102], [36, 30, 114, 147], [166, 172, 35, 14]]
y = [1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 1,
1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 1,
0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1,
1, 0, 0, 0, 0, 1, 1, 1, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 1, 0]
Sample Output:
[1 1 1 1 1 1 1 1 1 1 1 1
0 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 0 1 1 1 1 1 1 0 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 0 1 1 1 1 0 0 1 1 1 1 1 1 1 1 0 1 1 1
1 1 1 1 0 1]
Fill the Missing Code
import
numpy as np
np.random.seed(2)
#independent variables
X
= np.array(eval(input()))
#dependent
variable
y
= np.array(eval(input()))
m = X.shape[__] #no. of samples
n
= X.shape[__] #no. of features
c
= #no. of classes in the data and therefore no. of neurons in the
layer
#weight
vector of dimension (number of features, number of neurons in the layer)
w
= np.random.randn(___, ___)
#bias
vector of dimension (1, number of neurons in the layer)
b
= np.zeros((___, ___))
#(weighted
sum + bias) of dimension (number of samples, number of classes)
z
= ____
#exponential transformation of z
a
= np.exp(z)
#Perform the softmax on a
a
= ____
#calculate the label for each observation
y_hat
= ____
print(y_hat)
Final Code:
import
numpy as np
np.random.seed(2)
#independent variables
X
= np.array(eval(input()))
#dependent
variable
y
= np.array(eval(input()))
#‘m’
and ‘n’ refers to the no. of rows and columns in the dataset respectively.’c’
refers to the number of classes in y.
m = X.shape[0] #no. of samples
n
= X.shape[1] #no. of features
c
= len(np.unique(y)) #no. of classes in the data and therefore no. of
neurons in the layer
#Initializing
weights randomly
#weight
vector of dimension (number of features, number of neurons in the layer)
w
= np.random.randn(n, c)
#Initializing
biases as zero
#bias vector of dimension (1, number of neurons in the layer)
b
= np.zeros((1, c))
#Finding
the output ‘z’
#(weighted sum + bias) of dimension (number of samples, number of classes)
z
= np.dot(X, w) + b
#Applying
the softmax activation function on the output
#exponential transformation of z
a
= np.exp(z)
a
= a/np.sum(a, axis = 1, keepdims = True)
#Calculating
the label for each observation
y_hat = np.argmax(a, axis = 1)
print(y_hat)
Q3. Same layer still
different output
Why do two neurons in the
same layer produce different outputs even after using the same kind of function
(i.e. wT.x + b)?
Choose the correct answer
from below:
A. Because the weights are not the same for the neurons.
B. Because the input for each neuron is different.
C. Because weights of all neurons are updated using different learning rates.
D. Because only biases (b) of all neurons are different, not the weights.
Ans: A
Correct option: Because
the weights are not the same for the neurons.
Explanation :
- The weights for each neuron in a
layer are different. Thus the output of each neuron ( wT.x + b ) will be
different
- The input for each neuron in a layer
is the same. In a fully connected network, each neuron in a particular
layer gets inputs from each neuron in the previous layer.
- There may be different learning rates
for each model weight depending on the type of optimizer used, but that is
not the reason for the neurons to give different outputs.
- In a fully connected network, each
neuron has two trainable parameters : a bias and a weight. The values of
bias and weight for any two neurons in a layer need not be the same, since
they keep changing during the model training.
Q4. Will he watch
the movie?
We want to predict
whether a user would watch a movie or not. Each movie has a certain number of
features, each of which is explained in the image.
Now take the case of the
movie Avatar having the features vector as [9,1,0,5].
According to an algorithm, these features are assigned the weights [0.8,0.2,0.5,0.4] and bias=-10.
For a user X, predict whether he will watch the movie or not if the threshold
value(θ) is 10?
Note:
If the output of the neuron is greater than θ then the user will watch the
movie otherwise not.
Choose the correct answer
from below:
A. Yes, the user will watch the movie with neuron output = -0.6
B. No, the user will not watch the movie with neuron output = -0.6
C. No, the user will watch the movie with neuron output = 2.5
D. Yes, the user will watch the movie with neuron output = 2.5
Ans: B
Correct option :No,
the user will not watch the movie with neuron output with -0.6
Explanation :
The output of a neuron is
obtained by taking the weighted sum of inputs and adding the bias term to it.
The output of neuron is :
(0.8).(9)+(0.2).(1)+(0).(0.5)+(5).(0.4)−10 =−0.6<10,
therefore he won’t watch the movie.
Q5. Dance festival
The dance festival is in
this coming weekend and you like dancing as everyone does. You want to decide
whether to go to the festival or not. And the decision depends on the stated
three factors: ( x1, x2, x3 are boolean variables )
- x1: Is the
weather good? (x1=1 means good)
- x2: Will your
friend accompany you? (x2=1 means she will accompany)
- x3: Is the
festival near public transit? (x3=1 means ‘Yes’)
All of the above variables/factors are binary. You are using the following
rules for deciding whether to go or not (1 means you will go):
Output:
0:
if the ∑inwixi<threshold
1: if the ∑inwixi≥threshold}
Which of the given
options will be the values of the correct weights for deciding with threshold=5
using the above rules, given the condition that you won't go unless the weather
is good, and will definitely go if the weather is good?
Choose the correct answer
from below:
A. [6,2,2]
B. [4,2,4]
C. [3,2,1]
D. [4,2,2]
Ans: A
Correct option : [6,2,2]
Explanation :
We can try substituting
the values of weights for different values of variables:
a. x1=1,x2=0,x3=0:Weightedsum=6x1+2x0+2x0=6>5 , therefore Yes, you will go.
b. x1=0,x2=1,x3=1:Weightedsum=6x0+2x1+2x1=4<5 therefore No, you won’t go.
c. x1=1,x2=0,x3=1:Weightedsum=6x1+2x0+2x1=8>5, therefore Yes
According to the above
calculations weights [6,2,2] give the required outcomes for the necessary
condition of good weather (x1=1).
It gives a value greater than the threshold if x1=1, otherwise a smaller value
irrespective of the values of x2 and x3.
Other options don’t give
these desired outputs
Q6. Value of weight
While training a simple
neural network, this is what we get with the given input:
If the neural network is
defined as:
then what value of W will
satisfy the actual output values given that bias equals
to 1.
Choose the correct answer
from below:
A. 2
B. 5
C. 10
D. 7
Ans: A
Correct option : 2
Explanation:
- This problem can be solved by simple
substitution in equations. It is given that bias is equals to 1. And W is
same for all of them.
- We want to find the value W which
gives the results as the actual output given in the table.
Hence, for input = 1,
- w×1+1=3, which
gives the value of W = 2
Similarly for input
as 2, we will get
- w×2+1=5, again
the value of W = 2
And finally when input
= 4,
- w×4+1=9, again
the value of W = 2
Therefore 2 is the
correct answer
Q7. Vectorize
Consider the following
code snippet:
How do you vectorize
this?
Note:
All x, y and z are NumPy arrays.
Choose the correct answer
from below:
A. z = x + y
B. z= x * y.T
C. z = x + y.T
D. z = x.T + y.T
Ans: C
Correct option
: z=x+y.T
Explanation :
The shape of x is (10,6)
The shape of y is (6,1)
We can observe from the
question that all elements of y are added element-wise to z, while y[j] is
added to each element in the jth column in z.
For this to happen, we will have to convert the y array to shape (1,6) and then
add it to x, which will result in broadcasting of y into shape (10,6) .
Thus the answer is
z=x+y.T
Q8. Identify the
Function
Mark the correct option
for the below-mentioned statements:
(a) It
is possible for a perceptron that it adds up all the weighted inputs it
receives, and if the sum exceeds a specific value, it outputs a 1. Otherwise,
it just outputs a 0.
(b) Both
artificial and biological neural networks learn from past experiences.
Choose the correct answer
from below:
A. Both the mentioned statements are true.
B. Both the mentioned statements are false.
C. Only statement (a) is true.
D. Only statement (b) is true.
Ans: A
Correct option:
Both the statements are true.
Explanation :
Implementation of
statement (a) is called step function and yes it is possible.
Both of artificial and
biological neural networks learn from past experiences.
The artificial networks are trained on data to make predictions. The weights
assigned to each neuron continuously changes during the training process to reduce the error.
Q9. Find the Value
of 'a'
Given below is a neural
network with one neuron that takes two float numbers as inputs.
If the model uses the
sigmoid activation function, What will be the value of 'a' for the given
x1 and x2 _____(rounded off to 2 decimal places)?
Choose the correct answer
from below:
A. 0.57
B. 0.22
C. 0.94
D. 0.75
Ans: A
Correct option :
- 0.57
Explanation :
The value of z will
be :
- z= w1.x1+w2.x2+b
- z = (0.5×0.55) + (−0.35×0.45) + 0.15 = 0.2675
The value of a will
be :
- a= f(z) = σ(0.2675) = 1/(1+e(−z))=1.7652901=0.5664=0.57
NN: Introduction to Neural Network MCQs
NN : Introduction to Neural Network
Q1. Weights
impact
For
the neural network shown above, which of these statements is true?
Choose
the correct answer from below:
A. -5 weight is bad for the neural network.
B. The neuron with weight 10 will have the most impact on the output.
C. The neuron with weight -5 will have the most impact on the output.
D. The neuron with weight 2 will have the most impact on the output.
Ans:
B
Correct
option : The neuron with weight 10 will have the most
impact on the output.
Explanation :
There is no such thing that a neuron with a negative weight will be bad for the
output. The negative or positive weight of a neuron simply means whether it has
an increasing or decreasing effect on the output value. A neuron with the
largest magnitude will have the most significant effect on the output value.
Q2. Calculate
Forward Pass
The
neuron n has the weights 1,2,3,4, and 5. The
values of inputs are 4,10,5,20, and 0. We are using a linear
activation function with the constant of proportionality being equal to 2 here.
The
output will be:
Choose
the correct answer from below:
A. 193.5
B. 59.5
C. 238
D. 119
Ans:
C
Correct
option : 238
Explanation
:
Multiplying
weights with their corresponging inputs, and then adding everything together:
-> Output=(1∗4+2∗10+3∗5+4∗20+5∗0)=119
But
since the output is linear with a proportionality constant 2, hence :
-> Finaloutput=2∗119=238
Q3. Need
for NN
Are
classic ML Algorithms not powerful enough? Why exactly do we need to use Neural
Networks?
Check
all that apply.
Choose
the correct answer from below, please note that this question may have
multiple correct answers
A. Unlike Classic ML Algos, NN does not require us to do manual feature engineering.
B. NNs can work with both structured and unstructured data
C. For a large dataset, classic ML Algos can outperform NNs
D. NNs are able to work better with sparse data
Ans:
A, B, D
Correct
Options:-
- Unlike Classic ML Algos, NN does not
require us to do manual feature engineering.
- NNs can work with both structured and
unstructured data
- NNs are able to work better with
sparse data
Explanation:-
- In order to create a complex decision
boundary, classic ML Algorithms require us to do heavy feature
engineering, manually.
On the other hand, NN are able to find complex relations between features on their own. - NN are excellent in working with
unstructured data (image / text / audio data), whereas classic ML algos
are unable to handle this data.
- The performance of NN might be
comparable to that of classic ML Algos for small datasets.
However, if given a big enough dataset, NNs will always give better performance. - NNs work better with sparse data.
Q4. Scale
drives DL
Refer
to the given plot.
Which
of the following generally does not hurts an algorithm's performance, and may
help significantly?
Choose
the correct answer from below, please note that this question may have multiple
correct answers
A. Decreasing the size of a NN
B. Increasing the size of a NN
C. Decreasing the training set size
D. Increasing the training set size
Ans:
B, D
Correct
Options:-
- Increasing the size of a NN
- Increasing the training set size
Explanation:-
- According to the trends in the given
figure, big networks usually performs better than small networks.
- Also, bringing more data to a NN
model is almost always beneficial.
Q5. Factors
of DL performance
Which
of the following factors can help achieve high performance with Deep Learning
algorithms?
Choose
the correct answer from below, please note that this question may have
multiple correct answers
A. Large amount of data
B. Smaller models
C. Better designed features to use
D. Large models
Ans:
A, D
Correct
Options:-
- Large amount of data
- Large models
Explanation:-
- Over the last 20 years, we have
accumulated a lot of data. Traditional algorithms were not able to benefit
from this. Whereas, this large amount of data has been the fundamental
reason why DL took off in the past decade.
- In order to take advantage of this
large amount of data available to us, we need a big enough model also.
- Smaller models will not be able to
yeild very high performace, as they will not be able to take advantage of
the large amount of data.
- One main difference between classical
ML algos and DL algos is that DL models are able to “figure out” the
best features using hidden layers.
Q6. NN
true false
Mark
the following statement as true or false:-
"Neural
networks are good at figuring out functions, relating an input x to
an output y, given enough examples."
Choose
the correct answer from below:
A. True
B. False
Ans:
A
Correct
Option: True
Explanation:
- With NN, we don’t need to design
features by ourselves.
- The NN figures out the necessary
relations given enough data.
Q7. Why
NN
Why
might Neural Networks be preferred over Classic ML Algorithms?
B. NNs can work with both structured and unstructured data
C. For a large dataset, classic ML Algos can always outperform NNs
Choose
the correct answer from below:
A. Only Option A is correct
B. Only Options A and B are correct
C. Only Options B and C are correct
D. All the give options are correct
Ans:
A, B
Correct
answer: Only Options A and B are correct
Explanation:
- In order to create a complex decision
boundary, classic ML Algorithms require us to do heavy feature
engineering, manually. On the other hand, NN are able to find complex
relations between features on their own.
- NN are excellent in working with
unstructured data (image / text / audio data), whereas classic ML algos
are unable to handle this data
- The performance of NN might be
comparable to that of classic ML Algos for small datasets. However, if
given a big enough dataset, NNs will always give better performance.
Q8. Artificial neuron
Which
of the following statements is/are true about a single artificial neuron?
Statements:
A.
It is loosely inspired from biological neurons
B. It computes a weighted sum
C.
It applies an activation function
D. It is capable of performing non-linear classification using sigmoid
Choose
the correct answer from below:
A. Only statement A and B are correct
B. Only statement B,C and D are correct
C. Only statement A, B and C are correct
D. All the given statements are true
Ans: A, B, C
Correct
answer: Only statements A, B and C are correct
Explanation:
- The basic inspiration for artificial
neurons did come from biological neurons.
Biological neurons form a network a network within themselves.
Each connection, like the synapses in a biological brain, can transmit a signal to other neurons.
An artificial neuron receives signals then processes them and can signal neurons connected to it. - A neuron does it’s computation in 2
steps:
- First it computes the weighted sum
as: z=w1x1+w2x2+…+wdxd+b
- Then it applies an activation
function on top of this sum: a=f(z)
- A single neuron can perform binary
classification if it’s activation is the sigmoid function.
However, it cannot perform non linear classification on it’s own. We would need a network of multiple neurons to do that.
Q8. And
perceptron
We
want to design a perception that performs AND operation. Refer to the table
below:-
For
this, first, a weighted sum is calculated: z=w1x1+w2x2+b
Here, w1 and w2 are
weights, and b is the bias for the neuron.
The
activation function applied on this is as follows:-
Which of the following values of weights and bias will give the desired results?
A. w1=1, w2=1, b=-2
B. w1=1, w2=0, b=-2
C. w1=2, w2=1, b=4
Ans: A
Correct
answer: w1=1,w2=1,b=−2
Explanation:
- For the input (0,0) the perceptron
will perform calculation something like this:
z=w1x1+x2x2+b=(1)(0)+(1)(0)+(−2)=−2
Therefore f(z)=0 - Similarly for the inputs (0,1) and
(1,0)
For both cases, z=−1,
Therefore, f(z)=0 - But for the input (1,1) the value
of z=w1.x1+w2.x2+b=0
Therefore, f(z) = 1 - Therefore, among the options, 1,1,
and -2 give the required results.
- We can use any values for w1,w2,
and b which satisfy our conditions and output.
Q9.
Biological Neuron
Ans:
A neuron takes input, processes it, and sends output to other neurons
Q10:
Artificial Neuron
Ans:
A simplified model of a biological neuron used in neural networks
Q11: Dendrites
Ans:
Structures that receive input signals in a neuron, akin to features in ML
models
Q12:
Weights
Ans:
Parameters that determine the importance of features in a neural network
Q13:
Activation Function
Ans:
A function applied to the linear combination of inputs to introduce
non-linearity
Q14:
Sigmoid Function
Ans: An S-shaped activation
function used in logistic regression
Q15: Nonlinearity
Ans:
A
characteristic of models that can handle complex patterns beyond linear
relationships
Q16: Linear
Regression
Ans:
A
model representing a linear equation without an activation function
Q17: Logistic
Regression
Ans: A linear model with a
sigmoid activation function
Q18: Feature
Engineering
Ans: The process of creating
features to improve model performance
Q19: Encoder-Decoder
Ans: Neural network model for
compressing and then reconstructing data
Q20: Deep
Learning
Ans: A subset of ML involving
multi-layered neural networks to model complex patterns
Subscribe to:
Posts (Atom)