4. Introduction to Supervised, Unsupervised and Reinforcement Learning

 

4. Introduction to Supervised, Unsupervised and Reinforcement Learning

 

ü  The amount of data generated in the world today is very huge. This data is generated not only by humans but also by smartphones, computers and other devices. Based on the kind of data available and a motive present, certainly, a programmer will choose how to train an algorithm using a specific learning model.

·       Machine Learning is a part of Computer Science where the efficiency of a system improves itself by repeatedly performing the tasks by using data instead of explicitly programmed by programmers. Further let us understand the difference between three techniques of Machine Learning- Supervised, Unsupervised and Reinforcement Learning.

 

How Machine Learning Works?

Types of Learning Algorithms

       i.          Supervised learning

     ii.          Unsupervised learning

   iii.          Reinforcement learning

Supervised Learning

·       In Supervised learning, an AI system is presented with data which is labeled, which means that each data tagged with the correct label.

·       The goal is to approximate the mapping function so well that when you have new input data (x) that you can predict the output variables (Y) for that data.

·       As shown in the example, we have initially taken some data and marked them as ‘Spam’ or ‘Not Spam’. This labeled data is used by the training supervised model, this data is used to train the model.

·       Once it is trained we can test our model by testing it with some test new mails and checking of the model is able to predict the right output.

        

Types of Supervised learning

       i.          Classification: A classification problem is when the output variable is a category, such as “red” or “blue” or “disease” and “no disease”.

     ii.          Regression: A regression problem is when the output variable is a real value, such as “dollars” or “weight”.

 

Unsupervised Learning Algorithm

·       In unsupervised learning, an AI system is presented with unlabeled, uncategorized data and the system’s algorithms act on the data without prior training.

·       In the example, we have given some characters to our model which are ‘Ducks’ and ‘Not Ducks’.

·       In our training data, we don’t provide any label to the corresponding data.

·       The unsupervised model is able to separate both the characters by looking at the type of data and models the underlying structure or distribution in the data to learn more about it.

can 

 Types of Unsupervised Learning

       i.          Clustering: A clustering problem is where you want to discover the inherent groupings in the data, such as grouping customers by purchasing behavior.

     ii.          Association: An association rule learning problem is where you want to discover rules that describe large portions of your data, such as people who buy X also tend to buy Y.

 

 Reinforcement Learning

·       A reinforcement learning algorithm, or agent, learns by interacting with its environment.

·       The agent receives rewards by performing correctly and penalties for performing incorrectly.

·       The agent learns without intervention from a human by maximizing its reward and minimizing its penalty.

·       It is a type of dynamic programming that trains algorithms using a system of reward and punishment.

 

In the example, we can see that the agent is given 2 options i.e. a path with water or a path with fire.

·       A reinforcement algorithm works on reward a system i.e. if the agent uses the fire path then the rewards are subtracted and agent tries to learn that it should avoid the fire path.

·       If it had chosen the water path or the safe path then some points would have been added to the reward points, the agent then would try to learn what path is safe and what path isn’t.

·       It is basically leveraging the rewards obtained, the agent improves its environment knowledge to select the next action.

 

 

Applications of Machine Learning

•Recognizing patterns:

ü  Facial identities or facial expressions

ü  Handwritten or spoken words

ü  Medical images

•Generating patterns:

ü  Generating images or motion sequences

•Recognizing anomalies:

ü  Unusual sequences of credit card transactions

ü  Unusual patterns of sensor readings in a nuclear power plant or unusual sound in your car engine.

•Prediction:

ü  Future stock prices or currency exchange rates

 

Influence of Disciplines on Machine Learning

       i.          Artificial intelligence

Learning symbolic representations of concepts. Machine learning as a search problem. Learning as an approach to improving problem solving. Using prior knowledge together with training data to guide learning.

     ii.          Bayesian methods

Bayes' theorem as the basis for calculating probabilities of hypotheses. The naive Bayes classifier. Algorithms for estimating values of unobserved variables.

   iii.          Computational complexity theory

Theoretical bounds on the inherent complexity of different learning tasks, measured in terms of the computational effort, number of training examples, number of mistakes, etc. required in order to learn.

   iv.          Control theory

Procedures that learn to control processes in order to optimize predefined objectives and that learn to predict the next state of the process they are controlling.

     v.          Information theory

Measures of entropy and information content. Minimum description length approaches to learning. Optimal codes and their relationship to optimal training sequences for encoding a hypothesis.

   vi.          Philosophy

Occam's razor, suggesting that the simplest hypothesis is the best. Analysis of the justification for generalizing beyond observed data.

  vii.          Psychology and neurobiology

The power law of practice, which states that over a very broad range of learning problems, people's response time improves with practice according to a power law. Neurobiological studies motivating artificial neural network models of learning.

viii.          Statistics

Characterization of errors (e.g., bias and variance) that occur when estimating the accuracy of a hypothesis based on a limited sample of data. Confidence intervals, statistical tests.

 

 

	Supervised learning	Unsupervised learning	Reinforcement learning
Definition	Makes predictions from data	Segments and groups data	Reward-punishment system and interactive environment
Types of data	Labelled data	Unlabeled data	Acts according to a policy with a final goal to reach (No or predefined data)
Commercial value	High commercial and business value	Medium commercial and business value	Little commercial use yet
Types of problems	Regression and classification	Association and Clustering	Exploitation or Exploration
Supervision	Extra supervision	No	No supervision
Algorithms	Linear Regression, Logistic Regression, SVM, KNN and so forth	K – Means clustering,  C – Means, Apriori	Q – Learning,  SARSA
Aim	Calculate outcomes	Discover underlying patterns	Learn a series of action
Application	Risk Evaluation, Forecast Sales	Recommendation System, Anomaly Detection	Self-Driving Cars, Gaming, Healthcare

 

 

Which is the better Machine Learning technique?

ü  We learned about the three main members of the machine learning family essential for deep learning. Other kinds of learning are also available such as semi-supervised learning, or self-supervised learning.

ü  Supervised, unsupervised, and reinforcement learning, are all used for different to complete diverse kinds of tasks. No single algorithm exists that can solve every problem, as problems of different natures require different approaches to resolve them.

ü  Despite the many differences between the three types of learning, all of these can be used to build efficient and high-value machine learning and Artificial Intelligence applications. All techniques are used in different areas of research and development to help solve complex tasks and resolve challenges.

 

 

3. PERSPECTIVES AND ISSUES IN MACHINE LEARNING

 

3. PERSPECTIVES AND ISSUES IN MACHINE LEARNING

Issues in Machine Learning

·       What algorithms exist for learning general target functions from specific training examples? In what settings will particular algorithms converge to the desired function, given sufficient training data? Which algorithms perform best for which types of problems and representations?

·       How much training data is sufficient? What general bounds can be found to relate the confidence in learned hypotheses to the amount of training experience and the character of the learner's hypothesis space?

·       When and how can prior knowledge held by the learner guide the process of generalizing from examples? Can prior knowledge be helpful even when it is only approximately correct?

·       What is the best strategy for choosing a useful next training experience, and how does the choice of this strategy alter the complexity of the learning problem?

·       What is the best way to reduce the learning task to one or more function approximation problems? Put another way, what specific functions should the system attempt to learn? Can this process itself be automated?

·        How can the learner automatically alter its representation to improve its ability to represent and learn the target function?

6. A CONCEPT LEARNING TASK

 

6. A CONCEPT LEARNING TASK

Consider the example task of learning the target concept "Days on which Aldo enjoys his favourite water sport”

 

 

Example	Sky	AirTemp	Humidity	Wind	Water	Forecast	EnjoySport
1	Sunny	Warm	Normal	Strong	Warm	Same	Yes
2	Sunny	Warm	High	Strong	Warm	Same	Yes
3	Rainy	Cold	High	Strong	Warm	Change	No
4	Sunny	Warm	High	Strong	Cool	Change	Yes

 

Table: Positive and negative training examples for the target concept EnjoySport.

The task is to learn to predict the value of EnjoySport for an arbitrary day, based on the

values of its other attributes?

What hypothesis representation is provided to the learner?

·       Let’s consider a simple representation in which each hypothesis consists of a conjunction of constraints on the instance attributes.

·       Let each hypothesis be a vector of six constraints, specifying the values of the six attributes Sky, AirTemp, Humidity, Wind, Water, and Forecast.

 

For each attribute, the hypothesis will either

·       Indicate by a "?' that any value is acceptable for this attribute,

·       Specify a single required value (e.g., Warm) for the attribute, or

·       Indicate by a "Φ" that no value is acceptable

If some instance x satisfies all the constraints of hypothesis h, then h classifies x as a positive

example (h(x) = 1).

The hypothesis that PERSON enjoys his favorite sport only on cold days with high humidity

is represented by the expression

(?, Cold, High, ?, ?, ?)

The most general hypothesis-that every day is a positive example-is represented by

(?, ?, ?, ?, ?, ?)

The most specific possible hypothesis-that no day is a positive example-is represented by

(Φ, Φ, Φ, Φ, Φ, Φ)

Notation

·       The set of items over which the concept is defined is called the set of instances, which is denoted by X.

Example: X is the set of all possible days, each represented by the attributes: Sky, AirTemp,

Humidity, Wind, Water, and Forecast

·       The concept or function to be learned is called the target concept, which is denoted by c. c can be any Boolean valued function defined over the instances X

c: X→ {O, 1}

Example: The target concept corresponds to the value of the attribute EnjoySport

(i.e., c(x) = 1 if EnjoySport = Yes, and c(x) = 0 if EnjoySport = No).

·       Instances for which c(x) = 1 are called positive examples, or members of the target concept.

·       Instances for which c(x) = 0 are called negative examples, or non-members of the target

·       concept.

·       The ordered pair (x, c(x)) to describe the training example consisting of the instance x and its target concept value c(x).

·       D to denote the set of available training examples

 

The symbol H to denote the set of all possible hypotheses that the learner may consider regarding the identity of the target concept. Each hypothesis h in H represents a Boolean valued

function defined over X

h: X→{O, 1}

The goal of the learner is to find a hypothesis h such that h(x) = c(x) for all x in X.

___________________________________________________________________________

Ø  Given:

·       Instances X: Possible days, each described by the attributes

o   Sky (with possible values Sunny, Cloudy, and Rainy),

o   AirTemp (with values Warm and Cold),

o   Humidity (with values Normal and High),

o   Wind (with values Strong and Weak),

o   Water (with values Warm and Cool),

o   Forecast (with values Same and Change).

Hypotheses H: Each hypothesis is described by a conjunction of constraints on the

attributes Sky, AirTemp, Humidity, Wind, Water, and Forecast. The constraints may be "?" (any value is acceptable , “Φ” (no value e is acceptable , or a specific value.

·       Target concept c: EnjoySport : X → {0, l}

·       Training examples D: Positive and negative examples of the target function

Determine:

·       A hypothesis h in H such that h(x) = c(x) for all x in X.

-----------------------------------------------------------------------------------------------------------------

Table: The EnjoySport concept learning task.

 

The inductive learning hypothesis

Any hypothesis found to approximate the target function well over a sufficiently large set of training examples will also approximate the target function well over other unobserved examples.

5. CONCEPT LEARNING

 

5. CONCEPT LEARNING

·       Learning involves acquiring general concepts from specific training examples. Example: People continually learn general concepts or categories such as "bird," "car," "situations in which I should study more to pass the exam," etc.

·       Each such concept can be viewed as describing some subset of objects or events defined over a larger set

·       Alternatively, each concept can be thought of as a Boolean-valued function defined over this larger set. (Example: A function defined over all animals, whose value is true for birds and false for other animals).

Definition: Concept learning - Inferring a Boolean-valued function from training examples of its input and output