_edited_edited.png)
How Decision Trees Simplify Complex Data
What is a decision tree and how does it work?
A decision tree is one of the most popular and effective supervised learning techniques for classification problems that works well with both categorical and continuous variables.
It is a graphical representation of all the possible solutions to a decision that is based on a certain condition.
In this algorithm, the training sample points are split into two or more sets based on the split condition over input variables.
Decision Tree is Decision Making Understanding Decision Tree Algorithm:
A simple example of a decision tree can be - A person has to take a decision for going to sleep or restaurant based on parameters like he is hungry or has 30$ in his pocket.
Decision Flowchart
Types of Decision Tree Algorithm:
ID3 : This algorithm measures how mixed up the data is at a node using something called entropy. It then chooses the feature that helps to clarify the data the most.
C4.5 : This is an improved version of ID3 that can handle missing data and continuous attributes.
CART : This algorithm uses a different measure called Gini impurity to decide how to split the data. It can be used for both classification (sorting data into categories) and regression (predicting continuous values) tasks.
Terminologies of Decision Tree Algorithm:
Root node - Represent the entire set of the population which gets further divided into sets based on splitting decisions.
Decision node - These are the internal nodes of the tree, These nodes are expressed through conditional expression for input attributes.
Leaf node/Terminal node - Nodes that do not split further are known as leaf nodes or terminal nodes.
Splitting - The process of dividing a node into one or more sub-nodes.
Pruning - It is the reverse process of splitting where the sub-nodes are removed.
Typology of Decision Tree
Hierarchy of Decision Tree Algorithm:
Decision Tree algorithm works in simpler steps
Starting at the Root: The algorithm begins at the top, called the “root node,” representing the entire dataset.
Asking the Best Questions: It looks for the most important feature or question that splits the data into the most distinct groups. This is like asking a question at a fork in the tree.
Branching Out: Based on the answer to that question, it divides the data into smaller subsets, creating new branches. Each branch represents a possible route through the tree.
Repeating the Process: The algorithm continues asking questions and splitting the data at each branch until it reaches the final “leaf nodes,” representing the predicted outcomes or classifications.
How to Understand and relate the Decision Tree Algorithm in Python:
Node # 0 ??
Represents the number of node.
Checking balance <= 0.0 ??
Checking balance <= 0 means that every observation with a checking balance of 0 and less will the True arrow i.e. go to the left in the next level, and the rest will follow the False arrow that is going to the right node in the next level.
GINI ??: Refers to the quality of the split.
Refers to the quality of the split. Always a number between 0.0 and 0.5,
0.0 -> High purity in the node.
0.5 -> High impurity in the node
Samples ??
424 means that there are 424 observations in this node.
Value ??
List that tells how many samples at the given node falls into each category.
Value = [486,214] means that of these 700 observations at this level , 486 belong to class 0 and 214 belong to
class 1.f
these 700means
Class = y[0] ??
Class = y[0] represents a majority of nodes present in class 0.
Color Intensity
Color Intensity represents the concentration of the majority of the class in a node. The darker the color, the higher the concentration of the majority class.
Advantages:
Easy to interpret: The Boolean logic and visual representations of decision trees make them easier to understand and consume.
Little to no data preparation required: Decision trees have a number of characteristics, which make it more flexible than other classifiers.
More flexible: Decision trees can be leveraged for both classification and regression tasks, making it more flexible than some other algorithms.
Disadvantages :
Prone to overfitting: Complex decision trees tend to overfit and do not generalize well to new data.
High variance estimators: Small variations within data can produce a very different decision tree.
More costly: Given that decision trees take a greedy search approach during construction, they can be more expensive to train compared to other algorithms.
Conclusion:
Decision Trees are an effective and easy-to-understand tool for both classification and regression tasks, breaking down data into simpler decisions based on key features. Different algorithms like ID3, C4.5, and CART enhance the decision-making process, and understanding concepts like Gini impurity, entropy, and pruning is crucial for building accurate models.
To learn more about Decision Trees and other machine-learning techniques, follow my blog for in-depth insights and updates!