Python Code: Github Link

How Random Forest works? – It’s a forest of Trees!!

1. Simply, its a combination of many trees.

2. It’s always better to take more suggestions and decide on majority voting rather going for a single suggestion. Same principle works in our Random Forest.

3. In this way, the probability of error gets decreased.

4. So, it works similar to the decision tree.

5. The number of trees is the main parameter and remaining all similar to the decision tree.

How to know the optimal parameters where model performance is high?

1. Here comes the hyper-parameter tuning for increasing model performance.

2. There are two types of hyper-parameter tuning. i) Grid search and ii) Randomized search.

3. For both types, we have to give the ranges of parameters and algorithm searches in those ranges and gives us the optimal parameters.

4. Grid search searches in equal grids manner whereas randomized search searches randomly.

How our algorithm evaluates and gives optimal parameters?

1. This is where cross-validation comes into the picture.

2. Our algorithm divides the whole training set into k folds and trains the model with parameters on k-1 folds and test on the remaining 1 fold.

3. This gives us the accuracy of the model with the parameters. In this way, the algorithm evaluates and finds out the optimal parameters.

Python Code: Github Link

What is Decision Tree?

1. A simple IF-THEN rules through the branches of a tree till the leaf which tells us the output and why it is.

2. It splits the features into the branches and goes till leaves.

3. So, the parameters for a decision tree will be maximum depth of tree, minimum leaf size and the evaluation criterion.

4. How to evaluate a leaf? We always see for a homogeneous leaf.

5. What is homogeneous? Let’s have 2 leafs L1 and L2. L1 have 50 values Yes and 50 Values No.L2 have 90 vales Yes and 10 values No. Which one we will prefer? It’s obvious to prefer L2. So L2 is a homogeneous leaf which gives more information.

6. So, the evaluation criterion is based on how much information we gaining.

7. Because of this information gain, the decision tree always try to overfit the model.

8. So how to stop our tree for not getting overfit?

9. Here comes the pruning. Pruning is cutting down the tree based on information gain.

10. If there is no much information gain from one level to next level of the tree, there is no use of expanding our tree.