Dated: 10/Apr/2023
Developer: Prince Park, A Fatima
For School: American School of Bahrain
Department: SEN/Learning Support
Data Scientist With Microsoft
https://learn.microsoft.com/en-us/users/princeparkyohannanhotmail-8262/transcript/dlmplcnz8w96op1

ASSOCIATE CERTIFICATION: Microsoft Certified: Azure Data Scientist Associate
CERTIFICATION EXAM: Designing and Implementing a Data Science Solution on Azure (Exam DP-100)
Data Scientist Career Path
- Learning Paths: Create machine learning models
- Option 1: Foundations of data science for machine learning (complete course)
- Module: Introduction to machine learning
- Module: Build classical machine learning models with supervised learning
- Module: Introduction to data for machine learning
- Module: Explore and analyze data with Python
- Module: Train and understand regression models in machine learning
- Module: Refine and test machine learning models
- Module: Train and evaluate regression models
- Module: Create and understand classification models in machine learning
- Module: Select and customize architectures and hyperparameters using random forest
- Module: Confusion matrix and data imbalances
- Module: Measure and optimize model performance with ROC and AUC
- Module: Train and evaluate classification models
- Module: Train and evaluate clustering models
- Module: Train and evaluate deep learning models
- Option 2: The Understand data science for machine learning (selected modules from option 1)
- Module: Introduction to machine learning
- Module: Build classical machine learning models with supervised learning
- Module: Introduction to data for machine learning
- Module: Train and understand regression models in machine learning
- Module: Refine and test machine learning models
- Module: Create and understand classification models in machine learning
- Module: Select and customize architectures and hyperparameters using random forest
- Module: Confusion matrix and data imbalances
- Module: Measure and optimize model performance with ROC and AUC
- Option 3: The Create machine learning models (selected modules from option 1)
- Option 1: Foundations of data science for machine learning (complete course)
- Learning Paths: Microsoft Azure AI Fundamentals: Explore visual tools for machine learning
- Learning Paths: Build and operate machine learning solutions with Azure Machine Learning
- Learning Paths: Build and operate machine learning solutions with Azure Databricks
COURSES
DP-090T00: Implementing a Machine Learning Solution with Microsoft Azure Databricks – Training
Azure Databricks is a cloud-scale platform for data analytics and machine learning. In this course, you’ll learn how to use Azure Databricks to explore, prepare, and model data; and integrate Databricks machine learning processes with Azure Machine Learning.
- Module: Get started with Azure Databricks
- Module: Work with data in Azure Databricks
- Module: Prepare data for machine learning with Azure Databricks
- Module: Train a machine learning model with Azure Databricks
- Module: Use MLflow to track experiments in Azure Databricks
- Module: Manage machine learning models in Azure Databricks
- Module: Track Azure Databricks experiments in Azure Machine Learning
- Module: Deploy Azure Databricks models in Azure Machine Learning
DP-100T01: Designing and Implementing a Data Science Solution on Azure
This course teaches you to leverage your existing knowledge of Python and machine learning to manage data ingestion and preparation, model training and deployment, and machine learning solution monitoring with Azure Machine Learning and MLflow.
- Module: Design a data ingestion strategy for machine learning projects
- Module: Design a machine learning model training solution
- Module: Design a model deployment solution
- Module: Explore Azure Machine Learning workspace resources and assets
- Module: Explore developer tools for workspace interaction
- Module: Make data available in Azure Machine Learning
- Module: Work with compute targets in Azure Machine Learning
- Module: Work with environments in Azure Machine Learning
- Module: Find the best classification model with Automated Machine Learning
- Module: Track model training in Jupyter notebooks with MLflow
- Module: Run a training script as a command job in Azure Machine Learning
- Module: Track model training with MLflow in jobs
- Module: Run pipelines in Azure Machine Learning
- Module: Perform hyperparameter tuning with Azure Machine Learning
- Module: Deploy a model to a managed online endpoint
- Module: Deploy a model to a batch endpoint
My Learnings.
# Calculate the number of empty cells in each column
# The following line consists of three commands. Try
# to think about how they work together to calculate
# the number of missing entries per column
missing_data = dataset.isnull().sum().to_frame()
# Rename column holding the sums
missing_data = missing_data.rename(columns={0:'Empty Cells'})
# Print the results
print(missing_data)
## OR
print(dataset.isnull().sum().to_frame().rename(columns={0:'Empty Cells'}))
# Show the missing value rows
dataset[dataset.isnull().any(axis=1)]
EDA
import pandas as pd
# Load data from a text file
!wget https://raw.githubusercontent.com/MicrosoftDocs/mslearn-introduction-to-machine-learning/main/Data/ml-basics/grades.csv
df_students = pd.read_csv('grades.csv',delimiter=',',header='infer')
# Remove any rows with missing data
df_students = df_students.dropna(axis=0, how='any')
# Calculate who passed, assuming '60' is the grade needed to pass
passes = pd.Series(df_students['Grade'] >= 60)
# Save who passed to the Pandas dataframe
df_students = pd.concat([df_students, passes.rename("Pass")], axis=1)
# Create a figure for 2 subplots (1 row, 2 columns)
fig, ax = plt.subplots(1, 2, figsize = (10,4))
# Create a bar plot of name vs grade on the first axis
ax[0].bar(x=df_students.Name, height=df_students.Grade, color='orange')
ax[0].set_title('Grades')
ax[0].set_xticklabels(df_students.Name, rotation=90)
# Create a pie chart of pass counts on the second axis
pass_counts = df_students['Pass'].value_counts()
ax[1].pie(pass_counts, labels=pass_counts)
ax[1].set_title('Passing Grades')
ax[1].legend(pass_counts.keys().tolist())
# Add a title to the Figure
fig.suptitle('Student Data')
# Show the figure
fig.show()

# Create a function that we can re-use
# Create a function that we can re-use
def show_distribution_with_quantile(var_data, quantile = 0):
'''
This function will make a distribution (graph) and display it
'''
if(quantile > 0){
# calculate the quantile percentile
q01 = var_data.quantile(quantile)
print(f"quantile = {q01}")
var_data = var_data[var_data>q01]
}
# Get statistics
min_val = var_data.min()
max_val = var_data.max()
mean_val = var_data.mean()
med_val = var_data.median()
mod_val = var_data.mode()[0]
print('Minimum:{:.2f}\nMean:{:.2f}\nMedian:{:.2f}\nMode:{:.2f}\nMaximum:{:.2f}\n'.format(min_val,
mean_val,
med_val,
mod_val,
max_val))
# Create a figure for 2 subplots (2 rows, 1 column)
fig, ax = plt.subplots(2, 1, figsize = (10,4))
# Plot the histogram
ax[0].hist(var_data)
ax[0].set_ylabel('Frequency')
# Add lines for the mean, median, and mode
ax[0].axvline(x=min_val, color = 'gray', linestyle='dashed', linewidth = 2)
ax[0].axvline(x=mean_val, color = 'cyan', linestyle='dashed', linewidth = 2)
ax[0].axvline(x=med_val, color = 'red', linestyle='dashed', linewidth = 2)
ax[0].axvline(x=mod_val, color = 'yellow', linestyle='dashed', linewidth = 2)
ax[0].axvline(x=max_val, color = 'gray', linestyle='dashed', linewidth = 2)
# Plot the boxplot
ax[1].boxplot(var_data, vert=False)
ax[1].set_xlabel('Value')
# Add a title to the Figure
fig.suptitle('Data Distribution')
# Show the figure
fig.show()
# Get the variable to examine
col = df_students['Grade']
# Call the function
show_distribution(col)

def show_density(var_data):
fig = plt.figure(figsize=(10,4))
# Plot density
var_data.plot.density()
# Add titles and labels
plt.title('Data Density')
# Show the mean, median, and mode
plt.axvline(x=var_data.mean(), color = 'cyan', linestyle='dashed', linewidth = 2)
plt.axvline(x=var_data.median(), color = 'red', linestyle='dashed', linewidth = 2)
plt.axvline(x=var_data.mode()[0], color = 'yellow', linestyle='dashed', linewidth = 2)
# Show the figure
plt.show()
# Get the density of StudyHours
show_density(col)

Azure Databricks
Mount a remote Azure storage account as a DBFS folder, using the dbutils
module:
data_storage_account_name = '<data_storage_account_name>'
data_storage_account_key = '<data_storage_account_key>'
data_mount_point = '/mnt/data'
data_file_path = '/bronze/wwi-factsale.csv'
dbutils.fs.mount(
source = f"wasbs://dev@{data_storage_account_name}.blob.core.windows.net",
mount_point = data_mount_point,
extra_configs = {f"fs.azure.account.key.{data_storage_account_name}.blob.core.windows.net": data_storage_account_key})
display(dbutils.fs.ls("/mnt/data"))
#this path is available as dbfs:/mnt/data for spark APIs, e.g. spark.read
#this path is available as file:/dbfs/mnt/data for regular APIs, e.g. os.listdir
# %fs magic command - for accessing the dbutils filesystem module. Most dbutils.fs commands are available using %fs magic commands
We can override the cell’s default programming language by using one of the following magic commands at the start of the cell:
%python
– for cells running python code%scala
– for cells running scala code%r
– for cells running R code%sql
– for cells running sql code
Additional magic commands are available:
%md
– for descriptive cells using markdown%sh
– for cells running shell commands%run
– for cells running code defined in a separate notebook%fs
– for cells running code that usesdbutils
commands
Update RDP Source IP For Azure Virtual Machine
Remove-Variable * -ErrorAction SilentlyContinue; Remove-Module *; $error.Clear();
$config = @{
SubscriptionName = "";
VMName = ""
}
$myip = (Invoke-WebRequest -uri "http://ifconfig.me/ip").Content
Write-Host "My Public IP is", $myip;
if (-Not(Get-InstalledModule -Name "Az")) {
Write-Host "Installing Azure Package from PS Gallery...";
Install-Module -Name Az -Scope CurrentUser -Repository PSGallery -Force;
}
if (-Not(Get-Module -Name "Az") -and -Not(Get-Command -Name Get-AZContext)) {
Write-Host "Importing Azure Module..."
Import-Module -Name Az -Force;
}
if (-Not (Get-AZContext)) {
Write-Host "Connecting to Azure..."
Connect-AzAccount #-Credential $azCredF
}
if ([bool]((Get-AZContext))) { Write-Host "Connected to Azure using :", ((Get-AZContext).Account) }
else { Write-Error "Failed to connect to Azure"; return; }
if (((Get-AZContext).Subscription.Name) -ne $config.SubscriptionName) {
Write-Host "Switching Subscription Context...";
(Get-AzSubscription -SubscriptionName $config.SubscriptionName) | Set-AzContext | Out-Null;
}
Write-Host "Subscription =", ((Get-AZContext).Subscription.Name);
$vm = Get-AzVM -VMName $config.VMName
Write-Host "Located Azure VM '$($vm.Name)' within '$($vm.ResourceGroupName)' ResourceGroup";
$nic = $vm.NetworkProfile.NetworkInterfaces;
$networkinterface = ($nic.id -split '/')[-1];
$nicdetails = Get-AzNetworkInterface -Name $networkinterface;
$nsg = Get-AzNetworkSecurityGroup -Name (($nicdetails.NetworkSecurityGroup.Id -split '/')[-1]) -ResourceGroupName ($vm.ResourceGroupName)
# $rules = Get-AzNetworkSecurityRuleConfig -NetworkSecurityGroup $networkSecurityGroup
# $rdpRule = $rules | Where-Object { $_.Protocol -eq 'TCP' -and $_.DestinationPortRange -contains 3389 }
Write-Host "Before Updating public ip";
($nsg.SecurityRules | Where-Object { $_.Name -eq "RDP" }).SourceAddressPrefix
Write-Host ("*" * 10)
($nsg.SecurityRules | Where-Object { $_.Name -eq "RDP" }).SourceAddressPrefix = ([System.String[]] @("xxx.xxx.xxx.xxx", "xxx.xxx.xxx.xxx", $myip))
$nsg | Set-AzNetworkSecurityGroup | Get-AzNetworkSecurityRuleConfig -Name "RDP" | Out-Null
Write-Host "After Updating public ip";
($nsg.SecurityRules | Where-Object { $_.Name -eq "RDP" }).SourceAddressPrefix
Disconnect-AzAccount | Out-Null
OpenCV(cv2) Vs Pillow(PIL)
_ OpenCV is 1.4 Times faster than PIL _
Image is simply a matrix of pixels and each pixel is a single, square-shaped point of colored light. This can be explained quickly with a grayscaled image. grayscaled image is the image where each pixel represents different shades of a gray color.

I mostly use OpenCV to complete my tasks as I find it 1.4 times quicker than PIL.
Let’s see, how the image can be processed using both — OpenCV and PIL.
## Installation & importing
# cv2
pip install opencv-python
import cv2
---
# PIL
pip install Pillow
from PIL import Image, ImageEnhance
## Read the image
# Read/open a colorful image
pil_img = Image.open("your_image.jpg") # RGB
cv2_img = cv2.imread("your_image.jpg") # BGR
# Read/open a grayscale image:
pil_img = Image.open("your_image.jpg").convert("L")
cv2_img = cv2.imread("your_image.jpg", cv2.IMREAD_GRAYSCALE)
## Write/save an image
pil_img.save("new_image.jpg")
cv2.imwrite("new_image.jpg", cv2_img)
# Write/save a JPEG image with specific quality:
pil_img.save("new_image.jpg", quality=95)
cv2.imwrite("new_image.jpg", cv2_img, [int(cv2.IMWRITE_JPEG_QUALITY), 95])
## Conversion between both
# Pillow image to OpenCV image:
cv2_img = np.array(pil_img)
cv2_img = cv2.cvtColor(cv2_img, cv2.COLOR_RGB2BGR)
# OpenCV image to Pillow image
cv2_img = cv2.cvtColor(cv2_img, cv2.COLOR_BGR2RGB)
pil_img = Image.fromarray(cv2_img)
Note: OpenCV images are in BGR color format, while Pillow images are in RGB color format. So we have to manually convert the color format from one to another.
## Shape / Size
# cv2
if cv2_img.ndim == 2:
height, width = cv2_img.shape
depth = 1
else:
height, width, depth = cv2_img.shape
# PIL
width, height = pil_img.size
cv2_img = np.array(pil_img)
if cv2_img.ndim == 2:
depth = 1
else:
depth = cv2_img.shape[-1]
Note: It is hard to get the depth/channels directly from a Pillow image object, the easier way to do this would be to first convert it to an OpenCV image (ndarray) and then get the shape.
## Resize
# Resize without preserving the aspect ratio:
pil_img_resized = pil_img.resize((NEW_WIDTH, NEW_HEIGHT))
cv2_img_resized = cv2.resize(cv2_img, (NEW_WIDTH, NEW_HEIGHT))
Resize and preserve the aspect ratio:
# OpenCV:
scale_ratio = 0.6
width = int(img.shape[1] * scale_ratio)
height = int(img.shape[0] * scale_ratio)
dim = (width, height)
cv2_img_resized = cv2.resize(cv2_img, dim, interpolation=cv2.INTER_AREA)
# Pillow:
# scale ratio = min(max_width/width, max_height/height)
max_width = 256
max_height = 256
pil_img.thumbnail((max_width, max_height), Image.ANTIALIAS)
## RGBA to RGB
# Convert transparent pixels to white pixels (by pasting the RGBA image on a white RGB image).
#cv2
def cv2_RGBA2RGB(img):
b, g, r, a = cv2.split(img)
alpha = a / 255
r = (255 * (1 - alpha) + r * alpha).astype(np.uint8)
g = (255 * (1 - alpha) + g * alpha).astype(np.uint8)
b = (255 * (1 - alpha) + b * alpha).astype(np.uint8)
new_img = cv2.merge((b, g, r))
return new_img
# PIL
def pil_RGBA2RGB(img):
img.load() # for png.split()
bg = Image.new("RGB", img.size, (255, 255, 255))
bg.paste(img, mask=img.split()[3]) # 3 is the alpha channel
return bg
## Read an image from a URL.
# without request headers
url = ''
# cv2
import cv2
import numpy as np
import requests
cv2_img = cv2.imdecode(np.asarray(requests.get(url, stream=True).content, dtype=np.uint8), cv2.IMREAD_UNCHANGED)
# PIL
importt io;
import requests
pil_img = Image.open(io.BytesIO(requests.get(url, stream=True).content))
## Base64 Conversions
# Read image file as base64:
import base64
with open("your_image.jpg", "rb") as f:
base64_str = base64.b64encode(f.read())
# Conversion between Pillow & base64:
import base64
from io import BytesIO
from PIL import Image
def pil_to_base64(pil_img):
img_buffer = BytesIO()
pil_img.save(img_buffer, format='JPEG')
byte_data = img_buffer.getvalue()
base64_str = base64.b64encode(byte_data)
return base64_str
def base64_to_pil(base64_str):
pil_img = base64.b64decode(base64_str)
pil_img = BytesIO(pil_img)
pil_img = Image.open(pil_img)
return pil_img
# Conversion between OpenCV & base64:
import base64
import numpy as np
import cv2
def cv2_base64(cv2_img):
base64_str = cv2.imencode('.jpg', cv2_img)[1].tostring()
base64_str = base64.b64encode(base64_str)
return base64_str
def base64_cv2(base64_str):
imgString = base64.b64decode(base64_str)
nparr = np.fromstring(imgString, np.uint8)
cv2_img= cv2.imdecode(nparr, cv2.IMREAD_COLOR)
return cv2_img
- Pillow (PIL Fork) 9.4.0 documentation
- Image Processing — OpenCV Vs PIL – By Suraj Gurav
- The Ultimate Handbook for OpenCV & Pillow – By Steins
- https://note.nkmk.me/en/python-pillow-rotate/
- https://www.pyimagesearch.com/2021/02/03/opencv-image-translation/
- https://www.pyimagesearch.com/2017/01/02/rotate-images-correctly-with-opencv-and-python/
- https://stackoverflow.com/questions/14177744/how-does-perspective-transformation-work-in-pil
- https://www.pyimagesearch.com/2014/05/05/building-pokedex-python-opencv-perspective-warping-step-5-6/
- https://note.nkmk.me/python-opencv-warp-affine-perspective/
- https://stackoverflow.com/questions/25618756/how-to-convert-alpha-channel-of-the-image-to-white-color-using-opencv-python
- https://stackoverflow.com/questions/9166400/convert-rgba-png-to-rgb-with-pil
NWEA Builder / EnhancerNWEA
GlobalIDResolver
AttendanceAlert
ASBCustomization
ParentIDCards
Code Implementation of Machine Learning, Deep Learning & Artificial Intelligence Functions
Activation Functions
Sigmoid / Logistic Function


import math;
def sigmoid(x):
return 1 / (1 + math.exp(-x))
tanh Function


import math;
def tanh(x):
return (math.exp(x) - math.exp(-x)) / (math.exp(x) + math.exp(-x))
ReLU


import math;
def relu(x):
return max(0,x)
Leaky ReLU


import math;
def leaky_relu(x):
return max(0.1*x,x)
Loss \ Cost Functions
Mean Absolute Error/L1 Loss (Regression Losses)

Mean absolute error, on the other hand, is measured as the average sum of absolute differences between predictions and actual observations. Like MSE, this as well measures the magnitude of error without considering their direction. Unlike MSE, MAE needs more complicated tools such as linear programming to compute the gradients. Plus MAE is more robust to outliers since it does not make use of squares.
# Plain implementation
import numpy as np
y_hat = np.array([0.000, 0.166, 0.333])
y_true = np.array([0.000, 0.254, 0.998])
print("d is: " + str(["%.8f" % elem for elem in y_hat]))
print("p is: " + str(["%.8f" % elem for elem in y_true]))
def mae(predictions, targets):
total_error = 0
for yp, yt in zip(predictions, targets):
total_error += abs(yp - yt)
print("Total error is:",total_error)
mae = total_error/len(predictions)
print("Mean absolute error is:",mae)
return mae
# Usage : mae(predictions, targets)
# Implementation using numpy
import numpy as np
y_hat = np.array([0.000, 0.166, 0.333])
y_true = np.array([0.000, 0.254, 0.998])
print("d is: " + str(["%.8f" % elem for elem in y_hat]))
print("p is: " + str(["%.8f" % elem for elem in y_true]))
def mae_np(predictions, targets):
return np.mean(np.abs(predictions-targets))
mae_val = mae_np(y_hat, y_true)
print ("mae error is: " + str(mae_val))
Mean Square Error/Quadratic Loss/L2 Loss (Regression Losses)

Mean square error is measured as the average of the squared difference between predictions and actual observations. It’s only concerned with the average magnitude of error irrespective of their direction. However, due to squaring, predictions that are far away from actual values are penalized heavily in comparison to less deviated predictions. Plus MSE has nice mathematical properties which make it easier to calculate gradients.
# Plain implementation
import numpy as np
y_hat = np.array([0.000, 0.166, 0.333])
y_true = np.array([0.000, 0.254, 0.998])
def rmse(predictions, targets):
total_error = 0
for yt, yp in zip(targets, predictions):
total_error += (yt-yp)**2
print("Total Squared Error:",total_error)
mse = total_error/len(y_true)
print("Mean Squared Error:",mse)
return mse
print("d is: " + str(["%.8f" % elem for elem in y_hat]))
print("p is: " + str(["%.8f" % elem for elem in y_true]))
rmse_val = rmse(y_hat, y_true)
print("rms error is: " + str(rmse_val))
# Implementation using numpy
import numpy as np
y_hat = np.array([0.000, 0.166, 0.333])
y_true = np.array([0.000, 0.254, 0.998])
def rmse(predictions, targets):
return np.mean(np.square(targets-predictions))
print("d is: " + str(["%.8f" % elem for elem in y_hat]))
print("p is: " + str(["%.8f" % elem for elem in y_true]))
rmse_val = rmse(y_hat, y_true)
print("rms error is: " + str(rmse_val))
Log Loss or Binary Cross Entropy

import numpy as np
y_predicted = np.array([[0.25,0.25,0.25,0.25],[0.01,0.01,0.01,0.96]])
y_true = np.array([[0,0,0,1],[0,0,0,1]])
def cross_entropy(predictions, targets, epsilon=1e-10):
predictions = np.clip(predictions, epsilon, 1. - epsilon)
N = predictions.shape[0]
ce_loss = -np.sum(np.sum(targets * np.log(predictions + 1e-5)))/N
return ce_loss
cross_entropy_loss = cross_entropy(predictions, targets)
print ("Cross entropy loss is: " + str(cross_entropy_loss))
# OR
def log_loss(predictions, targets, epsilon=1e-10):
predicted_new = [max(i,epsilon) for i in predictions]
predicted_new = [min(i,1-epsilon) for i in predicted_new]
predicted_new = np.array(predicted_new)
return -np.mean(targets*np.log(predicted_new)+(1-y_true)*np.log(1-predicted_new))
For more functions refer ‘Common Loss functions in machine learning’ By Ravindra Parmar
Gradient Descent
Gradient Descent

# Single Feature
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
def gradient_descent(x, y, epochs = 10000, loss_thresold = 0.5, rate = 0.01):
w1 = bias = 0
n = len(x)
plt.scatter(x, y, color='red', marker='+', linewidth='5')
for i in range(epochs):
y_predicted = (w1 * x)+ bias
plt.plot(x, y_predicted, color='green')
md = -(2/n)*sum(x*(y-y_predicted))
yd = -(2/n)*sum(y-y_predicted)
w1 = w1 - rate * md
bias = bias - rate * yd
print ("m {}, b {}, cost {} iteration {}".format(m_curr,b_curr,cost, i))
# Usage
x = np.array([1,2,3,4,5])
y = np.array([5,7,9,11,13])
gradient_descent(x, y, 500)
---
# Multiple Feature
def gradient_descent(x1, x2, y, epochs = 10000, loss_thresold = 0.5, rate = 0.01):
w1 = w2 = bias = 1
rate = 0.5
n = len(x1)
for i in range(epochs):
weighted_sum = (w1 * x1) + (w2 * x2) + bias
y_predicted = sigmoid_numpy(weighted_sum)
loss = log_loss(y_predicted, y)
w1d = (1/n)*np.dot(np.transpose(x1),(y_predicted-y))
w2d = (1/n)*np.dot(np.transpose(x2),(y_predicted-y))
bias_d = np.mean(y_predicted-y)
w1 = w1 - rate * w1d
w2 = w2 - rate * w2d
bias = bias - rate * bias_d
print (f'Epoch:{i}, w1:{w1}, w2:{w2}, bias:{bias}, loss:{loss}')
if loss<=loss_thresold:
break
return w1, w2, bias
# Usage
gradient_descent(
X_train_scaled['age'],
X_train_scaled['affordibility'],
y_train,
1000,
0.4631
)
# custom neural network class
class myNN:
def __init__(self):
self.w1 = 1
self.w2 = 1
self.bias = 0
def sigmoid_numpy(self, X):
import numpy as np;
return 1/(1+np.exp(-X))
def log_loss(self, y_true, y_predicted):
import numpy as np;
epsilon = 1e-15
y_predicted_new = [max(i,epsilon) for i in y_predicted]
y_predicted_new = [min(i,1-epsilon) for i in y_predicted_new]
y_predicted_new = np.array(y_predicted_new)
return -np.mean(y_true*np.log(y_predicted_new)+(1-y_true)*np.log(1-y_predicted_new))
def fit(self, X, y, epochs, loss_thresold):
self.w1, self.w2, self.bias = self.gradient_descent(X['age'],X['affordibility'],y, epochs, loss_thresold)
print(f"Final weights and bias: w1: {self.w1}, w2: {self.w2}, bias: {self.bias}")
def predict(self, X_test):
weighted_sum = self.w1*X_test['age'] + self.w2*X_test['affordibility'] + self.bias
return self.sigmoid_numpy(weighted_sum)
def gradient_descent(self, age,affordability, y_true, epochs, loss_thresold):
import numpy as np;
w1 = w2 = 1
bias = 0
rate = 0.5
n = len(age)
for i in range(epochs):
weighted_sum = w1 * age + w2 * affordability + bias
y_predicted = self.sigmoid_numpy(weighted_sum)
loss = self.log_loss.log_loss(y_true, y_predicted)
w1d = (1/n)*np.dot(np.transpose(age),(y_predicted-y_true))
w2d = (1/n)*np.dot(np.transpose(affordability),(y_predicted-y_true))
bias_d = np.mean(y_predicted-y_true)
w1 = w1 - rate * w1d
w2 = w2 - rate * w2d
bias = bias - rate * bias_d
if i%50==0:
print (f'Epoch:{i}, w1:{w1}, w2:{w2}, bias:{bias}, loss:{loss}')
if loss<=loss_thresold:
print (f'Epoch:{i}, w1:{w1}, w2:{w2}, bias:{bias}, loss:{loss}')
break
return w1, w2, bias
# Usage
customModel = myNN()
customModel.fit(X_train_scaled, y_train, epochs=8000, loss_thresold=0.4631)
# Usage
customModel = myNN()
customModel.fit(
X_train_scaled,
y_train,
epochs=8000,
loss_thresold=0.4631
)

Batch Gradient Descent
def batch_gradient_descent(X, y_true, epochs, learning_rate = 0.01):
number_of_features = X.shape[1]
# numpy array with 1 row and columns equal to number of features. In
# our case number_of_features = 2 (area, bedroom)
w = np.ones(shape=(number_of_features))
b = 0
total_samples = X.shape[0] # number of rows in X
cost_list = []
epoch_list = []
for i in range(epochs):
y_predicted = np.dot(w, X.T) + b
w_grad = -(2/total_samples)*(X.T.dot(y_true-y_predicted))
b_grad = -(2/total_samples)*np.sum(y_true-y_predicted)
w = w - learning_rate * w_grad
b = b - learning_rate * b_grad
cost = np.mean(np.square(y_true-y_predicted)) # MSE (Mean Squared Error)
if i%10==0:
cost_list.append(cost)
epoch_list.append(i)
return w, b, cost, cost_list, epoch_list
w, b, cost, cost_list, epoch_list = batch_gradient_descent(
scaled_X,
scaled_y.reshape(scaled_y.shape[0],),
500
)
w, b, cost
Mini Batch Gradient Descent
def mini_batch_gradient_descent(X, y_true, epochs = 100, batch_size = 5, learning_rate = 0.01):
number_of_features = X.shape[1]
# numpy array with 1 row and columns equal to number of features. In
# our case number_of_features = 3 (area, bedroom and age)
w = np.ones(shape=(number_of_features))
b = 0
total_samples = X.shape[0] # number of rows in X
if batch_size > total_samples: # In this case mini batch becomes same as batch gradient descent
batch_size = total_samples
cost_list = []
epoch_list = []
num_batches = int(total_samples/batch_size)
for i in range(epochs):
random_indices = np.random.permutation(total_samples)
X_tmp = X[random_indices]
y_tmp = y_true[random_indices]
for j in range(0,total_samples,batch_size):
Xj = X_tmp[j:j+batch_size]
yj = y_tmp[j:j+batch_size]
y_predicted = np.dot(w, Xj.T) + b
w_grad = -(2/len(Xj))*(Xj.T.dot(yj-y_predicted))
b_grad = -(2/len(Xj))*np.sum(yj-y_predicted)
w = w - learning_rate * w_grad
b = b - learning_rate * b_grad
cost = np.mean(np.square(yj-y_predicted)) # MSE (Mean Squared Error)
if i%10==0:
cost_list.append(cost)
epoch_list.append(i)
return w, b, cost, cost_list, epoch_list
w, b, cost, cost_list, epoch_list = mini_batch_gradient_descent(
scaled_X,
scaled_y.reshape(scaled_y.shape[0],),
epochs = 120,
batch_size = 5
)
w, b, cost
Stochastic Gradient Descent
def stochastic_gradient_descent(X, y_true, epochs, learning_rate = 0.01):
number_of_features = X.shape[1]
# numpy array with 1 row and columns equal to number of features. In
# our case number_of_features = 3 (area, bedroom and age)
w = np.ones(shape=(number_of_features))
b = 0
total_samples = X.shape[0]
cost_list = []
epoch_list = []
for i in range(epochs):
random_index = random.randint(0,total_samples-1) # random index from total samples
sample_x = X[random_index]
sample_y = y_true[random_index]
y_predicted = np.dot(w, sample_x.T) + b
w_grad = -(2/total_samples)*(sample_x.T.dot(sample_y-y_predicted))
b_grad = -(2/total_samples)*(sample_y-y_predicted)
w = w - learning_rate * w_grad
b = b - learning_rate * b_grad
cost = np.square(sample_y-y_predicted)
if i%100==0: # at every 100th iteration record the cost and epoch value
cost_list.append(cost)
epoch_list.append(i)
return w, b, cost, cost_list, epoch_list
w_sgd, b_sgd, cost_sgd, cost_list_sgd, epoch_list_sgd = SGD(
scaled_X,
scaled_y.reshape(scaled_y.shape[0],),
10000
)
w_sgd, b_sgd, cost_sgd