Case Study: Quality Prediction In Mining Process¶
Introduction¶
In this project, machine learning and deep learning models will be train to predict the percentage of Silica Concentrate in the Iron ore concentrate per minute. This project could be practically used in Mining Industry to get the percentage of Silica Concentrate at a much faster rate compared to the traditional methods.
Freeport-McMoran is one of the largest mining companies in U.S.. They have made use of custom built AI model to boost the output in their mining facilities.
The 4th Industrial Revolution: How Mining Companies Are Using AI, Machine Learning And Robots:
Problem:
- Although the percentage of Silica is measured, its a lab measurement, which means that it takes at least one hour for the process engineers to have this value. So if it is posible to predict the amount of impurity in the process
- Pedict the percentage of Silica Concentrate in the Iron ore concentrate per minute.
Dataset:
- By leveraging past 3+ data, AI can help the mining facility sustain a faster production pace with no loss efficiency. Efficiency can be improve by performing operational tweaks to boost output. (Ex. Copper production could be increase if mili was fed with more ore per minute.)
Source: Kaggle Competition
Libraries And Data Importation¶
In [1]:
# Import libraries
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
import zipfile
In [2]:
# Import data
mining_df = pd.read_csv('mining_data.csv')
Data Exploraton¶
In [4]:
# Check data
mining_df.head()
Out[4]:
In [8]:
# Check data statistics
mining_df.describe()
Out[8]:
In [5]:
# Check data types
mining_df.dtypes
Out[5]:
In [7]:
# Check for missing values
mining_df.isnull().mean()
Out[7]:
Data Visualization¶
In [11]:
# Create bar chart
mining_df.hist(bins = 30, figsize = (20,20), color = 'b')
plt.show()
In [12]:
# Import library
from heatmap import corrplot
# Show correlation in the features by using heatmap
plt.figure(figsize=(12, 8))
corrplot(mining_df.corr(), size_scale=300)
In [14]:
# Create scatterplot
sns.scatterplot(mining_df['% Silica Concentrate'], mining_df['% Iron Concentrate'])
plt.show()
In [21]:
# Create scatterplot
sns.scatterplot(mining_df['% Iron Feed'], mining_df['% Silica Feed'])
plt.show()
Data Preprocessing¶
In [22]:
# Label Independent variables
df_iron = mining_df.drop(columns = '% Silica Concentrate')
# Label dependent variables
df_iron_target = mining_df['% Silica Concentrate']
In [23]:
# Check Independent variables shape
df_iron.shape
Out[23]:
In [24]:
# Check dependent variables shape
df_iron_target.shape
Out[24]:
In [25]:
# Transform into array
df_iron = np.array(df_iron)
# Transform into array
df_iron_target = np.array(df_iron_target)
In [30]:
# Reshaping the array
df_iron_target = df_iron_target.reshape(-1,1)
# Check dimension
df_iron_target.shape
Out[30]:
In [32]:
# Import library
from sklearn.preprocessing import StandardScaler, MinMaxScaler
# Scale the independent variables
scaler_x = StandardScaler()
X = scaler_x.fit_transform(df_iron)
# Scale the dependent variables
scaler_y = StandardScaler()
y = scaler_y.fit_transform(df_iron_target)
In [34]:
# Import libraru
from sklearn.model_selection import train_test_split
# Split the data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0)
In [37]:
# Check train dataset shape
X_train.shape, y_train.shape
Out[37]:
In [38]:
# Check train dataset shape
X_test.shape, y_test.shape
Out[38]:
Model Training And Evaluation¶
Linear Regression Model¶
In [43]:
# Import libraries
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, accuracy_score
# Train the model
LinearRegression_model = LinearRegression()
LinearRegression_model.fit(X_train, y_train)
Out[43]:
In [44]:
# Check Linear Regression model accuracy
accuracy_LinearRegression = LinearRegression_model.score(X_test, y_test)
accuracy_LinearRegression
Out[44]:
Decision Tree Regressor Model¶
In [46]:
# Import lirbary
from sklearn.tree import DecisionTreeRegressor
# Train the model
DecisionTree_model = DecisionTreeRegressor()
DecisionTree_model.fit(X_train, y_train)
Out[46]:
In [48]:
# Check Decision Tree model accuracy
accuracy_DecisionTree = DecisionTree_model.score(X_test, y_test)
accuracy_DecisionTree
Out[48]:
XGBoost Regressor Model¶
In [55]:
# Import library
from xgboost import XGBRegressor
# Train the model
xgb = XGBRegressor()
xgb.fit(X_train, y_train)
Out[55]:
In [56]:
# Check XGBoost model accuracy
xgb.score(X_test, y_test)
Out[56]:
Artificial Neural Network Model¶
In [58]:
# Import library
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.layers import Dense, Activation, Dropout
from tensorflow.keras.optimizers import Adam
# Create model architecture
optimizer = Adam(learning_rate=0.001, beta_1 = 0.9, beta_2 = 0.999, epsilon = 1e-07, amsgrad = False)
ANN_model = keras.Sequential()
ANN_model.add(Dense(250, input_dim = 22, kernel_initializer='normal',activation='relu'))
ANN_model.add(Dense(500,activation = 'relu'))
ANN_model.add(Dropout(0.1))
ANN_model.add(Dense(1000, activation = 'relu'))
ANN_model.add(Dropout(0.1))
ANN_model.add(Dense(1000, activation = 'relu'))
ANN_model.add(Dropout(0.1))
ANN_model.add(Dense(500, activation = 'relu'))
ANN_model.add(Dropout(0.1))
ANN_model.add(Dense(250, activation = 'relu'))
ANN_model.add(Dropout(0.1))
ANN_model.add(Dense(1, activation = 'linear'))
# Compile the model
ANN_model.compile(loss = 'mse', optimizer = 'adam')
# Check modelsummary
ANN_model.summary()
In [61]:
# Initiate model training
history = ANN_model.fit(X_train, y_train, epochs = 5, validation_split = 0.2)
In [63]:
# Evaluate the ANN model
result = ANN_model.evaluate(X_test, y_test)
accuracy_ANN = 1 - result
print("Accuracy : {}".format(accuracy_ANN))
Calculate Regression KPIs¶
In [72]:
# From the above results, it can be seen that, decision tree model out-performs the other models.
# Plot True values and Model Predictions
y_predict = DecisionTree_model.predict(X_test)
plt.plot(y_predict, y_test, '^', color = 'b')
plt.xlabel('Model Predictions')
plt.ylabel('True Values')
plt.show()
In [75]:
# Plot True values and Model Predictions with their proper range
y_predict_orig = scaler_y.inverse_transform(y_predict)
y_test_orig = scaler_y.inverse_transform(y_test)
plt.plot(y_test_orig, y_predict_orig, "^", color = 'r')
plt.xlabel('Model Predictions')
plt.ylabel('True Values')
plt.show()
In [74]:
# Import libraries
from sklearn.metrics import r2_score, mean_squared_error, mean_absolute_error
from math import sqrt
# Check evaluation metrics
k = X_test.shape[1]
n = len(X_test)
RMSE = float(format(np.sqrt(mean_squared_error(y_test_orig, y_predict_orig)),'.3f'))
MSE = mean_squared_error(y_test_orig, y_predict_orig)
MAE = mean_absolute_error(y_test_orig, y_predict_orig)
r2 = r2_score(y_test_orig, y_predict_orig)
adj_r2 = 1-(1-r2)*(n-1)/(n-k-1)
print('RMSE =',RMSE, '\nMSE =',MSE, '\nMAE =',MAE, '\nR2 =', r2, '\nAdjusted R2 =', adj_r2)
Conclusion¶
The Decision Tree Model was able to obtained a 98% accuracy in prediction and also outperforms an ANN model. Although the ANN model was only trained in five iterations and can bu further improve by adding more layers, it will still take a lot of time to train and consume a lot of computing power as well. Due to the good scores in evaluation metrics the model can be deploy into the production to save process engineers a lot of time in their workload.