Background¶
My friend is interested in acquiring an apartment building in particular locations like Humboldt Park, Lincoln Park, Logan Square, and West Town areas. I want to help by predicting prices, so I can tell him if properties are good deals or not. He gave me some information with 2-4 Unit and 5+ Unit apartment buildings in areas he wants to acquire. The given dataset has 27 variables and 125 observations.
Data Preparation/Cleaning¶
import seaborn as sns
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
%matplotlib inline
df = pd.read_csv('properties.csv')
df.head()
df.shape
We can see from the upcoming heatmap that some of the columns entirely lost. We can drop them; we also may toss out columns that only have text information; they will be hard to interpret for our future model.
sns.heatmap(df.isnull(),yticklabels=False,cbar=False,cmap='viridis')
df.columns
df.drop(['SOLD DATE','SQUARE FEET','$/SQUARE FEET','HOA/MONTH',
'NEXT OPEN HOUSE END TIME','NEXT OPEN HOUSE END TIME',
'URL','SOURCE', 'MLS#', 'LATITUDE', 'LONGITUDE', 'FAVORITE',
'STATUS','NEXT OPEN HOUSE START TIME','INTERESTED',
'ADDRESS','CITY', 'STATE','ZIP'], axis=1, inplace=True)
df.head()
df.shape
For the sake of readability when we will be doing our further analysis, I renamed all the columns.
df = df.rename(columns={'SALE TYPE':'SALE_TYPE','PROPERTY TYPE':'PROPERTY_TYPE','LOT SIZE':'LOT_SIZE',
'YEAR BUILT':'YEAR_BUILT','DAYS ON MARKET':'DAYS_ON_MARKET'})
df.head()
df.info()
df.columns
We need to prepare our dataset for the machine learning process. For that, first I decided to turn our object values into dummies.
for column in ['SALE_TYPE', 'PROPERTY_TYPE', 'LOCATION']:
print(df[column].unique())
dummies = pd.get_dummies(df[column],prefix=column).iloc[:,1:]
df = pd.concat([df, dummies],axis=1)
Since we will be using only integers in our machine learning process, we can get rid of the columns we turned into dummies. In our case, we will drop three columns.
df.drop(['SALE_TYPE', 'PROPERTY_TYPE', 'LOCATION'], axis=1, inplace=True)
We also notice that the LOT_SIZE and YEAR_BUILT columns have some missing values and we need to fill them.
df.isnull().sum()
If I drop the rows with missing values, I might end up losing valuable data. Since we don't have abundant observations, it would be better to try to keep as much data as we can.
df[['YEAR_BUILT', 'LOT_SIZE']].median()
df[['YEAR_BUILT', 'LOT_SIZE']]= df[['YEAR_BUILT', 'LOT_SIZE']].fillna(df[['YEAR_BUILT', 'LOT_SIZE']].median())
sns.heatmap(df.isnull(),yticklabels=False,cbar=False,cmap='viridis')
df.shape
Now we left with clean dataset for our future exploration.
df.head()
Data Visualization¶
We see from the next plot that 'BEDS', 'BATHS' variables are strongly correlated with PRICE.
sns.pairplot(df, x_vars=['BEDS', 'BATHS'], y_vars='PRICE', size=7, aspect=0.7, kind='reg')
sns.pairplot(df, x_vars=['YEAR_BUILT'], y_vars='PRICE', size=7, aspect=0.7, kind='reg')
This seems a little odd, and normally I would check with my friend to see if this data is incomplete. But for the sake of the exercise, I won't dig into it.
When we wanted to assess the correlation between LOT_SIZE and PRICE we see something strange happens. Let's find out.
sns.pairplot(df, x_vars=['LOT_SIZE'], y_vars='PRICE', size=7, aspect=0.7, kind='reg')
I decided to check the distribution of our LOT_SIZE by making a seaborn plot.
sns.distplot(df["LOT_SIZE"])
The chart above shows that one of the houses is 400 million square feet, which is too large to be correct. Wikipedia indicates that the largest homes in the world are only about 200,000 square feet.
Normally, I would check in with the data owner to understand more about why this value would appear, but in this case, I decided to proceed with the analysis after removing properties with LOT_SIZE values that were obviously too large (e.g. over 200000 square feet).
df.loc[df.LOT_SIZE>200000]
If we exclude these found bad data, we can see the improvement in our distribution.
df = df.loc[df.LOT_SIZE<200000]
df.shape
sns.distplot(df.loc[df["LOT_SIZE"] < 300000, "LOT_SIZE"])
The line still doesn't seem to follow the dots as closely as you'd expect, and it's probably because of that significant outlier on the right.
sns.pairplot(df, x_vars=['LOT_SIZE'], y_vars='PRICE', size=7, aspect=0.7, kind='reg')
Also, a one might expect a price decrease, on the properties, have been held on the market for a long time; we can see from our plot that this is not an attribute that affects the price much. I can think of that as an unwillingness of the property owner's to bend the price under the given amount. However, using an agent's service and getting a 5+ Unit Multi-Family properties indeed affect the price.
sns.pairplot(df, x_vars=['DAYS_ON_MARKET','SALE_TYPE_MLS Listing', 'PROPERTY_TYPE_Multi-Family (5+ Unit)'], y_vars='PRICE', size=7, aspect=0.7, kind='reg')
Let's analyze the effect of locations on our price.
df.head()
df[['LOCATION_Humboldt Park', 'LOCATION_Lincoln Park',
'LOCATION_Logan Square', 'LOCATION_West Town']].idxmax(axis=1).head()
From our violin plot, we see the highest medium price falls into a Lincoln Park location range and the least expensive into a Humboldt Park.
sns.violinplot(y=df[['LOCATION_Humboldt Park', 'LOCATION_Lincoln Park',
'LOCATION_Logan Square', 'LOCATION_West Town']].idxmax(axis=1), x=df['PRICE'], orient='h')
From our distribution plot below, we notice that no one is selling cheap houses and a few houses that are over $2,000,000.
sns.distplot([df['PRICE']])
plt.xlabel('PRICE')
plt.title('Distribution of the PRICE')
Machine Learning (Random Forest regression)¶
I will use a Random Forest Regressor since the target variable is a continuous real number.
from sklearn.ensemble import RandomForestRegressor
from sklearn.model_selection import train_test_split
X = df.drop('PRICE',axis=1)
y = df['PRICE']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
print(X_train.shape)
print(y_train.shape)
print(X_test.shape)
print(y_test.shape)
model = RandomForestRegressor()
model.fit(X_train, y_train)
y_pred = model.predict(X_test)
From here we can build our plot and add the prediction line.
plt.scatter(y_pred, y_test)
plt.xlabel('Prediction')
plt.ylabel('Real value')
diagonal = np.linspace(0, np.max(y_test), 100)
plt.plot(diagonal, diagonal, '-r')
plt.show()
from sklearn.metrics import mean_squared_log_error, mean_absolute_error
print('MAE:\t$%.2f' % mean_absolute_error(y_test, y_pred))
print('MSLE:\t%.5f' % mean_squared_log_error(y_test, y_pred))
From the table below we can evaluate the importance of our features.
feature_evaluation = pd.DataFrame(100*model.feature_importances_,X.columns).sort_values(by=0, ascending=False)
feature_evaluation.columns = ['feature_importance']
feature_evaluation
Linear Regression¶
Let's try a Linear Regression model and compare it with RF.
from sklearn.linear_model import LinearRegression
reg = LinearRegression()
reg.fit(X_train, y_train)
lin_pred = reg.predict(X_test)
plt.scatter(lin_pred, y_test)
plt.xlabel('Prediction')
plt.ylabel('Real value')
diagonal = np.linspace(0, np.max(y_test), 100)
plt.plot(diagonal, diagonal, '-r')
plt.show()
print('MAE:\t$%.2f' % mean_absolute_error(y_test, lin_pred))
print('MSLE:\t%.5f' % mean_squared_log_error(y_test, lin_pred))
A deviation of $188805.56,- which is mainly due to the extreme outliers.
We can evaluate the coefficients of each feature as well.
coefficients = pd.DataFrame(reg.coef_,X_train.columns).sort_values(by=0,ascending=False)
coefficients.columns = ['coefficients']
coefficients
Conclusion¶
We can see some coefficient discrepancies in both algorithms. There are some clear outliers in the dataset that we are predicting very badly, so it would be a good idea to understand why we're so far off with those.
Housing prices are correlated with location. With neighborhoods like Lincoln Park, it's far more expensive on average than neighborhoods like Humboldt Park. It's also apparent that the type and age of the property affect the price as well. 5+ Unit type properties cost more expensive compared to 2-4 Unit. If the data were available, it could be illuminating to analyze the impact of features like the presence or absence of a garage or porch, the overall condition of the house, air quality and safety of the neighborhood and so on.