Background¶
I worked with the "CTA_Ridership" dataset that I took from a data portal provided by the City of Chicago, which contains the number of rides per day for each CTA station name/ID since 2001. I used this data to assess CTA ridership and how it evolved from the beginning of 2001 until September of 2008.
Data Preparation/Cleaning¶
To get started I downloaded essential libraries of Python.
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from matplotlib.pyplot import figure
%matplotlib inline
data = pd.read_csv('CTA_-_Ridership_-__L__Station_Entries_-_Daily_Totals_UPDATED.csv')
data.head()
data.shape
Dataset has three object types and two integers.
data.info()
data.rename(columns={'stationname':'Station_Name','date':'Date','daytype':'Day_Type','rides':'Rides'},inplace=True)
data.head()
Here I'm comparing Saturdays with Sundays and Holidays. A - Saturday, U - Sundays and Holidays.
a_u = data.loc[data.Day_Type.isin(["A","U"]), ['Day_Type','Rides']].groupby('Day_Type').sum()
a_u
from matplotlib.ticker import FuncFormatter
ind = np.arange(len(a_u))
plt.bar(ind, height=a_u.Rides)
plt.xticks(ind, ["Saturdays", "Sundays/Holidays"])
plt.title("Total Rides by Day Type")
def millions(val, pos):
return f'{val*1e-6:.0f}M'
formatter = FuncFormatter(millions)
plt.gca().yaxis.set_major_formatter(formatter)
plt.ylabel("Number of Rides")
Here we see that Saturday has a higher average ridership than Sundays and Holidays.
for_sns = data.loc[data.Day_Type.isin(["U","A"]), ['Day_Type','Rides']]
sns.barplot(x='Day_Type',y='Rides',data=for_sns, order=["A", "U"])
To make further analysis more efficient, I converted the date column's object values to a non-null DateTime using a specific function, then derived years, months and weekdays by creating new variables for each.
import datetime
data.Date = pd.to_datetime(data.Date)
data.head()
data['Year'] = data.Date.dt.year
data['Month'] = data.Date.dt.month
data['Weekday_Name'] = data.Date.dt.weekday_name
data.head()
This data frame includes reports for 6481 days.
data.Date.max()-data.Date.min()
Data Visualization¶
It's evident from the upcoming plot that CTA ridership has been steadily increasing over the years.
grouped = data.groupby(['Year']).Rides.sum()
grouped.loc[grouped.index != 2018].plot(kind="line", y="Rides")
plt.ylim(bottom=0)
by_year = data.loc[data.Year !=2018].groupby('Year').Rides.sum()
by_year = by_year.to_frame()
by_year = by_year.reset_index()
by_year
fig, ax = plt.subplots()
index = ['2001','2002','2003','2004','2005','2006','2007','2008','2009','2010',
'2011','2012','2013','2014','2015','2016','2017']
plt.bar(index, by_year['Rides'])
plt.title('Rides over the years')
plt.xlabel('Year')
plt.ylabel('Rides')
plt.xticks(rotation=50)
Here I am creating a multi-index data frame for a heatmap.
multi = data.loc[data.Year!=2018].groupby(['Year','Month']).Rides.sum()
multi_plot = multi.unstack()
multi_plot
After generally rising from 2004 to 2015, CTA ridership dropped steadily in 2015 and early 2016.
multi_plot.sum(axis=1).plot()
plt.ylim(bottom=0)
multi_plot.loc[:, :].plot()
One might expect to see a significant decrease in the summertime because people are more active in the warm weather. They can walk, bike or skate, etc., to get to the desired spot, but from the plot below, we observe the most significant decrease happens during the winter time, especially January.
Why do we take a train? To get to the work, school, or seek entertainment. Admittedly, if the weather is unpleasant, and I can afford to use a Lyft or Uber service, I would instead do that than walk to the train station.
multi_plot.loc[:, [1, 3, 7, 10]].plot()
sns.heatmap(multi_plot,cmap="YlGnBu")
Ten busiest stations.¶
It's not surprising that the downtown stations are the busiest given the number of people that work and seek entertainment near those stations. A notable exception is O'Hare, which is understandable given that the CTA provides a convenient way to access one of the busiest airports in the world.
by_station = data[['Station_Name','Rides']]
by_station = by_station.groupby('Station_Name').sum()
by_station = by_station.sort_values(by='Rides', ascending=False)
by_station.head(10)
Ten least busy stations.¶
The low ridership at the Washington/Wabash, which is the downtown area, is explainable. It didn't open until August 31, 2017.
by_station.tail(10)
My next interest is to find out the cause of the decrease for the CTA ridership last couple of years. I found an 'L' stops dataset from the same data portal that provided me with location and basic service availability information for each place on the CTA system where a train stops, along with formal station names and stops descriptions. I also think that decrease could occur on only particular lines. Let's investigate it further with the new dataset.¶
stops_data = pd.read_csv("CTA_-_System_Information_-_List_of__L__Stops.csv")
stops_data.shape
stops_data.info()
stops_data.head()
stops_data.columns
For the better readability I converted column names. I decided to work only with line's columns to find out certain lines where decrease has occured.
stops_data.rename({"G": "GREEN", "BRN": "BROWN", "P": "PURPLE", "Y": "YELLOW", "Pnk": "PINK",
"O": "ORANGE"}, axis="columns", inplace=True)
stops_data.head()
colors = ['RED', 'BLUE', 'GREEN','BROWN', 'PURPLE', 'YELLOW', 'PINK', 'ORANGE']
stops = stops_data.groupby('MAP_ID').sum()[colors]
stops = stops > 0
stops['line_count'] = stops.apply(lambda x: sum(x), axis=1)
stops.head()
Here I'm merging stops dataset with the previous one.
merged = data.merge(stops, how="inner", left_on="station_id", right_index=True)
merged.shape
Here I divided Rides by line count so we can get the number of rides for every single line.
merged["rides_split"] = merged['Rides'] / merged["line_count"]
merged.head()
tidy = merged.melt(id_vars=["rides_split", "Year"], value_vars=colors)
agg = tidy.loc[tidy.value == True].groupby(["variable", "Year"]).sum()[['rides_split']].reset_index().pivot(index="Year", columns="variable", values="rides_split")
agg
Here we see an abrupt decrease in the yellow line in 2015. If you google, you'll run into an article that says; there was a five-month shutdown in the CTA Yellow Line this year.
agg.loc[agg.index != 2018, ["YELLOW"]].plot(kind="line", legend="right")
We also notice from the following plot that ridership on the pink line had increased at a fast speed during 2004-2015. Brown line is the least active line.
(agg / agg.loc[2001]).loc[agg.index != 2018].plot(kind="line", legend="right")
Conclusion¶
From the analysis above, we were able to extract insightful information like a decrease or increase of the ridership for particular lines. What could be the primary influence on the decline of ridership on some lines? Can you consider that it might be a matter of safety, that people try to avoid taking trains from certain lines or it's just a matter of location?! My further goal is to try to answer this question using "Crime data" from the Chicago data portal.
df = pd.read_csv('Crimes_-_2001_to_present.csv')
df.head()
df.rename(columns={'Description':'description'}, inplace=True)
df.rename(columns={'Location Description':'location_description'}, inplace=True)
df.info()