
## Trying out Clustering with K-Means

<Callout type="success" heading={null}>
	In this walkthrough we can are going to explore K-means on a crime data set
	across all US states. Which states have similar crime profiles?
</Callout>

### Import Libraries

```python
import numpy as np
import pandas as pd

from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split

import matplotlib.pyplot as plt
%matplotlib inline
```

<AIExplainer title="Explain this code...">

#### Importing necessary libraries

```python
import numpy as np
```

This imports the **numpy** library and gives it an alias `np`. **numpy** is a library in Python used for scientific computing. It provides a high-performance multidimensional array object and tools for working with arrays.

```python
import pandas as pd
```

This imports the **pandas** library and gives it an alias `pd`. **pandas** provides data structures and data analysis tools in Python. It's often used for working with structured data.

```python
from sklearn.linear_model import LinearRegression
```

From the **sklearn** (Scikit-learn) library, which is a machine learning library, we're importing the `LinearRegression` module. Linear regression is a statistical method that allows us to study relationships between two continuous variables.

```python
from sklearn.model_selection import train_test_split
```

This imports the `train_test_split` function from **sklearn**. The function is used to split datasets into training and test subsets.

```python
import matplotlib.pyplot as plt
```

This imports the **matplotlib** library's `pyplot` module and gives it an alias `plt`. `matplotlib` is a plotting library, and `pyplot` provides a MATLAB-like interface for making plots and charts in Python.

```python
%matplotlib inline
```

This is a magic command in Jupyter Notebook (or Jupyter Lab). It makes sure that any plot or graph created will be displayed directly below the code cell, inside the notebook itself, rather than opening in a new window.

</AIExplainer>

### Import Data Set and have a look

Let us have a look at the data.

```python
df = pd.read_csv("data_arrest.csv", delimiter=",", index_col=0)

df.head()
```

<AIExplainer title="Explain this code...">
#### Reading the CSV file into a pandas DataFrame

```python
df = pd.read_csv("data_arrest.csv", delimiter=",", index_col=0)
```

Here, we're using the `read_csv` function from the pandas library to read in a CSV file named "data_arrest.csv".

The argument "`data_arrest.csv`" is the path to the CSV file we want to read.

delimiter="," specifies that the delimiter in the CSV file is a comma (`,`). This is the standard delimiter for CSV files, so often, this argument is not necessary to specify. However, it's useful to specify if you're unsure about the delimiter or if it might be something other than a comma.

`index_col=0` tells **pandas** to use the first column of the CSV file (columns are zero-indexed in Python, so `0` refers to the first column) as the index column for the DataFrame. The index in a DataFrame is a special column that can be used to label rows. It's particularly useful for time series data, unique IDs, or any other kind of sequential or unique data.

#### Displaying the first few rows of the DataFrame

```python
df.head()
```

Using the `head()` method on a DataFrame (`df` in this case) will display the first five rows of the DataFrame by default. This method is particularly useful for getting a quick glimpse of the data after reading it into Python, especially if the dataset is large. If you want to see more (or fewer) than five rows, you can provide a number as an argument, like `df.head(10)` to display the first ten rows.

</AIExplainer>

<Image
	src={
		"/images/courses/foundations-in-machine-learning/trying-out-clustering-with-k-means/img1.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={600}
	height={251}
	caption={""}
/>

Rescaling data is important when using K means. This is because the different inputs can have different scales. Hence if not rescaled some variables will have a greater influence than others.

Here we use the normal `MinMaxScaler()` which rescales the data from 0 to 1 for each input variable.

```python
from sklearn.preprocessing import MinMaxScaler

minmax = MinMaxScaler()

hat_df = crime_rates_standardized = pd.DataFrame(minmax.fit_transform(df))
```

<AIExplainer title="Explain this code...">
#### Scaling the DataFrame using Min-Max Scaling

```python
from sklearn.preprocessing import MinMaxScaler
```

This line imports the `MinMaxScaler` class from the **sklearn.preprocessing** module. Min-Max scaling is a method to scale and transform features so that they are within a specific range, usually between 0 and 1.

```python
minmax = MinMaxScaler()
```

Here, we initialize an instance of the `MinMaxScaler`. By default, this scaler will transform features to lie between 0 and 1.

```python
hat_df = crime_rates_standardized = pd.DataFrame(minmax.fit_transform(df))
```

In this line, we're doing multiple things:

1. `minmax.fit_transform(df)`:

- `fit` computes the minimum and maximum value to be used for scaling.
- `transform` scales the dataset.
  Combining these, `fit_transform` will compute the minimum and maximum to be used for scaling and then transform the dataset. 2. pd.DataFrame(...): The result of the scaling is an array. To convert this array back into a DataFrame, we wrap it with the pd.DataFrame function.

3. `hat_df = crime_rates_standardized = ...`: We're assigning the resulting DataFrame to two variable names: `hat_df` and `crime_rates_standardized`. This is a chained assignment. Both variables will now reference the same DataFrame object.

To improve clarity and readability, it might be better to separate the assignments:

```python
crime_rates_standardized = pd.DataFrame(minmax.fit_transform(df))
hat_df = crime_rates_standardized
```

This block ensures that the data in the `df` DataFrame is now scaled such that each feature (column) lies between 0 and 1.

</AIExplainer>

### Clustering with K Means

Here we now fit a K Means model, and we try out different values of K (the number of clusters). In each case we can calculate the inertia (the total distance from the data to the centroid of the cluster it is assigned).

```python
from sklearn.cluster import KMeans

KMeans()
plt.figure(figsize=(8, 6))
inertia = []
for i in range(1, 11):
    kmeans = KMeans(n_clusters = i, init = 'k-means++', n_init='auto',random_state = 123).fit(hat_df)

    inertia.append(kmeans.inertia_) # criterion based on which K-means clustering works
```

<AIExplainer title="Explain this code...">
#### Performing K-means Clustering and Examining Inertia for Different Cluster Numbers

```python
from sklearn.cluster import KMeans
```

This line imports the `KMeans` class from the **sklearn.cluster** module. K-means clustering is a type of unsupervised learning algorithm used to find groups in data.

```python
KMeans()
```

This line initializes an instance of the `KMeans` class, but it doesn't do anything else. If you're not using it for any purpose, it might be redundant and could be omitted.

```python
plt.figure(figsize=(8, 6))
```

Here, we're initializing a new figure with a specified size of 8 inches in width and 6 inches in height for subsequent plotting.

```python
inertia = []
```

We're initializing an empty list named `inertia`. Inertia or the sum of squared distances of samples to their closest cluster center is a measure that can be used to determine the optimal number of clusters for K-means. Generally, as the number of clusters increases, inertia decreases. However, the rate of decrease sharply changes at some point, creating an "elbow" in the graph. The "elbow" or point of inflection is typically considered a good estimate for the optimal number of clusters.

```python
for i in range(1, 11):
```

Starting a `for` loop that will iterate over cluster numbers from 1 to 10.

```python
kmeans = KMeans(n_clusters = i, init = 'k-means++', n_init='auto',random_state = 123).fit(hat_df)
```

For each iteration (i.e., for each cluster number from 1 to 10):

- A KMeans clustering model is created with the specified number of clusters.
- `init='k-means++'` ensures a smart initialization method, speeding up convergence.
- `n_init='auto'` is incorrect as `n_init` expects an integer. It defines how many time the k-means algorithm will be run with different centroid seeds. The final results will be the best output of `n_init` consecutive runs in terms of inertia. By default, it is set to 10.
- `random_state=123` ensures reproducibility of results.

```python
inertia.append(kmeans.inertia_)
```

After fitting the model, we append its inertia value to the `inertia` list.

Once the loop completes, plotting the inertia values against the number of clusters can help determine the optimal number of clusters. Look for the "elbow" point in the plot.

</AIExplainer >

We can then plot the 'elbow' curve and see a good choice of number of clusters

```python
plt.plot(range(1, 11), wcss)
plt.title('The Elbow Method')
plt.xlabel('Number of clusters')
plt.ylabel('Inertia')
plt.show()
```

<AIExplainer title="Explain this code...">
#### Visualizing the Elbow Method for Optimal Number of Clusters

The provided code aims to plot the inertia values to visually find the "elbow", which represents an optimal number of clusters for k-means clustering. However, there seems to be a small inconsistency: the list where you stored inertia values is named `inertia`, but the provided code refers to it as `wcss`. The term WCSS stands for "Within-Cluster-Sum-of-Squares", which is another term for inertia.

Let's break down and correct the code:

```python
plt.plot(range(1, 11), inertia)
```

Here, we're plotting the inertia values (on the y-axis) against the range of cluster numbers from 1 to 10 (on the x-axis). This line will create a line chart.

```python
plt.title('The Elbow Method')
```

Setting the title of the plot as "The Elbow Method".

```python
plt.xlabel('Number of clusters')
```

Setting the label for the x-axis as "Number of clusters".

```python
plt.ylabel('Inertia')
```

Setting the label for the y-axis as "Inertia".

```python
plt.show()
```

This line displays the created plot. It's especially useful in script-based environments, but in Jupyter Notebooks, the plot often shows up automatically without needing this line.

In summary, this plot will allow you to visually identify the "elbow" point, which suggests an optimal number of clusters. The elbow is the point after which the inertia starts decreasing in a linear fashion. Before this point, the inertia decreases rapidly. Choosing the number of clusters right at the elbow often results in a good trade-off between the number of clusters and the reduction in inertia.

</AIExplainer>

<Image
	src={
		"/images/courses/foundations-in-machine-learning/trying-out-clustering-with-k-means/img2.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={680}
	height={542}
	caption={""}
/>

Looking at the elbow curve we see that K = 4 seems a good choice. So let us fit that model again, and look at the results.

```python
model = KMeans(n_clusters = 4, init = 'k-means++', n_init = 'auto', random_state = 42)

clusters = model.fit_predict(hat_df)

scatter = plt.scatter(df['Murder'], df['Assault'], c=clusters)

plt.xlabel('Murders Rates')
plt.ylabel('Assaults Rates')
plt.show()
```

<AIExplainer title="Explain this code...">
#### Visualizing Clusters Using K-means for Murder and Assault Rates

Let's break down the code step by step:

```python
model = KMeans(n_clusters = 4, init = 'k-means++', n_init = 'auto', random_state = 42)
```

You are initializing a KMeans clustering model with:

- `n_clusters = 4`: indicating that you want to partition the data into 4 clusters.
- `init = 'k-means++`': indicating that the initial placement of the centroids should be done in an optimized manner to speed up the convergence.
- `n_init = 'auto'`: This appears to be an error. `n_init` expects an integer specifying the number of times the KMeans algorithm will run with different centroid seeds. The best output in terms of inertia will be retained. Typically, a default value like 10 is used. The string 'auto' is not a valid input here.
- `random_state = 42`: To ensure reproducibility of the clustering.

```python
clusters = model.fit_predict(hat_df)
```

Here, you're fitting the model to the scaled data `hat_df` and then obtaining cluster labels for each data point. The `fit_predict` method effectively does a fit followed by a `predict` on the same data.

```python
scatter = plt.scatter(df['Murder'], df['Assault'], c=clusters)
```

This line is creating a scatter plot using the original (unscaled) `Murder` and `Assault` rates from the `df` DataFrame. The colors of the data points (c=clusters) represent the cluster to which each data point was assigned by the KMeans algorithm.

```python
plt.xlabel('Murders Rates')
```

Setting the label for the x-axis as "Murders Rates".

```python
plt.ylabel('Assaults Rates')
```

Setting the label for the y-axis as "Assaults Rates".

```python
plt.show()
```

Displaying the created scatter plot. In environments like Jupyter Notebooks, the plot often shows up automatically without needing this line, but in script-based environments, it's necessary to show the plot.

In summary, this code clusters the data into 4 clusters based on the features in the `hat_df` DataFrame and then visualizes how these clusters look when plotted against the `Murder` and `Assault` rates.

</AIExplainer>

There seems like a clustering which divided by the amount of crime. We see a clear correlation between murder rates and assault rates (no really surprise here). The clustering is clear visible along this relationship.

Now let plot this data set on a map. We do this using plotly which you can install using

`pip install plotly-express`

First we need to convert state names into the abbrevation of two letters, and then add them to the data set along with the assigned cluster numbers.

```python
import plotly.express as px # You can install this using `pip install plotly-express`

us_state_to_abbrev = {
    "Alabama": "AL",
    "Alaska": "AK",
    "Arizona": "AZ",
    "Arkansas": "AR",
    "California": "CA",
    "Colorado": "CO",
    "Connecticut": "CT",
    "Delaware": "DE",
    "Florida": "FL",
    "Georgia": "GA",
    "Hawaii": "HI",
    "Idaho": "ID",
    "Illinois": "IL",
    "Indiana": "IN",
    "Iowa": "IA",
    "Kansas": "KS",
    "Kentucky": "KY",
    "Louisiana": "LA",
    "Maine": "ME",
    "Maryland": "MD",
    "Massachusetts": "MA",
    "Michigan": "MI",
    "Minnesota": "MN",
    "Mississippi": "MS",
    "Missouri": "MO",
    "Montana": "MT",
    "Nebraska": "NE",
    "Nevada": "NV",
    "New Hampshire": "NH",
    "New Jersey": "NJ",
    "New Mexico": "NM",
    "New York": "NY",
    "North Carolina": "NC",
    "North Dakota": "ND",
    "Ohio": "OH",
    "Oklahoma": "OK",
    "Oregon": "OR",
    "Pennsylvania": "PA",
    "Rhode Island": "RI",
    "South Carolina": "SC",
    "South Dakota": "SD",
    "Tennessee": "TN",
    "Texas": "TX",
    "Utah": "UT",
    "Vermont": "VT",
    "Virginia": "VA",
    "Washington": "WA",
    "West Virginia": "WV",
    "Wisconsin": "WI",
    "Wyoming": "WY",
    "District of Columbia": "DC",
    "American Samoa": "AS",
    "Guam": "GU",
    "Northern Mariana Islands": "MP",
    "Puerto Rico": "PR",
    "United States Minor Outlying Islands": "UM",
    "U.S. Virgin Islands": "VI",
}

abbrev_state = []
for state in df.index:
    abbrev_state.append(us_state_to_abbrev[state])

df['state_code'] = abbrev_state
df['clusters'] = clusters

df.head()
```

<AIExplainer title="Explain this code...">
#### Creating Abbreviated State Codes Column and Adding Clusters to DataFrame

Let's break down the provided code:

```python
import plotly.express as px # You can install this using `pip install plotly-express`
```

This imports the **plotly express** module, a simplified interface for creating visualizations using the Plotly library.

```python
us_state_to_abbrev = { ... }
```

This is a dictionary that maps the full name of US states and territories to their corresponding abbreviated codes.

```python
abbrev_state = []
for state in df.index:
    abbrev_state.append(us_state_to_abbrev[state])
```

Here, we are iterating over the index of the DataFrame `df`, which appears to be state names. For each state name, the corresponding abbreviation is looked up in the `us_state_to_abbrev` dictionary and appended to the `abbrev_state` list.

```python
df['state_code'] = abbrev_state
```

A new column named `state_code` is added to the `df` DataFrame, where each row will have the abbreviated state code.

```python
df['clusters'] = clusters
```

Another new column named `clusters` is added to the `df` DataFrame. This column stores the cluster assignments from the previously executed KMeans clustering.

```python
df.head()
```

This line displays the first few rows of the updated `df` DataFrame. With this, you'll be able to see the newly added columns, `state_code` and `clusters`, alongside the original data in the DataFrame.

In summary, the code takes the original data of US states, adds an abbreviation for each state, and then appends the cluster assignments from KMeans clustering to facilitate further analysis or visualization.

</AIExplainer>

<Image
	src={
		"/images/courses/foundations-in-machine-learning/trying-out-clustering-with-k-means/img3.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={800}
	height={227}
	caption={""}
/>

We can now plot this map using `plotly-express`

```python
fig = px.choropleth(df,
                    locations='state_code',
                    locationmode="USA-states",
                    scope="usa",
                    color='clusters',
                    color_continuous_scale="Viridis_r",
                    )
fig.show()
```

<AIExplainer title="Explain this code...">
#### Creating a Choropleth Map with Plotly Express

This block of code visualizes the cluster assignments for US states on a choropleth map using the **plotly express** module. Let's dissect the code:

```python
fig = px.choropleth(df,
                    locations='state_code',
                    locationmode="USA-states",
                    scope="usa",
                    color='clusters',
                    color_continuous_scale="Viridis_r",
                    )
```

This line initializes the creation of a choropleth map using the provided DataFrame `df`.

- `locations='state_code'`,: Specifies which column in the DataFrame provides the locations that the map will be based on. Here, the `state_code` column (which contains abbreviated state codes) is used.

- `locationmode="USA-states"`,: This parameter tells the choropleth function how to interpret the `locations` mentioned earlier. "USA-states" means that the locations provided are US state abbreviations.

- `scope="usa"`,: This restricts the map's focus only to the USA.

- `color='clusters'`,: This parameter dictates which column in the DataFrame should be used to determine the colors of the regions. Here, the `clusters` column, which contains the cluster assignments, is used.

- `color_continuous_scale="Viridis_r"`,: Specifies the color scale for the map. "Viridis_r" is one of the predefined color scales in Plotly, and the "`_r`" indicates that the scale is in reverse order.

```python
fig.show()
```

Displays the created choropleth map.

In summary, this code visualizes how US states were clustered based on the previous KMeans analysis, presenting this information on a map where each state's color intensity represents its cluster assignment.

</AIExplainer>

<Image
	src={
		"/images/courses/foundations-in-machine-learning/trying-out-clustering-with-k-means/img4.jpg"
	}
	alt={"academy.digilab.co.uk"}
	width={1000}
	height={627}
	caption={""}
/>


Walking through an example of K Means in Pythong

Please log in to continue