Python modules

What is a module and why use it?

A module in Python is a file that contains reusable code - such as functions, classes, and variables that can be imported into other scripts.

Modules help organize code, avoid repetition, and provide specialized tools for different tasks.

Modules on this document

Numpy

NumPy is a powerful linear algebra library Python module for working with vectors, matrix, and arrays. It is widely used for scientific computing, data analysis, and machine learning because it performs mathematical operations extremely fast.

Panda

Pandas is an open-source library built as an extension of numpy for data manipulation and analysis in Python. It allows fast data cleaning, preparation, and visualization, and supports many input and output data sources.

Seaborn

Seaborn is a powerful Python library for creating statistical data visualizations. It is built on top of Matplotlib and provides a high-level interface to create elegant and informative charts with minimal code.

Plotly and Cufflinks

Plotly is a library for creating interactive visualizations in Python. It allows you to create rich, web-based graphics directly from Jupyter Notebooks or scripts.

Cufflinks connects Plotly with Pandas, making it possible to generate interactive plots directly from dataframes with simple syntax.

Scikit learn

Scikit learn includes pre-built algorithms and tools for performing machine learning tasks.

They save time, ensure reliability, and allow developers to focus on solving data problems instead of writing algorithms from scratch.

nltk

It is a powerful Python library for text analysis.

---

NumPy Module

NumPy is a powerful linear algebra library Python module for working with vectors, matrix, and arrays. It is widely used for scientific computing, data analysis, and machine learning because it performs mathematical operations extremely fast.

import numpy as np

This line imports the NumPy module and gives it the short alias np, so we can call its functions like `np.array()` or `np.arange()`.

Installing NumPy

pip install numpy

Creating Arrays

To start, import NumPy:

import numpy as np

You can create a 1D array (vector) from a list:

my_list = [1, 2, 3, 4]
array = np.array(my_list)
print(array)
# [1 2 3 4]

You can also create a 2D array (matrix) from a list of lists:

list2 = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
matrix = np.array(list2)
print(matrix)

Array generation functions

NumPy offers several ways to generate arrays automatically.

np.arange()

Creates a range of numbers.

import numpy as np
np.arange(5)         # [0 1 2 3 4]
np.arange(2, 10)     # [2 3 4 5 6 7 8 9]
np.arange(2, 10, 2)  # [2 4 6 8]

np.zeros() and np.ones()

Generate arrays filled with zeros or ones.

np.zeros((2, 4))  # 2 rows, 4 columns of zeros
np.ones((3, 3))   # 3x3 matrix of ones

np.linspace()

Creates evenly spaced numbers between two values.

np.linspace(3, 9, 5)
# [3. 4.5 6. 7.5 9.]

np.eye()

Creates an identity matrix (ones on the diagonal, zeros elsewhere).

np.eye(5)

Random numbers

NumPy’s random module can generate random values for testing or simulations.

np.random.rand()

Generates random numbers between 0 and 1 (uniform distribution).

np.random.rand(2, 3)

np.random.randn()

Generates random numbers from a normal (Gaussian) distribution.

np.random.randn(4, 5)

np.random.randint()

Generates random integers between two values.

np.random.randint(1, 20)       # One random integer
np.random.randint(1, 20, 5)    # Array of 5 random integers

Reshaping arrays

You can change the shape (rows and columns) of an array using reshape(), as long as the total number of elements matches.

arr = np.random.randint(1, 100, 25)
matrix = arr.reshape(5, 5)
print(matrix)

Maximum, minimum and index positions

Use these functions to find key values and their positions in arrays.

arr = np.random.randint(1, 100, 50)
print(arr.max())      # Maximum value
print(arr.argmax())   # Index of maximum
print(arr.min())      # Minimum value
print(arr.argmin())   # Index of minimum

Indexing and slicing

Access individual or multiple elements of arrays using indices.

One-Dimensional Arrays

arr = np.arange(0, 10)
print(arr[0])     # First element
print(arr[-1])    # Last element
print(arr[2:5])   # Slice from index 2 to 4

Make copies safely:

arr2 = arr.copy()
arr2[0] = 100
print(arr)   # Original unchanged

Two-dimensional arrays

Access elements by [row, column].

matrix = np.array([[1,2,3],[4,5,6],[7,8,9]])
print(matrix[0, 0])   # 1
print(matrix[1, 2])   # 6
print(matrix[:, 0])   # All rows, first column
print(matrix[:2, :2]) # Top-left 2x2 submatrix

Conditional selection

You can select elements based on conditions.

arr = np.arange(0, 10)
print(arr[arr > 5])   # Elements greater than 5
print(arr[arr % 2 == 0])  # Even elements

Array operations

You can perform arithmetic directly on arrays; operations are applied element-wise.

arr = np.arange(0, 10)
print(arr + 10)
print(arr * 2)
print(arr - arr)

Between two arrays:

a = np.arange(0, 10)
b = np.arange(5, 15)
print(b - a)

Common mathematical functions

NumPy provides built-in math functions.

arr = np.arange(0, 10)
print(np.sqrt(arr))   # Square root
print(np.max(arr))    # Maximum
print(np.min(arr))    # Minimum

---

pandas module

pandas is an open-source library built as an extension of numpy for data manipulation and analysis in Python.

It allows fast data cleaning, preparation, and visualization, and supports many input and output data sources.

Install it with:

pip install pandas

pandas series

A series in pandas is a one-dimensional labeled array that can hold any type of data (integers, strings, floats, etc.). Each element has a label, known as its index.

import numpy as np
import pandas as pd

labels = ['a', 'b', 'c']
data = [1, 2, 3]

series = pd.Series(data=data, index=labels)
print(series)

a    1
b    2
c    3

You can also create a series from a numpy array:

array = np.arange(5, 8)
labels = ['sale1', 'sale2', 'sale3']
series = pd.Series(array, index=labels)
print(series)

sale1    5
sale2    6
sale3    7

Access values by **label** or **position**:

print(series['sale1'])   # access by label
print(series[0])         # access by position

Series can hold different data types:

data = ['computer', 300, 'printer', 400]
series = pd.Series(data)
print(series)

0    computer
1         300
2     printer
3         400

You can perform operations between series:

s1 = pd.Series([1, 2, 3], index=['a', 'b', 'c'])
s2 = pd.Series([4, 8, 5], index=['a', 'b', 'c'])
sum_series = s1 + s2
print(sum_series)

a     5
b    10
c     8

dataframes

A **dataframe** is a two-dimensional table with rows and columns, similar to an Excel spreadsheet.

import pandas as pd

rows = ['sales1', 'sales2', 'sales3']
cols = ['zoneA', 'zoneB', 'zoneC']
data = [
    [123, 400, 200],
    [422, 234, 120],
    [256, 450, 210]
]

df = pd.DataFrame(data, index=rows, columns=cols)
print(df)

        zoneA  zoneB  zoneC
sales1    123    400    200
sales2    422    234    120
sales3    256    450    210

You can select rows or columns:

print(df.loc['sales1'])          # select one row
print(df.loc[['sales1','sales2']]) # select multiple rows
print(df['zoneA'])               # select one column
print(df[['zoneA','zoneC']])     # select multiple columns

Add a new column:

df['total_sales'] = df['zoneA'] + df['zoneB'] + df['zoneC']

        zoneA  zoneB  zoneC  total_sales
sales1    123    400    200          723
sales2    422    234    120          776
sales3    256    450    210          916

Delete a column permanently:

df.drop('total_sales', axis=1, inplace=True)

Delete a row permanently:

df.drop('sales3', inplace=True)

conditional selection

Conditional selection allows you to filter data that meets specific conditions.

condition = df > 200
print(df[condition])

        zoneA  zoneB  zoneC
sales1    NaN    400    NaN
sales2  422.0    234    NaN
sales3  256.0    450  210.0

Filter by column:

print(df[df['zoneA'] > 200])

        zoneA  zoneB  zoneC
sales2    422    234    120
sales3    256    450    210

Combine multiple conditions:

print(df[(df['zoneA'] > 200) & (df['zoneB'] > 300)])
print(df[(df['zoneA'] > 200) | (df['zoneB'] > 300)])

modifying index names

You can rename the row labels (index) of a dataframe.

df['days'] = ['day1', 'day2', 'day3']
df = df.set_index('days')
print(df)

handling null values

Pandas provides tools to remove or replace missing values (NaN).

import numpy as np
import pandas as pd

data = {'A':[4,5,np.nan], 'B':[6,1,5], 'C':[np.nan,4,np.nan]}
df = pd.DataFrame(data)
print(df)

     A  B    C
0  4.0  6  NaN
1  5.0  1  4.0
2  NaN  5  NaN

Remove rows or columns with nulls:

print(df.dropna())          # remove rows with NaN
print(df.dropna(axis=1))    # remove columns with NaN

     A  B    C
1  5.0  1  4.0

   B
0  6
1  1
2  5

Replace nulls:

df.fillna(value=100)
df.fillna(value=df.mean())

       A  B      C
0    4.0  6  100.0
1    5.0  1    4.0
2  100.0  5  100.0

     A  B    C
0  4.0  6  4.0
1  5.0  1  4.0
2  4.5  5  4.0

grouping data

Grouping allows you to aggregate data by one or more columns.

data = {'day':[1,1,2,2,3],
        'seller':['Antonio','Maria','Jose','Maria','Antonio'],
        'sales':[100,400,200,500,300]}
df = pd.DataFrame(data)

print(df.groupby('day').mean())
print(df.groupby('day').sum())
print(df.groupby('day').describe())

           seller  sales
day                     
1    AntonioMaria    500
2       JoseMaria    700
3         Antonio    300

    sales                                                      
    count   mean         std    min    25%    50%    75%    max
day                                                            
1     2.0  250.0  212.132034  100.0  175.0  250.0  325.0  400.0
2     2.0  350.0  212.132034  200.0  275.0  350.0  425.0  500.0
3     1.0  300.0         NaN  300.0  300.0  300.0  300.0  300.0

combining dataframes

You can concatenate dataframes vertically or horizontally.

df1 = pd.DataFrame({'A':[1,2,3], 'B':[4,5,6], 'C':[7,8,9]})
df2 = pd.DataFrame({'A':[11,12,13], 'B':[14,15,16], 'C':[17,18,19]})

print(pd.concat([df1, df2]))            # vertical
print(pd.concat([df1, df2], axis=1))    # horizontal

    A   B   C
0   1   4   7
1   2   5   8
2   3   6   9
0  11  14  17
1  12  15  18
2  13  16  19

   A  B  C   A   B   C
0  1  4  7  11  14  17
1  2  5  8  12  15  18
2  3  6  9  13  16  19

merging dataframes

Merging joins two dataframes based on a common key.

df1 = pd.DataFrame({'A':[1,2,3],'B':[4,5,6],'key':['C1','C2','C3']})
df2 = pd.DataFrame({'C':[7,8,9],'D':[10,11,12],'key':['C1','C2','C3']})

merged = pd.merge(df1, df2, on='key')
print(merged)

   A  B key  C   D
0  1  4  C1  7  10
1  2  5  C2  8  11
2  3  6  C3  9  12

joining dataframes

Joining combines dataframes by their index labels.

df1 = pd.DataFrame({'A':[1,2,3],'B':[4,5,6]}, index=['i1','i2','i3'])
df2 = pd.DataFrame({'C':[11,12,13],'D':[14,15,16]}, index=['i1','i2','i3'])

joined = df1.join(df2)
print(joined)

    A  B   C   D
i1  1  4  11  14
i2  2  5  12  15
i3  3  6  13  16

dataframe operations

You can perform operations such as counting unique values, applying functions, or sorting.

df = pd.DataFrame({'A':[11,12,10,12],'B':[14,18,16,17]}, index=['i1','i2','i3','i4'])
print(df['A'].unique())
print(df['A'].nunique())
print(df['A'].value_counts())

[11 12 10]

3

A
12    2
11    1
10    1

Apply a function:

def multiply(x): return x * 2
print(df['A'].apply(multiply))
print(df['A'].apply(lambda x: x * 2))

i1    22
i2    24
i3    20
i4    24

i1    22
i2    24
i3    20
i4    24

Sort values:

df.sort_values(by='B')

reading and writing excel files

Pandas can read and write Excel files easily.

import pandas as pd

df = pd.read_excel('example.xlsx')
print(df)

df['E'] = [20,22,23]
df.to_excel('output.xlsx', sheet_name='Sheet1')

reading html tables

You can extract data from HTML tables directly into pandas dataframes.

import pandas as pd

url = 'https://en.wikipedia.org/wiki/List_of_countries_and_dependencies_by_population'
tables = pd.read_html(url)
df = tables[0]
print(df.head(10))

working with sql databases

You can write and read dataframes from SQL databases.

from sqlalchemy import create_engine
import pandas as pd

df = pd.DataFrame({'A':[1,2,3],'B':[4,5,6]})
engine = create_engine('sqlite:///:memory:')
df.to_sql('table', con=engine, index=False)
data = pd.read_sql('table', con=engine)
print(data)

plotting dataframes

Pandas integrates with matplotlib to create visualizations directly from dataframes. Plots generated by Matplotlib (and by extension, Pandas plotting functions) are not automatically displayed in separate windows. plt.show() opens all the figures that have been created and displays them.[5][6] Your script will pause until you close all the plot windows. See Matplotlib module for further reference

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

df = pd.DataFrame(np.random.randint(0,200,(50,4)), columns=list('ABCD'))

df['A'].hist()
df.plot.area(alpha=0.3)
df.plot.bar(stacked=True)
df.plot.scatter(x='A', y='B', c='C', cmap='coolwarm')
df.plot.box()
df.plot.hexbin(x='A', y='B', gridsize=10)
df.plot.kde()
plt.show()

matplotlib module

matplotlib is a Python library for creating visualizations and plots from numerical data stored in lists or arrays. It provides full control over every aspect of a chart. from layout and colors to markers and line styles.

It is one of the most widely used libraries for data visualization in Python.

To install it use:

pip install matplotlib

basic example

A simple example of a line plot using matplotlib.

import numpy as np
import matplotlib.pyplot as plt

x = np.linspace(2, 9, 10)
y = x ** 2

plt.plot(x, y)
plt.show()

You can add colors and labels:

plt.subplot(1, 2, 1)
plt.plot(x, y, color='green')

plt.subplot(1, 2, 2)
plt.plot(y, x, color='blue')

plt.show()

subplots

Subplots allow displaying multiple plots in the same figure.

This creates two plots side by side (1 row, 2 columns).

To stack them vertically:

plt.subplot(2, 1, 1)
plt.plot(x, y, color='red')

plt.subplot(2, 1, 2)
plt.plot(y, x, color='blue')
plt.show()

object oriented approach

Matplotlib can also be used in an object-oriented way, providing more control.

import matplotlib.pyplot as plt
import numpy as np

x = np.linspace(2, 9, 10)
y = x ** 2

fig = plt.figure()
axes = fig.add_axes([0.1, 0.1, 0.8, 0.8])
axes.plot(x, y, color='green')
plt.show()

The list `[0.1, 0.1, 0.8, 0.8]` defines the left margin, bottom margin, width, and height of the axes as percentages of the figure size.

You can add multiple plots inside the same figure:

fig = plt.figure()

# main plot
ax1 = fig.add_axes([0.1, 0.1, 0.9, 0.9])
ax1.plot(x, y, color='red')

# inset plot
ax2 = fig.add_axes([0.15, 0.5, 0.3, 0.3])
ax2.plot(y, x, color='blue')
plt.show()

multiple plots in a grid

You can create several plots organized in rows and columns. This creates four subplots in a 2x2 grid.

fig, axes = plt.subplots(2, 2)

axes[0, 0].plot(x, y, color='blue')
axes[0, 0].set_title('blue')

axes[0, 1].plot(x, y, color='red')
axes[0, 1].set_title('red')

axes[1, 0].plot(x, y, color='yellow')
axes[1, 0].set_title('yellow')

axes[1, 1].plot(x, y, color='green')
axes[1, 1].set_title('green')

plt.show()

figure size

You can set the overall figure size using the figsize parameter.

fig = plt.figure(figsize=(8, 4))
ax = fig.add_axes([0.1, 0.1, 0.8, 0.8])
ax.plot(x, y, color='red')
plt.show()

The `figsize` values are in inches (width, height). For example, `(8, 4)` creates a rectangular figure 8 inches wide and 4 inches tall.

multiple lines and legends

You can draw multiple lines in a single plot and add a legend.

fig = plt.figure()
ax = fig.add_axes([0.1, 0.1, 0.8, 0.8])

x = np.linspace(0, 10, 20)
ax.plot(x, x**2, label='squared')
ax.plot(x, x**3, label='cubed')

ax.legend(loc=0)
plt.show()

The `loc=0` option automatically finds the best location for the legend.

colors

Matplotlib accepts many color formats, by name, shorthand, or hexadecimal code.

ax.plot(x, y, color='tomato')
ax.plot(x, y, color='#00FF00')  # hex code for green
ax.plot(x, y, color='lime')

You can find all available named colors in the official matplotlib color reference.

---

line styles

You can customize the appearance of plot lines, thickness, transparency, and style.

fig = plt.figure()
ax = fig.add_axes([0, 0, 1, 1])

ax.plot(x, y, color='red', linewidth=5, alpha=0.5, linestyle='--')
plt.show()

- **linewidth** controls thickness - **alpha** controls transparency (0 invisible, 1 fully visible) - **linestyle** can be `'--'`, `':'`, `'-.'`, or `'-'`

---

markers

Markers highlight data points on a plot.

fig = plt.figure()
ax = fig.add_axes([0, 0, 1, 1])

ax.plot(x, y, color='green', marker='o', markersize=5)
plt.show()

You can choose from various marker symbols: `'o'` (circle), `'s'` (square), `'d'` (diamond), `'+'`, `'*'`, `'x'`, and many others.

ax.plot(x, y, color='blue', marker='s', markersize=8)

Seaborn module

Seaborn is a powerful Python library for creating statistical data visualizations. It is built on top of Matplotlib and provides a high-level interface to create elegant and informative charts with minimal code.

installing seaborn

Install it with the following command:

pip install seaborn

loading data

Seaborn comes with several sample datasets. One of the most popular is tips, which contains restaurant bills and tips.

import seaborn as sns
import matplotlib.pyplot as plt

# Load built-in dataset
tips = sns.load_dataset("tips")

# Show the first 10 rows
print(tips.head(10))

   total_bill   tip     sex smoker  day    time  size
0       16.99  1.01  Female     No  Sun  Dinner     2
1       10.34  1.66    Male     No  Sun  Dinner     3
2       21.01  3.50    Male     No  Sun  Dinner     3
3       23.68  3.31    Male     No  Sun  Dinner     2
4       24.59  3.61  Female     No  Sun  Dinner     4
5       25.29  4.71    Male     No  Sun  Dinner     4
6        8.77  2.00    Male     No  Sun  Dinner     2
7       26.88  3.12    Male     No  Sun  Dinner     4
8       15.04  1.96    Male     No  Sun  Dinner     2
9       14.78  3.23    Male     No  Sun  Dinner     2

distribution plots

A distribution plot shows how data values are distributed. We can use `sns.histplot()` (modern replacement for the old `distplot`) to visualize the frequency of numerical values.

import seaborn as sns
import matplotlib.pyplot as plt

tips = sns.load_dataset("tips")

# Histogram for total bill
sns.histplot(data=tips, x="total_bill", bins=20, kde=True)
plt.title("Distribution of Total Bill")
plt.show()

We can also plot the distribution of tips:

import seaborn as sns
import matplotlib.pyplot as plt

tips = sns.load_dataset("tips")

sns.histplot(data=tips, x="tip", bins=20, kde=True)
plt.title("Distribution of Tips")
plt.show()

comparing two variables with joinplot

`joinplot()` shows the relationship between two numeric variables, combining scatter plots and histograms.

import seaborn as sns
import matplotlib.pyplot as plt

tips = sns.load_dataset("tips")

sns.jointplot(data=tips, x="total_bill", y="tip")
plt.show()

You can also use other visual styles such as hexbin or kde:

import seaborn as sns
import matplotlib.pyplot as plt

tips = sns.load_dataset("tips")

sns.jointplot(data=tips, x="total_bill", y="tip", kind="hex")
sns.jointplot(data=tips, x="total_bill", y="tip", kind="kde")

plt.show()

pairplot

The pairplot function compares all numerical columns in a dataset two by two. It’s useful to see correlations between multiple variables.

import seaborn as sns
import matplotlib.pyplot as plt

tips = sns.load_dataset("tips")

sns.pairplot(tips)
plt.show()

You can add a categorical variable to color the points:

import seaborn as sns
import matplotlib.pyplot as plt

tips = sns.load_dataset("tips")

sns.pairplot(tips, hue="sex")
plt.show()

categorical plots

Seaborn makes it easy to plot categorical variables, such as gender or day.

barplot

Shows average values of a numerical variable for each category.

import seaborn as sns
import matplotlib.pyplot as plt

tips = sns.load_dataset("tips")

sns.barplot(data=tips, x="sex", y="total_bill")
plt.title("Average Total Bill by Gender")
plt.show()

countplot

Counts the number of occurrences of each category.

import seaborn as sns
import matplotlib.pyplot as plt

tips = sns.load_dataset("tips")

sns.countplot(data=tips, x="sex")
plt.title("Count by Gender")
plt.show()

boxplot

Displays the distribution of values using quartiles.

import seaborn as sns
import matplotlib.pyplot as plt

tips = sns.load_dataset("tips")

sns.boxplot(data=tips, x="day", y="total_bill", hue="smoker")
plt.title("Total Bill by Day and Smoker Status")
plt.show()

violinplot

Similar to boxplot but shows the full data distribution.

import seaborn as sns
import matplotlib.pyplot as plt

tips = sns.load_dataset("tips")

sns.violinplot(data=tips, x="day", y="total_bill", hue="smoker", split=True)
plt.title("Total Bill by Day (Smoker vs Non-Smoker)")
plt.show()

stripplot and swarmplot

`stripplot` shows all individual data points; `swarmplot` spreads them out to avoid overlap.

import seaborn as sns
import matplotlib.pyplot as plt

tips = sns.load_dataset("tips")

sns.stripplot(data=tips, x="day", y="total_bill", hue="sex", dodge=True)
plt.title("Total Bill by Day and Gender (Stripplot)")
plt.show()

sns.swarmplot(data=tips, x="day", y="total_bill", hue="sex", dodge=True)
plt.title("Total Bill by Day and Gender (Swarmplot)")
plt.show()

You can also combine them with other plots:

import seaborn as sns
import matplotlib.pyplot as plt

tips = sns.load_dataset("tips")

sns.violinplot(data=tips, x="day", y="total_bill", inner=None)
sns.swarmplot(data=tips, x="day", y="total_bill", color="k", alpha=0.6)
plt.title("Combined Violin and Swarm Plot")
plt.show()

heatmaps

A heatmap displays values as colors in a grid. Here we use the `flights` dataset to show the number of passengers by month and year.

import seaborn as sns
import matplotlib.pyplot as plt

flights = sns.load_dataset("flights")

# Convert to matrix form
flights_matrix = flights.pivot_table(index="month", columns="year", values="passengers")

# Plot heatmap
sns.heatmap(flights_matrix, cmap="coolwarm", linecolor="white", linewidths=2)
plt.title("Heatmap of Flights Passengers")
plt.show()

grid plots

Grid plots allow us to create multiple charts at once for comparison.

pairgrid

Provides full control over the type of plots used in a pairwise grid.

import seaborn as sns
import matplotlib.pyplot as plt

iris = sns.load_dataset("iris")

# Create a PairGrid
g = sns.PairGrid(iris)
g.map_upper(sns.scatterplot)
g.map_lower(sns.kdeplot)
g.map_diag(sns.histplot)
plt.show()

facetgrid

Creates a grid of plots based on categorical variables.

import seaborn as sns
import matplotlib.pyplot as plt

tips = sns.load_dataset("tips")

g = sns.FacetGrid(tips, col="time", row="smoker")
g.map(sns.histplot, "total_bill")
plt.show()

regression plots

Regression plots show relationships between variables and a fitted regression line.

import seaborn as sns
import matplotlib.pyplot as plt

tips = sns.load_dataset("tips")

# Basic regression
sns.lmplot(data=tips, x="total_bill", y="tip")
plt.title("Regression Plot: Total Bill vs Tip")
plt.show()

# Regression separated by gender
sns.lmplot(data=tips, x="total_bill", y="tip", hue="sex", markers=["o", "x"])
plt.title("Regression by Gender")
plt.show()

You can separate by columns or rows:

import seaborn as sns
import matplotlib.pyplot as plt

tips = sns.load_dataset("tips")

sns.lmplot(data=tips, x="total_bill", y="tip", col="day", row="sex")
plt.show()

And adjust the aspect and size:

import seaborn as sns
import matplotlib.pyplot as plt

tips = sns.load_dataset("tips")

sns.lmplot(data=tips, x="total_bill", y="tip", height=6, aspect=1)
plt.show()

styles and colors

Seaborn provides easy ways to change styles, remove borders, and customize colors.

import seaborn as sns
import matplotlib.pyplot as plt

tips = sns.load_dataset("tips")

sns.set_style("darkgrid")

sns.countplot(data=tips, x="sex")
plt.title("Count by Gender with Darkgrid Style")
plt.show()

# Remove borders
sns.set_style("white")
sns.despine(left=True, bottom=True)

sns.countplot(data=tips, x="smoker")
plt.title("Smoker Count with White Style")
plt.show()

customizing figure size and font

plt.figure(figsize=(10, 4))
sns.countplot(data=tips, x="day")
sns.set_context("notebook", font_scale=1.5)
plt.title("Count by Day with Larger Font")
plt.show()

changing color palettes

You can use built-in color palettes or create your own.

sns.set_palette("Set2")
sns.countplot(data=tips, x="sex")
plt.title("Using Set2 Palette")
plt.show()

# Custom colors
custom_colors = ["#90ee90", "#1e90ff"]
sns.set_palette(custom_colors)
sns.countplot(data=tips, x="sex")
plt.title("Custom Colors")
plt.show()

---

plotly and cufflinks modules

Plotly is a library for creating interactive visualizations in Python. It allows you to create rich, web-based graphics directly from Jupyter Notebooks or scripts.

Cufflinks connects Plotly with Pandas, making it possible to generate interactive plots directly from dataframes with simple syntax.

This allows you to access Plotly’s interactive plotting capabilities and use them directly with Pandas dataframes.

installing plotly and cufflinks

Install them with:

pip install plotly cufflinks

importing the necessary modules

After installing, you need to import the required libraries and initialize the connection to work in offline mode, which is ideal for Jupyter Notebooks.

import numpy as np
import pandas as pd
import plotly.offline as pyo
import cufflinks as cf
import matplotlib.pyplot as plt

# Enable Plotly in offline mode
pyo.init_notebook_mode(connected=True)

# Connect Cufflinks to offline mode
cf.go_offline()

# This line ensures plots appear within Jupyter Notebook cells
%matplotlib inline

creating a sample dataframe

Let’s create a random dataframe with 100 rows and 4 columns (A, B, C, D) using NumPy and Pandas.

np.random.seed(42)

# Create a DataFrame with 100 rows and 4 columns
df = pd.DataFrame(np.random.randn(100, 4), columns=["A", "B", "C", "D"])

print(df.head())

basic interactive plot

We can visualize the data interactively using iplot(), which produces an interactive Plotly chart.

# Interactive plot using Cufflinks and Plotly
df.iplot(title="Interactive Plot with Plotly and Cufflinks")

You can zoom, pan, hover over points, and toggle data series directly in the graph.

scatter plot

A scatter plot compares two variables visually.

You can zoom into specific regions or isolate columns by clicking the legend.

bar chart

Bar charts show quantities across categories. We can visualize the sum of each column as a bar chart.

box plot

A box plot shows the data distribution, including minimum, maximum, median, and quartiles.

dataframe.iplot(
    kind="box",
    title="Box Plot for All Columns"
)

You can hide or show specific columns interactively by clicking the legend items.

histogram

A histogram shows the distribution of numerical data. You can create one for a specific column or for all columns.

# Histogram for one column
dataframe["A"].iplot(
    kind="hist",
    bins=30,
    title="Histogram of Column A"
)

# Histogram for all columns
dataframe.iplot(
    kind="hist",
    bins=30,
    title="Histogram of All Columns"
)

spread plot

The spread plot shows two variables along with their differences in a mirrored format.

dataframe[["A", "B"]].iplot(
    kind="spread",
    title="Spread Plot Between Columns A and B"
)

This helps compare two time series or numeric columns visually.

bubble chart

A bubble chart uses circle sizes to represent an additional variable.

dataframe.iplot(
    kind="bubble",
    x="A",
    y="B",
    size="C",
    title="Bubble Chart (A vs B, size based on C)",
    xTitle="A values",
    yTitle="B values"
)

3d surface plot

3D plots are useful for visualizing three variables in a single graph. You can rotate, zoom, and explore the plot interactively.

# Create a new DataFrame for 3D plotting
dataframe2 = pd.DataFrame({
    "A": [1, 2, 3, 4],
    "B": [30, 40, 20, 10],
    "C": [12, 16, 18, 15]
})

# 3D surface plot
dataframe2.iplot(
    kind="surface",
    title="3D Surface Plot (A, B, C)"
)

saving and exporting plots

Plotly allows exporting graphs as static images or HTML files.

# Save to an interactive HTML file
pyo.plot(df.iplot(asFigure=True), filename="interactive_plot.html")

# Export a static image (requires kaleido)
import plotly.io as pio
pio.write_image(df.iplot(asFigure=True), "plot.png")

Scikit learn module

Scikit learn includes pre-built algorithms and tools for performing machine learning tasks.

They save time, ensure reliability, and allow developers to focus on solving data problems instead of writing algorithms from scratch.

install it with:

pip install scikit-learn

basic workflow with scikit-learn

The standard steps to build a model with scikit-learn are:

1. Import the necessary modules.

2. Initialize the model with its parameters.

3. Split the data into training and testing sets.

4. Train the model with `.fit()`.

5. Predict new values with `.predict()`.

6. Evaluate model accuracy or error metrics.

Example:

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score

# Load dataset
iris = load_iris()
X, y = iris.data, iris.target

# Split into train and test
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# Create and train model
model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train, y_train)

# Make predictions
predictions = model.predict(X_test)

# Evaluate
accuracy = accuracy_score(y_test, predictions)
print("Accuracy:", accuracy)

evaluating models

After training a model, we need to measure how well it performs.

Common metrics include

-Accuracy for classification

- Mean squared error (MSE) for regression

- Confusion matrix for detailed error analysis

Example:

from sklearn.metrics import confusion_matrix, classification_report

print("Confusion Matrix:")
print(confusion_matrix(y_test, predictions))

print("\nClassification Report:")
print(classification_report(y_test, predictions))

NLTK module

installing and importing nltk

In Python, Natural Language Processing can be applied to analyze documents, process large amounts of text, and build models for automatic text classification (for example, spam detection).

We will use the Natural Language Toolkit (NLTK), a powerful Python library for text analysis.

To install NLTK, open your terminal and type:

pip install nltk

Once installed, import it into your Python environment:

import nltk

If it runs without errors, the library is ready to use.

loading the sms spam dataset

We’ll use the SMS Spam Collection dataset, a well-known dataset containing labeled SMS messages as *ham* (not spam) or *spam*.

Download it from public repositories (e.g., UCI Machine Learning Repository) and save it as `SMSSpamCollection` in your working directory.

# Import necessary modules
import pandas as pd

# Load the dataset
messages = pd.read_csv("SMSSpamCollection", sep="\t", names=["label", "message"])

# Display the first few rows
print(messages.head())

# Number of messages
print("Total messages:", len(messages))

This dataset contains over 5,500 messages, each labeled as either “ham” or “spam”.

adding message length column

Let’s add a new column called length that stores the number of characters in each message.

# Add a column with message lengths
messages["length"] = messages["message"].apply(len)

# Show basic statistics
print(messages["length"].describe())

We can now visualize the message lengths to understand their distribution.

visualizing message lengths

We’ll use Matplotlib and Seaborn to visualize the frequency of message lengths.

import matplotlib.pyplot as plt
import seaborn as sns

# Histogram of message lengths
messages["length"].plot(kind="hist", bins=100, figsize=(10, 5))
plt.title("Distribution of Message Lengths")
plt.xlabel("Message length")
plt.ylabel("Frequency")
plt.show()

To compare ham vs spam, we can plot two histograms side by side:

messages.hist(column="length", by="label", bins=100, figsize=(10, 5))
plt.show()

You’ll notice that spam messages tend to be longer on average than ham messages.

processing text: removing punctuation and stopwords

Before analyzing or classifying text, we must clean it by removing punctuation and stopwords (non-important words like “the”, “of”, “and”, etc.).

import string
from nltk.corpus import stopwords

# Example text
text = "¡Mensaje de ejemplo! Nota: este mensaje tiene signos de puntuación."

# Remove punctuation
no_punct = [c for c in text if c not in string.punctuation]
no_punct = ''.join(no_punct)

# Remove stopwords
words = no_punct.split()
clean_words = [word for word in words if word.lower() not in stopwords.words("spanish")]

print("Original:", text)
print("Cleaned:", clean_words)

creating the text processing function

Let’s create a function `process_text` that combines both cleaning steps, removing punctuation and stopwords.

def process_text(text):
    """
    Cleans a text string by removing punctuation and stopwords.
    """
    # Remove punctuation
    no_punct = ''.join([c for c in text if c not in string.punctuation])
    # Remove stopwords
    words = no_punct.split()
    clean_words = [word for word in words if word.lower() not in stopwords.words("spanish")]
    return clean_words

# Example
example = "Este es un mensaje de ejemplo, con palabras poco importantes!"
print(process_text(example))

This returns only the important words that carry meaning, ready for further analysis.

applying the function to the dataset

We now apply our cleaning function to the entire dataset using `apply`.

messages["cleaned"] = messages["message"].apply(process_text)
print(messages[["message", "cleaned"]].head())

Now each message has a new version containing only relevant words, ideal for machine learning.

vectorizing text

To use text in machine learning, we must convert it into numerical form.

We’ll use CountVectorizer from Scikit-learn, which transforms text into a “bag-of-words” matrix.

from sklearn.feature_extraction.text import CountVectorizer

# Create the vectorizer using our custom analyzer
vectorizer = CountVectorizer(analyzer=process_text)

# Fit and transform the messages
X = vectorizer.fit_transform(messages["message"])

print("Shape of the vocabulary matrix:", X.shape)

Each row represents a message, and each column represents a word in the entire vocabulary.

training a spam detection model

We’ll now train a Random Forest Classifier to predict whether a message is spam or not.

from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.pipeline import Pipeline
from sklearn.metrics import classification_report, confusion_matrix

# Split the data into training and test sets
X_train, X_test, y_train, y_test = train_test_split(
    messages["message"], messages["label"], test_size=0.3, random_state=42
)

# Create a pipeline that includes vectorization and classification
pipeline = Pipeline([
    ("vectorizer", CountVectorizer(analyzer=process_text)),
    ("classifier", RandomForestClassifier(n_estimators=100, random_state=42))
])

# Train the model
pipeline.fit(X_train, y_train)

# Make predictions
predictions = pipeline.predict(X_test)

evaluating the model

Let’s evaluate the accuracy of our spam detection model.

# Classification report
print("Classification Report:")
print(classification_report(y_test, predictions))

# Confusion matrix
print("Confusion Matrix:")
print(confusion_matrix(y_test, predictions))

Typical output:

         precision    recall  f1-score   support

    ham       0.96      1.00      0.98      1448
   spam       1.00      0.83      0.91       224

accuracy 0.97 1672

macro avg 0.98 0.92 0.95 1672

weighted avg 0.97 0.97 0.97 1672

This means our model correctly identifies most spam messages with 97% accuracy.

visualizing confusion matrix

Let’s visualize the confusion matrix for clarity.

sns.heatmap(confusion_matrix(y_test, predictions), annot=True, cmap="Oranges", fmt="d")
plt.title("Confusion Matrix - Spam Detection")
plt.xlabel("Predicted")
plt.ylabel("Actual")
plt.show()

The diagonal values indicate correctly classified messages.

Additional ML examples

See more implementation examples here