Python modules

What is a module and why use it?

A module in Python is a file that contains reusable code - such as functions, classes, and variables — that can be imported into other scripts. Modules help organize code, avoid repetition, and provide specialized tools for different tasks.

Modules on this document

Numpy

NumPy is a powerful linear algebra library Python module for working with vectors, matrix, and arrays. It is widely used for scientific computing, data analysis, and machine learning because it performs mathematical operations extremely fast.

Panda

pandas is an open-source library built as an extension of numpy for data manipulation and analysis in Python. It allows fast data cleaning, preparation, and visualization, and supports many input and output data sources.

NumPy Module

NumPy is a powerful linear algebra library Python module for working with vectors, matrix, and arrays. It is widely used for scientific computing, data analysis, and machine learning because it performs mathematical operations extremely fast.

import numpy as np

This line imports the NumPy module and gives it the short alias np, so we can call its functions like `np.array()` or `np.arange()`.

Installing NumPy

pip install numpy

Creating Arrays

To start, import NumPy:

import numpy as np

You can create a 1D array (vector) from a list:

my_list = [1, 2, 3, 4]
array = np.array(my_list)
print(array)
# [1 2 3 4]

You can also create a 2D array (matrix) from a list of lists:

list2 = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
matrix = np.array(list2)
print(matrix)

Array generation functions

NumPy offers several ways to generate arrays automatically.

np.arange()

Creates a range of numbers.

import numpy as np
np.arange(5)         # [0 1 2 3 4]
np.arange(2, 10)     # [2 3 4 5 6 7 8 9]
np.arange(2, 10, 2)  # [2 4 6 8]

np.zeros() and np.ones()

Generate arrays filled with zeros or ones.

np.zeros((2, 4))  # 2 rows, 4 columns of zeros
np.ones((3, 3))   # 3x3 matrix of ones

np.linspace()

Creates evenly spaced numbers between two values.

np.linspace(3, 9, 5)
# [3. 4.5 6. 7.5 9.]

np.eye()

Creates an identity matrix (ones on the diagonal, zeros elsewhere).

np.eye(5)

Random numbers

NumPy’s random module can generate random values for testing or simulations.

np.random.rand()

Generates random numbers between 0 and 1 (uniform distribution).

np.random.rand(2, 3)

np.random.randn()

Generates random numbers from a normal (Gaussian) distribution.

np.random.randn(4, 5)

np.random.randint()

Generates random integers between two values.

np.random.randint(1, 20)       # One random integer
np.random.randint(1, 20, 5)    # Array of 5 random integers

Reshaping arrays

You can change the shape (rows and columns) of an array using reshape(), as long as the total number of elements matches.

arr = np.random.randint(1, 100, 25)
matrix = arr.reshape(5, 5)
print(matrix)

Maximum, minimum and index positions

Use these functions to find key values and their positions in arrays.

arr = np.random.randint(1, 100, 50)
print(arr.max())      # Maximum value
print(arr.argmax())   # Index of maximum
print(arr.min())      # Minimum value
print(arr.argmin())   # Index of minimum

Indexing and slicing

Access individual or multiple elements of arrays using indices.

One-Dimensional Arrays

arr = np.arange(0, 10)
print(arr[0])     # First element
print(arr[-1])    # Last element
print(arr[2:5])   # Slice from index 2 to 4

Make copies safely:

arr2 = arr.copy()
arr2[0] = 100
print(arr)   # Original unchanged

Two-dimensional arrays

Access elements by [row, column].

matrix = np.array([[1,2,3],[4,5,6],[7,8,9]])
print(matrix[0, 0])   # 1
print(matrix[1, 2])   # 6
print(matrix[:, 0])   # All rows, first column
print(matrix[:2, :2]) # Top-left 2x2 submatrix

Conditional selection

You can select elements based on conditions.

arr = np.arange(0, 10)
print(arr[arr > 5])   # Elements greater than 5
print(arr[arr % 2 == 0])  # Even elements

Array operations

You can perform arithmetic directly on arrays; operations are applied element-wise.

arr = np.arange(0, 10)
print(arr + 10)
print(arr * 2)
print(arr - arr)

Between two arrays:

a = np.arange(0, 10)
b = np.arange(5, 15)
print(b - a)

Common mathematical functions

NumPy provides built-in math functions.

arr = np.arange(0, 10)
print(np.sqrt(arr))   # Square root
print(np.max(arr))    # Maximum
print(np.min(arr))    # Minimum

---

pandas module

pandas is an open-source library built as an extension of numpy for data manipulation and analysis in Python.

It allows fast data cleaning, preparation, and visualization, and supports many input and output data sources.

Install it with:

pip install pandas

pandas series

A series in pandas is a one-dimensional labeled array that can hold any type of data (integers, strings, floats, etc.). Each element has a label, known as its index.

import numpy as np
import pandas as pd

labels = ['a', 'b', 'c']
data = [1, 2, 3]

series = pd.Series(data=data, index=labels)
print(series)

a    1
b    2
c    3

You can also create a series from a numpy array:

array = np.arange(5, 8)
labels = ['sale1', 'sale2', 'sale3']
series = pd.Series(array, index=labels)
print(series)

sale1    5
sale2    6
sale3    7

Access values by **label** or **position**:

print(series['sale1'])   # access by label
print(series[0])         # access by position

Series can hold different data types:

data = ['computer', 300, 'printer', 400]
series = pd.Series(data)
print(series)

0    computer
1         300
2     printer
3         400

You can perform operations between series:

s1 = pd.Series([1, 2, 3], index=['a', 'b', 'c'])
s2 = pd.Series([4, 8, 5], index=['a', 'b', 'c'])
sum_series = s1 + s2
print(sum_series)

a     5
b    10
c     8

dataframes

A **dataframe** is a two-dimensional table with rows and columns, similar to an Excel spreadsheet.

import pandas as pd

rows = ['sales1', 'sales2', 'sales3']
cols = ['zoneA', 'zoneB', 'zoneC']
data = [
    [123, 400, 200],
    [422, 234, 120],
    [256, 450, 210]
]

df = pd.DataFrame(data, index=rows, columns=cols)
print(df)

        zoneA  zoneB  zoneC
sales1    123    400    200
sales2    422    234    120
sales3    256    450    210

You can select rows or columns:

print(df.loc['sales1'])          # select one row
print(df.loc[['sales1','sales2']]) # select multiple rows
print(df['zoneA'])               # select one column
print(df[['zoneA','zoneC']])     # select multiple columns

Add a new column:

df['total_sales'] = df['zoneA'] + df['zoneB'] + df['zoneC']

        zoneA  zoneB  zoneC  total_sales
sales1    123    400    200          723
sales2    422    234    120          776
sales3    256    450    210          916

Delete a column permanently:

df.drop('total_sales', axis=1, inplace=True)

Delete a row permanently:

df.drop('sales3', inplace=True)

conditional selection

Conditional selection allows you to filter data that meets specific conditions.

condition = df > 200
print(df[condition])

        zoneA  zoneB  zoneC
sales1    NaN    400    NaN
sales2  422.0    234    NaN
sales3  256.0    450  210.0

Filter by column:

print(df[df['zoneA'] > 200])

        zoneA  zoneB  zoneC
sales2    422    234    120
sales3    256    450    210

Combine multiple conditions:

print(df[(df['zoneA'] > 200) & (df['zoneB'] > 300)])
print(df[(df['zoneA'] > 200) | (df['zoneB'] > 300)])

modifying index names

You can rename the row labels (index) of a dataframe.

df['days'] = ['day1', 'day2', 'day3']
df = df.set_index('days')
print(df)

handling null values

Pandas provides tools to remove or replace missing values (NaN).

import numpy as np
import pandas as pd

data = {'A':[4,5,np.nan], 'B':[6,1,5], 'C':[np.nan,4,np.nan]}
df = pd.DataFrame(data)
print(df)

     A  B    C
0  4.0  6  NaN
1  5.0  1  4.0
2  NaN  5  NaN

Remove rows or columns with nulls:

print(df.dropna())          # remove rows with NaN
print(df.dropna(axis=1))    # remove columns with NaN

     A  B    C
1  5.0  1  4.0

   B
0  6
1  1
2  5

Replace nulls:

df.fillna(value=100)
df.fillna(value=df.mean())

       A  B      C
0    4.0  6  100.0
1    5.0  1    4.0
2  100.0  5  100.0

     A  B    C
0  4.0  6  4.0
1  5.0  1  4.0
2  4.5  5  4.0

grouping data

Grouping allows you to aggregate data by one or more columns.

data = {'day':[1,1,2,2,3],
        'seller':['Antonio','Maria','Jose','Maria','Antonio'],
        'sales':[100,400,200,500,300]}
df = pd.DataFrame(data)

print(df.groupby('day').mean())
print(df.groupby('day').sum())
print(df.groupby('day').describe())

           seller  sales
day                     
1    AntonioMaria    500
2       JoseMaria    700
3         Antonio    300

    sales                                                      
    count   mean         std    min    25%    50%    75%    max
day                                                            
1     2.0  250.0  212.132034  100.0  175.0  250.0  325.0  400.0
2     2.0  350.0  212.132034  200.0  275.0  350.0  425.0  500.0
3     1.0  300.0         NaN  300.0  300.0  300.0  300.0  300.0

combining dataframes

You can concatenate dataframes vertically or horizontally.

df1 = pd.DataFrame({'A':[1,2,3], 'B':[4,5,6], 'C':[7,8,9]})
df2 = pd.DataFrame({'A':[11,12,13], 'B':[14,15,16], 'C':[17,18,19]})

print(pd.concat([df1, df2]))            # vertical
print(pd.concat([df1, df2], axis=1))    # horizontal

    A   B   C
0   1   4   7
1   2   5   8
2   3   6   9
0  11  14  17
1  12  15  18
2  13  16  19

   A  B  C   A   B   C
0  1  4  7  11  14  17
1  2  5  8  12  15  18
2  3  6  9  13  16  19

merging dataframes

Merging joins two dataframes based on a common key.

df1 = pd.DataFrame({'A':[1,2,3],'B':[4,5,6],'key':['C1','C2','C3']})
df2 = pd.DataFrame({'C':[7,8,9],'D':[10,11,12],'key':['C1','C2','C3']})

merged = pd.merge(df1, df2, on='key')
print(merged)

   A  B key  C   D
0  1  4  C1  7  10
1  2  5  C2  8  11
2  3  6  C3  9  12

joining dataframes

Joining combines dataframes by their index labels.

df1 = pd.DataFrame({'A':[1,2,3],'B':[4,5,6]}, index=['i1','i2','i3'])
df2 = pd.DataFrame({'C':[11,12,13],'D':[14,15,16]}, index=['i1','i2','i3'])

joined = df1.join(df2)
print(joined)

    A  B   C   D
i1  1  4  11  14
i2  2  5  12  15
i3  3  6  13  16

dataframe operations

You can perform operations such as counting unique values, applying functions, or sorting.

df = pd.DataFrame({'A':[11,12,10,12],'B':[14,18,16,17]}, index=['i1','i2','i3','i4'])
print(df['A'].unique())
print(df['A'].nunique())
print(df['A'].value_counts())

[11 12 10]

3

A
12    2
11    1
10    1

Apply a function:

def multiply(x): return x * 2
print(df['A'].apply(multiply))
print(df['A'].apply(lambda x: x * 2))

i1    22
i2    24
i3    20
i4    24

i1    22
i2    24
i3    20
i4    24

Sort values:

df.sort_values(by='B')

reading and writing excel files

Pandas can read and write Excel files easily.

import pandas as pd

df = pd.read_excel('example.xlsx')
print(df)

df['E'] = [20,22,23]
df.to_excel('output.xlsx', sheet_name='Sheet1')

reading html tables

You can extract data from HTML tables directly into pandas dataframes.

import pandas as pd

url = 'https://en.wikipedia.org/wiki/List_of_countries_and_dependencies_by_population'
tables = pd.read_html(url)
df = tables[0]
print(df.head(10))

working with sql databases

You can write and read dataframes from SQL databases.

from sqlalchemy import create_engine
import pandas as pd

df = pd.DataFrame({'A':[1,2,3],'B':[4,5,6]})
engine = create_engine('sqlite:///:memory:')
df.to_sql('table', con=engine, index=False)
data = pd.read_sql('table', con=engine)
print(data)

plotting dataframes

Pandas integrates with matplotlib to create visualizations directly from dataframes. Plots generated by Matplotlib (and by extension, Pandas plotting functions) are not automatically displayed in separate windows. plt.show() opens all the figures that have been created and displays them.[5][6] Your script will pause until you close all the plot windows. See Matplotlib module for further reference

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

df = pd.DataFrame(np.random.randint(0,200,(50,4)), columns=list('ABCD'))

df['A'].hist()
df.plot.area(alpha=0.3)
df.plot.bar(stacked=True)
df.plot.scatter(x='A', y='B', c='C', cmap='coolwarm')
df.plot.box()
df.plot.hexbin(x='A', y='B', gridsize=10)
df.plot.kde()
plt.show()

matplotlib module

matplotlib is a Python library for creating visualizations and plots from numerical data stored in lists or arrays. It provides full control over every aspect of a chart. from layout and colors to markers and line styles.

It is one of the most widely used libraries for data visualization in Python.

To install it use:

pip install matplotlib

basic example

A simple example of a line plot using matplotlib.

import numpy as np
import matplotlib.pyplot as plt

x = np.linspace(2, 9, 10)
y = x ** 2

plt.plot(x, y)
plt.show()

You can add colors and labels:

plt.subplot(1, 2, 1)
plt.plot(x, y, color='green')

plt.subplot(1, 2, 2)
plt.plot(y, x, color='blue')

plt.show()

subplots

Subplots allow displaying multiple plots in the same figure.

This creates two plots side by side (1 row, 2 columns).

To stack them vertically:

plt.subplot(2, 1, 1)
plt.plot(x, y, color='red')

plt.subplot(2, 1, 2)
plt.plot(y, x, color='blue')
plt.show()

object oriented approach

Matplotlib can also be used in an object-oriented way, providing more control.

import matplotlib.pyplot as plt
import numpy as np

x = np.linspace(2, 9, 10)
y = x ** 2

fig = plt.figure()
axes = fig.add_axes([0.1, 0.1, 0.8, 0.8])
axes.plot(x, y, color='green')
plt.show()

The list `[0.1, 0.1, 0.8, 0.8]` defines the left margin, bottom margin, width, and height of the axes as percentages of the figure size.

You can add multiple plots inside the same figure:

fig = plt.figure()

# main plot
ax1 = fig.add_axes([0.1, 0.1, 0.9, 0.9])
ax1.plot(x, y, color='red')

# inset plot
ax2 = fig.add_axes([0.15, 0.5, 0.3, 0.3])
ax2.plot(y, x, color='blue')
plt.show()

multiple plots in a grid

You can create several plots organized in rows and columns. This creates four subplots in a 2x2 grid.

fig, axes = plt.subplots(2, 2)

axes[0, 0].plot(x, y, color='blue')
axes[0, 0].set_title('blue')

axes[0, 1].plot(x, y, color='red')
axes[0, 1].set_title('red')

axes[1, 0].plot(x, y, color='yellow')
axes[1, 0].set_title('yellow')

axes[1, 1].plot(x, y, color='green')
axes[1, 1].set_title('green')

plt.show()

figure size

You can set the overall figure size using the figsize parameter.

fig = plt.figure(figsize=(8, 4))
ax = fig.add_axes([0.1, 0.1, 0.8, 0.8])
ax.plot(x, y, color='red')
plt.show()

The `figsize` values are in inches (width, height). For example, `(8, 4)` creates a rectangular figure 8 inches wide and 4 inches tall.

multiple lines and legends

You can draw multiple lines in a single plot and add a legend.

fig = plt.figure()
ax = fig.add_axes([0.1, 0.1, 0.8, 0.8])

x = np.linspace(0, 10, 20)
ax.plot(x, x**2, label='squared')
ax.plot(x, x**3, label='cubed')

ax.legend(loc=0)
plt.show()

The `loc=0` option automatically finds the best location for the legend.

colors

Matplotlib accepts many color formats — by name, shorthand, or hexadecimal code.

ax.plot(x, y, color='tomato')
ax.plot(x, y, color='#00FF00')  # hex code for green
ax.plot(x, y, color='lime')

You can find all available named colors in the official matplotlib color reference.

---

line styles

You can customize the appearance of plot lines — thickness, transparency, and style.

fig = plt.figure()
ax = fig.add_axes([0, 0, 1, 1])

ax.plot(x, y, color='red', linewidth=5, alpha=0.5, linestyle='--')
plt.show()

- **linewidth** controls thickness - **alpha** controls transparency (0 invisible, 1 fully visible) - **linestyle** can be `'--'`, `':'`, `'-.'`, or `'-'`

---

markers

Markers highlight data points on a plot.

fig = plt.figure()
ax = fig.add_axes([0, 0, 1, 1])

ax.plot(x, y, color='green', marker='o', markersize=5)
plt.show()

You can choose from various marker symbols: `'o'` (circle), `'s'` (square), `'d'` (diamond), `'+'`, `'*'`, `'x'`, and many others.

ax.plot(x, y, color='blue', marker='s', markersize=8)