Python modules

What is a module and why use it?

A module in Python is a file that contains reusable code - such as functions, classes, and variables — that can be imported into other scripts. Modules help organize code, avoid repetition, and provide specialized tools for different tasks.

Modules on this document

Numpy

NumPy is a powerful linear algebra library Python module for working with vectors, matrix, and arrays. It is widely used for scientific computing, data analysis, and machine learning because it performs mathematical operations extremely fast.

Panda

pandas is an open-source library built as an extension of numpy for data manipulation and analysis in Python. It allows fast data cleaning, preparation, and visualization, and supports many input and output data sources.

NumPy Module

NumPy is a powerful linear algebra library Python module for working with vectors, matrix, and arrays. It is widely used for scientific computing, data analysis, and machine learning because it performs mathematical operations extremely fast.

import numpy as np

This line imports the NumPy module and gives it the short alias np, so we can call its functions like `np.array()` or `np.arange()`.

Installing NumPy

pip install numpy

Creating Arrays

To start, import NumPy:

import numpy as np

You can create a 1D array (vector) from a list:

my_list = [1, 2, 3, 4]
array = np.array(my_list)
print(array)
# [1 2 3 4]

You can also create a 2D array (matrix) from a list of lists:

list2 = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
matrix = np.array(list2)
print(matrix)

Array generation functions

NumPy offers several ways to generate arrays automatically.

np.arange()

Creates a range of numbers.

import numpy as np
np.arange(5)         # [0 1 2 3 4]
np.arange(2, 10)     # [2 3 4 5 6 7 8 9]
np.arange(2, 10, 2)  # [2 4 6 8]

np.zeros() and np.ones()

Generate arrays filled with zeros or ones.

np.zeros((2, 4))  # 2 rows, 4 columns of zeros
np.ones((3, 3))   # 3x3 matrix of ones

np.linspace()

Creates evenly spaced numbers between two values.

np.linspace(3, 9, 5)
# [3. 4.5 6. 7.5 9.]

np.eye()

Creates an identity matrix (ones on the diagonal, zeros elsewhere).

np.eye(5)

Random numbers

NumPy’s random module can generate random values for testing or simulations.

np.random.rand()

Generates random numbers between 0 and 1 (uniform distribution).

np.random.rand(2, 3)

np.random.randn()

Generates random numbers from a normal (Gaussian) distribution.

np.random.randn(4, 5)

np.random.randint()

Generates random integers between two values.

np.random.randint(1, 20)       # One random integer
np.random.randint(1, 20, 5)    # Array of 5 random integers

Reshaping arrays

You can change the shape (rows and columns) of an array using reshape(), as long as the total number of elements matches.

arr = np.random.randint(1, 100, 25)
matrix = arr.reshape(5, 5)
print(matrix)

Maximum, minimum and index positions

Use these functions to find key values and their positions in arrays.

arr = np.random.randint(1, 100, 50)
print(arr.max())      # Maximum value
print(arr.argmax())   # Index of maximum
print(arr.min())      # Minimum value
print(arr.argmin())   # Index of minimum

Indexing and slicing

Access individual or multiple elements of arrays using indices.

One-Dimensional Arrays

arr = np.arange(0, 10)
print(arr[0])     # First element
print(arr[-1])    # Last element
print(arr[2:5])   # Slice from index 2 to 4

Make copies safely:

arr2 = arr.copy()
arr2[0] = 100
print(arr)   # Original unchanged

Two-dimensional arrays

Access elements by [row, column].

matrix = np.array([[1,2,3],[4,5,6],[7,8,9]])
print(matrix[0, 0])   # 1
print(matrix[1, 2])   # 6
print(matrix[:, 0])   # All rows, first column
print(matrix[:2, :2]) # Top-left 2x2 submatrix

Conditional selection

You can select elements based on conditions.

arr = np.arange(0, 10)
print(arr[arr > 5])   # Elements greater than 5
print(arr[arr % 2 == 0])  # Even elements

Array operations

You can perform arithmetic directly on arrays; operations are applied element-wise.

arr = np.arange(0, 10)
print(arr + 10)
print(arr * 2)
print(arr - arr)

Between two arrays:

a = np.arange(0, 10)
b = np.arange(5, 15)
print(b - a)

Common mathematical functions

NumPy provides built-in math functions.

arr = np.arange(0, 10)
print(np.sqrt(arr))   # Square root
print(np.max(arr))    # Maximum
print(np.min(arr))    # Minimum

---

pandas module

pandas is an open-source library built as an extension of numpy for data manipulation and analysis in Python.

It allows fast data cleaning, preparation, and visualization, and supports many input and output data sources.

Install it with:

pip install pandas

pandas series

A series in pandas is a one-dimensional labeled array that can hold any type of data (integers, strings, floats, etc.). Each element has a label, known as its index.

import numpy as np
import pandas as pd

labels = ['a', 'b', 'c']
data = [1, 2, 3]

series = pd.Series(data=data, index=labels)
print(series)

a    1
b    2
c    3

You can also create a series from a numpy array:

array = np.arange(5, 8)
labels = ['sale1', 'sale2', 'sale3']
series = pd.Series(array, index=labels)
print(series)

sale1    5
sale2    6
sale3    7

Access values by **label** or **position**:

print(series['sale1'])   # access by label
print(series[0])         # access by position

Series can hold different data types:

data = ['computer', 300, 'printer', 400]
series = pd.Series(data)
print(series)

0    computer
1         300
2     printer
3         400

You can perform operations between series:

s1 = pd.Series([1, 2, 3], index=['a', 'b', 'c'])
s2 = pd.Series([4, 8, 5], index=['a', 'b', 'c'])
sum_series = s1 + s2
print(sum_series)

a     5
b    10
c     8

dataframes

A **dataframe** is a two-dimensional table with rows and columns, similar to an Excel spreadsheet.

import pandas as pd

rows = ['sales1', 'sales2', 'sales3']
cols = ['zoneA', 'zoneB', 'zoneC']
data = [
    [123, 400, 200],
    [422, 234, 120],
    [256, 450, 210]
]

df = pd.DataFrame(data, index=rows, columns=cols)
print(df)

        zoneA  zoneB  zoneC
sales1    123    400    200
sales2    422    234    120
sales3    256    450    210

You can select rows or columns:

print(df.loc['sales1'])          # select one row
print(df.loc[['sales1','sales2']]) # select multiple rows
print(df['zoneA'])               # select one column
print(df[['zoneA','zoneC']])     # select multiple columns

Add a new column:

df['total_sales'] = df['zoneA'] + df['zoneB'] + df['zoneC']

        zoneA  zoneB  zoneC  total_sales
sales1    123    400    200          723
sales2    422    234    120          776
sales3    256    450    210          916

Delete a column permanently:

df.drop('total_sales', axis=1, inplace=True)

Delete a row permanently:

df.drop('sales3', inplace=True)

conditional selection

Conditional selection allows you to filter data that meets specific conditions.

condition = df > 200
print(df[condition])

        zoneA  zoneB  zoneC
sales1    NaN    400    NaN
sales2  422.0    234    NaN
sales3  256.0    450  210.0

Filter by column:

print(df[df['zoneA'] > 200])

        zoneA  zoneB  zoneC
sales2    422    234    120
sales3    256    450    210

Combine multiple conditions:

print(df[(df['zoneA'] > 200) & (df['zoneB'] > 300)])
print(df[(df['zoneA'] > 200) | (df['zoneB'] > 300)])

modifying index names

You can rename the row labels (index) of a dataframe.

df['days'] = ['day1', 'day2', 'day3']
df = df.set_index('days')
print(df)

handling null values

Pandas provides tools to remove or replace missing values (NaN).

import numpy as np
import pandas as pd

data = {'A':[4,5,np.nan], 'B':[6,1,5], 'C':[np.nan,4,np.nan]}
df = pd.DataFrame(data)
print(df)

     A  B    C
0  4.0  6  NaN
1  5.0  1  4.0
2  NaN  5  NaN

Remove rows or columns with nulls:

print(df.dropna())          # remove rows with NaN
print(df.dropna(axis=1))    # remove columns with NaN

     A  B    C
1  5.0  1  4.0

   B
0  6
1  1
2  5

Replace nulls:

df.fillna(value=100)
df.fillna(value=df.mean())

       A  B      C
0    4.0  6  100.0
1    5.0  1    4.0
2  100.0  5  100.0

     A  B    C
0  4.0  6  4.0
1  5.0  1  4.0
2  4.5  5  4.0

grouping data

Grouping allows you to aggregate data by one or more columns.

data = {'day':[1,1,2,2,3],
        'seller':['Antonio','Maria','Jose','Maria','Antonio'],
        'sales':[100,400,200,500,300]}
df = pd.DataFrame(data)

print(df.groupby('day').mean())
print(df.groupby('day').sum())
print(df.groupby('day').describe())

           seller  sales
day                     
1    AntonioMaria    500
2       JoseMaria    700
3         Antonio    300

    sales                                                      
    count   mean         std    min    25%    50%    75%    max
day                                                            
1     2.0  250.0  212.132034  100.0  175.0  250.0  325.0  400.0
2     2.0  350.0  212.132034  200.0  275.0  350.0  425.0  500.0
3     1.0  300.0         NaN  300.0  300.0  300.0  300.0  300.0

combining dataframes

You can concatenate dataframes vertically or horizontally.

df1 = pd.DataFrame({'A':[1,2,3], 'B':[4,5,6], 'C':[7,8,9]})
df2 = pd.DataFrame({'A':[11,12,13], 'B':[14,15,16], 'C':[17,18,19]})

print(pd.concat([df1, df2]))            # vertical
print(pd.concat([df1, df2], axis=1))    # horizontal

    A   B   C
0   1   4   7
1   2   5   8
2   3   6   9
0  11  14  17
1  12  15  18
2  13  16  19

   A  B  C   A   B   C
0  1  4  7  11  14  17
1  2  5  8  12  15  18
2  3  6  9  13  16  19

merging dataframes

Merging joins two dataframes based on a common key.

df1 = pd.DataFrame({'A':[1,2,3],'B':[4,5,6],'key':['C1','C2','C3']})
df2 = pd.DataFrame({'C':[7,8,9],'D':[10,11,12],'key':['C1','C2','C3']})

merged = pd.merge(df1, df2, on='key')
print(merged)

   A  B key  C   D
0  1  4  C1  7  10
1  2  5  C2  8  11
2  3  6  C3  9  12

joining dataframes

Joining combines dataframes by their index labels.

df1 = pd.DataFrame({'A':[1,2,3],'B':[4,5,6]}, index=['i1','i2','i3'])
df2 = pd.DataFrame({'C':[11,12,13],'D':[14,15,16]}, index=['i1','i2','i3'])

joined = df1.join(df2)
print(joined)

    A  B   C   D
i1  1  4  11  14
i2  2  5  12  15
i3  3  6  13  16

dataframe operations

You can perform operations such as counting unique values, applying functions, or sorting.

df = pd.DataFrame({'A':[11,12,10,12],'B':[14,18,16,17]}, index=['i1','i2','i3','i4'])
print(df['A'].unique())
print(df['A'].nunique())
print(df['A'].value_counts())

[11 12 10]

3

A
12    2
11    1
10    1

Apply a function:

def multiply(x): return x * 2
print(df['A'].apply(multiply))
print(df['A'].apply(lambda x: x * 2))

i1    22
i2    24
i3    20
i4    24

i1    22
i2    24
i3    20
i4    24

Sort values:

df.sort_values(by='B')

reading and writing excel files

Pandas can read and write Excel files easily.

import pandas as pd

df = pd.read_excel('example.xlsx')
print(df)

df['E'] = [20,22,23]
df.to_excel('output.xlsx', sheet_name='Sheet1')

reading html tables

You can extract data from HTML tables directly into pandas dataframes.

import pandas as pd

url = 'https://en.wikipedia.org/wiki/List_of_countries_and_dependencies_by_population'
tables = pd.read_html(url)
df = tables[0]
print(df.head(10))

working with sql databases

You can write and read dataframes from SQL databases.

from sqlalchemy import create_engine
import pandas as pd

df = pd.DataFrame({'A':[1,2,3],'B':[4,5,6]})
engine = create_engine('sqlite:///:memory:')
df.to_sql('table', con=engine, index=False)
data = pd.read_sql('table', con=engine)
print(data)

plotting dataframes

Pandas integrates with matplotlib to create visualizations directly from dataframes. Plots generated by Matplotlib (and by extension, Pandas plotting functions) are not automatically displayed in separate windows. plt.show() opens all the figures that have been created and displays them.[5][6] Your script will pause until you close all the plot windows

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

df = pd.DataFrame(np.random.randint(0,200,(50,4)), columns=list('ABCD'))

df['A'].hist()
df.plot.area(alpha=0.3)
df.plot.bar(stacked=True)
df.plot.scatter(x='A', y='B', c='C', cmap='coolwarm')
df.plot.box()
df.plot.hexbin(x='A', y='B', gridsize=10)
df.plot.kde()
plt.show()