Python modules

What is a module and why use it?

A module in Python is a file that contains reusable code - such as functions, classes, and variables — that can be imported into other scripts. Modules help organize code, avoid repetition, and provide specialized tools for different tasks.

The NumPy Module

NumPy is a powerful linear algebra library Python module for working with vectors, matrix, and arrays. It is widely used for scientific computing, data analysis, and machine learning because it performs mathematical operations extremely fast.

import numpy as np

This line imports the NumPy module and gives it the short alias np, so we can call its functions like `np.array()` or `np.arange()`.

Installing NumPy

pip install numpy

Creating Arrays

To start, import NumPy:

import numpy as np

You can create a 1D array (vector) from a list:

my_list = [1, 2, 3, 4]
array = np.array(my_list)
print(array)
# [1 2 3 4]

You can also create a 2D array (matrix) from a list of lists:

list2 = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
matrix = np.array(list2)
print(matrix)

Array generation functions

NumPy offers several ways to generate arrays automatically.

np.arange()

Creates a range of numbers.

import numpy as np
np.arange(5)         # [0 1 2 3 4]
np.arange(2, 10)     # [2 3 4 5 6 7 8 9]
np.arange(2, 10, 2)  # [2 4 6 8]

np.zeros() and np.ones()

Generate arrays filled with zeros or ones.

np.zeros((2, 4))  # 2 rows, 4 columns of zeros
np.ones((3, 3))   # 3x3 matrix of ones

np.linspace()

Creates evenly spaced numbers between two values.

np.linspace(3, 9, 5)
# [3. 4.5 6. 7.5 9.]

np.eye()

Creates an identity matrix (ones on the diagonal, zeros elsewhere).

np.eye(5)

Random numbers

NumPy’s random module can generate random values for testing or simulations.

np.random.rand()

Generates random numbers between 0 and 1 (uniform distribution).

np.random.rand(2, 3)

np.random.randn()

Generates random numbers from a normal (Gaussian) distribution.

np.random.randn(4, 5)

np.random.randint()

Generates random integers between two values.

np.random.randint(1, 20)       # One random integer
np.random.randint(1, 20, 5)    # Array of 5 random integers

Reshaping arrays

You can change the shape (rows and columns) of an array using reshape(), as long as the total number of elements matches.

arr = np.random.randint(1, 100, 25)
matrix = arr.reshape(5, 5)
print(matrix)

Maximum, minimum and index positions

Use these functions to find key values and their positions in arrays.

arr = np.random.randint(1, 100, 50)
print(arr.max())      # Maximum value
print(arr.argmax())   # Index of maximum
print(arr.min())      # Minimum value
print(arr.argmin())   # Index of minimum

Indexing and slicing

Access individual or multiple elements of arrays using indices.

One-Dimensional Arrays

arr = np.arange(0, 10)
print(arr[0])     # First element
print(arr[-1])    # Last element
print(arr[2:5])   # Slice from index 2 to 4

Make copies safely:

arr2 = arr.copy()
arr2[0] = 100
print(arr)   # Original unchanged

Two-dimensional arrays

Access elements by [row, column].

matrix = np.array([[1,2,3],[4,5,6],[7,8,9]])
print(matrix[0, 0])   # 1
print(matrix[1, 2])   # 6
print(matrix[:, 0])   # All rows, first column
print(matrix[:2, :2]) # Top-left 2x2 submatrix

Conditional selection

You can select elements based on conditions.

arr = np.arange(0, 10)
print(arr[arr > 5])   # Elements greater than 5
print(arr[arr % 2 == 0])  # Even elements

Array operations

You can perform arithmetic directly on arrays; operations are applied element-wise.

arr = np.arange(0, 10)
print(arr + 10)
print(arr * 2)
print(arr - arr)

Between two arrays:

a = np.arange(0, 10)
b = np.arange(5, 15)
print(b - a)

Common mathematical functions

NumPy provides built-in math functions.

arr = np.arange(0, 10)
print(np.sqrt(arr))   # Square root
print(np.max(arr))    # Maximum
print(np.min(arr))    # Minimum

---

pandas module

pandas is an open-source library built as an extension of numpy for data manipulation and analysis in Python.

It allows fast data cleaning, preparation, and visualization, and supports many input and output data sources.

Install it with:

pip install pandas

pandas series

A series in pandas is a one-dimensional labeled array that can hold any type of data (integers, strings, floats, etc.). Each element has a label, known as its index.

import numpy as np
import pandas as pd

labels = ['a', 'b', 'c']
data = [1, 2, 3]

series = pd.Series(data=data, index=labels)
print(series)

You can also create a series from a numpy array:

array = np.arange(5, 8)
labels = ['sale1', 'sale2', 'sale3']
series = pd.Series(array, index=labels)
print(series)

Access values by **label** or **position**:

print(series['sale1'])   # access by label
print(series[0])         # access by position

Series can hold different data types:

data = ['computer', 300, 'printer', 400]
series = pd.Series(data)
print(series)

You can perform operations between series:

s1 = pd.Series([1, 2, 3], index=['a', 'b', 'c'])
s2 = pd.Series([4, 8, 5], index=['a', 'b', 'c'])
sum_series = s1 + s2
print(sum_series)

dataframes

A **dataframe** is a two-dimensional table with rows and columns, similar to an Excel spreadsheet.

import pandas as pd

rows = ['sales1', 'sales2', 'sales3']
cols = ['zoneA', 'zoneB', 'zoneC']
data = [
    [123, 400, 200],
    [422, 234, 120],
    [256, 450, 210]
]

df = pd.DataFrame(data, index=rows, columns=cols)
print(df)

You can select rows or columns:

print(df.loc['sales1'])          # select one row
print(df.loc[['sales1','sales2']]) # select multiple rows
print(df['zoneA'])               # select one column
print(df[['zoneA','zoneC']])     # select multiple columns

Add a new column:

df['total_sales'] = df['zoneA'] + df['zoneB'] + df['zoneC']

Delete a column:

df.drop('total_sales', axis=1, inplace=True)

Delete a row:

df.drop('sales3', inplace=True)

conditional selection

Conditional selection allows you to filter data that meets specific conditions.

condition = df > 200
print(df[condition])

Filter by column:

print(df[df['zoneA'] > 200])

Combine multiple conditions:

print(df[(df['zoneA'] > 200) & (df['zoneB'] > 300)])
print(df[(df['zoneA'] > 200) | (df['zoneB'] > 300)])

modifying index names

You can rename the row labels (index) of a dataframe.

df['days'] = ['day1', 'day2', 'day3']
df = df.set_index('days')
print(df)

handling null values

Pandas provides tools to remove or replace missing values (NaN).

import numpy as np
import pandas as pd

data = {'A':[4,5,np.nan], 'B':[6,1,5], 'C':[np.nan,4,np.nan]}
df = pd.DataFrame(data)
print(df)

Remove rows or columns with nulls:

df.dropna()          # remove rows with NaN
df.dropna(axis=1)    # remove columns with NaN

Replace nulls:

df.fillna(value=100)
df.fillna(value=df.mean())

grouping data

Grouping allows you to aggregate data by one or more columns.

data = {'day':[1,1,2,2,3],
        'seller':['Antonio','Maria','Jose','Maria','Antonio'],
        'sales':[100,400,200,500,300]}
df = pd.DataFrame(data)

print(df.groupby('day').mean())
print(df.groupby('day').sum())
print(df.groupby('day').describe())

combining dataframes

You can concatenate dataframes vertically or horizontally.

df1 = pd.DataFrame({'A':[1,2,3], 'B':[4,5,6], 'C':[7,8,9]})
df2 = pd.DataFrame({'A':[11,12,13], 'B':[14,15,16], 'C':[17,18,19]})

print(pd.concat([df1, df2]))            # vertical
print(pd.concat([df1, df2], axis=1))    # horizontal

merging dataframes

Merging joins two dataframes based on a common key.

df1 = pd.DataFrame({'A':[1,2,3],'B':[4,5,6],'key':['C1','C2','C3']})
df2 = pd.DataFrame({'C':[7,8,9],'D':[10,11,12],'key':['C1','C2','C3']})

merged = pd.merge(df1, df2, on='key')
print(merged)

joining dataframes

Joining combines dataframes by their index labels.

df1 = pd.DataFrame({'A':[1,2,3],'B':[4,5,6]}, index=['i1','i2','i3'])
df2 = pd.DataFrame({'C':[11,12,13],'D':[14,15,16]}, index=['i1','i2','i3'])

joined = df1.join(df2)
print(joined)

dataframe operations

You can perform operations such as counting unique values, applying functions, or sorting.

df = pd.DataFrame({'A':[11,12,10,12],'B':[14,18,16,17]}, index=['i1','i2','i3','i4'])
print(df['A'].unique())
print(df['A'].nunique())
print(df['A'].value_counts())

Apply a function:

def multiply(x): return x * 2
print(df['A'].apply(multiply))
print(df['A'].apply(lambda x: x * 2))

Sort values:

df.sort_values(by='B')

reading and writing excel files

Pandas can read and write Excel files easily.

import pandas as pd

df = pd.read_excel('example.xlsx')
print(df)

df['E'] = [20,22,23]
df.to_excel('output.xlsx', sheet_name='Sheet1')

reading html tables

You can extract data from HTML tables directly into pandas dataframes.

import pandas as pd

url = 'https://en.wikipedia.org/wiki/List_of_countries_and_dependencies_by_population'
tables = pd.read_html(url)
df = tables[0]
print(df.head(10))

working with sql databases

You can write and read dataframes from SQL databases.

from sqlalchemy import create_engine
import pandas as pd

df = pd.DataFrame({'A':[1,2,3],'B':[4,5,6]})
engine = create_engine('sqlite:///:memory:')
df.to_sql('table', con=engine, index=False)
data = pd.read_sql('table', con=engine)
print(data)

plotting dataframes

Pandas integrates with matplotlib to create visualizations directly from dataframes.

import numpy as np
import pandas as pd

df = pd.DataFrame(np.random.randint(0,200,(50,4)), columns=list('ABCD'))

df['A'].hist()
df.plot.area(alpha=0.3)
df.plot.bar(stacked=True)
df.plot.scatter(x='A', y='B', c='C', cmap='coolwarm')
df.plot.box()
df.plot.hexbin(x='A', y='B', gridsize=10)
df.plot.kde()

---