Python modules - Revision history

Root at 18:27, 29 October 2025

2025-10-29T18:27:15Z

← Older revision		Revision as of 19:27, 29 October 2025
Line 1,250:		Line 1,250:
	print(classification_report(y_test, predictions))		print(classification_report(y_test, predictions))
	</syntaxhighlight>		</syntaxhighlight>
	= ~~natural language processing (NLP)~~ =		= '''NLTK module''' =

	'''~~Natural Language Processing (NLP)''' is a field of '''computer science''', '''artificial intelligence''',~~ and '''~~linguistics''' that studies the interactions between computers and human language.~~		'''installing and importing nltk'''

	~~Its goal is~~ to ~~enable computers to understand~~, ~~interpret~~, and ~~generate human language in a useful way~~.		In Python, Natural Language Processing can be applied to analyze documents, process large amounts of text, and build models for automatic text classification (for example, spam detection).

	~~NLP uses computer programs that simulate communication between people and machines through '''natural languages''', the spoken and written languages of the world.~~		We will use the '''Natural Language Toolkit (NLTK)''', a powerful Python library for text analysis.

	~~In Python, NLP can be applied to analyze documents, process large amounts of text, and build models for automatic text classification (for example, spam detection).~~

	~~'''installing and importing nltk'''~~

	We will use the '''Natural Language Toolkit (NLTK)''', a powerful Python library for text analysis.

	To install NLTK, open your terminal and type:		To install NLTK, open your terminal and type:
Line 1,277:		Line 1,271:

	If it runs without errors, the library is ready to use.		If it runs without errors, the library is ready to use.



	'''loading the sms spam dataset'''		'''loading the sms spam dataset'''
Line 1,301:		Line 1,293:

	This dataset contains over '''5,500 messages''', each labeled as either “ham” or “spam”.		This dataset contains over '''5,500 messages''', each labeled as either “ham” or “spam”.



	'''adding message length column'''		'''adding message length column'''
Line 1,317:		Line 1,307:

	We can now visualize the message lengths to understand their distribution.		We can now visualize the message lengths to understand their distribution.



	'''visualizing message lengths'''		'''visualizing message lengths'''
Line 1,344:		Line 1,332:

	You’ll notice that spam messages tend to be '''longer''' on average than ham messages.		You’ll notice that spam messages tend to be '''longer''' on average than ham messages.



	'''processing text: removing punctuation and stopwords'''		'''processing text: removing punctuation and stopwords'''
Line 1,369:		Line 1,355:
	print("Cleaned:", clean_words)		print("Cleaned:", clean_words)
	</syntaxhighlight>		</syntaxhighlight>



	'''creating the text processing function'''		'''creating the text processing function'''
Line 1,394:		Line 1,378:

	This returns only the '''important words''' that carry meaning, ready for further analysis.		This returns only the '''important words''' that carry meaning, ready for further analysis.



	'''applying the function to the dataset'''		'''applying the function to the dataset'''
Line 1,407:		Line 1,389:

	Now each message has a new version containing only relevant words, ideal for machine learning.		Now each message has a new version containing only relevant words, ideal for machine learning.



	'''vectorizing text'''		'''vectorizing text'''
Line 1,429:		Line 1,409:

	Each row represents a message, and each column represents a word in the entire vocabulary.		Each row represents a message, and each column represents a word in the entire vocabulary.



	'''training a spam detection model'''		'''training a spam detection model'''
Line 1,508:		Line 1,486:


	=~~= Further~~ examples ==
			= Additional ML examples =
	See more implementation examples [[Machine learning algorithms\|here]]		See more implementation examples [[Machine learning algorithms\|here]]

	[[Category:Python]]		[[Category:Python]]

Root at 18:22, 29 October 2025

2025-10-29T18:22:06Z

Show changes

Root at 08:50, 28 October 2025

2025-10-28T08:50:31Z

← Older revision		Revision as of 09:50, 28 October 2025
Line 1,248:		Line 1,248:
	</syntaxhighlight>		</syntaxhighlight>
	[[Category:Python]]		[[Category:Python]]

			== Further examples ==
			See more implementation examples [[Machine learning algorithms\|here]]

Root at 08:48, 28 October 2025

2025-10-28T08:48:24Z

Show changes

Root at 08:36, 28 October 2025

2025-10-28T08:36:28Z

Show changes

Root: /* logistic regression */

2025-10-28T08:24:10Z

logistic regression

Show changes

Root at 17:45, 26 October 2025

2025-10-26T17:45:52Z

← Older revision		Revision as of 18:45, 26 October 2025
Line 1,405:		Line 1,405:
	A well-trained linear regression model for this dataset typically achieves a very low error, indicating good predictive accuracy.		A well-trained linear regression model for this dataset typically achieves a very low error, indicating good predictive accuracy.

	= logistic regression =		== logistic regression ==

	'''Logistic regression''' is a '''supervised learning algorithm''' used for '''classification problems''', where the target variable is '''categorical''' — meaning it can take a limited number of discrete values.		'''Logistic regression''' is a '''supervised learning algorithm''' used for '''classification problems''', where the target variable is '''categorical''' — meaning it can take a limited number of discrete values.
	It is used to model the '''probability''' that an event occurs based on one or more independent variables.		It is used to model the '''probability''' that an event occurs based on one or more independent variables.

Root at 17:43, 26 October 2025

2025-10-26T17:43:46Z

Show changes

Root at 17:32, 26 October 2025

2025-10-26T17:32:14Z

← Older revision		Revision as of 18:32, 26 October 2025
Line 1,405:		Line 1,405:
	A well-trained linear regression model for this dataset typically achieves a very low error, indicating good predictive accuracy.		A well-trained linear regression model for this dataset typically achieves a very low error, indicating good predictive accuracy.

	[[Category:~~Programming languages~~]]		[[Category:Python]]

Root at 11:11, 25 October 2025

2025-10-25T11:11:03Z

← Older revision		Revision as of 12:11, 25 October 2025
Line 1,250:		Line 1,250:

	For this linear regression example, we use a real dataset of '''U.S. housing data''' available on https://www.kaggle.com/datasets/vedavyasv/usa-housing		For this linear regression example, we use a real dataset of '''U.S. housing data''' available on https://www.kaggle.com/datasets/vedavyasv/usa-housing



	The dataset, called '''USA Housing''', contains information about house prices and their related features such as area population, number of rooms, and average income.<syntaxhighlight lang="python">import pandas as pd		The dataset, called '''USA Housing''', contains information about house prices and their related features such as area population, number of rooms, and average income.<syntaxhighlight lang="python">import pandas as pd
Line 1,284:		Line 1,282:
	The strongest correlation is typically between '''Average Area Income''' and '''Price''', meaning wealthier areas tend to have more expensive houses.		The strongest correlation is typically between '''Average Area Income''' and '''Price''', meaning wealthier areas tend to have more expensive houses.

	== preparing the data ==		'''preparing the data'''

	Before training the model, we split the dataset into '''features (X)''' and '''target (y)'''.		Before training the model, we split the dataset into '''features (X)''' and '''target (y)'''.

Line 1,296:		Line 1,295:

	print(X.head())		print(X.head())
	print(y.head())</syntaxhighlight>		print(y.head())</syntaxhighlight>'''splitting data for training and testing'''

	~~== splitting data for training and testing ==~~
	We divide the dataset into '''training data''' (used to build the model) and '''testing data''' (used to evaluate it).		We divide the dataset into '''training data''' (used to build the model) and '''testing data''' (used to evaluate it).

Line 1,312:		Line 1,310:

	# Training samples: 3500		# Training samples: 3500
	# Testing samples: 1500</syntaxhighlight>		# Testing samples: 1500</syntaxhighlight>'''training the model'''

	~~== training the model ==~~
	We now create and train a '''Linear Regression model''' using scikit-learn.		We now create and train a '''Linear Regression model''' using scikit-learn.

Line 1,340:		Line 1,337:
	</syntaxhighlight>Each coefficient represents how much the '''Price''' changes with one unit of change in the corresponding feature.		</syntaxhighlight>Each coefficient represents how much the '''Price''' changes with one unit of change in the corresponding feature.

	== making predictions ==		'''making predictions'''

	We can now predict house prices using the test data.		We can now predict house prices using the test data.

Line 1,357:		Line 1,355:
	1055 1.431508e+06 1.229242e+06		1055 1.431508e+06 1.229242e+06
	705 1.042374e+06 1.059353e+06		705 1.042374e+06 1.059353e+06
	</syntaxhighlight>		</syntaxhighlight>'''visualizing predictions'''

	~~== visualizing predictions ==~~
	We can visualize how close the predictions are to the actual prices. If the points align closely to the diagonal line, the model predictions are accurate.		We can visualize how close the predictions are to the actual prices. If the points align closely to the diagonal line, the model predictions are accurate.

Line 1,382:		Line 1,379:
	plt.show()		plt.show()
	</syntaxhighlight>		</syntaxhighlight>
	[[File:Python linear regression02.png\|none\|thumb\|495x495px]]		[[File:Python linear regression02.png\|none\|thumb\|495x495px]]'''evaluating the model'''

	~~== evaluating the model ==~~		We use three evaluation metrics to measure model performance:
	We use ~~'''~~three evaluation metrics~~'''~~ to measure model performance:

	1. '''MAE (Mean Absolute Error)''' average of absolute errors.		1. '''MAE (Mean Absolute Error)''' average of absolute errors.