top of page

Naive Bayes Classifier Using Pyspark

Pyspark
Stuck on a Naive Bayes project? Get help from Skilled & Vetted Data Science expert on Codersarts
Applied Machine Learning.jpg

Prerequisite :

  • You must have python 3.7 or more installed on your system.

  • You must have hadoop and pyspark installed on your system

  • You must have a Spyder, Jupyter notebook on your system. Spyder or jupyter notebook come up with anaconda. you just need to launch them after installing anaconda.

  • If you work on a google colab no need to install python or Any other IDE, you just need to sign in with google colab and install pyspark using "!pip install pyspark" this command.

  • Used a jupyter notebook to build this project.


Skilled required:

  • Python programming language

  • Basic Statistical analysis skills

  • Machine learning concept


What you’ll learn

  • How to read the data using pyspark dataframe

  • Perform Basic Exploratory Data analysis using pyspark

  • How to apply Naive Bayes Classification using pyspark

  • How to evaluate the model in pyspark.



Problem Statement or Description:

This project will show how to apply Naive Bayes Classification algorithms using pyspark on a titanic dataset. This dataset contains 11 features:passengerid, age, pclass, name, sex, age, sibsp, parch, ticket, ticket, fare, cabin and embarked. Target columns are the “Survived” column, the passenger is survived or not in the titanic disaster. In this project build the model which will predict if the passenger has survived or not on a testing set.


Key highlights of projects or Essence:

  • This project is about classification analysis.

  • This project shows you how to read the data and perform some basic Exploratory data analysis using pyspark

  • This project shows you how to perform data preprocessing.

  • This project shows you how to apply a Naive bayes classification using pyspark.

  • At the end of this project, Evaluate the model.


Packages and module used :

  • Pyspark

  • VectorAssembler

  • StringIndexer

  • StandardScaler

  • NaiveBayes

  • BinaryClassificationEvaluator

  • MulticlassClassificationEvaluator


Recommended projects:

  1. Chicago crime data analysis

  2. Census income dataset

  3. Student performance

  4. Divorce prediction

  5. Cervical cancer risk factor


Skills: 

Naive Bayes, Data Science, Machine Learning, Deep Learning, scikit-learn, pandas

Browser Sample projects

Are you working on this project idea?
if you need any assistance or mentorship, please send a help request

bottom of page