Download our e-book of Introduction To Python

Top Discussion

How can I write Python code to change a date string from "mm/dd/yy hh: mm" format to "YYYY-MM-DD HH: mm" format? Exception Type: JSONDecodeError at /update/ Exception Value: Expecting value: line 1 column 1 (char 0) Why Sigmoid is a not used in Multi-class classification? Getting this error 'collections.OrderedDict' object has no attribute 'status_code' How do you identify important variables while working on a data set in machine learning? What are metaclasses in Python? How to leave/exit/deactivate a Python virtualenvironment Join Discussion

Top Courses

Webinars

Why You Should Learn Data Science in 2024

Dec 2nd (7:00 PM) 4643 Registered

More webinars

Data Wrangling in Python

Shashank Shanu

2 years ago

Introduction
How to Merge different dataset?
How to Group Data in a Dataframe?
How to Concatenate different Dataframe?

Introduction

Data wrangling is a process to manipulate the data in various formats. It involves processes such as merging, grouping, concatenating etc. Data wrangling is required for analyzing or getting them ready to be used with other data. It is an important part of any data science projects and many data scientists spent much of their time in it.

Python provides us some good built-in features to apply these wrangling methods to various data sets to achieve the analytical goal as per requirements.

How to Merge different dataset?

Python, panda’s library provides us a function, merge, which help us to join different datasets according to our requirements. Its works similar to the join operations between the DataFrame as standard databases.

pd.merge(left, right, how='inner', on=none, left_on=none, right_on=none,

left_index=false, right_index=false, sort=true)

Let’s take an example to get better understanding

# import the pandas library

import pandas as pd

left_df = pd.DataFrame({

         'S.No.':[1,2,3,4,5],

         'Name': ['Alex', 'Amy', 'Allen',
'Alice', 'Ayoung'],

        
'subjects':['Maths','Physics','Chemistry','Biology','Civis']})

right_df = pd.DataFrame(

         {'S.No.':[1,2,3,4,5],

         'Name': ['Billy', 'Brian', 'Bran',
'Bryce', 'Betty'],

        
'subjects':['Physics','Chemistry','History','Biology','Civis']})

print(left_df)

print(right_df)

Output:

&nbsp;&nbsp; S.No.&nbsp;&nbsp;&nbsp; Name&nbsp;&nbsp;subjects

0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1&nbsp;&nbsp;&nbsp; Alex&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Maths

1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2&nbsp;&nbsp;&nbsp;&nbsp; Amy&nbsp;&nbsp;&nbsp;Physics

2&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 3&nbsp;&nbsp; Allen&nbsp;Chemistry

3&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 4&nbsp;&nbsp; Alice&nbsp;&nbsp;&nbsp;Biology

4&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 5&nbsp; Ayoung&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Civics

&nbsp;&nbsp; S.No.&nbsp;&nbsp; Name&nbsp;&nbsp;subjects

0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1&nbsp; Billy&nbsp;&nbsp;&nbsp;Physics

1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2&nbsp; Brian&nbsp;Chemistry

2&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 3&nbsp;&nbsp; Bran&nbsp;&nbsp;&nbsp;History

3&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 4&nbsp; Bryce&nbsp;&nbsp;&nbsp;Biology

4&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 5&nbsp; Betty&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Civics

How to Group Data in a Dataframe?

While perform Exploratory data analysis many times we need to group our dataframe where we need the result in terms of various groups present in the data set. Panadas provides us with in-built methods which can roll the data into various groups.

Let’s take an example

Below I have shown you how we can group the data by year and then get the result for a specific year.

# import the pandas library

import pandas as pd

ipl = {'Team name': ['Riders', 'Riders', 'Devils',
'Devils', 'Kings',

         'kings',
'Kings', 'Kings', 'Riders', 'Royals', 'Royals', 'Riders'],

         'Rank':
[1, 2, 2, 3, 3,4 ,1 ,1,2 , 4,1,2],

         'Year':
[2014,2015,2014,2015,2014,2015,2016,2017,2016,2014,2015,2017],

         'Total
Points':[876,789,863,673,741,812,756,788,694,701,804,690]}

df1 = pd.DataFrame(ipl)

grouped = df.groupby('Year')

print(grouped.get_group(2014))

Output:

&nbsp;&nbsp;&nbsp;&nbsp; Team&nbsp;Rank&nbsp; Year&nbsp; Points

0&nbsp; Riders&nbsp;&nbsp;&nbsp;&nbsp;1&nbsp; 2014&nbsp;&nbsp;&nbsp;&nbsp; 876

2&nbsp; Devils&nbsp;&nbsp;&nbsp;&nbsp;2&nbsp; 2014&nbsp;&nbsp;&nbsp;&nbsp; 863

4&nbsp;&nbsp; Kings&nbsp;&nbsp;&nbsp;&nbsp;3&nbsp; 2014&nbsp;&nbsp;&nbsp;&nbsp; 741

9&nbsp; Royals&nbsp;&nbsp;&nbsp;&nbsp;4&nbsp; 2014&nbsp;&nbsp;&nbsp;&nbsp; 701

How to Concatenate different Dataframe?

Many times, we also need to concatenate different dataframe for our task to accomplished. Python Pandas package also provides various facilities for easily combining together Series, DataFrame, and Panel objects.

Let’s take an example

import
pandas as pd

first_df
= pd.DataFrame({

         'Name': ['Alex', 'Amy', 'Allen',
'Alice', 'Ayoung'],

        
'subject_id':['sub1','sub2','sub4','sub6','sub5'],

         'Marks_scored':[98,90,87,69,78]},

         index=[1,2,3,4,5])

second_df
= pd.DataFrame({

         'Name': ['Billy', 'Brian', 'Bran',
'Bryce', 'Betty'],

        
'subject_id':['sub2','sub4','sub3','sub6','sub5'],

         'Marks_scored':[89,80,79,97,88]},

         index=[1,2,3,4,5])

print(pd.concat([first_df,second_df]))

Output:

&nbsp;&nbsp;&nbsp;&nbsp; Name subject_id&nbsp; Marks_scored

1&nbsp;&nbsp;&nbsp; Alex&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; sub1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 98

2&nbsp;&nbsp;&nbsp;&nbsp; Amy&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; sub2&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 90

3&nbsp;&nbsp; Allen&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; sub4&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 87

4&nbsp;&nbsp; Alice&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; sub6&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 69

5&nbsp; Ayoung&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; sub5&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 78

1&nbsp;&nbsp; Billy&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; sub2&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 89

2&nbsp;&nbsp; Brian&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; sub4&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;80

3&nbsp;&nbsp;&nbsp; Bran&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; sub3&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 79

4&nbsp;&nbsp; Bryce&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; sub6&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 97

5&nbsp;&nbsp; Betty&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; sub5&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 88

I hope you enjoyed reading this article and finally, you came to know about Data Wrangling in Python.

For more such blogs/courses on data science, machine learning, artificial intelligence and emerging new technologies do visit us at InsideAIML.

Thanks for reading…

Happy Learning…

Related Blog

Top Discussion

Top Courses

Webinars

Why You Should Learn Data Science in 2024

Data Wrangling in Python

Table of Contents

Introduction

How to Merge different dataset?

How to Group Data in a Dataframe?

How to Concatenate different Dataframe?

Submit Review