Data Visualization packages in Python - Pandas

In the previous article, we saw how dplyr and tidyr packages in R can be used to visualize and plot data to gain a better understanding of the underlying properties. Now let’s explore the data visualization techniques in Python and see how similar tasks can be performed.

Pandas:

Pandas is a Python package aimed toward creating a platform to carry out practical data analysis and manipulation tasks. It has data structures that are equivalent to dataframe in R, and it can handle multiple types of data like SQL or Excel data, information present in the form of matrices, time series data, and labeled/unlabeled data. Here’s a preview of the tasks that can be carried out using Pandas:

  1. groupby(..) function that is used to aggregate data through split-apply-combine operations
  2. Merging, joining, slicing, reshaping and subsetting of data
  3. Flexibility to change the dimension of data: addition and removal of columns
  4. Handling of missing data similar to libraries in R

You can import Pandas just like you would import any other library in Python.

First step of dealing with Pandas involves reading data from a csv file.

File_path: the location of the csv file to be read

Header: Can be None if you want the column heading to be Null. If column names are needed, then pass them as a list to the header argument.

After reading the data, placing it into a dataframe gives us the flexibility to perform various operations on it. To convert data into dataframe format use:

We are going to use the following dataframe in our later examples:

It is always important to have an estimate about the extreme values of the data. It is also convenient to have the data in a sorted format. To accomplish this, data can be sorted based on a column value in the dataframe using the sorting function:

column : column object of the dataframe

ascending: default value is True. If set to False, the data is sorted in descending order.

To get the entity with the  maximum value (which is the last value in the sorted dataframe), tail(n) function can be used.  n is the number of values from the last elements that need to be taken into consideration:

Similarly, head() collects values from the top:

Both head and tail, by default, will display 5 values from the top and bottom, respectively.

To get the information about the dataframe, use info():

In the examples that follow, we are going to use the following dataframe that contains the complete exam results of all the 6 students (there are 6 subjects):

When there are multiple entries for each object, the aggregate option comes into play. We use the groupby() function to accomplish it. To get the total marks for each student, we need to aggregate all the name objects using the sum function:

column_name: takes the list of columns based on which grouping needs to be done.

as_index: default value is True and means that the columns mentioned in list will be considered as indices for the new dataframe formed. When set to False, numerical numbering starting from 0 is given as the index.

Data can also be plotted using Pandas, but it requires pyplot from matplotlib:

Data Visualization - Pandas

Data Visualization in Python - Pandas

To get the frequencies of the values in a particular column, use value_counts():

To get the unique values in a column:

Dataframes can be accessed using the index too. ix() function is used to extract data using the index in numerical values:

Adding a column to data is quite easy in case of dataframe in Pandas:

loc()  can be used to extract subset of a dataframe:

iloc() is similar to loc() but here the index can be represented as numerals rather than as actual object names:

Subset of the dataframe can also be extracted by imposing a condition over the column values using logical operators:

In the above example, all the rows with ‘Pass’ column value as True are separated out using the logical equality condition.  

You can use the del function to delete a column.

Data can be changed into different storage formats. stack() and unstack() functions are used for this. stack() is used to bring down the column names into index values and unstack() is used to revert the stacking action. Give it a try and see the output.

The rows and columns interchange positions after unstacking. We can revert this using the transpose function.

Mean and standard deviation for a particular value of the data can be calculated using standard functions. Mean: mean() and standard deviation: std()

The above example adds a column to the dataframe containing the deviation from the mean value of Marks.

Generating a time series data:

Plotting the value of stock over 4 years using pyplot:

Data visualization in Python Pandas

 

Plot.bar() Data Visualization in Python

There are more plotting tools like the seaborn library that can create more sophisticated plots. With these data visualization packages in R and Python, we are ready to advance to the core concepts of Machine Learning.

We have our Machine Learning practice section coming soon. Stay tuned. 

About the Author

Krunal is passionate about exactly 3 things in life. Writing, travelling and marketing. In the rest of his time he learns, or fascinates himself with, Game Theory, Big Data, Artificial Intelligence and whole lotta food.
1