Lab: Exploring the Dataset

Estimated time needed: 30 minutes

Introduction

Data exploration is the initial phase of data analysis where we aim to understand the data's characteristics, identify patterns, and uncover potential insights. It is a crucial step that helps us make informed decisions about subsequent analysis.

Objectives

After completing this lab, you will be able to:

• Summarize the key characteristics of a dataset.
• Identify different data types commonly used in data analysis.

Install the required library

!pip install pandas

Collecting pandas
  Downloading pandas-2.3.3-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (91 kB)
Collecting numpy>=1.26.0 (from pandas)
  Downloading numpy-2.4.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (6.6 kB)
Requirement already satisfied: python-dateutil>=2.8.2 in /opt/conda/lib/python3.12/site-packages (from pandas) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /opt/conda/lib/python3.12/site-packages (from pandas) (2024.2)
Collecting tzdata>=2022.7 (from pandas)
  Downloading tzdata-2025.3-py2.py3-none-any.whl.metadata (1.4 kB)
Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.12/site-packages (from python-dateutil>=2.8.2->pandas) (1.17.0)
Downloading pandas-2.3.3-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (12.4 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.4/12.4 MB 166.3 MB/s eta 0:00:00
[?25hDownloading numpy-2.4.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (16.4 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.4/16.4 MB 160.5 MB/s eta 0:00:00
[?25hDownloading tzdata-2025.3-py2.py3-none-any.whl (348 kB)
Installing collected packages: tzdata, numpy, pandas
Successfully installed numpy-2.4.1 pandas-2.3.3 tzdata-2025.3

import pandas as pd

Load the dataset

Read Data

We utilize the pandas.read_csv() function for reading CSV files.

The functions below will download the dataset into your browser:

# Load the dataset directly from the URL
file_path = "https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/VYPrOu0Vs3I0hKLLjiPGrA/survey-data-with-duplicate.csv"

Utilize the Pandas method read_csv() to load the data into a dataframe.

df = pd.read_csv(file_path)

Hands on Lab

Explore the dataset

It is a good idea to print the top 5 rows of the dataset to get a feel of how the dataset will look.

Display the top 5 rows and columns from your dataset.

## Write your code here
df.head()

Find out the number of rows and columns

Start by exploring the numbers of rows and columns of data in the dataset.

Print the number of rows in the dataset.

## Write your code here
print(df.shape[0])

65457

Print the number of columns in the dataset.

## Write your code here
print(df.shape[-1])

114

Identify the data types of each column

Explore the dataset and identify the data types of each column.

Print the datatype of all columns.

## Write your code here
for col in df:
    print(col, df[col].dtype)

ResponseId int64
MainBranch object
Age object
Employment object
RemoteWork object
Check object
CodingActivities object
EdLevel object
LearnCode object
LearnCodeOnline object
TechDoc object
YearsCode object
YearsCodePro object
DevType object
OrgSize object
PurchaseInfluence object
BuyNewTool object
BuildvsBuy object
TechEndorse object
Country object
Currency object
CompTotal float64
LanguageHaveWorkedWith object
LanguageWantToWorkWith object
LanguageAdmired object
DatabaseHaveWorkedWith object
DatabaseWantToWorkWith object
DatabaseAdmired object
PlatformHaveWorkedWith object
PlatformWantToWorkWith object
PlatformAdmired object
WebframeHaveWorkedWith object
WebframeWantToWorkWith object
WebframeAdmired object
EmbeddedHaveWorkedWith object
EmbeddedWantToWorkWith object
EmbeddedAdmired object
MiscTechHaveWorkedWith object
MiscTechWantToWorkWith object
MiscTechAdmired object
ToolsTechHaveWorkedWith object
ToolsTechWantToWorkWith object
ToolsTechAdmired object
NEWCollabToolsHaveWorkedWith object
NEWCollabToolsWantToWorkWith object
NEWCollabToolsAdmired object
OpSysPersonal use object
OpSysProfessional use object
OfficeStackAsyncHaveWorkedWith object
OfficeStackAsyncWantToWorkWith object
OfficeStackAsyncAdmired object
OfficeStackSyncHaveWorkedWith object
OfficeStackSyncWantToWorkWith object
OfficeStackSyncAdmired object
AISearchDevHaveWorkedWith object
AISearchDevWantToWorkWith object
AISearchDevAdmired object
NEWSOSites object
SOVisitFreq object
SOAccount object
SOPartFreq object
SOHow object
SOComm object
AISelect object
AISent object
AIBen object
AIAcc object
AIComplex object
AIToolCurrently Using object
AIToolInterested in Using object
AIToolNot interested in Using object
AINextMuch more integrated object
AINextNo change object
AINextMore integrated object
AINextLess integrated object
AINextMuch less integrated object
AIThreat object
AIEthics object
AIChallenges object
TBranch object
ICorPM object
WorkExp float64
Knowledge_1 object
Knowledge_2 object
Knowledge_3 object
Knowledge_4 object
Knowledge_5 object
Knowledge_6 object
Knowledge_7 object
Knowledge_8 object
Knowledge_9 object
Frequency_1 object
Frequency_2 object
Frequency_3 object
TimeSearching object
TimeAnswering object
Frustration object
ProfessionalTech object
ProfessionalCloud object
ProfessionalQuestion object
Industry object
JobSatPoints_1 float64
JobSatPoints_4 float64
JobSatPoints_5 float64
JobSatPoints_6 float64
JobSatPoints_7 float64
JobSatPoints_8 float64
JobSatPoints_9 float64
JobSatPoints_10 float64
JobSatPoints_11 float64
SurveyLength object
SurveyEase object
ConvertedCompYearly float64
JobSat float64

Print the mean age of the survey participants.

## Write your code here
print(df['Age'].describe())
mean_age = []
for a in df['Age']:
    age = ''.join(c for c in str(a) if c.isdigit())
    if age == '':
        continue
    elif len(age) == 4:
        age = (int(age[:2]) + int(age[2:])) / 2
    else:
        age = int(age)
    mean_age.append(age)
print('Mean age:', round(sum(mean_age) / len(mean_age), 2))

count               65457
unique                  8
top       25-34 years old
freq                23913
Name: Age, dtype: object
Mean age: 33.03

The dataset is the result of a world wide survey. Print how many unique countries are there in the Country column.

## Write your code here
print(len(df['Country'].unique()))

186

Copyright © IBM Corporation. All rights reserved.