AICTE ATAL FDP · Data Science Essentials for Academicians

Python Programming for Data Science

Chandigarh Business School of Administration, Landran
Joseph Jude · @jjude · jjude.com

Ask→Collect→Clean→Analyze→Judge
·

Agenda

  1. AskWhat data science is, and why your college's data matters
  2. CollectThe room becomes the dataset
  3. CleanFind what's wrong in a marks sheet (Excel)
  4. AnalyzeSummarize in Excel, then the same steps in Python
  5. JudgeWhat the numbers can and can't claim
·

Today: one question, five steps

Ask

a question worth answering

→

Collect

get the data in

→

Clean

fix what's wrong, write it down

→

Analyze

summarize and show

→

Judge

decide which claim the numbers support

Excel by hand
Python the same steps, written down
No tool does this for you.

Today's question: Does evaluation time differ by department?We'll answer it by the end.

Not today: installing Python or memorizing syntax.

Ask

What is data science?

Using data to answer a question, from collection to a claim you can defend.

AnalyticsAnswer a focused question from data. StatisticsStudy design, inference, uncertainty. Data scienceThe whole sequence, from the question to a claim you can defend, using both.

The five steps from the agenda: Ask → Collect → Clean → Analyze → Judge.

@jjude · jjude.com
Ask

Will AI do this for us?

AI can

  • write the pandas code
  • draft a chart
  • summarize a table

You still must

  • choose the question that matters here
  • spot the duplicate row
  • know what your college's numbers mean
  • answer for the claim
Who has used an AI tool on a spreadsheet or dataset?
@jjude · jjude.com
Ask

Same AI models for everyone. Your data sets you apart.

The same AI model
↓
↓
↓
College Aits own records
College Bits own records
College Cits own records

Whose attendance? Whose feedback? Checked by whom?

@jjude · jjude.com
Ask

Your college's data journey

Admission

→

Attendance

→

Internal marks

→

Feedback

→

Placement

Reports that rest on them:NAACNBANIRF

Every stage is a place where a number can go wrong.

Which of these stages does your work touch?
@jjude · jjude.com
Collect

Three questions. You are the dataset.

  1. Your department
  2. Years of teaching experience
  3. Hours per week you spend evaluating student work (answer scripts, assignments, marks sheets)

Anonymous. No names or emails.

QR code for the room form

forms.gle/kc9ixFuGFM2WSB119

@jjude · jjude.com
Collect

Sample responses

What would you fix before trusting an average of this?
@jjude · jjude.com
Collect

Rows are people. Columns are questions.

Persondepartmentyearshours
AManagement105
BComputer Science48
CCommerce1510
DManagement76
EComputer Science2012
FCommerce97

Row = one record
one person (B is shaded)

Column = one variable
one question

Sample answers. hours = hours per week.

@jjude · jjude.com
Clean

Why start in Excel?

VisibleFamiliarForgiving

You can teach the logic of data work before you teach code.

@jjude · jjude.com
Clean

At least six things are wrong with this sheet. Find them.

@jjude · jjude.com
Clean

Average marks: 56 vs 50, until you remove one row

Average, with the duplicate

56Section A
50Section B

Section A: 38, 38, 74, 74

Average, without it

50Section A
50Section B

Section A: 38, 38, 74

Section B: 46, 50, 52, 52 in both. The blank and "78pt" are excluded for now.

A 6-mark gap (12% of Section B's mean) disappears.

@jjude · jjude.com
Clean

Six defects, six different failures

DefectWhat it breaksWhat to do
Duplicate rowinflates the mean, double-counts a personremove it, after confirming it is the same record
Blank markAVERAGE skips it, so n is smaller than headcountkeep it blank, report n; don't fill with 0
"78pt" in a number columnExcel's AVERAGE ignores text silentlyask the source; don't strip it blindly
SECTION A vs Section Aspelling variants split groupsone label for one section
Mixed or invalid dates
12/03/26, 2026-13-12		sorting and date maths fail; 12/03/26 is ambiguousagree one format (YYYY-MM-DD) and check impossible dates at the source
178 on a 100-mark testskews the meancheck the source sheet; don't delete without checking
@jjude · jjude.com
Clean

Six defects, six different failures

DefectWhat it breaksWhat to do
Duplicate rowinflates the mean, double-counts a personremove it, after confirming it is the same record
Blank markAVERAGE skips it, so n is smaller than headcountkeep it blank, report n; don't fill with 0
"78pt" in a number columnExcel's AVERAGE ignores text silentlyask the source; don't strip it blindly
SECTION A vs Section Aspelling variants split groupsone label for one section
Mixed or invalid dates
12/03/26, 2026-13-12		sorting and date maths fail; 12/03/26 is ambiguousagree one format (YYYY-MM-DD) and check impossible dates at the source
178 on a 100-mark testskews the meancheck the source sheet; don't delete without checking
@jjude · jjude.com
Clean

Six defects, six different failures

DefectWhat it breaksWhat to do
Duplicate rowinflates the mean, double-counts a personremove it, after confirming it is the same record
Blank markAVERAGE skips it, so n is smaller than headcountkeep it blank, report n; don't fill with 0
"78pt" in a number columnExcel's AVERAGE ignores text silentlyask the source; don't strip it blindly
SECTION A vs Section Aspelling variants split groupsone label for one section
Mixed or invalid dates
12/03/26, 2026-13-12		sorting and date maths fail; 12/03/26 is ambiguousagree one format (YYYY-MM-DD) and check impossible dates at the source
178 on a 100-mark testskews the meancheck the source sheet; don't delete without checking
@jjude · jjude.com
Clean

Six defects, six different failures

DefectWhat it breaksWhat to do
Duplicate rowinflates the mean, double-counts a personremove it, after confirming it is the same record
Blank markAVERAGE skips it, so n is smaller than headcountkeep it blank, report n; don't fill with 0
"78pt" in a number columnExcel's AVERAGE ignores text silentlyask the source; don't strip it blindly
SECTION A vs Section Aspelling variants split groupsone label for one section
Mixed or invalid dates
12/03/26, 2026-13-12		sorting and date maths fail; 12/03/26 is ambiguousagree one format (YYYY-MM-DD) and check impossible dates at the source
178 on a 100-mark testskews the meancheck the source sheet; don't delete without checking
@jjude · jjude.com
Clean

Six defects, six different failures

DefectWhat it breaksWhat to do
Duplicate rowinflates the mean, double-counts a personremove it, after confirming it is the same record
Blank markAVERAGE skips it, so n is smaller than headcountkeep it blank, report n; don't fill with 0
"78pt" in a number columnExcel's AVERAGE ignores text silentlyask the source; don't strip it blindly
SECTION A vs Section Aspelling variants split groupsone label for one section
Mixed or invalid dates
12/03/26, 2026-13-12		sorting and date maths fail; 12/03/26 is ambiguousagree one format (YYYY-MM-DD) and check impossible dates at the source
178 on a 100-mark testskews the meancheck the source sheet; don't delete without checking
@jjude · jjude.com
Clean

Six defects, six different failures

DefectWhat it breaksWhat to do
Duplicate rowinflates the mean, double-counts a personremove it, after confirming it is the same record
Blank markAVERAGE skips it, so n is smaller than headcountkeep it blank, report n; don't fill with 0
"78pt" in a number columnExcel's AVERAGE ignores text silentlyask the source; don't strip it blindly
SECTION A vs Section Aspelling variants split groupsone label for one section
Mixed or invalid dates
12/03/26, 2026-13-12		sorting and date maths fail; 12/03/26 is ambiguousagree one format (YYYY-MM-DD) and check impossible dates at the source
178 on a 100-mark testskews the meancheck the source sheet; don't delete without checking
@jjude · jjude.com
Clean

Cleaning is a set of decisions

  1. Keep the original untouched. Work on a copy.
  2. Write down every change: what, why, how many rows.
  3. Re-check: row count before and after, recompute the mean.

Section A's average: 56 with the duplicate, 50 without it, 57 if you read "78pt" as 78.

@jjude · jjude.com
Clean

The same cleaning, written down

import pandas as pd marks = pd.read_csv("marks.csv") # original stays untouched clean = marks.drop_duplicates() # 103 twice: 12 rows → 11 clean["Section"] = clean["Section"].str.title() # SECTION A → Section A clean["Mark"] = pd.to_numeric(clean["Mark"], errors="coerce") # 78pt → missing print(clean.groupby("Section")["Mark"].agg(["mean", "count"]))
           mean  count
Section               
Section A  50.0      3
Section B  50.0      4
Section C  89.0      2

The comments are the change log.

The 178 still needs a person to check it.

@jjude · jjude.com
Analyze

Does evaluation time differ by department?

departmenthours
Commerce10
Commerce7
Computer Science8
Computer Science12
Management5
Management6
group by department,
then take the mean→
departmentmean hoursn
Commerce8.52
Computer Science10.02
Management5.52

Sample answers, not all faculty. hours = hours per week.

@jjude · jjude.com
Analyze

Excel already does this

A pivot table is group-by plus an aggregate, in a grid you can rearrange.

@jjude · jjude.com
Analyze

Where Excel stops, and Python begins

Size

Very large files are slow, and a sheet has a row limit (1,048,576).

Pythonpandas is limited by memory, not a row cap.

Repeatability

Clicks leave no record, so rerunning means redoing.

PythonA script reruns every step.

Collaboration

Copies named final_v3.

PythonOne script, shared and versioned.

That thing you wish you never had to do by hand? This is where it stops.

@jjude · jjude.com
Analyze

One language, a few libraries

Collect · Clean · Analyzepandas · tables of data, built on NumPy Analyze, chartsMatplotlib · pandas draws through it When analysis needs a modelscikit-learn, statsmodels · beyond today JudgeNo library does this for you. Where you write itA notebook · Jupyter, or Google Colab
@jjude · jjude.com
Analyze

The whole workflow in six lines

·
import pandas as pd
Collect
data = pd.read_csv("responses.csv")
Clean
data["hours"] = pd.to_numeric(data["hours"], errors="coerce")
Clean
clean = data.drop_duplicates()
Analyze
summary = clean.groupby("department")["hours"].agg(["mean", "count"])
Analyze
summary["mean"].plot(kind="bar")

Judge is not a line of code.

@jjude · jjude.com
Analyze

What it prints and draws

                  mean  count
department                   
Commerce           8.5      2
Computer Science  10.0      2
Management         5.5      2
@jjude · jjude.com
Analyze

Which step is this line?

1
import pandas as pd
2
data = pd.read_csv("responses.csv")
3
data["hours"] = pd.to_numeric(data["hours"], errors="coerce")
4
clean = data.drop_duplicates()
5
summary = clean.groupby("department")["hours"].agg(["mean", "count"])
6
summary["mean"].plot(kind="bar")
Which step is the highlighted line?
@jjude · jjude.com
Analyze

Which step is this line?

1
import pandas as pd
2
data = pd.read_csv("responses.csv")
3
data["hours"] = pd.to_numeric(data["hours"], errors="coerce")
4
clean = data.drop_duplicates()
5
summary = clean.groupby("department")["hours"].agg(["mean", "count"])
6
summary["mean"].plot(kind="bar")
Which step is the highlighted line?
@jjude · jjude.com
Analyze

Which step is this line?

1
import pandas as pd
2
data = pd.read_csv("responses.csv")
3
data["hours"] = pd.to_numeric(data["hours"], errors="coerce")
4
clean = data.drop_duplicates()
5
summary = clean.groupby("department")["hours"].agg(["mean", "count"])
6
summary["mean"].plot(kind="bar")
Which step is the highlighted line?
@jjude · jjude.com
Analyze

Which step is this line?

1
import pandas as pd
2
data = pd.read_csv("responses.csv")
3
data["hours"] = pd.to_numeric(data["hours"], errors="coerce")
4
clean = data.drop_duplicates()
5
summary = clean.groupby("department")["hours"].agg(["mean", "count"])
6
summary["mean"].plot(kind="bar")
Which step is the highlighted line?
@jjude · jjude.com
Analyze

Which step is this line?

1
import pandas as pd
2
data = pd.read_csv("responses.csv")
3
data["hours"] = pd.to_numeric(data["hours"], errors="coerce")
4
clean = data.drop_duplicates()
5
summary = clean.groupby("department")["hours"].agg(["mean", "count"])
6
summary["mean"].plot(kind="bar")
What would break if we skipped line 4?
@jjude · jjude.com
Analyze

This runs without errors. What's the mistake?

1
import pandas as pd
2
data = pd.read_csv("marks.csv")
3
# average marks by section
4
avg = data.groupby("Section")["attendance"].mean()
5
print(avg)
@jjude · jjude.com
Analyze

This runs without errors. What's the mistake?

1
import pandas as pd
2
data = pd.read_csv("marks.csv")
3
# average marks by section
4
avg = data.groupby("Section")["attendance"].mean()
5
print(avg)

It averages the attendance column, not marks. The comment and the code disagree. It prints plausible numbers and no error.

What else would you check before trusting this?
@jjude · jjude.com
Analyze

The same table, two ways

Excel

Drag department to Rows and hours to Values, then change Sum to Average.

Python

summary = (clean .groupby("department")["hours"] .agg(["mean", "count"]))
Shift 1From cells to whole columns. Shift 2From clicking to writing it down.
@jjude · jjude.com
Judge

Judge: what can we claim?

  1. Could it be chance? Small groups vary by luck.
  2. Could something else explain it? Moving together is not causing.
  3. Compared to what? A number means little without a baseline.

The data can be clean and the claim still wrong.

@jjude · jjude.com
Judge

Could it be chance?

Management 5, 6 · Computer Science 8, 12 · gap 4.5 h

If department made no difference, any answer could sit in either group. Deal the four answers into two pairs:

pair 1pair 2gap
5, 68, 124.5
8, 125, 64.5
5, 86, 122.5
6, 125, 82.5
5, 126, 81.5
6, 85, 121.5

Gap ≥ 4.5 in 2 of 6 → p = 0.33

A p-value is the probability, assuming no real difference (the null hypothesis), of a result at least as extreme as the one observed.

Statistically significant = p below a threshold chosen in advance (often 0.05): unlikely if there were no real difference.

It does not mean large, important or proven.

@jjude · jjude.com
Judge

Attendance and marks move together. So raise attendance?

"r = 0.6 between attendance and marks, so raising attendance will raise marks."

r = 0.6 describes a positive linear association in this sample. It doesn't say which variable drives the other.

Reverse causation

Students who do well may attend more.

Confounding

A third factor (for example, motivation or timetable) may drive both.

Chance

In a small class, the pattern may not repeat.

Give one example of each from your own classroom.
@jjude · jjude.com
Judge

Computer Science: 10 hours a week. Is that a lot?

Compared to what? hours a week spent on evaluation

The roomAll six answers average 8.0. Computer Science is 2 above.we can check Student workload10 hours for how many students?our form didn't ask Last semesterMore or less than before?our form didn't ask

A fair comparison often needs data you planned to collect.

@jjude · jjude.com
Back to the question

Does evaluation time differ by department?

Analyze saidCommerce 8.5 · Computer Science 10.0 · Management 5.5 hours (n = 2 each) Could it be chance?Yes. p = 0.33: luck alone explains a gap this big. Something else?Possibly. Self-reported hours; how much student work each person has wasn't asked. Compared to what?Only to the room. No earlier figure to compare against. Judge saysCan't tell yet.

AI can write the code. You own the question.

@jjude · jjude.com
Back to the question

Questions to take back to your class

AskWhat decision will this answer change? What would count as an answer? CollectWhat does each column mean? Who filled it in, and how? CleanWhat did I change, why, and in how many rows? AnalyzeWhat is the n behind each number? JudgeCould it be chance? Could something else explain it? Compared to what?
@jjude · jjude.com

Thank you

Joseph Jude · @jjude · jjude.com