Introduction to Big Data Analytics, Fall 2021

This course offers an introduction to the principles and concepts in big data analytics (BDA), which is gaining more popularity in recent years.
The major topics covered in this course will include: data mining, distributed computing, and parallel programming.
The course is offered at undergraduate level.

Notes on Online Courses

All students enrolled in this course have been added to the team in Microsoft Teams for the corresponding course number.
You can attend the online course in Microsoft Teams at the following channel: Team created for Big Data Analytics Course [Course Number: 294321]

Course Information

Instructor: Jenq-Haur Wang
E-mail: mailbox at the domain of our university, and my username is jhwang
Office: R1534, Technology Building (ext. 4238)
Office Hours: Tue. and Wed. 9:10-12:00am
Class Hours: Mon. 13:10-15:00pm, Wed. 15:10-16:00pm, R325, 6th Teaching Building
TA: Ms. Chen (R1424, Technology Building)
E-mail: (TBA)
Prerequisite: data structures (and algorithms), database systems, discrete math (probability), operating systems, and working experience of high-level programming languages
Textbook: (Selected chapters)
- Jiawei Han, Micheline Kamber and Jian Pei, Data Mining: Concepts and Techniques, 3rd ed., Morgan Kaufmann Publishers, July 2011. ([DM3])
References:
- Tom White, Hadoop: The Definitive Guide, 4th ed., O'Reilly Media, 2015.
- Jimmy Lin and Chris Dyer, Data-Intensive Text Processing with MapReduce, Morgan & Claypool Publishers, 2010.
- Jure Leskovec, Anand Rajaraman, Jeffrey David Ullman, Mining of Massive Datasets, 3rd Edition, Cambridge University Press, Feb. 2020.
- Jules S. Damji, Brooke Wenig, Tathagata Das, Denny Lee, Learning Spark: Lightning-Fast Big Data Analysis, 2nd ed., O'Reilly Media, Jul. 2020.
- Others: online documents (Hadoop, Spark), tutorials, and selected academic papers.

Latest News

Dec. 6, 2021: [NOTE] HW#4 has been announced.
Nov. 15, 2021: [NOTE] HW#3 has been announced.
Nov. 1, 2021: [NOTE] HW#2 has been announced.
Oct. 13, 2021: [NOTE] HW#1 has been announced.
Sep. 15, 2021: The homepage for Introduction to Big Data Analytics (Fall 2021) has been setup.

(Tentative) Schedule

* All slides can be downloaded at the iSchool+ plaform in NTUT.

Week	Date	Content	Reading	Note
1	Sep. 20 & 22, 2021	(9/20: Compensation Leave for Mid-Autumn Festival) Course Overview
2	Sep. 27 & 29, 2021	Introduction to Big Data Analytics Ch.2, Getting to Know Your Data	DM3, Ch.1 DM3, Ch.2
3	Oct. 4 & 6, 2021	Ch.3, Data Preprocessing	DM3, Ch.3 (selected)
4	Oct. 11 & 13, 2021	(Compensation Leave for National Day) Ch.3		HW#1
5	Oct. 18 & 20, 2021	Ch.6, Frequent Pattern Mining	DM3, Ch.6	Term Project Proposal
6	Oct. 25 & 27, 2021	Ch.8, Classification: Basic Concepts	DM3, Ch.8	Due: HW#1 Team Registration
7	Nov. 1 & 3, 2021	Ch.8		HW#2 Due: Team Registration
8	Nov. 8 & 10, 2021	Ch.9, Classification: Advanced Methods	DM3, Ch.9 (selected sections)
9	Nov. 15 & 17, 2021	Ch.10, Cluster Analysis: Basic Concepts and Methods	DM3, Ch.10	HW#3 Due: HW#2
10	Nov. 22 & 24, 2021	(11/22: Midterm Exam)
11	Nov. 29 & Dec. 1, 2021	Distribtued Platforms: Hadoop, Spark Ref: Notes on installation, configuration, and management of Hadoop & Spark clusters		Due: HW#3 Due: Proposal
12	Dec. 6 & 8, 2021	Parallel Programming Paradigms & Concepts		HW#4
13	Dec. 13 & 15, 2021	MapReduce Programming (Lab: Spark cluster demo)
14	Dec. 20 & 22, 2021	Spark Programming (Lab: classification using Spark)
15	Dec. 27 & 29, 2021	Term Project Presentation (Week 1)		Due: HW#4
16	Jan. 3 & 5, 2022	Term Project Presentation (Week 2)
17	Jan. 10 & 12, 2022	Term Project Presentation (Week 3)
18	Jan. 17 & 19, 2022	Term Project Presentation (Week 4)

Programming Assignments and Projects

Please hand in your assignment before deadline according to the following instructions.

Submission Instructions

NOTE: Programs or projects in electronic files must be submitted directly to i-School+.

If you cannot successfully submit your work, please contact with the TA or the instructor.

Homeworks

There will be several written assignments and programming exercises that target at different data analysis tasks.

HW#1 : Ch.2-3 Data Preprocessing
Due: Oct. 27, 2021
HW#2 : Ch.6 Frequent Pattern Mining
Due: Nov. 15, 2021
[NOTE] For the programming projects in HW#2, the DBLP dataset can be downloaded in XML format at: https://dblp.org/xml/release/
However, since DBLP dataset is very large, it might not be easy to analyze. You can try to download the partial datasets collected by different sources. The details can be checked in the Notes for HW#2, for example:
- SNAP dataset for DBLP collaboration network: https://snap.stanford.edu/data/com-DBLP.html
- Co-auth DBLP: https://www.cs.cornell.edu/~arb/data/coauth-DBLP/
- AMiner citation network dataset: https://aminer.org/citation
HW#3 : Ch.8-9 Classification
Due: Nov. 29, 2021
HW#4 : MepReduce Programming
Due: Extended to Dec. 27, 2021

Projects

Term Project: paper presentation or system demonstration

Item	Description	Time
Proposal	You are required to submit a proposal for term project one week after midterm exam. One option is to join the competitions. You can check the details on recent competitions as potential topics for term project.	Nov. 22, 2021 (Mon.)
Topics	For paper presentations, the paper quality will greatly affect your score in term project. Please carefully select good papers to read.
Schedule	Due to our time limits, we have to start the term project presentation as early as Dec. 27, 2021 (Mon.). * [NOTE] All presentations must be finished within the scheduled time slots, which will be the last 4 weeks in this semester. No other time slots will be avbailable.	Dec. 27, 29, 2020 & Jan. 3, 5, 10, 12, 17, 19, 2022
Report	Each team is required to upload the final report after finishing your presentation. The final report should contain at least the following: presentation slides (for all teams), and source code, installation/execution instructions, team members and task responsibility (for system projects)	Jan. 22, 2022 (Sat.)

Exams

Midterm Exam: Nov. 15-19, 2021
- Date: Nov. 22, 2021 (Mon.)
- Time: 13:10-15:00pm
- Location: R325, 6th Teaching Building
- Range: Ch. 2, 3, 6, 8, 9, 10 (TBD)
Final Exam: Jan. 17-22, 2022
- Note: There will be no final exam in this course. Instead, you are required to finish a term project for system development or paper presentation.

Scores

Please check the homework submission site for more details.

You can check the current score (including HW#1 and the Midterm exam, as of Nov. 30, 2021)

E-mail: jhwang AT ntut . edu . tw
Created: Sep. 15, 2021.
Last Updated: Dec. 21, 2021.