Introduction to Big Data Analytics, Fall 2020

This course offers an introduction to the principles and concepts in big data analytics (BDA), which is gaining more popularity in recent years.
The major topics covered in this course will include: data mining, distributed computing, and parallel programming.
The course is offered at undergraduate level.

Course Information

Instructor: Jenq-Haur Wang
E-mail: mailbox at the domain of csie within our university, and my username is jhwang
Office: R1534, Technology Building (ext. 4238)
Office Hours: Mon. and Fri. 9:10-12:00am
Class Hours: Mon. 13:10-15:00pm, Wed. 15:10-16:00pm, R325, 6th Teaching Building
TA: Mr. Chen (R1424, Technology Building)
E-mail: (TBA)
Prerequisite: data structures (and algorithms), database systems, discrete math (probability), operating systems, and working experience of high-level programming languages
Textbook: (Selected chapters)
- Jiawei Han, Micheline Kamber and Jian Pei, Data Mining: Concepts and Techniques, 3rd ed., Morgan Kaufmann Publishers, July 2011. ([DM3])
References:
- Tom White, Hadoop: The Definitive Guide, 4th ed., O'Reilly Media, 2015.
- Jimmy Lin and Chris Dyer, Data-Intensive Text Processing with MapReduce, Morgan & Claypool Publishers, 2010.
- Jure Leskovec, Anand Rajaraman, Jeffrey David Ullman, Mining of Massive Datasets, 2nd Edition, Cambridge University Press, November 2014.
- Holden Karau, Andy Konwinski, Patrick Wendell, Matei Zaharia, Learning Spark: Lightning-Fast Big Data Analysis, O'Reilly Media, January 2015.
- Others: online documents (Hadoop, Spark), tutorials, and selected academic papers.

Latest News

Jan. 20, 2021: The final score has been announced. Please check if there's any questions before 5:00pm, Jan. 21, 2021 (Thu.).
Dec. 7, 2020: [NOTE] The current schedule of term project presentation has been arranged. If you haven't submitted your proposal, please do it as soon as possible.
Nov. 30, 2020: [NOTE] HW#4 has been announced.
Nov. 2, 2020: [NOTE] HW#3 has been announced.
Oct. 21, 2020: [NOTE] HW#2 has been announced.
Sep. 30, 2020: [NOTE] HW#1 has been announced.
Sep. 11, 2020: The homepage for Introduction to Big Data Analytics (Fall 2020) has been setup.

(Tentative) Schedule

* All slides can be downloaded at the iSchool plaform in NTUT.

Week	Date	Content	Reading	Note
1	Sep. 14 & 16, 2020	Course Overview Introduction to Big Data Analytics	DM3, Ch.1
2	Sep. 21 & 23, 2020	Ch.2, Getting to Know Your Data	DM3, Ch.2
3	Sep. 28 & 30, 2020	Ch.3, Data Preprocessing	DM3, Ch.3 (selected)	9/30: HW#1
4	Oct. 5 & 7, 2020	Ch.6, Frequent Pattern Mining	DM3, Ch.6
5	Oct. 12 & 14, 2020	Ch.8, Classification: Basic Concepts	DM3, Ch.8	Term Project Proposal 10/14 Due: HW#1
6	Oct. 19 & 21, 2020	Ch.8, Classification: Basic Concepts	DM3, Ch.8	10/21: HW#2 Team Registration
7	Oct. 26 & 28, 2020	Ch.9, Classification: Advanced Methods	DM3, Ch.9 (selected sections)	Due: Team Registration
8	Nov. 2 & 4, 2020	Ch.10, Cluster Analysis: Basic Concepts and Methods	DM3, Ch.10	HW#3 11/4 Due: HW#2
9	Nov. 9 & 11, 2020	Ch.10, Cluster Analysis: Basic Concepts and Methods	DM3, Ch.10
10	Nov. 16 & 18, 2020	(11/16: Midterm Exam)		Due: HW#3
11	Nov 23 & 25, 2020	Distribtued Platforms: Hadoop, Spark Ref: Notes on installation, configuration, and management of Hadoop & Spark clusters (11/25: Leave for MLN 2020)		Due: Proposal
12	Nov. 30 & Dec. 2, 2020	Parallel Programming Paradigms & Concepts		HW#4
13	Dec. 7 & 9, 2020	MapReduce Programming (Lab: Spark cluster demo)
14	Dec. 14 & 16, 2020	Spark Programming (Lab: classification using Spark)		Due: HW#4
15	Dec. 21 & 23, 2020	Term Project Presentation (Week 1) - 6 teams completed.
16	Dec. 28 & 30, 2020	Term Project Presentation (Week 2) - 7 teams completed.
17	Jan. 4 & 6, 2021	Term Project Presentation (Week 3) - 6 teams completed.
18	Jan. 11 & 13, 2021	Term Project Presentation (Week 4) - 4 teams completed.

Programming Assignments and Projects

Please hand in your assignment before deadline according to the following instructions.

Submission Instructions

NOTE: Programs or projects in electronic files must be submitted directly to i-School+.

If you cannot successfully submit your work, please contact with the TA or the instructor.

Homeworks

There will be several written assignments and programming exercises that target at different data analysis tasks.

HW#1 : Ch.2-3 Data Preprocessing
Due: Oct. 14, 2020
HW#2 : Ch.6 Frequent Pattern Mining
Due: Nov. 4, 2020
[NOTE] For the programming projects in HW#2, the DBLP dataset can be downloaded in XML format at: https://dblp.org/xml/release/
However, since DBLP dataset is very large, it might not be easy to analyze. You can try to download the partial datasets collected by different sources. The details can be checked in the Notes for HW#2, for example:
- SNAP dataset for DBLP collaboration network: https://snap.stanford.edu/data/com-DBLP.html
- Co-auth DBLP: https://www.cs.cornell.edu/~arb/data/coauth-DBLP/
- AMiner citation network dataset: https://aminer.org/citation
HW#3 : Ch.8-9 Classification
Due: Nov. 16, 2020
HW#4 : MepReduce Programming
Due: Dec. 14, 2020

Projects

Term Project: paper presentation or system demonstration

Item	Description	Time
Proposal	You are required to submit a proposal for term project one week after midterm exam.	Nov. 23, 2020 (Mon.)
Topics	For paper presentations, the paper quality will greatly affect your score in term project. Please carefully select good papers to read.
Schedule	Due to our time limits, we have to start the term project presentation as early as Dec. 21, 2020 (Mon.). Please check the current schedule of presentation. (as of Jan. 11, 2021) * [NOTE] All presentations must be finished within the scheduled time slots, which will be the last 4 weeks in this semester. No other time slots will be avbailable.	Dec. 21, 23, 28, 30, 2020 & Jan. 4, 6, 11, 13, 2021
Report	Each team is required to upload the final report after finishing your presentation. The final report should contain at least the following: presentation slides (for all teams), and source code, installation/execution instructions, team members and task responsibility (for system projects)	Jan. 15, 2021 (Fri.)

Exams

Midterm Exam: Nov. 9-13, 2020
- Date: Nov. 16, 2020 (Mon.)
- Time: 13:10-15:00pm
- Location: R325, 6th Teaching Building
- Range: Ch. 2, 3, 6, 8, 9, 10
Final Exam: Jan. 11-15, 2021
- Note: There will be no final exam in this course. Instead, you are required to finish a term project for system development or paper presentation.

Scores

Please check the homework submission site for more details.

Final score (as of Jan. 21, 2021)

E-mail: jhwang AT csie . ntut . edu . tw
Created: Sep. 11, 2020.
Last Updated: Jan. 21, 2021.