Introduction to Big Data Analytics, Spring 2018

This course offers an introduction to the principles and concepts in big data analytics (BDA), which is gaining more popularity in recent years.
The major topics covered in this course will include: data mining, distributed computing, and parallel programming.
The course is offered at undergraduate-level.

Course Information

Instructor: Jenq-Haur Wang
E-mail: mailbox at the domain of csie within NTUT, and my username is jhwang
Office: R1534, Technology Building (ext. 4238)
Office Hours: Mon. and Wed. 4:10-6:00pm
Class Hours: Tue. 9:10-10:00am, Wed. 10:10-12:00am, R227/R327, 6th Teaching Building
TA: Miss Lu (R1424, Technology Building)
E-mail: t106598005 AT NTUT . ORG DOT TW
Prerequisite: data structures (and algorithms), database systems, discrete math (probability), operating systems, and working experience of high-level programming languages
Textbook: (Selected chapters)
- Jiawei Han, Micheline Kamber and Jian Pei, Data Mining: Concepts and Techniques, 3rd ed., Morgan Kaufmann Publishers, July 2011. ([DM3])
- Tom White, Hadoop: The Definitive Guide, 4th ed., O'Reilly Media, 2015.
References:
- Jimmy Lin and Chris Dyer, Data-Intensive Text Processing with MapReduce, Morgan & Claypool Publishers, 2010.
- Jure Leskovec, Anand Rajaraman, Jeffrey David Ullman, Mining of Massive Datasets, 2nd Edition, Cambridge University Press, November 2014.
- Holden Karau, Andy Konwinski, Patrick Wendell, Matei Zaharia, Learning Spark: Lightning-Fast Big Data Analysis, O'Reilly Media, January 2015.
- Others: online documents (Hadoop, Spark), tutorials, and selected academic papers.

Latest News

Jul. 1, 2018: The total score has been announced. Please check if there's any questions before 5:00pm, Jul. 4, 2018 (Wed.).
May 22, 2018: [NOTE] Please check if you are included in the schedule of term project proposal. Note that 104590027 and 104590020 have the same title. Please check if it'is correct or not as soon as possible.
May 21, 2018: The score of midterm exam has been posted.
May 18, 2018: HW#4 has been announced.
May 17, 2018: The current schedule of term project presentation has been posted. There are 12 students who haven't submitted your proposals. Please do it as soon as possible.
May 4, 2018: The current score has been posted.
Mar. 27, 2018: HW#2 has been announced.
Mar. 20, 2018: The brochure of Industry Professional Assessment System (iPAS) is provided. Please check the website of iPAS for more details.
Mar. 13, 2018: [NOTE] HW#1 has been announced.
Mar. 5, 2018: The homepage for Introduction to Big Data Analytics (Spring 2018) has been setup.

(Tentative) Schedule

Week	Date	Content	Reading	Note
1	Feb. 27 & 28, 2018	Course Overview (2/28: 228 Peace Memorial Day)
2	Mar. 6 & 7, 2018	Introduction to Big Data Analytics Ch.2, Getting to Know Your Data	DM3, Ch.1 DM3, Ch.2
3	Mar. 13 & 14, 2018	Ch.3, Data Preprocessing Ch.6, Frequent Pattern Mining	DM3, Ch.3 (selected) DM3, Ch.6	HW#1
4	Mar. 20 & 21, 2018	Ch. 6: Frequent Pattern Mining	DM3, Ch.6
5	Mar. 27 & 28, 2018	Ch.8, Classification: Basic Concepts	DM3, Ch.8	HW#2 Due: HW#1 (3/28)
6	Apr. 3 & 4, 2018	(4/3: Compensation Leave for Sports Day 4/4: Children's Day)
7	Apr. 10 & 11, 2018	4/11: (TA) Term Project Proposal & Team Registration (Leave for ACM SAC 2018)		Term Project Proposal Due: HW#2
8	Apr. 17 & 18, 2018	Ch.9, Classification: Advanced Methods	DM3, Ch.9 (selected sections)	HW#3
9	Apr. 24 & 25, 2018	Ch.10, Cluster Analysis: Basic Concepts and Methods	DM3, Ch.10
10	May 1 & 2, 2018	(5/2: Midterm Exam)		Due: HW#3
11	May 8 & 9, 2018	Distribtued Platforms: Hadoop, Spark Ref: Notes on installation, configuration, and management of Hadoop & Spark clusters		Due: Proposal
12	May 15 & 16, 2018	Parallel Programming Paradigms & Concepts		HW#4: Hadoop & Spark
13	May 22 & 23, 2018	MapReduce Programming
14	May 29 & 30, 2018	Spark Programming Advanced Topics
15	Jun. 5 & 6, 2018	Term Project Presentation (Week 1): 8 teams completed.		Due: HW#4
16	Jun. 12 & 13, 2018	Term Project Presentation (Week 2): 7 teams completed.
17	Jun. 19 & 20, 2018	Term Project Presentation (Week 3): 9 teams completed.
18	Jun. 26 & 27, 2018	Term Project Presentation (Week 4)

Programming Assignments and Projects

Please hand in your assignment before deadline according to the following instructions.

Submission Instructions

NOTE: Programs or projects in electronic files must be submitted directly to the TA online.

If you cannot successfully submit your work, please contact with the TA or the instructor.

Homeworks

There will be several written assignments and programming exercises that target at different data analysis tasks.

HW#1: Ch.2-3 Data Preprocessing
Due: Mar. 28, 2018
HW#2: Ch.6 Frequent Pattern Mining
Due: Apr. 11, 2018
HW#3: Ch.8-9 Classification
Due: May 1, 2018
HW#4: distributed platform setup, and simlpe statistics using MapReduce programming
Due: Jun. 6, 2018

Projects

Term Project: paper presentation or system demonstration

Item	Description	Time
Proposal	You are required to submit a proposal for term project one week after midterm exam. Current schedule of term project presentation (34 teams in total, as of Jun. 26, 2018)	May 8, 2018 (Tue.)
Topics	For paper presentations, the paper quality will greatly affect your score in term project. Please carefully select good papers to read.
Schedule	Due to our time limits, we might have to start the term project presentation as early as Jun. 5, 2018 (Tue.). * [NOTE] All presentations must be finished within the scheduled time slots, which will be the last four weeks in this semester. No other time slots will be avbailable.	Jun. 5, 6, 12, 13, 19, 20, 26, 27, 2018
Report	Each team is required to upload the final report after finishing your presentation. The final report should contain at least the following: presentation slides (for all teams), and source code, installation/execution instructions, team members and task responsibility (for system projects)	Jun. 29, 2018 (Fri.)

Exams

Midterm Exam: Apr. 23-27, 2018
- Date: May 2 (Wed.)
- Time: 10:10-12:00am
- Location: R327, 6th Teaching Building
- Range: Ch.2, 3, 6, 8, 9, 10
Final Exam: Jun. 25-29, 2018
- Note: There will be no final exam in this course. Instead, you are required to finish a term project for system development or paper presentation.

Scores

Please check the homework submission site for more details.

You can check the total score. (including HW#1-HW#4, Midterm exam, and the Term Project, as of Jul. 4, 2018)
Please check if there's any questions about your score as soon as possible, before 5:00pm, Jul. 4, 2018!

E-mail: jhwang AT csie . ntut . edu . tw
Created: Feb. 25, 2018.
Last Updated: Jul. 4, 2018.