Big Data Analytics (IMFI), Fall 2019

This course offers an introduction to the principles and concepts in big data analytics (BDA), which is gaining more popularity in recent years.
The course is offered for the International Master Program in Financial Technology and Innovation Enterpreneur (IMFI). It's taught by two instructors: Chen-Shu Wang, and Jenq-Haur Wang.
This page is for the second part. The major topics covered in the second part include: data mining and distributed processing.

Course Information (for the second part)

Instructor: Jenq-Haur Wang
E-mail: mailbox at the domain of CSIE within our university, and my username is jhwang
Office: R1534, Technology Building (ext. 4238)
Office Hours: Tue. 9:10-12:00am, and Wed. 9:10-12:00am
Class Hours: Thu. 9:10-12:00am. R312, General Studies Building
TA: Mr. Chou (@ R1424, Technology Building)
Textbook:
- Jiawei Han, Micheline Kamber and Jian Pei, Data Mining: Concepts and Techniques, 3rd ed., Morgan Kaufmann Publishers, July 2011. ([DM3]) (selected chapters)
References:
- Holden Karau, Andy Konwinski, Patrick Wendell, Matei Zaharia, Learning Spark: Lightning-Fast Big Data Analysis, O'Reilly Media, January 2015.
- Bill Chambers and Matei Zaharia, Spark: The Definitive Guide, O'Reilly Media, 2018.
- Others: online documents (Hadoop, Spark), tutorials, and selected academic papers.

Latest News

Jan. 15, 2020: [NOTE] The total score (for Part II) has been announced. The final score still needs to be merged by Prof. Chen-Shu Wang.
Dec. 12, 2019: [NOTE] HW#3 has been announced.
Dec. 4, 2019: [NOTE] If you haven't submitted your proposal for term project, please do it as soon as possible.
Nov. 28, 2019: [NOTE] HW#2 has been announced.
Nov. 20, 2019: [NOTE] From the TA's feedback, it would be difficult to have programming labs in class due to the lack of programming concept for students. Thus, I would like you to really find some open dataset, and analyze it using any tool you are familiar with, for example, Weka, as the term project.
Oct. 30, 2019: [NOTE] HW#1 has been announced.
Sep. 11, 2019: The homepage for the second part of Big Data Analytics (IMFI, Fall 2019) has been setup.

(Tentative) Schedule (for the second part)

Week	Date	Content	Reading	Note
1	Sep. 12, 2019	Course Overview
2	Sep. 19, 2019	(The first part)
3	Sep. 26, 2019	(The first part) (Leave for TANET 2019)
4	Oct. 3, 2019	(The first part) (Leave for ROCLING 2019)
5	Oct. 10, 2019	(The first part) (National Day)
6	Oct. 17, 2019	(The first part)
7	Oct. 24, 2019	Introduction to Big Data Analytics Ch.2, Getting to Know Your Data	DM3, Ch.1 DM3, Ch.2
8	Oct. 31, 2019	Ch.3, Data Preprocessing	DM3, Ch.3	HW#1
9	Nov. 7, 2019	Ch.6, Frequent Pattern Mining	DM3, Ch.6
10	Nov. 14, 2019	(Leave for NCS 2019) TA: Introduction to distributed platform Introduction to Big Data Analytics Platforms: Hadoop, Spark, TensorFlow (Ref: Notes on installation, configuration, and management of Hadoop & Spark clusters)		Due: HW#1
11	Nov. 21, 2019	(Leave for TAAI 2019) TA: Membership registration for Term Project, Introduction to Open Data, dataset aquisition TA: Examples in Spark programming Programming in Spark & Colaboratory (Lab: Python Programming in Google Colab)
12	Nov. 28, 2019	Ch.8, Classification: Basic Concepts	DM3, Ch.8	HW#2 Proposal
13	Dec. 5, 2019	(Midterm Exam)		Due: Term Project Proposal
14	Dec. 12, 2019	Ch.10, Cluster Analysis: Basic Concepts and Methods (Lab: Classification Example using Spark)	DM3, Ch.10	Due: HW#2 HW#3
15	Dec. 19, 2019	(Lab: Clustering Example using Spark) The MapReduce Programming Paradigm Spark Programming
16	Dec. 26, 2019	(Term Project Presentation)		Due: HW#3
17	Jan. 2, 2020	(Term Project Presentation)
18	Jan. 9, 2020	(Term Project Presentation)

Homework Assignments, Labs, and Term Project (for the second part)

During the progress of the course, there will be several homework assignments for written exercises, and also some hand-on labs in class.

Homeworks

There will be about 3 written assignments for topics such as pattern mining, classification, and clustering.

HW#1 :Ch.2-3
Due: Nov. 14, 2019
HW#2 :Ch.6
Due: Dec. 12, 2019
HW#3 :Ch.8
Due: Dec. 26, 2019

Please hand in your assignment before deadline according to the following instructions.

Submission Instructions

NOTE: Programs or projects in electronic files must be submitted directly to the website.

If you cannot successfully submit your work, please contact with the instructor.

Labs

Due to the background of students this semester, it will be difficult to give hands-on labs or programming exercises in different platforms such as Spark and Jupyter Notebook.
Reference:

Notes on installation, configuration, and management of Hadoop & Spark clusters

Term Project

Instead, you are required to complete a term project in which open datasets can be anayzed using open source tools.
[NOTE] If you haven't submitted your proposal, please do so as soon as possible since we are arranging the presentation schedule.

The current scuedule for term project presentation. (as of Dec. 26, 2019)
Since we have less teams this semester, each team can have more time for presentation. The presentations that cannot finish will be delayed until next week.

Exams

Midterm Exam: Nov. 5-9, 2018
- Date: Dec. 5 (Thu.) (TBD)
- Time: 9:10-12:00am
- Location: R312, General Studies Building
- Range: Ch. 1, 2, 3, 6, 8 (DM3)
Final Exam:
- (N/A)

Scores

Please check the homework submission site for more details.

Current Score (Part II) (including HW#1-HW#3, Midterm exam, and Term Project, as of Jan. 15, 2020)

Created: Sep. 11, 2019.
Last Updated: Jan. 15, 2020.