Dipartimento di InformaticaUniversità degli Studi di Milano
Dario Malchiodi

Big scale analytics2016/17

MSc in Computer science · University of Milan · Lectures are in Italian · 6 credits · 48 hours

Archived edition: this page is kept for reference, and some materials or links may no longer be available.

This course introduces the principal techniques related to the analysis of large amounts of data.

News

  1. Slides for the seminar on «The role of the data scientist»

    The slides of the seminar on «The role of the data scientist» by A. Condorelli are available.

  2. Office hours canceled

    Starting Friday 04/20, regular office hour are canceled; students can arrange an appointment via e-mail.

  3. Office hours for the spring semester

    Office hours for the spring semester will be during Thursday from 14:30 until 17:30 in the teacher's office, starting from 9/3.

Earlier news (7)

  1. Office hours canceled

    Starting Thursday 02/17, regular office hour are canceled until next semester; students can arrange an appointment via e-mail.

  2. Classroom change for the Big scale analytics lecture of 10/1/2017

    The Big scale analytics class of 10/1/2017 will take place in Aula Beta.

  3. Office hours rescheduled on 22/12

    Office hours of 22/12 will end at 16:00.

  4. Office hours of November 24th canceled

    The office hours of November 24nd are canceled; students can arrange an appointment via e-mail.

  5. Schedule change for the Big scale analytics lectures

    Starting from September 27th, the Big scale analytics classes will take place in Aula Delta.

  6. Office hours for the fall semester

    Office hours for the fall semester will be during Thursday from 14:00 until 17:00 in the teacher's office, starting from 29/09.

  7. Office relocation

    Starting today, Prof. Malchiodi has moved his office in room P102 at the first floor in the via Comelico seat of the department.

Course schedule

Lectures will take place at the Computer science department, according to the following tentative schedule:

Day Hour Place
Monday 15:30 - 17:30 aula Delta
Tuesday 15:30 - 17:30 aula Delta

Any change to the schedule will be announced in class and published in paragraph News of this page.

Office hours

By appointment, room 5015 of the Computer Science Department. It is possible contact the teacher by e-mail, taking care to read in advance the guide prepared by Prof. Sebastiano Vigna and clearly specifying in the message the course name and the academic year. In particular, students are encouraged to always use their academic address (i.e. based on the domain studenti.unimi.it) signing with name and student ID number and recalling that the response time may vary depending on the teacher commitments.

Course material

The theoric part of the course is based on the following textbook: Anand Rajaraman and Jeff Ullman, Mining of Massive Datasets, available both as a freely downloadable PDF and published in hardcopy by Cambridge University Press (ISBN:9781107015357). The suggested readings for the practical part are Holden Karau, Andy Konwinski, Patrick Wendell, Matei Zaharia, Learning Spark. Lightning-Fast Big Data Analysis, O'Reilly, 2015 (ISBN:978-1-449-35862-4) and Sandy Ryza, Uri Laserson, Sean Owen, Josh Wills, Advanced Analytics with Spark. Patterns for Learning from Data at Scale, O'Reilly, 2015 (ISBN:978-1-491-91276-8)

The part on distributed file systems and MapReduce is based on the adopted textbook and on the Hadoop tutorial published by Yahoo!

Some labs refer to the Data Science and Engineering with Spark edX program.

Syllabus

The course explains the topics listed in the lecture calendar, covering the textbook contents as well as the contents of the remaining documents listed in Course material.

Lectures calendar

Lecture Date Topic Material
L01 26/09/2016 HDFS and MapReduce
L02 27/09/2016 Map-reduce and Hadoop
L03 03/10/2016 Apache Spark and Databricks
    L04 04/10/2016 Link analysis
    L05 10/10/2016 Link analysis. Map-reduce complexity analysis
      L06 11/10/2016 Map-reduce complexity analysis. Finding similar items
        L07 17/10/2016 Finding similar items
          L08 18/10/2016 Finding similar items
          L09 24/10/2016 Mining data streams
          L10 25/10/2016 Frequent itemsets
            L11 07/11/2016 Frequent itemsets
              L12 08/11/2016 noSQL data base: mongodb
              L13 14/11/2016 Clustering
                L14 15/11/2016 Clustering
                  L15 21/11/2016 Clustering
                  L16 22/11/2016 Web advertising
                    L17 28/11/2016 Recommendation systems
                      L18 29/11/2016 Recommendation systems
                        L19 05/12/2016 Perceptrons
                          L20 06/12/2016 Regression
                            L21 12/12/2016 Dimensionality reduction
                              L22 13/12/2016 Dimensionality reduction. Amazon Web Services: S3, EC2, and EMR
                              L23 09/01/2017 Deep learning and TensorFlow (Dr. Ceruti)
                                L24 10/01/2017 Fast data and introduction to Apache Flink (Dr. Bentivoglio)

                                  Exam modalities

                                  The exam consists in an oral test, by appointment.