Dipartimento di InformaticaUniversità degli Studi di Milano
Dario Malchiodi

Algorithms for massive datasets2020/21

MSc in Computer science · University of Milan · Lectures are in English · 6 credits · 48 hours

Archived edition: this page is kept for reference, and some materials or links may no longer be available. Current edition (2026-27) →
This page refers to the course Algorithms for massive dataset within the Master in Computer science. The module Algorihms for massive data, within the course «Algorithms for massive data, cloud and distributed computing» of the Master in Data science for economics is described in a specific page.

The course aims at describing the big data processing framework, both in terms of methodologies and technologies.

Expected results

Students:

  • will be able to use technologies for the distributed storage of datasets;
  • will know the MapReduce distributed processing framework and its leading extensions;
  • will know the principal algorithms used in order to deal with classical big data problems, as well as to implement them using a distributed processing framework;
  • will be able to choose appropriate methods for solving big data problems.

News

  1. Meeting with Radicalbit

    On Friday 4/6 a meeting is planned for students of the course of «Algorithms for massive datasets». In this meeting the Radicalbit company will present some opportunities for external theses. The meeting will take place remotely, using the same Zoom link of the lectures.

  2. Office hours change

    Starting Monday 31/05, office hour will be handled via e-mail.

  3. Availablity of recorded lectures of «Statistics and data analysis» (B.Sc. in Computer science) and of «Algorithms for massive datasets» (Master in Computer Science)

    The videorecorded lectures of the courses «Statistics and data analysis» (B.Sc. in Computer science) and of «Algorithms for massive datasets» (Master in Computer Science) will be available until June 19th.

Earlier news (7)

  1. Joint project for the «Algorithms for massive datasets» and «Statistical methods for machine learning» courses (Master in Computer Science)

    The description of a joint project for the courses «Algorithms for massive datasets» and «Statistical methods for machine learrning» for the Master in «Computer Science» is available.

  2. Projects for the «Algorithms for massive datasets» course (Master in Computer Science)

    The description of the projects for the course «Algorithms for massive datasets» for the Master in «Computer Science» are available. A joint project with the «Statistical methods for machine learning» course will be published shortly.

  3. Office hours on April, 30th

    The office hours of April, 30th are canceled.

  4. Office hours on April, 2nd

    The office hours of April, 2nd are canceled.

  5. Access to recorded lectures

    Access to the recorded lectures is allowed only to students who have signed up in the course moodle page. Should this not be possible, students are asked to contact the teacher.

  6. Office hours change for the course «Algorithms for massive datasets»

    The 11/03 office hours of the course «Algorithms for massive datasets» will start at 18:30. From next week office hours will take place every Friday at 16:30.

  7. Semester for the Algorithms for massive datasets course

    The class of Algorithms for massive datasets will be delivered in the spring semester.

Course schedule

Lectures take place (until further notice) online through authentication to a zoom link published on the moodle page of the course. The provisional schedule is as follows:

Day Hour Place
Monday 16:30 - 18:30 zoom
Wednesday 14:30 - 16:30 zoom

Any change to the schedule will be announced in class and published in paragraph News of this page. The recording of lectures, marked with (R) in the schedule, is available until the end of the course, through access to the corresponding moodle page.

Office hours

By appointment, room 5015 of the Computer Science Department. It is possible contact the teacher by e-mail, taking care to read in advance the guide prepared by Prof. Sebastiano Vigna and clearly specifying in the message the course name and the academic year. In particular, students are encouraged to always use their academic address (i.e. based on the domain studenti.unimi.it) signing with name and student ID number and recalling that the response time may vary depending on the teacher commitments.

Specific, group-based office hours for the course will be organized on a weekly basis, each thursday at 17:30 using the same zoom link of lectures.

Course material

Lectures are based:

  • on the textbook Mining of Massive Datasets, written by A. Rajaraman and J. Ullman (marked by RU in the calendar of lectures), available as a free download in the authors' Web site and published in hardcopy by Cambridge University Press (ISBN:9781107015357);
  • on the notes and sample code published in the calendar of lectures.

Syllabus

The course explains the topics listed in the lecture calendar (available at the beginning of the course), covering the textbook contents as well as the contents of the remaining documents listed in Course material.

Prereqs

The course requires knowledge of the main topics of bachelor-level computer programming, calculus, probability, and statistics.

Lectures calendar

Lecture Date Topic Material
L01 01/03/2021 Mathematical preliminaries.
L02 03/03/2021 Technical preliminaries.
L03 08/03/2021 Link analysis.
L04 10/03/2021 Link analysis.
L05 15/03/2021 HDFS and map-reduce.
L06 17/03/2021 Map-reduce algorithms. Spark.
L07 22/03/2021 Communication cost model.
L08 24/03/2021 Finding similar items.
L09 29/03/2021 Finding similar items.
L10 31/03/2021 Frequent itemsets.
L11 12/04/2021 Frequent itemsets.
L12 14/04/2021 noSQL data base: mongodb.
L13 19/04/2021 Clustering.
L14 21/04/2021 Clustering.
L15 26/04/2021 Recommendation systems.
L16 28/04/2021 Mining data streams.
L17 03/05/2021 Mining data streams.
L18 05/05/2021 Regression.
L19 10/05/2021 Regression and logistic regression.
L20 12/05/2021 Deep learning and TensorFlow.
L21 17/05/2021 Deep learning and TensorFlow.
L22 19/05/2021 Recurrent neural networks.
L23 24/05/2021 Dimensionality reduction.
L24 26/05/2021 Gradient boosting.

Exam modalities

The exam consists of a project and an oral test, both related to the topics covered in the course. The project requires to process one or more datasets through the critical application of the techniques described during the classes, and is described in a written report. The evaluation of the project, expressed with a pass/fail mark, considers the level of mastery of the topics and the clarity of the report. The oral test, which is accessed after a positive evaluation of the project, is based on the discussion of some topics covered in the course and on in-depth questions about the presented project. The evaluation of the oral test, expressed on a scale between 0 and 30, takes into account the level of mastery of the topics, clarity, and language skills.

Exam sessions

Session Date
June 17/06/2021
July 02/07/2021
September 08/09/2021
September 22/09/2021
January N/A
February N/A