Dipartimento di InformaticaUniversità degli Studi di Milano
Dario Malchiodi

Algorithms for massive datasets2022/23

MSc in Computer science · University of Milan · Lectures are in English · 6 credits · 48 hours

Archived edition: this page is kept for reference, and some materials or links may no longer be available. Current edition (2025-26) →
This page refers to the course Algorithms for massive dataset within the Master in Computer science. The module Algorihms for massive data, within the course «Algorithms for massive data, cloud and distributed computing» of the Master in Data science for economics is described in a specific page.

The course aims at describing the big data processing framework, both in terms of methodologies and technologies.

Expected results

Students:

  • will be able to use technologies for the distributed storage of datasets;
  • will know the MapReduce distributed processing framework and its leading extensions;
  • will know the principal algorithms used in order to deal with classical big data problems, as well as to implement them using a distributed processing framework;
  • will be able to choose appropriate methods for solving big data problems.

News

  1. Office hours on June, 15th

    The office hours of June, 15th are canceled.

  2. Joint project for the «Algorithms for massive datasets» and «Statistical methods for machine learning» courses (Master in Computer Science)

    The description of a joint project for the courses «Algorithms for massive datasets» and «Statistical methods for machine learrning» for the Master in «Computer Science» is available.

  3. Office hours on June, 8th

    The office hours of June, 8th will take place remotely, using the address https://meet.jit.si/ricevimento-malchiodi.

Earlier news (3)

  1. Projects for the «Algorithms for massive datasets» course (Master in Computer Science)

    The description of the projects for the course «Algorithms for massive datasets» for the Master in «Computer Science» are available. A joint project with the «Statistical methods for machine learning» course will be published shortly.

  2. Classroom change for the lectures

    Starting March 28th, he lectures of the course «Algorithms for massive datasets» will take place in the classrooms V8 (on Tuesday) and 304 (on Wednesday), from 14:30 till 16:30.

  3. Office hours on March, 23rd

    The office hours of March, 23rd are canceled.

Course schedule

Lectures take place in presence, during the spring semester, at the "Città studi" educational sector. The schedule is as follows:

Day Hour Place
Tuesday 14:30 - 16:30 110 V8
Wednesday 14:30 - 16:30 110 304

Any change to the schedule will be announced in class and published in paragraph News of this page.

Office hours

By appointment, room 5015 of the Computer Science Department. It is possible contact the teacher by e-mail, taking care to read in advance the guide prepared by Prof. Sebastiano Vigna and clearly specifying in the message the course name and the academic year. In particular, students are encouraged to always use their academic address (i.e. based on the domain studenti.unimi.it) signing with name and student ID number and recalling that the response time may vary depending on the teacher commitments.

Course material

Lectures are based:

  • on the textbook Mining of Massive Datasets, written by A. Rajaraman and J. Ullman (marked by RU in the calendar of lectures), available as a free download in the authors' Web site and published in hardcopy by Cambridge University Press (ISBN:9781107015357);
  • on the notes and sample code published in the calendar of lectures.

Syllabus

The course explains the topics listed in the lecture calendar (available at the beginning of the course), covering the textbook contents as well as the contents of the remaining documents listed in Course material.

Prereqs

The course requires knowledge of the main topics of bachelor-level computer programming, calculus, probability, and statistics.

Lectures calendar

Lecture Date Topic Material
L01 28/02/2023 HDFS and map-reduce.
L02 01/03/2023 Map-reduce algorithms.
  • RU 2.3
L03 07/03/2023 Communication cost model.
  • RU 2.5
L04 08/03/2023 Spark. Link analysis.
L05 14/03/2023 Link analysis.
L06 15/03/2023 Link analysis. Finding similar items.
  • RU 5.4-5.5, 3.1-3.3.
L07 21/03/2023 Finding similar items.
  • RU 3.3-3.5.
L08 22/03/2023 Finding similar items.
L09 28/03/2023 Frequent itemsets.
  • RU 6.1-6.2
L10 29/03/2023 Frequent itemsets.
  • RU 6.3-6.4.4
L11 04/04/2023 Mining data streams.
  • RU 4.1-4.3.
L12 05/04/2023 Mining data streams. Clustering.
  • RU 4.4-4.5.
L13 18/04/2023 Clustering.
L14 19/04/2023 noSQL data base: mongodb.
L15 26/04/2023 noSQL data base: mongodb. Regression.
  • RU 7.3-7.5.
L16 02/05/2023 Regression.
L17 03/05/2023 Logistic regression. Recommendation systems.
L18 09/05/2023 Recommendation systems.
L19 10/05/2023 Dimensionality reduction.
L20 16/05/2023 Dimensionality reduction.
    L21 17/05/2023 Gradient boosting.
    L22 23/05/2023 Deep learning and TensorFlow.
    L23 24/05/2023 Workshop (RadicalBit, Artea).
      L24 30/05/2023 Workshop (Microsoft).

        Exam modalities

        The exam consists of a project and an oral test, both related to the topics covered in the course. The project requires to process one or more datasets through the critical application of the techniques described during the classes, and is described in a written report.

        The evaluation of the project, expressed with a pass/fail mark, considers the level of mastery of the topics and the clarity of the report. The oral test, which is accessed after a positive evaluation of the project, is based on the discussion of some topics covered in the course and on in-depth questions about the presented project. The evaluation of the oral test, expressed on a scale between 0 and 30, takes into account the level of mastery of the topics, clarity, and language skills.

        Students should sign up at the chosen examination session through UniMia, and send en email to prof. Malchiodi within the project deadline (see table below), containing the link to the project. Students will be contacted after the project has been checked. The table below shows a tentative date for the oral exams.

        Exam sessions

        Session Date
        June 16/06/2023 (project deadline: 12/06)
        July 07/07/2023 (project deadline: 03/07)
        September 15/09/2023 (project deadline: 11/09)
        January 22/01/2024 (project deadline: 17/01)
        February 06/02/2024 (project deadline: 31/01)
        February 21/02/2024 (project deadline: 14/02)