Publications

International Conferences

[C1] Marco Mussi*, Simone Drago*, Marcello Restelli and Alberto Maria Metelli. Factored-Reward Bandits with Intermediate Observations. Proceedings of the 41st International Conference on Machine Learning (ICML), 2024. (A* Core Ranking - 2609/9473, Acceptance rate 27.5%)
[Link] [Paper] [Poster] [Slides]

[C2] Marco Mussi, Alessandro Montenegro, Francesco Trovò, Marcello Restelli and Alberto Maria Metelli. Best Arm Identification for Stochastic Rising Bandits. Proceedings of the 41st International Conference on Machine Learning (ICML), 2024. (A* Core Ranking - Spotlight - 335/9473, top 3.5%)
[Link] [Paper] [arXiv] [Poster]

[C3] Alessandro Montenegro, Marco Mussi, Alberto Maria Metelli and Matteo Papini. Learning Optimal Deterministic Policies with Stochastic Policy Gradients. Proceedings of the 41st International Conference on Machine Learning (ICML), 2024. (A* Core Ranking - Spotlight - 335/9473, top 3.5%)
[Link] [Paper] [arXiv] [Poster]

[C4] Gianmarco Genalti, Marco Mussi, Nicola Gatti, Marcello Restelli, Matteo Castiglioni and Alberto Maria Metelli. Graph-Triggered Rising Bandits. Proceedings of the 41st International Conference on Machine Learning (ICML), 2024. (A* Core Ranking - 2609/9473, Acceptance rate 27.5%)
[Link] [Paper] [Poster]

[C5] Francesco Bacchiocchi*, Gianmarco Genalti*, Davide Maran*, Marco Mussi*, Marcello Restelli, Nicola Gatti and Alberto Maria Metelli. Autoregressive Bandits. Proceedings of the 27th International Conference on Artificial Intelligence and Statistics (AISTATS), 2024. (A Core Ranking - 546/1980, Acceptance rate 27.6%)
[Link] [Paper] [arXiv] [Poster] [Slides]

[C6] Marco Mussi, Alberto Maria Metelli and Marcello Restelli. Dynamical Linear Bandits. Proceedings of the 40th International Conference on Machine Learning (ICML), 2023. (A* Core Ranking - 1827/6538, Acceptance rate 27.9%)
[Link] [Paper] [arXiv] [Poster] [Slides]

[C7] Marco Mussi*, Gianmarco Genalti*, Alessandro Nuara, Francesco Trovò, Nicola Gatti and Marcello Restelli. Dynamic Pricing with Volume Discounts in Online Settings. Proceedings of the Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence (IAAI), 2023. AAAI. Innovative Application of AI Award.
[Link] [Paper] [arXiv] [Poster] [Slides] [Award]

[C8] Marco Mussi, Gianmarco Genalti, Francesco Trovò, Alessandro Nuara, Nicola Gatti and Marcello Restelli. Pricing the Long Tail by Explainable Product Aggregation and Monotonic Bandits. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022. (A* Core Ranking - Oral Presentation - 54/753, top 7%)
[Link] [Paper] [Poster] [Slides]

Journals

[J1] Marco Mussi, Luigi Pellegrino, Oscar Francesco Pindaro, Marcello Restelli and Francesco Trovò. A Reinforcement Learning Controller Optimizing Costs and Battery State of Health in Smart Grids. Journal of Energy Storage, 82, 2024. (Q1 Scimago)
[Link] [Paper]

[J2] Marco Mussi, Davide Lombarda, Alberto Maria Metelli, Francesco Trovò and Marcello Restelli. ARLO: A Framework for Automated Reinforcement Learning. Expert Systems with Applications, 224, 2023. (Q1 Scimago)
[Link] [Paper] [arXiv]

[J3] Marco Mussi, Luigi Pellegrino, Marcello Restelli and Francesco Trovò. An Online State of Health Estimation Method for Lithium-Ion Batteries based on Time Partitioning and Data-Driven Model Identification. Journal of Energy Storage, 55, 2022. (Q1 Scimago)
[Link] [Paper]

[J4] Marco Mussi, Luigi Pellegrino, Marcello Restelli and Francesco Trovò. A voltage dynamic-based state of charge estimation method for batteries storage systems. Journal of Energy Storage, 44, 2021. (Q1 Scimago)
[Link] [Paper]

Workshops

[W1] Gianvito Losapio, Davide Beretta, Marco Mussi, Alberto Maria Metelli and Marcello Restelli. State and Action Factorization in Power Grids. Workshop on Machine Learning for Sustainable Power Systems at the European Conference on Machine Learning (ECML).
[Link - To Appear] [Paper - To Appear] [Poster - To Appear] [Slides - To Appear]

[W2] Simone Drago and Marco Mussi. Open Problem: Tight Bounds for Bernoulli Rewards in Kernelized Multi-Armed Bandits. Workshop on Aligning Reinforcement Learning Experimentalists and Theorists at the International Conference on Machine Learning (ICML). 2024.
[Link - To Appear] [Paper] [Poster]

[W3] Simone Drago, Marco Mussi, Marcello Restelli and Alberto Maria Metelli. Intermediate Observations in Factored-Reward Bandits. Adaptive and Learning Agents Workshop at the International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS). 2024.
[Link] [Paper] [Slides]

[W4] Francesco Bacchiocchi*, Gianmarco Genalti*, Davide Maran*, Marco Mussi*, Marcello Restelli, Nicola Gatti and Alberto Maria Metelli. Online Learning in Autoregressive Dynamics. European Workshop on Reinforcement Learning (EWRL). 2023.
[Link] [Paper] [Poster]

[W5] Alessandro Montenegro, Marco Mussi, Francesco Trovò, Marcello Restelli and Alberto Maria Metelli. Stochastic Rising Bandits: A Best Arm Identification Approach. European Workshop on Reinforcement Learning (EWRL). 2023.
[Link] [Paper] [Poster]

[W6] Alessandro Montenegro, Marco Mussi, Francesco Trovò, Marcello Restelli and Alberto Maria Metelli. A Best Arm Identification Approach for Stochastic Rising Bandits. Workshop on New Frontiers in Learning, Control, and Dynamical Systems at International Conference on Machine Learning (ICML). 2023.
[Link] [Paper] [Poster]

[W7] Gianmarco Genalti, Marco Mussi, Alessandro Nuara and Nicola Gatti. Dynamic Pricing with Online Data Aggregation and Learning. European Workshop on Reinforcement Learning (EWRL). 2022. (Oral Presentation - 10/96)
[Link] [Paper] [Poster] [Slides]

[W8] Marco Mussi, Alberto Maria Metelli and Marcello Restelli. Dynamical Linear Bandits for Long-Lasting Vanishing Rewards. Complex Feedback in Online Learning Workshop at International Conference on Machine Learning (ICML). 2022.
[Link] [Paper] [Poster]

Under Review

[R1] Marco Mussi and Alberto Maria Metelli. Generalizing the Regret: an Analysis of Lower and Upper Bounds. 2024. Submitted to the Journal of Artificial Intelligence Research (JAIR).

[R2] Marco Mussi*, Simone Drago*, Marcello Restelli and Alberto Maria Metelli. Factored-Reward Bandits with Intermediate Observations: Regret Minimization and Best Arm Identification. Submitted to the Artificial Intelligence Journal (AIJ).

[R3] Marco Mussi*, Andrea Cerasani*, Alessandro Lavelli and Marcello Restelli. Online Dynamic Pricing of Complementary Goods. Submitted to the Journal of Engineering Applications of Artificial Intelligence (EAAI).

[R4] Simone Drago*, Marco Mussi* and Alberto Maria Metelli. Sleeping Reinforcement Learning. 2024. Submitted to Neural Information Processing Systems (NeurIPS).

[R5] Federico Corso, Riccardo Zamboni, Marco Mussi, Marcello Restelli and Alberto Maria Metelli. No-regret Learning with Revealed Transitions in Adversarial Markov Decision Processes. 2024. Submitted to Neural Information Processing Systems (NeurIPS).

[R6] Alessandro Montenegro, Marco Mussi, Matteo Papini and Alberto Maria Metelli. Last-Iterate Global Convergence of Policy Gradients for Constrained Reinforcement Learning. 2024. Submitted to Neural Information Processing Systems (NeurIPS).

[R7] Gianmarco Genalti, Marco Mussi, Nicola Gatti, Marcello Restelli, Matteo Castiglioni and Alberto Maria Metelli. Bridging Rested and Restless Bandits with Graph-Triggering: Rotting and Rising. Submitted to the Journal of Machine Learning Research (JMLR).

In Preparation

[P1] Marco Mussi, Simone Drago and Alberto Maria Metelli. Open Problem: Tight Bounds for Kernelized Multi-Armed Bandits with Bernoulli Rewards.
[Paper] [arXiv]

[P2] Simone Drago, Marco Mussi and Alberto Maria Metelli. Notes on Minimax Regret Bounds for Reinforcement Learning.

[P3] Alberto Maria Metelli, Valentina Abbattista and Marco Mussi. Online Learning for PID Controllers.

[P4] AI4REALNET Consortium. AI for the Operation of Critical Infrastructures. Position/Consortium Paper.

[P5] Alessandro Montenegro, Leonardo Cesani, Marco Mussi, Matteo Papini and Alberto Maria Metelli. Deterministic Policy Gradients with Constraints.

[P6] Alessandro Montenegro, Marco Mussi, Matteo Papini and Alberto Maria Metelli. Deterministic Policy Gradients with Adaptive Variance.

[P7] Alessandro Montenegro, Federico Mansutti, Marco Mussi, Matteo Papini and Alberto Maria Metelli. Sample Reuse for Policy Gradients.

Education

Ph.D. in Information Technology - Politecnico di Milano (Nov 2020 - Jun 2024)
Focus on Reinforcement Learning and Online Learning.
Supervisor: Prof. Marcello Restelli
Co-supervisor: Prof. Alberto Maria Metelli
Tutor: Prof. Nicola Gatti
[Link] [Thesis] [Slides]

M.Sc. in Computer Science and Engineering - Politecnico di Milano (Sep 2017 - Dec 2019)
Main focus: Artificial Intelligence and Machine Learning
Scholarship: Tuition waiver for high academic performance
Relevant coursework: Machine Learning, Artificial Intelligence, Game Theory, Autonomous Agents and Multi- agent Systems, Foundations of Operational Research, Software Engineering, Principles of Programming Languages, Data Bases II

B.Sc. in Engineering of Computing Systems - Politecnico di Milano (Sep 2014 - Jul 2017)
Relevant coursework: Software Engineering, Theoretical Computer Science, Communication Networks and Internet, Information Systems, Data Bases I, Computer Architecture and Operating Systems, Automatic Control, Calculus I, Calculus II, Linear Algebra and Geometry, Logics and Algebra, Statistics and Probability, Physics, Applied Physics

High School Diploma in Computer Science - IIS Galileo Galilei Crema (Sep 2008 - Jul 2014)
Main Focus: C, Java, HTML, CSS, Javascript

Experience

Postdoctoral Researcher - Politecnico di Milano (Jun 2024 - now)
Supervisor: Prof. Marcello Restelli

Research Scientist - ML cube (Nov 2020 - now)
Goal: develop algorithms for dynamic pricing and advertising optimization

Research Assistant - Politecnico di Milano (Jan 2020 - Oct 2020)
Supervisor: Prof. Marcello Restelli

Industrial Projects

AD cube Marketing Mix Model - ML cube (Nov 2022 - now)
Focus: Budget optimization in advertising, considering advertising campaigns interactions

Data-driven Optimization Marketing Mix Models for Advertising - WebRanking (Feb 2022 - Aug 2022)
Focus: Implementation of a MMM to solve the attribution problem in digital advertising in contexts with scarce and noisy data

Dynamic Pricing for E-commerce - Euroffice (Feb 2021 - May 2022)
Focus: Implementation of a dynamic pricing model for an e-commerce with over 20000 products

AD cube Product Release - ML cube (Nov 2020 - Feb 2022)
Focus: Release of AD Cube, a product for advertising optimization in online campaigns

Last-mile Delivery Optimization - PaxMile (May 2020 - Oct 2020)
Focus: Delivery allocation using Reinforcement Learning and bikers load estimation using Supervised Learning techniques

Reinforcement Learning in Smart-grids - Ricerca Sistema Energetico (Feb 2020 - Oct 2022)
Focus: Exploit Reinforcement Learning solutions to preserve the battery State of Health in smart-grids, optimizing economic variables

European Projects

AI4REALNET - Fundamental Research Work Package (Oct 2023 - now)
Focus: The scope of AI4REALNET covers the perspective of AI-based solutions addressing critical systems (electricity, railway, and air traffic management) modeled by networks that can be simulated, and are traditionally operated by humans, and where AI systems complement and augment human abilities.

Academic Activities

Teaching

Exercise Sessions Lecturer - Politecnico di Milano
Lecturer for the Exercise Sessions of the Course of Foundations of Computer Science
26 Hours (February 2024 - June 2024)
Exercise sessions mainly on C and Fortran programming languages.
Course delivered in English.

Tutor - Politecnico di Milano and Cefriel
Tutor for a Master in AI/ML
30 Hours (September 2022 - July 2023)
Supervision of a team in the application of Reinforcement Learning algorithms to real-world control problems.

Organization of International Events

European Workshop on Reinforcement Learning
Organizing Committee - Communication Chair (19/21 September 2022)

Seminars

Distributed and Hierarchical Reinforcement Learning
AI4REALNET Dissemination Webinar - AI4REALNET Consortium (24 April 2024)

An introduction to Reinforcement Learning in Real World
DEIB Seminar - Politecnico di Milano (3 September 2021)

Un metodo data-driven per la stima dello stato di carica di batterie a ioni di litio
RSE Academy Seminar - Ricerca Sistema Energetico (23 October 2020)

Participation to International Conferences and Workshops

International Conference on Machine Learning - ICML 2024
Vienna, Austria. July 2024.

International Conference on Artificial Intelligence and Statistics - AISTATS 2024
Valencia, Spain. May 2024.

European Workshop on Reinforcement Learning - EWRL 2023
Brussels, Belgium. September 2023.

International Conference on Machine Learning - ICML 2023
Honolulu, Hawaii, USA. July 2023.

European Workshop on Reinforcement Learning - EWRL 2022
Milan, Italy. September 2022.

ACM International Conference on Knowledge Discovery and Data Mining - KDD 2022
Washington D.C., USA. August 2022.

International Conference on Machine Learning - ICML 2022
Baltimore, Maryland, USA. July 2022.

Participation to Summer Schools

Reinforcement Learning Summer School - RLSS 2023
Barcellona, Spain. June 2023.

DeepLearn Summer School - DeepLearn 2021
Virtual. July 2021.

Master's Students Supervision

Gianmarco Genalti - "A Multi-Armed Bandit Approach to Dynamic Pricing". Co-supervision. Supervisor: Prof. Nicola Gatti (M.Sc. in Mathematical Engineering, Dec 2021)

Amedeo Cavallo - "A Combinatorial Multi-Armed Bandit Approach to Online Advertising Budget Optimisation". Co-supervision. Supervisor: Prof. Marcello Restelli (M.Sc. in Computer Science and Engineering, Dec 2021)

Oscar Francesco Pindaro - "Controlling Lithium-Ion Batteries Through Reinforcement Learning". Co-supervision. Supervisor: Prof. Marcello Restelli (M.Sc. in Computer Science and Engineering, Apr 2022)

Davide Lombarda - "Towards Automated Reinforcement Learning". Co-supervision. Supervisor: Prof. Marcello Restelli (M.Sc. in Mathematical Engineering, Apr 2022)

Thomas Petrone - "Hidden Markov Model for Single User Response Prediction in Digital Advertising Campaigns". Co-supervision. Supervisor: Prof. Marcello Restelli (M.Sc. in Mathematical Engineering, Jul 2022)

Alessandro Montenegro - "Best Model Selection via Stochastic Rising Bandits". Co-supervision. Supervisor: Prof. Alberto Maria Metelli (M.Sc. in Computer Science and Engineering, May 2023)

Andrea d'Silva - "Integrating Behavioral Cloning into a Reinforcement Learning pipeline". Co-supervision. Supervisor: Prof. Francesco Trovò (M.Sc. in Computer Science and Engineering, May 2023)

Francesco Gonzales - "Stochastic Linear Bandit with Global-Local Structure". Co-supervision. Supervisor: Prof. Francesco Trovò (M.Sc. in Computer Science and Engineering, May 2023)

Vittorio Arianna - "Multi-Armed Bandits for Joint Pricing and Advertising". Co-supervision. Supervisor: Prof. Nicola Gatti (M.Sc. in Computer Science and Engineering, Oct 2023)

Marco Bonalumi - "An Online Learning Algorithm for Real-time Bidding". Co-supervision. Supervisor: Prof. Marcello Restelli (M.Sc. in Computer Science and Engineering, Dec 2023)

Alessandro Contù - "Budget Optimization in Marketing Mix Models". Co-supervision. Supervisor: Prof. Marcello Restelli (M.Sc. in Computer Science and Engineering, Dec 2023)

Andrea Cerasani - "An Online Dynamic Pricing Algorithm for Complementary Products". Co-supervision. Supervisor: Prof. Marcello Restelli (M.Sc. in Computer Science and Engineering, Dec 2023)

Federico Corso - "Smoothed OMD: an Algorithm for No-regret Learning in Adversarial MDPs with Revealed Transitions". Co-supervision. Supervisor: Prof. Alberto Maria Metelli (M.Sc. in Automation and Control Engineering, Jul 2024)

Valentina Abbattista. Co-supervision. (M.Sc. in Computer Science and Engineering, in progress)

Federico Mansutti. Co-supervision. (M.Sc. in Computer Science and Engineering, in progress)

Davide Beretta. Co-supervision. (M.Sc. in Computer Science and Engineering, in progress)

Fabio Patella. Co-supervision. (M.Sc. in Computer Science and Engineering, in progress)

Giacomo Cartechini. Co-supervision. (M.Sc. in Computer Science and Engineering, in progress)

Leonardo Cesani. Co-supervision. (M.Sc. in Computer Science and Engineering, in progress)

Review

Reviewer for International Conferences:
Neural Information Processing Systems (NeurIPS)
International Conference on Machine Learning (ICML)
International Conference on Learning Representations (ICLR)
International Conference on Artificial Intelligence and Statistics (AISTATS)
AAAI Conference on Artificial Intelligence (AAAI)
International Conference on Automated Machine Learning (AutoML)

Reviewer for International Journals:
Springer - Machine Learning (Q1)
IEEE - Transactions on Neural Networks and Learning Systems (Q1)
IEEE - Robotics and Automation Letters (Q1)
Elsevier - Engineering Applications of Artificial Intelligence (Q1)

Reviewer for International Workshops:
European Workshop on Reinforcement Learning (EWRL)
AutoRL @ ICML 2024
ARLET @ ICML 2024

Contacts

Email
marco DOT mussi AT polimi DOT it

Office

Office 19, First Floor of Building 21
Dipartimento di Elettronica, Informazione e Bioingegneria
Politecnico di Milano
Via Ponzio 34/5, Milan, 20133, Italy