Michael Kaisers, PhD

Michael Kaisers graduated from Maastricht University with a BSc in Knowledge Engineering in 2007 and a MSc in Artificial Intelligence in 2008. He earned the honor summa cum laude in both cases, while abbreviating the three-years bachelor's program to two years and complementing his master's program by an extra-curricular four-month research visit to Prof. dr. Simon Parsons at Brooklyn College, City University of New York.

In a nationwide competition, the Netherlands Organisation for Scientific Research (NWO) awarded him a TopTalent 2008 grant for his PhD research. In September 2008, he commenced his PhD position at Eindhoven University of Technology. From August 2009, the project continued at Maastricht University. He intensified his international research experience through a three-month research visit to Prof. dr. Michael Littman at Rutgers, State University of New Jersey, and published at various peer-reviewed workshops, conferences and journals.

From September 2010 to January 2012 he has chaired PhD Academy, which brings Maastricht PhD candidates together for social, educative and fun activities. He has also chaired the 4th Maastricht PhD conference (PhDC 2011) and the local organizing committee of the 9th European Workshop on Multi-agent Systems (EUMAS 2011). His dissertation coherently summarizes his PhD research and has been defended in December 2012.

The public thesis defense took place on Monday, 17 December 2012 at 4pm.

Michael remained affiliated with Maastricht University as a visiting postdoctoral research fellow while he filled the position of Director of Competence Center Agent Core Technologies in the Distributed Artificial Intelligence Laboratory at the Technical University Berlin from March to September 2013. Since November 2013 he is pursuing his research at the Centrum Wiskunde & Informatica in Amsterdam within the Intelligent and Autonomous Systems group, where he built a team working on intelligent joint decision making with applications to energy systems.

Following a successful tenure track evaluation, and passing the milestone of 1 million Euro total research funding acquired, he is a permanent staff member at CWI since 1 November 2017.

Keywords

Artificial intelligence, multi-agent reinforcement learning, evolutionary game theory, dynamical systems.

You can find partial lists of my publications on google scholar or DBLP. The list below is sporadically updated manually.

List of Conference and Workshop Publications

2021

Hendrik Baier and Michael Kaisers. ME-MCTS: Online Generalization by Combining Multiple Value Estimators. In Proc. of 30th Int. Joint Conf. on Artificial Intelligence (IJCAI), 2021.

2020

Hendrik Baier and Michael Kaisers. Guiding Multiplayer MCTS by Focusing on Yourself. In IEEE Conference on Games (CoG 2020), volume 2020-Augus, pages 550–557, 2020.

Hendrik Baier and Michael Kaisers. Opponent-Pruning Paranoid Search. In Proc. of Founda- tions of Digital Games (FDG 2020), 2020.

2019

Shantanu Chakraborty, Pablo Hernandez-Leal, and Michael Kaisers. An Exchange Mechanism to Coordinate Flexibility in Residential Energy Cooperatives. In Proc. of 20th IEEE Int. Conf. on Industrial Technology (ICIT), Melbourne, 2019.

Richard Klima, Daan Bloembergen, Michael Kaisers, and Karl Tuyls. Robust temporal differ- ence learning for critical domains. In Proc. of the 18th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS), Montreal, Canada, May 13-17, 2019. IFAAMAS.

Haralambie Leahu, Tim Baarslag, and Michael Kaisers. Automated Negotiation with Gaussian Process-based Utility Models. In Proc. of Twenty-Eighth Int. Joint Conf. on Artificial Intelligence (IJCAI), Macao, China, August 10-16, 2019. International Joint Conferences on Artificial Intelligence Organization.

Haralambie Leahu, Tim Baarslag, and Michael Kaisers. Preference Learning in Automated Negotiation Using Gaussian Uncertainty Models (Extended Abstract). In Proc. of the 18th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS), Montreal, Canada, May 13-17, 2019. IFAAMAS.

Georgios Methenitis, Michael Kaisers, and Han La Poutr ́e. Forecast – Based Mechanisms for Demand Response. In Proc. of the 18th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS), Montreal, Canada, May 13-17, 2019. IFAAMAS.

2018

Shantanu Chakraborty, Tim Baarslag, and Michael Kaisers. Energy Contract Settlements through Automated Negotiation in Residential Cooperatives. In Proc. of IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (Smart- GridComm), 2018.

Shantanu Chakraborty, Pablo Hernandez-Leal, and Michael Kaisers. Coordinating Distributed and Flexible Resources : A Case-study of Residential Cooperatives. In Proc. of Innovative Smart Grid Technologies (ISGT Europe), 2018.

Arzam Muzaffar Kotriwala, Pablo Hernandez-Leal, and Michael Kaisers. Load Classification and Forecasting for Temporary Power Installations. In Proc. of Innovative Smart Grid Technolo- gies (ISGT Europe), 2018.

Iliana Pappi, Tim Baarslag, Michael Kaisers, and Nikolaos G Paterakis. An Uncertainty-Aware Online Planning Tool for the Sustainable Electrification of Festivals. In Int. Conf. on Smart Energy Systems and Technologies (SEST). IEEE, 2018.

Dimitrios Tsimpoukis, Tim Baarslag, Michael Kaisers, and Nikolaos G. Paterakis. Automated Negotiations under User Preference Uncertainty: A Linear Programming Approach. In Proc. of Agreement Technologies (AT), 2018.

2017

Tim Baarslag and Michael Kaisers. The Value of Information in Automated Negotiation: A Decision Model for Eliciting User Preferences. In Proc. of 16th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2017). International Foundation for AAMAS, 2017.

Tim Baarslag, Michael Kaisers, Catholijn Jonker, Jonathan Gratch, and Enrico Gerding. When will negotiation agents be able to represent us? The challenges and opportunities for autonomous negotiators. In Proc. of the 26th Int. Joint Conf. on Artificial Intelligence (IJCAI), 2017. [Download]

Pablo Hernandez-Leal and Michael Kaisers. Learning against sequential opponents in repeated stochastic games. In Proc. of the 3rd Multidisciplinary Conference on Reinforcement Learning and Decision Making (RLDM), 2017.

Georgios Methenitis, Michael Kaisers and Han La Poutré. SLA–Mechanisms for Electricity Trading under Volatile Supply and Varying Criticality of Demand (Extended Abstract). In Proc. of 16th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2017). International Foundation for AAMAS, 2017.

Tim Baarslag, Michael Kaisers, Enrico H Gerding, Catholijn M Jonker, and Jonathan Gratch. When will negotiation agents be able to represent us? The challenges and opportunities for autonomous negotiators. In Proc. of the Int. Workshop on Agent-Mediated Electronic Commerce and Trading Agents Design and Analysis (AMEC/TADA), 2017.

Pablo Hernandez-Leal and Michael Kaisers. Towards a fast detection of opponents in repeated stochastic games. In Proc. of the 1st Workshop on Transfer in Reinforcement Learning (TiRL), 2017.

Georgios Methenitis, Michael Kaisers, and Han La Poutre. SLA Mechanisms for Electricity Trading under Volatile Supply and Varying Criticality of Demand. In Proc. of the Int. Workshop on Agent- Mediated Electronic Commerce and Trading Agents Design and Analysis (AMEC/TADA), 2017.

2016

Felix Claessen, Michael Kaisers and Han La Poutré. Efficient balancing by effort-based activation of demand response services. In Innovative Smart Grid Technologies-Europe (ISGT Europe), IEEE, 2016. [Download]

Aliene van der Veen and Michael Kaisers. Computing the value of flexibility in electricity retail, ahead and balancing markets. In Innovative Smart Grid Technologies-Europe (ISGT Europe), IEEE, 2016.

Christian Gitte, Huiwen Xu, Fabian Rigoll, Joeri van Eekelen, and Michael Kaisers. Multi-Commodity Energy Management Applied to Micro CHPs and Electrical Heaters in Smart Buildings. In 5th D-A-CH+ Energy Informatics Conference in conjunction with 7th Symposium on Communications for Energy Systems (ComForEn), pages 74-80. Vol. 84, Österreichischer Verband für Elektrotechnik. [Link]

Georgios Methenitis, Michael Kaisers and Han La Poutré. Incentivizing Intelligent Customer Behavior in Smart-Grids: A Risk-Sharing Tariff & Optimal Strategies. In Proceedings of the 25th International Conference on Artificial Intelligence (IJCAI), 2016. AAAI Press. [Link]

Timon V. Kanters, Frans A. Oliehoek, Michael Kaisers, Stan R. van den Bosch, Joep Grispen, and Jeroen Hermans. Energy- and Cost-Efficient Pumping Station Control. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 2016. [Link]

2015

Georgios Methenitis, Michael Kaisers and Han La Poutré. A Multi-Scale Energy Demand Model suggests sharing Market Risks with Intelligent Energy Cooperatives. In Innovative Smart Grid Technologies-Asia (ISGT Asia), IEEE, 2015. [Link]

Felix Claessen, Bart Liefers, Michael Kaisers and Han La Poutré. The value of online information for demand response in Walrasian electricity markets. In Innovative Smart Grid Technologies-Asia (ISGT Asia), IEEE, 2015.

Sherief Abdallah and Michael Kaisers. Improving Multi-Agent Learners Using Less-biased Value Estimators. In Proc. of Int. Conf. on Web Intelligence and Intelligent Agent Technology (WI-IAT 2015). IEEE/WIC/ACM, 2015.

Tobias Linnenberg, Michael Kaisers and Alexander Fay. Open Energy Exchange - A new way of social networking. In Proc. of SmartER Europe 15, Essen, February 2015.

Tobias Linnenberg, Alexander Fay and Michael Kaisers. Bottom-up Demand Response by Following Local Energy Generation Voluntarily (Demonstration). In Proc. of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI 2015), pages 4282-4283, 2015.

2013

Sherief Abdallah and Michael Kaisers. Addressing the Policy-bias of Q-learning by Repeating Updates. In Proc. of 12th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2013), pages 1045-1052. International Foundation for AAMAS, 2013. [Download]

2012

Daniel Hennes, Daan Bloembergen, Michael Kaisers, Karl Tuyls, and Simon Parsons. Evolutionary advantage of foresight in markets. In Proc. of the Genetic and Evolutionary Computation Conference (GECCO 2012), pages 943-949. Sheridan Printing, 2012. [Download]

Michael Wunder, John R. Yaros, Michael Littman, and Michael Kaisers. A Framework for Modeling Population Strategies by Depth of Reasoning. In Conitzer, Winikoff, Padgham, and van der Hoek, editors, Proc. of 11th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2012), pages 1393-1394. International Foundation for AAMAS, 2012. [Download]

Michael Kaisers, Daan Bloembergen, and Karl Tuyls. A Common Gradient in Multi-agent Reinforcement Learning (Extended Abstract). In Conitzer, Winikoff, Padgham, and van der Hoek, editors, Proc. of 11th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2012), pages 947-954. International Foundation for AAMAS, 2012. [Download]

2011

Sjriek Alers, Daan Bloembergen, Daniel Hennes, Steven de Jong, Michael Kaisers, Nyree Lemmens, Karl Tuyls, and Gerhard Weiss. Bee-inspired foraging in an embodied swarm (Demonstration). In Tumer, Yolum, Sonenberg, and Stone, editors, Proc. of 10th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2011), pages 1311-1312. International Foundation for AAMAS, 2011. [Download]

Daan Bloembergen, Michael Kaisers, and Karl Tuyls. Empirical and Theoretical Support for Lenient Learning (Extended Abstract). In Tumer, Yolum, Sonenberg, and Stone, editors, Proc. of 10th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2011), pages 1105-1106. International Foundation for AAMAS, 2011. [Download]

Michael Kaisers and Karl Tuyls. FAQ-Learning in Matrix Games: Demonstrating Convergence near Nash Equilibria, and Bifurcation of Attractors in the Battle of Sexes. In Workshop on Interactive Decision Theory and Game Theory (IDTGT 2011). Assoc. for the Advancement of Artif. Intel. (AAAI), 2011. [Download]

Daniel Mescheder, Karl Tuyls, and Michael Kaisers. Opponent Modeling with POMDPs. In Proc. of 23nd Belgium-Netherlands Conf. on Artificial Intelligence (BNAIC 2011), pages 152-159. KAHO Sint-Lieven, Gent, 2011. [Download]

Michael Wunder, Michael Kaisers, J.R. Yaros, and Michael Littman. Using iterated reasoning to predict opponent strategies. In Tumer, Yolum, Sonenberg, and Stone, editors, Proc. of 10th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2011), pages 593-600. International Foundation for AAMAS, 2011. [Download]

2010

Daan Bloembergen, Michael Kaisers, and Karl Tuyls. A comparative study of multi-agent reinforcement learning dynamics. In Proc. of 22nd Belgium- Netherlands Conf. on Artificial Intelligence (BNAIC 2010), pages 11-18. University of Luxembourg, 2010. [Download]

Daan Bloembergen, Michael Kaisers, and Karl Tuyls. Lenient frequency adjusted Q-learning. In Proc. of 22nd Belgium-Netherlands Conf. on Artificial Intelligence (BNAIC 2010), pages 19-26. University of Luxembourg, 2010. [Download]

Michael Wunder, Michael Kaisers, Michael Littman, and John Robert Yaros. A Cognitive Hierarchy Model Applied to the Lemonade Game. In Workshop on Interactive Decision Theory and Game Theory (IDTGT 2010). Assoc. for the Advancement of Artif. Intel. (AAAI), 2010. [Download]

Michael Kaisers and Karl Tuyls. Frequency Adjusted Multi-agent Q-learning. In van der Hoek, Kamina, Lespérance, Luck, and Sen, editors, Proc. of 9th Intl. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2010), pages 309-315, 2010. [Download]

Daniel Hennes, Michael Kaisers, and Karl Tuyls. RESQ-learning in stochastic games. In Adaptive and Learning Agents (ALA 2010) Workshop, 2010. [Download]

2009

Michael Kaisers. Replicator Dynamics for Multi-agent Learning - An Orthogonal Approach. In Toon Calders, Karl Tuyls, and Mykola Pechenizkiy, editors, Proc. of the 21st Benelux Conference on Artificial Intelligence (BNAIC 2009), pages 113-120, Eindhoven, 2009. Eindhoven University of Technology. [Download]

Michael Kaisers, Karl Tuyls, and Simon Parsons. An Evolutionary Model of Multi-agent Learning with a Varying Exploration Rate (Extended Abstract). In Proc. of 8th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2009), pages 1255-1256. International Foundation for AAMAS, 2009. [Download]

2008

Michael Kaisers, Karl Tuyls, Frank Thuijsman, and Simon Parsons. Auction Analysis by Normal Form Game Approximation. In Proc. of Int. Conf. on Web Intelligence and Intelligent Agent Technology (WI-IAT 2008), pages 447-450. IEEE/WIC/ACM, December 2008. [Download]

Michael Kaisers, Karl Tuyls, and Frank Thuijsman. Discovering the game in auctions. In Proc. of 20th Belgian-Netherlands Conference on Artificial Intelligence (BNAIC 2008), pages 113-120. University of Twente, 2008. [Download]

List of Journal Publications, Books and Chapters

2021

Daniel Willemsen, Hendrik Baier, and Michael Kaisers. Value targets in off-policy AlphaZero: a new greedy backup. Neural Computing and Applications, 5, 2021. [View]

2019

Shantanu Chakraborty, Tim Baarslag, and Michael Kaisers. Automated Peer-to-peer Negotiation for Energy Contract Settlements in Residential Cooperatives. Applied Energy, 2019.

Georgios Methenitis, Michael Kaisers, and Han La Poutr ́e. Degrees of Rationality in Agent–Based Retail Markets. Computational Economics, 2019.

2018

Fernando Lezama, Joao Soares, Pablo Hernandez-Leal, Michael Kaisers, Tiago Pinto, and Zita Vale. Local Energy Markets: Paving the Path Towards Fully Transactive Energy Systems. IEEE Transactions on Power Systems, pages 1–8, 2018.

Georgios Methenitis, Michael Kaisers, and Han La Poutr ́e. Renewable electricity trading through SLAs. Energy Informatics, 1:1–57, 2018.

2017

Tim Baarslag, Michael Kaisers, Enrico H Gerding, Catholijn M Jonker, and Jonathan Gratch. Computers That Negotiate on Our Behalf: Major Challenges for Self-sufficient, Self-directed, and Interdependent Negotiating Agents. In Autonomous Agents and Multiagent Systems: AAMAS 2017 Workshops, Visionary Papers, pages 143-163. Sao Paulo, Brazil, May 8-12, 2017. Revised Selected Papers, Springer International Publishing.

Pablo Hernandez-Leal and Michael Kaisers. Towards a fast detection of opponents in repeated stochastic games. In Autonomous Agents and Multiagent Systems: AAMAS 2017 Workshops, Best Papers, pages 239-257. Sao Paulo, Brazil, May 8-12, 2017. Revised Selected Papers, Springer International Publishing.

2016

Sherief Abdallah and Michael Kaisers. Addressing environment non-stationarity by repeating Q-learning updates. Journal of Machine Learning Research, 17.46, 1-31. 2016. [Open Access]

2015

Daan Bloembergen, Karl Tuyls, Daniel Hennes and Michael Kaisers (2015). Evolutionary Dynamics of Multi-Agent Learning: A Survey. Journal of Artificial Intelligence Research, 53, 659–697. [Open Access]

2013

Marco Lützenberger, Tobias Küster, Thomas Konnerth, Alexander Thiele, Nils Masuch, Axel Heß ler, Jan Keiser, Michael Burkhardt, Silvan Kaiser, Jakob Tonn, Michael Kaisers, and Sahin Albayrak. A Multi-agent Approach to Professional Software Engineering. In Proceedings of the first workshop on Engineering Multi-Agent Systems (EMAS), pages 158-177. Lecture Notes in Computer Science. Springer, 2013.

2012

Michael Kaisers. Learning against Learning - Evolutionary Dynamics of Reinforcement Learning Algorithms in Strategic Interactions. Doctoral dissertation, Maastricht University. 2012. [Dissertation, Propositions]

Haitham Bou Ammar, Karl Tuyls, and Michael Kaisers. Evolutionary Dynamics of Ant Colony Optimization. In Ingo J. Timm and Christian Guttmann, editors, Multiagent System Technologies. 10th German Conference, MATES 2012, pages 40-52. Lecture Notes in Computer Science, Vol. 7598. Springer, 2012. [Download]

Marcel Neumann, Karl Tuyls, and Michael Kaisers. Using Time as a Strategic Element in Continuous Double Auctions (Short Paper). In Ingo J. Timm and Christian Guttmann, editors, Multiagent System Technologies. 10th German Conference, MATES 2012, pages 106-115. Lecture Notes in Computer Science, Vol. 7598. Springer, 2012. [Download]

Michael Kaisers, Daan Bloembergen and Karl Tuyls. Multi-agent Learning and the Reinforcement Gradient. In Massimo Cossentino, Michael Kaisers, Karl Tuyls, and Gerhard Weiss, editors, Multi-Agent Systems. 9th European Workshop, EUMAS 2011, pages 145-159. Lecture Notes in Computer Science, Vol. 7541. Springer, 2012. [Download]

2010

Michael Kaisers and Karl Tuyls. Replicator Dynamics for Multi-agent Learning - An Orthogonal Approach. In Matthew E. Taylor and Karl Tuyls, editors, Adaptive and Learning Agents. Second Workshop, ALA 2009, pages 49-59. Lecture Notes in Computer Science, Vol. 5924. Springer, 2010. [Download]

2009

Marc Ponsen, Karl Tuyls, Michael Kaisers, and Jan Ramon. An evolutionary game-theoretic analysis of poker strategies. Entertainment Computing, 1(1):39-45, January 2009. [Download]

2007

Jaap H. van den Herik, Daniel Hennes, Michael Kaisers, Karl Tuyls, and Katja Verbeeck. Multi-agent learning dynamics: A survey. In Proceedings of the 11th International Workshop on Cooperative Information Agents (CIA 2007), pages 36-56. Lecture Notes in Computer Science, Vol. 4676. Springer, 2007. [Download]

List of potential Master's thesis topics

See the group's internship website.

Research topics and related scientists

Energy/Smart Grid
Damien Ernst, Daniel Freund, Han La Poutré, Sven Seuken, Rainer Unland, Mihail Mihaylov (Mike)

Reinforcement learning
Daniel Hennes, Sherief Abdallah, Ardeshir Kianercy, Ari Weinstein, Michael Wunder, Eduardo Gomes

Automated mechanism design
Sofia Ceppi, Peter McBurney

(Evolutionary) Game theory
Frank Thuijsman, Philippe Uyttendaele

List of academic resources

Job search Academic Transfer (Dutch), Academics (German)

Call for proposals CORDIS

Call for papers WikiCFP

Conferences Overview, AAMAS 2010, AAAI, GECCO 2010, ICML 2010, ECAI 2010, ECML 2010, BNAIC, IJCAI 2011

Journals (EF/AI) ML (80.9/94.7), JMLR (87.1/94.6), AI (83.1/92.1), JAIR (68.1/86.9), AAMAS (24.7/41.7), EC (n/a), InTech Open Access (n/a)

Contests RL competition, Trading competitions (TAC/CAT), Starcraft AI competition, The Lemonade Game

Lifestyle guides Pocketmod, 43 folders, 4HWW/Timothy Ferriss, Reading guide

Quality information Udacity, Learn anything online, iTunes U, Wikipedia, Google

Interesting reads Reinforcement Learning: An Introduction, Multiagent Systems, Networks, Crowds, and Markets, Predictably Irrational

Applications MindMapping, Research article management, Publish or perish, Detexify, Word cloud, H-index, Erdos numebr

Valorisation Maastricht Centre for Entrepreneurship, BioMedbooster, Service Science Factory

My productivity toolbox

Email

Short emails with a self-containing subject line.

Signature: I prioritize short emails with a clear subject line, and process email in batches. Please know your Email etiquette.

Inbox Zero by Merlin Mann to process email effectively, try the mailboxapp.

Lists

Getting Things Done (GTD) by David Allen for the project overview and next action lists.

Five types of actions: read, write, contact (email, call, meet), think (brainstorm, plan), do. Nothing else belongs on a todo list.

Time

Eisenhauer time matrix: important beats urgent.

Project milestones and rewards.

Contact is expensive: all contact needs a clear purpose and agenda. Try standing meetings or calls.

The pomodoro technique to focus time and attention.

Scrum and sprints to manage projects and responsibilities

A paper diary to capture my progress in a timeless way.

List of personal recommendations

Paragliding XC planner, DHV (tests and second hand), Paragliding 365, Google wheather map, Wing infos @ Para 2000, Papillon Flugschule und Shop, Windfinder, Wetter online, USAF map, Buienradar, Windguru, Weather Spark, Forum, CTR Europe

Flights Hipmunk, QFly, Billigflieger, Expedia, Opodo

Accomodation Hostelworld, Booking

Entertainment PhD comics

You can contact me in German, English, or Dutch for professional requests. I would also like to improve my Spanish and I'll appreciate opportunities to practice. [download vCard]