Reinforcement Learning e
a Era da Experiência
Instituto de Informática
Universidade Federal do Rio Grande do Sul (UFRGS)




“Máquinas conseguem pensar?”
“Um computador mereceria ser chamado de inteligente se pudesse enganar um humano a acreditar que também é humano.”


“We normally associate punishments and rewards with the teaching process.”
“The machine has to be so constructed that events which shortly preceded the occurrence of a punishment signal are unlikely to be repeated, whereas a reward signal increased the probability of repetition of the events which led up to it.”
“What we want is a machine that can learn from experience”.







Agente
Ação
Próx. Estado
Recompensa
Ambiente

Agente
Ambiente

estado (s_t)
ação (a_t)
reward (r_t)
Política \textcolor{#00B050}{\pi : S \mapsto A}
\pi^* = \max\limits_{\pi} \sum\limits_t \gamma^t r_t


Towards Superhuman Intelligence. David Silver (RLC 2024)
Human-Level Control through Deep Reinforcement Learning. Mnih et al. (Nature 518, 2015)

Towards Superhuman Intelligence. David Silver (RLC 2024)
Mastering the Game of Go with Deep Neural Networks and Tree Search. Silver et al. (Nature 529, 2016)

Towards Superhuman Intelligence. David Silver (RLC 2024)

A General Reinforcement Learning Algorithm That Masters Chess, Shogi, and Go through Self-Play. Silver et al. (Science 362, 2018)

Towards Superhuman Intelligence. David Silver (RLC 2024)
BDX Droids – Disney Research
Walk, Run, Crawl, RL Fun | Boston Dynamics | Atlas


2020
Towards Superhuman Intelligence. David Silver (RLC 2024)



AlphaProof
Towards Superhuman Intelligence. David Silver (RLC 2024)


IMO 2024
28/42
Prata
IMO 2025
35/42
Ouro
Treinado com novas técnicas de RL para raciocínio em múltiplas etapas e prova de teoremas
Advanced version of Gemini with Deep Think officially achieves gold-medal standard at the IMO. Google DeepMind (2025)

AlphaProof???
Towards Superhuman Intelligence. David Silver (RLC 2024)

Silicon Valley bets big on ‘environments’ to train AI agents. TechCrunch (2025)


Aggressive behavior
Safe behavior
Outracing champion Gran Turismo drivers with deep reinforcement learning. Wurman et al. (Nature 602, 2022)

Goal: Learn a set of policies \Pi = \{\pi_1, \dots, \pi_n\} guaranteed to contain an optimal policy for any preferences over objectives

Infeasible in real-world settings!
e.g. autonomous driving, healthcare, robotics
How to learn a set of policies containing optimal policies for
any user preference in a sample-efficient manner?


Time vs. Killing Enemies

Coins vs. Time
Sample-Efficient Multi-Objective Learning via Generalized Policy Improvement Prioritization. Alegre et al. (AAMAS 2023)




AMOR: Adaptive Character Control through Multi-Objective Reinforcement Learning. Alegre et al. (SIGGRAPH 2025)
NeurIPS 2023

NeurIPS 2025

Reinforcement Learning e
a Era da Experiência
Lucas N. Alegre
Muito Obrigado!

