Preview of the Autonomous agent trained by neuroevolution project
Data / AICompleted

Autonomous agent trained by neuroevolution

Designing the rewards and penalties that guide the evolution of a virtual robot, driven by a neural network, to reach a target without collisions.

Status

Completed

Period

January 2025 → March 2025

Context

School

Field

Data / AI

Technologies

JavaNeural networkGenetic algorithmNeuroevolutionReward functions

01The project

This project, carried out during my Bachelor's degree, consisted of training a virtual robot to move on its own in a 2D environment. The goal was for it to reach a target as quickly as possible, without ever hitting an obstacle.

We worked on EvoAgentStudent, a simulation platform developed by François Suro, a researcher at LIRMM. My work wasn't about the platform's code, but about what guides the agent's learning, its rewards and its penalties.

02The agent's brain

The robot is controlled by a small neural network. As input, it receives the values of its sensors, such as the direction of the target or the distance to obstacles. As output, it controls its motors. In between, layers of neurons turn what the robot perceives into decisions.

Nobody programs its behavior. Everything depends on the network's weights, the importance given to each connection between neurons. The challenge is finding the right ones.

03Learning through evolution

To find these weights, the platform uses neuroevolution, inspired by natural selection.

  • A population of agents is created, each with random weights.
  • Each agent is tested in the simulation, and receives a score based on its actions.
  • The best agents are selected and "reproduce", by mixing and slightly modifying their weights.
  • The new generation is tested in turn, and so on.

Generation after generation, more and more efficient behaviors emerge, without anyone ever writing them. To go faster, the agents of the same generation are tested in parallel, on several processor cores.

The chart below shows what happens during a training run, in the environment where the agent has to reach the target. With each generation, the best score improves, tripling in just 26 generations. The plateaus correspond to generations where no agent did better than the previous champion, before a new improvement appeared.

04Three challenges of increasing difficulty

Training took place in three environments.

  • Reaching the target, in an open space
  • Avoiding obstacles, moving without hitting anything
  • Combining both, reaching the target in the middle of obstacles

05My role, rewards and penalties

Each agent's score depends entirely on the reward functions, and that was the core of my work. I designed the rewards that encourage the robot to get closer to the target and reach it quickly, and the penalties that punish it when it hits an obstacle.

I then observed the behaviors obtained across generations, to adjust these rules and correct unwanted behaviors.

06What I take away from it

This project taught me that in artificial intelligence, an agent learns exactly what you reward, and not necessarily what you had in mind. A poorly designed reward can produce surprising behavior, which maximizes the score without really solving the problem. Designing the right rewards therefore means anticipating how the agent could exploit them.

I also discovered another way of learning, neuroevolution, where you don't program the behavior but let it emerge, generation after generation. Observing and analyzing these behaviors taught me to reason about a system I don't directly control, and to adjust my rules based on what I observed.

Above all, this is the project that introduced me to neural networks. Watching a robot learn on its own to reach its target was a real turning point, and it's largely what made me want to specialize in artificial intelligence.

07Tech stack

Java, with the JBox2D physics engine for the simulation, a multilayer neural network for the agent's brain, and a genetic algorithm for its learning.