These are small machine learning projects/experiments, with the goal of understanding and learning from ML implementation methods to later be used on a separate project. Most of them explore evolutionary machine learning, where i try to mimick biological processes using ideas learned from machine learning and use them in realtime applications. Many were made quickly, either as coursework or as personal experimentation.

Basic Reinforcement Learning

I created a reinforcement learning demo in Pygame where a square learns to escape a boxed environment through a single goal opening. It uses four directional “eyes” to sense distances to walls, forming a state vector stored in long-term memory. Initially, the square explores the space randomly, guided by an epsilon value that balances exploration and exploitation. Colliding with any wall resets it to the starting point. Over time, it refines its policy through Q-learning, eventually learning an efficient path to the goal. If I revisit this project, I'll add obstacles or a maze. Below are examples with varying epsilon values.

The Titanic Experiments

These experiments explore evolutionary algorithms as an alternative to gradient-based training of neural networks, using the Titanic Kaggle Dataset. The challenge is a classic binary classification problem: predicting whether a passenger survived the Titanic disaster based on several features.

The dataset includes structured data points such as age, sex, ticket fare, passenger class, number of siblings/spouses aboard, number of parents/children aboard, and port of embarkation. Labels indicate survival (1) or death (0). The full dataset and competition details are available here on Kaggle.

Would you survive the Titanic?

I decided to go ahead and export the model I trained locally so you can input your own Titanic passenger data into the model and see if it thinks that passenger would've survived the Titanic disaster!

I chose to evaluate my neural networks using Q-values instead of accuracy because I wasn’t trying to predict a correct label — I was trying to train agents to learn optimal behaviors over time, and Q-values reflect how valuable each decision is in the long run, not just whether it was “right” in the moment. Even if the shapes of the Q-value and accuracy curves mirror each other, the Q-value gives a smoother, more sensitive view of learning progress, especially when improvements are subtle or when the agent is still exploring. For me, it felt more informative than a flat accuracy metric that doesn't tell the whole story.

Particle Swarm Optimization (PSO)

In this approach, each neural network is treated as a particle flying through a high-dimensional space of weights. These particles adjust their position based on both their own best-known performance and that of their neighbors. Over iterations, the population “swarms” toward better solutions. PSO proved especially useful for quickly finding reasonable accuracy without fine-tuning learning rates or gradients.

PSO was the least consistent of the three, but it was the most exploratory, with strong exploration and a tendency to avoid premature convergence. Here's a link to the code.

Evolutionary Strategies (ES)

Evolutionary Strategies operate by perturbing neural network weights with random noise and measuring how performance changes. The best-performing variants are selected and used to guide the next generation. ES was the most consistent of the three, striking a balance between exploration and exploitation, and yielding solid validation scores across multiple runs.

How were they made?

The projects use Python (PyTorch, HuggingFace, OpenCV, etc), local models, and some custom training. Some are just scripts, others are interactive UIs or browser demos.