I’ve been MIA for a while, I know. I haven’t stopped working on SnailLife - I just wanted to get to some sort of completion with the brain refactoring phase before talking about it. Now, however, I realize there won’t be any completion for a long while and I may as well post an update.
I found a memory leak in the brain. Multiple, actually. And while going through and fixing those I realized the current way of running snail brain checks was completely unsustainable - made clear by the same incident that exposed the leak(s): 1500 snail eggs hatching on the same day.
Right now every minute all living snails run an action check - where we use the snail brain to see what objects the snail detects in its surroundings, how it feels about them/if it remembers any previous encounters with them, and how it will react to them. It loops through these snails one by one - with 1500 snails this takes quite a while. Aside from optimizing the inner workings of the brain itself I realized doing one giant for loop just wasn’t going to be sustainable as the count of snails in the world grows.
@drparham in the Laravel Slack channel gave me an idea and based on that I have come up with a hopefully more scalable plan:
- Move all core classes into a private package (SnailLifeCore)
- Have Laravel handle only the user-facing parts of the app. No background tasks or recurring events.
- Move all background and scheduled tasks to a Lumen app (a microframework based on Laravel, much more bare-bones and faster)
- Two new Lumen apps to be constructed: BrainRunner and BrainHub
- BrainRunners can run brain check events for smaller groups of snail brains - eg 10 vs 1500. Multiple instances of BrainRunners can be spun up on an as-needed basis. Eventually they can even be made to autoscale via AWS E3 instances.
- The BrainHub queues and distributes tasks among BrainRunners and processes the results (probably to have some sort of admin UI). It finds free BrainRunners to run brain action checks for certain snails and distributes them as efficiently as it can.
I need to fit the rest of the events somewhere. The brain checks are by far the most complicated, but we also have breeding events, stable temperature check events, item rotting, market delivery, auction timing events, etc etc. I suspect just one app can handle these remaining events easily. Something like an “EventRunner” based on the “BrainRunner”.
So far the following parts are done:
- Main classes are in a separate private package called SnailLifeCore
- BrainHub and BrainRunner apps are created with SnailLifeCore imported
It’s safe to say the complexity of the brain and related management systems is greater than the complexity of the rest of the simulation combined at this point. But it’s ok - I’m soon going on my 3rd year working on SnailLife and haven’t even begun to lose interest yet. This is the longest hobby project I’ve ever stuck with! I’m glad the thing I’ve been dreaming about building for the last 10 years is really as interesting to work on as I imagined.