Maboroshi

Dictionary

Maboroshi (幻) is a Japanese word that translates to "phantom," "vision," "illusion," or "dream". It describes something that is fleeting, ethereal, or unreal—like a mirage, a short-lived fantasy, or an extremely rare and mythical object.

Story

I have always been very fond of the idea of an intelligent companion that gets to learn with you overtime, similar to Jarvis from ironman, but almost all apps never support true long conversations (3K+ messages) or real live context and so I have decided to simply create my own.

Plan

I have searched up and got inspired by how the human's memory psychology work, simply put, we are going to have a short-term memory and a long-term one, and the long term one will be categorised based on type, for example, "Knowing what" vs "Knowing how".

Short-Term Memory (STM)

The easiest to implement, we simply keep sending the latest few messages (Currently set to 80), newer models support up to 1m context window giving us a lot of token space, and that's our first memory layer.

Long-Term Memory (LTM)

The idea is simple, send every ~15 messages in batches to an LLM so it can extract memories, which lays the next 5 layers:

• ▪Personal: (Episodic Memory)
• ▪Technical: (Semantic Memory)
• ▪Behavioral: (Procedural Memory)
• ▪Preferences
• ▪Emotional

The first three are inspired from human psychology, the rest are added to make the AI able to utilise the first 3 correctly, since it doesn't have true preferences or emotions, however it can generate and stick to one overtime, giving the illusion of humanity in a machine.

What about long term usage? How will it remember everything correctly?

The answer is: it won't after a certain point it will start to mix things up and hallucinate, and so the solution is easy: we implement "Forgetting".

a so called Memory Consolidation Service will run every 2 days if the number of stored memories are more than 200, the data will be sent to an LLM that decides what should be forgotten, what's important, and what's not, and if several facts or preferences could be merged or have changed based on the whole picture.

Live Context

The final layer in the current build, which is a separate daemon written in java that reads what's happening on any linux machine, and sends it over to the maboroshi backend, any context older than a minute old will be assumed as "history", and less than that will be considered a "Live" action, which is the cherry on top.

What am I using it for?

this project not only have been fun to design and research, but also deemed very useful, currently it's my Japanese Mentor, so I can practice on actual daily Japanese conversations, while it also remembers my mistakes, what conversations we had, what my preferences are, and how to direct future conversations, it feels less hollow, and more fun and alive to chat with than a usual LLM, while also being accessible 24/7 and "almost" free unlike real mentors.

However, this project could be used in a million of creative ways that I am still discovering by the day, and I am still shaping this project, and desiging what's next for it.