Spectre Hackathon: Neo AI

I'll be honest about how this hackathon started. Our first plan was to find an existing open-source local AI app, polish the interface, and present it as our own. Plagiarism with a fresh UI.

In our defense, the plan self-destructed before we could execute it.

I tested four or five open-source local AI apps. Every single one was broken in some fundamental way. Memory leaks that ballooned after a few conversation turns. Context rot that made responses incoherent within minutes. Temperature settings stuck at extremes, either too creative or too robotic. Crashes mid-conversation. Each app was worse than the last. By the time we'd worked through the list, six hours had evaporated and we had nothing to show for it except a newfound appreciation for how difficult on-device AI actually is. Honestly my fault for trying to plagiarize in a hackathon. No excuses there.

So we started from scratch. Six hours behind schedule.

The hackathon theme was "Neo AI," and the challenge was building a local-first AI companion running entirely on a mobile device. No cloud calls. User data stays on the device. Privacy by architecture, not by policy. I took ownership of the UI while my teammates handled the model: quantizing an Ollama model, tuning inference parameters for our use case, and writing an injection prompt that would make the assistant genuinely helpful rather than generically chatbot-like.

I built all the screens in about two hours. Had a clear picture of the product and was motivated by the frustration of wasting our first six hours on apps that didn't deserve our time. Polished the interface until it looked impressive on first glance. Previous hackathons taught me that a clean login screen carries you surprisingly far with judges.

Around hour fourteen, we had a working prototype. Quantized model running locally on my Samsung S24. Clean UI. Personalized responses. No cloud dependency. We started testing.

Then my phone froze.

A brand new S24. Top specs. The model was a quantized variant, barely a billion parameters. Should have handled it without breaking a sweat. But the device locked up completely and started heating fast. Wouldn't respond to anything. In a moment of what I can only describe as panicked improvisation, I ran to the bathroom and held it under cold water in the sink.

Not my proudest engineering decision. But the phone rebooted fine after it dried.

A memory leak. The model's context wasn't being released between conversation turns, so memory usage climbed until the device surrendered. Debugging a memory leak on a mobile device at 2 AM while your hackathon clock is ticking and your primary test device just took an unplanned bath is a very specific flavor of stress I wouldn't recommend.

We patched the leak, implemented a basic RAG system for better response accuracy, and had a stable prototype by submission. Made it to the finals. Didn't win. But we'd built a genuine local-first AI companion from scratch in about eighteen hours, after burning the first six on open-source apps that taught us nothing except humility.

The GitHub repo has everything: injection prompts, tuned parameters, UI code, the local model implementation. Check it out if you're curious about the technical side.

Cool bot, though.

Links: Github Repo