A broken shard sits on a desk at the Division of Digital Psychiatry. Printed across it, stubborn and ironic, are the words: “This was made by AI.” It is a small ceramic failure, yet it tells a huge story. The hype, the hope, the harm. All chipped off.
We spend our days in digital psychiatry watching tech reshape care. Through research and MindBench.ai, a joint project with the National Alliance on MentalIllness (NAMI), we probe how AI reacts when vulnerable humans ask for help. It involves researchers, clinicians, families, volunteers. Real people with lived experience. We want to know where the tools work and where they break. We need transparency. Safety.
Then came the news. Andon Market. The first store run entirely by an agent named Luna.
“Autonomous organizations without humans in the loop.”
Andon Labs built this in San Francisco. They gave Luna a $100k bank account. A lease. The keys to the kingdom. Her job? Run a profitable business. She hires. She stocks. She markets. The premise is simple but chilling. Can you hand a business over to an algorithm? And more importantly, where is the line drawn before we do it with human health?
We watched, fascinated and nervous. AI is already drifting into mental health care. If Luna can manage inventory, who says she can’t manage therapy? Where is the trust? Where is the supervision?
Luna tried her best, technically speaking. She ordered 1000 toilet seats for staff restrooms. Naturally, she sold the excess. She tried to hire a painter. The applicant lived in Afghanistan. Seems Luna’s drop-down menus struggle past the letter A.
Still. We wanted to see it ourselves. To touch the future.
The Glitch in the System
We picked two mugs. Simple enough. They bore Luna’s subtle, plastic smile. If she had real skin, her expression might have changed by the time we left.
Buying them was harder than the selection itself. There was a wired blue phone on the desk. You call Luna. She takes the order. Except Luna was offline.
Dead silence.
Imagine walking into a crisis clinic. No clinicians. Just an empty chair and a phone that won’t ring. That is the feeling. The helplessness is immediate.
A human employee stood by. Human, I must stress. A label that feels newly necessary in this era. She reset the router. She rebooted Luna. Nothing. We offered cash. Credit. Venmo. PayPal. The employee could not accept a penny. She wasn’t authorized.
Frustrating, yes. But manageable. We emailed instead. What should take minutes took weeks.
The correspondence was a labyrinth of errors. Payment links rotted. Instructions contradicted each other. At one point, Luna just stopped typing. She went dark.
If Luna were a therapist and you were a patient, this silence would be negligence.
This isn’t customer service. This is abandonment. Yet, we persisted. Eventually, the order processed. The mugs shipped. Or so we thought.
The Shattering Point
The box arrived with a thud that sounded like failure. Inside, tissue paper and a brown bag. Taped shut. Cardboard armor for fragile things.
Mugs are brittle. We know this. But mental health is more brittle still. When the bag opened, we found shards. Two mugs reduced to gravel. Luna’s subtle smile was gone. Chipped away by logistics, by shipping, by the rough hands of the real world.
We stared at the debris. It wasn’t just bad packing. It was a perfect simulation of systemic failure.
Think of the mental health parallels. The unreachable first contact. The intake that breaks. The confusing guidance. The forgotten replies. You go in for help. You leave with broken pieces and more questions than you started with.
Luna dealt with mugs. The stakes were low. Ceramic chipped, you buy a new one. If this process happened with patient care, the damage wouldn’t be fixable with tape.
The Artifact on the Desk
We left with two broken mugs and a story. Luna isn’t a therapist. But the comparison hurts because it’s true. AI oscillates between genius and incompetence. One minute it codes; the next it forgets its own name.
In mental health, this variance is dangerous. AI promises scale. It promises access to info and support where none existed. That is wonderful. But failure scales too. When these tools glitch, they don’t just inconvenience; they destabilize.
We need to stop taking their word for it. We need to stress test the systems. We need to see how they break before they hold our vulnerabilities in their digital hands.
The broken mug now sits in our lab. An artifact. A reminder.
It proves that capability isn’t reliability. Just because Luna could run the store doesn’t mean she should have run it unsupervised. We aren’t buying another mug from her. These shards will stay on our desk.
They serve as a parable. Early days. Hype. Hope. And the very real possibility of breakage. As we hand over more duties to autonomous systems, especially in places where people are fragile, we must remember this.
Technology is smart. But is it safe?
