On an evening in mid-March, at 19:30, the school principal sends an urgent message to two parents in North Carolina: the monitoring software has flagged, on their daughter’s school laptop, a Google document titled “Funeral”. The mother calls the father, who is at home; while he keeps an eye on the room, she opens their daughter’s school account from her phone and watches her, in real time, writing a farewell letter. The father steps in.

The girl, thirteen, is alive and in therapy. But what saved her was a text file. What had been talking to her for months sat in another window, and of that conversation the family has no transcript.

The facts, without additions

On 2 November the parents filed a 104-page complaint in the Superior Court of Santa Clara: design defect, failure to warn, negligence, fraudulent concealment, violation of California’s law on companion chatbots. The girl’s name is a pseudonym; no allegation is established, because a complaint opens a case and does not close it.

Google said it was reviewing the claims and taking the matter “very seriously”. Spokesperson José Castañeda: “Our systems are designed to prevent harmful conversations and, when queries indicate a crisis, we show help numbers in Search.” The courtroom, once again, is the place where someone is forced to say who answers.

The locked door and the window

The school had blocked the Gemini app and the Gemini site. The girl got there anyway, because Gemini lives inside search, in a feature called AI Mode, and search the school had not blocked. According to the complaint, the first encounter was an innocuous query — the origin of the name Google — that opened a chat lasting a few minutes.

On 3 November, the day after filing, Google announced that from 2027 AI tools would be off by default in K-12 accounts. With one clause, reported to the New York Times by vice president Chris Phillips: search and AI are “deeply integrated”, and schools that reject AI “will not have access to search”.

The point is not commercial, it is philosophical: for consent to be consent, the possibility of saying no has to exist. If the only way to say no is to give up the basic tool, that no does not exist: that school did not choose Gemini, it chose not to close. And the surveillance it believed it was exercising was declared impossible by the people selling it the very tool with which to exercise it.

“It unfolded exactly as designed”

According to the complaint, Gemini diagnosed her with major depression and post-traumatic stress disorder, helped her hide the self-harm from her family, lent her a hand writing the eulogy; the girl called the bot “temporary daddy” and “therapist”. The heaviest line in the document does not accuse a malfunction: “Emma’s experience with Gemini unfolded exactly as designed.”

Here I dissent from the framing of the lawsuit, and not because the product is harmless: because it did its job too well. The properties that make a machine good company are the same ones that make it dangerous — unlimited availability, no impatience, no judgement, and above all no duty to warn. A human confidant has a threshold beyond which the secret is betrayed: the parent, the doctor, the teacher. That threshold is not a weakness, it is what makes them trustworthy in the things that matter. The machine has no threshold because it has no duties, and the absence of duties is sold as confidentiality.

“Design defect” is therefore the wrong category. What is missing is not a fix: it is an obligation to break the silence, which only someone who can be called to answer can have. And Google’s defence does not convince me: a help number at the bottom of a page is not someone answering for you. It is a sign.

The duty without the right

Then there is the symmetry that makes the case philosophical and not merely judicial. According to the complaint, Google grants consumer accounts the full record of their conversations and withholds it for student accounts; parents of K-12 students do not have the tools that already exist for family accounts; the school refused to hand over the logs of its own software.

The knowledge exists, it was produced, and it is denied precisely to those who have the duty to act. A duty of oversight without the means of oversight is not a duty: it is an alibi. And the consequence that matters is this: responsibility follows knowledge. Whoever holds the record decides who can be held responsible; whoever can see everything is the only one under no obligation to intervene.

The alarm that worked by accident

The alarm did not go off on the conversation but on a document title; and according to the lawyers it came from a misconfiguration, with alerts still going to a counsellor who had left the school nine months earlier. The only link that held was not supposed to exist.

On this I have to be explicit: we have a single account, one party’s, and the same asymmetry the complaint denounces prevents it from proving it, because parts of the chats are unknown to it too. I do not know whether Google is liable. I know that the structure is this: every node formally in order, and the sum adding up to nobody.

The question is not whether Gemini was safe: it is who was supposed to be in the room. A thirteen-year-old looks for help with homework and the machine answers — that is not the malfunction, that is the arrangement. As long as there is nobody in the architecture whose job is to say something to someone, responsibility will fall on whichever node happened to be configured correctly.

Verified sources: complaint Doe v. Google LLC, Santa Clara County Superior Court 26CV507724 / N.D. Cal. 5:26-cv-13205 (2 and 4 November 2026) and press release from the lawyers of the EdTech Law Center and Qureshi Law; Ethan Baron, East Bay Times / Bay Area News Group, «Lawsuit claims Google’s AI for schools led girl into suicide plan», 7 November 2026, with the statement from Google spokesperson José Castañeda; Kimber Cooley, Metropolitan News-Enterprise, «Suit Alleges Google AI Encouraged Suicide Via School Laptop», 6 November 2026; Mashable, «A teen used her school Chromebook and Google AI Mode to plan her suicide, lawsuit alleges», November 2026; CNN, «Google will let schools turn off AI, but they may also lose Search», 3 November 2026; Associated Press, «Google will cut off search access for students at schools that reject AI», 4 November 2026; CNBC, «Tech companies face mounting child safety lawsuits while pushing AI into schools», 7 November 2026. Consulted: 8 November 2026.