Apple Rolls Out Visual AI, Unintended Hallucinated Stereotypes Flood Millions of Devices

Apple’s Visual AI, branded “Apple Intelligence,” is built on an on‑device model of roughly three billion parameters, according to Apple’s own technical report cited by The Decoder. The system automatically generates concise summaries of notifications, texts, emails and other user‑generated content across “hundreds of millions of iPhones, iPads and Macs.” In a forensic audit of more than 10,000 of those summaries, the non‑profit AI Forensics group found a consistent pattern: when the source text identified a Hispanic man, the ethnicity was retained in the summary, but when the same narrative described a white man, the ethnicity was omitted. The researchers reproduced the bias by feeding 200 fabricated news stories—each with four ethnic variations—into the model and observing that “whiteness functions as an invisible default” (The Decoder).

Beyond ethnicity, the audit uncovered gender‑stereotype reinforcement. Ambiguous sentences that could refer to any gender were routinely rendered with masculine pronouns, while references to traditionally female‑coded roles (e.g., nursing, teaching) were more likely to retain gendered language. The bias was not triggered by any user prompt; it emerged automatically as the model parsed everyday communications. Because the feature runs locally on each device, the distortions propagate to every user without a central server that could be patched or audited in real time.

The scale of deployment raises regulatory concerns. Under the EU AI Act, systems that present a “systemic risk” to fundamental rights—such as bias in identity‑related processing—must be classified as high‑risk models and subjected to conformity assessments, transparency obligations and post‑market monitoring. The Decoder notes that Apple has not signed the voluntary AI Code of Practice, a step that many large tech firms have taken to demonstrate compliance. If European regulators deem Apple Intelligence a high‑risk AI, the company could face fines of up to 6 % of global revenue or be forced to roll out corrective updates across its entire device fleet.

Apple’s leadership has framed Visual AI as the cornerstone of its upcoming wearable and AR initiatives, a narrative echoed in Bloomberg Technology’s coverage of Tim Cook’s push for “Visual Intelligence” as the defining feature of the next product wave. However, the bias findings arrive just weeks before the scheduled March 2 launch of the iPhone 18 Pro and the anticipated rollout of iOS 26.4, both of which are expected to integrate deeper AI capabilities. The timing suggests that Apple may need to allocate engineering resources to remediate the model before it becomes a selling point, potentially delaying feature rollouts or prompting a public acknowledgment of the issue.

Analysts see the episode as a litmus test for Apple’s broader AI strategy. The company has historically relied on on‑device processing to differentiate itself on privacy, yet the forensics report demonstrates that privacy‑by‑design does not automatically guarantee fairness. If Apple can quickly retrain or fine‑tune the three‑billion‑parameter model to eliminate the ethnicity and gender gaps, it could preserve its reputation for responsible AI while still leveraging the hardware advantage that underpins its ecosystem. Failure to act, however, may erode consumer trust and invite scrutiny from regulators already eyeing the AI‑driven features of major platforms. The next few weeks will reveal whether Apple treats the bias as a technical glitch to be patched or as a strategic inflection point for its AI ambitions.