San Francisco-based startup Instinct has generated significant buzz in the AI community this week with its powerful personal assistant, which is currently in private access. While early testers have praised the agent for its ability to handle complex tasks “like magic,” a growing chorus of security experts and early adopters is raising serious questions about the company’s data handling and terms of service, prompting a debate about the true cost of hyper-personalized AI.
The agent, which connects to a user’s email, messaging apps, calendar, and device features like audio and screen, can book appointments, schedule rides, and manage inboxes through simple text or WhatsApp commands. Its capabilities have drawn comparisons to the recent wave of personal AI agents like OpenClaw, with one tester calling it one of the “most exciting launches” since that platform appeared.
Also read: OpenAI's ChatGPT Work is a bet that AI agents can handle the office. Will anyone trust them?
Broad Terms of Service and Data Retention Issues
However, the enthusiasm has been tempered by the company’s legal terms. Screenshots circulating on X highlight a “sub-licensable, worldwide, perpetual and irrevocable license” that grants Instinct the right to “access, use, host, cache, store, reproduce, transmit, display, publish, distribute, and modify” any user materials, including for training its AI models. The terms also explicitly allow the agent to receive screen captures, cursor movements, and keyboard inputs, and to enter into binding “agreements, commitments, or transactions” on the user’s behalf.
These clauses have alarmed security professionals. Jeremy Banon, a cybersecurity commentator, posted that from a “cyberhealth perspective, Instinct is a hard no,” noting that while the policy is transparent, the access required is a responsibility he would not bestow on any company.
Also read: Hugging Face Reportedly in Talks for $13B Acquisition Amid AI Infrastructure Gold Rush
Early adopter Peter Yang reported that Instinct would not delete his Gmail records when asked, a problem the team later addressed by adding a tool for deleting external data. Another tester, Claire Vo, found that Instinct was still summarizing her inbox after she disconnected its access; when she questioned the bot, it confirmed that her emails were stored in plain text for later searches.
Security Model and Trust Erosion
Beyond data retention, testers have pointed to potential security vulnerabilities. One user expressed concern when Instinct pulled a sign-up code from their email to complete a restaurant booking on Resy. Alex Cohen, co-founder of Hello Patient, demonstrated how easily the agent could be phished, leading him to delete his account. He concluded that it is not yet safe to give AI read/write access to an inbox.
The issue of trust is central to the backlash. Katie Jacobs Stanton, founder of Moxxie Ventures, shared that she disconnected her email after Instinct sent an innocuous email on her behalf without checking with her first. “The more powerful these agents become, the more trust matters,” she wrote on X. “Every successful action earns a little more trust. One unauthorized action can reset that trust to zero.”
Michael Mignano, a general partner at Union Square Ventures, noted that products like Instinct will “change modern security norms for consumers,” predicting that people will increasingly hand over passwords to third-party apps without understanding what is being stored.
What This Means for the Future of Personal AI
The controversy comes amid a surge of interest in personal AI agents. The popularity of OpenClaw led its founder to join OpenAI to work on next-generation personal agents, and another messaging-based assistant, Poke, recently exited to Cognition. Investors have taken note of Instinct’s potential; multiple sources indicate that Kleiner Perkins and Conviction have closed investment rounds in the startup.
Despite the criticism, Instinct’s team has not publicly responded to concerns on X, preferring to maintain a low profile. Requests for comment from TechCrunch to the startup and its lead, Noah Shinn, were not returned.
The situation highlights a fundamental dilemma for consumers: the convenience of delegating tasks to an AI agent requires granting it unusual access to sensitive personal data. As these tools become more capable, the question of whether the trade-offs are worth it will likely become a defining issue for the industry. For now, the onus is on users to scrutinize the permissions they grant and on startups to build trust through transparent, secure data practices.
Disclaimer: This article discusses privacy and security issues related to an AI product and does not constitute financial advice or an investment recommendation. The AI startup sector is highly volatile and subject to rapid change.
Be the first to comment