Before you connect company data to an AI assistant

Connecting an AI assistant to company data is a permissions decision, not a product rollout. Plan on the basis that whatever the assistant can read, anyone who can talk to it may be able to extract, and whatever it can do, anyone who can influence its input may be able to trigger. Settle five things before the pilot starts: what data is allowed, what the assistant may do, who can use it, what gets recorded, and who owns the risk.

A note on attribution. Points taken from OWASP or NIST are named as such and linked. Everything else in this checklist is our own recommendation, not a requirement from either body.

Decide what data may go in

Start with classification, because every later control depends on it. If you have no scheme, a three-tier one is enough to begin with, and it must be written down before anyone connects a mailbox, a file share or a CRM.

Two points catch teams out. First, connectors usually ingest whole repositories or drives, so a folder nobody remembers sharing ends up indexed. Clean up sharing before you connect, not after. Second, secrets in documents (API keys in wiki pages, passwords in tickets) turn a search feature into a credential leak.

Assume the assistant will read hostile text

Treat everything the assistant reads (inbound email, web pages, shared documents, uploaded files) as content that someone else may have written to steer it. We recommend designing the deployment so that this does no harm, rather than making a system prompt your only control.

In practice:

• Give the assistant the minimum tools it needs. Read-only access first.
• Require human confirmation for any action with side effects: sending email, changing records, running code, making payments.
• Keep untrusted content away from assistants that also have access to restricted data.
• Treat the assistant's output as untrusted input to whatever consumes it. If a downstream system executes or renders it, validate it as you would user input.

For a reference list of LLM application risks, use the OWASP Top 10 for Large Language Model Applications. The project page describes the OWASP GenAI Security Project as a global open-source initiative covering security and safety risks of generative AI, including LLMs, agentic AI systems and AI-driven applications. It says the Top 10 remains a core component, and it lists a 2025 version. The page points to genai.owasp.org/llm-top-10/ for the latest list. Read that list and use its categories as the structure for your threat model rather than inventing your own.

Access control and logging

The assistant should act with the permissions of the person asking, not with a broad service account. If it uses one shared identity with access to everything, a junior employee may be able to retrieve board documents by phrasing a question well. Check how the connector handles per-user permissions, and test it with two accounts of different seniority before go-live.

Then decide what to log. At minimum you want enough to reconstruct an incident:

timestamp, user id, assistant/tool name,
source documents retrieved, actions requested,
actions executed, confirmation (yes/no), result status

Decide in advance how long you keep these logs, who can read them, and whether the prompts themselves contain personal data. Conversation logs are a data store in their own right and need the same classification as the content they hold. Send them to whoever does your monitoring, so an unusual volume of retrievals from one account gets noticed.

Review the vendor before the contract

Ask the vendor direct questions and get the answers in writing:

1. Is our prompt and document data used to train or improve models? Can that be switched off by contract, not just by a setting?
2. Where is data processed and stored, and which sub-processors are involved?
3. How long are prompts, outputs and uploaded files retained, and can we delete them?
4. Which admin controls exist: single sign-on, role-based access, audit export, connector allow-lists?
5. How are security incidents notified, and within what time?

Check that the answers match the actual admin console. A policy page and a configurable setting are different things. If the tool offers plug-ins or third-party connectors, review those as separate suppliers.

Give the risk an owner

Governance sounds like paperwork, and it is, but it is the part that stops shadow deployments. The NIST AI RMF Playbook provides suggested actions aligned to the four functions of the AI Risk Management Framework: Govern, Map, Measure and Manage. NIST states that the Playbook is neither a checklist nor a set of steps to follow in full, and that its suggestions are voluntary. It also notes that AI RMF 1.0 is being updated and the Playbook will follow, so check for the current version. Borrow what fits a company your size.

Our recommendation is a short register of every AI tool in use, each with an owner, the data tiers it may touch, and a review date. Add one more column: whether legal has confirmed that EU AI Act or GDPR obligations apply to that use case, and what they concluded. We have not assessed those obligations here, so ask legal counsel or a compliance adviser to decide. If you want help setting up the register and the review process, our consulting service covers readiness assessments.

Where to start

1. List every AI assistant already in use, including free accounts employees signed up for themselves, and name an owner for each.
2. Write the three-tier classification above and mark which tier each connected data source falls into. Remove restricted sources from scope.
3. Run a pilot with read-only access, per-user permissions and human confirmation for every action. Test it with two accounts of different seniority.
4. Turn on logging, set a retention period, and send the logs to whoever monitors your systems.
5. Send the vendor questions, keep the written answers with the contract, and ask legal to confirm whether EU AI Act or GDPR obligations apply.
6. Before wider rollout, have the setup tested by someone who will try to attack it. A penetration test with scenarios built around your actual connectors shows what a threat model on paper misses.