A vendor’s model is someone else’s computer, trained and run where you do not sit. Client data does not belong there by default. It belongs there only when you can name the purpose, the place, and the person who can make it stop. If any of those three is missing, keep the file in the building.
This is a sorting rule for a firm that already uses a writing tool for its own drafts and is now being asked to paste a customer’s names, contracts, or numbers into the same box. It is not a product review, and it is not a claim that a small model on your own machine is always safer. It is a claim that you should know which of the two you are doing before you upload.
Name the purpose before the file moves
The UK Information Commissioner’s Office guidance on AI and data protection is not a statutory code, and the page says it is under review because of the Data (Use and Access) Act. What it does say is that a risk-based approach means assessing risks to the rights and freedoms of individuals and putting proportionate measures in place. It also says you may have to stop a planned project if you cannot sufficiently mitigate those risks. “The demo was impressive” is not a mitigation.
Write the purpose in one sentence a client could read. “We will use the model to draft an internal summary of this contract, and a lawyer in this firm will check it before anyone outside sees it.” If the sentence is “we will see what the tool says”, the file stays put. The ICO’s point is about people, not about novelty. A customer list, a patient’s note, a staff review, and a set of unpaid invoices are all capable of harming someone if they leak or if the model invents a fact about them.
The same page says the guidance is about data protection law as the ICO reads it, and that other laws still apply, including equality law. Do not treat a UK page as the statute of Mauritius, Kenya, or Nigeria. Do treat it as a worked example of the question every firm owes its clients: why does this file need to leave?
A shared model is a different product from a private draft
There are two different machines hiding in the phrase “AI tool”. One keeps your prompt inside a tenant you control, does not use it to train a model other customers use, and lets you delete it. The other adds your text to a model, or to a review queue, that is not yours. The contract questions for telling those apart are on the firm’s page about an AI software contract. This page is the prior decision. If you do not yet have a yes or no on training, do not upload client files while you wait for the clause.
NIST’s AI Risk Management Framework is voluntary. Version 1.0 was released on 26 January 2023. NIST’s page says it is meant to build trustworthiness into the design, development, use, and evaluation of AI products, services, and systems, and that version 1.0 is being revised. The four functions set out in NIST AI 100-1, govern, map, measure, and manage, are a way to ask what the system touches and how you will know it has gone wrong. They are not a certificate that makes an upload lawful. On 26 July 2024 NIST also released a Generative Artificial Intelligence Profile, NIST AI 600-1, for the risks that generative systems add. Read it if the tool writes. Do not read it as permission to paste a client’s confidential schedule into a public chatbot because the profile exists.
When client data should stay in the building
Keep the file off someone else’s model when any line below is true.
- You cannot name the purpose in a sentence the client would accept.
- The vendor has not said, in the contract, whether the file trains a model other customers use.
- The file identifies a person and you have no retention date.
- A wrong sentence from the model could be sent onward as if it were yours, with no one named to read it first.
- The work is a decision about a person, and you have not asked whether a high-risk deployer duty applies where you operate.
That last line is narrow. Article 26 of Regulation (EU) 2024/1689, on EUR-Lex, applies to deployers of high-risk AI systems. Where the deployer controls the input data, the input must be relevant and sufficiently representative for the intended purpose. That is not a rule for every draft email. It is a rule for the systems the Regulation classes as high-risk, in the places the Regulation reaches. A firm outside that scope still has the plain version: do not feed a hiring or credit decision with a file you would not show the person it describes.
A small model is not a magic lock
Running a model on a machine you own can be the right answer when the file cannot leave. It can also be a new copy of the same problem. The machine still needs an owner, a way to delete the file, and a person who reads the output. A laptop in a bag is not a control. A server in a cupboard with no backup and no named administrator is not a control either.
Use the small or local model when the work is narrow and the data is sensitive: a first pass over a contract that must not sit in a foreign tenant, a sort of your own invoices, a draft that never includes the customer’s name. Use the vendor’s model when the data is already public, or when the contract forbids training on your files and you have a deletion date, and a person still checks the result. The firm’s note on connecting an assistant to tools you already pay for is about the second case, the plumbing. Plumbing is not a reason to connect the client archive.
Staff habits decide this more than architecture. If people paste client work into personal accounts, a careful company tenant does not unwind the paste. Pair any tool with the rule in the AI working agreement: what never gets pasted, and who reviews what leaves the firm.
Decide in one sitting
Take one live client file, not a hypothetical. Answer the three names: purpose, place, person who can delete it. If you cannot, the file stays. Tell the team the answer in a sentence, not in a policy no one opens. Revisit the sentence when the vendor changes its terms. A checkbox you ticked last year is not a purpose.
Stopping is a complete outcome. The ICO says as much for the projects whose risks you cannot bring down. A client who trusted you with a file did not agree to be someone else’s training example. If you cannot rule that out, do not upload.
Key takeaways
- The ICO’s AI guidance is not a code, and it is under review. It does say you assess risks to people and you may have to stop.
- Do not upload client files until the contract says whether they train a model other customers use.
- NIST’s AI RMF is voluntary. Version 1.0 is dated 26 January 2023 and is being revised. The July 2024 generative profile does not authorise a paste.
- Article 26’s input-data duty is for deployers of high-risk systems, not for every draft.
- A model on your own machine still needs an owner, a deletion path, and a human reader.
FAQ
Can we paste a client name into a free chatbot to try a prompt?
No. If you cannot say whether that chat trains a shared model, or who can delete it, the name stays in your own files.
Does a local model solve the problem?
Only if someone owns the machine, can delete the file, and still reads the output. A laptop in a bag is not that control.
Is the NIST framework a legal permission?
No. NIST describes the AI RMF as voluntary. It is a way to ask what the system touches. It does not replace data protection law where you operate.
Does the EU AI Act ban cloud models?
No. Article 26 sets duties for deployers of high-risk systems, including input data where the deployer controls it. A drafting tool is not automatically in that class.
What do we tell the team this week?
One sentence: client files stay off tools that have not said, in the contract, that they will not train a shared model on those files.
Sources
- Information Commissioner’s Office, “About this guidance”, Guidance on AI and data protection. ico.org.uk
- NIST, AI Risk Management Framework. nist.gov/itl/ai-risk-management-framework
- NIST, AI 100-1, Artificial Intelligence Risk Management Framework (AI RMF 1.0), January 2023. nvlpubs.nist.gov
- Regulation (EU) 2024/1689, Article 26, EUR-Lex. eur-lex.europa.eu
This article is general information, not investment advice. See section 4 of our Terms & Conditions.
This is the kind of decision we advise on. If you are weighing it for your own business, see how we work or start a conversation.
