Most pilot planning assumes the customer buys. Many don’t.
Ninety days in, the customer says the results were interesting and the budget went elsewhere. By then, the startup has fine-tuned an adapter on the customer’s documents, built a retrieval index from its knowledge base, saved a set of its hardest queries as evaluation cases, and possibly folded some of its data into the next version of the shared model.
Now someone asks: what do we have to delete, and what can we keep?
SaaS AI Lawyer Andrew S. Bosin, a two-time SaaS founder, sees this question arrive at the worst possible moment: after the pilot has ended, when the customer has no reason to be generous and the agreement never answered it.
This article covers what an AI startup can keep, what it must remove, and the pilot terms that decide the answer. For the pilot agreement as a whole (fees, success criteria, conversion pricing), see Andrew’s guide to the AI proof of concept agreement for paid pilots. For how to get training consent in the first place, see Can We Train Our AI Model on Beta Customer Data?
Quick Answer
When a pilot ends without a purchase, the startup’s rights in anything built from the customer’s data usually depend on four things:
- What the pilot agreement said about training. Service-only language means the data, and often what was built from it, has to go. Express training consent that survives termination may let the startup keep a shared model.
- What was actually built. Raw data, logs, and retrieval indexes can be deleted. A customer-specific model should almost always be deleted. Training that went into a shared model is much harder to undo.
- The confidentiality clause. A “return or destroy” obligation that reaches materials “derived from” confidential information can be argued to cover a model trained on it.
- Privacy law. If personal data was involved, deletion obligations under GDPR and state privacy laws may apply regardless of the contract.
The fix is to decide all of this in the pilot agreement, before any data comes in.
Step 1: Inventory What the Pilot Actually Created
“The customer’s data” isn’t one thing by the end of an AI pilot. Before deciding what to delete, the startup needs an honest list:
- Raw data: documents, records, tickets, or files the customer uploaded
- Prompts, outputs, and logs: every interaction during the pilot
- Retrieval indexes and embeddings: vector representations built from customer content
- Evaluation sets: customer queries and answer keys saved to measure quality
- Customer-specific models: fine-tuned models, adapters, or configurations built for that customer
- Shared model contributions: customer data used to train or fine-tune a model offered to other customers
- Know-how: what the team learned about the domain, the prompts that worked, and the failure modes it found
Each item carries a different legal risk and a different technical cost to remove. The first four are usually straightforward to delete. The last three are where disputes happen.
Step 2: Read What the Pilot Agreement Actually Says
Most pilot agreements fall into one of three patterns.
Pattern 1: Service-Only Use
The agreement lets the startup use pilot data only to provide the pilot and measure results, which is the position most enterprises insist on. Andrew’s no-training clause guide covers how that position is negotiated.
Under this pattern, anything built from the data is hard to justify keeping once the pilot ends. A customer-specific model or a saved evaluation set is still a use of the customer’s data, and the startup’s permission to use it ended with the pilot. If any pilot data leaked into a shared model, the startup has a problem regardless of whether the customer converts.
Pattern 2: Silence
The agreement doesn’t mention training at all, or relies on vague language like “to improve our services.”
Silence usually works against the startup. The customer will read the data license as limited to the pilot and ending with it. The confidentiality clause, which almost every pilot has, will typically require the startup to return or destroy the customer’s confidential information when the pilot ends. And if the startup drafted the agreement, ambiguity is often read against it.
Pattern 3: Express Training Consent
The customer agreed in writing that the startup could use pilot data to train or improve its models. This is the strongest position, but it still leaves questions:
- Does the license survive termination? A license granted “during the Pilot Period” may end with it. If the startup needs to keep using a model trained on the data, the license, or at least the right to keep using models already trained, has to survive expressly.
- Is the consent tied to conversion? Some customers agree to training on the assumption they’ll become paying customers. If the consent is conditional, non-conversion may end it.
- Does the confidentiality clause conflict? A training license and a “destroy all derivatives” obligation in the same agreement contradict each other. The agreement needs to say which one controls.
Step 3: The Confidentiality Clause Is the Hidden Issue
Founders focus on the data and training clauses and overlook the confidentiality clause. It’s often the one that decides the outcome.
A typical clause requires the receiving party, at the end of the agreement, to “return or destroy all Confidential Information, including all copies, extracts, and materials derived from it.” A customer can argue that a model fine-tuned on its confidential documents is a material derived from them.
Whether that argument succeeds depends on the drafting and the facts. But the startup doesn’t want to be having that argument at all. The pilot agreement should say expressly how models, indexes, and evaluation sets are treated, instead of leaving them to a clause written for documents and spreadsheets.
Step 4: Check Whether Privacy Law Decides It for You
If the pilot data included personal data, the contract isn’t the only thing that matters.
GDPR
When the startup processes personal data on the customer’s behalf, it’s usually a processor. Article 28(3)(g) of the GDPR requires a processor, at the controller’s choice, to delete or return all personal data at the end of the services and to delete existing copies, unless EU or member state law requires storage.
The European Data Protection Board’s Opinion 28/2024 on AI models adds that an AI model trained on personal data isn’t automatically anonymous. If a model could still reveal personal data it was trained on, it may itself be treated as containing personal data. In that case, deleting the raw data while keeping the model may not satisfy the deletion obligation.
CCPA
California’s regulations let a service provider use personal information internally to build or improve the quality of its services, “provided that the service provider or contractor does not use the personal information to perform services on behalf of another person” (11 CCR § 7050(a)(3)). That allowance can support some model improvement, but it’s narrower than it sounds. It doesn’t let the startup reuse one customer’s personal information to serve another customer, and it doesn’t override the startup’s contract with that customer.
FTC Model Deletion Orders
The FTC has required companies to delete not just improperly used data but what was built from it. In its 2021 Everalbum settlement, the company had to delete users’ photos and videos “and the models and algorithms it developed” with them. A startup that keeps a model trained on data it had no right to keep carries the same kind of risk, at least where consumers’ personal data is involved.
The data processing terms should be consistent with all of this. Andrew’s AI SaaS DPA guide covers the deletion and return commitments enterprise customers expect.
Step 5: Handle Each Asset Deliberately
Raw Data, Logs, Indexes, and Evaluation Sets
Delete them on a stated schedule after the pilot ends, and confirm deletion in writing if the customer asks. These are the easiest items to remove, and keeping them rarely justifies the risk.
Evaluation sets deserve special attention. A startup may want to keep a customer’s hardest queries as test cases. Unless the agreement allows it, they belong to the customer and should go with everything else. Andrew’s AI model evaluation beta agreement guide covers who owns evaluation data.
Customer-Specific Models
A model, adapter, or configuration built only for one customer should almost always be deleted if that customer doesn’t convert.
The startup may own the weights, and many pilot agreements say so. But ownership of the weights doesn’t give the startup the right to keep using the customer’s data embedded in them. A customer-specific model has no legitimate use once the customer is gone, and serving it to anyone else would be hard to defend.
Shared Model Contributions
This is the hardest case. Once data has been used to train a shared model, removing its influence usually means retraining from a dataset without it.
If the startup had express consent and the right to keep trained models survives termination, the startup can usually keep the model. If it didn’t, it has three choices, none of them cheap:
- Retrain the shared model without the customer’s data
- Negotiate a release or license with the customer after the fact
- Keep the model and accept the legal and diligence risk
That risk shows up again in fundraising and M&A. Investors increasingly ask whether the company had the rights to the data its models were trained on. Andrew’s article on AI model ownership: weights, training data, and outputs explains why training data provenance matters in diligence.
Know-How
Engineers remember what they learned: which prompts worked, where the model failed, what the domain looks like. That general knowledge usually isn’t the customer’s property. But it’s a different thing from a saved copy of the customer’s data or prompts.
Some agreements address this with a residuals clause that lets each party use general knowledge retained in unaided memory. Enterprise customers often resist residuals clauses, and they don’t cover saved files, trained weights, or anything written down. Don’t rely on one as a substitute for a clear training term.
What the Pilot Agreement Should Say
Every AI pilot agreement should answer these questions before the pilot starts:
- What happens to each asset if the pilot ends without a purchase: raw data, logs, indexes, evaluation sets, customer-specific models, and shared model contributions
- Whether any training license survives termination, and whether it’s conditional on conversion
- How the confidentiality clause treats models and indexes, so “derived materials” doesn’t sweep them in by accident
- Deletion timing and confirmation: a set number of days, with written certification on request
- Carry-over on conversion: if the customer does buy, which assets move into the subscription so nothing has to be rebuilt
The SaaS MSA guide for AI companies covers how those carry-over terms should appear in the agreement that replaces the pilot.
Sample Clauses
These are illustrative starting points, not ready-to-sign terms. The right language depends on the product, the data, and the customer.
Treatment of Pilot Assets on Non-Conversion
If Customer does not enter into a subscription agreement within thirty (30) days after the end of the Pilot Period, Company will, within sixty (60) days thereafter, delete (a) all Pilot Data, including associated logs, embeddings, retrieval indexes, and evaluation sets, and (b) any model, adapter, or configuration trained or tuned on Pilot Data and made available solely to Customer (a “Customer-Specific Model”). On Customer’s written request, Company will certify such deletion in writing.
Survival of Training License (Where Training Was Agreed)
The license granted in Section [Training] to use Training Data to train and improve Company’s models will terminate at the end of the Pilot Period as to Training Data not yet used. Company may continue to use, license, and commercialize any model trained before the end of the Pilot Period, and Company will not be required to retrain any such model to remove Training Data. This Section survives expiration or termination of this Agreement.
Confidentiality Carve-Out
Company’s obligations to return or destroy Confidential Information apply to Pilot Data as provided in Section [Deletion] and do not require Company to delete or retrain any model trained in accordance with Section [Training]. No model, weights, or parameters will be deemed to contain or be derived from Customer’s Confidential Information solely because Training Data was used in its training, provided such use complied with this Agreement.
The second and third clauses only make sense where the customer gave real training consent. Without it, the first clause is the honest default.
Checklist: Before the Pilot Starts
- Decide whether pilot data may be used for anything beyond the pilot
- List each asset the pilot may create, and state what happens to each one if the customer doesn’t buy
- Make any training license survive termination expressly, or state that it doesn’t
- Say whether consent depends on conversion
- Align the confidentiality clause with the training and deletion terms
- Exclude personal and sensitive data from training unless the DPA and privacy law allow it
- Set deletion timelines and offer written certification
- Keep customer-specific models separate from shared models in both the contract and the architecture
- Define what carries over if the customer converts
Frequently Asked Questions
If the pilot customer doesn’t buy, do we have to delete the model we built for them?
Usually yes for a customer-specific model. It embodies the customer’s data and has no legitimate use once the customer is gone. The answer for a shared model depends on whether the customer gave training consent that survives termination.
We own the model weights under the pilot agreement. Doesn’t that settle it?
No. Owning the weights doesn’t mean the startup can keep using the customer’s data embedded in them. The data license and confidentiality terms still apply.
The pilot agreement says nothing about training. Can we keep what we trained?
That’s the weakest position. The customer will argue the data license ended with the pilot and the confidentiality clause requires destruction of derived materials. If personal data was involved, privacy law may require deletion as well.
Can we keep the customer’s queries as evaluation test cases?
Only if the agreement allows it. Without that permission, evaluation sets built from pilot data should be deleted with the rest of it.
What if the data is already in a shared production model?
The options are to retrain without it, get a release from the customer, or keep it and accept the risk. None is free, which is why the answer belongs in the pilot agreement.
Does it matter if the pilot data was de-identified?
It can help, if the de-identification meets the legal standard that applies and the model can’t reveal the underlying data. But de-identification doesn’t override a contract that limits use of the customer’s data to the pilot.
Get the Post-Pilot Terms Right Before the Pilot Starts
Andrew S. Bosin is a New Jersey technology attorney and two-time SaaS founder who represents AI and SaaS startups nationwide on a flat-fee basis. He drafts pilot, proof of concept, beta, and design partner agreements that settle, before any data comes in, what the startup can keep if the customer walks away, so a pilot that doesn’t convert doesn’t leave a problem inside the product.
He personally drafts every agreement, gives every client his personal cell phone number, and responds quickly.
Andrew S. Bosin LLC
36 Highland Road, Glen Rock, NJ 07452
📞 201-446-9643
📧 andrewbosin@gmail.com
🌐 www.njbusiness-attorney.com
Call for a consultation. Flat-fee legal services for AI and SaaS startups, nationwide.
#AISaaSLawyer #AIStartupLawyer #BestAISaaSLawyer #AIPilot #AITraining #DataRights #AIContracts #SaaSContracts
This post is provided for general informational purposes and does not constitute legal advice.