What an AI BAA actually covers

By Mat Steinlin, Head of Information Security

Last updated: July 2026

If you're building a healthcare product and want to use an AI vendor, the BAA is where the conversation usually starts. Someone in legal or security asks: "Do they have a BAA?" You confirm they do. The review moves on.

That's the right first question. It's not the last one.

A BAA (Business Associate Agreement) is a legal contract required under HIPAA before any vendor can receive, access, or process protected health information (PHI) on your behalf. Without one, sharing PHI with a third-party service (including an LLM API) is a HIPAA violation. (45 CFR § 164.308(b)(1))

But what a BAA actually establishes is more limited than most teams assume. Understanding the boundaries is what separates a team that's genuinely compliant from one that has a signed contract and an audit problem waiting to surface. As one practitioner put it: "HIPAA compliance is 20% software design and 80% processes and procedures." A BAA covers a slice of that 20%.

What a BAA legally establishes

When you execute a BAA with an AI vendor, you're creating a legally binding agreement that:

Makes the vendor a business associate under HIPAA. This is the core function. HIPAA's privacy and security rules apply to covered entities (healthcare providers, health plans, clearinghouses) and their business associates. A BAA brings the vendor into scope for those obligations. Without it, the vendor is operating outside the HIPAA framework regardless of their security posture.

Defines the vendor's permitted uses of your data. The BAA specifies that the vendor can only use PHI for the purposes you've authorized, typically to provide the service you've contracted for. Anything outside that scope, including using your data to train models, requires your explicit permission. Every major LLM vendor's enterprise BAA explicitly prohibits using your data for model training. (Anthropic BAA terms; OpenAI Enterprise terms; Microsoft DPA)

Requires the vendor to implement appropriate safeguards. The vendor must maintain administrative, physical, and technical safeguards for PHI they handle. This is enforceable.

Establishes breach notification obligations. The vendor is required to report security incidents and breaches to you within the timeframes defined in the agreement and under HIPAA's Breach Notification Rule. (45 CFR § 164.410)

Creates shared liability. Both parties have obligations. The vendor's liability is for their handling of PHI within their infrastructure. Your liability is for everything outside it.

One way practitioners frame this: "A BAA is basically just a contractual risk/liability transfer. It says nothing about their actual security practices." The contract is required and enforceable. It is not an independent assessment of how the vendor has configured their systems.

What a BAA requires of you

The flow of obligations runs in both directions. A BAA doesn't just bind the vendor, it binds you too.

Most vendor BAAs include a "Customer Obligations" section that requires you to: use PHI only as permitted by HIPAA; provide the vendor with only the minimum necessary PHI; notify the vendor of any restrictions on PHI use that may affect them; and comply with any applicable implementation guides the vendor has published as a condition of the BAA.

Anthropic's BAA, for example, explicitly requires compliance with their Implementation Guide for HIPAA Entities, and states that data submitted outside the terms of that Guide is not treated as PHI under the BAA (meaning Anthropic has no HIPAA obligations around it). Implementation guides typically specify: prohibited uses (what the service cannot be used for), configuration requirements (authentication, access controls, session management), restrictions on third-party integrations, and workforce training obligations.

Before you assume your BAA provides complete coverage, read what it requires of you. The vendor-specific chapters in this guide cover the customer obligations for each provider where that information is publicly available or accessible through signed agreements.

What a BAA does not cover

This is where the gaps appear.

A BAA covers the vendor's handling of PHI within their infrastructure from the moment they receive it. It does not cover what happens in your system before the request is sent, or after the response is returned. "HIPAA compliance is on you, not the service. It's not a switch you turn on."

Your audit logging. HIPAA's security rule requires audit controls: mechanisms to record and examine activity in systems that contain or use PHI. (45 CFR § 164.312(b)) No LLM vendor generates HIPAA-compliant audit logs of your application's interactions with their API. That logging is your responsibility to build and maintain. If you cannot produce a log showing who sent what PHI to which model and when, you have an audit control gap. Your vendor's interface may generate some logs by default. Those must either be disabled or stored in a HIPAA-compliant system; that configuration decision belongs to your team, not the vendor.

Storage of those logs. The logs you generate contain PHI. They must be encrypted at rest (45 CFR § 164.312(a)(2)(iv)), access-controlled, and retained for six years (45 CFR § 164.316(b)(2)). This is the most common failure point: teams correctly log every LLM interaction, then ship those logs to an observability stack designed for operational telemetry, not regulated data. The logs end up in unencrypted storage with broader access than required.

PHI handling before the model. The BAA covers PHI after your request arrives at the vendor's infrastructure. What happens before then (what you put in the prompt, how you construct the context, whether PHI is present at all) is your responsibility. De-identification before the prompt and re-identification after the response is not required by the BAA, but it reduces your exposure if controls elsewhere fail. See PHI de-identification for the technical framework.

Key management. The BAA establishes that the vendor will handle your data securely on their end. How you manage the API keys used to access that vendor (scoping by environment, rotation schedule, revocation procedures) is your responsibility. Compromised API keys are your incident, not the vendor's. See Key management.

Access controls. HIPAA requires that access to PHI be limited to those who need it for authorized purposes (the minimum necessary standard). (45 CFR § 164.502(b)) Which users and systems in your organization can send PHI to an LLM, and under what conditions, is your decision to make and enforce.

Other vendors in your stack. Every system that touches PHI needs appropriate controls and, where applicable, its own BAA. Your database, logging pipeline, observability stack, data warehouse, and any downstream system that handles PHI are each in scope independently.

The common failure pattern

Here is how this plays out in practice.

A team ships a healthcare AI feature. They've done the right things: signed a BAA with their LLM vendor, restricted API access to production systems, and built logging so they can see every prompt and response. The security review passes.

The failure happens downstream.

Those logs get shipped to the team's existing observability platform, a tool designed for application telemetry, not regulated data. In that system:

• Logs are stored in plaintext or with encryption that doesn't meet HIPAA requirements

• Access is granted broadly to engineers and support staff, giving developers full access to unredacted PHI in logs for debugging purposes is itself a HIPAA violation, separate from any storage issue

• Retention is set to 30 or 90 days, nowhere near the six-year HIPAA requirement

• Data gets copied to development environments for debugging

The BAA with the LLM vendor is valid. The interaction with the model is compliant. But the system storing and surfacing that same data is not. The compliance boundary shifted the moment the logs landed in a system without appropriate controls.

The signed BAA offers no protection here. It covers what the vendor does with PHI. Not what you do with it afterward.

There is a related failure that appears earlier in the development process: founders sometimes skip BAAs for third-party services entirely because compliance "seemed inconvenient" at the time. The BAA with the LLM vendor gets signed; the logging platform, data warehouse, or internal analytics tool does not. Each gap is its own violation.

The infrastructure layer

Every vendor chapter in this guide concludes with the same section, because the requirement doesn't change based on which vendor you use.

HIPAA's technical safeguards apply to any system that handles PHI. (45 CFR § 164.312) When your application sends patient data to an LLM, those safeguards apply to the full interaction chain.

What you need regardless of vendor:

Security Risk Analysis. Before any of the controls below can be considered sufficient, HIPAA requires a documented Security Risk Analysis (45 CFR § 164.308(a)(1)(ii)(A)) (an assessment of the risks and vulnerabilities to PHI in your environment, and documentation of why your controls adequately address them). This is the umbrella process that ties all other controls together. It's also the single most commonly cited failure point in actual OCR enforcement settlements. A team can have audit logging, encryption, and access controls in place and still fail a HIPAA audit if they cannot document the analysis that demonstrates why those controls are sufficient for their specific AI vendor relationship. The Risk Analysis must be updated when you add a new AI vendor, change your data flows, or make significant changes to your environment.

Audit logging. Every prompt and response involving PHI, logged with timestamps, user attribution, and model identity. The vendor does not generate this for you.

Log retention and access controls. Logs containing PHI must be retained for six years and access-controlled so only authorized personnel can view them. Retention is typically the harder problem; encryption at rest is often provided by default by cloud storage services, but maintaining six years of logs in a queryable, access-controlled system requires explicit infrastructure decisions. Broad developer access to logs containing PHI is a HIPAA violation even when encryption is in place.

PHI de-identification (defense in depth, not required). De-identification is not required by HIPAA or by any vendor BAA; if you have a valid BAA in place, sending PHI to an LLM is a permitted use. The more pressing concern is often PHI in logs: if unredacted PHI flows into logs that engineers access for debugging, you have a HIPAA violation regardless of what the model received. HIPAA requires access to PHI on a least-privilege, need-to-know basis. Broad developer access to logs containing PHI violates that requirement even when the model interaction itself was fully covered.

Key management. API keys scoped by application, environment, and team. Rotation and revocation procedures in place.

Access controls. Enforcement of who can send PHI to which models under which conditions. Documented and auditable.

For teams who don't want to build this infrastructure themselves, Aptible LLM Gateway provides it as a managed layer across all supported LLM vendors, with one Aptible BAA covering the full interaction chain.

FAQs

Do I need a BAA with every AI vendor I use?

Yes. Every vendor that receives, accesses, or processes PHI on your behalf requires a BAA before you send any PHI. This includes LLM API providers, logging infrastructure, observability tools, and any downstream system that handles PHI. A single BAA with your primary LLM vendor does not cover other systems in your stack.

Does a BAA mean the AI vendor is HIPAA compliant?

It means they've agreed to operate as a compliant business associate for the PHI you share with them. Your full implementation, including the infrastructure your team builds, is what determines whether your system is compliant. The vendor's compliance posture and your system's compliance are separate questions.

Microsoft makes this explicit in their public documentation: "having a BAA with Microsoft does not ensure your organization's compliance with HIPAA." Google structures HIPAA as a shared responsibility between Google and the customer. The major cloud providers are telling you directly that a signed BAA is a starting point, not a finish line.

What happens if I send PHI to an AI tool without a BAA?

It's a HIPAA violation. Depending on the circumstances, this falls under HIPAA's tiered penalty structure. The statutory minimums are outdated; the 2026 inflation-adjusted values range from $145 per violation for unknowing violations to $73,011 per violation for willful neglect. For willful neglect not corrected within 30 days, penalties are no less than $50,000 per violation and no more than $1,500,000 for identical violations in a calendar year, with a maximum adjusted exposure per incident of $2,190,294 (45 CFR § 160.404). In the event of a breach, the absence of a BAA is an aggravating factor. More practically, it makes your organization liable for the vendor's handling of that data without any contractual protections.

Can I use consumer AI tools with PHI?

No. Consumer tiers of Claude.ai, ChatGPT, Gemini, and similar products do not offer BAAs. They are not designed for regulated data. The enterprise tiers of these products (Claude for Enterprise, ChatGPT Enterprise, Vertex AI) are the covered offerings. See the individual vendor chapters for detail.

Is a BAA the same as HIPAA certification?

There is no such thing as HIPAA certification. HIPAA is a federal law, not a certification program. A BAA is a contractual agreement. Compliance is determined by implementation, audits, and, when violations occur, OCR enforcement. Vendors who claim to be "HIPAA certified" are using a term that doesn't exist in the regulation.

HHS does not certify cloud service providers as HIPAA compliant. The only formal vendor designation under HIPAA is "HIPAA eligible": the vendor is willing to sign a BAA and will operate as a business associate. That is a legal status, not a security assessment.

What's next

If you're evaluating a specific vendor, the chapters below go deep on each one's BAA process, what's covered, and what isn't:

• Claude (Anthropic)

• OpenAI / ChatGPT

• Gemini / Vertex AI

• Azure OpenAI

• AWS Bedrock

For day-to-day tools your team uses beyond LLM APIs, see HIPAA-compliant AI tools for healthcare teams.

__

For the full technical requirements for HIPAA-compliant LLM usage, see HIPAA-Compliant AI: What Developers Need to Know. For BAA requirements broadly, see What is a HIPAA BAA.