OMB Issues AI Acquisition Guidance
A recent guidance memorandum from the
Office of Management and Budget establishes AI-related contract requirements.
By Stephen L. Bacon
During the past
several years, Congress and the Biden administration have taken steps to ensure
that the federal government can harness the extraordinary potential of
artificial intelligence (AI) while managing the unique risks it poses. These
efforts led to the issuance of Office of Management and Budget (OMB) Memorandum
M-24-18, “Advancing the Responsible Acquisition of Artificial Intelligence in
Government,” on September 24, 2024.
M-24-18
will begin to shape the acquisition of responsible AI systems by prescribing
certain requirements that agencies must include in AI-related contracts.
Building upon previous OMB guidance issued in March 2024, M-24-18 places
particular emphasis on managing risks associated with contracts for
“safety-impacting AI” and “rights-impacting AI.” M-24-18 directs agencies to
impose new requirements in contracts for “safety-impacting AI” and
“rights-impacting AI” by December 1, 2024, and in solicitations for all other
AI-related contracts no later than March 23, 2025.
M-24-18
is a first step towards establishing a standard framework for AI-related
contract requirements. Contractors and agency acquisition professionals should
expect that these requirements will continue to evolve and expand as AI
adoption and innovation continues to accelerate.
Background
In late 2022,
Congress passed the Advancing American AI Act to encourage responsible use of
AI in agency programs and initiatives. (1) The law required the Director of OMB
to “develop an initial means by which to…ensure that contracts for the
acquisition of an artificial intelligence system or service” address certain
matters including “protection of privacy, civil rights, and civil liberties”
and the “ownership and security of data and other information.” (2)
On October 30, 2023,
President Biden issued Executive Order 14110 that established guiding
principles and priorities for the development and use of AI. Executive Order
14110 instructed the Director of OMB to “issue guidance to agencies to
strengthen the effective and appropriate use of AI, advance AI innovation, and
manage risks from AI in the federal government.” (3)
OMB issued that
guidance on March 28, 2024 in Memorandum M-24-10, “Advancing Governance,
Innovation, and Risk Management for Agency Use of Artificial Intelligence.”
M-24-10 included new requirements and recommendations to agencies regarding how
to “address specific risks from relying on AI to inform or carry out agency
decisions and actions, particularly when such reliance impacts the rights and
safety of the public.” (4)
OMB Memorandum M-24-10
Among other
requirements, M-24-10 directed agencies to apply certain minimum risk
management practices for “safety-impacting AI” and “rights-impacting AI” by
December 1, 2024. OMB’s guidance defines “safety-impacting AI” and
“rights-impacting AI” and specifies certain uses that are presumed to impact
rights and safety.
Before agencies may
use “safety-impacting AI” or “rights-impacting AI,” M-24-10 requires them to
complete an “AI impact assessment” of benefits, risks, and the quality of data
used in the AI design and training; conduct adequate testing of the AI in its ‘intended
real-world context;’ and perform an independent agency evaluation of the AI.” (5)
Once the AI is in use, agencies must conduct ongoing monitoring of the AI
including periodic human reviews to assess benefits and risks including
emerging risks to rights and safety.
M-24-10 also
prescribes several additional practices that apply specifically to
“rights-impacting AI.” This includes an assessment and ongoing monitoring of
the AI’s impact on equity and fairness, and mitigation of risks associated with
potential algorithmic discrimination. In addition, agencies are required to
consult and incorporate feedback from the public and affected communities
regarding the design, development, and use of AI.
Agencies must also
provide notice to individuals when the use of AI results in an adverse decision
or action that impacts their rights. And, where practicable, agencies must
include an appeal or secondary human review process for AI-based decisions and
provide a mechanism for individuals to “opt out” of the AI functionality in
favor of a human alternative.
OMB Memorandum M-24-18
M-24-18 expands upon and implements the guidance
in M-24-10 by establishing requirements applicable to AI-related contracts.
These requirements do not apply to the intelligence community or to AI acquired
for use in a National Security System but are otherwise applicable to AI
systems or services acquired by federal agencies.
M-24-18 defines “AI systems” broadly to include data
systems, software, applications, tools, or utilities that integrate AI
functionality. However, the definition specifically excludes “any common
commercial product within which artificial intelligence is embedded, such as a
word processor or map navigation system.” (6)
OMB’s guidance also does not govern certain limited uses
of AI including, most notably, “AI used incidentally by a contractor during
performance of a contract (e.g., AI used at the option of a contractor when not
directed or required to fulfill requirements).” (7)
M-24-18 recognizes that agencies must understand when AI
is being acquired to effectively manage the risks and performance of AI
systems. Toward this end, agencies are supposed to “consider requirements
language asking vendors to report any proposed use of AI as part of their
proposal submission” or in the integration of new features or components after
award. (8)
These disclosure obligations are likely to present
difficult questions for contractors and agencies including whether the use of
certain features or components constitute a covered AI system, “incidental use”
of AI by the contractor, or qualify under the “common commercial product”
exemption.
If an acquisition involves AI covered by M-24-18, the
applicable requirements will depend on the type of AI acquired. Some
requirements are generally applicable to all AI acquisitions, while others
address specific concerns associated with “safety-impacting AI,”
“rights-impacting AI,” AI-based biometrics, and “general use enterprise-wide
generative AI.”
General AI Acquisition Practices
M-24-18 generally requires agencies to leverage
performance-based acquisition techniques to proactively understand and evaluate
potential risks and benefits of AI systems prior to contract award.
Agencies must
establish performance-based requirements that include adequate safeguards
against inaccurate outputs and other AI risks, and that ensure the system or
service will be appropriate for its intended use.
OMB’s guidance
emphasizes the need for agencies to carefully negotiate intellectual property
(IP) licensing rights to accomplish the government’s long-term objectives while
preventing vendor “lock-in.”
Moreover, contracts
must include systems and procedures for data management, and “permanently
prohibit the use of inputted agency data and outputted results to further train
publicly or commercially available algorithms.” (9) Agencies must also require
disclosure when copyrighted materials are used for training data, and may
require disclosure of synthetic or third-party data used for that purpose.
M-24-18 instructs
agencies to include contractual requirements to obtain documentation that is
needed to understand the contractor’s model training and integrity. In
addition, contracts should mandate compliance with data protection
requirements, including software controls for privacy and security.
Practices for Rights-Impacts and Safety-Impacting AI
Contracts for
“rights-impacting” and “safety-impacting” AI must require contractors to
disclose information and documentation to agencies about the AI’s functionality
and design including, for example, real-world performance metrics, as well as
training, testing, and validation data.
Contractors should understand that certain information
disclosed regarding the AI’s functionality may become publicly available in the
agency’s AI use case inventory published online.
AI-related contracts
must also contain terms that permit agencies to regularly monitor and evaluate
the AI’s performance and risks. These terms must allow agencies sufficient time
and access to conduct independent evaluations of the AI using agency validation
and training datasets.
Contracts should
specify the applicable procedures and frequency of examination, testing, and
validation of AI systems, and require contractors to provide the results to the
government. If problems are discovered, contractors must implement corrective
measures, including model retraining as needed.
Contractors will be
required to disclose “serious AI incidents and malfunctions of the acquired AI
system or service within 72 hours, or a timely manner based on the severity of
the incident, after the vendor reasonably believes the incident occurred.” (10)
The definition of a “serious AI incident or malfunction” will be determined at
the agency level, but may include “unexpected malfunctions, or unintended
outcomes that directly result in harms to rights or safety.” (11)
To implement M-24-10’s
requirements specific to the use of “rights-impacting AI,” contracts must
specify the actions “needed to support agency plans for notifying individuals
when the use of AI results in an adverse decision…or providing those individuals
with an opportunity to appeal.” (12) The contract also must require the
contractor to provide additional access, information or documentation the
agency needs to implement applicable notice and appeal procedures.
AI-Based Biometrics
Contracts for
AI systems that are used to identify individuals using biometric identifiers
(e.g., faces, fingerprints, etc.) must require verification that those systems
“are not trained on data collected in violation of applicable law or federal
policy, and that such systems are sufficiently accurate to support reliable
biometric identification and verification across different groups based on the
results of testing and evaluation in operational contexts.” (13) These systems
also must meet certain properties specified in the guidance to ensure their
quality and reliability.
Practices for Generative AI
OMB-24-18 requires implementation of additional
practices for acquisitions of “general use enterprise-wide generative AI,”
which refers to generative AI acquired for general purposes (e.g., workplace
productivity) and acquired for use by end users in more than one agency
component or through a contract that accommodates more than one organizational
component.
Under such contracts, contractors will be required to
provide transparency regarding generated content, mitigate the risk of
inappropriate use, and provide documentation to cover evaluation, testing, and
red-teaming performed on content.
Implementation Timeline and Potential Challenges
The guidance
in M-24-18 will apply to any contracts awarded under a solicitation issued on
or after March 23, 2025, and to any option to renew or extend the period of
performance for an existing contract that is exercised after that date.
However, M-24-18 includes an accelerated implementation timeline for contracts
involving “safety-impacting AI” and “rights-impacting AI.”
Agencies were required
to first identify all such contracts by November 1, 2024. Also, no later than
December 1, 2024, all existing and new contracts must incorporate the
applicable requirements discussed above that are specific to “safety-impacting
AI” and “rights-impacting AI.”
Because M-24-18
directs changes to existing contracts, agencies and contractors will
need to negotiate the specific terms and pricing adjustments to be included in
a contract modification. If the changes cannot be negotiated in a bilateral
modification, it is conceivable that agencies could direct contractors to
implement the new requirements pursuant to the contract’s “Changes” clause.
Moreover, if
contractors cannot or will not implement the new requirements, agencies may
resort to terminating the contract for convenience or potentially default.
Indeed, OMB’s guidance clearly states that “[a]gencies must cease use of
AI systems and services that impact rights or safety in cases where required
risk management practices cannot be sufficiently implemented, as determined by
the agency.” (14)
Conclusion
Although M-24-18 is the government’s “initial
means” for establishing AI-acquisition guidance, it is very likely just a
steppingstone to future updates to the Federal Acquisition Regulation (FAR) and
agency FAR supplements that will establish formal rules and standard
contract clauses for AI-related contracts. Thus, as the use of AI continues to
grow and evolve, contractors and agency acquisition professionals will need to
continuously monitor developments in the requirements applicable to these
contracts. CM
Stephen L. Bacon is a shareholder
in the Washington, D.C. office of the law firm Rogers Joseph O’Donnell, where
he represents government contractors in bid protests, claims, investigations,
and suspension and debarment proceedings. He frequently litigates cases at the
Court of Federal Claims, the Government Accountability Office, the Boards of
Contract Appeals, and the Small Business Administration’s Office of Hearings
and Appeals. He also provides advice and counsel to clients on a broad range of
contractual and regulatory compliance issues that confront government
contractors.
The views expressed in this article are those of the
author and do not necessarily reflect the views of Rogers Joseph O’Donnell or
its clients. This article is for general information purposes and is not
intended to be and should not be construed as legal advice.
ENDNOTES
1 Pub. L.
No. 117-263, div. G, title LXXII, subtitle B, § 7222(1).
2 Id. §
7724(d)(1)(A).
3 E.O.
14110, § 10.1(b).
4 OMB
Memorandum M-24-10 at 2.
5 Id. at
17-19.
6 OMB
Memorandum M-24-18 at 4.
7 Id.
8 Id. at
9.
9 Id. at
13.
10 Id. at
17.
11 Id.
12 Id. at
18.
13 Id. at
10.
14 Id. at
14 (emphasis added).