AI has reshaped the business world and changed how government agencies accomplish their missions. Artificial intelligence provides solutions to many public sector challenges by automating routine tasks, making better decisions, and cutting down risks.
Artificial intelligence can reshape the scene of government operations if used ethically in procurement. AI systems can analyze large datasets to review suppliers based on cost, quality, and compliance. These systems predict demand trends and spot suspicious activities that might signal fraud. The training data’s existing societal biases can reflect and magnify in AI systems.
The responsible use of AI in government needs careful review. The U.S. government must adopt these opportunities to stay competitive, while ensuring fairness, transparency, and accountability that protect public institution integrity. Public bodies want to achieve these goals by using procurement to set standards for ethical development and machine learning implementation in government.
This piece outlines the key steps to ethically procure AI solutions for government use. We’ll explore AI’s role in public services and build internal capacity for responsible implementation.
Step 1: Understand the Role of AI in Government Services
Image Source: cef-see.org
AI is reshaping how governments deliver services and meet public needs, even though adoption remains in early stages. A 2023 Bloomberg Philanthropies survey reveals that all but one of these local governments are learning about AI’s potential, while just 2% currently use it. Before making any procurement decisions, understanding how AI works in government settings is vital.
How AI supports public sector goals
Government agencies now see AI as a solution to maintain service levels while controlling costs, especially with tight budgets and staff shortages. AI-powered government operations could boost public administration productivity by up to 3% and increase real GDP by up to 4% by 2035.
AI helps the public sector through several mechanisms:
- Operational efficiency: AI automates routine tasks so the core team can focus on higher-value work. Government employees who use AI save about 3.25 hours weekly. These time savings add up to major productivity gains across organizations.
- Better decision-making: AI analyzes large datasets to find patterns that help improve service delivery. To name just one example, 58% of cities are learning about generative AI for data analysis, while 76% use it for data-driven policymaking.
- Cost reduction: AI automation and targeted services could reduce fiscal deficits by up to 22% by 2035 compared to baseline projections. This includes efficiency savings and better compliance in areas like tax collection.
- Improved citizen experience: AI powers 24/7 self-service options that citizens now expect. Almost 98% of governments believe citizens prefer to interact through new technologies. Digital transformation has become essential to meet public expectations.
Examples of AI in local government and federal agencies
Local governments across America show tangible benefits from their AI solutions:
Pittsburgh uses an AI-powered traffic management system that provides immediate analysis of key intersections. Traffic managers can adjust signals to optimize vehicle flow and reduce idle time. This supports the city’s goal to cut transportation emissions 50% by 2030.
Phoenix created myPHX311, a web portal and app that answers common questions in Spanish and English. The system connects residents to city agencies for water service requests, graffiti reporting, and streetlight outage notifications.
Washington, DC’s AI system helps city workers inspect 1,800 miles of sewer pipes. This automation cut reporting time from 75 minutes to just 10 minutes per hour of video inspection.
Wilmington, Delaware used AI to send targeted ads on social media platforms that prompted residents to pay overdue bills. The city recovered $1.10 million in unpaid water bills while avoiding uncomfortable collection calls for employees.
Federal agencies are adopting AI faster than ever. By December 2024, they reported over 1,700 AI use cases. The Department of Health and Human Services leads with 271 use cases. The Department of Homeland Security has added DHSChat, an internal agency chatbot, to its growing AI implementations.
These agencies use AI for various purposes. The top three categories are mission-enabling internal support, health and medical applications, and government services including benefits delivery. Defense and intelligence agencies use AI for computer vision, graph analysis, and deep neural networks to spot suspicious activities in large datasets.
AI brings the most value when organizations treat it as a foundation across departments rather than isolated pilot projects. Government organizations can tap into the full potential of AI by starting with quick wins while building momentum and investing in infrastructure, data quality, and workforce skills.
Step 2: Identify Ethical Risks Before Procurement
Image Source: SoluLab
AI tool procurement needs more than just understanding their benefits. Government agencies must spot ethical risks that could break public trust and cause harm before signing contracts. We’ve seen what happens when these risks get ignored – just look at Google’s withdrawal from Project Maven in 2018 when employee protests over ethical issues blindsided their procurement process.
Recognizing bias and fairness issues
AI systems can copy and increase existing societal biases when trained on historical data. These algorithms become more than just math formulas in government settings – they turn into moral actors that affect citizens’ lives. Several cases show us these risks clearly:
- The COMPAS recidivism algorithm unfairly labeled Black defendants as high-risk, which raised serious questions about fair sentencing
- Detroit police wrongfully arrested a Black man in 2020 because of bad facial recognition matches. This showed how the technology makes more mistakes with people of color
- A healthcare algorithm didn’t properly assess Black patients’ needs because it looked at costs instead of actual health outcomes
Bias can sneak into AI systems in many ways. It happens through training on skewed data, choosing efficiency over fairness, and feedback loops that keep disadvantages going. What’s fair isn’t simple either – different situations need different fairness measures, and you can’t satisfy all of them at once.
Agencies should use structured frameworks like NIST’s AI Risk Management Framework to deal with these issues. They also need review councils to check high-impact systems before launch and development teams that bring different viewpoints to the table.
Assessing data quality and privacy risks
Data quality makes or breaks AI systems – state government CDOs know this well. Hawaii’s CDO put it simply: “Data quality is the biggest challenge for data and AI across the board”.
Bad data leads straight to biased or wrong AI results – it’s classic garbage in, garbage out. Common issues include:
- Old or incomplete information that messes up service delivery and policy decisions
- Systems that don’t work well together because they use different standards
- Some groups get left out of the data, so the AI doesn’t work equally for everyone
AI systems also create privacy and security weak spots. Government agencies have lots of sensitive citizen data, which makes them prime targets for attacks. The biggest privacy risks are:
- Data leakage: AI models might accidentally reveal sensitive stuff like patient records, credit card numbers, or where people live
- Consolidation vulnerabilities: Putting data from different agencies in one place creates an attractive target. Hackers only need to break into one system instead of four
- Unchecked surveillance: AI-powered analysis of surveillance data might end up targeting certain communities unfairly[123]
Procurement processes must check data quality, review privacy impacts, and require strong security measures to reduce these risks. Virginia’s CIO stressed this point – you need reliable, high-quality data for ethical, responsible, and clear AI results.
A good risk review during procurement helps agencies avoid expensive mistakes and keep public trust. One procurement officer summed it up well: “Every AI contract needs transparency, auditability, explainability, impact assessments, change management strategies, crisis response plans, and business continuity plans”.
Step 3: Evaluate AI Vendors and Their Claims
Government agencies looking to buy AI solutions must guide through marketing claims to find vendors that deliver truly responsible technology. The next vital step after identifying ethical risks involves a full assessment of AI providers and their often lofty promises.
What to ask about ‘AI-powered’ tools
Vendors often market products as “AI-powered” with minimal transparency about how these systems actually work. Public interest protection requires procurement officers to ask specific questions about these technologies:
Data ownership and usage: Almost 92% of AI vendors claim broad data usage rights beyond what’s needed for service delivery—this is a big deal as it means that the market average of 63%. Agencies should ensure they keep ownership and control of their data when working with third-party providers. Ask whether vendors use customer data to train their models; some platforms like AWS Bedrock clearly state they never use customer data for model training.
Security and compliance measures: Security stands paramount for public agencies, so verify that vendors build security guardrails into their AI products. Ask about encryption, access controls, data residency policies, and compliance certifications. The vendor should provide Transparency Notes—documents explaining how their AI technologies work, including capabilities and limitations.
Risk management documentation: Documentation should show the vendor’s approach to managing risks at the model, system, and application levels. All the same, agencies shouldn’t require vendors to disclose sensitive technical information like model weights. The Office of Management and Budget suggests that agencies must request four distinct data sets for minimum transparency, including the acceptable use policy and information about the model.
Verifying transparency and explainability
AI systems’ ability to understand and explain their decisions is vital for public agencies bound by transparency requirements.
Assessment frameworks: Public administrators can use vendor transparency rubrics to determine what specific information to request and assess the quality of information provided during the formal solicitation process. Transparency upstream in AI procurement leads to better public transparency downstream after deployment.
Transparency standards: High-impact AI systems require enhanced documentation covering:
- Risk management practices
- Impact assessments
- Test environments
- Human oversight mechanisms
- Performance monitoring approaches
Evaluate explainability tradeoffs: Complex AI models often create a tradeoff between accuracy and explainability. A Tax AI data scientist explained: “Even as a data scientist, when we run a neural network, we still don’t really fully understand what is happening inside… random forest is much easier because we can visualize the decision tree and show how the decision is made”.
Request model disclosure formats: AI systems affecting citizen rights or safety need vendor verification about notifying affected populations about AI use. They should provide mechanisms to contest decisions and allow individuals to opt out of automated processes. Neither agencies nor the public can properly assess accuracy or reliability without clear documentation of these systems’ operation and data dependencies.
RFP response evaluation needs a diverse committee with expertise in AI, ethics, and procurement. Standardized scoring rubrics help assess AI-related risks, benefits, and compliance to ensure fair and objective evaluation. Checking vendor references and examining similar work products provides valuable ground evidence of a vendor’s capabilities.
Note that AI vendor contracts do more than just serve as legal agreements—they actively shape AI governance, liability structures, and compliance standards long before formal regulations take effect. Government agencies can set a precedent for responsible AI use that benefits both the public sector and society at large through careful evaluation and thoughtful procurement practices.
Step 4: Draft Contracts with Ethical Safeguards
Government agencies must take a completely different approach when contracting AI systems compared to regular technology procurement. AI’s growing presence in government operations means contracts need specific ethical safeguards against unique risks.
Mandating algorithmic transparency
Government contracts for AI systems should specify clear transparency requirements. The Office of Management and Budget’s Memorandum M-25-22 tells agencies to address AI system transparency in contracts, which might involve protecting proprietary data. These terms must clearly state what vendors should reveal about their algorithms and decision-making processes.
Vendors must meet these transparency requirements:
- Document AI system purposes, capabilities, and limitations
- Provide information about data sources and model development processes
- Explain how the system produces outputs that affect citizens
- Maintain audit logs of system decisions and changes
Agencies should ask for detailed documentation about risk management practices, impact assessments, and performance monitoring approaches for AI systems with significant impact. These rules help agencies learn about AI tool operations without exposing sensitive technical details like model weights.
Including audit rights and data usage terms
Contracts must address data usage and audit rights alongside transparency requirements. The terms should describe ownership and intellectual property rights for both parties. These rules should stop vendors from using nonpublic agency data to train public or commercial AI algorithms unless the agency agrees.
Government data protection rules must cover:
- Data use that follows applicable laws and policies
- Clear guidelines on using, accessing, and keeping government data
- Limits on how vendors use government data beyond service delivery
- Privacy protections, especially for personal information
Michigan showed this approach after replacing a faulty unemployment insurance fraud detection system. The new vendor agreed to a “source code escrow” provision that let an independent auditor check the system’s accuracy. This shows how audit rights protect against system failures.
Setting accountability and human oversight clauses
Human oversight throughout AI development and use helps prevent harm and ensures fairness in government programs. Contracts should create ways for meaningful human involvement in automated processes.
Contracts should include these oversight elements:
- Rules for human review of important decisions
- Clear responsibility for AI system decisions
- Ways for citizens to challenge automated decisions
- Steps for ongoing monitoring and assessment
The Office of Management and Budget requires agencies to include “ongoing testing and monitoring of performance, risks, and effectiveness of an AI system or service” in their contracts. This rule ensures systems stay aligned with government goals and ethical standards.
to learn if your agency’s contract templates have these ethical safeguards before your next AI purchase.
Section 6602 of recent legislation requires intelligence agency leaders to “track and evaluate performance of procured and agency-developed artificial intelligence, including efficacy, safety, fairness, transparency, accountability, appropriateness, lawfulness, and trustworthiness”. Adding these elements to contracts helps agencies follow new requirements and protect public interests.
Step 5: Monitor AI Systems Post-Implementation
Successful deployment of AI in government marks just the beginning—ongoing monitoring ensures these systems remain effective, ethical, and compliant throughout their lifecycle. The General Services Administration (GSA) emphasizes that AI systems must meet ethical, legal, and technical standards not only before but also after implementation.
Tracking performance and compliance
Post-implementation monitoring requires systematic approaches to evaluate AI systems’ continued effectiveness. GSA has established continuous monitoring protocols that track AI system interactions at the network level and is developing strategies to increase capacity for monitoring system behaviors and performance. Likewise, the Government Accountability Office (GAO) identifies “monitoring” as a critical practice, calling for agencies to ensure AI systems remain reliable and relevant over time.
Key monitoring requirements include:
- Documenting results of monitoring activities and corrective actions to promote traceability and transparency
- Establishing acceptable ranges for data and model drift to maintain system reliability
- Implementing automated alerts and reporting systems to detect deviations from compliance standards
- Conducting regular AI risk assessments, especially for high-impact systems
Should a high-impact AI system be found non-compliant after deployment, GSA has developed a defined termination process involving revoking system access, ceasing operations, and securing processed data. Furthermore, an incident response team coordinates shutdown procedures and conducts post-termination reviews to assess impact and identify corrective actions.
Using AI CoE (Center of Excellence) for governance
The AI Center of Excellence (CoE) plays a vital role in government-wide AI governance and implementation. As Krista Kinnard from the AI CoE explains, they embed “a clear and transparent human-centered approach into all AI engagements”. The CoE helps agencies develop infrastructure to support regular evaluation of data, models, and outcomes to assess both intended and unintended impacts.
Presently, the Partnership for Public Service is advancing these efforts through their AI Center for Government, which holds working sessions with leaders from agencies like the Government Accountability Office and IRS to apply enterprise risk management practices to federal AI initiatives.
Responding to public concerns and audits
Public trust remains fundamental to AI adoption in government. Agencies must implement mechanisms for citizens to report concerning outputs from AI systems. GSA policy requires updated processes that include reporting pathways for agency users of AI tools that violate established principles.
AI observability tools have become essential in addressing public concerns by reducing hallucinations in AI outputs, mitigating risks through transparency in decision-making, addressing disparities, and optimizing costs. As AI adoption increases, a new industry focused on AI assurance, certification, and compliance is emerging.
Third-party assessments and audits are increasingly recognized as crucial for ensuring AI systems remain responsible, equitable, traceable, reliable, and governable. Some experts advocate for “watchdog AIs” or “AI auditors” that test, verify, and monitor other AI models in real-time.
Step 6: Build Internal Capacity for Responsible AI Use
Responsible AI adoption needs a capable government workforce as its foundation. Agencies must invest in their people after implementing systems to ensure ethical and effective AI use.
Training staff on AI tools and ethics
Government employees need specialized training to grasp AI capabilities, limitations, and ethical considerations. The General Services Administration cooperates with the Office of Management and Budget to deliver the AI Training Series. Over 12,000 government employees registered in 2024—a 41% increase from the previous year. This complete program has three distinct tracks:
- Acquisitions: Understanding risk management and ethics in AI procurement
- Leadership and Policy: Learning about AI policy development and ethical leadership
- Technical: Breaking down complex AI concepts into plain language
Agencies should make training a priority that emphasizes core ethical principles. These principles include fairness, transparency, accountability, privacy, safety, and human oversight.
Lining up with federal AI frameworks and EO 14110
Executive Order 14110 mandates “chief artificial intelligence officer” positions across major federal agencies. These officers must work through the Chief Artificial Intelligence Officer Council (CAIOC). CAIOC serves as the main venue for interagency coordination on AI adoption.
The National AI Initiative Act guides agencies to work together toward a coherent AI strategy. Agencies must also update their internal acquisition procedures to ensure AI use meets federal guidelines.
Promoting cross-agency collaboration
The AI Center of Excellence (AI CoE) brings agencies together to advance collaboration and enhance workforce capacity. The AI Readiness Project expands this effort through the State Chief AI Officer Community of Practice. Government leaders use this platform to share lessons and create tools for responsible AI.
This community leads three national working groups that focus on immediate priorities:
- Creating practical guardrails for emerging AI systems
- Looking at AI’s effects on the public-sector workforce
- Building shared frameworks for AI evaluation and monitoring
Agencies should promote information sharing through structured communities of practice while developing their internal AI expertise.
Conclusion
Government agencies have a vital duty to procure AI systems ethically while modernizing their operations and keeping public trust. We’ve outlined six key steps that help create a framework for responsible AI adoption in the public sector. The first step requires agencies to understand how AI fits into public services. They must then spot potential ethical risks before starting procurement. A detailed vendor evaluation protects against overblown claims and keeps the process transparent.
Well-crafted contract language acts as the cornerstone of accountability. It makes ethical safeguards non-negotiable parts of AI procurement. Agencies should to check if they’re ready for AI implementation. This helps identify gaps in their procurement process before moving ahead. The work doesn’t stop after implementation. AI systems need constant evaluation to work as intended and meet ethical standards.
Staff training and cooperation between agencies builds the human foundation needed for responsible AI governance. This all-encompassing approach balances new technology with ethical responsibility. It serves the public interest while reducing potential risks. Government agencies that use these procurement best practices protect themselves from ethical issues. They also tap into the full potential of AI benefits for public service delivery.
AI reshapes how governments work. A careful procurement approach builds systems that stay fair, transparent, and accountable. AI offers great possibilities to improve government services. Yet only ethical implementation will deliver on this promise while keeping the public trust that democratic institutions need.
Key Takeaways
Government agencies must approach AI procurement strategically to harness transformative benefits while protecting public trust and ensuring ethical implementation.
• Identify ethical risks early: Assess bias, fairness issues, and data quality problems before procurement to prevent costly mistakes and maintain public trust.
• Demand vendor transparency: Ask specific questions about data usage, security measures, and algorithmic explainability rather than accepting vague “AI-powered” marketing claims.
• Include ethical safeguards in contracts: Mandate algorithmic transparency, audit rights, human oversight clauses, and clear accountability mechanisms in all AI procurement agreements.
• Monitor systems continuously post-deployment: Track performance, compliance, and public concerns through systematic monitoring protocols and AI Centers of Excellence.
• Build internal AI capacity: Train staff on AI ethics and tools while fostering cross-agency collaboration to ensure responsible implementation and governance.
The success of AI in government depends not just on the technology itself, but on the procurement processes, contractual safeguards, and human oversight that guide its implementation. Agencies that follow these evidence-based practices will maximize AI’s benefits while minimizing risks to citizens and democratic institutions.
FAQs
Q1. How can government agencies ensure ethical AI procurement? Government agencies can ensure ethical AI procurement by identifying risks early, demanding vendor transparency, including ethical safeguards in contracts, continuously monitoring systems post-deployment, and building internal AI capacity through staff training and cross-agency collaboration.
Q2. What are some key ethical risks to consider when implementing AI in government? Key ethical risks include algorithmic bias, fairness issues, data quality problems, privacy concerns, and potential lack of transparency in AI decision-making processes. These risks can lead to unintended discrimination or erosion of public trust if not properly addressed.
Q3. How can government contracts protect against AI-related risks? Government contracts can protect against AI-related risks by mandating algorithmic transparency, including audit rights and data usage terms, and setting clear accountability and human oversight clauses. These provisions ensure ongoing compliance and allow for intervention if issues arise.
Q4. What role does post-implementation monitoring play in ethical AI use? Post-implementation monitoring is crucial for ensuring AI systems remain effective, ethical, and compliant throughout their lifecycle. It involves tracking performance, assessing compliance, responding to public concerns, and conducting regular risk assessments to maintain system reliability and relevance.
Q5. Why is building internal AI capacity important for government agencies? Building internal AI capacity is important because it enables agencies to effectively oversee AI systems, make informed decisions about their use, and ensure ethical implementation. This includes training staff on AI tools and ethics, aligning with federal frameworks, and encouraging cross-agency collaboration to share best practices.