The National Institute of Standards and Technology’s Center for AI Standards and Innovation is teaming up with the nonprofit OpenMined to develop methods for evaluating artificial intelligence systems while preserving data confidentiality.
The organizations signed a collaborative research and development agreement to utilize OpenMined’s software infrastructure, including PySyft, to enable AI evaluations that adhere to security requirements and maintain scientific rigor, NIST said Friday.
PySyft enables researchers to perform data science and analysis using non-public information without seeing or obtaining a copy of sensitive datasets.
Balancing innovation and modernization with security as agencies increase adoption of AI will be a key topic at the Potomac Officers Club’s 2026 Digital Transformation Summit, happening on April 22. The event will bring together government and industry leaders to explore how agencies are integrating AI into high-security federal environments to support various missions. Sign up today to gain insights from the figures shaping federal digital transformation.
How Is the CAISI-OpenMined Partnership Intended to Improve AI Evaluations?
By developing privacy-preserving evaluation methods, stakeholders can conduct rigorous AI evaluations to measure system performance even when underlying data, models or benchmarks cannot be shared due to intellectual property, data protection or national security constraints.
According to NIST, insights from the effort will inform the agency’s development of voluntary standards, best practices and recommendations for AI evaluation.
The partnership builds on CAISI’s prior work with the General Services Administration to provide evaluation frameworks, testing methodologies, and performance measurement tools for agencies before and after deployment in support of USAi, a governmentwide platform that accelerates agency adoption of generative AI.
What Is CAISI?
The National Institute of Standards and Technology’s Center for AI Standards and Innovation, or CAISI, serves as the U.S. government’s primary interface with industry for testing and collaborative research on commercial AI systems.
The organization works with federal partners to develop guidelines to improve AI security. It also evaluates AI capabilities that may pose risks to national security.
The center recently launched the AI Agent Standards Initiative to promote secure, interoperable and trustworthy autonomous AI systems. Under the initiative, CAISI aims to develop industry-led AI agent standards, support community-driven open-source protocols, address security risks associated with agentic AI, and reinforce U.S. leadership in international AI governance.
