The recent release of president Joe Biden’s executive order on artificial intelligence (AI) marks a pivotal step toward establishing standards in an industry that has long operated without comprehensive regulations. The move has been endorsed by tech leaders, who have long advocated for responsible AI regulations from Congress. However, what’s concerning is the order’s broad language, particularly the role of red-teaming, and the voluntary nature of many provisions, prompting doubts about its practical implementation and effectiveness.

While the call to develop standards, tools and tests for AI system safety and security is commendable, achieving this goal in practice is likely to pose significant challenges.

Defining Red Team Scope in AI Security Testing

The executive order doesn’t provide a precise definition of what a red team entails in the context of AI, which could create ambiguity about the scope of security testing. AI is essentially an application environment, so the kind of red team services available for AI testing should encompass a wide range of assessments, from evaluating the model’s performance and logic to ensuring data security.

What’s intriguing is that they’re using the term ‘red team’ not just in the cybersecurity sense but also in the context of data integrity, content awareness, fraud prevention and AI model testing. This represents a significant expansion of the term, especially considering that traditional red teaming has predominantly focused on physical and electronic security.

Uncertainties in Reporting Security Findings

The executive order emphasizes reporting security findings to the government, but the specifics of how this reporting will occur are yet to be determined. The language used in the order is quite broad and lacks a clear categorization of the risks posed by AI models to society or broader ecosystems.

Another critical consideration is the substantial volume of data handled by AI systems and the associated potential risks. When a flaw or compromise occurs, the data stored within these systems becomes vulnerable. Presently, there is a lack of compartmentalization and isolation in the AI sphere, setting it apart from how data is managed in other domains, such as the military.

While the executive order highlights the need to report security findings to the government, the details of this reporting mechanism are yet to be specified. The language used in the order is broad, lacking a precise classification of the potential risks posed by AI models to society or broader ecosystems.

Additionally, it seems that the provisions outlined in the executive order are voluntary, introducing potential challenges in practical implementation. The stipulation for developers of powerful AI systems to share test results represents a significant stride. This includes disclosing models posing risks to national security, economic security and public health and safety, even before the development phase commences. This raises inquiries about the extent of government involvement in the private sector for national security reasons.

Challenges in Addressing Bias in AI Datasets

Addressing bias in AI datasets is a critical concern, given its potential impact on the fairness and equity of AI applications. The executive order acknowledges this issue but falls short of providing well-defined guidelines for avoiding bias in various types of datasets. In some instances, bias may be inherent in data collected by government sources, necessitating specific guidance that the order and AI, for that matter, currently lack.

The mandate to report threats generated by AI systems to critical infrastructure is also important. Still, the order’s ambiguity in terms of regulating AI models’ impact and categorizing risks, especially in areas like privacy and data handling, poses challenges to effective implementation. The order’s broad provisions on addressing equity and civil rights within AI may require further clarification to ensure practical application.

The order also emphasizes that AI systems must adhere to privacy and discrimination prevention. Nevertheless, existing discrimination-preventing mechanisms, such as FICO scores that exclude considerations of race, sex, or orientation in credit checks, already exist. The executive order’s provisions related to equity and civil rights within AI are notably broad and may benefit from additional clarification to facilitate effective implementation.

While President Biden’s executive order on AI regulation and security takes commendable steps, it leaves critical questions unanswered. The broad language used, coupled with the voluntary nature of many provisions, raises concerns about practical implementation and effectiveness. To navigate the complexities of the evolving AI landscape, further refinement may be necessary. The order, though emphasizing the need to address issues, falls short of offering concrete solutions, leaving certain areas, such as bias in datasets, somewhat vague. As the AI industry continues to advance, ongoing dialogue and collaboration will be crucial to crafting regulations that strike the right balance between innovation and responsible governance.