1.1 Identification

1.1 Identification

Inspired by and mirroring the work of Jobin et al.[2], this study performed a scoping review to identify the characteristics and trends of AI soft law programs at a global scale [3]–[5]. Scoping reviews are a particularly effective means of gathering information in areas where evidence on a subject is nascent or emerging [3]. One factor complicating this type of research is the heterogeneity in the creation and distribution of soft law programs. Any organization can create them without having to comply with paradigms as to how they are promulgated, published, or broadcast to the world. Therefore, there is no centralized repository or systematic means of classifying them. To overcome this challenge, we established a three-pronged eligibility criteria to identify AI programs dedicated to soft law:

1. Comply with a definition of soft law: programs in this database are ones who set forth substantive expectations that are not directly enforceable by government [6]. Governance tools that fall within this scope include: strategies, partnerships, codes of conduct, professional guidelines, among others [6]–[8]. This effort explicitly excludes initiatives or programs that are considered hard law, those where a public entity may force an individual or organization to perform an act via the threat of a penalty or punishment
2. Emphasis on the governance or management of AI: Considerable debate exists on how to define AI [9], [10]. In determining the soft law programs to include in this database, the research team was instructed to add any program directed at an AI technology, regardless of how its author characterized or defined it. This includes soft law programs created to govern AI applications (e.g. facial recognition and autonomous vehicles) or methods (machine learning and neural networks).
3. Temporal restriction: Our research team limited the inclusion of programs to those in development or published by December 31st, 2019. Considering the constantly evolving AI governance space, our team wanted to focus on published initiatives.

Once the eligibility criteria were established, programs were detected through three parallel identification strategies: linkhubs, keyword searches, and citation chaining. The intention of this multi-pronged approach was to maximize the number of relevant AI soft law programs within this study

1.1.1 Linkhubs

Linkhubs are resources that compile potentially relevant soft law programs. Our team began evaluating a small number of well-known sources related to the governance of AI (such as [1], [2], [11]–[13]). As our search progressed, the research team added more resources to the list. Eventually, 84 linkhubs were evaluated (see Appendix 1). They range from databases created by multilateral organizations, governments, non-profits, independent researchers that utilized crowdsourcing, popular media, and academic publications.

1.1.2 Keyword searches

Our team developed and implemented a keyword search protocol. It consisted of combing through the first ten pages of a private-browsing session of a search engine (Google) using three levels of terms [1]. Our intent in creating this segmentation was to broaden our reach into the grey literature in order to detect an assortment of programs. The first level of keywords depict types of soft law found in the literature [6]. The second level of keywords consists of themes, applications, and methods of AI generated through a brainstorm session performed by the research team. The last level indicates the technology of interest, AI. Overall, the team performed 370 searches (all of the keywords can be found in Appendix 2).

1.1.3 Citation chaining

The text and references within every screened-in program were vetted by at least one team member for mention of other programs. This was done in an effort to identify additional soft law programs that complied with the project’s inclusion criteria.