STEP 2: PROBLEM DEFINITION, EVALUATING AI/ML SUITABILITY

Is AI/ML a feasible and desirable solution to implement in this context?

Even if the use of AI/ML is viable for a given problem, it is important to also consider whether it represents both a feasible and desirable solution which is in the interest of both local populations and external stakeholders.Feasibility refers to the resources available in an environment that will allow an efficient and sustainable adoption of the AI technology, while desirability refers to the will of the local population and stakeholders to adopt such technology. This includes understanding the upsides and downsides of developing an AI/ML solution, assessing risk appetite, extracting key learning from countries/agencies that have adopted similar solutions in the past and considering alignment with national regulations.. Both feasibility and desirability are crucial in defining and predicting the efficiency and sustainability of a program.

When developing AI/ML solutions, a unique challenge to reflect upon is the expertise differential. There is often a significant delta between the level of technical expertise of those designing and testing AI models and tools, those commissioning the projects, and those who may be the end users of AI, or impacted by its use. This section therefore invites reflections on ecosystem mapping and on the identification of relevant stakeholders operating in the AI/ML space in a given context.

Please find below a legend of what can be found within the framework:

📚Resources - e.g. reports, articles, and case studies

🛠Tools - e.g. guidelines, frameworks and scorecards

🔗Links - e.g. online platforms, videos, hubs and databases

❌Gap analysis - tools or resources are currently missing

👥 List of stakeholders which should be included in the specific decision point

  • 👥 General public, civil society, local tech service providers, journalists, local entrepreneurial ecosystem

    🔗🛠 Global Digital Health Index and Maturity Model - A platform for countries to document, benchmark, and track their maturity in adopting digital health across a series of defined metrics. It includes an interactive digital resource to assess, monitor, and improve the environment for effective use of digital health technology, to strengthen health systems and improve health outcomes. The tool also provides a maturity model with five levels to set a baseline, generate a scorecard and a benchmark against global averages, and learn from other countries to inform and target investments in digital health at the country, regional, and global levels

    📚Government AI Readiness Index 2021 - Yearly index ranking countries based on 42 indicators across three pillars: Government; Technology Sector; and Data and Infrastructure. The index answers the following question: how ready is a given government to implement AI in the delivery of public services to their citizens?.

    📚🛠Framework of Multi-layered Innovation Ecosystem Mapping (MIEM) - Framework to explore and identify both knowledge and business ecosystems

    🛠 Open Data Ethnography - TThis tool is designed to build understanding of in-depth and context-specific experiences of individuals in seeking, accessing and using information. This granular understanding is important for designing relevant interventions that speak to user needs (community or individual) and their social and cultural context

    ❌ Tools to assess human capacities (e.g. digital literacy of the local population and stakeholders that will adopt such technology

    • See step 1, Decision 3 (stakeholder mapping)

  • 👥 Legal experts, in-country government ministries, people in the community who might not use / benefit directly from the product but may be indirectly impacted

    🔗📚OECD AI Policy Observatory - Live repository of over 700 AI policy initiatives from 60 countries, territories and the EU. This information can be accessed by policy instrument (such as a national strategy, a regulation or a grant), or by group targeted by the policy (such as SMEs). A list of stakeholder initiatives is also provided

    🛠 Algorithmic Impact Assessment Tool - A questionnaire that determines the impact level of an automated decision-system. It is composed of 48 risk and 33 mitigation questions. Assessment scores are based on many factors including systems design, algorithm, decision type, impact and data

    🛠 Practical Framework for Public Agency Accountability - The Algorithmic Impact Assessment (AIA) framework proposed in this report is designed to support affected communities and stakeholders as they seek to assess claims made about these systems, and to determine where – or if – their use is acceptable

    ❌ AI/ML inclusive risk assessments to help communities understand their levels of risk exposure when providing their data; tools to assess the political, social, and economic implications of adopting an AI/ML solution; tools to evaluate specific risks (e.g., disparate impact on protected groups, arbitrary results, mischaracterizations, privacy harms) and measures to mitigate them, assessments of funding and technical expertise required to maintain and update solutions over time

  • 👥 Civil society, local population, general public, end users

    📚 UN OHCHR guide on conducting interviews

    📚WHO focus group on “AI for Health” - Case study of focus group conducted by ITU/WHO to establish a standardised assessment framework for the evaluation ​of AI-based methods for health, diagnosis, triage or treatment decisions

    🛠 Designing for Motivation, Engagement and Wellbeing in Digital Experience - Framework grounded in psychological research designed to enable researchers and practitioners to form actionable insights with respect to how technology designs support or undermine basic psychological needs

    ❌ Linguistic and epistemological translation tools to understand both local languages, values, and technical systems