<strong><em>Why is meal planning a recommended financial strategy for women but not for men with children, according to ChatGPT? And why is this important for financial services?</em></strong>

Natural language processing (NLP) tools can create human-like conversations. Perhaps the most well-known application of an NLP tool is ChatGPT, which <a href="https://www.theguardian.com/technology/2023/feb/02/chatgpt-100-million-users-open-ai-fastest-growing-app" target="_blank" rel="noopener noreferrer">reached over 100 million users two months after its launch</a> in November 2022. Many people have increasingly used ChatGPT as a tool for work or personal use. I wanted to test if ChatGPT showed gender bias when looking at finance-related applications, so I inputted a few prompts about financial services. The results are a reminder that while AI is a powerful tool, it is not free from problematic social norms and potential bias against women. This week CFI <a href="https://www.centerforfinancialinclusion.org/equitable-ai-for-inclusive-finance">released two new publications</a> to help strengthen industry conversations to mitigate against, monitor for, and correct harmful gender bias in inclusive finance. These publications are the result of CFI’s winning proposal to USAID’s <a href="https://www.usaid.gov/innovation/competitions/equitable-ai-challenge" target="_blank" rel="noopener noreferrer">Equitable AI Challenge</a>, implemented by DAI’s Digital Frontiers.
<blockquote class="pullquote--right">One simple prompt revealed three problematic differences. However, they reflect a complex problem requiring an urgent response.</blockquote>
When I asked ChatGPT, or GPT-3.5, to write financial advice to help men and women with children, the answers showed important differences by gender, as seen in the following screenshots. For instance, the tool recommended that men, not women, create or update their wills and designate beneficiaries for their assets. This result demonstrates an existing assumption that <em>men</em> with children are more likely to have assets or income than <em>women</em> with children. Moreover, GPT-3.5 advised only men, not women, to have a work-life balance and spend quality time with their families. Again, this shows a bias about gender roles in work and within families. Another blatant example was the recommendation GPT-3.5 gave to women, but not to men, about planning meals. The tool advised women to cook larger meals at home and plan their grocery shopping to save money.

One simple prompt revealed these three problematic differences. However, they reflect a complex problem requiring an urgent response — especially as many companies, including fintechs, incorporate AI tools, such as NLP, into their business models.

<img class="aligncenter size-full wp-image-46358" src="https://content.centerforfinancialinclusion.org/wp-content/uploads/sites/2/2023/08/EqAI-ChatGPT-01.png" alt="" width="4038" height="3555" />
<h2><strong>Growing Use Cases of AI in Inclusive Finance</strong></h2>
AI has <a href="https://www3.weforum.org/docs/WEF_AI_in_Financial_Services_Survey.pdf" target="_blank" rel="noopener noreferrer">numerous applications in the financial sector</a>, such as revenue generation, process automation, risk management, client acquisition, and customer service. Companies are increasingly using AI to accelerate lending decisions and use alternative data — including social media, mobile phone activity, and satellite data — to measure creditworthiness. AI presents an opportunity to quickly expand access to financial services for populations without long credit histories, including women. Companies use facial patterns, voice recognition, and biometric data for know-your-customer (KYC) requirements and to verify customers’ identities. Insurance companies use AI algorithms to speed up claims processing. Companies are also using AI tools for client acquisition and customer service. Chatbots and virtual assistants help prospective and existing customers find information about financial products that meet their needs.
<blockquote><span class="ui-provider kt b c d e f g h i j k l m n o p q r s t u v w x y z ab ac ae af ag ah ai aj ak" dir="ltr">Instead of offering helpful customization, the financial advice given perpetuates outdated and unhelpful stereotypes. The resulting harm from biased AI systems used in financial services contradicts the goals of financial inclusion and has potential business and regulatory costs for fintechs and their investors.</span></blockquote>
However, while AI tools are making high-stakes decisions for the economic prospects and future of individuals, their businesses, and their communities, these systems are also introducing new risks, including data privacy vulnerabilities and unfair outcomes for certain consumers. Women have experienced negative effects, such as being unfairly rejected as “false negative” in credit decisions, or receiving higher pricing, lower credit limits, and more limited choices.  As the above experiment with GPT-3.5 showed, algorithms can provide different financial advice depending on a customer’s gender. When designed well, tailored solutions for different customer segments can increase usage by satisfying the specific needs of the group. However, in this case, instead of offering helpful customization, the financial advice given perpetuates outdated and unhelpful stereotypes. The resulting harm from these issues contradicts the goals of financial inclusion and has potential business and regulatory costs for fintechs and their investors.
<h2><strong>Sources of Harmful Gender Bias</strong></h2>
<img class="wp-image-46360 alignright" src="https://content.centerforfinancialinclusion.org/wp-content/uploads/sites/2/2023/08/Screenshot-2023-08-25-at-7.40.18-AM.png" alt="" width="405" height="408" />

CFI has written extensively on the opportunities and risks of AI for inclusive finance, and in 2021, we released a <a href="https://content.centerforfinancialinclusion.org/wp-content/uploads/sites/2/2021/03/The-Stories-Algorithms-Tell-CFI-publication-MAR21.pdf">framework to help understand algorithmic bias</a>. The framework included three buckets to categorize sources of harmful AI bias: 1) inputs (such as datasets and AI teams), 2) code, and 3) context.

Applying this framework to my experiment with ChatGPT illustrates a few examples of where differences in financial advice by gender may originate. However, it’s important to highlight that these possible explanations are purely theoretical, as we have not tested them empirically.

<strong>Inputs:</strong> NLP and AI algorithms are trained on data to make predictions and inform decisions. In this case, <a href="https://medium.com/@dlaytonj2/chatgpt-show-me-the-data-sources-11e9433d57e8" target="_blank" rel="noopener noreferrer">GTP-3.5 processed input data from the web, books, and WebText2</a>, which is culled from Reddit submissions. As you may imagine, in aggregate these input datasets compile a series of beliefs and social norms, such as the one that suggested that women save money through meal prepping.

<strong>Code and Context:</strong> As a model is being built, developers can use techniques to reduce biased responses by, for instance, penalizing the model for generating harmful or stereotypical content. AI developers, or other actors, can periodically review model outputs after launch and provide feedback and adjustments over time. However, if the teams testing the system are primarily men, problematic outputs — like from my experiment — may be less likely to be flagged.
<h2><strong>A Path Forward </strong></h2>
As with any technology that experiences exponential growth, stakeholders — like investors, donors, and regulators — must play catch-up, and many questions remain. How, amid all these changes, can concerned stakeholders understand, let alone contribute to, shaping the future of AI in a way that is responsible and equitable, particularly for achieving our target financial inclusion outcomes?

In the inclusive finance sector, many actors are increasingly interested in incorporating principles and tools that guide the responsible and equitable deployment of AI. However, although there is a proliferation of techniques and academic resources to identify, mitigate, and correct bias, our extensive review of over 120 guides and checklists showed a lack of tools appropriately designed for the financial sector.

Recognizing this gap, CFI developed and this week released a <a href="https://content.centerforfinancialinclusion.org/wp-content/uploads/sites/2/2023/08/Equitable-AI-Guide.pdf">practical guide</a> to help impact investors identify harmful AI gender bias among their investee companies, and an <a href="https://content.centerforfinancialinclusion.org/wp-content/uploads/sites/2/2023/08/Equitable-AI-Brief.pdf">accompanying brief</a> that presents some of the key challenges with building accountability and transparency for AI in inclusive finance and offers opportunities to strengthen the industry conversation. Recommended practices in the guide touch on data, inputs, and model building as well as crucial business practices including governance. For instance, in the case of OpenAI, the creator of GPT-3.5, the company publicly committed that AI should benefit all of humanity and wrote <a href="https://openai.com/safety" target="_blank" rel="noopener noreferrer">principles to guide their mission</a>. This stance may explain OpenAI’s six-month effort to make the new version of ChatGPT, GPT-4, safer and with hopefully fewer problematic outputs. I’ll be sure to check back with some prompts for GPT-4!

Read more about our work around equitable AI <a href="https://www.centerforfinancialinclusion.org/equitable-ai-for-inclusive-finance">here</a>.

<img class="aligncenter size-full wp-image-46361" src="https://content.centerforfinancialinclusion.org/wp-content/uploads/sites/2/2023/08/Screenshot-2023-08-25-at-7.42.00-AM.png" alt="" width="2054" height="1034" />

<em>*Note: In this blog, gender gaps refer to the gap between women and men. However, CFI acknowledges that gender includes non-binary populations such as LGTBQ communities. It is essential to highlight that this blog does not intend to dismiss the existence of non-binary people.</em>

Gender Bias in AI: An Experiment with ChatGPT in Financial Inclusion

<p style="font-weight: 400;">Artificial intelligence (AI) enables innovations in digital finance that can boost efficiency, cut costs, and expand consumer reach. AI provides scalable ways to determine the creditworthiness of individuals and businesses that historically lacked identification, collateral, and credit history. And by automating processes, AI can enable higher volumes of low-value transactions that make harder-to-reach segments more viable customers. However, while AI presents tremendous opportunities, deploying AI also comes with significant risks – one being inequitable outcomes for marginalized consumers, especially women. When applied at scale, the harms caused by AI counter the financial inclusion goals of many impact investors and their portfolio companies. While impact investors are well positioned to take a proactive role in supporting companies in building and deploying equitable AI systems in inclusive finance services, they often do not have the tools required to assess and evaluate the fairness of algorithms. Recognizing this gap, the Center for Financial Inclusion (CFI) developed a practical guide for impact investors to help them identify harmful AI gender bias.</p>
<p style="font-weight: 400;">The <em>Investing in Equitable AI for Inclusive Finance: A Risk Management Guide for Impact Investors</em> gives an overview of the use cases of AI in inclusive finance and some of the drivers of harmful AI bias towards women. It also presents a snapshot of the state of practice in bias identification and mitigation and then provides a user-friendly checklist with an actionable set of questions to help impact investors understand the use of AI among their investee companies.</p>
<p style="font-weight: 400;">The accompanying brief, <em>Prompts for Equitable Artificial Intelligence in Inclusive Finance: Strengthening the Industry</em> <em>Conversation,</em> presents some of the key challenges with building accountability and transparency for AI in inclusive finance and discusses how CFI’s new guide can help raise awareness around the risks and costs of gender bias in AI. The brief also offers opportunities to strengthen the industry conversation on equitable AI, including advancing research, supporting consumer advocacy and journalism, and operationalizing data rights for consumers.</p>
<p style="font-weight: 400;"><em>This work is the result of USAID’s <a href="https://www.usaid.gov/innovation/competitions/equitable-ai-challenge" target="_blank" rel="noopener noreferrer">Equitable AI Challenge</a>. Implemented by DAI’s Digital Frontiers, the challenge aimed to identify innovative approaches to address artificial intelligence inequitable outcomes. The Equitable AI Challenge asked for proposals that critically consider holistic and creative approaches to identify and address gender biases in AI systems within global development contexts. This work is the result of CFI’s winning proposal.</em></p>

Equitable AI for Inclusive Finance

CFI Advisory Council Member Rosita Najmi has over 20 years of experience in global development, but she attributes her initial connection to financial inclusion to a personal relationship with poverty as a young refugee who came to the United States from Iran with her family.

<span data-contrast="auto">Originally a field researcher and social entrepreneur, she has also worked across leading international NGOs to advocate for and employ financial services as a tool to empower and improve the lives of low-income people. As a development economist, she helped stand up the first World Bank global practice for financial inclusion. She then transitioned from operations and development policy functions to funder roles -- across impact investing, private philanthropy, and now corporate philanthropy. She is a long-time and devoted champion of gender equality, and her contributions include efforts to advance women’s economic empowerment, including the creation and funding of </span><a href="https://www.findevgateway.org/finequity" target="_blank" rel="noopener noreferrer"><span data-contrast="none">FinEquity</span></a><span data-contrast="auto"> and the </span><a href="https://data2x.org/financial-inclusion/" target="_blank" rel="noopener noreferrer"><span data-contrast="none">Data2X Women’s Financial Inclusion Data Partnership</span></a><span data-contrast="auto">. Currently, </span><span data-contrast="none">Rosita serves as </span><span data-contrast="none">a</span><span data-contrast="none"> VP of Social Impact at The UPS Foundation, leading strategy, innovation, grantmaking, and measurement, evaluation, and learning (MEL).</span>

<span data-contrast="auto">As Rosita reflects on her career, she contemplates the meaningful way that her professional roles, and the specific order in which they happened, have enabled her to optimize her impact. As a CFI Advisory Council member since January 2022, she continues to work toward achieving a goal that her late mother regularly encouraged: “Make your life count!”</span>
<h3><b><span data-contrast="auto">1. How did you become involved in inclusive finance and what are you working on now?</span></b><span data-ccp-props="{}"> </span></h3>
<span data-contrast="auto">RN: My connection to financial inclusion is rooted in a personal relationship with poverty. My family — my late mom, three older siblings, and I — </span><a href="https://www.hbs.edu/mba/student-life/people/Pages/portrait-project.aspx?profile=rnajmi~pp" target="_blank" rel="noopener noreferrer"><span data-contrast="none">came to the United States from Iran as refugees</span></a><span data-contrast="auto">. We brought three suitcases and $3,000 cash, which my then 15-year-old brother carried in his pockets.</span>

<span data-contrast="auto">I remember well the judgment and discrimination my family and I faced while living in poverty. I grew up in Tennessee, where there was racism and judgment of families who received government welfare, such as food stamps, which mine did. At that time, food stamps were pieces of paper so it was easy for others to notice when my mom used them at the grocery store.</span>

<span data-contrast="auto">As early as elementary school, I used to help my mother with her bank account and finances. I couldn’t even reach the counter to see the bank teller! But, because of her accent and English being her second language, my mother was not confident, and I would often help communicate with the bank teller on her behalf.</span>

<span data-contrast="auto">This personal journey with access to financial services and resources has shaped my priorities and instilled a strong desire to help and empower others, with an emphasis on designing for dignity. I have worked in social and economic development for many years, but my top priority has been to transform financial services into a tool to empower low-income and marginalized people.</span>
<blockquote class="pullquote--right"><span data-contrast="auto">This personal journey with access to financial services and resources has shaped my priorities and instilled a strong desire to help and empower others, with an emphasis on designing for dignity.</span></blockquote>
<span data-contrast="auto">I started my career as a field researcher, social entrepreneur, and founder of an international NGO. These experiences were the best education I could have asked for — more than my formal education — to understand families’ financial hardships around the world and the impossible choices that so many are forced to make every day. When I joined the World Bank as a development economist, I helped develop the Bank’s first global practice for financial inclusion. We created the support template to help countries develop and measure progress against national financial inclusion strategies. After that, and for over a decade, I have been a funder of different types of capital, from impact investing to private and corporate philanthropy.</span>

<span data-contrast="auto">Currently, I oversee grantmaking, strategy, innovation, and measurement, evaluation, and learning (MEL) at </span><a href="https://about.ups.com/us/en/social-impact.html" target="_blank" rel="noopener noreferrer"><span data-contrast="none">The UPS Foundation</span></a><span data-contrast="auto">. </span><a href="https://about.ups.com/us/en/social-impact/the-ups-foundation/the-ups-foundation-mission-and-purpose.html" target="_blank" rel="noopener noreferrer"><span data-contrast="none">One of the four focus areas</span></a><span data-contrast="auto"> is equity and economic empowerment, which includes a focus on women. Whether in accessing finance or moving their goods to markets or across borders, women face unique challenges because of </span><a href="https://www.centerforfinancialinclusion.org/normative-constraints-to-womens-financial-inclusion-what-we-know-and-what-we-need-to-know"><span data-contrast="none">social and cultural norms</span></a><span data-contrast="auto">, limits on mobility and agency, and time poverty, among other barriers. We seek to do our part to remove such barriers. UPS’ equity and economic empowerment portfolio also includes career and college readiness for youth, entrepreneurship, and workforce development.</span>
<h3><b><span data-contrast="auto">2. What key lessons have you learned from past work experiences?</span></b></h3>
<span data-contrast="auto">RN: While funding the </span><a href="https://www.betterthancash.org/" target="_blank" rel="noopener noreferrer"><span data-contrast="none">Better than Cash Alliance</span></a><span data-contrast="auto"> (BTCA) during my tenures at both the Omidyar Network and the Bill &amp; Melinda Gates Foundation, I deepened my understanding of the potential of advocacy. Advocacy is a skill set -- a combination of art and science -- that can transform lives, change policies, and move resources to support underserved or marginalized communities. I also learned how blended finance can play a vital role in global sustainable development. I believe we need varying types of capital — a mixed capital stack — for scale and long-lasting impact.</span>
<blockquote><span data-contrast="auto">Advocacy is a skill set -- a combination of art and science -- that can transform lives, change policies, and move resources to support underserved or marginalized communities.</span></blockquote>
<span data-contrast="auto">Furthermore, learning and sharing experiences through coalitions and alliances is much more efficient and effective than doing it alone. I am a collaborator and connector and have worked on building and supporting multiple global coalitions and alliances around different topics like impact investing, </span><a href="https://www.findevgateway.org/finequity" target="_blank" rel="noopener noreferrer"><span data-contrast="none">women’s economic empowerment</span></a><span data-contrast="auto">, </span><a href="https://www.ictd.ac/programme/digitax/" target="_blank" rel="noopener noreferrer"><span data-contrast="none">tax policy and administration</span></a><span data-contrast="auto">, and </span><a href="https://www.cifaralliance.org/" target="_blank" rel="noopener noreferrer"><span data-contrast="none">climate equity and resilience</span></a><span data-contrast="auto">.</span>

<span data-contrast="auto">Finally, because of my tenure at </span><a href="https://www.mercycorps.org/" target="_blank" rel="noopener noreferrer"><span data-contrast="none">Mercy Corps</span></a><span data-contrast="auto">, I became a strong believer in market-based approaches that lead to sustainable development and empowerment. In contrast to a beneficiary-focused approach, market-oriented models position populations living in poverty as consumers, producers, employees, or entrepreneurs. Market-based approaches use business models and work to catalyze markets to solve development and humanitarian challenges more sustainably and at scale than time and resource-bound interventions.</span><span data-ccp-props="{&quot;335551550&quot;:6,&quot;335551620&quot;:6}"> </span>
<h3><b><span data-contrast="auto">3. If you weren’t in your current professional role, what would you be doing right now?</span></b></h3>
<span data-contrast="auto">RN: I would spend more time advocating to end human rights violations in Iran, especially in support of </span><a href="https://web.archive.org/web/20160229174442/https:/www.hbs.edu/mba/student-life/people/Pages/wdydwyd.aspx?profile=rnajmi" target="_blank" rel="noopener noreferrer"><span data-contrast="none">women and girls</span></a><span data-contrast="auto"> and members of the religion the </span><a href="https://www.bahai.org/" target="_blank" rel="noopener noreferrer"><span data-contrast="none">Bahá'í Faith</span></a><span data-contrast="auto">.</span>
<h3><b><span data-contrast="auto">4. What unique expertise do you bring to CFI’s Advisory Council? </span></b></h3>
<span data-contrast="auto">RN: I strive to bring the following to CFI: (i) my network so I can be a connector for CFI’s applied research; (ii) strategic thinking, especially around creating intersectional outcomes; and (iii) personal experience and passion for designing for dignity and empowerment.</span>

<span data-contrast="auto">If I had the chance to re-design my career, I would take the same path. The order of my experiences was meaningful; I am a better funder, because I was a grantee first, and I can be an innovator, because I worked in the field, in operations, and on the front lines. I aim to bring my experience wearing these diverse hats to CFI’s Advisory Council.</span>
<h3><b><span data-contrast="auto">5. In one word, what is the current state of financial inclusion today? </span></b><span data-ccp-props="{}"> </span></h3>
<span data-contrast="auto">RN: Transitioning. Many people and organizations have worked hard to build the case for why the sector needs to go beyond solely advancing access. It’s now understood that we cannot declare victory simply when someone opens a bank account and that instead, the customer must build towards resilience to escape poverty and stay out of poverty. Digital financial inclusion can be a tool for both empowerment and equity in development, and it supports government accountability and fiscal transparency, particularly for government subsidies to its citizens and tax administration. Lastly, when I think about the future, especially in terms of climate shocks and war, the need for humanitarian response is increasing at a rapid pace and scale. We must swiftly integrate digital payments into humanitarian responses. </span><span data-ccp-props="{&quot;335551550&quot;:6,&quot;335551620&quot;:6}"> </span>
<h3><span data-ccp-props="{}"> 6. </span><b><span data-contrast="auto">What can we learn from fintech that can be applied to financial inclusion?</span></b></h3>
<span data-contrast="auto">RN: Friction management is important. Typically, the goal of payments, especially for access to credit and savings, is to be as frictionless as possible. However, I believe there is a role for friction when it comes to credit. For example, providing notifications or other reminders highlighting an outstanding debt balance would be advantageous when a customer has multiple loans or payment plans. This approach serves the dual purpose of protecting consumers' interests and empowering consumers to stay informed about their financial obligations. In this way, the financial provider is signaling its focus on its bottom line and its commitment to the financial health of its customer.</span>

<span data-contrast="auto">On the data side, people are already researching and discovering issues related to data misuse, especially data privacy. Fortunately, new laws are helping, but the financial services industry and development actors could also have proactive strategies for the misuse and </span><i><span data-contrast="auto">missed</span></i><span data-contrast="auto"> use of data.</span><span data-ccp-props="{&quot;335551550&quot;:6,&quot;335551620&quot;:6}"> </span>
<blockquote class="pullquote--left"><span data-contrast="auto">The effectiveness of AI relies on the quality of the data employed. If marginalized groups, such as women or people with disabilities, are invisible in the data, these groups are at serious risk of being left further behind.</span></blockquote>
<span data-contrast="auto">There is an abundant amount of data being produced, but unfortunately, not all data is used or used optimally. This underutilization can be because of a fear of breaking the rules, especially around data management and governance. Other times, the barrier can be because skill sets are missing in data science. Nevertheless, like how we developed frameworks to guide countries to develop national strategies for driving and measuring financial inclusion, I think there is a need to develop diagnostics and templates that help governments and financial services providers use data to benefit their customers. For example, financial services providers can use data to measure product equity — showing who is using products and services and to what outcomes. Further, governments, academia, civil society, and development organizations can leverage anonymized and aggregated data sets to triangulate with other data sets for insights, innovations, and consumer protection.</span>

<span data-contrast="auto">I would like to see more ‘data for good’ strategies in the private sector, which can include disaggregating data by </span><a href="https://data2x.org/" target="_blank" rel="noopener noreferrer"><span data-contrast="none">gender</span></a><span data-contrast="auto">, race, and other classifications and considering the quality of data sets that educate artificial intelligence. The effectiveness of AI relies on the quality of the data employed. If marginalized groups, such as women or people with disabilities, are invisible in the data, these groups are at serious risk of being left further behind.</span>
<h3><b><span data-contrast="auto">7. What are the most crucial issues in inclusive finance that we should prioritize over the next five years?</span></b></h3>
<span data-contrast="auto">RN: We are seven years from the 2030 Sustainable Development Goals (SDG) deadline, and the COVID pandemic eliminated years of progress. We must compensate for this loss by pursuing interventions with intersectional outcomes. Therefore, the issues I would prioritize in the next five years include: </span><span data-ccp-props="{&quot;335551550&quot;:6,&quot;335551620&quot;:6}"> </span>
<ul>
 	<li data-leveltext="" data-font="Symbol" data-listid="35" data-list-defn-props="{&quot;335552541&quot;:1,&quot;335559684&quot;:-2,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}" aria-setsize="-1" data-aria-posinset="1" data-aria-level="1"><span data-contrast="auto">The misuse and missed use of data</span></li>
 	<li data-leveltext="" data-font="Symbol" data-listid="35" data-list-defn-props="{&quot;335552541&quot;:1,&quot;335559684&quot;:-2,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}" aria-setsize="-1" data-aria-posinset="2" data-aria-level="1"><span data-contrast="auto">Climate risks related to food security, a just transition, adaptation, and resilience</span></li>
 	<li data-leveltext="" data-font="Symbol" data-listid="35" data-list-defn-props="{&quot;335552541&quot;:1,&quot;335559684&quot;:-2,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}" aria-setsize="-1" data-aria-posinset="3" data-aria-level="1"><span data-contrast="auto">Gender equality issues related to health and education outcomes, and the social and cultural norms that thwart the full contributions of women and girls</span></li>
</ul>

Designing for Dignity: Creating Equal Opportunity through Financial Inclusion – Q & A with Rosita Najmi, CFI Advisory Council Member

In 2022, advances in artificial intelligence captured our collective imaginations in unprecedented ways. Whether it was the artistic text-to-image renderings of <a href="https://openai.com/dall-e-2/" target="_blank" rel="noopener noreferrer">Dall-E2</a> shown in this blog post or in-depth conversations with <a href="https://openai.com/blog/chatgpt/" target="_blank" rel="noopener noreferrer">ChatGPT</a>, the implications of data advances in analysis, prediction, and generation became clearer than ever to the “non-techy” public. Last year, governments around the world raced to stay on top of data advances through various data governance approaches, such as <a href="https://www.dlapiperdataprotection.com/" target="_blank" rel="noopener noreferrer">data protection legislation</a>, or draft approaches to regulating AI including the <a href="https://www.whitehouse.gov/ostp/ai-bill-of-rights/" target="_blank" rel="noopener noreferrer">White House’s AI Bill of Rights</a> and the <a href="https://artificialintelligenceact.eu/" target="_blank" rel="noopener noreferrer">EU’s AI Act</a>. What has become very clear is that there is both tremendous excitement at the promise of these advances, as well as significant concern.

Several sessions at <a href="https://financialinclusionweek.org/" target="_blank" rel="noopener noreferrer">Financial Inclusion Week</a> (FIW) 2022 focused on data-driven advances in inclusive finance that grappled with the duality of exciting opportunity and concerning risk – particularly with regards to low-income and vulnerable consumers, like female-owned MSEs and migrants.
<h1>Data Opportunities</h1>
A few sessions shared how customer data is helping to shape product design for traditionally underserved consumers. The UN Capital Development Fund hosted a session on <a href="https://www.youtube.com/watch?v=g15noAC2vWU&amp;list=PL30fMMRd3G_4MWcg0tioe63vFk1WA8Clb&amp;index=5" target="_blank" rel="noopener noreferrer">Driving Women’s Financial Inclusion and Resilience</a>, with a focus on how consumer data, whether administrative or survey based, can drive innovative financial products for women. In this session, a speaker from <a href="https://imeremit.com.np/" target="_blank" rel="noopener noreferrer">IME Nepal</a> talked about how the company is leveraging remittance transaction data during COVID to better understand consumers and expand its digital offerings, including a wallet. UNCDF highlighted demand-side research that showed the challenges women face in onboarding to digital remittance services. My colleague <a href="https://www.centerforfinancialinclusion.org/bio/anindita-chakraborty">Anindita Chakraborty</a> contributed to the discussion with insights from CFI’s research on the difficulties <a href="https://www.centerforfinancialinclusion.org/research/covid-19-and-msmes-data-and-analysis-to-understand-impact">female-led MSEs</a> faced during the pandemic, and how that data should inform products to help MSEs be more resilient in future shocks.
<blockquote>Customer data is helping to shape product design for traditionally underserved consumers.</blockquote>
In addition to leveraging consumer data, looking at data on a larger scale can have significant benefits. In a <a href="https://www.youtube.com/watch?v=F0aZV-Zfzd0&amp;list=PL30fMMRd3G_4MWcg0tioe63vFk1WA8Clb&amp;index=5" target="_blank" rel="noopener noreferrer">short session by Principal Financial Group</a>, the Principal team shared the launch of the Global Financial Inclusion Index, a ranking of 42 markets based on what governments, financial systems, and businesses are doing to advance financial inclusion for their populations. This index aims to give investors insights into the strength and resilience of these markets.

Several fintechs and providers shared insights on their data-driven business models. A <a href="https://www.youtube.com/watch?v=WwecOBYOyi4&amp;list=PL30fMMRd3G_4MWcg0tioe63vFk1WA8Clb&amp;index=6" target="_blank" rel="noopener noreferrer">session</a> hosted by Korean fintech giant <a href="https://www.sentbe.com/" target="_blank" rel="noopener noreferrer">SentBe</a> presented how its cross-border money transfer services leverage data on traditionally underserved consumers – including migrants from Bangladesh and Cambodia working in Korea – to provide customer-centric receipt options. Senegalese fintech <a href="https://www.lebfay.com/" target="_blank" rel="noopener noreferrer">LebFay</a> presented <a href="https://www.youtube.com/watch?v=DA8wqNKl2f4&amp;list=PL30fMMRd3G_4MWcg0tioe63vFk1WA8Clb&amp;index=7" target="_blank" rel="noopener noreferrer">its business model</a> that uses an app and artificial intelligence-driven scoring model to lend to traditionally underserved MSEs and tradesmen. And, <a href="http://imbita.org/" target="_blank" rel="noopener noreferrer">Imbita Swaziland Women’s Finance Trust</a> spoke about the digitalization journey in offering its savings and credit cooperative society (SACCO) members a USSD-based platform to save, borrow, and make mobile payments.
<h1>Data Risks</h1>
<blockquote class="pullquote--right">Challenges, including privacy, bias, and other human rights issues, can stem from the exponential growth of digital lending.</blockquote>
At the same time, several sessions highlighted the risks around data-driven innovation and what inclusive finance stakeholders need to do about it. <a href="https://aapti.in/" target="_blank" rel="noopener noreferrer">Aapti Institute</a>, an India-based think-tank that conducts research at the intersection of technology and digital rights, hosted a session <a href="https://www.youtube.com/watch?v=lXLY7BsG_fg&amp;list=PL30fMMRd3G_4MWcg0tioe63vFk1WA8Clb&amp;index=3" target="_blank" rel="noopener noreferrer">on AI and Financial Inclusion</a>, discussing risks and opportunities for risk mitigation with representatives from <a href="https://www.dvara.com/research/" target="_blank" rel="noopener noreferrer">Dvara</a>, <a href="https://rbihub.in/" target="_blank" rel="noopener noreferrer">RBI Innovation Hub</a>, and <a href="https://www.undp.org/india" target="_blank" rel="noopener noreferrer">UNDP</a>. This session raised challenges, including privacy, bias, and other human rights issues, that could stem from the exponential growth of <a href="https://www.centerforfinancialinclusion.org/digital-credit-algorithms-in-india-expanding-access-or-cementing-exclusion">digital lending in India</a>. I was honored to moderate a session on the <a href="https://www.youtube.com/watch?v=J0QaPM6sDDk&amp;list=PL30fMMRd3G_4MWcg0tioe63vFk1WA8Clb&amp;index=2" target="_blank" rel="noopener noreferrer">Potential of Privacy by Design for Inclusive Finance</a> with inputs from <a href="https://www.paypal.com/us/home" target="_blank" rel="noopener noreferrer">PayPal</a>, <a href="https://foundation.mozilla.org/en/" target="_blank" rel="noopener noreferrer">Mozilla Foundation</a>, and the <a href="https://www.dlak.co.ke/" target="_blank" rel="noopener noreferrer">Digital Financial Services Association of Kenya</a>. Representatives from PayPal and Mozilla discussed how Privacy by Design has been implemented in their business models, and Kevin Mutiso of DFSAK shared how startup fintechs can integrate and benefit from strong privacy practices.
<h1>Looking Ahead</h1>
As we look ahead to 2023, CFI will be monitoring how advances in AI are integrated into the inclusive finance space. Will there be chatbots or financial capability curricula that use ChatGPT? We’ll also be watching for countries to codify the use of AI and enforce data protection legislation. With so much happening in this space at the moment, there is a lot to pay attention to as we continue to grapple with the duality of data opportunities and risks.

View the full <a href="https://www.youtube.com/playlist?list=PL30fMMRd3G_4MWcg0tioe63vFk1WA8Clb" target="_blank" rel="noopener noreferrer">Data Risks &amp; Opportunities</a> Playlist from FIW 2022.

Financial Inclusion Week 2022 had over 3,300 registered participants from 2,000 organizations across 148 countries. With almost 400 speakers and 125 sessions, FIW 2022 was the largest event to date. All sessions from FIW 2022 can be viewed here.

And be sure to save the date for FIW 2023 which will take place October 16-19, 2023.

FIW Recap: Grappling with Data Opportunities and Risks in Inclusive Finance

The theme for Financial Inclusion Week (FIW) 2022 was “Inclusive Growth in a Digital Era.” For years, financial products and services have been moving digital, and the pandemic further accelerated the trend toward digitalization. As digital becomes the new norm, we must ensure that these services are safe and inclusive of all people and do not exacerbate the exclusion of marginalized and vulnerable groups. FIW 2022 explored how the global financial inclusion community can harness digital innovation to expand inclusive financial services for all people across the globe, ensuring that products meet customer needs, have appropriate safeguards in place, and contribute to inclusive economic growth.

FIW 2022 hosted 125 live and on-demand sessions, featuring nearly 400 speakers from around the world. With over 3,330 registered participants from 2,000 organizations and 146 countries, FIW 2022 was the largest event to date.

To capture the learnings and key takeaways from the many Financial Inclusion Week 2022 sessions, CFI is compiling short summaries that highlight content and ideas from FIW 2022 sessions on select topics.

FIW 2022 Recap

This report and infographic examine data from the 250+ eligible fintech applicants from the fourth Inclusive Fintech 50 competition to better understand the state of early-stage fintechs. CFI analyzes all applications to identify trends and to find innovations with disruptive potential. The massive scale that is only possible with digitalization matters greatly to the financial inclusion agenda, and this year’s IF50 applicants are already collectively reaching 54 million people, many of whom were previously underserved, across 101 countries of operation. The fintechs are young and small now, but as we’ve seen in past years, they grow to reach millions more people and raise millions more in investment. However, despite this tremendous outreach and potential, there are also areas that require attention: funding and gender gaps, inconsistent practices around data privacy and consent, and limited data collection for outcomes management.

Insights from the 2022 Inclusive Fintech 50 Applicant Pool

Consumer privacy risks have proliferated in the last decade in parallel with the increased adoption of digital technologies and the availability of consumer data. The great onboarding onto digital finance necessitated by COVID-19 also opened up vulnerable consumers to data misuse through scams in markets like the Philippines, Kenya, and India, among many others. Privacy violations can also have serious long-term consequences for financial inclusion and, even as access has improved, trust remains a critical ingredient in translating that progress into sustainable gains for consumers.

Evidence shows that when consumers fall prey to privacy-related harms, the loss of trust dissuades them from the opportunities afforded by digital finance. Privacy implications not only affect consumers but also impact businesses. While compliance with new data protection regulations has brought stronger safeguards, the rapid speed at which consumer data is collected, used, and shared in digital finance has far outpaced consumers’ understanding. For low-income digital financial consumers, the limits of informed consent and choice are even starker.

This report is intended to be an introduction to Privacy by Design (PbD) and to review the main PbD approaches in literature and wider market practice for their potential applications in the inclusive finance sector. The analysis is based on a literature review and key informant interviews with privacy and inclusive finance experts conducted by the Center for Financial Inclusion (CFI) between May and August 2022. The paper groups findings into four approaches to PbD, each of which attempts to embed privacy through different means, often by combining various tools, methods, and areas of application. The paper also identifies a series of gaps and barriers that limit the potential of PbD, particularly in emerging markets and developing economies. The report closes with considerations for the adaption of PbD for fintechs working with low-income and vulnerable consumers. CFI aims to build off this initial report to develop practical guidance on PbD for fintech product teams.

Embedding Trust: The Potential of Privacy by Design for Inclusive Finance

<span data-contrast="auto">“</span><i><span data-contrast="auto">Essentially all models are wrong, but some are useful,</span></i><span data-contrast="auto">” British statistician George E. P. Box declared. Despite progress made in machine learning and artificial intelligence, digital credit models risk being considered “wrong, but useful” – useful for certain groups such as urban salaried workers, but wrong as they leave behind large segments of the economy whose data trails are too faint or complex for a credit scoring algorithm. </span><span data-ccp-props="{&quot;201341983&quot;:0,&quot;335559739&quot;:160,&quot;335559740&quot;:259}"> </span>

<span data-contrast="auto">CFI recently explored how automated decision-making tools for digital credit are playing out for one demographic in India – </span><a href="https://www.habitat.org/sites/default/files/documents/Leaving-the-City-Behind_Rapid-assessment-with-migrant-workers-Maharashtra.pdf" target="_blank" rel="noopener noreferrer"><span data-contrast="none">the more than 54 million</span></a><span data-contrast="auto"> interstate migrants – </span><span data-contrast="none">and what this might teach us about the ways data inputs and algorithms intersect with the digital lives of those at the margins.</span>
<h1><span class="TextRun MacChromeBold SCXW59723147 BCX0" lang="EN-US" xml:lang="EN-US" data-contrast="auto"><span class="NormalTextRun SCXW59723147 BCX0" data-ccp-charstyle="Strong">Digital </span><span class="NormalTextRun SCXW59723147 BCX0" data-ccp-charstyle="Strong">C</span><span class="NormalTextRun SCXW59723147 BCX0" data-ccp-charstyle="Strong">redit in India is </span><span class="NormalTextRun SCXW59723147 BCX0" data-ccp-charstyle="Strong">B</span><span class="NormalTextRun SCXW59723147 BCX0" data-ccp-charstyle="Strong">ooming, </span><span class="NormalTextRun SCXW59723147 BCX0" data-ccp-charstyle="Strong">Y</span><span class="NormalTextRun SCXW59723147 BCX0" data-ccp-charstyle="Strong">et </span><span class="NormalTextRun SCXW59723147 BCX0" data-ccp-charstyle="Strong">L</span><span class="NormalTextRun SCXW59723147 BCX0" data-ccp-charstyle="Strong">eaving </span><span class="NormalTextRun SCXW59723147 BCX0" data-ccp-charstyle="Strong">B</span><span class="NormalTextRun SCXW59723147 BCX0" data-ccp-charstyle="Strong">ehind </span><span class="NormalTextRun SCXW59723147 BCX0" data-ccp-charstyle="Strong">L</span><span class="NormalTextRun SCXW59723147 BCX0" data-ccp-charstyle="Strong">arge (and</span> <span class="NormalTextRun SCXW59723147 BCX0" data-ccp-charstyle="Strong">P</span><span class="NormalTextRun SCXW59723147 BCX0" data-ccp-charstyle="Strong">ossibly</span> <span class="NormalTextRun SCXW59723147 BCX0" data-ccp-charstyle="Strong">V</span><span class="NormalTextRun SCXW59723147 BCX0" data-ccp-charstyle="Strong">iable) </span><span class="NormalTextRun SCXW59723147 BCX0" data-ccp-charstyle="Strong">S</span><span class="NormalTextRun SCXW59723147 BCX0" data-ccp-charstyle="Strong">egments of the </span><span class="NormalTextRun SCXW59723147 BCX0" data-ccp-charstyle="Strong">E</span><span class="NormalTextRun SCXW59723147 BCX0" data-ccp-charstyle="Strong">conomy</span></span></h1>
The fintech ecosystem has ballooned in India in recent years, with an estimated 2,000 fintech startups and 300 digital lenders, including Buy-Now-Pay-Later (BNPL) players, operating by the end of 2020. Digital lending alone is expected to reach <a href="https://www.reuters.com/article/india-tech-financing-idUSL5N2NK4XL" target="_blank" rel="noopener noreferrer">$1 trillion by 2023</a>. While digital lenders existed prior to the pandemic, they have boomed over the last two years in the rush to conduct business and daily tasks remotely.

Earlier this year, CFI conducted desk research and interviews across the digital finance ecosystem in India and learned that most digital lenders are focused on serving urban, young, and employed consumers. With higher and more predictable income streams, this customer segment creates large volumes of digital data that can be mined to determine creditworthiness. And given the overall size of the Indian market, there is still room to grow before even this segment is saturated. But where does that leave the rest of consumers, including migrants?

While migrants have propelled the growth of vital industries such as <a href="https://cprindia.org/wp-content/uploads/2022/04/Push-the-policy-needle-forward-on-migrant-support.pdf" target="_blank" rel="noopener noreferrer">construction, hospitality, and commercial agriculture</a>, they historically have faced hurdles in accessing formal financial services and government benefits. And now with the increasing reliance on digital data to allocate benefits and make credit decisions, migrants potentially could be further marginalized.
<blockquote class="pullquote--right">A migrant's data trails do not fit the mold built for the urban, young, employed segment digital lenders were designed to serve.</blockquote>
Take the case of Mohan Kumar, an archetypal migrant whose data trails systematically exclude him from financial services. Mohan works in Surat, India in a garment factory, but is originally from a village near Ganjam, Odisha where his family still lives. Recently married, he visits his wife, who now lives with his parents back in Ganjam, for two weeks twice a year. On his last visit home, Mohan and his wife decided to open a stationary store. When seeking advice from friends, Mohan learned about digital loans available through the phone. Although small, these loans helped a few of his peers with fledgling, entrepreneurial ventures. Despite having a steady, full-time job in Surat for the last four years, Mohan is rejected again and again from numerous digital lenders. Simply put, his data trails do not fit the mold built for the urban, young, employed segment they were designed to serve.

Millions of migrants face similar challenges as Mohan. In general, migrants have fainter data trails due to lower levels of digital activity. Many low-income migrants have limited financial and digital capability and do not possess smartphones. And those who do have a smartphone typically only use mobile data when necessary due to the high cost. Additionally, migrants often rely on agents, such as kirana store owners, to assist them with digital transactions, and these transactions may not be directly associated with their identity.

Furthermore, digital lending algorithms do not understand the data trails of migrants very well. Migrants’ mobility causes multiple barriers. Some migrants change phones and phone numbers between their destination and home, which can impede the verification of transactions or authorizing identity via one-time-passwords (OTP), leading to KYC failures. Additionally, a mismatch between the location of the cell tower where a potential borrower applies for a loan and the location where the phone’s SIM is registered could cause the applicant to be rejected. While providers rationalize that the geographic mismatch might imply that a customer would be hard to reliably reach, unfortunately, this can unintentionally screen out viable borrowers who are inherently on the move, such as migrants.
<blockquote class="pullquote--left">Migrants' payment patterns are not well-understood or formalized, organized, or digitized.</blockquote>
When it comes to assessing cashflows to determine repayment capacity, migrants’ digital data trails make it difficult to see the full picture of their economic lives. For example, even if a migrant has established a stable income over multiple years, as is the case with Mohan, often their payment patterns are not well-understood or formalized, organized, or digitized. For example, some contractors provide an advance payment while others pay weekly or monthly, and some cover living expenses for migrants but deduct the costs from their salaries. In addition, migrants’ data trails might show varied consumption patterns in their two locations – for instance, in Surat, Mohan spends his money on accommodations and food, while back in Odisha he spends on household expenses and children’s education. Such variations in earning and spending tend not to fit within lending models’ range of what makes a viable borrower.

Finally, there is the issue of accessibility. Examining the top 30 downloaded digital credit apps in India (15 on the App Store and 15 on the Google Play Store), we found that 28 out of 30 worked only in English, with just 2 offering Hindi. Such limitations favor highly educated segments and inherently exclude vast numbers of potential clients.
<h1>More Effort is Needed to Ensure Digital Credit Does Not Further Exclude Marginalized Groups</h1>
The Reserve Bank of India (RBI) has taken an interest in digital lending and consumer protection issues over the last two years, asking Google to <a href="https://economictimes.indiatimes.com/tech/technology/google-removes-personal-loan-apps-violating-user-safety-policies-from-play-store/articleshow/80267043.cms?from=mdr" target="_blank" rel="noopener noreferrer">remove at least 30 lending apps from the Play store</a> that did not comply with local laws, <a href="https://www.rbi.org.in/Scripts/BS_PressReleaseDisplay.aspx?prid=52589" target="_blank" rel="noopener noreferrer">convening a working group</a> under the chairmanship of executive director Jayant Kumar Dash, and empowering <a href="https://faceofindia.org/index.html" target="_blank" rel="noopener noreferrer">FACE</a> to become a self-regulatory organization (SRO). The working group focused on a range of consumer protection issues including transparency and disclosure, aggressive sales, consent, and algorithmic design. For instance, one of the group’s recommendations suggests that providers have verifiable audit trails for their algorithms and models should be documented to ensure transparency. While a promising direction, little is understood about what this will look like in practice and how it will balance accountability with the proprietary nature of the models.

Moving forward, digital lenders should find ways to understand and assess the unique activities of groups like migrants through more nuanced models and data inputs that give a more complete picture of a migrant’s economic life. For example, do workers travel back to the same destination each working cycle? How many trips home do they make? How have income patterns shifted within their employment – is it consistent or is it varied?
<blockquote>Digital lenders should find ways to understand and assess the unique activities of groups like migrants through more nuanced models and data inputs that give a more complete picture of a migrant’s economic life.</blockquote>
Providers could draw inspiration from the work of <a href="https://indiamigrationnow.org/" target="_blank" rel="noopener noreferrer">India Migration Now</a> (IMN), which is working to understand the nuances of the data trails of migrants. They are preparing to launch a pilot for a customized, digital lending product for migrants that will leverage both traditional and non-traditional indicators to determine creditworthiness. IMN’s scoring model will leverage data inputs such as frequency of travel, place of origin, destination for migration, frequency of remittance transfer, length of stay at migration location, GPS, and mobile top-up amounts. IMN’s focus on migrants requires commitment, research, time, and resources, which may be less attractive to other digital lenders when there are still potential customers with more robust data trails who are easier to reach.
<h1>What Can Be Done to Maximize the Inclusivity of Algorithms?</h1>
<strong>Research</strong>: We must do more to understand <a href="https://escholarship.org/content/qt3gt9x28k/qt3gt9x28k.pdf?t=qx7nai" target="_blank" rel="noopener noreferrer">whether specific segments that are already at risk for exclusion</a> are further included or excluded by algorithmic decisions. Researchers could help by leveraging context-specific research about specific segments – for instance, migrants or <a href="https://ncst.nic.in/" target="_blank" rel="noopener noreferrer">scheduled tribes</a> in India – to help data scientists better understand the data trails of these groups and incorporate insights into models.

<strong>Guidance and Incentives for Providers</strong>: While advancements in auditing and disparate impact testing can measure models' results across a set of sensitive characteristics (e.g. gender, race, religion, ethnic group), providers may not prioritize the inclusivity of their algorithms. In markets like India, for instance, financial service providers continue to focus on growth among customer segments with more “intuitive” data trails and on keeping customer acquisition costs low. In our view, stakeholders such as impact investors and industry associations can engage with providers to provide guidance and/or incentives for inclusive algorithms. CFI <a href="https://dai-global-digital.com/how-four-awardees-are-paving-the-way-for-a-more-equitable-ai-powered-future.html" target="_blank" rel="noopener noreferrer">recently launched a project</a> to help impact investors engage with fintechs on their algorithms during their due diligence processes.

<strong>Government Engagement</strong>: Because this is a very nascent field, more work is needed to test regulatory approaches and efforts to build the public sector’s capacity for supervision. Government entities, such as the Reserve Bank of India, have begun to think <a href="https://www.centerforfinancialinclusion.org/accountability-for-algorithms-a-balancing-act-for-governments">more strategically about the supervision</a> of high-stakes algorithmic decisions. Ongoing work is needed to understand and advance data protection regulation, national AI strategies, and sector-specific initiatives, such as the RBI working group or Project Veritas in Singapore, to name a few.

Digital Credit Algorithms in India: Expanding Access or Cementing Exclusion?

<span class="NormalTextRun SCXW205969893 BCX0">Platforms </span><span class="NormalTextRun SCXW205969893 BCX0">provide </span><span class="NormalTextRun SCXW205969893 BCX0">critical employment opportunities for many underserved populations and are increasingly interconnected with financial services. To advance financial inclusion and support development goals, </span><span class="NormalTextRun SCXW205969893 BCX0">however, </span><span class="NormalTextRun SCXW205969893 BCX0">platforms must focus on the unique digital, financial, and livelihood needs of micro and small enterprises. This </span><span class="NormalTextRun SCXW205969893 BCX0">publication</span> <span class="NormalTextRun SCXW205969893 BCX0">provides an overview</span><span class="NormalTextRun SCXW205969893 BCX0"> of the increasing interconnectedness of</span><span class="NormalTextRun SCXW205969893 BCX0"> platforms and financial service offerings, how platforms can push the frontiers of inclusive finance, how platforms support women, and what is needed from policymakers for platforms to serve broader development goals. </span><span class="NormalTextRun SCXW205969893 BCX0">The audience of this report includes</span><span class="NormalTextRun SCXW205969893 BCX0"> policymakers, platform leaders, donors, international development actors</span><span class="NormalTextRun SCXW205969893 BCX0">,</span><span class="NormalTextRun SCXW205969893 BCX0"> and anyone interested in the potential for financial services and/or platforms to improve livelihoods.</span>

Platforms, Livelihoods, and Inclusive Finance

Imagine you’re a young mother living on the outskirts of Kampala, Uganda. Due to unexpected medical expenses for your son, you decide to apply for a digital loan. You download a credit app that you heard about from a neighbor, answer a few questions about yourself, consent to have data on your phone shared, and wait 30 minutes. Then, you receive a rejection message: “Your credit limit is 0 UGX.”

You heard that having an active mobile wallet would increase your chances of being approved for a loan and made sure your MTN mobile wallet was active before applying. You’re confused about why your loan application was rejected and wonder how to improve your future chances. Most importantly, you’re still unsure how you will cover your son’s medical expenses.

Understanding the reasons behind credit decisions is increasingly more difficult as algorithms, rather than front-line staff, make and communicate outcomes. <a href="https://centerforfinancialinclusion.org/accountability-for-algorithms-a-balancing-act-for-governments/">Complex algorithms</a>, fueled by alternative data, are driving finance decisions about and for customers -- from underwriting to insurance to <a href="https://centerforfinancialinclusion.org/responsible-social-protection-lessons-from-covid-19-digital-cash-transfers/">eligibility for social protection</a>. As algorithms become more commonplace, we must look for ways to increase transparency around decisions and inform low-income customers about their options for rectification.
<h1>In the Dark on Automated Decisions</h1>
In honor of <a href="https://www.consumersinternational.org/what-we-do/world-consumer-rights-day/" target="_blank" rel="noopener noreferrer">World Consumer Rights Day</a>, CFI is researching how credit decisions are made and communicated to low-income consumers, the rights consumers deserve, and the tools they need to understand – and be able to challenge – those decisions.

In the not-so-distant past, most financial institutions relied on front-line staff to make and communicate decisions to customers. If a customer was rejected for a loan or insurance product, she could ask a loan officer to explain how her history was evaluated and could work with the officer to correct data errors. However, as digital models leveraging algorithms gain traction, the transparency that customers used to rely on from their in-person engagements with loan officers or agents has disappeared.
<blockquote class="pullquote--right">As digital models leveraging algorithms gain traction, the transparency that customers used to rely on from their in-person engagements with loan officers or agents has disappeared.</blockquote>
Because most customers are unaware of how an underwriting algorithm works or what data inputs are used, many customers are also unclear about how a decision is made or whom to ask for more information. This lack of clarity was evident from <a href="https://centerforfinancialinclusion.org/trust-of-data-usage-sources-and-decisioning-perspectives-from-rwandan-mobile-money-users/">CFI’s survey-based research</a> in Rwanda. Out of a sample of 30 digital borrowers in Rwanda, 16 respondents had been rejected for a digital loan but only 10 of those recalled an explanation for the denial. Of those 10 who received an explanation, six were dissatisfied with the provider’s communication; some received only a basic explanation such as, “Your credit limit is zero.”

When the full survey sample was asked to describe what an acceptable explanation of loan denial might look like, there was a clear ask for more specificity and more communication. A 52-year-old woman shared her experience: “They denied me a loan because I delayed to repay when I was in a hospital and very sick. If they had called to explain the reason for the denial, I would have explained my condition.” A 38-year-old woman reasoned: “When you delay to repay, they call to remind you about the loan. They should then do the same when you are denied a loan.”
<h1>Help for Consumers: Legislation, Awareness, Support</h1>
So, what can consumers do in this increasingly digital era? Most data protection frameworks give consumers the right to access and rectify their data. However, our Rwanda research suggests that consumers are largely unaware of what is being inputted in the first place. While data protection efforts – like the 2019 Kenyan Data Protection Act that gives individuals the right to request the rectification of personal data that is “inaccurate, out-of-date, incomplete or misleading” – are a good first step, more needs to be done to communicate to customers about the inputs, decisions, and digital rights.
<blockquote class="pullquote--left">More needs to be done to communicate to customers about the inputs, decisions, and digital rights.</blockquote>
Many nascent data protection frameworks offer consumers the right to be informed if they’ve been subjected to an automated decision by an algorithm, for example. For instance, the <a href="https://www.minijust.gov.rw/fileadmin/user_upload/Minijust/Publications/Official_Gazette/_2021_Official_Gazettes/October/OG_Special_of_15.10.2021_Amakuru_bwite.pdf" target="_blank" rel="noopener noreferrer">Rwandan Data Privacy Law</a> mandates that individuals should be informed about the logic involved in their automated decision at the time of personal data collection. <a href="https://www.dataguidance.com/notes/brazil-data-protection-overview" target="_blank" rel="noopener noreferrer">Brazil’s Data Protection Act</a> gives consumers the ability to ask for a review of a decision made with their personal data taken solely through automated processing; included in the review should be the criteria and procedures used for the decision.

But what this will look like in practice, and how it will be enforced, remains to be seen. And given what we know about the challenges facing low-income consumers in accessing and feeling empowered to use grievance redressal mechanisms, there is concern that these data rights will not be fully exercised in a way that keeps companies accountable.

Consumer advocacy organizations are a potential avenue to help raise the voice of consumers vis-à-vis their digital rights. For instance, <a href="https://centerforfinancialinclusion.org/responsible-social-protection-lessons-from-covid-19-digital-cash-transfers/">our research</a> on government-to-person digital payments during COVID-19 found that civil society organizations often were the first to sound the alarm about digital systems failing or customers poorly served.

Whether it’s a mother in Kenya applying for a credit product to pay for medical expenses, or a farmer looking to insure his crops in the face of an increasingly unpredictable climate, we must support consumers’ voices and increase transparency around how digital decisioning is made.

<em>Along with other leading consumer advocates around the world and in partnership with <a href="https://www.consumersinternational.org/" target="_blank" rel="noopener noreferrer">Consumers International</a>, we’re excited to participate at the <a href="https://www.consumersinternational.org/fair-digital-finance-forum-2022/" target="_blank" rel="noopener noreferrer">Fair Digital Finance Forum</a> this week. Take a look at their <a href="https://www.consumersinternational.org/fair-digital-finance-forum-2022/" target="_blank" rel="noopener noreferrer">program</a> to get a lay of the land and stay up to date with CFI’s <a href="https://www.centerforfinancialinclusion.org/research/consumer-protection">consumer protection</a> workstream.</em>

Understanding the Decision: Consumer Rights and Algorithmic Decision-Making

Lucciana Alvarez Ruiz

Alexandra (Alex) Rizzi

Eda Dokle

Devina Srivastava

Mayada El-Zoghbi

Loretta Michaels

Jayshree Venkatesan

Pia  Roman Tayag

Ezra Mannix

Tanwi Kumari

Daryl  Collins

Derry Moore

Maria Gabriela Coloma Ponce de Leon

Alex Kessler

Jacobo Menajovsky

India

South Asia

Rwanda

Sub-Saharan Africa

Kenya

Alternative Data

Responsible Product Design and Delivery

Women

Digital Financial Services

Fintech

Impact Investing

Underserved Groups

Enabling Environment

Financial Inclusion Strategy

Fintechs

Consumer Protection

Data Privacy

Blockchain

Challenges and Crises

Small Business Owners

Credit

Digital Payments

Insurance

Remittances

Refugees and Migrants

Partnerships

Policy, Regulation and Government Initiatives

Data Transparency

Financial Capability

Human Centered Design

Mobile Banking

Responsible Digital Finance

Underwriting

Blog Post

Toolkits and Guides

Report

Announcement

Brief

India is a global leader in financial inclusion. For over a decade, Accion has worked with and learned from India’s innovators, investors, and policymakers to expand economic opportunity for its people and businesses.

Consumers all over the world may not know it, but many decisions about and for them are increasingly made by algorithms. Eighty percent of viewing activity on Netflix is driven by recommendations from algorithms; a doctor might recommend a course of treatment based on an algorithm that uses patient DNA; <a href="https://www.poverty-action.org/study/using-mobile-phone-and-satellite-data-target-emergency-cash-transfers-togo" target="_blank" rel="noopener noreferrer">social protection programs</a> might determine eligibility for an assistance payment using an algorithm that crunches your mobile phone transactions; and financial providers can determine your creditworthiness or insurability based on algorithms that leverage alternative data.

A persistent challenge in reaching low-income and vulnerable populations with financial services has been a limited availability of data. In recent years the growth of consumer data trails in emerging markets, largely generated by mobile phones, has helped change the equation alongside powerful new analytical tools. But what data goes into consequential algorithms, how it is used, and whether it leads to inclusion — or bias and exclusion — is largely a black box to consumers, as well as governments. While the expansion in consumer data and companies leveraging artificial intelligence (AI) technology can advance financial inclusion, these technologies introduce new risks and may reinforce historical patterns of bias and exclusion, with potentially harmful consequences for consumers.
<blockquote class="pullquote--right">These technologies introduce new risks and may reinforce historical patterns of bias and exclusion, with potentially harmful consequences for consumers.</blockquote>
Central banks and policymakers have long faced the challenge of balancing financial innovation, market growth, and inclusion alongside competition, stability, and consumer safety. That balancing act has been made more difficult by the speed and scope of financial innovation, technology, and algorithms of increasing sophistication. In emerging markets and developing countries in particular, the increased use of mobile phone data for consequential decisions has raised the profile and potential risks of these tools.

There is a growing recognition that governments require ways to assess consumer risks and opportunities in the deployment of these algorithms and AI systems, as well as methods to amplify the opportunities and mitigate risks — all without quashing innovation. But, how?

Earlier this month, H.M. Queen Máxima of the Netherlands, the United Nations Secretary-General’s Special Advocate for Inclusive Finance for Development (UNSGSA), <a href="https://www.centerforfinancialinclusion.org/unsgsa-and-center-for-financial-inclusion-co-host-virtual-roundtable-for-central-banks-to-tackle-algorithm-biases-in-financial-services">co-hosted a virtual workshop</a> with the Center for Financial Inclusion (CFI) to raise awareness and address algorithm biases for financial inclusion. Queen Máxima and CFI Managing Director Mayada El-Zoghbi co-chaired the meeting, which convened leadership from select central banks and other subject matter experts. The workshop examined how to identify and assess these emerging risks, discussed existing risk mitigation tools, and identified opportunities for further research.

The challenge of bias in financial services is not new and many countries already have financial sector anti-discrimination laws that hold financial institutions to fair lending practices, including just under half of emerging markets. The Equal Credit Opportunity Act in the United States, which was passed in 1974, for instance, makes credit discrimination based on protected classes illegal. But it runs into limits in the face of algorithmic lending as it also <a href="https://www.centerforfinancialinclusion.org/reducing-bias-in-algorithmic-decisions-cannot-rely-on-blind-approaches">restricts lenders</a> from collecting demographic information on borrowers. Given the current methods for testing algorithms, this missing demographic data makes disparate impact testing much harder. While the Equal Credit Opportunity Act and similar antidiscrimination laws could be used to hold companies accountable, they will grapple with understandable privacy concerns.

A second potential avenue for accountability could be through data protection laws, the most well-known being the European Union’s General Data Protection Regulation (GDPR).  While GDPR requires that processors assess any high-risk data system, like a new credit scoring algorithm, it does not amount to an audit. GDPR and the laws it has inspired also enumerate individual rights for consumers including the right to an explanation of an automated decision and the right to have your data rectified by a provider if it is incorrect. But we worry, how likely is it that low-income consumers (or consumers writ-large, really) will be aware enough to be empowered by these rights?  If <a href="https://www.centerforfinancialinclusion.org/trust-of-data-usage-sources-and-decisioning-perspectives-from-rwandan-mobile-money-users">consumers are not aware of how their data is being used</a>, they are less likely to exercise their rights, raise their collective voice, and seek accountability for algorithmic decisions.
<blockquote>If consumers are not aware of how their data is being used, they are less likely to exercise their rights, raise their collective voice, and seek accountability for algorithmic decisions.</blockquote>
In recognition of the limits of existing approaches, some countries have drafted national AI policies or frameworks. These policies work to establish the level of risk for discrete AI systems and how to assign levels of supervision. For instance, the EU introduced a draft law that would require <a href="https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A52021PC0206" target="_blank" rel="noopener noreferrer">Algorithmic Impact Assessments</a> for high-risk systems, such as automated credit decisions. Such proposed auditing would require technical expertise within governments and some degree of disclosure of the source code and data. We foresee challenges in implementing such audits in many markets due to gaps in government resources and staffing, as well as challenges in creating standards for auditing that balance supervision with innovation and the proprietary nature of the codes.

One of the ways that governments are engaging is through test and learn partnerships with fintechs. For instance, in the US the Consumer Financial Protection Bureau (CFPB) has granted a <a href="https://www.consumerfinance.gov/about-us/newsroom/cfpb-announces-first-no-action-letter-upstart-network/" target="_blank" rel="noopener noreferrer">no action letter</a> to Upstart Networks, an online lender that uses alternative data. The letter essentially immunizes Upstart from regulatory action and in exchange, Upstart provides information about its loan applications, methodology for decision-making, and whether its loans expand financial inclusion. Other governments have also conducted test and learn engagements with fintechs, as well as create fora for enhanced private-public engagement such as the AI Forum in the Netherlands and Project Veritas in Singapore. Other emerging approaches include working across regulators, such as the Digital Markets Taskforce in the UK.

While there are some strong examples and initial work that could be shared, clearly, there is also still much to learn. What enhancements to existing legal frameworks are necessary to address the new or shifting risks in the use of AI in digital financial services? What digital public goods can help mitigate attendant risks? How can supervisors and regulators build capacity to understand data inputs and use cases, and develop principles and assessment methodologies that are regularly evaluated? These questions would best be answered through research and dialogue among and across regulators, as well as the private sector. Only through a concerted effort will the enormous potential of algorithms and AI extend the reach of financial services in a manner that is inclusive, safe, and fair for customers.

Accountability for Algorithms: A Balancing Act for Governments

The past year was one of building on what became our new reality in 2020.

While the full impact of the pandemic remains unknown, we know that financial services play a crucial role in mitigating the continued effects on livelihoods. We know that financial services are important for changing norms to achieve women’s economic empowerment. We also know that understanding data risks is needed for protecting low-income consumers against fraud and other hazards. And we know that financial services are needed for underserved customers so they can be resilient in the face of a changing climate.

So how have we advanced our learning on these key lines of inquiry in 2021?

The list below – 10 pieces of selected research, analysis, and recommendations from the year – is not exhaustive: CFI published 15 publications and nearly 30 blog posts this year. But it offers a cross-section of how CFI is thinking about the biggest challenges in our sector.

We look forward to furthering our learning in 2022 and continuing to advance inclusive financial services. We wish everyone a healthy new year with shared prosperity for all.
<h3><a href="https://www.centerforfinancialinclusion.org/the-stories-algorithms-tell-bias-and-financial-inclusion-at-the-data-margins">The Stories Algorithms Tell</a></h3>
<em>By Alexandra Rizzi, Alex Kessler, and Jacobo Menajovsky</em>

This flagship publication grounds universal challenges on the use of algorithms, automated decisions, alternative data, and bias in the context of inclusive financial services to present the current state of play among inclusive finance actors. The report is based on desk research and interviews from a sample of fintechs, regulators, and other experts.
<h3><a href="https://www.centerforfinancialinclusion.org/normative-constraints-to-womens-financial-inclusion-what-we-know-and-what-we-need-to-know">Normative Constraints to Women’s Financial Inclusion: What We Know and What We Need to Know</a></h3>
<em>By Julia Arnold, Mayada El-Zoghbi, and Alex Kessler</em>

Financial services design and delivery play a part in ensuring women retain control over resources. But norms are increasingly recognized as a key constraint to women’s financial inclusion. This report reviews evidence on social and gender norms and financial inclusion and identifies areas for further improvement to understand and address the norms that influence women’s participation in the economy and the financial system.
<h3><a href="https://www.centerforfinancialinclusion.org/green-inclusive-finance-a-framework-for-understanding-how-financial-services-can-help-low-income-people-weather-climate-related-risks">Green Inclusive Finance</a></h3>
<em>By Howard Miller</em>

The role of financial services in helping low-income consumers respond to climate change is increasingly gaining acceptance in the inclusive finance realm as a key component of the global response to the great crisis of our time. This video and blog post map out CFI’s framework for green inclusive finance and provide four distinct but interrelated pathways for understanding how inclusive finance can help low-income populations prepare for and respond to climate change while improving green outcomes.
<h3><a href="https://www.centerforfinancialinclusion.org/creating-prosperity-in-an-era-of-uncertainty">Promoting Prosperity in an Era of Uncertainty</a></h3>
<em>By Mayada El-Zoghbi</em>

In her welcome letter to <a href="https://www.centerforfinancialinclusion.org/about/what-we-do/financial-inclusion-week">Financial Inclusion Week</a>, CFI’s Managing Director discusses the biggest trends in and challenges facing inclusive finance amid a pandemic that already has pushed an estimated 97 million more people into poverty. She provides a bird’s-eye view of the measures taken to avert the crisis, noting the bright spots – rapid mobile money adoption, impressive investments in inclusive fintechs, the flowering of the platform and gig economies – while striking a cautious tone regarding the challenges ahead.
<h3><a href="https://www.centerforfinancialinclusion.org/the-precarious-state-of-msmes-understanding-the-impact-of-covid-19-and-opportunities-to-support-their-recovery">The Precarious State of MSMEs: Understanding the Impact of COVID-19 and Opportunities to Support their Recovery</a></h3>
<em>By Shradha Modi, Katia Huayta Zapata, Lauren Braniff, and Aeriel Emig</em>

This brief shares how COVID-19 has impacted MSMEs over the last year and a half in four emerging markets: Colombia, India, Indonesia, and Nigeria. Backed by <a href="https://www.centerforfinancialinclusion.org/research/covid-19-and-msmes-data-and-analysis-to-understand-impact">six waves of survey data</a> collected from owners and operators of micro, small and medium enterprises (MSMEs), this brief offers four areas to focus on to navigate today’s challenges and build resilience for tomorrow.
<h3><a href="https://www.centerforfinancialinclusion.org/weathering-the-storm-ii-tales-of-survival-from-microfinance-crises-past">Weathering the Storm II: Tales of Survival from Microfinance Crises Past</a></h3>
<em>By Daniel Rozas</em>

A follow up to the <a href="https://www.centerforfinancialinclusion.org/weathering-the-storm-hazards-beacons-and-life-rafts-lessons-in-microfinance-crisis-survival-from-those-who-have-been-there">original report</a> and case studies, published in 2011 in the aftermath of the 2007-09 financial crisis, Weathering the Storm II looks at the decade that followed the original paper and shares what became of the organizations profiled within it and how they’re facing the COVID-19 crisis. Nearly every type of financial institution – from NGO to bank – is included in this report that synthesizes experiences and lessons from 16 institutions in 14 countries on four continents over 15 years (including five new case studies from Azerbaijan, Bosnia and Herzegovina, India, Nicaragua, and Palestine).
<h3><a href="https://www.centerforfinancialinclusion.org/yes-covid-is-the-big-bang-of-digital-payments">Yes, COVID is the Big Bang of Digital Payments</a></h3>
<em>By Loretta Michaels</em>

With over 300 million monthly active users in 2020 (up 17 percent year-over-year), mobile money has seen explosive growth. Not only is it an extremely useful tool for businesses and individuals, but mobile money has also been integral for rapid government-to-person (G2P) transfers in many countries in the first months of the pandemic. But there’s still untapped potential for mobile money in a post-pandemic world, especially for governments seeking universal access to financial services.
<h3><a href="https://www.centerforfinancialinclusion.org/trust-of-data-usage-sources-and-decisioning-perspectives-from-rwandan-mobile-money-users">Trust of Data Usage, Sources, and Decisioning: Perspectives from Rwandan Mobile Money Users</a></h3>
<em>By Alex Rizzi and Tanwi Kumari</em>

Who is fairer: loan officers or digital lenders? The publication shares highlights of a qualitative study of 30 mobile money users in Rwanda that aims to answer this question. The survey focuses on user perceptions of and opinions on consumer data in underwriting and the evolving data ecosystem.
<h3><a href="https://www.centerforfinancialinclusion.org/leveraging-digital-financial-capability-to-drive-womens-financial-inclusion">Leveraging Digital Financial Capability to Drive Women’s Financial Inclusion</a></h3>
<em>By Julia Arnold and Marina Dimova</em>

Financial capability must evolve to include digital financial capability. While digital tools offer promise, there is a risk of excluding women, oral communities, and others from the formal financial system, if focus is not given to helping people gain the knowledge and skills to effectively use digital financial services. This post provides a framework for developing effective digital financial capability programs and action steps to guide the way.
<h3><a href="https://www.centerforfinancialinclusion.org/reducing-bias-in-algorithmic-decisions-cannot-rely-on-blind-approaches">Reducing Bias in Algorithmic Decisions Cannot Rely on “Blind” Approaches</a></h3>
<em>By Alex Kessler and Jacobo Menajovsky</em>

What happens when gender-blind credit scoring is applied to financial products and services? We see products like the infamous Apple Card, which gave women consistently lower lines of credit in what researchers call “fairness through unawareness,” an approach that intentionally excludes data on sensitive attributes like race, ethnicity, or sex to reduce discriminatory outcomes. This post addresses the need for a better understanding of algorithms transforming the financial sector to mitigate against discriminatory outcomes while respecting limitations of data gathering needed to protect privacy.

Top 10 of 2021: CFI Research that Furthered Our Thinking in a Pivotal Year

The <a href="https://www.inclusivefintech50.com" target="_blank" rel="noopener noreferrer">Inclusive Fintech 50</a> application data from the third year signals continued progress on innovations to onboard low-income customers, despite the challenges posed by the pandemic:
<ul>
 	<li><strong>Inclusive fintechs are using creative approaches to serve new customers</strong>. From retail partnerships that help drive customer acquisition, to insurance platforms offering micro-coverage, to native language chatbots, to enhanced biometric identity verification solutions, early-stage inclusive fintechs are pursuing outside-the-box solutions to reach un/underbanked people around the world.</li>
</ul>
Nevertheless, there are two areas of concern:
<ul>
 	<li><strong>Funding concentration persists</strong>. While fintech investments across emerging markets have been increasing, early-stage inclusive fintechs continue to face disparities based on geography, funding status, and gender.</li>
 	<li><strong>Potential customer risks persist around data usage</strong>. Although data is central to the business models of inclusive fintechs, some applicants require customer consent related to data, and some still do not.</li>
</ul>

Insights from the 2021 Inclusive Fintech 50 Applicant Pool

H.M. Queen Máxima of the Netherlands, the United Nations Secretary-General’s Special Advocate for Inclusive Finance for Development (UNSGSA), today co-hosted a virtual workshop with the Center for Financial Inclusion (CFI) at Accion to raise awareness and address algorithm biases for financial inclusion.

Queen Máxima and CFI Managing Director Mayada El-Zoghbi co-chaired the meeting, which convened leadership from select central banks and other subject matter experts. Both the UNSGSA and the Managing Director provided brief opening remarks. President and CEO of Accion Michael Schlein also offered brief remarks.

[caption id="attachment_43591" align="alignright" width="559"]<img class="size-full wp-image-43591" src="https://content.centerforfinancialinclusion.org/wp-content/uploads/sites/2/2021/12/unsgsa-cfi-workshop-algorithm-bias-web-story-foto-21.jpg" alt="" width="559" height="322" /> <em>UNSGSA Queen Máxima (top left) is pictured during a virtual workshop for central banks to address algorithm biases for financial inclusion, which the Special Advocate co-hosted with Center for Financial Inclusion Managing Director Mayada El-Zoghbi (top right). Also pictured are Office of the UNSGSA Director Pia Tayag (bottom left) and Accion President and CEO Michael Schlein. Other attendees are not shown, in accordance with Chatham House Rule.</em>[/caption]

Artificial intelligence (AI)-driven digital financial services hold tremendous potential for financial inclusion. For example, the rapid growth in mobile phone usage and increase in new customer data have resulted in novel methods to deliver financial services by leveraging AI and data — especially in emerging markets and economies.

However, these new innovations can be accompanied with new risks or reinforce previous biases and exclusion issues, with potentially harmful consequences for consumers. This is particularly relevant for vulnerable groups like the poor, women, smallholder farmers, and small businesses.

Attendees used the opportunity to exchange knowledge and learn lessons from the experiences of their counterparts from different institutions and countries on these issues. They were encouraged to examine how to more effectively diagnose problems and identify risks related to AI-driven digital financial services.

Participants also examined how governments could better monitor and mitigate the risks associated with using algorithms for credit and insurance products in their own markets. Further, attendees used the time to consider the role of the private sector in finding solutions and ensuring future innovations are safe and fair for consumers.

The workshop was designed to help participants better understand the risks associated with algorithm biases, as well as to create a research agenda to develop possible strategies and solutions to tackle these problems.

The meeting was conducted under Chatham House Rule. For further reading on the topic, please see CFI’s report, <a href="https://www.centerforfinancialinclusion.org/the-stories-algorithms-tell-bias-and-financial-inclusion-at-the-data-margins">"The Stories Algorithms Tell: Bias and Financial Inclusion at the Data Margins."</a>

UNSGSA and Center for Financial Inclusion Co-Host Virtual Roundtable for Central Banks to Tackle Algorithm Biases in Financial Services

Trust of Data Usage, Sources, and Decisioning: Perspectives From Rwandan Mobile Money Users

<blockquote> “Our data is stored in servers, so government can make strict rules regarding not hampering privacy of citizens. We need to have trustworthy individuals in charge of managing the servers."</blockquote>
&nbsp;
<blockquote>“I felt scared [when looking at the data Google stores] – it wasn’t just that they were collecting my location, they were collecting a pattern of my locations so they knew where I would be at a certain time.”</blockquote>
These quotes illustrate a subtle but important difference in the approach to digital experiences expressed by men and women. The first comes from Prajit, a 57-year-old Indian man, and reflected a dispassionate policy-oriented viewpoint. The second comes from, Alison, an 18-year-old Asian woman in California. Her sense of personal violation is not unusual. From Ohio to Assam, a shared discourse of fear and invasion in digital experiences runs through many women’s digital worldviews.

Thanks to generous sponsorship from PayPal, we have spent the past several months speaking to a small sample of low- to middle-income men and women in the U.S. and India with the intent to probe themes relating to experience, behavior, and attitudes in the digital world, including digital financial services. These results add to a body of work -- such as work on the exposure of negative bias towards women in the algorithms used by digital lenders (see <a href="https://www.womensworldbanking.org/insights-and-impact/report-algorithmic-bias-financial-inclusion-and-gender-primer-opening-up-new-credit-to-women-in-emerging-economies/" target="_blank" rel="noopener noreferrer">here</a> and <a href="https://centerforfinancialinclusion.org/the-stories-algorithms-tell-bias-and-financial-inclusion-at-the-data-margins/">here</a>) -- about how women’s digital data can work against rather than for them.

Our conversations took place in two parts. In the first interview, we mapped out the digital lives of respondents. Through this mapping, we were able to capture the whole of respondents’ “portfolios” of digital tools and services -- including not only digital financial services, but also social media, e-commerce, communications, and any other digital tool or service that might capture and use customer digital data.

In preparing for the second conversation, we informed respondents about how their most-used providers kept -- and used -- specific data. For example, with users of any Google product, we explained -- and, where possible, showed -- how they could see in Google Takeout the locations they had been, their YouTube videos, their search activity, their email, their contacts, and many other pieces of data. We explained that this data helps to create user profiles that suppliers can then target.

We then had an open-ended conversation about what they knew -- or did not know -- and how they felt about this information. What emerged from these conversations was a bifurcation in digital philosophies, a gendered digital worldview perpetrated by and perpetuated in the specific discourses of these men and women, particularly in discussions of digital exposure.
<blockquote class="pullquote--right">What emerged from these conversations was a bifurcation in digital philosophies, a gendered digital worldview.</blockquote>
Men and women in our sample approached the topic of digital exposure -- the extent to which personal, demographic, and behavioral data is accessible or shared with digital service providers -- in very different ways.

In discussions about digital exposure, men overwhelmingly were philosophical. Their concerns lay in abstracted notions of right and wrong, what should be done, and by whom, and tended toward a generalized, high-level discourse.

For example, Collin, a 23-year-old Kenyan migrant living in New York City, spoke in terms of a moral code on the topic of the sale of personal data:
<blockquote class="blockquote">“[For a company to] share your secrets, it's something that it's not in line with accepted code of conduct.”</blockquote>
<h2>Through a Lens of Fear</h2>
Women, however, were more inclined to frame their exposure on a personal level and usually through a lens of fear. For these women, their concerns parallel the challenges and threats they encounter in their physical lives. They tended not to debate about morality or responsibility, or digital exposure as a society-level concern. Rather, they spoke at length -- and in visceral analogies -- about their safety and invasion of their privacy.

For example, Sonam, a 27-year old Indian woman working in banking, expressed personal fear when discussing real-time location tracking:
<blockquote class="blockquote">“You never know who can follow you from your location data, the thought is quite scary.”</blockquote>
Alisha, a 50-year-old woman from Texas, when informed that her phone or device is recording her, drew on lived experience and spoke of violation:
<blockquote class="blockquote">“It reminds me of when my home was broken into… and he rummaged through my bedroom drawers and that was the part that bothered me the most because he went through my underwear drawer… you went somewhere no one goes.”</blockquote>
Worse, women’s comments suggest they are even less sure how to protect themselves from these personal invasions in the digital world compared to invasions in the physical one. Exposure in their digital lives is a mirror to the risks they face in their physical lives – but they have less information about how they navigate them.

In response to threats, many women find it necessary to curtail their behavior and activities in the physical world to avoid feeling or being violated. While the digital world could -- and should -- be different, where women can take full part in all it has to offer without feeling threatened, the reality is starkly contrary. When it comes to physical safety, women provide advice to each other about how to protect themselves in different contexts – do not walk alone at night if you can avoid it; lock your car doors; if you put your drink down, do not pick it up again. These constraints are frustrating and limiting, but at least they are known.

While <a href="https://www.icrw.org/publications/technology-facilitated-gender-based-violence-what-is-it-and-how-do-we-measure-it/" target="_blank" rel="noopener noreferrer">technology-facilitated gender-based violence</a>, such as cyberbullying and online harassment, has been acknowledged and discussed, the difference here is that these women are not reacting to a stranger harassing them online. This is a sense of violation from a service they need to use to engage in their personal and business lives. The strategies for protecting oneself against this kind of invasion are nebulous at best. Indeed, the threats themselves are often unknown and obscured. And this lack of knowledge is perceived as a violation among the women we interviewed.

Danielle, for example, analogizes this lack of knowledge to date rape:
<blockquote class="blockquote">“If you’re a woman and someone drugs you and rapes you and you wake up and you don’t know that you’ve been drugged and raped. Then you see the video and you realize that this has happened, but before you see the video you did not even know you had been violated. So, I feel there’s some aspect of this that feels off about it — like I’ve been raped, and I don’t even know that I’ve been raped so I feel not only violated but tricked into not knowing.”</blockquote>
Through these conversations, a gendered narrative emerges that belies a split between men's and women’s digital worldviews. Worldviews that influence and determine their digital behaviors and lives. These interviews <a href="https://dalberg.com/our-ideas/privacy-line/" target="_blank" rel="noopener noreferrer">and others</a> suggest a common desire from many women who say they have or would like to curtail their activities on specific digital products because they are concerned about safety. But opting out of digital tools is at an end of a behavioral spectrum. Simply limiting the use of products creates a digital divide of its own.

Governments and digital service providers alike benefit from an increasingly digital population. For example, government-to-person transfer programs in developing countries are increasingly dependent on digital technologies to deliver services and provide an interface to the public. Digital service providers benefit from a large and growing consumer market. However, both governments and private providers bear increased responsibility in meeting expectations of digital safety for these benefits to be realized.

Both will fail if governments and providers do not better understand the nuances of consumer concerns. This is not currently being done. Eliciting concerns about data security and privacy must consider how experiences are gendered if we want to build – and maintain – trust among women.

Going forward, we will seek to further investigate the nature of women’s digital worldviews and their effects on digital behavior, especially in the realm of digital financial services. Specifically, we look to understand specific barriers women face in using digital services, investigate perceptions of the value of these services, and evaluate the design and function of digital services through a gendered lens.

Gender and Digital Worldviews: Divergent User Perspectives On Data Collection and Use

The majority of Kenyans have mobile money accounts; their activity demonstrates digital financial services’ potential. Accion helps Kenyan innovators harness emerging technologies to create new financial services.

In April 2021, a <a href="http://kenyalaw.org/kl/fileadmin/pdfdownloads/bills/2021/TheCentralBankofKenya_Amendment_Bill_2021.pdf" target="_blank" rel="noopener noreferrer">bill</a> that proposed licensing digital lenders was passed by the Kenyan parliament and is now pending approval from the national assembly. Among other things, this bill would require all digital lenders to submit product pricing to the Central Bank of Kenya for approval before launching, a move that seeks to lower predatory interest rates. Further, non-performing loans would be capped at <a href="https://www.businessdailyafrica.com/bd/economy/mobile-phone-lenders-risk-closure-in-cbk-bill-3376538" target="_blank" rel="noopener noreferrer">twice the amount of the initial loan defaulted</a>, which means lenders won’t be able to perpetually increase the amount to be paid back through late-payment fees.

Much has been <a href="https://www.bloomberg.com/news/features/2020-02-12/tech-startups-are-flooding-kenya-with-apps-offering-high-interest-loans" target="_blank" rel="noopener noreferrer">written</a> on the fintechs that have flooded the Kenyan market since 2016. FSD and CGAP published alarming <a href="https://www.cgap.org/blog/kenyas-digital-credit-revolution-five-years">evidence</a> in 2018 showing that in addition to a saturated digital lending market, an increasing number of digital borrowers in Kenya became over-indebted, which furthered them into <a href="https://www.microsave.net/2019/09/18/11310/" target="_blank" rel="noopener noreferrer">poverty traps</a>. Digital lenders such as OKash and Branch reportedly publicly shamed debtors by <a href="https://restofworld.org/2020/okash-microlending-public-shaming/" target="_blank" rel="noopener noreferrer">texting their friends, family, and colleagues</a>. Further, high-interest rates and non-transparent fees resulted in APR’s above <a href="https://qz.com/africa/1975202/kenya-prepares-to-crack-down-on-high-interest-loan-apps/" target="_blank" rel="noopener noreferrer">400 percent</a>, making it more difficult for some low-income borrowers to repay. In some instances, repayment was possible, yet it meant sacrificing other payments like food, medicine, or school fees. And finally, millions of borrowers were reported to credit bureaus for <a href="https://www.cgap.org/research/publication/consumer-protection-digital-credit" target="_blank" rel="noopener noreferrer">loans smaller than $10</a>.

During the COVID crisis, the Kenyan Central Bank issued a number of measures to ease the debt burden on borrowers, such as <a href="https://www.centralbank.go.ke/uploads/press_releases/850440997_Press%20Release%20-%20Credit%20Reference%20Bureau%20Regulations%20-%20April%202020.pdf" target="_blank" rel="noopener noreferrer">forbidding</a> unregulated mobile-based and credit-only digital lenders from reporting any late borrowers to credit bureaus. Although this helped ease debt stress on consumers during the crisis, this drove many digital lenders out of the market as levels of default skyrocketed, leaving <a href="https://www.theelephant.info/data-stories/2020/09/18/covid-19-regulatory-measures-could-widen-kenyas-financial-access-gap/" target="_blank" rel="noopener noreferrer">many Kenyan borrowers underserved</a>. The entire loan market in Kenya shrank due to COVID-19; today, these platforms lend around <a href="https://www.businessdailyafrica.com/bd/opinion-analysis/editorials/cbk-regulation-of-digital-lenders-is-long-overdue--3378386" target="_blank" rel="noopener noreferrer">2 billion shillings a month</a>, compared to 4 billion shillings in early 2020.

The proposed bill by the Central Bank of Kenya unfortunately will not address the issue of making digital credit both safe and accessible. The bill focuses on product pricing only, and it doesn’t address the underlying challenge of digital lending platforms: a weak credit assessment model. Such a model <a href="https://www.ted.com/talks/shivani_siroya_a_smart_loan_for_people_with_no_credit_history_yet?language=en" target="_blank" rel="noopener noreferrer">extracts large amounts of data from the borrower’s phone</a> and runs it through algorithms that approve or deny the loan. Many digital lenders mitigate the risk of a light-touch credit assessment with above-market APRs and hidden late payment fees, thus being able to generate profit and returns to their investors.

The primary problem is digital lenders do not measure the capacity of someone to repay, but their willingness to repay, and these are drastically different concepts. Willingness refers to the genuine disposition of a person to pay back their loan. Capacity, on the other hand, refers to the person’s ability to generate enough money in the future to pay back the loan without having to forgo essential expenses.
<blockquote class="pullquote--left">Digital lenders do not measure the capacity of someone to repay, but their willingness to repay and these are drastically different concepts.</blockquote>
For example, I might score very well on a digital risk assessment. My geo-trace shows I go to work every day, my call log shows I call my mom regularly, and my browsing history shows I do not engage in sports bets. These measurements indicate I am a responsible citizen who complies with her duties, suggesting that I have a high willingness to repay a loan. However, I might have astronomical medical expenses that the digital lender is unable to detect. Or I might be sending significant remittances to my family. Both of these relate to my repayment capacity and are measures that digital lending apps tend to ignore in favor of automated decision-making based on more readily available data.

By comparison, traditional banks run thorough credit risk assessments, sometimes so thorough that they are slow and expensive. These banks have credit committees and risk analysis departments that challenge relationship managers on the loan proposals they present. The result is that most people granted a loan have the means to pay it back. This method excludes the majority of the world’s population from the financial system, however, it also ensures that most people who are granted a consumer loan actually have the capacity to repay.

So where do we draw the line? How do we expand access to credit to low-income people and informal workers and also protect them from default? How can we measure their capacity to repay in a cheap, digital, and rapid manner? Should we prohibit alternative, digital, phone-based metrics that only assess someone’s willingness to repay?
<h2>A Possible Solution</h2>
The answer could be a measure in between. One idea is that the Central Bank of Kenya could only grant a license to digital lenders that incorporate the borrower’s capacity to repay in their credit assessment algorithms. This means, finding a way to measure, prove and integrate the person’s real income and expenses. Another option could be asking lenders to integrate proxies for repayment capacity. For example, mobile money records could be matched with the borrower's geolocation to determine where and why they are being paid and predict how stable his or her income is. Coupled with strong data protection measures, this and similar metrics could help curb the high default rate and loan-stacking among digital borrowers in Kenya without causing financial exclusion.

Further, as suggested in a <a href="https://centerforfinancialinclusion.org/making-digital-credit-truly-responsible/" target="_blank" rel="noopener noreferrer">publication produced by CFI</a>, the Central Bank of Kenya should withdraw a digital lender’s license upon surpassing a threshold percentage of non-performing loans as a share of the total loan portfolio. This would force the lenders to improve their algorithms and creatively design more capacity-measuring metrics. And finally, the CBK should explicitly require licensed digital lenders to cross-check an individual’s loan performance with each other before issuing further credit to prevent loan-stacking. Although the Digital Lenders Association of Kenya (DLAK) advocates for digital lenders to be able to receive and submit borrower information to credit bureaus, this practice has not addressed loan stacking in the past and only served to blacklist millions of borrowers for small amounts. Therefore, digital lenders should be allowed to view and report outstanding loans, yet with very limited power to blacklist a late borrower.
<blockquote class="blockquote">The CBK should explicitly require licensed digital lenders to cross-check an individual’s loan performance with each other before issuing further credit to prevent loan-stacking.</blockquote>
Kenya cannot go back to the old bank-driven model which financially excluded most of the population, yet Kenyans cannot keep paying the price for the financial experimentation of tech startups. Digital credit should be responsible, affordable, and inclusive. With this new bill, the Central Bank of Kenya is getting closer to achieving it. However, the root cause of the problem is the evaluation algorithm and the business model that stems from it. Without addressing this, little will change for Kenyan borrowers.

Digital Lending in Kenya: Willingness vs. Capacity to Repay

When Goldman Sachs came under fire for offering women lower lines of credit than men on the now <a href="https://www.nytimes.com/2019/11/10/business/Apple-credit-card-investigation.html" target="_blank" rel="noopener noreferrer">infamous Apple Card</a>, they defended the product against charges of sexism by explaining that gender had not been an input to the credit scoring model. If an applicant’s gender was never fed into the algorithm, how did credit decisions end up discriminating against women who had similar credit histories as men?

Gender-blind credit scoring is an example of what many AI bias researchers call “fairness through unawareness,” an approach that intentionally excludes data on sensitive attributes like race, ethnicity, religion, or sex with the goal of reducing discriminatory outcomes. Behind this approach is the idea that eliminating markers of different group identities will prevent basing decisions on those characteristics. This has led to failures in the past, notably <a href="https://cpb-us-e1.wpmucdn.com/sites.suffolk.edu/dist/3/1172/files/2014/01/Rice-Swesnik_Lead.pdf" target="_blank" rel="noopener noreferrer">redlining</a>. Recent research has shown it can also make <a href="https://arxiv.org/abs/2009.06251" target="_blank" rel="noopener noreferrer">discrimination harder to detect and address</a> in the era of algorithmic decisions.
<blockquote class="pullquote--right">“Fairness through unawareness” intentionally excludes data on sensitive attributes like race, ethnicity, religion, or sex with the goal of reducing discriminatory outcomes.</blockquote>
As algorithms <a href="https://www.centerforfinancialinclusion.org/the-stories-algorithms-tell-bias-and-financial-inclusion-at-the-data-margins" target="_blank" rel="noopener noreferrer">transform the financial sector</a>, we need a better understanding of how to mitigate against discriminatory outcomes to ensure that those left out of the digital ecosystem today do not face further financial exclusion. Otherwise, we may not know how many Apple Card equivalents exist in inclusive finance.

This post explores the challenges and tradeoffs of using sensitive attribute data for detecting discrimination in the context of evolving data protection norms and regulations.
<h2>The Perils of Fairness through Unawareness</h2>
The idea may seem counterintuitive: if you remove gender from the data inputs of a credit model, how does the algorithm consistently assign a lower limit to women? The reason is that credit scoring algorithms are crunching larger amounts and new sources of data, which means that the model can inadvertently use proxies for sensitive attribute data in decisions, even if that was not the intention of the model developers. For instance, the <a href="https://journalofbigdata.springeropen.com/articles/10.1186/s40537-019-0180-9" target="_blank" rel="noopener noreferrer">type of phone</a> someone uses or the <a href="https://dl.acm.org/doi/10.1145/2721896.2721908" target="_blank" rel="noopener noreferrer">apps a person installs on their phone</a> can act as proxies for gender, age, or other characteristics.
<blockquote class="pullquote--left">Models can inadvertently use proxies for sensitive attribute data in decisions, even if that was not the intention.</blockquote>
A very simple method for testing algorithmic bias within a credit scoring model requires only a few variables like the actual default rate, the actual score of the individual when approved, and the variable you want to test with your algorithm, such as gender or race.
<h2>Data Tensions: Privacy vs. Anti-Discrimination</h2>
So, why isn’t this done? The answer lies in data limitations, sometimes based on very legitimate concerns about histories of injustice. It illustrates one of the challenges of detecting and monitoring for bias in algorithmic systems without infringing on privacy rights.

Many countries around the world prohibit collecting sensitive data or basing credit decisions on protected categories either under their banking regulations or under new data privacy legislation. In India, lenders are prohibited from discriminating on the <a href="https://www.rbi.org.in/Scripts/PublicationReportDetails.aspx?ID=764" target="_blank" rel="noopener noreferrer">grounds of sex, caste, or religion</a>. In the U.S., the Equal Credit Opportunity Act (ECOA) made it illegal for credit providers to discriminate based on sex, race, national origin, and other protected characteristics in response to widespread denials of credit to women and people of color. ECOA prohibits lenders from collecting this data in the first place, even for the purpose of detecting discrimination, because of fears it will be used improperly to further perpetuate bias in consumer lending. Research in the US shows that discriminatory trends have been <a href="https://arxiv.org/pdf/1912.06171.pdf" target="_blank" rel="noopener noreferrer">hard to detect in consumer lending</a> and that a lack of sensitive attribute data has also created enforcement challenges.

While not all lending regulations prohibit the collection of sensitive data, data privacy legislation introduces new limits on this data. The General Data Protection Regulation (GDPR), which has become the model for legislation in many developing markets, bars certain sensitive data from being collected or processed -- including data on race and ethnic origin, political opinions, philosophical beliefs, sexual orientation, and biometrics -- without individual consent.
<blockquote class="pullquote--right">Concerns about safely and properly using sensitive attribute data are valid given histories of discrimination and surveillance of minority groups in both wealthy and developing countries.</blockquote>
On top of GDPR, the European Commission has recently put forward <a href="https://digital-strategy.ec.europa.eu/en/library/proposal-regulation-laying-down-harmonised-rules-artificial-intelligence" target="_blank" rel="noopener noreferrer">draft legislation to regulate artificial intelligence</a>, with specific provisions for AI systems classified as high risk, which includes credit scoring. The draft legislation suggests that AI providers should process “special categories of personal data” to monitor and correct for bias in the public interest, but does not clarify collection practices or consent processes for such data. Concerns about safely and properly using sensitive attribute data are valid given histories of discrimination and surveillance of minority groups in both wealthy and developing countries. Changes in regulatory frameworks to allow for use of this data to mitigate against further discrimination require public trust bolstered by technical solutions that protect against misuse. Depending on a country’s history or in places experiencing active or simmering conflict, technical solutions may not be enough. For example, in a country like Rwanda where <a href="https://www.nytimes.com/2004/04/09/world/a-decade-after-massacres-rwanda-outlaws-ethnicity.html" target="_blank" rel="noopener noreferrer">ethnic distinctions are banned</a>, the tensions around sensitive identities make it difficult to fathom collecting and using such information, especially by the government.
<h2>Data Ecosystems in Emerging Markets: It’s Complicated</h2>
This tension between privacy and anti-discrimination may be exacerbated in developing markets by the <a href="https://centerforfinancialinclusion.org/rethinking-financial-inclusion-at-the-data-margins/" target="_blank" rel="noopener noreferrer">limited digital footprints</a> of customers at the data margins, as well as paltry knowledge on how data trails and protected groups intersect.

Sex-disaggregated data is still <a href="https://data2x.org/wp-content/uploads/2020/03/MappingGenderDataGaps_Economic.pdf" target="_blank" rel="noopener noreferrer">largely unavailable</a> in the financial sector and for many economic indicators in developing markets. Beyond availability, there is less understanding of how women’s lives and the barriers that shape their economic behaviors are reflected in data used for credit decisions. The reality is that gender, ethnic, or religious identities interact with other indicators (like income, asset accumulation, or technology use) and will impact individuals’ data trails in ways that may misrepresent individuals’ potential and result in disparate outcomes (i.e. access to a financial product). Fairness through unawareness approaches are blind to this intersectionality.
<h2>Where Do We Go From Here?</h2>
As an industry, we can do more to understand the data inputs for digital financial services and how socioeconomic realities present in these datasets, especially for traditionally excluded groups. There will still be tradeoffs between privacy and anti-discrimination, but managing those tradeoffs requires a conversation between providers, regulators, donors, and the public to identify what is possible and explore the right balance.

These conversations can start with the many open questions that exist from a technical perspective. For instance, can <a href="https://ieeexplore.ieee.org/document/8970908" target="_blank" rel="noopener noreferrer">third parties</a> play a role in securely storing sensitive data, or are there options to use <a href="http://proceedings.mlr.press/v80/kilbertus18a/kilbertus18a.pdf" target="_blank" rel="noopener noreferrer">encryption</a> or <a href="https://spectrum.ieee.org/artificial-intelligence/machine-learning/engineering-bias-out-of-ai" target="_blank" rel="noopener noreferrer">synthetic data</a> techniques? Can regulators develop tools that allow them to estimate the demographic characteristics of a dataset, like the <a href="https://www.consumerfinance.gov/data-research/research-reports/using-publicly-available-information-to-proxy-for-unidentified-race-and-ethnicity/" target="_blank" rel="noopener noreferrer">Consumer Financial Protection Bureau has done</a> in the U.S.? Can investors encourage investees to design monitoring and evaluation indicators to help answer questions about how their algorithms perform on inclusion?

While the road ahead may be complex, starting these conversations and bringing the public into them, especially those whose lives will be affected by algorithmic decisions, will be crucial for building trust. We need to engage with emerging research on possible solutions to see what might be appropriate in developing markets. At the very least, financial inclusion stakeholders should know enough to respond to any gender-blind or fairness through unawareness approaches in digital finance with healthy skepticism.
<blockquote>Starting these conversations and bringing the public into them, especially those whose lives will be affected by algorithmic decisions, will be crucial for building trust.</blockquote>