Series Three, Episode Seven: Diversity in AI and the challenges of removing bias

Julia: Hello, my name is Julia Streets and welcome to DiverCity Podcast, talking about diversity and inclusion in financial services. On the podcast we seek to shine a light on positive progress, call out areas requiring further focus, and offer lots of ideas to help drive change. Today I’m joined by Tracey Groves and Sylvia Carrasco. Tracey Groves is the founder and director of her advisory business Intelligent Ethics, passionate about helping boards and leaders think differently about responsible leadership, diversity, inclusive culture and ethical behaviours as drivers of a high performing business. She’s a leading voice in the analysis and the impact of Artificial Intelligence, or AI, on business ethics, and works with organisations to help them better understand and prepare for the consequences and impact of AI on leadership and accountability, decision making, integrity and judgement. Tracey, welcome to the show.

Tracey: Hello.

Julia: Sylvia Carrasco heads the board of directors of the FinTech company, Goldex, focused on reinventing how gold is traded. She has enjoyed an 18 year career in investment banking, managed teams, negotiated global commercial deals, and has been responsible for delivering distribution strategies at Credit Suisse and Mann Financial. In 2008, she successfully founded the first FCA regulated firm to provide execution trading services to global institutions. Sylvia, welcome to the show.

Sylvia: Hello.

Julia: So at the beginning of the show, we invite each of our guests to take a moment to talk about the things that they’re most focused on. Tracey, what are you working on at the moment?

Tracey: The key things that I’m focusing on are two pronged, in effect. One is around the role of leadership in taking organisations to a place where they’ve never been before. What I mean by that is being able to exceed expectations in terms of performance, but by doing that, using integrity and inclusion as core drivers of that performance. The second prong of what I’m absolutely passionate about is thinking about how technology and emerging intelligence systems are going to enrich all of our lives for the good. And this comes with an ‘only if’ we are able to hardwire ethical principles and integrity at their very core. So my work is about combining those two elements, and thinking about what can I do to help shape influence, design, and monitor how leaders and organisations are going to open up possibilities and avenues that we’ve never seen in terms of human performance as well as organisational performance.

Julia: Fabulous, and there’s a lot in there that we will certainly unpick as we go through the show. Thank you very much indeed. Sylvia, let me turn to you. What are you focused on at the moment?

Sylvia: Thank you very much for having me here today. Basically, I dedicate my time to two things mainly, professionally speaking. The first one is obviously running Goldex, which is the first ethical gold provider that has ever been launched in the world so that has taken a lot of time and a lot of dedication, because we had to change the business model from scratch. The second thing that I’m very passionate about is to help women set up their own businesses. I’m a mentor at the London Business School. I mentor some very intelligent ladies who needs some help to keep moving forward in their ideas for setting up their businesses. I do quite a lot of promotions, for example, with our UK government, with the Department of International Trade, that are now being very vocal about helping female founders in the FinTech space. So that’s pretty much how I’m dividing my professional life. One of them, you can call it more philanthropic attitude, and the other one is obviously to run the business successfully.

Julia: Clearly, there are some very interesting parallels. The word “ethics” comes up quite a lot in what you were both talking about. Tracey, you advise organisations at the very, very highest level about how to drive performance, and the contribution that diversity and inclusion can make. To what degree do you see that, that awareness has shifted over time?

Tracey: It’s a great question, Julia and I would say awareness has shifted. However, it’s the use of the word awareness that really, what I would say ignites my passion for wanting to make a difference, because awareness is not enough. So there’s a huge amount of talking and I do not doubt any of the sincerity and the intent of initiatives and policies around returners to work schemes, around sponsorships, around policies such as shared parental leave. I’m seeing things happening now such as recruitment strategies with no male short lists for candidates and even no male panels on conferences and debates taking place. Big tick from me, brilliant.

However, the challenge comes in how do we then convert that into action? And although that is taking place in terms of these initiatives being written down, is there a belief, an inherent sense of “We’re doing this because not only is it the right thing to do, but also it drives better outcomes from a business perspective”. That’s the bit I feel that yes, the awareness has gone up, but there still needs to be a greater level of integrity in carrying out and executing those initiatives and policies.

Julia: Where do the initiatives lie? Is it with heads of D&I? Is it with the businesses themselves, where do you think the accelerating change will come from?

Tracey: It has to come from the very top, absolutely. There needs to be support, there needs to be endorsement and there needs to be a belief at the top that this is the right thing to do. However, I genuinely believe it’s down to every single one of us, at whatever level of the organisation, to hold each other to account as and when these initiatives are rolled out, and necessarily not executed in the way that they should be. So if we are seeing inappropriate behaviour, if we are seeing behaviour which is exclusive and is discrimination in terms of the way it’s not including every single one of us, then what are we doing to raise our concerns? And in an area whereby I feel psychologically safe to be able to escalate issues and make those concerns heard. So it’s not about, it’s all down to the leadership and it’s their fault, it’s actually the leadership is defining what the standards of behaviour are, it’s down to every single one of us to be able to live up to that and then to demonstrate at all levels what it means to do that right thing.

Julia: And that takes a degree of courage, actually, throughout an organisation, because what we talk about a lot on the podcast is that there are certain entrenched behaviours that have always been there over time, so middle management behaviours. But the millennials for example, will not hesitate to call out bad behaviour, because for them it’s just part of the way that they work because they work at a faster pace of feedback loops, if you like. Are you seeing that there is a wave of change, or are there still sticking points about people going, “Yes, we understand the intention.” And how can we help people hold businesses to account?

Tracey: I think the “Me Too” movement has genuinely opened doors and given people more permission to be able to speak up now. So you talk about courage and bravery to voice these concerns. I think now women are feeling empowered and feeling more comfortable. It doesn’t necessarily alleviate the fear, but I think the solidarity and collaboration that comes from being part of a collective that is outlining exactly what is appropriate, and what is inappropriate, is making a big difference. I’ll give you a real life example of a client that I’m working with, whereby they were using all levels of management to be able to deliver messages around inappropriate behaviour, and they were using this at levels of management they’d never done before, normally it just used to come from the top and then that would be cascaded down via using loads of communication channels. Now they said, “No, actually this has got to come personally from middle and junior managers.”

As a consequence of doing that, the number of whistleblowing calls and allegations of inappropriate behaviour went up significantly. As a result, they did some investigation to understand, well that could be a good thing, it’s not necessarily a bad thing. But why? What’s driving this increased confidence and ability to make those concerns heard? What they found as a result of their investigations was actually the number of individuals who are looking at their manager and saying, “How dare you? How dare you sit there and tell me about what this is in terms of our corporate values, or what we stand for or our purpose, when I know on a day to day basis you do not enact those in terms of your behaviours, in terms of your decision making.” So as a consequence of that, it actually inspired people to be able to say, “Actually this is not the top leadership that we need to be held to account here. It’s everyone else. That includes people in my team, in my office, in my local geographical function area.”

So I genuinely think that it’s something that we need to hold to collective accountability. But also I think to ensure that we reward those individuals who take that step forward because we can all benefit from it.

Julia: We see so many organisations coming out with those classic three to five corporate statements that they’ll put all over coffee rooms and that sort of thing, with integrity and ethical behaviour. It strikes me, that’s a real example of how, as a culture of an organisation, all the employees get behind the principle of integrity and ethical behaviour as well.

Tracey: Yes, Julia, and I’m seeing actually companies now, and I’m talking about probably there’s more established and mature organisations, who are moving away from those more bland statements of ‘we operate with honesty’, ‘we are transparent and brave in our decision making’ and actually becoming much more personal in the motive. So I’m seeing things such as ‘we’re in it together’, ‘here for tomorrow, not just today’. So these statements are actually a lot more now about how can we collectively come together to create a more sustainable environment where we all feel that we’re meant having a role and the contribution to play? As opposed to words which we potentially can find hard to relate to.

Julia: What’s really interesting there, you talk about organisations of a certain size, so I’m imagining in my mind’s eye, thousands of employees around the world in multiple locations, et cetera. The other dynamic that I’m really fascinated by in the world of FinTech is of course smaller organisations at the earlier stage of that journey of change. Sylvia, when I think about the world of gold, that is probably the one of the most traditional asset classes in many ways. As a FinTech founder, and also a champion and a mentor of women entrepreneurs, how do you think about incorporating diversity and inclusion into your commercial strategy?

Sylvia: The foundations of Goldex as a Company are based on both diversity and inclusion. Starting with inclusion, is very very simple. Basically the whole journey of Goldex started because I personally started trading gold and, I started seeing things that I didn’t like. Because gold is not a regulated market, there is no FCA policing the behaviour of the content providers. When there is no police, unfortunately bad things happen. It was very clear for me at the very beginning coming from such a regulated markets like equities where everything is basically structured in a way to always protect clients, in gold, there was no such a protection for the customers. So customers who purchase gold, generally speaking, they get market abuse from the providers, the providers are always the ones who purchase the gold, and then they sell it to their customers. That creates a conflict of interest because the provider is mixing their own flow with that of their customers. So you can call it that they have insider dealing information, and that’s when the customer, being totally unaware, is actually abused on prices.

Now I thought that there was a perfect opportunity to change that. That’s why the way the whole business was created was to say, okay, what about if we become the Airbnb’s of gold, or the Ubers of gold, where ultimately Uber doesn’t own any taxis. Airbnb does not own any flats. What they do is they have set up a marketplace, a technology system, to allow customers to get in contact and get a taxi or get a flat. So that is the ethos of Goldex. The whole ethos was, I want to make sure that we do not abuse our clients. We’re going to make sure that we will never own one gram of gold, and that all the others will be given to a peer-to-peer market, where the price will be dictated by people like me or you or Tracey. So the entire foundation of the business is about being ethical. Being an ethical provider and just offering a very good trading experience and trading systems for our clients.

Julia: And presumably the technology comes into play quite heavily with that in terms of setting up, as you say, the marketplace, a platform, essentially. When you set up the organisation, thinking about the talent that you need to have around you to make that successful, they’re not necessarily the institutional gold traders personality types. They’re actually the younger data scientists, et cetera. I don’t want to make an assumption they’re younger, necessarily, because we’re finding that there’s retraining at every level of career journeys. Can you shine some light on how do you find that talent?

Sylvia: I think good talent is very difficult to find. However, we are in London and London is probably the best place at the moment to recruit FinTech specialists. Interesting, I think it was last week somebody said to me, and I cannot remember who said this to me, but somebody who was apparently, if I remember correctly, adequate to say this, said to me, “I know that female founders have a lot of problem recruiting people.” And I said, “Really?” And she said, “Yes they do, because apparently there are still men who are not happy to work for females.” And I was quite surprised and I said, “Look, personally, I’ve never had that issue.”

I have asked probably about a dozen female founders in FinTech companies about whether they have had issues recruiting employees. Interestingly enough, they all said no. Well, they said we do have issues like everybody else, but they didn’t say that they were having problems. So I don’t know if it’s a myth or not. Personally, I think that hiring talent is difficult. However, the way I look at hiring is very, very simplistic, and I think it is normal to do, despite the fact that I know that is not what everybody does. When we look to fill a position, we interview people. If it’s a woman, if it’s a gay man, if it’s a black man or a black lady or a Chinese woman, we don’t care. All we care about is how they fit with the values of the company, which is the first and most important thing. Then their professional skills to fill that vacancy. If we think that they are the best for the job, we will offer them the job.

I think the main problem comes when in certain organisations, they already have an idea in their head about the type of individual they want to hire, and that I think is really wrong, because that’s discrimination already. Then, what is even worse, is that if a company finds, let’s say a woman who could be an amazing developer for example, and then despite the fact that she’s the most adequate to fill that job, they don’t give it to her. That is just because she’s a woman. That is where I think discrimination really kicks in. In our heads, we are a small organisation, however we have Spanish, English, French, Argentinians, Indians – we are all sorts of different cultural mixes, which I think is very good for the organisation because it creates different perspectives from different communities.

Julia: As you look beyond your business into relationships with suppliers and strategic partners, do you look to them to demonstrate to you that they have similar attitudes towards skills, talent, diversity? How do you engage with them? And are people such as VCs looking to you and going, well, we want to understand your diversity mix as well?

Sylvia: Well, to be honest with you, I don’t have an agenda to analyse my provider’s diversity and inclusion policies, which maybe I should change. I’m not saying that I’m not doing it wrong, but I definitely don’t do this. I choose providers because I think they are the best at what they do or they best fill the needs that we have. But all I can say, however, regarding this, is that again we don’t discriminate based on the fact of where they are based. We have providers from all over the world. We have providers from India, from Spain, from Italy, from the US, from the UK of course. Again, it’s this cultural mix that we find interesting to work with.

In regards to the investors, I must admit that only one investor of our roll of investors have ever asked me about our policies. It was an interesting talk. He said, “Look, I was very interested about your company. I was very interested about investing. However, what really made me invest is that at the very end of your presentation, you talk about social good. And that social good definitely was the last bit for me to invest in your company.” Unfortunately not many people ask for it, but it’s something that we definitely, we promote the social good.

Julia: And it’s interesting when it comes back to this whole point about ethics and social good. We’re more and more hearing that investors, the VC community, are paying more attention, but there’s certainly a lot more attention to be paid to it. Some firms are using it as a competitive edge, which I think is a fascinating dynamic when there’s clearly an appetite for change, but some people are making … And this is the whole point is about the podcast, to talk about the commercial benefits. I touched on it around the type of data scientists and technologists that we need in business today, and I’m really fascinated about the impact of Artificial Intelligence. I know that there are a lot of concerns, and there’s more and more coverage and discussion about some of those inherent biases that might exist in AI. Tracey, I know this is something you think about a lot. Does that concern you? Are people worried about it? Should people be more worried about it?

Tracey: I’m a genuinely optimistic person so I don’t want to sound worried, but I am, which is, we talk about values being hardwired into intelligent systems and Artificial Intelligence as these new emerging technologies arrive. But we can’t actually apply these values on a human level, never mind within systems. So for me, the opportunity for us to think long and hard about these aren’t AI’s or intelligent systems that are biased, it’s us, and the machines are learning it from us. I think the sooner we start to get our heads around effectively that it’s not about the ethical system itself being something that we can programme, it’s more about what are we doing and holding ourselves to account in terms of how we design, develop, implement and then most importantly measure, and maybe this is the auditor coming out to me from my previous profession, measure and assess that the outcomes from the AI is in line with its intention and its purpose.

That for me is the worrying bit, because there are so many opportunities for AI to bring about a richness in our lives that we’re not yet even aware of. We’re already seeing it on a day-to-day level, whether or not we’re conscious of it. But if we’re not careful, it could actually abuse us and rob us off power, as opposed to give us more power in our ability to not only enrich us from a social and economic perspective, but also from an ecological perspective as well.

Julia: For large firms that are trying to go through that transition and that transformation, the digital journey, if you like, are they paying attention to some of those inherent biases? Because I wonder from an early stage business, you were saying, Sylvia, that you set up the organisation based upon fundamental principles, avoiding the conflicts of interest, understanding that market abuse takes place, so it needs to be a very clean model. It’s an open conversation for both of you; as new technologies come through, how do you ensure that some of those biases are removed over time? Is that something you think about as well, Sylvia?

Sylvia: Yes we do. Again like Tracey, I’m not going to sound negative, but, if you think about it, they are not, and I’m referring to women here, there are not many women who take investment decisions at home. So it’s usually the man who will say, “Darling, I’m buying some bonds.” She’s lucky if she’s even told. But generally speaking, men are the ones driving the investment decisions and what to do with the cash that they have. I still don’t understand why they don’t take more control of this. It’s a little bit like the division – “you do the repairing of the DIY at home and I do the dishes”. So I think it’s important that women start taking more control in certain aspects such as the investment decisions.

From our perspective, one thing that we’ve done from the very beginning is for example, to include women in our marketing campaigns. We actually have a few posts about diversification and we played with the word diversification, not just as diversification in investments but diversification for female. And there was a picture of a woman actually trading or using that on an iPad. So of course our target as a company, we need to focus on the characteristic investor who is our target audience. But, nevertheless, we are trying everything we can to also include women into taking control of investment decisions.

Tracey: I’m definitely seeing a lot more activity now, to get more women involved in the technology and engineering subjects, whether that’s from university onwards, but also in internships and university sponsorships as well with big corporates. But I think it’s starts earlier than that. I think we need to start educating our children, and I think it now needs to be part of our national curriculum that we are highlighting the ethical principles upon which AI and technology needs to be based, because the children of today are growing up with it. They don’t know how not to trust a machine. For them, that intelligence system will be the one that they will trust because they don’t know any other difference.

So their ability to ask questions, to be curious, to apply judgement, is going to be significantly curtailed unless they are brought up to understand and have that confidence to have a line of inquiry which will mean, was that meant to be what it was set out to be? Has it given us the right outcome? Is it in line with what it stands for? And if not, what are the consequences of that? Who’s going to be held to account if that decision was the wrong one? That for me is the concern around diversity in terms of the systems.

I think what’s hopeful is that the government is doing a huge amount to raise the awareness of this, and I know the new Centre for Data Ethics and Innovation, which is being set up by the UK government, is going to be a significant role player in acting as a catalyst and setting standards for what good looks like, because there is no regulation. There is an uneven playing field, shall we say, at the very least when it comes to ethics and the way that these machines are being developed, but it’s not too late if we just take action now. It will be too late if we take action tomorrow.

Julia: Great, and I think that’s a perfect moment to take a pause there and turn to Robert and Cynthia who have been doing some research to support today’s discussion.

Robert: We are pretty familiar with the gender pay gap in financial services, but what about the less well known gender investment gap? Research carried out in the US by Boston Consulting Group found that when pitching for early stage capital, female business owners secured significantly less money than male business owners. In fact, on average, the disparity was more than a million dollars.

Cynthia: However, the same study showed that businesses founded by women ultimately delivered higher revenue – more than twice as much per dollar invested than those founded by men, making women owned companies better investments for financial backers.

Robert: Artificial Intelligence, or AI, is currently being described as the new diversity. Reshaping business with Artificial Intelligence, the 2017 MIT Sloan Management Review shows that 85% of executive surveyed believed that AI will give their companies a competitive advantage. 60% said that an AI strategy is urgent for their organisations. Executives will need to take the time and energy to research how best to forge partnerships between humans and AI.

Julia: Thanks Cynthia and Robert, and links to the research can be found on our website www.divercitypodcast.com, where you can find it on our episodes and sign up for early notifications for future recordings. Please do follow us on Twitter @divercitypod, and DiverCity Podcast is available on BrightTalk and all good podcast channels and we’d love a rating, it all helps to promote the show.

So I think there’s some interesting questions, Tracey, as I come back on this thought process around AI. I guess there’s the central question about who do you trust?

Tracey: Yes. And in order to be trusted, I need to be trustworthy, and the same applies both to leadership and it applies to AI and intelligence systems as well. And that requires honesty, it requires competence, but it also requires reliability. And that’s my wish, that we create an ecosystem, whereby both machines and humans come together and collaborate in order to achieve the best outcome.

Julia: This really matters because when we think about, and it came up in conversation earlier, about reaching out to engaged investors, essentially, and reaching out to those who aren’t already engaged as investors. Sylvia, I know this is something that you think a lot about.

Sylvia: Yes. It was very important for us to make sure that everybody could be a client of Goldex, that everybody should be allowed to use gold as a protection of their wealth. For that reason, we decided not to put a minimum investment to our clients so that they can start investing from £10. This way, we make sure that people’s conception that you have to be a millionaire to buy gold is something that we want to make sure that disappears from people’s heads. Our service is basically to be open to all and not just the very selected few.

Julia: Fabulous. Well, I think it has been a fascinating discussion around this central question around ethics, integrity, and at the very heart of business models to ensure that it’s available for all, thinking about the impact of AI in technology and what that means today and also tomorrow. Sylvia and Tracey, thank you so much for joining us.

Sylvia: Thank you very much.

Tracey: Thank you.

Kieron: This episode of DiverCity Podcast was produced by me, Kieron Yates, on behalf of Julia Streets productions. Thanks to Cynthia Akinsanya and Robert Pinto-Fernandes for their insights. You can find out more about guests on this week’s show on our website divercitypodcast.com. Whilst you are there, you can also sign up to our newsletter for all our latest updates. To be sure of catching all our future podcasts, subscribe to our feed in iTunes or your favourite podcast app. And if you’ve enjoyed this episode of DiverCity Podcast, remember to give us a rating or a review, it’s all helps promote the show to a wider audience. Finally, our twitter handle is @divercitypod. Thanks for listening.