Open Source to Empire: Can AI ever be an ethical actor?

“‘AI’ is used to describe a range of technologies. We should be worried about it…When we look at algorithms - they can influence and be persuasive.” - Joe Parker

Ever since ChatGPT launched in November 2022, AI has become the latest technological buzzword that everyone, from major corporations to the lowliest intern, suddenly claims to be leveraging. A few years ago, it was blockchain. Before that, machine learning. Before that, “big data”. Corporate jargon moves in trends much like dating apps do: everyone scrambles to look as though they arrived early to the party, had been looking forward to it, and actually knew about this party last year, but perhaps you weren’t invited?

But unlike most previous tech trends, artificial intelligence feels different. Not simply because of what it can do, but because of what it may eventually become. AI is no longer confined to recommending Netflix documentaries about cults you’ll never finish watching (and they must know that, right?). Increasingly, it is shaping our choices, influencing our behaviour, filtering information and silently mediating our relationships with the world around us.

Which raises an uncomfortable question: can a system optimised for engagement, persuasion and (profit) growth ever truly behave ethically? In this episode of The Great RomCon?, I sat down with student and AI ethics researcher, Joe Parker, to discuss the moral architecture emerging beneath modern AI systems - and what this means for politics, autonomy, dating and the future of human agency itself.

Joe belongs to the first generation to have grown up fully inside algorithmic culture. Unlike older generations who can vaguely remember boredom, analogue childhoods and speaking to acquaintances without them first stalking them online, Gen Z have lived alongside recommendation systems, social media feeds and digital persuasion technologies for much of their conscious lives. Which perhaps makes them uniquely qualified to recognise how strange all this has become.

“‘AI’ is used to describe a range of technologies,” Joe explains early on. Machine learning, neural networks, large language models, reinforcement learning: all grouped together under the increasingly mythical umbrella term of “AI”. But despite the marketing gloss now surrounding these systems, Joe believes people should be concerned. “We should be worried about it,” he tells me. “When we look at algorithms - they can influence and be persuasive.” The real issue with AI may not ultimately be whether it becomes conscious or sentient. It may be whether it subtly reshapes human behaviour at scale long before that point arrives.

Joe references concepts from behavioural economics: nudging, choice architecture and Karen Yung’s work on “hypernudging”: algorithmic systems capable of continuously adapting environments around users in order to influence behaviour. If traditional advertising was based on reason, logic and persuasion, modern algorithmic systems increasingly resemble behavioural management of potential customers. ‘Choice architecture’, the way in which different options are arranged, cannot be neutral - they must be arranged in a certain way - and whatever choices are made by the choice architect will influence behaviour, intentionally or not.

Every platform has incentives. Every recommendation system contains assumptions about what an “enjoyable experience” looks like. YouTube’s autoplay does not emerge naturally from the laws of physics. Dating app algorithms are not passive observers of human desire. Platforms shape behaviour because shaping behaviour is the business model.

Joe’s concern centres not on the dramatic, cinematic fears around AI: killer robots, rogue superintelligence, Terminator skulls rolling across apocalyptic landscapes. But on something subtler: the gradual erosion of civil autonomy. “The media focuses on the worst cases,” he says, “rather than the power being amassed by new tech platforms and less concern about how individual preferences are being changed.” This echoes the work of Shoshana Zuboff, whose concept of “surveillance capitalism” argues that modern platforms increasingly seek not just to predict human behaviour, but to modify it. We are not merely using algorithms; algorithms are shaping us in return.

And nowhere feels more revealing of this than online dating. Joe notes that dating apps increasingly mirror wider social media culture. “There is a lot of continuity with how people present on social media,” he says. “People have an idea of this idealised person they are looking for.” But do people actually know what they want? Dating apps present attraction as though it were a rational sorting exercise. 6’5’’, works in finance, blue eyes etc. Compatibility reduced to searchable and filterable attributes. We discussed height filtering on apps like Hinge -  arguably one of the few socially acceptable forms of open discrimination remaining in mainstream culture. Dating is meant to be discriminating, the search for love has always favoured being choosy, but many people rebuff the idea that people should be rejected for something that they were born with or cannot change, such as height or age. Two metrics that are less easy to discern in person, but easy to filter out online.

And yet despite all this apparent optimisation and personalisation, many young people seem increasingly disillusioned by app dating itself. Joe himself found the experience bleakly repetitive. “It just felt like a big waste of time to me,” he admits. “There’s an imperative to find out why you matched with someone - there’s something completely wrong about that.”

In real life, attraction often emerges gradually through context, chemistry, humour and revealed ambiguity. On apps, however, two strangers begin with the assumption of mutual interest before knowing anything meaningful about each other - ‘which thing about my sanitised profile did you like?’ The result can feel oddly performative: people attempting to reverse-engineer intimacy from a ‘you matched with someone’ notification.

Much of this, Joe suggests, intensified after Covid. “Covid invited a sense of isolation.” The pandemic accelerated trends already underway: remote working, digital dependency, mediated communication and algorithmic socialisation. Young people increasingly studied online, socialised online, dated online (if at all) and consumed media online, often simultaneously through one device. The consequences are still unfolding psychologically and socially, such as the rise in concern about NEETs (Not in Education, Employment, Training or Skills) in the UK.

One particularly troubling part of our discussion focused on how AI systems themselves are built. I referenced reporting from Karen Hao’s ‘Empire of AI’ (2025) book , which details the hidden human labour behind supposedly frictionless AI systems. Many large language models were trained using low-paid workers in developing countries tasked with reviewing disturbing and traumatic content in order to moderate datasets. If the AI was going to be given guardrails to not produce harmful content, then it needed reinforcement learning to know what that material looks. In other words, some of the world’s most advanced AI systems were partially built upon invisible human suffering.

Which raises the central moral tension surrounding AI development: do the ends justify the means? When posed with that classical moral dilemma, most people ask: what means, and what ends? If these technologies eventually help solve climate change, healthcare diagnostics or scientific research problems, does that morally offset the exploitative labour structures that helped create them? It is the kind of ethical question Silicon Valley tends to move quickly past in favour of product launches and t-shirt cannons.

Joe repeatedly returns to the importance of intention in ethical systems, when I press him on whether consumers value the freemium model. “Intention seems to matter for moral culpability,” he tells me. This matters because modern AI systems increasingly operate in morally ambiguous territory. Recommendation algorithms may not “intend” harm in the human sense, but they are still optimised around outcomes - engagement, retention, monetisation - that can produce harmful social effects. Filter bubbles, outrage amplification, compulsive usage patterns and emotional dependency often emerge not because platforms explicitly desire social breakdown, but because outrage and stimulation are extremely profitable. And perhaps that is what makes AI ethics so difficult. We are trying to apply human moral frameworks to systems that do not possess human understanding, empathy or accountability, but have been built with human material and human incentives of commercialisation.

We also discussed freedom itself (“Freedom isn’t free, it belongs to you and me…”), drawing on thinkers like Isaiah Berlin and contemporary debates around autonomy in gamified digital systems. How much free will do people actually possess inside environments engineered to shape behaviour? In this, we discussed the merits of whether Amazon’s efforts to gamify the workplace of warehouse staff to incentivise them to work harder, as captured in Dr Charlotte Unrah’s ‘Benevolent Algorithmic Managers’ 2024 paper from Southampton University. If platforms deliberately exploit cognitive biases, default settings and frictionless design to maximise engagement, are users meaningfully “choosing” their actions in the traditional sense? This is exemplified by the power of default settings, precisely because most people very rarely change them, even when if is relatively easy to do so. Users technically can opt out of algorithmic recommendation feeds in some cases, but platforms often create enough friction to discourage doing so. “There is an asymmetry of knowledge in that data transaction,” Joe says. “The transparency is not there.” That asymmetry increasingly defines modern digital life. Most people do not fully understand how the systems shaping their attention operate. They simply live inside them.

Towards the end of the conversation, we touched on the potential rise of AGI (artificial general intelligence) and whether machines may eventually surpass human cognitive capabilities entirely. Joe remains thoughtful rather than apocalyptic. But what interested me more was a simpler question lingering beneath everything we discussed: What kind of society are we building while chasing these technologies? Some of the biggest companies in the world are spending billions in the race to create AGI as quickly as possible.

But AI is not simply a technical revolution. It is a philosophical one, because it is happening to people, it will affect their lives - for good and bad. It forces us to confront uncomfortable questions about autonomy, persuasion, authenticity, labour, truth and even love itself. Can an algorithm ethically mediate relationships? Can systems optimised for profit ever genuinely prioritise human flourishing? Can technologies built around prediction avoid gradually reducing human complexity and quirks?

Talking to Joe left me feeling that the greatest danger of AI may not be machine consciousness at all. It may be the effect on passive human unconsciousness: the slow adaptation to systems that increasingly shape what we see, want, believe and desire. When considering what makes us real, sure to exist - “I think, therefore I am”, mused Descartes. We need to make sure, as AI will assist our work and personal lives more and more, that we continue to think critically for ourselves.