the ethics of algorithms - coding values into systems
Can a computer algorithm ever be truly neutral, or do we always program our own moral biases into our code?
The BIG Question
Can a computer algorithm ever be truly neutral, or do we always program our own moral biases into our code?Required Pre-Learning
Watch Craig 'n' Dave SLR19 Part 4 video (Moral and cultural values in software).
Ensure familiarity with creating and accessing keys in Python dictionaries.
Ensure familiarity with basic Python functions and conditional
ifelifelseDeclarative Outcomes
Understand that software and its algorithms inherently embed the moral and cultural values of their creators.
Explain the concepts of Utilitarianism, QALY (Quality-Adjusted Life Year), and Deontology within the context of automated decision-making.
Recognise how the scale of software deployment amplifies minor algorithmic biases into systemic societal impacts.
Procedural Outcomes
Trace and execute a baseline Python triage algorithm to determine its outputs (Use).
Adjust mathematical weighting variables to reflect a specific utilitarian ethical framework (Modify).
Synthesise new algorithmic rules using selection statements to enforce a deontological ethical framework (Create).
The Moral Algorithm
When we write code, it is easy to assume that because computers process binary logic, our programs are totally objective and neutral. But this is a dangerous myth. Every time a software engineer writes an algorithm that sorts, ranks, or decides something, they are making a moral choice.
Imagine you are programming an automated triage system for a hospital that is completely out of beds. A new patient arrives, and your algorithm has to generate a "Priority Score" between 0 and 100. Who gets the bed? The 18-year-old student, or the 75-year-old retired nurse? If you write code that adds +10 points to the student's score because they are younger, you have just programmed a specific moral value into your system.
When tackling these dilemmas, computer scientists and ethicists often look at three distinct philosophical frameworks to guide how an algorithm should behave:
1
Utilitarianism
This is the idea that the "right" choice is the one that results in the greatest good for the greatest number of people. A utilitarian algorithm doesn't care about individual fairness; it cares about the total overall outcome. In a hospital, a utilitarian algorithm would focus purely on saving the highest total number of lives, even if it means leaving the most severely injured patients untreated because they take up too many resources.
2
QALY (Quality-Adjusted Life Year)
This is a specific, real-world metric used by health services to measure the outcomes of treatments. It doesn't just measure if someone survives, but the quality and length of their survival. 1 QALY equals one year of life in perfect health. If an algorithm uses a QALY framework, it mathematically deprioritises elderly people or those with chronic illnesses, because saving a 20-year-old yields more "Quality-Adjusted Life Years" than saving an 80-year-old.
3
Deontology (Duty-based Ethics)
Deontology is the opposite of Utilitarianism. It argues that there are strict moral rules and duties you must follow, regardless of the consequences. A deontological algorithm operates on absolute rules. For example, a deontological rule might be: "Always treat patients in the exact order they arrived (first come, first served)" or "Always treat children first." It does not calculate the "greatest good"; it simply follows its programmed moral duty.
Task The UMC Triage Algorithm Task
0
Get Organised!
Open your favourite Python IDE
Create a new script called
triage-ethics.pyCreate a blank word processed document in order to save your work.
Add a suitable title and introduction.
Your teacher will set you an assignment through which you will submit your attempts at the three graded challenges.
1
Level 1: Mild
Copy the following base code into your IDE. This algorithm calculates a patient's priority score. Run the code and observe the output.
def calculate_priority(patient):
# Base score starts at 0
score = 0
# 1. Severity of illness (Scale 1 - 10)
# The higher the severity, the higher the priority
score = score + (patient["severity"] * 5)
# 2. Projected recovery chance (Scale 0.0 - 1.0)
# Higher chance of recovery adds a flat 20 points
if patient["recovery_chance"] > 0.5:
score = score + 20
# 3. Penalise for lifestyle risk (Scale 1 - 5, e.g., smoking)
score = score - (patient["lifestyle_risk"] * 2)
# Ensure score stays between 0 and 100
if score > 100:
score = 100
elif score < 0:
score = 0
return score
# Test Patients
patient_a = {"name": "Alex", "age": 22, "severity": 8, "recovery_chance": 0.8, "lifestyle_risk": 4, "is_child": False}
patient_b = {"name": "Sam", "age": 78, "severity": 9, "recovery_chance": 0.3, "lifestyle_risk": 1, "is_child": False}
print(patient_a["name"], "Priority Score:", calculate_priority(patient_a))
print(patient_b["name"], "Priority Score:", calculate_priority(patient_b))
Task: Look at the scores generated. Write a comment explaining which patient gets the bed, and why the algorithm favoured them based on the math.
2
Level 2: Spicy
The hospital board has decided that the algorithm must strictly follow a QALY (Quality-Adjusted Life Year) framework. They want to prioritise younger patients who will live longer, healthier lives if saved.
Task: Modify the mathematical weightings in the calculate_priority function.
Add a rule that subtracts 15 points from the score if the patient's "age" is over 65.
Add a rule that adds 15 points if the patient's "age" is under 30.
Run the code again. How have the scores changed? Add a comment on the answer document explaining whether this feels "fair" to you.
3
Level 3: Hot
The algorithm has been heavily criticised by the public for being too utilitarian and heartless. An independent ethics board demands you program a Deontological rule (an absolute moral duty) into the system.
Task: Create two new rules using if statements at the very end of your function (just before returning the score):
The Duty of Care Rule: If a patient's "is_child" value is True, their score must automatically be set to 100, overriding all other calculations.
The First Responder Rule: Add a brand new key to the dictionaries called "is_healthcare_worker" (set to True or False). If True, add a flat 25 points to their score as a moral reward for their service.
Create a third test patient (a 10-year-old child) to test that your Deontological duty rule works perfectly.
Copy and paste your completed code into your answer document.
Summary: You have just witnessed how math is not strictly neutral. By deciding which variables to include, which to ignore, and how heavily to weight them, you baked your own moral philosophy directly into the application. When these algorithms are deployed at a national scale, a simple +15 points for age can result in systemic discrimination against thousands of people.
Homework Task: Write a 200-word reflection addressing the "Big Question" from the start of the lesson: Can a computer algorithm ever be truly neutral? In your answer, clearly explain the difference between a Utilitarian algorithm and a Deontological algorithm, using your Python code as evidence.
Last modified: September 14th, 2026
