[Music]
welcome back everyone
um
here we have lily ryan who is going to
be
presenting on
an interesting segue down the
misguided roads of
how we're measuring
our lives with smart devices lilly is a
penetration tester a digitally digital
security consultant and public speaker
who serves on the board of digital
rights watch lily specializes in web
application security privacy education
and the history of technology related
issues no my lily
hey everyone thank you for coming along
to my talk today um
took a sip of water at just the wrong
time but anyway
um thank you very much for being here
before i wanted uh before i get into
this i wanted to say that i'm speaking
to you from the stolen lands of the
warandra people and i acknowledged that
sovereignty was never seeded
so
um
my name's lily ryan this is how you can
find me on twitter if you would like to
i spend most of my time breaking
computers and
helping other people not break computers
um and because i am or being a pen
tester and i get paid to be
professionally paranoid about things
that means i ask these kinds of
questions a lot
about what it means to be human how we
identify other humans authentication all
of those kinds of things from that kind
of security standpoint but because my
background is in history i spend a lot
of time asking the same questions from a
philosophical and human rights based
standpoint too so today i wanted to look
at the history of the systems that we
have tried to develop for identifying
each other whether or not they're any
good from a scientific point of view and
then to look at some thoughtful ways
that we can approach the topic of
defining humans when it comes to
building new apps and services i also
want to say at the outset that the topic
that we're getting into is heavy both
emotionally and in terms of content
i have uh not very long to cover about
200 solid years of history philosophy
and scientific development so i am going
to have to skip some things and simplify
others
an entire ocean of ink has been spilt on
this topic and this talk is only a drop
so if you want to explore the rest of
the ocean with me um please you know get
in touch i'd be very happy to chat and
with that out of the way let's get into
it
there's this really famous bit in sir
arthur conan doyle's a study in scarlet
where sherlock holmes meets dr watson
for the very first time he looks him up
and down and he says
you have been in afghanistan i perceive
and watson is really stunned that this
stranger could know this about him
without any advance warning
later on we find out that sherlock
holmes was able to deduce the fact that
dr watson had been in afghanistan by
observing the way he stands
his skin color and his clothing all in a
matter of seconds
this kind of moment has been used over
and over again to introduce detective
characters in films and books and
television shows in the years since then
and it's known as the sherlock scan it's
supposed to show it's supposed to show
just how smart and observant the
detective is how wise and knowledgeable
about the world and how transparent the
secrets of human nature are to such a
superpowered deduction machine
but if you've ever tried this out for
yourself
perhaps you've been trying to deduce
something about a stranger at a coffee
shop or a new acquaintance at a party
you'll realize pretty quickly that it's
also a complete load of junk
in the real world
things very rarely work this way
while sherlock holmes said that he knew
about watson's past partly because of
how he was standing a person might stand
in a particular way because they really
need to pee or because they went too
hard on leg day at the gym the day
before or because they're wearing
uncomfortable shoes or for an infinite
number of other reasons
the character of sherlock holmes grew
out of a very particular very victorian
english post-industrial revolution world
view where scientific development
scientific developments
seem to be coming in thick and fast and
new industries sprung up almost
overnight and the world changed rapidly
and those people who were in power who
were leading this new scientific charge
they were heady with success they began
to believe
that science really could tell us
everything that we needed to know about
the universe
that there was nothing that a measuring
tape and a calculating eye couldn't
deduce about the world and the people in
it and that it was only a matter of time
before we hit upon this one unifying
system that allowed us to definitively
categorize everything
sherlock holmes was their superhero the
ultimate rational being a person who had
it all figured out and who could read
the world around him and translate its
secrets back to the rest of us with
unnerving ease
the problem with this kind of thinking
was that human beings remained and
remain stubbornly difficult to
categorize in any meaningful way
although it hasn't stopped a lot of
people from trying
one of the main ways that we've tried to
get the measure of our man so to speak
is by using biometrics
when i say biometrics in this talk i'm
using the term to encompass a wide range
of different methods that have been
developed for measuring the human body
historically we've used a lot of words
for this kind of activity sometimes
interchangeably
anthropometry is another common word
that has sometimes had the same meaning
and sometimes been distinct
there are also words that have been used
to mean some of the things that we mean
by biometrics today like craniometry
which is a system for measuring the
shape of the human skull
there's also physiognomy which is
supposed to be the study of what you can
tell about the kind of person someone is
by looking at their physical features
and as a subset of this you also have
phrenology which has sometimes been used
interchangeably with craniometry and
sometimes not but has generally come to
mean what you can tell about the kind of
person someone is by measuring the shape
of their skull
a lot of these approaches have since
been rightly discarded as pseudoscience
um
it's not generous to lump them in with
other body measuring systems that we use
for real benefit in modern medicine
and in criminal science but my talk
today is all about the problems that we
have as we work out whether any
particular
system of measuring people is scientific
and ethical or not
the question of why we want to measure
these things at all is that we're a
species with large brains and way too
much time on our hands
and on a practical level the ability to
recognize another human being is a
safety measure you know is this person
that you're seeing in your living room
at night the person you share your house
with or is it someone who's broken in
we also have surveillance measures you
know is this person the same person that
you saw breaking into your house last
month or just someone who kind of looks
like that person
and the more that a group of people is
in a position of power over other groups
of people the more it is important to
them to know who someone is where
they're going or who else they might be
spending time with and for what purpose
this kind of knowledge
can
arguably be used for public good to keep
out intruders for example but
in the wrong hands and even in ignorant
hands it can be used as a form of social
control
this is why most authoritarian
governments end up heavily invested in
categorizing groups of humans and they
make extensive use of whatever biometric
tools and systems that they have
available
but also human beings
just like to know more about themselves
we just really like quizzes anything
that claims to offer personalized
insights into someone's life or
character is gonna make money and it
doesn't matter what kind of scientific
evidence it's based on
we have been trying to get this kind of
insight from at least the fourth century
bc when one of aristotle students wrote
an early physionomic personality manual
based on face types
this was based on the idea that people
who looked like certain kinds of animals
would have traits associated with those
animals so the person in this picture is
supposed to be perceptive and determined
like a sighthound
students of medicine have taught have
sought insight about humans by
dissecting the bodies of the dead and
mapping the bodies of the living testing
out their theories and refining them
and in the 1940s the myers-briggs type
indicator test was released this is
stuck around recruitment and hr
departments to this day as a way of
learning more about candidates and
employees
despite the fact that it does have
well-documented methodological problems
buzzfeed knows this about humans which
is why the quiz section on the site is
so huge
they rake in money by placing a long
line of ads on our screens right next to
the place where we spend our down time
working out whether we're more of a paul
or a ringo or what our taste in 90
sitcoms says about our fashion sense
and i should add at this point that
there is nothing wrong with enjoying
taking a quiz or filling out a
questionnaire it's just another part of
being human
the problems creep in
when quizzes get weaponized
the questionnaire was invented by a guy
called francis galton this is him
galton was obsessed with categorizing
humans he made it his life's work the
questionnaire was a tool that he used to
help his research he was also charles
darwin's cousin and also the guy who
introduced the idea of using
fingerprints as a uniquely uh a way of
uniquely identifying criminals and
matching them to crime scenes and also
the guy who coined the term eugenics
which is the idea that some humans are
biologically superior to others and that
we could create a master race of human
beings by breeding superior humans with
each other and killing off or enslaving
the rest
and if you're thinking that sounds
familiar it's because yes galton's work
was directly referenced by the nazi
party in order to justify the holocaust
it has also been used not just by the
nazis to justify racial segregation the
removal of children from their families
uh the forced sterilization of disabled
people and people with mental illnesses
and prison inmates and queer people and
people of color
and other categories of people deemed to
be quote unquote bad for humanity
somehow
yeah
um so at around the same time as galton
and his theories there were a lot of
other people working on similar ideas in
europe
in france alphonse bertion this is him
invented the mugshot and a system called
anthropometrics this was mostly used to
keep track of repeat criminals by
measuring things like the circumference
of their head the length of their
forearms and taking those front and side
on pictures
for record keeping purposes this is a
mug shot of bertion himself with some of
his own measurements demonstrating his
technique
and in italy a guy called cesare
lombroso was one of the first people to
use the term criminology to define the
study of criminals and their
characteristics
unlike criminology today however
which covers a wide range of the study
of criminal behavior lombroso was most
interested in categorizing the humans
who had committed crimes
he was a proponent of the idea that
uh well i mean it's been long since
discredited he was a proponent of the
idea that some people are born criminals
who are biologically more likely to
commit crimes no matter what because of
this according to lombroso
he said there would be physical ways to
tell who those born criminals were so
the system that he developed to tell
criminals from normal people included a
lot of the things that will sound
familiar to you at this point like
measuring the size of the head and
analyzing the facial features
lombroso's work incorporated a lot of
classic physionomy of that sort of
aristotle area era
it drew from animal imagery
so according to him people who look like
lions were fierce and also gun leaders
and people who looked like bulls were
going to be stubborn and determined and
so on
you can likely imagine the comparisons
that got used for people who looked less
white skinned and european
um
they were far more distasteful and
degrading and degrading and i'm not
going to repeat them here
sometimes physionomy also worked the
other way as you can see here this dog
is apparently ambitious and jealous
don't ask me how you could know this i
have no idea
lombroso also believed left-handedness
was a sign of the born criminal just
like most spiral notebook manufacturers
do
and this would be more amusing if he
wasn't trying to sell these ideas to
actual law enforcement
in germany franz josef gaul this dude
came up with the pseudoscience of
phrenology which was supposed to explain
things about human character by
measuring the shape of a person's skull
his reasoning was that because the brain
was the seed of the personality it would
be possible to tell what sort of
personality you had by which parts of
your head were bigger than others
larger forwards for example meant that
you were supposed to be really smart
and as i said this is a picture of goal
so you could see why he might have a
vested interest in saying that
phrenology along with physionomy entered
the realm of popular entertainment
really quickly
as the idea spread throughout europe and
the united states people published
handbooks that you could use as a guide
to
reading the personalities of others
two american businessmen brothers by the
name of fowler started manufacturing
phrenology busts like the one you see
here for use as a reference tool
these kinds of busts are mostly seen
these days as a prop in the office of a
psychiatrist on tv even though most real
psychiatrists wouldn't go near one of
them with a 12-foot pole
people held phrenology parties where
they would literally have their heads
examined and their personalities
diagnosed you could even use phrenology
to choose the ideal spouse as you can
see in this page out of a phrenology
handbook from 1902
a lot of these images that you've been
seeing as you will probably have noticed
are extracts from different popular
handbooks on phrenology and physionomy
that are published that were published
around this time i'm showing you these
partly because some of the illustrations
make no sense at all even in the context
of the second weird suit of science but
also because these illustrations
demonstrate the kinds of things people
sometimes have really genuinely believed
about one another
i should add that at the time this was
happening most mainstream scientists
were absolutely not on board with
physionomy or phrenology or eugenics by
the early 20th century as is the case
today the general consensus among the
scientific community was that none of
these personality revealing methods were
especially scientific
there was no way that you could use
methods like physionomy or phrenology to
prove anything at all about a human
being's personality a skull is a skull
and while measuring a skull might tell
you something about that skull's
structure or density or shape
that was all a skull has no moral value
or social role it just is
but it didn't really stop the popular
appeal of the idea of some kind of
universal system where you could look at
a person's face and know all about their
character
and it didn't stop authoritarian regimes
from seeing the potential in investing
in these kinds of ideas
one of the popular phrenology handbooks
that i looked at during my research was
this one it's literally called his power
it was published in 1923 just as some
world governments were using these bunk
philosophies to establish their own
political power
and all of these ideas drifted in and
out of each other's spheres in the late
19th and early 20th centuries they
bounced and branched off each other in
countless ways
you will have seen that some of the
things that came out of the effort to
measure and categorize human beings
ended up being absolutely useless except
for a quirky way for some people to pass
the time
other things that came out of this were
among the greatest and most terrible
atrocities perpetrated by human beings
against each other in the history of our
existence as a species
and still other things that came out of
this were things that have become an
accepted or even arguably useful part of
life today in many places
the study of criminology the use of
fingerprints and mug shots and police
work questionnaires and surveys as
research tools and the measurement of
human bodies in order to learn more
about them for medical reasons
this is the thing about biometrics
generally
while all measurements are themselves
numbers on a page or in a database the
questions that we try to answer and the
problems that we try to solve using
these measurements can have profound
impacts and consequences whether they're
good or evil
so it's now the year 2022 and even
though there are human beings living in
space and we have the ability to hold
events like this online in real time
we haven't fundamentally changed we
still want the same kinds of things like
knowing about each other and ourselves
to find a way of organizing the world in
our heads that makes sense to us and for
feeling like if we have enough data we
might be able to establish some kind of
control over our lives
we even succeed at it some of the time
wearable devices like the fitbit and the
apple watch have become ways for us to
measure the things that we want to know
about ourselves like the number of steps
that we take in a day or our heart rate
or how well we slept last night
some of us like to measure our weight
and our bowel movements and our
menstrual cycles and some of us lived
vastly improved lives because we have
the ability to do this i mean i'm i'm
wearing fitbit right now
the tools we have for
uh for using this these days have become
more complex than calipers and measuring
tapes
the current model of an iphone for
example this is not a picture of a
current iphone iphone users will know
this but the current model of iphone has
a true depth camera with infrared depth
perception capabilities
another 12 megapixel camera i think two
now a microphone an ambient light sensor
a moisture sensor an accelerometer a
gyroscope a compass a barometer and a
thumbprint sensor and we can carry it in
our pocket
if you add a specialized device like a
smartwatch or an external camera our
capability grows and as developers we
can use these capabilities in our own
software to help personalize them for
our users and make the things that they
do more relevant and meaningful
some of you who are watching this might
have built things using things like
apple's research kit and care kit or
health kit maybe all three these
frameworks really super powerful they're
useful in the potential that they
represent for having people help for
helping people sorry research and detect
and manage human health using them
people have done awesome things like
learning to predict epileptic seizures
and manage asthma and track medication
use and care for patients after surgery
and learn more about parkinson's disease
but getting meaningful results usually
comes down to whether or not we're
collecting the right data and using it
to answer questions that it can actually
answer properly and then accepting
feedback and reviewing our hypotheses
if you've ever worked with user input
data
you probably know
just how difficult it is to handle
everything that your users throw at you
things about humans are hard
names genders addresses time zones
geographic location language you name it
you will find an exception to your rule
this classic list of falsehoods that
programmers believe is a fun way to dive
in and see just how many ways that
people can be
human beings are impossibly varied and
anyone we're not thinking about at the
time when we build stuff will probably
be left out
some of you will know
in 2014 apple became infamous for this
because at its launch the health app
contained no way for users to record
menstrual data despite the fact that
close to half of iphone users did
menstruate
this feature was added in 2015 and
they've clearly taken this on board in
the years since then some of you have
probably seen that you know there's a
cycle tracker by all accounts the
features that that it now has the
language it uses show that it's been
given a lot of thought
but even when you've accounted for the
most standard use cases it's hard to
come up with a system of human
measurement that's true for everybody
sentiment analysis from facial
expressions is a particularly flawy area
our faces don't all work the same way it
might be all well and good as long as
you're in total control of everything
that your face is doing but not all of
us are some folks will have ticks that
they can't do anything about or they may
have neurochemistry that makes it tricky
to express emotions externally or
perhaps not the expected ones
this software also won't work well for
you if you've had a stroke and you've
lost control of some of your facial
muscles or even if you've been to the
dentist that day and you can't really
feel your mouth
we get similar problems with gait and
posture analysis to try and predict
things about someone's mood what an
algorithm might have been trained to
perceive as threatening or angry
movement might also easily be the result
of a panic attack or a lack of control
over motor functions and the same is
true of failing to account for the way
that human beings change over time and
sometimes rapidly there's been more than
one example of pregnant people being
hounded by their health trackers to lose
weight when they were supposed to be
gaining it and then having the health
trackers yell at them again for not
sleeping poorly
for for sleeping poorly once the baby's
been born
there have been forums filled with
first-hand accounts from people with
mobility issues being berated for not
completing their 10 000 steps when some
people just want to celebrate doing 50
steps
and then we have issues with all of that
data that we collect being used for
purposes that have nothing to do with
helping the people that the app is
ostensibly for
once we got the hang of collecting
menstrual data it was great for period
tracking but we've also learned that
companies like ovia health have
developed family planning apps that
people's employers can monitor so in the
april of 2019
there was a report that activision
blizzard the video game company were
encouraging menstruating employees to
use this app which could provide their
bosses with data about their menstrual
cycles and pregnancy stages and fertile
windows
in exchange in exchange for getting a
front row view to all this uterus
activity for honestly god knows what
reason the company paid these employees
an extra one dollar a day
given the fact that
an entire
californian government department has
now filed a suit against activision
blizzard for their despicable workplace
culture i cannot imagine what extra hell
it must have been like to work there
while knowing your bosses were tracking
your periods
something some insurance companies like
in in the united states in particular
also have policies that mandate that
their customers wear a fitbit or they'll
pay extra premiums for the healthcare
cover
and this continues to ensure that the
only people who can sidestep this this
cycle of data gathering and surveillance
and those who have enough money to
afford it leaving the privacy risks for
the most vulnerable people
then we have the straight up old-school
phrenological style uses the kinds of
services where data is gathered about
people and used to draw conclusions that
there is no scientific basis for check
out this screenshot here this is a
company called face session face ception
they've been around for a while now they
have a whole business model centered
around inferring things about people
from an image of their face things like
whether or not someone might be a poker
player or a terrorist
in 2017 two stanford university
researchers conducted a study that
claimed machines were better than humans
at looking at someone's face and
determining if they were gay or straight
not only did this study lack rigor and
completely ignore the existence of
things like bisexuality or ethics it
also re-popularized the idea that there
was something inherent about looking at
another person that could tell you
whether or not they were attracted to
people of the same gender as themselves
a researcher at the university of
pretoria in south africa tried to
replicate the study in november 2018 and
found that the results didn't bear up to
much scientific scrutiny but that report
didn't receive anywhere near as much
publicity
and then in the january of 2021 one of
the authors of this weird gaydar study
did it again claiming that facial
recognition tech could tell you
someone's political orientation
if that's not enough during the late
trump administration parts of the u.s
government started considering whether
they should create a national database
of people with mental illnesses this
would have been combined with a program
to harvest data from health trackers
like the fitbit and the apple watch to
create a model that would watch for
quote unquote neurobehavioral signs that
might indicate someone with a mental
illness was going to go on a shooting
spree
the group that pitched this called it
the safe home project this is a backroom
for stopping aberrant fatal events by
helping overcome mental extremes
and while the problem of gun violence in
the united states is definitely
something that needs addressing
weaponizing biometric tech against
people with mental illnesses is unlikely
to do anything except create fear among
a vulnerable population
quite apart from the fact that there's
no way we can tell whether someone's
planning a violent rampage just from the
readings off of like an apple watch
there's also the problem that internet
of things devices especially consumer
grade personal health devices aren't
that always that accurate that might be
enough to tell you you got as much
physical activity as you wanted compared
to other days but there have also been
cases like this one where the microsoft
band made rest in peace was shown to be
getting a heart rate of a chicken breast
and there are also reports that using a
green light as an optical sensor for
tracking heart rates means that devices
like the fitbit aren't accurately
measuring this info for people with dark
skin
consumer grade health trackers will
often go with green light sensors
instead of infrared because they're
cheaper and they lower the cost of the
device but it's been shown that melanin
will absorb green light so those with
more melanin in their skin aren't going
to get the same results as people with
less of it
so once again
we're at a point where scientific and
technological advances are coming at us
thick and fast and the world is changing
and we think we can solve all the
world's problems by classifying and
measuring the humans in it
but by now you should be clear that i
think some of the questions that we're
trying to answer with biometrics aren't
always the right ones and in some cases
they're going to lead us or they are
actually leading us down a really
dangerous path
we have a world
where there are actual nazis again
and people are writing apparently
serious articles about craniomatry and
race again
and the last time around weaponized
categorization was one of their most
potent tools
so knowing what we know what do we do
i mean most of us mean well we want to
do our best we want to build cool apps
and software and all kinds of things
that help people
but not everyone's thinking about the
data or using that data to answer
focused and answerable questions
so how do we make sure that we get the
best out of our scientific advancements
and acknowledge our boundaries and our
biases
right now the main way of addressing
this seems to be coming up with a
proposal for an ethical framework that
can be used to guide us
so when i searched this a couple of
hours ago google scholar returned me
almost 700 hits for articles on
biometric data ethics frameworks
published just this year um i should
remind you that i am delivering this
talk two weeks into january
in the last several years other ethical
frameworks for artificial intelligence
applications to massdev
for artificial intelligence applications
for mass data gathering have been
proposed by microsoft and ibm
and google
and the australian government through
the csiro
and countless other research bodies and
governments like the ethics institute
which has published
this responsible machine learning
principle
and there are technology consortiums
like the partnership on ai and the
information and technology industry
council which have both published
ethical framework principle statements
that their members have to agree with
a study that was done at the western
sydney university in 2018 analyzed a
whole lot of these ethics frameworks and
distilled them into these five focus
points privacy governance fairness
shared benefit and transparency
these are all great and important things
to think about i'm glad that we have
frameworks like this and more than one
that i'm glad that peop there are people
who want to talk about them
but what most of them lack is any actual
accountability or requirement or
reporting or oversight
they're mostly a list of nice things to
have but they don't actually represent
any obligation
and in an industry where it is difficult
for some organizations to realize that
they could be doing the most good of all
simply by paying tax in the countries
where they operate the chances of these
frameworks being adopted in practical
terms may vary
it also doesn't look great when
companies fire the people they hire to
help them with getting these things
right
like when google whose ceo i showed just
before touting their ethical ai
principles fired tim nitzgebru
a an extremely prominent ai ethics
researcher
because she pointed out biases in their
systems
this slide isn't working but it is a
screenshot of an article
describing what i just said from the new
york times please go and look that up
it's a real thing that happened if you
missed it at the time
so
legislating about against the worst of
these abuses on the other hand this
might have a stronger effect
the gdpr is one of the more prominent
examples of an attempt to address
inappropriate data use
and it includes biometric data
it's improved things not just in europe
where it applies even some of us in
australia probably most of us at this
point will have had to work on making
sure that our systems comply with it
if we do business in europe or we handle
data about europeans and most of us do
either or both of those things
it's no coincidence that europe with its
strong laws is where a lot of this
phrenology and physiology stuff first
started being developed and applied
the attitude to data collection there is
strongly informed by the fact that they
lived through some of the worst of what
can happen because of this
but we still lack good legislation
sometimes any relevant legislation in so
many other countries
sometimes older laws might be bent to
apply to this situation but there isn't
much that prevents biometric data from
being abused this is partly because of
the well-known problems with technology
and law where tech capabilities have
grown quicker than the law is able to
keep up and partly because it's not
always easy to understand how new tech
might bring up old problems again
there's also the fact that in australia
we still don't have a bill of rights so
we don't really have a legally enshrined
right to privacy
which is a concern given the absolute
torrent of pro-surveillance legislative
changes that have been made recently
which affect the information that's
gathered about us by both governments
and companies
lots of us are time poor and in the
third year of a pandemic many of you may
have bigger problems than campaigning
about biometrics ethics which is fair
enough
there's another other crap going on in
the world that we have to know how to
pick our battles but if you want to
contribute in some way to helping out
you can always donate to the groups that
are keeping their eye on these things
holding organizations accountable and
campaigning for proper legislative
reform
in australia specifically you can donate
to both digital rights watch and
electronic frontiers australia who are
collectively doing that absolute best to
look out for the interest of all of us
in this space
and because i am full disclosure on the
board of digital rights watch i can
attest wholeheartedly that we would not
have been able to handle the amount of
junk the last few years has thrown at us
without the support of donors so we're
actually able to hire staff now which is
great because holding the government to
account on these issues is literally
several full-time jobs if you want to
take action but you're not sure how or
you don't have time but you do have a
few spare dollars consider donating to
either or both of these organizations
because it actually helps seriously
and in terms of what else you can do if
you work in tech and you're asked to
implement systems that you think are
feeding into the same problem
push back say no
we are the people with the skills needed
to make these things real and we get to
choose how we use those skills
that said i realize it's not that easy
for a lot of us to just do this you know
it requires enormous privilege and
confidence to be able to stand up and
say no in a workplace not everybody's in
a position to do that that is okay
but if you want support in how to speak
up in your workplace this is where
unions can help you out there are unions
supporting tech workers in australia
like professionals australia and we're
seeing growing numbers of tech workers
all around the world unionize because
collective action is the only way that
we can really fight against mega
corporations owned by men who have
enough money to fire themselves into
this into space for the lulls
they didn't stay there anyway
to fight back against them as well as
against all the systems and standards
that they prop up as normal
the other thing that we can do to stop
biometric abuse becoming part of
everyday life
is to educate each other about where
humanity's already been and the plethora
of futures that are possible that don't
involve perpetuating these systems
this is why i'm here right now i talk
about history so that i can help give
historical context to other software
developers because knowing more about
what came before helps us to build
better futures
so this talk is one resource i'm adding
to the pile that you can use to help
other people learn about these issues
and here are some other resources by
people who have studied this topic in
much further depth
they cover off everything from the
history of scientific racism to the bias
in google search results
some of them most of them will make you
angry but also most of them will give
you hope
if you want to check out some of the old
phrenology and physionomy handbooks like
the ones i've been showing you pieces of
today
honestly maybe don't
a lot of the content in these books is
disgustingly racist and sexist and
classist and just about every other kind
of ist you can imagine there are better
things to read
if this talk is about anything it's
really about the fact that biometric
measurements and data don't always give
us good answers for the problems that
we're trying to solve no matter how much
that we've been told that data will save
everything solve everything sorry and
save everything i guess
we have been trying to get somewhere
with the whole categorizing humans thing
since literally before the time of
aristotle and we've been repeating the
same questions to ourselves over and
over again
there is an interesting irony in us
wanting to harness biometric data
because we want neat answers and systems
for our lives but that the answer that
we keep coming up with is that there is
no one good system
everything works on a case-by-case basis
human beings are all kinds of strange
and wonderful things and they can't be
categorized we can't be categorized
sherlock holmes for all that he was the
pinnacle of the victorian rationalist he
knew this too in the short story a case
of identity he observed that life is
infinitely stranger than anything which
the mind of man could invent
people are weird and messy and if we
write software that respects and works
with that
maybe we'll do a bit better
hopefully we can make decisions about
what's right by being informed about the
past and by learning from the worst of
our mistakes
if you want to keep talking about the
topics that i've covered today
feel free to get in touch
um here's me on twitter again
thank you very much
to knock away lily that was uh amazing
um
and i think you know
so much of what you've put there is as
conversations i've had for the last five
years with friends and family like they
don't understand why i don't have a
fitness tracker or a smart watch or why
i don't opt into these things and it's a
really difficult conversation to have
because you end up
going into this rabbit hole of this is
what's going on this is how it's been
collected and harvested and yes there's
some benefit to for having that cloud
service
because you get some insight but where
is where is the rest of it going yeah
it's funny
yeah my fitbit went off during this talk
because it's like your heart rate's
really high and i was like i know i'm
giving a talk it's nerve-wracking
um we do have a couple of questions
coming in um and
i think there's more coming in as i'm
speaking so that's good sign um the
feedback in the channel has been amazing
i know you can't see it but people no i
got that close engaged
and really engaged so i'll come back and
check it later and i'll be um i'll be in
the post uh talk conversation room
um
in venulis after this
um so question
what can we do in a digitally connected
world to reduce our risk exposure as
individuals and as societies
okay just starting off with the small
small questions yeah
i think you know individuals are
probably the thing that we have the most
control over societies are harder being
that they're larger scale
um part of that can be about thinking
about the tech that you want to bring
into your lives and whether that's worth
it clearly i made a decision that i
wanted to use a fitbit
i use that within my own specific
parameters and
[Music]
control as much as i can where that
information goes but
certainly asking yourself the questions
about the kinds of things that you want
to be using in your life is important
and if you have no choice
um besides using some of this stuff in
your life because i know that there are
people who for various medical reasons
are going to have to
um and probably for other reasons too
um
it's worth thinking about the ways that
you you might be able to limit that and
if you can't also getting in touch with
the companies and rattling a few cages
to ask more about why there aren't as
many options
yeah i mean one thing i found is that
it's almost as if this is the new
digital divide those that are privileged
positions enough to be able to push back
against these sort of technologies and
this data metric that's almost the new
digital divide it's not access anymore
it's about the ability to opt out
yeah and whether or not definitely
position to do that
yeah definitely because it's always
about you know paying
for that extra bit and if you don't have
the money
you can't
and and that's really you know
the the main thing that causes those
issues that the people who don't who
can't afford it are going to have
this sort of inescapable path towards
having all of their data gathered and
harvested and being surveilled in all of
these ways
um another question
is there a standard for new software
engineers who might not be aware that
they code
the code that they write may have
ethical issues
good questions that's a good question
i mean
i know that there are quite a few
frameworks i run through a lot of them
um and there are far far more than i
than i was able to put into this talk or
we would have been here forever um
i think just you know even
sending them a link to this talk
um after after it's published or
giving people one of those books that i
listed earlier might be a way of doing
it um i know that um particularly for
new graduates um there are there is a
lot more focus now on bringing ethics
into computer science and software
development courses which is great um
but it's it doesn't mean that it's going
to reach everybody in quite the same
ways and unfortunately a lot of the time
most people only get that realization
when something happens to them what
happens to someone they know that they
realize is unfair or they haven't been
able to opt out of it or it's creepy um
i know a few people who had that aha
moment when the taliban took back over
in afghanistan um recently and
they started using all the biometric
data that the us had gathered
about afghani citizens and particularly
afghani us collaborators to start
persecuting those people so
yeah that i mean that is one one example
of how it's not just this academic
slippery slope argument you see the same
thing with the way that facial
recognition is used on the uyghurs in
china as well
that these things do get used in these
ways and there are ways of stopping them
and thinking about them that start with
the very small scale stuff and by
thinking about what we're going to
accept in our lives
um at the risk of possibly opening a
hornet's nest that i think you've very
very well avoided but i mean this
pandemic obviously has has raised a lot
of the similar questions around the data
privacy and and you know there's the
same kind of
mold as we've been discussing um and i
guess yeah i think that that certainly
has created a bit more of a zeitgeist
than the general public
around these issues
what's really gotten me has been
particularly the way that it's affected
kids during the pandemic with academic
monitoring software that has the camera
on all the time and you know they're
trying to use it to make sure that
attendance is assured and that people
aren't cheating and copying answers from
each other in a remote way all this
proctoring software that um gets loaded
on into like children's laptops and
turned on and
you know you can't opt out of it
um and i know that that happens too in a
lot of workplaces where
particularly for those and again it's
usually people who aren't in
particularly high-paying jobs that
aren't trusted as much for no particular
reason but their work their employers
don't trust them and then they get
monitored through their cameras all the
time every day while they're working to
see whether they're working to see what
they're clicking on to see how fast
they're typing to make sure it's really
them
um it's creepy and it's insidious
and that's that's just one of the ways
that the pandemic has made things worse
for a lot of people in these ways
yeah i mean there's some comments in the
chat remote proctoring is weird for
professional certs i i mean we do it at
red hat and i when i learned that i was
just like well i guess i'm not getting a
red hat cert now because it's just
ridiculous i mean there's actually a
bunch of studies and academia looking at
how it doesn't really achieve what it
sets out to do right and i think that's
it's just silly silly costs but
someone's selling something and
someone's buying it right
yeah thank you very much for your talk
lily that was really good and really
engaging um as i said there's been some
really good conversation going on on the
channel
um
again thank you
no worries
um i'll be in the chat later
um just a couple of notes before uh
before i hit off um so professional
delegates uh networking session um will
be happening in about an hour so that's
at eight
sorry
6 30 australian eastern standard time
um so if you come back into venulis then
there will be a special room open if
you're a professional delegate ticket
holder
and for speakers and i think the
volunteers as well so please join us
back to there we have a an interesting
talk by anthony green who i had no i had
no idea who this person i'm kiwi so
apparently he's quite a a prolific
political commentator
um that most australians know so that
should be really interesting um to hear
his thoughts on
politics analysis stuff which sounds
cool
all right thank you very much for today
everyone it's um been great being your
host in this room
enjoy the rest of your evenings