[Music]
so welcome back it's time for our first
uh invited presentation by brett downing
uh he'll be talking about microarrays
and advanced robotics i'm really looking
forward to this talk um brett's a a
really really interesting fellow very
talented i met him first been about 10
years ago at the uh just prior to lca
2012 with his sister jenna and daniel um
they come over from perth and these
mysterious strangers help us
set up tidy up and they're going who are
these people and uh yeah and since then
if you um if you ever need a sonic
screwdriver brett will have one on him
so uh
i think these days he carries around the
pine saw soldering uh iron so yeah
without any more to do over to you brett
um take it away and also i believe brett
will will take some questions so um yeah
i'll line them up on the message system
please
i would do thanks fred great um
i'd like to begin today by acknowledging
the traditional owners of the lines we
meet on today and i'd like to pay my
respects to elders past and present
i've come here today to talk about
advanced robotics on microcontrollers
particularly
a bit of background roz the robot
operating system is a
software framework that provides publish
and subscribe messaging and packaging
tools
it was started in about 2007 as a
response to prolific abandonware in
academic robotics departments every new
student would build a new robot with new
ways of talking to new components and
the rest would get scrapped year after
year
and
you were lucky if you ever got a whole
robot even a simple one even for a
moment roz offered messaging and
software packaging conventions that were
extremely strict and
prescriptive but fairly versatile
the defining feature of roz
was that it encouraged modularity
consideration for system integrators and
it allowed researchers to share basic
functionality
and package their research for others to
fork and cite
roz is based on a data flow
framework
most nodes don't have flow control like
if or switch case
nodes just receive
receive messages and emit messages so in
a simple
remote control example you might have
a joystick node which deals with the
hardware of a joystick
it might emit a message
with a strong typing that describes a
joystick
state
that might get interpreted by a
teleoperation node which handles a loss
of input
and converts it into like a
a motion vector in a coordinate space
this doesn't care about
the shape of the robot but it handles
you know dead man switches and stuff
another motion might subscribe to
the
the motion vector
and describe how the robot should move
but it's another it's yet another node
that's responsible for actually
figuring out how the the joints should
all behave
this separation of intent is what makes
roz so appealing every robot that moves
can be driven by
every met every motion planner
every remote control if you can prove
your path planner on a milk crate with
drill motors zip tied to the bottom you
can expect it to work on a
state-of-the-art humanoid machine
likewise if some research lab has proven
a computer vision mapping
uh mapping and planning suite
you can reasonably expect it to command
a hacked vacuum cleaner maybe you'll
need to provide some image processing
power image processing power on a second
machine
these aren't hyperbolic statements
either the messaging convention is so
strict and the packaging is so
prescriptive that it's actually very
difficult to leak information
around the abstractions roz provides
more often than not i found that raw's
rose packages fresh off github just work
when you want to deal with the real
world you first need to collect some
data think about what that means to you
and then do something about it
sense and act
happen need to happen in particular
places and in particular ways this is
where software meets hardware
hardware is hard and it throws up all
sorts of constraints for a practical
application you might need high
resolution timers you might need to deal
with esoteric industrial protocols you
might have low power requirements or
heat tolerance issues
these are things you just can't get on a
regular computer
the thinking part is pretty flexible
though it only requires some computing
power there's no exotic hardware
involved it doesn't need to happen
anywhere in particular
things worth really thinking about like
map generation often don't need answers
right away
things that need answers right away such
as obstacle avoidance often don't need
much thought
the separation of duty
ros two nodes don't need to be on the
same machine
in the example here
there's only a couple of nodes that
actually exist
on the robot
and those are over here
and there's
a robot that's not showing up on a
note over here that's not showing up
uh the rest is all on my laptop all of
the inverse kinematics all of the
joystick handling all of the
everything else
the ros 2 nodes automatically discover
each other over the local network ros 2
transport layers are deeply configurable
and do support encryption and
authentication the hard work is all done
on the laptop while the robot can focus
on doing things and measuring things
this separation of duty is why i'm
so excited about micro ros it takes it a
step further than just having a big
machine and a little machine
going back to sense think act there's an
implicit step between
act that goes back around to sense
which is why the world has a think about
you and stuff moves about
the trouble is the world doesn't let you
have your turn in peace and stuff
continues to move while
stuff continues to move about while
you're trying to have a think
i'm going to take detour from robots to
talk about time
until about 2010 pcs came with parallel
ports
and parallel ports could interrupt the
cpu in a cnc machine you could measure
the length of a cutting tool by lowering
it onto a conductive pad
and stopping when you complete an
electrical circuit through it
it was very simple to write software
that could use the parallel port to
detect tool contact
stop all the motors within tens of
microseconds the parallel port became
obsolete and was replaced by usb
adapters
the communication round trip of usb
pushed the latency out to about a
millisecond the port throughput was the
same so consumer experiences were good
but motor commands would get buffered in
the adapter so an instruction to stop
would get delayed behind instructions to
advance
the additional buffering made the
difference between an accurate
measurement and a broken tool
as computers got more powerful we
expected more from them more threads
more things at once modern task
schedulers are amazing things some of
them allow you to schedule operations
to happen by some deadline
instead of indicating some relative
priority
this allows things like real-time audio
processing on linux
mouse cursors that don't stutter
deadline schedulers are useful for
robotics but they're not enough on their
own
very often you need better timing
resolution so you need something to
happen
not just before some deadline but at a
specific time
if you've ever tried to use addressable
leds on an 80 tiny processor and then do
something else as well you know it
doesn't work
to get really good timing guarantees on
a cpu you need an ability for the cpu to
drop whatever it's doing
and respond to an external event a cpu
interrupt
unfortunately cpus take time to do
anything and things have a habit of
happening all at once
the more interrupts you want the cpu to
service the more likely it is that your
interrupts will collide with each other
and one will get delayed or missed
the more strictly timed code you try to
write the less of your code will be on
time
i mean on time in the strictest hardware
sense here running a specific cpu
instruction on a specific instant
bit bash protocols and physics
experiments
the promise
of time sharing of a cpu is incompatible
with the promise of handling multiple
unpredictable
events on time every time
the solution doesn't exist to be written
down in a software framework
the way out is you compromise you relax
some constraints limit the timing at the
resolution of your timing guarantees and
then only guarantee that code will run
before
before a deadline
that compromise is what a real-time
kernel offers and it's plenty for a lot
of things that i didn't come here to
talk about
the other constraint you can relax is
the assumption that we have fewer cpus
than things we want to do more hardware
less sharing of resources
more cpus servicing fewer interrupts
each simpler cpus that have more
deterministic timing or even specialized
devices that can implement desired
behavior without cpu intervention
a lot of embedded systems engineering
for battery-powered devices
boils down to corralling microcontroller
peripherals to do all of the things
while the cpu takes a nap
but less sharing of more hardware is not
a solution
it's just a different problem
fortunately the throw hardware out of
problem space is a little more tractable
these days
if i have to leave the comfort of gcc on
linux and write my time sensitive stuff
on a microcontroller i don't also want
to write about passing messages back and
forth between the micro and some host
system
and then also write about uh write that
on both sides of the communication
channel
using an arduino for an out of chassis
experience for a robot was so common
that the ros community made a couple of
compelling attempts at getting ros to
work nicely with microcontrollers
roz serial provided a generic host side
node that could be instructed by an
arduino
to interact with a host side
data stream the arduino side library let
you set callbacks to be executed when
messages arrived and pack messages to be
sent
overall it was very clean but
it was very
it was a very clean implementation for
ros one by the time roz 2 came about
it had a few issues though
most arduino boards have a physical
serial physical serial port behind a usb
converter so they don't move data
particularly quickly
raw serial also couldn't be extended to
wi-fi or ethernet when the
esp8266 became popular or when ethernet
shields became cost effective
library support for the various messages
had to be generated from host side code
and copied to an arduino library it was
automated but still a little a little
rough on beginners
because the serial port on almost all
arduinos and clones used usb serial
emulation they inherited a 10 meter
limit to the tether to a host pc
it was common to see wireless bridges
open wrt routers
or xp radios but they often doubled the
cost of the actual robot
these limitations needed a systematic
fix
instead of wrapping some key functions
on microsite code and then leaning on
the host for actual work
micro ros builds ros2 natively for the
target microcontroller
microros provides a full-blown native
ros 2 interface for arduino esp
stm32
because roz uses mainline ros2 source
the developers were able to get
incredible automated test coverage which
made for a really pleasant pull pull
request experience on
github peer discovery is
important for flexibility by forcing a
decades worth of robot developers into a
strict and modular design pattern any
host side nodes ever written can talk to
any microcontroller on the network
without modification
micropros integrates with free rtos so
you can have fancy threading stuff or
you can just write event handlers
i mean you can take it as uh
as shallow or deep as you please
a lot of
roz developers these days are having to
square with automotive safety integrity
coding guidelines so there's actually
very little memory thrashing at run time
in micro roles
micro ros inherits the plugable
transport layers
of of ros 2
which became way more significant than i
expected
when i proposed this talk
none of the regular ros transports would
build on a microcontroller so microroz
relied on hosts on a host side node to
broker transactions through a simpler
communication protocol
the microros agent supported serial udp
can bus
but the agent sat on a host like the end
of a tether lending a similar topology
to raw serial
that tether could be wi-fi but the agent
was still a central point of failure on
an otherwise distributed system
these simpler protocols were implemented
as uh
as proper ros transport layers though
they just lacked the killer features
that made ros 2 so powerful
and then
someone announced embedded rtps
i don't have a slide for this one
um
ros 2 ships with the default transport
layers based on dds the data
distribution system
ros 2 treats it like a
big old name space that holds the
current states of a bunch of values and
updates them on any machine that cares
behind the key behind the curtain dds is
a lot more than that
dds is provided for ros 2 by a couple of
vendors and they're all notionally
intercompatible
for at least the basic features
one such basic feature is rtps real time
publish subscribe which handles most of
the network transport tasks
rtps
is not to be confused with any of rtp
rtsp
rtmp
rtmps
realtime publish subscribe
yeah
acronyms
and a cluttered namespace embedded rtps
had its first commits in 2018 and built
an implementation of rtps that
works on microcontrollers with lwip the
lightweight internet protocol and free
rtos allowing it to target
the
threading and networking capabilities of
stm32 arduino and esp
embedded rtps announced on the roz
discord
that it had passed some compatibility
tests with fast dds the
the host side dds implementation for ros
foxy
in january 21
uh then in october the micro ros
maintainers
had included it as a transport option in
micro ros eliminating the need for the
agent
bringing raw micro rods into a
decentralized topology with the rest of
roles
i think
i'm going to go into demo time now
so i'm going to hope this all works
um
so these are all the nodes that i
currently have running
and
see
if i visualize what the kinematics is
doing that's actually up behind me
that's this one
where the camera pointed at it from a
second robot
and then on the joystick
i can
move all of the parts like so
so
i've got
a joystick input with a deadman switch
which is all handled on the host side
computer i reuse all of these nodes for
any robot that drives around
i've got an openscad model that has all
the inverse kinematics
all the inverse kinematics runs on the
host side machine and then it sends the
updated states
to
the microcontroller
um
so that's about all i've got
uh the microcontroller in in this demo
is the esp32
and it's just running uh
um
it's accepting server messages for
several messages
an array of where am i
i can interrogate
the topics that i'm using
and i can
read off the commands that are being
sent to
if i can type it
uh read off the commands that are being
sent to the claw
so these are just um
this is just an array of
signed integers each one represents the
angle of a server motor
the joystick
again is a standard message
with stick inputs
uh stick inputs has floating point
numbers buttons as i think also floating
point numbers
um
and this this kind of standardization
means that any
any developers can sort of share basic
functionality of all of their robots so
the same messages i use here i use on
the other robot behind me
they all use very similar hardware so
the timing performances demo is not
great i've kind of used the
the simplest implementation
of sending just the the updated state
instead of sending like a schedule of
motions to to follow
so normally in a like the reason you
would want to use roles on hardware is
because you want you know strict timing
you want to have a
motion plan coming from a host host
system
you can have motion commands coming from
a host system with a sort of default
estop behavior if something happens
instantly on the
on the microcontroller on the
if the microcontroller needs to respond
to an event quickly you can you can do
that on the microcontroller
uh we've got a question about inverse
kinematics
where are we
inverse kinematics um
sure
kinematics is describing
um
describing how a joint should move in
order to achieve a certain result so
in the case of
uh in the case of this claw i've got the
contact points where the claw should be
moving around a circle
kinematics is
essentially where that contact point
ends up for a given mode of motion
inverse kinematics is figuring out how
the motors should move to achieve that
desired motion so
here i have a gate synthesizer which
describes a path plan
it describes a path through space and
the motors
the inverse kinematics engine tries to
find joint joint poses that achieve
those
um
that achieve those positions
uh
and i did actually implement
inverse kinematics on openscad i kind of
got a bit carried away
um
openscad being a
uh
having its variables are
openscad's a bit weird it's the simplest
possible cad language that you could
use
um
so it's the simplest language that could
solve a problem
which means that it doesn't have like
for loops or mutable variables or stuff
like that
it just has like tail recursion if you
want a loop you have to use taylor
equation so in this case i have uh the
animation loop figures out places for
the contact points to be
and then it um
it uses tail recursion to optimize
the poses of all the motors
to achieve those joint positions and
that all happens within open skirt that
was really useful um
for
figuring out like do these components
all collide with each other have i
made joints long enough to reach the
positions they want to get to
um
that was also very good because i could
export the model
into roles but i could also export the
kinematics chain
so
i've just got a bunch of
lengths of
uh lengths of all the
uh joints of the
hinges and stuff
and i could just export those directly
into the ros simulation to generate
this
model uh the connection
the connection between the computer and
the robot is
you can take your pick so at the moment
i'm using the micro ros agent over
wireless
so i've just got udp transport over wifi
that thing is just battery powered off a
drill motor
for drill battery
um
and you can use a serial can bus i mean
you you can make an any transport
over anything as long as you implement
it as a
like as an actual ros transport
uh what sort of limits are there in
micro raws for hardware support i'm
assuming you can't do more complicated
stuff that you would do on a full linux
host eg connecting to lidars funny you
mentioned that
um
lidars produce a fairly huge amount of
data
and a lot of the time they just put it
straight onto an ethernet cable so
most of the time for a lidar you've got
uh lidar data coming
from the unit over ethernet into the
host system and then you've got a driver
on the host system
but
for vacuum cleaners
they have a little spinny lidar that
just does a 2d scan
those end up producing a fairly small
like a very modest amount of
data
so
those ones you can just
attach to a microcontroller and then
spool all of the all of the
ranges and angles out to a host system
or deal with them on the
on the local device
so
like i mentioned for map generation
versus obstacle avoidance
if you have a 2d lidar you can
you can set up little zones where if you
detect a hit within a within a zone
you know you've got an obstacle that you
need to avoid so you can have
an uh
a function on the microcontroller that
avoids the obstacle
um
like in a in a hardware real-time loop
and then you've got the host system
picking up all of those points
intercepting all that data and
generating a map and then building a
motion plan
for the robot to follow
so that way you've got
all of the hardware real-time safety
guarantees of an on-board
hardware-backed obstacle avoidance
system
with the niceties of a more complex map
generation and path planning tool
so a lot of the
communications for micro rows are not
encrypted
um i i'm not sure that embedded rtps has
an encryption profile
so
generally ros is all about
researching quickly so it's more about
rapid iteration than
uh
safe networking practices
so
dds has automatic discovery on the local
network um
you can encrypt it and
sros secure ros
basically provides an api
to
manipulate the dds security layers
all dds vendors
implement a security profile uh it's
just a mandatory aspect of like the
minimum dds spec
so
you can configure that but you end up
locking nodes into
you know they stay in one they're one
lane
um they don't read or write to
uh messages uh topics they're not
allowed to
stuff like that
uh which gets kind of in the way of
um it gets a little bit in the way of
like iterating rapidly on
new topologies for control systems
uh
oops
uh
how much time do we have
i think we're about wrapping up
hey
i'll just jump back in again
thank you very much brett
that was fascinating and the i've got a
couple of comments about what you just
had on the screen a moment ago one of
the questions i don't think um you
followed up was what is the inspiration
for this design when i first saw it i
thought it was something like a
a hexapod on its back
right now so this is
this is a reference to
an art sculpt that has a kinematic
sculpture by mark cetrakine um
i actually saw it first on
uh on adam's have just tested
and
when i looked at what they were using to
implement it
i looked up the motors they were using
and it was like 500 bucks a motor
for 15 motors
yeah
so my pockets aren't nearly that deep
but i wanted to dabble with the you know
for mark citrocaine it was a
sharpening exercise so i wanted to kind
of
pay homage to it and sharpen my own
skills by essentially replicating it
that looks amazing so in the animation
you have the sphere sitting on the five
fingers have you done that physically
i have
unfortunately these servos so
the reason
the original artist used the fancy
motors was because he can tune the
control loop of the motor
at runtime
whereas these are just you know three
dollar hobby servos
um
so i can't modify the controls the
controller within the server because
it's it's an asic it's just raw silicon
yeah
i find these are a bit squishy so if you
move them from their target position
they just apply some torque to get back
but
it
feels a bit springy
so
i had this thing rotating a salad bowl
it rotated but it was noisy and rattly
and
the
the contact points weren't actually
where i wanted them to be
yeah
okay well it's a it's an amazing demo
anyway i love the um
the visualization where you have the 3d
representation and the physical object
and they're almost synchronized it's
very impressive
yeah
um i'm just having a quick check um
a final question i think and we're just
about out of time it's almost lunch
is
do you have any examples of micro ros
running in production or commercial
systems
so there are um the microros team have a
regular
a regular meet up
online where they they present to each
other
and there's a
there's a robot chassis that they're
aiming for like
uh urban cargo delivery and they use
micro rolls to handle communication
between ros on the host box and micro
rolls on the motor controller so instead
of having some
weird driver that you have to import
dependencies for
you can push all of the dependencies out
to the motor controller and have the
motor controller interact with
the native raw system on the host
machine
so that allows them to iterate very
quickly on the nature of motion control
on their platform
okay interesting so it allows a little
bit more
modularity or separation of concerns
yeah
okay great
well thank you very much brett thank you
i
yeah i've experimented a little bit with
roz in the past and
found it a little bit impenetrable but
this is inspiring it makes me want to
look into it more
so yeah thank you very it's been a lot
of fun
that's great
and for everybody else we are now going
into the lunch break we'll be back at
about 1 20
and where i will be doing a talk about
some problems and i'm going to be trying
to do some stuff live on the bench
and
hopefully it will go wrong because
that's when it's entertaining
so thanks everybody and we'll see you
soon