The Power of Swift for Machine Learning (TensorFlow Meets)
Articles,  Blog

The Power of Swift for Machine Learning (TensorFlow Meets)


♪ (intro) ♪ Hi everybody, and welcome to this episode of TensorFlow Meets. I’m absolutely delighted
to be chatting with Chris Lattner. And Chris, I hear you’re the inventor of this language
that everybody’s talking about, Swift. Yeah, it’s something
I’ve been working on for some time. I guess my background is
I’ve been working on compilers and tools for quite some time; worked on this LLVM compiler for a while. Swift is an awesome new language. It’s got some really cool geeky
language stuff on the side of it, but the thing I love about it,
it’s designed to be easy to use. Yeah, and that seems to be
a lot of the passion around it, is that even for new developers
who are coming in that it’s all of this easy-to-use stuff. Yeah, exactly. The real goal here is to bring
usability first and foremost, and this takes a lot of hard
system engineering to make things feel easy, but it’s absolutely worth it and it’s a great challenge. So when it comes to now
using Swift for TensorFlow, it seems like…
I’ve started playing with it and kicking the tires a little bit, and I saw things like even Keras layers
are pretty straightforward. Could you tell us a little bit
about the thinking behind how you designed the API? Yeah, absolutely. So Swift as a language
has a lot of similarities to Python, and so wherever possible, we’re trying
to make the APIs feel the same because we want people to be able
to learn one set of technologies and move back and forth
without big road blocks. But on the other hand,
Swift as a language has new capabilities and advantages that Python doesn’t. It’s just a lot newer and so we want to be able
to take advantage of that and built things out
so it can be familiar, yet powerful. I see, okay, cool. So one thing that may not be
immediately apparent to developers or immediately understandable
is this concept you spoke a lot about, differentiable code. Ah, yeah. It sounds like a really powerful thing but can you help us understand
a little bit more what’s it all about? Sure, absolutely. So this is a big “differentiator”
for Swift for TensorFlow. I hear it’s integral to the product. Yeah, it’s integral to everything we do. So if you think about
machine learning models, when you define your forward function, you’re defining and composing
your model in this way, but then you need to train it. So when you train it,
what you end up wanting to do is compute the gradient of the values
as they flow through your model and how they contribute to your loss. So the way that works is through calculus, and calculus has this underlying principle
called the chain rule. Chain rule is something
that’s been known for a really long time and so what differentiable programming is doing is it’s automatically
computing this for you in the language. And the cool thing about this
is it makes it super extensible, so you can do new kinds of things and you can experiment and research
new kinds of concepts. Or, if you don’t want to worry about
that level of thing, you can just build on top of
somebody else’s libraries. Right. So things like optimizers
like stochastic gradient descent– (Chris) Yes, that’s exactly how they work. So then it’s a case of–
what I really like and what I find interesting
in this is that instead of just trusting those libraries
it gives you the tools to be able to– because your code is differentiable– to be able to build your own
or to at least understand what’s going on. Yeah, exactly.
Again this comes back to the principle of making it so that everything
is an open box. And so you can look in
and you can get around, you can customize and tweak and change and everything is right there
for you to play with. Right, so it’s one of the things
I really like about Swift, for TensorFlow in particular, is that
you can come in right at the top and just build your layers
in Keras and train. Maybe that’s all you’ll ever want to do, but if you really need to kick the tires
and see what’s going on underneath. Even people like me
who have forgotten more calculus – than they’ve probably ever learned.
– Me too. I agree. But if I do brush up on my calculus
and I really want to tweak and optimize and that kind of stuff,
then it’s all there for me. Yeah, that’s the idea of this–
“no boundaries.” You can go wherever
your inspiration takes you. The thing I found about researchers is they don’t want to have
artificial boundaries. They don’t want to say:
“I can do this much in Python and then I have to switch
to C++ to do more.” If you can make it continuous,
you can make it so that– By being a seamless experience you can enable more things
to practically happen, just because it’s more natural and easy and you’re removing those barriers. And you can trust
your debugging more, right? Because you’re not thinking about maybe
something changed crossing the barrier. Yeah, you don’t have
to switch debuggers, in some cases. Exactly. Cool. Now one of the things– maybe it’s related to that– but one of the things
that also impressed me when you showed in your talk
was really interoperability– that I can just pull Python stuff in,
or I can pull other stuff in. Could you tell us
a little more about that? Sure, so Swift works
very naturally with C, and the way it does that
is that it pulls in the client compiler directly into Swift and it interoperates
at a very low level of the compiler with this. Python, on the other hand,
is a super dynamic language. And so Swift has now… We’ve implemented
new dynamic language features in Swift to allow it to directly talk
to the Python runtime. And so when you’re using Python and Swift, you’re not using wrappers
or some weird Python-esque thing. You’re using real Python right in Swift. And one of the things that I love
about that is it gives you perhaps the world’s best
progressive typing system for Python also because you can use a Python dictionary, or you can use a Swift dictionary
of Python objects, or you can use a Swift dictionary
of Swift strings and Python objects, and you can choose whatever level
of Python that you want. It’s really natural
and it just composes properly. Super cool. Now a lot of people of course
will know Swift from it being for iOS development. (Chris) Yes. Of course, it goes beyond
iOS development now with TensorFlow. Could you give us some guidance
on where’s the best place to get started? Yeah, so Swift
is a cross-platform language and a lot of iOS developers
use it for sure, but it’s also been very popular
on the server, for example. So a lot of people have been building
Linux servers and things like that which it’s really great for. If you want to get started,
the easiest place to go is to go toGitHub.com/TensorFlow/Swiftand we have a nice landing page
and we have all the information, you can join our community, and there’s tons of information there
they can get [going]. One of the really cool things
is that all the demos we showed are available in Colab,
and so you can use it on any device– a Chromebook…
Anything that you have, it just works. Cool. And I know you’ve been working
on a training course with the folks from fast.AI, so if we want to be trained in Swift,
we can go there too. Absolutely. We’re just
getting that of the ground. I’m really excited about that. A little bit terrified
about that as well. But I’m sure it will be really great and I’m looking forward
to working with Jeremy Howard. I’m sure. I met him this morning,
and he’s awesome, so… – He’s so passionate.
– Yes, definitely. So thank you so much, Chris.
As always, this was amazing. As always, it was inspirational, so… And thanks everybody for watching
this episode of TensorFlow Meets. If you have any questions for me,
if you have any questions for Chris, please leave them in the comments below. We’ll also put links to everything
that we spoke about in this show in the comments below
so that you can follow them from there. – Sounds great.
– Thanks again, Chris. ♪ (outro) ♪

13 Comments

  • Aaron Hilton

    Can't wait to see what the research community can do with swift for tensorflow. Seems very liberating, and Swift itself is a great language. Thanks for all your amazing contributions to truly advancing the computer sciences. I don't think most people really appreciate just how fundamentally this changes the whole world. Not just for machine learning.

  • John Wayne

    I thought that auto differentiation is in Swift master for a while, given the Swift for Tensorflow was announced more than a year ago.
    Was very surprised to find out that it's in an feature branch.
    I wonder if it's planned for AD to be merged into Swift master, and added to the standard, or if it will always be a fork.

  • Michael Mantion

    is swift designed for mac users or something? there are no good tutorials for it, everyone who does a tutorial is on a mac. So weird.

  • Alexito's World

    The best thing for me is how the TF team is working closely with the Swift core team and community to include all these cool improvements into Swift itself so the general public can benefit from it! Thanks for that!

Leave a Reply

Your email address will not be published. Required fields are marked *