t’s been quite a year since I posted my last curriculum on getting started with Python! So much has happened (like a whole BOOK DEAL - what?!) and it brings me with great joy to say that my interests have evolved and I’m now pursuing a new track in tech - Spatial Computing!
Thanks to a product demo from Microsoft (hey - that’s my company!), I basically became interested in this technology overnight. I was literally sitting in an airport on my way from La Guardia to LAX when this new venture blossomed.
I’ve since learned the fundamentals of C# (thanks, Codecademy!) and attended a workshop with the awesome folks over at Magic Leap! I’ve met up with some of the spatial computing and HoloLens folks at Microsoft + have also had a chance to meet some people online and IRL that are doing awesome stuff in this space.
I also have had a chance to get my hands on a couple of devices (read: toys) and I’m dying to try them all out! However, if you know me, I like to approach learning methodically. So, I put together an entire curriculum for myself to get up to speed with all things virtual reality, augmented reality, mixed reality and spatial computing.
Are the resources below an exhaustive list of every resource on this planet? No! However, I feel that the resources below will help me achieve general understanding so that I can create with my Magic Leap One, HoloLens 1 (here’s the link for HoloLens 2) and Google VR devices.
I’m following the stuff below in order + will be tweeting my heart out about it as I learn.
What’s a curriculum without books, am I right? So, I’m reading the first book from cover to cover to get a better understanding of just what the heck all this tech is and where it came from. For me, it’s better for me to know the past in order to make sense of the now and the future. Not to mention, this technology is full of a bunch of words and theories I’d never heard of before. Striving to read a chapter a day! This book was suggested by Thomas over at Microsoft - he’s basically my hero.
As for the Unity book - this was gifted to me by Tricia (thanks, Tricia! you’re so sweet) I never used Unity until I became interested in this space. I’m using this book more as a reference as I get caught up or stuck working in Unity.
Creating Augmented & Virtual Realities: Theory & Practice for Next-Generation Spatial Computing
Authors: Erin Pangilinan, Steve Lukas (SO helpful), and Vasanth Mohan
Unity Game Development in 24Hours
I personally learned C# on Codecademy, however, I later refreshed that knowledge by watching the C# 101 video series on Channel 9. The great thing about the video series is that Scott and Kendra follows along with the docs that we have over on Docs!
- Codecademy Learn C# - https://www.codecademy.com/learn/learn-c-sharp
- C# 101 Video Series - https://channel9.msdn.com/Series/CSharp-101
Getting Started with Unity
Unity has a lot of content that I've found to be helpful given that I was completely new to using Unity. I'd suggest completing the courses/project in the order below. However, feel free to skip ahead if you already have proficiency in one area over another.
Getting Started with Unity is pretty lengthy, however, the first two sections (Create Your First Unity Project and Beginner In-Editor Tutorials) will walk you through the Unity interface. I HIGHLY recommend completing those two sections at a minimum - especially if you haven't used the Unity interface.
Beginner Scripting gives a pretty good overview of how to write scripts for whatever you create in Unity. There's also a decent walk-through of how to read Unity's reference documentation.
Create with Code provides a rather thorough walk-through of how to create a game in Unity. Consider saving some of the scripts that Carl walks through in the videos to use for your own projects.
- Getting Started with Unity - https://learn.unity.com/course/getting-started-with-unity
- Beginner Scripting - https://learn.unity.com/course/beginner-scripting
- Create with Code - https://learn.unity.com/course/create-with-code
Introduction to VR
Even though I’m most interested in spatial computing, VR is starting to look real interesting! So, I’m dabbling into that area first. I’m a video learning type of gal and fortunately Udacity has videos in their catalog related to virtual reality! I even added a physics course because physics wasn’t something I really did in school. I did take a light, speed and digital phenomena course in undergrad but I’d be lying if I said I remembered anything beyond how to make a pin-hole camera.
- Introduction to Virtual Reality - https://www.udacity.com/course/introduction-to-virtual-reality--ud1012
- VR Scenes and Objects - https://www.udacity.com/course/vr-scenes-and-objects--ud1013
- Intro to Physics - https://www.udacity.com/course/intro-to-physics--ph100
- VR Software Development - https://www.udacity.com/course/vr-software-development--ud1014
Took me long enough, however, I went on ahead and purchased one of those Google Cardboard devices that works for smartphones! However, I couldn’t just get a plain one. I got this alligator red embossed one from Powis VR. Since I’m starting out on the VR train, I figured I should go ahead and learn how to create for this device.
- Quickstart for Google VR SDK for iOS - https://developers.google.com/vr/develop/ios/get-started
- Building a Mobile VR App with Google VR - https://www.arvracademy.io/mobilevr
The super friendly and passionate folks over in the land of fun (aka HoloLens) at Microsoft was kind to loan me a HoloLens 1 device! So, I’ll be heading over to the Mixed Reality Academy over on Microsoft Docs for what looks to be a couple of weeks of getting acclimated with the device. Maybe sometime in 2020 I’ll have enough pennies saved for a HoloLens 2! Jesse (another super awesome person) on the HoloLens team let me test drive the HoloLens 2 - WOAH! Omg! There’s so many features that I’m missing out on but I think it’s totally worth the investment. I also had a chance to try out VR with Aysegul (probably one of the most brilliant people I know in the spatial computing world) during my visit to Microsoft SF. All I have to say is…I WANNA CREATE!
- MR Basics 100: Getting Started with Unity - https://docs.microsoft.com/en-us/windows/mixed-reality/holograms-100
- MR Basics 101: Complete project with device - https://docs.microsoft.com/en-us/windows/mixed-reality/holograms-101
- MR Basics 101E: Complete project w/ emulator - https://docs.microsoft.com/en-us/windows/mixed-reality/holograms-101e
- MR Input 210: Gaze - https://docs.microsoft.com/en-us/windows/mixed-reality/holograms-210
- MR Input 211: Gesture - https://docs.microsoft.com/en-us/windows/mixed-reality/holograms-211
- MR Input 212: Voice - https://docs.microsoft.com/en-us/windows/mixed-reality/holograms-212
- MR Input 213: Motion Controllers - https://docs.microsoft.com/en-us/windows/mixed-reality/mixed-reality-213
- MR Spatial 220: Spatial Sound - https://docs.microsoft.com/en-us/windows/mixed-reality/holograms-220
- MR Spatial 230: Spatial Mapping - https://docs.microsoft.com/en-us/windows/mixed-reality/holograms-230
- Periodic Table of Elements - https://docs.microsoft.com/en-us/windows/mixed-reality/periodic-table-of-the-elements
HoloLens w/ Azure
So we have a bunch of pretty cool cognitive services stuff that can integrate with HoloLens. I’ve only used some of these features on simple apps that I’ve built - but never in a device. Interested to see what I can do - especially with…bots!
- MR and Azure 301: Language Translation - https://docs.microsoft.com/en-us/windows/mixed-reality/mr-azure-301
- MR and Azure 302: Computer vision - https://docs.microsoft.com/en-us/windows/mixed-reality/mr-azure-302
- MR and Azure 303: Natural Language Understanding - https://docs.microsoft.com/en-us/windows/mixed-reality/mr-azure-303
- MR and Azure 304: Face Recognition - https://docs.microsoft.com/en-us/windows/mixed-reality/mr-azure-304
- MR and Azure 310: Object detection - https://docs.microsoft.com/en-us/windows/mixed-reality/mr-azure-310
- MR and Azure 312: Bot Integration - https://docs.microsoft.com/en-us/windows/mixed-reality/mr-azure-312
The Magic Leap One was the first device that I actually tried out when I was first introduced to spatial computing. At the time of writing up this curriculum, this device is probably the only one that I can honestly say that I know how to build for - which is saying a lot because whew, I was lost in the beginning! However, the folks over at Magic Leap have a lot of resources available on their site to help you get started! Lately, I’ve been playing with apps from the community but it’s time that I build something for myself (maybe something that I could one day publish?).
- Getting Started with Unity - https://creator.magicleap.com/learn/guides/get-started-developing-in-unity
- Unity Tutorials
- Hello, Cube! - https://creator.magicleap.com/learn/guides/gsg-create-your-first-unity-app
- Introduction to Meshing - https://creator.magicleap.com/learn/tutorials/introduction-to-meshing-unity-r-edition
- Eye Tracking - https://creator.magicleap.com/learn/guides/eye-tracking-tutorial-unity
- 6 DOF - https://creator.magicleap.com/learn/guides/control-6dof-tutorial-unity
- Hand Pose - https://creator.magicleap.com/learn/guides/gestures-in-unity
- Hand Tracking Key Points - https://creator.magicleap.com/learn/guides/hand-tracking-key-points-unity
- Unity API Samples
- Hand Tracking - https://creator.magicleap.com/learn/guides/sdk-example-gesture-input
- Image Tracking - https://creator.magicleap.com/learn/guides/sdk-example-image-tracking
- Meshing - https://creator.magicleap.com/learn/guides/sdk-example-meshing
- Magic Kit - https://creator.magicleap.com/learn/guides/magickit-overview
Don’t know much about this but I figured “Sure, why not! You have some iOS devices!”.
- Learn ARKit Using Unity - https://www.udacity.com/course/learn-arkit-using-unity--ud114
Ayesgul usually talks about Babylon.js and because I trust her, I’m going to tackle this one as well.
- Babylon101 - https://doc.babylonjs.com/babylon101/
There’s SO much to learn! As I mentioned, I’ll be sharing everything on Twitter so feel free to connect with me there!