CS291A – Mixed and Augmented Reality

Exercise 1: Installing and Testing XR SDKs

Change Log:
Thu., Jan 6, noon: original post
Fri, Jan. 7, 9:00am: small wording changes.
Fri, Jan. 7, 5pm: Submission questionnaire posted.

In this homework exercise, you will install and run examples in several XR-enabling SDKs. It is possible that you will run into installation problems with some of these platforms on your specific development platform. In that case, we are interested in how far you got and where the stumbling blocks are.

The deliverable on this assignment is a questionnaire submission that gathers brief experience reports for all of these. Depending on your OS and platform, there will be several additional steps that you will likely proactively need to figure out (e.g. version mismatches, or smth like registering a developer account in Xcode on Mac). Be a researcher, and try to get as far as you can, but feel free to limit your time investment on each of these SDKs to one hour each (you don't have to).

Depending on your platform choices, the instructions may turn out to be confusing. Please use Slack to ask questions, and also use your best judgment!

Let's get started with something that doesn't require an installation: web-based AR in the browser:

• We'll look at AR.js, https://github.com/AR-js-org/AR.js and AFrame https://aframe.io/.
  • A somewhat dated walkthrough (but hopefully still functional) is at https://aframe.io/blog/arjs/
  • A slightly newer tutorial is at https://levelup.gitconnected.com/simple-augmented-reality-ar-integration-with-a-frame-f625e9dc66b8
    
  
  Your task: show a model of your choice on your webcam on top of a marker (either printed or shown on a phone/tablet)
  
  

  If you want to do something more sophisticated with AFrame and have an AR-capable Android phone, check out https://medium.com/samsung-internet-dev/making-an-ar-game-with-aframe-529e03ae90cb

  [One of the underlying libraries for AR.js is ARToolkit, one of the oldest open-source AR marker libraries (we'll cover the history in class). There is a company-supported, but still open-source version of it: http://www.artoolkitx.org. The last time I offered this class, only a few people managed to get it running, so I am taking it out of this homework]

Then, please install and test the following SDKs (again, no problem if you get stuck! Limit your time to maximal 1h for each of these and report how far you got). Also, please utilize Slack and the newly set up platform_... channels in there to give and receive tips for installing any of these on the OS's you try the

• ARCore in the Android Emulator (even if you have an ARCore-approved phone, please try the emulator for this homework. There will be a question on device deployment for those of you with suitable Android phones)
• https://developers.google.com/ar/develop/java/quickstart
• https://developers.google.com/ar/develop/java/emulator or:
• https://developers.google.com/ar/develop/c/emulator
• Walk around in the emulated 3D room and place android models on detected surfaces

• Vuforia w. Unity 
• https://store.unity.com/ , Individual—>Personal (or Student, but it takes a bit of time to verify) 
• In Unity Hub Installs, select IOS, Android, or Universal Windows support
• Select your platform of choice in File->Build-Settings...
• https://developer.vuforia.com/downloads/sdk —> Add Vuforia Engine to a Unity Project or upgrade to the latest version
• https://developer.vuforia.com/downloads/samples --> Download from Unity Asset Store 
  
  
• There are plenty of videos on YouTube to walk you through Vuforia/Unity installation and a first demo project deployment. Some require an AR-capable smart phone, but others only a webcam. Two older example YouTube tutorials to get started with Vuforia are here (caveat: they may use older versions of the software than you):
• https://www.youtube.com/watch?v=9XikHnTiukk&list=PLX2vGYjWbI0Thl0pOCbKWrbbiw7RWiRG7&index=1
• https://www.youtube.com/watch?v=MtiUx_szKbI&t=250s          
  

• Wikitude SDK
  • Pick one platform of choice for this assignment that interests you most: Unity, Windows Native, iOS, Android, or one of several Cross-Platform SDKs (also requires a mobile device). Free version for 45 days of testing only (fine for us for now).
  Compile and run an example of your choice.

 

• Pick one more AR platform of your choice to test. A few to check out are MaxST, XZIMG, or KudanAR. There is also a long list of Javascript-based AR libraries here, for example.
  If you have an XR device that you would like to program with for the quarter and a library that hasn't been covered yet with the above options, you can go for that (e.g. ARKit with RealityKit/SceneKit on an iPhone or iPad or Meta Quest 2, or HoloLens-2). Just be aware that you can easily spend much more than an hour setting up a thorough development platform for any specific device. A potentially simple pathway if you have access to an AR/VR device (HoloLens, Meta/Oculus, Vive...) is to use WebXR or ARgon.js in a browser on the device!
  • Pick the platform that interests you most and demonstrate a simple first HelloAR app!

 

• Finally, please install OpenCV (no requirement for AR yet. We will come back to OpenCV, so your time setting this up correctly will be well-spent.)
• https://opencv.org/releases/
• For platforms other than Windows, maybe this is of help: https://www.pyimagesearch.com/opencv-tutorials-resources-guides/
• Compile and run the examples in the ‘sample' subdirectory

 

The questionnaire for this assignment (that's how you submit the assignment and screen shots as evidence of your efforts) is here: https://bit.ly/xr-hw1-q

Submission

Your questionnaire must be submitted by Tue., Jan 11, 12:00pm (we will discuss results at class time)