Projects

Mobile Imaging - Winter Quarter 2014


Course Project - Due March 18th

This is a hands-on programming project. You are to work in teams of two or three people - no exceptions. You can use the course mailing list to find partners if you're not sure who to work with.

For this project, you are to choose from the project choices below or propose a topic of your own. You may use any mobile programming environment you wish (Android, iOS, Windows Phone, etc.). By March 18th, you are to turn in the code and demonstrate the working program to the instructor.

You are not expected to solve any of these problems completely and provide an "app store ready" application. You are expected, however, to implement something substantial that makes sense and demonstrates significant thought, planning, and effort. Brainstorming and regular group meetings are highly encouraged, as is getting started early. If the project looks like it was thrown together in the last week, the grade will suffer.

Note: Working with others will be important throughout your career. The group sinks or swims together - "I couldn't get in touch with him," "she didn't finish her part until yesterday," and such are not valid excuses for not getting the project done well. Make sure early on that you can work together effectively.

The project proposal is due on February 6th. This is just a short email that includes:

We'll discuss details in class with each group, such as the development language/environment you plan to use, your key resources (will you read journal articles or conference papers? make use of existing code?), etc.

Here is a list of projects to choose from:

Project #1 - 3D scene models from images

Build an application that, for a given image of a scene (e.g., the interior or exterior of a building), allows a user to create an approximate 3D model of that scene and then manipulate the model in 3D (e.g., allow the user to change viewpoint, zoom in and out, etc.). An old example of this from multiple images is at http://xenia.media.mit.edu/~sbeck. A journal article from 2000 that was influential in this area is at http://www.cs.illinois.edu/~dhoiem/courses/vision_spring10/sources/criminisi00.pdf, and a relevant 2003 paper is at http://iris.usc.edu/outlines/papers/2003/hlk_Lee_Nevatia.pdf.

The scene in your project could be fairly simple, like a hallway with a floor, ceiling, and walls. Or an image of some cardboard boxes, or some things with simple shapes like that.

The model you view in 3D could just be a wireframe of the structure, but even nicer would be a textured version (with texture from the original image). This is easiest if you specify the scene model to be piecewise planar (consisting of various planes in the 3D space), or perhaps mixing planes with other simple shapes (parts of cylinders, spheres, etc.). Onto each plane (or shape segment) can be texture mapped the pixels from the original image.

Project #2 - 3D object modeling

Create an app that will build a 3D model of an object from multiple images (or video) taken by a user. The imaging scenario can include known markers (e.g., on a table) to help with registration.

Project #3 - Image mosaicing

Create an app that stitches together N images of a scene to form one larger image – i.e., a panoramic or mosaic image. The new single image should minimize (or eliminate) artifacts like ghosting. Here is an example: http://www.slrobertson.com/article-stitched-mosaics.htm. As part of the project, discuss the theoretical limitations (under what conditions - e.g., scene geometry, camera motions, lighting - should the technique work or not work?) and show real examples that corroborate this (both positive and negative examples).

Project #4 - Image retargeting

Build an application that resizes an image for a different aspect ratio ("retargets" it) that takes into account image content - keeping the important elements of a scene (with proper shapes and aspect ratio) intact. For example, see:

Seam Carving for Content-Aware Image Resizing

A Survey of Retargeting Techniques

Project #5 - High dynamic range imaging

Create an application that produces a high dynamic range (HDR) image from a set of input images, for display on the mobile device.

Project #6 - Gesture control

Image the user's hand to provide gestural control for a game, map application, etc.

Project #7 - Video journal

Summarize your day by capturing key images from your always-on (e.g., Google Glass) device. The key is determining when to capture images and which images to keep in a "summary of my day" presentation.

Project #8 - Image quality advisor

Build an app that gives a user automatic and useful information about how to capture better quality images in the current imaging environment.

Project #9 - Capture on smile

The app takes a picture when the person in front of the camera smiles (or makes a particular gesture, etc.).

Project #10 - Poor man's Kinect

Use two devices together to determing the 3D location of one or more points - e.g., someone's fingertip - to enable 3D user interaction.

Project #11 - Lip tracking for speechreading

Use the front camera on a device to track a user's lips and produce information useful for "speechreading" - i.e., using visual information about the mouth area to help in speech recognition.

Project #12 - Document rectification

When taking a picture of a document (using your device as a scanner), automatically improve the geometry and photometry so the document is easier to read than the original.

Project #13 - Social image sharing

Create an application that keeps track of your friends' locations so that when you take a picture that is likely to have your friend in it, the image gets sent to your friend automatically.

Project #0 - Roll your own

Come up with a project of your own, a specific mobile imaging-related problem that you can address and implement. Must be approved by the instructor.

 

Project Reports and Presentations

Here's what is due on March 18th: