Manifest Tech Blog

What's all this about "augmented reality" for mobile apps? Well, Virtual Reality (VR) has to do with being immersed in a synthetic 3D graphics environment, i.e., experienced in a head-mounted display -- whether for action games, or for serious research like exploring the virtual surface of Mars or experiencing chemical bonds at molecular scales.

Then Augmented Reality (AR) combines graphics overlays on top of (i.e., augmenting) the real world, as in a heads-up display or the familiar yellow first down line for football games.

On a smartphone, the device already can display location-based information about near-by businesses and points of interest. And it has a camera that can display live video on the display. So combine these and you get AR -- hold up the phone and look around to see the local scene as live video, with text and graphics overlays identifying items in that direction.

This idea is already implemented in mobile apps including Google Goggles (for Android) and Yelp (iPhone 3GS only).

Or try Wikitude (shown here), which adds geo-tagged user-created content including Flickr photos, YouTube videos, and Wikipedia info (iPhone, Android, Symbian), and Layar, which imports user-created or partner content "layers" for customized tours of local areas (iPhone, Android).

However, these apps still are limited by the handheld mobile platform. It's awkward to hold up a device and look around with it, especially when the small screen can only show a rather limited field of view within the surrounding scene. In addition, the inexpensive sensors in handheld devices can have significant errors. As a result, current mobile AR apps cannot precisely register information to the background scene, and instead typically indicate objects of interest in a general direction, and show their relative positioning.

So augmented reality is just beginning on smartphones. Still, these devices are good enough to do real-time GPS navigation, for example, and support AR-style apps that display information about upcoming traffic cameras, or help you retrace your route to where you parked your car.

Maybe it's time to put on wireless eyewear to go with our Bluetooth headsets.

See my full article - Augmented Reality Goes Mobile - for more on the development of mobile AR, from early research prototypes to desktop AR to mobile devices

See my Smartphone Apps Gallery for more on apps and app stores.

Cameras are for more than taking pictures, and smartphones are more than communications devices. Combine the two, and you have a handheld device that can "look" at objects and tell you about them.

For example, the Shop Savvy app (free for iPhone, Android, and Nokia) and similar apps can scan a barcode and return information about that product, with web links and reviews.

But why require barcodes? The new Google Shopper app for Android offers three forms of product search: plain old text search, voice search (say the product name), and image search.

With image search, Shopper continuously scans the image from the camera, looking for a recognizable product -- not only barcodes, but also cover art from books, CDs, DVDs, and video games.

In seconds, it identifies the object and returns information including prices, reviews, detailed specs, and links to sellers. You also can save history and favorites to access even if the phone is offline.

And Shopper does not even need to see the entire cover -- I was able to use it to quickly recognize books from just the top of the cover, and CDs from just the artist's face.

Shopper packages the product search features demonstrated in the earlier Google Goggles app (also free for Android), which does more general visual search. It identifies products, also including wine labels, as well as generally recognizable objects including landmark structures (buildings and bridges) and famous artworks.

Plus, Goggles does text recognition for business cards to extract contact information, and Google has demonstrated a new version that does more general text recognition and transition, for example for highway signs. While Shopper just constantly looks for something recognizable, Goggles waits until you take a picture to start the search.

These are amazing -- and useful -- apps, harnessing the Google image search and product search engines and databases in the cloud to turn your phone into an almost real-time recognizer. And this is just the beginning -- Google is not terribly specific about exactly what kind of stuff these apps recognize. For example, it turns out that Goggles also does general image search to find matching images on the web, whether or not they are further identified as known products, landmarks, or artworks.

So keep a look out for further updates -- There are plenty of other objects around us that would be useful to recognize, including faces. Hmmm ...

See my Smartphone Apps Gallery for more on mobile apps

See my article Augmented Reality Goes Mobile - for more on AR apps, including Google Goggles

I've been really spoiled lately by using the Google Android search by voice on the Verizon Droid phone -- and not just for doing translations (see previous post). When looking up crossword clues, for example, it's so much easier to speak a phrase instead of typing tricky words on the virtual keyboard -- although the Droid did have a problem today understanding "Cy Young award" (it heard "scion").

But even occasional misunderstandings aren't a big deal, with the impressive ability of Google search to figure out what you probably meant, even if you couldn't express it fully. This kind of flexibility is really helpful for searching, since you can get to the right results even if you misspell some words, or don't have all the right words. But what if you don't even have the right letters to make up a word?

The new Google Gesture Search app takes this idea to the next level. The idea is that you draw letters on the touch screen, and it instantly displays matching items on your phone -- contacts, bookmarks, apps, and music. Just enter a couple of letters to display any names that contain that sequence.

But how can this work? Can you really depend on each letter being correctly recognized? It doesn't matter with Gesture Search -- it just searches on multiple possible letters for each of your gestures, and shows all possible matches.

As a result, you can write quite loosely. You can enter upper or lower case. You can draw the full character with multiple strokes (like the cross bar on "A" or "F") -- but why bother? Just keep entering more loose letter gestures, and the list of matches will reduce down to what you are looking for. (Of course, this works so well because Gesture Search is working with a known list of items on your phone -- this would be less useful for doing general text searches of the web.)

This is another fun Android application from Google Labs, experimenting with new ways to use our mobile devices (see previous posts on Google Shopper and Google Goggles). Gesture Search is currently English only, and has options to select the items to search (i.e., omit the music library), to regenerate the search index, and to send gesture data to Google to help improve the application.

See my Smartphone Apps Gallery for more on mobile apps

Find the Verizon Droid from Motorola on Amazon.com

I've you've been interested in trying out a smartphone, but have been put off by the idea of paying $200 for an iPhone, Verizon Wireless has just started a new promotion for you -- the Palm Pixi Plus is now only $29.99 (originally $99), and you get a second one for free!

That's a full-fledged smartphone, with Web browsing, e-mail, camera and media playback, and downloadable apps to customize your experience and connect to your social networks (see earlier post). And, oh yeah, it's also a mobile phone and an organizer, with syncing of contacts and calendars to the desktop and online. The Pixi design is relatively small and light, with a built-in keyboard to help you keep communicating.

And if you want a little more power, the Palm Pre Plus from Verizon is now only $49.99 (originally $149), with a larger screen, double the memory, and higher-res camera, in a slide-out keyboard design. And you get a second one for free.

Even better, these both support 3G Mobile Hotspot -- which turns your smartphone into a local Wi-Fi hotspot for up to five devices. And Verizon now offers this service with up to 5 GB a month for free (was $40 for 5 GB and 5 cents per megabyte overage).

So, for example, the Palm Pixi Plus can act like the Verizon / Novatel MiFi (see earlier post), connecting to the cellular broadband service wherever you are, so your laptop or other portable devices (like an iPod touch) can quickly go online using their built-in Wi-Fi networking.

Some gory details: These prices are with a two-year activation. All Verizon smartphones require at least a $29.99 monthly data plan (which includes unlimited personal e-mail and Web). However, while any Wi-Fi devices are connected to the phone all data traffic is billed to the Mobile Hotspot plan (and not the smartphone data plan). The 3G Mobile Hotspot application itself is a free download, and supports both open and protected Wi-Fi service.

Also note that Palm Pre and Pixi come in two flavors, the original versions sold through Sprint, and the newer Plus versions sold through Verizon, with the mobile hotspot feature. The Pixi Plus also added Wi-Fi and the Pre Plus doubled the memory.

And shop do around: Walmart is offering big discounts on smartphones -- the Palm Pre Plus is free with online discount, and the Pixi Plus is not only free, you actually get paid $50 from a mail-in rebate.

The bottom line is that you can get a nice smartphone (or two!) for under $30, and also have it serve as a free mobile hotspot. Of course, while the 5 GB limit is great for checking e-mail and basic Web browsing, don't plan to be streaming movie videos to your laptop over Wi-Fi.

See my full article on the design of these new Palm smartphones: Palm Pre: Product Summary

See my Mobile Communications Gallery for more on the Palm and other smartphones.

Find the Palm Pixi Plus and Palm Pre Plus on Amazon.com.

I've been having a lot of fun this year giving talks and demos about smartphone apps, especially on the Apple iPhone and Google Android.

The trick for demoing smartphones, of course, is to find a good way to let the audience see what you're doing on the small screen. You can use a webcam display from a laptop, but it's tricky trying to hold up the camera to the phone, and you can get bad reflections and glare on the phone's screen.

My answer is to set up a simple rig with the Joby Gorillapod SLR -- a highly flexible mini-tripod that you can bend to fit to any surface, from a table to a slanted podium (see earlier post).

The Gorillapod SLR model is big enough that you can position the camera about 8 inches off the surface, and then spread out the legs to give enough clearance above the phone and to the sides so that your fingers can work comfortably on the touch screen. You also can tweak the angle of the tripod so that the phone image is squared up, without perspective distortion. The Gorillapod SLR is priced at around $39.

The webcam that I'm using is the Logitech Webcam C905 (formerly the QuickCam Pro for Notebooks) -- a small design around 2 x 1 inches for easy carrying (see earlier post). The webcam has a clip that holds securely to the base plate of the Gorillapod -- so there's no screws or other assembly required. It's also helpful to add a USB extension cable to allow you to position the tripod further away from your laptop if needed to fit a particular facility. The Logitech webcam is priced at $99, or around $75 street.

I typically run the QuickCam at 1280x1024 resolution, which clearly shows the details on the smartphone screen. Even with video flowing at this higher resolution, there's no obvious degradation of the system performance oven on an older lower-end laptop.

When showing demos, the phone fills almost the full height of the video window. The Logitech software also can apply a small digital zoom if desired to enlarge the size of the phone's screen. Since I'm not running full-screen, it then just takes a click on the laptop to switch back and forth between showing the live camera feed and other presentation material.

The result is quite good, even when you're trying to shoot a bright screen in a dark room, or with glare from overhead lighting. I've found it best to override the auto settings and manually adjust the focus and brightness / contrast to pull out the detail on the phone's screen. It also helps to place a plain background under the phone (even a white piece of paper).

One other issue with shooting phones is reflections and glare on the screen. I have experimented with polarizing filters to knock down some of the glare. You can get packs of 2 x 2 inch polarizing film from companies like Edmund Optics, cut it to fit over the camera lens, and simply tape it in place. However, these are not going to make serious reflections magically disappear. Instead, I've been able to adjust the tripod setup to control the lighting issues, sometimes aided by turning off particularly troublesome podium lighting.

The result is an easy to carry set-up that also simple to set up, and has produced good results in a variety of venues. The trick is to spend some time to optimize the set-up for the particular facility before the talk starts -- and be sure to check that the organizers aren't planning to change the lighting as you get underway.

See my Digital Photo Cameras Gallery for more on the Gorillapod and other tripod accessories

See my Home Networked Media Gallery for more on the Logitech Webcam C905 and other webcams

Find the Joby Gorillapod SLR and
Logitech Webcam C905 on Amazon.com.

The Android 2.1 software upgrade that originally shipped with the Google Nexus One phones is now rolling out on the Verizon Wireless Droid by Motorola.

This finally adds Pinch to Zoom for web browsing and other apps including Gallery and Google Maps, so you can just pinch or spread your finger and thumb to zoom smaller or larger (yes, just like the iPhone).

But the bigger addition is the expansion of Speech to Text -- all text entry is now voice enabled, for any app. The virtual keyboard has a new microphone icon next to the space bar. Tap and just speak your search query or sentences to write (in English), and Android enters the text field for you.

This is not perfect, but it's typically plenty good enough for searching, and perhaps adequate for quick notes -- and a lot faster than typing on a small keyboard (see earlier post on Gesture Search).

The 2.1 upgrade also includes a new version of the Gallery app that now browses media stored online at Picasa, with a "3D" view for browsing photos and videos in stacks. There's also an enhanced Music app with new navigation tabs, and a new News and Weather app and widget customized for your location.

Other nice improvements include Live Wallpapers for animated home screens and Night mode to automatically dim the screen.

Also -- Google Earth now runs on the Droid, as well as the Nexus One and iPhone. You can fly around the globe and then zoom in on a location, now also with search by voice, plus local information and a roads layer as in Google Maps.

Your phone will automatically notify you when a system update is available, or you can check using Settings, and selecting About phone (at the bottom of the list), and then System updates. This update is identified under About phone as Firmware version 2.1-update1.

See my full article, Verizon Droid from Motorola: Android 2.0, for more on the Droid's design and features and technical specifications.

See my Smartphone Apps Gallery for more on these and other mobile apps.

See my Mobile Communications Gallery for more on smartphones.

Find the Verizon Droid from Motorola on Amazon.com

Update details ...

The not so modestly named Verizon Wireless Droid Incredible from HTC actually lives up to its name. This slender device is wicked fast, packs some nice hardware upgrades, and features the highly-customizable HTC Sense interface.

The Incredible is the latest incarnation of the Google Android smartphone platform, designed as a close cousin to the Google Nexus One (both are thinner and lighter without the slide-out keyboard of the Motorola Droid) and now available on the Verizon Wireless network

(Google had originally planned to bring out the Nexus One on Verizon, but now recommends that customers should get the Droid Incredible instead.)

UPDATED: The Incredible has a faster 1 GHz Snapdragon processor than the Droid's Arm 550 mHz, and you feel the difference. For example, when downloading multiple apps in the Google Marketplace the Droid will hesitate for multiple seconds when bringing up confirmation screens, but the Incredible just keeps on responding.

The Incredible also steps up the hardware features from the Droid and the Nexus One in several interesting ways, with an 8 MP camera (from 5 MP), built-in 8 GB internal memory for storing documents and media (in addition to the removable MicroSD card), and a FM radio.

The HTC Sense interface extends the standard Android Home page with five panels and a "Leap" feature to show thumbnails of all five panels to jump quickly between them. The interface also offers plethora of large widgets for direct access to your e-mail, text messages, and other favorite information on one of the Home screen panels -- although the full app for each is also just a tap away with the same responsiveness as flicking to a widget.

The HTC Sense interface also extends many of the core Android apps -- Browser, Mail, People, Calendar, etc. -- to add additional features, especially for a more integrated view of your information. For example, the People app combines your contact information with associated communications, including email, updates, texts, tweets, and the Photo Gallery app combines photos on the phone with online Picasa Web albums.

These apps also need to be modified for a second reason -- to support the internal 8 GB memory, which is not found by standard Android apps, but is a nice place to stash media files and saved documents that you can continue to access even when you swap the removable MicroSD card.

The Droid Incredible is priced at $199.99 from Verizon after $100 mail-in rebate (as a debit card), with a new 2-year agreement. It requires a Nationwide Talk plan (from $39.99/month), and an Email and Web for Smartphone plan (from $29.99 for unlimited monthly access). Verizon is also offering a free 2 GB memory card for purchases before May 31, 2010.

See my full article on the Incredible - Droid Incredible from Verizon Wireless / HTC

See earlier article on the Android interface - Verizon / Motorola Droid -- Android 2.0

See my Mobile Communications Gallery for more on smartphones

Find the Verizon Droid Incredible on Amazon.com.

How much can you do with a camera on a smartphone? The folks at Google Labs are determined to keep opening the aperture by pouring technology into their Google Goggles app to do all kinds of visual searches (see earlier post on Google Shopper and Google Goggles).

With the new Google Goggles version 1.1 released this week, you now can scan photos of text in five languages (English, French, Italian, German, Spanish), copy the text, and/or translate it to these and twelve other languages.

Goggles already recognized all kinds of objects -- logos, products, artwork, landmarks -- and looks up information about them. Plus it scans business cards to extract the contact information.

So now you can shoot the cover of a book or CD to look up the product information, zoom in on artwork to look up the artist, and then focus on the text to scan and translate it.

Goggles also has added a crop rectangle that you can position to select the portion of the image that you want to scan, and control for the flash to better illuminate what you are shooting. And you can do searches on images in the phone's photo gallery.

And beyond visual searches of images. Goggles also has a heads-up "Augmented Reality" mode in which it overlays Google Local information about near-by places as you pan the live camera around your location (see earlier post on AR apps).

Google Goggles is a free app, available for Android devices running Android 1.6 and above (like, of course, the new Verizon / HTC Droid Incredible -- see previous post).

What's up next? -- How about face finding and face recognition ...

See my Smartphone Apps Gallery for more on mobile apps

See my article Augmented Reality Goes Mobile - for more on AR apps, including Google Goggles

Find the Verizon Droid Incredible on Amazon.com.

What's new in Google Goggles version 1.1 ...

Time for a tasty frozen treat -- Google announced the details of the upcoming release of Android 2.2 at last week's Google I/O developers conference. This is the seventh platform release since Android 1.0 was launched in September 2008.

The code names for the Android releases are desserts, in alphabetical order (Cupcake, Donut, Eclair), and version 2.2 is named Froyo, for frozen yogurt. After all, this is not just a minor point release, it's a yummy-looking treat promising a wide range of improvements, for users and developers alike.

Some of the highlights include:

- Significant speedups, promising 2x-5x improvements for processing-intensive apps, plus faster loading of web pages.
- Support for running applications from SD card, so you can install large apps like games without using up all the internal memory.
- Support for using the phone as a shared Wi-Fi hotspot for laptops or other portable devices, or tethered to a laptop over USB.
- Expanded camera / camcorder features including portrait orientation, LED flash for video, and expanded controls for zoom, flash, exposure, and geo-tagging.
- Enhanced Microsoft Exchange support, especially for security and administrator policies.
- Expanded Bluetooth support including voice dialing, sharing contacts, and car docks.
- Expanded developer support for services including messages from cloud services, backup to cloud storage, speech recognition, and "car mode" and "night mode" controls -- all now available to be built in to any Android app.

The new Android 2.2 is apparently already starting to roll out to Nexus One phones, but will take longer to move though the release process for other manufacturers and carriers, like the new Verizon Wireless Droid Incredible from HTC (see earlier post).

See the Android 2.2 Platform Highlights for more information on the new release, including screen shots and an introductory video.

See my Mobile Communications Gallery for more on smartphones

Find the Verizon Droid Incredible on Amazon.com.

More details on Android 2.2 features ...

The difference is clear with the new Verizon Wireless Droid X from Motorola -- it's got a big, big display. For some, this is a great blessing, to view your apps and websites and messages more clearly, and to better enjoy and share photos and videos. But for others, the result is a handheld device that can stretch the limits of what fits comfortably in your pocket or purse.

The Droid X display is significantly bigger at 4.3 inches, compared to the Droid Incredible at 3.7 inches (see earlier post). The resolution also steps up, to 854 x 480 pixels from 800 x 400. (In comparison, the Apple iPhone 4 display is 3.5 inches, but with very fine detail at 960 x 640 (see earlier post), up from the iPhone 3G at 480 x 320.)

The larger display not only means bigger icons and text, but also more information on the screen. The size also helps when you play widescreen video, especially if other people are trying to watch at the same time.

Not surprisingly, the bigger screen on the Droid X results in a bulked-up case -- 5 x 2.6 x 0.4 inches, compared to 4.6 x 2.3 x 0.47 for the Droid Incredible. (Yes, the Droid X actually is thinner, with a slimmer profile that swells out at the top for the camera. The Droid X also is almost an ounce heavier at 5.5 vs. 4.6 ounces.

The result pushes the bounds of today's understanding of "handheld." If you lay the Incredible on top of the Droid X, both the bottom buttons and side frame are still visible.

But this is the whole point of the Google Android platform. While Apple designs the iPhone in only one current model, for which it selects the features that you may have, the Android platform is open for different manufacturers and wireless carriers to sell different configurations, so you can find the model that best fits your needs.

Just from Verizon Wireless, your Android options are the original Motorola Droid with slide-out keyboard (see earlier post and article), the HTC Incredible with a faster processor, higher-res 8 MP camera, and optical joystick (see earlier post and article), and now the Droid X with the bigger display. Plus, the various Droid models offer features including FM radio, expansion memory (up to 32 GB with micro SD cards), and wireless mobile hotspot (to share the 3G cellular connection as a Wi-Fi hotspot).

As a multimedia device, the Droid X steps up to 720p HD video capture with the 8 MP camera, and offers HD playback on your HDTV display with the built-in micro HDMI connector. It also supports wireless sharing of music, photos, and videos to devices on your home network with DLNA technology.

However, the flip side of the open Android approach, versus a strictly controlled platform, is the potential for anarchy in different versions of different software on different devices with different features. Yes, the smartphone has reached a crucial mass of capability and corresponding complexity, so the open PC vs. more closed Mac platform debate will play out again in handheld devices. We've seen these issues before.

Another issue, though, is the desire of manufacturers like Motorola and HTC to differentiate their Android products by enhancing and improving the Google software with customized interfaces and changes to various applications.

For example, the Droid X home screen has permanent soft keys for Phone and Contacts and Apps, plus a pop-up strip to help you navigate the seven home screen panels that you can customize with shortcuts and widgets. The panels come preloaded with scads of widgets, some taking up a quarter or half the screen, for news and tips and photos and contacts and app launcher and music playback and weather and email and airplane modes and calendar and social networking. Whew! That's too much even for a computer screen, so it really seems out of place on a smartphone.

The bottom line, however, is that you have options. The Android platform is evolving rapidly, with the latest Google Android 2.2 release (see earlier post) is due out on the Droid X later this summer, with Adobe Flash Player 10.1. You may choose the Droid X with the big display for enjoying and sharing media, or prefer the original Droid for lots of messaging on the slide-out keyboard, or select the Droid Incredible for a compact and powerful Android experience.

See my full article on the Droid Incredible - Droid Incredible from Verizon / HTC

See my earlier article on the Android interface - Verizon / Motorola Droid -- Android 2.0

See my Mobile Communications Gallery for more on smartphones

See my Smartphone Apps Gallery for more on mobile apps

Find the Verizon Droid X, Droid Incredible, and Droid on Amazon.com